CA Stress Testing Results

[TheKitchenAbode]

Rene - Just did the same thing on my desk top, disabled the 1060 and switched to the hd530 chip. I get the same message. If I check in CA preferences video card status it shows nothing, it's as if it does not recognize the chip at all.

 

I have X11 running on my Spectra, it has an intel 620 chip. If I open up preferences it is shown. The library works ok.

 

What I have noticed is that if I go to windows control panel on my Spectra there is an Intel Graphics Settings option. But if I go to the Control Panel on my desktop there is no Intel Graphics Settings option, just my Nvida one. Somethings not right.

[TheKitchenAbode]

Here is a very simple plan.

 

Just 1 p-solid, 12 surfaces, default concrete. File size is only 2,845 KB

 

Let me know if you can work fluidly with this in plan view.

 

While doing this have task manager open, tab-performance and observe your systems activity.

 

P Solid Stress.plan

[Kbird1]

2 hours ago, TheKitchenAbode said:

But if I go to the Control Panel on my desktop there is no Intel Graphics Settings option, just my Nvida one. Somethings not right.

 

Two Slightly different Drivers version I beleive, your Spectre may have the full Intel Driver version, while the Desktop may have the built in Windows 10 Intel (MUC) driver in use which may not be fully featured, typically these don't include the 3D options Panel etc , I'd think a google search would find more info on the missing right click menu.

 

Thanks for looking into this , interesting read.......

 

M.

[TheKitchenAbode]

 

18 hours ago, Alaskan_Son said:

 

Multi page capability for starters maybe?

 

I assume you are referring to the ability to select pages within a PDF doc prior to importing. if so, yes this is a convenience. However, from what I can see is that once the pages are loaded they are essentially dumb from that point on. Not really any different then importing a very high resolution pic which can also result in simialr lag. Looks to me lke the issue relates to what CA has to do to scale these as one zooms and pans. For example, text components are not fonts anymore so traditional efficient font scaling algorithims can't be used so very single pixel must be calculated and extrapolated which could be very time consuming.

[HumbleChief]

Want to thank you again for the effort Graham, it will help in the future when choosing a new system...

[Alaskan_Son]

1 hour ago, TheKitchenAbode said:

I assume you are referring to the ability to select pages within a PDF doc prior to importing.

 

Actually no.  I'm referring to the fact that multi-page PDF's are imported with all pages.  When we import a PDF, we're not importing a snapshot, but rather the whole document.  In other words, we can display any page that we wish at any time.  The whole entire file is imported and can optionally be stored with the plan.

[BrownTiger]

I do not know how you make all these observations...

 

Here is my experience:

Software runs about 52 thread, when rendered 6 more threads are created.

 

Startup:

Creates a log file: C:\Users\<username>\AppData\Local\Chief Architect Premier X11\Message Log.txt

Mapps every single font used by this plan

Mapps (preload) every single *.calib

Lots of HASP Security delay

Sets up Named Pipe: \Device\NamedPipe\Chief Architect Premier X11 Single Instance Lock

Initializes QtWebEngine

Software executes file:///<install location>/resources/html/startup-options.js?cache-bust=

Executes  https://www.chiefarchitect.com/go?title=Chief Architect Premier&version=21.1.1.2&platform=x64&resource=startup-dynamic-content&dc-version=4&product-key=abcdefgh&academic=0&student=0&presentation=0

Builds up new ini's and sets up toolbars based on the plan file.

 

Startup observations:

More fonts and more calibs you have => Slower startup.

Issues with the software: check Message Log.txt

 

2D rendering is pretty quick  - heavily optimized, precomputed. 

CAModel.DLL and CACommon.DLL 

It appears that they re-wrote the clunky undo/redo engine.

The only way to measure app real performance appears to be force UNDO/REDO so it rebuilds the display trees.

Anything else gets you some sort of cache(s), and the performance will depend on the plan.

 

3D rendering 

<PERFORMANCE HIT> If your plan file is dirty file autosaved

<PERFORMANCE HIT>First render: Every referenced Zip file associated with plan calib is opened and read [C:\ProgramData\Chief Architect Premier X11\Referenced Files], so if you are referencing a zip that is large, rendering incurs the penalty. Damn you: bonusPlants.zip 

Renderer executed results stored in the pageFile <Virtual Memory>

 

Second 3D rendering 

Nothing is really executed other than PageFile (Virtual Memory is read)

 

The faster your virtual memory [hard drive page] - the less delay you have.

ALL about HardDrive, dont think Video Card or CPU will make a difference.

 

Overall very nice job CA.

 

 

 

[BrownTiger]

Overall I believe that CA took time profiling the application and tuning slow functions. So if your system does not 

perform - Hard Drive [excluding something that has millions of polygons].

[Renerabbitt]

2 hours ago, BrownTiger said:

Overall I believe that CA took time profiling the application and tuning slow functions. So if your system does not 

perform - Hard Drive [excluding something that has millions of polygons].

 

‘CA is a cache monster in my opinion”

 

I posted that somewhere else around here...totally agree about hard drive from testing and observation...great to see it broken down so eloquently...cheers and +1

still say that custom textures with 4K resolution requires a good vid card... I noticed a difference when I switched to pascal

[TheKitchenAbode]

4 hours ago, Renerabbitt said:

 

still say that custom textures with 4K resolution requires a good vid card... I noticed a difference when I switched to pascal

 

From the testing everything you add increases complexity, some items such as high resolution textures contribute to complexity greater than other items. I'm certainly not saying that better hardware will not result in improved performance, just attempting to gain a sense as to the degree of improvement and where that improvement might be realized.

 

Just a simple example, my main systems has a GTX1060 6GB, my Spectra has an Intel HD 620 integrated graphics chip. Technically the GTX 1060 is 10 times more powerful than the HD 620, however when I run my test models the difference is very little, also as my Spetra's CPU is half of what my main system is it's possible that some of the slightly slower video performance is related to it.

 

On the other hand when I did my last main system upgrade the primary difference between the build was the CPU, same memory(8GB), same type of hard drive(std HD), minor graphics card difference, however in this case the performance boost was very noticeable.

 

 

Possibly the below post was missed, would be interesting if other users would run this and report their experience. It's not a trick, just an example of how other things can impact on performance.

 

On 4/29/2019 at 5:25 PM, TheKitchenAbode said:

Here is a very simple plan.

 

Just 1 p-solid, 12 surfaces, default concrete. File size is only 2,845 KB

 

Let me know if you can work fluidly with this in plan view.

 

While doing this have task manager open, tab-performance and observe your systems activity.

 

P Solid Stress.plan

[TheKitchenAbode]

On 4/30/2019 at 11:12 AM, Alaskan_Son said:

 

Actually no.  I'm referring to the fact that multi-page PDF's are imported with all pages.  When we import a PDF, we're not importing a snapshot, but rather the whole document.  In other words, we can display any page that we wish at any time.  The whole entire file is imported and can optionally be stored with the plan.

Not 100% sure that the PDF is actually being imported into to plan. I just imported a 4 page pdf, saved my plan, closed it and then went to the original referenced PDF and changed it's name. When I re-opened the plan CA said referenced file missing and nothing showed up in my plan. Regardless, the concern is how rapidly CA's performance can degrade. I can do a similar PDF procedure in Microsoft OneNote and everything is fast and smooth, pan, zoom, scroll, resize, all actions are instant.

 

Edit - Ignore first statement, forgot to click Save in plan.

[Alaskan_Son]

22 minutes ago, TheKitchenAbode said:

Not 100% sure that the PDF is actually being imported into to plan. I just imported a 4 page pdf, saved my plan, closed it and then went to the original referenced PDF and changed it's name. When I re-opened the plan CA said referenced file missing and nothing showed up in my plan. Regardless, the concern is how rapidly CA's performance can degrade. I can do a similar PDF procedure in Microsoft OneNote and everything is fast and smooth, pan, zoom, scroll, resize, all actions are instant.

 

Edit - Ignore first statement, forgot to click Save in plan.

 

I hadn't been speaking to the performance degradation issue, only to the fact that there are some reasons Chief doesn't just convert the PDF to an image.  They're really not the same (Imported PDF's and Picture files) and have some different capabilities...multiple pages being one of them.

[TheKitchenAbode]

1 minute ago, Alaskan_Son said:

 

I hadn't been speaking to the performance degradation issue, only to the fact that there are some reasons Chief doesn't just convert the PDF to an image.  They're really not the same (Imported PDF's and Picture files) and have some different capabilities...multiple pages being one of them.

 

I understand that aspect. But the purpose of my post is performance related. It's just that I'm seeing things that seem to contradict common belief, including mine. In particular what appears to be the underutilization of my GPU. I posted a small test plan hoping others would provide some feedback but so far none has been received. I have no problem if it is something unique to my setup, just would be nice to know.

[Chrisb222]

On 4/29/2019 at 5:25 PM, TheKitchenAbode said:

Let me know if you can work fluidly with this in plan view.

 

No. Everything I do in plan view causes my computer to hang for 2-4 seconds.

[TheKitchenAbode]

Just now, Chrisb222 said:

 

No. Everything I do in plan view causes my computer to hang for 2-4 seconds.

 

Thanks Chris, that's what happens on my main system and my Spectra 360.

[Alaskan_Son]

1 hour ago, TheKitchenAbode said:

 

Thanks Chris, that's what happens on my main system and my Spectra 360.

 

Its gonna happen on everybody’s system because of the fill you used on that polyline.  Chief is having to draw an absolutely bonkers number of vector lines (somewhere in the neighborhood of 65 million lines).  

[TheKitchenAbode]

6 minutes ago, Alaskan_Son said:

 

Its gonna happen on everybody’s system because of the fill you used on that polyline.  Chief is having to draw an absolutely bonkers number of vector lines.  

Yes, that is correct. The point is two fold, first lag is not just surface related. The second point is whether or not those with very high end systems experience greater fluidity. In other words if my 6700K 1060 lags for 3-4 seconds then does a much more powerful system lag significantly less. If not then why not?

[Chrisb222]

7 minutes ago, TheKitchenAbode said:

Thanks Chris, that's what happens on my main system and my Spectra 360.

 

Well I'm pretty proud that my little Mac Mini is running with the big dog$. 

 

$700, quiet as a mouse, and not much bigger. Can't PBR but I prefer RT anyway. Yeah, I'm happy...

 

[TheKitchenAbode]

16 minutes ago, Chrisb222 said:

 

Well I'm pretty proud that my little Mac Mini is running with the big dog$. 

 

$700, quiet as a mouse, and not much bigger. Can't PBR but I prefer RT anyway. Yeah, I'm happy...

 

 

This is what's conflicting. My main systems CPU is technically ranked at least 2 times faster and my GPU is technically ranked 10 times faster but we both experience similar lag. Theoretically this should not be the case.

[TheKitchenAbode]

Time permitting I have continued to further explore this subject and rerun many of the preliminary tests to ensure repeatability. So far all results have been consistent and supportive with the statements made to date. Direct comparisons were made between my two systems, I7 6700k Alienware and I5 Spectra 360, all indications are that the CPU differences were the primary reason for the differences in overall performance between the two systems. Technically the I 6700K is on average twice as fast as the I5 in the Spectra 360 and all of the results reflected this.

 

Besides the CPU, the only other significant difference between the two systems is the GPU, GTX 1060 versus Intel HD 620 integrated graphics in the Spectra 360. Technically the GTX 1060 is 10 times more powerful than the HD 620, however results did not reveal this anticipated level of improved graphical performance. In all cases indications are that the CPU is restraining(limiting) the GPU's potential. As mentioned in previous posts, CA must compile(prepare) things before sending this to the GPU for final processing and this compiling is primarily CPU based. As a result, it appears that the GPU spends most of it's time waiting for something to do. Even when PBR'ing, it can be seen that the majority of scene processing time relates to CPU preparation time, this is of particular importance when a scene is being generated for the first time. Once the scene has been generated and is now primarily under GPU control then the GPU plays a much greater role as one pans and zooms within the scene. This is an important aspect to keep in mind as graphics generation is a two part process, the first being the initial preparation of the scene and the second being the handling of the scene thereafter. As a result higher end graphics cards will likely be held back by the CPU during the initial scene generating stage but should provide improved performance when moving around within the scene.

 

What must also be explored, in respect to the above, are the processes involved when working within a scene once it is primarily under GPU control. When working in the scene the GPU only handles certain aspects such as panning and zooming, when any change to the model is made the model must be updated which is primarily a CPU based process and as such the GPU will in most cases be waiting for this new updated model. Once again that high end graphics card is likely being performance limited by the CPU.

[HumbleChief]

On 4/29/2019 at 2:25 PM, TheKitchenAbode said:

Let me know if you can work fluidly with this in plan view.

P Solid Stress.plan

No, my computer hangs for as little as 8 and as much as 17 seconds on a simple zoom via mouse wheel. 13 seconds to simply select the slab. 14 seconds to move an edge. I would say unusable with my system. ...and 12 seconds to simply close the file...That thing is evil and I've deleted it off my system just in case it will wake me in the middle of the night....

[TheKitchenAbode]

1 hour ago, HumbleChief said:

No, my computer hangs for as little as 8 and as much as 17 seconds on a simple zoom via mouse wheel. 13 seconds to simply select the slab. 14 seconds to move an edge. I would say unusable with my system. ...and 12 seconds to simply close the file...That thing is evil and I've deleted it off my system just in case it will wake me in the middle of the night....

 

Very descriptive!!!, It was designed to stress systems so the fact that you experienced a lag is not unexpected and I greatly appreciate you reporting back some numbers. The lag you experienced is primarily due to the fact that those operations require considerable CPU processing and to make matters worse most of those processes appear to be single or lightly threaded at best. What is also important to note is that your 1080 GPU is technically almost double the performance of my 1060, yet your lag appears to be somewhat no less than what I am experiencing. You also have 24 logical cores versus my 8 logical cores, again this did not appear to be beneficial nor having double the memory. The result you saw is as would be predicted via the stress testing results and the provided analysis which indicates that under these circumstances CPU base and turbo frequency will be the primary determining performance factor.

 

This does not mean that those dual Zeon's and 1080 GPU can't shine, it's just that the task must be one that favors their particular qualities.

 

[HumbleChief]

14 minutes ago, TheKitchenAbode said:

 

Very descriptive!!!, It was designed to stress systems so the fact that you experienced a lag is not unexpected and I greatly appreciate you reporting back some numbers. The lag you experienced is primarily due to the fact that those operations require considerable CPU processing and to make matters worse most of those processes appear to be single or lightly threaded at best. What is also important to note is that your 1080 GPU is technically almost double the performance of my 1060, yet your lag appears to be somewhat no less than what I am experiencing. You also have 24 logical cores versus my 8 logical cores, again this did not appear to be beneficial nor having double the memory. The result you saw is as would be predicted via the stress testing results and the provided analysis which indicates that under these circumstances CPU base and turbo frequency will be the primary determining performance factor.

 

This does not mean that those dual Zeon's and 1080 GPU can't shine, it's just that the task must be one that favors their particular qualities.

 

Interesting to say the least. What makes that .plan such a beast? Or did I miss the explanation above?

[TheKitchenAbode]

Just now, HumbleChief said:

Interesting to say the least. What makes that .plan such a beast? Or did I miss the explanation above?

 

It's the fill that causes the issue. It's set at 1/16" and given the size of the poly-solid CA has to really work to calculate it all. I did this primarily to provide an example to demonstrate that surfaces are not the only determining factor concerning lag. In this example there are only 12 surfaces yet the lag in plan view is considerable. I have a test model that has almost 20 million surfaces and it functions in plan view far better than this one single poly-solid. It's also worth noting that plan view performance differs from camera view performance, different elements affect the view types differently and the involved processes.

[glennw]

8 hours ago, HumbleChief said:

No, my computer hangs for as little as 8 and as much as 17 seconds on a simple zoom via mouse wheel. 13 seconds to simply select the slab. 14 seconds to move an edge. I would say unusable with my system. ...and 12 seconds to simply close the file...That thing is evil and I've deleted it off my system just in case it will wake me in the middle of the night....

 

Graham,

Thanks for all your hard work and making the results available to us all.

I thought I would load this file on my relatively low spec mac to see how it compares with Larry's times.

I tested the same operations and got greatly different (shorter) times - I have included Larry's times in brackets:

Zoom 2 seconds (8-17)

Select slab 4 seconds (13)

Move edge 4 seconds 14)

Close file instant (12)

 

How could you explain these differences?

My processor is only a 3.5GHz intel i5