CA Stress Testing Results


TheKitchenAbode
 Share

Recommended Posts

All these tests seem to point a bad analogy. Like the engine in our cars and the CPU in our computers, if you want to go fast more horsepower is faster. Other factors for sure, like turbocharging = more HP but the thing that surprises me, and not in a good way, is how little the GPU factors in to the speed of Chief...Will NOT be easy to ignore the fastest vid card out there but it seems to be a pretty bad investment for Chief speed.

 

Also I have never really considered AMD chips but may as the numbers, again, are hard to deny..

 

EDIT: ...and not a burly Diesel pumping out tons of low speed torque, that will run all day for a million miles, like the Xeons, but a high revving horsepower machine putting out maximum GHZ..(HP). Seems there's really no substitute.

Link to comment
Share on other sites

48 minutes ago, HumbleChief said:

Also I have never really considered AMD chips but may as the numbers, again, are hard to deny..

 

I have asked if Mike if this could be confirmed as the number of 11 sec to open the camera in the BBQ test seems way too low according to his systems specs and when compared to the numbers being reported by those with I7 -9700K and I9 - 9900K CPUs. The AMD is a good processor but I have never seen a comparison review that shows it beating those processors.

Link to comment
Share on other sites

1 hour ago, HumbleChief said:

but the thing that surprises me, and not in a good way, is how little the GPU factors in to the speed of Chief

 

There's no doubt so far in the three posted stress tests that the GPU's role is very minor and that certainly raises the question as to the value of spending big bucks on the most powerful GPU out there. Also, other than PBR'ing there are no indicators that one needs massive amounts of GPU memory, it's just not being used.

 

However, keep in mind that these tests are focused on CA's needs only, you may be running other software on your system that may benefit greatly from a high end GPU.

Link to comment
Share on other sites

I agree with your statement Graham, I upgraded my gpu recently from a gtx960 4gig saw very little improvement. Was disappointed for the amount of money I spent.

Also when I ran my test all I had open was task manager and fire fox window for email and the forum. Also Bit defender security software.

 

 

Link to comment
Share on other sites

9 minutes ago, MPDesign said:

What I dont understand is why the p-solid stress plan is slower than snails with one object?

 

 

That's due to the fill I used, if you open up the object and reduce the fill spacing from say my 1/16' to say 1/4' you will see a significant improvement in performance. That fill creates a crazy number of vector lines that must be recalculated when scaling up and down when zooming.

Link to comment
Share on other sites

Really starting to have a disdain for this thread though I have tried to avoid voicing too much of my own opinion..a lot of hard work has gone into this thread, which I appreciate...

 

This has not been standardized testing.

No real methods of testing have been established.

No specialized software has been implemented for monitoring the tests.

NO COMPUTER COMPONENTS HAVE TRULY BEEN ISOLATED...No one here is using a test bench with the same components and swapping out one hardware component.

The tests have only demonstrated a couple of scenarios that are completely unrealistic for nearly every CA user.

 

Testing based on bias will always have a higher probability of supporting the bias. Why are we doing known CPU intensive tasks and being surprised by the results?

Load in 4k textures on a PBR and tell me how great your CPU is or go swap in a laptop 5400 RPM for your NVME and try browsing the library with your awesome CPU.

Excuse the sarcasm but by no means has this been an indicator of a singular system component being the biggest bang for the buck. I've said it so many times I can't even count, All 5 of the machines I have used extensively have performed as well as the next in real world use with little to no noticeable difference, which includes a gamut of various clock speeds and core configurations both AMD and Intel. Spending $2K on a CPU is not going to make CA feel 10 times faster than spending $200. 

This post by @BrownTiger got brushed over and no one really responded to it which blows my mind, again we're discussing some industry known topics here:

 

Not meaning to sound like a jerk but this thread does not paint the whole picture of using this software and it is painfully obvious to me and misleading to people out there shopping components.

  • Upvote 2
Link to comment
Share on other sites

41 minutes ago, TheKitchenAbode said:

 

That's due to the fill I used, if you open up the object and reduce the fill spacing from say my 1/16' to say 1/4' you will see a significant improvement in performance. That fill creates a crazy number of vector lines that must be recalculated when scaling up and down when zooming.

Thank you Graham, now I get it.

Link to comment
Share on other sites

8 minutes ago, Renerabbitt said:

Spending $2K on a CPU is not going to make C feel 10 times faster than spending $200. 

 

Correct, that's what the stress test results are demonstrating. The same can be said for the GPU, as we have seen so far systems with top end GPU's do not demonstrate improved performance relative to their stated(independently reviewed) gamming performance differences.

 

The tests demonstrate that depending on what type of function one is doing that what impacts on performance varies. It is clear from say the BBQ test that processing surfaces/textures is a mult-threaded operation and as such high core count processors will be beneficial. On the other hand the P-solid test demonstrates that in plan views vector graphics manipulations are mostly single threaded and as such high core count processors do not provide a significant improvement.

 

Though I do not have a test bench here to swap out specific pieces of hardware there are all of the differing systems out here in the forum. They can run these and report their findings and as being demonstrated there are indicators of certain trends.

 

I'm more than open to other testing examples/methods to demonstrate other performance aspects. I'm even open to other tests that disprove the conclusions being drawn upon based on my tests.

Link to comment
Share on other sites

25 minutes ago, Renerabbitt said:

Really starting to have a disdain for this thread though I have tried to avoid voicing too much of my own opinion..a lot of hard work has gone into this thread, which I appreciate...

 

This has not been standardized testing.

No real methods of testing have been established.

No specialized software has been implemented for monitoring the tests.

NO COMPUTER COMPONENTS HAVE TRULY BEEN ISOLATED...No one here is using a test bench with the same components and swapping out one hardware component.

The tests have only demonstrated a couple of scenarios that are completely unrealistic for nearly every CA user.

 

Testing based on bias will always have a higher probability of supporting the bias. Why are we doing known CPU intensive tasks and being surprised by the results?

Load in 4k textures on a PBR and tell me how great your CPU is or go swap in a laptop 5400 RPM for your NVME and try browsing the library with your awesome CPU.

Excuse the sarcasm but by no means has this been an indicator of a singular system component being the biggest bang for the buck. I've said it so many times I can't even count, All 5 of the machines I have used extensively have performed as well as the next in real world use with little to no noticeable difference, which includes a gamut of various clock speeds and core configurations both AMD and Intel. Spending $2K on a CPU is not going to make CA feel 10 times faster than spending $200. 

This post by @BrownTiger got brushed over and no one really responded to it which blows my mind, again we're discussing some industry known topics here:

 

Not meaning to sound like a jerk but this thread does not paint the whole picture of using this software and it is painfully obvious to me and misleading to people out there shopping components.

Hi Rene

I did read Brown Tigers reply and it sounded like a very professional inspection of whats going on. ( Thank you Brown Tiger!)  My system was built 1-2012 so I will need to upgrade the whole thing in the near future. I have been watching your threads about ram disk and the nvme drives. I cant wait to get that newer hardware. :)

Link to comment
Share on other sites

17 hours ago, TheKitchenAbode said:

Mike - Would it be possible to run this one again.

Sorry guys. I must have ran the test twice without realizing the difference in time it would make. I ran it again like Graham asked - 22 sec. I closed CA and opened the plan again. Same results - 22 sec. Haven't ran the Parade of Homes test yet; Monday struggles.:)

Link to comment
Share on other sites

Just now, Ridge_Runner said:

Sorry guys. I must have ran the test twice without realizing the difference in time it would make. I ran it again like Graham asked - 22 sec. I closed CA and opened the plan again. Same results - 22 sec. Haven't ran the Parade of Homes test yet; Monday struggles.:)

 

Thanks Mike, that makes sense. Really appreciate your participation.

Link to comment
Share on other sites

1 hour ago, Renerabbitt said:

Really starting to have a disdain for this thread though I have tried to avoid voicing too much of my own opinion..a lot of hard work has gone into this thread, which I appreciate...

 

This has not been standardized testing.

No real methods of testing have been established.

No specialized software has been implemented for monitoring the tests.

NO COMPUTER COMPONENTS HAVE TRULY BEEN ISOLATED...No one here is using a test bench with the same components and swapping out one hardware component.

The tests have only demonstrated a couple of scenarios that are completely unrealistic for nearly every CA user.

 

Testing based on bias will always have a higher probability of supporting the bias. Why are we doing known CPU intensive tasks and being surprised by the results?

Load in 4k textures on a PBR and tell me how great your CPU is or go swap in a laptop 5400 RPM for your NVME and try browsing the library with your awesome CPU.

Excuse the sarcasm but by no means has this been an indicator of a singular system component being the biggest bang for the buck. I've said it so many times I can't even count, All 5 of the machines I have used extensively have performed as well as the next in real world use with little to no noticeable difference, which includes a gamut of various clock speeds and core configurations both AMD and Intel. Spending $2K on a CPU is not going to make CA feel 10 times faster than spending $200. 

This post by @BrownTiger got brushed over and no one really responded to it which blows my mind, again we're discussing some industry known topics here:

 

Not meaning to sound like a jerk but this thread does not paint the whole picture of using this software and it is painfully obvious to me and misleading to people out there shopping components.

Agree 110%

Link to comment
Share on other sites

47 minutes ago, Renerabbitt said:

Really starting to have a disdain for this thread though I have tried to avoid voicing too much of my own opinion..a lot of hard work has gone into this thread, which I appreciate...

 

This has not been standardized testing.

No real methods of testing have been established.

No specialized software has been implemented for monitoring the tests.

NO COMPUTER COMPONENTS HAVE TRULY BEEN ISOLATED...No one here is using a test bench with the same components and swapping out one hardware component.

The tests have only demonstrated a couple of scenarios that are completely unrealistic for nearly every CA user.

 

Testing based on bias will always have a higher probability of supporting the bias. Why are we doing known CPU intensive tasks and being surprised by the results?

Load in 4k textures on a PBR and tell me how great your CPU is or go swap in a laptop 5400 RPM for your NVME and try browsing the library with your awesome CPU.

Excuse the sarcasm but by no means has this been an indicator of a singular system component being the biggest bang for the buck. I've said it so many times I can't even count, All 5 of the machines I have used extensively have performed as well as the next in real world use with little to no noticeable difference, which includes a gamut of various clock speeds and core configurations both AMD and Intel. Spending $2K on a CPU is not going to make CA feel 10 times faster than spending $200. 

This post by @BrownTiger got brushed over and no one really responded to it which blows my mind, again we're discussing some industry known topics here:

 

Not meaning to sound like a jerk but this thread does not paint the whole picture of using this software and it is painfully obvious to me and misleading to people out there shopping components.

 

Pretty sure that the testers are not fooled or mislead by this test in to thinking that it's a definitive test of all computer components and all possible scenarios and hopefully no tester is stupid enough to think that spending 2K on a CPU will be 10 times faster than a $200 CPU, but maybe we're not that smart?

 

It's only a test as defined by the limited parameters of the test and that's all it is. Does that make it worthless? Maybe through some eyes but through my eyes it's an attempt to set some parameters that allow a comparison no matter how flawed.

 

Will I buy the fastest CPU I can afford based on this test? No. I'll buy the fastest CPU I can afford because I know that's a good long term investment. Will I ignore the fastest SSD hard drive because of this test? No way. Will I buy the fastest vid card? Probably just because I will never trust a test like this to tell the whole story and hopefully no one else will either.

 

With that said I would, again, to thank Graham for the effort as I have at the very least, learned.

Link to comment
Share on other sites

3 hours ago, HumbleChief said:

All these tests seem to point a bad analogy. Like the engine in our cars

 

1 hour ago, Renerabbitt said:

The tests have only demonstrated a couple of scenarios that are completely unrealistic for nearly every CA user.

 

I had the same thoughts. My puny system performs well in everything I do, which is designing average sized single family residences. Would it be more beneficial if folks had an idea of what their money gets them in those situations, what basics are needed to perform fairly well, and whether it's really worth it to buy the baddest and most expensive?

 

Or going by Larry's analogy, does it matter whether the Corvette's 6.2L supercharged LTI or the 6.2 Hemi Hellcat is more powerful, if all we're doing is going to the grocery store?

 

These tests help determine which hardware configurations are the fastest, but what about real-life situations? Sure, some people have huge projects and use lots of embedded files, but I suspect that for the average CA user, just knowing the basics they need to get the job done is more useful...

Link to comment
Share on other sites

7 minutes ago, HumbleChief said:

With that said I would, again, to thank Graham for the effort as I have at the very least, learned.

Many thanks to Graham, I have told him in PMs and publicly that his donated time is extremely generous.

 

I've also shared some PMs with him voicing frustrations for the lack of guidance from the developers on system specs in these forums.

 

This is a fun thread, and its interesting, I would not call it difinitive

 

To respond to what you said, and speaking for myself, I agree with you whole heatedly, but I was once a dumb 20 year old kid looking to get into the industry and wondering what system specs I should focus on, would've been nice if I was spared all of my failures by a truly definitive approach to system building...then again maybe that's what I can attribute some success to, perseverance.

...I still can't successfully build an ad hoc network..been trying for 10 years

Link to comment
Share on other sites

6 minutes ago, HumbleChief said:

With that said I would, again, to thank Graham for the effort as I have at the very least, learned.

 

Agreed, I participated in the test and thank Graham for taking the time, it's definitely an interesting thread. 

Link to comment
Share on other sites

1 minute ago, Chrisb222 said:

 

 

I had the same thoughts. My puny system performs well in everything I do, which is designing average sized single family residences. Would it be more beneficial if folks had an idea of what their money gets them in those situations, what basics are needed to perform fairly well, and whether it's really worth it buy the baddest and most expensive?

 

Or going by Larry's analogy, does it matter whether the Corvette's 6.2L supercharged LTI or the 6.2 Hemi Hellcat is more powerful, if all we're doing is going to the grocery store?

 

These tests help determine which hardware configurations are the fastest, but what about real-life situations? Sure, some people have huge projects and use lots of embedded files, but I suspect that for the average CA user, just knowing the basics they need to get the job done is more useful...

The problem for me and Chief is not trips to the grocery store but long fast marathon drives that tax the system as much possible and genuinely slow Chief down to where large plans are unworkable, very much like (but of course not exactly) like the test plans posted here. Will I ever need a Hemi Hellcat? Maybe not but if do and don't have one what then? Just work through the pain of 5 - 7 second delays with each move in Chief? Or invest in the HellCat for when I actually need the power? I'll probably, however unwise it might be, choose the later. 

Link to comment
Share on other sites

5 minutes ago, HumbleChief said:

The problem for me and Chief is not trips to the grocery store but long fast marathon drives that tax the system as much possible and genuinely slow Chief down to where large plans are unworkable, very much like (but of course not exactly) like the test plans posted here. Will I ever need a Hemi Hellcat? Maybe not but if do and don't have one what then? Just work through the pain of 5 - 7 second delays with each move in Chief? Or invest in the HellCat for when I actually need the power? I'll probably, however unwise it might be, choose the later. 

just rent Amazon EC2 if it ever gets to the point... sit in your living room working from a small touch screen tablet linked to 100 amazon machines that just laugh at CA.

I may set this up next year, So I can truly work from anywhere

Link to comment
Share on other sites

35 minutes ago, HumbleChief said:

The problem for me and Chief is not trips to the grocery store but long fast marathon drives that tax the system as much possible and genuinely slow Chief down to where large plans are unworkable, very much like (but of course not exactly) like the test plans posted here. Will I ever need a Hemi Hellcat? Maybe not but if do and don't have one what then? Just work through the pain of 5 - 7 second delays with each move in Chief? Or invest in the HellCat for when I actually need the power? I'll probably, however unwise it might be, choose the later. 

 

C'mon, who really NEEDS a Hellcat?? :P

 

 

Link to comment
Share on other sites

6 minutes ago, Renerabbitt said:

Many thanks to Graham, I have told him in PMs and publicly that his donated time is extremely generous.

 

Much appreciated Rene, my thoughts are mutual.

 

I knew from the beginning of this exercise that aspects of this would raise controversy. The purpose here was not to purposely prove or disprove anything, it was intended to allow one, including myself, to better understand some of the things that are going on when working in CA that can impact on ones workflow experience. The derived results are not to be taken in absolute terms, they are to be viewed from a point of relativity to and with the other results.

 

It is extremely important that one does not attempt to extrapolate these result beyond the context of the tests design parameters. The tests were not purposely designed to be biased towards any specific hardware setup, but they are biased from the perspective that they were designed to evaluate a very specific function, and it is the processes involved in executing this function that defines the role played by the hardware. The fact that the tests force CA to process plans beyond what one normally works with is by design, otherwise the involved processes would not be revealed. It's no different than using a high speed camera to allow one to examine fast moving things through slow motion. If not for this type of technique we would still be arguing over whether or not all 4 hoofs of a horse left the ground while galloping.

 

Do other elements factor into this performance issue such as caching, memory speed, ram disks and the likes, I'm certain there are but to date I have not seen any quantifiable data specific to CA  to demonstrate this. I have read many technical documents concerning the impact these can have, the issue is that in many cases their degree of benefit is highly dependent upon very specific processes. Added to this is the fact that some of these techniques do not, due to other significant hardware improvements, have the same degree of benefit as they did at the time of their original evaluation. I'm also a bit hesitant to go to far into these as those techniques are in most cases outside of the average users expertise.

Link to comment
Share on other sites

14 minutes ago, TheKitchenAbode said:

 

Much appreciated Rene, my thoughts are mutual.

 

I knew from the beginning of this exercise that aspects of this would raise controversy. The purpose here was not to purposely prove or disprove anything, it was intended to allow one, including myself, to better understand some of the things that are going on when working in CA that can impact on ones workflow experience. The derived results are not to be taken in absolute terms, they are to be viewed from a point of relativity to and with the other results.

 

It is extremely important that one does not attempt to extrapolate these result beyond the context of the tests design parameters. The tests were not purposely designed to be biased towards any specific hardware setup, but they are biased from the perspective that they were designed to evaluate a very specific function, and it is the processes involved in executing this function that defines the role played by the hardware. The fact that the tests force CA to process plans beyond what one normally works with is by design, otherwise the involved processes would not be revealed. It's no different than using a high speed camera to allow one to examine fast moving things through slow motion. If not for this type of technique we would still be arguing over whether or not all 4 hoofs of a horse left the ground while galloping.

 

Do other elements factor into this performance issue such as caching, memory speed, ram disks and the likes, I'm certain there are but to date I have not seen any quantifiable data specific to CA  to demonstrate this. I have read many technical documents concerning the impact these can have, the issue is that in many cases their degree of benefit is highly dependent upon very specific processes. Added to this is the fact that some of these techniques do not, due to other significant hardware improvements, have the same degree of benefit as they did at the time of their original evaluation. I'm also a bit hesitant to go to far into these as those techniques are in most cases outside of the average users expertise.

Was hoping you really didn't have to post that disclaimer as I for one understood those parameters from the beginning, but thanks anyway for those who did not.

 

Now the remaining question, a HellCat Processor? Or a Hellephant processor? Maybe a Hellephant with a turbo cache? Could you run a couple tests please? :P:P

Link to comment
Share on other sites

1 minute ago, HumbleChief said:

Now the remaining question, a HellCat Processor? Or a Hellephant processor? Maybe a Hellephant with a turbo cache? Could you run a couple tests please? :P:P

 

I'll send you my address so you can have one dropped off in my driveway for a test run. Results won't be very good though, with our traffic conditions about 60 miles an hour is tops and there are traffic lights about every 2 blocks.:)

Link to comment
Share on other sites

56 minutes ago, TheKitchenAbode said:

 

I'll send you my address so you can have one dropped off in my driveway for a test run. Results won't be very good though, with our traffic conditions about 60 miles an hour is tops and there are traffic lights about every 2 blocks.:)

What? Not a real world test?  I dunno man..........:o:o

Link to comment
Share on other sites

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now
 Share