Back in the fall of last year Architosh began talking to numerous developers -- who have been in the Macintosh market for years -- about what they thought of Apple's new Power Mac G5 with its new 64-bit IBM chip. We were interested to know just how they felt about these new machines and what kind of performance advantage they really gave Apple, if any at all. We even began talking about a new Architosh Benchmark Suite for Architecture -- co-developed with a set of developers who support that market space. To our surprise everyone we talked to said to "count them in" on the benchmark suite because they were 100 percent behind the Mac and 100 percent excited about the new G5 processor inside the new Power Mac.

One of the things I would like to point out to those reading this is that Nemetschek North America is not alone in being excited about the new G5 and the Mac platform. A number of developers, some new some old to the Mac space, are incredibly excited about the OS X platform in general and the hardware architecture Apple had created with the Power Mac G5.

Let's take a look at the scores and get into the discussion about them. Nemetschek has some internal testing files that test VectorWorks performance across three critical areas: 1) 2D Redraw speed - an area that test the general 2D CAD drawing performance, 2) Rendering performance - testing the calculation times to finish various renders, and 3) Solids modeling - testing the length it takes for various modeling operations to complete. At the very end of this article we have a summary graph (click on all graphics for a larger image you can read):

Interview: Fall 2003

AFR. These scores are wonderful Sean. And very honest as you can see when comparing to the older G4 hardware. Are they created out of a test suite like Maxon's CineBench render performance test suite?

Sean: We don't have a simple canned test. Instead we use a number of files with scripts that write out timing results to text files. They are very process dependent right now to get repeatable results (making sure comparable screen settings are used, no background apps, etc.) and thus are not particularly user friendly. We have a single and dual G5 in house which we used for the scores you have.

AFR. So this isn't something VectorWorks users can do at home on their own?

Sean: Correct.

AFR. So let's look at the 2D scores. The 2D redraw speeds are so fast on the G5 1.8Ghz compared to the G4 before and the 2.8Ghz Intel Xeons. Why is the G5 so darn fast in this department? What's going on to make this machine so quick?

1.8Ghz G5 Power Mac is 51 percent faster than a 2.8Ghz Intel Xeon workstation at 2D Redraw VectorWorks performance.

Sean: This surprised us also. Unfortunately, we haven't had a chance to profile the results and figure out all the details.

AFR. The numbers the single G5 1.8Ghz turns in with Solids Operations are so much better than the previous G4 machines and even the dual G4. Why is that? Is OpenGL faster under the G5 than the G4?

Sean: Again we haven't been able to dive in deep to the results yet. Solids is an interesting test for me because the code is almost pure C++; there is very little platform-specific code involved. After some consideration, we consider the two main areas likely for this improvement consist of system architecture and floating point.

AFR. So we are talking about the new motherboard architecture on the G5...

Sean: Some of the gain is no doubt due simply to the faster hardware specs. The processor is 45 percent faster, the bus is twice as wide -- which especially helps floating point -- and memory access is significantly faster. Although not as memory intensive as some other parts of VectorWorks, solids modeling does require a lot of information to be moved around in the registers and memory for calculation. So certain amount of the gain is due to the raw gains in the throughput of the G5.

AFR. Can you say more about floating point?

Sean: Floating point has been a big area of tuning for us. Although the various chip architectures seem to be close and constantly swapping their [performance] leads in floating point speed, we saw consistently slower performance starting on the Motorola G3 compared to Intel chips. When the well-known, publicized benchmarks are run, they usually have compiler flags set to compile code that puts floating point values into registers and on the stack on a way that is optimal for that chip.

AFR. So as many say, benchmark suites have compiler tuning advantages that don't translate to real world performance.

Sean: In the real world of legacy applications and cross-platform support working applications often can't have their code optimized as much. We found that when values were not optimally aligned, floating point calculations on the G3 could take 80 times longer than aligned ones. This compares to a three to four times speed hit on Intel chips. We've done a lot of tuning in this area, but it is a possibility that the G5 does not suffer the same problem.

1.8Ghz G5 Power Mac is 50 percent faster than a 2.8Ghz Intel Xeon workstation at Rendering VectorWorks performance.

AFR. Looking at the scores the single G5 turned in a combined total score of 240 compared to the 2.8Ghz Intel Xeon processor's score of 306.3 (lower is better); that's a 28 percent faster difference. That's roughly equal or better than the type of processor upgrade improvement that typically comes when a company like Intel introduces a new processor architecture or moves to a smaller manufacturing process. In other words, it's 'upgrade time'. Does an IT manager not have an excellent case for recommending these systems as the next VectorWorks workstation? Wouldn't that kind of improvement warrant a substantial improvement in CAD productivity?

Sean: Individual users need to assess the performance increase based on how they use it; just adding up our performance indices is not an accurate measure of the performance gain to a customer. We try to measure some of the more critical speed areas in VectorWorks, but there are many we don't measure and only the user can weigh them based on their value. It is also important to note that if you take any session of use for an application like VectorWorks, that nearly 90 percent of the total time measured is spent waiting for user input.

That being said, I think the G5 to G4 comparison shows a performance increase far beyond the normal incremental chip upgrade and is a high value for the money spent. I don't think we've seen a performance jump like this since the PowerPC replaced the Motorola 68040 chip.

AFR. Thanks for the info Sean.

Sean: You are welcome.

So The Overall Test Summary (combining the 2D Redraw test, the Rendering test and the Solid Modeling test) is graphed below (in seconds). As you can see both the dual 2.0Ghz G5 and the single 1.8Ghz G5 were substantially faster than the workstation-grade Intel Xeon machine at 2.8Ghz. (respectively 47 percent and 27 percent faster).

Quick Facts: Nemetschek North America is a cross-platform CAD developer (VectorWorks) with approximately equal share of Macintosh and Windows users worldwide. The firm develops VectorWorks and the VectorWorks industry series applications using Microsoft's developer tools for the Windows version and Metrowerk's PowerPlant for the Macintosh version. Both versions are built on top of the same exact code base written in C++ with additional functionality built using the application's VectorScript scripting language.