Softpanorama (slightly skeptical) Open Source Software Educational Society

May the source be with you, but remember the KISS principle ;-)

Solaris vs. Linux: Framework for the Comparison

by Dr Nikolai Bezroukov

5. Hardware: SPARC vs. X86

The strongest selling point of linux for enterprise customers is actually not the OS per se, but the Intel-based hardware on which linux is running. Sun is looking good here. Sun's response to dot com bubble burst was to invest in R&D while the market settled. Part of this research (and acquisitions) was to get the best Opteron based technology - and today they are very visible in the high end Opteron market ( Dell only recently (mid May, 2006) declared that they will sell Opteron systems too). 

But let's look at the situation from a computer science standpoint. In uniprocessor mode Intel (or Opteron CPUs, which are better Intel then Intel ;-) CPUs are faster then UltraSparc. But UltraSparc  is more fault-tolerant, better integrated with the rest of hardware and far more scalable. As John C. Shoemaker (Sun Microsystems's Executive Vice President and General Manager of Computer Systems from 1995 to 2002) recollected on his article A personal view of Sun Microsystems:

SPARC was never competitive. Its RISC architecture was an advantage in technical applications, and it made certain tradeoffs in input/output and cache sizes to achieve this targeted application success; however, it was not a competitive general purpose uni-processor, ever! It was always about the system for Sun, not the chip. The system targeted technical application. When we pushed to commercial markets, it was through the implementation of a highly scalable vertical architecture, taking full advantage of a well-aligned strategy with the software developers at Sun. They were, and still are, brilliant at creating an amazingly robust and highly scalable OS (Solaris). No competitor could come close to Sun. We were able to attack aggressively and take the commercial market with system price/performance leadership on an open platform, running basically all (well over 12,000) major commercial and technical industry applications.

At the point in time when we began to achieve significant wins in commercial account applications, we acquired Cray Business Systems Division from Silicon Graphics (SGI) in 1995. Cray had previously chosen SPARC/Solaris, on a level playing field basis, as their best bet for entry into the commercial market. Cray BSD brought a world-class technical team to Sun, one with specific expertise in data center class systems design and quality requirements. This acquisition brought Sun a breakthrough 64-processor product with dramatic industry leading reliability, availability, scalability, and cost performance. Intel, IBM, and others had better uni-processor performance but, because they did not have Solaris, no one could scale above three or four processor systems. Microsoft made many promises to customers about future scalability, but those promises turned into nothing but delays.

Linux is essentially an OS that flourishes only on Intel. It never achieved any significant success on RISC architecture although IBM now is trying to change this situation pushing Linux on Power5 and potentially cannibalizing its own AIX sales (at least on low level servers).  Actually running several competing with each other hardware and software offerings is nothing new for IBM.

If you look on the general quality of the servers, Sun is a solid engineering company and Sun hardware engineers has a small but devoted camp of followers even on Intel platform (with Opteron line of servers), to say nothing about UltraSparc enthusiasts.

5.1.  UltraSparc is an expensive but pretty cool CPU :-)

While an excellent CPU with advanced architecture, UltraSparc has higher price/performance ratio then either Intel or AMD Opteron CPU offerings. This situation improved with the introduction of T1 CPU[Wikipedia].  The are really open processors: on March 21, 2006, Sun made the UltraSPARC T1 source code available under the GPL.

The main advantages of UltraSparc are:

1. Energy efficiency: it consumes less energy then either Intel or Opteron and much less then IBM's Power5 CPUs.  This is probably not enough to make them noticeably more attractive for server farms in hosting environment where the margins are extremely thin and you need  many cheap units as well as the ability to upgrade them without incurring huge capital costs, but it is incentive for some large companies datacenters that run into air-conditioning or electricity supply limits to their growth as the move to a new building in such cases is a very expensive proposition.

2. Fault tolerance. Sun server hardware can do amazing things with faulty components calculating the number of faults pr 24 hours.  Almost any duplicate component can be switched off. Even low level Sun server like V240 can survive bad memory chips, one faulty CPU or one burned power supply. In some sense it is an cheap cluster. Sun servers also behave reasonably well (shut down gracefully and usually are able to be restarted) after overheating. 

3. Cleaner architecture. Being big Endean CPU with RISC instruction set provides some complier level advantages in comparison with convoluted instruction set of X86 line.

4. Availability of hardware-based stack protection. From the security standpoint UltraSparc permit implementation of stack overflow protection that currently makes Solaris an order of magnitude more secure then Linux. Solaris 10 on Opteron also has this advantage over Linux (in 64 bit edition. )

5. Partitioning. Similar to high-end Sun SMP systems, the UltraSPARC T1 can be partitioned under Solaris 10.  Thus, several cores can be partitioned for running a single or group of processes and/or threads, whilst the other cores deal with the rest of the processes on the system.

Now with 2 GHz UltraSparc CPUs from Fujitsu and T1 from Sun the difference in price/performance ratio is less, but still a for computationally extensive applications some gap still exists (for Web servers and other multi-threaded applications T1 may actually be faster).  You can also use Solaris 10 on Opteron or Intel, but you need to be careful with the compatibility list and probably would be better off buying hardware from Sun or HP: currently the level of support of  PC hardware in Solaris for Intel is far more limited that in Linux.  And that's not as bad as you might try think as quality of some drivers for Linux leaves much to be desired. At least when Sun does provide drivers it provide reliable drivers.  Still Sun slashed quite a lot in price in recent years and managed to dramatically improved performance of UltraSparc. Paul Murphy  stated this consideration the best in his March 2005 article "Face-off: SPARC/Solaris vs. Intel/Linux" [Murphy&Golden] :

Suppose, for example, that back in mid-2000, you installed a pair of Sun 450s running Oracle, each with two 327 GB T3 workgroup arrays, 4 CPUs at 450 MHz, and 8 GB of RAM. Taken together that might have cost you around $700,000 initially, and something like $2,900 a month for hardware support after the first year. Now, five years later, users are complaining about those servers' performance. Also, some of your colleagues are saying you'd get a six-month payout just on support costs by replacing this stuff with a couple of Dell boxes running Linux. They add that you'd get an enormous performance boost into the bargain.

Your colleagues are about half right on the money, absolutely right about the performance opportunity and dead wrong about the implicit exclusion of SPARC/Solaris from the operation of Moore's Law.

That dual 450/T3 combination gave you a good bang for the buck in its day. Do a detailed review of the options available to replace it today, and you'll find that vendor pricing relationships haven't changed much on, say, Dell's 6600 quad Xeons, Sun's V40z quad Opteron, IBM's newest PowerRISC 720, or Sun's 440 UltraSPARC.

Back in 2000, both Compaq and IBM had gear that could compete on performance, but not on price, with the Sun 450/T3 combination. Today that range is narrower with Sun, Dell and IBM all clustered around $40,000 for a single system capable of handling the entire load. Even so, the cheapest and fastest options are still from Sun.

The fastest option, built around Sun's four-way Opteron V40z, is also the cheapest from a hardware perspective. At about $38,000 inclusive, it should handle the load formerly shared by the two 450s while cutting response time in half. The most expensive option is probably the Dell 6600 (I could not get fully configured pricing on the IBM 720 from IBM's Web site) at about $45,300 inclusive.

The most reasonable option, the Sun 440, is neither the cheapest nor the fastest at about $41,000 and including a 3310 external disk array. It has, however, the benefit that you can put it in place without procedural, technical or software change.

But if we talk about openness, then SPARC architecture in Sun's systems is more entitled to be called an open architecture than is anything that comes from Intel's fabs. And it's a pure hypocrisy (similar to Linus Torvalds stance in Transmeta) to ignore this fact for any open source evangelist.

Intel's microprocessors are more of an industry commodity than a standard. The term standard should be reserved for specifications that have received the imprimatur of an officially recognized standards body such as ANSI, the IEEE, the ISO, the IETF, or the W3C. When a technical specification for a protocol, programming language or other technology receives the endorsement of one of these organizations, that specification is considered to be ratified as a standard.  From this point of view SPARC is the industry standard, Intel X86 is not and will not be in the foreseeable future.

Moreover architecturally UltraSparc is a very interesting microprocessor. And as we mentioned before it is a big Endean microprocessor, which is actually the only right way to build microprocessors :-). And as the amount of memory increases beyond 4G it has some advantages due to Alpha-style organization of memory bus. Old, 32 bit Intel servers scale very weakly beyond 4G of memory. UltraSparc scales to 64G without major problems. Interesting benchmarks for T1 can be found at  http://www.rz.rwth-aachen.de/computing/hpc/hw/niagara.php  and related slides http://www.rz.rwth-aachen.de/computing/hpc/hw/niagara/Niagara_ZKI_2006-03-31_anMey.pdf

And the ability to access huge amount of memory is more important then raw speed in many typical enterprise applications. If we are taking about memory intensive applications like databases doubling the amount of RAM always wins doubling of the number of CPUs.  Most organizations usually overpay CPUs and underpay for memory when ordering new servers. There are just too many enterprise servers that are completely unbalanced CPU/memory wise and/or over-specified to the extent that they run with CPU utilization of less then 10%. Generally it usually does not make much sense to order less then 4G per CPU on a typical transaction oriented servers. For example to order server with "top of the line" (at the time of writing) two 3.6GHz Intel X86-64 CPU and just 2G of memory while similar priced server with 3 GHz CPUs (which are twice cheaper then 3.6GHz CPU at the time of writing) and 8G of memory (that can be bought for a difference in CPU prices)  has much higher price performance ratio on all major transaction-oriented enterprise applications. If we are talking about Oracle and sizable databases, 16G of memory typically is more optimal then 4G and produce the speed-up that easily compensate much slower speed of CPU (or less number of CPUs).

Also if you think about it Intel 8086 design was extended for way too long...  AMD has done an excellent job of extending it to 64 bit but still old warts of initial 16 bit design are still present.

In is not accidental that in 2001 Moreover UltraSparc III  got prestigious Analysts Choice Award for Best Server/Workstation Processor by Microprocessor Reports):

PALO ALTO, CA -- January 22, 2001 -- Sun Microsystems, Inc. (Nasdaq: SUNW) today announced that its 64-bit UltraSPARC[tm] III processor received MicroDesign Resources (MDR) Microprocessor Report 2001 Analysts Choice Award in the Best Server/Workstation processor category. Sun Microsystems accepted the award at MDRs second annual awards ceremony, Thursday, Jan. 18 at the Hyatt Saint Claire in San Jose, Calif.

The MDR awards confirm the top technology picks from the Microprocessor Reports analyst team, recognizing excellence in technology innovation, design and implementation. The Sun Microsystems, Inc. UltraSparc III processor won first place for Best Server/Workstation Processor in a competitive field of nominees which also included Intel's Itanium, IBM's Freeway for eServer z900, and Intel's Pentium III Xeon processors.

The prestigious MDR awards celebrate the companies and products that shaped the electronics industry in 2000. Microprocessors for servers and workstations do the heavy lifting in this industry. The immense processing requirements placed on this class of processors require them to incorporate many advanced features, said Max Baron, editor-in-chief and principal analyst, Microprocessor Report. Because of its advanced multiprocessing architecture, we gave the Sun UltraSparc III processor the Microprocessor Report 2001 Analysts Choice Award for Best Server/Workstation Processor.

Partially due to UltraSparc CPU architecture Solaris on UltraSparc scales very well on multi-CPU machines, compared to Linux. After 8 CPUs the difference in quality of SMP implementation became really noticeable. Also you can add the advantages of Solaris 10 being a 64 bit OS, with large max file sizes and the ability to access huge amount of RAM without any special hacks, then the advantage of Solaris for mid-range Opteron servers became more prominent. Again I am not talking here about running workstation loads or Web servers, I am talking about typical mid-range servers loads in large enterprises, typically SAP and database related loads.

On a negative side Solaris 10 on UltraSparc suffers from lower performance of classic UltraSparc CPUs in comparison with Intel and AMD CPUs. Only new T1 chips from Sun are competitive on transactional benchmarks (as they have 6 or 8 cores). For all major practically important enterprise applications T1 based computers and high end Fujitsu UltraSparc compatible CPUs are the only one  competi tive with Opteron. On SPECint2000/SPECfp2000 benchmarks only Fujitsu UltraSparc compatible CPUs are competitive (T1 consists of several relatively slow cores  - 1GHz each -- so without multithreading it provides less advantages).  That puts Sun in some disadvantage on the low and midrange servers (up to 4 CPUs). 

If we are talking about workstation Sun generally lost an edge here. Still hardware-wise UltraSparc IIe and III are one of the cheapest 64 bit RISC CPU available (along with Alpha). But Sun desktop offerings are unique in a sense that they are the only one that permit using all Windows application in Solaris environment via so called PC-card (called SunPCi IIIpro card) - a fully functional PC on the card that can use the same partitions as Solaris (shared disks).  It's rather expensive for individuals but OK for companies. That's why UltraSparc workstation (for example $995 Sunblade150 ) with  $495 SunPCi IIIpro card represent such an attractive offering for consultants and small teams who need maximum productivity from each developer while preserving reasonably low cost of hardware per developer. In addition to its own Ethernet port, the card also supports the sharing of Sun system's Ethernet ports for increased networking flexibility. It also has three USB 2.0 ports and a 1394a port, as well as serial and parallel ports, provide support for PC-standard I/O devices such as digital media devices, keyboards, mice, storage, and printers.

This offering which was never properly marketed by Sun makes Solaris Sparc workstations one of the most interesting Unix software development environments in existence providing a unique level of PC integration with the Solaris, including shared files,  monitor, and keyboard (as well as CD-ROM and floppy drive). I personally have been never able to achieve on Unix the same level of productivity as I can achieve using Windows based editor and I definitely prefer Windows version of SlickEdit to Solaris version. PC launcher 1.0 lets SunPCi users get seamless access and power to view, edit, and print many popular types of PC files by automatically launching the associated Windows application from Solaris. By incorporating PC launcher into the Solaris CDE desktop, users can share files created by Microsoft Office. Although you can accomplish many of those tasks with StarOffice and good X emulator using NFS or Samba as common filesystem, this environment is as close to ideal PC VM running under Solaris as one can get. Of course the fashion now is to use tandem Windows and linux or Windows and Solaris under VMware (which is really an excellent environment for a laptop) but an extra real CPU is always extra real CPU :-).  

5.2. Solaris on Opteron

Now let's briefly disuses Sun Opteron line of servers. Here the price advantage of Dell is much less that on UltraSpeac and  for two CPUs servers Sun competes with Dell (and HP) neck-to-neck both in the price and support costs. Moreover you can buy "bare-bone" Opteron server (for example bare bone X2100 with one Opteron 146 2.0GHz CPU and 512M of memory costs less then $800) and install additional memory and harddrive yourself saving some percentage of the cost (sometimes one third).  Now note that Solaris 10 is cheaper then Suse (Red Hat Enterprise is ridiculously expensive) and support contact is cheaper too. It is interesting to note that many Opteron-based servers from HP are Solaris 10 compatible. HP's support matrix for 64-bit Solaris 10 can be viewed here.  In Feb 2, 2006 article Register published an interesting article HP confirms plan to attack Sun via Solaris  which stated the following:

HP has long "officially" supported various versions of Solaris on its Xeon- and Opteron-based servers. Now, however, it's kind of ready to talk about this support. The company this week "announced" support of Sun's version of Unix in a statement to staff, according to insiders.

... ... ...

"Enabling 64-bit Solaris 10 on Opteron-based ProLiant servers is an extension of HP's Sun Attack program - a way to provide a solution for customers who are interested in moving from Sun/Solaris to an Industry-Standard HP solution."

... ... ...

Since reviving Solaris x86, Sun has now managed to push HP, IBM and Dell into admitting that they will sell the OS if customers ask for it.

HP has long been the most able Solaris x86 supporter. This is largely a result of a decent sized Solaris x86 business at Compaq. IBM follows with its recent agreement to let Sun service Solaris running on Big Blue's blades. Dell only ships Solaris when customers order a ton of servers and demand the OS.

The impressive Solaris x86 download figures quoted by Sun and buzz around Open Solaris seem to center on academic users and hobbyists. This makes it difficult to tell how many large customers have decided to run Solaris x86 in production. If you are such a customer, please let us know.

Created Jan 2, 2005.  Last modified: February 28, 2008