|Contents||Bulletin||Scripting in shell and Perl||Network troubleshooting||History||Humor|
Low end servers are one socket server that are actually, not that different from desktops or laptops. That means that they can have up to eight cores (12 if AMD is used) which actually makes them equal to high end server of ten years ago.
As the line between low end servers and desktops is fuzzy some companies use desktops instead. Google is a classic example of this approach. Now this approach is even more viable as you can buy desktop with solid state drive and put additional storage, if necessary, on eSCSI connected array of disks.
In this case reliability is achieved by prompt replacement of a failed server with identical and the cost is lower (often substantially lower). Also you can use then without maintenance contract as replacement cost is sufficiently low. This is probably the most price/performance efficient way to deploy low end server those days, but few large corporation use it. Conservatism is one reason.
Fashion also play role. Despite pretty high costs, the fastest growing segment of low end enterprise server market are blades. It is important to understand that they are mixed blessing as in many blade architectures the cost of chassis is substantial and nullifies savings from the deployment of blades. Also hardware is often half-baked and reliability is much lower that for equivalent number of 1U servers. Additional class of errors connected with interface between blade and enclosure is actually making them a very risky proposition in the datacenter without 24 x 7 people present as often you need physically delete and reinsert the blade to reboot it successfully.
They are marginally more electrically efficient as power supplies are shared between blades which is impossible in 1U servers or desktops.
Solaris disappeared from low end after Oracle acquisition so that this not much to talk about it. Also RISK CPUs were smashed by Intel (starting from 2006 Sun almost completely disappeared from SpecCPU tables., and Intel version of Solaris was rather late and failed to attract enough following. Oracle stopped this wasteful practice completely.
So linux is the king of the hill in this segment. Essentially hardware-wise linux rides Windows bandwagon and it is Windows, not other Unixes that is Linux major competitor on this segment.
That does not mean that Linux is cheaper, actually commercial distribution are very expensive for this segment. They just hide the total cost of ownership behind annual maintenance fees. That why Windows 2003 and Windows 7 server managed to get some previous Linux gains back. Annual subscription from Red Hat is typically $350 a year (patches only) or $750 (patches and phone support). Of course, you can use CentOS or Academic OS but that path, while perfectly OK for research labs and small startups, is not for large corporations.
As an OS, Linux more quickly improves on low end as low-end servers are its core specialization and is the area where it achieved significant penetration. All four major low end enterprise class servers vendors (Dell, HP, IBM and CISCO) ships Linux preinstalled on their servers. For the last 5 years Linux peneration dramatically increased on low end and now in many datacenters Linux represent a dominant bland of Unix on the low end. Its far from perfect and its perma bata status did not change, nut it baceme tolerable and reliability baceme acceptatble.
As hardware can be overspecified it can actually demostrate reliability euql to Solaris SPARC hardware or IBM Power hardware. Input output stack remained messed but it works.
I would say that as of 2007 Linux dominates key applications areas (web-servers, departmental file servers, DNS and mail servers, etc). As version 2.6 of Linux kernel matures, its a adequate, twenty years old OS, that can run major enterprise application such as databases, mail, DNS, etc perfectly well.
Linux completely dominates Web ISP field (and almost completely killed FreeBSD in this segment). Short lived attempt of Solaris 10 to get some traction in this territory is history. Nothing happened. MySQL acquisition and release of Solaris based AMP stack did not help. Those ISPs which tried Solaris 10 model quickly folded.
Moreover, Oracle acquisition of Sun helped to derail Sun low end server market share and essentially conceded the field to Linux (which Oracle also resells in the form of Oracle Linux -- a clone of Red Hat which can be run with a different, in Oracle opinion better debugged and tuned, kernel).
Please note that the validity of price-related arguments below is limited to mainland USA. For other regions prices can be very different. For example some software companies (Microsoft is one example) often charge higher prices in Eastern Europe in comparison with mainland USA where their prices are very reasonable (price discrimination under the disguise of compensating for piracy ).
It's important to understand that on low-end servers hardware prices and hardware compatibility list greatly influence competitiveness of the OS: hardware prices and the hardware compatibility list and reliability are as important as OS prices, support prices and reliability. So our discussion will be fuzzy here: we will jump from OS to hardware and hardware compatibility issues multiple times without any warnings.
For example, in the past (before Oracle acquisition) Sun practiced rather questionable policy of supplying servers mostly with 80G drives (probably in a hope that customers will buy NAS for anything larger then 80G). That was pretty irritating and on low end it limited capabilities of customers to effectively use Solaris with Sun's Intel servers -- by definition the most compatible and thoroughly tested Intel hardware for Solaris. The only way to avoid this Sun trap was to switch to HP. Several Proliant server were at the time Solaris-compatible (see HP.com - Solaris support on HP ProLiant Servers ). But HP as a company and HP hardware has its own set of problem. Ordering process is horrible, probably worse that with Sun when it was still an independent company. Servers are over-engineered and some software like ILO is not well debugged (Version of ILO 3.0 shipped with servers and blades from 2011 until 2013 (version 2.5) locked access completely on timeout, which was tremendously irritating). Support is not that great iether. Dell is a better, more customer oriented small server vendor and reliability of its server is in my opinion higher then HP servers which carry significant price premium.
Generally on low end you should not expect good customer service from vendors. Such an expectation is a fallacy. Margins are too thin. Many giant corporations moved its low end customer service to bored, ill-treated, underpaid people in countries like India who are desperate to move on from customer support to better jobs. Dell predominantly does direct sales and among four major low end server vendors (Dell, IBM, HP, and, in the past, Sun, now Cisco) they are the only one with usable Web site, where you can price your order without speaking to customer rep. Also in case of Dell any large company usually already have a rep to talk to and he is ready to discuss detail of the quotes that you already got on the WEB using their small business division and get large company discounts that you particular company is entitled due to volume agreements with Dell. They are good partners even if you are buying is one server.
Generally comparing OSes on low end servers the following factors look as the most important:
Returning to OS comparison, linux is attractive on low end, but not due to its low cost (for large enterprise this is an urban myth, as we will show below) and unsubstantiated claims like better security or, god forbid, better kernel quality. The key is application support (especially open source applications support) and drivers availability. In essence linux is Microsoft Windows of Unix: popular OS of average quality running on the most popular hardware and the most price effective platform in existence and as such it is and always will be the most insecure flavor of Unix in existence. Like in workstations Linux owes much of its success on low end to Microsoft as it is Microsoft that is owner of PC workstations and Intel servers standards, the standard that it releases for free to all major manufactures. Sun with its UltraSparc line (much like IBM with Power5 line) is more like an icebreaker that need to charter its own course in cold waters.
For many open source applications (as well as some commercial applications) as it is not the OS itself but the applications that determine the attraction of linux or Solaris for large enterprise customers. Here linux (actually only Red Hat) is definitely preferable as the number of supported precompiled open source applications in Solaris while reasonable and includes all major application is still a tiny fraction of packages supported for Red Hat.
Large corporation typically over specify low end servers. Even when they are buying blades. Also they are often afraid to move application to other OS. I know many cases where enterprise upgraded their server not to Intel but to outdated and very expensive HP boxes (one example of such replacement is the replacement of multiple boxes with new HP servers that have 550MHz PA-RISC CPU, two 73G drives and 6G of memory. Each costs over $6K each; note that HP charges for CD-ROM for such a box $500 -- the same price that a low end server from Dell would cost). Often the environment is so conservative that migration to more modern systems can be done only after big corporate earthquakes like mergers and acquisitions. Also realistically not all servers need 3.2GHz CPUs with four of six cores or a couple of them. Many can work well with much less horsepower if they have adequate amount of memory installed.
Servers from Dell and HP comes with 3 year manufacturer warrantee which can be expended to five years. As Five years is a typical depreciation schedule it is fair to say that hardware support is effectively outsourced to vendors. In case of HP there are some games with resellers that you ca play but generally they do not save money or save money while making support more cumbersome.
If we assume that a typical low end server consumers 300 watts (of them only 120 watts for CPU) and cost of electricity is 10 cents per kilowatt/hour then annual consumption of electricity is approximately $262 per server. For 1U servers in five years it can exceed the cost of hardware. For 2U servers (assuming $5000 price tag) it's more then 20% of cost of hardware.
I would like to stress it again that for a low-end server the flexibility of configuration (for example the ability to install wide range of hard drives on the server) is of paramount importance.
This is one of the reasons why blades picked up their presence in corporate environment for the last five years.
Support costs significantly influence the total cost of ownership of low end servers. For example for a typical lifespan of low-end server (five years) the cost of support $750 per year is equal to $3750 or more then a halve of the cost of the $5K server. That explains attractiveness of CentOS as an alternative to Red Hat for small business.
The licensing terms for Red Hat looks sounds really menacing for low end servers. Here is the relevant quote from the licensing agreement:
4. REPORTING AND AUDIT. If Customer wishes to increase the number of Installed System, then Customer will purchase from Red Hat additional Services for each additional Installed System. During the term of this Agreement and for one (1) year thereafter, Customer expressly grants to Red Hat the right to audit Customer's facilities and records from time to time in order to verify Customer's compliance with the terms and conditions of this Agreement. Any such audit shall only take place during Customer's normal business hours and upon no less than ten (10) days prior written notice from Red Hat. Red Hat shall conduct no more than one such audit in any twelve-month period except for the express purpose of assuring compliance by Customer where non-compliance has been established in a prior audit. Red Hat shall give Customer written notice of any non-compliance, and if a payment deficiency exists, then Customer shall have fifteen (15) days from the date of such notice to make payment to Red Hat for any payment deficiency. The amount of the payment deficiency will be determined by multiplying the number of underreported Installed Systems or Services by the annual fee for such item. If Customer is found to have underreported the number of Installed Systems or amount of Services by more than five percent (5%), Customer shall, in addition to the annual fee for such item, pay liquidated damages equal to twenty percent (20%) of the underreported fees for loss of income and administration costs suffered by Red Hat as a result.
If your organization wants to run any of the RHEL-certified enterprise applications and receive support from the software vendors you have two choices: to buy it from Red Hat or to buy it from Oracle. In this sense RHEL is a regular proprietary OS with rather strict licensing and all PR ink about "evil proprietary Microsoft" and "open source leader" (with Windows being cheaper then Red Hat if you factor in the support costs :-) should probably be spend for more noble purposes.
Red Hat business is based on the fact that GPL permits arbitrary restrictions on binary distribution rights as long as source code is available. In this sense Red Hat is a regular OS vendor that provides access to its binaries for a price. The only unique feature is that due to GPL it is suffering from clones due to availability of source (Oracle and CentOS are two most prominent examples).
You can read more about my views on GPL licensing and potential problems with it in ebook Labyrinth of Software Freedom (BSD vs GPL and social aspects of free licensing debate).
Taking into account that for each instance of the server you need a valid license and annual support costs are integral part of the cost of ownership. Here are approximate annual costs [RedHat2006]:
Enterprise Linux ES
|Price Per System||$799||$349||1299|
As we can see as an OS RHEL does not has cost advantages in comparison with Solaris 10. It has larger presence in this segment and complete more like Microsoft of Linux.
Note: if you buy hardware from Dell you will get 50% discount for RHEL standard edition and that makes RHEL quote competitive with Solaris at least for the initial three years of service covered by the subscription. Both Red Hat and Solaris charge for each additional CPU (so multicore chips benefit from this treatment). Solaris scale is flat. Red Hat doubles the support costs after two CPUs (you need AS version). That makes service plans for Red Hat competitive with Sun only on low end (up to two CPUs).
Intel decimated all RISK manufactures. On low end this is the end of story. If you look at Spec CPU benchmark the impression is that only IBM can still (barely) float of the surface. All other risk manufactures including Sun UltraSparc already sank. So the key question now becomes whether our can get more hardware for the same price from Dell or HP or Cisco. My bet would be on Dell.
Sun has basic remote management capabilities but they exist on console level. But Dell and HP servers provide VNC-based GUI interface to console. Althouth they are very similar Dell DRAC look more capable and less buggy then ILO.
DRAC firmware has an embedded Web server so that you can connect to it from a any PC that has a Web browser. Web browser interface works very similar to VNC but currently in working with linux it has problems with mouse: it is quirky and not very pleasant to use. Also unlike working with Windows you see two mouse pointers: one for "external world" and the other for "internal world" (if you connect to Windows server those two mouses are merge into one all the time). That creates quite a lot of confusion as moving "wrong cursor" and clicking on it can result in unpredictable behavior if "right cursor" accidentally also was position of some kind of button or menu.
But to be fair you need to understand that this is a recovery console, not your every day tool (althouth it can be used as such); so standards are more relaxed here and the fact that it works with GUI is a significant advantage. It, for example, greatly simplifies reconfiguration of network after server move as you can use RHEL GUI widgets for that (much like in Windows) and do not need to remember all the intricacies of linux command line commands like ifconfig that are slightly different from classic Solaris version (FSF behaves a lot like Microsoft in the area preaching a variant of "divide and conquer" theme for software world -- embrace and extend; see GNU echo parody).
Dell also have integrated deployment and change management solution called Dell OpenManage 5.0. With DRAC5 it delivers comprehensive hardware deployment, monitoring and management with features including [Itreseller2006]:
Integrated hardware and software management with partner applications like Altiris Server Management Suite, Microsoft SMS and MOM, and Novell ZENworks Linux Management - Dell Edition, providing single console management of enterprise resources
Dell OpenManage Connections, which allows comprehensive hardware monitoring within industry management frameworks
IMPI 2.0 and SMASH, which deliver simplified management capabilities today and in the future via a standardized management infrastructure
Improved Linux Integration with full support for Novell SuSE Linux SLES9
Improved integration between Dell OpenManage and VMware ESX 3.0, allowing simplified monitoring for customers' hardware devices dedicated to virtual machines.
There are still advanced administrative technologies that are more commonly used in large corporate IT then in startups (outside ISPs). One of them is custom automated installation. This area is represented by Jumpstart in Solaris and its Red Hat clone Kickstart. The latter was first released with Red Hat Linux version 5 in 1998 (not documented). Later similar scheme was also implemented in Suse (autoyast).
Both Kickstart and Autoyast permit automation of key elements of installation process including but not limited to:
Solaris Jumpstart has also a variant called webstart in which you can create an image of the system and them replicate it on as many systems as you wish, introducing customarization on the level of already fully configured system. The image of the system is called flash archive and it can be installed. Flash archives are a very important productivity tool and can save 80% of the time in replication of complex server setups to remote locations.
Kickstart has similar capabilities, so in this area two OSes are even.
All server from major manufactures are Linux compatible, but quality of drivers varies. HP provides its own optimized for Proliant set of drivers, but to what extent they are better is unclear. Dell prefers to work directly with vendors of linux distributions such as Red Hat and Novell.
Mouse support is linux's weak spot and it hurts in enterprise environment. It often freezes on KVM switches (even expensive corporate-level switches with full emulation like Avocent). Recently it became better. As most large enterprise buy servers with remote control (Drac/ILO) this is not a critical deficiency.
Hot swappable disk are only marginally useful on low end servers. This is not a critical feature for low end servers as replacement of the whole servers is matter of minutes if not seconds. Still for certain types of servers (for example DNS, Sendmail, etc) it has value. But large enterprises IT brass often make this feature "enterprise standard", demand it on all low end server that they buy and as a result overpay. That's a typical behavior of large enterprise IT and we can do nothing about it. Sums lost are not that big (much less then stupid over-deployment of SAN that is real plague of large IT environments).
There are only two players in this race AMD and Intel. Until mid 2006 release of Intel dual core CPUs (51xxx series) Opteron was the king of the hill in raw CPU efficiency. After that leader changed and Intel is the leader.
Anyway this race for faster CPUs between Intel and AMD goes for a long time. Current Intel position is strong but everything can happen. Situation can reverse again.
UltraSparc disappeared from SpecCPU results long ego so now the only battle is between AMD and Inter, the battle that AMD is losing. But on low end AMD is still is force to recon with. They have close scores on most computational benchmarks. Large differences are visible only in more complex benchmarks, for example in transactional benchmarks.
As for computational benchmarks such as SpecCPU situation is not that good for AMD. Transactional benchmarks are better. SPECweb2005 which is one of the most relevant benchmarks for low end servers now we will provide some information about benchmark itself:
SPECweb2005 is the next-generation SPEC benchmark for evaluating the performance of World Wide Web Servers. As the successor to SPECweb99 and SPECweb99_SSL, SPECweb2005 continues the SPEC tradition of giving Web users the most objective and representative benchmark for measuring a system's ability to act as a web server. In response to rapidly advancing Web technology, the SPECweb2005 benchmark includes many sophisticated and state-of-the-art enhancements to meet the modern demands of Web users of today and tomorrow:
- Measures simultaneous user sessions
- Relevant dynamic content: PHP and JSP implementations included
- Page images are requested using 2 parallel HTTP connections
- Multiple, standardized workloads: Banking (HTTPS), E-commerce (HTTP and HTTPS), and Support (HTTP); agreed to by major players in the WWW market
- Simulates browser caching effects by using If-Modified-Since requests
- File accesses more closely matching today's real-world web server access patterns
- Full disclosures available on this web site
- Stable implementation with no incomparable versions
- Java-based client for cleaner, more portable code
In terms of SpecCPU for all RISK CPU vendors this situation looks like uphill battle. And not only for Sun, but also for IBM with its "last hurrah": recent Power push. Big Blue is spending billions to upgrade its chip plants -- and getting thrashed by rivals that were manufacturing CPUs at much higher volumes and offering bargain-basement prices.
Power efficiency of low endse3rvers is an interesting topic. For example one problem with replacing one powerful server with several smaller one (path often advocated by linux enthusiasts) creates energy consumption problem as efficiency of converters from alternative current to DC in small servers and mid range servers is pretty low. DC power supplies typically have only 70% efficiency (50% on low loads). So additional costs of air-conditioning of a tropical landscape of a typical server room with servers instead of palms are substantial. It also limits expandability and creates the necessity of more expensive and more powerful power feeds. I know one company that instead of moving datacenter from one building to another when it moved to a small headquarters retrofitted additional (former factory) building as cost of providing additional power feed and air-conditioning in smaller building would eat all savings from the move from larger headquarters to smaller.
Here blades can help as they centralize power generation and have some energy optimization circuitry.
With relentless growth of the number of servers in the datacenter, energy efficiency is especially acute problem for those with datacenters that are located in high rise buildings. The latter usually have limitations on both power and air conditioning expandability. A typical Intel 51xx dual-core processor server now matches in power consumption both Opteron and UltraSparc. Previous Sun energy-consumption advantage when Intel consumed around 300 watts, which requires additional 50% (~150) watts of power for air conditioning disappeared in mid 2006.
In any case due to low efficiency of power supplies using low end servers you need to pay approximately 0.3 KWH per server. With a hundred servers (typical large datacenter figure) that creates power consumption of 30 KWH. Assuming the cost of generation $0.05 per KWH and the cost of delivery $0.04 per KWH the annual cost of energy per server is approximately $250, the figure which represents a significant share of maintenance costs.
Energy efficiency presents a formidable problem for Goggle which is a poster child of using low end linux server farms. Over the last three generations of Google's computing infrastructure, performance has nearly doubled. But because performance per watt remained nearly unchanged, that means electricity consumption has also almost doubled. That is a typical problem for any firm that tries to building massive server clusters on commodity hardware. The hardware is cheap, but it is designed to solve a different problem, powering a box in a desktop environment. As Google vice president of operations Urs Hoelzle noted "'Over four years, the power costs of running a PC can add up to half of the hardware cost". If we assume that price of electricity can double like the price of gas did, the four-year cost of a server's electricity bill will soon be larger than the $3,000 initial price of a typical low-end server with x86 processors.
Google try to fight the problem using more efficient (and more expensive) power supplies and to work with component makers to accelerate the time-to-market of more efficient devices, such as motherboards with a smaller number of DC voltage inputs [TG Daily]. "Saving power is still the name of the game, even to the extent that we shut off the lights in them when no-one is there" noted Hoelzle.
Xen, more powerful and elegant para-virtualization engine for X86
architecture developed at Cambridge. It provides capability
similar to VMware but with overhead more similar to Solaris Zones
overhead. The key advantage of VMware is dramatic simplification
of system-to-system migration path which is a very important for low end
servers. Xen has similar capabilities: copy the boot drive using
dd command to another server, point Xen
at that file, and boot the VM. It also provides the capabilities for
migrating a running VM from one physical server to another "on the fly",
the key attraction of VMware Virtual Center. In both cases most hardware
differences are resolved on virtualization level.
As for images Solaris has an excellent tool called flash archives. Zones also provide rudimentary "image swapping" capabilities. Also like linux vendors Sun is working on integrating Xen in Solaris so it will be the second virtualization capability available for the OS. Sun will release working Xen support code in the second half of 2006. This code will give OpenSolaris the ability to run on Xen as a Dom0 (host system), with support for 32-bit and 64-bit guests (DomU). Sun plans to integrate XEN into regular releases of Solaris 10 in 2007.
Linux has important home field advantage for LAMP stack. At the same time the lion share of testing and almost all documentation for major open source applications including databases is currently limited to 32-bit packages. Due to this reason, sometimes Sun supplies 32-bit packages of LAMP stack even on 64-bit UltraSparc. For example, in CSQamp package (a precompiled with Studio 11 LAMP stack) which is a part of Cool Tools set of optimized for T1 architecture packages, PHP and MySQL are included as 32-bit applications (CSQamp package will work with the CSQmysql -- 64-bit MySQL database server):
Currently only Solaris on Intel is competitive in speed with Linux running LAMP stack. T2000 is also a contender but it is more expensive.
See Blades (discussed in Virtual Servers and VMware Fallacies part of the paper)
Groupthink : Understanding Micromanagers and Control Freaks : Toxic Managers : Bureaucracies : Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Two Party System as Polyarchy : Neoliberalism : The Iron Law of Oligarchy : Libertarian Philosophy
Skeptical Finance : John Kenneth Galbraith : Keynes : George Carlin : Skeptics : Propaganda : SE quotes : Language Design and Programming Quotes : Random IT-related quotes : Oscar Wilde : Talleyrand : Somerset Maugham : War and Peace : Marcus Aurelius : Eric Hoffer : Kurt Vonnegut : Otto Von Bismarck : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce : Oscar Wilde : Bernard Shaw : Mark Twain Quotes
Vol 26, No.1 (January, 2013) Object-Oriented Cult : Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks: The efficient markets hypothesis : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Vol 23, No.10 (October, 2011) An observation about corporate security departments : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law
Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds : Larry Wall : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS : Programming Languages History : PL/1 : Simula 67 : C : History of GCC development : Scripting Languages : Perl history : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history
The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month : How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Haterís Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite
Most popular humor pages:
Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor
The Last but not Least
|You can use PayPal to make a contribution, supporting hosting of this site with different providers to distribute and speed up access. Currently there are two functional mirrors: softpanorama.info (the fastest) and softpanorama.net.|
The statements, views and opinions presented on this web page are those of the author and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.
Last modified: February, 16, 2013