Softpanorama (slightly skeptical) Open Source Software Educational Society

May the source be with you, but remember the KISS principle ;-)

Fighting Raymondism:
Software Realism vs Software Idealism

Something is rotten in the state of Denmark

Shakespeare, Hamlet

Note: In the Switchboard below all links to additional pages are in bold, for example Donald Knuth is a link to a special page devoted to Professor Knuth, the author of TAOCP and TeX.

In the vision of Software Realists programmers like all humans have weaknesses and guided primarily by self-interest and as such requiring formal organization and incentives (direct or indirect) to act outside the limits defined by their own self-interest. Software Realists see the evils of the software world as given and derived from the limited and unhappy choices available, given the inherent moral and intellectual limitations of human beings and existing hardware.   Some of them try to create better software like programmers involved in creation of all BSD flavors but they consider commercial programmers as equals (actually many of them wear two hats) and do not object to reusing the code that they developed pro bono in proprietary software. 

In this slightly tragic worldview the software development is a hard and often underappreciated labor that requires special, pretty rare, talent  People with this talent like talented sport stars (for example tennis players or ice ballet dancers) risk their health while they are young creating short living but tremendously complex artifacts (large software systems are probably the most complex system even created by humans) and for this reason should be remunerated accordingly.  It is important to understand that like in case of artists creating paintings on the sand beach programmers creation are short-lived and the new wave of hardware often wipes them clean.  For example, who now remembers the creators of PL/1 optimizing and debugging compilers, the compilers that were real breakthrough in many areas of complier writing art and in comparison with which some modern compliers still look like junk despite the fact that they were written 30 years ago.

In other words they see that due to immense complexity of those artifacts all software sucks. It is just that some software sucks less. It can be proprietary software, it can be free software -- all depends of the talent of the creators not so much on a particular ideology (which, by the way, can be completely wrong: Soviets invented quite a few new technologies and managed to launch the first man into space).  Thus, in view of Software Realists school the perfection of software is unachievable and old good Unix with its classic codebase might sometimes be preferable to new Turks be it Windows or Linux. That does not mean that Linux or Windows codebase is all crap. That just means that they are just another OSes in a long historical line of such systems. In some areas they might be better then competition, in some worse, but none has the monopoly on innovation (actually Linux is a pretty conservative kernel despite a radical ideology behind it). 

Software Realists are unconvinced by claims of linux superiority and want to see hard facts.  Furthermore, history guided them that the proper tradeoffs between different subsystems of OSes can be ironed out only via long, expensive and painful process that requires strong highly paid managers, programmers and testers who are ready to sacrifices their health for the success of their creation. The real art requires sacrifices and it is better when such sacrifices are properly remunerated, although stores of talented artists who died in poverty are nothing new. 

Because real talents are rare good software is a very expensive thing.  Software realism school  presuppose that modern software is almost always a compromise between the demands of the talent and demands of the marketplace.

The Software Idealist school (both in its Stallmanism and Raymondism incarnations) holds that mankind in general and programmers in particular has not yet achieved their full moral potential, and that they are (at least in principle) perfectible if guided by wonderful new software development ideology.  Foolish and immoral choices inherent in the creation of proprietary software explains the all the evils of the existing software world  and revolution was needed and actually already  came in the form of either free software movement or its less pure form called open source software movement. Both major Software Idealism schools rely on volunteer labor of programmers connected via Internet to produce immortal gems of software wisdom that will crush proprietary software developers like cockroaches. 

In Software Idealism  worldview, whether purely moral incentive actually work in a long run or not and what will happen when Linus Torvalds will become yet another retired dot-com multimillionaire with his own yacht is irrelevant to the achievement of true software justice, justice for all.  This utopian view holds that  volunteer programmer potential is immense and can do everything including improving human nature that should get rid of those outdated  economic rewards for software development and be satisfied  working part time in McDonalds in order to be able to participate in the movement. So the Software Idealism vision promotes pursuit of the highest ideals which somehow guarantee the best software solutions. At the end of the day new liberated men should all storm this evil Bastille of software oppression which is of course Microsoft and dance on its ruins. Sometimes in their enthusiasm  they can attack other sinful old fashioned proprietary software vendors instead of  Microsoft.  Until opening Solaris (and even after that) sometimes their target was Sun.

And if the unwashed masses who corrupt their soils by using Windows are slow in catching on, then it is the role of the intellectual vanguard (the keepers of programming craft who in Eastern Europe might be called "programming intelligentsia") to lead them  - even if in the short run, the masses can be unhappy with the results because they might not be able to use full capabilities of their laptops, cameras or scanners.  In general Software Idealists think that higher considerations should guide us and that those consideration somehow guarantee creation of a better software, the software that is not only better but which is as perfect as it is free.

Following Orwell's advice this page views the Open Source Software (OSS) phenomena without rose-colored glasses. I am convinced that we need to understand both strong and weak points of OSS and the former is impossible without the latter. Both exists. 

The main point here is that the idea of sacrificing yourself to save humanity is very seductive to certain types of individuals. Probably instead of saving the world it is often wiser to learn to live in it. The latter is also more difficult.

ESR's paper does has historical value but it was mainly used as a "manifesto" used to teach "open source" converts how they should look at their place in the Movement. Specifically, ESR promotes the view that "open source" is a marvelous utopia. Like all the great utopias, it is free of personality conflicts and everyone freely works for the Common Good under the benevolent dictatorship of el Linusimo.

Paradoxically in "free vs open source discussion" I am on the RMS side (Stalmanism side, if you wish ;-) and I think that RMS is right by saying that he's not sure to what extent the Free Software is compatible with corporate desire for profit. It's much more straightforward and truthful to say it that way, rather than jump over the head trying to sell open source projects to the highest bidder as ESR attempts.

There is one terminological problem: some people (RMS is one example) distinguish free software ("free software"="GPL-based software" in RMS interpretation ;-) from Open Source (umbrella term that includes BSD license, Artistic license and LGPL), some do not. Open source is snappier, clearer, less ambiguous, and close enough to the same thing. As such it's preferable to the  99% of people. I know that RMS disagree, but so be it. And actually if you are semantic fundamentalist you can see the GPL has problems with coercing the word  "free" (that's why so much material on GNU site is devoted to it ;-).  BSD license is more free that GPL in both "free like in beer" and "free like in freedom" meanings of this word. 

The principal advantage of open source means that for simple programs the possibility of adapting program for your needs largely  compensates for the shortcomings of this program.  Of course you need to be a programmer to use this advantage, but the programming code is useful for adaptation only if it is really short and simple. You can convert any open source project into an analogy of closed source project just by overcomplicating the code base. That means that commercializing of open source ("Linus revolution") is internally contradictive undertaking as Red Hat behavior clearly demonstrates. As RMS said:

... I would choose a bare-bones unreliable free program rather than ... reliable proprietary program...

Again the key advantage of open source for me that "bare-bone" open source program does have additional value that might compensate for many other real or perceived faults. This opportunity is not automatic and to a large extent disappear with the growth of the size of the program. So KISS principle is of paramount importance for open source.

Again it's important to understand that the principal advantage of open source exists only up to certain amount of lines in a program. That's why scripting languages are so important and Perl, TCL, PHP and Python, not Linux can be considered to be flagships of open source. Linux is a pretty conservative reimplementation of Unix that introduced almost nothing new into operating system kernel design. And BTW Unix introduced at least seven: C language as system programming language, hierarchical filesystem, pipes and a set of pipes friendly utilities/filters, regular expressions, devices as files, shell as the mother of all modern scripting languages, first built-in TCP/IP stack). If one compares Linux with BE OS, Inferno or even with OS/2 and Amiga one can see that in major design decisions Linux is an extremely conservative OS. As Rob Pike noted in his "Systems Software Research is Irrelevant" (http://plan9.bell-labs.com/cm/cs/who/rob/utah2000.pdf) Linux can be considered as a sign that computer science research became irrelevant and he claimed that the whole situation in OS design is generally bad and requires action.

The second important point is that Raymondism statement that open-source software as a new economic force for producing software is inherently or inevitably superior to alternatives is plain vanilla Vulgar Marxism.  As I mentioned in My responce to the letter by Paolo Pumilia to the FM:

I would like to reiterate that ERS's views on the economic superiority of open source are close to vulgar Marxism with it's economic determinism. Contrary to your impression "vulgar Marxism " is a legitimate scientific term. As Professor Robert M. Young stated in his work "Marxism and the history of science" [see R.C. Olby, G.N. Cantor, J.R.R. Christie and M.J.S. Hodge (editors), Companion to the History of Modern Science. (1996), pp. 77-86.]:

"The defining feature of Marxist approaches to the history of science is that the history of scientific ideas, of research priorities, of concepts of nature and of the parameters of discoveries are all rooted in historical forces which are, in the last instance, socio-economic. There are variations in how literally this is taken and various Marxist-inspired and Marxist-related positions define the interrelations among science and other historical forces more or less loosely. There is a continuum of positions. The most orthodox provides one-to-one correlations between the socio-economic base and the intellectual superstructure. This is referred to as economism or vulgar Marxism."

All in all, I tried to communicating a more objective message that can mobilize developers by giving them a clear sense of what OSS is about, what are major pitfalls and difficulties and how to avoid them or at least lessen their influence on the project. I do not propose ready-made answers in the best CatB fairy-tale style, I have more questions that answers. Anyway, Brook's Law is not negated by a fairy-tail about a dog and pony show project (Fetchmail) by a true believer who cannot even understand that this project should use a scripting language instead of C [I assume you've heard of Brook's law].

My approach to researching this phenomena is to consider OSS as a vital part of Unix Renaissance with Linux as one of several free kernels (and like FreeBSD and Solaris kernels is too complex in the latest incarnations and far from being the best available Unix kernel; it's probably dead last in quality and architectural refinement among the three kernels mentioned above).

Special attention is devoted to the limitations of the OSS because I strongly believe that understanding limitations of OSS is probably one of the most important component of general understanding of OSS phenomena. For example I think that attempt to complete with NT in both client and server space for Linux will harm the kernel development and introduce the level of bloat that is comparable with Microsoft with corresponding problems in stability. Also contrary to CatB I think that there was a process of  switching of the  development of kernel from volunteers to paid developers, the process that started around 1997 and was mainly completed after 1999 Linux IPOs. I suspect that since Linus Torvalds became Transmeta  chief PR person it's simply incorrect to think about Linux kernel as a pure volunteer-driven project as CatB tries to present it. Similarly it's incorrect to think about ESR as a volunteer Linux evangelist since his election to the VA Linux board. Actually ESR is an extremely well paid Linux evangelist that contributed a lot to the Linux Gold Rush. And there were some Gold rush victims. Sound that this has nothing to do with open source? Read on. Here is the message  that I had found on the Linuxtoday forum about "absurdly rich" Eric Raymond:

I tend to think that 'Sudden Wealth Syndrome'  is applicable not only to children of rich (In working class neighborhoods, a kid might say "My daddy can beat up your daddy," but in the neighborhoods of "Sudden Wealth Syndrome" the brag is "My daddy can buy your daddy."). And cosmic valuations and consolidation frenzy is only one danger  to be aware of.  The second one is financial problems including shady accounting practices to justify them, no real business plan, etc. It's really sad, but greed is greed even if it's connected with open source. One can expect press releases like  "The company denied that it expects to find evidence of financial irregularities either in its revenue recognition or expenses". May be accompanied by resignation of CFO, CEO or both. Who just a little bit overstep regulations trying to turn a profit. The Me use Linux too IPO open sore Linus open ebiz ASP solutions satire aptly described crazy atmosphere of this gold rush bubble that partially was inspired by Raymondism (See Linus Midas Touch for more information):

The final destruction of what used to be a charming little OS scene arrived today, Monday, December 13, 1999.

Linuxtoday is spewing forth "me-use-linux-too-IPO-open-sore-Linus-open-ebiz-ASP-solutions" press releases from every backwater, buzzless Joe Q. Corp with a hotmail account...

OS figureheads are being courted for interviews with a veracity that is usually reserved only for pathological child molesters and internet CEOs...

Forty thousand "Embedded Internet eSolution Firewall Privacy Biz Remote" solutions are being deeply discounted to the five people who care enough to add one more yeahd00ditssecure.pl script to their boxes...

2-bit players are buying half-bit companies without a dime to their names just to get at the word Linux in their press releases...

Along with research materials and a collection of critical materials about the Cathedral and the Bazaar(CatB) I authored two papers on the subject. the first of them coined the term Raymondism as a negative phenomenon connected with naive, on the border of blind-folded chauvinism, view of OSS. I see it as an attempt to create an open source mythology and rally people around it (like semi organized high demand cult) that leads to the loss of credibility on the movement, and betrayal of trust of people who support it. See also What is Raymondism and Commentary to the First Paper on OSS problems.

Of course not everybody would agree with my views (actually ESR even reacted to my first paper in a rather revealing as for his own personality way)  but I think that my concerns are pretty legitimate to think about them even if you disagree. See also responses to the first FM paper   It's interesting to note that most responses are limited to just one day October 8, 1999. News last just one day in the atmosphere of information overload  ;-)

I understand perfectly well that it's easy to refute my statements citing ERS's other papers and he really revised some of his most utopian views expressed in CatB in his later papers. The problem with such defense is that ESR's views are pretty much opportunistic and eclectic. Therefore radically different approaches happily coexist in his papers and as any opportunist he switches between mutually incompatible views as a matter of convenience. For example he often advertise his closeness to anarcho-capitalism, ironically called "libertarianism" in the USA (an eclectic political movement often called anarchism for the rich). But in many papers including CatB he is much more close to the anarcho-communism -- a variety of pre-Marxist grassroot communist philosophy. For example the idea of  a gift economy that he advocated in CatB is extremely close to anarcho-communism and no amount of words or citations can refute this fact.

As J.Salinas put it in his response to LinuxWorld Eric Raymond's tips for effective open source advocacy

Surely, the real reason that Free Software took off in 1998 is that it had hit a critical mass, and re-labelling it as OpenSource probably made very little difference in practice.

The truth is that the GNU effort had been adopted by Unix users and, more importantly, by academic institutions, at an increasing rate during the late eighties and nineties, and this produced a critical mass among "those in the know" by 1997/98 that made it inevitable that Free Software would be noticed and make inroads.

Eric Raymond may want to have been more important, but really his attitudinising was just a component, albeit valued, in the inevitable flow of events, and not the decisive factor.

The take up owed everything to the CS students during the previous N years who had been taught on Free Software, and respected the model, pushed it, and sold it. ESR was just one of those people who pushed it around. Nobody where I worked had ever heard of him or of OpenSource. But they did know about Gnu and Emacs. These constituted the practical elements that made recognition of Free Software inevitable.

That's my theory, and I am sticking to it.

Nikolai Bezroukov

Notes: Those pages are written by people for whom English is not a native language. Some amount of grammar and spelling errors should be expected. This is a Spartan WHYFF (We Help You For Free) site. It cannot replace the best teachers and the best books. The site contain some obsolete pages as it develops like a living tree... Some links on older pages are broken. Please try to use Google, Open directory, etc. to find a replacement link (see HOWTO search the WEB for details). We would appreciate if you can mail us a correct link. Search Amazon by keywords:     Open directory Research Index

Old News ;-)

[Dec 11, 2003] ONLamp.com Myths Open Source Developers Tell Ourselves by chromatic

One persistent misfeature of open source development is thoughtless mimicry, copying the behaviors of other projects without considering if they work or if there are better options under the current circumstances. At best, these practices are conventional wisdom, things that everybody believes even if nobody really remembers why. At worst, they're lies we tell ourselves.

Perhaps "lies" is too strong a word. "Myths" is better; these ideas may not be true, but we don't intend to deceive ourselves. We may not even be dogmatic about them, either. Ask any experienced open source developer if his users really want to track the latest CVS sources. Chances are, he doesn't really believe that.

In practice, though, what we do is more important than what we say. Here's the problem. Many developers act as if these myths are true. Maybe it's time to reconsider our ideas about open source development. Are they true today? Were they ever true? Can we do better?

Some of these myths also apply to proprietary software development. Both proprietary and open models have much room to improve in reliability, accessibility of the codebase, and maturity of the development process. Other myths are specific to open source development, though most stem from treating code as the primary artifact of development (not binaries), not from any relative immaturity in its participants or practices.

Not every open source developer believes every one of these ideas, either. Many experienced coders already have good discipline and well-reasoned habits. The rest of us should learn from their example, understanding when and why certain practices work and don't work.

Publishing your Code Will Attract Many Skilled and Frequent Contributors

Myth: Publicly releasing open source code will attract flurries of patches and new contributors.

Reality: You'll be lucky to hear from people merely using your code, much less those interested in modifying it.

While user (and developer) feedback is an advantage of open source software, it's not required by most licenses, nor is it guaranteed by any social or technical means. When was the last time you reported a bug? When was the last time you tried to fix a bug? When was the last time you produced a patch? When was the last time you told a developer how her work solved your problem?

Related Reading Dancing Barefoot By Wil Wheaton

Some projects grow large and attract many developers. Many more projects have only a few developers. Most of the code in a given project comes from one or a few developers. That's not bad — most projects don't need to be huge to be successful — but it's worth keeping in mind.

The problem may be the definition of success. If your goal is to become famous, open source development probably isn't for you. If your goal is to become influential, open source development probably isn't for you. Those may both happen, but it's far more important to write and to share good software. Success is also hard to judge by other traditional means. It's difficult to count the number of people using your code, for example.

It's far more important to write and to share good software. Be content to produce a useful program of sufficiently high quality. Be happy to get a couple of patches now and then. Be proud if one or two developers join your project. There's your success.

This isn't a myth because it never happens. It's a myth because it doesn't happen as often as we'd like.

Feature Freezes Help Stability

Myth: Stopping new development for weeks or months to fix bugs is the best way to produce stable, polished software.

Reality: Stopping new development for awhile to find and fix unknown bugs is fine. That's only a part of writing good software.

The best way to write good software is not to write bugs in the first place. Several techniques can help, including test-driven development, code reviews, and working in small steps. All three ideas address the concept of managing technical debt: entropy increases, so take care of small problems before they grow large.

Think of your project as your home. If you put things back when you're done with them, take out the trash every few days, and generally keep things in order, it's easy to tidy up before having friends over. If you rush around in the hours before your party, you'll end up stuffing things in drawers and closets. That may work in the short term, but eventually you'll need something you stashed away. Good luck.

By avoiding bugs where possible, keeping the project clean and working as well as possible, and fixing things as you go, you'll make it easier for users to test your project. They'll probably find smaller bugs, as the big ones just won't be there. If you're lucky (the kind of luck attracted by clear-thinking and hard work), you'll pick up ideas for avoiding those bugs next time.

Another goal of feature freezes is to solicit feedback from a wider range of users, especially those who use the code in their own projects. This is a good practice. At best, only a portion of the intended users will participate. The only way to get feedback from your entire audience is to release your code so that it reaches as many of them as possible.

Many of the users you most want to test your code before an official release won't. The phrase "stable release" has special magic that "alpha," "beta," and "prelease" lack. The best way to get user feedback is to release your code in a stable form.

Make it easy to keep your software clean, stable, and releasable. Make it a priority to fix bugs as you find them. Seek feedback during development, but don't lose momentum for weeks on end as you try to convince everyone to switch gears from writing new code to finding and closing old bugs.

This isn't a myth because it's bad advice. It's only a myth because there's better advice.

The Best Way to Learn a Project is to Fix its Bugs and Read its Code

Myth: New developers interested in the project will best learn the project by fixing bugs and reading the source code.

Reality: Reading code is difficult. Fixing bugs is difficult and probably something you don't want to do anyway. While giving someone unglamorous work is a good way to test his dedication, it relies on unstructured learning by osmosis.

Learning a new project can be difficult. Given a huge archive of source code, where do you start? Do you just pick a corner and start reading? Do you fire up the debugger and step through? Do you search for strings seen in the user interface?

While there's no substitute for reading the code, using the code as your only guide to the project is like mapping the California coast one pebble at a time. Sure, you'll get a sense of all the details, but how will you tell one pebble from the next? It's possible to understand a project by working your way up from the details, but it's easier to understand how the individual pieces fit together if you've already seen them from ten thousand feet.

Writing any project over a few hundred lines of code means creating a vocabulary. Usually this is expressed through function and variable names. (Think of "interrupts," "pages," and "faults" in a kernel, for example.) Sometimes it takes the form of a larger metaphor. (Python's Twisted framework uses a sandwich metaphor.)

Your project needs an overview. This should describe your goals and offer enough of a roadmap so people know where development is headed. You may not be able to predict volunteer contributions (or even if you'll receive any), but you should have a rough idea of the features you've implemented, the features you want to implement, and the problems you've encountered along the way.

If you're writing automated tests as you go along (and you certainly should be), these tests can help make sense of the code. Customer tests, named appropriately, can provide conceptual links from working code to required features.

Keep your overview and your tests up-to-date, though. Outdated documentation can be better than no documentation, but misleading documentation is, at best, annoying and unpleasant.

This isn't a myth because reading the code and fixing bugs won't help people understand the project. It's a myth because the code is only an artifact of the project.

Packaging Doesn't Matter

Myth: Installation and configuration aren't as important as making the source available.

Reality: If it takes too much work just to get the software working, many people will silently quit.

Potential users become actual users through several steps. They hear about the project. Next, they find and download the software. Then they must brave the installation process. The easier it is to install your software, the sooner people can play with it. Conversely, the more difficult the installation, the more people will give up, often without giving you any feedback.

Granted, you may find people who struggle through the installation, report bugs, and even send in patches, but they're relatively few in number. (I once wrote an installation guide for a piece of open source software and then took a job working on the code several months later. Sometimes it's worth persisting.)

Difficulties often arise in two areas: managing dependencies and creating the initial configuration. For a good example of installation and customization, see Brian Ingerson's Kwiki. The amount of time he put into making installation easier has paid off by saving many users hours of customization. Those savings, in turn, have increased the number of people willing to continue using his code. It's so easy to use, why not set up a Kwiki for every good idea that comes along?

It's OK to expect that mostly programmers will use development tools and libraries. It's also OK to assume that people should skim the details in the README and INSTALL files before trying to build the code. If you can't easily build, test, and install your code on another machine, though, you have no business releasing it to other people.

It's not always possible, nor advisable, to avoid dependencies. Complex web applications likely require a database, a web server with special configurations (mod_perl, mod_php, mod_python, or a Java stack). Meta-distributions can help. Apache Toolbox can take out much of the pain of Apache configuration. Perl bundles can make it easier to install several CPAN modules. OS packages (RPMs, debs, ebuilds, ports, and packages) can help.

It takes time to make these bundles and you might not have the hardware, software, or time to write and test them on all possible combinations. That's understandable; source code is the real compatibility layer on the free Unix platforms anyway.

At a minimum, however, you should make your dependencies clear. Your configuration process should detect as many dependencies as possible without user input. It's OK to require more customization for more advanced features. However, users should be able to build and to install your software without having to dig through a manual or suck down the latest and greatest code from CVS for a dozen other projects.

This isn't a myth because people really believe software should be difficult to install. It's a myth because many projects don't make it easier to install.

It's Better to Start from Scratch

Myth: Bad or unappealing code or projects should be thrown away completely.

Reality: Solving the same simple problems again and again wastes time that could be applied to solving new, larger problems.

Writing maintainable code is important. Perhaps it's the most important practice of software development. It's secondary, though, to solving a problem. While you should strive for clean, well-tested, and well-designed code that's reasonably easy to modify as you add features, it's even more important that your code actually works.

Throwing away working code is usually a mistake. This applies to functions and libraries as well as entire programs. Sometimes it seems as if most of the effort in writing open source software goes to creating simple text editors, weblogs, and IRC clients that will never attract more than a handful of users.

Many codebases are hard to read. It's hard to justify throwing away the things the code does well, though. Software isn't physical — it's relatively easy to change, even at the design level. It's not a building, where deciding to build four stories instead of two means digging up the entire foundation and starting over. Chances are, you've already solved several problems that you'd need to rediscover, reconsider, re-code, and re-debug if you threw everything away.

Every new line of code you write has potential bugs. You will spend time debugging them. Though discipline (such as test-driven development, continual code review, and working in small steps) mitigates the effects, they don't compare in effectiveness to working on already-debugged, already-tested, and already-reviewed code.

Too much effort is spent rewriting the simple things and not enough effort is spent reusing existing code. That doesn't mean you have to put up with bad (or simply different) ideas in the existing code. Clean them up as you go along. It's usually faster to refine code into something great than to wait for it to spring fully formed and perfect from your mind.

This isn't a myth because rewriting bad code is wrong. It's a myth because it can be much easier to reuse and to refactor code than to replace it wholesale.

Programs Suck; Frameworks Rule!

Myth: It's better to provide a framework for lots of people to solve lots of problems than to solve only one problem well.

Reality: It's really hard to write a good framework unless you're already using it to solve at least one real problem.

Which is better, writing a library for one specific project or writing a library that lots of projects can use?

Software developers have long pursued abstraction and reuse. These twin goals have driven the adoption of structured programming, object orientation, and modern aspects and traits, though not exactly to roaring successes. Whether proprietary code, patent encumbrances, or not-invented-here stubbornness, there may be more people producing "reusable" code than actually reusing code.

Part of the problem is that it's more glamorous (in the delusive sense of the word) to solve a huge problem. Compare "Wouldn't it be nice if people had a fast, cross-platform engine that could handle any kind of 3D game, from shooter to multiplayer RPG to adventure?" to "Wouldn't it be nice to have a simple but fun open source shooter?"

Big ambitions, while laudable, have at least two drawbacks. First, big goals make for big projects — projects that need more resources than you may have. Can you draw in enough people to spend dozens of man-years on a project, especially as that project only makes it possible to spend more time making the actual game? Can you keep the whole project in your head?

Second, it's exceedingly difficult to know what is useful and good in a framework unless you're actually using it. Is one particular function call awkward? Does it take more setup work than you need? Have you optimized for the wrong ideas?

Curiously, some of the most portable and flexible open source projects today started out deliberately small. The Linux kernel originally ran only on x86 processors. It's now impressively portable, from embedded processors to mainframes and super-computer clusters. The architecture-dependent portions of the code tend to be small. Code reuse in the kernel grew out of refining the design over time.

Solve your real problem first. Generalize after you have working code. Repeat. This kind of reuse is opportunistic.

This isn't a myth because frameworks are bad. This is a myth because it's amazingly difficult to know what every project of a type will need until you have at least one working project of that type.

I'll Do it Right *This* Time

Myth: Even though your previous code was buggy, undocumented, hard to maintain, or slow, your next attempt will be perfect.

Reality: If you weren't disciplined then, why would you be disciplined now?

Widespread Internet connectivity and adoption of Free and Open programming languages and tools make it easy to distribute code. On one hand, this lowers the barriers for people to contribute to open source software. On the other hand, the ease of distribution makes finding errors less crucial. This article has been copyedited, but not to the same degree as a print book; it's very easy to make corrections on the Web.

It's very easy to put out code that works, though it's buggy, undocumented, slow, or hard to maintain. Of course, imperfect code that solves a problem is much better than perfect code that doesn't exist. It's OK (and even commendable) to release code with limitations, as long as you're honest about its limitations — though you should remove the ones that don't make sense.

The problem is putting out bad code knowingly, expecting that you'll fix it later. You probably won't. Don't keep bad code around. Fix it or throw it away.

This may seem to contradict the idea of not rewriting code from scratch. In conjunction, though, both ideas summarize to the rule of "Know what's worth keeping." It's OK to write quick and dirty code to figure out a problem. Just don't distribute it. Clean it up first.

Develop good coding habits. Training yourself to write clean, sensible, and well-tested code takes time. Practice on all code you write. Getting out of the habit is, unfortunately, very easy.

If you find yourself needing to rewrite code before you publish it, take notes on what you improve. If a maintainer rejects a patch over cleanliness issues, ask the project for suggestions to improve your next attempt. (If you're the maintainer, set some guidelines and spend some time coaching people along as an investment. If it doesn't immediately pay off to your project, it may help along other projects.) The opportunity for code review is a prime benefit of participating in open source development. Take advantage of it.

This isn't a myth because it's impossible to improve your coding habits. This is a myth because too few developers actually have concrete, sensible plans to improve.

Warnings Are OK

Myth: Warnings are just warnings. They're not errors and no one really cares about them.

Reality: Warnings can hide real problems, especially if you get used to them.

It's difficult to design a truly generic language, compiler, or library partially because it's impossible to imagine all of its potential uses. The same rule applies to reporting warnings. While you can detect some dangerous or nonsensical conditions, it's possible that users who really know what they are doing should be able to bypass those warnings. In effect, it's sometimes very useful to be able to say, "I realize this is a nasty hack, but I'm willing to put up with the consequences in this one situation."

Other times, what you consider a warnable or exceptional condition may not be worth mentioning in another context. Of course, the developer using the tool could just ignore the warnings, especially if they're nonfatal and are easily shunted off elsewhere (even if it is /dev/null). This is a problem.

When the "low oil pressure" or "low battery" light comes on in a car, the proper response is to make sure that everything is running well. It's possible that the light or a sensor is malfunctioning, but ignoring the real problem — whether bad light or oil leak — may exacerbate further problems. If you assume that the light has malfunctioned but never replace it, how will you know if you're really out of oil?

Similarly, an error log filled with trivial, fixable warnings may hide serious problems. Any well-designed tool generates warnings for a good reason: you're doing something suspicious.

When possible, purge all warnings from your code. If you expect a warning to occur — and if you have a good reason for it — disable it in the narrowest possible scope. If it's generated by something the user does and if the user is privy to the warning, make it clear how to avoid that condition.

Running a program that spews crazy font configuration questions and null widget access messages to the console is noisy and incredibly useless to anyone who'd rather run your software than fix your mess. Besides that, it's much easier to dig through error logs that only track real bugs and failures. Anything that makes it easier to find and fix bugs is nice.

This isn't a myth because people really ignore warnings. It's a myth because too few people take the effort to clean them up.

End Users Love Tracking CVS

Myth: Users don't mind upgrading to the latest version from CVS for a bugfix or a long-awaited feature.

Reality: If it's difficult for you to provide important bugfixes for previous releases, your CVS tree probably isn't very stable.

It's tricky to stay abreast of a project's latest development sources. Not only do you have to keep track of the latest check-ins, you may have to guess when things are likely to spend more time working than crashing and build binaries right then. You can waste a lot of time watching compiles fail. That's not much fun for a developer. It's even less exciting for someone who just wants to use the software.

Building software from CVS also likely means bypassing your distribution's usual package manager. That can get tangled very quickly. Try to keep required libraries up-to-date for only two applications you compiled on your own for awhile. You'll gain a new appreciation for people who make and test packages.

There are two main solutions to this trouble.

First, keep your main development sources stable and releasable. It should be possible for a dedicated user (or, at least, a package maintainer for a distribution) to check out the current development sources and build a working program with reasonable effort. This is also in your best interests as a developer: the easier the build and the fewer compile, build, and installation errors you allow to persist, the easier it is for existing developers to continue their work and for new developers to start their work.

Second, release your code regularly. Backport fixes if you have to fix really important bugs between releases; that's why tags and branches exist in CVS. This is much easier if you keep your code stable and releasable. Though there's no guarantee users will update every release, working on a couple of features per release tends to be easier anyway.

This isn't a myth because developers believe that development moves too fast for snapshots. It's a myth because developers aren't putting out smaller, more stable, more frequent releases.

Common Sense Conclusions

Again, these aren't myths because they're never true. There are good reasons to have a feature freeze. There are good reasons to invite new developers to get to know a project by looking through small or simple bug reports. Sometimes, it does make sense to write a framework. They're just not always true.

It's always worth examining why you do what you do. What prevents you from releasing a new stable version every month or two? Can you solve that problem? Solve it. Would building up a good test suite help you cut your bug rates? Build it. Can you refactor a scary piece of code into something saner in a series of small steps? Refactor it.

Making your source code available to the world doesn't make all of the problems of software development go away. You still need discipline, intelligence, and sometimes, creative solutions to weird problems. Fortunately, open source developers have more options. Not only can we work with smart people from all over the world, we have the opportunity to watch other projects solve problems well (and, occasionally, poorly).

Learn from their examples, not just their code.

chromatic is the technical editor of the O'Reilly Network and the co-author of Perl Testing: A Developer's Notebook.

[Sep 27, 2005] The dotCommunist Manifesto by Eben Moglen

[Sep 27, 2005] [PDF] Info-communism? A Critique of the Emerging Discourse on Property Rights  by Milton Mueller (Syracuse University School of Information Studies). A very interesting paper that foes more deeply into the connection of "software commonists" to communists.  The author's discussion of GPL and Raymondism is much weaker that this part of the article.

Intellectual property has emerged as the central communication-information policy issue of our time. This paper analyzes the legal and political discourse on property rights in information that has followed the success of several related movements: free software, opposition to software patents, copyright resistance, and various other methods of promoting “information commons,” including the debate over spectrum rights. By challenging the concept of exclusive property rights over information, this intellectual and social movement proposes to transform some basic economic institutions of the information society. (For a summary, see Kranich, 2004)As the movement has gained momentum, the word “communist” has re-entered political discourse. The rhetorical exchanges around “info-communism” are dogged by metaphors and parallels drawn from industrial-era communism. Both “leftists” and “rightists” in the ICT policy space are getting carried away with the metaphor. Free software advocate Eben Moglen pens a “dotCommunist Manifesto.”1

A Forbes columnist accuses Lessig of being a “radical” who advocates “stealing intellectual property.”2

Bill Gates accuses free software adherents of being “modern-day communists.”

Dan Hunter, a writer who is supportive of the movement, nevertheless argues that “the culture war is a Marxist war” and repeatedly refers to the copyright resistance as “Marxism-Lessigism.”3

This is not an inconsequential debate. Labels and frames are important in shaping social movements. A movement that is self-defined, or allows itself to be defined by, labels as redolent as “communist” will follow a different social and political trajectory than one that is defined differently.

A number of powerful symbolic and political factors conspire to fuel this turn in the discourse. Since industrial-era manifestations of communism have been thoroughly discredited, info-communism offers hope and revitalization to those on the left who could never quite bring themselves to accept the economic failure of social systems that attempted to abolish markets based on the exchange of private property.

The concept is also useful to their rightist counterparts, who hope to hang the albatross of old communism around the neck of emerging social movements of information.

Thus, a “new information right,” capitalizing on rights-holding businesses willing to put their money where their mouthpieces are, is responding withknee-jerk defenses of intellectual property. (Liebowitz and Margolis, 2004; DeLong, 2004)1Moglen, http://emoglen.law.columbia.edu/publications/dcm.html January 2003.2Stephen Manes, “The Trouble with Larry,” Forbes Magazine 29 March 2004.3Hunter, 2004. see also Legal Affairs, Nov-Dec.2004. http://www.legalaffairs.org/issues/November-December-2004/feature_hunter_novdec04.html

An obvious danger of this dialectic is a polarization of the debate over property rights in the information economy into a caricature of the 19thcentury debate over communism.

If Marxism is to be invoked, we should recall the opening sentences of Marx’s The Eighteenth Brumaire of Louis Bonaparte (1852).

“Hegel remarks somewhere that all great, world-historical facts and personages occur, as it were, twice. He has forgotten to add: the first time as tragedy, the second as farce.”

This paper has three goals. The first is to salvage the rationality of the debate over the nature of property institutions in information and communications, by criticallyexamining the metaphors and parallels to old-line communism.

The second goal is identify and call attention to a deep-seated tension within the information left that contributes mightily to this framing. In some cases the movement pushes shared or nonexclusive rights on practical, voluntaristic grounds, and presents it to the public as a choice they can exercise freely, based on their own desires or self-interest.

In other instances non-exclusivity is urged upon people as an ethical or moral imperative, and considerable effort is expended to use various forms of leverage to push people into that alternative. The information left needs to reflect more deeply on the implications of relying on either type of appeal. In making this argument, the paper notes that the need to motivate and sustain collective action plays an important role in shaping responses to this problem. Ethical and spiritual appeals to communal property provide stronger glue forholding together a social movement. But in this case they are also more prone to legitimate charges that the movement is “communist”and hostile to private property in any form.My third goal is to take sides regarding the dichotomy described above.

I will argue that it is impossible for moral/ethical arguments alone to justify either common or private property in all cases and that a free, contractually based economy offers the best hope of finding the right mix. Drawing on property rights theory, I argue that there is an important place for both models (commons and private property) in the present and future economy, and that there is often a dynamic interaction between the two that is superior to any attempt to push the economy into one of the two extremes.

The movement should view information commons as a vital and constructive part of a free and open market economy, not as its enemy. As Merges (2004) has argued, contractual arrangements to build commons and nonexclusive access to informational resources can be seen as a rational market response to the legal and political overreaching of rent-seeking copyright and patent holders.

Red-Baiting the Commonists: Frame or be FramedIt didn’t start with Bill Gates. Back in 1998, a Slashdot feature article was already noting that “in the debate about open source software the term ‘communism’ comes up quite frequently.”4

About three and a half years later (in 2001), Eben Moglen was using Marx’s language of class struggle and of revolutionary changes in the “means of cultural production” to produce what he called a “dotCommunist Manifesto,” which he eventually turned into the turgid piece that sits on the web.

What happened on January 6, 2005 merely raised the media profile of a discussion that had been going on for years. On that date, CNET news published the following exchange, in which Bill Gates was asked about movements to reform intellectual property laws:

REPORTER: In recent years, there's been a lot of people clamoring to reform and restrict intellectual-property rights. It started out with just a few people, but now there are a bunch of advocates saying, "We've got to look at patents, we've got to look at copyrights." What's driving this, and do you think intellectual-property laws need to be reformed?

GATES: No, I'd say that of the world's economies, there's more that believe in intellectual property today than ever. There are fewer communists in the world today than there were. There are some new modern-day sort of communists who want to get rid of the incentive for musicians and moviemakers and software makers under various guises. They don't think that those incentives should exist.

[Mar 28, 2005] Source Access Direct Benefits?  by Jason Matusow, program manager of the Shared Source Initiative at Microsoft Corp. There is a Russian proverb: An "objective view of a person is more common for people who do not like this person" :-) In this sense Microsoft views on open source are very valuable. The key idea of this note is that benefits are indirect as "transparency increases trust." It also contains an interesting arguments that "The availability for source code does not deliver direct benefit to the end user" which are true if and only if the user is a non-programmer and/or we are talking about large and complex systems (Linux, gcc, Gnome, etc). This statement is probably wrong for:

• programmers and medium to small size systems, especially written in scripting languages;
• advanced users and small scripts and components: the border between programmers and non-programmers in scripting languages is rather fuzzy and advanced users often possess considerable programming skills that are just limited to particular scripting languages (for example VBA in Office environment):

Here is the full text (Jason Matusow blog contains several other interesting notes): 

On Thursday last week I met with a delegation representing Eastern European members of the press (about 40 of them) at our facilities in Redmond.  We talked through issues of source licensing and how they are perceived in their respective countries.

 

One of the questions asked struck me as a particularly simple, and good, question: How does this help end users?  

 

I would like to say that source licensing (open, shared, whatever) has great direct benefit for the end user – but in truth it doesn’t. For the person sitting in front of a machine trying to get work done, who has no development experience, there is no direct benefit from source licensing. All value is indirect in nature.  

• There is a strong argument to be made that transparency increases trust. However, I do not believe this leads to the idea that transparency must be absolute to engender real trust. (I’ll have to tackle this one in another blog entry.) Trust therefore is as direct a relationship as possible in this discussion.

• If the applications running in that environment are better because the developers had deeper knowledge of the system on which there are building, then that is indirectly good for the end user.

• If the feedback loops within the technical community are stronger because of the source access vehicles, then that is indirectly good for the end user.

The argument that source availability fundamentally improves the quality of a given piece of software is a specious argument. It can help, but it certainly isn’t a compulsory result.

 

In the same vein of thought is the assumption that the software would be more secure because of source availability. Both arguments have the same flaw.

 

Just because the code is there does not mean people are looking at it. More importantly, it does not mean that the right people are looking at it. And, as the code base matures and evolves, there is no guaranteed rigor in testing or ongoing compatibility resulting exclusively from source access.

 

Where does this leave us? The availability for source code does not deliver direct benefit to the end user. Direct benefits are reserved for the development community (individual and organizational) and business strategy.   

 

This posting is provided "AS IS" with no warranties, and confers no rights.

O'Reilly Whence the Source Untangling the Open Source-Free Software Debate

This circumstantial evidence makes it pretty easy to perceive Stallman's generous, virtuoso effort as the technical foundation of the movement. Throngs of Free Software Foundation enthusiasts do, and thus seem to implicitly accept his radically socialist ideology as the One True Philosophy of source code liberation.

But there's another problem: Stallman wasn't the first.

Years before he or Eric Raymond ever hit on the idea of liberating source code, the UNIX operating system was being developed at AT&T Bell Laboratories. As a government-regulated monopoly, AT&T was barred from competition in the computer industry. Though UNIX source code was not then "free" in either the FSF or OSI sense, it could be licensed at nominal cost.

Universities were among the first to take advantage. As a result, UNIX ended up in the hands of hundreds of collaborating academic programmers. In particular, the UNIX effort at the University of California at Berkeley spawned a West Coast hacker culture to rival Stallman's MIT cohort. Ultimately, the student programmers at Berkeley created their own variation of the operating system so potent that it became a major fork in the UNIX lineage -- the Berkeley Software Distribution, or BSD.

It is difficult to overestimate the role of BSD UNIX in modern computing. Not only did it beget many key features of all future versions of UNIX, but it was also under the BSD flag that UNIX met the Internet (though at the time it went by its more ancient name, Arpanet). Much of the most common system software surrounding the TCP/IP protocol was developed at UC Berkeley, and was introduced to the world as part of BSD.

In the years since, BSD has enjoyed not only a substantial commercial run, but has also found its way into a commerce-free distribution of its own, one to rival Linux. Though not as popular or mediagenic as Linux, FreeBSD can nevertheless be widely found on the machines of hobbyists, ISPs, and major corporations alike.

So the shared source collaboration concept had received significant validation long before Raymond or Stallman showed up on the field. That would make AT&T the unlikely mother of the movement, having quietly accomplished the feat with neither Stallman's righteous rhetoric nor Raymond's theoretical grandstanding.

First Monday Papers

I wrote two papers for the First Monday in 1999: Open Source Software Development as a Special Type of Academic Research and A second look at The Cathedral and The Bazaar

• The first paper Open Source Software Development as a Special Type of Academic Research (Russian Translation by Sergey Korop is also available) should not be listed as a critique of the Cathedral and the Bazaar(CatB). It analyses a dangerous attempt to create an open source mythology using Vulgar Marxist arguments and to rally people around it (like semi organized high demand cult). I called this phenomenon Raymondism. As I pointed out in the Bad Linux Advocasy FAQ that leads to the loss of credibility on the movement, and betrayal of trust of people who support it. But main content was devoted to the exploration of  the limitations of the open source development model. I never intended to write a point-by-point critique of ESR's essay (although I did have an annotated collection of quotes), it just contains criticism of the ideas expressed in the ESR's writings in general. The main point was that ESR describes Open Source as a revolutionary phenomenon, whereas for me it is another form of scientific community. Similarly for me Linux model is not a new and revolutionary development model, but a logical continuation of the famous GNU project of the FSF -- the project with strong connections to the MIT. I am convinced that this connection was crucial to the success of GNU project like the connection to the University of Helsinki immensely helped the Linux project during early, most difficult stages.  See my commentary for details.

  As one Slashdot author(see #192) put it:

  It is a fairly good critique the whole essence of which, IMHO, can be phrased like so: OSS is not exactly such a novel thing--it has been known in scientific community for a long time. Current hype and success of it has to be largely attributed to Linux, but it is too naive to claim that OSS as a new software development paradigm means the total obliteration of any other way of developing software. ...Bezroukov does attack ESR as much as having his name in the article name. Why? Because ESR represents exactly this naive, on the border of blind-folded chauvinism, view of OSS. ESR, propaganda is one thing, reality is whole a lot different.

  (to save space it was considerably edited and shortened; see the last beta version)

  • My commentary

  • My responce to the letter by Paolo Pumilia to the FM
    

    I would like to reiterate that ERS's views on the economic superiority of open source are close to vulgar Marxism with it's economic determinism. Contrary to your impression "vulgar Marxism " is a legitimate scientific term. As Professor Robert M. Young stated in his work "Marxism and the history of science" [see R.C. Olby, G.N. Cantor, J.R.R. Christie and M.J.S. Hodge (editors), Companion to the History of Modern Science. (1996), pp. 77-86.]:

    "The defining feature of Marxist approaches to the history of science is that the history of scientific ideas, of research priorities, of concepts of nature and of the parameters of discoveries are all rooted in historical forces which are, in the last instance, socio-economic. There are variations in how literally this is taken and various Marxist-inspired and Marxist-related positions define the interrelations among science and other historical forces more or less loosely. There is a continuum of positions. The most orthodox provides one-to-one correlations between the socio-economic base and the intellectual superstructure. This is referred to as economism or vulgar Marxism."

     

•  The second paper A second look at The Cathedral and The Bazaar is explicitly devoted to the critique of CatB (or to be more exact the version of CatB published in First Monday). I do feel that Cathedral vs. Bazaar model is primitive and incorrect.

  This paper can be considered a continuation of the earlier paper Open Source Software Development as a Special Type of Academic Research. One of the important aspects of the previous paper was a critique of the description of Open Source software (OSS) as a revolutionary phenomenon and argumentation that it is just another form of a scientific community. The publication of Eric Raymond's new book The Cathedral and the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary (O'Reilly)  makes a fresh and critical review of his most influential paper The Cathedral and The Bazaar (CatB) that is central to this book even more necessary. This his paper provides an overview of the weaknesses of the CatB (the idea of inapplicability of Brooks' Law,  the idea that "given enough eyeballs, all bugs are shallow", the view of the source code as the best thing since sliced bread, etc.) as well as the more coherent demonstration of the fact that the bazaar metaphor is internally contradictive and that in some parts Linux can be considered as belonging to the Cathedral model, while Microsoft can be considered as belonging to the Bazaar model.  The complex nature and pitfalls of status competition are discussed. Along with a critique of CatB views, a more objective picture of the status competition in the OSS environment is provided.

  As one of the founders of Cygnus and prominent GNU C compiler developer, Michael Tiemann said in his recent interview:

  Now, Eric Raymond wrote a very popular piece called The Cathedral and the Bazaar that contrasts those two development models. Abstractly, he brings up a lot of very interesting points, but I would not say we've discovered the truth yet about GNU versus Linux, in terms of development models. There are cathedral aspects to Linux and bazaar aspects to GNU. Eric Raymond's story makes for interesting reading and raises interesting points, but in my mind, having been part of the cathedral model, those points are much more theoretical.

  If Eric Raymond were absolutely correct about the cathedral model, it would have been impossible for a 23-year-old to download the GNU C compiler and two weeks later spin out a brand-new port to a completely new architecture - and, after signing the FSF assignment papers, have it accepted back by Stallman. That's not the cathedral model. The cathedral model is "centralized everything," with somebody dictating: "We're going to put the narthex here, and the nave there, and the chancery over here." I did six ports of the GNU C compiler in a year while writing the GNU C++ front end. Other people used those data points to say, "Well gosh, if this guy can do it, then so can I." And that really caused the whole blossoming of the GNU ports. The GNU compilers from Cygnus support over 150 different host-target combinations, and they can theoretically be configured to support over a thousand.

  So I don't think it's fair to characterize the GNU tools from a purely "cathedral" point of view. I really think there are elements of the central control and the haphazard-and-serendipitous development models at work there.

  See also:

  • Bad Linux Advocacy FAQ

  • The last beta version

  • Short Introduction into Lysenkoism -- my (very limited) attempt to describe this very complex phenomenon.

  • Open Source Chronicle -- some research materials for the paper. Very raw

  • Nikolai Bezroukov. Portraits of Open Source Pioneers alpha version

  • Softpanorama Lysenkoism and Pseudoscience Page -- problems in the academic research

Comments about this page

[ Nov 27, 2002] Google Search raymondism -- pretty amusing views from some Linux enthusiasts ;-)

From: Angerthas.Daeron (daeron@demon.com) Subject: Max Burke spreads Microsoft fud   View: Complete Thread (4 articles) Original Format Newsgroups: comp.os.linux.advocacy Date: 2002-11-27 08:28:39 PST

In a previous post Max Burke (mlvburke@%$%#@.nz) wrote the following :

<snip>
".. # Before I get flamed for this, please understand that a holy war,
"Linux uber alles" of sorts,  is a self-defeating strategy. I hope
that  there is a healthy "silent majority" of the open source
community (that why I actually am writing this FAQ) who are just
writing code as best they can, and/or submitting patches bug reports.
But that does not mean that we can just ignore the ranting and raving
of the zealots. But the public tend to define the open source
community in terms of its most outspoken members which in this
particular case means zealots...
http://www.softpanorama.org/OSS/Bla_faq/raymondism.shtml  .."

He accuses us on COLA of promoting a holy war and of being zealots.
He points to a web page to back up his arguments. It's written by one
"Nikolai Bezroukov" of Kiev University of commerce and economics,
Ukraine which makes him eminently qualified to comment on the issue.

I make this point as Mr Bezroukov himself doesn't think Linus Torvalds
is qualified to give opinions. He purports to be an unbiased
commentator but from the tone and content I for one suspect his
motives. I had not been aware of it's existence so here is a belated
response.

Firstly I am not a zealot. I am just a user of the technology. I come
here to disguise Linux with like minded individuals. The same cannot
be said for Mr Burke and others with the same hidden agenda as
himself. I suspect that they are in fact disguised proponents for the
Microsoft Corporation. I guess you knew that already.

I don't use the Microsoft product and can go for ages without
mentioning it. I have no axe to grind either way. What I do object to
is these WinTrolls coming over here pretending to engage in dialogue.
But secretly trying to undermine the Open Source Community in general.

It cannot be a co-incidence that the language, deceptions and
mis-information they use is strikingly similar to the product coming
out of Redmond. You might suspect that it has been written by the same
people. It's also strange how most of the fud mentioned here at one
time or another bears a striking resemblance to that web page. It's
remarkable how they are all so ON MESSAGE.

There has been a change of tack recently. Rather that the direct
assault they are going for a more suttle approach.
In the web page referred to above the author one  "Nikolai Bezroukov"
resorted to the personal attack. Referring to something called
"Raymondism". This I assume is a gratuitous personal insult aimed at
"Eric S. Raymond" author of "The Cathedral and the Bazaar" amongst
other things. "Raymondism" can also be refered to as "childish
diseases" and  "bad advocacy" he informs us.

He defines the affliction as "naive .. blind fold Linux chauvinism
("Linux uber alles")". The quotes are himself quoting himself. Linux
advocates he don't agree with are just like  Nazis - (it's the German
quotation - get it !).

What he does like is "a credible OSS advocacy" - what this is he
doesn't say.  O.S.S  ".. can play a positive role in developing
countries .." he says, get that people ? Some colonials might have a
use for it. Don't even think of going head to head against W2K. He
attacks ERS for ".. primitive anti Microsoft rhetoric ..". 

http://www.zdnet.com/sr/stories/issue/0,4537,384326,00.html
".. Since Jan. 1998 Eric Raymond successfully promoted "open source"
as a distinct and slightly anti-Stallman movement. See for example his
interview with Smart Reseller Straight From The Source where Eric was
called a Godfather of Linux ;-) .."

There was no reference to a "Godfather of Linux" in the quoted
article. Is this a case of someone making up their own quotes again ?
I've tried to find the quote on google but to no avail.

He also quotes  ESR and attacks him because some of the same arguments
could be applied to Microsoft. Here he betrays his true position, not
as an "objective" reviewer of OSS but as an apologist for the
Microsoft corporation. This me-too-ism runs through the article(s). 
I'm curious as to his motives for this hatchet job on OSS. He even
quotes Pope Boniface VII at one point to bolster his arguments. OSS
you see - it's the devils work !

".. At the same time the movement is still in its early stages (and
not last days, as some predict) .." Who ever said OSS was on its last
days ? Nobody. You print a falsehood only to retract it in the same
sentence. This bares similarities to an "Ericism". Are they by any
chance related ?

".. What ESR and Co failed to realize is that people who are
developing and using Solaris, Novell and Microsoft products are also
professionals and many of them are of a caliber far superior to the
author of low to middle-range open source products like EMACS editor
macros, a mail utility, and like ;-). For any intelligent professional
an open demonstration of arrogance naturally creates a strong negative
reaction, a backlash that is damaging to the movement credibility and
future .."

Why is such a superior company desperately trying to gather cudos by
comparing themselves to a bunch of sandle wearing OSS advocates. The
words  Inferiority Complex comes to mind. See how he has to rope in
Solaris and Novell to bolster his arguament. I suspect neither of them
would be quick to defend the beast. Again he tacks a negative
signifier on to the end of the sentence possible in the hope that no
one would notice.

This again reminds me of a typical fud posting here. See how he gets
'author' 'low' and 'open source products' together in the same
sentence. For such an expert on OSS the only apps he can think of are
"EMACS" "editor macros" and  "a mail utility".

A Microsoft defender accuses the OSS movement of "arrogance". Has he
ever heard the expression Pot Kettle Black ?

The only war is the one being prosecuted from One Microsoft Way. OSS
people have neither the time or the inclination to mount "WARS". He
uses the word on more than one occasion. Bill Gates may be at war with
the rest of the world but that's his own paranoia.

".. The same problems exist with primitive anti Microsoft rhetoric .."
er the TRUTH ! This arrogant bastard then goes on to abuse Linus
Torvalds ".. technical judgments are very suspect .. things about
which he actually has very little real knowledge due to the specifics
of his career .."

Could Mr Bezroukov please enlighten us as to his own qualifications.
As he is not impressed with Richard Stallman, Eric Raymond" or Linus
Torvalds. There is an old football expression around here - if you
can't go for the ball then go for the man. We can be sure which
philosophy fires up Bezroukov - go for the man.

"..  We should suspect any OSS advocacy that includes the following
features .."

Is that the royal we or do you have an invisible friend sitting on
your sholder as you type ?

".. open source software .. is called economism or Vulgar Marxism .."
Get that folks OSS is communist. YOUR ALL A BUNCH OF NO GOOD COMMIES !

Sorry I lost it there for a minute - to continue. We can take it that
Nikolai has embraced the one true Church of the All Mighty Dollar and
as such is displaying all the zeal of the convert. The Good Lord loves
a believer.

".. See Is "Vulgar Marxism" a legitimate scientific term .." - Answer
NO it's just more of the same abuse from a vulgar troll!

"..  concealment of the facts about the true economic origin of ..
(OSS) .. products .. 'taxpayer-funded'  (university-funded) .."

Do Microsoft see the Universities as a threat now. 'my god there are
people actually thinking there - without a licence'

Didn't his BillNess use an unauthorized terminal to bash out code in
his early years - all paid for by his college ?

".. Linus Torvalds was financed .. remunerated him quite nicely ..
most highly paid developers in the Unix word .."

What is the point - are we supposed to feel jealous. Yes he makes
money. He probably lives in a house eats food and sleeps in his own
bed.  Bezroukov cannot support his position on it's merits so he
trashes the personal reputation of OSS advocates instead. Do these
people have no integrity ?

".. disrespect of other developers .. especially Microsoft .." -
Finally we come to the kernal of the matter.

We've hurt their feelings. Sitting hunched out there in Redmond
hacking out "Dog Food" all day is bad enough but getting maligned by
your peers - that really hurts.

" Instigation of hatred of the members of the commercial community is
unproductive and unethical .."

OH The irony ! Blackmailing and intimidating your own commercial
partners is also unethical. Getting lectures on ethics from you people
is ludicrous  as well as insulting !

Linux Kernel Mailing List, Archive by Week Closed-door develop

http://www.softpanorama.org/OSS/index.shtml

CatB in a new light. This fall Raymond has been touring Europe promoting his book. He most kindly made his way by Trondheim where he gave an unforgettable series of speeches. In one of these speeches he poses the question why nobody had articulated the bazaar mode of development prior. He says there are a handful of intelligent, articulated hackers that had already observed the phenomena, but none had spelled it out. Why is that? Raymond asks. His suggestion is that hackers like to think that their success in developing software is due to their own brilliance. All hackers liked to think so, and that is why nobody had tried to look into the matter more closely before Raymond did.

Let's permutate Raymonds question a bit, and ask: why is it that the hacker community is not questioning the apparent flaws in the 'Cathedral and the Bazaar'. Paul Feyerabend writes:

There comes then a moment when the theory is no longer an esoteric discussion topic for advanced seminars and conferences, but enters the public domain. There are introductory texts, popularizations; examinations questions start dealing with problems solved in its terms. Scientists from distant fields and philosophers, trying to show off, drop a hint here and there, and this often quite uninformed desire to be on the right side is taken as a further sign of the importance of the theory.

Unfortunately, this increase in importance is not accompanied by better understanding; the very opposite is the case. Problematic aspects which were originally introduced with the help of carefully constructed arguments now become basic principles; doubtful points turn into slogans; debates with opponents become standardised and also quite unrealistic, for the opponents, having to express themselves in terms which presupposes what they contest, seem to raise quibbles, or to misuse words. Alternatives are still employed but they no longer contain realistic counter-proposals; they only serve as a background for the splendour of the new theory. Thus we do have success; but it is the success of a manoeuvre carried out in a void, overcoming difficulties that were set up in advance for easy solution. (1993, p. 30)

Answering almost with Raymond's own words: can it be that we hackers like to think that the success of our software is due to a genial, new way of development that we have come up with ourselves? Is it truly so? CatB is not a software engineering essay. It is an anthropological study. However, it contains material about the bazaar, the hackers' way of doing software engineering. Central traits to the bazaar is the open process, the freedom to do with the code what each individual developer wants, and a high degree of
cooperation. Raymond mentions Linux as an archetypal bazaar, yet in a letter to the author David Miller, a central Linux developer, writes:

Date: Tue, 5 Oct 1999 18:40:06 +0200 (CEST)
From: David S. Miller

> is it so that you core Linux kernel developers are doing much of the discussing and
> planning outside of the Linux kernel mailinglist?

It is true to a large extent, and in my opinion it's the gem that keeps us at such high productivity rates.
It's a surefire method by which us core developers can obtain the best signal to noise ratio. Discussions happen
more efficiently and productively when you know you're talking to someone with a clue and you don't get barraged with responses from folks who are perhaps not so clueful and not so weathered on the topic as the core developers.
I.e. there is a mismatch between the map and the terrain, the map here being Raymond's bazaar and the terrain being how things work in the real world. While there is an open forum, areas for community building, the development in itself is being done in a closed
fashion. The results, i.e. the source code, is up for public review, so the product itself is still open. The process, however, is a closed one, and it is the process more than the product that Raymond emphasize as the bazaar model.

Comments on the second paper (SLcatB)

Text of the paper:

• First Monday: A Second Look at the Cathedral and Bazaar

• Local copy: A Second look at the Cathedral and Bazzar

• Russian translation Povtornyij vzglyad na Sobor i Bazar

Feedback:

• Critical Review of Essence of Distributed Work: The Case of The ...

  Open source has many benefits which can be pretty inviting.  However, it does have its limitations or downfalls as well.  One idea here is the principal value of simplicity.  The availability of source code is useful but the ability to look and modify the tool diminish rapidly with the growth of the code base. Very large and complex open source projects are probably as closed as proprietary commercial projects of the same size.[1]  Another criticism is that open source does not automatically provide quality feedback.  More emphasis is put into development; information and instructions on how to use a particular piece of software are hard to understand or non-existent.  One of the major criticisms is that the pool of talented developers that can spend their time on open source projects is very limited and as such open source software projects usually suffer from lack of development resources and luck of feedback. While everyone pays tribute to Linux's original author, Torvalds, as "a benevolent dictator" guiding Linux kernel development, no one has a very clear idea of what would happen if he were to retire or otherwise disappear from the scene. Most open source projects (probably 90%) have just one main developer.[2] Even highly popular projects like Linux have problems with attracting volunteer developers for particular tasks and now essentially switch to paid developers model. Quality feedback for open source software projects is difficult to get and is usually insufficient for guiding project development.  

   

• IBM DeveloperWorks Open source Linux Features - Library - Papers  DeveloperWorks Open source Linux Features - Library - PapersBrooks' Law and open source: The more the merrier? Does the open source development method defy the adage about cooks in the kitchen?Paul Jones
  Director of MetaLab, University of North Carolina May 2000

  While Raymond tells us that "the more the merrier is the rule in an open source project" and Stallman bemoans the scarcity of programmers, Jamie Zawinski, formerly of Netscape and of the Mozilla project disagrees. "The dynamic of development inside a company is definitely different from development in an open source way. However, I don't think that saying 'adding programmers makes it later' isn't true for open source projects because of their (alleged) lack of deadlines: I think that it's true (in any sense that it is true) only because the definition of 'programmer' is slippery in that case. If you have a project that has five people who write 80% of the code, and a hundred people who have contributed bug fixes or a few hundred lines of code here and there, is that a '105-programmer project?'"

  What Zawinski and others have seen is that in such undertakings as the Apache project, Linux Kernel, and other large software projects, the bulk of the work is done by a few dedicated members or a core team -- what Brooks calls a "surgical team."

  "In most open source projects," says Zawinski, "there is a small group who do the majority of the work, and the other contributors are definitely at a secondary level, meaning that they don't behave as bottlenecks."

  Zawinski goes on: "Most of the larger open source projects are also fairly modular, meaning that they are really dozens of different, smaller projects. So when you claim that there are ten zillion people working on the Gnome project, you're lumping together a lot of people who never need to talk to each other, and thus, aren't getting in each others' way."

  Brian Behlendorf of Apache and of Collab.net agrees. "We don't consciously think about it, but I think that the philosophy of keeping things simple and pushing out almost anything extraneous or nonessential to external modules has been followed fairly carefully in Apache. We've also been fairly successful (I think) in 'federalizing' the Apache process to sister projects."

  Bezroukov agrees that modularity is essential, but there are limits: "For example if a project logically consists of three parts and we have two developers, than adding third developer can be highly beneficial -- the structure of the project will now correspond to the structure of the team. If the project is decomposable into three major parts and we add fourth developer, the effect will be much less."

   

• The Realm of Utoastria -- some discussion of my approach

  The cathedral and the bazaar

  "What truths do the majority flock about? It is the truths that are becoming obsolete. For when a truth is so old, it is on good way of becoming a lie. A normal truth is viable for no more than 20 years, and yet that is when the majority takes it to them and recommends them to the society as wholesome spiritual fuel."
  - Dr. Stockman in Henrik Ibsen's drama 'An enemy of the people'

  A great deal of sociological and anthropological studies has been performed on the hacker community, but there is scarcely any research on the community's software engineering practices. CatB was the first essay to specifically address the community's practices. However, it is still very much an anthropological study dealing with some aspects of software engineering, but with emphasis on the anthropology. Another strain has been to focus on the social dynamics of the bazaar (< >). Nicolai Bezroukov is the first to address the software engineering perspectives of CatB in a critical manner. Unfortunately his two articles suffer from being the first efforts at formulating a systematic and thorough critique of Raymond's observations.

  Critical remarks

  Books such as Levy's Hackers and Raymond's The New Hacker Dictionary show that the hacker community loves to tell and hear cool and neat stories and epics of great deeds of hacking. There is almost something tribal about how these stories are told. I believe anyone who has ever been part of a community of hackers can remember how the oldtimers late at night with everybody sitting around them, told epic stories of great hacking achievements. Not only their own, but those of the hacker lore like the Story of Mel and the Magic Button. How these legends are being recounted, almost ritualistic, to the listeners undivided delight. (Incidentally, I don't think it is a coincident that so many of us hackers are into fantasy and sci-fi literature, but that's only my personal little anecdote).

  Where am I going with this?

  I am of the opinion, that CatB isn't free of recounting great deeds of hacking. Wouldn't it be great if a loosely knit group of hackers, with no Dilbert managers at the top telling them what to do and how, could create an industry strength operating system? It's the total liberty trip, that which most software engineers dream of. Nobody is told what to do. Everybody's code is equal. And at the top the maintainer is simply coordinating this mob by giving credits where credits are due. That is basically how Raymond portrays the bazaar. Reality seems in fact to be somewhat different, more like herding cats (as Bruce Perens once observed).

  Linux, to take an example, has a small, devoted core developers team very much like Brook's surgical team (Brooks, 1995). Looking at other highly successf