Why the postulate about high level of quality feedback to be partially a myth

Softpanorama
(slightly skeptical) Open Source Software Educational Society

May the source be with you, but remember the KISS principle ;-)

Why the postulate about high level of quality feedback to be partially a myth ?

I think that, other things equal, OSS products have some edge because the level of conflict between commercial developers and voluntary developers is lower in case of GPL products. There are other advantages including the trivial one: just availability of the both source and tools to build it free of charge that lower barrier of entrance and attract talented developers from less wealthy countries. But CatB reasoning is either primitive or plan wrong. There is no understanding of the complexity of the situation. Quality is not automatic consequence of opening code or adopting open source approach from the beginning. There are many badly written open source products.

Unless they are engaged in the "vainly fair" game of finding security holes described above, most people hate reading other code. Moreover, even if people are reviewing the code, that doesn't mean they're qualified to do so. In the scientific world, peer review works because reviewers are pre-selected and generally as a group posses a comparable, or higher, technical caliber on the subject matter. The quality of technical publication is maintained by attracting the best reviewers to magazines and publishing houses. There is no comparable process in open source. As Andrew Leonard put it in Salon Free Software Project BSD Unix Power to the people, from the code:

"Most people are bad programmers," says Joy. "The honest truth is that having a lot of people staring at the code does not find the really nasty bugs. The really nasty bugs are found by a couple of really smart people who just kill themselves. Most people looking at the code won't see anything ... You can't have thousands of people contributing and achieve a high standard."

Here are also some relevant critical postings from the Slashdot discussion:

Yet again the open source model fails to deliver
by Anonymous Coward on Tuesday May 09, @08:41AM EST (#19)

Well, what can I say? I'd say I've told you so, but I didn't, but anyway it's certainly not unexpected that the open source model so praised by /. and other similar communities has failed to deliver on its quite amazingly grandiose claims.

When people praise open source, they point to the legions of developers, the "many eyes" to find bugs, and the ability to quickly and efficiently incorporate user feedback and requests. But the truth seems to be a "little" different from this Garden of Eden coding paradise.
Most open source projects are really only the work of one or two people, who occasionally can be bothered to actually write a few lines of code and then release it as the next version (hence version numbers like 0.99beta-pre3). There is none of the dedication which occurs in a closed source software house, and certainly none of the encouragment or rewards, unless you count the mythical "kudos" which are often heard of, but never seen.
And as for the bugs issue, well, how many people ever read the source code to something they download? Not very many from what I can see. And when you've got projects as convulted and bloated as the Linux kernel, how many people can actually find the bug in the first place? Linus and a few of his cronies is about all I'd say. So this argument seems to be little more than hand-waving.
User feedback ties in with both the previous two points - it all depends on whether the coder(s) actually bother with doing anything often, and whether bugs and requests can actually be fixed.
Two high profile open source projects - Mozilla and the HURD - have both been pretty much vapourware for the last God-knows how long, and if you ignore the extremely buggy versions of Mozilla which we've seen, they appear to be so for the forseeable future.
Closed source may not be the best thing in the world, but at least it delivers what its customers want and on time.

"Troll"rating was unfair / Open source in general
by Master of Kode Fu (joeydevilla@hotmail.com) on Tuesday May 09, @09:38AM EST (#93)
(User Info)

I don't think that a "troll" rating for the previous post is completely fair. Whether open source is better than closed source is a matter of opinion, and the poster is simply voicing his/hers.
The points made in the comments "Most open source projects are really only the work of one or two people" and "how many people ever read the source code to something they download?" are well made. I can't say for certain, because I haven't carried out a poll or seen any statistics. What I know from the literature (such as CatB), experience and observation is that most open source projects start out as an itch that a programmer decides to scratch. The majority of people who find out about the project don't do anything except download the binary, a few download the code, a fraction of them comments and a fraction of that fraction hand back some modifications. Only a small portion of the much-vaunted "many eyes that make bugs shallow" are actually being harnessed. That's not a bad thing -- the closed source model exposes the software to even fewer eyes. It's just that there are many obstacles to fixing or modifying someone else's code: how the original programmer wrote it, availability of documentation, availability of the programmer to answer questions, whether it's written in a language you know, the complexity of the program's problem domain, how good a programmer you are, and the big one -- how much time you have.
The primary advantage provided by open source to a program's developer is the potential peer review and fine-grained testing that freely handing out your source gives you. There's a large body of evidence that code review is one of the best ways for incresing software quality. However, as Fred Brooks says, there's no silver bullet for the problems of software development, and peer review is only one step in making good software. Without good planning and process, all you have is a bickering committee. And we know why they don't make statues, painting or memorials of committees, right?
There's a very good article by Steve McConnell (author of Code Complete, Rapid Development, Software Project Survival Guide and After the Gold Rush) covering some of the issues around open source software. He points out that if open source wants to really reach its potential, the following needs to be done:
1. Create a central clearinghouse for the open-source methodology so it can be fully captured and evolved.
2. Kick its addiction to Code and Fix.
3. Focus on eliminating upstream defects earlier.
4. Collect and publish data to support its claims about the effectiveness of the open-source development approach.

While theoretically the more people are reviewing a program, the more likely they will find bugs. But reality is different. Mongol horde approach does not work. A single well-trained reviewer who understands the subject area will be more effective than a hundred people who just recently learned how to program.

The grim reality of OSS projects that you may have users. But getting quality feedback, reviews or help in development is not that simple, actually it's more difficult than for closed source commercial products where you can pay money to attract people of necessary qualification. CatB myth about huge volume of high quality voluntary feedback is just another myth. Like rare metals, talent is a very rare commodity that naturally concentrates on development and high quality feedback is impossible without talent. And the reason that a talented person would thoroughly analyze mundane code for free without any strong incentives looks like stretching the truth. For anybody who wrote a substantial program it's clear that most of the code is just mundane. It's error recovery, input checking, GUI(if any), etc, with code that is directly related to functionality of the program often around 10% or less.

But it's only one side of the coin. The other side of the coin is that for innovative projects the level of feedback cannot be high, because not many people can understand the innovation. I am convinced that if we are talking about innovation, not imitation, the author need to be ready to pursue his dream on his own regarding the support of rejection of the crowd. With very few exceptions, the applauds of the crowd for really innovative products usually come too late ;-). This is the case in mathematics and this is the case in programming because programs can be considered as a special class of the applied mathematical theories. Please remember that in no way Linux kernel development can be considered as innovative activity; this was just an attempt to replicate functionality of existing Posix kernels.

Copyright © 1996-2007 by Dr. Nikolai Bezroukov. www.softpanorama.org was created as a service to the UN Sustainable Development Networking Programme (SDNP) in the author free time. Submit comments This document is an industrial compilation designed and created exclusively for educational use and is placed under the copyright of the Open Content License(OPL). Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

Standard disclaimer: The statements, views and opinions presented on this web page are those of the author and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.