Brooks Law and open Source -- Does Open Source defy the gravity ?

Brooks' Law
This little book by Fredrick Brooks, The Mythical Man Month (or M3 among its most ardent followers), has become a kind of bible for software developers. Brooks received the Turing Prize, often called "the Nobel Prize of Computing," from the Association for Computing Machinery this year.

Fredrick Brooks headed the team at IBM that created the first large-scale computer operating system in the early 1960s. During his work at IBM, he realized that programmers were not like ditch diggers or house painters. Instead, he wrote, "The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by exertion of the imagination. Few media of creation are so flexible, so easy to polish and rework, so readily capable of realizing grand conceptual structures..."

Further, he defends software architects against ignorant management with Brooks' Law. "Oversimplifying outrageously, Brooks' Law [can be stated] as 'Adding manpower to a late software project makes it later.'" In other words and with mixed metaphors, art and artists cannot be rushed, and too many cooks will surely spoil the broth.

Importantly and uniquely, The Mythical Man Month and Brooks' Law are now over 25 years old. In a world in which "old" is defined in days or weeks, such longevity is nothing short of a miracle.

But proponents of open source and free software development, including Linux developers, are not completely satisfied with the Law. Most famously (among geeks at any rate), Eric Raymond in his "The Cathedral and the Bazaar," declared Brooks' Law obsolete, if not simply limited, saying "if Brooks' Law were the whole picture, Linux would be impossible."

Although Raymond now says that he has somewhat modified his views or was misunderstood, some still would say he is given to oversimplifying and outrageousness himself. "I don't consider Brooks' Law 'obsolete' any more than Newtonian physics is obsolete; it's just incomplete. Just as you get non-Newtonian effects at high energies and velocities, you get non-Brooksian effects when transaction costs go low enough. Under sufficiently extreme conditions, these secondary effects dominate the system -- you get nuclear explosions, or Linux."

Defying gravity
But how do open source and free software efforts defy the crux of Brooks' Law, the need for small, smart groups and time flexibility for the artist/architect programmers? Perhaps by not having a schedule in the first place?

"I have yet to see an open source project commit to a schedule, and most of them don't ship anywhere near when folks think they will (some ship early, and some ship late -- it's hard to predict). I don't know how you take a rule about 'making a late project later' and apply it to development which has little formal scheduling," says Erik Troan, author of the popular Red Hat Package Manager.

The Free Software Foundation's McArthur Fellow Richard Stallman echoes Troan's sentiments. "GNU projects rarely have a schedule. When people ask me when something will be finished, I respond, 'It will be ready sooner if you help.'"

Ken Coar, member of the Apache Web Server group, will admit to having goals, but not deadlines. "Goal dates, yes -- but our release process more closely resembles the exchange between Pope Julius II and Michelangelo than a conventional software house's product development."

Yet, Coar gives himself (and us) a warning with his newly coined "Coar's Contention." "Most open-source fixes will not be proposed until a release date [for the project] is announced." In other words, deadlines do influence work and workers even in the artistically liberated world of open source.

"While the 'no deadlines' idea is not true in a general sense, we still can talk about speed of development -- just as critical a factor as deadlines. Just imagine what would happen if Linux kernel were frozen and no new version will appear until 2005!" stresses Nikolai Bezroukov of Fairleigh Dickinson University, a critic of both Stallman and Raymond.

Yet, Stallman sees projects without deadlines as a way to avoid the temptation to place untrained hands on the job at the wrong time and place. "If you don't have a deadline, you won't be likely to try to put more people on the job in that way. You will make use of more helpers only when they can actually help. Thus, I think that Brooks' Law does apply to [free software], but we have no inclination to try to violate it."

For Stallman, free software is ultimately about freedom of all sorts. "In free software projects there is usually a scarcity of resources, and no boss who can order people to do things this way instead of that way. We tend not to have too many people trying to help one project." In fact, free software -- as opposed to open source -- is not about products, per se. "In the free software movement, freedom and community are the focus. We talk about the deeper issues, such as what way of life we want."

Surgical teams
While Raymond tells us that "the more the merrier is the rule in an open source project" and Stallman bemoans the scarcity of programmers, Jamie Zawinski, formerly of Netscape and of the Mozilla project disagrees. "The dynamic of development inside a company is definitely different from development in an open source way. However, I don't think that saying 'adding programmers makes it later' isn't true for open source projects because of their (alleged) lack of deadlines: I think that it's true (in any sense that it is true) only because the definition of 'programmer' is slippery in that case. If you have a project that has five people who write 80% of the code, and a hundred people who have contributed bug fixes or a few hundred lines of code here and there, is that a '105-programmer project?'"

What Zawinski and others have seen is that in such undertakings as the Apache project, Linux Kernel, and other large software projects, the bulk of the work is done by a few dedicated members or a core team -- what Brooks calls a "surgical team."

"In most open source projects," says Zawinski, "there is a small group who do the majority of the work, and the other contributors are definitely at a secondary level, meaning that they don't behave as bottlenecks."

Zawinski goes on: "Most of the larger open source projects are also fairly modular, meaning that they are really dozens of different, smaller projects. So when you claim that there are ten zillion people working on the Gnome project, you're lumping together a lot of people who never need to talk to each other, and thus, aren't getting in each others' way."

Brian Behlendorf of Apache and of Collab.net agrees. "We don't consciously think about it, but I think that the philosophy of keeping things simple and pushing out almost anything extraneous or nonessential to external modules has been followed fairly carefully in Apache. We've also been fairly successful (I think) in 'federalizing' the Apache process to sister projects."

Bezroukov agrees that modularity is essential, but there are limits: "For example if a project logically consists of three parts and we have two developers, than adding third developer can be highly beneficial -- the structure of the project will now correspond to the structure of the team. If the project is decomposable into three major parts and we add fourth developer, the effect will be much less."

Back to the head chef

The human wave attack -- the thousand geeks hacking through the night -- too has limits. Limits are made less painful by modularity, but that goes only so far. At some point, management, whether they are software core team leaders or CEOs, will be tempted to make the wrong decision and send in more busy little hands.

Let us now return to Bob Young and Red Hat 4.0 in 1996 as they face a cash short run, mounting deadline pressure, and an anxious Board of Directors. Should Young add more programmers in an effort to get the product out the door or should he heed what may well have been seen as a bit of techie egotism on the part of Erik Troan and just stand back and watch his software team flourish or flounder?

To try to reduce his own stress level, Bob Young went out and bought Brooks' book. "While I didn't completely buy the arguments, (I had no frame of reference)," he says , "it at least did give me a more authoritative source making the same argument that Marc and Erik were making.

"Our small development team did ship 4.0 on the schedule to which they had committed. It was a big success, and Brooks' book about Brooks' Law contributed to reducing the (already small) chance that we were going to make the mistake of adding developers at a late stage of a software project."

And even Eric Raymond has lavish praise for Brooks. "The Mythical Man Month is still, despite time and tide, one of the most penetrating and humane books on programming ever written; every programmer and everyone who manages programmers should read it."

Fredrick Brooks himself may have the final say about how software development should be understood: "Programming is fun because it gratifies creative longings built deep within us and delights sensibilities we have in common with all men." That is a happy thought that is sure to please many in the open source community.

But if this strikes too optimistic a note for some of our more skeptical readers, bear in mind that Brooks has other sweeping statements that may not soon be hailed as law by most members of the community. He concludes: "No major improvement in the software engineering area will ever appear."

Resources