Softpanorama (slightly skeptical) Open Source Software Educational Society

May the source be with you, but remember the KISS principle ;-)

Softpanorama Classic Series

"No scene from prehistory is quite so vivid as that of the mortal struggles of great beasts in the tar pits. The fiercer the struggle, the more entangling the tar, and no beast is so strong or so skillful but that he ultimately sinks. "

The Mythical Man-Month. Essays on Software Engineering by Frederick Brooks Jr. Anniversary Edition  Paperback, 322 pages Published by Addison-Wesley Pub Co in July 1995.

The book was first published in 1975 and was based on Fred Brooks experience with the design of  operating systems for famous IBM/360 series.   Actually both hardware and descendants of operating systems designed for those machines are still  sold by IBM (IBM mainframe business). Not everything in those OS was a success. Actually a lot were dismal failure. And it is actually failures that survived best. For example, PL/1 an innovative programming language that pioneered the use of exceptions (and inspired C) is dead, but some ugly things from 60th like JCL are still widely used.

Most of the material in the book is as relevant today as it was when originally written. Understandably, though, part of the material is outdated. In the 20th Anniversary Edition, rather than update the original text, which is considered "classical," Brooks has wisely decided to add update chapters. These discuss the issues presented while shedding fresh light from the 90s on them. Personally, I recommend reading the relevant parts of Chapter 18 after reading each previous chapter. Chapter 18 is titled, "Propositions of The Mythical Man-Month: True or False?", and it contains updated information about each of the chapters from 1 to 17.

Contains in addition to the original book his influential 1986 essay "No Silver Bullet"  which was originally an invited paper for the IFIP '86 conference in Dublin, and later published in Computer magazine. In this paper, Brooks speculated that no technology will be found, within ten years of its publication (in 1986), which will enhance the process of software development by an order of magnitude. Nine years later, in retrospect, Brooks sadly notes that he was right.

Frederick P. Brooks

Frederick P. Brooks, Jr. is the "father of the IBM System 360". While manager of the 360 project it was Dr. Brooks who specified that a byte would consist of 8 bits and that word consists of 4 bytes. Whether or not you agree with his decision, it's hard to argue that this has not had a huge impact on the computer field.

No book on software project management has been so influential and so timeless as The Mythical Man-Month. Even now,  20 years after the initial publication it's still a very important. He was born in 1931 in Durham, North Carolina and grew up in nearby Greenville, NC.  He was interested in computers during his teenage years, and went on to double major in math and physics at Duke University.  At Harvard, he studied for his Ph.D. under Howard Aiken, the inventor of the early Harvard computers.  He joined IBM in 1956, working in Poughskeepie and Yorktown, NY and in 1957 Brooks and Dura Sweeney patented a Stretch interrupt system for the IBM Stretch computer that introduced most features of today's interrupt systems. 

In the early 1960s he was the leader of the development team for the IBM 360 Series.  The 360 quickly became the most popular mainframe computer on the market.  In 1964, Brooks left IBM to found the Computer Science department at the University of North Carolina at Chapel Hill and to become a professor of Computer Science.  He has chaired the department for the past twenty years. 

Throughout his career, Brooks has authored two books in the field of Computer Science, The Mythical Man-Month: Essays on Software Engineering and Computer Architecture: Concepts and Evolution.

The Mythical-Man Month is undoubtedly his most famous work in which he compiled essays about his work on the IBM 360 Series.  It also contains the famous “Silver Bullet Essay” where he likens his work on the 360 Series to a werewolf that can only be killed by a “Silver Bullet.”  Brooks was also known for his famous aphorisms, his most popular being Brooks’ Law: “Adding manpower to a late software project makes it later”   Many of his quotes are applicable in a wider context, such as “Complexity is the fatal foe” (P 344). 

Brooks is still a professor of Computer Science at UNC today.

See also

• Faculty Biography Frederick P. Brooks Jr. (UNC-CH Computer Science)
• The Mythical Man-Month - Wikipedia, the free encyclopedia
• Fred Brooks - Wikipedia, the free encyclopedia

Old News ;-)

[Dec 15, 2006] Computerworld - The mythical open source miracle by Neil McAllister

Actually Spolsky does not understand the role of scripting languages. But hi is right of target with his critique of OO. Object oriented programming is no silver bullet.

Dec 14, 2006 (InfoWorld) Joel Spolsky is one of our most celebrated pundits on the practice of software development, and he's full of terrific insight. In a recent blog post, he decries the fallacy of "Lego programming" -- the all-too-common assumption that sophisticated new tools will make writing applications as easy as snapping together children's toys. It simply isn't so, he says -- despite the fact that people have been claiming it for decades -- because the most important work in software development happens before a single line of code is written.

By way of support, Spolsky reminds us of a quote from the most celebrated pundit of an earlier generation of developers. In his 1987 essay "No Silver Bullet," Frederick P. Brooks wrote, "The essence of a software entity is a construct of interlocking concepts ... I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation ... If this is true, building software will always be hard. There is inherently no silver bullet."

As Spolsky points out, in the 20 years since Brooks wrote "No Silver Bullet," countless products have reached the market heralded as the silver bullet for effortless software development. Similarly, in the 30 years since Brooks published " The Mythical Man-Month" -- in which, among other things, he debunks the fallacy that if one programmer can do a job in ten months, ten programmers can do the same job in one month -- product managers have continued to buy into various methodologies and tricks that claim to make running software projects as easy as stacking Lego bricks.

Don't you believe it. If, as Brooks wrote, the hard part of software development is the initial design, then no amount of radical workflows or agile development methods will get a struggling project out the door, any more than the latest GUI rapid-development toolkit will.

And neither will open source. Too often, commercial software companies decide to turn over their orphaned software to "the community" -- if such a thing exists -- in the naive belief that open source will be a miracle cure to get a flagging project back on track. This is just another fallacy, as history demonstrates.

In 1998, Netscape released the source code to its Mozilla browser to the public to much fanfare, but only lukewarm response from developers. As it turned out, the Mozilla source was much too complex and of too poor quality for developers outside Netscape to understand it. As Jamie Zawinski recounts, the resulting decision to rewrite the browser's rendering engine from scratch derailed the project anywhere from six to ten months.

This is a classic example of the fallacy of the mythical man-month. The problem with the Mozilla code was poor design, not lack of an able workforce. Throwing more bodies at the project didn't necessarily help; it may have even hindered it. And while implementing a community development process may have allowed Netscape to sidestep its own internal management problems, it was certainly no silver bullet for success.

The key to developing good software the first time around is doing the hard work at the beginning: good design, and rigorous testing of that design. Fail that, and you've got no choice but to take the hard road. As Brooks observed all those years ago, successful software will never be easy. No amount of open source process will change that, and to think otherwise is just more Lego-programming nonsense.

McGee's Musings

The Mythical Man-Month Essays on Software Engineering REVIEWED BY TAL COHEN

From A Second Look at the Cathedral and Bazaar by Nikolai Bezroukov,
http://firstmonday.org/issues/issue4_12/bezroukov/index.html 

One of the most indefensible ideas of CatB (i.e. Raymond) is that Brooks' Law is non-applicable in the Internet-based distributed development environment as exemplified by Linux

The real problem with the CatB statement is that due to the popularity of CatB this statement could discourage OSS community from reading and studying The Mythical Man-Month, one of the few
computer science books that has remained current decades after its initial publication.

Actually the term "Brooks' Law" is usually formulated as "Adding manpower to a late software project makes it later".

I believe that the illusion of the non-applicability of "mythical man-month postulate" and Brooks' law is limited only to projects for which a fully functional prototype already exists and most or all architectural problems are solved...
 

Table of Contents

Recommended Links

Faculty Biography Frederick P. Brooks Jr. (UNC-CH Computer Science)

The Mythical Man-Month - Wikipedia, the free encyclopedia

Fred Brooks - Wikipedia, the free encyclopedia

Mythical Man-Month quotes

The mythical man-month (anniversary ed.) ACM ref list

ERCB The Mythical Man-Month, Anniversary Edition

Brooks' Law and open source The more the merrier

ONLamp.com The Mythical Man-Month Revisited -- a very superficial, but funny paper...

Amazon.com- Books- The Mythical Man-Month- Essays on Software ...

Mythical Man-Month, The- Essays on Software Engineering ...

Slashdot Review The Mythical Man Month Essays on Software Engineering

Ian Alexander's Reviews of Books on Requirements Engineering and Related Subjects - Brooks on Mythical Man-Month

FabTime Book Reviews The Mythical Man-Month by Frederick Brooks, Jr.

Reviews

There is an interesting review of the book by Ray Duncan in DDJ.

What is a mythical man-month anyway? Consider a moderately complex software application from the early microcomputer era, such as the primordial version of Lotus 1-2-3, Ashton-Tate dBASE, or Wordstar. Assume that such a program might take one very smart, highly-motivated, expert programmer approximately a year to design, code, debug, and document. In other words, 12 man-months. Imagine that market pressures are such that we want to get the program finished in a month, rather than a year. What is the solution? You might say, "get 12 experienced coders, divide up the work, let them all flog away for one month, and the problem will be solved. It's still 12 man-months, right?"

Alas, time cannot be warped so easily. Dr. Brooks observed, while he was managing the development of Operating System/360 (OS/360) in the early 1960's, that man-months are not -- so to speak -- factorable, associative, or commutative. 1 programmer * 12 months does not equal 12 programmers * 1 month. The performance of programming teams, in other words, does not "scale" in a linear fashion any more than the performance of multi-processor computer systems. He found, in fact, that when you throw additional programmers at a project that is late, you are only likely to make it more late. The way to get a project back on schedule is to remove promised-but-not-yet-completed features, rather than multiplying worker bees.

When you stop to think about it, this phenomenon is easy to understand. There is an inescapable overhead to yoking up programmers in parallel. The members of the team must "waste time" attending meetings, drafting project plans, exchanging EMAIL, negotiating interfaces, enduring performance reviews, and so on. In any team of more than a few people, at least one member will be dedicated to "supervising" the others, while another member will be dedicated to housekeeping functions such as managing builds, updating Gantt charts, and coordinating everyone's calendar. At Microsoft, there will be at least one team member that just designs T-shirts for the rest of the team to wear. And as the team grows, there is a combinatorial explosion such that the percentage of effort devoted to communication and administration becomes larger and larger.

The Mythical Man-Month -- an interesting review by Lane Core Jr.

Few books on software project management have been as influential and timeless as The Mythical Man-Month. With a blend of software engineering facts and thought-provoking opinions, Fred Brooks offers insight for anyone managing complex projects. [outside back cover, Twentieth Anniversary Edition, with four new chapters; Addison-Wesley, 1995]

Chapter 2: The Mythical Man-Month

Good cooking takes time. If you are made to wait, it is to serve you better, and to please you. Menu of Restaurant Antoine, New Orleans [page 13]

More software projects have gone awry for lack of calendar time than for all other causes combined. Why is this cause of disaster so common?

First, our techniques of estimating are poorly developed. More seriously, they reflect an unvoiced assumption which is quite untrue, i.e., that all will go well.

Second, our estimating techniques fallaciously confuse effort with progress, hiding the assumption that men and months are interchangeable.

Third, because we are uncertain of our estimates, software managers often lack the courteous stubbornness of Antoine's chef.

Fourth, schedule progress is poorly monitored. Techniques proven and routine in other engineering disciplines are considered radical innovations in software engineering.

Fifth, when schedule slippage is recognized, the natural (and traditional) response is to add manpower. Like dousing a fire with gasoline, this makes matters worse, much worse. More fire requires more gasoline, and thus begins a regenerative cycle which ends in disaster. [page 14]

Optimism

In many creative activities the medium of execution is intractable. Lumber splits; paints smear; electrical circuits ring. These physical limitations of the medium constrain the ideas that may be expressed, and they also create unexpected difficulties in the implementation. [page 15]

Computer programming, however, creates with an exceedingly tractable medium. The programmer builds from pure thought-stuff: concepts and very flexible representations thereof. Because the medium is tractable, we expect few difficulties in implementation; hence our pervasive optimism. Because our ideas are faulty, we have bugs; hence our optimism is unjustified. [page 15]

The Man-Month

The second fallacious thought mode is expressed in the very unit of effort used in estimating and scheduling: the man-month. Cost does indeed vary as the product of the number of men and the number of months. Progress does not. Hence the man-month as a unit for measuring the size of a job is a dangerous and deceptive myth. It implies that men and months are interchangeable.

When a task cannot be partitioned because of sequential constraints, the application of more effort has no effect on the schedule. The bearing of a child takes nine months, no matter how many women are assigned. Many software tasks have this characteristic because of the sequential nature of debugging. [page 17; emphasis in original]

Systems Test

In examining conventionally scheduled projects, I have found that few allowed one-half of the projected schedule for testing, but that most did indeed spend half of the actual schedule for that purpose. Many of these were on schedule until and except in system testing. [page 20]

Gutless Estimating

Observe that for the programmer, as for the chef, the urgency of the patron may govern the scheduled completion of the task, but it cannot govern the actual completion. An omelette, promised in two minutes, may appear to be progressing nicely. But when it has not set in two minutes, the customer has two choices--wait or eat it raw. Software customers have had the same choices.

The cook has another choice; he can turn up the heat. The result is often an omelette nothing can save--burned in one part, raw in another. [page 21]

Regenerative Schedule Disaster

Oversimplifying outrageously, we state Brooks's Law:

 

Adding manpower to a late software project makes it later.

This then is the demythologizing of the man-month. The number of months of a project depends upon its sequential constraints. The maximum number of men depends upon the number of independent subtasks. From these two quantities one can derive schedules using fewer men and more months. (The only risk is product obsolescence.) One cannot, however, get workable schedules using more men and fewer months. More software projects have gone awry for lack of calendar time than for all other causes combined. [pages 25-26; emphasis in original]

Chapter 11: Plan to Throw One Away

Two Steps Forward and One Step Back

The fundamental problem with program maintenance is that fixing a defect has a substantial (20-50 percent) chance of introducing another. So the whole process is two steps forward and one step back.

Why aren't defects fixed more cleanly? First, even a subtle defect shows itself as a local failure of some kind. In fact it often has system-wide ramifications, usually nonobvious. Any attempt to fix it with minimum effort will repair the local and obvious, but unless the structure is pure or the documentation very fine, the far-reaching effects of the repair will be overlooked. Second, the repairer is usually not the man who wrote the code, and often he is a junior programmer or trainee. [page 122]

Chapter 14: Hatching a Catastrophe

How does a project get to be a year late?... One day at a time. [page 153]

But the day-by-day slippage is harder to recognize, harder to prevent, harder to make up. Yesterday a key man was sick, and a meeting couldn't be held. Today the machines are all down, because lightning struck the building's power transformer. Tomorrow the disk routines won't start testing, because the first disk is a week late from the factory. Snow, jury duty, family problems, emergency meetings with customer, executive audits--the list goes on and on. Each one only postpones some activity by a half-day or a day. And the schedule slips, one day at a time. [page 154]

Milestones or Millstones?

For picking the milestones there is only one relevant rule. Milestones must be concrete, specific, measurable events, defined with knife-edge sharpness. Coding, for a counterexample, is "90 percent finished" for half of the total coding time. Debugging is "99 percent complete" most of the time. "Planning complete" is an event one can proclaim almost at will. [page 154]

Review  by Orville R. Weyrich, Jr.

I met this book shortly after its first publication, while I was a graduate student in chemistry who had just recently mastered the art of programming a Texas Instruments TI-59 calculator. I realized the limitations of the hand-held calculator when I managed to program some chemical analyses which required days of dedicated calculator-time to complete. At that point, I asked for an IBM-360 mainframe account and taught myself programming. Several thousand lines of Fortran-G later, I was ready for bigger and better things, and found The Mythical Man Month in the university bookstore.

Perhaps it is too melodramatic to say that The Mythical Man Month changed my life, so suffice it to say that after reading the book, I went on to get a job programming a PDP-11 for Knoxville Utility Board. After I got my degree in chemistry, I changed my career to software engineering and never looked back. Now almost twenty years later, after three years as Assistant Professor of Computer Science specializing in software engineering and a decade in private industry, this book still has a special place in my heart.

With the melodrama now dispatched, let me say that I wholeheartedly recommend this book for any manager of a software engineering project, and especially for anyone involved in a Year-2000 project.

A few more comments follow, arranged by chapter.

Chapter 1: The Tar Pit

Programming in-the-large seems to be like the tar pits that entrapped the prehistoric creatures such as the giant sloth, whose bones today grace the science library at the University of Georgia. Brooks writes:

 

Large-system programming has over the past decade been such a tar-pit, and many great and powerful beasts [including the IBM 360 project team] have thrashed violently in it. Most have emerged with running systems -- few have met goals, schedules, and budgets .... No one thing seems to cause the difficulty -- any particular paw can be pulled away. But the accumulation of simultaneous and interacting factors brings slower and slower motion. 

Chapter 2: The Mythical Man-Month

Develops the justification for "Brooks's Law"

 

Adding manpower to a late software project makes it later

Chapter 3: The Surgical Team

Following the lead of Harlan Mills, Brooks constructs an analogy between the roles necessary for a successful software engineering project, and the roles necessary for a surgical team. For many years, I have related best to the roles of "toolsmith" and "tester." 

Chapter 4: Aristocracy, Democracy, and System Design

In 1972 I was privileged to tour France as part of a joint expedition of the French and Religion departments of my college. Although I was there as a student of French, visits to many a Cathedral were on the itinerary due to the influence of the religion department. Brooks manages to make my experience relevant to software engineering:
 

Most European cathedrals show differences in plan or architectural style between parts built in different generations by different builders. The later builders were tempted to "improve" upon the designs of the earlier ones, to reflect the changes in fashion and differences in individual taste ... and the result proclaims the pridefulness of the builders as much as the glory of God.

Brooks goes on to relate this observation to the need for conceptual integrity in the design of a software system, and makes observations on how this can be achieved. 

Chapter 5: The Second System Effect

Beware the software engineer who has just completed his first successful system -- his next project runs the risk of dying of "featuritis". 

Chapter 6: Passing the Word

Say what you mean and mean what you say -- on the necessity of making the documentation match the implementation. 

Chapter 7: Why Did the Tower of Babel Fail?

Using additional analogies, Brooks points out that "The job least well done by project managers is to utilize the technical genius who is not strong on management talent." He then proceeds to illustrate his point with an example of how to do it right, taken from the fictional account of Heinlein's The Man Who Sold the Moon. 

Chapter 16: No Silver Bullet

This chapter, which was first published in IEEE Computer, [April, 1987] after the first edition of The Mythical Man Month, is included in the revised edition. It argues that CASE tools are useful adjuncts to software engineering, but are not a panacea.

Epilogue

I will give Frederick P. Brooks, Jr. the last word:
 

The tar pit of software engineering will continue to be sticky for a long time to come. Once can expect the human race to continue attempting systems just within or just beyond our reach; and software systems are perhaps the most intricate and complex of man's handiworks. The management of this complex craft will demand our best use of new languages and systems, our best adaptation of proven engineering management methods, liberal doses of common sense, and a God-given humility to recognize our fallibility and limitations.

Review The Mythical Man-Month Essays on Software Engineering

Review Discussion Mythical Man Month

Probably the biggest insight I took from the book (i.e. They talk about the "Mythical Man Month") is that the what kills timelines software projects more than anything else is communication overhead. Reducing the communication requirements between groups working on software is the single most important thing you can do during software development
----------------
Eric Raymond's "loophole" to Brooks law - primary development does not scale, debugging does.
-----------------
From A Second Look at the Cathedral and Bazaar by Nikolai Bezroukov, http://firstmonday.org/issues/issue4_12/bezroukov/index.html One of the most indefensible ideas of CatB (i.e. Raymond) is that Brooks' Law is non-applicable in the Internet-based distributed development environment as exemplified by Linux The real problem with the CatB statement is that due to the popularity of CatB this statement could discourage OSS community from reading and studying The Mythical Man-Month, one of the few computer science books that has remained current decades after its initial publication. Actually the term "Brooks' Law" is usually formulated as "Adding manpower to a late software project makes it later". I believe that the illusion of the non-applicability of "mythical man-month postulate" and Brooks' law is limited only to projects for which a fully functional prototype already exists and most or all architectural problems are solved
----------------
Open Source vs. Brook (i.e. more progs later projects).
----------------------
Now I am reading the newly published Extreme Programming Explained by Kent Beck. He describes a fear of touching another programmer's code as a factor in slowing down many projects. I first read MMM over twenty years ago. Since then, I have been looking for any sign that any manager I worked for had even heard of the book. So far, no joy. Which is really scary, considering who my present employer
-------------------------------------
Brook also managed the development of the IBM 360 Operating System." Note that while manager of the 360 project it was Dr. Brooks who specified that a byte would consist of 8 bits. Whether or not you agree with his decision, it's hard to argue that this has not had a huge impact on the computer field. Tanner Lovelace UNC Chapel Hill

My 1 1/2 Cents

Wow! Nobody seems to have noticed a minor detail... IBM OS 360 looks to have been a major screw up.
And that's the lesson: If you screw up with style, then people will applaud you anyway... Especially if you are a computer scientist. (:-)

After all, computer science is after the first principles of the game, not just the "Implementation business"...

---

Mr. Fred Brooks must have been a politician in his time. Being in command for several thousand programmers must be more complex than the Job of a General.

And that leads us to Mr. Machiavelli's work "The Prince". In his time, Mr. Machiavelli was a seniour politician in the Republic of Florence. Later the Medicis returned to Florence, the Republic was junked and so was Mr. Machiavelli. Out of favors, out of Job, he found himself in his Palazo under house arrest. The only thing to do was to write a book with his reflections on the process of gaining and maintaining power.
And what? He is sending his writings to the new powers. Do you think the Medicis are stupid enough to hire an all to clever civil servant of the previous regime? Bingo.

So people like Fred Brooks and Machiavelli are probably very clever but not clever enough. Fouche, for that matter, never wrote books - he was a man of action.

MMM vs. OSS advocado.com discussion

Off the top of my head..., posted 13 Nov 2000 by rillian

For example, according to Brooks, opening development up wide should result in basically an infinite time-to-completion for a project, since all effort gets sucked up in inter-developer training and communication.

 

I think this one is easily explained by the different organizational structures. When a commercial development team hires someone, there's an expectation on both sides now that they're "inside the wall" that someone must be supported and involved and given work to do. Else, why did you hire them? The lower chemical potential of open development means those doing the work are more-or-less free to ignore the training of new contributors, who must learn on their own. With no schedule, or a recognizance of the value of an educational budget, this is no problem.

 

Microsoft chimes in to remark that the reason OSS has worked is that anarchy is OK at producing copies of systems which are already fully architected. The "taillights are easy to follow" principle.

 

The advantage of "chasing taillights" is you don't have to do much work to achieve a shared vision among the contributors. It's that vision that is hardest to pass on. I understand taking advantage of this is one of the reasons behind GNOME's emulation of Microsoft products. But the effect applies just as well to designs just over the horizon. How often have you read a new project description and said "Yes, I see what they want to do"?

Others have remonstrated that OSS projects aren't really bazaars, but autocratic cathedrals run by the elect, and so are perfectly understandable.

It is true that one effective model is to have circles of contributors, where new contributors are "trained" by a circle of more experienced developers, who in turn defer to core members. But the term "autocratic cathedral" poorly captures the complexity of the situation. This model is only autocratic in that--for reasons of resource efficiency, lack of distinct vision and respect for their technical competence--everyone follows the decisions made by the core members. And one is free to move between these roles as time and interest permit. In other words, not autocratic at all.

Perhaps (to further mix metaphors) this is a surgical team, drawing their new members from recent graduates of a custom-tailored university degree program, crossed with the social freedom one has in starting/joining a minor shared-interest club in a large city. I don't think there's much interest to be found in calling that "traditional top-down design" and understood.

On a more philosophical note, stephan's point about historical context is a valuable one, and applies just as well to the The Cathedral and the Bazaar. We shouldn't expect to find directly applicable rules in seminal works of this sort. Initial theories often have limited scope or are found to be wrong on some (even many) details. But the central idea still has value, and is either incorporated in later theories, or develops into a sophisticated field in its own right, sharing only the name and central premise of the original. (Darwin is a good example here.) We should keep this in mind and not be surprised when bold claims don't apply to our situation.

As a great friend used to say, "I think we can complicate this a bit." :-)

Mythical Man-Month

The essays in this book are concise, clear, and eminently readable. Brooks has a way of forcing you to question your implicit assumptions. In the title chapter on schedule slippage, he drives the reader humorously, but relentlessly, to this conclusion:

Adding manpower to a late software project makes it later.

In another essay, Brooks points out:

Chemical engineers learned long ago that a process that works in the laboratory cannot be implemented in a factory in one step. An intermediate step called the pilot plant is necessary....In most [software] projects, the first system is barely usable. It may be too slow, too big, awkward to use, or all three. There is no alternative but to start again, smarting but smarter, and build a redesigned version in which these problems are solved.... Delivering the throwaway to customers buys time, but it does so only at the cost of agony for the user, distraction for the builders while they do the redesign, and a bad reputation for the product that the best redesign will find hard to live down. Hence, plan to throw one away; you will, anyhow.

The 20th anniversary edition includes all of the chapters of the original book, plus a lot of new material. The chapter "No Silver Bullet--Essence and Accident in Software Engineering" was first published in 1986. Brooks asserted that no single software engineering development would produce an order-of-magnitude improvement in programming productivity within ten years. While this paper caused a lot of rebuttal in the software engineering community, Brooks was right--there has been "No Silver Bullet."

The other chapters discuss why this is so and Brooks points out one of his mistakes in the first edition--"David Parnas Was Right, and I Was Wrong About Information Hiding." Brooks states, "I am now convinced that information hiding, today often embodied in object programming, is the only way of raising the level of software design."

If you create, maintain, manage, or are involved in part of the software engineering process, you must read The Mythical Man Month. From the hard-nosed advice to the enjoyable anecdotes, you will enjoy this book.

Notes on Fred Brooks' 'The Mythical Man-Month'

Excerpt from The Mythical Man-Month

Why is programming fun? What delights may its practitioner expect as his reward?

First is the sheer joy of making things. As the child delights in his mud pie, so the adult enjoys building things, especially things of his own design. I think this delight must be an image of God's delight in making things, a delight shown in the distinctiveness of each leaf and each snowflake.

Second is the pleasure of making things that are useful to other people. Deep within, we want others to use our work and to find it helpful. In this respect the programming system is not essentially different from the child's first clay pencil holder "for Daddy's office."

Third is the fascination of fashioning complex puzzle-like objects of interlocking moving parts and watching them work in subtle cycles, playing out the consequences of principles built in from the beginning. The programmed computer has all the fascination of the pinball machine or the jukebox mechanism, carried to the ultimate.

Fourth is the joy of always learning, which springs from the nonrepeating nature of the task. In one way or another the problem is ever new, and its solver learns something: sometimes practical, sometimes theoretical, and sometimes both.

Finally, there is the delight of working in such a tractable medium. The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by exertion of the imagination. Few media of creation are so flexible, so easy to polish and rework, so readily capable of realizing grand conceptual structures. (As we shall see later, this tractability has its own problems.)

Yet the program construct, unlike the poet's words, is real in the sense that it moves and works, producing visible outputs separately from the construct itself. It prints results, draws pictures, produces sounds, moves arms. The magic of myth and legend has come true in our time. One types the correct incantation on a keyboard, and a display screen comes to life, showing things that never were nor could be.

Programming then is fun because it gratifies creative longings built deep within us and delights sensibilities we have in common with all men.

No Silver Bullet Essence and Accidents of Software Engineering

From The Mythical Man-Month, Anniversary Edition, pages 7-8.

No Silver Bullet Essence and Accidents of Software Engineering -- Famous paper by F. Brooks

Fashioning complex conceptual constructs is the essence; accidental tasks arise in representing the constructs in language. Past progress has so reduced the accidental tasks that future progress now depends upon addressing the essence.

Of all the monsters that fill the nightmares of our folklore, none terrify more than werewolves, because they transform unexpectedly from the familiar into horrors. For these, one seeks bullets of silver that can magically lay them to rest.

The familiar software project, at least as seen by the non-technical manager, has something of this character; it is usually innocent and straightforward, but is capable of becoming a monster of missed schedules, blown budgets, and flawed products. So we hear desperate cries for a silver bullet--something to make software costs drop as rapidly as computer hardware costs do.

But, as we look to the horizon of a decade hence, we see no silver bullet. There is no single development, in either technology or in management technique, that by itself promises even one order-of-magnitude improvement in productivity, in reliability, in simplicity. In this article, I shall try to show why, by examining both the nature of the software problem and the properties of the bullets proposed.

Skepticism is not pessimism, however. Although we see no startling breakthroughs--and indeed, I believe such to be inconsistent with the nature of software--many encouraging innovations are under way. A disciplined, consistent effort to develop, propagate, and exploit these innovations should indeed yield an order of-magnitude improvement. There is no royal road, but there is a road.

The first step toward the management of disease was replacement of demon theories and numerous theories by the germ theory. That very step, the beginning of hope, in itself dashed all hopes of magical solutions. It told workers that progress would be made stepwise, at great effort, and that a persistent, unremitting care would have to be paid to a discipline of cleanliness. So it is with software engineering today.

Does it have to be hard? --Essential difficulties

Not only are there no silver bullets now in view, the very nature of software makes it unlikely that there will be any--no inventions that will do for software productivity, reliability, and simplicity what electronics, transistors, and large-scale integration did for computer hardware.

We cannot expect ever to see two fold gains every two years.

First, one must observe that the anomaly is not that software progress is so slow, but that computer hardware progress is so fast. No other technology since civilization began has seen six orders of magnitude in performance-price gain in 30 years. In no other technology can one choose to take the gain in either improved performance or in reduced costs. These gains flow from the transformation of computer manufacture from an assembly industry into a process industry.

Second, to see what rate of progress one can expect in software technology, let us examine the difficulties of that technology. Following Aristotle, I divide them into essence, the difficulties inherent in the nature of software, and accidents, those difficulties that today attend its production but are not inherent.

The essence of a software entity is a construct of interlocking concepts: data sets, relationships among data items, algorithms, and invocations of functions. This essence is abstract in that such a conceptual construct is the same under many different representations. It is nonetheless highly precise and richly detailed.

I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation. We still make syntax errors, to be sure; but they are fuzz compared with the conceptual errors in most systems.

If this is true, building software will always be hard. There is inherently no silver bullet.

Let us consider the inherent properties of this irreducible essence of modern software systems: complexity, conformity, changeability, and invisibility.

Complexity. Software entities are more complex for their size than perhaps any other human construct because no two parts are alike (at least above the statement level). If they are, we make the two similar parts into a subroutine--open or closed. In this respect, software systems differ profoundly from computers, buildings, or automobiles, where repeated elements abound.

Digital computers are themselves more complex than most things people build: They have very large numbers of states. This makes conceiving, describing, and testing them hard. Software systems have orders-of-magnitude more states than computers do.

Likewise, a scaling-up of a software entity is not merely a repetition of the same elements in larger sizes; it is necessarily an increase in the number of different elements. In most cases, the elements interact with each other in some nonlinear fashion, and the complexity of the whole increases much more than linearly.

The complexity of software is an essential property, not an accidental one. Hence, descriptions of a software entity that abstract away its complexity often abstract away its essence. For three centuries, mathematics and the physical sciences made great strides by constructing simplified models of complex phenomena, deriving properties from the models, and verifying those properties by experiment. This paradigm worked because the complexities ignored in the models were not the essential properties of the phenomena. It does not work when the complexities are the essence.

Many of the classic problems of developing software products derive from this essential complexity and its nonlinear increases with size. From the complexity comes the difficulty of communication among team members, which leads to product flaws, cost overruns, and schedule delays. From the complexity comes the difficulty of enumerating, much less understanding, all the possible states of the program, and from that comes the unreliability. From complexity of function comes the difficulty of invoking function, which makes programs hard to use. From complexity of structure comes the difficulty of extending programs to new functions without creating side effects. From complexity of structure come the unvisualized states that constitute security trapdoors.

Not only technical problems, but management problems as well come from the complexity. It makes overview hard, thus impeding conceptual integrity. It makes it hard to find and control all the loose ends. It creates the tremendous learning and understanding burden that makes personnel turnover a disaster.

Conformity. Software people are not alone in facing complexity. Physics deals with terribly complex objects even at the "fundamental" level [BJC:]. The physicist labors on, however, in a firm faith that there are unifying principles to be found, whether in quarks or in unified field theory. Einstein argued that there must be simplified [BJC: undecipherable] of nature, because God is not capricious or arbitrary.

No such faith comforts the software engineer. Much of the complexity that he must master is arbitrary complexity, forced without rhyme or reason by the many human institutions and systems to which his interfaces must conform. These differ from interface to interface, and from time to time, not because of necessity but only because they were designed by different people, rather than by God.

In many cases, the software must conform because it is the most recent arrival on the scene. In others, it must conform because it is perceived as the most conformable. But in all cases, much complexity comes from conformation to other interfaces; this complexity cannot be simplified out by any redesign of the software alone.

Humor

Discussion Mythical Man Month

My 1 1/2 Cents

Wow! Nobody seems to have noticed a minor detail... IBM OS 360 looks to have been a major screw up. And that's the lesson: If you screw up with style, then people will applaud you anyway... Especially if you are a computer scientist. (:-)

After all, computer science is after the first principles of the game, not just the "Implementation business"...

---

http://www.onlamp.com/lpt/a/4900

Published on ONLamp.com (http://www.onlamp.com/)
 http://www.onlamp.com/pub/a/onlamp/2004/06/17/mmm_revisited.html
 See this if you're having trouble printing code examples

The Mythical Man-Month Revisited

by Ed Willis 06/17/2004

Surely everyone in development has heard of The Mythical Man-Month if for no other reason than its presentation of Brooks' Law: "Adding manpower to a late project makes it later." Having finally read it in its entirety, though, I can say that it's like a time capsule — simultaneously you can see just how much the field has changed since the original writing and just how much has stayed stubbornly the same.

I have had comparatively recent introduction to the field of software development, being only eight or so years out of school. Excepting my efforts in shell scripting to get in touch with my ancient Unix ancestors, I really know nothing about the history of my profession and how it was for those who worked through the early days that defined our industry. Brooks' writing style is fairly dry and formal — seemingly the prose comes to the reader from the time of Arthur Conan Doyle rather than from the time of the Rolling Stones — and this serves to intensify the perception of antiquity.

To say that this book has deepened my appreciation of just how much I take for granted in my field and how hard-won all those little gains were would be to understate the matter.

It was hard to keep my thoughts on what I was reading, as my reactions to the material took off in my mind. That said, I might characterize my varied reactions as:

• Impudent reactions to the modus operandi of the elder days of yore.
• Recognizing ideas that foreshadowed significant advents in the field later on.
• Recognizing ideas that time has not been overly kind to.
• Reacting to the essential ideas Brooks presents.

And so, without further ado ...

Impudent Reactions to the Modus Operandi of the Elder Days of Yore

Ultimately, most of the amusing excerpts I've singled out below trace back to assumptions regarding the existence of a technique, process, or technology that we take for granted now but which had yet to be invented or was in its infancy at the time of the writing. The field of software development, after all, started out not so long ago with only some hardware and some mathematics before it blossomed into a field full of craft and its own science. Brooks wrote from a time far closer to those origins. In a sense, that these things below seem funny now is an indication of how far the field has come.

I'm not likely doing myself any favors etching these in electronic stone. I can already hear the condemnation of the older professors I've worked with ringing in my ears as I write this. But the truth is, this book had me in hysterics so frequently that to avoid this aspect would seem utterly remiss to me. For example:

"Teams building new supervisors or other system-heart software will of course need machines of their own. Such systems will need operators and a system programmer or two who keeps the standard support on the machine current and serviceable."

I just bet this is the root of all my problems — I have not one but two machines all to myself at work. Do I have any systems programmers or operators? Not a one. It's a miracle I can accomplish anything at all, under the circumstances.

Regarding source code documentation:

"The most serious objection is the increase in the size of the source code that must be stored. As the discipline moves more and more toward on-line storage of source code, this has become a growing consideration. I find myself being briefer in comments to an APL program, which will live on disk, then on a PL/I one that I will store as cards."

For who among us is this not true? Honestly, you just can't shut me up on cards.

On estimation:

"My guideline in the morass of estimating complexity is that compilers are three times as bad as normal batch application programs, and operating systems are three times as bad as compilers."

Pretty much everything I've ever worked on would be "batch application programs" then. At least they're only one ninth as bad as operating systems.

"Consider the IBM APL interactive software system. It rents for $400 per month and, when used, takes at least 160K bytes of memory. On a Model 165, memory rents for about $12 per kilobyte per month. If the program is available full-time, one pays $400 software rent and $1920 memory rent for using the program."

I sure hope someone's been paying my software and memory rent. I haven't been, at least the last little while.

"... we went to allocating machine time in substantial blocks. The whole 15-man sort team, for example, would be given a system for a four-to-six-hour block."

A 15-man sort team — a whole baseball team, with pinch-hitters and relievers — for sorting!

"Operating systems, loudly decried in the 1960s for their memory and cycle costs, have proved to be an excellent form in which to use some of the MIPS and cheap memory bytes on the past hardware surge."

Is the jury really in on operating systems? Let's not be too hasty now. The irony of the situation is that, in at least embedded systems, this question is still very much up for debate.

"Many users now operate their own computers day in and day out on varied applications without ever writing a program. Indeed, many of these users cannot write new programs for their machines, but they are nevertheless adept at solving new problems with them."

Strange but true. They're also pretty good at causing new problems with them too, in my experience.

At various points through the book Brooks discusses the effective use of the "transient area" and how vital an issue it is to the success of any project. I was, of course, deeply alarmed to discover that I had no idea what he was talking about. At the end of the book, Brooks, writing much closer to the present day, critiques his earlier work and revisits this point:

"The size of the transient area, hence the amount of program per disk fetch, is a crucial decision, since performance is a super-linear function of that size. [This whole decision has been obsoleted, first by virtual memory, then by cheap real memory. Users now typically buy enough real memory to hold all the code of major applications.]"

I feel much better now.

Ideas That Foreshadowed Significant Advents in the Field

The earlier section aside, there is much to recommend in this book. Brooks is a smart guy looking critically at his field and finding much opportunity for improvement. Many later development practices embody his observations.

"The sooner one puts the pieces together, the sooner the system bugs will emerge. Somewhat less sophisticated is the notion that by using the pieces to test each other, one avoids a lot of test scaffolding. Both of these are obviously true, but experience shows that they are not the whole truth — the use of clean, debugged components saves much more time in system testing than that spent on scaffolding and thorough component test."

The key distinction here for me is the notion of the components being tested and debugged in isolation, separated from the rest of the system. This is essentially the same technique as is practiced in Extreme Programming (XP), that is, XUnit-style unit testing.

There, as well, adherents argue that creating these separate test scaffoldings for each component costs less than the expense of fixing component-level defects at integration time. In fact, XP's argument is even stronger; with a highly evolutionary and iterative model, these test sets are part of the weight that is moved around when it comes time to make new changes. Still it's telling that the value of this approach to the creation of quality was evident to Brooks 15 years or so in advance of XP. In another spot, Brooks mentions:

"... interesting results show that three times as much progress in interactive debugging is made on the first interaction of each session as on subsequent interactions."

The real irony here is that his main point is that debugging sessions should be short, whereas the XUnit take on this would be that the value of debugging drops dramatically as the developer does it, so we should aim to do as little of it as possible. This is certainly one of the motivations of XUnit testing. It's unsurprising that this idea would have eluded Brooks, as the interactivity of the short think-code-test-debug cycle upon which XUnit rests was a long way off in the future when Brooks wrote this.

"For picking milestones there is only one relevant rule. Milestones must be concrete, specific, measurable events, defined with knife-edge sharpness. Coding, for example, is '90 percent finished' for half of the total coding time. Debugging is '99 percent complete' most of the time. 'Planning complete' is an event one can proclaim almost at will."

The first time I encountered similar statements was in Steve McConnell's Rapid Development. As Brooks himself points out, software is ethereal "thought-stuff." Many things come out of this intangibility. William Burroughs talks about the inability to fake certain things (a good meal, for example). The only milestone that is like this in software development is the release itself. All others may be fake, and the further they are from the release, the easier they are to fake. This is all the more reason to be careful in defining and evaluating non-implementation milestones.

Going further, many Agile enthusiasts want to do away with non-implementation phases of development (or at least reduce the effort put into non-implementation work products) and put the emphasis much more squarely on the implementation itself.

Regarding non-implementation work products, especially documentation, Brooks wrote:

"A basic principle of data processing teaches the folly of trying to maintain independent files in synchronization ... Yet our practice in programming documentation violates our own teaching. We typically attempt to maintain a machine-readable form of a program and an independent set of human-readable documentation, consisting of prose and flowcharts ... The solution, I think, is to merge the files, to incorporate the documentation in the source program."

Obviously this foreshadows the movement toward placing ever greater emphasis on internal documentation as is witnessed in the advent of Javadoc, Doxygen, and the remaining host of source code documentation tools. I see these efforts leading to the minimization of external design artifacts as these tools make it tempting to forgo formal design documents altogether in favor of embedding the design documentation in the code itself.

XUnit-style testing reduces the need for requirements documents in a similar, though less dramatic fashion; unit requirements, at least, exist in an executable and readable form in the unit tests themselves. One motivation for both of these techniques is that, of all the work products we can create, we can't avoid creating and maintaining the implementation itself. Let's load the implementation with as much value as humanly possible. That Brooks saw the value of some of these possibilities so early is encouraging.

Ideas Time Has Not Been Overly Kind To

Brooks wrote about waterfall lifecycle models:

"Much of present-day software acquisition procedures rests upon the assumption that one can specify a satisfactory system in advance, get bids for its construction, have it built, and install it. I think this assumption is fundamentally wrong, and that many software acquisition problems spring from that fallacy. Hence they can not be fixed without fundamental revision, one that provides for iterative development and specification of prototypes and products."

Here Brooks is clearly decrying the waterfall lifecycle model and is on the verge of embracing true iterative development, stopping seemingly just shy of recommending iterative development of the actual shipping implementation. Elsewhere he notes:

"Lehman and Belady offer evidence that quanta [of updates to software] should be very large and widely spaced or else very small and frequent. The latter strategy is more subject to instability according to their model. My experience confirms it: I would never risk that strategy in practice."

Recent history, at least, favors the opposite position. I think most organizations, likely in many cases without any real decision on the matter, practice something akin to the continuous integration favored by XP and end up with small and frequent quanta. I doubt there's much support for the other position now, but consider this later passage:

"In most projects, the first system built is barely usable. It may be too slow, too big, awkward to use, or all three. There is no alternative but to start again, smarting but smarter, and build a redesigned version in which these problems are solved. The discard and redesign may be done in one lump, or it may be done piece-by-piece. But all large-system experience shows that it will be done."

The piece-wise redesign sounds like refactoring, but I don't think it is. I think he's saying you will invariably throw away the whole implementation either all in one go or a little bit at a time, so it's wise to "plan to throw one away." I still hear people say this sometimes. This is probably not acceptable now — certainly I'd be embarrassed to have to do this. But this is the world in which Brooks lived when he wrote this book. Even in a lifecycle that tried to reject change after gathering the requirements (or maybe because of this) we still end up throwing one away.

It's easy to see why Brooks couldn't fully justify the essential invitation of change through the development cycle that characterizes evolutionary prototyping, XP, and other iterative methodologies, although he could certainly see the possible value in it.

Bear in mind that Boehm hadn't yet finished his work in estimating the costs of change when Brooks wrote this. This work would ultimately lead to the widely held belief that failing to catch defects very close to the point of their introduction imposed costs exponential in the amount of time between introduction and discovery. This latter idea is still entrenched in our industry despite the fact that, if it were true, practices that allow for change throughout the development lifecycle would be exponentially more expensive than would be the waterfall model, which they clearly are not. (N.B. They are inarguably more expensive, they are just not exponentially so). Equally one still hears variants of this:

"The fundamental problem with software maintenance is that fixing a defect has a substantial (20-50 percent) chance of introducing another."

I do not believe the risks to be this high now in any reasonably well-run organization. They may come close to 20 but should be nowhere near 50 percent. In short, we can claim have become better at maintenance over the past 30 years. Brooks, though, had to play the ball where it lay at the time he was writing and so would not have seen some possibilities we enjoy today as being legitimate or responsible then.

On overall system design and requirements:

"Often the fresh concept does come from an implementer or from a user. However, all my own experience convinces me, and I have tried to show, that the conceptual integrity of a system determines its ease of use. Good features and ideas that do not integrate with a system's basic concepts are best left out. If there appear many such important but incompatible ideas, one scraps the whole system and starts again on an integrated system with different basic concepts."

There is a certain smugness at work in the idea that the architect will make better decisions here than the user will. Certainly this view is out of favor now. We normally try to find out what the user wants (somehow) and then find a way to design our software to provide this to them in the most sensible manner we can envision. I can't imagine saying "no" to the user regarding a feature just because it doesn't fit into my current conceptual view of the system, and the notion of throwing out the current system so we can devise a better one that embodies all the features we want is a luxury no one can afford.

Plainly put, our job as software developers is to distill the system's conceptual integrity given the user's requirements. It's not our job to pick over the user's requests, looking for some set of functions that makes sense as a whole to us. It is also our job to take our lemons and make lemonade. We don't have the option to throw out our organization's software inventory when it doesn't match up well enough with new requirements. We must find a way to refactor that inventory toward a design that accepts (however grudgingly) the complete requirements, both old and new.

"A discipline that will open an architect's eyes is to assign each little function a value: capability x is worth not more than m bytes of memory and n microseconds per invocation. These values will guide initial decisions and serve during implementation as a guide and warning to all."

Even in embedded development where I make my living, I rarely see anything like this level of budgeting detail. I'm sure it was an absolute necessity in the hardware-poor past, though it makes me awfully glad to live in the current age of hardware excess.

On source code control and configuration management:

"First, each group or programmer had an area where he kept copies of his programs, his test cases, and the scaffolding he needed for component testing. In this playpen area there were no restrictions on what a man could do with his own programs.... When a man had his component ready for integration into a larger piece, he passed a copy over to to the manager of that larger system, who put this copy into a system integration sublibrary..."

I shudder to think how miserable and invariably risky this manual approach must have been, but what alternative was there? Even the advent of RCS was still many years in the future.

Brooks spends a lot of time in the book on the subject of identifying and training architects and designers: "How to grow great designers? ... systematically identify top designers ... assign a career mentor to be responsible for the development of the prospect ... devise and maintain a career development plan for each prospect ..."

The sad thing here is that in most development organizations I know of, design is not a desirable thing in its own right. Outside of the development group itself no one knows how to design, or even if anyone does, the issues of how to identify, train, use, and retain top design talents never actually come up.

Essential Ideas

Brooks presents several essential ideas to consider.

The Surgical Team

The surgical team is Brooks' proposed development team model. At its head is the chief programmer or surgeon. Everyone else supports him. It defines the following roles:

• Surgeon/Chief Programmer, who does actual development, design, testing, and documentating.
• Copilot, the Surgeon's right hand man.
• Administrator, who handles money, space, equipment, etc.
• Editor, a technical writer who finishes the Surgeon's documentation.
• Secretaries, one each for the Administrator and Editor.
• Program Clerk, who manages technical records for the team./li>
• Toolsmith, who manages custom tools for the team and the development environment.
• Tester, who defines and executes test cases.
• Language Lawyer, a programming language expert for some programming language of interest.

Even with the provisos listed in the book (one Language Lawyer can support two or three Surgeons, the Administrator may be able to look after two teams) this all seems excessive. I'm not clear at all on what the Programming Clerk does even after reading through the description a couple of times. I doubt the two Secretaries are truly necessary.

From what I do understand, one good Secretary or Administrator could likely look after all of the duties assigned to the two Secretaries, the Administrator, and the Programming Clerk. Also, the Tester or Toolsmith could handle many of the Programming Clerk's duties. I expect that the Toolsmith role would be controversial now — it's not clear to me that permitting each development team to vary with respect to their tool sets and development environments is either desirable or necessary, but people with these skills are obviously needed in the organization. I'd expect teams to share this role in practice.

Paring this back to the Surgeon, Copilot, Tester, Secretary/Administrator, and Editor would likely suffice for the team itself. An organization that supports multiple teams could provide the Language Lawyer and Toolsmith, or the team could subsume the roles itself.

Would such teams use their manpower effectively? Absolutely — if for no other reason than the well-defined roles. Each person knows what he must do and, perhaps more importantly, what's outside his purview. This alone is a welcome change from the relative chaos of role definition in most organizations. Frequently more than one person is doing the same thing, while unnervingly whole aspects of the work exist that no one is actually doing.

In many organizations also, technical decision-making has no clear process, with people haphazardly CC:d on emails, email threads that go on with no clear direction, and subjects eventually exhausted with no obvious outcome. At least in the proposal, technical decision-making clearly rests first and foremost with the Surgeon, who may delegate at her discretion. Being unambiguous about who is responsible for making the decisions is the first step in making them, I expect. It's clear that this is the Surgeon's responsibility.

This model also takes advantage of the well-documented disparity in measured productivity between programmers. The Surgeon is your star, the Copilot the understudy. Everyone else must ensure that these two people can maximize their contributions. That's smart.

Brooks puts forward his proposal as much for its scalability as for its optimization of productivity. Larger projects would have a set of these teams. Decisions requiring the input or agreement of more than one team — and ultimately the entirety of the architecture, which presumably the whole set of teams would have to ratify — would require the attention and involvement of the Surgeons, rather than the whole teams. In this, the surgical team proposal is reminiscent of the Scrum of Scrums and similar proposals for scaling XP projects.

The main disadvantage of the proposal is that it may reinforce the tribal boundaries within a larger organization where the characteristics and standards of each team diverge based on the tastes of the Surgeons that lead them. All in all, though, I'd expect the good points here to outweigh the bad. Allowing your A guys to do the A work and being painfully clear on who should make decisions would help many organizations enormously.

No Silver Bullets

If there's one thing that will stay with me from reading The Mythical Man-Month, it's Brooks' discussion of accident and essence in this essay. The central conjecture that drives this essay is this:

"There is no single development, in either technology or management technique, which by itself promises even one order-of-magnitude improvement within a decade in productivity, in reliability or simplicity."

Likely most have heard (or mis-heard) this as I did in a degenerate form of something like "development productivity can not increase by an order of magnitude." This is most definitely not what he's saying. He admits freely the possibility that combinations of improvements may yield this order-of-magnitude improvement — he draws the line at single factors. So there is no one, single silver bullet. This is an important distinction, because once understood, it becomes clear that this statement was probably true then and is in all likelihood true today. Knowing this is a tremendous boon in sorting out the nonsense from the truth in development.

As recently as a few weeks ago, I saw more than one order of magnitude improvement in development productivity attributed to the adoption of a particular set of process improvements. Without even trying to sort out whether manipulating a single factor produced this effect or more than one, I felt confident that either the presenter or Brooks was wrong. My money's on the presenter.

These people are typically not trying to pull the wool over your eyes intentionally — they believe what they're saying. Consider the many people still beating the "defect phase containment will save you orders of magnitude in effort" drum. Today, this is probably true only at the extremes, in really large projects, projects with really stringent quality requirements, or projects staffed with unusually bad teams. Counter evidence is ubiqutuous, but people still tell this story. As professionals, we have a responsibility to sort the wheat from the chaff. Brooks's conjecture is a great tool to bring to bear in this effort.

Brooks supports his conjecture with an inspired discussion that divides the world of software development into accident and essence:

"The essence of a software entity is a construct of interlocking concepts: data sets, relationships among data items, algorithms, and invocations of function. The essence is abstract, in that the conceptual construct is the same under many different representations. It is nonetheless highly precise and richly detailed. ... I believe the hard part of building software to be the specification, design and testing of this conceptual construct, not the labor or representing it and testing the fidelity on the representation."

This is the essence. The accident is everything else involved in software development. The details of the programming language, the configuration management, the modeling language, the packaging, documentation tools, libraries, build tools and so on are all accidental work in software development. Clearly there's lots of accidental work. Here's why what Brooks is saying makes so much sense — no one single area of software development is so badly burdened by accidental work that improving it can yield an order-of-magnitude improvement in overall productivity, reliability or simplicity.

My other realization from this reading is that, while I have never characterized myself (as so many do) as a C guy, a C++ guy, a Java guy, or what have you, I have most definitely prided myself on being a good generalist with a decent background on programming languages, build environments, libraries, installers, and operating systems.

The idea of the essence and accident of software development makes plain where continuing study has the most effect. Anything that improves skills in the accidents of development can only be of benefit in particular software niches whereas study to improve skills in the essence of development necessarily has to be a benefit in all domains.

Ed Willis works in telecommunications.

Producer's Note: Check out this discussion on The Mythical Man-Month that took place on java.net in May. The discussion has since been archived but there are many interesting posts and opinions.

Quotes

Good cooking takes time. If you are made to wait, it is to serve you better, and to please you. Menu of Restaurant Antoine, New Orleans. [page 13]

More software projects have gone awry for lack of calendar time than for all other causes combined. Why is this cause of disaster so common?

First, our techniques of estimating are poorly developed. More seriously, they reflect an unvoiced assumption which is quite untrue, i.e., that all will go well.

Second, our estimating techniques fallaciously confuse effort with progress, hiding the assumption that men and months are interchangeable.

Third, because we are uncertain of our estimates, software managers often lack the courteous stubbornness of Antoine's chef.

Fourth, schedule progress is poorly monitored. Techniques proven and routine in other engineering disciplines are considered radical innovations in software engineering.

Fifth, when schedule slippage is recognized, the natural (and traditional) response is to add manpower. Like dousing a fire with gasoline, this makes matters worse, much worse. More fire requires more gasoline, and thus begins a regenerative cycle which ends in disaster. [page 14]

In many creative activities the medium of execution is intractable. Lumber splits; paints smear; electrical circuits ring. These physical limitations of the medium constrain the ideas that may be expressed, and they also create unexpected difficulties in the implementation. [page 15]

Computer programming, however, creates with an exceedingly tractable medium. The programmer builds from pure thought-stuff: concepts and very flexible representations thereof. Because the medium is tractable, we expect few difficulties in implementation; hence our pervasive optimism. Because our ideas are faulty, we have bugs; hence our optimism is unjustified. [page 15]

The second fallacious thought mode is expressed in the very unit of effort used in estimating and scheduling: the man-month. Cost does indeed vary as the product of the number of men and the number of months. Progress does not. Hence the man-month as a unit for measuring the size of a job is a dangerous and deceptive myth. It implies that men and months are interchangeable.

When a task cannot be partitioned because of sequential constraints, the application of more effort has no effect on the schedule. The bearing of a child takes nine months, no matter how many women are assigned. Many software tasks have this characteristic because of the sequential nature of debugging. [page 17; emphasis in original]

In examining conventionally scheduled projects, I have found that few allowed one-half of the projected schedule for testing, but that most did indeed spend half of the actual schedule for that purpose. Many of these were on schedule until and except in system testing. [page 20]

Observe that for the programmer, as for the chef, the urgency of the patron may govern the scheduled completion of the task, but it cannot govern the actual completion. An omelette, promised in two minutes, may appear to be progressing nicely. But when it has not set in two minutes, the customer has two choices--wait or eat it raw. Software customers have had the same choices.

The cook has another choice; he can turn up the heat. The result is often an omelette nothing can save--burned in one part, raw in another. [page 21]

Adding manpower to a late software project makes it later.

This then is the demythologizing of the man-month. The number of months of a project depends upon its sequential constraints. The maximum number of men depends upon the number of independent subtasks. From these two quantities one can derive schedules using fewer men and more months. (The only risk is product obsolescence.) One cannot, however, get workable schedules using more men and fewer months. More software projects have gone awry for lack of calendar time than for all other causes combined. [pages 25-26; emphasis in original]

The fundamental problem with program maintenance is that fixing a defect has a substantial (20-50 percent) chance of introducing another. So the whole process is two steps forward and one step back.

Why aren't defects fixed more cleanly? First, even a subtle defect shows itself as a local failure of some kind. In fact it often has system-wide ramifications, usually nonobvious. Any attempt to fix it with minimum effort will repair the local and obvious, but unless the structure is pure or the documentation very fine, the far-reaching effects of the repair will be overlooked. Second, the repairer is usually not the man who wrote the code, and often he is a junior programmer or trainee. [page 122]

How does a project get to be a year late?... One day at a time. [page 153]

But the day-by-day slippage is harder to recognize, harder to prevent, harder to make up. Yesterday a key man was sick, and a meeting couldn't be held. Today the machines are all down, because lightning struck the building's power transformer. Tomorrow the disk routines won't start testing, because the first disk is a week late from the factory. Snow, jury duty, family problems, emergency meetings with customer, executive audits--the list goes on and on. Each one only postpones some activity by a half-day or a day. And the schedule slips, one day at a time. [page 154]

For picking the milestones there is only one relevant rule. Milestones must be concrete, specific, measurable events, defined with knife-edge sharpness. Coding, for a counterexample, is "90 percent finished" for half of the total coding time. Debugging is "99 percent complete" most of the time. "Planning complete" is an event one can proclaim almost at will. [page 154]

Etc

Brooks on optimism of programmers

All programmers are optimists. Perhaps this modern sorcery especially attracts those who believe in happy endings and fairy godmothers. Perhaps the hundreds of nitty frustrations drive away all but those who habitually focus on the end goal. Perhaps it is merely that computers are young, programmers are younger, and the young are always optimists. But however the selection process works, the result is indisputable:
"This time it will surely run," or "I just found the last bug."

-- Frederick Brooks, Jr., The Mythical Man Month

Brooks' Law and open source The more the merrier

Last modified: February 28, 2008