Software Craftsmanship: The New Imperative

Pete McBreen

A Book Review by

Glen B. Alleman

www.niwotridge.com

October 2, 2001

Copyright ©, 2001

 

Overview

Software Craftsmanship presents a view that software developers should return to their craftsman roots in order to deal with the increasing complexity of today's development demands. McBreen makes the case that building software systems requires a set of skills and experiences beyond just book learning, training courses, methodologies, and CASE tools. A craftsman paradigm is presented in which an apprentice software developer learns from journeyman in much the same way other craftsman based professions do.

This concept is both interesting and timely in light of the recent failures of systems ranging from the FAA and IRS to dot–coms. McBreen presents this concept as a counter to software engineering methods (SWE), which he proceeds to blame for many of the woes of the industry.

Starting with the Preface, McBreen confuses the engineering aspects of software with software manufacturing. He quotes a 1969 NATO report from Naur and Randell, which is not only out of date, but inappropriate in todays software engineering discipline. This use of dated reference materials continues in many chapters and creates the impression that McBreen may not have done his homework.

His case against software engineering has useful points but many of the issues he brings up are well worn red herrings. His Big Project example is nearly 20 years old and in a domain unfamiliar to many – a ballistic missile defense system (the first such system). Surely we have learned something about software engineering in the intervening time since 1975.

McBreen approaches the issue of software engineer licensing as a potential threat to software craftsman. But he does not provide a balanced view for this topic, since the licensing of process system engineers writing software for mission critical systems has been in place for decades. A look at the European TÜV and SINTEF regulations would have provided some background on addressing this issue in the US.

One issue with the book is that it does not establish a context in which to discuss the topic of craftsmanship–based software development. Customer relationships, development processes, project management processes, examples of success and failure are not placed in a context. One place to look for a taxonomy of software projects and their attributes would have been the Capers Jones series. [1] These texts provide a broad as well as deep view of the complexities of software development and the risks of naively apply one size fits all to a problem.

This one size fits all approach is addressed by McBreen in a later chapter. He states that ones size does not fit all, but then proceeds to provide advice and direction to the reader on how to apply craftsmanship methods in the absence of a specific problem domain. The process of developing craftsman–based software for a business application with an on–site customer is significantly different than developing software within a highly regulated or globally distributed corporation using COTS products.

Although there is much to offer in the book, I have some specific issues with the material, its sources - or lack of sources – as well as the overall tone for what is presented as a New Imperative.

Early Chapters

The thesis of the book, that Software Engineering has run its course and it is time to return to the master/apprentice approach starts by citing the early days of software engineering. His working example of large scale software engineering is the SAFEGUARD system completed in 1975. This example is not very timely nor is it representative of a modern large scale development effort. Although SAFEGUARD presented many challenges to the software engineers of the time, it's lessons have long been absorbed into the practice of software engineering. In addition the SAFEGUARD system was one of the early discovery design projects. Not only did new hardware have to be built, but new and innovative software processes invented as well.

Problems with Software Engineering

McBreen's comment that large scale development projects are systems engineering projects makes an important point. At what size does the project move from programming to systems engineering? The definition of the domain of a process is vital to the discussion of methods and their application. One Size Does Not Fit All is developed later, but the concept domain context should underlie the discussion of methods. McBreen's conjecture that in system engineering projects the developers must wait for the hardware fails to address the COTS integration methods used in todays commercial, government, weapons, and space systems.

Referring to the Site Defense example, these are inappropriate connections to modern systems engineering processes. A more appropriate defining characteristic might be that the developers must wait for the next major release of software (SAP, Oracle, Siebel, JVM, application library) before the full needs of the client can be fulfilled. Or that the developers must wait until a synchronized set of features is available for all the moving parts of the system. This is typical in the manufacturing world, where CAD, ERP, CRM, and other infrastructure systems all seem to be moving in different directions at the same time.

McBreen then proceeds to ask the question is software engineering a good choice for your projects, without defining the domain of the project or its characteristics, or even a very good definition of software engineering. See [9] for a starting point for the definition of software engineering. I also recommend the Capers Jones taxonomy of projects as a starting point for such a discussion on what's an appropriate method in what domain. One starting point that McBreen missed is the broad collection of definitions for this topic. [2]

Understanding Software Development Through Metaphor

Chapter 3 develops an understanding of software development using the one size does not fit all approach. The reference materials for these questions are light though. Highsmith and Cockburn are the most modern, others are dated in the mid-80's. One Humphries reference is given, but the bulk of Humphries work is not referenced. There is a vast array of material available to address the question what is software engineering? Too bad he picked materials that lead only to the conclusion that SWE is bad and master/apprentice is the only way to salvation.

Chapter 4 starts to lay the ground work for master/apprentice. The collaborative intellectual aspects of software development are described, leading to the statement that software development is a social process (ala Cockburn). XP is injected into the discussion as the example of such a practice. This of course leaves out other practices that meet these criteria ranging from internal methods (IBM's internal methods for example) to other light weight and agile processes that we all know and love.

An interesting point here is that McBreen references articles in several journals, but they appear to be hand picked. One example is the Laitnen Oct/Sep 2000 IEEE Software, Point - Counter Point discussion of scaling down a process to fit the problem. McBreen fails to mention the details of the article and the counter points. This approach of clipping quotes out of context may be appropriate for an article or conference presentation, but a book length work deserves a more balanced view.

Another interesting point in Chapter 4 is the reference to Paul MacCready's talk on the Gossamer Condor and the analogy to software development. The aircraft MacCready built was targeted at a prize of flying a human powered vehicle across a specific distance. This aircraft only had to do one job, get around a course with a human providing the power. Examples of no documentation are used as the lead in to developers wanting to write code rather than write design documents.

McBreen's conjecture that aircraft designers strive to create the perfect design on the drawing board is just plain wrong. He needs to spend some time with an aircraft manufacturer like Boeing before making such rash statements. He will then see the 1,000's of iterations between design, test, and analysis that are performed for nearly every part of a commercial airliner. These uninformed statements appear throughout the book and take away from the valuable material that lies between them.

Sidebar


In this weeks Aviation Week & Space Technology 9.24.01 there is a product announcement piece about AirplanePDQ. This product provides the experimental aircraft builder a set of automated tools for generating drawings and documentation as well as sizing and analysis tools. The notion that experimental builders don't do documentation and skip all the good engineering practices is nonsense. McBreen needs to look further into the process of designing, constructing, and maintaining even the most rudimentary of aircraft to see that lightweight processes are used, but there is a process and it is based on sound engineering practices.


End Sidebar

Using this Gossamer analogy, would the builders of aircraft at Boeing behave in the same way as MacCready? Of course not. Would the developers who only want to write code and produce no documentation be welcomed in the domain of realtime flight control systems at Boeing? Probably not. Establishing a context for an analogy adds to its value. Using MacCready's work as the basis of craftsmanship while not putting the aircraft he produces in a context, misses an important point - can the lessons learned from one domain be transferred to another? Gossamer Condor carried one very powerful bicyclist around a closed course in pursuit of a prize. Can the methods used to construct the aircraft be extended to other domains? Well that's the big question isn't it.

Some of the discussion about repairability of the Condor is interesting, but one must remember it was an experimental aircraft not a production machine. So extending the analogy – is there a difference between experimental or one-off software and commercially robust production software? Probably.

The conjecture at the end of Chapter 4 that there is a resurgence of software craftsmanship needs to have a context as well. Well crafted applications are common in some industries and rare in others. Flight control software, although highly regulated, is hand crafted to meet the needs of the aircraft designer. Process control systems are hand crafted for specific products and processes. I've built software for gas turbine controls within a highly regulated framework of TÜV and SINTEF. The code was carefully crafted under the tutelage of turbine controls engineers, then mass produced as an embedded product, complete with loads of documentation, test verifications, and certifications. McBreen's generalizations provide a disservice to those who practice their trade within structured and regulated domains, with the suggestion that software engineering is inappropriate for such environments.

Part 2 - What is Software Craftsmanship?

Part 2 opens with the statement – a key failing of the SWE metaphor is that it fails to place people at the center of the software development process. McBreen does not differentiate between the theory of SWE and its application or inappropriate misapplication. A simple search of the SEI materials would have led McBreen to conference proceedings and books addressing this very issue.

Authors ranging from Jones to DoD planners ALL talk about the human element of the development process. The notion of tagging SWE with the failures of natural language processing is a red herring. Requirements elicitation is a complex and domain specific process. Getting requirements out of a customer in the same room with the programmers is different than getting requirements out of a corporate customer spread over 3 continents for a $30M ERP system deployment. McBreen again fails to establish a context for the blanket statement that craftsmanship is better than SWE.

McBreen makes other sweeping statements without reference materials. Engineering relied for a long time on the craft tradition ... and moved ... into the academic tradition in the last few decades. Really? The English bridge builders of the late 19th century had no academic tradition of stress and beam calculations? McBreen then slams academic processes by claiming that mathematical models are not up to reality and therefore craftsman are needed to bridge the gap (so to speak).

McBreen would be advised to read a bit further the SAFEGUARD materials and discover that modeling and simulation are identified as the major contributors to the success of the project. This anti-academic approach has no place in a book of such importance. It simply sours the rest of the material, which could in fact add value to the discipline.

Putting People back into software development

Chapter 5 is a description of how businesses (I assume since he doesn't really say which ones) retrained workers to be programmers and their resulting failure. CASE is painted as one source of failure, along with the industrialization and de-skilling of programmers. No references are provided for this conjecture, so its hard to tell if this is a national trend or just personal anecdotal evidence.

Later in the chapter is a call to arms (I prefer a call to action, since business practices and military actions have little in common). The call to action is to insist that developers really know their craft before we trust them to create systems for us or with us. Without stating the context, one would be hard pressed to find an industry where this call to action is not currently applicable. The first place to look though might be the dot–com businesses.

The natural selection process has played a role in sorting out the mess created by poorly trained software engineers. In my world of ERP and realtime systems it would be rare to allow an untrained development team to spend customer or investors money. It does happen, but with the usual disastrous results. It was common in the beginnings of ERP to spend way too much money on the integration, but people learned to control the requirements creep and restrict the development dollars to measurable productivity efforts. Much of McBreen’s concern should be directed to management rather than developers (at least in this chapter), since the decisions to deploy inexperienced teams is rooted in the management process.

Programming is a craft is a legitimate statement, just as designing and building any complex product is. Skill and experience are needed. The best process in the world will not save a project from failure … is an obvious statement. Highsmith’s comments are referenced here, but they too are obvious to successful organizations.

The Red Herring of Software Licensing

Chapter 6 launches into a diatribe about software licensing. This is an interesting topic in the pages of trade journals, but a book chapter risks that the topic will have passed by long before the book comes to market. I skipped this chapter in the first reading and should have on the second.

It seems this topic will not go away and will continue to elicit comments. But at the same time new and interesting issues continue to rise regarding some way to provide the means of producing and assuring quality software, just as McBreen is proposing in this book.

What Does Craftsmanship Add to the Process of Software Development?

Chapter 7 starts with a misinformed notion about products and repairability. McBreen claims that products made my machines - not craftsman - have lost their attribute of repairability. I'm assuming he is excluding automobiles, computers, houses, aircraft, clothing, and the myriad of other non-disposable goods we use every day. This type of generalization simply sets the tone to discount other important concepts.

McBreen then makes the conjecture that software craftsmanship works because software is easy to copy. But copying of the articles of production is what mass production systems do, not what McBreen claims he wants us to move toward. The market forces between craftsman and mass production becomes confusing when McBreen introduces the shrink wrapped, time to market, good enough software, and software bloat arguments. Craftsman have a different relationship with their users heads a section. But what users? The 10 million users of a shrink wrapped product? I think not. The one user in the same room as the developers? Probably so. The user three levels removed from an embedded OEM component? Hardly. What user? He doesn’t say.

The conjecture of Alan Cooper [3] of designing for a single user needs to be tested against the product marketing strategies of the likes of IBM, HP, and Microsoft. Software products are broad and deep in scope and functionality. No context is established for this blanket quote, so it is difficult to determine if and where it is valid.

The ideas of sold as is licenses fails to address the legal issues associated with product development. McBreen might be better served by referencing some commercial issues of product warranty and liability before suggesting software products be sold to the general public with craftsman like warranties. Many have objected to the corporate approach  to software development, but the large software business structures of the early 21st century are not the same as the cottage industries of the 19th century.

McBreen may be in fact wishing they were, and the book is building a case to return to such a business model. Much of this chapter leads me to that end as well. This is an interesting approach and may be at the heart of many of the agile and lightweight development processes. This is an idea that is outside my current interest, having come from large environments, most of which are regulated by government agencies or risk indemnification processes (petrochemicals, electric utilities, manufacturing, etc.). So I’m not in a position to say whether this will work out in the end or not…this craftsman approach with the absence of a corporate structure.

The Role of Customers

Chapter 8’s title is Customers have a different relationship with craftsman and indeed they do. But does the craftsmanship approach to the customer scale beyond the face–to–face interactions of customer/developer co–located team? How about software development in a global market place? How about software development in the OEM, ISV, or product based environments, where many times customers are not only unavailable, they are not even known to the developers. McBreen takes the approach used in XP, that having a customer close by is better than not - which is obvious. But provides no suggestions when this situation - no matter how desirable - is not possible.

The discussion of the failings of good enough software is a repeat of other presentations and books. Many have addressed these issues, with similar results. Nothing has changed much here, since the market forces still overwhelm good development practices. But, we as consumers have created a two–headed master. We want products faster, cheaper and with more features, no bugs, and free upgrades. A marketing mans nightmare. It’s not clear who the customer is in this chapter. An end user, an in–house development organization, contractors? In all cases though, the market pressures are tremendous. Something more than lip service needs to be paid to the consequences of this quest for faster, better, cheaper.

The follow on discussion about a craftsman’s interaction with a customer is interesting – hire small teams of developers, manage them as you would a craftsman doing custom work, pay them according to their skill, not the market norms, measure their deliverables, have the customer make the cost/quality tradeoffs. Trying to deploy such a strategy in a formal organization would be a challenge, since simple job descriptions and labor regulations come to mind as impediments. This is not to say these goals are not invaluable in todays rapidly changing environment. But again no context is discussed as to where such an approach is viable.

Managing Craftsman

The discussion that craftsman have specialized skills and need to be managed as such, seems a bit counter to XP’s everyone knows everything about the project. Mixing ASD concepts with XP concepts gets a bit confusing in this section of the book. One concept of having the customer in a long term relationship with the craftsman needs some context again. The reality of supplying products to customers would set the stage here.

Commercial vendors are usually removed from customers either through distributors, resellers, or integrators. Or at least through a customer service department. Having developers interact with the customer doesn’t always make business sense. Customer support departments of many software companies provide this connection, while maintaining an arms length relationship with the developers. A rethinking of the economics of customer support is needed but none is given.

Another out of the blue statement is found in the middle of Chapter 13. Journeymen rarely work alone. Says who? Where are some background references? This constant opinion making is just that – opinion. What this book needs is a good copy editor to weed this stuff out or come up with some references to support this conjecture.

Repositioning Software Engineering

McBreen states that craftsmanship does not replace SWE but is complimentary to SWE. If so, then why all the ragging on SWE and how it is not appropriate for building modern software? McBreen claims SWE was created for building really large, multi–year systems development project like air traffic control systems. This is just plain wrong and again taints the good parts of this section with mis–informed and mis–guided blanket statements. It is just these large scale development projects that need innovative solutions, like craftsmanship based programmers. The challenge is to integrate these new practices with the structured approaches. McBreen provides no such suggestions.

Chapter 14 kicks off the topic of repositioning software engineering. The claim that waterfall is the predominate method has no references or backup. He ties job descriptions to methods, an interesting concept, but not one supported by any field evidence. Again a look a Capers Jones [1] numbers would have revealed a broad array of methods and best practices in diverse industries. A look at Royce’s Software Project Management [6] book would be a better starting point for how commercial and industrial companies develop software. Ambler’s two book series on Software Patterns [7] is another good starting point for building a picture of how commercial product developers do their work.

McBreen uses statistics in an unconventional way. He claims a small project is 100 person years and a large project 10,000 person years. McBreen provides no source for these numbers, unlike the Capers Jones surveys conducted between 1995 and 1999. The Jones data describes projects ranging from 5 to 20 persons. A thorough read of the Jones book would have revealed that the dynamics of projects varies across domains while the distribution of the project sizing is remarkably narrow.

The Hazards of the Software Engineering Metaphor

Chapter 15 states that the Software Engineering approach may be appropriate for some projects, it is an inappropriate choice for most commercial software developers. By commercial software developers, I’m assuming McBreen means software sold to the public. If McBreen means in house development or purpose built projects then he may have a point. There is no reference material cited to support this conjecture, so I must assume it is the opinion of the author.

McBreen goes on to cite NASA’s report on the Faster, Better, Cheaper approach to project management. This study is listed in the footnote, but readers should follow the links to the entire report as well as the Phase I report on the MCO Mishap. In both reports people are listed as the most important aspects of mission success. Contrary to what McBreen claims software engineering practices do.

McBreen conjecture that software engineering denigrates anecdotal evidence is wrong. The space craft construction business, for example, is based on four processes: Normative, Rational, Participative, and Heuristic. Mr. McBreen would do well to read Rechtin’s [8] books on systems architecture in the space craft business before generalizing that software engineering restricts anecdotal evidence. As a side note the MCO report is over 200 pages in length with several dozen major recommendations. McBreen summarizes the findings to his need in 2 paragraphs. Granted these reports are partially political instruments, but simplifying their outcomes is a disservice to the professional working in the field.

McBreen’s discussion of reuse in regard to Ariane 5 is misinformed. His reference is to a news release early in the analysis of the accident. Numerous reports are available with a little searching effort. A more thorough reference set can be found at [4]. Although tacitly similar to the more detailed reports (floating point conversion error), the real problem with Ariane 5 was the improper inclusion of the software in 5’s flight control configuration. It was a configuration management issue which eventually resulted in the floating point conversion error. The software generating the error should have never been executed. There are 6 failures identified in later reports:

  1. Unprotected code that should have had error handling.
  2. Inclusion of code that was meaningless after liftoff.
  3. Ariane 5’s high initial acceleration was different than 4’s which led to a build up of horizontal velocity that was 5 times more rapid than 4’s. This created an excessive value for the variable being converted.
  4. The halting of both (redundant) processors after the floating point conversion error is rooted in the culture of Ariane design. This culture is focused on random failures that can be handled by redundant hardware executing the same software. Common Cause Faults were not addressed by this approach. This Common Cause Fault mode is addressed in detail in NASA space craft as well as fault tolerant process control systems. See [5] for an overview of the issues associated with Common Cause Faults and fault tolerant diagnostics in the presence of such faults.
  5. The errant software was not re–tested prior to 5’s launch. Reuse in mission critical systems requires re–testing as well.
  6. A decision was made in 1992 to remove the SRI’s from the test loop since they were considered fully qualified. In addition the simulation of failure  modes was not possible with real equipment, only with a model. Both bad choices in the end.

Mr. McBreen’s over simplification of the Ariane 5 failure in support of his thesis that reuse is somehow associated with software engineering’s worst traits is not untypical of the popular  press. Further research would have served his readers better, since the cause of the failure is traceable to software management procedures unique to the Ariane project.

Learning from Software Engineering

Chapter 16 discusses the learnings from software engineering. In the size matters paragraph, McBreen states the craftsmanship projects are 10 to 20 developer years, yet the Jones numbers have nearly the same level of effort for a broad set of application domains, ranging from commercial to defense systems. McBreen claims software engineering projects are 50 to 100 times that size, with no references for the sources of the number. He then proceeds to reuse the SAFEGUARD numbers as an estimating example. Since the reference is 10 years old and the project is over 20 years old, one would think a more recent example of estimating failure could be found. I might suggest the IRS, FAA, and AX aircraft as examples.

At this point in my second read I started to skip chapters. Since I grew up building weapons systems and realtime fault tolerant products, I was still reacting (not a good trait for a reviewer I realize) to Mr. McBreen’s lack of understanding of the causal faults in the Ariane 5 example and the misuse of this information in his thesis that SWE practices need to be replaced with craftsmanship practices.

Chapter 17 reviews some of the guidelines for good project management. These should be read and re–read by managers.  But once again he uses the Borland Craftsmanship project. A broader set of references would allow the reader a better base by which to judge the comments. These are small criticisms but indicative of a narrowly focused thesis. Since the Borland report uses line of code as a productivity measure rather than function points it is difficult to calibrate their results with the more extensive Jones numbers. One new source of project management guidelines is Anti-Patterns in Project Management, Brown, McCormick III, and Thomas, John Wiley & Sons, 2000.

Chapter 18 makes the nearly unbelievable claim that Java is hazardous to the health of your projects. Java is too new and it is proprietary. McBreen surely has not done his home work or is so narrowly opinionated that his readers will be stunned to learn this news. Having just come off a very large scale CORBA/Java project, I can state that Java was a major contributor to our success. McBreen’s example of internationalization and the associated problems is plain silly. McBreen claims his browser displays the English page correctly but displays the Japanese page in something that is vaguely reminiscent to line noise. I might suggest he install the Japanese character sets available with the browser – all will then be clear (assuming he can read Japanese).

The outsourcing issues presented in this chapter bring up real problems. It has been shown that outsourcing can be successful if a clear and concise specification is available. In the absence of such a specification, problems can occur. Insisting on the software craftsmanship as a solution to this problem begs the question – if the craftsman is light on documents (ala MacCready) and depends on direct contact with the customer, how does the outsourcing process work? Outsourcing is sometimes done on other continents, with skilled programmers speaking another language. If the outsource supplier is involved in all phases of the project, from requirements to life cycle support, then the definition of outsourcing has be redefined.

Conclusion and Recommendations

I found the book a stimulating read, although frustrating at times when I encountered comments like the Java one above. The major gaps in understanding might be attributed to McBreen’s lack of experience in those areas. Gaps that could have been easily filled with research. These gaps I also attribute to poor copy editing, but that is another issue all together.

I would recommend the book, but with qualifications and caveats. The read should try to put the ideas into a context and ask are these concepts appropriate for my domain? Do the ideas make sense for the specifics of my problem? Does the specific statement actually make sense from my own experience and the experience of others? Have I seem a similar concept in a peer reviewed journal or are these simply the opinions of the author with an axe to grind?

McBreen gives credit to the SEI, Jones, Coplien, Highsmith, Cockburn and others, but he appears to have left much of their contribution out of this book and on the shelf.

Some Counter Points

McBreen provides little in the way of research behind his thesis. This is a personal issue with me, since my training and experience come from the hard sciences (particle physics realtime data analysis and radar/sonar signal processing). McBreen's conjecture that a new method is needed is shared by many others as well. Here's some references to fill in the blanks.

The factual gaps in Ariane 5 can be understood, since McBreen may not be working in the domain of spacecraft design. The opinion that Java is unacceptable for commercial development seems odd, since he positions himself as a leading trainer in OO technologies.

Footnotes

[1] Software Assessments, Benchmarks, and Best Practices, Capers Jones, Addison Wesley, 2000. Assessment and Control of Software Risk, Capers Jones, Prentice Hall, 1994.

[2] There are numerous definitions of Software Engineering. So of course there are web sites that collect these definitions. Here's one that is useful for the discussion here. http://wwwsel.iit.nrc.ca/sedefn/SEdefn.html. The IEEE definition used by McBreen is only one and is a old one at that.

[3] The Inmates are Running the Asylum, Alan Cooper, SAMS, 1999.

[4] http://www.niwotridge.com/Resources/Ariane5Failure.htm Follow the links to the Ariane 5 materials for a survey of the more technical reports. It's breathtaking at times how the press, even the technical press mis-inform when it comes to a complex technical topic.

[5] http://www.niwotridge.com/PDFs/Fault%20Tolerance.pdf (860K download). See pp. 93 for an overview discussion of this complex topic.

[6] Software Project Management, Walker Royce, Addison Wesley, 1998.

[7] Process Patterns and More Process Patterns, Scott Ambler, Cambridge University Press, 1998 and 1999.

[8] The Art of Systems Architecting, Second Edition, Mark Maier and Eberhardt Rechtin, CRC Press, 2000. The first edition of this work was published in 1997. New material on distributed systems has been added. Rechtin is a major contributor to the process of space craft design

[9] http://wwwsel.iit.nrc.ca/sedefn/SEdefn.html.