Friday, December 26, 2003

Re-evaluating Agile Affiliations

This thread of discussion at Alan Francis ’ blog has prompted me to re-evaluate my affiliation with the “agile community” (to use my own phrase).

First, let’s define “agile community” as the Agile Alliance and its members, those who have signed the Agile Manifesto or those who have written a paper, spoken at a conference or written a book on the topic of agile software development.

I often refer to the “agile community” as if I were an observer of it rather than a part of it. Why should this be so? Firstly and foremostly it is about objectivity. It is important to be able to hold a neutral mindset and try to evaluate “agile” as others see it - those others are many of my colleagues in management positions in Fortune 1000 companies. So a standoffish objectivity is important for me because it is important to my audience.

However, there are other reasons why I don’t feel part of the “agile community”. One is that I don’t work as a consultant and I don’t make a profession out of attending conferences and hanging out in hotel lobbies and bars with the other “agilists”. Fair enough! But there is something more subtle - I worry that the agile community gets sucked into perpetual naval gazing. A state where it is more important to worry about whether something is “agile” or not, or whether something is “more agile” than something else. To me these are not important questions. The important question is whether software development is aligned with the stockholders interests and whether software development can be performed in such a manner as to improve the return to the stockholders. It’s all about better software, faster! Good, fast, cheap - pick 3!

Hence, I have no time for discussion about “but is this self-organizing” or not. I’m not interested in the observed attributes of some “agile” methods - only in the results.

As such I will continue to pursue the study of management science and software engineering, as I see it happening in the Fortune 1000 on medium and large scale projects, for continual improvement of shareholder aligned performance. What is the goal of the organization? How best can the day-to-day activities of software engineers be organized to align with that?

I am not interested in being a member of a club for the sake of membership, nor am I interested in political association for the benefits of that association (though I recognize that political association is a key part to being a successful manager). I am not interested in aligning with a movement because I have something to sell. I don’t work as a consultant. I don’t sell anything. I may make some pocket money from selling a few books but it’s small reward for the hours spent developing the material (including the on-going effort of this site).

As long as the “agile community” continues to focus on the right questions, “How can we govern software engineering better? How can we align our day-to-day activities with the stockholders interests? How can we improve the return to the stockholders through software development? How can we report our progress transparently and in a customer valued manner? and How can we continue to build better software, faster and cheaper?” then I will want to part of that community. It is these principles that were at the heart of Feature Driven Development and why I am proud to be part of it. If the “agile community” degenerates into discussion of “my method is more self-organizing than yours” or “my method has faster feedback than yours” or “my method is more fun than yours” then I’m not interested. As Jeff De Luca has said (paraphrasing) at the start of many projects, “You’re not here to have fun. You’re not here to make your resume look good. You’re not here to use cool technology. All those things may be important but they are secondary. You are here first and foremost to make a profit for the stockholders. Now let’s get started.”

Thursday, December 25, 2003

More Thoughts on Product Management

Chapter 16 of “Agile Management…” looks at Product Management. It’s an area that arguably doesn’t get enough coverage in the book. In fact, I point out that there is a whole other book to be written on the topic. In the Foreword, Eli Schragenheim tries to compensate for this shortcoming by giving some examples of how to better understand and define the value associated with the features in a product.

This post on Venchar points out how valuable the role of usability engineering and both product and interface design are to realizing value and its associated attributes such as market share acquisition. I’ve commented elsewhere in online posts that if I did another book it would look at this “marketing” aspect of software development management and focus on increasing Throughput through better value acquisition. I observed that such a book would need to merge the best of product management with the best of usability. Initial feedback showed that few people could see a role for such a combination of topics.

I intend to explore this subject more - product management, design and usability - over at my uidesign.net blog. If you’ve got opinions on how usability and design can be used to enhance product management and ultimately enhance ROI through improved realization of Throughput, then I’d love to hear from you.

Saturday, December 20, 2003

Smells Like Teen Spirit

A couple of recent posts have reminded me of one of my rules of thumb - developers who spend too much time on the debugger are not good developers and should be rolled off my team.

Keith Ray writes in Cost of Debugging quoting Christian Sepulveda,

One half of each day was completely wasted. The developer couldn’t really doing anything productive during the four-minute debug cycle because he had to navigate to the correct place in the application and execute the right steps so he could observe the results of the change he just made; he was forced into tedium. In other words, half the development budget was being spent on performing repeated, monotonous tasks. This isn’t a very effective use of resources or talent.

It was my former mentor Duncan Smeed who pointed at Robert C. Martin’s Debuggers are a Wasteful Timesink, which took me back to experiences as a contractor working at IBM’s PC Company and my earlier roots as a machine code programmer in the games industry in the early 1980’s. [Aside: In fact if you ever owned a Sinclair/Timex Spectrum there is a strong chance you owned and played one of my games.]

In a world where programmers are measured by activity and dedication to “activity” then they get rewarded for spending hours on the debugger. I can specifically recall one developer who was earning almost twice as much as me to sit and play with the debugger until 9pm every night when I was going home at 3.36pm (the earliest available moment under the flexitime rules). Why did I go home so early? I had finished my code and it was tested and working.

In the 1980’s, we didn’t really have debuggers and when we did they were useless because we were testing real-time performance and so much was running off timers and interrupt driven. It just wasn’t possible to test with a debugger. It was time wasting and useless. I learned to code by putting logging and instrumentation into my code. I learned to develop by small increments fully unit tested. And the biggest lesson I ever learned when working as a team is - you agree the interface (the method signatures) at design time and you b****y well stick to them. With assembly language, the concept of “breaking the build” blows the machine up - and in those days we didn’t have multi-process spaces to protect the memory and keep the machine alive.

The smell of an over-revving debugger resembles the teen spirit which encourages idle youth to spin tires with the parking break on. It makes a lot of noise, it creates smoke, it provides an illusion of activity and intelligence, but ultimately the velocity is zero.

[Updated 12/21 1300hrs PST]

Thursday, December 18, 2003

Being Too Nice

Blogging has been light this month and it will get lighter as Christmas approaches. My dad has been staying with me for the holidays. It’s his first trip outside Europe. I’ve been trying to spend as much time with him as possible. I’ve had a couple of chances to do that this year.

Back in September, he was reminiscing about his short spell as a line manager. He worked for 30 years at the Nobel’s Explosive Company. Few people remember that Alfred Nobel - famous for the Peace Prize (and others) - was the inventor of Nitroglycerin and later Dynamite. After scouring Europe looking for a manufacturing site, he was granted permission to build a factory on 7 square miles of sand dunes on the north bank of the river Irvine in Scotland at Ardeer where the British Dynamite Company opened in 1871. Over 100 years later, the plant was still operating, mostly producing detonators for the mining industry.

For 1 year of his career, he managed 300 women employed to hand assemble these miniature explosive devices. After only a year in the job, he was moved because he was “too nice for the job” and instead became an instructor in the factory’s apprentice school. During his tenure, he build a good rapport with the staff. He averted 3 strikes and increased the productivity whilst avoiding lost production due to threatened industrial action. In Britain in the 1970’s, strikes were common and the unions strong. Managers who were good at industrial relations were few and far between.

It’s so important that managers are measured objectively using metrics which are aligned with the stock holders interest. Objective measurement should lead to rational decisions about good and bad managers and good and bad management techniques. It is all too easy to fall into the arrogance of a “them” and “us” attitude - to think that being a manager makes you (somehow) better than other people. It’s counterintuitive in a world where people arrogantly think they are better than their staff that the right way to manage people is to treat them with respect and empower them with the ability to perform better. To see things the right way, objective, rational measurement is required. This insures that managers who produce more by being “nice” to their staff will be rewarded not punished.

Jason Marshall points out why it is so important (more than ever) to treat staff fairly and to focus managers on keeping them well motivated - to balance the “emotional capital”. Knowledge workers have a lot of power to undermine without appearing to be doing anything other than their job. When a development team is poorly motivated by bad management, the results will show up in the productivity figures.

If an organization is not measuring the flow and delivery of client-valued functionality then managers can not be judged objectively and the trend in staff productivity cannot be monitored.

Saturday, December 13, 2003

Chapter 6 - Agile Project Management

Chapter 6 revisits a theme from Chapter 4, the main variables of scope, budget and schedule, and examines how traditional software engineering methods choose a different constraint than Agile or RAD methods. Traditional SDLC methods fix the scope and allow the budget and schedule to vary. They define “quality” as conformance to scope delivery. This choice is rooted in the project management methods of the PMI and is indelibly fixed into the SEI SW-CMM (capability maturity model) and the definition of ISO 9000-13 software development quality assurance. Agile and RAD methods fix the delivery date and allow the budget and scope to vary.

The Agile choice is the better choice and Chapter 4 explained why - the scope has much greater uncertainty than either the budget or a date. In fact as sure as the Sun rises, any date chosen will arrive. Dates are absolutely certain. What is unknown is how much functionality can be delivered by that date. The Traditional method of fixing the scope means that the variable with the greatest uncertainty is set as the mark by which quality is measured. This inevitably means that any estimates of delivery date and budget based on a fixed scope have a high margin of error. Error on the high side is failure under ISO 9000 standard and CMM quality definitions.

Next, Chapter 6 looks at the effect of cost accounting and its focus on tracking effort expended and value-added. This leads to the false security of recording work-in-progress as an asset. This is somewhat of a historical perspective as, at least in the United States, it is no longer normal practice to capitalize intangibles such as software development in-progress. However, even as recently as 3 years ago, this was common - particularly in high technology companies which were losing money. Recording WIP on the balance sheet takes it out of the P&L and makes the loss look lower.

Throughput Accounting assigns all costs incurred in software development to OE (operating expense) whilst only recording value-added on completion and delivery. Hence, value is only recorded at two points - the input and the output. There is no value-added for partially completed work.

When you start to measure things this way, it naturally leads to a different way of tracking projects - tracking the flow of value. This is a concept from Lean Production and really it isn’t a commonly accepted project management concept at all. The notion of using it for Agile project management was floated academically by Koskela and Howell in the paper, The Underlying Theory of Project Management is Obsolete published in 2002 by the Project Management Institute.

I recognized this proposal as reflecting how FDD projects are managed. With a slight tweaking of the FDD Feature Complete graph, to use the Lean Production reporting tool of Cumulative Flow Diagrams (CFD), as I described at in this article at the FDD website, it can be shown that Agile software methods were already employing the technique advocated by Koskela and Howell. CFDs had been advocated by Donald Reinertsen in his 1997 book Managing the Design Factory as the ideal method for tracking the flow of added information during a design process. As software development is an information discovery, knowledge work endeavor, it seemed only natural that CFDs were the correct choice.