Monday, March 31, 2008

Stop Negotiating, Start Collaborating

Stop Negotiating! No really! Cut it out! Banish it from your thoughts, your actions and your organization. Refuse to negotiate. When you need to work with another department, group or team inside your organization, and you find yourself in negotiations to get what you need, just say “Enough!” And refuse to negotiate.

What is so bad about negotiation?

Negotiation implies that you have an internal market and a market implies that you need a contract between a supplier and a consumer of a product or service (usually a service in an IT or technology product company context). Markets and their contracts carry transaction costs [1] – one of those costs is negotiation. If you can replace your internal market with a collaborative network, you’ll gain an economic boost from what are known as the network externalities [2] of being in the network. Collaborative networks do away with the transaction costs associated with marketplaces - specifically in this case, the cost of negotiation.

Negotiation also implies that information is being hidden. The supplier is hiding their availability, or the cost of the job, or some information that might give the consumer an advantage. Our 20^th Century education taught us that “information is power” and business schools teach negotiation emphasizing that leverage involves hiding information. It’s become second nature in business to assume hidden information and to probe for it during negotiations. Hence, consumers of your IT service naturally distrust your estimates or resource requests.

Let’s pause for a moment and reflect on that word advantage. Within your organization, why would any one group or team require an advantage over another? Doesn’t the organization have a set of common goals? And isn’t the organization supposed to be collaborating to realize those goals? What can obtaining an advantage over another group possibly have to do with collaboration?

If you are beginning to think that there ought not to be a position of advantage in a truly collaborative organization, you’re thinking along the right lines. Having an advantage and collaboration are incompatible. By implication then, information hiding is incompatible with collaboration. And negotiation ought to be unnecessary.

Trust is the essence of a highly collaborative organization.

People can argue over the essence of agility or Lean Thinking but for me optimizing a software development organization for high performance starts with building a high trust organization. High trust organizations are flat. They feature a high degree of empowerment and delegation. They encourage joint responsibility and mutual accountability. High trust organizations adapt dynamically to the needs of the organization regardless of reporting structure or formal organizational hierarchy. High trust organizations are social networks of highly collaborative knowledge workers.

High trust organizations are also lean. They expunge the overheads of low trust environments. High levels of trust and a flat structure mean that audits typical of hierarchies are eliminated and contracts are dispensed with. There is no concept of an internal market. In short, there is no negotiating. Negotiation is waste! The resultant contracts, their documentation, agreement, review and subsequent audit or enforcement are all considered as more waste! Subsequent negotiation for corrective action when required is yet more waste!

Negotiation is a symptom of an organization that has a lot of growth potential in its social capital. If you find yourself negotiating, you know there is room for improvement.

Be the Naked Organization

So if negotiation is undesirable, how do you go about eliminating it? Just saying “No!” won’t cut it. You need to get naked!

To put it another way, you need to unilaterally disarm! Stop hiding information. Make everything your team does transparently available to anyone who wants to take an interest. When everything is on show for all to see, you arrive at the negotiating table naked, exposed, no cards up your sleeve, nothing hidden - no sleeves. You have nothing to negotiate. All you can offer is your capability and availability.

Doing this is counter-intuitive. It goes against the grain of much of our education and training. But amazingly it is truly disarming. And it will change the culture of your whole business, not just technology. Rather than leave you vulnerable to attack, it generates trust in your relationships and it turns negotiation in to collaborative problem solving.

Think about it for a minute. “Vulnerable to attack.” From whom? Do you really think the marketing department wants to invade the IT shop and write the code themselves? Perhaps you think transparency leaves your team open to abuse from consumers of your services? Actually, the reverse is true. When you are negotiating from a position of hidden information, there is low trust. The consumer will believe you are hiding something which you are and will assume that he has to bargain hard. It’s likely that the consumer went to business school and you as an IT or software engineering manager did not. You are at a disadvantage. It is lack of transparency leaves you and your team open to abuse. Transparency protects you – despite the insecurity of nakedness and its associated vulnerability.

A Recipe for Success

So what is it that you are making transparently available to anyone who asks? You need to track the customer-valued work that you and your team does, from initial request to delivery. You need to report:

• how many things you’ve delivered
• how long they took to process (the lead time, from the customer’s perspective)
• how many things you have in process
• the quality of your deliverables

In my Recipe for Success [3] Cutter Fellow Jim Highsmith discussed in a series of articles for Cutter’s Agile Project Management advisory service [4-8], I identify four things your organization needs to do to deliver a high trust culture that is lean and exhibits agility. These are:

1. focus on quality
2. reduce (or limit) work-in-progress
3. balance demand against throughput
4. prioritize

Focusing on quality eliminates waste from rework, improves productivity and lead time, and builds trust with consumers.

Reducing work-in-progress sufficiently, allows team members to single-task. It reduces lead time and non-obviously has a huge positive effect on quality, further reducing lead time and improving productivity. It also forces the need for prioritization.

Balancing demand against throughput recognizes that a team can only effectively carry so much work-in-progress. Taking on too much clogs the system, affects quality and reduces customer service. Balance also eliminates abuse of the team - work/life balance or the agile idea of sustainable pace are achieved by agreeing to balance demand against capacity.

Finally, prioritizing forces the consumer to think hard about what is important. Given a supplier system that offers transparency on throughput, lead time, work-in-progress and quality, the consumer is left to figure out how best to utilize such a system for maximum value.

Collaboration Games

Transparency offers us the ability to turn negotiation in to collaborative problem solving. There is a simple question to be answered, “How best can we select job requests in order to maximize the value delivered through the supplier service?” Together your department and your value-chain partners can analyze and solve this problem. There is no negotiation. Negotiation is replaced by a puzzle of team optimization. Transparency, in this case, creates the opportunity for a collaboration game between consumer and supplier.

Naturally, there are a few snags. The work orders in a transparent system must be of a somewhat similar size. There must be a system of analysis that breaks work down in to types suitable for processing through a transparent system. If work items vary greatly in size, it leads to need negotiation. “If I give you two small ones, can you process them as if they were one regular item?” “How do I know they are two small ones?” Or, “We know this one is kind of big but it is really important, could you just squeeze it through?”

Soon you find yourself needing to estimate everything and to analyze the effort involved. Suddenly the problem to be solved revolves around trying to fit effort estimates against a calendar of available work hours. Since everyone knows that estimates are always wrong, and hence, the customer negotiator sharpens up her pencil and once again puts the squeeze on the supplier. I’ve recommended that organizations stop estimating [9] simply because it opens the door for abusive relationships through negotiation. Now I’m going a step further and asking you to stop negotiating. As part of that plan you need to stop estimating. Play with the facts! Use the hard, objective data, transparently. The hard facts are historical throughput (number of work orders delivered), lead time, quality, and quantity of work-in-progress. Estimates are not bad. They simply open the door to negotiation, reduce trust and leave, hard to build, social capital on the table.

It’s Non-negotiable: IT Services at Corbis

At Corbis, I’ve encouraged my organization to stop negotiating, to refuse to estimate, and to make all our data transparently available to anyone who wants it – often through popular software tools such as Microsoft Outlook or Sharepoint. In addition, we put things on whiteboards and post reports on walls.

We deal with different sized work items by offering different classes of service. For example, my build and configuration management team gets three different request types:

1. build requests
2. machine configuration and system maintenance requests
3. environment build-outs or replacements

Builds take tens of minutes, system maintenance can take up to a day and environment build outs can take a month or more. The department splits its capacity across the three classes of service. For example, we will agree to build out or refresh no more than one environment per month. This represents a specific resource allocation from the available pool of staff and time.

At a grander scale, we allocate 10% of our entire software engineering department resources to sustaining engineering, while we dedicate the other 90% to major initiatives. Our sustaining work takes work orders that represent 1 day to 15 days of coding effort and we offer a sustaining release to production every two weeks. We have analysis techniques in place to identify work orders that are too large and to either break them in to smaller items that we process sequentially, or to reject the item and redirect it to a major initiative.

With major initiatives, we also process work orders – usually product features – and we batch these up for integration events every month and for release every 3 months.

Within our sustaining system, we offer several classes of service:

• change requests (new features requested by business owners)
• production bug fixes (escaped defects found in production)
• additional bug fixes (special class of service that utilizes slack developer capacity where testing is also performed by developers)
• maintenance work (IT system upgrades, patches, API refreshes and so forth)
• production data changes (refreshing meta-data for data-driven applications)

We use a kanban system [10] to control the resource allocation across the classes of service and to limit the work-in-progress in sustaining to the committed 10% of total resources.

The kanban system also facilitates prioritization. Each Monday morning at 10am, a prioritization board consisting of six vice presidents (or their representatives) from around the company works collaboratively to democratically select change requests or production bugs to fill available kanban slots. For instance, the board members may be told on Friday that there is one free kanban slot available. They are provided a list of the approved backlog of change requests and production bugs and given a simple instruction, “Please select one item.” At the Monday meeting, they get the opportunity to collaborate and select the optimal item to promote into the kanban slot for engineering to work.

This collaborative process emerged over several months. Initially, the board was in a bargaining mode. They would argue, “If I give you two small ones, can you treat it as only one slot?” but we would refuse to play that negotiating game. After two months, a “democratic” period emerged, in which the board self-organized into a democracy with multi-voting to select winners each week. However, after three more months, it was evident that democracy was not selecting the requests to maximize business value, and scarce capacity was being squandered. Out of this emerged the current collaboration game approach, in which the board members work together to puzzle over and select the best choice for the business regardless of functional boundaries or ownership of requests.

This is an example, of where the introduction of an IT process and its foundation in a culture of high trust and transparency, actually resulted in positive behavioral change across the wider business. Six previously competing business units have stopped negotiating and started collaborating and the result is a high performance organization that has released valuable new software to production every 9 business on average since September 2006.

Acknowledgements

This article was broadly inspired by an article in Harvard Business Review, by Philip Evans and Bob Wolf entitled “Collaboration Rules” [11]. Corey Ladas also contributed his thoughts and ideas inspiring the comparison of free market economics with those of collaborative networks.

References

[1] Coase, Ronald H., The Nature of the Firm, Economica, 1937

[2] Liebowitz, S. J., and Stephen E. Margolis, Network Externalities (Effects), The New Palgrave’s Dictionary of Economics and the Law, MacMillan, 1998

[3] Anderson, David J., Recipe for Success, http://www.agilemanagement.net/index.php/blog/Recipe_For_Success, January 2007

[4] Highsmith, Jim, A Recipe for Success, Part 1, Cutter Consortium, Agile Project Management Advisory Service E-mail Advisor, February 2007

[5] Highsmith, Jim, A Recipe for Success, Part 2, Cutter Consortium, Agile Project Management Advisory Service E-mail Advisor, February 2007

[6] Highsmith, Jim, A Recipe for Success, Part 3, Cutter Consortium, Agile Project Management Advisory Service E-mail Advisor, March 2007

[7] Highsmith, Jim, A Recipe for Success, Part 4, Cutter Consortium, Agile Project Management Advisory Service E-mail Advisor, March 2007

[8] Highsmith, Jim, A Recipe for Success, Part 5, Cutter Consortium, Agile Project Management Advisory Service E-mail Advisor, April 2007

[9] Anderson, David J., Stop Estimating, http://www.agilemanagement.net/index.php/blog/Stop_Estimating, March 2005

[10] Anderson, David J., Kanban in Action, http://www.agilemanagement.net/index.php/blog/Kanban_in_Action, March 2007

[11] Evans, Philip and Bob Wolf, Collaboration Rules, Harvard Business Review, July-August 2005

Revision History

Revision 1.4, August 2007

Download

[Download in PDF 152KB]

Related post: Blog entry: Stop Negotiating, Start Collaborating Technorati tag: Agile, Lean, Kanban, Software+Engineering, Cutter, David+Anderson, Pollyanna+Pixton, APLN

SPIP: Providing Value to Customers in Software Engineering Development Through Lean Principles

Co-authored with Merwan Mehta and David Raffo

Providing value to customers in software development through lean principles

Abstract

Lean thinking has created substantial benefits for a variety of industries. Although it is not easily apparent, the same principles can be utilized in reducing lead times for software development. Fundamentally, lean principles leverage the benefits of software engineering best practices to improve the work flow and the tactical management of the project. By managing the project more efficiently and reducing waste, tremendous performance benefits can be achieved.

This article seeks to present and articulate a number of key lean principles in the context of software development. The goal is to provide insight into practical strategies for making trade-offs between the core lean principles of providing the highest possible customer value, maximizing work flow, and eliminating waste in the context of specific real world software development situations. Copyright © 2008 John Wiley & Sons, Ltd. Technorati tag: Lean, Kanban, Software+Engineering, David+Anderson, David+Raffo, Merwan+Mehta

Providing Value with Lean

While I haven’t been blogging much over the winter, I wasn’t entirely idle. I co-authored two important papers. The second of which is the forthcoming Technical Note from the SEI entitled “CMMI and Agile: Why not embrace both?” with Mike Konrad, Hillel Glazer and Jeff Dalton [More on this when the SEI publishes it.] The first is an academic paper that appeared in the Journal of Software Process: Improvement and Practice, co-authored with Merwan Mehta and David Raffo “Providing Value to Customers in Software Development Through Lean Principles.” [Unfortunately, this paper though available online is a download from Wiley, both the journal subscription and the paper re-print are very expensive. If you have an interest in this paper, send me an email.]

I’m very pleased with both of these papers. The co-authoring experience in both cases was very enjoyable and rewarding. The outcomes were superior to anything I might have written on my own.

There are many lessons in the Lean paper, but perhaps the most important is this message…

Value first, then flow, then waste reduction/elimination

Too often I see Lean being taught as “the elimination of waste” when in fact waste reduction is a tertiary concern. While waste elimination is important, waste should not be reduced in detriment to flow, and smooth flow can be sacrificed to improve value delivery.

So for example, a queue in a kanban system can be considered as waste (probably necessary waste). As queues are a type of waste, it makes sense to reduce or eliminate queues. I see this advice coming from others who talk about Lean in software development.

However, it is important to understand why the queue is there. It is there to absorb variation in size and complexity of work items or in the availability of someone to process those work items. If you reduce the size of the queue or eliminate it altogether, it may impeded the smooth flow of the system, causing a stop-go effect, lowering the overall throughput of the process.

For example, (and I talk about this when I present kanban at conferences) at Corbis we had a non-instant availability bottleneck in build engineering, due to the time-slicing (multi-tasking) required of the build engineers. As a result, we increased the size of the queue in front of build engineering in order to smooth the flow through the whole system. In this example, the variability comes from the availability (or lack thereof) of the resource, not from the sizing of the work items. So we increased the amount of “waste” in the system to smooth flow and increase throughput.

It is also known that expediting is bad. Expediting impedes flow and causes WIP to increase and lead times to lengthen. We have demonstrated this empirically with the kanban implementations we’ve done so far. If we are to focus on smooth flow we would never expedite.

However, this general rule would be wrong. Occasionally there will be times where an expedite request carries a very high monetary value. The impact on flow and WIP and lead times to other items in progress is outweighed by the monetary benefit of accepting the expedite request. Hence, it makes sense to pursue the value in the expedite request despite its impact on flow and its introduction of waste in the system.

Hence, value trumps flow and flow trumps waste elimination in all process operation and process improvement decisions. Technorati tag: Agile, David+Anderson, Software+Engineering, Management, Lean, Kanban, David+Raffo, Merwan+Mehta