Friday, June 11, 2010

Reflections on a Couple of Others’ Thoughts on Scrum Compared to Kanban

A couple of guys both longer in the tooth than me, who have authored more books than me, have also published their experienced thoughts on Scrum and Kanban this past week.

First Alan Shalloway, The Real Differences Between Scrum and Kanban.

Alan lists a number of practices that he sees as different between the two approaches: some of them are core fundamental elements of the Kanban method - the seed properties that generate the emergent Lean behavior in the organization.

Kanban differs from Scrum in the following ways…

1. Make process policies explicit
2. Visibility of process, not merely results
3. Inclusion of Management
4. Flow as an option instead of time-boxed iterations
5. Controlled change management

You need to read Alan’s full post. I feel it is fair and balanced and represents an alternative perspective that broadly follows mine

Secondly, Ken Schwaber posted his own thoughts. Ken had evidently been attending the Microsoft TechEd conference and had seen a presentation on Scrum and Kanban by Joel Semeniuk and Steven Borg. This prompted him to post, Waterfall, Lean/Kanban, and Scrum

This article talks a lot about complexity theory and understanding processes as simple, complicated, and chaotic, and how Ken’s motivation for creating Scrum was based on providing something that worked in the chaotic space. This is all fair enough and interesting in itself. However, the post seems biased by Ken’s assumption that we’re doing manufacturing kanban systems on software development. Ken has no actual experience using Kanban. His post also highlights a fundamental principle of Scrum that the underlying process for software development cannot be mapped. This point is reinforced in an older post by Tobias Mayer, Scrum and Kanban - different animals, that was referenced by Alan Shalloway.

This reveals a truly fundamental difference between Scrum and Kanban that I missed in my post yesterday or more accurately between the Scrum community and the Kanban community. It’s not an accident that Tobias’ blog is called Agile Anarchy. The underlying assumption is that trying to map the creative process that produces software is futile and there is no point in trying. It simply has to be contained in a black box. Ken refers to this as a “container” in his post and that container is a Sprint. The Kanban community believes in a scientific approach and the use of models to understand the natural philosophy of what is at work in our universe. While we recognize that all models are wrong, we recognize the value in some models. A value-stream map is a model of a process, it is probably not an entirely accurate model, more an approximation. The question is whether it is useful or not?

Once we have that model, and can observe actual behavior against it, we can then use other models such as the Theory of Constraints, Theory of Profound Knowledge, Systems Thinking and Lean’s Waste (or the economic models of transaction and coordination costs), to analyze what is happening and suggest changes. We accept that these models are incomplete and that any suggested changes will not work in all circumstances. The question is whether they work often enough to be useful?

It seems to me that we have an overwhelming quantity of evidence that has been reported from firms large and small from all corners of the world at the Lean & Kanban 2009, UK Lean 2009, Lean & Kanban Exchange, Lean Software & Systems 2010 and much more to come at Lean & Kanban Europe 2010, and LESS 2010 later this year, to demonstrate that this scientific approach is working and genuinely helping organizations and teams improve both their economic performance and also the social capital of their organizations - resulting in a better sociological outcome for everyone involved.

[I’ve decided to update this post on June 12th following a private email from a Scrummaster who has been a lifelong believer in the anarchist ideal]

It appears that few practicing Scrum followers realize the underlying anarchist ideals held by the leadership of the community and designed into Scrum as a method. It appears that this Scrummaster sees Scrum as very prescriptive…

“I think of scrum as being a very strict system of prescribed practices that are imposed whether they fit your needs or not.”

and then provides an explanation of true anarchy as

“Anarchy means “no government”, and as practiced by modern anarchists,  is a system of total self regulation, living a life of personal responsibility, and the abstention from imposition of one’s own will onto others as happens with violence, oppressive social behavior (even aggressive driving) and voting.  It is closer to a religion in this way and it is more like perfect order than chaos.”

It appears that these ideals are not obvious in Scrum but I believe it is a fair assessment to characterize at least some of the leadership in the Scrum community as true anarchists. They believe in “no government”, “total self regulation”, “a life of personal responsibility”, and “abstention from imposition of one’s own will onto others.” This particular Scrummaster went on to say that…

“I can’t even tolerate a queue that is force ranked. Too command and control.”

And this highlights an incongruence in how Scrum is articulated, taught and practiced. Both Scrummaster and Product Owner training contain advice, guidance, and practices (belonging to the role) that are considered antithetical by a true anarchist.

The second issue that became obvious to me from this email exchange is that few people appear to understand my linking of the “use of models” and “science.” Perhaps this is a reflection of how poorly science is taught in our schools? So let me explain. Newton’s Law of Gravity is a model for a small part of the natural philosophy of the universe. It explains, in part, how things are attracted to each other and stick together, and why objects fall to earth. Newton’s Law of Gravity is known to be in inaccurate model. It only works at relatively large scale and has needed modified several times since Newton created it to accommodate concepts like distance from the center of gravity, varying levels of gravitational pull, and the other core forces of the Universe that were not known to Newton in his day. However, we need to ask whether Newton’s Law is useful or not despite its lack of completeness? The answer to this is surely “Yes!” for many common everyday commercial, scientific and military applications around planet Earth Newton’s Law is perfectly adequate. It is this pragmatic approach that is commonly held in the Kanban community. Models can be useful. They can allow us to make improvements even when they do not work in all circumstances. There are many in the Agile/Scrum communities who do not share this pragmatism and believe that in the absence of a complete model that defines the full natural philosophy of the domain that only an emergent solution based on empirical observation, experimentation and feedback is possible. The concept of predicting an outcome based on a scientific model is not acceptable nor does it exist within their paradigm. This can lead to discussions where individuals talk past each other. Kanban community folks using a scientific paradigm talking with Scrum advocates with an Edge of Chaos paradigm fail to comprehend what the other is saying. This pattern has happened frequently in the Kanbandev Yahoo! group when new folks arriving with a strong Scrum background try to understand Kanban by force fitting it to their existing mindset.

It appears that Scrum leaders like Ken and Tobias openly position Scrum as as a solution for Chaos, that values Anarchy and Revolution. While Lean and Kanban leaders like me, Alan Shalloway and Donald Reinertsen position Kanban and Lean as a scientific approach that approximates chaos as merely complicated or complex and delivers an Economic, Scientific and Evolutionary approach. This juxtaposition seems useful as a selection criteria for organizations. Do you wish to use a framework for change that tames chaos through anarchy and revolution, or a framework for change that models complexity incompletely but usefully and uses science to generate significant evolutionary economic and sociological improvement? (even if those improvements are theoretically less than optimal?) Choice is good! It is only right that one approach should not fit all and that you should have the best information available in order for you to make an informed choice that is appropriate for your business and organization. It’s been refreshing that folks like Ken and Tobias have published their thoughts on how Kanban differs from Scrum and we’ve been able to raise the quantity and quality of information available to the market.

Thursday, June 10, 2010

Thoughts on how Kanban differs from Scrum

I’ve largely stayed out of the debate comparing Scrum to Kanban or those wishing to position the techniques as rivals. I’ve actively encouraged Mattias Skarin and Henrik Kniberg who have done a very good job analyzing and comparing in their book Scrum and Kanban making the best of both!

I feel that I can add some insights that Henrik and Mattias didn’t cover - insights that have emerged while I’ve been touring the world this past nine months working with teams and agile coaches on 5 continents from small innovative startup firms to some of the world’s largest industrial and technology businesses.

At one level Scrum is presented as quite a prescriptive project management process - Sprints, Scrums, Sprint Planning, Retrospectives, Demos etc etc. The leadership of the Scrum community is anxious to point out that Scrum is much more than this - it is a framework for provoking change and emergent behavior in organizations.

Kanban is not a project management or software development lifecycle method. It is an approach to change management - a framework for catalyzing change in an organization. So it differs from Scrum in that it cannot be used as a process to get work done. It needs to be applied to an existing process. That existing process can be Scrum, or equally any other lifecycle method with which you are familiar, or something that has no name - an ad hoc process. However, as a framework for change Kanban and Scrum can be compared as alternatives.

It is in this area, as frameworks for change, that I feel I can add to the existing literature as Kniberg & Skarin did not address this aspect.

Change Catalyst

Kanban uses the WIP limit as its control mechanism to provoke conversations about change. Failure to respect the WIP limits and discuss problems will lead to stagnation and a failure to improve. Improvement discussions are objective as the visualization, measurement, explicitness of policies and the models from Lean, Theory of Constraints and the teachings of W. Edwards Deming, allow a team to scientifically analyze their problems and propose solutions.

Scrum uses commitment as its control mechanism for provoking changes. Commitment exists at two levels: at the personal daily level - this is reinforced with the Scrum and the 3 questions “what did you do for us yesterday?” “what will you do for us today?” and “Is anything impeding you from meeting your commitment today?”; the second commitment is at the team level, the Sprint commitment, a promise to deliver something on a certain date. Failure to meet a commitment should lead to a discussion about the failure that should provoke a process improvement. While we could debate the effectiveness of this approach, let’s accept that it works.

So Kanban uses a WIP limit as a change agent and Scrum uses commitments. This is a fundamental difference in approach.

Evolution rather than Revolution

Ken Schwaber has talked about the “hard words of Scrum” such as Scrum, Scrummaster, Sprint, and so forth. Scrum is intended to be shock treatment for an organization. It often involves people taking on new job titles, new roles and responsibilities and it is generally introduced in managed fashion with training and a defined date to switch to the new approach. Introducing Scrum represents a revolution. It shakes up the social hierarchy of an organization and affects (both negatively and positively) the professional self-esteem and egos of team members. Not every project manager values being told they need to retrain and assume a new job title! For some organizations this will be the right approach - they need shock treatment to avoid disaster. For others, all it does is raise emotional resistance amongst the workforce and encourage passive aggressive behavior.

Kanban is designed as the antithesis of the Scrum approach to change. With Kanban you start with the process you have now. You “kanbanize” it by mapping the value-stream, visualizing it, and limiting work-in-progress to create a pull system. You leave existing roles, responsibilities, job titles and practices intact. The only changes are to the interface with upstream and downstream partners, such as business owners and systems operations. Any changes made here are specifically chosen to avoid shaking up the social hierarchy or invoke an emotionally defensive response from affected people.

Kanban provokes evolutionary change. Initially this means process optimization - Kaizen - but gradually as the organization matures in capability it leads to larger managed changes - Kaikaku. It has been observed that progressive kaizen events lead to improved organizational maturity and capability enabling more dramatic kaikaku level changes.

So Scrum is introduced in a diametrically opposed fashion to Kanban. Kanban is softly softly and Scrum is hard shock treatment!

Conformance to Process

While Scrum is advertised as a starting point and a framework that will provoke change, there has been a growing market in practice-based conformance and assessments. The best known of these, endorsed by Jeff Sutherland is referred to as the Nokia Test. These tests determine whether you team is following Scrum. If not then the team is said to be “Scrum-But” and Ken Schwaber has taken to calling practitioners of non-conformant Scrum by the insidious term “Scrum-Butt"s. The effect of these conformance tests is to discourage innovation and deviation from textbook definitions. The net effect is confusion as on the one hand the leadership of the community tells practitioners that Scrum is about catalyzing emergent behavior in their organizations, while the same leadership prescribes a test that is designed to punish those who deviate from a standard definition.

Kanban is designed as an approach that will customize and evolve an existing process regardless of what that existing process is at the start. It is therefore impossible to define a conformance test for Kanban. Kanban uses 5 seed properties to catalyze the emergent behavior of process evolution. These 5 properties are: Visualize the Workflow; Limit Work-in-Progress; Measure Flow; Make Process Policies Explicit; Use Models to Evaluate Improvement Opportunities. These represent the 5 practices that must be present for the Kanban approach to change to work. The team’s process for software development and project management will always be unique and over time will be tailored and optimized to the value-stream, risk profile, skills and capabilities of the team, customer demand, bottlenecks and sources of variability that affect the team. There can be no test for conformance. Any measurement that might apply should measure whether a better economic and sociological outcome was affected by the introduction of the Kanban approach.

Containers versus Whole System

There are a number of other properties that emerge with teams using Kanban that differ from practices of teams doing Scrum. These emergent differences are mostly a side-effect of not pursuing the “Container” approach of Scrum. Scrum uses a container known as a Sprint that time-boxes a batch of development work and discourages interference from outside. Ideally there should be no interference. Scrum seeks, like good software architecture, to minimize the interfaces from the container, to achieve a loose coupling. The desired effect is to make the activity within the container - the Sprint development - as predictable as possible, in order to meet the Sprint Commitment.

Kanban does not introduce such a container, instead it encourages a whole system view. A number of mechanisms simplify the coordination of elements in the whole system. For example, the combination of visualization and a WIP limited pull system enable a very simple interface with business owners. As a result most organizations adopting Kanban do not need the single Product Owner concept of Scrum and can easily cope with multiple competing business owners attending queue replenishment meetings.

Kanban daily standup meetings have been shown to be effective with up to 50 or more people. The reason for this is that the team are implicitly trusted to be doing the work that is shown in the visualization of the workflow. There is no need to use the standup to reinforce personal commitment and hence the standup can focus on the work and not the people. Teams will iterate over the work tickets rather than through the team members. The three questions are obviated. More mature Kanban teams reduce discussion only to work that is impeded or defective, focusing only on exceptions rather than work that is proceeding normally.

Organizations using Kanban have also been observed to merge smaller teams to take advantage of the reduced coordination costs and better utilize their labor pool. It’s become common to see teams of 20 to 30 and sometimes up to 50 being created often from multiple (former) Scrum teams, or from a mix of Scrum and non-agile teams. The workflow visualization often involves multiple rows (or swimlanes) to represent different streams of development. Some team members may be assigned to a swimlane as permanent team members, accountable and responsible for work in that lane, while others are allowed to float across lanes to provide staff augmentation or specialist skills. The result is more effective and efficient use of the available people, resulting in more throughput and shorter lead times.

Closing Thoughts

I believe that we are only beginning to understand the differences between Scrum and Kanban. I believe that as more emergent properties of Kanban organizations are observed as recurring patterns in the field, we will grow in our understanding. Kanban differs from Scrum. Where we are still learning is how introduction of Kanban to an organization using Scrum changes that organization, its culture and its approach. I believe that Kanban is compatible with the mechanics of Scrum. It is compatible with Scrum, the project management method. Adding WIP and visualization to Scrum will help improve Sprint Commitment effectiveness. However, it is also introducing the WIP limit as a mechanism to catalyze incremental changes. Scrum teams adopting this approach are said to be Scrumban teams. The WIP limit obviates the need for commitment to drive change, reduces any dysfunctional reliance on heroic effort, and improves overall systems thinking when considering potential improvements. Scrumban will replace the Scrum philosophy and framework with the Kanban approach. It may still look somewhat like Scrum at the practice level but at the cultural level it will be Kanban - softly softly evolution rather than shock treatment and revolution.

When considering whether Scrum or Kanban is right for you and your business consider the culture and maturity. If your organization has low maturity, limited capability at risk management, change management and decision making, and is riddled with specialization and defensiveness then Kanban is probably a better choice. If you have time to let the culture and performance evolve and improve over months and years then Kanban is the right choice. If on the other hand, your organization is highly mature and capable of assessing risk, evaluating process alternatives, making good quality decisions, and managing high impact change gracefully then Scrum may be the right choice for you. If you are facing an extinction level event and you don’t have time to let evolution work its magic on your organizational performance then perhaps a Scrum revolution is worth the risk.

Tuesday, June 02, 2009

Lean Software Austin

Looks like new sponstaneously affiliating groups are shooting up all over. Some of the folks who attended Lean & Kanban 2009 are behind Lean Software Austin. Another potential hub for the Limited WIP Society. Reading Scott Bellware’s tweets on this, it seems that Agile Austin is too Scrum-centric and those who want to innovate and try something else feel squeezed out. How many more are like this? Eric Lefevre asks, Is Scrum Evil? Technorati tag: Lean, Kanban, Software+Development, Project+Management, Software+Engineering, Scrum

Wednesday, April 01, 2009

Recent Blogospehere Commentary

Israel Gat (Agile Executive) posted his thoughts on my Kanban & Retrospectives blog and made some comparisons with Scrum.

Israel Gat also posted his thoughts on Continuous Deployment (or Customer Driven Testing) following an interesting Twitter discussion mainly between James Shore and me. What’s interesting about this one is the consideration that even if the technical team is capable of continuous deployment it is undesirable if the end customer is incapable of absorbing such frequent releases or the economic cost to that customer is unacceptable.

And Dave Nicolette’s been in on the action too, explaining how Kanban is an Agile method. This post from Dave was prompted by my Kanban & Planning and Estimation post to which he posted a comment.

And Eric Willeke spotted this blog with lots of Kanban posts… WingingIT  Technorati tag: David+Anderson, Agile+Management, Agile, Lean, kanban, Software+Engineering, Project+Management, Israel+Gat, Dave+Nicolette

Wednesday, November 26, 2008

Trashing Scrum or Reflecting Reality?

Tobias Mayer thinks the Lean folks in our community have been trashing scrum!

While his criticism was aimed at Mary Poppendieck, others associated with Lean in software development, most notably Alan Shalloway and I have said things that the Scrum community don’t like. So I think it worth expressing my point of view on this.

For 6 years, since I developed the manuscript for my book, I’ve been saying that Scrum is a useful process that helps immature teams achieve success. However, it only takes them so far and that more is needed if they are to keep improving. Based primarily on the available literature at the time - 2002 - I observed that Scrum is designed at the practice level to eliminate a lot of external variability that affects the performance of most development teams. As such, this recipe of practices, or prescription will have a quick positive effect on performance, but without a true focus on organization level continuous improvement and cultural change, it will fail to generate further improvements. The result will be a [more or less] unit step improvement in productivity and reliability [and possibly quality]. The Scrum community doesn’t like this observation.

However, many of us working in the field find senior people in large companies telling us - we’ve tried Scrum and it worked for a while but now we are looking for what is next, for the thing that will take us to the next level. This isn’t an issolated comment. And those saying it are not critical of Scrum. They simply recognize that it gave them a boost in performance and reliability and now they need something more to continue improving. While the leaders in the Scrum community like to promote the concept of a culture of continuous improvement, Scrum does not appear to have enough depth in its guidance, literature, training and coaching to get teams there. Perhaps these teams are amongst the teams that Ken Schwaber describes as having “failed with Scrum.” If that’s true then as Alan Shalloway suggested, Scrum is failing us. If what it takes to really do Scrum right, you need Jeff Sutherland or one of his folks to be present, then more work is needed - more guidance, more depth, more theory, more practices. The literature simply leaves too much as an exercise for the reader.

Jeff Sutherland has been the voice of reason in the Scrum community over the years. He has been the one talking about the challenge of achieving hyper-productivity [4x productivity improvements] and the one publishing metrics. It turns out that few Scrum teams achieve the hyper-productivity and of those who do when you look at the absolute numbers the performance is good but not stellar. Again, this reflected an observation I made years ago, that many agile teams were anecdotally reporting relative improvement but not absolute numbers for productivity or quality. I strongly suspected that FDD teams were out-performing many other agile teams significantly and this turned out to be a correct assumption when years later metrics began to appear. Even the original FDD team in Singapore building a huge enterprise system produced productivity at the higher end of scale that Sutherland reports. The team at Motorola that I was running, particularly on the OTA Device Management project exceeded the productivity of the Borland Quattro Pro project that Jeff likes to quote as one of the best ever and not only did we have the code production metrics, we also had the initial quality as escaped defects to system test numbers. Both were in the 99th percentile for the industry. It’s only recently that, as a community, we’ve started to have an open debate about absolute performance of agile teams and look at what truly affects performance. What’s driving this is enterprise scale adoption. It is no co-incidence that big corporations want to see productivity data. So while, Scrum has clearly helped a lot of teams improve and the relative improvements have often been enthusiastically received, the absolute numbers, seem to imply that more is needed. As Tobias reports Mary Poppendieck reflecting on Jeff Sutherland’s approach, Jeff does more. He includes many software engineering and risk management practices that in my observation improve the organizational maturity and ultimately produce significant productivity improvements that are remarkable in absolute terms, not just in relative terms. I often lament that Jeff Sutherland has not written a book.

Apparently, trying to have this kind of rational discussion about Scrum and its practices is unacceptable. The Scrum community doesn’t want to hear anything other than “Scrum is great!” and “Scrum is the answer to all problems.” Any form of discussion or debate is regarded as dissent. This leads me nicely to my second observation…

My second observation about Scrum relates to the community itself. In recent years, any form of debate that suggesting there may be challenges with Scrum or implementing it has been rooted out and abolished. There were several public ex-communications from the Scrum Yahoo! group including prominent figures like Scott Ambler and Alan Shalloway. Others like me, were warned off by Ken Schwaber and chose not to participate further. My observation from the outside is that the Scrum community reflects the antithesis of our agile values. It is run from the top. The message is strictly controlled. Dissent is not permitted. It resembles a cult of personality and appears to be the very definition of command’n'control in its execution. Speaking from my own experience forming the APLN, we were very conscious that we must create an agile organization that truly reflected the values that we espoused. So we created an organization that encouraged spontaneous affiliation, encouraged diversity, had a small government run at almost no cost, requiring no annual fee, that encouraged innovation and dictates no rules to members or local chapters. If you are trying to create a culture where people feel free to speak up and raise issues at daily scrums, I cannot see how this is possible, when the organization doing the coaching discourages any form of dissent.

So when you hear someone from the Scrum Alliance accusing others of trashing Scrum look deeper. Apparently, if you say, Scrum really helped a whole bunch of projects, teams and companies improve productivity and reliability on project delivery, but the improvements topped out and now they need something more to really keep moving you , are trashing Scrum. This leads me to the conclusion that if Scrum is so easily trashed, how can it have any real substance or worth? If the leaders in the Scrum community truly believe in its worth and value then they should show some maturity and open themselves to constructive professional objective debate.

Related articles: Why we lost focus on development practices Technorati tag: David+Anderson, Agile+Management, Agile, Lean, Scrum, Extreme+Programming