Tuesday, November 27, 2007

Enterprise Scale Continuous Integration

Back in January when I spoke at the OOP conference in Munich, I described how I didn’t believe that continuous integration scaled to enterprise level. Indeed, we hadn’t managed to make it work. What we were doing was taking a more Lean approach to integration - little and often - and moving to fewer codelines. We were achieving this by implementing latent code patterns that enabled several projects to live in the same environment and avoid the problem of new code accidentally escaping in to the wild when another project was due for release.

This was all very well and given enough time and enough focus on Lean principles we might yet have evolved to a continuous integration approach for the enterprise but it would have taken years.

For the past six months I’ve been lucky enough to have Troy Magennis on my staff as our Enterprise Architect. Troy brought a wealth of experience in .Net and C# and large scale Microsoft technology projects to our team. Given that we suffer numerous impacts to our productivity with the constant challenge of code line management, integration, build and environment build out and reset, Troy refused to accept that continuous integration wasn’t possible, and indeed in his view it was essential to eliminating waste and maintaining a regular flow of valuable software in to our production environments.

So together with one of our architecture team and a toolsmith from our configuration management group, they spent some considerable time and effort tackling the problem of how to build our entire enterprise code base in a continuous fashion. A few weeks ago, they finally achieved it and we now have cruise control reports on the state of our enterprise code base at any given time.

This doesn’t mean that our enterprise DEV environment can be pushed to production whenever we choose. At least not without successful implementation of latent code and full regression testing to show that the new code is truly latent, but the result is that we have reduced our code line maintenance to a single line for all major projects in our portfolio plus a branch for sustaining engineering (released on a 2 week cycle).

At my prompting Troy has gathered his thoughts on Does Continuous Integration Scale to Enterprise Projects? what it takes to achieve enterprise scale continuous integration and how to implement it, in a white paper available over on his blog. You can get it in PDF here. The main takeaway from this article isn’t a revelation about configuration management or build tools. The key to enterprise scale continuous integration is solid enterprise architecture and enforcement of good software engineering principles of loose coupling and high cohesion across all projects in the portfolio.

Yes, all you agilistas out there! Architecture does matter, if you are to scale agile techniques to the enterprise! Technorati tag: Agile, Software+Engineering, Enterprise+Architecture

Bugs and Kanban

Corey Ladas takes a look at two ways to treat bugs in a kanban system. The second option is the more challenging. It requires patient courageous management. My feeling is that option 2 will produce the net higher velocity (and throughput) in the long term because it teaches the team to really focus down on prevention of bugs while option 1 treats bugs as part and parcel of the business of software development. While option 1 will suffer a throughput reduction as bugs increase, there is far less incentive to focus on eliminating them altogether.

Ideally, I’d like to run a side-by-side comparison over a 6 to 9 month period to compare these two options. How are you dealing with bugs in your kanban system? Like option 1 or option 2 or do you have you own alternative approach? Technorati tag: Agile, Lean, Kanban, Software+Engineering

Wednesday, November 14, 2007

How big can an effective standup be?

This is a picture of a standup meeting on a large project at Corbis. Today I counted 41 attendees. The attendance has averaged 39 or 40 every day for 6 weeks. Daniel Vacanti can be seen far-left leading the meeting. The kanban board is behind him out of shot. [Not everyone is standing as some are sitting at their actual desks in the communal work area. This meeting happens every day at 9.30am. It’s generally over within ten minutes.

Corey Ladas has blogged about how the Lean Kanban process scales differently and how a standup can be effective with very large numbers of people. The key is that we don’t enumerate through the people asking them what they have done and will do and whether they are blocked. We enumerate through the kanban cards on the board and discuss whether work if flowing and what if anything is blocking it.

This project has 6 dev leads and 6 dev teams working concurrently. If this were a scrum project we’d have 6 independent scrums followed by a scrum-of-scrums with only the leads (assuming the lead is playing the scrummaster role.)

I see pros and cons to the big meeting. It is efficient. It gets the whole team together, not just the sub-teams. But it clearly isn’t as personal. And the claimed peer-pressure aspect of a scrum standup meeting is missing.

What we are finding is that some of the dev teams hold a brief huddle afterwards for just their own team. This is entirely optional, doesn’t happen every day and doesn’t (generally) happen in front of the kanban board - more a design coordination meeting than a project tracking meeting.

Is anyone else running standup meeting with larger than the recommended scrum size of 6 to 12? What are your results? What’s the largest effective standup meeting you’ve been a part of?

Photography: courtesy Laurence Cohen
Related Post: Lean Scales Differently than Agile, Corey Ladas Lean Scales Differently than Agile Technorati tag: Agile, Lean, Kanban, Software+Engineering, Scrum, David+Anderson, Corey+Ladas

Kanban Workshop Dec 3rd

Have you been wondering about how a Kanban board really works for software development?  Come to the next Seattle APLN meeting in three weeks and find out.

Who:      Darren Davis, Manager at Corbis
What:     Kanban workshop
When:    Monday December 3, 5:30pm - 7:30pm
Where:   Avanade Inc. ( 2211 Elliott Avenue, Seattle, WA)
Why:      Learn the advantages of a true single-piece flow pull system and how to implement it in your organization

From Darren:

The software engineering team at Corbis has spent the last year creating and running a development process based on the principals of Lean Manufacturing.  Using this process, we have released over 190 change requests and bug fixes at regular two week intervals with very high quality and minimal management overhead.  Critical to the success of this process has been the use of a Kanban board.  Using such high-tech implements as post-it notes and Sharpies, we have created an information-rich way of controlling the amount of work in progress, minimizing inefficient multitasking, identifying and addressing blocking issues and exposing untracked work.

This presentation, in workshop format, will define a software development workflow and create a Kanban board to track work through it.  We will demonstrate the use of post-it notes and other visual clues to provide transparency into the development process in a way that technical and non-technical people can easily understand.  We’ll finish up by running a standup, at the board, to illustrate how the board is used day-to-day to organize and manage development work.

My earlier blog post Kanban in Action discusses this process and shows Darren at the Kanban board.

Please RSVP to Dragos Dumitriu (dragosd at avanade dot com).  If the door is locked call Dragos (425.260.9283) or David Socha (206.418.8201).

To receive all Seattle APLN emails, subscribe to our Yahoo! Group.

More information is at the Seattle APLN wiki.

Monday, November 05, 2007

APLN Leadership Summit in Seattle?

The APLN Seattle Chapter would like to organize a Leadership Summit next spring. We’re looking for project managers in the Seattle area to help with the event planning. If you’d like to help organize a fun conference for project managers and are interested in getting involved in the APLN locally, please leave a comment or drop me an email to david dot anderson at corbis dot com. We’ve already secured a venue and we have volunteers for business development to drive the sponsorship program. We need someone to help coordinate the agenda, the invitations, the speakers and organize a fun social event in a nearby hotel. If you’re up for it and perhaps have some previous event planning experience, we’d love to hear from you. Technorati tag: APLN, Project+Management