One of the workshops I run is to help team members understand root cause analysis. I use it with operations teams as well as product development teams. My workshop goal is to have people leave with a basic understanding and some practice.

I created the diagram below to situate this workshop in a larger context of Kaizen (Continuous improvement).

One starting point is with a team using a Scrum board or Kanban to create BIG visible information. This supports teams in identifying problems or waste to stop the production line and investigate the problem using root cause analysis tools. I introduce two tools and have the participants practice with each:

• Ask Why five times – powerful technique for drilling down into problems.
• Fishbone (Ishikawa) Diagram – looks at the big picture around the problem.

Both of these improve quality to help teams go fast. 5S (sort, straighten, shine, standardize, sustain) is also totally applicable for software – it’s called clean code: coding standard, refactoring, etc.

Finally, the foundations for Kaizen and root cause analysis are:

• NO BLAME. Most problems are related to the system, not individuals.
• Team work and trust. If 100 people each help find 1% improvement, this will be sustainable.
• Genchi Genbustsu – Go and See.  When you work on a problem, go to the source and get the facts for yourself. Root cause is about investigation and problem solving – see and think for yourself.

If you want to learn more, check out Implementing Lean Software Development: From Concept to Cash or Toyota Production System: Beyond Large-Scale Production.

Workshop Slides

Here is a slide deck I have used in training. It’s from last year, and would benefit from a refresh to cut down on the text – still very usable. As usual, you are welcome to use this as well as the diagram under Creative Commons license.

I recently attended a ThoughtWorks QTB – Rapid, Reliable Releases (AKA It’s not making money until its in production) by Rolf Russell and Andy Duncan. It was a solid presentation around the importance of managing environments effectively.

I will walk through the diagram below starting with …

a reliable continuous integration system creates a foundation for rapid reliable releases. It provides early integration and gives the teams rapid feedback on how they are doing. Everything else follows from this.

A key ingredient for successful management of production and other environments is collaboration with dev and ops. There is even a new type of role appearing – DevOps – that handles both development and operations. Either way, someone needs to champion this important interface.

The next topic was about using value stream mapping to see what it takes to actually get something into production (I heartily agree with this approach as I have found it very effective). What we often see as a result is that the work is optimized for each department and is not aligned with the best needs of the whole business and there are lot’s of delays and handoffs.

The next step is to envision future state - what items can go in parallel to get feedback as soon as possible. In one environment is was functional testing, performance testing, and UAT.

A practical idea is to use the cloud for scalable parallel Continuous Integration. In this situation we need lot’s of computers for a short burst, so this lines up nicely with the cloud’s pay per use model and lets testing go really fast.

The final bits of advice were around creating deployment recipes that go with the code (including database changes). In order to accomplish this you will need to create consistent environments and encapsulate differences (such as number of servers, specialized hardware, etc). Once you explicitly handle all the risky bits, then it is easy to support reliable and automated deploys.

If you haven’t done this sort of thing before, then it may seem like a lot of work. For me, it’s seems just normal. I personally implemented a lot of this when I worked at a startup in 2000 and have been using it ever since. I can say firsthand that work on this really pays off.

Video is available on InfoQ.

Derek Longmuir presented ThoughtWorks QTB on working with legacy systems. You can see the video and slides on InfoQ.

I like the definition given by Michael Feathers:

Legacy code is simply code without tests.

Legacy Systems have Value. They are usually business critical and feature rich. They may even be stable and reliable (YMMV). Hint: in the MindMap below, start at 3 o’clock and go clockwise.

Why change from a legacy system? There are number of good reasons: obsolete technology, can’t add features, system is fragile.

This is an important problem since most systems we have 10% of the effort to build and 90% effort to maintain. So to manage costs, we need maintainable systems.

What to do?  Traditional approaches such as Big Bang (think explosion) and wrapping/hiding legacy systems rarely achieve business objectives.

Do a system health check. Look at the code. Get complexity measures. Look at test code ratio. What state is the system in?

Before getting started, there are some tools that you will need for basic technical hygiene. The equivalent of brushing and flossing your teeth is test and build automation.

How to tackle this?

With the Strangler approach, the goal is to strangle the existing application by building a new system that runs in parallel. The idea is to put a new interface in place and begin migrating the functionality in a piecewise fashion. This approach takes a lot of effort and makes sense when there is no hope for the existing code base.

The Phased approach is about rehabilitating the system piece by piece. How do you eat an elephant? One bite at a time.

Strangely, there was no mention of the bible on this topic: Working Effectively with Legacy Code by Michael Feathers. This book is a must-read on this topic. As is Refactoring, Refactoring to Patterns, and Clean Code.

Ken Schwaber has a great presentation where he talks about the Innovator’s Dillemma and how companies build a (design) dead core.

A typical (success) story

• Management – we need to hit the date.
• Developers – OK, we’ll cut quality but won’t tell you.
• Success! We made the date. We’re heroes!

BUT, this is a horrible long term strategy because you get a design-dead core and can no longer ship product.

Design-dead core

Do you have  a design-dead core?  Here’s a quick checklist (see mindmap below):

1. The code is fragile: difficult to work with and things break unpredictably
2. Little or no automated test harness.
3. Few experts who really understand the technology.

Innovator’s Dilemma

The purported dilemma is that you need to choose between fast delivery and maintainability. So, if you want to get to market fast you need to take shortcuts that are going to hinder you in the long run.

This is also called the inventor’s dilemma.

Agile to the Rescue

Teams that follow Agile practices avoid this peril in two ways.

By managing features and scope, teams can find the most valuable software to deliver by a certain date.

Technical practices such as automated testing, continuous integration and refactoring keep a code base healthy and maintainable. They also helps teams go faster.

Release Burndown to illustrate the Innovator’s Dilemma

Consider the chart below. Companies start at burndown line A. If they use Agile, they will stay there. Most companies don’t. So, release by release, they accumulate technical debt and the code base decays.  After a few years, they build a design-dead core.

As a coach, I like to show teams the chart below and vote on their code base. Many companies are at line C with some area’s that are D.

Help! I have a dead core!

OK, so you’ve got a dead core. Sad news. There are ways to recover. I’d suggest you start with Michael Feather’s book Working Effectively with Legacy Code.

Watch the video

The whole video is great, but for the part explaining the Innovator’s dilemma check out:

• Start: 35:38
• Stop: 45:07

This is a list of some resources that are useful for getting started or growing your understanding of Agile.

The permanent page for this content on my website is here (so this is better place to link to since it will be updated).

Getting Started

Short articles for printing out and reading while you are on the train/subway.

• Agile Manifesto – Web page
• Agile Manifesto Principles – Web page
• Article on the history of Iterative developement
• Connections between Lean and Agile
• Short Scrum Guide – 13 page article/guide
• Scrum on a page – This you can put on a wall near you.

Intro to Scrum/Agile

• A Gentle Introduction to Agile – Presentation
• Agile Executive Briefing – Presentation
• Scrum in 10 minutes – Video (10 min)
• A Day in the Life of a Scrum Team – Video (6 min)
• Scrum Overview by Ken Schwaber – Video (60 min)
• Overview Powerpoint by Mike Cohn

Other Stuff you need to know to get your project started

• Getting started with User Stories – Book Excerpt
• Agile Estimation and Planning by Mike Cohn – Video (90 min)

Next Steps

• Check out some of the other resources below.
• Start reading some of the books.
• You have started a journey of learning – be patient and enjoy the trip.

Additional Learning Resources

• Scrum
  • Scrum Overview by Ken Schwaber – Website
  • Scrum Overview by Mike Cohn – Website
• XP
  • eXtreme Programming on C2 Wiki.
• User Stories
  • An Introduction to User Stories – Presentation
• Agile Estimation and Planning
  • Agile Estimation and Planning – Presentation
• Lean
  • Demo showing impact of bottlenecks
  • Lean for Business – Website
• Fun Stuff
  • Punchy video about how Agile can improve on Waterfall
• Agile Team Room
  • Agile Team Room – Photos
  • How does radical collocation help a team succeed?
• Pairing – team collaboration on tasks
  • Benefits of Pair Programming – C2 Wiki
  • Benefits Research Paper – Cockburn & Williams
  • How Pair Programming works - General concepts behind pairing and pair programming] (used in training)
  • Promiscuous Pairing - Some ideas on how rapid pairing can cultivate creativity. Also, How-to Article
• Agile Documentation Practices – Web Article
• Agile Testing – Video (60 min)
• Automated Testing
  • Automated Acceptance Testing – Presentation
• How Agile are you? (Agile Adoption)
  • Nokia Test – basic Agile test
  • Uncle Bob’s more extensive test
• Crystal Clear – low ceremony Agile process
  • 7 Properties of Crystal Clear – Article
  • 7 Properties Mind Map

Books to Read

Stage 1: Getting the basics in place

• Agile and Iterative Development: A Manager’s Guide
• Scrum I: Agile Software Development With Scrum – Basic Instructions
• Scrum II: Agile Project Management With Scrum – Stories about Scrum Usage
• User Stories Applied: For Agile Software Development
• Scrum and XP from the Trenches
• Agile Estimating and Planning

Deepening the practice

• Implementing Lean Software Development: From Concept to Cash
• Agile Software Development: The Cooperative Game
• The Software Project Manager’s Bridge to Agility
• Scrum III: Enterprise Scrum

eXtremeProgramming

• Extreme Programming Explained: Embrace Change
• Extreme Programming Installed
• Art of Agile Development

Technical Practices

• Domain-Driven Design: Tackling Complexity in the Heart of Software
• Refactoring: Improving the Design of Existing Code
• Agile Testing: A Practical Guide for Testers and Agile Teams

Lean

• Lean Software Development: An Agile Toolkit
• The Goal: A Process of Ongoing Improvement
• Critical Chain
• Lean Thinking: Banish Waste and Create Wealth in Your Corporation

Other good ones

• Crystal Clear: A Human-Powered Methodology for Small Teams
• Managing Agile Projects
• Fearless Change: Patterns for Introducing New Ideas

Games & Simulations

• XPGame – learn how Agile really works
• Leadership Game - learn different leadership styles and how you relate to them
• Bottleneck Game – learn how to improve your processes to eliminate bottlenecks
• Business Value Game – learn strategies and challenges with prioritizing work (product backlog)

Below are the slides from my first presentation at AgileTourToronto. It is based on ideas from Alistair Cockburn (among others) and has been a work-in-progress since I started sharing Agile ideas in 2002.

Presentation Overview

There are a lot of choices and alternatives for getting started with Agile. It can be confusing. This talk will give you a brief guided tour of Agile methodologies so that you have some understanding of how they are similar and how they differ. We’ll cover some of the history of iterative development and waterfall as well as the Agile Manifesto to provide context. At the end of this, you will have an understanding of key principles and the Agile landscape.

Slides on Slideshare

A Gentle Introduction To Agile

At Agile 2009, Mary Poppendieck presented on “Deliberate Practice” – how people become experts. The video and slides are available from InfoQ.

Consider the fifth value statement proposed for the Agile Manifesto by Bob Martin:

Craftsmanship over Crap

This presentation follows in the theme craftsmanship – How do we as a community bring it about?

The answer given in this talk is we need to consider what it takes to develop elite level skills in other professions – deliberate practice.  Consider the visual note below:

It seems to me that virtually every company I have every worked for or with has done virtually nothing to bring about excellence in technical (or other) skills.  Imagine what the world would be like if companies viewed their employees as assets and invested in them with mentoring and challenges so that they get deliberate practice.  This requires companies to think about Production Capability and not just Production.  More than just thinking about hitting the deadline.  This is an essential component in build lasting success.

Ever heard of this crazy-sounding approach called eXtreme Programming (XP)? Maybe they were on to something.

Jared Spool gave a pretty good banquet keynote for Agile 2009 on User Interface Engineering.  The main point is that is that very successful companies use amazing interfaces as a key competitive advantage.  And the key to this is good user experience design.

Case and point is Apple.  We watched an Apple video from the 80’s that showed Apple’s vision for the future of the computer (funny/sad was a plot element so long ago was about global climate change).  Everyone at Apple knew what the vision was and could explain it.  So what this means is that everyone is thinking about the long-term vision.  Every time they have a choice to make on design they can take a step in the right direction so that the future can be achieved in part through many many tiny shifts.

Good design is INVISIBLE.  NetFlicks is destroying their competition through a ridiculously high net promoter score (how likely someone is to refer a friend).  No one ever mentions their Web UI even though it rocks – it’s invisble: It does what it needs to and it’s easy.  How do they do it?  In part through a culture of excellence – employees are the top priority and the CEO puts this ahead of board meetings.  To support this they hire fast and fire fast.  They also pass the culture test.

How do you learn good user interface design?  Mentoring!  Good designers know what good design is, but cannot explain it.  Like chicken sexing.  (Makes me wonder how much we are kidding ourselves that good design can simply be taught through patterns or what not.)

The company test is a recipe for successful cultures.  All the companies with awesome user experience met the test even though they were radically different in many ways.

The last point about celebrating failure is critical.  If people are not making mistakes, then they are playing it safe.  And if they are playing it safe, the result is mediocrity not excellence.  There is a similar field of thought around innovation – it also requires support for failure.

If you need help building better UI, start with #2 – spend some time observing your users and then make it better.

Tonight Amr Elssamadisy from GembaSystems presented at the XPToronto/Agile User group.  There was a big turnout of almost 40 people and I think one person didn’t actually have a seat.

What follows are a series of notes from the talk without too much editing…

People’s reasons for wanting to hear about Technical Debt tonight

• People with short-term focus
• Technical Arguements
• Is it worth refactoring
• Currently living in Legacy Land
• Business value of paying down technical debt
• How?  (to fix)
• Best Practices
• Root causes

Examples of Technical Debt

• I have spent 3 years building legacy code web site with no unit tests.
• Code that has been copy and pasted.
• Code that is difficult to get under test: large classes, methods.
• Branching and duplication of the application.
• Duplication of functionality written by different people.

How does technical debt affect behaviour?

New requirements

• Dumbed-down solutions
• Fear of change.  Not sure what will happen.
• Question to ask: How hard is work this year compared to last year?
• Splinter IT group into siloed green team

Design

• What design?
• No new design
• No changed design
• Tons of band-aid solutions
• New features need to be workaround

Defects

• Only critical ones
• Lots and time and effort in regression ? $$$
• Many bugs don’t get fixed – bad code
• Often sketchy manual testing

Maintainability

• Goes down with technical debt
• Dead core after 5 years
• Inventor’s dilemma – productivity goes down over time.
• Difficult to get new people who can work with the code base.

Why do teams fall into this trap?

People follow cost of change curve.  It rises exponentially: requirements to analysis to design to test to production.  Code is fragile for everyone but the author.

Test-first development makes the cost of change linear.  Courage?  Not needed so much since you have a test harness.  Tests act as a signaling system to people working on the system that there is something they need to pay attention to.

“Programming as Theory Building” – this can only be learned through apprenticeship which is not practiced in corporate America.  So what do you use?  Documents don’t work – this results in a new paradigm and the level of mismatch leads to additional cost.  Perhaps pairing is the new apprenticeship model.

Breaking dependencies and adding test to legacy code

Low coupling and high cohesion are main design principles. –> Get teams to agree to “do no harm” to the design

Example of adding code into a function.  This will make cohesion worse.

1. Take the code to be modified and move the code to a new function.  (Extract method refactoring)
2. How hard is it to create the main object?  Probably really hard.
3. So, refactor to a new class (a sprout) instead so you can create it and test it.
4. And may need to create mock objects to make this work due to parameters.
5. Use refactoring to aggregate related sprouts.

Read Michael Feather’s book - Working Effectively with Legacy Code.

Like most things, this is a people problem.  And then we had an interesting digression into Christopher Avery’s material on personal responsibility.