February 2009 Entries
Tester involvement and burn down charts

Having recently made the burn down chart burn down for a change the team were actually motivated by putting their time into our agile tool, version one, (funny, "individuals and interactions over process and tools")...  But, I then got thinking, what does it actually mean, what is the burn down measuring?  It measures how much effort has been expended against how much effort is left for a given period of time (based on the time estimates).  The thing is, the burn down has no concept of your definition of done.  Developer adds time against tasks and that makes the burn down burn down.  They do all the tasks, burn down reaches 0 job done, project manager gets all excited and arranges team drinks.  The fact that the effort was expended doing something wrong and that effort still remains to fix it is not considered on the chart...

The big bit of value which iterative incremental (agile if you like) methods bring to the table is the frequent testing of working software.  The value comes from the shortening of the validation loops (see diagram below).  The quicker you find out you've done something wrong the quicker and more cheaply it can be addressed (remember it's not just external change which costs money).

image 

On most projects it is not possible to actually deliver to the stakeholder at the end of each iteration (normally only works when you work internally for a customer or have some kind of framework agreement in place - another topic of discussion!).  This is where the word increment comes, you iteratively develop software increments, at some point the customer will take an increment of that software and start using it.  So, why is all this important to a burn down?

Progress on a project is a measure of how many features are done (definition of done done).  The only true test of a complete feature is when the stakeholder accepts it, hence the acceptance criteria of a user story (whether they managed to get their thoughts down properly is another story, but again, another topic of discussion!).  That's where testers come in, they make sure the developers have met the acceptance criteria.  So, if a burn down is to be of any value it needs to show a burn down based on time to set a feature to done.  This also means that the time spent on the project by the test team is also considered in the burn down, as well as the time spent by developers on defect resolution.  Interestingly though, the customer might see it and say "that's not what I meant" (acceptance criteria was wrong..?) and then you start again...

However, is it possible for the test resource to match the pace of the developers?  When do the testers start testing?  There's obviously a lag between the feature being code complete and a test resource being able to test it.  This depends on the availability and number of testers available to the project.  This also depends on the level of automated testing and time taken to write automated tests.  Also, as the product grows does it take longer to test each time?  There will be a level of regression testing for each drop...  That's where automation can add a lot of value (see Selenium, TFS and Agile)!

So what does the burn down mean?  Nothing unless it considers the above.  All it shows is how much time has been spent doing stuff, of which that stuff might yield no value to the project (back to being on a waterfall process again).  We could go into earned value management now, but I think that would be a little to much process for an agile process.

Really though, the only measure of progress is customer satisfaction and tools can't tell you that!

Agile fundamentalism

I have been busy writing software of late, but have been thinking about what I could blog about next.  The blog idea that was going round my head is how fundamentalist some people are about software development.  I'm sure many people would have heard me jest about using the Nike methodology "Just do it" (I expect I'm in breach of some kind of branding laws?).  Anyway, I don't need to spend the time writing a well researched, thought out and intellectual article (some people would argue that's beyond my capability, but that's another argument :)).  This is great, and so are the comments, go read:  "The Decline and Fall of Agilists".  And thanks, Clive for making me aware of it!

 

Now, Tom is getting on at me about finishing some work, better do that!