Tom Peplow's Blog

In the previous post I discussed waste and how to reduce with TDD and refactoring. In this post I’m going to focus on how you defer decisions and remove dependencies.

Wrong decisions that have to be undone or worked around are a massive cause of waste. Making the right decision can only happen when you have enough information available – thus you make an informed choice. The old fashioned (well not that old!) way of solving this problem is write a spec, speak to people and validate your approach with them – essentially gathering information which will help you make the right informed choice. There’s another way, experimentation – come up with a hypothesis of how you think something should work and test it (experiment) – validate your hypothesis. This is essentially “working software over comprehensive documentation” from the agile manifesto. What often stops this frequent demonstration of software is dependency on things that are needed but aren’t yet ready. Commonly you’ll have concurrent teams working on features that are required for each other to work. Sometimes hundreds of lines of code are written before you validate the need with a demo (or better through use) – thus those hundreds of lines of code could be wasted. However, these dependencies are artificial – they don’t need to be there.

So, how do you defer decisions and remove dependencies? Interestingly, the two can be achieved at the same time. You may want to demonstrate some functionality to a user but the underlying capabilities are not yet delivered (and will not be for a considerable amount of time). But you do have a high level understanding of what that capability will look like – its interface. That understanding will evolve as you build the solution, so making it easy to grow and change is crucial (refactoring should be easy). A premise behind service–orientation is that what an organisation does remains pretty stable, what changes is how they do it. Knowing what something needs to do and defining the input and output. You can then create a mock of the implementation (i.e. mock out the how). Thus, you’ve removed the dependency and deferred the decision about how to do it until you need to solve it. This links in with the I ‘interface separation’ and D ‘dependency inversion’ from SOLID. From a high level architectural view it is the beginnings of the development of service interfaces and message exchange patterns... The plus to this is separate people can work without dependency on each other but together on the development of an entire system. Inverting your dependency on something also means that if you do make a wrong choice, or something better comes along, it can be easily changed.

There are times where you can’t defer a decision until later.  For example, having a consistent architecture is important, Brooks talks about it in terms of conceptual integrity and the advantages it gives in terms of maintainability. There are some decisions about the approach you’re going to take that you must make early on in the process... You can do your best to make sure you’ve got dependency inversion between your components and layers so that if you make a wrong decision you can swap it out. But changing your mind on large, often crucial parts of the system, costs time and money... So even if you could swap out something you might not want too. The key thing to remember is make a decision when you have enough information to make it. So, in this case, candidate architectures or proof of concepts are a great way to help create that information. They are great at breaking assumptions – and have proved invaluable when building solutions in new technologies or new business domains. Those candidate architectures can be thrown away, but they aren’t waste because they generated information valuable in making the right choice, which if made wrong would create a huge amount of waste.

In my previous post I introduced a few lean concepts and set out to tell a tale how those concepts can be applied to the day to day task of cutting code... I picked 3 areas, reducing waste, removing artificial dependencies and building quality into the process.

Let’s start with waste. Waste in the software process has many disguises... In lean manufacturing a large stock inventory is waste. Tom and Mary highlighted that unused features are essentially an inventory; an unused feature is a waste. Another, and probably a much more obvious waste to non-lean thinkers is a bug. Think about this, bugs in unused features are utter waste!

What are unused features though? The clear unused feature to a user is a screen they never use. But to a developer, it’s over engineered code, generic frameworks that have a single application, developing something that already exists (the not made here syndrome)... Agile methods work hard to reduce the statistics of unused features by not having big up front specs, by constant demonstration of software and by re-prioritisation of work backlogs. But developers must also work hard not to create code waste. YAGNI – you aren’t going to need it. It’s a simple statistic, the more code the more bugs. No code, no bugs. We can’t yet write no code to implement solutions (unless you’re a SharePoint guru :) ) so we must aim to write as little as possible. I said before I wouldn’t focus on process, but it is absolutely crucial that you develop the least amount of features for a system that will leave your customer delighted. With that digression out of the way I’ll assume you’re now working on a feature that will be used 90% of the time – how do you make sure you don’t introduce waste in code?

Well here’s a paradox – the answer is to write more code, specifically – write test code. Why isn’t test code waste, the user doesn’t use it the business doesn’t get value from it – its waste? That argument is valid sometimes, sometimes there just isn’t time, sometimes you literally do have a day or two to get something out the door, then test code would be waste. One thing to remember in everything you do is that being pragmatic is the most important thing... The pragmatic guy needing to meet a new regulatory requirement (ignored by the business for months) will get something out the door. The same pragmatic guy also will write tests, he’ll make sure when he’s not commercially constrained, he builds time into his project to build tests.

I can remember having a discussion once with a finance director about project timescales, and being open and honest with him I explained the need to do certain things. He doesn’t care about the software development process or the tasks I have to do to give him software that’s going to make him hundreds of thousands of pounds a month. All he cares about is the fact that if he had it now, he’d be making that money and every day he doesn’t have it he’s losing that money. But, he also understood that me not doing a job properly could result in his call centre not being able to operate and not making any money – so reluctantly, he let me do it properly (pragmatically) (he still squeezed me, but we dropped features not quality).

I think it’s clear enough that if you write tests it will take longer to deliver the solution (there’s simply more to do). In fact, Microsoft have done some empirical research and found it does take longer (have a read here http://research.microsoft.com/en-us/news/features/nagappan-100609.aspx). But they also found that teams using TDD (Test Driven Development) had a lot fewer live defects. TDD measurably increased in process quality. Again, there are two sides to the argument – if a commercial pressure dictates that you simply must get something done you may be willing to have a few live defects (I’ll talk more about quality in a later post...).

A commonly used approach with TDD is “red green refactor”. Write a test, run it; it will fail (red). Make the test pass (green). Then refactor. The idea behind it is you write the least amount of code possible to get the test passing, thus ingrained into the approach is the concept of writing less code.

The refactor step is crucial. Without it, you would repeat yourself... Repeating yourself results in more code (thus more bugs). So, refactoring is also a critical part in writing less code. The key idea with the refactor approach is that you only introduce something when you know you need it. You don’t start off from a point thinking you’re going to need it and write more code than you actually need. The likelihood is you won’t need it, so it’s waste.

TDD should also mean you write fewer tests too. Bashing out a load of code (with loads of things you’ve added in that you don’t need) means that when you retrospectively add tests you need to test more stuff, thus write more tests.

TDD also meets another core lean principle; defer the decision about something until have the information. You only write production code when you know you need it; you only refactor code to use a design pattern when you know you need it... You don’t start off thinking you need it, you wait until you know...

Another aspect of writing less code is using more code that has already been written and tested. The downside of using solutions that already exist is that they will probably do 80% of what you need and have features which you only use 10% of. Now, this is a problem if you have to pay for the software – sinking a lot of money into a something you only use 10% of is wasteful. If that 10% is very hard to do yourself then you’ll probably still spend the money (the best example of this is a database, pretty much all projects need a database, you’d never write your own but you probably only use a small proportion of its features from one project to another...). This is what’s great about Open Source Software (OSS). It’s free. So, there is no money spent thus no incurred waste. There may still be code in that OOS solution that you don’t use, but you didn’t write it, hopefully someone else is using it, so it’s not waste. If you need something, you can contribute to the OSS project and add it in. Or if it’s very specific, you can extend it yourself. Extending existing solutions is also possible with existing paid for solutions, as long as those solutions are extensible. Extending something is again less code from building it all from scratch. Picking extensible components to build a solution on top of is a great way to reduce the amount of code you have to write and test.

I’ve focused on tests teasing out production code, I’ve not touched on the impact tests have on quality – for that, read the last post in this series...

Having spent the day with the Poppendiecks last month I came out enthused again about software development. When working at the coal face it’s great to have an opportunity to sit back and look at the bigger picture. Tom and Mary are fantastically passionate speakers that meeting them is a really great opportunity and I highly recommend if they are speaking anywhere near you, you make a point of going to listen. If not, you can read their books, which are full of interesting and pertinent real world anecdotes (just like their talks).

Picture to the right is Mike on a night out, sporting his own Kanban board…

I’m going to try not to regurgitate their content but rather add some thoughts about how a lean approach is applicable, every day, to software development... I’m going to focus on how the underpinning concepts of lean actually correlate quite tightly with good software design and architecture. I don’t think it’s anything new, but I think it’s interesting anyway...

As a developer while listening to Tom and Mary the three things that struck home were... Firstly, that the most important concepts of lean is kaizen, or improvement. The reason for improvement is to eliminate waste. Specifically waste is things that don’t add any value. Secondly, in lean continuous improvement is supported by a fundamental belief that quality should be built into the process. To address ‘in process quality’ lean manufacturing advocates a stop the line mentality – when a problem occurs, stop and fix it. Thirdly, another important concept is the need to remove artificial dependencies (as process dependencies can create waste). Lean recognises that another reason for waste is making the wrong decisions; so lean advocates deferring a decision until you have as much information as possible (thus reducing the likelihood of making a wrong decision).

I did say I wouldn’t regurgitate material, but this sets up some context for what I’m going to talk about – it’s very high level so to find out more have a read of Lean Software Development. Also, there is much more to lean, I’ve just picked out some key ideas...

There are glaringly good reasons why lean concepts align so well with development process – and if you don’t specifically do lean you’re probably doing a lot of what being lean requires. For example, test driven development, continuous integration, frequent drops of demonstrable software etc. Just a question though, how many of you, when the build breaks, drop tools and fix the build before going on?

What I am going to focus in on is the importance good coding practices can have on being lean. There are quite a lot of opposing views on this out there, about whether SOLID is good or not, whether you should do TDD etc. I don’t want to get drawn into that argument here, rather just show how some concepts really line up with being lean.

This started out as one big post but I’ve split it up into smaller posts, the next post will look at ways of reducing waste, followed by reducing dependency hierarchies and finally bringing it all together with building quality into the process.