I’ve fielded a few questions recently about how to manage tests in an agile environment. Most of these questions are about how to store and maintain test cases and also how to manage exploratory tests.
Over the last few projects I’ve been experimenting with ideas, tools and concepts and think I’ve finally settled on one way that suites me. It might not work for you but hopefully it will give you some ideas and may lead you to finding the right way for you.
Firstly let’s start with the tool. I am currently using TestLink (http://testlink.org/wordpress/). It is incredibly simple and at times basic, but that’s what I need. It’s also free. I have the main install on a VM which is backed up and I also produce a MySQL backup daily to a central location, which also gets backed up. I therefore shouldn’t loose any data. I also perform manual backups when I feel I need to. Phew – backup process out of the way.
I access the product through my browser pointing at the website hosted in the VM. The below screen shot shows the TestLink main page. One thing to note is that if you view the site in Chrome when editing a test case you don’t get the Rich Text editor options. Works fine in IE and Firefox though.
TestLink main page
I’m going to start by explaining the terms and process flow used by the tool and how I have altered these or associated these with the way I work.
The tool has the concept of a ‘Specification’. This to me is the main test case repository. It’s the product home. It’s where the standard, generic, base test cases exist. On the following diagram it is referred to as ‘Test Cases and Ideas’. I’ve included the term idea as most of my test cases are questions I’m going to ask the application under test (AUT). At the start they are ideas that I will flesh out to fully fledged questions. These ideas are generally very guidance level and have a few lines about the aim of the question. I try to keep tests high level anyway so that they are cheaper to maintain, but that’s just me, here, on my projects.
TestLink then offers a breakdown within the ‘specification’ in the form of a test plan. A test plan to me in Waterfall used to be the project level test plan. It was a mammoth document that was very often out of date. I’m now working in an agile environment where the concept of a test plan has shrunk and moved to cover just the sprint I’m working on – in my opinion anyway. It could be pushed up a level to a release, but I like to think small and short term when it comes to test case management. The long term plan is outside of my scope essentially. That’s down to the product owner to define what they want in each sprint.
So in TestLink I have a test plan, which directly maps to a sprint. TestLink also then allows you to create different builds within the test plan. I work in a rapid environment where the AUT is dropped to the test stack in minutes through an automated build and deploy process so I don’t make a note of each individual build for testing. So my build stays at the same level as the test plan. I.e The build is a sprint.
TestLink then allows you to assign test cases from the repository against a test plan. You can then assign out to individual testers. You then execute the tests against a build.
So this whole process allows me to add test ideas to a repository, assign them to a sprint (through concept of test plan and build) and then execute them per sprint. I can then view all of the results per sprint, per plan or per specification. Excellent.
But what about exploratory testing. Well, exploratory testing to me has a defined charter otherwise you are wading around in the dark. The charter (in my world) is essentially a test idea, a guide, a tour or a hypothesis. So in essence I can create a holder in the repository for that idea and assign it to a sprint as we go. This therefore allows me to report on it, but more importantly, it also keeps a record of the exploratory notes. And this is done through the execution part of TestLink where I can add notes to the test and then mark it as passed or failed or blocked etc. These notes are then available against each of the tests in the results section for a permanent reference. Job done. If you really get stuck then you can also add attachments so you could keep the session information in a text file and attach it.
The end results is no change at all to my way of working or process, yet a flexible and simple way of managing my tests. I get results out of the back and somewhere to store my exploratory testing where it can sit alongside traditional test cases. Makes my daily testing much more efficient.