As much as I would like to make software testing out to be glamorous, international and dangerous, more often than not, it is not. In fact sometimes it can be quite dull. We don’t get to wear disguises, play with the latest spy technology or flit around the world in fast cars. Far from it. Only the other day I drove to work in a family hatchback, sat at my desk testing all day, drove home again and got an early night because I had a headache.
However, there is one thing that testing does have in common with the CIA which I discovered the other night whilst researching something for my wife. I stumbled across the following site: http://fnopress.com/SYN/modules/mod9.htm and realized I employ many of the techniques that the CIA do. Digging further I found that Cem Kaner has also drawn the comparison with exploratory testing and the Phoenix checklist. http://www.kaner.com/pdfs/ExploringExploratoryTesting.pdf
When looking at problems and formulating plans the CIA operatives asks themselves a series of questions; these questions are known as the Phoenix checklist (link here). These questions can be applied to any problem, scenario or plan to tease out more information and find out exactly what information is present or missing. The questions are applied to problems and plans by a CIA operative in the same way a tester would apply the questions to a specification, test case, test plan or software planning discussion. It just happens the subject matter is different. It does not even stop at testing; product managers, business analysts, project managers and software designers can also use these questions (and probably do use a variation of them). The questions are not just limited to the CIA or the IT industry – they are often used in the advertising and media industries as well.
So software testing is loosely associated to the CIA, if only in terms of the questioning strategy. I’ve therefore made myself slightly more interesting by association…..
So what type of questions do software testers and the CIA operatives ask? Well here is the list of questions that the Phoenix checklist recommends. http://blog.futurelab.net/2007/01/the_phoenix_checklist.html
For Problems
-
Why is it necessary to solve the problem?
-
What benefits would be received by solving the problem?
-
What is the unknown?
-
What is it you don’t yet understand?
-
What is the information you have?
-
What isn’t the problem?
-
Is the information sufficient? Or is it insufficient? Or redundant? Or contradictory?
-
Where are the boundaries of the problem? Can you separate the various parts of the problem? Can you write them down? What are the relationships of the parts of the problem?
-
What are the constants (things that can’t be changed) of the problem?
-
Have you seen this problem before?
-
Have you seen this problem in a slightly different form?
-
Do you know a related problem?
-
Try to think of a familiar problem having the same or similar unknown?
-
Suppose you find a problem related to yours that has already been solved. Can you use it? Can you use its method?
-
Can you restate your problem? How many different ways can you restate it? More general? More specific?
-
Can the rules be changed?
-
What are the best, worst, and most probable cases you can imagine?
For Plans
-
Can you solve the whole problem? Part of the problem?
-
What would you like the resolution to be? Can you picture it?
-
How much of the unknown can you determine?
-
Can you derive something useful from the information you have?
-
Have you used all the information?
-
Have you taken into account all essential notions in the problem?
-
Can you separate the steps in the problem-solving process? Can you determine the correctness of each step?
-
What creative thinking techniques can you use to generate ideas? How many different techniques?
-
Can you see the result? How many different kinds of results can you see?
-
How many different ways have you tried to solve the problem?
-
Can you intuit the problem? Can you check the result?
-
What should be done? How should it be done?
-
Where should it be done?
-
When should it be done?
-
Who should do it?
-
What do you need to do at this time?
-
Who will be responsible for what?
-
Can you use this problem to solve some other problem?
-
What is the unique set of qualities that makes this problem different from another?
-
What milestones can best mark your progress?
-
How will you know when you are successful?
Looking at the list above I have been asking these questions of specifications, test plans, test cases and the software itself throughout my whole career. Fair enough, I may not have phrased them in the same way, or asked them all at the same time in the same order, but none the less I have questioned everything in the same way.
In summary, questioning things thoroughly at all stages of the Software Development Lifecycle is what software testers bring to the table and formulating those questions around the Phoenix Checklist is not a bad place to start.
http://www.kaner.com/pdfs/ExploringExploratoryTesting.pdf