I’m looking forward to the new Visual Studio 2010 (with Camario). The Lab management means that testers can spin up a new test stack (using Hyper-V or ESX) with any prerequisites already install, take a build (deployed using workflow) and start testing. They can raise defects with a full environment details attached and even a “snapshot” of the VM. They will also be able to add automated steps to their test cases using recorded UIA.
We kind of do this already... We have templates to create environments but for speed and to save space we snapshot the VM’s. Each night (or on demand from a TFS build) a msbuild script calls a VMTool that rolls back to a snapshot (that contains the prereqs). The msbuild script is also currently used to deploy the latest build to the “clean” VM box. A tester can then test the build on the VM, and raise defect in TFS. We can also run automation (currently only a web site) using Selenium RC by reading the scripts and running them as standard mstests, just line unit tests in a CI build. Other tests are standard c# tests but we hope to move this to a framework soon.
The one thing I look forward to is a complete management system that we can write a mix of manual and automated tests in one place. It would also be nice to attach the delta of the VM to the defect so the developer can see what happen when the bug was found. The bit I’m not too sure about is when the developer uses the VM diff attached to snapshot it could rollback the VM I’m currently using. I guess this can be solved by good communication. I might also start video recording tests to provide that the bug did exist and attached the compress video/link with a time of crash to the defect.
Nice to see that we are kinda on the same wave length as MS when it comes to VM testing.
http://www.4fem.info/?p=14 & http://seleniumhq.org/
http://www.microsoft.com/visualstudio/en-us/products/2010/default.mspx