The test process Part 1

Now that I've posted a few things about the different kinds of tests, and the differences of functional and non-functional testing I will go a little into the testing process.  Test activities can be divided into 5 phases of the testing process:


1. Planning and control
2. Analysis and design
3. Implementation and Execution
4. Evaluating exit criteria and reporting
5. Test closure activities.




A test plan is defined by ISTQB as 'A document describing the scope, approach, resources, and schedule of intended test activities.'  With a test plan a tester can identify many things including features to be tested, tasks, and who will do the testing tasks, test environments, design techniques entry/exit criteria, choice rationale and risks.  Basically a test plan records the entire test process.  A related task in this phase is Test monitoring where the status of the project is checked periodically through reports.




In Test Analysis and Design testers take the test objectives and turn them into test conditions and test cases. The activities in this stage of testing goes in the following order as stated by ISTQB:
1. Review Test basis (such as the product risk analysis, requirements, architecture, design specifications, and interfaces), and examine specifications of the software certain types of tests such as black box testing can be designed at this point as well. As we study the test basis, we often identify gaps and ambiguities in the specifications, because we are trying to identify precisely what happens at each point in the system, and this also pre- vents defects appearing in the code.
2. Identify the test conditions based on analysis of test items, specifications, and what we know about their behavior and structure. This gives us a high-level list of what we are interested in testing. In testing, we use the test techniques to help us define the test conditions. From this we can start to identify the type of generic test data we might need.
3. Design test cases using techniques to help select representative tests that relate to particular aspects of the software which carry risks or which are of particular interest, based on the test conditions and going into more detail. I'll create a post later about test design.
4. Evaluate testability of the requirements and system. The requirements may be written in a way that allows a tester to design tests; for example, if the performance of the software is important, that should be specified in a testable way. If the requirements just say 'the software needs to respond quickly enough' that is not testable, because 'quick enough' may mean different things to different people. A more testable requirement would be 'the software needs to respond in 5 seconds with 20 people logged on'. The testability of the system depends on aspects such as whether it is possible to set up the system in an environment that matches the operational environment and whether all the ways the system can be configured or used can be understood and tested. For example, if we test a website, it may not be possible to identify and recreate all the configurations of hardware, operating system, browser, connection, firewall and other factors that the website might encounter.
5. Design the test environment set-up and identify any required infrastructure and tools. This includes testing tools and support tools such as spreadsheets, word processors, project planning tools, and non-IT tools and equipment - everything we need to carry out our work.


In the test implementation and execution phase, we take the test conditions and make them into test cases and set up the test environment, then we actually do the tests.  For the test implementation we develop and prioritize our test cases and create test data for those tests. We will also write instructions for carrying out the tests and organize related test cases into test case collections AKA test suites to allow for a more efficient test execution.  Then for the test Execution the testers will actually perform the tests they have created, and compare the results with the expected results.