Black box and White box testing are mentioned a lot in Chapter 4 of the ISTBQ book. The text book definition of black box in the ISTQB book states that it is "testing either functional or non-functional, without reference to the internal structure of the component or system". Basically Black box testing is when a tester knows what the software does from having access to things like requirements, but has no knowledge as to how the software does these things, or anything about the branching logic. Black box testing tends to involve a lot of exploratory testing as the tester goes through the system to understand how the system works. Techniques of black box testing include but are not limited to equivalence partitioning, and boundary value analysis, the use of data tables, and state transition testing
Meanwhile White box testing is defined as "Testing based on an analysis of the internal structure of the component or system". In other words this type of testing is when the tester knows all the details about not only what the software supposed to do, but how it works and does what it's supposed to do. These kinds of tests allow tester to increase their test coverage by finding new tests that haven't been created. It uses techniques like statement coverage, decision coverage, and branch coverage.
Black box testing is good when the tester is starting to run tests on a software they know little about, while white box testing is good for going more in depth into understanding the system.
Thursday, October 2, 2014
Wednesday, September 24, 2014
Static Vs. Dynamic Testing
In the ISTQB book one of the first things mentioned in Chapter 3 is dynamic testing and static testing.
Dynamic testing involves executing a software with a set of given input values and then checking the outputs values and comparing them to the expected results. Static testing involves using either a set of tools or manually examining software work products such as user guides, test cases, and scenarios but not actually executing them, reviews are actually an example of static testing at work. Basically dynamic testing actually tests the software code while static testing looks through the documents.
Static and dynamic testing techniques are complimentary due to the fact that dynamic testing works on areas static testing does not and visa versa. When they are used together they can greatly improve the quality of testing and help find a lot more defects. Static testing methods are good for finding deviations from standards, missing requirements, design defects, non-maintainable code and inconsistent interface specifications. Dynamic testing is good for finding defects in the code itself, as well as the quality of the software. Static testing is good in the early stages of testing and in reviews while dynamic testing is good for the main executions.
Since a lot of the info in the ISTQB book is about the dynamic aspects of testing I'll go more in depth with static testing According to ISTQB the use of static testing, e.g. reviews, on software work products has various advantages:
Since static testing can start early in the life cycle, early feedback on quality issues can be established, e.g. an early validation of user requirements and not just late in the life cycle during acceptance testing.
By detecting defects at an early stage, rework costs are most often relatively low and thus a relatively cheap improvement of the quality of software products can be achieved.
Since rework effort is substantially reduced, development productivity figures are likely to increase.
The evaluation by a team has the additional advantage that there is an exchange of information between the participants.
Static tests contribute to an increased awareness of quality issues.
Dynamic testing involves executing a software with a set of given input values and then checking the outputs values and comparing them to the expected results. Static testing involves using either a set of tools or manually examining software work products such as user guides, test cases, and scenarios but not actually executing them, reviews are actually an example of static testing at work. Basically dynamic testing actually tests the software code while static testing looks through the documents.
Static and dynamic testing techniques are complimentary due to the fact that dynamic testing works on areas static testing does not and visa versa. When they are used together they can greatly improve the quality of testing and help find a lot more defects. Static testing methods are good for finding deviations from standards, missing requirements, design defects, non-maintainable code and inconsistent interface specifications. Dynamic testing is good for finding defects in the code itself, as well as the quality of the software. Static testing is good in the early stages of testing and in reviews while dynamic testing is good for the main executions.
Since a lot of the info in the ISTQB book is about the dynamic aspects of testing I'll go more in depth with static testing According to ISTQB the use of static testing, e.g. reviews, on software work products has various advantages:
Since static testing can start early in the life cycle, early feedback on quality issues can be established, e.g. an early validation of user requirements and not just late in the life cycle during acceptance testing.
By detecting defects at an early stage, rework costs are most often relatively low and thus a relatively cheap improvement of the quality of software products can be achieved.
Since rework effort is substantially reduced, development productivity figures are likely to increase.
The evaluation by a team has the additional advantage that there is an exchange of information between the participants.
Static tests contribute to an increased awareness of quality issues.
Thursday, September 18, 2014
Formal Review Process
Since Formal reviews seem to be more elaborate and complex than informal reviews I have decided to go more in depth. While informal reviews are more common due to less structure and planning Formal reviews happen less often, usually every few weeks as opposed to informal review that can happen every few days, and have a lot more structure and planning involved.
Typical formal reviews have 6 steps:
1. Planning
This is the beginning of the review process and starts when the author of the software requests a review from the moderator who's job is to lead the inspection. The moderator then has to schedule the start and end dates of the review, and work with the author to come up with test documents including entry criteria. The document can't be too long as most people can't retain a large number of pages of info. during this phase the people who will be involved in the review will be decided on, and from these chosen reviewers the moderator hands out duties and tasks to them.
2. Kick-off
This step is stated by ISTQB to be optional but is highly recommended as it gives the review team members a degree of understanding about the document. During this phase reviewers are given a short intro on the objective of the review and document. The relationship between the review document and other documents used as sources are explained as well.
3. Preparation
This part of the review is where the individual reviewers gather up all of their data and materials they have compiled in testing to work on the document. In this phase the reviewers identify defects, questions and comments and all the issues are recorded. Using checklists in this phase is a way to make the testing more efficient. The efficiency can also be checked by number of pages checked in an hour which varies greatly depending on the number of pages, and the complexity of the software.
4. Review Meeting
This is the actual review, where the review team gets together to talk about the defects that they have found as well as ask questions and collaborate with one another. Bugs talked about in the review is recorded by scribe. The severity of every defect is also recorded by the scribe and there are 3 levels of severity Critical, Major, and Minor
5. Rework
This is the phase where the reviewers try to retest and improve the documents step by step. It's up to the author to decided if the defects need to be fixed. Then the author updates the software and the document.
6. Follow up
This is done by the moderator and is when they determine if the changes done by the author to the software and document are adequate, and the bugs have been removed from the affected areas.
The tasks of a formal review are divided up to include the author 9program writer), a moderator (inspection leader), a scribe (recorder), the reviewers (testers), and a manager (the boss).
Typical formal reviews have 6 steps:
1. Planning
This is the beginning of the review process and starts when the author of the software requests a review from the moderator who's job is to lead the inspection. The moderator then has to schedule the start and end dates of the review, and work with the author to come up with test documents including entry criteria. The document can't be too long as most people can't retain a large number of pages of info. during this phase the people who will be involved in the review will be decided on, and from these chosen reviewers the moderator hands out duties and tasks to them.
2. Kick-off
This step is stated by ISTQB to be optional but is highly recommended as it gives the review team members a degree of understanding about the document. During this phase reviewers are given a short intro on the objective of the review and document. The relationship between the review document and other documents used as sources are explained as well.
3. Preparation
This part of the review is where the individual reviewers gather up all of their data and materials they have compiled in testing to work on the document. In this phase the reviewers identify defects, questions and comments and all the issues are recorded. Using checklists in this phase is a way to make the testing more efficient. The efficiency can also be checked by number of pages checked in an hour which varies greatly depending on the number of pages, and the complexity of the software.
4. Review Meeting
This is the actual review, where the review team gets together to talk about the defects that they have found as well as ask questions and collaborate with one another. Bugs talked about in the review is recorded by scribe. The severity of every defect is also recorded by the scribe and there are 3 levels of severity Critical, Major, and Minor
5. Rework
This is the phase where the reviewers try to retest and improve the documents step by step. It's up to the author to decided if the defects need to be fixed. Then the author updates the software and the document.
6. Follow up
This is done by the moderator and is when they determine if the changes done by the author to the software and document are adequate, and the bugs have been removed from the affected areas.
The tasks of a formal review are divided up to include the author 9program writer), a moderator (inspection leader), a scribe (recorder), the reviewers (testers), and a manager (the boss).
Tuesday, September 9, 2014
Formal Vs Informal Review
Section 3.2 of the ISTQB book is about the test review process. There are 2 types of review, informal review, and formal review. While each one has it's differences they both share the goal of finding defects, and to help fellow testers gain a better understanding of the software, and test requirements, and cases.
Informal reviews are the more simple and commonly used of the 2 types. They can be done at anytime during the early stages of testing, it can only take 2 people to do an informal review, and they can be conducted with little planning, they don't required to be led by a trained moderator. There are many informal testing types but one thing they all share in common is that they are undocumented meaning that no one needs to record a list of defects, and there is no compilation of test cases and defects given out to the review members.
Formal reviews on the other hand are a lot more complex. They are usually done in the later stages of testing and because of that they require more people. They have six main steps:
1. planning
2. kick-off
3. preparation
4. review meeting
5. rework
6. follow up
The formal meetings also have a number of roles that are allocated in the planning phase. The roles are the Moderators, Scribes, Authors, Reviewers, and Managers. Like informal reviews there are different types of formal testing such as technical reviews, and walkthroughs, and not every role is present in every review type, for instance, walkthroughs have the authors explaining the software and leading the review, while technical reviews have trained moderators leading the review instead and the authors are usually absent. Scribes and reviewers however are present in most if not all formal review types.
Informal reviews are the more simple and commonly used of the 2 types. They can be done at anytime during the early stages of testing, it can only take 2 people to do an informal review, and they can be conducted with little planning, they don't required to be led by a trained moderator. There are many informal testing types but one thing they all share in common is that they are undocumented meaning that no one needs to record a list of defects, and there is no compilation of test cases and defects given out to the review members.
Formal reviews on the other hand are a lot more complex. They are usually done in the later stages of testing and because of that they require more people. They have six main steps:
1. planning
2. kick-off
3. preparation
4. review meeting
5. rework
6. follow up
The formal meetings also have a number of roles that are allocated in the planning phase. The roles are the Moderators, Scribes, Authors, Reviewers, and Managers. Like informal reviews there are different types of formal testing such as technical reviews, and walkthroughs, and not every role is present in every review type, for instance, walkthroughs have the authors explaining the software and leading the review, while technical reviews have trained moderators leading the review instead and the authors are usually absent. Scribes and reviewers however are present in most if not all formal review types.
Wednesday, September 3, 2014
The Ed Kit Principle
| When I took a practice test for ISTQB certification I came across the question asking 'The quality and effectiveness of software testing are primarily determined by the quality of the test processes used is stated in' a) Bill Hetzel Principle b) Ed Kit principle c) IEEE 829 d) IEEE 8295 The correct answer was the Ed Kit Principle, and since I didn't know what that was I decided to do some research. Basically the Ed Kit principle states that "The quality and effectiveness of software testing are primarily determined by the quality of the test processes used.". So in other words the way that a tester's testing methods are determines how good the tests they run are. Like the Bill Hetzel Principle the Ed Kit Principle isn't really mentioned by name in the ISTQB software testing book despite it appearing once in a while on online practice tests. However that doesn't mean it isn't present in the book, and in fact has a significant presence. If you have a copy of the book the principle is referenced a lot in at least in section 1.4 Fundamental Test Process. Just remember if you're looking for the principle by name you will not be able to find it, just remember that the principle is about how quality of the test processes determine the quality of the tests. |
Wednesday, August 27, 2014
The Bill Hetzel Principle
After taking a practice test for ISTQB certification I came across a question asking where 'testing must be planned' is stated and the Bill Hetzel Principle was the answer. I wasn't sure what it was, and since this was a chapter 1 quiz I tried to find where it was mentioned in ISTQB book with no success, so I decided to do some research. So I found that Bill Hetzel wrote a book on software testing called 'The Complete Guide to Software Testing' in 1988, and it is still considered a good one from what I can tell.
While I haven't been able to get access to it I found that some of his main testing principles are:
Testing must be planned can be found in section 1.2 'What is Testing'
Testing requires independence can be found as one of the code of ethics, and can be found (more or less) along with the remaining principles in section 1.5 The Psychology of Testing.
The principles of 'Testing is creative and difficult', 'An important reason for testing is preventing errors', and 'testing is risk-based' may not necessarily be found word for word in the book, but some of the stuff the book talks about in CH 1 is mostly the same basic principle.
While I haven't been able to get access to it I found that some of his main testing principles are:
- Complete testing is not possible
- Testing is creative and difficult
- An important reason for testing is to prevent defects
- Testing is risk based
- Testing must be planned
- Testing requires independence
Testing must be planned can be found in section 1.2 'What is Testing'
Testing requires independence can be found as one of the code of ethics, and can be found (more or less) along with the remaining principles in section 1.5 The Psychology of Testing.
The principles of 'Testing is creative and difficult', 'An important reason for testing is preventing errors', and 'testing is risk-based' may not necessarily be found word for word in the book, but some of the stuff the book talks about in CH 1 is mostly the same basic principle.
Wednesday, August 20, 2014
The Test Process Part 2
leaving off from last post I wrote that Test activities can be divided into 5 phases of the testing process:
1. Planning and control
2. Analysis and design
3. Implementation and Execution
4. Evaluating exit criteria and reporting
5. Test closure activities.
I covered the first 3 so now I'll finish the list.
For evaluating exit criteria this is the point in testing when the tester takes what was done in test execution and assesses it against the defined objectives. ISTQB suggests that this should be done for each test level, so we can know when we have done enough testing and can move on to the next level. Also in order to assess the risks of deeming a component, activity or level is complete and can move on we need to come up with a series of exit criteria. Exit criteria should be set and evaluated for each test level. According to ISTQB evaluating exit criteria has the following major tasks:
• Check test logs against the exit criteria specified in test planning
• Assess if more tests are needed or if the exit criteria specified should be changed
• Write a test summary report for stakeholders
The last on the list of test activities is the test closure activities.
During test closure activities, the testers gather up the data from finished test activities from each of the testers involved, this includes checking and filing test ware, and analyzing facts and numbers. We may need to do this when software is delivered. There are a number of reasons why a testing team would close testing, these include the testers getting all the info they needed from testing, the project could be cancelled at anytime, a particular milestone is achieved where we no longer need to continue testing after, or when a maintenance release or update is done. Test closure activities include the following major tasks:
• Check which planned deliverables we actually delivered and ensure all incident reports have been resolved through defect repair or deferral.
• Finalize and archive test ware, such as scripts, the test environment, and any other test infrastructure, for later reuse. This can save time later if the testers have to test a new version of the same software later.
• Hand over testware to the maintenance organization who will support the software and make any bug fixes or maintenance changes, for use in con firmation testing and regression testing. This group may be a separate group to the people who build and test the software; the maintenance testers are one of the customers of the development testers; they will use the library of tests.
• Evaluate how the testing went and analyze lessons learned for future releases and projects. This might include process improvements for the soft ware development life cycle as a whole and also improvement of the test processes.
1. Planning and control
2. Analysis and design
3. Implementation and Execution
4. Evaluating exit criteria and reporting
5. Test closure activities.
I covered the first 3 so now I'll finish the list.
For evaluating exit criteria this is the point in testing when the tester takes what was done in test execution and assesses it against the defined objectives. ISTQB suggests that this should be done for each test level, so we can know when we have done enough testing and can move on to the next level. Also in order to assess the risks of deeming a component, activity or level is complete and can move on we need to come up with a series of exit criteria. Exit criteria should be set and evaluated for each test level. According to ISTQB evaluating exit criteria has the following major tasks:
• Check test logs against the exit criteria specified in test planning
• Assess if more tests are needed or if the exit criteria specified should be changed
• Write a test summary report for stakeholders
The last on the list of test activities is the test closure activities.
During test closure activities, the testers gather up the data from finished test activities from each of the testers involved, this includes checking and filing test ware, and analyzing facts and numbers. We may need to do this when software is delivered. There are a number of reasons why a testing team would close testing, these include the testers getting all the info they needed from testing, the project could be cancelled at anytime, a particular milestone is achieved where we no longer need to continue testing after, or when a maintenance release or update is done. Test closure activities include the following major tasks:
• Check which planned deliverables we actually delivered and ensure all incident reports have been resolved through defect repair or deferral.
• Finalize and archive test ware, such as scripts, the test environment, and any other test infrastructure, for later reuse. This can save time later if the testers have to test a new version of the same software later.
• Hand over testware to the maintenance organization who will support the software and make any bug fixes or maintenance changes, for use in con firmation testing and regression testing. This group may be a separate group to the people who build and test the software; the maintenance testers are one of the customers of the development testers; they will use the library of tests.
• Evaluate how the testing went and analyze lessons learned for future releases and projects. This might include process improvements for the soft ware development life cycle as a whole and also improvement of the test processes.
Subscribe to:
Posts (Atom)