ISTQB – CTFL Certified Tester Foundation Level – Test Management Part 2

7. Test Estimation [CC]

Test estimation. When we put a plan, we need to estimate the effort needed to execute the plan, the time needed and the number of resources needed. There are many techniques to estimate the different elements for the plan. The Http curriculum mentioned only two techniques matrix based and expert based. Let’s look at each one of them in detail.

The metrics Based Approach to understand this technique, let’s give an example first. For example, if your previous project was 1000 hours long and the testing effort in that project was 300 hours out of the 1000 hours, now that the new project is 2000 hours, can you estimate the testing efforts? Yes, it will be around 600 hours. Now, do you think you can estimate the number of defects that you can expect to find in the new project? Actually, yes. If you have found 500 defects in the 1000 hours project, then you would expect the defects in the new project to be around 1000 bugs. But you know, your development team now has that knowledge of such software domain. Then you would expect the defects to be less than 1000, so say 800 bugs.

However, there is a module in the new software that will use a new technology you have never dealt with before. So you can estimate the number of defects to be around $850. So in this technique, we used collected data and some sort of equations to estimate the project. The way I see it, we used some information about our history, which was how many bugs were found in the 1000 our project. And we used some information about our present, which is the knowledge of the current development team. And we use some information about our future, which is the expectation of the complexity of a specific module or man and good. Other kind of data that can be estimated include the number of distant conditions, the number of district cases written, the number of district cases executed, the time taken to develop test cases, the time taken to run testing cases, and the number of defects found. The accuracy of this technique will depend heavily on the accuracy of the collected data.

The second approach is expert based approach, which depends on using the experience of some stakeholders to drive an estimate. In this context, experts could be business experts, test consultants, developers, analysts and designers or anyone with knowledge about the applications to be tested or the tasks involved within the process. You can use either abroad is to estimate the project individually or mixing and matching between the two techniques. Many factors affect the level of efforts required to fulfill the test requirements of a budget and hence those factors will affect the estimation process. Those factors can be found in three main categories.

One product characteristics, for example, the quality of the test basis, the requirement or specification. If it’s a good quality, then the estimate will be lower. If it’s a bad quality, then the estimate will be higher the size of the product, complexity of the product, the amount of nonfunctional requirements, the security requirements and how much documentation is required. The more documentation required, the higher the effort. The less documentation required, the lower the effort. Second category development process characteristics, the stability of the organization, the tools used, for example manual tools versus automation tools, the test process, the time pressure, the skills of those involved in the testing and development activity. The lower the skill level in development, the more defects could be introduced and the higher the effort. The lower the skill level in testing, the more detailed the test documentation needed to be and more the effort and so on. Last category expected outcome of testing such as the number of defects and amount of rework required. The more number of defects found or the more amount of rework required will increase the estimate. Thank you.

8. Test Progress Monitoring [CC]3

Test Blog Rest monitoring a blend won’t mean anything without monitoring the execution of the blend. Test monitoring can serve various purposes during the project, including give the test team and the test manager feedback on how the testing activity is going, allowing opportunities to guide and improve the testing and the project provides a project team with visibility about the test results, measure the status of the testing, test coverage and test items. Again is the exit criteria to determine whether the test work is done or not, gather data for use in estimating future test efforts and above all, prove that the plan itself is right and following it will eventually lead us to the test objective. Common test metrics include percentage of work done in test case preparation or percentage of bland test cases prepared, percentage of work done in test environment preparation, test case execution for example, number of district cases run or number of test cases not run and test cases passed versus test cases filled defect information for example defect density defects found and fixed failure rate and meets results test coverage of requirement risks or code subjective confidence of the testers in the product, dates of test milestones and finally, test costs, including the cost compared with the benefit of finding the next defect or to run. The next test metrics may be collected manually or automatically by using software tools and may be used to measure exit criteria such as coverage. Metrics may be also be used to assess progress against the blend schedule and budget. I have heard this question many times who will collect this data? And the answer is anyone and everyone. Thank you.

9. Test Summary Reporting [CC]

Test reporting is about communicating our findings from the testing activities to project stakeholders. The purposes of test reporting are notifying budget stakeholders about test results and exit criteria status, helping stakeholders understand and analyze the results of a test period, helping stakeholders to take decisions about how to guide the project forward and assuring that the original test plan will lead us to achieve our testing goals. The report should include analyzed information and metrics required to support recommendations and decisions about future actions, such as an assessment of defects, remaining outstanding risks, the economic benefits of continuing the testing activity, the level of confidence in tested software.

Additional report content might vary depending on the test objectives, test plans and the collective metrics. For example, if we are using a risk based testing, then stakeholders would expect to see the updated list of product and project risks, responses and effectiveness of the responses. If we were using requirement based testing, then we would measure coverage in terms of requirement of functional areas and so on and so forth. The Italy 829 standard includes an outline of the test summary board that could be used for test reporting. The template contains the following sections one test summary report Identifier a specific identifier allocated for this specific document. Two summary identify or relevant support material such as test items, environment and references so that the reader of the report knows which version and release of the project software is being reported.

Variances Document any changes or deviations from those areas agreed on in the reference documents, especially in areas that may cause concern to the group accepting the test results. Four comprehensive assessment measures the actual progress made against the exit criteria and explains why any differences have arisen. Summary Results Report on the overall status of the incident provides an overview of the results of the test activities. It should include details of defects raised and fixed, as well as those that remain unsolved.

Evaluation Provides an evaluation of the quality of each test item, including a view of the risks of failure in production of these test items based on the test result metrics and test item pass failed criteria. Seven summary of Activities A summary of the major test activities and events, such as staff time used, hours per day, elapsed time versus staff time, is staff working excess hours per week or not, and so on. And it changes to the project scope and direction requirements and design changes and lost surprising defect to trends loss of personnel. Finally, approvals identifies all approvals of the document. Gretchen’s exam in this area are mainly asking about what section lies in which document. Thank you.

10. Test Control [CC]

Test control. If you have heard of Murphy’s Law before, then you know that it’s very hard that anything goes as planned. Risks happen, customer changes his mind every now and then, stakeholders interfere, software crashes, stuff quit and so on. When bland do not execute the way you want, then control is needed to get things back to its tracks. Test control is about guiding and corrective actions to try to achieve the best possible outcome of the project. The specific corrective or guiding actions depend on of course what we are trying to control. Consider the following examples consider for example a module or component will not be ready on time for testing. Then test control might involve rebuilderizing the test so that we start testing again as to what’s available now. Also consider that you discovered that most of the executed test cases have failed resulting in too many defect logged again as the software. After investigation you discover that the easy test cases were the ones that run first.

Test control could be to tighten the entry criteria for testing as it seems that the developers do not do proper unit testing. Examples of test control activities are adjusting the scope of testing. Perhaps the amount of tests to run to manage the testing of later change requests. It change the test schedule due to the availability of test environment rebellionutorized tests tighten the inter criteria review of product risks and perhaps it changing the risk ratings to meet the target. Corrective actions taking do not have to be testing related only. For example, you might need to descope the functionality of the software by removing some less important blend deliverables from the initial delivered solution to reduce the time and effort required to achieve the solution. Or maybe you can delay release date altogether until exit criteria have been met. Thank you.

11. Configuration Management [CC]

Configuration Management how many times have you heard questions like what is the correct version of the software module that I have to continue its coding? Or who can provide me with the accurate copy of the last year’s version four, one of the Expert Wave software package? Or what version of the design document matches the version we are adapting to the new customer? Or what version of the software system is installed at ABC Industries? Or maybe you have heard what changes have been introduced in the version installed at the ABC Industries site? Or maybe you have heard, can we be sure that the version installed at ABC Industries doesn’t include undcummented changes? The answers to such questions are within configuration management. The purpose of configuration management is to establish and maintain the integrity of the products of the software or system through the project and product lifecycle. Configuration management is about keeping track of the different versions and iterations of the project artifacts such as documents, components and data. From the testing perspective, configuration management is the management of all the test items. For example, management of the requirement documents design documents, software tools and software plans. And it should ensure the following one all items of testware are identified, version controlled, tagged for changes related to each other and related to the development items such as the test objects, so that traceability can be maintained throughout the test process. In other words, we should know which version of the requirements document maps to which version of the design document maps to which version of the test case or test procedures documents maps to which version of the test object of the software, or as it’s called, which build version of the software. Also, we should ensure that all identified documents and software items are referenced unambiguously in test documentation. The configuration management procedures and infrastructures and tools should be chosen and recommended during tested planning. Thank you.

12. Incident Management [CC]

Hi an incident is any unplanned event occurring that requires further investigation. Some of those incidents are questions suggestions about the software, but most of them will turn into defects. To ensure reliable and fast elimination of failures detected and to ensure handling of incidents, a wellfunctioning boca for communication and administration of those incidents is needed. Instant management ensures that instance are tracked from recognition to correction to closure. It’s important that organizations document their instant management process. Aside from the big words, instant management is the bug reporting tool that you are actually using right now. It’s a sort of database to store and track instance or defects. But instead of calling defect or bug management, we call it instance management. Because such database can contain other types of incidents such as questions and suggestions. Incidents can be raised at any time throughout the software development lifecycle, from reviews of the test basis, like requirements, design and so on, to test specification and test execution. Each instance should be documented as an instant report.

Some companies use Microsoft Excel, others use sophisticated instance management tools or paper. I have seen companies use email system to communicate instance or defects and then how many problems have raised the form. This for example a developer claims that he didn’t receive the email or the manager cannot know how many defects are currently opened or fixed. It’s a total mess. Instant reports have the following objectives to provide developers and other parties with feedback on the problem to enable identification, isolation and correction as necessary to provide test leaders with a means of tracking the quality of the system under test and the progress of the testing. We can measure how many incidents are open, how many incidents bear developer, how many incidents they are tested, how many high priority defects, how many defects per module and so on and last tubulewide ideas for testing bosses improvement. For each instance, the point of injection should be decremented. For example, a defect in requirements of code.

Subsequent bosses improvement can focus on that particular area to stop the same defect from occurring again. To achieve these objectives, the instant reboot has to be again very effective, to the point and very efficient. Not too many details and not too little details. I view this instantry board as some sort of an art. It should help any and all of its readers to understand the report, the developer, the manager, the customer, the new developer who just joined the team, who has no idea how the software actually works in the first place and the tester who has to do regression testing using this instant reboot maybe a few years after releasing the software. All those should understand the report very easily. The report should be as helpful as possible to help the developer reuse the bug easily. Every step should be unambiguous and clear enough for anyone to understand. That’s why it’s highly recommended that the tester try the bug scenario. Few times and on different configurations to make sure it will always be viducible. The details that are normally included on an instant reboot. According to IEEE test standard 829 are date of issue issuing organization which could be your company or the customers or a third party doing the testing the author the name of the ozone to of the report expected and actual. Results identification of the test item in other words, configuration item which item we are testing and which module and which component hardware and software environment software or system lifecycle process in which the incident was observed did we discover the bug in the requirement phase design phase or system testing phase description of the incident to enable reproduction and resolution, including logs, database, DOMS, all screenshots and developers love screenshots. This is about that should be very detailed. We will talk more about that later.

The scope or degree of impact on stakeholders interests severity of the impact on the system how bad is the bug? How urgent do we need the bug? Fix it we could have a high severity bug where the system crashes but it doesn’t affect our testing activity and we have a workaround for it. So in this case the bug would be high severity, but it will also be a low barrierity to fix. Or we could have a very low severity bug, like a taboo for example somewhere. But we need this bug to be fixed as soon as possible because we have a demonstration to the customer tomorrow. So in that case it will be a low severity bug and a high priority at the same time. Status of the incident for example open, deferred, duplicate, waiting to be fixed closed this status actually depends heavily on the tool itself. Conclusions recommendations, approvals global issues such as other areas that may be affected by a change resulting from the incident a change history such as the sequence of actions taken by project team members with respect to this incident to isolate, rebel and confirm it as fixed.

I have seen bugs that have a very long change history that it can be printed over tens of papers, so it’s good to know the history of the bug and last references, including the identity of the test case specification that revealed the problem. Let’s now look at an example of an instant reboot to help us understand it better. Summary Application crash on clicking the save button while creating a new user bug ID for example 1234 this ID is automatically created by the tool module which component that have this problem. In our case, it’s new user’s menu build number version number five by one severity high priority high assigned to developer X reverted by your name reverted on the date of the report status new or open or active? Again, it depends on the tool we are using environment Windows Eight SQL Server 2010 steps to reproduce and this is where the author starts first step log on into the application. We are starting the scenario from the very beginning, from logging on into the application in the first place. Two, navigate to the user’s menu new users.

Three fill all user information field. Look carefully at this step. It says fill all the user information field. It didn’t say, for example, put Jean in the first name, but Smith in the last name. It didn’t give us details. Why? Because the tester here is trying to help the developer not to concentrate on the data itself. This bug has nothing to do with the data. Any data will reproduce the bug. So this is some sort of a message for the developer. Don’t concentrate about the data, just fill in any valid information and the bug will be reproducible. Four click on the save button. Five, seen an error page or a 1090 exception in set to values error, whatever. Six sees the attached logs for more information. We can attach more logs related to the bugs if any. Seven also sees the attached screenshot for the arrow report. And again, developers love screenshots. Expected result on clicking save button should be prompted to US success. Message new user has been created. Successfully attach application crash screenshot. So I hope you get an idea how to create a bug report or an instance.

• Category: Uncategorized