Research Article

Team Exploratory Testing Sessions

Table 2

The approach’s risks.

Risk and consequencesMitigation plan

If a clear TET session goal is missing, results will be minimal, time will be lost, and participants and management will feel the method unsatisfactory. On the other hand, too many or contradictory session goals can deteriorate the results.Each session has to have a clear, context fitting goal. If there are several goals, they have to be in line with each other, and there should not be too many of them. When planning a new session, the facilitator can consult the TET domain expert and management to select best possible goal(s).
If participants lack testing experience, the target software is in poor condition, and the session goal is to find defects, defects are not found or reports might be too incomplete, and results will be lost.The target software condition and participant testing skills have to be in balance and taken into account during participant selection.
If participants lack an attitude, only obvious defects will be found, and rest of the time will be spent getting bored. If teamwork skills are bad, both information sharing and atmosphere will suffer.Participant selection has to be done carefully. The facilitator can talk with participant candidates to determine their motivation and attitude.
With a team that is too homogeneous, knowledge sharing suffers. On the other hand, if a team is too heterogeneous, understanding and coordinating with one another can be challenging.The aspiration in team composing is to strike the right balance between homogeneity and heterogeneity: members should have a variety of talents and perspectives, yet are similar enough to be able to work with each other [27]. To keep the TET session team in balance, participants should have different backgrounds and knowledge bases. This has to be taken into account during participant selection.
Having too many participants causes confusion, and having too few reduces the benefits of the session.It is good to keep the number of participants between five and ten to ensure results and avoid fuss. For declining participants, there should be back-up candidates.
If the target software version or configuration is not controlled, the results are not reliable and might be caused, for example, by improper settings. With old versions, time is wasted on already fixed defects.All participants have to have the same, preferably the latest software versions and configurations in use. Participants should not be asked to bring test devices but must prepare to use them.
If the target software is huge and no specific focus area is given, testing efficiency might suffer. On the other hand, if the focus area is already well tested, using it might cause frustration and poor results.The sessions should have well-chosen focus areas. Analyzing TCT asset and defect metrics can help in selecting the focus area. The facilitator can consult the TET domain expert and management to select best possible focus area(s).
If the session duration is too long, it decreases the number of participants. Neither it can be too short, which would prevent sufficient test cycles and reacting to findings.A one-hour duration is the minimum session duration. The duration has to be adjusted in accordance with the session feedback and realized session duration. During this research, two hours was found to be sufficient.
If the frequency of sessions is too high, there is a risk of discovering already detected defects.Leaving at least two development iterations between sessions ensures that there is enough time for earlier findings to be fixed and new features to be made ready for testing.
If the session is organized during the busiest days of the development cycle, it decreases the number of participants and weakens their motivation.The target software’s development cycle has to be taken into account when organizing sessions. At the beginning or in the middle of the iteration, there is more space for additional exercises.
Time is wasted if testing tools are improper or test accounts or data is missing. Inappropriate test accounts or data can cause security problems.Tools have to be checked to ensure that they are working before sessions take place. Personal accounts or data are not permitted to be used in TET sessions. Specific test accounts and data must be prepared and used.
Without proper reporting means, the results of the session may vanish. If tools are laborious to use, using them might isolate from the situation.Reporting session findings must be made easy, for example, by using templates that reflect other reporting means of the target project. It is good to ask the name of the reporter if the report needs to be clarified after the session.
If session findings are not reported further, they vanish.Completion work is an equally important part of the process and, during that, results are reported further. Enough time must be allocated for this work by the facilitator and domain management to make this possible.
If testing of the target software requires specific testing tools/techniques and all participants are not familiar with them, time and effort will be wasted.If the testing tool/technique utilized in the session is demanding, the goal of the session should reflect that, and learning should be the main target of the session.
If the technical environment is unstable, results are harder to gain, and results might be unreliable.The technical environment has to be checked to ensure that it is working (and fixed, if needed), so that testing can be done without interruption.
If the physical environment is restless or too formal, it will disturb concentration and the atmosphere suffers.The physical environment has to be checked to ensure that it is convenient, so testing can be done in a proper setting. Before the session, the room has to be prepared, for example, templates made available.
If sessions are not guided and focused, it will cause ad hoc testing, testing beside other activities, and resources to be wasted.The facilitator leads the sessions and keeps the participants focused. The person responsible for facilitator role has to be able to lead a team.
If the session or target software introductions are too formal and long, the atmosphere will stiffen, and time will be wasted.When preparing introductions, the facilitator and domain expert have to take the informal nature of the session into account. They can also review each other’s introductions.
If the session result reporting is missing or inadequate, the benefits of the method are not verifiable and domain management will not be able to do the proper followup. Also planning new sessions will be more laborious and can lead to poor parameter values.A summary report of the session results must be delivered to all interested parties, and it must reflect the session goals. The report has to document the parameter values used so that they can be taken into account when planning the next session.
If preparation work, that is, parameter definition, is done inadequately, it may cause ad hoc testing, unsatisfactory results, and dropping the method usage.The facilitator has to allocate enough time for the preparation work; domain management has to make this possible. When defining the parameters, the facilitator can use domain management and expert support.
If the domain expert does not have firm domain area knowledge, the participants will not get the support they need and may end up reporting already known defects and incomplete features.The domain expert has to be a person who knows the target area and its current situation and is able to express it clearly. This has to be acknowledged when nominating the responsible for this role.
If the TET domain management does not see the benefits of the approach, utilization might fail completely as resources will not be allocated.Those seeing potential in the approach have to sell the approach to the domain management. Experiences from the other practitioners may help with this. After the first session has taken place, management has to be properly reported to make sure that they have enough information to assess the approach.
No quantitative results gained.Careful parameter preparing, especially participant selection, lowers the risk. Possible qualitative results, like learning, compensate the outcome.