Exploratory Testing Explained
Exploratory testing is a method for executing testing activities, in which the design and execution phases are performed simultaneously. This means that the tester dives into testing, without having clear instructions on what to test. Instead, the tester decides what and how to test, during the testing itself, by exploring the product. This provides the tester freedom and responsibility for the test performed. Mainly used for functional testing, exploratory testing enables the tester to learn about the product while testing, while aiming to make tests more efficient every time.
Think about the way you start using a new app on your mobile phone. You play around with it, using your knowledge from other products but also investigating new options, to learn how your new phone works. By trial and error, you focus on the main aspects you are interested in. You try to make the most of the short time you have, checking if and how it can answer your needs. This is what happens during exploratory testing.
Exploratory testing is a very good choice if there’s not much time available for testing or there is lack of information about the system available. The exploratory testing results can provide the developer with the big picture of the product, and with an idea of the main functionalities to focus the future tests on, without needing to read the requirements. In fact, one of the main premises when adopting the exploratory testing approach, is to learn from the testing, in order to improve the quality of future tests performed. For example, if while conducting a test something looks suspicious, the tester will probably try to go in that way in the next, more formal, testing activity. In addition, there is no need for documentation before testing starts, so more testing time is available.
Exploratory Testing & Planned Testing
Exploratory Testing complements Planned Testing. In planned testing, the design and execution phases are created and conducted in different times and for more than one person. This means that the tester who performed the tests will get a set of directions and steps in advance, to follow while testing.
One of planned testing’s advantages, is the possibility to share the test plan with others and perform the resource analysis before execution starts. This is often used when there is knowledge of the product and time resources are less of an issue. This also enables more sophisticated options: during the planned testing design phase, different planned testing tools can be used. These include equivalence classes, decision tables, decision trees, state machines or use cases, for example.
On the other hand, one of the disadvantages of planned testing, is that due to the structure process and the need to follow a step by step procedure as specified in the designing phase, some product aspects might stay hidden, if they were not considered in the plan. In exploratory testing, they could have been spontaneously discovered.
In my opinion, a mix between both Exploratory and Planned testing is the best recipe for an excellent test plan. So even if you already execute planned testing, it would be good practice to add exploratory testing to the beginning of your process. You could also add exploratory testing later in the cycle, this will mitigate the possibility of undiscovered areas. Finally, in some cases, performing the same tasks in a different way may bring different results, so it’s important to try.
How to Perform Exploratory Testing
There are three ways to perform Exploratory testing:
1. Ad Hoc Testing - the most informal and less recommended way to test. Anyone is able to perform ad hoc testing even if no testing skills exist. You just need to ask the person to try the product without giving any directions or setting expectations.
In Ad Hoc testing, there is no order, objective or method. This way can be helpful if the only objective is to report bugs. But, since exploratory tests are not registered, there is no way of follow up nor trace tester actions.
2. Strategy Based Testing - probably the most used technique by more experienced testers. In this case, the tester will apply the various types of techniques, such as limit values or equivalence partitioning to create test cases while testing. This way the tester can use her/his knowledge and implement techniques he/she considers most appropriate for a successful testing.
3. Session Based Testing - the most common way to apply the exploratory testing strategy. The idea is to provide a period of time, usually between 30 minutes to 2 hours, in which the tester will test without interruptions. During this time, the tester will take into consideration “The Mission”, i.e what we are going to test. This “mission” can be a functionality, a form, or any aspect of the product we need to focus on.
After the testing is completed, the tester will complete a document with notes from the session, including the date, name of the tester, risks, bugs, coverage and metrics. Some of these metrics are “total time spent in the session”, “relation between time spent in the mission vs time spent in opportunities” and ¨percentage of time for testing, defects and session built”.
Then the tester will analyze these metrics. For example, if the relation between time spent focused in the mission is lower than the acceptance value agreed for this system, a new session may be needed. However distractions are allowed and may also provide information as the need for new missions or more time for testing.
Exploratory Testing Main Recommendations - A Summary
- If you need to learn about the tool or product, exploratory testing is a useful approach. This may help you make the first move to your testing plan.
- Use exploratory testing if you do not have enough time or too much knowledge about the product.
- Learn from previous tests performed in order to conduct more efficient test in the future.
- While you perform exploratory testing, take good notes, as they will be the inputs for decision taking.
- When conducting session based testing, the objective is to test and achieve our mission. However, distractions aren’t necessarily problematic, because different things may arise from distractions.
That’s it! I hope this approach improves your testing quality. Now is your turn to try it!