Using the JMeter AutoStop Listener - How to Do it Right
Listeners are Apache JMeter™ elements that check JMeter requests, evaluate results and monitor system health. They do so by assembling data, enabling customization and manipulating information. The results or data gathered by listeners can be shown in the form of a tree, tables, graphs, log file or even in simple text in some cases.
The JMeter AutoStop Listener is a listener that can be used when you want to stop a test on a certain runtime basis: average response time, average latency or error rate. When executing tests, you can configure the AutoStop Listener to stop the test if the target environment exceeds a certain defined value, say if the error failure rate exceeds the expected tolerable limit, for a defined period of time. This threshold setting is useful to ensure the test stops when the tested service becomes unresponsive or diverts from the expected response time, latency or error rate.
Add the AutoStop Listener through your JMeter Plugins Manager.
AutoStop on Response Time/Latency
The response time is the aggregated time it takes the system to process a request, from the first byte of data leaving the user until the last byte is received by the user.
Latency is the measurement of how long it takes from sending a request to receiving the first response, including the time of the sending and the receiving.
The average response time (or average latency) is the arithmetical mean of all response times (or latencies) if they were divided equally among all requests.
- Request I - 3 ms
- Request II - 5 ms
- Request III - 4 ms
So, the average response time would be (3 + 5 + 4) / 3 = 4 milliseconds.
With the AutoStop listener, the test is stopped if the defined response time/latency exceeds N ms for the period of time defined. In this example, if the AutoStop listener was set to 4 ms the test would stop, but if it were set to 2 it wouldn’t. To disable the AutoStop listener, set the time value to 0.
AutoStop on Error Rate
The error rate is the threads/user failure percentage.
For example, let’s say we run test for 100 users, and 10 of them are unable to get the expected response, and fail. The error rate will be ‘10/100 * 100’, which is 10%. The Error rate can also be a float number, i.e. any real number that contain a fractional part. If 9 users from a total of 200 users fail, the error rate would be ‘9/200 * 100’ that is 4.5%.
With the AutoStop listener, the test is stopped if the defined error rate exceeds N percent for the period of time defined. In the first example, if the AutoStop listener was set to 8% the test would stop, but if it were set to 15% it wouldn’t. To disable the AutoStop listener, set the time value to 0.
How to Use the JMeter AutoStop Listener - Step-by-Step
1. Add a thread group. Here, I have set 10 threads with 0 ramp up time. The test will run for these 10 concurrent users.
2. Add the sampler request to specify the target environment on which we are going to execute the load test. As shown in the attached screenshot, this request is targeting the page “blazedemo.com”. I named the Request “BlazeDemo Request”.
To monitor the output response of the executed test, I added some listeners:
- “View Results Tree” to ensure the output response meets the actual desired response.
- “Summary Report” to view the value of various output parameters, like average response time, error rate, etc.
After executing the test, I observed that the average response time is around 600 ms and the error rate is 0% for 10 concurrent users. So to check the AutoStop listener, I decided to shut down the test when the average response time exceeds 500 ms over 1 sec.
For the executed test, if the average response time is more than 500 ms for 1 second, the thread group will stop working. Since the average response time is around 600 ms, the AutoStop listener will activate and the mark for autostop will be recorded and shown in the jmeter.log file, as shown in the attached screenshot below.
Similarly, we can set the error rate or average latency parameter.
That’s it! You now know how to use the JMeter AutoStop listener. To learn more JMeter, check out our free JMeter academy.
To try out BlazeMeter, which enhances JMeter abilities, put your JMX file or URL in the box below, and your test will start in minutes.