How to Use the Arrivals Thread Group and the Free-Form Arrivals Thread Group
Apache JMeter’s basic functionality includes a Thread Group element. This element is used to define the schedule and level of the load. The Thread Group element is capable of handling simple performance tests. However, JMeter has a lot of great Thread Group extensions that can help us with more complex scenarios.
One such extension is the Custom Thread Groups plugin, which allows us to create more complex load scenarios. This plugin extends the basic JMeter thread group functionality with five awesome thread group types: Stepping Thread Group, Ultimate Thread Group, Concurrency Thread Group, Arrivals Thread Group and Free-Form Arrivals Thread Group. We haven’t covered the Arrivals Thread Group and Free-Form Arrivals Thread Group in our blog before, so let’s take a look at them now. Both of these Thread groups were developed and then improved by BlazeMeter’s Andrey Pokhilko and Artem Fedorov.
When to Use the Arrivals Thread Group and Free-Form Arrivals Thread Group
Before we start working with Thread Groups we need to clarify one important thing - what does “arrival” mean? One "arrival" is one start of a thread iteration (one iteration start of a virtual user).
The Arrivals Thread Group is used to schedule “arrivals”. The scheduling of the "arrivals" is one more way to describe a load. Since an “arrival” is a script iteration start, scheduling arrivals is the scheduling of iteration starts. The Free-Form Arrivals Thread Group is similar to the Arrivals Thread Group with one major difference - the scheduling is more flexible.
The Arrivals Thread Group is good for more traditional load testing. A possible scenario could be something like ‘Is our system performing well when users are buying 10 tickets per minute for a duration of one hour?’.
The Free-Form Arrivals Thread Group is good for load testing, but it is also great for Stress testing scenarios like Spike testing. A possible scenario could be something like “Can our system handle 30,000 ticket sales in 5 minutes after the start of a promotion?”. For more information about the differences between Load Testing and Stress Testing you should head on over to this link.
Great! We are ready to play with the Arrivals Thread Group and Free-Form Arrivals Thread Group elements to see how they work.
Setting Up Your JMeter Script
If you are not sure how to install JMeter plugins, please follow this link to learn how.
We need a test script to demonstrate the two Arrivals Thread Groups in action. Let’s use the BlazeDemo website to create some JMeter scripts. I developed a simple script that has three steps:
- Open Landing page
- Search for flights
- Open Purchase Ticket page
I added constant timers to the Search and Purchase page requests. We will use them to change the overall duration of the script in future experiments. I also added some listeners to the script. The most interesting to us are the Active Threads Over Time, Hits per Second and Summary Report listeners.
Load Testing with the Arrivals Thread Group
As mentioned, the Arrivals Thread Group is used to schedule “arrivals”, i.e iteration starts. Therefore, if all existing users are in the middle of the iterations JMeter will automatically create new threads. This could create a huge number of threads and cause “out of memory” problems. So, please, be careful using this element and use the Concurrency Limit field to avoid such problems.
To find the Arrivals Thread Group: Test plan -> Add -> Thread (Users) -> bzm - Arrivals Thread Group
Fill in the following values:
- Target Rate (arrivals/min): 120
- Ramp Up Time (min): 1
- Ramp-Up Steps Count: 2
- Hold Target Rate Time (min): 2
- Target Rate defines how many iterations start in one minute while the target load is being achieved. In our case the target load is 120 “arrivals” per minute (2 “arrivals” per second).
- Ramp Up Time defines the time to achieve the target load level. In practice, this time is less than the duration of one step, due to JMeter design. In our case we have two steps. So, a 1 minute target up time in theory becomes 30 seconds in practice. We can see this in the graph above.
- Ramp-Up Steps Count defines the number of steps to achieve the target load level. In our case it is 2 steps.
- Hold Target Rate Time defines the target load level duration. In practice we need to add the duration of the last step to this number. In our case it is 2 minutes in theory and 2.5 minutes in practice.
- Time Unit defines the unit of measurement for time in the interface of this Thread Group. Here we have two options: minutes or seconds. In our case we have chosen minutes.
- Thread Iterations Limit defines the number of the iterations to run for each virtual user. It could be useful if we need to run each user only once to simulate users’ real behaviours. In our case this field is empty, so each user will run indefinite iterations until end of the schedule.
- Log Threads Status into File defines the name of the file to save the logs about “arrivals”. An example can be found in the screenshot below.
- Concurrency Limit defines the maximum number of virtual users to avoid out of memory problems. In our case it is 1000, quite a large number.
- Graph displays a schedule of “arrivals” (number of “arrivals”/min) and the approximate number of total “arrivals”. In our case we expect about 330 total “arrivals”.
Let’s run our first experiment with a short script. I updated the constant timers to 0 milliseconds. This means that each iteration of the script will pass quickly and should take about 50 milliseconds.
So, let’s run the script!
The Active Threads Over Time listener shows that one virtual user is almost enough to cover a load schedule such as this. It means that the iteration has enough time to finish, before the next one should start.
The Summary Report listener shows that the script was run 331 times and it is near 330 “arrivals”. Good, everything looks right.
Let’s run another test and update both timers to a 1 minute (60000 ms) duration. Now the overall duration of the script will be about 2 minutes.
We’ll keep the settings of the Arrivals Thread Group the same and run the test again.
JMeter started 240 threads to satisfy the expected load as we can see in the screenshot above.
The script started 331 times and it meets our ~330 “arrivals” requirement, but the Purchase page request was run only 211 times. This means that iterations were interrupted by JMeter because of the duration of the timers. We should take this into account when working with the Arrivals Thread Group. A script will start the expected number of times, but it doesn’t means each iteration will run until the end.
Now is time to try the Concurrency Limit option!
Update the Concurrency Limit value to 100. Note that expected number of “arrivals” is still the same.
The script run was limited with 100 threads and we can see this in the screenshot above.
The expected number of “arrivals” wasn’t achieved. It is only 289 while expected value is 330. This is also an important understanding that we should take into consideration.
Stress Testing with the Free-Form Arrivals Thread Group
To find the Free-Form Arrivals Thread Group: Test plan -> Add -> Thread (Users) -> bzm - Free-Form Arrivals Thread Group
Add two rows:
- Start value: 10; End value: 120; Duration 1;
- Start value: 120; End value: 120; Duration 2.
One row of the Threads Schedule area describes one test period:
- Start value means “arrival” rate at the beginning of the test period.
- End value means “arrival” rate at the end of the test period.
- Duration defines the duration of this test period.
Update the value of the constant timers to 0 ms again. So, we will run a short script. We expect about 305 “arrivals”. Let’s run it and check the results.
Similar to our experiment with the Arrivals Thread Group, most of the time one virtual user is enough to keep this “arrival” schedule.
Open the Summary Report listener and check the number of “arrivals”.
We see that the actual number of “arrivals” is 304 and this is close to the expected value (~305 total “arrivals”), and the script was started and finished 304 times.
Let’s change the duration of the script as in the example above. Set the duration of the timers to 1 minute (60000 ms). The duration of the script will become about 2 minutes.
Keep the settings of the Free-Form Arrivals Thread Group without changes.
Check the Active Threads Over Time listener after the test run.
The maximum number of active threads achieved 240 virtual users. The results are similar to the results of our long script test for the Arrivals Thread Group. So, we are good here.
The Summary Report listener shows that the script was started 304 times, but the last request was run only 184 times. Take this fact into account. “Arrival” means that the script will start, but it doesn’t mean that the script will run until the end.
Time to test the Concurrency Limit option.
Set Concurrency Limit to 100. This setting should disallow JMeter to start more than 100 virtual users.
Start the script.
We see on the graph that the limit was not exceed. Limitation works as expected.
The expected number of total “arrivals” wasn’t achieved. This is the same behaviour as for the Arrivals Thread Group experiment. So my suggestion is to make schedules that don’t reach the Thread Group limitation.
Alright, above was a demonstration of the Free-Form Arrivals Thread Group for Load testing. How about Stress testing? The Free-Form Arrivals Thread Group is perfect for that!
Let’s create a Spike test scenario. A Spike test is a type of Stress test that includes a sharp rise in the load level.
Update the Threads Schedule with the following values:
- Start value: 10; End value: 50; Duration 0.5;
- Start value: 500; End value: 500; Duration 0.5;
- Start value: 50; End value: 10; Duration 2.
This is an example of a Spike test scenario. This scenario contains a sharp rise of expected “arrivals” during the test run. Let’s run the script and see how the BlazeDemo server will cope with this situation.
The Hit per Second listener shows a significant increase in requests to the server.
The Summary report listener shows that all requests passed. This means that the server successfully coped with the load.
To learn more advanced JMeter like this, go to our free advanced JMeter academy.
Running the Arrivals Thread Groups in BlazeMeter
If you want an easy way to scale your tests, or you need to collaborate on tests and reports, run your JMeter script in BlazeMeter. You will also get advanced reporting and be able to see trends over time.
Just upload your JMX script and run your test. For example, here is the JMeter script we created here, in BlazeMeter:
To try out BlazeMeter’s abilities, request a demo, or put your URL in the box below and your test will start in minutes.
Nice! We have considered the Arrivals Thread Group and Free-Form Arrivals Thread Group. That’s it for this post! Thanks for your time!