API Performance Testing Tools: JMeter, Taurus, and BlazeMeter
May 4, 2022

API Performance Testing Tools: JMeter, Taurus, and BlazeMeter

Performance Testing
API Testing

By

Let's talk about APIs from a performance testing perspective. Imagine you have a web app that consumes an API. Once your web application is publicly available, multiple clients will be hitting the same service at the same time. That being the case, you would definitely want to execute performance tests to be sure your servers are accessible to all users, even under a heavy load, and that there are no API delays.

But how can you do it? Many engineers are not sure how to run software performance testing at the API level. In this article, I am going to walk you through different ways for running API performance testing tools using three of the most useful API load testing tools: JMeter, Taurus, and BlazeMeter. 

We will be using a public API called Deck of Cards in all of the examples. You can find the documentation here.

Table of Contents

  1. What are API Performance Testing Tools?
  2. API Performance Testing Tool: JMeter 
  3. API Performance Testing Tool: Taurus
Back to top

What are API Performance Testing Tools?

APIs are usually consumed by multiple systems, which makes them an important testing target. API performance testing tools will not only detect failures on the API itself, but also on other, third-party components. 

Back to top

API Performance Testing Tool: JMeter 

JMeter is an open-source tool used for performance testing. It can be used on any platform that supports Java, which makes it a good option for automating and executing API performance tests.

Let’s see how to create an API test:

1. Open JMeter

2. Right-click on Test Plan - Add - Threads (Users) - Thread Group

JMeter load testing

11. As we saw in step 7, the response to the request is:

{"success":true,"deck_id":"3p40paa87x90","shuffled":true,"remaining":52}

 

So if we want to extract the value of deck_id, we may add a regular expression like this one: "deck_id":\s"(.*)",

12. Going back to JMeter, we will complete the fields as follows:

JMeter load testing

Remember to remove or disable the View Results Tree when running high load tests, in order to improve the performance of the load generator.

📕 Related Resource: Learn more about JMeter Performance Testing: Upload and Download Scenarios 

Back to top

API Performance Testing Tool: Taurus

Taurus is an open-source API testing tool that automates and runs performance tests. Unlike JMeter, Taurus doesn’t have a GUI, but it enables simplified running of other tests from tools like Gatling, Locust, Selenium, and others.

In Taurus, you can write your script on a YAML File and run it on a terminal. YAML is an easy-to-understand language (which leads to a lower learning curve). This makes it easy to maintain using version control tools such as GitHub. It is also an advantage when adding your tests to continuous integration tools like Jenkins. 

Here you can read the complete guide to API testing with Taurus. Now let’s see how to test our Deck of Cards API.

Every YAML file starts with an “execution” section. This is where you define the parameters corresponding to the load and the execution engine to be used (which is JMeter by default).

execution:
  - executor: jmeter
    concurrency: 5
    ramp-up: 10s
    hold-for: 60s
    scenario: deck of cards

 

  • scenario: it is the name of the scenario that is going to be executed. It must be defined in the “scenarios” section (which we will see next). It is important to keep in mind that this field does not have a default value, therefore we must specify it. If not, Taurus will not know which scenario to run.
  • concurrency: the number of target concurrent virtual users
  • ramp-up: the ramp-up time for reaching target concurrency
  • hold-for: the time to hold the target concurrency

After defining the test, it’s time to add the “scenarios” section to the YAML file. This section tells Taurus which steps to execute during the test. 

1. First, we are going to name the scenario. This name matches the scenario in the “execution” section.

execution:
  - executor: jmeter
    concurrency: 5
    ramp-up: 10s
    hold-for: 60s
    scenario: deck of cards 
scenarios:
 deck of cards:

 

2. Then we will add requests and transactions. The transactions are the same as the Transaction Controllers on JMeter. They are useful to identify the different steps of our tests.

execution:
  - executor: jmeter
    concurrency: 5
    ramp-up: 10s
    hold-for: 60s
    scenario: deck of cards 
scenarios:
 deck of cards:
  requests:
   - transaction: Shuffle the cards
     do:

 

3. Inside those, we will list the individual requests to be executed. We have a GET HTTP request to Shuffle the cards. 

execution:
  - executor: jmeter
    concurrency: 5
    ramp-up: 10s
    hold-for: 60s
    scenario: deck of cards 
scenarios:
 deck of cards:
  requests:
   - transaction: Shuffle the cards
     do:
       - url: 'http://deckofcardsapi.com/api/deck/new/shuffle/?deck_count=1'
         method: GET

 

  1. Now we are going to add an assertion to check if we are getting the right response.
execution:
  - executor: jmeter
    concurrency: 5
    ramp-up: 10s
    hold-for: 60s
    scenario: deck of cards 
scenarios:
 deck of cards:
  requests:
   - transaction: Shuffle the cards
     do:
       - url: 'http://deckofcardsapi.com/api/deck/new/shuffle/?deck_count=1'
         method: GET
         assert:
         - contains:
           - deck_id

 

5. As we did on JMeter, on the “Shuffle the cards” request we extracted the value of deck_id and used it later on the “Draw a card” request.

execution:
  - executor: jmeter
    concurrency: 5
    ramp-up: 10s
    hold-for: 60s
    scenario: deck of cards 
scenarios:
 deck of cards:
  requests:
   - transaction: Shuffle the cards
     do:
       - url: 'http://deckofcardsapi.com/api/deck/new/shuffle/?deck_count=1'
         method: GET
         assert:
         - contains:
           - deck_id
         extract-regexp:
          deck_id:
           regexp: '\"deck_id\":\s\"(.*)\",'
           default: NOT_FOUND  
           match-no: 1  
           template: 1  
           subject: body  
           scope: all

 

  • regexp: the regular expression
  • default: the default value to use when the regular expression doesn’t match. 
  • match-no: If multiple values have matched, you specify which match to use. In this example, we are matching the first result.
  • template: Which capture group to take
  • subject: the subject for search (body, headers,http-code, url)
  • scope: If we set the scope to “all”, it will check main and sub-samples
  1. Let’s add a transaction called “Draw a card”, which is the second step of our test.
execution:
  - executor: jmeter
    concurrency: 5
    ramp-up: 10s
    hold-for: 60s
    scenario: deck of cards 
scenarios:
 deck of cards:
  requests:
   - transaction: Shuffle the cards
     do:
       - url: 'http://deckofcardsapi.com/api/deck/new/shuffle/?deck_count=1'
         method: GET
         assert:
         - contains:
           - deck_id
         extract-regexp:
          deck_id:
           regexp: '\"deck_id\":\s\"(.*)\",'
           default: NOT_FOUND  
           match-no: 1  
           template: 1  
           subject: body  
           scope: all
   - transaction: Draw a card
     do:

 

7. Inside this transaction, we’ll add a GET HTTP request to draw a card. 

execution:
  - executor: jmeter
    concurrency: 5
    ramp-up: 10s
    hold-for: 60s
    scenario: deck of cards 
scenarios:
 deck of cards:
  requests:
   - transaction: Shuffle the cards
     do:
       - url: 'http://deckofcardsapi.com/api/deck/new/shuffle/?deck_count=1'
         method: GET
         assert:
         - contains:
           - deck_id
         extract-regexp:
          deck_id:
           regexp: '\"deck_id\":\s\"(.*)\",'
           default: NOT_FOUND  
           match-no: 1  
           template: 1  
           subject: body  
           scope: all
   - transaction: Draw a card
     do:
       - url: 'http://deckofcardsapi.com/api/deck/${deck_id}/draw/?count=2'
         method: GET

 

  1. Finally, we are going to add an assertion to the “Draw a card” request.
execution:
  - executor: jmeter
    concurrency: 5
    ramp-up: 10s
    hold-for: 60s
    scenario: deck of cards 
scenarios:
 deck of cards:
  requests:
   - transaction: Shuffle the cards
     do:
       - url: 'http://deckofcardsapi.com/api/deck/new/shuffle/?deck_count=1'
         method: GET
         assert:
         - contains:
           - deck_id
         extract-regexp:
          deck_id:
           regexp: '\"deck_id\":\s\"(.*)\",'
           default: NOT_FOUND  
           match-no: 1  
           template: 1  
           subject: body  
           scope: all
   - transaction: Draw a card
     do:
       - url: 'http://deckofcardsapi.com/api/deck/${deck_id}/draw/?count=2'
         method: GET
         assert:
         - contains:
           - deck_id
 

The final script will look like this:

execution:
  - executor: jmeter
    concurrency: 5
    ramp-up: 10s
    hold-for: 60s
    scenario: deck of cards 
scenarios:
 deck of cards:
  requests:
   - transaction: Shuffle the cards
     do:
       - url: 'http://deckofcardsapi.com/api/deck/new/shuffle/?deck_count=1'
         method: GET
         assert:
         - contains:
           - deck_id
         extract-regexp:
          deck_id:
           regexp: '\"deck_id\":\s\"(.*)\",'
           default: NOT_FOUND  
           match-no: 1  
           template: 1  
           subject: body  
           scope: all
   - transaction: Draw a card
     do:
       - url: 'http://deckofcardsapi.com/api/deck/${deck_id}/draw/?count=2'
         method: GET
         assert:
         - contains:
           - deck_id

 

To execute the script, navigate to the location where you saved the script in your command line window and execute bzt.yml. Where “.yml” is the name of the script. 

As the script executes, a live report will be displayed showing various metrics.

 

https://imagecompressor.com/

 

 

All of these tools are great options for API performance testing. Each offers different benefits:

  • JMeter allows you to create tests with a GUI, making it more user-friendly and letting them run from your machine.
  • Taurus enables simplified coding with quick results.
  • BlazeMeter lets you scale your JMeter and Taurus scripts to thousands of users and run your test from different locations using a user-friendly dashboard for looking at the live results.

START TESTING NOW

 

Related Resources: 
Back to top
Mikaella Mateos

Mikaella Mateos

Performance Tester, Abstracta Inc.

Mikaella is a technical tester with experience in Performance, Automation, and Functional testing. She's been working at Abstracta since 2020 where she has participated in many Performance Engineering projects. Mikaella is also an instructor of functional and performance testing courses at Abstracta Academy and has written multiple testing articles.