Yuri Bushnev is a Head of System Quality at AlphaSense, with more than 5 years of experience in Quality Assurance, DevOps and Development.

Learn JMeter in 5 Hours

Start Learning
Slack

Run massively scalable performance tests on web, mobile, and APIs

Nov 08 2017

Locust Assertions - A Complete User Manual

Performance and scalability are extremely important aspects of our web applications. Due to this fact, there are many tools available for free that allow us to verify how our application or servers behave under a custom amount of load. Each tool comes with a list of pros and cons that require you to think twice before choosing one or another. You can go with strong mature Apache JMeter™ if you prefer a GUI for your tests creation; you can try writing scripts yourself in Scala language by using Gatling; or you can try load testing with Locust.

 

Locust is an extremely powerful and lightweight performance framework, which allows you to write performance scripts in pure Python. Locust was created by developers, for developers. And while it is not a full replacement for the aforementioned tools, at the same time it is designed to solve specific problems and it does that extremely well. Creating lightweight Python based performance scripts allows you to simulate thousands of test users on any laptop, without stepping outside your current code development IDE.

 

When I used Locust for the first time, I liked its powerful capabilities and the simplicity of the test creation workflow. But at the same time, after many years of JMeter experience, I was a bit lost in the Locust documentation when trying to find familiar functionalities and “how to” steps for my goals.

 

One of the topics I couldn’t find the answer to right away was Assertions. We all love JMeter for its powerful assertions capabilities. There are hundreds of JMeter articles about each assertion element, while for Locust you can barely find any mention about any assertion. But that doesn’t mean this functionality doesn’t exist.

 

Locust was created in such a way that you don’t have many out-of-the-box functionalities. But at the same time, you can easily extend its capabilities with any custom functionality, with only a small block of Python-based code.

 

In this article, we are going to implement some of the most important assertions you might need for your performance tests when using the Locust framework. Let’s go!

 

Response Body Assertions With Regex

 

The Response Body Assertion is one of the commonly used assertions that you might find useful for performance tests validation. If you are not familiar with regex, I highly recommend you go over tutorials (e.g. this live coding regex tutorial), because Regex is a powerful technique that is used across different development and quality assurance tasks.

 

The main idea behind Regex is that we can specify any pattern we expect to see in the response body of our test requests. In Locust, you can use the built-in ‘re’ python module that contains the functionality to work with regex patterns.

 

To verify a response body in the Locust script, all you need to do is:

  • Prepare a regex using ‘re.compile(“[your_expected_regex_string]”)’
  • Use the Python default assertions function with this syntax: assert [some_condition], [“failure_message”]

 

For our handy example we are going to validate if the http://blazedemo.com/ home page was loaded successfully by verifying the title that should be presented on the home page. The script looks pretty simple:

 

import re
from locust import HttpLocust, TaskSet, task

class BrowseDocumentation(TaskSet):
   HOME_PAGE_TITLE_REGEX = re.compile(".*Welcome to the Simple Travel Agency!.*")

   @task
   def index_page_with_regex_assertion(self):
       r = self.client.get("/")
       assert self.HOME_PAGE_TITLE_REGEX.search(r.text) is not None, \
           "Expected title has not been found!"

class AwesomeUser(HttpLocust):
   task_set = BrowseDocumentation
   host = "http://blazedemo.com"
   min_wait = 1 * 1000
   max_wait = 5 * 1000

 

To test that the assertion works fine, you can run the script above in non-GUI mode by using the command below:

 

locust -f PowerfullAssertionsWithLocust.py --no-web -c 50 -r 10

 

how to run assertions in locust

 

As you can see, there are no failed assertions during test execution, which is good, except we still cannot be sure that our assertion works as expected. Let’s change HOME_PAGE_TITLE_REGEX with any invalid value like:

 

HOME_PAGE_TITLE_REGEX = re.compile(".*Just invalid article.*")

 

And if you run the same script again, you should see lots of failed assertions:

 

locust assertions - how to use them

 

As you can see, we added just a few Python code lines and now we are able to parse and validate responses using inbuilt Python functionality.

 

Response Body Assertions With XPath

 

Regex assertions are quite powerful, but sometimes we want to validate not only the text presence of the response but also the exact location of a certain HTML tree element. One of the common ways to do that is to use XPath locators (more details in this article). The XPath locators parse the HTML and get a specific value from the HTML tree.

 

For these needs, Python also has inbuilt functionality, which is located in the ‘html’ Python module. By using functions provided by the ‘html’ module, we can convert the text representation of the response into a XML tree. After that, we can verify if the number of found elements by the specified XPath equals 1. This is the code snippet that explains how to use the response XPath assertions in Locust:

 

import re
from locust import HttpLocust, TaskSet, task
from lxml import html

class BrowseDocumentation(TaskSet):
   HOME_PAGE_TITLE_XPATH = "//h1[contains(text(),'Welcome to the Simple Travel Agency!')]";

   @task
   def index_page_with_regex_assertion(self):
       r = self.client.get("/")
       tree = html.fromstring(r.text)
       assert len(tree.xpath(self.HOME_PAGE_TITLE_XPATH)) == 1, \
                    "Expected title has not been found!"

class AwesomeUser(HttpLocust):
   task_set = BrowseDocumentation
   host = "http://blazedemo.com"
   min_wait = 1 * 1000
   max_wait = 5 * 1000

 

Response Body Assertions With CSS

 

Some of us prefer CSS locators instead of XPath. Therefore, it’s interesting to check if we can use CSS assertions the same way we use XPath assertions. CSS is one of the commonly used and well-known languages in web development. All functions needed to work with CSS selectors are also presented in the ‘lxml’ Python module, which is good news for us because we don’t need to import any additional modules.


In fact, CSS assertions look very similar to XPath assertions, except for one big difference. In the XPath we used in the first example, we have a function that verifies the presence of text. CSS doesn’t provide the same functionality, because CSS works only with web elements and not with text nodes. So if you need to verify the presence of an element containing text, you can do it this way:

 

from locust import HttpLocust, TaskSet, task
from lxml import html

class BrowseDocumentation(TaskSet):
   HOME_PAGE_TITLE_CSS = ".jumbotron h1";

   @task
   def index_page_with_css_assertion(self):
       r = self.client.get("/")
       tree = html.fromstring(r.text)
       assert len(tree.cssselect(self.HOME_PAGE_TITLE_CSS)) == 1, \
           "Title element is not unique"
       assert tree.cssselect(self.HOME_PAGE_TITLE_CSS)[0].text == "Welcome to the Simple Travel Agency!",\
           "Expected title has not been found!"

class AwesomeUser(HttpLocust):
   task_set = BrowseDocumentation
   host = "http://blazedemo.com"
   min_wait = 1 * 1000
   max_wait = 5 * 1000

 

First, we take the CSS locator that should contain the expected text and verify if this element is unique. On the second step, we verify that this unique element has the expected text. It is not enough to have only step number two because there might be a situation where there are a few elements returned by mentioned CSS locator, and we cannot be sure which is used in the current assertion.

 

Response Code Assertions

 

The Response Code Assertion in Locust is extremely simple. A response object contains a property called ‘status_code’, which you can use for assertions. We can verify that when we request the home page, the response code is 200, which means that the request was executed successfully:

 

from locust import HttpLocust, TaskSet, task
from lxml import html

class BrowseDocumentation(TaskSet):
   @task
   def index_page_with_response_code_assertion(self):
       r = self.client.get("/")
       assert r.status_code is 200, "Unexpected response code: " + r.status_code

class AwesomeUser(HttpLocust):
   task_set = BrowseDocumentation
   host = "http://blazedemo.com"
   min_wait = 1 * 1000
   max_wait = 5 * 1000

 

Response Headers Assertions

 

Response Headers Assertions also require a very easy technique, which can be used without any additional Python modules. Response objects which return after request execution contain python dictionary “headers”. That’s why you can verify headers using this small code snippet:

 

from locust import HttpLocust, TaskSet, task
from lxml import html

class BrowseDocumentation(TaskSet):
   @task
   def index_page_with_response_header_assertion(self):
       r = self.client.get("/")
       assert r.headers['Connection'] == 'keep-alive', "Unexpected connection header: " + r.headers['Connection']

class AwesomeUser(HttpLocust):
   task_set = BrowseDocumentation
   host = "http://blazedemo.com"
   min_wait = 1 * 1000

 

In this example we verify that the “keep-alive” header configuration is switched on for our application under test. We chose it because it is a handy example of how we can show the assertion in action.

 

The Headers Assertion is not a popular type of assertion, so you should have quite a unique test scenario (for example, verification of security related headers). But if you want to learn more about the “keep-alive” property, check out this article, which has a very clear explanation of this option.

 

Duration Assertions

 

The Duration Assertion verifies that the response was received within a specified time. All responses that take longer than a given number of seconds are marked as failed. This type of assertion is also very simple to create.

 

A response class object that returns after the request execution contains an ‘elapsed’ property, which represents the amount of time elapsed between the moment a request was sent to the moment it was received. All you need to keep in mind is that the ‘elapsed’ property is presented as a timedelta object. That’s why if you want to verify if your requests won’t take more than 1 second, you can do it this way:

 

import datetime
from locust import HttpLocust, TaskSet, task

class BrowseDocumentation(TaskSet):
   @task
   def index_page_with_response_duration_assertion(self):
       r = self.client.get("/")
       assert r.elapsed < datetime.timedelta(seconds = 1), "Request took more than 1 second"

class AwesomeUser(HttpLocust):
   task_set = BrowseDocumentation
   host = "http://blazedemo.com"
   min_wait = 1 * 1000
   max_wait = 5 * 1000

 

JSON Response Assertions

 

JSON is one of the most commonly used formats for data exchange between a browser and a server. That’s why it is very important to know how to create assertions for JSON based responses.

 

To show you the JSON Response Assertion in action, we are going to use a test web application designed for JSON response testing: https://jsonplaceholder.typicode.com/. We need to add an additional Python module called “json”. Let’s assume that we have a request “https://jsonplaceholder.typicode.com/posts/1”, which should return this response:

 

{
  "userId": 1,
  "id": 1,
  "title": "sunt aut facere repellat provident occaecati excepturi optio reprehenderit",
  "body": "quia et suscipit\nsuscipit recusandae consequuntur expedita et cum\nreprehenderit molestiae ut ut quas totam\nnostrum rerum est autem sunt rem eveniet architecto"
}

 

And let’s say you want to assert that the userId returned by this request is correct and equals 1. Then we can assert the json elements this way:

 

import json
from locust import HttpLocust, TaskSet, task

class BrowseDocumentation(TaskSet):
   @task
   def index_page_with_json_assertion(self):
       r = self.client.get("/posts/1")
       json_response = json.loads(r.text)
       assert json_response['userId'] is 1

class AwesomeUser(HttpLocust):
   task_set = BrowseDocumentation
   host = "https://jsonplaceholder.typicode.com"
   min_wait = 1 * 1000
   max_wait = 5 * 1000

 

Final Thoughts

 

Assertions allow you to make functional checks at the same time you are doing nonfunctional performance verification. By using all the assertions mentioned above, you can test almost any functionality of your web application.

 

Of course, this list is not final. There are plenty of other assertions that you can apply based on your needs. We gave you the main and mostly useful assertions, which are usually enough to verify up to 90% of possible test cases.

 

However, as you can see, the implementation  of assertions in Locust is easy. All you need is some basic Python programming language knowledge. You most probably have such knowledge if you already chose Locust as the main tool for your performance verification.

 

Now you should have enough knowledge to make powerful assertions in your Locust performance scripts yourself. But you can always find all the ‘ready to use’ examples combined together through this link.

 

Click here to subscribe to our newsletter.

 

Run your Locust tests with BlazeMeter! Get scalability, multiple geo-locations and advanced reporting. To learn more, request a demo. Or, put your URL in the box below and your test will start in minutes.

 

Interested in writing for our Blog?Send us a pitch!

Your email is required to complete the test. If you proceed, your test will be aborted.