Andrey Pokhilko is Chief Scientist at BlazeMeter. Andrey is a renowned thought-leader and innovator within the Apache JMeter performance testing community. He has worked extensively to create and develop tools that enhance the Apache JMeter protocol coverage and analytical reporting capabilities. In 2009, he founded JMeter-Plugins.org and led this open source project to its undisputed status as the premier source of JMeter Plugins in the world. Loadosophia was similarly born from Andrey’s desire to complete JMeter’s offerings. He has world-class experience with large scale performance testing, having led the load testing team for five years at search engine giant Yandex.

Learn JMeter in 5 Hours

Start Learning
Slack

Run massively scalable performance tests on web, mobile, and APIs

Apr 07 2019

Performance Testing with Docker Containers

The Docker approach to virtualization can provide advantages when testing applications with open source performance testing tools. By making it replicable, Docker enables sharing the tests between users and replicating the test environment. This can be helpful for sharing knowledge between users of different levels, in addition to saving time by not running unnecessary tests.

 


This blog post will cover an overview of Docker, the specifics of using Docker containers for generating load, and how to involve open source Taurus for universal automation of performance tests. We’ll also cover how dockerization solves specific issues of performance tests inside CI, and how to use the cloud for scaling performance tests for massive loads.


Let’s get started.

 

What is Docker, and What Are Docker Containers?

 

Before we get into how to run performance tests with Docker containers, let’s get an overview on what Docker is.


Docker is a tool designed to make it easier to create, deploy, and run applications by using containers. Containers allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies, and ship it all out as one package. By doing so, thanks to the container, the developer can rest assured that the application will run on any other machine regardless of any customized settings that machine might have that could differ from the machine used for writing and testing the code.

 

performance testing with docker


Key features of Docker

 

  • An all inclusive application packaging approach 


Everything you need to run your application sits within a simple container image. You can copy it, share it with your team, develop it locally, and then put it into production.

 

  • Resource-efficient

With Docker, you can run many different applications at the same time. For example, you can run a full set of microservices without them interfering with each other. In the past, applications had dependency conflicts, it was challenging to run different applications on the same operating systems. We used to use virtual machines, but that required a lot of hardware to run. With Docker you can run isolated processes without running virtual machines, so it is resource efficient.

 

  • Portable

You can build a Docker container in one environment, and transfer it to another environment.  You can also multiply containers and run it on the same image on many different machines. There’s even Windows based Docker, so it’s not just for Linux users anymore.

 

  • Disposable


Docker containers require minimal effort to shut them down by destroying everything, allowing you to avoid a huge clean up process.

 

  • Open Source

Docker is a free open source solution, which means that you don’t need to pay for licensing, and it is frequently updated with new releases.


Benefits of Using Docker

 

There are many benefits for using Docker. Firstly, versioned images simplify deployments. This means you can simplify the evolution of your application with deploying just a new version with a new image.
Docker also provides a huge repository of free images for popular apps. We’ll show one of them when we delve deeper using a Taurus image.

 

Finally, Docker’s success has led to additional solutions such as Kubernetes and “Cloud-Native” which have really expanded cloud capabilities for developers and testers.
Now that we’ve covered what Docker is,  we're going to dig deeper into how to use open source Taurus to automate performance tests on Docker containers.

 

What is Taurus?


Taurus an open source command-line test automation tool. It’s a wrapper on top of JMeter, Gatling and many others.
A typical Taurus-based test starts with creating a YAML file describing the test configuration, like the one below:

 

load testing with Taurus and Docker


Then, the YAML file is launched with the help of Taurus command-line tool, like this:

bzt test.yml

 

As a result, you see a live dashboard with KPIs of your test, You can also configure Taurus to send these results in a web report.

 

performance testing with docker

 

From here, all the nuances of how your test will behave is based on your YAML file instructions.

The “bzt” command is used to run the tests and you will see the results in real-time on your dashboard.

 

Before we can run a performance test, we need our Taurus Docker image.

 

Using a Pre-Built  Taurus Image

 

What are the advantages of using a pre-built image?


There are several advantages of using a pre-build Docker image including:

  • not needing to install Taurus
  • no Python requirement
  • no JMeter installation needed
  • no Java requirement
  • pre-installed Selenium
  • as well as Chrome & Firefox

 

The command to run pre-built Taurus image is very simple, like this:

 

sudo docker run -it blazemeter/taurus http://blazedemo.com

 

Performance tests with Docker and Jmeter

 

Docker will download the image (if it is not present in the cache) and will run Taurus inside the container. Then, Taurus will generate real-time results in its dashboard..                                                      

 

Making Your Own Docker Image

 

If you want to have your own, maybe customized, version of Docker image with Taurus, it’s not too hard to build one from scratch:

 

Create a file named “Dockerfile” with the following contents:


FROM python
RUN pip install bzt
RUN apt-get update && apt-get -y install default-jre-headless
RUN bzt -install-tools -o modules.install-checker.include=jmeter
ENTRYPOINT ["bzt"]

 

performance testing with docker

 

Create the Docker image from these instructions by issuing the following command:

sudo docker build . -t bzt1

This will create the image, based on Alpine with Python, Java, and Taurus inside. Also, JMeter will be pre-installed. The name for the image will be “bzt1”, and we’ll use it to point to this image when we run the container.

 

perf testing with docker and taurus

 

Now that you’ve created your Docker image, let’s run a test.


Testing Our Docker Image

 

Testing our own Docker image will be just the same as with the pre-built one, only the name of the image changes.

 

First, run the following command:

 

sudo docker run -it bzt1 http://blazedemo.com

 

performance testing with blazemeter and docker


And it runs and you will see the results in the dashboard:

 

blazemeter with docker


Now that you’ve run a performance test, how can we use Docker within our continuous integration process?


Advantages of Using Docker in Your Continuous Integration Process

Conflicting dependencies are not a problem anymore when using Docker. For example, if you  want to build one project with Java 8, and one project that requires Java 7, Docker can help separate these processes. Resulting images can also be put into a registry for future use and you can use dedicated images for intermediary build steps.


Another advantage is you don’t have to build the image as part of your Jenkins job, you can have several images, eg- one for unit testing, one for integration testing etc.
Docker layer cache speeds things up. You don’t need to start from scratch, which saves development time.Jus be aware of HDD space running out! When you rely heavily on building images as part of your CI integration ,you need to keep on top of maintenance for deleting old images.

 

performance testing with blazemeter and docker


Now we know the advantages, let’s look at how to transfer files in and from your Docker container.


Test-related files need to get into your Docker container, and the resulting files need to get out of the container. In order to do this, you can use the following command to mount the local directory into the container directory: docker run -v /local-dir:/container-dir…


For example: sudo docker run -it -v `pwd`:/bzt-configs bzt1 requests.jmx

 

blazemeter with docker

 

docker and blazemeter

 

We see our JMeter Script running, but how do we get to the JMeter logs, in order to get to the artifacts?

 

For this we have to add the mount command to image launch, by adding the following:

 

sudo docker run -it -v `pwd`:/tmp bzt1 /tmp/requests.jmx

 

performance testing with jmeter and docker

 

I can now check in the JMeter logs, and in the Taurus logs to further investigate any problems

docker perf testing

 

As you can see here, all original files have root permissions:

 

docker performance testing

 

Taurus is a CI friendly tool so deleting root permissions shouldn’t add too much work to your flow.

 

Limitations


While Docker has many advantages, it has its limitations.

 

  • When you run several programs within the same machine you can consume major CPU, and it can affect other programs while running parallel runs, which can distort your results. Also, Docker does not solve HW scaling problems. So if your computer can only run 1,000 Virtual Users, you can’t run more than that when using Docker Containers.
  • Host OS and container still need settings tuned
  • You still need to be able to configure your docker image and your host machine
  • Disposable nature => easy to lose results
  • You will need to store your results, because of the disposable nature of Docker
  • Load Testing Reporting When Using Docker
  • We usually want proper reporting features and those aren’t available through Docker.

 

The first thing to try is an online web report generated from your Taurus test.

 

taurus load tests with docker

 

In Taurus, there is an additional parameter for command-line, called “-report”. This parameter makes Taurus to send all report KPIs into cloud storage. Also, those KPIs are presented live on web report page, which you can share with your colleagues. By using this feature, you avoid Docker’s trap of losing results after test has been completed.

 

load testing with blazemeter and docker

 

If you have the BlazeMeter API key, you can configure Taurus to store results into your personal workspace, by providing API key via “-o modules.blazemeter.token=...” option.

 

Scaling your performance tests in the Cloud with BlazeMeter


With Docker you can only scale with the limitations of your machine. By uploading your tests to BlazeMeter you can scale tests up to 2 million virtual users, and share reporting easily.

 

scaling docker performance tests


BlazeMeter provides full support for JMeter and Taurus, as well as up to 20 other open source tools.


You can share these reports with your team, compare reports from previous tests etc.

 

scaling docker tests in the cloud

To start running your tests in BlazeMeter, just enter your url in the box below to start testing
 

Interested in writing for our Blog?Send us a pitch!

Your email is required to complete the test. If you proceed, your test will be aborted.