Performance Testing with Docker Containers
The Docker approach to virtualization can provide advantages when testing applications with open source performance testing tools. By making it replicable, Docker enables sharing the tests between users and replicating the test environment. This can be helpful for sharing knowledge between users of different levels, in addition to saving time by not running unnecessary tests.
This blog post will cover an overview of Docker, the specifics of using Docker containers for generating load, and how to involve open source Taurus for universal automation of performance tests. We’ll also cover how dockerization solves specific issues of performance tests inside CI, and how to use the cloud for scaling performance tests for massive loads.
Let’s get started.
What is Docker, and What Are Docker Containers?
Before we get into how to run performance tests with Docker containers, let’s get an overview on what Docker is.
Docker is a tool designed to make it easier to create, deploy, and run applications by using containers. Containers allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies, and ship it all out as one package. By doing so, thanks to the container, the developer can rest assured that the application will run on any other machine regardless of any customized settings that machine might have that could differ from the machine used for writing and testing the code.
Key features of Docker
- An all inclusive application packaging approach
Everything you need to run your application sits within a simple container image. You can copy it, share it with your team, develop it locally, and then put it into production.
With Docker, you can run many different applications at the same time. For example, you can run a full set of microservices without them interfering with each other. In the past, applications had dependency conflicts, it was challenging to run different applications on the same operating systems. We used to use virtual machines, but that required a lot of hardware to run. With Docker you can run isolated processes without running virtual machines, so it is resource efficient.
You can build a Docker container in one environment, and transfer it to another environment. You can also multiply containers and run it on the same image on many different machines. There’s even Windows based Docker, so it’s not just for Linux users anymore.
Docker containers require minimal effort to shut them down by destroying everything, allowing you to avoid a huge clean up process.
- Open Source
Docker is a free open source solution, which means that you don’t need to pay for licensing, and it is frequently updated with new releases.
Benefits of Using Docker
There are many benefits for using Docker. Firstly, versioned images simplify deployments. This means you can simplify the evolution of your application with deploying just a new version with a new image.
Docker also provides a huge repository of free images for popular apps. We’ll show one of them when we delve deeper using a Taurus image.
Finally, Docker’s success has led to additional solutions such as Kubernetes and “Cloud-Native” which have really expanded cloud capabilities for developers and testers.
Now that we’ve covered what Docker is, we're going to dig deeper into how to use open source Taurus to automate performance tests on Docker containers.
What is Taurus?
Taurus an open source command-line test automation tool. It’s a wrapper on top of JMeter, Gatling and many others.
A typical Taurus-based test starts with creating a YAML file describing the test configuration, like the one below:
Then, the YAML file is launched with the help of Taurus command-line tool, like this:
As a result, you see a live dashboard with KPIs of your test, You can also configure Taurus to send these results in a web report.
From here, all the nuances of how your test will behave is based on your YAML file instructions.
The “bzt” command is used to run the tests and you will see the results in real-time on your dashboard.
Before we can run a performance test, we need our Taurus Docker image.
Using a Pre-Built Taurus Image
What are the advantages of using a pre-built image?
There are several advantages of using a pre-build Docker image including:
- not needing to install Taurus
- no Python requirement
- no JMeter installation needed
- no Java requirement
- pre-installed Selenium
- as well as Chrome & Firefox
The command to run pre-built Taurus image is very simple, like this:
sudo docker run -it blazemeter/taurus http://blazedemo.com
Docker will download the image (if it is not present in the cache) and will run Taurus inside the container. Then, Taurus will generate real-time results in its dashboard..
Making Your Own Docker Image
If you want to have your own, maybe customized, version of Docker image with Taurus, it’s not too hard to build one from scratch:
Create a file named “Dockerfile” with the following contents:
RUN pip install bzt
RUN apt-get update && apt-get -y install default-jre-headless
RUN bzt -install-tools -o modules.install-checker.include=jmeter
Create the Docker image from these instructions by issuing the following command:
sudo docker build . -t bzt1
This will create the image, based on Alpine with Python, Java, and Taurus inside. Also, JMeter will be pre-installed. The name for the image will be “bzt1”, and we’ll use it to point to this image when we run the container.
Now that you’ve created your Docker image, let’s run a test.
Testing Our Docker Image
Testing our own Docker image will be just the same as with the pre-built one, only the name of the image changes.
First, run the following command:
sudo docker run -it bzt1 http://blazedemo.com
And it runs and you will see the results in the dashboard:
Now that you’ve run a performance test, how can we use Docker within our continuous integration process?
Advantages of Using Docker in Your Continuous Integration Process
Conflicting dependencies are not a problem anymore when using Docker. For example, if you want to build one project with Java 8, and one project that requires Java 7, Docker can help separate these processes. Resulting images can also be put into a registry for future use and you can use dedicated images for intermediary build steps.
Another advantage is you don’t have to build the image as part of your Jenkins job, you can have several images, eg- one for unit testing, one for integration testing etc.
Docker layer cache speeds things up. You don’t need to start from scratch, which saves development time.Jus be aware of HDD space running out! When you rely heavily on building images as part of your CI integration ,you need to keep on top of maintenance for deleting old images.
Now we know the advantages, let’s look at how to transfer files in and from your Docker container.
Test-related files need to get into your Docker container, and the resulting files need to get out of the container. In order to do this, you can use the following command to mount the local directory into the container directory: docker run -v /local-dir:/container-dir…
For example: sudo docker run -it -v `pwd`:/bzt-configs bzt1 requests.jmx
We see our JMeter Script running, but how do we get to the JMeter logs, in order to get to the artifacts?
For this we have to add the mount command to image launch, by adding the following:
sudo docker run -it -v `pwd`:/tmp bzt1 /tmp/requests.jmx
I can now check in the JMeter logs, and in the Taurus logs to further investigate any problems
As you can see here, all original files have root permissions:
Taurus is a CI friendly tool so deleting root permissions shouldn’t add too much work to your flow.
While Docker has many advantages, it has its limitations.
- When you run several programs within the same machine you can consume major CPU, and it can affect other programs while running parallel runs, which can distort your results. Also, Docker does not solve HW scaling problems. So if your computer can only run 1,000 Virtual Users, you can’t run more than that when using Docker Containers.
- Host OS and container still need settings tuned
- You still need to be able to configure your docker image and your host machine
- Disposable nature => easy to lose results
- You will need to store your results, because of the disposable nature of Docker
- Load Testing Reporting When Using Docker
- We usually want proper reporting features and those aren’t available through Docker.
The first thing to try is an online web report generated from your Taurus test.
In Taurus, there is an additional parameter for command-line, called “-report”. This parameter makes Taurus to send all report KPIs into cloud storage. Also, those KPIs are presented live on web report page, which you can share with your colleagues. By using this feature, you avoid Docker’s trap of losing results after test has been completed.
If you have the BlazeMeter API key, you can configure Taurus to store results into your personal workspace, by providing API key via “-o modules.blazemeter.token=...” option.
Scaling your performance tests in the Cloud with BlazeMeter
With Docker you can only scale with the limitations of your machine. By uploading your tests to BlazeMeter you can scale tests up to 2 million virtual users, and share reporting easily.
BlazeMeter provides full support for JMeter and Taurus, as well as up to 20 other open source tools.
You can share these reports with your team, compare reports from previous tests etc.
To start running your tests in BlazeMeter, just enter your url in the box below to start testing