Tips For Testing with Docker Containers

The Docker approach to virtualization can provide advantages when testing applications with open source performance testing tools. By making it replicable, performance testing with Docker containers enables sharing the tests between users and replicating the test environment. This can be helpful for sharing knowledge between users of different levels, in addition to saving time by not running unnecessary tests.

This blog post will cover an overview of Docker, the specifics of using Docker containers for generating load, and how to involve open source Taurus for universal automation of performance tests. We’ll also cover how Dockerization solves specific issues of performance tests inside CI, and how to use the cloud for scaling performance tests for massive loads.

Let’s get started.

What is Docker?

Docker is a tool designed to make it easier to create, deploy, and run applications by using containers.

What Are Testing Docker Containers?

Docker containers allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies, and ship it all out as one package.

By doing so, thanks to the container, the developer can rest assured that the application will run on any other machine regardless of any customized settings that machine might have that could differ from the machine used for writing and testing the code.

Key Features of Docker

• An all-inclusive application packaging approach 

Everything you need to run your application sits within a simple container image. You can copy it, share it with your team, develop it locally, and then put it into production.

• Resource-efficient

With Docker, you can run many different applications at the same time. For example, you can run a full set of microservices without them interfering with each other. In the past, applications had dependency conflicts, it was challenging to run different applications on the same operating systems. We used to use virtual machines, but that required a lot of hardware to run. With Docker, you can run isolated processes without running virtual machines, so it is resource-efficient.

• Portable

You can build a Docker container in one environment, and transfer it to another environment.  You can also multiply containers and run it on the same image on many different machines. There’s even Windows-based Docker, so it’s not just for Linux users anymore.

• Disposable

Docker containers require minimal effort to shut them down by destroying everything, allowing you to avoid a huge clean-up process.

• Open Source

Docker is a free open-source solution, which means that you don’t need to pay for licensing, and it is frequently updated with new releases.

Why Test Docker Containers?

There are many benefits to using Docker containers for software performance testing. Firstly, versioned images simplify deployments. This means you can simplify the evolution of your application just by deploying a new version with a new image.

Docker also provides a huge repository of free images for popular apps. We will show one of them when we delve deeper using a Taurus image.

Finally, Docker’s success has led to additional solutions such as Kubernetes and “Cloud-Native” which have really expanded cloud capabilities for developers and testers.

Now that we’ve covered what Docker is,  we're going to dig deeper into how to use open-source Taurus to automate performance tests on Docker containers.

Automating Performance Testing Docker Containers

Use Taurus

Taurus an open-source command-line test automation tool. It’s a wrapper on top of JMeter, Gatling, and many others.

A typical Taurus-based test starts with creating a YAML file describing the test configuration, like the one below:

Then, the YAML file is launched with the help of the Taurus command-line tool, like this:

bzt test.yml

As a result, you see a live dashboard with KPIs of your test, You can also configure Taurus to send these results in a web report.

From here, all the nuances of how your test will behave is based on your YAML file instructions.

The “bzt” command is used to run the tests and you will see the results in real-time on your dashboard.

Before we can run performance testing with Docker containers, we need our Taurus Docker image.

Using a Pre-Built Taurus Image

There are several advantages of using a pre-built Docker image including:

• not needing to install Taurus
• no Python requirement
• no JMeter installation needed
• no Java requirement
• pre-installed Selenium
• as well as Chrome & Firefox

The command to run the pre-built Taurus image is very simple, like this:

sudo docker run -it blazemeter/taurus http://blazedemo.com

Docker will download the image (if it is not present in the cache) and will run Taurus inside the container. Then, Taurus will generate real-time results in its dashboard.                                                      

Make Your Own Docker Image

If you want to have your own, maybe customized, version of Docker image with Taurus, it’s not too hard to build one from scratch:

Create a file named “Dockerfile” with the following contents:

FROM python
RUN pip install bzt
RUN apt-get update && apt-get -y install default-jre-headless
RUN bzt -install-tools -o modules.install-checker.include=jmeter
ENTRYPOINT ["bzt"]

Create the Docker image from these instructions by issuing the following command:

sudo docker build . -t bzt1

This will create the image, based on Alpine with Python, Java, and Taurus inside. Also, JMeter will be pre-installed. The name for the image will be “bzt1”, and we’ll use it to point to this image when we run the container.

Now that you’ve created your Docker image, let’s run a test.

Test Your Docker Image

Testing our own Docker image will be just the same as with the pre-built one, only the name of the image changes.

First, run the following command:

sudo docker run -it bzt1 http://blazedemo.com

And it runs and you will see the results in the dashboard:

Now that you’ve run a performance test with a Docker container, how can we use Docker within our continuous integration process?

Advantages of Using Docker Containers

Conflicting dependencies are not a problem anymore when using Docker. For example, if you want to build one project with Java 8, and one project that requires Java 7, Docker can help separate these processes. The resulting images can also be put into a registry for future use and you can use dedicated images for intermediary build steps.

Another advantage is you don’t have to build the image as part of your Jenkins job, you can have several images, eg- one for unit testing, one for integration testing etc.
Docker layer cache speeds things up. You don’t need to start from scratch, which saves development time. Just be aware of HDD space running out! When you rely heavily on building images as part of your CI integration, you need to keep on top of maintenance for deleting old images.

Now we know the advantages, let’s look at how to transfer files in and from your Docker container.

Test-related files need to get into your Docker container, and the resulting files need to get out of the container. In order to do this, you can use the following command to mount the local directory into the container directory: docker run -v /local-dir:/container-dir…

For example: sudo docker run -it -v `pwd`:/bzt-configs bzt1 requests.jmx

We see our JMeter Script running, but how do we get to the JMeter logs, in order to get to the artifacts?

For this we have to add the mount command to the image launch, by adding the following:

sudo docker run -it -v `pwd`:/tmp bzt1 /tmp/requests.jmx

I can now check in the JMeter logs, and in the Taurus logs to further investigate any problems

As you can see here, all original files have root permissions:

Taurus is a CI-friendly tool, so deleting root permissions shouldn’t add too much work to your flow.

Limitations of Docker Containers

While Docker has many advantages, it has its limitations.

• When you run several programs within the same machine you can consume major CPU, and it can affect other programs while running parallel runs, which can distort your results. Also, Docker does not solve HW scaling problems. So if your computer can only run 1,000 Virtual Users, you can’t run more than that when using Docker Containers.
• Host OS and container still need settings tuned
• You still need to be able to configure your docker image and your host machine
• Disposable nature => easy to lose results
• You will need to store your results, because of the disposable nature of Docker
• Load Testing Reporting When Using Docker
• We usually want proper reporting features and those aren’t available through Docker.

The first thing to try is an online web report generated from your Taurus test.

In Taurus, there is an additional parameter for the command-line, called “-report”. This parameter makes Taurus to send all report KPIs into cloud storage. Also, those KPIs are presented live on the web report page, which you can share with your colleagues. By using this feature, you avoid Docker’s trap of losing results after the test has been completed.

If you have the BlazeMeter API key, you can configure Taurus to store results into your personal workspace, by providing API key via “-o modules.blazemeter.token=...” option.

Test Docker Containers With BlazeMeter

With Docker, you can only scale with the limitations of your machine. By uploading your tests to BlazeMeter you can scale tests with Docker containers up to 2 million virtual users, and share reporting easily.

BlazeMeter provides full support for JMeter and Taurus, as well as many other open-source tools.

You can share these reports with your team, compare reports from previous tests, etc.

START TESTING NOW

Related Resources

• JMeter Performance Testing
• How to Use Docker with JMeter