How to Scale Selenium Tests in a Kubernetes Cluster

We live in a world where nothing sits still. In today’s rapidly changing IT world, companies strive towards scalable and fault-tolerant distributed systems, and not only in regards to software development. A lot of digital transformations are going on in the Quality Assurance area as well. Companies are ‘shifting to the left’, focusing more on how they can automate their test processes to make them quicker, easier to configure and more efficiently deployed.

Selenium Grid: Advantages and Disadvantages

When it comes to web-application automation, Selenium Grid has become the first fundamental part of the revolutionizing tool sets, allowing to perform Selenium tests on different machines and with multiple browsers at the same time. In the good old days you would just plug a couple of computers together, run a couple of Selenium agent scripts on each of those machines and that was that. But nothing stays relevant forever in its initial form, and the wildly popular automation solution ‘Selenium Grid’ has been given a breath of fresh air.

But before jumping into the most interesting part, let’s think about what was not perfect in Selenium Grid’s infrastructure in its original form:

• The Grid has an ‘Achilles heel’ - Selenium Hub: The hub-and-node concept has a serious weak spot - Selenium Hub. Selenium Hub is a single browser access point that makes all nodes unavailable when the hub goes down.
• Selenium Grid doesn’t scale well: It’s a widely known fact that the Selenium hub can only handle a limited number of connected nodes.
• Stale infrastructure for ongoing test runs: Selenium Grid nodes usually end up in a poor state left up and running for quite a long time. Hence, the node restart is exactly what the doctor ordered, but it can only be done manually.
• Frustrating version synchronization: Keeping the project up-to-date with the latest Selenium release and having it synchronized with the browser and webdriver version can cause difficulties and takes ages. Grid maintenance and updates are a time consuming node-by-node process.

Introducing Zalenium

This is where Zalenium comes in! According to its official page, Zalenium is a flexible and scalable container based Selenium Grid, with video recording, live preview, basic auth & a dashboard. Moreover, it has out-of-the-box Docker and Kubernetes integration. This fact makes Zalenium an attractive choice to get Selenium based infrastructure up and running.

When you dig into Zalenium, you will realize all the benefits it brings to web automation.

Selenium/Zalenium -Kubernetes Integration

Out-of-the-box integration with Kubernetes significantly reduces test execution time by spinning up Docker containers of the necessary web browsers and running them in parallel. This actually means horizontal scaling of your tests. Moreover, it lets you avoid the headache of Grid set up and maintenance. Therefore, you don’t have to deal with containers/pods directly. Instead, Kubernetes automatically manages the desired state of the system.

Running Zalenium in a Kubernetes cluster becomes a powerful solution to get Selenium based infrastructure up in a more efficient and simpler way. This is true especially if you’ve already orchestrated, shipped and run the company’s software with Kubernetes.

Let’s dive into Zalenium and take advantage of its integration with Kubernetes together. The desired configuration is represented in the diagram below.

Kubernetes Cluster configuration on the local machine

Creating our Selenium based Zalenium-Kubernetes Test

The main prerequisite for this configuration is to have a running Kubernetes cluster. To experiment with Kubernetes locally, you can use Minikube. Minikube is a tool that allows spinning up a single-node virtual cluster on your machine.

The easiest way to set up Minikube on a Mac is to execute the following brew command (the installation steps for other platforms you can find here on GitHub):

Alright!

In order to interact with the cluster, we can trigger the Kubernetes API via Kubectl. This tool allows running containerized applications on top of the Kubernetes cluster. If you setup Minikube with the command above, it downloads Kubectl as a dependency during the installation process.

You just need to make sure that the sufficient version has been installed:

For example, the Kubectl version I use is

In case of separate Kubectl installation, use the instructions provided in the official documentation.

After we have all the necessary tools installed, we can instantiate our own Kubernetes cluster.

Just run the following:

It will start a local Kubernetes cluster:

Check the status and IP of the cluster:

Running Zalenium on Top of the Kubernetes Cluster

In order to spin up Zalenium in Kubernetes, we need to pull Zalenium and Selenium Docker images inside the cluster. For this to be possible, we should switch Docker to Kubernetes context by executing the docker-env command:

This will provide a cmd to use Docker from the command line of your local machine against the Kubernetes VM.

Let’s download the Selenium and Zalenium Docker images for spinning them up in the cluster.

As was mentioned before, guys from Zalando already prepared all necessary scripts to run Zalenium with Kubernetes using a one line command.

All the ‘magic’ scripts and configurations are located in the Zalenium GitHub repo:

So, create a resource from the cloned files:

It will start Kubernetes pods under a namespace ‘Zalenium’.

Check that the service is up by listing all the container images in the cluster.

Moreover, you can open the  Kubernetes dashboard to verify all the running services under the ‘Zalenium’ namespace:

An Advanced (alternative) Way to Start Zalenium in Kubernetes using Helm

In order to simplify deploying applications to Kubernetes, we can use its own package manager  Helm, which helps with the installation and maintenance of any software on the cluster.

After Minikube is set up, install the Kubernetes package manager.

Initialize Helm on both client and server sides

Helm packages, aka charts, might be installed in several ways. Zalenium developers have already provided a chart that bootstraps a Zalenium deployment on a Kubernetes cluster.

Navigate to the helm directory in the cloned zalenium repo:

The chart’s installation command might include arguments for Zalenium customization. For example, in the string below the desired number of pods is increased to 3.

Great! All Zalenium pots are running on top of the Kubernetes cluster!

Web-based Test Execution with Zalenium

After you set up Zalenium, open the service zalenium/zalenium in the default browser via the command:

minikube service zalenium --namespace zalenium

You can get a URL of the service deployed on Kubernetes by the following command:

Now all you need is to point the Selenium webdriver configuration to use this URL in order to run your tests with Zalenium.

If you want to practice Zalenium with an already existing test framework, let’s use the project from Yuri Bushnev ‘s blog about Top 15 UI Test Automation Best Practices.

You just need to modify the base.properties file in order to integrate the framework with Zalenium.

Now you are ready to run Selenium tests with the new configuration:

Zalenium also gives a great possibility of a live preview of your running tests. Moreover, you can interact with the dockerized container via VNC to watch a life stream of the test during the execution process. It can be accessed at the URL: 

And it will look like this:

Moreover, Zalenium with its nice ‘video/logs recording’ feature enables to monitor the test from a browser. Just check the dashboard by the URL:

Congratulations! You’ve just practiced Selenium tests running on the local Kubernetes cluster using Zalenium! But please notice that Minikube is a limited version of Kubernetes. This essentially means that you can not really scale your Selenium Grid by running Minikube (It’s restricted by the power of your local machine). But the installation process of Zalenium on Kubernetes is the same as on Minikube, so you can get a good understanding of this workflow.

The best way to run your test would be to deploy Kubernetes on a cloud provider like AWS and enjoy a  real cloud-native scalable Selenium Grid. If you are ready to try out a cloud service,  Amazon EKS User Guide has plenty of resources to get you started.

The combination of Zalenium and Kubernetes has become a powerful solution for testing web-based applications in a more efficient manner. In the QA space test automation is a rapidly growing area and shifting more towards a cloud-native approach.

Use BlazeMeter’s End User Experience Monitoring to easily run Selenium tests as part of a load test, to get insights into your front end performance under load. You can either add your URL after uploading your load testing script and we will run it in Selenium for you, or you can upload a Selenium script. The test results enable you to monitor your front end performance under a load: Get combined KPIs for HTTP/JMeter response times and for GUI page load times. Analyze requests in a Waterfall chart with webpage screenshots to get a complete client side breakdown.

START TESTING NOW