Which Language Bindings should be used with Selenium?

Selenium is the de facto standard for Test Automation of web browser applications. Selenium WebDriver API has various features to emulate final user interactions with web browser to help Test Automation Engineers.

The secret of Selenium’s success is due to its open source philosophy that allows it to be used to support more browser functionalities and ever-evolving developing languages.

Which language bindings should be used with Selenium? This is one of the first decisions taken on each new Test Automation project. In this article we will discuss some (of many!) language bindings that Selenium offers: Java, Python and Robot Framework. Moreover during this paper we present a running test case approached with different Selenium bindings.

What do we mean by Language Bindings?

The term language binding generally refers to mapping a software library/API to another language in the manner to be used in two different developing ecosystems and/or environments with the same functionalities.

The number of Selenium users has grown significantly over time. Many users means many development stacks to be supported, and community documentation to use Selenium with different languages, using WebDriver API as the main functionality to be shared via bindings.

From the picture above you can see:

• On the left some examples of languages where is possible to find a Selenium bindings
• On the right there are WebDriver implementation according target browser/operating system 
• In the middle there is WebDriver API that is the contract to be granted among bindings.

Java was first released in 1995 as an alternative to complex programming languages (mainly C/C++). Pillars of its philosophy are: Object-Oriented Programming, variable static typing, automatic memory management, architecture neutral and multi-threading.

Python’s first release was in 1990 after around a decade of incubation. Main pillars of its philosophy are: multi paradigms (OOP, functional, etc.), variable dynamic typing, automatic memory management, easy to read and nice to modify.

The Robot Framework idea was born in 2005 and after some years under the Nokia umbrella it becomes an open source project. It was developed as a Python extension with both Java and .NET implementation. At the heart of its philosophy there is keyword-driven testing, where each action is described via keyword closer to natural language to avoid complexity.

Generic test case to be implemented

In this section we describe a generic test case used as comparative bench for bindings. For simplicity, test cases have only a few steps so we can better focus on the language bindings comparison.

Test Case Setup

The Chrome instance is created under Selenium WebDriver and it’s ready to be used.

Selenium WebDriver displays the label to inform that it is running the Test Automation session.

Go to under test page

The first step of our test requires that Chrome loads the web page where the test will be executed

http://the-internet.herokuapp.com/hovers

The above picture displays that the page is correctly loaded.

Find first avatar picture

Find first showed avatar and highlight it with red dashed line.

The above picture displays the avatar correctly highlighted.

Mouse over first avatar picture

Simulate mouse over the first avatar picture and highlight displayed name with blue dashed line.

Name under avatar is visible and highlighted as shown above.

Test Case Teardown

After test completion the Chrome instance is closed on your local PC.

For all the bindings analyzed in this paper you can find the complete test code available on Github at the following link.

To execute locally all or one single binding implementation, you need to take these precondition steps first:

• clone repository locally (required local Git application)
• install Chrome browser with proper procedure
• install and execute locally Chrome WebDriver implementation (see this link).

Comparing Selenium Bindings

Before we get started, I will explain the specific preconditions to understand and execute each implementation.

Java

A test case implemented in Java is available directly here on GitHub. To be able to run this implementation knowing only Java and Selenium API are not enough. You need additional skills with JUnit5 test framework in order to run Test Automation actions like setup, teardown and assertion.

To execute this test case from your source code it's necessary to provide additional software libraries (e.g. JUnit5, Selenium) both for compilation and for the execution phases. You can use Maven for this, which provides one shell command to handle all necessary functionality for test cases and resolving required dependencies using your network connection.

Python

Test cases implemented in Python are available directly here on GitHub. As already seen in the previous section, only Python and Selenium API knowledge is not enough and you need an additional skill on unittest test framework to run Test Automation actions like setup, teardown and assertion.

To execute this Python test case you need to first provide some software library like unittest and Selenium. The first one typically is provided natively in each Python installation whereas for Selenium we need to use a package installer like pip.

Robot Framework

Test case implemented in Robot Framework are available directly here on GitHub. No additional knowledge is required except Robot Framework and Selenium API. Robot Framework already embeds Test Automation actions like setup, teardown and assertion.

Robot Framework is developed as a Python library, so it can be installed easily with a package installer like pip. The most recent dependency for Selenium with Robot Framework that can be installed with package manager (see here).

Test case implementation analysis

In this section we will compare three independent pieces of code functionally of the same test automation case. Each script produces identical requirements coverage using Selenium library. We will then compare the code, focusing on how Selenium API is employed and cover pros and cons in every implementation in term of:

• readability
• length
• structure
• maintenance

Readability

Java and Python are two of the most well known development languages in the world. They are also generic purpose languages and require an additional software layer to be used for test automation. This additional layer can add issues of readability, as opposed to Robot Framework, which is designed for Test Automation and its semantic is very descriptive in this context.

Java is strongly object oriented and consequently every Selenium functionality is organized by dedicated entity (e.g. JavascriptExecutor, TakesScreenshot, etc.). Python binding for Selenium seems less object oriented and some functionality can be accessed directly (e.g. execute javascript) without passing through an intermediary object.

WebDriver instance creation is relatively easy for all of the language bindings that we are investigating, but in cases of specific configuration (e.g. proxy) creation can involve burden of code that affects readability. At this point it is necessary to provide specific additional configuration:

• Java is handled by proposed objects (e.g. ChromeOptions, etc.)
• Python, it can be handled in pythonic way with dictionary of strings
• Robot Framework, uses inline Python code for specific config

To provide more details on Robot Framework, we can easily customize WebDriver creation but there is a limit based on configuration type. To bypass this limit the only way is to add Python code, and this affects readability.

With Robot Frameworks code there is a general focus on what test logic will do, whereas with Java/Python code there is a focus on how the test actions are coded. Robot Framework has limits when a test action cannot be obtained with existing keyword, and then you need to mix Python code.

Code Length

Comparing code length is a simple check and it can provide some interesting data. Java version is longest compared to others and Python version is shorter than Java by around 30%. The main reasons are:

• each block parentesys used in Java will take one line
• Selenium API in Java requires some intermediate object to setup WebDriver than Python
• JUnit5 requires some lines to identify functional block via annotation.

Looking at Robot Framework version the lines number is reduced by around 50% compared to other versions.

This is due to:

• syntax dedicated for Test Automation
• using descriptive actions instead of declarative ones.

Structure

An additional note about two of our “contenders”: Java and Python were both created within five years of each other, and they share some concepts, as well as some big differences.

Java code for Test Automation is more lengthy than Python, and consequently the same functional code in Python can be described with significantly less characters, as shown below:

• command line terminator in Java with character “;”
• command block delimiter in Java with characters “{” and “}”
• type bindings and declaration, in Java every variable must explicitly declares its type during creation

Python code has a more complex structure, eg:

• code blocks are defined using character “:” and through indentation
• variable type is determined dynamically at run time.

With Python you can write more concise code and potentially this can result in your code being less readable. Based on this observation it’s important that during Python coding an Integrated Development Environment that support your Test Automation Engineer to handle both code block and variable type definition properly.

Robot Framework file structure is tailored for Test Automation and each standard code file represents a test suite with multiple test cases. The Code file is organized in sections recognizable by three asterix notation (e.g. Settings, Variables, Test Cases, Keyworks). The Importing library in our example is reduced to one line and variables are easily identified because they are marked with “$”. Test cases and keywords share the same code structure and it’s section that identify implemented code scope.

Maintenance

In Test Automation the same test suite can be developed by multiple colleagues and it’s not uncommon that the maintenance activities are executed by different people than the original engineer. The tool/process designed for developing the tool chain becomes critical because it aims to minimize maintenance cost. This cost can increase after every software cycle in order to answer to more strict requirements in terms of effort and coverage.

Java and Python at a high level share the same maintenance complexity and therefore a similar required effort of original code quality and code maintenance.

Robot Framework thanks to its dedicated file structure for Test Automation provides a great help during maintenance. What is entity of maintenance? When is it necessary to modify/remove a test case or keyword? Being closer to natural language is a great help because we can simply remap business rules into coded script without adding an additional layer of complexity.

Conclusion

Regardless of your role in Test Automation, the integration of Selenium is an important step to increase quality and efficiency in each release. You can read more about running Selenium GUI Functional Tests in BlazeMeter, or request a demo to find out how BlazeMeter can help you with your Test Automation projects.

START TESTING NOW