Robot Framework: The Ultimate Guide to Running Your Tests

This blog post will give an overview of the Robot Framework and explain how to create your Selenium tests in Robot as well. Get ready for these insightful tips and best practices, and finally, learn how to run Robot in Taurus.

What is Robot Framework Used For?

Robot Framework is an open source test automation framework for acceptance testing and acceptance test-driven development (ATDD). Robot Framework also enables teams to do Selenium testing with a keyword driven approach.

Acceptance testing determines whether a system answers the acceptance criteria, defined by user needs and requirements. When we execute acceptance testing, we verify the whole system as a single unit on a high level, to understand if the system under test can be used by the end users.

Usually, test engineers act as system users by executing steps and scenarios that come from requirements and business processes, by forming a set of predefined keywords. This approach to testing, which is based on a set of keywords that can be reused across all tests, is called keyword-driven.

Robot Framework utilizes this keyword-driven approach in a very good fashion. It doesn’t limit you to any approach or format to keywords: if you wish to use keywords on a high level, then it’s ok. You wish to manipulate variables and do programming with keywords? Please, go ahead. You like Gherkin syntax and want to use it to describe your tests? Feel free to do it.

Additionally, Robot Framework has a rich ecosystem of internal and external libraries that consist of many useful keywords that you can reuse in your ecosystem. But don’t worry, if you didn’t find a keyword you need, you can develop a new library yourself. You can develop tests with the Robot Framework on Java and Python. In the blog post, we will use Python since the framework itself is implemented on Python and there are more external libraries on Python.

Sounds like a great framework, doesn’t it? Let’s have a look at how to use it. You can follow the test source code usage of the Robot Selenium library. You can find the test version with custom Selenium keywords, too.

Does Robot Framework Use Python?

The short is answer is yes. Because the robot framework is built on top of Python, the integration process is simple. Using the two together offers significant flexibility to your code and "helps you create your own custom keywords tha are better suited for your task than their Robot counterparts."

Robot Framework Prerequisites

First, we need to follow the requirements:

1. Install Python 3.6.x or above. (Get the download.) Code snippets in the blog post will be given for version 3.6.4. If there is any difference for version 3.6.4. It’s up to the reader to choose which version to install.

2. Install the Python package manager (pip). It can be downloaded from its download page. All further installations in the blog post will make use of pip so it’s highly recommended to install it.

3. Download a development environment. The PyCharm Community edition will be used in the blog post. You can use any IDE of your choice since code snippets are not IDE dependent.

Ready to start upgrading your testing using the Robot framework? Start testing with BlazeMeter for FREE today!

Start Testing Now

Creating your Robot Framework Project

Now, let’s create our project.

1. Create the project in PyCharm IDE with File -> New Project.

In Windows:

In MacOS:

2. Specify the location for the project (the last part of the path will be the project’s name)

When developing a Python application, it’s good practice to isolate its dependencies from others. With this, you can be sure that you are using the right version of the library in case there are multiple versions of it in your PYTHON_PATH. (The PYTHON_PATH variable is used in python applications to find any dependency that is declared via import statement in python modules).

3. To do this, you need to install the virtual environment.

• Install the Virtual Environment tool with the command pip install virtualenv in the prompt.
• In the project root directory, create the environment with virtualenvvenv in the prompt where venv is the environment name.

4. If the environment is activated, your prompt will be prefixed with the environment’s name as below:

5. If the environment has been activated successfully, install the Robot Framework with pip:

Creating Your Selenium Test in the Robot Framework

Once you are done with the prerequisites, let’s choose an application that you are going to automate tests for. 

Next, you will create a simple test case that allows you to search for any flight. This will be a UI test; hence we need a tool that will help you manipulate the elements on a web page. Selenium WebDriverwill suit this purpose as it’s open-source, stable and easy to use. Selenium WebDriver gives us as test developers a simple programming interface to interact with graphical elements on web pages. The only thing you need to do is to choose the appropriate WebDriver depending on the browser you would like to use.

In general, any test forms a test case. In the Robot Framework, test cases are created in test cases files that can have any extension from the following: .html, .xhtml, .htm, .tsv, .txt, .rst, .rest or .robot

Any test cases file forms a test suite file, regardless of the number of tests it has. Sometimes you need to group test suite files into higher-level suites. This can be achieved by creating directories. You can create a tree of directories with any depth by forming a logical structure. To do that you should follow the application business logic or any other grouping criteria.

Implementing Keywords in Your Test

A test case is composed from keywords. You can use keywords created on your own (they are named as user keywords) or import keywords from Robot Framework libraries.

It’s up to a test developer to choose which one to use but keep in mind the following recommendation: in a test case try to avoid any programming and low-level keywords by using high-level user keywords instead. It will make such a test case easy to read and easy to understand.

How should you apply this recommendation? Consider the following issue: when automating UI tests with Selenium and the Robot Framework, you need to choose between already developed Selenium keywords or your own high-level business keywords to interact with web elements. The choice depends on answers to the following questions:

1. Will other people, besides the test developers, implement new tests and keywords?
2. Is the business logic simple and not complex and do you not need to use object-related models?

If the answer to these two questions is yes, then feel free to use keywords that were already developed. But, if you need to implement complex business logic and you can’t do it without a business model, then encapsulate manipulation with web pages in internal classes and create keywords for business steps instead.

The Selenium2Library from the Robot Framework will be used as there will not be any complex business logic — any flow can be implemented with the available keywords.

Have a look at the test below:

Robot Framework Selenium Test Example

To book a flight we need to perform the following steps:

1. Open a http://blazedemo.com/ webpage.
2. Choose any departure city from a drop-down list.
3. Choose any destination city from a drop-down list.
4. Click on “Find Flights” button.
5. Verify if there are flights found on Flights search result page.

To implement the test, the steps above need to be converted to keywords. If you tried to implement a test case without user defined keywords, your test would look like the following (consider that we are using the SeleniumLibrary from the Robot Framework).

search_flights.robot:

*** Settings ***
Library  SeleniumLibrary

*** Test Cases ***
The user can search for flights
    [Tags]	    search_flights
    Open browser    http://blazedemo.com/   Chrome
    Select From List By Value   xpath://select[@name='fromPort']  Paris
    Select From List by Value   xpath://select[@name='toPort']    London
    Click Button    css:input[type='submit']
    @{flights}=  Get WebElements    css:table[class='table']>tbody tr
    Should Not Be Empty     ${flights}
    Close All Browsers

To be able to use keywords from external libraries (like SeleniumLibrary) you need to import it. This should be done in the “Settings” section of the code in Robot with the setting “Library”. The “Settings” section is used not only for importing external libraries and resources, but also for defining metadata for test suites and test cases. You will come back to the “Settings” section later in the blog post.

Underneath the “Settings” section there is the “Test Cases” section where you should add all the test cases within a test cases file. As you can see, there are keywords used from SeleniumLibrary to open the browser and to select appropriate values from departure and destination drop-down lists.

In the Robot Framework, any keyword can accept any number of arguments. In this case, for example, the keyword “Open browser” accepts two arguments: the URL to open and the browser this URL should be opened in. The keyword “Select From List by Value” accepts the selector of the web element as the first argument, and accepts the value to select as the second argument.

The Tags section is used to assign a logical group to a test case. Furthermore, it can be used to execute tests only with the specified tag. I will explain how to achieve this later in the blog post.

There is one more thing that needs to be explained in more detail here. The result of the “Get WebElements” keyword is assigned to the variable “@flights”. You can use the variable to describe keyword arguments, to store a value in a test suite or to save the results of keyword execution. The names of variables are case-insensitive, and as well as names of keywords, and spaces and underscores are ignored. The test case can be re-written with usage of variables in the following way:

*** Settings ***
Library  SeleniumLibrary

*** Variables ***
${URL}          http://blazedemo.com/
${BROWSER}      Chrome

*** Test Cases ***
The user can search for flights
    Open browser    ${URL}   ${BROWSER}

Such styling adds more readability to the test case. Variables can be scalar (with the $ prefix), lists (with the @ prefix, dictionaries (with the & prefix) and environment (with the % prefix).

After saving the available flights as web elements in the list “flights”, you can use the built-in keyword “Should Not Be Empty” to verify there is at least one flight found. As a last step we close all opened browsers.

Now, it’s high time to make some improvements using Robot Framework’s capabilities. If you want to add more test cases to the test suite, you will notice that anytime you want to interact with the page, you need to open it. After the test passed, you need to close the browser. With the help of the “Settings” section that was mentioned previously, you can manage these tear up and tear down operations.

Updated test case:

*** Settings ***
Library  SeleniumLibrary
Suite Setup     Open browser    ${URL}   ${BROWSER}
Suite Teardown  Close All Browsers

*** Variables ***
${URL}          http://blazedemo.com/
${BROWSER}      Chrome

*** Test Cases ***
The user can search for flights
    [Tags]	   search_flights
    Select From List By Value   xpath://select[@name='fromPort']  Paris
    Select From List by Value   xpath://select[@name='toPort']    London
    Click Button    css:input[type='submit']
    @{flights}=  Get WebElements    css:table[class='table']>tbody tr
    Should Not Be Empty     ${flights}

As you can see, the values  open browser and closing all browsers have changed to suite setup and suite teardown, respectively.

Creating Keywords

As another improvement, you can develop custom keywords to select departure and destination cities and to find flights.

Search_flights_keywords.robot:
 

*** Settings ***
Library     SeleniumLibrary


*** Variables ***
${URL}          http://blazedemo.com/
${BROWSER}      Chrome

*** Keywords ***

Open Home Page
    Open browser    ${URL}   ${BROWSER}

Close Browsers
    Close All Browsers


Select Departure City
    [Arguments]      ${departure_city}
     Select From List By Value   xpath://select[@name='fromPort']  ${departure_city}

Select Destination City
    [Arguments]      ${destination_city}
    Select From List by Value   xpath://select[@name='toPort']    ${destination_city}

Search For Flights
    Click Button    css:input[type='submit']

There are available Flights
    @{flights}=  Get WebElements    css:table[class='table']>tbody tr
    Should Not Be Empty     ${flights}

All keywords we are intending to use should be created in the “Keywords” section.  You can notice that we don’t have any test cases in the file. Such files are considered as resource files and they have slightly different properties than test cases files: in the “Settings” section you cannot use metadata settings such as SuiteSetup, SuiteTearDown or Tags settings. You can only use the import settings (Library, Resource, Variables) and Documentation.

Going back to the created keywords: In “Select Departure City” and “Select Destination City” keywords we have an [Arguments] property, which means these keywords require arguments. A keyword can have any number of arguments but it’s advised not to overload it with too many. If a keyword has many arguments it’s harder to understand what it does and it becomes prone to errors.

This test cases file "search_flights.robot" was changed to reflect the modifications:
 

*** Settings ***
Resource        search_flights_keywords.robot
Suite Setup     Open Home Page
Suite Teardown  Close Browsers


*** Test Cases ***
The user can search for flights
   [Tags]	   search_flights
    Select Departure City   Paris
    Select Destination City    London
    Search For Flights
    There are available Flights

Resources files are loaded via the setting “Resource”. Now this test looks more elegant and it is easy to understand what the test steps are.

Initialization Files

The next thing we can do is to add a setup suite and a teardown suite to the test suite initialization file. As mentioned previously in the blog post, a test cases file forms a test suite. But test suite files can be put to a higher-level test suite by creating directories. On its own, a directory cannot have the setup and teardown information, but the Robot Framework has initialization files for that. The name of the initialization file should be __init__.ext, where ext should be one of the supported formats.

Let’s create a directory for our test suite and name it search_flights.

Our initialization file will be search_flights/__init__.robot (don’t confuse it with python __init__.py)
 

*** Settings ***
Suite Setup     Open Home Page
Suite Teardown  Close Browsers
Resource        search_flights_keywords.robot

Like resource files, initialization files cannot have test cases and not all settings are supported. Variables and keywords that were created or imported will not be available in the lower level suites. Use initialization files to provide common tags, documentation and setup/teardown operations.

With the initialization file, our test case looks even simpler:

*** Settings ***
Resource        search_flights_keywords.robot


*** Test Cases ***
The user can search for flights
    Select Departure City   Paris
    Select Destination City    London
    Search For Flights
    There are available Flights

It’s a good practice to keep test cases as simple as possible. It will make them more stable and easy to modify when needed.

Creating Keywords from Functions or Methods

Ok, you learned how to create a test case using existing keywords that are grouped into user-defined keywords. But what if we need to save data to a database when selecting the departure city? Or to log information to a file? Those are cases when you need to form a keyword from a function or method of the class. The Robot Framework allows you to do that.

Let’s modify this test to use keywords formed from python class methods.

1. First, we need to install the Selenium library with pip:

2. Download the Chrome WebDriver.

3. Let’s create a Python class that will manage the WebDriver session:

class WebDriverManager(object):

    __driver = None

    @classmethod
    def get_web_driver(cls, browser):
        if cls.__driver is None:
            if (browser.lower()) == "chrome":
                cls.__driver =  webdriver.Chrome("C:/drivers/chromedriver.exe")

        return cls.__driver

The WebDriverManager class stores the created driver session in the __driver variable and returns it when the get_web_driver method is called.

4. Since web pages elements are not loaded simultaneously, you need to wait until the element becomes visible on the page. In Selenium, this can be achieved through the WebDriverWait class. You will create the class Web that will encapsulate the usage of the WebDriverWait class instance by exposing simple methods to search for web elements on the page by xpath.

from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

from web_driver.web_driver_manager import WebDriverManager


class Web(object):
    _driver = None

    def __init__(self, browser):
        self._driver = WebDriverManager.get_web_driver(browser)
        self._wait = WebDriverWait(self._driver, 10)

    def get_web_element_by_xpath(self, xpath):
        return self._wait.until(EC.presence_of_element_located((By.XPATH, xpath)))

    def get_web_elements_by_xpath(self, xpath):
        return self._wait.until(EC.presence_of_all_elements_located((By.XPATH, xpath)))

    def open(self, path):
        self._driver.get(path)

    def close_all(self):
        self._driver.quit()

5. Now, you need a class that will expose methods to interact with elements on the Flights search page.

from web_driver.web import Web


class SearchFlightPage(object):

    __url = "http://blazedemo.com/"

    def open(self):
        self._web.open(self.__url)

    def __init__(self, browser):
        self._web = Web(browser)

    def select_departure_city(self, city):
        self._web.get_web_element_by_xpath("//select[@name='fromPort']/option[@value='{}']".format(city)).click()

    def select_destination_city(self, city):
        self._web.get_web_element_by_xpath("//select[@name='toPort']/option[@value='{}']".format(city)).click()

    def search_for_flights(self):
        self._web.get_web_element_by_xpath("//input[@type='submit']").click()

    def get_found_flights(self):
        return self._web.get_web_elements_by_xpath("//table[@class='table']/tbody/tr")

    def close(self):
        self._web.close_all()

This class uses a Web instance to access web elements on the web page and has a method to select values from departure and destination drop-down lists, clicking on “Find Flights” button, and getting found flights.

This class also has the methods open() and close(), which are self-explanatory.

Let’s have a look at the modified test case:

*** Settings ***
Library         pages//SearchFlightPage.py  Chrome
Suite Setup     Open
Suite Teardown  Close

*** Test Cases ***
The user can search for flights
    Select Departure City   Paris
    Select Destination City    London
    @{flights}=     Get Found Flights
    Should Not Be Empty     ${flights}

Here, the SearchFlightsPage.py library is used to import class methods as keywords.

6. There is still one thing you can improve here. As you see, there's a little programming to be done by saving a list of flights into a variable and then passing it further to the ‘Should Not Be Empty” keyword.

You can bypass this by saving a test variable using the keyword “Set Test Variable” in the resource file.

Have a look at search_flights_keywords.robot:

*** Settings ***
Library         pages//SearchFlightPage.py  Chrome


*** Keywords ***
Open search page
    Open

Close pages
    Close

Select departure
    [Arguments]  ${city}

    select departure city  ${city}

select destination
    [Arguments]  ${city}
    select destination city  ${city}

Search Flights
    search for flights
    @{flights}=     Get Found Flights
    set test variable  ${flights}

Flights are found
    Should Not Be Empty     ${flights}

In the “Search Flights” keyword you are saving search results into the test variable “${flights}”. Later, in “Flights are found” keyword you will read this variable.

Our modified test:

*** Settings ***
Resource         search_flights_keywords.robot
Suite Setup     Open search page
Suite Teardown  Close pages

*** Test Cases ***
The user can search for flights
    Select Departure   Paris
    Select Destination    London
    Search Flights
    Flights are found

The choice to use an already developed keyword or to create your own is always up to a test developer. Try to keep things as simple as possible: if you don’t need additional behavior that can be implemented only in python modules or classes, use already developed keywords – it will save your time.

Executing Your Selenium Tests in the Robot Framework

To execute robot tests in your prompt, type: robot path/to/tests.

‘path/to/tests’ should be a name of a suite file or a suite directory.

You can select which tests to be executed from a suite by filtering by tags. Any robot test or suite can have a tag assigned. To execute tests with an assigned tag, type the following in your prompt:

Robot –include [tag]    path/to/tests

Executing Your Robot Test in Taurus

You can also create your test scripts with Taurus or upload existing ones, in a much easier way. In Taurus, you can run your test in loops to get transaction time statistics. By using the command -cloud you can delegate the test to the cloud through BlazeMeter, getting an online report and without having to have your own infrastructure.

Create and execute a configuration for your Robot test in Taurus and you will get a detailed result of how your application works.

Consider the following configuration to check if our “Find Flights” scenario will work for 1 min:

execution:
- executor: selenium
  runner: robot
  hold-for: 1m
  scenario:
    script: search_flights/

reporting:
  - final-stats
  - blazemeter

This will start Taurus and execute our scenario for 1 min:

BlazeMeter will soon have exciting updates about GUI testing! For stress testing, request a live BlazeMeter demo with one of our performance engineers.

START TESTING NOW

Related Resources

• WebdriverIO Tutorial: How to Run WebdriverIO in Taurus