Integrating Load Testing into CircleCI for iOS
February 7, 2023

Integrating Load Testing into CircleCI for iOS

Open Source Automation

In this blog post, we show a hands-on tutorial on how to do CircleCI iOS tests, specifically relating to load and API testing. Not only will this tutorial show you how to build an iOS test in JMeter and then integrate it into CircleCI, but we will then show you how to do the same more easily with BlazeMeter. 

Let us get started.

Table of Contents:

Why Use CircleCI for iOS Load Testing & API Testing?

By using CircleCI for iOS performance and API tests, we can automate these iOS tests as part of a build and CI/CD pipeline. 

In conventional load testing approaches, testers usually have to coordinate with the developers, wait until all the commits are integrated into the main code, and then manually trigger a bunch of performance and API tests. 

However, with multiple developers constantly integrating their code, there can be several changes to the code in a single day which makes keeping track of all these changes and then manually triggering parallel tests tedious and prone to mistakes. Hence, it becomes very difficult to perform thorough performance testing and API testing for every build, as and the time and efforts to coordinate with developers and execute tests manually for validating

This compatibility with multiple developers’ work is where CircleCI proves to be extremely useful. With test automation, whenever there is a code commit, a set of tests will run and provide the output, whether the build is pass or fail. 

Knowing the outcome of the test allows us to automate different tasks that can be auto-triggered such as sending a test report via email, stopping the test environment, or triggering the next test. This will save a lot of the manual efforts of coordinating with development teams and triggering the tests. It also reduces any manual errors, such as forgetting to test a particular code commit or incorrectly reporting the test status. The testers will only have to worry about adding the test scripts and configuring the tests. The execution will be taken care of by CircleCI build pipelines.

Moreover, because of the ability to perform tests earlier in the delivery lifecycle, CircleCI enables identifying potential performance and functional issues, much earlier. Also, since each and every code commit gets tested, we can pinpoint a particular commit that could have potentially broken the performance or functionality of the code. 

How to Start the CircleCI-iOS Load Test Integration Process

There is an increasing trend of web traffic originating from mobile apps. With iOS being a dominant mobile OS, load testing an iOS application is often critical for businesses. In the following sections, let's dive into how we can make use of CircleCI for running automated load tests for an iOS application. 

Create a JMeter Script for the iOS App

We will use open-source JMeter to create the load test script for our iOS app, as it is one of the widely used open-source tools for load testing. Please note that your PC and iOS device needs to be connected to the same network for this to work.

1. Open JMeter and add a Thread Group.

Add thread group to JMeter

2. Add a Recording Controller.

Add recording controller to JMeter

3. Add an HTTP Test Script Recorder.

Add HTTP test script recorder to JMeter

4. Set the port in HTTP Script Recorder and click Start.

5. On your iOS device, open the proxy settings (Settings->Wifi->Modify->Proxy->Manual) and enter the IP address of the PC running JMeter and the port provided in HTTP Test Script Recorder.

Enter proxy port in JMeter

6. After saving the above settings on your iOS device, you can open your iOS app and navigate through the app. All the underlying HTTP calls/API calls will be recorded in your JMeter script.

7. You can stop recording and enhance the JMeter script with Extractors, Assertions, Timers, etc. For more details on how to enhance your JMeter script, you can refer to this JMeter tutorial.

When you are done, you will have a working JMeter script in the form of a jmx file, which you can use in the next steps to run your CircleCI iOS load test.

Configuring the CircleCI Pipeline

Now that we have the load testing script ready, we will integrate the iOS app load test in the CircleCI pipeline.

1. Create a Git Repository or open your existing code repository and add your JMeter script.

Add JMeter script to code repository

2. Login to CircleCI with your GitHub credentials and click on “Projects”. You will see all your repositories listed there.

3. Select the appropriate project. You will see an option to create a CircleCI config file.

Create a config file for your CircleCI iOS test.

4. In this step, you can select the branch of your project that you want to configure in CircleCI. If you choose the third option, it will add a “circleci-project-setup” branch to your repo where you can commit. In real-life cases, you would choose the first option but for the sake of this tutorial, we will go with the third option.

This is the config file where you can define your build pipeline along with all the automated tests that you want to trigger. You can select the third option if you are doing this for the first time as it will show a preconfigured template that you can edit.

5.  Here is a sample config file that triggers the JMeter test script that we created earlier. It also stores your JMeter result in artifacts at the end of the run. This way you can track your detailed JMeter results for any historic pipeline runs.

# Use the latest 2.1 version of CircleCI pipeline process engine.
# See: https://circleci.com/docs/2.0/configuration-reference
version: 2.1
 
# Define a job to be invoked later in a workflow.
# See: https://circleci.com/docs/2.0/configuration-reference/#jobs
jobs:
  run-jmeter:
	# Specify the execution environment. You can specify an image from Dockerhub or use one of our Convenience Images from CircleCI's Developer Hub.
	# See: https://circleci.com/docs/2.0/configuration-reference/#docker-machine-macos-windows-executor
	docker:
  	- image: justb4/jmeter
	# Add steps to the job
	# See: https://circleci.com/docs/2.0/configuration-reference/#steps
	steps:
  	- checkout
      - run:
      	name: "Run Jmeter"
      	command: "jmeter -n -t blazemeter_test.jmx -l reports/run.log -e -o reports"
  	- store_artifacts:
      	path: reports
      	destination: artifact-file
 
# Invoke jobs via workflows
# See: https://circleci.com/docs/2.0/configuration-reference/#workflows
workflows:
  run-jmeter-workflow:
	jobs:
  	- run-jmeter

With the above configuration, we are now all set to trigger our first CircleCI iOS load test.

Triggering Your CircleCI iOS Load Test

There are multiple ways your iOS load test can be triggered from CircleCI. The ideal way is to have it triggered on every commit so that we do not miss testing any commit. Otherwise, we can also use the CircleCI API or the CircleCI console to manually trigger the test. 

Let us look at how the test is triggered automatically upon committing to the code repository.

1. Go to your code repository console, edit your code, and make a commit in your code repository.

Making a commit in your code repository.

2. Go back to the CircleCI console, click on “Dashboard” from the left pane and you will see your test is automatically triggered in the CircleCi console. You can click on the Job to see the detailed status of each step.

Automatically triggered iOS load test in the CircleCI console.


3. You can expand the “Run JMeter” step to see the summary of your run. This is also where you can review some of the high-level statistics from your test.

JMeter test run summary

4. If you want to see the detailed report in an HTML format, you can also download the JMeter test result from the “Artifacts” tab.

JMeter "Artifacts" tab

We have now successfully integrated the JMeter Load Test for your iOS app into CircleCI. Instead of JMeter, you can also use BlazeMeter to run your functional, performance, and API monitoring tests.

Triggering Scheduled Runs From CircleCI

There might be cases where we would want to schedule the pipeline to execute daily or at specified intervals. This is sometimes useful when we have to get daily performance benchmarks of our system or quarterly regression tests.

We can do this in CircleCi using triggers.

1. Navigate to Projects and click on Project Settings.

CircleCI project settings

2. Go to “Triggers” from the left side pane and Click on “Add Triggers.”

Adding triggers for CircleCI iOS load testing

3. Enter Schedule details, and you have successfully created a trigger that will start your test at the specified schedule, such as nightly or weekly.

Setting CircleCI scheduling details

Triggering BlazeMeter Tests With CircleCI

BlazeMeter is one of the most widely used platforms for running performance and API tests, as well as continuous testing capabilities. Plus, it enables teams to build upon JMeter’s load-testing capabilities.

BlazeMeter provides the RESTful APIs to perform various actions such as creating, updating, and running a functional or performance test over HTTP requests. Let's take a look at how we can make use of this API to integrate our load tests into CircleCI for iOS testing.

 1. Create an API Key

The very first step is to create an API Key that will be used later on to access the API. To create this key, go to your BlazeMeter console, navigate to the account settings and click on the “API Keys” option.

BlazeMeter API key

You will see a “Create a new API Key” button. Click on it and provide the name of the API key that you want to create. Select the expiry of this API Key as per your requirements and click on the “Generate” button as shown below.

Generating BlazeMeter API key

You will see the “api-key-id” & “api-key-secret”. Copy these values, especially the secret value as it will not be displayed again and save it securely. We will be using these values in our API calls to BlazeMeter.

 2. BlazeMeter’s API Will Start the Test

 A sample API request to trigger a BlazeMeter test looks like the following:

 curl -X POST https://a.blazemeter.com:443/api/v4/tests/{TestID}/start -H “Content-Type: application/json” --user 'api_key_id:api_key_secret'

 {TestID} – Test Id of your JMeter performance test

api_key_id:api_key_secret – these are the values we generated in the steps above

 We can use this cURL command in any third-party tool or command line to trigger the BlazeMeter test remotely, without having to access the JMeter console.

 There are many other API requests associated with the performance tests, which you can refer to from the BlazeMeter documentation. 

 3. Update Your CircleCI Configuration File

 Now, add your cURL command in the “Command” section in your CircleCI configuration file. A sample configuration file will look like the following:

jobs:
  run-jmeter:
	# Specify the execution environment. You can specify an image from Dockerhub or use one of our Convenience Images from CircleCI's Developer Hub.
	# See: https://circleci.com/docs/2.0/configuration-reference/#docker-machine-macos-windows-executor
	docker:
  	- image: justb4/jmeter
	# Add steps to the job
	# See: https://circleci.com/docs/2.0/configuration-reference/#steps
	steps:
  	- checkout
  	- run:
      	name: "Run Jmeter"
      	command: "curl -X POST https://a.blazemeter.com:443/api/v4/tests/{TestID}/start -H \"Content-Type: application/json\" --user 'api_key_id:api_key_secret'"

Once these three steps are completed, the integration of the BlazeMeter performance test with the CircleCI pipeline is complete. Now you can go to your repository again and perform a commit. As soon as you commit, you will see the CircleCI pipeline getting triggered, which will in turn trigger your BlazeMeter performance test. At this point, you can visit the BlazeMeter console to monitor the test if you want or you can download the results at any later point.

Bottom Line

With the advancement of various CI/CD tools, the software delivery cycles are getting shorter and code is getting evermore complex. Multiple developers can simultaneously update their bit of code and merge it to the main repository at any time and with increased frequency. It is critical for testing to also adapt to these changing dynamics and keep up with the pace of development, the complexities of code, and the environment.

With the use of the CircleCI tool, we can integrate our performance and API tests in build pipelines so that the tests are triggered automatically on every code commit without manual intervention. This way we can ensure that every code commit gets thoroughly tested. Moreover, using CircleCI for iOS load and API testing allows us to test and find issues early in the delivery cycle, which in turn gives enough time for developers to take any corrective actions and resolve the issues.

Thanks to all these advantages, integrating your performance and API testing with CircleCI can be extremely beneficial. Learn how your CircleCI testing can be even better with BlazeMeter by starting a free trial. 

Start Testing Now