BlazeMeter in a Continuous Integration Cycle: A CA Technologies Use Case
CA Technologies implements BlazeMeter in its Continuous Integration process: CA uses BlazeMeter to run performance tests on an OAuth authorization scenario. This performance test is implemented into CircleCI, a continuous integration tool. This blog post will explain how it’s done.
The Performance Test
OAuth (the Open Standard for Authorization) is a protocol that provides token-based authentication and authorization - as opposed to the standard username and password requirements. OAuth allows third party services to use the end-user information without revealing their personal credentials.
CA Technologies uses OAuth to verify Agile Central users, to ensure they are receiving proper support services. OAuth includes a login and logout process, which needs to be tested every time the code is changed. This is a condition for deploying the code. The performance testing is run by BlazeMeter, and implemented into continuous integration tool CircleCI.
The Continuous Integration Process
Continuous Integration and agility ensure quicker development and deployment with less errors. By automating testing and treating the whole development, testing and deployment process as one cycle, products work smoother, resulting in better customer experience and better sales. Just as important, Continuous Integration and Continuous Deployment cycles free up developers’ and engineers’ time from boring tasks, leaving them with more time to research, be creative and think out of the box when writing code.
Continuous integration cycles include many types of tests and jobs - checking code, unit testing, functional testing and more. Performance testing is an important part of the CI cycle, to ensure that code changes don’t affect software performance. This can be easily achieved by integrating a load or performance test as a job in the CI build.
CA Technologies uses different CI tools, including open source Jenkins and commercial CircleCI, which enable managing jobs and builds from the user’s browser, in the cloud. CircleCI is used in this case.
Nikhil Kasthurirangan and Naveen Gunukula, who manage this project, integrate the BlazeMeter performance test into the CircleCI workflow as one of the eight steps that are automatically carried out every time a code is committed. Because CircleCI is SaaS, the code goes into production quickly.
BlazeMeter in CircleCI
This is the CircleCI workflow. As you can see, there are eight jobs. Each job is responsible for its own task.
For example, the first job, checkout_code, is a GitHub integration. This job downloads the code onto the CircleCI machine and generates the build. The second job gets dependencies, the third persists them on a virtual machine through CircleCI and so on. BlazeMeter is the sixth test (“big_ole_test”) and it ensures the login and logout flows are working properly.
These jobs are run automatically every time the code is changed and committed. After the fourth test is run, the unit test and the performance test are run in parallel.
The BlazeMeter Performance Test Scenario
Let’s take a closer look at the BlazeMeter test through the IDE. This is an overview of all the tests in the Continuous Integration flow:
Let's look at the steps that are taken each time we want to validate performance with BlazeMeter:
In the BlazeMeter test, Agile Central is started locally and a mock client is set up, to replace the CA support website. Then, BlazeMeter checks a login and logout scenario, after the new code was pulled from GitHub and the rest of the tests were completed. If the test results don’t perform according to the KPIs specified, the test will fail and an alert will be sent to the developer to fix the code.
Running BlazeMeter Through Taurus
In CircleCI, BlazeMeter is run through Taurus, an open source test automation framework. A shell script is used to install BlazeMeter Taurus (bzt) on a virtual machine, and the Agile Central authentication test is orchestrated through a YAML config file for bzt:
Configuring the Performance Scenario with JMeter
The YAML config file runs a preconfigured JMeter script, that tests the login and logout flow of the support URL. The JMeter script was initially created by using the free BlazeMeter Chrome recorder. With the recorder, the process was recorded and then converted to JMX. The YAML file provides a single easy to read and source controlled location to define the duration and intensity of the test, specify pass/fail criteria and control the naming of the reports that will appear in BlazeMeter’s portal. Another separate configuration file, .bzt-rc, contains global settings including the API token for communication with BlazeMeter. In the script. CA added passfail criteria and BlazeMeter configurations like the BlazeMeter token.
Analyzing Continuous Integration Test Results
After running the BlazeMeter part of the test, performance results are viewed and analyzed on the BlazeMeter workspace, which is reached through CircleCI. On CircleCI, CA can see the results of all the tests in their workflow, including the performance test. If needed, CA can fix their code. Otherwise, the next part of the CI process is run.
Conclusion and Looking Forward
After the integration with Blazemeter, CA were able to test each commit against the performance metrics and block the commit in case of performance degradation. This enabled CA to continuously deploy improvements almost realtime. In the future, these performance tests will also be run at scale.
Interested in Continuous Testing? Sign up for our free course.
To learn how you can add BlazeMeter to your Continuous Integration cycle, request a demo. You can also run your own performance test by putting your URL in the box below, and your test will start in minutes.