Mobile Testing on Real Devices with BlazeMeter: Learn How
69% of digital media today is consumed on mobile platforms, according to comScore. This is a 99% increase in digital media consumption on mobile platforms compared to 2013. This mobile revolution has forced companies to focus on a mobile first strategy - either by building mobile apps or mobile-optimized sites.
Mobile apps and mobile sites need performance testing. However, mobile applications are bit harder to test than web-based applications. BlazeMeter enables developers to simply run load and functional tests on mobile apps, as part of their shift left and continuous integration performance testing strategies.
Mobile testing with BlazeMeter requires only 3 steps:
- Package your app and test scripts
- Run a simple command
- Analyze results
This post will provide a step by step explanation of how to run a functional mobile test with BlazeMeter on a real mobile device. Currently, our mobile testing solution is in Beta, so we would appreciate your feedback after completing the steps in this blog post, here.
You can either run the test for your own app, or follow the demo we created for you. Steps will be shown for both ways. The demo shown in this blog post will use a sample ios application and run a basic Appium test. The demo test will show how to run a functional GUI test for an iphone app that is purchasing a ticket on the BlazeDemo website. You will be able to download the test results as well as the video recording. BlazeMeter mobile testing also works for android.
Let’s get started.
- Make sure you have a Python 2.7 environment setup.
- You will need a BlazeMeter API key and secret. If you don’t already have an account, you can sign up for a free one here.
- Clone the BlazeMeter mobile test Github repo. The repo comes with a sample app and test.
- Go to the root of the cloned repository. Install the required libraries
Preparing Your App
If you are testing your own app, follow these steps. If you’re following the demo, skip to “Running the Mobile Functional GUI Test” section.
Step 1 - Package Your Application for the Test
If you are using your iPhone application, make sure that it is signed for ad-hoc deployment. We support Android apps but are working to build APIs for Android and also allow device selection.
Add the mobile app that you want to test to the resources directory. The bundled mobile app ipa uses the source code from this repository.
Step 2 - Zip Your Test File
Add your test to the test-files/python/tests directory. Replace the existing blaze_demo_test.py script. Make sure you add your additional requirements for your Appium test in the requirements.txt file under the test-files/python directory.
Since we are running this test on a real device that is pointing to your application, the only configuration you need to pass in the desired capabilities is the bundle id.
Now, bundle the tests by zipping them with the create_bundle script provided in the python directory.
This zipped artifact is what BlazeMeter uses to upload and run the tests.
Running the Mobile Functional GUI Test
Go to the root of the cloned directory and type the following command
This will create and initialize the test. After a few seconds, the program will then prompt you for an iPhone application. The default application is already pre-selected. Hit enter, or, if you are using your own application, pass the path to the application.
The next prompt is to enter the zipped test files. The default is already selected so go ahead and hit enter. If you want to pass your own test files, pass the path to your zipped file.
The system will now install the app on a real device and run the tests. This will take about 5 minutes or so. The output will include a video recording of the entire test, screenshots and the tests results.
Analyzing Your Mobile GUI Test Results
The response contains a JSON results object that has a list of entities. Each entity has a type, like VIDEO or DEVICE_LOG, as well as a URL that hosts the video or log file.
The video captures how the application ran under test on the device. This is a very useful tool for debugging and understanding how your test ran.
JUnit XML Result
The response also contains a results object that has a list of entities. Look for the type APPIUM_JAVA_XML_OUTPUT. This file contains the results of the test run in JUnit-XML format. This format makes it easy to integrate into CI systems like Jenkins. In addition, you will be able to see the time it took to run the scenario (34.998 in our case). This KPI is especially important when you want to compare the GUI test results to load testing results.
We are also working on the UI for the results on Blazemeter platform. This is what the results will look like. If you are interested, please reach out to us so that we can get you access.
What about more frameworks and languages?
We are working on this, please let us know what you want. We also plan on adding support for Java soon.
That’s it! As I mentioned earlier, this is a beta feature and we are interested in your feedback. If you are interested in joining this beta program, please reach out.
Learn more about BlazeMeter from here.