Load Testing Challenges: Present & Future Interview With Alex Podelko
Alex Podelko is a prominent thought leader in the world of load and performance testing. For the past sixteen years, Alex has supported numerous performance initiatives for several large companies. He is currently a consulting member of technical staff at Oracle, and the owner of a very influential performance engineering blog. This week, we had the pleasure of interviewing Alex and hearing his thoughts on the main challenges in load testing and the future of the field:
Q: What are the main challenges facing load testing today?
The continuous delivery approach brings a huge challenge for development. New features are getting rolled out with increasing speed, making it ever more vital to find the best way to conduct load testing in this new age. Many different approaches are being tried, and a uniform process has yet to emerge. One thing that does seems clear is that all the routine parts of load testing should be automated in order to make load testing a standard part of the workflow.
I believe software updates must be properly tested before being sent to production. Since even a small change to the application can seriously harm performance, I think every precaution must be taken prior to production. Although there are other opinions around the IT world.
Another challenge facing load testing is the increase in the scale of systems. One way to address it may be constant load testing of the production environment. There are organizations that deal with this challenge mainly by closely monitoring trends in the production environment in order to spot irregularities. While close monitoring is indeed vital, I believe that it does not eliminate the need for periodic load tests.
Conducting periodic load tests on a functioning production environment is no easy task. This entails running the test while your users are using the product in real time, meaning that any change to the production environment may affect the user experience. If the testing requires writes as well, this would involve separating your test writes from the constant stream of writes from the users. It gets even more complex if the need arises to test elaborate workflows with multiple steps. The ways to overcome this challenge must be tailored to each specific situation.
Q: Which scenarios and technologies are hard to load test?
In the past, load testing has been used mostly to simulate basic user actions on a protocol level. Today there are many more complex user actions and rich data that must be taken into account, such as streaming audio and video. Scenarios of this nature require load testers to find solutions for recording these intricate user actions.
As for technologies, each new technology requires the load testers to figure out ways to generate significant and meaningful loads on the new systems. Sometimes this entails making actual changes to the product, or using a specific API. From the mainframe to the cloud, technologies keep getting more complex and testing them becomes more difficult. Due to this complexity, testing cannot be fully automated, and load testers must solve the challenge that each new technology presents as it emerges.
Q: Why do some enterprises decide to conduct load testing in-house?
Many organizations, especially big corporate ones, avoid using the public cloud for security reasons unless absolutely necessary. They keep all their systems behind a firewall, and mostly rely on their in-house resources. This is very understandable, especially for internal applications if high scalability isn't required. While this is a valid decision from a data security point of view, it causes them to miss out on the benefits that the public cloud provides, one of which is instantly scalable load testing. The public cloud allows testers to provision cloud servers for a limited time and to simulate any amount of user load at reasonable costs. One solution may be to use a private cloud.
Q: How do you see the future of load testing?
In my opinion, the load test process needs to undergo a change in culture. Traditionally the load testing team joined the development process only at the very end. A better solution, however, might be to introduce load testing into the process from the beginning, making sure that every part of the application can withstand the anticipated load. This can save the testers a lot of trouble down the line, as there will be no major surprises regarding performance when the application is complete. Fortunately, there are ways to accomplish this without having to pay for a load testing team during the entire development process.
One solution is to make the testing process as automated as possible. This way the performance engineers only need to be involved with complex cases. Another important change may be to give more of the load testing responsibilities to the developers and functional testers. These team members should constantly pay attention to performance aspects during the development stages. Developers are usually educated in the field of performance well enough, but lack the time to devote attention to it due to their production schedules. While they shouldn't devote too much of their time to actual load testing, both developers and functional testers should pay attention, for example, to the response time of functional tests in order to nip performance issues in the bud.
Want to find out more? Check out our performance testing training center.