How BT Group Saved Millions By Transforming Testing Efficiency With BlazeMeter’s Service Virtualization and AI-Driven Test Data

The Challenge

BT faced significant challenges with the integration of over 50 BSS, OSS, and network components from multiple vendors — each with their own release cycles — leading to complex testing environments.  

BT sought to modernize and update their developing and testing methods. In their original testing ecosystem, they faced several challenges:  

• Tech Stack Complexity - Service Virtualization was essential to handle the integration of over 50 BSS, OSS, and network components from multiple vendors, each with their own release cycles, leading to complex testing environments.
• Third-Party Simulation Challenges - As part of the digital complexity, BT was finding it challenging to simulate third parties at scale, to ensure quality testing and deliveries.
• Delayed Release Cycle - As a result, BT’s release cycle was a bottleneck, hindering development and organizational efficiency and making it difficult to meet tight deadlines.
• Combating Spam and Phishing Messages - One of the projects BT needed to test was SpamShield, a product designed to identify SMS Spam and Phishing attacks using Artificial Intelligence and Machine Learning. This was urgent, due to these issues becoming prevalent and threatening the customer experience.  

The Solution

British Telecom chose BlazeMeter to advance their software testing practices and implement continuous testing by using its AI and Service Virtualization capabilities. With these virtual services, BT could simulate third-party systems and identify issues early and maintain uninterrupted development.

How Service Virtualization Impacted BT

BT integrated Service Virtualization by initially mapping their entire tech stack and identifying key integration points where Service Virtualization could provide immediate benefits. The deployment was phased to ensure minimal disruption, and initial tests focused on high-impact areas like third-party system simulations and high-volume transactional systems. 

1. Initial Setup: BT's tech team began by identifying critical third-party services and their interactions within the tech stack. Virtual services were then created to replicate these interactions, ensuring no dependency on actual third-party systems during testing.
2. Gradual Rollout: The Service Virtualization capabilities were rolled out in phases, starting with non-critical systems to test the integration and gradually moving to more critical systems. This phased approach ensured stability and minimized disruption.
3. Ongoing Management: Using Kubernetes and Docker, BT managed the test environments efficiently. Karpenter was used to dynamically adjust resources, optimizing performance and cost. 

BlazeMeter’s Service Virtualization capabilities enabled BT to realize the following benefits:  

Simulate Complex User Scenarios

BlazeMeter’s Virtual Services allowed BT to simulate complex user scenarios by configuring a sequence of requests to be executed in a specific order. This ensured that BT could test performance and reliability under various user scenarios and conditions, leading to more robust and realistic testing outcomes. By defining the order of these requests, BT was able to replicate real-world workflows, ensuring their applications could handle different user interactions seamlessly.

Generate Dynamic Response Status Codes

BlazeMeter’s dynamic response status codes feature enabled BT to set or generate response status codes dynamically based on certain conditions. This flexibility allowed BT to create more realistic API Virtual Services by simulating different responses during testing. For instance, by mimicking various HTTP status codes such as 200 OK, 404 Not Found, and 500 Internal Server Error, BT could effectively test their application's ability to handle different response codes and improve error-handling logic. 

Mitigate Security Risks

The SSL alias feature in BlazeMeter’s Virtual Services allowed BT to specify a custom alias for SSL configurations, ensuring secure and encrypted communication between clients and API endpoints. This capability was crucial for managing multiple SSL certificates and maintaining secure communication channels during testing. By using SSL aliases, BT could easily distinguish and manage different SSL configurations within their Virtual Services, enhancing the security and reliability of their tests. 

Leverage AI-Based Synthetic Test Data

BlazeMeter’s conditional routing logic feature enabled BT to define rules for routing requests to different processing actions based on specific conditions. This capability allowed BT to create dynamic and realistic scenarios in their virtual services by simulating complex API behaviors and routing configurations. By setting up rules based on request methods, headers, parameters, or payloads, BT could ensure their applications responded correctly under various conditions. This feature was enabled by AI Synthetic Test Data Orchestration, which allows the creation of synthetic data to simulate various scenarios during testing. 

Improve API Performance

Data-set profiles in BlazeMeter’s Virtual Services allowed BT to use predefined sets of data as inputs or responses during simulations. This feature helped BT test various scenarios and data conditions, ensuring their APIs performed reliably under different circumstances. By defining data-set profiles that included user profiles, transaction data, and other relevant information, BT could validate the functionality and reliability of their APIs comprehensively.

BlazeMeter’s feature allowing the re-ordering of HTTP processing steps and webhook calls gave BT the flexibility to customize the sequence in which these actions were executed during API simulations. This ensured that requests and notifications were handled in the desired order, tailoring the execution flow to match specific testing scenarios and ensuring realistic API behaviors.

BlazeMeter’s optional/dependent mandatory tag validation feature also enabled BT to define certain tags or elements within API responses as mandatory. This ensured that the response structure complied with the expected format, containing all essential data elements required by the client application. By setting mandatory tags, BT could accurately simulate API behavior and validate the correctness of responses generated by their Virtual Services.

Furthermore, the proxy config redirect feature in BlazeMeter’s Virtual Services allowed BT to redirect incoming requests to specified destinations based on predefined rules. This capability was invaluable for simulating complex API behaviors and testing different routing configurations. By setting up proxy redirects, BT could control the flow of incoming requests and optimize resource allocation, ensuring their applications handled redirected requests effectively. 

XPath & CDATA Matching

BlazeMeter’s feature for displaying XML node attributes in the selection wizard for XPath matching enabled BT to use XPath expressions to match specific nodes within XML documents. This capability allowed BT to define precise criteria for selecting and processing XML data, ensuring their virtual services could accurately interpret and respond to XML messages during API testing.

Additionally, the CDATA matching and parsing feature in BlazeMeter’s Virtual Services enabled BT to handle XML CDATA sections within incoming requests and responses accurately. This capability ensured that data within CDATA sections was processed correctly, maintaining the integrity of data exchanges in virtual service testing and preventing unintended parsing or encoding issues. 

Streamline Testing Processes

BlazeMeter’s ability to run multiple Virtual Services on a single port number streamlined BT’s testing processes and simplified configuration management. This feature allowed BT to simulate and manage various APIs or endpoints within the same virtual environment, optimizing resource utilization and ensuring comprehensive test coverage. 

How AI Enhanced BT’s Testing

In addition to Service Virtualization, BT utilized BlazeMeter’s AI capabilities to further enhance their testing processes. AI was instrumental in automating repetitive testing tasks by generating test data and predicting potential issues before they occurred. By incorporating AI/ML, BT could analyze vast amounts of testing data to identify patterns and anomalies, leading to proactive problem resolution and continuous improvement.

BT leveraged BlazeMeter’s Test Data Pro to generate synthetic data, allowing them to create realistic and varied test data sets efficiently. This ensured that all edge cases were tested, and the applications were robust against different data inputs. The use of synthetic data helped BT test how their Virtual Services handled various inputs, edge cases, and unexpected data without relying on actual production data.

The combination of AI and Service Virtualization provided BT with unparalleled testing capabilities, ensuring high-quality software delivery and operational efficiency.

1. Seamless Integration: AI tools were integrated into BT's existing testing framework, working in tandem with Service Virtualization. This seamless integration allowed for continuous testing and real-time feedback, enhancing the overall testing process.
2. Continuous Learning and Improvement: The AI models continuously learned from new data, improving their accuracy and effectiveness over time. This adaptive learning ensured that BT's testing processes evolved with changing requirements and technologies.
3. Comprehensive Coverage: AI and Service Virtualization together provided comprehensive test coverage, from unit tests to end-to-end testing scenarios. This ensured that BT’s applications were tested thoroughly, reducing the risk of defects and improving reliability. 

The Results

With BlazeMeter, BT saw the following ROI:

• Millions of Pounds Saved - Service Virtualization and AI contributed significantly to cost savings by reducing the need for physical testing environments and minimizing the risk of errors in production through improved test coverage and reliability.
• Four Week Decrease in Time-to-Market - By leveraging BlazeMeter’s suite of tools, including AI/ML capabilities, and adopting a comprehensive testing approach with Service Virtualization at the 5G SA core for resiliency testing, BT accelerated delivery cycles, with new updates and developments reaching production four weeks earlier than before. This is a significant improvement in the ability to deliver high-quality services to users.
• Successful Implementation of SpamShield – Service Virtualization and AI were key components in SpamShield’s success, resulting in a 93% initial reduction in spam reporting and 280 million blocked spam texts.
• Higher Quality Software – Service Virtualization and AI’s automation and efficiency resulted in higher quality software, which was more reliable and secure. This resulted in enhanced customer satisfaction, a primary goal for BT’s management.
• Shift Left – Service Virtualization and AI allow BT to shift-left even further, and test integrations early. This allowed for streamlined processes and minimal errors, ensuring faster cycles and higher adaptability. 
• Test Simplification and Democratization - One of BlazeMeter’s primary objectives is to provide access to testing for every team member regardless of skill level. Service Virtualization and AI’s user-centric approach simplified testing tasks for users of all skill levels and provided easy access to resources to further their knowledge and skills.
• Skilled Users - BlazeMeter provided comprehensive training materials, learning management systems, and tutorials to ensure each BT member obtained in-depth knowledge. These included platforms like BlazeMeter University and webinars. In addition, BlazeMeter paired BT with experienced professionals for enhanced learning and skill development. 

Bottom Line

BT’s strategic partnership with BlazeMeter, leveraging advanced Service Virtualization and AI capabilities, has led to a significant transformation in their software testing practices. The deployment not only streamlined BT’s testing processes but also resulted in substantial cost savings, improved time-to-market, and enhanced software quality. By simulating complex scenarios, automating test data generation, and continuously learning from new data, BT was able to proactively address potential issues and deliver high-quality services to their customers. This case study highlights the profound impact of Service Virtualization and AI on BT’s operational efficiency and showcases the potential for other organizations to achieve similar success through innovative testing solutions.

Experience how BlazeMeter can make a positive impact on your organization. Try BlazeMeter for free today or explore one of our on-demand demos.

Start Testing Now        Watch Demo