Plutora Improves Software delivery for O2/ Telefónica UK

Telecommunications is a fast-moving industry. As a leading brand at the forefront of the industry, O₂ has continually innovated new mobile products and has championed customer experience. In order to maintain and extend its leadership position, O₂ maintains the highest standards in its IT application delivery in order to support its massive customer base which comprises over 33 million connections. However, its IT infrastructure experienced a number of challenges which could slow down the planned improvements. 

The UK IT test environment management group supports a large number of internal teams. They were faced with over 30,000 emails and remedy requests relating to non-production environments. In addition, scheduling environments were not centrally managed allowing each team to manually add their bookings into spreadsheets with little or no collaboration or arbitration on conflicts. Management of communication and schedules was an unsustainable situation without more effective tools or a lot more people. In the case of unplanned outages (i.e. failures in non-production systems or applications) it could take several days to work through the various architectures to understand where the root cause of a test system outage originated. Furthermore, it drew heavily upon many different support teams which had to be coordinated using manual lists. Ultimately, IT testers were often unable to carry out their testing projects whilst issues with the test environments were being investigated. 

Mark Lewis, UAT Test Environment Manager at O₂ commented, “One solution we sought was visibility of all of our test environments to avoid double-bookings and uncontrolled changes and make sure sufficient management is in place to be effective.” 

The UK IT test team – in conjunction with its quality test partner, NTT DATA – decided to undertake a review of IT test environment management tools to provide a solution to address these issues. 

“We are rolling out a dynamic, company-wide IT test environment view in the UK where key stakeholders will be able to self-serve with customised dashboards and reports and see environment utilisation centrally for the first time. Plus they can be alerted to any potential system conflicts, enabling them to have a direct conversation with the stakeholders affected and take immediate action.” – Rob Hutchinson, Senior Test Manager, IT Billing and Charging, O₂

“One of the main problems was that we just didn’t have visibility of all of our test environments and that frequently resulted in double-bookings and uncontrolled changes because there wasn’t sufficient management in place to be effective.” – Mark Lewis, UAT Test Environment Manager at O₂

PARTNERING WITH PLUTORA

After a two-month external review of IT test environment tools, O₂ and NTT DATA selected Plutora because it is an independent, dedicated, test environment management platform that could be rolled out across the business. Plutora was brought in to deliver the following outcomes: 

• 100% visibility of the test environment planning 
• Enable unified scheduling for the first time 
• Centralised environment booking and change requests for testing and change management 
• Automation of manual improvement processes to increase productivity and cut resource duplication 
• Integrate seamlessly with Plutora’s standard APIs so other systems and tools used in the O₂ ecosystem can benefit and further reduce effort and manual updates 
• New self-service capability for the senior leadership team via Plutora’s reporting feature 
• Up-to-date dashboards, alerts, notifications and analysis to engage stakeholders and reduce unplanned outages 

“Plutora was the only tool dedicated to IT test environment management, rather than it just being an add-on, and the team was really flexible and supportive.” – Mark Lewis, UAT Test Environment Manager at O₂

The focus of the O₂ IT test team is to support the IT environments, including physical hardware and application upgrades that support their customers during their test lifecycles. This spans simulation of buying a new mobile phone to services that support their provisioning such as operations and billing upgrades. The implementation of Plutora was carried out in two phases to ensure a smooth transition from the existing two tools which only included IT testing as an add-on. 

In phase one, O₂ and its QA test partner, NTT DATA ran a pilot and was satisfied that the tool contained all mission-critical processes to ensure that when any service exits the test phase it can be fully integrated into production with minimal risk. 

After four months, phase two of the implementation extended a roll out to 60 test environments including support of the company’s compliance with the European Union GDPR data compliance regulation which came into force in 2018. 

“Our main priority for the next two years is to ensure that test environment and release management are given the focus they need and we are as efficient as possible.” – Mark Lewis, UAT Test Environment Manager at O₂

“Plutora gives us a solution that helped reduce unplanned non-production outages by 50% in a year.” – Rob Hutchinson, Senior Test Manager, IT Billing and Charging at O₂

BENEFITS

The telecommunications company has made immediate gains following the Plutora implementation. Many IT testing problems that the company experienced in the past have disappeared due to the hard work of O₂’s IT Environment Management team with quality test partner, NTT DATA and Plutora. Standardising the handling of IT test environments with Plutora has meant a considerable reduction in testing effort and a much easier, more efficient way of managing IT schedule conflicts. Now key stakeholders can be alerted automatically to any issues or changes in IT environment status. 

Release and environment teams are no longer relying on spreadsheets. Manual improvement processes have been automated and standardised. Consequently, a single, dynamic view of IT test environments has been created from which vital intelligence is garnered to prevent any detrimental impact on internal project delivery. Implementing Plutora has contributed to O₂’s IT modernisation and drastically reduced unplanned non-production downtime by 50% in a year. Before, these outages were managed by identifying incidents manually. Then they were communicated to affected stakeholders and project teams using a shared mailbox. Now, the Environment request module in Plutora is used to capture environment bookings and system changes. Also, reports and workflows have been automated to broadcast communication to projects and support groups. 

Furthermore, O₂ is using Plutora’s reporting feature to automate monthly KPI reports, thereby improving measurement and giving a foundation for productivity improvement. 

Summary of Benefits 

• Centralisation, management and automation of IT test environments for the first time 
• Live visibility into status of releases and environments including current configurations 
• Improved conflict scheduling through Plutora’s visual ‘Environment Scheduling’ feature 
• Increase in test environment utilisation enabling decommissioning or repurposing of underutilised environments 
• Productivity improvement by reducing time spent collaborating, tracking configurations, and conflicts. Team can now focus on process improvements including more automation
• Vital IT intelligence created for faster, more effective decision-making