A testing center of excellence (TCoE) can help you leverage automated testing efforts.

The “World Quality Report 2018-19” claims that test automation is almost a heal-all concept in QA and one of the most booming trends for the near future. Still, the general percentage of automated testing in overall test efforts is quite modest. Apart from technical and financial impediments, test automation can be hampered by a number of organizational challenges. Here, testing centers of excellence (TCoEs) may be of practical value.

Step 1 of 7

14%

Which of the following best describes how you currently monitor and manage systems across your hybrid cloud environment?(Required)

Multiple separate tools for different environments

Multiple separate tools for different monitoring and management functions

Single consolidated dashboard

Custom built monitoring and management solution

No full monitoring and management solution

Don’t know

How valuable would it be for your monitoring and management solution to incorporate predictive analytics to help you identify and address system issues that are critical and not waste time or effort working through all the data and “noise” that is not critical?(Required)

Extremely valuable

Moderately valuable

Somewhat valuable

Slightly valuable

Not valuable

How do you most typically address potential system issues BEFORE they impact your operations?(Required)

Automated workflows and playbooks to remediate potential issues as soon as they are detected

An orchestrated response process that leverages automation, AI, and human expertise to rapidly respond to potential issues

Advanced monitoring tools such as predictive analytics and anomaly detection to proactively identify and address potential issues

Respond to issues as they arise, using manual processes and troubleshooting techniques

A combination of automated remediation, orchestrated response, and/or proactive monitoring to address potential system issues

Don’t know

Other

What methods does your organization use to control and monitor cloud/IT spending?(Required)

Centralized procurement processes

Pre-approved budget thresholds

Automated cost monitoring tools

Department-level spending controls

Regular cost audits and reviews

Cloud cost optimization platform

Approval workflows for resource provisioning

Native financial operations tools provided by the cloud provider

No formal controls in place

Don’t know

Other

Please specify:

Which methods do you rely on most to detect security vulnerabilities in your systems?(Required)

Automated scanning tools

Manual assessments

Vendor notifications

Threat modeling (for early detection)

None of these

Don’t know

Other

Please specify:

What are the TWO most common ways you prioritize vulnerabilities to address within your environment?(Required)

Based on potential impact

Based on exploitability

Regulatory compliance requirements

Based on CVSS scores

Risk-based analysis of multiple factors

First-in-first-out approach

No formal prioritization process

Don’t know

How do you typically handle system updates and security patches?(Required)

Automated updates and patches

Manually review and apply updates and patches on a regular schedule

A combination of automation and manual review

Apply updates and patches on an as-needed basis.

Outsourced update and patch management

No defined process

Don’t know

Other

Δ

Let’s find out what TCoE is, why test automation is a must and how TCoE may come in handy on your way to higher test automation levels.

TCoE: Capsule Review

A TCoE provides QA optimization and control due to consolidated testing best practices, tools, standardized methodologies and testing metrics. It aims to centralize QA resources dispersed around the whole enterprise, thus being more relevant for large-scale organizations with a number of departments or even subsidiaries. Capgemini research results prove that TCoE can bring such tangible benefits as optimized QA resources (people, budgets and tools), reduced testing time (by 30 percent) and costs (by 35 percent in a three-year time frame) and increased test automation rate (up to 50 percent).

Why Level Up Test Automation

Test automation is not just hype, but a proven way to get more from your testing. The ever-growing popularity of Agile and DevOps/continuous delivery projects necessitates that testing be performed within a shorter time frame. Test automation tools are highly efficient in performing the most time-consuming types of testing and the ones comprising large amounts of data, such as regression, performance, data warehouse or load testing. With its ability to execute thousands of test cases during every run, test automation provides coverage unachievable with manual testing resources.

Test Automation Challenges That TCoE Can Solve

The promotion of test automation is acute as it can result in tangible benefits. For now, the average level of automated test activities is only around 16 percent, as reported by the leading CIOs from around the world. However, it is pointless to automate all testing activities. Test automation is not a panacea for every testing project or testing type. Each project’s automated testing efforts require a careful consideration and ROI assessment, as test automation has a number of limitations. For example, it can save the testing budget only in the long run. So, there’s no need to introduce automation for small-scale testing projects. Besides, even in projects with a large testing scope, test cases that are unique for each release or source code modification, rarely repeated test cases and those with difficult-to-measure or not purely objective output shouldn’t be automated.

However, the low rate of test automation comes not only from its natural technical impediments but also because of the organizational problems bound. Luckily, TCoE can handle some of test automation organizational challenges.

Test Strategy Lacking Test Automation

If the test strategy doesn’t fully correspond to software requirements and its peculiarities, it may lack testing types appropriate for automation. TCoE may be of help here as TCoE managers can assist test leads in building a more effective test strategy with respect to high-level QA priorities and help BAs to come up with multilevel software requirements, if a software requirements specification turns out to be incomplete.

Besides, the initial part of test automation requires a significant share of testing time (choosing the repeatable test cases and scenarios, scripting and maintaining them) and budget. Without TCoE, some test leads may try to economize and strike automated testing off the test strategy, sure that the moves will help them meet the rigid deadline and budget. If it’s truly impossible to implement relevant test automation and ensure acceptable testing quality stays within time and budget, TCoE managers can suggest revising the testing time and financial limits for the project.. The alternative, of course, often results in even more expenditures because of the release of heavily bugged software.

The Lack of Skilled Test Automation Engineers

QA teams consist mostly of manual test engineers without code writing skills needed to script and maintain automated test scenarios, while developers don’t have the knowledge of relevant automation testing tools or the experience in creating appropriate test cases. So, neither manual test engineers nor developers can fill the gap without sufficient training.

TCoE can help, with its managers knowing each test engineer and having an established knowledge-transfer process. TCoE managers can find test team members (preferably, manual test engineers) who are enthusiastic about test automation and provide training to them on a broadly applicable programming language (for example, Java or Python) and the most adjustable test automation tools (Selenium, Ranorex, TestComplete, etc.).

Silo Testing Processes

Uncoordinated and varied testing processes can hinder test automation. In different QA teams, test leads may choose disparate test management tools and test automation engineers are likely to use diverse test automation frameworks. Besides, with QA resources being dispersed around the company, it’s complicated to create a common base of automated scripts that can be reused in different testing projects.

Efficient test automation requires unified testing processes around the enterprise, which can be attained with TCoE. From the earliest stages of its maturity, TCoE focuses on aggregation, analysis and standardization of testing processes and test automation tools and frameworks. In addition, the TCoE team captures test patterns, creates modular automated test scripts and gathers them in a library to be fully or partially used through the same testing types or similar testing projects.

Excessive Test Automation Spend

Test automation may be pricey due to automation software license costs. Software license purchase for almost every testing project anew can turn test automation into a nightmare for the finance department and lead to excessive testing costs. TCoE handles this challenge by providing conventional licensed test automation software tools that can be used for a number of projects.

Also, test automation requires ongoing engineering resources for maintenance. Application under test (AUT) changes from iteration to iteration and all the automated scenarios should be checked for their validity. Only a few of them can remain unchanged while the majority needs restructuring and correction. A number of TCoE’s qualified test engineers ready to cater to continuous automated testing can be relocated to the project where they are most needed, not being bound to a distinct one. Besides, common test cases and scripts that were automated once can be reused through different projects.

Wrong Time for Test Automation

Another test automation pitfall is the difficulty of choosing the right time to introduce it during a development life cycle. If a test strategy implies a too-early start with scripting test cases, a test automation engineer risks finding the prepared test scripts and suites outdated in the next iteration due to requirements changes. At the same time, starting with test automation too late may result in excessive testing time and costs.

TCoE consolidates test automation best practices and QA professionals with the deep knowledge of software and its requirements. Thus, it’s much easier for TCoE managers than for each test lead separately to evaluate a certain project and decide when it’s the safest time to develop and implement automated tests.

On a Final Note

The relevance of test automation is here and is predicted to gradually grow in the future. Still, automated testing has a number of constraints and trying to reach higher levels of its implementation is bound to organizational challenges. TCoE can assist in solving such organizational test automation problems as an unbalanced test strategy, the lack of test automation engineers, uncoordinated testing processes, excessive test automation costs and inappropriate time for test automation.

— Andrei Mikhailau