Leapwork report reveals flawed AI testing, highlights need for change

Recent research from Leapwork has highlighted concerns regarding the efficiency of software testing practices among businesses, especially in the context of AI integration.

According to the study, only 16% of companies believe their testing processes are efficient. The research surveyed 401 senior and technical professionals in the United States and the United Kingdom.

One of the key findings of the research is that although 85% of the respondents have incorporated AI applications into their technology stacks over the past year, a significant majority—68%—have encountered issues related to performance, accuracy, and reliability of these applications. The deficiencies in testing processes have negatively affected the usability of AI apps, which are becoming increasingly prevalent.

The research shows a disparity in perceptions between different levels of expertise within companies. Among C-suite executives, who comprised 50% of the respondents, 73% reported perceiving significant issues with AI app performance, compared to 62% of software engineering or technical leads. This highlights a growing demand for more rigorous software testing practices.

Robert Salesas, CTO at Leapwork, commented on the growing realisation of AI's limitations. "For all its advancements, AI has limitations, and I think people are coming around to that fact pretty quickly," he said. "The rapid automation enabled by AI can dramatically increase output, but without thorough testing, this could also lead to more software vulnerabilities, especially in untested applications."

The survey pinpointed specific issues with AI applications, with 23% of respondents citing security vulnerabilities, and 21% identifying integration failures as common bugs. Additionally, some of the primary challenges in incorporating AI into workflows were organisational resistance to change (20%), inconsistent performance and reliability (19%), and managing the fast pace of AI advancements and updates (19%).

Despite recognising the importance of testing AI—77% of respondents deemed it essential—substantial gaps were identified in existing testing resources and practices. Notably, 24% of organisations do not have a dedicated team or individual responsible for testing AI apps, and 26% lack a commercial testing platform. Furthermore, nearly a third (30%) of respondents believe that their current testing processes are inadequate for ensuring reliable AI applications.

Christian Brink Frederiksen, CEO of Leapwork, underscored the critical need for comprehensive testing in today's interconnected digital environments. "There have been too many outages this year alone, many of which affected millions of customers for big brands. We've been given a wake-up call no one can ignore," he said. "What makes digital infrastructure today so tricky to test is the copious amount of complex, interconnected applications. A tiny error in one application could have a monumental cascading effect and shut down businesses."

Frederiksen stressed the importance of moving beyond traditional, isolated testing methods: "Whether big or small, all updates need appropriate testing, but many businesses have outdated, siloed approaches. It shouldn't be about testing one individual app—it should be about testing the entire user journey."

As AI becomes embedded across various facets of business operations, ensuring these systems perform flawlessly has become an imperative. Leading technology officers must transition to a holistic testing approach, where every application and user journey is comprehensively assessed. This shift is essential for maintaining operational resilience and customer trust in an era increasingly dominated by AI-powered solutions.