HI Technology & Innovation has warned that UK engineering leaders may be overstating the productivity impact of AI coding tools because most organisations are not measuring the effect quantitatively.
The technology advisory firm’s report, AI in Software Engineering: Making Sense of the Noise, is based on interviews with more than 100 CTOs, CISOs, VPs of Engineering, and senior engineers. It finds measurable gains where AI adoption is tracked, but also shows that leadership confidence often runs ahead of the available data.
Among organisations that track AI’s effect on engineering productivity, measured gains average around 20%. Some teams reported project timelines being halved and sprint output rising by 20% to 25% after adopting modern coding assistants, although 83% of businesses are not tracking any quantitative metrics to assess AI’s impact on engineering productivity.
AI investment is already widespread enough for that absence of measurement to create a management problem. According to the report, 91% of businesses are investing in AI tools within engineering, but only 22% have a formal documented AI strategy, while around half of engineers, 52%, use AI tools even where their employer has not formally invested in them.
“AI is producing genuine productivity gains in UK engineering teams, the data on that is clear. But the gains people feel are often larger than the gains they would actually find if they measured them,” said Mike Daniel of HI Technology & Innovation. “The leaders who will get the most out of AI over the next 12 months are the ones who measure what is happening in their teams and then double down where the evidence is strongest, not the ones with the most tools or most extreme adoption.”
Developers may feel faster when AI reduces the mental burden of code generation, debugging, or boilerplate work, yet downstream effects can remain hidden without reliable metrics. Rewrite rates, review time, architectural rework, security checks, and quality assurance can all offset apparent velocity gains if organisations rely only on output volume or qualitative feedback.
Quality remains a particular concern. Only 57% of engineering leaders said they were pleased with AI output quality, while 79% reported no increase in time spent on code review, quality assurance, or testing despite the growth in AI-generated code.
HI Technology & Innovation said teams that focus AI on coding and debugging report roughly twice the velocity gain of teams that spread AI across four or more stages of the software development lifecycle. The finding points towards targeted adoption supported by a written strategy and a small number of trusted productivity and quality metrics, rather than blanket deployment across every engineering process.
The report concludes that the opportunity is not to reduce AI use, but to direct it more carefully. Engineering leaders are being pushed towards better instrumentation of the development lifecycle, where productivity, quality, risk, and rework can be assessed together rather than treated as separate conversations.
The full report can be downloaded from the AI in Software Engineering report page.
