Decoding the AI productivity paradox in software development

The introduction of AI coding tools is influencing the software development landscape, with engineering teams reporting changes in productivity alongside increasing challenges for existing measurement frameworks to keep pace with these developments.

According to The State of Engineering Excellence 2026 report by Harness, which gathered insights from 700 engineering practitioners and managers across multiple countries, AI adoption is now widespread in software engineering. However, organisations are assessing how to interpret reported productivity changes and their associated costs.

A reported 89% of engineering leaders indicate improvements in developer productivity, while 88% report increased developer satisfaction. At the same time, 81% of respondents note that developers are spending more time on manual work, particularly code review.

The report also highlights that approximately 31% of developer time is now spent on tasks such as reviewing AI-generated code, fixing bugs, and switching between tools, much of which is not formally tracked.

The findings suggest limitations in existing measurement frameworks:

• 89% of leaders say their current metrics reflect AI’s impact, while 94% also indicate that key factors are not captured in those same systems
• Only 6% believe current frameworks are sufficient to address these gaps

The study identifies measurement itself as a key challenge, particularly in assessing productivity, code quality, and return on investment, with organisations relying on dashboards designed for earlier stages of software development.

The report also notes differences in perception between leaders and developers regarding AI-related metrics. While 15% of managers report no concerns about how AI productivity data is used, this compares with 4% of practitioners.

Concerns about monitoring and evaluation are also present, with 54% of respondents expressing concern about performance evaluations based on AI-generated data. Additionally, 55% of developers call for a clearer separation between improvement data and performance evaluation, along with greater transparency and involvement in defining metrics.

While established frameworks such as velocity, DORA, and cycle time remain in use, the report suggests they may not fully capture the effects of AI on development workflows.

The report outlines several approaches, including:

• Expanding metrics to include code quality, validation time, cognitive load, and burnout indicators
• Treating AI performance as a separate measurement area with defined benchmarks
• Increasing developer involvement in how metrics are defined and applied

Overall, the findings indicate that organisations are adapting measurement approaches to account for changes introduced by AI coding tools, particularly in how productivity and engineering output are assessed.