In recent years, there has been a loud buzz about how generative AI (gen AI) is making software developers’ lives easier since it generates code snippets, suggestions, and related documentation on demand. Many developers feel that productivity is pushed intuitively, of course. Now we have hard data to back up that premise.
However, it’s notable that AI is more of a help to developers newer to the job than their senior counterparts, a new study, conducted by researchers from MIT, Microsoft, Princeton University, and the University of Pennsylvania, concludes. The researchers analyzed data from three trials conducted at Microsoft, Accenture, and an unnamed Fortune 100 electronics manufacturing company involving 4,867 developers. Regrettably, 40% of the participants in the Accenture segment were laid off midway through the experiment, the researchers noted. Hence, a real-life twist mitigating the glories of AI.
Also: Early adopters are deploying AI agents in the enterprise now, with scaled adoption in 2025
These experiments were run by the companies as part of their ordinary course of business, with developers using GitHub Copilot –an open-source AI-based coding assistant that suggests intelligent code completions. “Our analysis reveals a 26% increase in the number of completed tasks among developers using the AI tool,” the researchers state.
Importantly, less experienced developers showed higher adoption rates and greater productivity gains from AI. “Copilot significantly raises task completion for more recent hires and those in more junior positions, but not for developers with longer tenure and in more senior positions,” the researchers noted. Short-tenure developers increased their output by 27% to 39% while long-tenure developers had more marginal gains of 8% to 13%.
Also: Businesses still ready to invest in Gen AI, with risk management a top priority
The researchers even speculated that mature developers abandoned CoPllot after initially trying it. “Copilot is relatively easier and less costly compared to other AI tools in the workplace,” they state. “Copilot does not require any complementary investment, can be adopted at the individual level, and is already integrated into the software development environment.”
<!–>
However, despite these advantages, the adoption rate for CoPilot “is significantly below 100% in all three experiments, with around 30% to 40% of the engineers not even trying the product,” they pointed out. “Moreover, the adoption rates are remarkably similar across the experiments. This suggests that factors other than access, such as individual preferences and perceived utility of the tool, play important roles in engineers’ decisions to use this tool.”
Along with an overall productivity increase, the study unearthed boosts in specific coding tasks. There was a 14% increase in the number of code updates (commits) and a 38% increase in the number of times code was compiled.
Also: 1 in 5 top companies mention generative AI in their financial reports, but not in a good way
Fortunately, software work is relatively easy to measure, following a highly structured workflow, where specific tasks are defined and tracked through version control software. “Internally defined goals and tasks are, therefore, quantifiable,” the researchers pointed out.
