↵ select ↓ ↑ navigate esc close

Paper Feed: Artificial Intelligence in the Firm: Bottlenecks in Software Production

Paper Feed ·

Listen: https://serve.buildtall.systems/27d724374ef81843e3a34533943198acb19cad7d33364f2e3f151c2d3fd54359.mp3

Source: https://fion.ac/jellyfish.pdf


The document is Artificial Intelligence in the Firm: Bottlenecks in Software Production, a job market paper by Fiona Chen and James Stratton of Harvard University. It asks what happens to a firm's output and staffing when its software engineers adopt AI coding tools.

The question matters because AI often looks impressive on individual tasks, but firms do not run on individual tasks. Production involves several steps that depend on one another, and the slowest step can cap what the whole process delivers. The authors study software engineering because it has seen fast gains in AI capability and quick uptake, and because the steps from writing code to shipping it are well documented. They separate two kinds of tools. AI coding assistants suggest code in real time as an engineer types. AI coding agents take a higher-level task and carry out multi-step work on their own, writing code across several files, running tests, and iterating until the task is done.

The data come from Jellyfish, a company whose analytics platform tracks engineering teams. The authors use roughly 300 million work events from 718 firms, from January 2021 to March 2026. These include code commits, which are incremental saved changes to a codebase, and pull requests, which are formal submissions of code for review. They also include Jira issues and epics, which are tracked units of engineering work and larger projects, along with calendar events, LinkedIn-based headcounts, and records of AI tool usage. The main method is a staggered difference-in-differences design. Firms adopt AI tools at different times, so the authors compare each firm's activity before and after its adoption with firms that have not yet adopted. The key assumption is that, without adoption, early, late and non-adopting firms would have followed similar paths. The authors check this against the trends before adoption, and they control for firm size, since larger firms tended to adopt earlier.

On productivity, the findings are clear. AI assistants raise commits by about nine percent, a statistically significant result, while their effects on lines of code, about twelve percent, and pull requests, about five percent, are not significant. AI agents have large effects: lines of code rise about thirty percent, commits about twenty percent, and pull requests about twenty-three percent, and all three are significant. The authors note that these estimates are on the lower end of other studies. They suggest this is because they measure adoption at the firm level, which averages in engineers who never use the tools much.

The next question is whether this extra coding turns into more finished software. The paper measures finished software by Jira issues resolved, which count only once code is deployed to production, and by epics. Neither rises significantly after adoption of either tool. For agents, the confidence intervals rule out an increase in issues resolved larger than about twelve percent of the baseline. That is far below the thirty percent gain in lines of code. The authors also test whether issues simply became larger or more complex, which would hide real growth in output, and find no shift in issue size. As the paper puts it, "these gains do not translate into proportional changes in software output or employment."

On employment, the authors find no significant change in overall headcount, measured through LinkedIn profiles, and no significant change in the number of junior or senior workers. For agents, they can rule out an overall decline larger than about three percent. For active engineers tracked on the Jellyfish platform, agents show no significant change, but the confidence interval is wider and allows for a decline of up to about fourteen percent. Assistants show a small, marginally significant rise in engineering employment, but the authors caution against reading it causally, because employment was already trending upward among early adopters before they adopted.

The central explanation is a bottleneck in code review. The paper's own time-use estimate, built from platform activity, finds engineers spend about forty-two percent of their time writing code. A survey of one hundred engineers recruited through Prolific put the figure closer to thirty percent. The rest goes to complementary work such as planning, review, and management. After agents are adopted, the time from submitting a pull request to merging it rises by about 3.45 days, a forty-nine percent increase over a baseline of about seven days. The share of pull requests that receive a formal request for changes nearly doubles, rising from about thirteen percent to about twenty-five percent. Comments per pull request rise by about thirty-five percent. These changes are not explained by larger pull requests, because the size of each pull request did not change significantly. Assistants show no significant effect on any of these review measures.

Firms respond by moving people into review. The share of workers who write code does not change significantly after agent adoption, but the share who perform code reviews rises by about four percentage points, a fourteen percent increase. A junior software engineer interviewed for the paper described the problem this way: "People could make a huge number of commits quickly, but code review was still the bottleneck."

Firms have also started using AI in review itself. By March 2026, nearly eighty percent of firms in the sample had used AI code review tools. Yet AI generated only about 23 percent of review comments, and only about 11 percent of pull requests received at least one AI comment. Humans remain central. A senior engineer quoted in the paper said, "Even with AI tools, you absolutely still need humans involved in the review process."

To explain these patterns, the authors build a simple model of two sequential stages. Coders write draft code, which may contain bugs. Reviewers catch bugs, but each extra hour of review finds fewer of them. AI can affect the model in two ways. It can raise coding productivity, which means more draft code arrives at review. It can also change the bug rate, meaning the quality of the draft code. The model shows that a productivity gain alone raises good output, but by less than it would if review capacity could grow with it. A rise in the bug rate requires more review time per unit of code and lowers good output. Either channel shifts the balance of labor toward review. The authors read the rise in change requests and comments as evidence of the quality channel, inferring this through the model rather than observing bug rates directly. The model also suggests a practical lesson: if the bottleneck comes from productivity, reallocating labor toward review can ease it, but if it comes from lower code quality, the fix requires better technology.

The paper is candid about its limits. The employment estimates are short-run, and they describe a single firm adopting AI, so they cannot show what happens across the whole economy. Some growth at adopting firms could come at the expense of competitors. Take-up is measured only through business licenses, so individual engineers who use tools on personal accounts would not appear in the data. The assistant employment result depends on trends that were already rising before adoption. The authors also note that their data partner, Jellyfish, reviewed the manuscript only to check for proprietary information, and that the conclusions belong to the authors.

The main takeaway is that AI can speed up writing code much more than it speeds up delivering software. As the paper's conclusion states, "code review becomes the bottleneck." Its practical advice is that managers should measure the whole production process rather than relying on lines of code or commits, because, in the authors' words, "Changes in one production stage may shift the bottleneck to another, and hence necessitate reallocation of labor."

  1. Artificial Intelligence in the Firm: Bottlenecks in Software Production