How We Measured 65% AI-Written Code (And Why Lines Don't Matter)
AI AdoptionJanuary 2, 20264 min read

How We Measured 65% AI-Written Code (And Why Lines Don't Matter)

After publishing what we learned from our Claude Code pilot, I got a flood of questions. Not about adoption strategies or developer feedback - about measurement. How do you actually quantify "65% AI-written code"?

The answer is simpler than you'd think. But it requires letting go of a metric that's been with us for decades.

Forget Lines of Code

Most AI coding tools - Claude Code, Cursor, GitHub Copilot - give you a "Lines of Code Accepted" counter out of the box. It sounds useful. It isn't.

That metric was designed for the autocomplete era. You'd ask the AI to implement a function, it would offer a suggestion, and you'd accept or reject it. In that world, "accepted" meant something.

In the agentic world, it means nothing.

When you work with an agent, you plan together, then the agent codes autonomously - writing, building, testing, iterating - and you review the output. Every line the agent writes during that autonomous phase counts as "accepted" automatically. The metric inflates to meaninglessness.

Worse: if you throw away the entire output because the plan was wrong, that counter doesn't adjust. You could "accept" 500 lines that never ship.

Lines of code also fail for a simpler reason: they never mattered. Two developers implementing the same task can produce wildly different line counts depending on style, patterns, and preferences. Same work, different numbers. That's not measurement - that's noise.

Commits as Units of Work

We went back to basics. A commit is a unit of work. You take a task, implement it, commit it. If it's a big task, you split it into reviewable pieces - the same practice we've followed for years.

Claude Code handles this naturally. When you ask Claude to commit your changes, it writes clear commit messages and adds a header automatically:

Co-Authored-By: Claude <[email protected]>

No extra steps. No friction. The metadata is just there.

From that point, measurement becomes straightforward: count commits with the header, count commits without. That ratio is your AI-written percentage.

The Technical Implementation

We ran this on GitLab. I knew who was in the pilot. GitLab knows what commits each person made. Using scripts that crawled commit history - scripts Claude wrote for me, which felt appropriate - we generated a report of every commit, its message, and whether it carried the Co-Authored-By header.

The analysis from there was trivial. Filter, count, calculate percentages.

This same approach scales to thousands of developers. You can slice it by person, team, repo, or the entire org. The data is already in your version control system. You just need to query it.

GitHub works the same way. The header convention is identical. If you're on GitHub, the same methodology applies.

The Gap Between Measured and Actual

Our pilot measured 65%. The real number was closer to 80%.

The gap came from engineers who didn't adopt the commit practice. They'd work entirely with Claude Code but commit manually out of habit, missing the header. When I followed up with them directly, they swore they couldn't go back - that every line of code during the pilot was written with Claude.

That's the limitation of any opt-in tracking: you measure the floor, not the ceiling. But 65% as a floor was more than enough to validate what we were seeing.

What I'd Tell Other Teams

If you're trying to measure AI adoption, start here: commits with co-author attribution, compared to total commits. It mirrors how you already work. It doesn't require new tooling. It scales.

If you have a better method, I'd genuinely like to hear it. But after running this pilot across 30-40 engineers and preparing to scale it to 300+, I haven't found one.

The metric that matters isn't how many lines the AI wrote. It's how many units of work the AI delivered. Commits capture that. Lines don't.

FAQ

Frequently Asked Questions

Common questions about this article

Jonathan Barazany

Jonathan Barazany

Chief AI at Nayax. Previously 10 years at Microsoft building data systems and leading engineering teams. Writes about AI agents, data engineering, and technical leadership.

Found this helpful? Share it!