Fast PRs but shallow understanding

Your sprint metrics look great. But when the next production incident hits, will anyone on your team actually know how the system works?

Feb 10, 2026

👋 Hey, it’s Stephane. I share lessons, and stories from my journey to help you lead with confidence as an Engineering Manager. To accelerate your growth see: 50 Notion Templates | The EM’s Field Guide | CodeCrafters | Get Hired as an EM | 1:1 Coaching

Paid subscribers get 50 Notion Templates, The EM’s Field Guide, and access to the complete archive. Subscribe now.

Something strange is happening on your team right now.

Pull requests are moving faster. Sprint velocity looks great. Your engineers seem productive and your stakeholders are happy. But when a production incident happens (and it will) you’re going to discover that nobody actually understands the system they built. Not deeply. Not in the way that lets someone trace a cascading failure across three services at 2am.

This is the comprehension gap, and it’s growing on every team that’s adopted AI-assisted development without thinking about what they’re trading away.

The speed you’re measuring isn’t the speed that matters

When your team reports that AI tools are making them faster, they’re almost certainly telling the truth. Generating code is faster. Scaffolding a new endpoint, writing boilerplate, translating a pattern from one service to another - these things that used to take an afternoon now take twenty minutes.

But here’s the thing you’re not measuring: how long does it take your team to debug an issue they didn’t anticipate? How quickly can they trace a performance regression through code they generated but never deeply reasoned about? How confident are they proposing architectural changes to systems they assembled rather than built?

Speed to first commit is not the same as speed to production-grade, maintainable software. And it’s definitely not the same as speed to resolve the incident that happens because nobody understood the implications of what they shipped.

A friend of mine who is Director of Engineering told me his teams adopted AI tooling aggressively and their throughput numbers looked incredible for a few months. Then they hit a subtle data consistency issue that crossed three services. The engineer on call had generated significant chunks of the code in question. She could read it, but she couldn’t reason about it the way you can when you’ve wrestled with every design decision yourself. What should have been a two-hour investigation turned into a two-day ordeal that pulled in four engineers.

The code wasn’t bad. The understanding was thin though.

Comprehension

There’s a mental model I keep coming back to: the difference between navigating a codebase and inhabiting it. Navigation means you can find things, follow the flow, read the code and roughly understand what it does. Inhabiting means you have a mental model of the system’s behaviour that extends beyond what’s written - you understand why certain trade-offs were made, what will break if you change that one seemingly innocuous configuration value.

AI-assisted development is excellent at helping people navigate. It’s terrible at helping them inhabit.

This matters because your most valuable engineers aren’t the ones who write code the fastest. They’re the ones who carry rich mental models of how systems actually behave - the ones who can say “that change looks fine but it’ll cause issues with our rate limiter under sustained load” before anyone discovers it in production.

Every time an engineer delegates the thinking to a tool, they get the output without building the mental model. Do it once, no big deal. Do it systematically for months, and you’re slowly replacing deep expertise with surface-level familiarity.

As a manager, you can’t see this happening. Your dashboards won’t show it. It only becomes visible when the problems get hard enough.

If you’re enjoying this article, consider subscribing to get:

✉️ Free: 1 original post every Tuesday, my favourite posts of the week every Sunday + 10 Notion Templates for Engineering Managers
🔒 Paid: Full archive + 50+ EM templates & playbooks + The EM Field Guide

What you can actually do about this

I’m not here to tell you to ban AI tools. That ship has sailed, and frankly, they are useful. The question is how you structure your team’s relationship with these tools so you get the speed benefits without gutting the expertise that makes your team resilient.

Create “understanding checkpoints” in your development process. Before a PR that used significant AI generation gets merged, the author should be able to answer a simple question: “Walk me through the three riskiest assumptions in this code”. If they can’t, it isn’t ready to be merged - not because it doesn’t work, but because nobody owns the understanding of it. You want to ensure someone on your team can debug this at 2am without starting from scratch.

Distinguish between “delegation tasks” and “learning tasks”. Not all work is equal. Generating a standard CRUD endpoint for your fifteenth entity? Delegate to AI. Implementing a new caching strategy or integrating with an unfamiliar third-party system? That’s a learning task, and the cognitive work of figuring it out is the point. Help your team recognise the difference. A useful heuristic: if the task touches a part of the system nobody deeply understands yet, it’s a learning task.

Watch for the “confident but shallow” pattern. AI tools produce code that looks authoritative - clean variable names, reasonable structure, helpful comments. This creates a dangerous illusion of understanding. The engineer feels confident because the code looks right. You feel confident because the PR looks clean. But neither of you has pressure-tested the underlying reasoning. In your 1:1s, start asking questions like “What surprised you about this implementation?” or “What would break if we doubled the traffic to this endpoint?” If the answers feel vague, you’ve found a gap.

Make exploration and investigation part of the job. One of the effects of AI-assisted development is that it makes deep-diving through a codebase feel unnecessary. Why spend an afternoon tracing through the authentication flow when you can just ask an AI to explain it? Because the afternoon of tracing builds a kind of understanding that no summary can replace - the accidental discoveries, the “wait, why is this here?” moments, the connections between subsystems that only become visible when you walk through them yourself. Protect time for this. Explicitly.

Reframe how you talk about velocity. If your team has internalised that “more PRs merged = better performance”, AI tools will turbocharge that metric while potentially degrading the things that actually matter. Start talking about outcomes instead: incidents resolved quickly, architectural decisions made confidently, onboarding new team members efficiently. These are the metrics that reflect genuine team capability, not just throughput.

The manager’s job just got harder

AI tools don’t reduce the need for strong engineering leadership. They actually increase it. When your team was writing every line of code by hand, the understanding was baked into the process. It happened automatically. Now it doesn’t.

That means it’s on you to create the conditions where deep understanding still develops - to notice when your team is accumulating comprehension debt, to push back when “ship faster” becomes the only message from above, and to make the case that a team that understands its systems is worth more than a team that can generate code quickly.

The organisations that get this right will have teams that are both fast and resilient. The ones that don’t will have teams that look productive right up until the moment they aren’t.

Your team’s speed is impressive. Just make sure someone still knows how it all works.