Blog

How to Verify AI-Written Code Before You Merge It

AI coding agents write code that looks right. That's the problem. A function that reads cleanly, matches your naming conventions, and compiles without errors can still be calling the wrong endpoint, never actually getting invoked, or silently swallowing an error three layers down. "It looks plausible" and "it's correct" are different claims, and the gap between them is where AI-assisted code quietly breaks things.

Reviewing AI-written diffs the same way you'd review a junior dev's PR isn't enough, because the failure modes are different. A junior dev who doesn't understand the codebase usually writes code that looks wrong — bad names, obvious gaps. An agent that doesn't understand the codebase writes code that looks right and is subtly disconnected from it. You need to ask it different questions.

Here's the checklist I actually use, organized around five questions, before merging anything an agent wrote.

1. Did it actually get wired up?

This is the one people skip most, because the code review makes the new function look complete. But "the function exists" and "something calls it" are separate facts.

  • Where did the AI place the code it just wrote?
  • Is this new function actually being called?
  • Did the AI wire this up to the right endpoint?
  • Where is the API key being used in the generated code?
  • What functions did the last AI change touch?

If you can't answer "what calls this" with a concrete file and line number, don't merge it yet — you don't know if you shipped a feature or an orphaned function.

2. What's the blast radius?

Agents are good at making a local change work. They're much less reliable at knowing what else depends on the thing they just changed. This is the question that catches the "fixed it here, broke it three files over" class of bug.

  • What could break if I change this function?
  • Show upstream and downstream callers for this function.
  • What functions were impacted by this commit?
  • What would break if this feature was removed?
  • What is the user-facing impact of this change?

If the agent touched a function with a dozen callers, you want that list in front of you before approving, not discovered in a bug report next week.

3. Does it touch anything sensitive?

Agents don't know which code paths are sensitive unless you tell them, and they won't flag it themselves. This is the pass that catches security regressions that a purely functional review misses entirely.

  • Where are database writes performed?
  • Is authentication checked here?
  • Where are secrets or tokens used?
  • Show all SQL queries built with string concatenation.
  • Show all API endpoints with no authentication check.
  • Find all places where user input is not validated.

If the change is anywhere near auth, input handling, or a database write, run this pass even if the diff looks small. Small diffs are exactly where this gets skipped.

4. Did it leave dead ends?

Agents frequently generate a helper, an abstraction, or a fallback path that never actually gets exercised. It's not wrong, exactly — it's just weight the codebase now carries for no reason, and it compounds across every future agent session that has to read past it.

  • Find functions that may be unused.
  • Find unreachable code paths.
  • Detect duplicate logic.
  • Find all functions with no error handling.
  • Find all hardcoded values that should be config.

This one matters more over time than it looks like it does in the moment — a codebase full of AI-generated dead branches gets harder for both humans and future agents to reason about.

5. Does it handle failure?

The happy path is what gets demoed. The failure path is what agents most often skip, because nothing in the prompt asked for it explicitly.

  • Where does this exception get thrown?
  • Is this error handled anywhere, or does it bubble up?
  • Where is this error being swallowed silently?
  • What happens when this external service fails?
  • Where are retry attempts handled?
  • What code runs when this API returns a 500?

If you can't trace what happens when the new code's dependency times out or errors, that's the thing that surfaces at 2am, not in review.

Why this is slower to skip than it looks

Every one of these questions is answerable by hand — grep for callers, read the diff, trace the import graph in your head. The problem isn't that it's impossible, it's that it's slow enough that it gets skipped under deadline pressure, especially on a change that "looks fine." That's exactly the condition under which AI-written code most needs the check.

I built codemd.dev because I wanted these five passes to be a five-minute step instead of a reason to skip review entirely — it builds a navigable graph of a codebase and answers exactly these kinds of questions with evidence pulled from the actual call graph, not a guess based on how the code reads. There's a free local version for VS Code that builds the graph on your machine, and a hosted version for connecting GitHub repos and running the same checks from a dashboard.

Happy to answer questions on how it works or what it gets wrong — that feedback is most of why I'm posting this instead of just shipping quietly.

Audit your AI agent's work before it merges

CodeMD.dev builds the callgraph and repo map your agent's diff should be judged against, so these five questions take minutes, not a deadline-day scramble.