PR Risk Review: Catch Incidents Before Merge (2026)

Key Takeaways

• A pull request risk review is a pre-merge assessment that reads a change against the live system it is about to affect, and returns a verdict on whether it is likely to cause an incident. It is not a linter and not a code-style review: the evidence it needs is your running infrastructure, not just the diff.
• The premise is Google's, not ours. The SRE book states that "roughly 70% of outages are due to changes in a live system". If most incidents arrive by deploy, the cheapest place to stop one is the review that already exists.
• Static policy tools cannot do this job, by their own documentation. Open Policy Agent evaluates the Terraform JSON plan, and its docs are explicit that some information "may not be available at plan time." A plan file does not know that the service you are touching paged someone twice last week.
• The verdict should be advisory. Aurora posts an approving or commenting GitHub review and never blocks the merge. A reviewer that cannot be overruled becomes a reviewer that gets switched off, and a false block during an incident is worse than a missed finding.
• Measured AI results in this area are real but modest, and worth stating honestly. Meta reported 42% accuracy at putting the root cause in its top five suggested changes on its web monorepo, and Microsoft's RCACopilot paper reports RCA accuracy up to 0.766. Anyone promising near-certainty is selling something.
• Aurora's implementation is open source, so you can check these claims rather than take them. The reviewer has its write paths removed from the toolset, treats the pull request text as data rather than instructions, and emits exactly one of two verdicts. Every mechanism described here, including where the restrictions stop, is auditable in the repository.

A pull request risk review is an automated pre-merge assessment that reads a proposed change alongside the state of the production system it will affect, then returns a verdict on whether merging it is likely to cause an incident. The distinguishing feature is not the model, it is what the reviewer is allowed to look at: the diff plus read access to your monitoring, deploy, and CI tooling. A code reviewer sees the diff. An incident-risk reviewer can also ask whether the service this change touches is healthy right now.

This post is for platform and SRE teams who already run linters, tests, and policy checks in CI, and still get paged for changes that passed all of them.

Why review for incident risk before merge?

Because the arithmetic of change-induced failure is not close.

Google's SRE book puts it plainly: "roughly 70% of outages are due to changes in a live system". That single sentence reorganizes where reliability effort belongs. If most incidents are delivered by your own deploy pipeline, then detection, paging, and investigation are all downstream of a moment you already control.

DORA formalizes the same idea as a metric. Change fail rate is defined as "the ratio of deployments that require immediate intervention following a deployment. Likely resulting in a rollback of the changes or a 'hotfix' to quickly remediate any issues." Note that DORA now publishes five metrics rather than the original four keys, splitting throughput from instability, and that its own guidance cautions against using them for broad organizational comparison rather than at the service level.

So the question is not whether pre-merge review is valuable. It is what the reviewer needs to know to be right.

What can static checks catch, and what can they not?

This is the part most writing on the topic skips, and the answer comes from the tools' own documentation.

Open Policy Agent's Terraform guidance describes evaluating the JSON plan produced by terraform show -json, inspecting resource_changes for types and actions. It then states the limitation directly: some information "may not be available at plan time," specifically unknown values, dynamic blocks, and function calls. The advice is to "confirm that the information you need is available in the plan JSON output" before writing a policy.

That is an honest description of a static analyzer, and it defines the boundary precisely.

Check	Reads	Catches	Cannot know
Linters and type checks	Source text	Syntax and type errors, style violations	Anything about runtime behavior
Unit and integration tests	Code under a harness	Broken logic you thought to test for	Production data shapes, real traffic, real scale
OPA and plan policy	The Terraform plan JSON	Forbidden resource types, missing tags, open security groups	Values unknown at plan time; whether this service is currently unhealthy
Progressive delivery like Argo Rollouts	Live metrics after partial deploy	Regressions that show up in the canary window	Nothing pre-merge; it runs after you ship
Incident-risk review	The diff plus live system state and incident history	Changes that are individually valid but risky for this service right now	Anything genuinely novel; it is a probability, not a proof

The pattern is that every existing layer is either blind to production or runs after merge. Argo Rollouts is genuinely excellent at what it does: an AnalysisTemplate defines how to evaluate a canary, an AnalysisRun executes it, and a failed analysis "causes the Rollout to abort, setting the canary weight back to zero," leaving the rollout in a degraded state. That is real protection, and it is protection you receive after the change is already in front of users. We covered that half of the loop in the CI/CD auto-remediation guide.

The gap is a reviewer that knows both the change and the current state of the thing being changed, before anyone merges.

How does a pull request risk review actually work?

Here is Aurora's implementation end to end. It ships as a per-repository toggle labeled Incident Prevention, and every mechanism below is in the open-source repository.

Every cheap check runs before any tokens are spent

A webhook arrives from a GitHub App with an HMAC-SHA256 signature verified in constant time against the raw body. Then a chain of cheap deterministic checks, in order: the feature flag is on, the action is one of opened, reopened, ready_for_review, or synchronize, the pull request is not a draft, the base branch is the repository default, an App installation exists, the payload carries the fields needed to identify the change, a Redis check deduplicates on repository plus pull request plus head SHA, the installation is not suspended, and the repository has the review toggle explicitly enabled.

The per-repository toggle defaults to off, so nothing reviews your code because a webhook happened to fire. Note that the global feature flag defaults to on, which is the opposite default and worth knowing: the thing standing between a webhook and an agent run is the per-repository opt-in, not the global switch.

The ordering is the interesting design choice. Every filter is deterministic and nearly free, and they all run before the expensive part. A pull request that fails any of them costs nothing: no agent, no tokens, no latency.

The reviewer's write paths are removed

A reviewer that can write to production is not a reviewer, it is a risk. Aurora restricts the reviewer in two places, and it is worth describing them accurately rather than as a slogan.

The first is a flag specific to pull request review, applied when the agent's tool set is built. It removes cloud command execution, the Terraform tool, the commit tool, remote SSH, and on-prem kubectl before anything else runs, so those capabilities are simply absent from the reviewer.

The second is the agent's read-only mode, which drops the remaining mutation-capable tools by name and narrows infrastructure tooling to inspection verbs: plan, state inspection, outputs, refresh.

These are complementary rather than individually sufficient, and one gap is worth naming: a general shell tool remains in the tool set, restrained by the command policy and safety gates described in our AI agent guardrails post rather than by removal. So the accurate claim is that the reviewer's write path to your infrastructure and repository is removed, while general command execution is governed by the same policy layers that govern every other Aurora command. If that distinction matters for your threat model, and it should, our kubectl safety guide covers how those layers work.

There is a subtler threat here that deserves naming. A pull request's title, body, filenames, and diff are all written by whoever opened it, which makes them untrusted input in the prompt injection sense. A malicious or merely mischievous author could try to talk the reviewer into declaring a risky change safe. Aurora escapes that text, wraps it in explicit delimiters, and instructs the agent to treat it strictly as data to review rather than instructions to follow. The read-only enforcement means injection cannot make the agent do anything, so the residual risk is the verdict itself, and that is what the escaping protects.

What the reviewer reads

The diff, pre-fetched and rendered file by file. This is the only evidence gathered up front. It is split per file rather than concatenated into one blob, and budgeted rather than unbounded, so the agent attends to each file in turn instead of skimming a wall of text.

Read access to your operational tooling, which it is instructed to use. The prompt directs the agent to check recent alerts on the affected services, deploy history and CI status, and service health. This is the part a static analyzer structurally cannot do, and it is what can change a verdict from "this code is fine" to "this code is fine but this service is already degraded."

Two honest qualifications on that second point, since this is where AI review posts usually get vague. First, these are tools offered rather than a fixed pipeline: what the agent can actually consult depends on which connectors you have configured, and whether it consults them on a given pull request is a decision the model makes. Second, an agent given a read tool does not reliably use it, which is one more reason the output belongs in a comment a human weighs rather than in a gate.

One verdict, two values

The output is exactly SAFE or RISKY. Two values, not a five-point severity scale, because a scale invites the reviewer to hedge and the reader to ignore the middle.

Findings carry a severity, a file, a line range, a title, and an explanation. Two normalization rules keep the output honest: a SAFE verdict carries no findings, and a RISKY verdict that names no specific finding is demoted to SAFE. An agent that says "this feels risky" without pointing at a line is not allowed to say anything.

Each finding gets a stable fingerprint derived from its file path and normalized title, deliberately excluding the line number, so as lines shift across commits the same issue keeps the same identity instead of being re-reported as new.

Why should the verdict be advisory rather than blocking?

Because a blocking reviewer optimizes for the wrong failure.

The GitHub reviews API offers three review actions: APPROVE, REQUEST_CHANGES, and COMMENT. Aurora uses the first and third. It never requests changes, never creates a blocking status check, and never gates a merge.

Consider the two ways a risk reviewer is wrong.

A false negative costs you the thing you already had: a change ships and might cause an incident, exactly as it would have without any reviewer. You are no worse off than the status quo.

A false positive on a blocking reviewer costs you a mitigation. The change that most needs to ship quickly is the one fixing a live incident, and it is precisely the change most likely to look alarming: it touches production configuration, under time pressure, on an already-unhealthy service. A reviewer that blocks that merge is actively causing harm during an outage.

There is a second-order effect that matters more. A blocking check that is wrong even occasionally gets bypassed, and a bypass path that gets used becomes a bypass path that is always used. The reviewer ends up disabled, which means its true positives are lost too. Advisory findings survive being wrong.

This is the same reasoning behind the human approval gates in our automated incident remediation guide. The agent's job is to put evidence in front of a person, not to hold the door shut.

One consequence worth designing for: if the reviewer approved an earlier commit and later commits introduce new risk, the stale approval has to go, or the pull request shows a green check that is no longer true. Aurora dismisses the prior approval with an explicit reason. Incremental reviews also never approve, only comment, since an incremental pass has only seen part of the change.

How accurate is this, honestly?

The published numbers from teams operating at scale are useful and modest, and quoting them accurately matters more than making the category sound better than it is.

Meta reported that its AI-assisted root cause system achieved "42% accuracy in identifying root causes for investigations at their creation time related to our web monorepo", measured by backtesting historical investigations using only information available at their start, counting a success when the root cause appeared in the top five suggested code changes. Meta also states it deliberately prioritized precision over reach to avoid misleading engineers.

Microsoft's RCACopilot paper reports root cause analysis accuracy up to 0.766 on a year of real incidents, with the diagnostic collection component in use at Microsoft for over four years.

Both are strong results. Neither is a system you would let merge your code unsupervised, and both teams say so in their own framing. Meanwhile Gartner predicts that over 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls.

The most useful corrective comes from a study run inside real engineering organizations rather than a benchmark. A mixed open and closed-source user study at Mozilla and Ubisoft, covering more than 587 patch reviews, found that 8.1% and 7.2% of LLM-generated review comments were directly accepted by reviewers, with a further 14.6% and 20.5% marked as valuable as review or development tips even when not accepted. Read those two numbers together. Direct acceptance was low, and yet roughly a fifth to a quarter of comments were judged worth having. That is a fair description of what this class of tool is: a source of context that is often ignored and occasionally important, which argues for keeping it cheap, advisory, and easy to skip.

So, plainly: a pre-merge risk reviewer is a probability engine that surfaces context a human would need minutes to assemble. Treat its output as a prompt for human attention. That is exactly why advisory is the correct posture, and it is why we say plainly that Aurora does not forecast incidents.

Where does this fit alongside AI code review?

It is a different job, and the distinction is worth being precise about because the tools look superficially similar.

Reviewer type	Question it answers	Evidence
AI code review	Is this code correct, idiomatic, and free of defects?	The diff and the repository
Incident-risk review	Is merging this likely to break production right now?	The diff plus live system state and incident history

Both are useful and they overlap very little. A tool reading only your repository cannot know that the payments service has been flapping since Tuesday. A tool reading your infrastructure is not the right thing to catch an off-by-one error. Running both is reasonable, and we have said before that Aurora is complementary to code-review tools rather than a replacement.

Aurora's scope is deliberately narrow: infrastructure, deploy, and configuration blast radius. The broader cross-cloud dependency picture Aurora builds during investigations is described in our multi-cloud incident management guide.

What this does not do

Being explicit about the boundaries, since this is a category where vendors are vague.

• It does not block merges. Advisory only, by design, for the reasons above.
• It does not predict incidents. It assesses a specific proposed change against current state. There is no forecasting model, no anomaly detection, and no time-series prediction in this reviewer.
• It does not review every pull request. Default branch only, non-draft only, and only for repositories where somebody turned it on.
• It does not write to your repository or infrastructure. The reviewer cannot commit, apply infrastructure changes, or merge.
• It is only implemented for GitHub today. The adapter is structured so other forges can be added, but only GitHub ships.

Where this leaves the review you already do

Nothing here replaces human review. The claim is narrower and, we think, more useful: the human reviewing your pull request is missing one category of information, and it is the category that causes most outages.

They can read the diff. They usually cannot, in the ninety seconds they have, check whether this service alerted overnight, what its dependency fan-out looks like, whether a similar change caused an incident last quarter, and whether anything downstream is currently degraded. That information exists. It is just spread across four tools nobody opens during review.

Putting it in the pull request, as a comment a human can disagree with, is a small change to the workflow. Given that roughly 70% of outages come from changes, it is a small change at the highest-leverage point in the system.

• GitHub: github.com/Arvo-AI/aurora
• Docs: Aurora documentation
• Related guides: Pre-Incident Detection · CI/CD Auto-Remediation · AI Agent Guardrails · Root Cause Analysis for SREs

All claims sourced. Accuracy figures are quoted from the publishing organizations with their stated methodology and caveats. Tool limitations are cited to each tool's own documentation. Aurora's behavior and its limits are described from its open-source implementation. Last verified: July 24, 2026.