Why "CI is green" isn't a quality gate
Most Azure DevOps branch policies gate on build success, a minimum number of reviewers, and linked work items. Those are necessary, and they catch broken builds. But they answer a narrow question — does this compile and pass the tests we already wrote? — not the one reviewers actually care about: does this change deliver what it was supposed to?
The gap is easy to miss. A pull request can be fully green and still implement three of five acceptance criteria, quietly leave an edge case uncovered, or touch a file that has caused regressions before. CI has no opinion on any of that. It never read the work item.
What a meaningful PR gate checks
A quality gate worth the name reasons about two things a build server can't: whether the change covers the intent of its work item, and how risky the change is given what has broken before.
| Check | Question it answers | Source |
|---|---|---|
| CI status | Does it build and pass existing tests? | Pipeline |
| Acceptance-criteria coverage | Does the change satisfy each linked criterion? | Work item |
| Uncovered criteria | Which criteria does it leave unaddressed? | Work item |
| Historical risk | Has this area regressed before? | History |
Acceptance-criteria coverage of the change
Every pull request in Azure DevOps should link to the work item it implements, and that work item carries acceptance criteria. The most useful thing a gate can do is compare the actual change to those criteria and report, criterion by criterion, which ones the change covers and which it leaves uncovered. That turns review from a vibe check into a checklist grounded in what was promised — and it surfaces the missing criterion while it's still cheap to fix, rather than in the next sprint's bug triage.
Historical risk signals
Not all changes carry equal risk. A one-line copy tweak and a refactor of a module that has regressed twice this quarter are not the same merge, even if both are green. A gate that weighs historical risk signals lets reviewers spend their attention where it actually matters, instead of distributing it evenly across every diff.
Turning checks into a merge-readiness score
Individual signals are useful, but reviewers need a single, defensible read on whether a PR is ready. Helix scores each pull request against the acceptance criteria linked to its work item and against historical risk signals, producing a merge-readiness score and a plain-language view of which criteria are covered versus uncovered — surfaced before the merge, where it can still change the outcome.
Two properties make this safe to put in the path of a real merge:
- Governed application. Any suggested change is human-approved — the gate scores and explains; a reviewer decides. Nothing is rewritten automatically.
- Audit-grade by design. Only the AI-derived analysis is stored. Raw code, build logs, and work-item bodies are not retained, so the gate can run against sensitive repositories without becoming a data-exposure risk of its own.
Helix works across both places the work lives: a native Azure DevOps extension where the pull request happens, and a web app where quality is governed across the org.
Rolling it out
You don't have to block merges on day one. A sensible path: run the gate in non-blocking mode first so reviewers see merge-readiness scores and uncovered-criteria lists alongside their normal review, calibrate what "ready" means for your teams, and only then decide where a score should become a required check. Because the signal is tied to acceptance criteria your teams already write, adoption doesn't require new process — just a clearer read on work you're already tracking.
See merge-readiness scored on your PRs
Helix scores each pull request against its work item's acceptance criteria and historical risk, and shows what's covered versus uncovered before merge — with any suggested change applied only on human approval. Helix is sales-assisted; a short walkthrough is the fastest way to see it on your repositories.
Request a walkthrough →Frequently asked questions
What is a PR quality gate in Azure DevOps?
A check that decides whether a pull request is ready to merge. Beyond confirming CI passed, a meaningful gate evaluates whether the change actually covers the acceptance criteria of its linked work item and what historical risk signals apply — producing a merge-readiness score before the merge.
Why is a green CI build not enough to merge?
CI confirms the code builds and existing tests pass. It does not confirm the change did what the work item asked. A PR can be fully green and still miss half of its acceptance criteria, which is why merge-readiness needs a check of acceptance-criteria coverage, not just build status.
How do you measure acceptance-criteria coverage on a PR?
Helix scores each pull request against the acceptance criteria linked to its work item, showing which criteria the change covers and which it leaves uncovered. Combined with historical risk signals, that produces a merge-readiness score reviewers see before merge.
Does an AI PR gate change my code automatically?
No. Any suggested change is governed — applied only with human approval. The gate scores and explains merge-readiness; a reviewer decides what, if anything, to act on. Only AI-derived analysis is stored, not raw code or work-item bodies.