Why work-item metadata is the right place to look
Delivery quality lives in how teams write, test, and close work — not in how many story points they burn down. All of that is recorded in Azure DevOps work items: acceptance criteria, linked test cases, linked bugs, states, and timestamps. You can read those signals across an entire portfolio without ever touching a line of source code.
That distinction matters for scale and for trust. Because the measurement uses metadata only, code never leaves the Azure DevOps boundary, so you can safely scan dozens of projects at once — which is exactly what you need to compare teams fairly.
The six signals that actually indicate quality
Not every field is a quality signal, and no single ratio tells the whole story. These six, computed per project, cover specification, testing, defect load, flow, and organizational consistency.
| Signal | What it tells you | Healthy |
|---|---|---|
| Acceptance-criteria discipline | Is "done" defined before work starts? | High |
| Test cases per story | Is work verified, not just built? | ≥ 0.8 |
| Bugs per story | Defect load relative to output | < 0.3 |
| Story closure / throughput | Is work actually finishing? | Steady |
| 90-day staleness | Is the project alive at all? | Any = Red |
| Process fragmentation | How many process templates run | Few |
1. Acceptance-criteria discipline
The strongest leading indicator of quality is whether teams define what "done" means before they build. There's a subtlety here that trips up most reporting: acceptance criteria only count if they're in the dedicated AcceptanceCriteria field. Many teams write perfectly good criteria but bury them in the Description, where tooling and done-criteria enforcement can't read them. Measuring the strict, dedicated-field rate — not just "AC appears somewhere" — is what separates real discipline from the appearance of it.
2. Test cases per story
Divide linked test cases by stories in a project. A ratio at or above 0.8 suggests work is being verified rather than shipped on faith. A ratio near zero means the project has essentially no automated evidence that features do what they claim.
3. Bugs per story
Divide bugs by stories. Below roughly 0.3 is healthy. A high ratio points either to weak up-front specification (see signal 1) or thin testing (signal 2) — the three signals reinforce each other, which is why you read them together.
4. Story closure and throughput
Stories that open and never close, or a throughput that has quietly flatlined, indicate work is getting stuck. This is a flow signal rather than a specification one, but a portfolio full of open-forever stories is rarely a healthy one.
5. 90-day staleness
A project with zero work-item activity in the last 90 days is auto-flagged Red. It doesn't matter how good its historical ratios were — a project nobody is touching is either abandoned, undergoing silent scope creep elsewhere, or a licensing and governance liability.
6. Process fragmentation
Count the distinct process templates the portfolio runs (Agile, Scrum, CMMI, custom variants). Every extra template is another set of fields, states, and rules that reporting has to reconcile. High fragmentation makes portfolio-wide measurement harder and usually reflects governance drift.
Rolling projects into a red/amber/green portfolio read
Per-project numbers are only useful if a leader can act on them. The move is to convert each signal into a red/amber/green band against its threshold, then give every project one worst-driven verdict — a project is only Green if it clears every gate, and any Red signal (staleness especially) pulls the whole project Red.
Sort the portfolio by verdict and dozens of teams collapse into a single screen that answers "where do I look first?" In one illustrative portfolio scan of 52 projects, the Reds clustered tightly around two problems: acceptance criteria written into the Description instead of the dedicated field, and test-case ratios well under 0.8 — a pattern you can only see once every project is measured the same way. (Figures illustrative.)
From verdict to behaviour change
A verdict is a diagnosis, not a fix. The value comes from pairing each Red or Amber with a concrete behaviour-change target — "move acceptance criteria into the dedicated field," "add test cases until the ratio clears 0.8," "close or archive stories idle past 90 days." That turns a dashboard into a punch list a delivery lead can actually run.
See your portfolio's delivery-quality read — free
HAPI (Helix ADO Portfolio Inspector) scans your Azure DevOps portfolio using work-item metadata only — code never leaves your ADO boundary — and returns a red/amber/green verdict per project with concrete behaviour-change targets, plus a shareable executive report and a per-project workbook.
Run a free scan →Frequently asked questions
What signals indicate delivery quality in Azure DevOps?
The most reliable signals come from work-item metadata: acceptance-criteria discipline, test cases per story, bugs per story, story closure and throughput, 90-day staleness, and how many distinct process templates the portfolio runs. Together they tell you whether work is well-specified, tested, and actually moving.
What is a healthy ratio of test cases and bugs per story?
As a rule of thumb, test cases per story is healthy at roughly 0.8 or higher, and bugs per story is healthy below about 0.3. A project with zero work-item activity in the last 90 days is auto-flagged Red regardless of its other ratios.
How do you roll project quality into a portfolio view?
Compute each signal per project, convert each to a red/amber/green band against its threshold, then give every project a single worst-driven verdict. Sorting projects by verdict turns dozens of teams into a one-screen portfolio read that shows where to look first.
Does measuring delivery quality require access to source code?
No. HAPI measures quality from work-item metadata only — acceptance criteria, test links, bug links, states, and timestamps. Code never leaves the Azure DevOps boundary, so a scan is safe to run across an entire portfolio.