DORA metrics without the gaming

The DORA metrics come from years of research by the DevOps Research and Assessment program, published in the annual State of DevOps reports and the book Accelerate. Their appeal is that four numbers describe both how fast a team delivers and how stable that delivery is, and the research found the two tend to improve together rather than trade off.

They are also easy to misuse. Once a number becomes a target, people optimize the number. The goal here is to compute the metrics honestly from real data and use them to find where delivery gets stuck.

The four metrics

Two measure throughput and two measure stability:

The last one was called "time to restore service" in earlier reports. The rename narrowed it to failures caused by changes, which is easier to measure consistently than every incident. The 2024 report also introduced a fifth metric, rework rate: the share of deployments that were unplanned fixes for problems in production.

Computing them from real data

Surveys work for research, but a team improving its own delivery needs numbers from its systems. You need three sources of events:

• Deployments: from the CD system or deploy pipeline, with service, version, commit SHA, environment, time and result.
• Commits: from the version control system, with SHA and commit time, so each deployment can be traced back to the changes it contains.
• Failures: from the incident tool or a label on rollbacks and hotfixes, linked to the deployment that caused them.

The linking is the hard part. Record the commit SHA on every deployment, and make "which deploy caused this?" a required field when an incident is closed. Without those links, the stability metrics are guesses.

With a deployments table and a deployment_commits table that maps each deployment to the commits it shipped for the first time, the throughput metrics are short queries:

SELECT
  d.service,
  date_trunc('week', d.finished_at) AS week,
  count(DISTINCT d.id) AS deployments,
  percentile_cont(0.5) WITHIN GROUP (
    ORDER BY extract(epoch FROM d.finished_at - c.committed_at) / 3600
  ) AS median_lead_time_hours
FROM deployments d
JOIN deployment_commits dc ON dc.deployment_id = d.id
JOIN commits c ON c.sha = dc.sha
WHERE d.environment = 'production'
  AND d.status = 'succeeded'
GROUP BY d.service, week
ORDER BY d.service, week;

Use the median or a percentile for lead time, not the average. One commit that sat on a branch for a month will drag an average far from what a typical change experiences.

Change failure rate and recovery time come from the failure links:

SELECT
  d.service,
  date_trunc('month', d.finished_at) AS month,
  count(*) AS deployments,
  count(f.id)::float / count(*) AS change_failure_rate,
  percentile_cont(0.5) WITHIN GROUP (
    ORDER BY extract(epoch FROM f.resolved_at - f.started_at) / 60
  ) AS median_recovery_minutes
FROM deployments d
LEFT JOIN failures f ON f.caused_by_deployment_id = d.id
WHERE d.environment = 'production'
GROUP BY d.service, month;

Decide the definitions once

Most arguments about DORA numbers are arguments about definitions. Write them down before anyone looks at a dashboard:

• What counts as a deployment? A production rollout of a service. Not a config reload, not a staging deploy, not a re-run of the same version.
• When does lead time start? First commit time is the common choice and the most honest. Merge time is easier to collect but hides time spent in review.
• What counts as a failure? A deployment that led to a rollback, a hotfix, a fix-forward under incident, or user-facing degradation. Decide whether a failed deploy that never received traffic counts. Usually it should not, since the pipeline did its job.
• When does recovery end? When users stop being affected, not when the postmortem is written.

Apply the definitions the same way to every team, and publish them next to the numbers.

How the metrics get gamed

Each metric has an obvious way to improve it without improving anything:

• Deployment frequency rises if teams deploy empty or trivial changes. Count deployments that contain at least one new commit.
• Lead time drops if large changes are squashed into a single fresh commit just before merge. Measuring from the first commit on the branch, or from pull request creation, makes this harder.
• Change failure rate falls if incidents are not linked to deployments, or are reclassified as "infrastructure". Make the link part of closing an incident and review unlinked incidents.
• Recovery time shrinks if incidents are closed early and reopened as new ones.

The best defense is not clever detection. It is never attaching rewards, rankings or performance reviews to the numbers. When the metrics are used to find problems rather than judge people, there is little reason to game them.

For teams, not against them

DORA's own guidance is explicit that the metrics describe a team's delivery system, not individual performance, and that comparing teams with different products, architectures and risk profiles says little. A payments service and an internal reporting tool should not deploy at the same rate.

Useful ways to use them:

• Show each team its own trend. The question is whether delivery is getting easier for this team, not who is ahead.
• Use them to find constraints. Long lead time with frequent deploys points to slow review or long queues before merge. A high change failure rate points to testing or release practices. Slow recovery points to observability, rollback tooling or runbooks.
• Pair them with the team's view. A short regular survey on how hard it is to ship catches what the numbers miss.
• Use them for platform decisions. If a golden path is supposed to make delivery easier, the lead time of services on it compared with services off it is direct evidence.

Where to start

1. Make every production deployment emit an event with service, version, commit SHA, time and result.
2. Add a required "caused by deployment" field to incident closure.
3. Write down the definitions above, adjusted to your systems, and share them.
4. Build one dashboard per team showing the four metrics as monthly trends.
5. Review it in team retrospectives, not in management reporting, and pick one constraint to work on at a time.