Article
How to Diagnose a Disputed Metric
A disputed metric is a number people keep using but no longer fully trust.
The dashboard says one thing. The spreadsheet says another. Finance has a different version. Someone in the meeting asks which number is right, and the conversation shifts from the decision to the reporting argument.
That is usually the wrong starting point.
The better question is:
What would make this number safe to use for this decision?
This article gives you a practical way to diagnose one contested KPI without turning the exercise into a full data transformation programme. Start with the metric people already argue about. Then inspect definition, source, transformation, filters, timing, ownership, caveats, and decision use.
Start with one contested number
Do not begin with the whole dashboard estate.
Choose one number that repeatedly slows decisions down. It might be revenue, margin, pipeline, churn, active customers, conversion, stock availability, delivery performance, or cost per acquisition.
The best first metric usually has three properties:
- It appears in more than one place.
- Different teams explain it differently.
- Leaders still use it for important decisions.
If the number is visible, politically sensitive, and frequently reconciled by hand, it is a strong candidate.
The aim is not to prove one person wrong. The aim is to expose why the metric is disputed and what needs to be agreed before people can use it confidently.
Step 1: Name the decision
Metrics do not need to be perfect for every possible use. They need to be trusted for a specific decision.
Ask:
- What decision is this metric meant to support?
- Who makes that decision?
- Is the decision operational, financial, commercial, or strategic?
- Does it need a live number, a final approved number, or a forecast?
- What action changes if the number moves?
This matters because two versions of the same metric can both be valid if they support different decisions.
For example, sales revenue by order date may be useful for pipeline and sales management. Finance revenue after credits, refunds, and recognition rules may be authoritative for board reporting. A forecast spreadsheet may be useful for cash planning.
The dispute starts when those different decision contexts are hidden under one label.
Step 2: Write the plain-English definition
Next, write what the metric means in normal business language.
Avoid starting with SQL, dashboard filters, or warehouse models. Those matter later, but first the business has to agree the meaning.
For a disputed metric, capture:
- The metric name.
- The plain-English definition.
- What is included.
- What is excluded.
- The intended decision.
- The business owner.
- The authoritative report, if one exists.
If people cannot agree the plain-English definition, the dispute is not primarily a dashboard issue. It is a KPI definition issue.
That is often the central reporting trust gap.
Step 3: Compare the versions people are using
List every version of the metric that appears in the business.
For each version, write down:
- Where it appears.
- Who uses it.
- Which decision it supports.
- Which source it uses.
- Whether it is manually adjusted.
- Whether it is final, live, forecast, or provisional.
This simple inventory often explains the argument.
The sales dashboard may be live. The finance pack may be closed at month end. The spreadsheet may include a manual forecast adjustment. The board pack may contain a finance-approved snapshot. Those are not automatically contradictions. They may be different answers to different questions.
The problem is that the difference is invisible.
When the distinction is useful and agreed, it may be tolerable divergence. When it is hidden or unmanaged, it becomes a Reporting Trust problem.
Step 4: Trace the source-to-report path
Once the definition is clear, inspect the path.
Ask where the number starts and how it reaches the report.
At minimum, capture:
- Source system.
- Important source fields.
- Extract or ingestion route.
- Main transformations.
- Joins and grain.
- Filters.
- Manual adjustments.
- Final report or dashboard.
- Known caveats.
This is the source-to-report lineage of the disputed metric. It shows how captured reality becomes the number people see in a meeting.
You do not need a perfect lineage diagram to start. A rough path is usually enough to reveal whether the problem sits in source capture, transformation logic, semantic meaning, manual adjustment, or the final reporting layer.
If nobody can explain the path, the metric is not ready for high-stakes interpretation.
Step 5: Inspect filters, timing, and grain
Many disputed metrics are caused by boring differences that have large business consequences.
Check timing first:
- Order date vs invoice date.
- Created date vs closed date.
- Calendar month vs accounting period.
- Live data vs closed reporting.
- Time zones.
- Late-arriving records.
- Backdated changes.
Then check filters:
- Test records.
- Cancelled orders.
- Refunds.
- Internal accounts.
- Inactive customers.
- Unpaid invoices.
- Trial users.
- Regional exclusions.
Then check grain:
- Customer vs account vs contract.
- Order vs order line.
- Lead vs opportunity.
- Invoice vs payment.
- User vs organisation.
Numbers often look plausible while being wrong for the decision because the report joins, filters, or groups data at the wrong grain.
The goal is not to make the metric more complicated. The goal is to make the important assumptions visible.
Step 6: Find the owner and caveats
A disputed metric usually has either no owner or too many unofficial owners.
Ask:
- Who owns the business meaning?
- Who owns the technical implementation?
- Who approves changes?
- Who communicates caveats?
- Where should users check whether the metric is safe to use?
Ownership is not bureaucracy. It is how the business stops the same argument recurring every month.
Caveats matter as much as ownership.
If the metric excludes a region, depends on manual finance adjustments, uses a lagging source system, or should not be used before month end, that context should be visible. These visible warnings are trust signals. They help people decide whether the number is ready for the decision in front of them.
Step 7: Decide what to fix first
After the inspection, the fix is usually clearer.
If the definition is unclear, agree the KPI definition before changing the dashboard.
If the source path is unclear, map the source-to-report lineage before rebuilding the report.
If manual adjustments are hidden, decide whether they belong in the trusted reporting process or need to remain a controlled finance step.
If two versions are both valid, rename them and document where each is safe to use.
If the metric is going into a semantic layer or AI workflow, make sure the approved meaning, caveats, owner, and safe-use context are visible at the Semantic Layer Gate. Otherwise the same disagreement will reappear as a faster automated answer.
What not to do first
Do not ask the data team to “just make the numbers match” before the business has agreed which number belongs to which decision.
Do not rebuild the dashboard before naming the disagreement.
Do not treat the spreadsheet as automatically wrong. It may contain real business judgement, manual corrections, or caveats the formal reporting layer has not captured.
Do not define every metric at once. Start with the one contested metric that causes the most friction.
A quick diagnostic template
Use this as a first pass:
Metric:
Decision it supports:
Business owner:
Technical owner:
Plain-English definition:
Included:
Excluded:
Authoritative source:
Authoritative report:
Other versions in use:
Timing rules:
Filters:
Manual adjustments:
Known caveats:
Where the number is safe to use:
Where the number is not safe to use:
First fix:This is not a full governance programme. It is a practical way to stop a metric dispute becoming a recurring meeting tax.
What to do next
Pick one metric that already causes argument.
Run the inspection above. If the lowest-trust area is definition, start there. If it is lineage, trace the source path. If it is caveats, make the limits visible. If it is duplication, decide which version is authoritative for which decision.
For a lightweight first pass, use the Reporting Trust Scorecard. If the dispute is already showing up across dashboards, spreadsheets, and finance packs, the dashboard reconciliation checklist gives you a broader inspection path.
For the wider strategy, read why your business numbers don’t match or get the free opening chapter. For working artefacts around KPI definitions, source-to-report lineage, ownership, caveats, and improvement planning, use the Reporting Blueprint Toolkit.