The security dashboard data problem

If your security dashboards have ever disagreed with each other, or you have ever been unable to recreate a number from last quarter, you have hit this. It is not a tooling failure and a new dashboard will not fix it. The problem lives in the data underneath, and it has four specific causes worth naming.

Cause 1: Sources count things differently

Two vulnerability scanners do not agree on what "critical" means. Two asset tools do not agree on what counts as an asset. Your EDR and your CMDB have different ideas about how many endpoints you have.

Stack these on a dashboard and sum them and you get a number that is precise and meaningless. The dashboard shows it with total confidence, because a dashboard cannot tell the difference between a real number and a nonsense one. It just draws.

The fix is normalization, reconciling those definitions before anything gets charted. No dashboard tool does this for you.

Cause 2: History rolls over

Most security tools are built to show you now. Many quietly discard or overwrite older data. So when the board asks for a trend across the last four quarters, you cannot produce it, because two of those quarters no longer exist in the source.

Boards live on trends. A point in time number tells them almost nothing. The fix is keeping your own recorded history of computed numbers, independent of whether the source tool still holds the raw data. Again, not something a dashboard does.

Cause 3: No provenance

Someone asks "how did you arrive at this number?" In a lot of programs the honest answer is "an analyst exported some data into a spreadsheet a few months ago and did some math we cannot fully reconstruct."

That is not defensible. In a regulated industry it is worse than not defensible. The fix is provenance, keeping a trace of where every number came from and how it was computed, so any figure can be explained on demand. Dashboards have no concept of this.

Cause 4: Manual assembly that does not scale

For a while, willpower works. A diligent person pulls the numbers, reconciles them by hand, updates the spreadsheet, feeds the dashboard. Then sources multiply, quarters accumulate, the person gets busy or leaves, and the whole thing quietly degrades. You usually discover it has broken at the worst possible moment, in front of an audience.

The fix is automation of the collection and computation, on a schedule, not by heroics.

Why tools do not fix any of this

Look at the four fixes. Normalize, record, trace, automate collection. Now look at what a dashboard tool does: it reads a data source and draws charts.

There is no overlap. The dashboard tool sits entirely above all four problems. It assumes they are already solved and renders whatever it is given. That is why swapping Grafana for Power BI, or buying a more expensive tool, changes nothing about whether your numbers are trustworthy. You are upgrading the layer that was never the problem. The tool comparison is still worth reading, but go in knowing the tool is not where this gets solved.

What actually fixes it

All four problems share one root: there is no disciplined, automated, recorded, traceable layer that produces your numbers before they reach the dashboard. Build or buy that layer and all four problems go away at once. It is the data layer beneath the dashboard, and treated as its own system it has a name: a security KPI system of record.

That is the thing to evaluate. Not which dashboard draws the prettiest chart, but whether you have a record underneath it that you can defend.