Companion to slide 11 of the Builders' Day deck. A one-page index of every error we found in the UN System Data Commons, with the scale of each and a link to its full evidence. The evidence itself lives in Finding: data quality across the graph, which is the document prepared for the platform team — this page exists so a reader coming from the deck can see the whole list at once and jump to the one they care about.

How they were found. probe/smell.py swept the governed graph — 1,661 indicators, 1,701,211 observations, every reporting country and every year — with checks that need no subject-matter knowledge: a percentage outside 0–100, a negative count, a rate exceeding its own denominator, a value far outside its own indicator's distribution. The sweep produced 3,844 findings. Seven survived a hand check against the indicator's own distribution. A cluster of flags is not a count of errors, which is why the number on the slide is seven and not 3,844.

The seven

# Indicator Where What is wrong Scale Full evidence
1 VC_SNS_WALN_DRK feel safe walking alone after dark Kyrgyzstan 2021–23 6,710–6,990 on a Percent field; every other observation is 22.8–95.0 ×100 detail
2 EN_MWT_RCYV municipal waste recycled South Africa 2018–23 up to 3.44×10⁹ t — more municipal waste than Earth generates in a year ×1,000 (kg as t) detail
3 EN_HAZ_PCAP hazardous waste per capita Brunei 2016–23 8.7–36 million kg per person; global median is 22 kg national total in a per-capita field detail
4 EN_EWT_* e-waste, four indicators Guadeloupe 2022 13.71 → 13,951 kg/capita and 5,472 → 5,367,000 t, collected and recycled ×1,000, propagated detail
5 SI_RMT_COST average remittance cost Malawi 2023–24, Myanmar 2022 a cost of −0.10, −0.93, −0.56 %, between normal years sign detail
6 STR_WORK_NB workers in strikes Brazil 2015–19 1.28 bn workers — six times the population, fourteen times any country's record ×400 against its own series detail
7 EAR_INEE_NB_PPP minimum wage, PPP int'l $ Slovenia 2000–06 110–175k against a global maximum of 10,259; breaks at 2007, the year it adopted the euro unconverted tolar detail

Four of the seven are unit or scale slips, one is a sign, one is a field mix-up and one is a currency that was never converted. None needs subject expertise to see once the indicator's own distribution is in front of you — which is the argument for running checks like these from outside the platform.

And one that is not an error

VC_DSR_MORT — deaths due to disaster — United States, 2020: 345,950. 2021: 470,644. Across the other 163 countries the median is 42. These are COVID-19 deaths, classified as a disaster and reported here; most countries did not. Presumably correct reporting, and that is the problem: two numbers share a variable, a unit and an axis and are not the same measurement, and nothing in the observation metadata would let a tool tell them apart.

This is the single clearest case for the comparability grading this project is built around, and the one we most want to discuss. Full note.

Counted carefully

Malaysia's 147.7 % municipal waste recycled (2019) is not among the seven. It appears on the demo clipped off the axis in red rather than deleted, because a silent drop is how a dataset launders its own errors. But a share above 100 % can be legitimate for an indicator like this, so it is unresolved, not verified — the same reason Kuwait's 3,850 % water stress turned out to be correct.

What we checked and dismissed is published alongside the findings, so the checks' precision can be judged rather than taken on trust: Kuwait's water stress (desalination and fossil groundwater), 132,810 disaster-affected persons per 100,000 in the Marshall Islands (a person counts once per disaster), euro-area and CFA countries sharing a conversion factor, Iran's minimum wage at 23× the 99th percentile (the unit is local currency), and 23,172 negative percentages. The full list.

Three structural observations

Not errors, but properties of the graph that make errors harder to catch. Each is argued in full in the finding.

  1. Percent covers bounded proportions and signed rates of change, with nothing in the unit string to tell them apart — which is why a flat "percentages are 0–100" rule produced 23,172 false positives. A distinct unit, or a bounded: [0,100] property, would make a large class of errors mechanically detectable. Our workaround is to treat each indicator as its own control group.
  2. 170 of the 689 SDG base indicators carry no name, and every unnamed one we have examined holds zero observations. Unnamed and empty are the same set — so a search over names cannot tell you a concept is absent from the framework.
  3. Enumeration depends on where you start. undata/topic/Root yields 1,661 base indicators; the seventeen SDG goal trees yield 689 — and six indicators are reachable from the goal trees and not from Root, although all seventeen trees are Root's direct children. ->relevantVariable is not transitive, so a client enumerating from a single root gets a silently incomplete set. Detail.

Reproducing any of it

git clone https://github.com/sarapis/undatacommons-nyc
python3 probe/corpus.py --roots all
python3 probe/smell.py --all --corpus probe/cache/corpus-all.json

Stdlib only. Writes docs/artifacts/smell-<date>.{json,md}; the JSON carries every finding with its DCID, place, year, value, unit and provenance.