"Can I trust AI to get my emissions right?" is the right question, and there's a harder one underneath it: how would you even know? A carbon number looks the same whether it's right or invented. Our team came from safety-critical engineering, building self-driving systems at Cruise, where you never trust a number you can't check against a known-correct outcome. We brought that discipline to carbon accounting. Here's how we measure the analyst, so you can judge the accuracy yourself instead of taking our word for it.
Anyone can put "99% accurate" on a slide. A percentage on its own tells you nothing, because it doesn't say what it was measured against, how often, or whether anyone can check it. So here's how we measure the analyst. Hold your own tools to the same test.
We test against answers we already know
You can't measure accuracy without a right answer to measure against. So we build them. We take real general ledgers and work out the correct footprint by hand, line by line, and lock that in as a known-correct reference. Then we run the analyst over the same data and measure how close it lands against that reference. A system worth trusting checks its accuracy against answers someone already knows.
We run it many times, and measure two different things
Run the same ledger through an AI twice and it can land in a slightly different place each time. A single pass hides that, so we run it repeatedly and measure two separate things people usually blur together:
Consistency. Does the answer wobble from one run to the next? A footprint that changes when nothing about your data changed is a red flag, and you'd never see it if you only ran it once.
Accuracy. Is it landing on the known-correct answer, or systematically off to one side? A tool can be perfectly consistent and consistently wrong. We measure both, because they're different problems with different fixes.
We separate a defensible difference from a mistake
Not every difference from our reference answer is an error. Two experienced analysts can look at the same freight line and defensibly reach for slightly different factors, and both are within the rules. If we treated every such choice as a failure, we'd be tuning the analyst to match one person's judgement call rather than to be correct. So our benchmark allows for genuinely defensible alternatives and only counts the real misses. Being honest about where the accounting really has more than one right answer is part of measuring accuracy properly.
When something's off, we find which step caused it
A single accuracy score tells you something is wrong but not what. So when a result drifts, we trace it to the step that caused it: did the analyst read the wrong number off the document (data extraction), or read it correctly and pick the wrong emission factor (the mapping step)? Those are different failures with different fixes. Naming the cause is what lets us fix the actual problem instead of playing whack-a-mole with symptoms.
Every code change is re-checked against the benchmark
The most dangerous kind of error is the one that creeps in quietly. A change to the model or the underlying factor data can improve one thing and quietly degrade another. So we re-score against the benchmark on every change, tied to the exact version under test, and keep the history. If a change would make a result worse, we see it before it reaches a customer's report, not months later when an assurer does.
“The AI worth trusting is the one whose accuracy is measured, in the open, against answers someone already knows, not the one that promises accuracy in a sentence.”
The test to hold any tool toAnd it tells you when it's unsure
Measured accuracy only matters if the tool is honest about the edges. When a factor match is low-confidence, the analyst flags it for you rather than folding it into a tidy-looking total. Being told "these lines are uncertain, take a look" is a feature. A tool that flags nothing is just hiding where it guessed.
None of this removes you from the loop, and it isn't meant to. You stay the reviewer; the measurement is what lets you review with confidence instead of taking a black box on faith. It's the same principle behind why we ground every factor decision in your own data and why an AI you can check beats one you have to believe.
See it measured on your own data
Bring a real ledger. We'll run the analyst, show the working on every figure, and flag anything it's unsure about, so you can judge the accuracy yourself.
Book a Sustainability Check-up