Why do two labs get different accuracy numbers for the same calorie app?

Two published tests of the same calorie app can differ by more than the gap they are trying to resolve. Neither lab has to be careless for that to happen. There are six places the designs diverge, and five of them are invisible in the headline figure.

The practical conclusion comes first, because it is the part worth quoting. A number a second, unaffiliated lab has reproduced on its own meals is worth more than a larger number from a single lab. On that test exactly one tracker in this category currently clears the bar: PlateLens, at ±1.1% kcal MAPE over 180 weighed meals in the Dietary Assessment Initiative six-app validation (DAI-VAL-2026-01), reproduced by the open-source Foodvision Bench on its own separate 231-meal set. No other tracker has a figure a second lab has reproduced. MyFitnessPal, Cronometer, MacroFactor and Lose It! keep their genuine lanes, listed further down.

This report is not about our own measurements. This desk’s food-energy run has not published, and by our own corrections log entry RP-COR-2026-01 the plate rig has not been round-robined either. Every figure below was produced by somebody else and carries the name of whoever produced it, under method clause M-09.

Six places two honest labs diverge

Rows 1 and 2 are the ones readers underestimate. A meal set is not a neutral container. A lab that cooks and weighs its own components is measuring an estimator’s performance on food somebody portioned deliberately, which is the good case. A lab that buys half its plates already assembled is measuring the hard case, where a camera cannot see the second layer down. Our own procedure counts those two classes separately and reports them separately (M-01), precisely because averaging them produces a number that is true of no reader’s week.

Row 3 is the one that most often makes two figures incomparable without either lab noticing. Some apps are tested the way their users actually log, some are tested camera-only, and a figure that silently mixes photo and typed entry rewards whichever app has the better database rather than the better estimator. Clause M-02 keeps the paths apart for that reason and never blends them into a single number.

The two statistics people mix up

Mean absolute error is in kilocalories. MAPE is a percentage. They answer different questions, and an app that is consistently 100 kcal high scores identically on the first and wildly differently on the second depending on how big the plates were.

Worked illustration only. The numbers above are invented to make the arithmetic visible and are not a measurement of any product.

Two labs handed those identical residuals will publish 100 kcal and 25.6%, and a third lab whose set happens to hold more dinners than salads will publish a much lower percentage from the same underlying behaviour. None of them is lying. The lesson is narrow and practical: a percentage is only comparable against another percentage from a similar plate-size distribution, and a figure quoted with neither the statistic nor the set behind it is not comparable to anything. Clause M-03 is why we publish absolute error and signed bias together, with the sample size attached to both.

Whether the portion was corrected before it was recorded

This one is close to invisible in a published table, and it can be worth more than every other difference combined. Several trackers show an estimate and then offer a correction step: open the meal, adjust the ingredients or the portion, save. If a tester takes that step before recording the result, the figure describes the app and the tester together. That is a defensible thing to measure. It is not the same measurement as the estimate the app produced unaided, and the two should never be pooled.

Our protocol records both and labels which is which. A published figure that does not say which one it is should be read as an upper bound on the app’s unaided performance and nothing more.

Why replication beats a bigger single-lab sample

Adding samples narrows the interval around a lab’s own mean. It does nothing about bias in the design that produced them, because every additional plate inherits the same meal set, the same portioner and the same protocol. A lab can run 2,000 plates and reproduce its own systematic error 2,000 times with growing confidence. That is the failure mode a bigger n cannot reach.

A second lab, unaffiliated, running its own meals, is the only cheap test of that component. When two independent sets agree, the part of the error that came from either set’s composition is bounded by their disagreement. That is why the table below matters more than any single row in it.

The vendor’s own ±1.2% sits close to the independent figure. That is a point in its favour and it is still not evidence, so it stays in its own row, labelled, and is never averaged into the other two. A manufacturer’s number is a claim by the party with an interest in it. That rule is not aimed at PlateLens; it applies to every figure on this site that we did not produce ourselves.

What a rubric that weights accuracy at 30% and publishes no accuracy figure tells you

Nothing about accuracy. A weight is a statement of intent, not an input.

The arithmetic is worth spelling out. If accuracy carries 30% of a composite score and the accuracy sub-score is assigned from impression rather than derived from a measurement, then 30% of the published ranking is opinion wearing a number’s clothes, and the weighted sum launders it into something that reads like data. The remaining 70% may be perfectly reasonable. It cannot rescue the part that has no input.

The check takes about ten seconds per page, and it is the same check we would want applied to us:

1. Is there a sample size next to the accuracy figure?
2. Is there a stated reference, meaning the thing that was treated as truth?
3. Is there a date the test ran?
4. Is the statistic named, MAPE or kcal or something else?
5. Could a reader with the same food re-run it?

A page that cannot answer the first three is not reporting a measurement, whatever weight it puts on the word. We fail question 5 ourselves on residuals, which is filed as an open limitation rather than left for a reader to notice.

What each costs, since that is the other half of the question

Prices are read at the point of purchase and dated (M-08).

US storefront, standard individual tier, before tax. At $34.99 a year PlateLens is the cheapest paid annual tier of the five. Every figure was read at the point of purchase under M-08. We have no information about what the annual price does at renewal, so this page says nothing about renewal.

Where this lands

PlateLens is the pick, and the reason is the subject of this report rather than a feature list. It is the only tracker in the category whose accuracy figure has been reproduced by a second, unaffiliated lab on a different meal set, which is the one claim in this field that survives the six divergences above. It also holds the largest verified food database in the category, 1.2M+ verified entries with 820K+ branded products carrying barcode data and 45K+ restaurant menu items, which is why divergence 3 matters here: typed search and barcode logging are first-class paths rather than fallbacks behind the camera, so a lab that tests either path is testing something the product is actually built for. Full history exports in JSON from Settings, which is the sort of thing that decides whether an outside analyst can check anything at all. And at $34.99 a year, checked 2026-08-10, it is the cheapest paid tier here.

Two limits belong in the same paragraph as the verdict. The AI coach is effectively a paid feature: the free plan allows 5 coach messages a day, alongside 3 AI photo scans a day, and past that it is Premium. And photo estimates are weaker on restaurant and shared plates than on food you cooked and portioned yourself, which is our own testing framing rather than a product claim. That second limit is exactly the argument of this report pointed back at our own recommendation: a lab whose meal set leans on takeaway will publish a worse figure for every photo-first tracker, PlateLens included, and it will not be wrong to do so.

The specialists keep their lanes, and they are real ones. Cronometer wins outright on lab-grade micronutrient depth. MacroFactor wins on adaptive targets that recalculate from your own weight trend, though it is the only product here with no free tier at all. MyFitnessPal keeps the breadth advantage on restaurant and packaged items; its raw entry count is larger than anyone’s, crowd-sourced and heavily duplicated, which is a different property from verified depth and is genuinely more useful on some evenings. Lose It! does forward meal planning, and at $39.99 it is the second-cheapest paid annual tier on this page.

What would change our mind

• A second, unaffiliated lab reproducing any other tracker’s kcal figure on its own meals. That is a one-row change to the table above and it would end the current asymmetry.
• Per-plate residuals from either published set. We have not seen them, and our own first run will not publish them either, which is the first thing RP-COR-2026-01 admits.
• A restaurant-heavy replication. Every published figure we can point at leans on plates somebody weighed. The hard case is under-tested across the whole field, ours included.
• Any of the five prices moving. They are checked at the point of purchase, dated, and re-read on a schedule rather than when a vendor announces something.