These five cases come from the test suite of the figure filter that runs on every analysis answer. They are not illustrations. Cases 1, 2 and 4 are test sentences word for word, case 3 is a shortened one, and case 5 joins several into one line. Together they are the clearest description of what the filter does that we can give.
Each case shows the sentence the model wrote and the sentence the reader is allowed to see.
Case 1: a figure that was rounded
The model wrote: Roughly 800.000 DKK across 4 users.
You see: Roughly … DKK across … users.
The ledger returned 812.400 across 3 users. Neither number in the sentence appears in the rows, so neither number survives.
This case is the reason the filter exists. The sentence is not a lie. It is a summary, and it is the kind of summary a helpful assistant writes. It is also wrong about the user count, and nobody reading it would check. A figure that was rounded, softened or misremembered is not the figure that came back.
Case 2: a figure that is grounded
The model wrote: Revenue was 812.400 DKK, a 12,3% share of 3,901 rows.
You see: Revenue was 812.400 DKK, a 12,3% share of 3,901 rows.
Every figure appears in the rows the query returned, in one of the formats those rows can be written in. The sentence passes unchanged.
Number formats differ between Danish, English, German and Spanish. A filter that only recognises one of them redacts correct figures, which trains people to ignore the redaction. Format handling is not a detail here. It is the difference between a filter people trust and a filter people learn to skip.
Case 3: a number the model wrote itself
The model wrote: The caption said 777 and the prompt 555.
You see: The caption said … and the prompt ….
This is the subtle case. The caption above the block was written by the model, and the model then quoted its own caption as evidence for the figure in its prose.
Text the model authored is not evidence. If it were, any model could ground any number by writing it twice. Four things can ground a figure: the rows a tool returned this turn, a numeric value the model sent as a tool input, a figure the user typed in the question, and a figure an earlier answer already showed.
Case 4: a caveat with invented counts
The model wrote: Only 500 of 3,901 rows are in this file. The export is incomplete.
You see: Only … of … rows are in this file. The export is incomplete.
The warning is correct and important. No tool returned those counts this turn, and neither the question nor an earlier answer contained them, so they do not survive.
This case looks harsh and it is deliberate. A warning that carries invented numbers is worse than a warning without numbers, because the reader acts on the size of the problem. The sentence keeps its meaning. The reader learns that the export is incomplete and has to ask by how much.
Case 5: numbers that are not figures, and one that is grounded
The model wrote: Last 7 days (3 September – 10 September 2026), after 14:30, ticket #3, project PR0017.
You see: Last 7 days (3 September – 10 September 2026), after 14:30, ticket #3, project PR0017.
Dates, years, clock times, week numbers, quarters, list markers and
identifiers are not figures, in English, Danish, German and Spanish. The 7 in
"Last 7 days" is different. It is a count, and it survives only because the
query ran with the period P7D, an ISO 8601 duration the model sent as a tool
input. Without that input, the 7 would be replaced like any other count.
A filter that redacts a date is not strict. It is broken, and it makes every answer unreadable.
What the five cases have in common
| Case | Survives? | Rule |
|---|---|---|
| Rounded up | No | A figure must appear in the returned rows |
| Grounded | Yes | Any format those rows can be written in |
| Model's own caption | No | Model-authored text is not evidence |
| Caveat with counts | Partly | The sentence survives, the invented counts do not |
| Dates and identifiers | Yes | Not figures in any language; the 7 is grounded by P7D |
The count of replacements travels with the message. You can see that a sentence was changed, and how many times.
Why publish the test cases?
Because the claim is checkable only if the cases are public.
Any vendor can say that their assistant does not invent figures. The useful version of that claim is a list of sentences the system refuses, including the ones where the refusal costs something. Case 4 costs something. We publish it for that reason.
If you evaluate assistants, take these five sentences to the vendor and ask what their system does with each one. The answers will differ more than the marketing pages do.
Questions and answers
- What happens to a number that no query returned?
- The platform replaces the number with an ellipsis and counts the replacement. The sentence around the number stays, so a warning survives even when its counts do not.
- Can a caption written by the model ground a figure?
- No. Text the model authored is not evidence. A number the model wrote into a caption cannot ground the same number in its prose.
- Where does verification happen?
- Verification happens on the server, before the text is streamed and before it is stored. Every client, the database and the model's own history get the verified text. A withheld figure is kept beside it, and the reader can reveal it, marked as not verified.
Sources
We check every external claim at the date shown. Microsoft moves feature states between release waves, so check the page again before you rely on it.
- 01FinGround: Detecting and Grounding Financial Hallucinations via Atomic Claim VerificationarXiv · Sources checked
- 02FAITH: A Framework for Assessing Intrinsic Tabular Hallucinations in FinancearXiv · Sources checked