A semantic layer is a set of business terms and the method to compute each one. It turns a word such as "overdue" into a query that everybody in the company agrees on.
Business intelligence teams have built these for thirty years. What is new is that an AI assistant needs one more than a dashboard ever did, and that it cannot be built the way the old ones were.
What does a semantic layer hold?
Two kinds of entry, and they behave differently.
A business term. A name, a definition in words, and the query that computes it. For example:
Over credit limit. A customer whose open customer ledger entry balance exceeds the Credit Limit (LCY) on the customer card. Blocked customers are excluded.
The entry carries the query as an executable recipe, not as a description. The next answer replays it instead of deriving it again. A term is always company scope, because a definition that is true for one person is not a definition.
A memory. A lesson, a preference or a correction. For example: this company reports in DKK although several projects are invoiced in EUR, so convert before you total across projects. A memory can be company scope or user scope, because a way of working can belong to one person.
Why does an assistant need one?
Because your question is shorter than your meaning.
"How much did we invoice last month?" contains four decisions the assistant has to make: which document types count, which date field decides the month, which company or companies, and whether credit memos subtract. Your finance team settled these years ago and no longer notices them.
Without a shared layer, the assistant makes those four decisions itself on every question. It will be consistent with itself and inconsistent with you, and the difference will be small enough to survive review.
Why must the layer build itself?
Because a layer that needs a project stays empty.
This is the part the traditional approach gets wrong. The classic semantic layer is built first: someone models the terms, someone else validates them, and the tool becomes useful at the end of that work. In a consultancy engagement that takes months, and companies will not fund it before they have seen value.
So the order has to reverse. The assistant must be useful on day one, and write the layer as a by-product of being used.
The loop works like this:
- It notices. When something gets settled in a conversation, the assistant writes an entry. You corrected a figure. You named what a term means. It worked out which table actually holds the answer.
- A second agent checks it. A smaller reviewing model reads the conversation that produced the entry, including the queries that ran, and the entries nearest to it. It approves, rewrites it more precisely, replaces an older entry it contradicts, or denies it with a reason.
- It starts being used. From then on the entry reaches conversations: by name, by the tool it belongs to, or by meaning.
- It stays honest. Entries that carry a query are re-run on a schedule. One that stops returning data is marked stale with the reason.
- You can step in. The whole layer is a list. Read it, sort by how often each entry was used, edit, delete or add by hand.
Nobody writes the first entry. That is the requirement, not a feature.
Who approves an entry?
By default the reviewing agent, which publishes and tells the administrators.
A company can switch the policy to human review. A company-wide entry then waits as a recommendation until an administrator confirms it. User-scope memories are not affected.
The reason to make the second agent the default is that the alternative does not happen. A review queue that requires a person is a queue that grows. A model review that a person can audit afterwards produces a layer that exists.
How does an entry reach a conversation?
Four routes, and the difference matters for how the layer behaves at scale.
By name. A term is included in full when its title or a trigger keyword appears in the message. Below that threshold, the assistant sees a short list of titles and can search for the rest.
By tool. An entry can attach itself to a result. A rule about project ledger entries arrives with the first project ledger query, and stays for the session.
By meaning. Memories are retrieved by vector search after each tool call. A close match arrives in full, a weaker one as a title the assistant can open.
By connection. An entry can be held to one company, one environment or every connection of a kind, so a rule about a sandbox does not follow you into production.
Retrieval by name alone fails as the layer grows, because nobody uses the exact title. Retrieval by meaning alone is unpredictable. Both are needed.
What can go wrong?
The layer learns something wrong. This is why review exists, why an entry can be superseded rather than duplicated, and why the list is editable.
A definition goes out of date. A term that was true before a chart of accounts change is now wrong and still confident. Re-running the stored query on a schedule catches a good share of these, because a definition that stops returning data is usually a definition that stopped being true.
The layer becomes a second source of truth. It must not. The entry stores how to compute a figure from your system. It does not store the figure.
Retrieval breaks. If the index is unreachable, the turn should run without the layer and say nothing about it. A broken knowledge layer must never break an answer.
The test that tells you whether it works
Ask the assistant the same question twice, a month apart, with a correction in between.
If the second answer needs the same correction, there is no layer. If it does not, the layer is doing its work, and the value compounds from there.
Questions and answers
- What is a semantic layer?
- A semantic layer is a set of business terms and the method to compute each one. It turns a word such as overdue into a query that the same company agrees on.
- Why must a semantic layer build itself?
- A layer that requires a project stays empty. Companies do not have the time to write definitions before they see value, so the assistant must write an entry when a definition gets settled in a conversation.
- Who approves an entry in the semantic layer?
- A second, smaller model reads the conversation that produced the entry and approves, rewrites, replaces or denies it. A company can require an administrator to confirm company-wide entries.
Sources
We check every external claim at the date shown. Microsoft moves feature states between release waves, so check the page again before you rely on it.
- 01FinGround: Detecting and Grounding Financial Hallucinations via Atomic Claim VerificationarXiv · Sources checked 2026-09-17
- 02Aged Accounts Receivables (report 120)Microsoft Learn · Sources checked 2026-09-17