An internal knowledge assistant is only as good as its last wrong answer

Every company past a certain size has the same knowledge problem, and it isn't that the knowledge is missing. It's in Confluence, in Drive, in the wiki nobody updated since the reorg, in a thread from March. What's missing is the path to it: searching is slow, the results are stale, and asking a person in Slack is faster and more trusted. So people ask in Slack — the same questions, every week, to the same three people.

An assistant that answers those questions from the documents is an obvious idea. Most of the ones that get built are ignored within a month. The difference between the two is not the model.

Why most of them get ignored

A knowledge assistant is trusted the way a colleague is trusted: one confident wrong answer about the time-off policy and nobody asks it again. Three things cause that answer.

It answers from memory instead of from the document. A general model asked about your policy will produce a fluent, plausible, wrong paragraph. The assistant has to retrieve first and answer only from what it found — and show where it found it, so the reader can check.

It doesn't know the document changed. The policy was updated on Tuesday; the index still has Monday's version. The assistant confidently reports the old rule. Freshness is not a nice-to-have; it is the whole reason people stopped trusting the wiki.

It shows people things they shouldn't see. The compensation bands are a document too. If the assistant indexes everything and answers everyone, it is a leak with a friendly interface. Retrieval has to respect the same permissions the documents already have, per user, at query time.

What a good one does

• Answers with citations, or says "I don't know." Every answer links to the passage it came from. When retrieval comes back empty or ambiguous, the assistant says so and offers the person who would know. A correct "I don't have that" is worth more than a fluent guess.
• Re-indexes on change, not on a schedule. When a document is edited, its embeddings are rebuilt then — so the assistant is never behind the source.
• Lives where the questions already are. Inside Slack or Google Chat, not on a portal people have to remember. The point is to intercept the question in the place it is asked.
• Escalates gracefully. For the questions it can't answer, it routes to the owner — and that routing becomes the list of documents that need writing.

The work nobody budgets for

The assistant is honest about your documentation in a way the wiki never was. The first weeks of questions surface every page that is outdated, contradictory, or missing, because the assistant is now reading them out loud. That is not a failure of the project; it is the most useful output of it. Budget time for an owner to fix what it finds.

The other piece is an evaluation set: fifty real questions people actually asked, with the answer a knowledgeable colleague would give. Run it every time the documents, the prompts, or the model change. Without it, "seems to work" is the only signal, and it is the signal that led to the wiki nobody trusts.

The stack, briefly

None of this needs exotic infrastructure. Embeddings and vector search over the documents — Postgres with a vector extension is enough for most companies — a retrieval step with permission filtering, a language model to write the answer from the retrieved passages, and the chat integration. It runs inside your cloud and your permissions, because the documents are already there and the questions are about them.

What to measure

The share of questions answered with a citation. The share the assistant declined, and what those were about. Corrections reported by users. And the one that matters to the people who used to answer everything: how many repeated questions stopped reaching them.

Start with one team's questions — HR and operations are usually the loudest — and one channel. If it earns trust there, the same foundation extends to engineering docs, support runbooks, and sales material; if it doesn't, you'll know why within a month, and the answer will be in the documents, not the model.

What we learned prototyping it

We built this as a proof of concept on our own documentation: embeddings and vector search in Postgres (pgvector), retrieval and indexing with LlamaIndex, an open-weight model (Gemma 3) writing the answers from the retrieved passages, and a Google Chat integration so the questions could be asked where they already were.

The model was the least interesting decision. A small open-weight model answered well when the retrieval was good and badly when it wasn't — which is the point: quality lived in what was indexed, how it was chunked, and whether the index was current. The first version re-indexed nightly; a policy edited in the morning was wrong until the next day, and that one wrong answer cost more trust than a week of right ones. Re-indexing on change went in immediately.

Permissions were the design constraint that shaped everything else. Filtering at query time, per user, against the documents' own access rules, is not an add-on; it decides the index layout. Build it first.

And the assistant was brutal about our own docs. Within days it had surfaced the contradictions and the pages nobody had touched in a year — which is exactly the output that justifies the project, if someone owns fixing them.

What production adds: named-entity handling for people and products so the retrieval matches the way questions are actually phrased, an evaluation set from real questions, and an owner for the documentation debt the assistant will keep finding.