Blog · Published · Updated
Why an AI knowledge base misses answers
Short answer
When the document exists and the assistant answers „I don't know”, start with retrieval. The most common causes are a single search method, no reranking and outdated versions of documents. According to Anthropic (2024), combining contextual embeddings with BM25 search cut failed retrievals by 49%, and by 67% with reranking.
How we workDetails ↓
- 01Call30 minutes with an engineer, free
- 02PrototypeA working system on your process and data, free
- 03ImplementationTarget and fixed price in the contract; 50% at signing
- 04Acceptance50% once the agreed target is met
- net per application, fixed price before contract
- from €3,000
- from signing the contract to acceptance
- 4–10 weeks
- free prototype on your documents
- €0
We belong to the OpenAI and Anthropic partner programmes and learn through their official courses. What this means ↓
Author: Syntalith · 3 min read
Statistics
AI knowledge bases: statistics 2026
−67%
failed retrievals according to Anthropic (2024) after combining contextual embeddings, BM25 search and reranking. Embeddings with BM25 alone gave −49%.
49%
According to Anthropic (2024), combining contextual embeddings with BM25 search cut failed retrievals by 49%, and by 67% with reranking.
11 of 13
According to the NoLiMa study (2025), at 32,000 tokens of context 11 of 13 models fell below half of their short-context performance.
146
According to Bartanowicz and Jassem (LREC 2026), in a RAG system on technical documentation, scored on 146 reference questions, Bielik-11B-v2.3-Instruct consistently outperformed PLLuM-12B-nc-chat.
83%
According to Smits and Walkowiak (EACL 2026), in RAG on Polish Wikipedia, scored on 1,000 PolQA questions, PLLuM-12B matched the quality of 70B models while using 83% less energy.
Figures checked: 22 October 2026
Three causes behind most failures
The model sees only what it is given in the context. If retrieval did not find the right paragraph, that paragraph does not exist for the model, even if it has sat in the database for a year.
| Symptom | Cause | What helps |
|---|---|---|
| It cannot find an invoice number or an error code | search by meaning only | add search by words, for example BM25 |
| It cannot answer a question put in other words | search by words only | add search by meaning |
| The right passage is there, but far down the list | no reranking | a second model that scores the candidates |
| A confident answer with a quotation, but out of date | old versions and duplicates in the sources | order in the documents and ageing of results |
Hybrid search and reranking
Search by meaning understands the question „how do we pay overtime at the weekend” when the procedure talks about „work on days off”. It loses exact strings, though: the number FV/2026/08/117, the code E-4471, a part number. That is why both paths run in parallel and the results are merged into one list. Anthropic, in its Contextual Retrieval publication, reports that this combination cut failed retrievals by 49%, and by 67% with reranking.
- 01Questionwords and meaning searched in parallel
- 02Mergeone list of candidates from both paths
- 03Rerankinga more precise model sets the order
- 04Contexta few best passages with their version and date
- 05Answerwith a quotation, or a refusal when there is no basis
More text does not help. The Context Rot study describes quality falling as input grows, and according to NoLiMa (2025), at 32,000 tokens 11 of 13 models fell below half of their short-context performance (NoLiMa).
Outdated and conflicting documents
No model fixes this cause. If a 2024 price list sits on the drive next to the 2026 one, retrieval finds both, and the model picks one or merges them into an answer that sounds confident. Tidying the documents is a separate item in the project: duplicates, contradictions, OCR for documents without a text layer, and an owner for every collection.
- Does search combine meaning and words
- Is there reranking, and how many passages go into the context
- How does the system treat an older version of the same document
- Are retrieval and answers scored separately on the team's questions
How to test a knowledge base before you buy
The assistant cannot find exact numbers and codes
search by words next to search by meaning
a configuration change
The right source is in the results, but too far down
reranking and the number of passages in the context
a configuration change
Confident answers from outdated documents
order in the sources and owners for each collection
the team's work
The test starts with 50 questions the team has already asked, each with the document and the passage that should appear. First you measure whether the right passage reaches the top five results. Only then do you judge the answer. When retrieval finds the source and the answer is wrong, the problem lies in the generator or the instructions. When retrieval does not find the source, changing the language model will achieve nothing.
For Polish documents there are already public measurements: Bartanowicz and Jassem (LREC 2026) compared four retrieval models and vector, full-text and hybrid methods on 146 questions about technical documentation. The results apply to their corpus, so the decision comes from a test on your own questions. We build knowledge bases with hybrid search as part of our AI apps, from €3,000 net, and for the free analysis bring the questions your assistant answers with „I don't know” today.
Worked example · Scenario
How we count the payback: answers from company documents
We run the same count for your documents in the free analysis.
€3,723
a month for finding answers to 800 questions in documents
Payback: 1 mo.
Our assumptions are 8 minutes to find an answer in documents or ask a colleague, and checking in 15% of the old time. €34.90 an hour is the EU average hourly labour cost for 2025 according to Eurostat, for companies with 10 or more employees. Model cost: Claude Sonnet 5.5, about 6,000 + 500 tokens per answer from documents, at the ECB rate of 1.1205 USD per euro on 22 October 2026.
See the full calculation →
