Blog · Published · Updated
AI Knowledge Base Without RAG: Which Route Fits?
Short answer
With a corpus below about 200,000 tokens, roughly 500 pages, first test the whole material in the model's context, and only then build search (RAG). That is Anthropic's advice. At 100 questions a day and a corpus of 150,000 tokens it costs about €589 a month without prompt caching and €41 with it. Document permissions need the full architecture at any size.
How we workDetails ↓
- 01Call30 minutes with an engineer, free
- 02PrototypeA working system on your process and data, free
- 03ImplementationTarget and fixed price in the contract; 50% at signing
- 04Acceptance50% once the agreed target is met
- net per application, fixed price before contract
- from €3,000
- from signing the contract to acceptance
- 4–10 weeks
- free prototype on your documents
- €0
We belong to the OpenAI and Anthropic partner programmes and learn through their official courses. What this means ↓
Author: Syntalith · 3 min read
Statistics
AI knowledge bases: statistics 2026
200,000
tokens, about 500 pages: below this threshold Anthropic advises putting the whole corpus into the model's context first, before building RAG.
11 of 13
According to the NoLiMa study (2025), at a context of 32,000 tokens 11 of 13 models fall below half of the performance they have with a short context.
Source: NoLiMa, ICML 2025 (2025) ↗
49%
According to Anthropic (2024), combining contextual embeddings with BM25 search cut the number of failed retrievals by 49%.
67%
According to Anthropic (2024), adding reranking to contextual retrieval cut the number of failed retrievals by 67%.
15.1%
According to Eurostat (2025), 15.1% of people aged 16 to 74 in the EU used generative AI tools for work.
20.0%
According to Eurostat (2025), 20.0% of EU enterprises with at least 10 employees used AI.
Figures checked: 22 October 2026
The decision for a corpus of up to 200,000 tokens
In its Contextual Retrieval article, Anthropic says it plainly: if the knowledge base is smaller than about 200,000 tokens, about 500 pages, you can put the whole corpus into the model's context with no retrieval. That is a starting heuristic you check on your own documents. Permissions, how often documents change, the token cost and the result on the team's questions also decide.
A few dozen documents, rare changes, one department
tidy files and the search you already have
€0
The corpus fits in the context, everyone sees everything
the whole corpus in context with prompt caching
the token cost
The corpus outgrows the context or the trial goes badly
hybrid search with reranking
from €3,000
Different roles see different documents
RAG with permissions before the model
€3,000–17,500
The last rung is the only one where the size of the corpus stops deciding. The risk decides, that a restricted document affects an answer for a person without access.
How much 200,000 tokens is for you
Languages such as Polish use more tokens than English, because tokenisers split their words into more pieces. Tables and price lists exported to text take many times more space than on screen, and scans add noise after reading.
- 01SampleTake 20 representative documents.
- 02CounterRun them through your vendor's token counter.
- 03AverageWork out the average per document.
- 04TotalMultiply by the number of documents in the catalogue.
What the whole corpus costs in every question
You pay the token cost for every question separately. Prompt caching changes the bill the most.
- Input tokens a day
- 100 × 150,000 = 15 million
- Without caching, Sonnet 5.5 ($2 per million)
- €589 a month
- With caching ($0.10 per million on reads)
- €41 a month
Difference
€548 a month
22 working days, no output tokens. A 1-hour cache, refreshed by the following questions, written once a day at $4 per million. Anthropic's rates for Claude Sonnet 5.5, converted at the ECB rate of 1.1205 USD per euro on 22 October 2026. The number of questions and the size of the corpus are the scenario's assumptions.
Why a long context is not enough for ever
Context windows grow, but quality does not grow with them. The NoLiMa study (ICML 2025) tested 13 models: at 32,000 tokens 11 of them fell below half of their short-context performance when the question and the information sought share no vocabulary. Even GPT-4o fell from 99.3% to 69.7%.
Permissions before the model
If different people see different documents, the permission filter has to work before the model, because an instruction such as "do not show HR documents" blocks nothing. The model gets only the fragments the person asking has access to. So even a wrong answer will not reveal the content of a restricted document. This requirement alone settles the architecture, even with a few dozen files. We test on the account of the person with the least access.
Our own knowledge base
Syntalith's operating knowledge lives as versioned Markdown in a repository. The service runs deterministic full-text search, and we have the semantic layer switched off today. With a few dozen files it would add variability with no gain. Do not copy this setup without checking, because your corpus looks different.
When we advise against a rollout
- The corpus is small and stable, and everyone sees everything
- The documents are in a state no search will rescue
- The problem is one-off, for example 40 contracts once a quarter
- Employees ask colleagues because the document does not exist and the decision is in someone's head
Gather 20 real questions from employees and check how many of them the search you already have can handle. Next to each question, write down the document that holds the answer. The same set will later serve to compare the full context with search. If the trial justifies a separate solution, the scope is described on the private ChatGPT for your company page.
Worked example · Scenario
How we count the payback: the team's questions to documents
In the free analysis we also work out whether you need RAG at all.
€3,723
a month for finding answers to 800 questions in documents
Payback: 1 mo.
Our assumptions are 8 minutes to find an answer in documents or ask a colleague, and checking in 15% of the old time. €34.90 an hour is the EU average hourly labour cost for 2025 according to Eurostat, for companies with 10 or more employees. Model cost: Claude Sonnet 5.5, about 6,000 + 500 tokens per answer from documents, at the ECB rate of 1.1205 USD per euro on 22 October 2026.
See the full calculation →
