AI Knowledge Base Without RAG: When You Really Do Not Need It (2026)
If all your knowledge fits into roughly 200,000 tokens, start with the cheapest test there is: the whole corpus in the model's context. Anthropic itself suggests it. What settles the choice afterwards is that test, your permissions and your token bill. Here is the decision ladder and an honest do-not-buy.
The cheapest knowledge base is the one you never build. If your entire corpus fits inside the context window, the cheapest test is also the first one: hand it to the model whole and measure the answers.
8 min read
With a corpus below roughly 200,000 tokens, about 500 pages, test the whole thing in the model's context first. That is a starting heuristic. Permissions, how often the documents change, token costs and the answers you measure on real questions decide the rest. A dedicated Syntalith knowledge base starts from €6,000 net.
Quick answer
In its "Contextual Retrieval" write-up (September 2024) Anthropic says it plainly: if your knowledge base is smaller than about 200,000 tokens, roughly 500 pages, you can just include the entire knowledge base in the prompt. That sentence tells you where to start testing. It sets no economic or quality cutoff, and it does not mean retrieval becomes mandatory above that size.
The order that follows from it is a testing order:
- Before you build anything: tidy your file names, delete stale versions, check the search you already have. The cost is your team's time and zero licence fees.
- The cheapest test: put the whole corpus in the model's context and score the answers on twenty real questions. If they hold up, the RAG question can wait. Simple automation of one channel starts at Syntalith from €3,500 net.
- When that test fails, or once permissions enter the picture: retrieval comes back and you need full RAG. Access control can require retrieval at any corpus size. A dedicated build starts from €6,000 net, typical full projects fall in the €6,000–35,000 net range, and maintenance is priced individually. Rates are on the pricing page.
This article exists so that you do not buy something from us you do not need. At a free process scan (€0) we say the same thing, run on your numbers.
How many pages is that in your case?
The figure of 200,000 tokens sounds precise, but converting it to pages depends on what you keep. Several things can wreck the estimate.
Polish costs more tokens than English, because tokenizers split Polish words into more pieces. Tables and price lists exported to text can take several times more than the same content looks like to a human. Scans run through OCR add noise, page numbers and repeated headers, and those cost tokens too.
A practical method: take 20 representative documents, run them through your provider's token counter, work out the average per document, and multiply by the number of documents in the repository. If you land at 60,000 tokens, the whole corpus fits in context comfortably and that is where to start. If you land at 400,000, that route is closed for the full corpus, and the comparison with retrieval happens on a question set.
Add one line to the calculation: with the whole-corpus-in-the-prompt approach you pay for those tokens on every question rather than once. At a hundred questions a day and a 150,000-token corpus, that is 15 million input tokens a day unless you enable prompt caching. It can still beat the cost of a build, but do the arithmetic before assuming it is free.
Why does long context not settle it for good?
It is tempting to extend the same logic: context windows keep growing, so in a year the whole corpus will fit in the prompt and retrieval will disappear. The measurements say otherwise.
The "Context Rot" study (Chroma, July 2025, 18 models) showed that long-context quality degrades unevenly and starts falling well before the advertised window limit. An advertised 200k or one million tokens describes the technical capacity of the window. It does not describe the length over which the model works equally well.
NoLiMa (ICML 2025) adds a result that matters for company knowledge: when the question and the target information share no vocabulary, 11 of the models tested drop below 50% of their short-context accuracy already at 32k tokens. And employee questions rarely use the procedure's vocabulary. Somebody asks about "same-day leave" while the document says "unplanned absence".
Hence the practical conclusion: when the full-context test performs poorly, you go back to retrieval, and to retrieval that combines a lexical path with a semantic one. Why one method is not enough, and what breaks without reranking, is in the piece on why your AI knowledge base cannot find answers.
The decision ladder
Four rungs. Step onto the lowest one that solves your problem and stop there.
| Corpus size and character | The right approach | Cost order |
|---|---|---|
| A few dozen documents, rare changes, one department | good file names, one location, the search you already have in Google Workspace, Microsoft 365 or Confluence | your team's time, zero licences |
| The corpus fits in the context window, knowledge is stable, everyone sees everything | test full context first: the whole corpus in the prompt, prompt caching, one question channel | token cost per question, automation from €3,500 net |
| The corpus outgrows the window or the test performs poorly, many formats, frequent changes | compare full context with retrieval on a question set, then hybrid RAG with reranking and document versioning | dedicated build from €6,000 net |
| Any size, but different roles see different documents | RAG with scope enforced before the model, audit trail and logs | €6,000–35,000 net, depending on sources and requirements |
The last row is the only one where corpus size stops deciding. Even a small base needs the full architecture if an HR document must not shape an answer given to a salesperson. That rung is governed by leak risk, and corpus size becomes secondary. We cover it separately in the piece on permission-aware RAG enforced before the model.
Boring architecture wins: our own knowledge base
An example from our own back yard, since advising others is easy.
Syntalith's operational knowledge lives as versioned Markdown in git. The service always keeps a deterministic full-text path in the BM25 tradition, and an optional semantic index can run alongside it. Today we keep that index switched off, and it is a decision one configuration change reverses. Every answer our team or our agent receives points to a file, a line number and the commit it came from, in either mode.
There is one reason and it is not technical. At our corpus size, on the order of dozens of files and a few hundred kilobytes of text, a semantic layer would add variance without gain today: the same question could return different results after an index rebuild, and we need repeatability. A citation with a commit hash also shows when a fact changed and who changed it. At this stage a checkable trail back to the source is worth more to us than flexible matching.
This is not a recipe to copy, because your corpus looks different. It is an argument about order: start with the most boring architecture that gives checkable answers, and add layers only once you have measured that questions go unanswered without them.
When we will say "do not buy this from us"
Four situations where a scan ends in a refusal instead of an offer.
The corpus is small and stable. A few dozen files, updated quarterly, everyone sees everything. The search in a tool you already pay for, plus half a day tidying names. A €6,000 build buys you nothing here.
The documents are in a state no retrieval can rescue. Three undated versions of the price list, procedures living in email, nobody sure which one applies. Document owners and cleanup first, tooling second. RAG run over that material produces confident errors with citations attached, which is worse than today's chaos, because it looks credible.
The problem is a one-off. Once a quarter somebody has to read 40 contracts and pull the termination dates out of them. That is one prompt with attachments. A system built around four hours of quarterly work never pays for itself.
The real problem is the process, and search only masks it. People ask colleagues not because they cannot find the document, but because the document does not exist and the decision lives in someone's head. Building a knowledge base then covers the problem with a layer of technology. Start by writing those decisions down.
If the first or third point describes your situation and you still want a comparison of ready tools, the market map is in the piece on company brain AI: a ready platform or your own knowledge base.
How to start
- Measure your corpus in tokens on a sample of 20 documents. That takes 30 minutes and settles most of this decision.
- Collect 20 real employee questions and check how many the search you already own handles correctly.
- Book a free process scan: 30 minutes with an engineer and a written takeaway in two business days. If the conclusion is that you need no implementation, you get that in writing.
Book a free process scan | See pricing
Related articles
Frequently asked questions
- When do I not need RAG for a company knowledge base?
- When the whole corpus fits inside the model's context window and the answers on real questions are good. In its "Contextual Retrieval" write-up (September 2024) Anthropic notes that for knowledge bases below about 200,000 tokens, roughly 500 pages, you can simply include the entire corpus in the prompt. That is a starting heuristic. Permissions, how often documents change, token costs and your measured results decide the rest.
- How many pages is 200,000 tokens?
- Roughly 500 pages of text, though the conversion depends on language and format. Polish typically costs more tokens than English, and tables or OCR'd scans can take several times more than clean prose. Measure it on your own material before deciding the corpus fits in context.
- Does long context replace RAG?
- Not in the general case. The "Context Rot" study (Chroma, July 2025, 18 models) showed that long-context quality degrades unevenly and starts falling well before the advertised window limit. NoLiMa (ICML 2025) found that when the question and the target information share no vocabulary, 11 of the models tested drop below 50% of their short-context accuracy already at 32k tokens.
- Where should I start if the corpus is small?
- With tidy file names, deleting stale versions, and checking whether the search you already have in Google Workspace, Microsoft 365 or Confluence is enough. That costs your team's time and zero licence fees. Only when that fails is it worth discussing automation from €3,500 net or a dedicated knowledge base from €6,000 net.
Free process scan
Start with a free process scan.
- A 30-minute call with the engineer who would lead the work.
- A review of the processes that cost you the most time and money.
- A written summary of what to automate first and the likely cost range.
The scan is free and creates no obligation. If automation is unlikely to pay off, the written recommendation will say so.
€0
30 minutes · written takeaway within 2 business days
Times are shown in your own time zone. We work with clients across time zones.
Describe the process in the form