Long context or RAG? Qwen and Polish RAG measurements

Qwen3.8-27B recalled every checked fact from a 230,085-token prompt. The run took 563.4 seconds. The same type of test at 50,059 tokens finished in 59.3 seconds. That real result captures both the advantage and the cost of long context: a model can read a large packet, but sending that whole packet again for every question is not automatically a good knowledge system.

RAG answers another question. It retrieves likely passages first and sends only those passages to the generator. Its quality therefore has two measurable stages: whether retrieval found the right source and whether generation used that source correctly.

Our long-context measurement

We ran three llama.cpp profiles on a home RTX 3090. They used different weights, KV cache and batch settings because each had to fit in 24 GB of VRAM. These are practical deployment profiles. This was not a one-variable laboratory experiment.

We planted verifiable facts at the beginning, middle and end of the material. Every profile had to return them:

The conclusion is narrow: the model recalled the planted facts in these inputs. The test did not measure citation correctness, document freshness, conflicting versions, refusal when the source is absent or access control. We do not call it a company-brain benchmark.

What a real Polish RAG study measured

Evaluation of Two Leading Polish Language Models in a Real-world RAG Scenario describes a system built over real technical documentation for a low-code platform. The corpus contained about 1,200 chunks and the evaluation used 146 reference questions selected as likely user questions. Employees helped prepare the reference answers.

The researchers compared four embedding models and vector, full-text and hybrid search. Their best setup, OrlikB/KartonBERT-USE-base-v1 with vector search, produced:

These are the paper's metrics under its relevance definitions. Increasing the returned passages from five to seven raised recall and NDCG while lowering F1. More generator context did not improve every measure at once.

For generation, the model received five retrieved documents. The mean 1–5 score was 4.521 for Bielik-11B-v2.3-Instruct and 4.025 for PLLuM-12B-nc-chat. That is a result for those checkpoints, corpus and procedures. It is not a Qwen3.8-27B result.

What the two tables cannot establish

Our table does not show that long context beats RAG. The paper does not show that RAG is always faster or better. There is no common corpus, question set, hardware or generator. The sources answer different useful questions:

• our Qwen run records what happened when 50k, 115k and 230k tokens were physically placed in one prompt;
• the RAG paper measures retrieval and generation on real Polish documentation;
• neither replaces an evaluation on one company's sources.

When long context is a candidate

Long context is useful when the question concerns a complete one-off packet and relationships between distant document sections matter. It can also simplify an early prototype because no chunking or index is needed yet.

The system still needs checks for citation correctness, conflicting versions and justified refusal. Passing a needle test does not provide those controls.

When RAG is a candidate

RAG becomes natural when one large corpus serves many questions, documents change frequently or access depends on the user's role. Retrieval can exclude unauthorised sources before generation and expose the passages that influenced the answer.

RAG is not fixed by choosing a fashionable model. The LREC study found measurable differences between k=5 and k=7. A company test needs to separate parsing, chunking, embeddings, filters, reranking and generation.

A defensible company test

Begin with questions employees actually ask and the sources that should settle each answer. Record the effective document version, permitted access scope, relevant passages and the cases where the system must refuse.

Run a long-packet profile and a RAG profile against the same questions. Retain the source ranking, answer, citations, wall time, model version and reviewer decision. Only then can the team say whether an index beats the simpler packet.

What Syntalith can deploy

Syntalith can build that evaluation, run local checkpoints and an API baseline, and then implement the winning architecture. The scope can include parsers, indexing, permission filters, citations, a review panel, regression monitoring and team training. Qwen is one generator candidate. The buyer's evaluation determines the winner.

See our AI process audit and custom AI applications, or begin with a free process scan. The answer may be RAG, long context, ordinary search or simply better document ownership.