Long context or RAG? Qwen and Polish RAG measurements

With long context, an application sends the entire material with the question. In RAG, or retrieval-augmented generation, a retrieval module selects relevant passages first and a generator writes the answer from that selection. Long context simplifies the path. RAG limits the text sent with each question and separates retrieval from generation.

Qwen3.8-27B recalled three checked facts from an input containing 230,085 tokens. That single run took 563.4 seconds from launch to the end of the answer. The same test at 50,059 tokens finished in 59.3 seconds. The result shows that the model accepted a very large packet and that sending the whole packet again carries a substantial time cost.

RAG requires two separate checks. First, we ask whether retrieval returned the right sources. Then we ask whether generation used them correctly. Context length alone answers neither question for a complete production knowledge system.

Our long-context measurement

We ran three llama.cpp configurations on a home PC with an RTX 3090. They used different model weights, context memory and batch sizes because each had to fit in 24 GB of VRAM. The configurations target practical use, with several variables changing together.

We placed one verifiable fact near the beginning, middle and end of synthetic material. Each length was run once. A pass means that all three facts were returned, or 3/3:

Prompt processing is the rate at which the server processes input tokens before generating an answer. Generation is the rate at which it creates output tokens. Elapsed time runs from launch to completion, and peak VRAM is the highest measured card-memory use. The conclusion is narrow: the model recalled the three facts we placed in these inputs. The test did not measure citation correctness, document freshness, conflicting versions, refusal when the source is absent or access control. It therefore cannot evaluate a complete company knowledge system.

What a real Polish RAG study measured

The paper Evaluation of Two Leading Polish Language Models in a Real-world RAG Scenario used real technical documentation for a low-code platform. The corpus contained about 1,200 chunks, and the evaluation used 146 reference questions selected as likely user questions. Employees helped prepare the reference answers.

The researchers compared four embedding models and vector, full-text and hybrid search. Their best setup, OrlikB/KartonBERT-USE-base-v1 with vector search, produced:

In this table, precision is the share of returned passages matching the reference set under the paper's definition, while recall is the share of relevant passages that retrieval found. F1 is the harmonic mean of those two measures, so it falls when they diverge. NDCG accounts for rank: a relevant passage near the top earns more credit than one found lower down. The values follow the paper's definitions and annotations.

Increasing the returned passages from five to seven raised recall and NDCG while lowering F1. More material in the generator packet did not improve every measure at once.

For generation, the model received five retrieved documents. The authors used three separate automated evaluator models, gpt-oss-20b, Mistral-Small-3.2-24B-Instruct-2506 and Qwen3-30B-A3B-Instruct-2507, to score the answers. They report mean answer scores on a 1–5 scale: 4.521 for Bielik-11B-v2.3-Instruct and 4.025 for PLLuM-12B-nc-chat. A score of 5 meant fully correct and comprehensive, while 1 meant incorrect or irrelevant. A higher value means a better rating under that study's criteria. These numbers belong to those checkpoints, corpus and evaluation procedure. Qwen3.8-27B was not part of this comparison.

What the two measurements cannot establish

Our table does not establish that long context is better than RAG. The paper does not establish that RAG will be faster or better for every company. The measurements used different corpora, questions, hardware and generators. They answer two useful questions:

• our Qwen run records the result of placing approximately 50k, 115k and 230k tokens in one prompt;
• the RAG paper measures retrieval and generation on real Polish documentation;
• a deployment decision still needs the company's own questions and sources.

When long context is a candidate

Long context is worth considering when the question concerns a complete, one-off packet and relationships between distant sections matter. It can also simplify an early prototype because chunking and an index are not required yet.

The system still needs checks for citation correctness, conflicting versions and justified refusal when the sources contain no answer. Passing a three-fact test does not provide those controls.

When RAG is a candidate

RAG is worth considering when one large corpus serves many questions, documents change frequently or access depends on the user's role. Retrieval can exclude unauthorised sources before generation and expose the passages that influenced the answer.

RAG quality depends on the whole chain. The LREC study found measurable differences between k=5 and k=7, where k is the number of passages sent onward. A company test needs to separate parsing, chunking, embeddings, filters, reranking and generation.

A defensible company test

Begin with questions employees actually ask and the sources needed for each correct answer. Record the effective document version, permitted access scope, relevant passages and the cases where the system must refuse.

Run at least two architectures against the same questions: the complete packet and RAG. Retain source rankings, answers, citations, elapsed time, model version and the reviewer's decision. That record lets the team assess whether an index beats the simpler packet.

What Syntalith can deploy

Syntalith prepares the question-and-source set, runs local checkpoints and an API baseline, and delivers the comparison. After the architecture is selected, we implement the needed parsers, indexing, permission filters, citations, review panel, regression monitoring and team training. Qwen is one generator candidate. The buyer's evaluation determines the winner.

See our AI process audit and custom AI applications, or begin with a free process scan. The answer may be RAG, long context, ordinary search or simply better document ownership.