Qwen3.8-27B: 60k, 120k, 150k or 250k context?

Qwen3.8-27B retrieved the three requested facts from both a 50k-token input and a 230k-token input. The shorter run took 59.3 seconds; the longer one took 563.4 seconds. For this task, waiting about nine times longer did not produce an additional correct answer.

In a separate code repair, the 150k profile on patched vLLM performed best. It was much faster than the 120k llama.cpp profile, although the server, weights and KV cache also changed. We separate those two tests below and show how to choose context for the work. Each measurement row comes from one run on one GPU.

Official 262k and our local configurations

Qwen3.8-27B has a native 262,144-token context window. Its model card also describes extending the window to one million tokens through YaRN. We did not run the million-token variant on the RTX 3090, so that extension is outside this experiment.

Our llama.cpp path used three practical configurations:

This table answers what fits on one 24 GB card. It does not isolate context length, because the configurations also differ in weight format, KV-cache precision and batch size. We changed those settings together to run each variant on the same GPU.

Finding facts in a long input

We placed exact values near the beginning, middle and end of a synthetic record. The task required all three values in the answer. “Facts returned” means the number of correct values out of three; 3/3 means that this specific retrieval check passed. The result shows that the planted facts were found. It does not establish understanding of arbitrary long documents.

“Input tokens” counts the tokens before the answer. Prompt-processing speed measures how quickly that input is prepared, generation speed measures output creation and total elapsed time covers the full run. Maximum VRAM is the highest observed card memory use. The 250k run contained 4.6 times as many tokens as 60k, took 9.5 times as long and used 23,623 MiB of the 24 GB card.

What history compaction changes for an agent

A long coding task can fill its window faster than the repository size suggests. An agent's history includes system instructions, tool definitions, calls, tool results and answers. Once the useful limit is reached, the system compacts earlier turns, and the agent may need to reread files and reconstruct dependencies.

In one recorded agent run, Codex gave the result 87 out of 100 points under the Syntalith criteria for correctness, regression tests, compatibility, scope discipline, verification and documentation. The evaluation identified a material nondeterministic fallback defect. This is a score for one task against concrete criteria, and it does not estimate model reliability as a percentage. Compaction itself is not proof of a defect; the cost appears when the same context has to be rebuilt repeatedly.

Why 250k does not settle the choice

Every configuration returned 3/3 planted facts in the retrieval check. The longest configuration generated at 28.05 tok./s and took 563.4 seconds, compared with 49.24 tok./s and 59.3 seconds at 60k. We did not measure answer quality on arbitrary documents in this check, so the result cannot support a claim that a larger window improves engineering quality.

A larger window can also make history more expensive to search. Continue a session when its earlier reasoning is needed; start an independent task with a fresh context.

Why we kept 150k for later runs

The selected patched-vLLM configuration accepted up to 150,000 tokens and used Qwen Code 0.21.13 at medium effort. In a task repairing two json.Marshal error paths, it finished without compaction in 502.14 seconds. OpenAI Codex evaluated the result and awarded 100 of 100 points under the Syntalith criteria: correctness 40/40, regression tests 20/20, compatibility 15/15, scope discipline 10/10, verification 10/10 and documentation 5/5. The single result had no deductions against those criteria; it is not an external audit or a repeatability measure.

The same request and baseline on the 120k llama.cpp configuration received 98 of 100 points from Codex, compacted once and took 1,468.76 seconds. This compares complete configurations, including engine, weight format, KV cache, serving path and context limit. It is a practical comparison of two stacks rather than a clean test of 120k against 150k.

How to choose a window for company work

First measure the real input packet:

task context = system instructions
  + tool definitions
  + input documents or code
  + prior answers and tool results
  + response allowance

Choose the smallest window that covers the 95th percentile of packet length with a safety margin. A repeatable document analysis may fit in 60k. A long repository change may need 120k or compaction. A company knowledge system will often gain more from retrieving a few relevant passages than from loading 250k for every question.

What Syntalith measures

As part of private LLM deployment, Syntalith measures real input lengths, quality after trimming and queue time. We use the results to select context capacity, memory and document retrieval. This gives you a hardware estimate before buying equipment.

The experiment's full measurements are in the public JSON summary.