Qwen3.8-27B: 60k, 120k, 150k or 250k context?

The largest window was not the most useful choice in the home-PC experiment. Qwen3.8-27B recovered one exact fact from the beginning, middle and end of inputs containing 50k, 115k and 230k tokens. The 230k run took 563.4 seconds, compared with 59.3 seconds for 50k. In a separate code-repair task, the 150k configuration on patched vLLM finished much faster than the 120k llama.cpp configuration.

Each row below describes one run on one GPU. A “profile” is a concrete combination of server, weights, KV-cache settings and batching. Those parameters have to be judged as a set. Choose context length for the job instead of the largest number on a model card.

Official 262k and our local configurations

Qwen3.8-27B has a native 262,144-token context window. Its model card also describes extending the window to one million tokens through YaRN. We did not run the million-token variant on the RTX 3090, so that extension is outside this experiment.

Our llama.cpp path used three practical configurations:

This table answers what fits on one 24 GB card. It does not isolate context length, because the configurations also differ in weight format, KV-cache precision and batch size. We changed those settings together to run each variant on the same GPU.

Finding facts in a long input

We placed exact values near the beginning, middle and end of a synthetic record. The task required all three values in the answer. “Facts returned” means the number of correct values out of three; 3/3 means that this specific retrieval check passed. The result shows that the planted facts were found. It does not establish understanding of arbitrary long documents.

“Input tokens” counts the tokens before the answer. Prompt-processing speed measures how quickly that input is prepared, generation speed measures output creation and total elapsed time covers the full run. Maximum VRAM is the highest observed card memory use. The 250k run contained 4.6 times as many tokens as 60k, took 9.5 times as long and used 23,623 MiB of the 24 GB card.

What history compaction changes for an agent

A long coding task can fill its window faster than the repository size suggests. An agent's history includes system instructions, tool definitions, calls, tool results and answers. Once the useful limit is reached, the system compacts earlier turns, and the agent may need to reread files and reconstruct dependencies.

In one recorded agent run, Codex gave the result 87 out of 100 points under the Syntalith criteria for correctness, regression tests, compatibility, scope discipline, verification and documentation. The evaluation identified a material nondeterministic fallback defect. This is a score for one task against concrete criteria, and it does not estimate model reliability as a percentage. Compaction itself is not proof of a defect; the cost appears when the same context has to be rebuilt repeatedly.

Why 250k does not settle the choice

Every configuration returned 3/3 planted facts in the retrieval check. The longest configuration generated at 28.05 tok./s and took 563.4 seconds, compared with 49.24 tok./s and 59.3 seconds at 60k. We did not measure answer quality on arbitrary documents in this check, so the result cannot support a claim that a larger window improves engineering quality.

A larger window can also make history more expensive to search. Continue a session when its earlier reasoning is needed; start an independent task with a fresh context.

Why we kept 150k for later runs

The selected patched-vLLM configuration accepted up to 150,000 tokens and used Qwen Code 0.21.13 at medium effort. In a task repairing two json.Marshal error paths, it finished without compaction in 502.14 seconds. OpenAI Codex evaluated the result and awarded 100 of 100 points under the Syntalith criteria: correctness 40/40, regression tests 20/20, compatibility 15/15, scope discipline 10/10, verification 10/10 and documentation 5/5. The single result had no deductions against those criteria; it is not an external audit or a repeatability measure.

The same request and baseline on the 120k llama.cpp configuration received 98 of 100 points from Codex, compacted once and took 1,468.76 seconds. This compares complete configurations, including engine, weight format, KV cache, serving path and context limit. It is a practical comparison of two stacks rather than a clean test of 120k against 150k.

How to choose a window for company work

First measure the real input packet:

task context = system instructions
  + tool definitions
  + input documents or code
  + prior answers and tool results
  + response allowance

Choose the smallest window that covers the 95th percentile of packet length with a safety margin. A repeatable document analysis may fit in 60k. A long repository change may need 120k or compaction. A company knowledge system will often gain more from retrieving a few relevant passages than from loading 250k for every question.

What Syntalith measures

In a Syntalith AI process audit, we measure input-length distribution, quality after trimming and queue time. We use those measurements to name a configuration, a fallback and the threshold for changing hardware. Book a free process scan when the workload and typical document are still unknown.

The measurements are available in our public JSON summary.