Qwen3.8-27B: 60k, 120k, 150k or 250k context?

The largest window was not the most useful configuration in the home-PC experiment. Qwen3.8-27B recovered three facts from the beginning, middle and end of 50k, 115k and 230k prompts, but the 230k run took 563.4 seconds against 59.3 seconds for 50k. The 150k patched-vLLM configuration avoided compaction and finished the long Go repair much faster than the 120k llama.cpp configuration.

Let the job determine context length. The largest number on a model card should not.

Official 262k and our local profiles

Qwen3.8-27B has a native 262,144-token context. The model card also describes extension to one million tokens through YaRN. We did not test one million tokens on the RTX 3090 and do not treat that extension as a free property.

Our llama.cpp path used three practical profiles:

This is not an isolated context-length experiment. Weight format, cache precision and batch settings differ because all three profiles had to fit on one 24 GB card. The table answers “what can this machine run”. It cannot isolate the causal effect of one parameter.

Long-context fact retrieval results

We placed exact facts at the beginning, middle and end of a long input. Each profile had to return them.

The 250k configuration returned all three facts, but the cost was different. The prompt held 4.6 times as many tokens as 60k, while elapsed time increased by 9.5 times. VRAM also approached the full 24 GB capacity.

Why 60k was too small for a long agent task

The short profile generated faster and used higher-precision Q5 weights, but a long coding task exceeded its useful window. The agent compacted history, reread files and spent time rebuilding context. The completed run plus continuation took 42.77 minutes and scored 87/100.

Compaction is not inherently a failure. It becomes costly when the system repeatedly rediscovers the same dependencies. An agent's context contains system instructions, tool definitions, calls and outputs as well as source files.

Why 250k did not win either

The 250k profile avoided compaction but generated more slowly, used Q4 KV cache and did not improve engineering quality in the task we ran. Both primary harnesses missed an established empty-body compatibility case.

Filling a window can also make the session worse. Old tool traces compete with the current problem. Sessions should be task-scoped: resume when previous reasoning is required, and start fresh for an independent job.

Why we retained 150k for later runs

The selected patched-vLLM configuration advertised 150,000 tokens and used Qwen Code at medium effort. It completed the Go repair without compaction in 502.14 seconds. Codex awarded the patch 100/100 under Syntalith's rubric; this was not an external-auditor score.

The same prompt and baseline on llama.cpp 120k scored 98/100, compacted once and used 1,468.76 seconds. This comparison covers complete profiles and isolates no effect of context length. It still shows that usable context, serving and cache choices affect completion time.

Choosing a window for company work

First measure the real packet:

task context = system instructions
  + tool definitions
  + input documents or code
  + prior answers and tool results
  + response allowance

Then choose the smallest window that covers the 95th percentile with a safety margin. A repeatable document analysis may fit in 60k. A long repository change may compact at 120k. A company knowledge system that loads 250k for every question will often lose to retrieving a few relevant passages.

What the client buys

In a Syntalith AI process audit, we measure input-length distribution, quality after trimming and queue time. The document selects a profile, names an alternative and defines the trigger for changing hardware.

If the workload and typical document are still unknown, begin with the free process scan. Do not buy a GPU from the label “262k”.

The measured dataset is available in our sanitized JSON ledger.