When not to deploy Qwen locally: five results that should stop a project

One local run lasted 121 minutes 38 seconds and delivered no working change. A second run took more than three hours and two user interventions, then still broke an existing contract. That is enough to reject a setup even when the model fits in the card's memory and looks impressive in a short demo.

The cases below come from an after-hours experiment on a home PC with an RTX 3090. Each describes a recorded run and leads to a concrete decision before buying infrastructure. Single runs expose risks to test; they do not establish model repeatability.

Result 1: 121 minutes, 4.28 million input tokens and no completion

We connected Claude Code 2.1.235 to Qwen3.8-27B through a local endpoint that followed Anthropic's API format. The llama.cpp configuration had a 120k context and medium reasoning effort. The agent had the standard Bash, Edit, Read, Write, Glob and Grep tools in a clean writable worktree.

The run had no artificial wall-time limit. After 7,298.5 seconds, the client sent 131,265 tokens to a server limited to 131,072. The server returned HTTP 400. Before that, the orchestration layer made 66 tool calls and repeatedly sent earlier input again, for 4,283,936 cumulative input tokens. Its 16-test file did not compile. Documentation remained unchanged, and no final report was produced.

This run identified a failure in the combination of orchestration layer, endpoint and 120k context. It cannot rate Qwen itself or Claude Code used with Claude models. We therefore removed this configuration from the experiment's reference set.

Deployment decision: when a representative task cannot finish on its own, inspect context management, repeated prompt processing, the protocol and the orchestration layer first. Consider more hardware after those checks. A different client or a managed API may be the right answer.

Result 2: a small context turned one task into a three-hour session

Qwen Code in medium mode, running on the corrected 60k configuration, received an ordinary handoff-error repair. Its first turn lasted 152.99 minutes and repeated this cycle:

generate the complete test file
-> hit the output or context limit
-> automatic summary
-> lose the record of what was read before writing
-> reread
-> generate the complete file again

Two ordinary user corrections eventually brought the session to completion. Total time was about 192.1 minutes, with 1,069.1 Wh of energy use, or 1,069.1 watt-hours. The change set became disciplined only after an explicit request to restore files outside the task. The program still rejected an empty REST request body even though the existing contract allowed it. Codex then awarded 78/100 under the Syntalith rubric. The 100 points cover correctness (40), regression tests (20), compatibility (15), scope discipline (10), verification (10) and documentation (5). Codex was the evaluator for this run; no external auditor took part.

The same kind of task on the 120k/medium configuration finished much sooner. A larger window does not ensure quality, while a window that is too small can spend the session on compaction and rereading.

Deployment decision: if a session repeatedly compacts before its first material change, pause that configuration. Record compactions, rereads and user interventions alongside token generation speed.

Result 3: the maximum window increased cost without improving quality

The 250k configuration avoided compaction and retained about 119k of active context. It completed in 30 minutes 19 seconds and reached 23,179 MiB of peak VRAM. The result looked promising, yet empty REST-body handling remained broken and Codex awarded 80/100. The 120k/medium result preserved the existing contract and was the better choice.

In a separate test, the model had to retrieve three facts placed near the beginning, middle and end of a long input. It returned all three with both 230,085 and 50,059 input tokens. The longer run took 563.4 seconds, compared with 59.3 seconds for the shorter one.

Deployment decision: when a large window does not improve the process result, shorten the packet, add retrieval or split the work. A server accepting 250k tokens does not by itself justify the cost of that configuration.

Result 4: public tests passed, hidden validation found a defect

On the CSV import task, Qwen Code in medium mode finished in 93.14 seconds. It passed 12/12 public tests but only 4/5 hidden checks because it accepted a blank name and a blank role. Codex and OpenCode, driving the same local model, passed every check for this task.

That is a small defect in a small test project. In account provisioning, it could create invalid records, require manual repair and complicate the audit trail.

Deployment decision: edge cases are part of production evidence. If an expensive defect cannot be caught automatically or reviewed quickly by a person, narrow the system, choose a stronger model or put the rule in deterministic code.

Result 5: an impressive frontend failed browser checks

Qwen Code generated an ATS interface from an empty directory in 26:03. It added search, sorting, stages, a detail panel and 10,000 synthetic applicants. The list had no bounded scroll area, so the browser rendered 8,652 matching rows and 251,327 DOM elements. The first run also ended with a false loop-detector signal before full browser checking could begin.

Repairing six defects and then two visual issues brought total model time to 48:25. Independent checking confirmed only then that the row count was bounded, the page had no horizontal overflow and the desktop and mobile geometry was correct. DOM, the document tree of rendered page elements, is the size measure used here.

Deployment decision: an interface without behaviour and outcome checks remains a prototype. UI work needs behaviour, performance, responsive checks and visual review. A document workflow needs equivalent checks for citations, handling of missing sources and permissions.

Project stop table

What a company receives before deciding to build

Syntalith can run a short evaluation designed to find reasons to stop. We choose one process, build 20–50 cases, compare a local checkpoint with a sensible API baseline and measure time, cost, corrections and risk. The result may favour local Qwen, a smaller model, deterministic code, an API or ending the project.

The free process scan helps select the case. An AI process audit starting at PLN 4,990 net covers the comparison, acceptance criteria, architecture and fixed quote. The company gets a measured decision before infrastructure spend. A model chosen in advance receives no special treatment.