Maintaining local Qwen: turn failures into tests

A local model needs care across the client, server, memory, tools and test repository. In our RTX 3090 experiment, each failure pointed to a different control. This article turns those observations into checks so the next operator can reproduce the working profile and locate a fault quickly.

These are examples from an after-hours home-PC experiment. They are not production incidents or client history.

Eight failures from one experiment

The failures belonged to different layers. Number 1 was client configuration, numbers 2 and 6 were launcher issues, number 3 was the harness, number 4 was context management, number 5 was the test repository and number 8 was the upgrade process. A general judgement about answer quality would have found none of them.

From symptom to repeatable test

Each entry should state the symptom, reproduction, expected behaviour, evidence, owner and version decision:

id: QWEN-OPS-004
symptom: request exceeds server context and returns HTTP 400
reproduction: resume a long session until the preflight threshold
expected: client compacts or refuses before sending an oversized request
artifacts: event stream, request length, server error, profile version
owner: agent-platform
version_change: switch client or profile; do not enlarge the limit without a test

Run the check at the layer that failed. A wrong key needs /health, /v1/models, a 401 for the bad secret and the expected model ID for the good one. The 8k defect needs a long tool write because a short smoke test passes.

Upgrade gate

Before changing the model, vLLM image, Qwen Code or driver, run three levels of checks.

Five minutes: startup and protocol

• endpoint becomes healthy;
• expected model and context are reported;
• bad secret is rejected;
• response stream terminates correctly;
• small JSON and a tool call satisfy their schemas.

About thirty minutes: client and harness

• a write longer than 8k is not truncated;
• a resumed session can reread and edit;
• client does not submit more context than the server accepts;
• loop detection requires evidence of no progress;
• git status shows no cache or unrelated files.

One process task: quality and cost

Use the same natural request, a fresh worktree, hidden checks, diff review, elapsed time and system metrics. The retained profile used the Go json.Marshal error-handling task, which took 8:22 through Qwen Code. In this single run, Codex awarded 100/100 under Syntalith's criteria: correctness (40 points), regression tests (20), compatibility (15), scope and maintainability (10), verification (10) and documentation (5). The run driven by the Codex harness introduced nondeterministic fallback behaviour and failed review.

A new release number is not an acceptance criterion. The new profile should clear critical checks and stay within the agreed regression budget.

What to record at handover

deployment/
  profiles/
    daily.yaml
    fallback.yaml
  manifests/
    model.json
    server-image.json
    licenses.md
  runbooks/
    start-stop-status.md
    context-overflow.md
    gpu-oom.md
    key-rotation.md
    prior-version.md
  tests/
    smoke/
    client-regressions/
    process-cases/
  failures/
    QWEN-OPS-001.md
    ...
  training/
    operator-lab.md
    reviewer-criteria.md

The manifest should contain the model version, format and hash, image, patch commit, driver, tool parser, sampling settings, context and client. latest is not reproducible.

A runbook supplies the command, expected output and first diagnostic step. The person taking over should perform startup, testing and switching to the prior version without relying on the configuration author.

Two monitoring layers

Operations monitoring records endpoint availability, queue, time to first token, total time, input and output lengths, history compactions, provider and tool errors, VRAM, GPU use, process exits and profile version.

Quality monitoring records hidden-check results, acceptance without correction, human correction category, permission or citation violations, diff scope and justified or incorrect refusal rates.

A GPU panel cannot see a blank candidate name. A "good answer" comment cannot show that the next turn is about to overflow context. Both layers are needed.

Practice switching to the prior version

On one GPU, switching should be short and familiar:

1. stop admitting new work;
2. finish or terminate the queue according to policy;
3. select the prior image and profile from the manifest;
4. verify model ID, context and secret;
5. run startup checks and one critical case;
6. restore traffic and retain failure records.

The home profile switched from vLLM to an llama.cpp fallback with a 120k context. A company also needs a recovery-time target, permission test and user communication. A slower fallback is useful when the team knows its behaviour.

Train the team

An operator should start a profile, identify a wrong address, read metrics, rotate a secret and switch versions. A reviewer needs criteria and hidden cases to separate fluent prose from accepted output. The process owner defines critical errors. Security checks data, network, tools and retention.

The best exercise uses a copy of the environment. One person deliberately introduces a ledger failure, another diagnoses it from logs and the runbook, then runs the repair check.

What Syntalith hands over

An implementation includes agreed profiles, manifests and licences, integration code, quality tests, status panel, alerts, version switching, known-failure records and training material. The team performs startup, testing and version switching before acceptance.

Syntalith can deliver the system through custom AI applications and train the owning team through AI-Native courses. A free process scan helps decide whether the next step is a pilot, audit, training or repair of an existing deployment.