One RTX 3090 for a team: measured Qwen concurrency

One RTX 3090 may handle short requests from several users, but headcount alone cannot answer the question. Prompt length, answer length, concurrent requests and the waiting time the business can accept all matter. Our home configuration processed one request at a time, so it measured the service time of a single agent. A public test on the same GPU class shows what changes when several streams run together.

This article brings together two sources. The first is our measured time for individual tasks. The second is a public Qwen3.8-27B test designed for 1, 2, 4 and 8 concurrent streams on one RTX 3090. Every figure cited here comes from a recorded run. Together, the sources separate model capacity from queue and server design and identify the measurements needed before deployment.

What we measured with one active job

On an after-hours home PC, we ran one request at a time. Other requests waited in the queue. VRAM means the graphics card's memory. Duration varied by task:

Codex awarded the 100/100 score after the Qwen Code run, using the Syntalith rubric. The 100 points cover correctness (40), regression tests (20), compatibility (15), scope discipline (10), verification (10) and documentation (5). The score describes this run; an independent laboratory did not produce it.

Each row is one run. These durations cannot be converted into jobs per hour under load. The measurement did not include concurrent prompt processing, configuration changes, queueing or retries after failure.

A public multi-stream test on an RTX 3090

The syv-ai/qwen38-27b-rtx3090 repository reports a Qwen3.8-27B test on one RTX 3090 with a patched vLLM stack. Each stream received its own 4k-token prompt and generated 256 tokens. The server admitted at most eight sequences.

MTP uses an additional model head to predict subsequent tokens. In this test, the author measured the server configuration with MTP:

Per-stream speed is the number of answer tokens generated per second for one user. Aggregate speed sums the streams. The model-pass time, or forward-pass time, is the compute time for one step.

Aggregate throughput rose while each user's response slowed. At eight streams, each received 23 tok/s compared with 126 tok/s for one stream. tok/s means answer tokens generated per second. A single tokens/s value hides the concurrency cost.

The DFlash2 measurement reached 137 tok/s for one stream, 97 for two and 46 for four. When eight streams were requested, only five sequences remained resident, so the author did not report a stable C8 result, meaning the eight-stream case. The one-stream configuration with the highest speed had less capacity.

Long prompts can dominate the result

The same repository ran eight independent 16k-token prompts. Aggregate throughput fell to 15.8 tok/s and mean time to first token reached 71.7 s. Processing roughly 131k input tokens, called prefill, consumed most of the work before answer generation.

When eight requests shared the same 16k prefix, prefix caching could reuse that work. Two streams produced 128.4 tok/s aggregate with a 1.3 s mean TTFT, where TTFT means time to first token. Four streams produced 147.8 tok/s and 2.6 s. These values describe this test. They do not guarantee the same result for a company chat service, and they show that prompt-length distribution and shared-prefix reuse affect service capacity.

Why these results do not go straight into a proposal

The public test uses a tuned, modified stack, 256-token answers and tightly specified prompts. Our configuration used another server, while the agents handled multi-step tasks with tools. The results cannot be directly compared or turned into a team forecast.

A deployment needs measurements on the target server for:

• time to first token and full-task latency for every task class;
• per-user and aggregate throughput at 1, 2, 4 and 8 requests;
• resident sequence count before a memory error;
• the effect of long prefill and shared-prefix caching;
• 95th-percentile waiting time under representative traffic, the value below which 95% of observations fall;
• the share of tasks accepted by the test suite, since a fast wrong answer adds no process throughput.

What follows from the evidence today

One RTX 3090 can process more than one short sequence, but per-user speed falls and capacity depends on the configuration. Our server allowed one active job at a time, which was suitable for a coding experiment and does not describe a team architecture. The public test sustained eight short streams in one specific MTP stack, while DFlash2 stopped at five resident sequences.

The target TTFT, queue length and accepted-result rate should set the trigger for a second GPU. Headcount alone is not a throughput metric.

How Syntalith runs this pilot

We can run the model on client hardware or build an isolated pilot. We replay peak traffic and report latency, throughput, VRAM and quality measurements. The design includes queueing, concurrency limits, data classes, a controlled fallback and an explicit scaling trigger.

An AI process audit starts at PLN 4,990 net. If the test supports local deployment, custom AI applications can include the API, control panel, monitoring and upgrade plan. Start with a free process scan.