One RTX 3090 for a team: measured Qwen concurrency

Can one RTX 3090 serve a team? Our experiment did not answer that question because the home profile had exactly one inference slot. We will not turn single-task durations into theoretical “jobs per hour” or guess a user count.

We do have two honest sources. The first is our measured service time for single agents. The second is a public Qwen3.8-27B benchmark designed for 1, 2, 4 and 8 concurrent streams on one RTX 3090. Together they show where model fit ends and service design begins.

What our one-slot profile actually showed

We served one request at a time on an after-hours home PC. Subagents waited in sequence. Task duration varied substantially:

These are real single runs. They do not establish hourly capacity under load. They exclude concurrent prefill, profile switching, queues and retries after a failure.

A public concurrency test on the same GPU class

The syv-ai/qwen38-27b-rtx3090 repository publishes a Qwen3.8-27B test on one RTX 3090 using a patched vLLM stack. Each stream received a distinct 4k-token prompt and generated 256 tokens. The server admitted at most eight sequences.

The MTP profile produced:

Aggregate throughput rose while each user's response slowed. At eight streams, each stream received 23 tok/s rather than 126 tok/s. A single tokens/s number hides this trade-off.

The DFlash2 profile reached 137 tok/s for one stream, 97 for two and 46 for four. When eight were requested, only five sequences remained resident, so the author did not report a steady C8 result. The fastest single-user profile was not the highest-capacity profile.

Long prompts can dominate the concurrency count

The same public repository ran eight independent 16k-token prompts. End-to-end aggregate throughput fell to 15.8 tok/s and mean time to first token reached 71.7 s because roughly 131k tokens of prefill dominated the run.

When eight requests shared the same 16k prefix and prefix caching could work, two streams produced 128.4 tok/s aggregate with 1.3 s mean TTFT. Four streams produced 147.8 tok/s and 2.6 s. These are not promises for a company chat service. They demonstrate that prompt-length distribution and shared prefixes are capacity inputs. They are operationally significant.

Why these numbers do not go straight into a proposal

The public benchmark uses a tuned, modified stack, 256-token answers and tightly defined prompts. Our acceptance profile used another runtime and multi-turn agents that called tools. The cells are not interchangeable.

A team deployment needs measurements on the target server:

• time to first token and full-task latency for each task class;
• per-user and aggregate throughput at 1, 2, 4 and 8 requests;
• resident sequence count before memory failure;
• the effect of long prefill and prefix caching;
• 95th-percentile queue delay under representative traffic;
• accepted-task rate, because a fast wrong answer is not business throughput.

What can be concluded now

One RTX 3090 can serve more than one short sequence, but the per-user slowdown is measurable and capacity depends on the profile. Our one-slot setup was appropriate for a coding experiment. It provides no evidence for a team architecture. The public benchmark shows MTP sustaining eight short streams in one specific stack while DFlash2 stopped at five resident sequences.

Target TTFT, queue delay and accepted-result rate should determine the trigger for a second GPU. An unsupported employee count should not.

How Syntalith runs this pilot

We can test the model on client hardware or build an isolated pilot, replay peak traffic and deliver latency, throughput, VRAM and quality evidence. The design includes queueing, concurrency limits, data classes, controlled fallback and an explicit scaling trigger.

See the AI process audit and custom AI applications, or start with a free process scan. If local deployment wins the test, we can build the endpoint, control plane, monitoring and upgrade procedure, then teach the team to operate it.