Concurrency and throughput are related, but they are not the same metric.
Quick Definitions
- Concurrency: how many requests are in-flight at the same time.
- Throughput: how many requests are completed per second.
A system can have high concurrency and low throughput if latency is high.
Useful Relationship
A practical approximation is:
throughput ≈ concurrency / latency_seconds
This helps estimate expected throughput from known concurrency and average response time.
What to Optimize First
- If saturation is CPU-bound, optimize compute and query efficiency.
- If saturation is I/O-bound, optimize downstream latency and connection strategy.
- If queueing is the issue, improve backpressure and request prioritization.
Tooling
Use the Concurrency Calculator and API Throughput Calculator together to validate scenarios before running full load tests.