Kafka / Event Stream Partition Calculator
A deterministic tool for calculating the minimum safe partition count for a Kafka topic based on producer and consumer throughput targets.
Who this Lab is for
Designed for
- Data and backend engineers
- Event streaming architects
- Platform administrators
Use it when
- Creating a new high-throughput Kafka topic
- Tuning consumers that cannot keep up with production
- Reviewing event stream topologies
A complete run, step by step
Determine target throughput
Identify the total peak bandwidth the topic needs to handle in MB/s.
Measure producer speed
Benchmark the maximum throughput a single producer instance can push.
Measure consumer speed
Benchmark the maximum throughput a single consumer instance can process.
Apply partitioning formula
Calculate the exact partitions required to satisfy the throughput constraints of both producers and consumers.
What you will need
Prepare the following information before starting. Use measured evidence where possible; defaults are examples and should not be treated as recommendations.
Target throughput (MB/s)
The total peak bandwidth the topic needs to process.
Single producer speed (MB/s)
The maximum throughput a single producer instance can push to a partition.
Single consumer speed (MB/s)
The maximum throughput a single consumer instance can read and process from a partition.
What the result tells you
Your report includes
- Minimum required partitions for throughput
- Optimal partition count for parallelism
- Clear constraints for ordering guarantees
How it is determined
The tool applies Confluent's standard partitioning formula: Max(Target / Producer Speed, Target / Consumer Speed). It ensures sufficient parallelism without over-partitioning, which can degrade cluster performance.
Follows standard Confluent partitioning formulas by finding the maximum bottleneck between producer and consumer throughput boundaries.
Model assumptions
- • Messages have an even key distribution without significant partition skew.
- • Hardware can support the network and disk I/O demands of the calculated partitions.
- • Broker instances are correctly provisioned for the replication factor.
Authoritative references
500 MB/s ingestion stream
Situation
The system must process 500 MB/s. A producer can push 50 MB/s, and a consumer can read 20 MB/s.
Result
The calculation divides 500 by 20, dictating an absolute minimum of 25 partitions to achieve the consumer throughput target.
Use the result with engineering judgement
- It assumes even distribution of keys. Key skew can bottleneck specific partitions.
- It does not calculate replication factor impact on disk I/O.
Questions before you begin
Why not just use 100 partitions for everything?
Over-partitioning increases metadata overhead in ZooKeeper/KRaft and increases end-to-end latency and recovery time during broker failures.
Can I change partitions later?
Yes, but increasing partitions changes the key hashing, breaking strict message ordering for existing keys.
Should I over-provision partitions for future growth?
It is generally better to plan partitions for your 1-2 year target rather than over-provisioning massively, to save on metadata overhead.
Ready to run Partition Calculator?
Log in to open the full interactive workspace. Your completed result can be saved, revisited and exported as a private report.
Log in and start