测试环境
3台阿里云ECS，每台ECS部署单个节点，3台Broker
机器配置
- 系统 ：CentOS 7.6 64位
- CPU ：4核
- 内存 ：8GB
- 磁盘 ：500GB 云SSD
- Java ：OpenJDK Runtime Environment (build 1.8.0_212-b04)
- Kafka ：2.12-2.0.0
测试数据一览
| batch.size(Byte) | buffer.memory(Byte) | record.size(Byte) | TPS | IOPS(MB/s) | avg.latency(ms) | max.latency(ms) | 50th latency(ms) | 95th latency(ms) | 99th latency(ms) | 99.9th latency(ms) |
|---|---|---|---|---|---|---|---|---|---|---|
| 16384 | 33554432 | 100 | 302005.3153 | 28.8 | 798.04 | 1335 | 830 | 1240 | 1297 | 1329 |
| 16384 | 33554432 | 200 | 174137.1504 | 33.21 | 867.44 | 1181 | 921 | 1108 | 1141 | 1165 |
| 16384 | 33554432 | 500 | 72406.05315 | 34.53 | 885.8 | 2090 | 886 | 1593 | 1920 | 2052 |
| 16384 | 33554432 | 1000 | 37120.90278 | 35.4 | 872.44 | 2581 | 661 | 2280 | 2482 | 2555 |
| 16384 | 33554432 | 2000 | 17908.30946 | 34.16 | 876.73 | 1506 | 911 | 1214 | 1433 | 1502 |
| 16384 | 33554432 | 3906 | 9412.119044 | 35.06 | 854.83 | 1779 | 972 | 1527 | 1648 | 1755 |
| 16384 | 67108864 | 100 | 297743.1072 | 28.39 | 1521.64 | 2088 | 1796 | 1978 | 2004 | 2039 |
| 16384 | 67108864 | 200 | 168588.5764 | 32.16 | 1726.93 | 2454 | 1780 | 2158 | 2411 | 2448 |
| 16384 | 67108864 | 500 | 75266.06554 | 35.89 | 1687.83 | 2466 | 1823 | 2269 | 2368 | 2416 |
| 16384 | 67108864 | 1000 | 37109.05609 | 35.39 | 1733.59 | 3768 | 1851 | 3251 | 3405 | 3750 |
| 16384 | 67108864 | 2000 | 18557.00713 | 35.39 | 1664.47 | 2061 | 1761 | 2010 | 2049 | 2058 |
| 16384 | 67108864 | 3906 | 9252.748066 | 34.47 | 1708.58 | 3161 | 1665 | 2841 | 2983 | 3139 |
| 32768 | 33554432 | 100 | 636083.7595 | 60.66 | 67.62 | 486 | 30 | 253 | 354 | 444 |
| 32768 | 33554432 | 200 | 335363.3998 | 63.97 | 462.28 | 1440 | 297 | 1262 | 1351 | 1414 |
| 32768 | 33554432 | 500 | 140394.227 | 66.95 | 450.3 | 1010 | 431 | 866 | 930 | 1004 |
| 32768 | 33554432 | 1000 | 72127.17463 | 68.79 | 447.05 | 1043 | 479 | 841 | 913 | 1019 |
| 32768 | 33554432 | 2000 | 36563.87343 | 69.74 | 444.37 | 1855 | 236 | 1234 | 1441 | 1795 |
| 32768 | 33554432 | 3906 | 17221.18895 | 64.15 | 462.8 | 1072 | 406 | 833 | 1008 | 1065 |
| 32768 | 67108864 | 100 | 635428.9781 | 60.6 | 57.68 | 829 | 28 | 214 | 367 | 581 |
| 32768 | 67108864 | 200 | 328757.0354 | 62.71 | 936.46 | 2382 | 978 | 2079 | 2201 | 2288 |
| 32768 | 67108864 | 500 | 133757.7914 | 63.78 | 929.35 | 1975 | 930 | 1786 | 1928 | 1958 |
| 32768 | 67108864 | 1000 | 69072.22192 | 65.87 | 929.2 | 2500 | 863 | 1836 | 2138 | 2455 |
| 32768 | 67108864 | 2000 | 35931.15591 | 68.53 | 900.59 | 2602 | 731 | 2227 | 2441 | 2550 |
| 32768 | 67108864 | 3906 | 17583.34506 | 65.5 | 900.1 | 1329 | 996 | 1189 | 1257 | 1309 |
压测结果
- batch-size 参数对TPS影响非常大，提升一倍后TPS也相应提升一倍
- 消息大小同样非常影响TPS，消息体越大，TPS越低
- buffer.memory越大，平均发送延迟也相应增大
- 默认client配置（不压缩，acks=all）下100B大小的消息体，集群写入TPS约为30W
