RocketMQ--性能测试

1,672 阅读8分钟

工具清单

consumer.sh:消息消费的benchmark工具 producer.sh: 消息生产benchmark工具(同步非批处理模式)

 [root@rocketmq1 benchmark]# pwd
 /home/rocketmq-all-4.9.0-source-release/distribution/target/rocketmq-4.9.0/rocketmq-4.9.0/benchmark
 ​
 [root@rocketmq1 benchmark]# sh producer.sh -h
 [root@rocketmq1 benchmark]# Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=320m; support was removed in 8.0
 Java HotSpot(TM) 64-Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release.
 usage: benchmarkProducer [-a <arg>] [-h] [-k <arg>] [-l <arg>] [-m <arg>] [-n <arg>] [-q <arg>] [-s <arg>] [-t
        <arg>] [-w <arg>]
  -a,--aclEnable <arg>         Acl Enable, Default: false
  -h,--help                    Print help
  -k,--keyEnable <arg>         Message Key Enable, Default: false
  -l,--tagCount <arg>          Tag count, Default: 0
  -m,--msgTraceEnable <arg>    Message Trace Enable, Default: false
  -n,--namesrvAddr <arg>       Name server address list, eg: 192.168.0.1:9876;192.168.0.2:9876
  -q,--messageQuantity <arg>   Send message quantity, Default: 0, running forever
  -s,--messageSize <arg>       Message Size, Default: 128
  -t,--topic <arg>             Topic name, Default: BenchmarkTest
  -w,--threadCount <arg>       Thread count, Default: 64

命令参数:

-k 是否使用message key,true则使用timestamp作为message的key

-n 指定nameserver地址, 例如:192.168.31.205:9876;192.168.31.206:9876;192.168.31.207:9876

-q 发送信息的条数,默认为0一直发送。

-s 指定消息大小,默认128字节

-t 指定topic,默认使用BenchmarkTest,如果指定其他记得先创建对应的topic

-w 开启的发送生产消息的线程数

返回结果:

SendTPS:生产消息的TPS(Transactions Per Second 的缩写,每秒处理的事务数目)

Max RT:最大响应时间(毫秒)

Average RT:平均响应时间(毫秒)

Send Failed:发送失败的总请求数

Response Failed:返回失败的总响应数

 [root@rocketmq1 benchmark]# sh consumer.sh -h
 [root@rocketmq1 benchmark]# Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=320m; support was removed in 8.0
 Java HotSpot(TM) 64-Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release.
 usage: benchmarkConsumer [-a <arg>] [-e <arg>] [-f <arg>] [-g <arg>] [-h] [-m <arg>] [-n <arg>] [-p <arg>] [-r
        <arg>] [-t <arg>] [-w <arg>]
  -a,--aclEnable <arg>             Acl Enable, Default: false
  -e,--expression <arg>            filter expression content file path.ie: ./test/expr
  -f,--filterType <arg>            TAG, SQL92
  -g,--group <arg>                 Consumer group name, Default: benchmark_consumer
  -h,--help                        Print help
  -m,--msgTraceEnable <arg>        Message Trace Enable, Default: false
  -n,--namesrvAddr <arg>           Name server address list, eg: 192.168.0.1:9876;192.168.0.2:9876
  -p,--group suffix enable <arg>   Consumer group suffix enable, Default: false
  -r,--fail rate <arg>             consumer fail rate, default 0
  -t,--topic <arg>                 Topic name, Default: BenchmarkTest
  -w,--threadCount <arg>           Thread count, Default: 20

命令参数:

-e 配合filter参数使用,过滤的条件表达式

-f 过滤方式

-g 指定消费组,默认为benchmark_consumer

-n 指定nameserver地址,例如:192.168.31.205:9876;192.168.31.206:9876;192.168.31.207:9876

-p 是否给消费组添加后缀,默认会给指定的消费组后添加后缀

-r 指定消费失败率,只要没有超过消费失败率,消费失败都会重试

-t 指定topic,默认使用BenchmarkTest,如果指定其他记得先创建对应的topic

-w 开启的接收消息的线程数

返回结果:

TPS: 消费的TPS(Transactions Per Second 的缩写,每秒处理的事务数目)

FAIL:消费失败的总数

AVG(B2C):broker到Consumer的平均响应时间(毫秒)

AVG(S2C):nameserver到Consumer的平均响应时间(毫秒)

MAX(B2C): broker到Consumer的最大响应时间(毫秒)

MAX(S2C): nameserver到Consumer的最大响应时间(毫秒)

 sh consumer.sh -t BenchmarkTest -n 192.168.31.206:9876 > /tmp/consumer.log &

结束进程:

 ps -ef |grep org.apache.rocketmq.example.benchmark
 [root@rocketmq1 ~]# cd /home/rocketmq-all-4.9.0-source-release/distribution/target/rocketmq-4.9.0/rocketmq-4.9.0/benchmark/
 [root@rocketmq1 benchmark]# sh producer.sh -n 192.168.31.206:9876
 [root@rocketmq1 benchmark]# sh consumer.sh -n 192.168.31.206:9876

生产者TPS:

 [root@rocketmq1 benchmark]# sh producer.sh -n 192.168.31.206:9876
 [root@rocketmq1 benchmark]# Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=320m; support was removed in 8.0
 Java HotSpot(TM) 64-Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release.
 topic: BenchmarkTest threadCount: 64 messageSize: 128 keyEnable: false propertySize: 0 tagCount: 0 traceEnable: false aclEnable: false messageQuantity: 0
 19:46:12.973 [main] DEBUG i.n.u.i.l.InternalLoggerFactory - Using SLF4J as the default logging framework
 RocketMQLog:WARN No appenders could be found for logger (io.netty.util.internal.PlatformDependent0).
 RocketMQLog:WARN Please initialize the logger system properly.
 Current Time: 1627818382916 Send TPS: 5276 Max RT(ms): 462 Average RT(ms):  12.123 Send Failed: 0 Response Failed: 0
 Current Time: 1627818392916 Send TPS: 4511 Max RT(ms): 462 Average RT(ms):  14.199 Send Failed: 0 Response Failed: 0
 Current Time: 1627818402917 Send TPS: 3457 Max RT(ms): 1051 Average RT(ms):  18.555 Send Failed: 0 Response Failed: 0
 Current Time: 1627818412916 Send TPS: 3854 Max RT(ms): 1051 Average RT(ms):  16.634 Send Failed: 0 Response Failed: 0
 Current Time: 1627818422916 Send TPS: 3812 Max RT(ms): 1051 Average RT(ms):  16.821 Send Failed: 0 Response Failed: 0
 Current Time: 1627818432916 Send TPS: 4615 Max RT(ms): 1051 Average RT(ms):  13.858 Send Failed: 0 Response Failed: 0
 Current Time: 1627818442916 Send TPS: 4528 Max RT(ms): 1051 Average RT(ms):  14.138 Send Failed: 0 Response Failed: 0
 Current Time: 1627818452916 Send TPS: 4578 Max RT(ms): 1051 Average RT(ms):  13.987 Send Failed: 0 Response Failed: 0
 Current Time: 1627818462916 Send TPS: 4731 Max RT(ms): 1051 Average RT(ms):  13.545 Send Failed: 0 Response Failed: 0
 Current Time: 1627818472916 Send TPS: 4719 Max RT(ms): 1051 Average RT(ms):  13.554 Send Failed: 0 Response Failed: 0
 Current Time: 1627818482916 Send TPS: 4651 Max RT(ms): 1051 Average RT(ms):  13.765 Send Failed: 0 Response Failed: 0
 Current Time: 1627818492916 Send TPS: 4247 Max RT(ms): 1056 Average RT(ms):  15.065 Send Failed: 0 Response Failed: 0
 Current Time: 1627818502916 Send TPS: 4615 Max RT(ms): 1056 Average RT(ms):  13.873 Send Failed: 0 Response Failed: 0
 Current Time: 1627818512916 Send TPS: 4461 Max RT(ms): 1056 Average RT(ms):  14.302 Send Failed: 0 Response Failed: 0
 Current Time: 1627818522920 Send TPS: 4673 Max RT(ms): 1056 Average RT(ms):  13.672 Send Failed: 0 Response Failed: 0
 Current Time: 1627818532916 Send TPS: 3952 Max RT(ms): 1056 Average RT(ms):  13.844 Send Failed: 0 Response Failed: 26
 Current Time: 1627818542916 Send TPS: 4862 Max RT(ms): 1056 Average RT(ms):  13.147 Send Failed: 0 Response Failed: 26

消费者TPS:

 Current Time: 1627818393853 TPS: 15813 FAIL: 0 AVG(B2C) RT(ms): 8927713.640 AVG(S2C) RT(ms): 8927150.286 MAX(B2C) RT(ms): 8957517 MAX(S2C) RT(ms): 8957268
 Current Time: 1627818403853 TPS: 121414 FAIL: 0 AVG(B2C) RT(ms): 8809188.316 AVG(S2C) RT(ms): 8808765.007 MAX(B2C) RT(ms): 8958431 MAX(S2C) RT(ms): 8958177
 Current Time: 1627818413853 TPS: 114379 FAIL: 0 AVG(B2C) RT(ms): 8194979.611 AVG(S2C) RT(ms): 8194555.552 MAX(B2C) RT(ms): 8778402 MAX(S2C) RT(ms): 8778149
 Current Time: 1627818423853 TPS: 61572 FAIL: 0 AVG(B2C) RT(ms): 6167205.915 AVG(S2C) RT(ms): 6166942.961 MAX(B2C) RT(ms): 8589939 MAX(S2C) RT(ms): 8589690
 Current Time: 1627818433853 TPS: 4635 FAIL: 0 AVG(B2C) RT(ms):  16.493 AVG(S2C) RT(ms): -390.830 MAX(B2C) RT(ms): 144 MAX(S2C) RT(ms): 0
 Current Time: 1627818443853 TPS: 4576 FAIL: 0 AVG(B2C) RT(ms):  16.859 AVG(S2C) RT(ms): -360.405 MAX(B2C) RT(ms): 110 MAX(S2C) RT(ms): 0
 Current Time: 1627818453853 TPS: 4550 FAIL: 0 AVG(B2C) RT(ms):  16.788 AVG(S2C) RT(ms): -371.261 MAX(B2C) RT(ms): 119 MAX(S2C) RT(ms): 0
 Current Time: 1627818463853 TPS: 4726 FAIL: 0 AVG(B2C) RT(ms):  15.794 AVG(S2C) RT(ms): -402.021 MAX(B2C) RT(ms): 107 MAX(S2C) RT(ms): 0
 Current Time: 1627818473853 TPS: 4805 FAIL: 0 AVG(B2C) RT(ms):  16.345 AVG(S2C) RT(ms): -388.560 MAX(B2C) RT(ms): 129 MAX(S2C) RT(ms): 0
 Current Time: 1627818483853 TPS: 4576 FAIL: 0 AVG(B2C) RT(ms):  15.676 AVG(S2C) RT(ms): -395.823 MAX(B2C) RT(ms): 280 MAX(S2C) RT(ms): 30
 Current Time: 1627818493853 TPS: 4287 FAIL: 0 AVG(B2C) RT(ms):  18.059 AVG(S2C) RT(ms): -344.192 MAX(B2C) RT(ms): 103 MAX(S2C) RT(ms): 0
 Current Time: 1627818503853 TPS: 4660 FAIL: 0 AVG(B2C) RT(ms):  16.543 AVG(S2C) RT(ms): -391.125 MAX(B2C) RT(ms): 178 MAX(S2C) RT(ms): 0
 Current Time: 1627818513853 TPS: 4423 FAIL: 0 AVG(B2C) RT(ms):  17.403 AVG(S2C) RT(ms): -348.632 MAX(B2C) RT(ms): 133 MAX(S2C) RT(ms): 0
 Current Time: 1627818523853 TPS: 4734 FAIL: 0 AVG(B2C) RT(ms):  15.806 AVG(S2C) RT(ms): -400.024 MAX(B2C) RT(ms): 209 MAX(S2C) RT(ms): 0
 Current Time: 1627818533853 TPS: 3989 FAIL: 0 AVG(B2C) RT(ms):  15.950 AVG(S2C) RT(ms): -402.737 MAX(B2C) RT(ms): 487 MAX(S2C) RT(ms): 225
 Current Time: 1627818543853 TPS: 4827 FAIL: 0 AVG(B2C) RT(ms):  15.824 AVG(S2C) RT(ms): -402.517 MAX(B2C) RT(ms): 140 MAX(S2C) RT(ms): 0
 Current Time: 1627818553853 TPS: 4663 FAIL: 0 AVG(B2C) RT(ms):  15.906 AVG(S2C) RT(ms): -367.178 MAX(B2C) RT(ms): 141 MAX(S2C) RT(ms): 0
 Current Time: 1627818563853 TPS: 4405 FAIL: 0 AVG(B2C) RT(ms):  17.627 AVG(S2C) RT(ms): -345.836 MAX(B2C) RT(ms): 172 MAX(S2C) RT(ms): 0
 Current Time: 1627818573854 TPS: 4463 FAIL: 0 AVG(B2C) RT(ms):  16.608 AVG(S2C) RT(ms): -348.164 MAX(B2C) RT(ms): 159 MAX(S2C) RT(ms): 0