kafka最好用的脚本三:kafka-config

4,366 阅读2分钟

kafka-config 使用手册

官方说明

Add/Remove entity config for a topic, client, user or broker.
Option                      Description
------                      -----------
--add-config <String>       Key Value pairs of configs to add. Square brackets
                              can be used to group values which contain commas:
                              'k1=v1,k2=[v1,v2,v2],k3=v3'. The following is a
                              list of valid configurations: For entity_type
                              'topics':
                            	cleanup.policy
                            	compression.type
                            	delete.retention.ms
                            	file.delete.delay.ms
                            	flush.messages
                            	flush.ms
                            	follower.replication.throttled.replicas
                            	index.interval.bytes
                            	leader.replication.throttled.replicas
                            	max.message.bytes
                            	message.format.version
                            	message.timestamp.difference.max.ms
                            	message.timestamp.type
                            	min.cleanable.dirty.ratio
                            	min.compaction.lag.ms
                            	min.insync.replicas
                            	preallocate
                            	retention.bytes
                            	retention.ms
                            	segment.bytes
                            	segment.index.bytes
                            	segment.jitter.ms
                            	segment.ms
                            	unclean.leader.election.enable
                            For entity_type 'brokers':
                            	follower.replication.throttled.rate
                            	leader.replication.throttled.rate
                            For entity_type 'users':
                            	request_percentage
                            	producer_byte_rate
                            	SCRAM-SHA-256
                            	SCRAM-SHA-512
                            	consumer_byte_rate
                            For entity_type 'clients':
                            	request_percentage
                            	producer_byte_rate
                            	consumer_byte_rate
                            Entity types 'users' and 'clients' may be specified
                              together to update config for clients of a
                              specific user.
--alter                     Alter the configuration for the entity.
--delete-config <String>    config keys to remove 'k1,k2'
--describe                  List configs for the given entity.
--entity-default            Default entity name for clients/users (applies to
                              corresponding entity type in command line)
--entity-name <String>      Name of entity (topic name/client id/user principal
                              name/broker id)
--entity-type <String>      Type of entity (topics/clients/users/brokers)
--force                     Suppress console prompts
--help                      Print usage information.
--zookeeper <String: urls>  REQUIRED: The connection string for the zookeeper
                              connection in the form host:port. Multiple URLS
                              can be given to allow fail-over.

使用方法

kafka-config 相对于 kafka-topics 而言, 在配置修改方面做得更全面,用户可以指定 config 的 对象 : topic, client, user or broker.

修改 topic 的参数:

可在创建topic时,通过--config进行指定项的参数配置,覆盖默认配置:
> bin/kafka-topics.sh --zookeeper localhost:2181 --create --topic my-topic --partitions 1 --replication-factor 1 --config max.message.bytes=64000 --config flush.messages=1

也可以在创建topic之后通过config.sh文件对其中的特定指标进行修改,下面操作对my-topic相关指标进行配置:
> bin/kafka-configs.sh --zookeeper localhost:2181 --entity-type topics --entity-name my-topic --alter --add-config max.message.bytes=128000

查看是否修改成功:
> bin/kafka-configs.sh --zookeeper localhost:2181 --entity-type topics --entity-name my-topic --describe

也可以撤销/删除某些指定配置,将该项重置为默认配置:
> bin/kafka-configs.sh --zookeeper localhost:2181 --entity-type topics --entity-name my-topic --alter --delete-config max.message.bytes

修改 broker 的参数:

改变当前broker 0上的log cleaner threads可以通过下面命令实现:
> bin/kafka-configs.sh --bootstrap-server localhost:9092 --entity-type brokers --entity-name 0 --alter --add-config log.cleaner.threads=2

查看当前broker 0的动态配置参数:
> bin/kafka-configs.sh --bootstrap-server localhost:9092 --entity-type brokers --entity-name 0 --describe

删除broker id为0的server上的配置参数/设置为默认值:
> bin/kafka-configs.sh --bootstrap-server localhost:9092 --entity-type brokers --entity-name 0 --alter --delete-config log.cleaner.threads

同时更新集群上所有broker上的参数(cluster-wide类型,保持所有brokers上参数的一致性):
> bin/kafka-configs.sh --bootstrap-server localhost:9092 --entity-type brokers --entity-default --alter --add-config log.cleaner.threads=2

查看当前集群中动态的cluster-wide类型的参数列表:
> bin/kafka-configs.sh --bootstrap-server localhost:9092 --entity-type brokers --entity-default --describe