前言经历:今天想用canal admin同步rocketmq,配了一个早上没配起来,原因是在官网看漏了一些东西(以为不重要),浪费了很多时间,特此记录下!!!
1.环境准备
1.1Canal下载
官网下载地址:github.com/alibaba/can…
1.2MySQL环境配置
- 对于自建 MySQL , 需要先开启 Binlog 写入功能,配置 binlog-format 为 ROW 模式,my.cnf 中配置如下
log-bin=mysql-bin # 开启 binlog
binlog-format=ROW # 选择 ROW 模式
server_id=1 # 配置 MySQL replaction 需要定义,不要和 canal 的 slaveId 重复
-
注意:针对阿里云 RDS for MySQL , 默认打开了 binlog , 并且账号默认具有 binlog dump 权限 , 不需要任何权限或者 binlog 设置,可以直接跳过这一步
-
授权 canal 链接 MySQL 账号具有作为 MySQL slave 的权限, 如果已有账户可直接 grant
GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'canal'@'%';
-- GRANT ALL PRIVILEGES ON *.* TO 'canal'@'%' ;
FLUSH PRIVILEGES;
1.3修改Admin配置
vi conf/application.yml
port: 8089
spring:
jackson:
date-format: yyyy-MM-dd HH:mm:ss
time-zone: GMT+8
spring.datasource:
address: 127.0.0.1:3306
database: canal_manager
username: canal
password: canal
driver-class-name: com.mysql.jdbc.Driver
url: jdbc:mysql://${spring.datasource.address}/${spring.datasource.database}?useUnicode=true&characterEncoding=UTF-8&useSSL=false
hikari:
maximum-pool-size: 30
minimum-idle: 1
canal:
adminUser: admin
adminPasswd: 123456
修改配置化启动admin服务
bin/startup.sh
1.4访问控制台
查看 admin 日志
2019-08-31 15:43:38.162 [main] INFO o.s.boot.web.embedded.tomcat.TomcatWebServer - Tomcat initialized with port(s): 8089 (http)
2019-08-31 15:43:38.180 [main] INFO org.apache.coyote.http11.Http11NioProtocol - Initializing ProtocolHandler ["http-nio-8089"]
2019-08-31 15:43:38.191 [main] INFO org.apache.catalina.core.StandardService - Starting service [Tomcat]
2019-08-31 15:43:38.194 [main] INFO org.apache.catalina.core.StandardEngine - Starting Servlet Engine: Apache Tomcat/8.5.29
....
2019-08-31 15:43:39.789 [main] INFO o.s.w.s.m.m.annotation.ExceptionHandlerExceptionResolver - Detected @ExceptionHandler methods in customExceptionHandler
2019-08-31 15:43:39.825 [main] INFO o.s.b.a.web.servlet.WelcomePageHandlerMapping - Adding welcome page: class path resource [public/index.html]
此时代表canal-admin已经启动成功,可以通过 http://127.0.0.1:8089/ 访问,默认密码:admin/123456
1.4安装canal-deployer
- canal_local.properties 为注册admin配置
- canal.properties 为本地配置
# 修改配置
vim canal_local.properties
# register ip 本服务在admin中的显示ip,名称也是用这个
canal.register.ip = 10.2.55.55
# canal admin config
#canal-admin的访问地址
canal.admin.manager = 10.2.55.55:8011
# canal-deployer的控制端口
canal.admin.port = 11110
# canal-admin的访问账号
canal.admin.user = admin
# canal-admin的访问密码,与adminPasswd对应,这里是密文
canal.admin.passwd = 6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9
# admin auto register 集群配置
# 启动后自动注册到admin中
canal.admin.register.auto = true
# 集群名称 ,这里是单点可空着即可
canal.admin.register.cluster =
# 节点名称名称,里是单点可空着即可
canal.admin.register.name =
- 启动canal-deployer
# 启动服务 以canal_local.properties 配置方式
bin/startup.sh local
# 查看启动日志
tail -f logs/canal/canal.log
- 刷新admin列表
- 这里刷新后,这里的server会自动出现!(假如说不带上local的命令是出不来的!!)
可以通过admin修改server配置,集群模式必须在集群管理中统一配置。
重点注意下这几块的配置
######### common argument #############
#################################################
# tcp bind ip
canal.ip =
# register ip to zookeeper
canal.register.ip =
canal.port = 11111
canal.metrics.pull.port = 11112
# canal instance user/passwd
# canal.user = canal
# canal.passwd = E3619321C1A937C46A0D8BD1DAC39F93B27D4458
# canal admin config
#canal.admin.manager = 127.0.0.1:8089
canal.admin.port = 11110
canal.admin.user = admin
canal.admin.passwd = 6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9
# admin auto register
#canal.admin.register.auto = true
#canal.admin.register.cluster =
#canal.admin.register.name =
#################################################
######### destinations #############
#################################################
canal.destinations =
# conf root dir
canal.conf.dir = ../conf
# auto scan instance dir add/remove and start/stop instance
canal.auto.scan = true
canal.auto.scan.interval = 5
# set this value to 'true' means that when binlog pos not found, skip to latest.
# WARN: pls keep 'false' in production env, or if you know what you want.
canal.auto.reset.latest.pos.mode = false
##################################################
######### RocketMQ #############
##################################################
rocketmq.producer.group = test
rocketmq.enable.message.trace = false
rocketmq.customized.trace.topic =
rocketmq.namespace =
rocketmq.namesrv.addr = 127.0.0.1:9876
rocketmq.retry.times.when.send.failed = 0
rocketmq.vip.channel.enabled = false
rocketmq.tag =
创建实例
基本配置下mysql的表和mq的队列就好啦
最终效果
之后修改mysql的表,就会同步到mq啦
附参考博客:
拯救我出来的博客!:www.cxyzjd.com/article/liz… 带点小坑的官网quickStart github.com/alibaba/can…