安装Canal Admin1.1.5并同步RocketMq

1,394 阅读3分钟

前言经历:今天想用canal admin同步rocketmq,配了一个早上没配起来,原因是在官网看漏了一些东西(以为不重要),浪费了很多时间,特此记录下!!!

1.环境准备

1.1Canal下载

image.png 官网下载地址:github.com/alibaba/can…

1.2MySQL环境配置

  • 对于自建 MySQL , 需要先开启 Binlog 写入功能,配置 binlog-format 为 ROW 模式,my.cnf 中配置如下
log-bin=mysql-bin # 开启 binlog
binlog-format=ROW # 选择 ROW 模式
server_id=1 # 配置 MySQL replaction 需要定义,不要和 canal 的 slaveId 重复
  • 注意:针对阿里云 RDS for MySQL , 默认打开了 binlog , 并且账号默认具有 binlog dump 权限 , 不需要任何权限或者 binlog 设置,可以直接跳过这一步

  • 授权 canal 链接 MySQL 账号具有作为 MySQL slave 的权限, 如果已有账户可直接 grant

GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'canal'@'%';
-- GRANT ALL PRIVILEGES ON *.* TO 'canal'@'%' ;
FLUSH PRIVILEGES;

1.3修改Admin配置

vi conf/application.yml
  port: 8089
spring:
  jackson:
    date-format: yyyy-MM-dd HH:mm:ss
    time-zone: GMT+8

spring.datasource:
  address: 127.0.0.1:3306
  database: canal_manager
  username: canal
  password: canal
  driver-class-name: com.mysql.jdbc.Driver
  url: jdbc:mysql://${spring.datasource.address}/${spring.datasource.database}?useUnicode=true&characterEncoding=UTF-8&useSSL=false
  hikari:
    maximum-pool-size: 30
    minimum-idle: 1

canal:
  adminUser: admin
  adminPasswd: 123456

修改配置化启动admin服务

bin/startup.sh

1.4访问控制台

查看 admin 日志


2019-08-31 15:43:38.162 [main] INFO  o.s.boot.web.embedded.tomcat.TomcatWebServer - Tomcat initialized with port(s): 8089 (http)
2019-08-31 15:43:38.180 [main] INFO  org.apache.coyote.http11.Http11NioProtocol - Initializing ProtocolHandler ["http-nio-8089"]
2019-08-31 15:43:38.191 [main] INFO  org.apache.catalina.core.StandardService - Starting service [Tomcat]
2019-08-31 15:43:38.194 [main] INFO  org.apache.catalina.core.StandardEngine - Starting Servlet Engine: Apache Tomcat/8.5.29
....
2019-08-31 15:43:39.789 [main] INFO  o.s.w.s.m.m.annotation.ExceptionHandlerExceptionResolver - Detected @ExceptionHandler methods in customExceptionHandler
2019-08-31 15:43:39.825 [main] INFO  o.s.b.a.web.servlet.WelcomePageHandlerMapping - Adding welcome page: class path resource [public/index.html]

此时代表canal-admin已经启动成功,可以通过 http://127.0.0.1:8089/ 访问,默认密码:admin/123456

1.4安装canal-deployer

  • canal_local.properties 为注册admin配置
  • canal.properties 为本地配置
# 修改配置
vim canal_local.properties
# register ip 本服务在admin中的显示ip,名称也是用这个
canal.register.ip = 10.2.55.55

# canal admin config
#canal-admin的访问地址
canal.admin.manager = 10.2.55.55:8011  

# canal-deployer的控制端口
canal.admin.port = 11110 

# canal-admin的访问账号
canal.admin.user = admin 

# canal-admin的访问密码,与adminPasswd对应,这里是密文
canal.admin.passwd = 6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 

# admin auto register 集群配置
# 启动后自动注册到admin中
canal.admin.register.auto = true
# 集群名称 ,这里是单点可空着即可
canal.admin.register.cluster = 
# 节点名称名称,里是单点可空着即可
canal.admin.register.name =
  • 启动canal-deployer
# 启动服务 以canal_local.properties 配置方式
bin/startup.sh local

# 查看启动日志
tail -f logs/canal/canal.log
  • 刷新admin列表
  • 这里刷新后,这里的server会自动出现!(假如说不带上local的命令是出不来的!!)

image.png

可以通过admin修改server配置,集群模式必须在集群管理中统一配置。

image.png 重点注意下这几块的配置

######### 		common argument		#############
#################################################
# tcp bind ip
canal.ip =
# register ip to zookeeper
canal.register.ip =
canal.port = 11111
canal.metrics.pull.port = 11112
# canal instance user/passwd
# canal.user = canal
# canal.passwd = E3619321C1A937C46A0D8BD1DAC39F93B27D4458

# canal admin config
#canal.admin.manager = 127.0.0.1:8089
canal.admin.port = 11110
canal.admin.user = admin
canal.admin.passwd = 6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9
# admin auto register
#canal.admin.register.auto = true
#canal.admin.register.cluster =
#canal.admin.register.name =



#################################################
######### 		destinations		#############
#################################################
canal.destinations = 
# conf root dir
canal.conf.dir = ../conf
# auto scan instance dir add/remove and start/stop instance
canal.auto.scan = true
canal.auto.scan.interval = 5
# set this value to 'true' means that when binlog pos not found, skip to latest.
# WARN: pls keep 'false' in production env, or if you know what you want.
canal.auto.reset.latest.pos.mode = false



##################################################
######### 		    RocketMQ	     #############
##################################################
rocketmq.producer.group = test
rocketmq.enable.message.trace = false
rocketmq.customized.trace.topic =
rocketmq.namespace =
rocketmq.namesrv.addr = 127.0.0.1:9876
rocketmq.retry.times.when.send.failed = 0
rocketmq.vip.channel.enabled = false
rocketmq.tag = 

创建实例

基本配置下mysql的表和mq的队列就好啦

image.png

最终效果

之后修改mysql的表,就会同步到mq啦

image.png

附参考博客:

拯救我出来的博客!:www.cxyzjd.com/article/liz… 带点小坑的官网quickStart github.com/alibaba/can…