MaterializeMySQL引擎同步问题

1,111 阅读29分钟

MaterializeMySQL是最新的数据库引擎,目前属于实验阶段。此引擎可以实时同步MySQL数据库整个库的数据。

问题一

创建MaterializeMySQL引擎后,没有发现数据同步,查看错误日志,如下:

2021.01.26 21:17:52.947907 [ 124159 ] {} MaterializeMySQLSyncThread: Code: 48, e.displayText() = DB::Exception: The uni_center.t_log cannot be materialized, because there is no primary keys., Stack trace (when copying this message, always include the lines below):

该错误提示MySQL表中未建立主键,增加主键后错误消失。

问题二

2021.01.25 19:26:25.569419 [ 23151 ] {} MaterializeMySQLSyncThread(uni_center): Query EXTERNAL DDL FROM MySQL(uni_center, uni_center) CREATE TABLE `t_log` ( `id` bigint DEFAULT NULL, `resource` varchar(255) DEFAULT NULL, `operateTime` datetime DEFAULT NULL, `isCheck` bit(1) DEFAULT b'0', `uni_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL COMMENT '缁熶竴缂栧彿', `operate` int DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8 wasn't finished successfully: Code: 538, e.displayText() = DB::Exception: Cannot parse MySQL create query., Stack trace (when copying this message, always include the lines below):

导致该错误的原因是建表语句中的isCheck字段数据类型为bit,Clickhouse不支持bit类型;其他不支持的类型还有比如:enum,point等等,目前支持的字段如下:

问题三

2021.02.08 17:19:19.670602 [ 23186 ] {dc37cbb8-a79f-4242-b154-5a432442e566} DynamicQueryHandler: Code: 33, e.displayText() = DB::Exception: Cannot read all data. Bytes read: 8113. Bytes expected: 11572., Stack trace (when copying this message, always include the lines below)

解决:

<yandex>
    <merge_tree>
        <in_memory_parts_enable_wal>0</in_memory_parts_enable_wal>
    </merge_tree>
</yandex>

service clickhouse-server restart