Kafka Connect相关插件配置文档之十一

聆听金生
195 阅读5分钟

本文已参与「新人创作礼」活动，一起开启掘金创作之路。

Kafka Connect相关插件配置文档之一

Kafka Connect相关插件配置文档之二

Kafka Connect相关插件配置文档之三

Kafka Connect相关插件配置文档之四

Kafka Connect相关插件配置文档之五

Kafka Connect相关插件配置文档之六

Kafka Connect相关插件配置文档之七

Kafka Connect相关插件配置文档之八

Kafka Connect相关插件配置文档之九

Kafka Connect相关插件配置文档之十

3.7.6 Retries

  • max.retries

    任务失败前重试错误的最大次数。类型：int默认值：10有效值：[0，…]重要性：中等

  • retry.backoff.ms

    错误尝试重试之前等待的时间（以毫秒为单位）。类型：int默认值：3000有效值：[0，…]重要性：中等

3.8 示例

------------------------------------流处理---------------------------------------
curl -i -k  -H "Content-type: application/json" -X POST -d  '{
    "name":"mysql-sink",
    "config":{
        "connector.class":"io.confluent.connect.jdbc.JdbcSinkConnector",
        "connection.url":"jdbc:mysql://mysql:3306/test",
        "connection.user":"root",
        "connection.password":"root",
        "dialect.name":"MySqlDatabaseDialect",
        "topics":"dataxhive",
        "insert.mode":"upsert",
        "table.name.format":"test1",
        "pk.mode":"record_value",
        "pk.fields":"id",
        "tasks.max":"1",
        "key.converter":"org.apache.kafka.connect.storage.StringConverter",
        "value.converter":"io.confluent.connect.avro.AvroConverter",
        "value.converter.schema.registry.url":"http://master:8081"
    }
}' http://localhost:8083/connectors

------------------------------------mudesa--------------------------------------
curl -i -k  -H "Content-type: application/json" -X POST -d  '{
    "name":"mysql-sink",
    "config":{
        "connector.class":"io.confluent.connect.jdbc.JdbcSinkConnector",
        "connection.url":"jdbc:mysql://mysql:3306/test",
        "connection.user":"root",
        "connection.password":"root",
        "dialect.name":"MySqlDatabaseDialect",
        "topics":"dataxhive",
        "insert.mode":"upsert",
        "table.name.format":"test1",
        "pk.mode":"record_value",
        "pk.fields":"id",
        "tasks.max":"1",
        "key.converter":"org.apache.kafka.connect.storage.StringConverter",
        "value.converter":"io.confluent.connect.avro.AvroConverter",
        "value.converter.schema.registry.url":"http://master:8081",
        "type":"key",
        "offsetKey":"dataxhive"
    }
}' http://localhost:8083/connectors
  1. user原表数据

1 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 2 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 3 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 4 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 5 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 6 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 7 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 8 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 9 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 10 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 11 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 12 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 13 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 14 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 15 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 16 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 17 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 18 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 19 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 20 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 21 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 22 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 23 1.20 1.055 1.25 2020-05-16 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000

  1. 导入mysql数据

1 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 2 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 3 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 4 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 5 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 6 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 7 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 8 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 9 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 10 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 11 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 12 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 13 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 14 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 15 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 16 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 17 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 18 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 19 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 20 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 21 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 22 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000 23 1.20 1.055 1.25 2020-05-15 dadadas 16:04:42.000000 2020-05-16 16:04:47.000000

avatar
文章
阅读
粉丝