背景：之前通过wal2json+flinksql 同步到sr中，对于业务库的delete操作，在sr中并不能实际删除，但是 业务测希望做到物理删除。
解决办法: 发现sr中提供routine load的方式实时监听kafka topic，并通过打标识在识别是删除，还是新增和更新。在routine load中，最后一位type ，0为 新增和更新即 I，U
创建routine load方式如下：
`
CREATE ROUTINE LOAD dw_sr.bus_fund_label_value_mapping_portfolio ON bus_fund_label_value_mapping_portfolio_test COLUMNS(id,inner_code,label_code,label_value,report_period_id,created_by,create_time,updated_by,update_time,delete_flag,label_value_first,label_value_second,type ,__op=type) PROPERTIES ( "desired_concurrent_number" = "3", "format" = "json", "max_batch_interval" = "20", "max_batch_rows"= "250000", "max_error_number" = "1000" ) FROM KAFKA ( "kafka_broker_list" ="etl01:9092", "kafka_topic" = "jgy05", "property.kafka_default_offsets" ="OFFSET_BEGINNING" );CREATE ROUTINE LOAD dw_sr.bus_fund_tag_value ON bus_fund_tag_value_test COLUMNS(id,tag_id,inner_code,label_code,label_value,created_by,create_time,updated_by,update_time,delete_flag,type ,__op=type) PROPERTIES ( "desired_concurrent_number" = "3", "format" = "json", "max_batch_interval" = "20", "max_batch_rows"= "250000", "max_error_number" = "1000" ) FROM KAFKA ( "kafka_broker_list" ="etl01:9092", "kafka_topic" = "jgytag02", "property.kafka_default_offsets" ="OFFSET_BEGINNING" ); |
