binlog梳理-1

253 阅读3分钟

BingLogEvent

每个LogEvent分为Header、Post-Header、Body、EventFooter四部分, Header和Footer每个event都相同格式,但PostHeader和Body,不同EventType不一样。

  @anchor Binary_log_event_format
  Any @c Binary_log_event saved on disk consists of the following four
  components.

  - Common-Header
  - Post-Header
  - Body
  - Footer

  Common header has the same format and length in a given MySQL version. It is
  documented @ref Table_common_header "here".

  The Body may be of different format and length even for different events of
  the same type. The binary formats of Post-Header and Body are documented
  separately in each subclass.

  Footer is common to all the events in a given MySQL version. It is documented
  @ref Table_common_footer "here".

EventHeader

EventHeader为固定长度19字节,主要记录了event的创建时间、类型、大小、在日志中的偏移量等信息. image.png 各字段详细说明参考: Log_event_header

EventType

/**
  Enumeration type for the different types of log events.
*/
enum Log_event_type {
  /**
    Every time you add a type, you have to
    - Assign it a number explicitly. Otherwise it will cause trouble
      if a event type before is deprecated and removed directly from
      the enum.
    - Fix Format_description_event::Format_description_event().
  */
  UNKNOWN_EVENT = 0,
  /*
    Deprecated since mysql 8.0.2. It is just a placeholder,
    should not be used anywhere else.
  */
  START_EVENT_V3 = 1,
  QUERY_EVENT = 2,
  STOP_EVENT = 3,
  ROTATE_EVENT = 4,
  INTVAR_EVENT = 5,

  SLAVE_EVENT = 7,

  APPEND_BLOCK_EVENT = 9,
  DELETE_FILE_EVENT = 11,

  RAND_EVENT = 13,
  USER_VAR_EVENT = 14,
  FORMAT_DESCRIPTION_EVENT = 15,
  XID_EVENT = 16,
  BEGIN_LOAD_QUERY_EVENT = 17,
  EXECUTE_LOAD_QUERY_EVENT = 18,

  TABLE_MAP_EVENT = 19,

  /**
    The V1 event numbers are used from 5.1.16 until mysql-5.6.
  */
  WRITE_ROWS_EVENT_V1 = 23,
  UPDATE_ROWS_EVENT_V1 = 24,
  DELETE_ROWS_EVENT_V1 = 25,

  /**
    Something out of the ordinary happened on the master
   */
  INCIDENT_EVENT = 26,

  /**
    Heartbeat event to be send by master at its idle time
    to ensure master's online status to slave
  */
  HEARTBEAT_LOG_EVENT = 27,

  /**
    In some situations, it is necessary to send over ignorable
    data to the slave: data that a slave can handle in case there
    is code for handling it, but which can be ignored if it is not
    recognized.
  */
  IGNORABLE_LOG_EVENT = 28,
  ROWS_QUERY_LOG_EVENT = 29,

  /** Version 2 of the Row events */
  WRITE_ROWS_EVENT = 30,
  UPDATE_ROWS_EVENT = 31,
  DELETE_ROWS_EVENT = 32,

  GTID_LOG_EVENT = 33,
  ANONYMOUS_GTID_LOG_EVENT = 34,

  PREVIOUS_GTIDS_LOG_EVENT = 35,

  TRANSACTION_CONTEXT_EVENT = 36,

  VIEW_CHANGE_EVENT = 37,

  /* Prepared XA transaction terminal event similar to Xid */
  XA_PREPARE_LOG_EVENT = 38,

  /**
    Extension of UPDATE_ROWS_EVENT, allowing partial values according
    to binlog_row_value_options.
  */
  PARTIAL_UPDATE_ROWS_EVENT = 39,

  TRANSACTION_PAYLOAD_EVENT = 40,

  HEARTBEAT_LOG_EVENT_V2 = 41,
  /**
    Add new events here - right above this comment!
    Existing events (except ENUM_END_EVENT) should never change their numbers
  */
  ENUM_END_EVENT /* end marker */
};

目前总共的事件类型有41种,部分是老版本使用的。最常见的有TABLE_MAP_EVENT、QUERY_EVENT、WRITE_ROWS_EVENT、UPDATE_ROWS_EVENT和DELETE_ROWS_EVENT等。

EventFooter

EventFooter主要是描述checksum算法,校验算法主要是CRC32 image.png

/**
  Enumeration spcifying checksum algorithm used to encode a binary log event
*/
enum enum_binlog_checksum_alg {
  /**
    Events are without checksum though its generator is checksum-capable
    New Master (NM).
  */
  BINLOG_CHECKSUM_ALG_OFF = 0,
  /** CRC32 of zlib algorithm */
  BINLOG_CHECKSUM_ALG_CRC32 = 1,
  /** the cut line: valid alg range is [1, 0x7f] */
  BINLOG_CHECKSUM_ALG_ENUM_END,
  /**
    Special value to tag undetermined yet checksum or events from
    checksum-unaware servers
  */
  BINLOG_CHECKSUM_ALG_UNDEF = 255
};

Events

TableMapEvent

mysql对一行数据做修改时,会产生两个Event: TableMapEvent和RowsEvent(Write_rows_event/Update_rows_event/Delete_rows_event)。TABLE_MAP_EVENT描述了表的结构信息。

In row-based mode, every row operation event is preceded by a Table_map_event which maps a table definition to a number.

The table definition consists of database name, table name, and column definitions.

Post-Header部分包括两部分: table_id和flags,flags目前属于保留位。table_id是递增的,每次修改表结构都会产生Table_map_event,并分配新的table_id。 image.png

Body部分包括了库名、表名、列数、列字段类型以及每列的详细信息。其中metadata根据列的类型不同而不同,有些类型没有的。 image.png

RowsEvent

RowsEvent是row模式下,insert/update/delete事件的基类,其Post-Header主要是包含table_id信息。 image.png 而body部分,则根据操作不同,决定是否包含更新前后的数据columns_before_image/columns_after_image。

image.png

## MySQL Binlog 源码入门