RocketMQ部署及使用-Go （三）

本章介绍了rocketmq 5.0版本的一些概念和使用，keeping learning。

1. 概念先行

Topic

A topic is a top-level container that is used in Apache RocketMQ to transfer and store messages that belong to the same business logic.

Message Type

Apache RocketMQ support message type：

• NORMAL
• FIFO
• TRANSACTION
• DELAY

Starting from version 5.0, Apache RocketMQ supports enforcing the validation of message types, that is, each topic only allows messages of a single type to be sent. This can better facilitate operation and management of production systems and avoid confusion. However, to ensure backward compatibility with version 4.x, the validation feature is enable by default.

Message Queue

MessageQueue is a container that is used to store and transmit messages in Apache RocketMQ. MessageQueue is the smallest unit of storage for Apache RocketMQ messages.

Message

A message is the smallest unit of data transmission in Apache RocketMQ. A producer encapsulates the load and extended attributes of business data into messages and sends the messages to a Apache RocketMQ broker. Then, the broker delivers the messages to the consumer based on the relevant semantics.

Message View

MessageView is read-only interface to message from a development perspective. The message view allows you to read multiple properties and payload information inside a message, but you cannot make any changes to the message itself.

Message Tag

MessageTag is a fine-grained message classification property that allows message to be subdivided below the topic level. Consumers implement message filtering by subscribing to specific tags.

Message Offset

Messages are stored in the queue in order of precedence, each message has a unique coordinate of type Long in the queue, which is defined as the message site.

Consumer Offset

A message is not removed from the queue immediately after it has been consumed by a consumer, Apache RocketMQ will record the last consumed message based on each consumer group.

Message Key

MessageKey is The message-oriented index property. By setting the message index, you can quickly find the corresponding message content.

Producer

A producer in Apache RocketMQ is a functional messaging entity that creates messages and sends them to the server. A producer is typically integrated on the business system and serves to encapsulate data as messages in Apache RocketMQ and send the messages to the server.

Transaction Checker

Apache RocketMQ uses a transaction messaging mechanism that requires a producer to implement a transaction checker to ensure eventual consistency of transactions.

Consumer Group

A consumer group is a load balancing group that contains consumers that use the same consumption behaviors in Apache RocketMQ.

Consumer

A consumer is an entity that receives and processes messages in Apache RocketMQ. Consumers are usually integrated in business systems. They obtain messages from Apache RocketMQ brokers and convert the messages into information that can be perceived and processed by business logic.

Subscription

A subscription is the rule and status settings for consumers to obtain and process messages in Apache RocketMQ. Subscriptions are dynamically registered by consumer groups with brokers. Messages are then matched and consumed based on the filter rules defined by subscriptions.

2. 基础模块

• A producer generates a message and sends it to a Apache RocketMQ broker.
• The message is stored in a topic on the broker.
• A consumer subscribes to the topic to consume the message.

Message Production

The running entity that is used to generate messages in Apache RocketMQ. Producers are the upstream parts of business call links. Producers are lightweight, anonymous, and do not have identities.

Message Storage

Topic

The grouping container that is used for message transmission and storage in Apache RocketMQ. A topic consists of multiple message queues, which are used to store messages and scale out the topic.

MessageQueue

The unit container that is used for message transmission and storage in Apache RocketMQ. Message queues are similar to partitions in Kafka. Apache RocketMQ stores messages in a streaming manner based on an infinite queue structure. Messages are stored in order in a queue.

Message

The minimum unit of data transmission in Apache RocketMQ. Messages are immutable after they are initialized and stored.

Message consumption

ConsumerGroup

An independent group of consumption identities defined in the publish/subscribe model of Apache RocketMQ. A consumer group is used to centrally manage consumers that run at the bottom layer. Consumers in the same group must maintain the same consumption logic and configurations with each other, and consume the messages subscribed by the group together to scale out the consumption capacity of the group.

 Consumer

The running entity that is used to consume messages in Apache RocketMQ. Consumers are the downstream parts of business call links, A consumer must belong to a specific consumer group.

 Subscription

The collection of configurations in the publish/subscribe model of Apache RocketMQ. The configurations include message filtering, retry, and consumer progress. Subscriptions are managed at the consumer group level. You use consumer groups to specify subscriptions to manage how consumers in the group filter messages, retry consumption, and restore a consumer offset.

The configurations in a Apache RocketMQ subscription are all persistent, except for filter expressions. Subscriptions are unchanged regardless of whether the broker restarts or the connection is closed.

3. 领域模型 Domain Model

Commucation Model

According to the concept of distributed system architecture, a complex system can be split into multiple independent modules, such as microservice modules. Remote communication between the modules must be ensured in the system. There are two typical communication models for this purpose:

• RPC-based synchronous communication model;

• Middleware-based asynchronous communication model.

RPC-based synchronous model

In this model, remote systems communicate with each other directly. Each request is sent directly from the caller to the callee, and the callee returns the call result immediately to the caller. Notice The word "synchronous" does not refer to the mode of the programming interface. RPC also supports the programming mode of asynchronous non-blocking calls, in which case the caller still expects a direct response from the callee within a specified period.

Asynchronous communication model

In this model, subsystems are not connected in a tightly coupled manner. The caller needs only to convert a request into an asynchronous event, or message, and send it to the agent. As long as the message is sent, the call is considered complete. The agent delivers the message to the called downstream subsystem and ensures that the task is accomplished. The role of agent is typically assumed by a message middleware.

Asynchronous communication benefits

• Simple system topology. Because the caller and callee both communicate only with the agent, the system works in a star structure that is easy to maintain and manage.

• Weak upstream and downstream coupling. Weak coupling enables the system structure to be more flexible. The agent performs buffering and asynchronous recovery. Systems deployed at the upstream and downstream can be upgraded and changed independently without affecting each other.

• Load shifting. Message-oriented agents typically provide a large traffic buffer and powerful traffic shaping capability. This prevents traffic peaks from drowning downstream systems.

Message transmission model

Message middleware services have two common transmission models:

• the point-to-point model

• publish/subscribe model.

Point-to-point model

has the following characteristics:

• Consumer anonymity: The queue is the only identity used during upstream-downstream communication. Downstream consumers cannot declare an identity when they obtain messages from the queue.

• One-to-one communication: Consumers do not have identities. All consumers in a consumer group consume the subscribed messages together. Each message can be consumed only by one specific consumer. For this reason, this model supports only one-to-one communication.

Publish/Subscribe model

the following characteristics:

• Independent consumption: In this model, consumers use the identity of a consumer group, or a subscription, to receive and consume messages. Consumer groups are independent of each other.

• One-to-many communication: Based on the design of independent identity, this model allows a topic to be subscribed to by multiple consumer groups, each having full access to all the messages. Therefore, the publish/subscribe model supports one-to-many communication.

Comparison between transmission models

The point-to-point model is simpler in structure, while the publish/subscribe model offers better scalability. Apache RocketMQ uses and has the same high scalability as the publish/subscribe model.

Topic

A topic is logically a collection of queues; we may publish messages to or receive from it.

Message categorization and message isolation Identity and permission management: Messages in Apache RocketMQ are anonymous.

In Apache RocketMQ, a topic is a top-level storage container in which all message resources are defined. A topic is a logical concept and not the actual unit that stores messages.

A topic contains one or more queues. Message storage and scalability are implemented based on queues.All constraints and attribute settings for a topic are implemented based on the queues in the topic.

• Topic name: the name of a topic. A topic name identifies the topic and is globally unique in a cluster.

• Queues: the actual storage unit that stores messages.A topic must contain at least one queue.

• Message type : the message type that is specified for a topic.

Message Queue

A queue is a container that is used to store and transmit messages in Apache RocketMQ. A queue is the smallest unit of storage for Apache RocketMQ messages. A topic in Apache RocketMQ consists of multiple queues. This way, queues support horizontal partitioning and streaming storage.

Queues provide the following benefits: Ordered Storage Queues are ordered in nature: Messages are stored in the same order in which they are queued. The earliest message is at the start of the queue and the latest message is at the end of the queue. Offsets are used to label the locations and the order of messages in a queue. Streaming Operation Semantics: The queue-based storage in Apache RocketMQ allows consumers to read one or more messages from an offset. This helps implement features such as aggregate read and backtrack read. These features are not available in RabbitMQ or ActiveMQ.

Model relationship

In Apache RocketMQ, a queue is part of a topic. Messages are operated in queues even though they are managed by topic. For example, when a producer sends a message to a specific topic, the message is sent to a queue in the topic. You can change the number of queues in Apache RocketMQ to scale out or scale in.

Internal attributes

Read and write permissions

• Definition: whether data can be read from or written to the current queue. Values: defined by the broker. The following describes the enumerations: 6: read and write. Messages can be written to and read from the current queue. 4: read-only. Messages can be read from but not written to the current queue. 2: write-only. Messages can be written to but not read from the current queue. 0: The read or write status is unavailable. The current queue does not allow read or write operations.

• Constraint: The read and write permissions are related to O&M operations. We recommend that you do not frequently modify the permissions.

Version compatibility

Queue names vary based on the versions of Apache RocketMQ brokers. The following describes the differences: Broker versions 3.x and 4.x: A queue name consists of the topic name, broker ID, and queue ID, and is bound to physical nodes. Broker versions 5.x: A queue name is a globally unique string that is assigned by the cluster, and is decoupled from physical nodes. We recommend that you do not construct queue names or bind them to other operations. Otherwise, the queue names may fail to be resolved when the broker is updated.

Message

Definition

A message is the smallest unit of data transmission in Apache RocketMQ. A producer encapsulates the load and extended attributes of business data into messages and sends the messages to a Apache RocketMQ broker. Then, the broker delivers the messages to the consumer based on the relevant semantics. The characteristics of the message model in Apache RocketMQ are:

• Immutability: A message is an event that is generated. After the message is generated, the content of the message does not change. Even if the message passes through a transmission channel, the content of the message remains the same. The messages that consumers obtain are read-only messages.
• Persistence: By default, Apache RocketMQ persists messages. The received messages are stored in the storage file of the Apache RocketMQ broker to ensure that the messages can be traced and restored if system failures occur.

Model relationship

Messages are initialized by producers and are sent to the Apache RocketMQ broker. Messages are stored in queues in the order in which the messages are received on the Apache RocketMQ broker. Consumers obtain and consume messages from the Apache RocketMQ broker based on the specified subscriptions.

Internal attributes

System retention attributes

• Topic name

  Definition: the name of the topic to which a message belongs. The topic name is globally unique in a cluster.

  Values: obtained from the SDK of the client.

• Message type

  Definition: the type of a message. Values: obtained from the SDK of the client. Apache RocketMQ supports the following message types:

  • Normal: Normal messages. A normal message does not require special semantics and is not correlated with other normal messages.
  • FIFO: Fifo messages. Apache RocketMQ uses a message group to determine the order of a specified set of messages. The messages are delivered in the order in which they are sent.
  • Delay: Delayed messages. You can specify a delay to make messages available to consumers only after the delay has elapsed, instead of delivering messages immediately when they are produced.
  • Transaction: Transaction messages. Apache RocketMQ supports distributed transaction messages and ensures transaction consistency of database updates and message calls.

• Message queue

  Definition: the queue to which a message belongs.

  Values: specified and populated by the broker.

• Message offset

  Definition: the location where the current message is stored in the queue.

  Values: specified and populated by the broker. Valid values: 0 to Long.Max.

• Message ID

  Definition: the unique identifier of a message. The ID of each message is globally unique in the cluster.

  Values: automatically generated by the producer client. A message ID is a string of 32 characters that consists of digits and uppercase letters.

• (Optional) Message keys

  Definition: the list of index keys for messages. You can configure different keys to distinguish between messages and quickly find messages.

  Values: defined by the producer client.

• (Optional) Message tag

  Definition: the tag that is used to filter messages. Consumers can filter messages by tags and receive only messages that contain specified tags. Values: defined by the producer client. Constraint: Only one tag can be specified for each message.

• (Optional) Scheduled time

  Definition: the millisecond-level timestamp that is used when a message triggers delayed delivery in a scheduled time scenario.

  Values: defined by the message producer.

  Constraint: The maximum duration is 40 days.

• Message sending time

  Definition: the local millisecond-level timestamp of the producer client when the message is sent.

  Values: populated by the producer client.

  Note: The client time may be different from the broker time. In this case, the message sending time is based on the client time.

• Message store timestamp

  Definition: the local millisecond-level timestamp of the Apache RocketMQ broker when the message is stored. For delay messages and transaction messages, the message retention time is the broker time that is displayed for the consumer when the message takes effect.

  Values: populated by the broker.

  Note: The client time may be different from the broker time. In this case, the message retention time is based on the broker time.

• Retry times

  Definition: the number of times that the Apache RocketMQ broker redelivers a message after the message fails to be consumed. After each retry, the maximum number of retries is increased by one.

  Values: labeled by the broker. The first time that a message is consumed, the number of retries is zero. The first time that a message fails to be consumed, the number of retries is one.

• Custom attributes for messages

  Custom attributes

  Definition: the extended information that can be specified by the producer. Values: specified by the producer based on key-value pairs from a string. Message load

  Definition: the actual message data of the service message.

  Values: serialized by the producer and transmitted in binary bytes. Constraints: see Parameter limits.

Behavior constraints

The size of a message cannot exceed the upper limit. If the size of a message exceeds the corresponding upper limit, the message fails to be sent. The following describes the default limits for messages:

max size of message: 4 MB

Usage notes

• Overloaded transmission is not recommended for a single message.

Apache RocketMQ is a messaging middleware that transmits data for business events. If the size of a message is large, the network transmission layer may be overloaded. This affects retries upon errors and throttling. We recommend that you limit the data size of a single message event. If an overloaded transmission is required in the production environment, we recommend that you split the message based on a fixed size or use the file storage method.

• Immutability of messages

Messages cannot be modified in Apache RocketMQ broker versions 5.x and the messages that consumers obtain are read-only messages. No strong constraints related to immutability are imposed on versions 3.x and 4.x. We recommend that you re-initialize messages if you want to transmit messages.

• Correct example:

    Message m = Consumer.receive();
    Message m2= MessageBuilder.buildFrom(m);
    Producer.send(m2);

• Incorrect example：

    Message m = Consumer.receive();
    m.update()；
    Producer.send(m);

Producer

Definition

A producer in Apache RocketMQ is a functional messaging entity that creates messages and sends them to the server.

A producer is typically integrated on the business system and serves to encapsulate data as messages in Apache RocketMQ and send the messages to the server. The following message delivery elements are defined on the producer side: - Transmission mode: A producer can specify the message transmission mode in an API operation. Apache RocketMQ supports synchronous transmission and asynchronous transmission. - Batch transmission: A producer can specify batch transmission in an API operation. For example, the number or size of messages sent at a time can be specified. - Transactional behavior: Apache RocketMQ supports transaction messages. Producers are involved in transaction checks to ensure eventual consistency of transactions.

Producers and topics have an n-to-n relationship. A producer can send messages to multiple topics, and a topic can receive messages from multiple producers. This many-to-many relationship facilitates performance scaling and disaster recovery.

Model relationship

The message is initialized by the producer and sent to the Apache RocketMQ server. Messages are stored in the specified queue of the topic in the order in which they arrive at the Apache RocketMQ server. The consumer obtains and consumes messages from the Apache RocketMQ server based on the specified subscription relationship.

Internal attributes

• Client ID

  Definition: the identity of a producer client. This attribute is used to distinguish between different producers. A client ID is globally unique within a cluster.

  Value: The client ID is automatically generated by Apache RocketMQ SDKs. It is mainly used for O&M purposes such as log viewing and problem locating. The client ID cannot be modified.

• Communication parameters

  (Required) : the endpoint used to connect to the serverv. This endpoint is used to identify the cluster. The access point must be configured in the format. We recommend that you use domain names to avoid using IP addresses to prevent node changes from failing to perform hotspot migration.

  (Optional) : the credential used by the client for authentication. Transmission is required only when identity recognition and authentication are enabled on the server.

  Request Timeout (Optional) : the timeout period of the network request.

• Prebound topic list

  Definition: the list of topics to which a producer of Apache RocketMQ sends messages. Prebound topics provide the following benefits:

  Transaction messages (Required): The prebound topic list attribute must be specified for transaction messages. In transaction messaging scenarios, when a producer recovers from a fault or is restarted, the producer checks whether a transaction message topic contains uncommitted transaction messages. This prevents latency caused by uncommitted transaction messages in the topic after the producer sends new messages to the topic.

  Non-transaction messages (Optional): The server checks the access permissions and validity of the destination topics based on the list of prebound topics during producer initialization, instead of performing the check after the application is started. We recommend that you specify the prebound topic list attribute for non-transaction messages. If the prebound topic list attribute is not specified for non-transaction messages or destination topics are changed, Apache RocketMQ dynamically checks and identifies destination topics.

  Limit: For transaction messages, prebound topics must be specified and used together with a transaction checker.

• Transaction checker

  Apache RocketMQ uses a transaction messaging mechanism that requires a producer to implement a transaction checker to ensure eventual consistency of transactions. When a producer sends transaction messages, a transaction checker must be configured and used together with prebound topics.

• Send retry policy

  Send retry policy specifies how a producer retries the delivery of messages upon a failed message delivery attempt.

Version compatibility

Starting from Apache RocketMQ version 5.x, producers are anonymous, and producer groups are discontinued. For Apache RocketMQ version 3.x and version 4.x, existing producer groups can be discontinued, without affecting your business.

Usage notes

• We recommend that you limit the number of producers on individual processes.

In Apache RocketMQ, producers and topics provide a many-to-many form of communication. A single producer can send messages to multiple topics. We recommend that you create and initialize the minimum number of producers that your business scenarios require, and reuse as many producers as you can. For example, in a scenario that requires message delivery to multiple topics, you do not need to create a producer for each topic.

• We recommend that you do not create and destroy producers on a regular basis.

The producers of Apache RocketMQ are underlying resources that can be reused, like the connection pool of a database. You do not need to create producers each time you send messages or destroy the producers after you send messages. If you regularly create and destroy producers, a large number of short connection requests are generated on the broker. This imposes a high level of load on your system.

• Example of correct usage

  Producer p = ProducerBuilder.build();
  for (int i =0;i<n;i++)
  {
    Message m= MessageBuilder.build();
    p.send(m);
  }
  p.shutdown();

Consumer Group

Definition

A consumer group is a load balancing group that contains consumers that use the same consumption behaviors in Apache RocketMQ.

Unlike consumers that are running entities, consumer groups are logical resources. Apache RocketMQ initializes multiple consumers in a consumer group to achieve the scaling of consumption performance and high availability disaster recovery.

In a consumer group, consumers consume messages based on the consumption behaviors and load balancing policy that are defined in the group. The following section describes the consumption behaviors that are defined:

• Subscription: Apache RocketMQ manages and traces subscriptions based on consumer groups.

• Delivery order: The Apache RocketMQ broker delivers messages to consumers by using ordered delivery or concurrent delivery. You can configure the delivery method in the consumer group.

• Consumption retry policy: the retry policy that is used when a consumer fails to consume a message. The policy includes the number of retries and the setting of dead-letter queues.

Model relationship

The message is initialized by the producer and sent to the Apache RocketMQ server. Messages are stored in the specified queue of the topic in the order in which they arrive at the Apache RocketMQ server. The consumer obtains and consumes messages from the Apache RocketMQ server based on the specified subscription relationship.

Internal attributes

• Consumer group name

  Definition: the name of a consumer group. Consumer group names are used to distinguish between consumer groups. Consumer group names are globally unique in a cluster.

  Values: created and configured by users.

• Delivery order

  Definition: the order in which Apache RocketMQ delivers messages to a consumer client. Apache RocketMQ supports ordered delivery and concurrent delivery based on different consumption scenarios.

  Values: The default delivery method is concurrent delivery.

• Consumption retry policy

  Definition: the retry policy that is used when a consumer fails to consume a message. If a consumer fails to consume a message, the system re-delivers the failed message to the consumer for re-consumption based on the policy. For more information, see Consumption retry.

  Values:A consumption retry policy contains the following items:

  • Maximum retries: the maximum number of times that a message can be re-delivered. If a message fails to be consumed and the maximum number of retries is exceeded, the message is delivered to the dead-letter queue or is discarded.

  • Retry interval: the interval between which the Apache RocketMQ broker re-delivers a failed message.

  • Constraint: Retry interval is available only for push consumers.

• Subscription

  Definition: the set of subscription relationships that are associated with the current consumer group. A subscription includes the topics to which the consumers subscribe and the message filter rules that are used by consumers. Consumers dynamically register subscriptions for consumer groups. The Apache RocketMQ broker persists subscriptions and matches the subscriptions to the consumption progress of messages.

Behavior constraints

In the Apache RocketMQ domain model, consumer management is implemented through consumer grouping, and consumers in the same group share messages for consumption. Therefore, to ensure the normal load and consumption of messages in a group, Apache RocketMQ require all consumers in the same group to keep the following consumption behaviors consistent:

• Delivery Order
• Consumption retry policy

Consumer

Definition

A consumer is an entity that receives and processes messages in Apache RocketMQ. Consumers are usually integrated in business systems. They obtain messages from Apache RocketMQ brokers and convert the messages into information that can be perceived and processed by business logic.

The following items determine consumer behavior:

Consumer identity: A consumer must be associated with a consumer group to obtain behavior settings and consumption status.

Consumer type: Apache RocketMQ provides a variety of consumer types for different development scenarios, including push consumers, simple consumers and pull consumers.

Local settings for consumers: These settings specify how consumer clients run based on the consumer type. For example, you can configure the number of threads and concurrency settings on consumers to achieve different transmission effects.

Model relationship

The message is initialized by the producer and sent to the Apache RocketMQ server. Messages are stored in the specified queue of the topic in the order in which they arrive at the Apache RocketMQ server. The consumer obtains and consumes messages from the Apache RocketMQ server based on the specified subscription relationship.

Internal attributes

• Consumer group name

  Definition: the name of the consumer group associated with the current consumer. Consumers inherit their behavior from the consumer groups.

  Values: Consumer groups are the logical resources of Apache RocketMQ{#product-name}，You must create consumer groups by using the console or calling API operations in advance.

• Client ID

  Definition: the identity of a consumer client. This attribute is used to distinguish between different consumers. The value must be unique within a cluster.

  Values: The client ID is automatically generated by the Apache RocketMQ SDK. It is mainly used for O&M purposes such as log viewing and problem locating. The client ID cannot be modified.

• Communication parameters

  Endpoints (Required) : the endpoint used to connect to the server. This endpoint is used to identify the cluster. The access point must be configured in the format. We recommend that you use domain names to avoid using IP addresses to prevent node changes from failing to perform hotspot migration.

  Credential (Optional) : the credential used by the client for authentication. Transmission is required only when identity recognition and authentication are enabled on the server.

  Request Timeout (Optional) : the timeout period of the network request.

• Pre-bound subscription list

  Definition: the subscription list of the specified consumer. The Apache RocketMQ broker can use the pre-bound subscription list to verify the permissions and validity of the subscribed topic during consumer initialization instead of after the application is started.

  Values: We recommend that you specify the subscription or the list of subscribed topics during consumer initialization. If the subscription is not specified or the subscribed topics are changed, Apache RocketMQ dynamically verifies the topics.

• Message listener

  Definition: the listener that a consumer uses to invoke the message consumption logic after Apache RocketMQ broker pushes a message to the consumer.

  Values: The value of a message listener is configured on the consumer client.

  Constraints: When you consume messages as a push consumer, you must configure the message listener on the consumer client.

Behavior constraints

In the Apache RocketMQ domain model, consumer management is implemented through consumer grouping, and consumers in the same group share messages for consumption. Therefore, to ensure the normal load and consumption of messages in a group, Apache RocketMQ require all consumers in the same group to keep the following consumption behaviors consistent:

Delivery Order

Consumption retry policy

Usage notes

• We recommend that you limit the number of consumers on individual processes. The consumers of Apache RocketMQ support the non-blocking transmission mode at the communication protocol level. The non-blocking transmission mode has higher communication efficiency and supports concurrent access by multiple threads. Therefore, in most scenarios, only one consumer needs to be initialized for a consumer group in a single process. Avoid initializing multiple consumers with the same configurations during the development phase.

• We recommend that you do not create and destroy consumers on a regular basis. The consumers of Apache RocketMQ are underlying resources that can be reused, like the connection pool of a database. You do not need to create consumers each time you receive messages or destroy the consumers after you consume messages. If you regularly create and destroy consumers, a large number of short connection requests are generated on the broker. This imposes a high level of load on your system.

• Correct example

Consumer c = ConsumerBuilder.build();
for (int i =0;i<n;i++)
{
  Message m= c.receive();
  //process message
}
c.shutdown();

Subscription

Definition

A subscription is the rule and status settings for consumers to obtain and process messages in Apache RocketMQ.

Subscriptions are dynamically registered by consumer groups with brokers. Messages are then matched and consumed based on the filter rules defined by subscriptions.

By configuring subscriptions, you can control the following messaging behaviors:

• Message filter rules: These rules are used to define which messages in a topic are consumed by a consumer. By configuring message filter rules, consumers can effectively obtain messages that they want and specify message receiving ranges based on different business scenarios.
• Consumption status: By default, the Apache RocketMQ broker provides persistent subscriptions. In other words, after a consumer group subscribes to a broker, consumers in the group can continue consuming messages from where the consumers left off after they reconnect.

Rules for determining a subscription

The subscriptions of Apache RocketMQ are designed based on consumer groups and topics. Therefore, a subscription refers to the subscription of a specified consumer group to a topic. The following describes the rules for determining a subscription:

One topic to many subscribersThe following figure shows two consumer groups (Group A and Group B) subscribed to Topic A. These two subscriptions are independent of each other and can be defined separately.

One subscriber to multiple topicsThe following figure shows a consumer group (Group A) subscribed to two topics: Topic A and Topic B. Consumers in Group A have two separate subscriptions to Topic A and Topic B. The two subscriptions are independent of each other and can be defined separately.

Model relationship

The message is initialized by the producer and sent to the Apache RocketMQ server. Messages are stored in the specified queue of the topic in the order in which they arrive at the Apache RocketMQ server. The consumer obtains and consumes messages from the Apache RocketMQ server based on the specified subscription relationship.

Internal attributes

• Filter types

  Definition: the type of a message filter rule. After a message filter rule is set for a subscription, the system matches the messages in a topic based on the filter rule. Only the messages that meet the conditions are delivered to consumers. This feature helps you classify messages sent to consumers based on your requirements.

  Values: Tag filter: filters and matches the full text based on tag strings. SQL92 filter: filters and matches message attributes based on SQL syntax.

• Filter expressions

  Definition: the expression of a custom filter rule. Values: For more information, see Syntax for filter expressions.

Behavior constraints

• Subscription consistency Apache RocketMQ manages subscriptions based on consumer groups. Therefore, consumers in the same consumer group must maintain the same consumption logic. Otherwise, consumption conflicts occur, which in turn causes some messages to be incorrectly consumed.

• Correct example

//Consumer c1
Consumer c1 = ConsumerBuilder.build(groupA);
c1.subscribe(topicA,"TagA");
//Consumer c2
Consumer c2 = ConsumerBuilder.build(groupA);
c2.subscribe(topicA,"TagA");

Incorrect example
//Consumer c1
Consumer c1 = ConsumerBuilder.build(groupA);
c1.subscribe(topicA,"TagA");
//Consumer c2Consumer 
c2 = ConsumerBuilder.build(groupA);
c2.subscribe(topicA,"TagB");