rocketmq 快速入门

400 阅读5分钟

快速安装

环境要求

  • 64bit OS, Linux/Unix/Mac is recommended;
  • 64bit JDK 1.8+;
  • Maven 3.2.x;
  • 4g+ free disk for Broker server

快速安装

  • 下载http://rocketmq.apache.org/release_notes/release-notes-4.7.0/
  • 解压安装
    1. unzip rocketmq-all-4.7.0-source-release.zip

    2. cd rocketmq-all-4.7.0/

    3. mvn -Prelease-all -DskipTests clean install -U

    4. cd distribution/target/rocketmq-4.7.0/rocketmq-4.7.0

  • 启动 Name Server
    1. nohup sh bin/mqnamesrv &

    2. tail -f ~/logs/rocketmqlogs/namesrv.log

       看到The Name Server boot success...在运行broker
      
  • 启动 Broker
    1. nohup sh bin/mqbroker -n localhost:9876 &

    2. tail -f ~/logs/rocketmqlogs/broker.log

       boot success. serializeType=JSON and name server is localhost:9876
      
  • 关闭
    1. sh bin/mqshutdown broker

    2. sh bin/mqshutdown namesrv

  • 如果启动失败怎么办

启动失败他一般会把日志放在nohup.out 中,只需要把按照提示信息进行修改就行了

名词介绍

RocketMQ的功能和现实生活中的邮局收发信件很类似,我们类比地说一下相应的模块。现实生活中的邮政系统要正常运行,离不开下面这四个角色,一是发信者,二是收信者,三是负责暂存、传输的邮局,四是负责协调各个地方邮局的管理机构。对应到RocketMQ中,这四个角色就是Producer、Consumer、Broker和NameServer。

  • name server(协调各个地方邮局的管理机构)
  • Broker(暂存、传输的邮局)
  • Producer(发信者)
  • Consumer(收信者)
  • Topic(表示一类消息的集合,每个主题包含若干条消息,每条消息只能属于一个主题,是RocketMQ进行消息订阅的基本单位。)
  • Producer Group(同一类Producer的集合,这类Producer发送同一类消息且发送逻辑一致。如果发送的是事务消息且原始生产者在发送之后崩溃,则Broker服务器会联系同一生产者组的其他生产者实例以提交或回溯消费。)
  • Consumer Group(同一类Consumer的集合,这类Consumer通常消费同一类消息且消费逻辑一致。消费者组使得在消息消费方面,实现负载均衡和容错的目标变得非常容易。要注意的是,消费者组的消费者实例必须订阅完全相同的Topic。)
  • Message Queue(用于存储消息的物理地址,每个Topic中的消息地址存储于多个 Message Queue 中)
  • 消息走向

demo运行

demo运行的时候只需要quickstart一个项目就行了。整合其他的可能会遇到意向不到错误

  • mvn 添加依赖
    <dependency>
        <groupId>org.apache.rocketmq</groupId>
        <artifactId>rocketmq-client</artifactId>
        <version>4.3.0</version>
    </dependency>

生产者请把超时时间设置的大一点!!!

producer.setSendMsgTimeout(6000);

  • 异步传输(需要Broker返回确认信息) 这个demo需要注意的producer.shutdown();请把这个注释起来,否则会报no Topic
    public static void main(String[] args) throws Exception {
        //Instantiate with a producer group name.
        DefaultMQProducer producer = new DefaultMQProducer("please_rename_unique_group_name");
        // Specify name server addresses.
        producer.setNamesrvAddr("localhost:9876");
        //Launch the instance.
        producer.start();
        producer.setRetryTimesWhenSendAsyncFailed(0);
        producer.setSendMsgTimeout(6000);
        for (int i = 0; i < 100; i++) {
            final int index = i;
            //Create a message instance, specifying topic, tag and message body.
            Message msg = new Message("TopicTest",
                    "TagA",
                    "OrderID188",
                    "Hello world".getBytes(RemotingHelper.DEFAULT_CHARSET));

            producer.send(msg, new SendCallback() {
                @Override
                public void onSuccess(SendResult sendResult) {
                    System.out.printf("%-10d OK %s %n", index,
                            sendResult.getMsgId());
                }
                @Override
                public void onException(Throwable e) {
                    System.out.printf("%-10d Exception %s %n", index, e);
                    e.printStackTrace();
                }
            });
        }
        //Shut down once the producer instance is not longer in use.
        //producer.shutdown();
    }
  • 同步传输(需要Broker返回确认信息)

这里需要把消息catch一下,然后把超时时间设置的大一点,否则第一个消息发送失败,发送会被打断

public static void main(String[] args) throws MQClientException, UnsupportedEncodingException, RemotingException, InterruptedException, MQBrokerException {
        //Instantiate with a producer group name.
        DefaultMQProducer producer = new
                DefaultMQProducer("please_rename_unique_group_name");
        // Specify name server addresses.
        producer.setNamesrvAddr("localhost:9876");
        producer.setSendMsgTimeout(60000);
        //Launch the instance.
        producer.start();
        for (int i = 0; i < 100; i++) {
            try {
                //Create a message instance, specifying topic, tag and message body.
                Message msg = new Message("TopicTest" /* Topic */,
                        "TagA" /* Tag */,
                        "OrderID188",
                        ("Hello RocketMQ " +
                                i).getBytes(RemotingHelper.DEFAULT_CHARSET) /* Message body */
                );
                //Call send message to deliver message to one of brokers.
                SendResult sendResult = producer.send(msg);
                System.out.printf("%s%n", sendResult);
            }catch (Exception e) {
                e.printStackTrace();
            }

        }
        //Shut down once the producer instance is not longer in use.
        producer.shutdown();
    }
  • 单向传输(消息发送之后就没了)
public static void main(String[] args) throws Exception{
        //Instantiate with a producer group name.
        DefaultMQProducer producer = new DefaultMQProducer("please_rename_unique_group_name");
        // Specify name server addresses.
        producer.setNamesrvAddr("localhost:9876");
        //Launch the instance.
        producer.setSendMsgTimeout(6000);
        producer.start();
        for (int i = 0; i < 100; i++) {
            //Create a message instance, specifying topic, tag and message body.
            Message msg = new Message("TopicTest" /* Topic */,
                    "TagA" /* Tag */,
                    ("Hello RocketMQ " +
                            i).getBytes(RemotingHelper.DEFAULT_CHARSET) /* Message body */
            );
            //Call send message to deliver message to one of brokers.
            producer.sendOneway(msg);

        }
        //Shut down once the producer instance is not longer in use.
        producer.shutdown();
}
  • 消费者接收消息
public static void main(String[] args) throws InterruptedException, MQClientException {
        // Instantiate with specified consumer group name.
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("please_rename_unique_group_name");

        // Specify name server addresses.
        consumer.setNamesrvAddr("localhost:9876");

        // Subscribe one more more topics to consume.
        consumer.subscribe("TopicTest", "*");
        // Register callback to execute on arrival of messages fetched from brokers.
        consumer.registerMessageListener(new MessageListenerConcurrently() {

            @Override
            public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs,
                                                            ConsumeConcurrentlyContext context) {
                System.out.printf("%s Receive New Messages: %s %n", Thread.currentThread().getName(), msgs);
                return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
            }
        });

        //Launch the consumer instance.
        consumer.start();

        System.out.printf("Consumer Started.%n");
    }

No route info of this topic, TopicTest

  • 确保生产者可以连接到nameserver,并能够从中获取路由元信息。
  • 确保nameserver确实包含该topic的路由元信息。您可以使用管理工具或Web控制台通过topicRoute从名称服务器查询路由元信息。
  • 确保您的代理将心跳发送到生产者连接到的相同name servers列表。
  • 确保主题的权限为6(rw-)或至少2(-w-)。
  • 如果找不到此topic,请通过管理工具命令updateTopic或Web控制台在代理上创建它。
  • 还有可能发送消息超时

整合spring boot

  • 添加依赖
<!-- https://mvnrepository.com/artifact/org.apache.rocketmq/rocketmq-spring-boot-starter -->
    <dependency>
        <groupId>org.apache.rocketmq</groupId>
        <artifactId>rocketmq-spring-boot-starter</artifactId>
        <version>2.1.0</version>
    </dependency>
  • 写配置
rocketmq:
  name-server: localhost:9876
  producer:
    group: test-prodct-group #这个是必须要加的,否则项目启动不起来
    send-message-timeout: 6000
  • 发送消息
    @Autowired
    private RocketMQTemplate rocketMQTemplate;

    @GetMapping("/send")
    public String send() {
        rocketMQTemplate.convertAndSend("TopicTest","--- send body chunlai ---");
        return "success";
    }
  • 接收消息
@Service
@RocketMQMessageListener(consumerGroup = "test-consumer-group",topic = "TopicTest")
public class StringListener implements RocketMQListener<String> {//这里的String是生产者发送的消息类型

    @Override
    public void onMessage(String s) {
        System.out.println("rocketMq body = " + s);
    }
}

producer是如何启动的

DefaultMQProducer 没有太多的逻辑,它只是封住了其他实现类的调用,主要的实现类就是DefaultMQProducerImpl

  • 进入producer#start();
  • 调用DefaultMQProducerImpl#start();
  • 使用变量来标记serviceState:只有开始CREATE_JUST才会运行下面的状态
public void start(boolean startFactory) throws MQClientException {
        switch(this.serviceState) {
        case CREATE_JUST:
            this.serviceState = ServiceState.START_FAILED;
            //... 参数检查
            
            //获取MQClientInstance实例,没有则自动创建
            this.mQClientFactory.registerProducer(this.defaultMQProducer.getProducerGroup(), this);
            if (!registerOK) {
                // 异常
            } else {
                // 启动mQClientFactory 
                this.topicPublishInfoTable.put(this.defaultMQProducer.getCreateTopicKey(), new TopicPublishInfo());
                if (startFactory) {
                    this.mQClientFactory.start();
                }

                this.log.info("the producer [{}] start OK. sendMessageWithVIPChannel={}", this.defaultMQProducer.getProducerGroup(), this.defaultMQProducer.isSendMessageWithVIPChannel());
                this.serviceState = ServiceState.RUNNING;
            }
        default:
            // 给所有 broker 发送心跳
            this.mQClientFactory.sendHeartbeatToAllBrokerWithLock();
            return;
        case RUNNING:
        case START_FAILED:
        case SHUTDOWN_ALREADY:
            throw new MQClientException("The producer service state not OK, maybe started once, " + this.serviceState + FAQUrl.suggestTodo("http://rocketmq.apache.org/docs/faq/"), (Throwable)null);
        }
}
  • this.mQClientFactory.start()
public void start() throws MQClientException {
        synchronized(this) {
            switch(this.serviceState) {
            case CREATE_JUST:
                //启动请求响应通道
                this.mQClientAPIImpl.start();
                // 各中定时
                this.startScheduledTask();
                // 启动拉消息服务
                this.pullMessageService.start();
                // 启动 rebalanceService
                this.rebalanceService.start();
                //启动Producer
                this.defaultMQProducer.getDefaultMQProducerImpl().start(false);
            case RUNNING:
            case SHUTDOWN_ALREADY:
            default:
                return;
            case START_FAILED:
                throw new MQClientException("The Factory object[" + this.getClientId() + "] has been created before, and failed.", (Throwable)null);
        }
    }
}