全网最简洁的TinyWeb它来了(无任何第三方依赖)

631 阅读9分钟

大家好,我是春哥,一名拥有10多年Linux后端研发经验的BAT互联网老兵。

在上一篇文章《万字长文!带你吃透Reactor并发模型》发表之后,我收到了很多小伙伴的良好反馈。有小伙伴私信春哥说这个echo协议太简单了,能不能支持http协议。春哥的答复是马上给安排!于是,春哥继续爆肝,为大家编写了一个名为TinyWeb的项目,供大家学习使用。TinyWeb是一个支持http协议的Web服务器,可以处理http请求并返回响应。如果您对Web开发或者网络编程感兴趣,不妨来学习一下TinyWeb,相信会对你有所帮助。

如果之前没有看过《万字长文!带你吃透Reactor并发模型》文章的小伙伴,这里是传送门:

《万字长文!带你吃透Reactor并发模型》

1.问题分解

我们来对TinyWeb项目进行问题分解,我们要实现TinyWeb需要解决以下2个主要问题:

  • 实现一个高性能的并发框架。
  • 实现http协议的解析。

1.1 并发框架

我们使用epoll来实现Reactor-MS的并发框架。整体的架构如图1-1所示。

图1-1 Reactor-MS并发框架

Reactor并发模型相关的细节,这里就不展开,可以去看春哥之前的文章《万字长文!带你吃透Reactor并发模型》

1.2 http协议解析

http协议是纯文本协议,http协议格式如图1-2所示。

图1-2 http协议格式

由于http协议本身就不复杂,因此我们完全可以自己实现http协议的解析。

2.TinyWeb实现

讲完了如何解决两个主要的问题,现在我们来看一下如何实现这个TinyWeb项目。

2.1 目录结构

我们先来看一下TinyWeb的目录结构。


    TinyWeb
    ├── cmdline.cpp
    ├── cmdline.h
    ├── conn.hpp
    ├── epollctl.hpp
    ├── handler.hpp
    ├── httpcodec.hpp
    ├── httpmessage.hpp
    ├── makefile
    ├── mp_account.png
    ├── packet.hpp
    ├── README.md
    ├── tinyweb.cpp
    └── WebBench
        ├── makefile
        └── webbench.cpp
  • cmdline.h和cmdline.cpp实现了命令行的解析,之前在另一篇文章有介绍过了,大家感兴趣的可以阅读下面这篇文章:
  • conn.hpp实现了连接的管理。
  • epollctl.hpp实现了epoll事件的管理。
  • handler.hpp实现了http业务处理注册。
  • httpcodec.hpp实现了http协议的解析。
  • httpmessage.hpp实现了对http请求和应答的封装。
  • packet.hpp是二进制缓冲区的封装。
  • tinyweb.cpp是TinyWeb的入口程序。
  • WebBench则是压测工具的代码目录。

2.2 详细介绍

这一小节,我们对相关代码做一下详细介绍。

2.2.1 连接管理

下面的代码实现了TCP连接的管理。


    #pragma once

    #include "httpcodec.hpp"

    class Conn {
     public:
      Conn(int fd, int epoll_fd) : fd_(fd), epoll_fd_(epoll_fd) {}
      bool Read() {
        do {
          ssize_t ret = read(fd_, codec_.Data(), codec_.Len());  // 一次最多读取4K
          if (ret == 0) {
            perror("peer close connection");
            return false;
          }
          if (ret < 0) {
            if (EINTR == errno) continue;
            if (EAGAIN == errno or EWOULDBLOCK == errno) return true;
            perror("read failed");
            return false;
          }
          codec_.Decode(ret);
        } while (true);
      }
      bool Write() {
        do {
          if (send_len_ == (ssize_t)send_pkt_.UseLen()) return true;
          ssize_t ret = write(fd_, send_pkt_.DataRaw() + send_len_, send_pkt_.UseLen() - send_len_);
          if (ret < 0) {
            if (EINTR == errno) continue;
            if (EAGAIN == errno && EWOULDBLOCK == errno) return true;
            perror("write failed");
            return false;
          }
          send_len_ += ret;
        } while (true);
      }
      HttpMessage* GetReq() { return codec_.GetMessage(); }
      void SetResp(HttpMessage* resp) { codec_.Encode(resp, send_pkt_); }
      bool FinishWrite() { return send_len_ == (ssize_t)send_pkt_.UseLen(); }
      int Fd() { return fd_; }
      int EpollFd() { return epoll_fd_; }

     private:
      int fd_{0};            // 关联的客户端连接fd
      int epoll_fd_{0};      // 关联的epoll实例的fd
      ssize_t send_len_{0};  // 要发送的应答数据的长度
      Packet send_pkt_;      // 发送应答数据的二进制缓冲区
      HttpMessage* req_;     // http请求消息
      HttpMessage* resp_;    // http应答消息
      HttpCodec codec_;      // http协议的编解码
    };

2.2.2 事件管理

我们这里的事件管理特指的是epoll事件管理,下面的代码展示了相关的实现。


    #pragma once

    #include "conn.hpp"

    inline void AddReadEvent(Conn *conn) {
      epoll_event event;
      event.data.ptr = (void *)conn;
      event.events = EPOLLIN;
      assert(epoll_ctl(conn->EpollFd(), EPOLL_CTL_ADD, conn->Fd(), &event) != -1);
    }

    inline void ModToWriteEvent(Conn *conn, bool isET = false) {
      epoll_event event;
      event.data.ptr = (void *)conn;
      event.events = EPOLLOUT;
      if (isET) event.events |= EPOLLET;
      assert(epoll_ctl(conn->EpollFd(), EPOLL_CTL_MOD, conn->Fd(), &event) != -1);
    }

    inline void ClearEvent(Conn *conn, bool isClose = true) {
      assert(epoll_ctl(conn->EpollFd(), EPOLL_CTL_DEL, conn->Fd(), NULL) != -1);
      if (isClose) close(conn->Fd());  // close操作需要EPOLL_CTL_DEL之后调用,否则调用epoll_ctl()删除fd会失败
    }

2.2.3 http业务处理注册

http业务处理我们设计成主动注册的方式来实现,处理函数和http请求方法和url做绑定,下面的代码实现了http业务处理的注册。


    #pragma once

    #include <functional>
    #include <map>
    #include <string>

    #include "httpmessage.hpp"

    enum Method {
      kGet = 1,   // get 方法
      kPost = 2,  // post 方法
    };

    class Handler {
     public:
      void Register(Method method, std::string url, std::function<void(HttpMessage* req, HttpMessage* resp)> handler) {
        if (method == kGet) {
          get_handlers_[url] = handler;
        } else if (method == kPost) {
          post_handlers_[url] = handler;
        }
      }

      void Deal(HttpMessage* req, HttpMessage* resp) {
        std::string method;
        std::string url;
        req->GetMethodAndUrl(method, url);
        if (method == "GET") {
          if (get_handlers_.find(url) == get_handlers_.end()) {
            resp->SetStatusCode(NOT_FOUND);
          } else {
            get_handlers_[url](req, resp);
          }
        } else if (method == "POST") {
          if (post_handlers_.find(url) == post_handlers_.end()) {
            resp->SetStatusCode(NOT_FOUND);
          } else {
            post_handlers_[url](req, resp);
          }
        } else {
          resp->SetStatusCode(NOT_FOUND);
        }
      }

     private:
      std::map<std::string, std::function<void(HttpMessage* req, HttpMessage* resp)>> get_handlers_;
      std::map<std::string, std::function<void(HttpMessage* req, HttpMessage* resp)>> post_handlers_;
    };

2.2.4 http消息封装

我们通HttpMessage类实现了对http请求和应答消息进行封装,下面的代码实现了http消息封装。


    #pragma once

    #include <map>
    #include <string>

    // 目前只支持4个状态码
    enum HttpStatusCode {
      OK = 200,                     //请求成功
      BAD_REQUEST = 400,            //错误的请求,目前body只支持json格式,非json格式返回这个错误码
      NOT_FOUND = 404,              //请求失败,未找到相关资源
      INTERNAL_SERVER_ERROR = 500,  //内部服务错误
    };

    // http消息
    typedef struct HttpMessage {
      void SetHeader(const std::string &key, const std::string &value) { headers_[key] = value; }
      void SetBody(const std::string &body) {
        body_ = body;
        SetHeader("Content-Type", "application/json");
        SetHeader("Content-Length", std::to_string(body_.length()));
      }
      void SetStatusCode(HttpStatusCode statusCode) {
        if (OK == statusCode) {
          first_line_ = "HTTP/1.1 200 OK";
        } else if (BAD_REQUEST == statusCode) {
          first_line_ = "HTTP/1.1 400 Bad Request";
        } else if (NOT_FOUND == statusCode) {
          first_line_ = "HTTP/1.1 404 Not Found";
        } else {
          first_line_ = "HTTP/1.1 500 Internal Server Error";
        }
      }
      int32_t GetStatusCode() {
        std::string status_code;
        bool start = false;
        for (size_t i = 0; i < first_line_.size(); i++) {
          if (first_line_[i] == ' ') {
            if (start) {
              break;
            } else {
              start = true;
              continue;
            }
          }
          if (start) {
            status_code += first_line_[i];
          }
        }
        return std::atoi(status_code.c_str());
      }
      std::string GetHeader(const std::string &key) {
        if (headers_.find(key) == headers_.end()) return "";
        return headers_[key];
      }
      void SetMethodAndUrl(const std::string &method, const std::string &url) {
        first_line_ = method + " " + url + " HTTP/1.1";
      }
      void GetMethodAndUrl(std::string &method, std::string &url) {
        int32_t spaceCount = 0;
        for (size_t i = 0; i < first_line_.size(); i++) {
          if (first_line_[i] == ' ') {
            spaceCount++;
            continue;
          }
          if (spaceCount == 0) method += first_line_[i];
          if (spaceCount == 1) url += first_line_[i];
        }
      }
      void ParserBody() {
        std::string key;
        std::string value;
        bool is_key = true;
        auto add_param = [this, &is_key](std::string &key, std::string &value) {
          if (key != "" && value != "") {
            params_[key] = value;
          }
          key = "";
          value = "";
          is_key = true;
        };
        for (size_t i = 0; i < body_.size(); i++) {
          if (body_[i] == '&') {
            add_param(key, value);
          } else if (body_[i] == '=') {
            is_key = false;
          } else {
            if (is_key) {
              key += body_[i];
            } else {
              value += body_[i];
            }
          }
        }
        add_param(key, value);
      }
      void GetParam(std::string key, int64_t &value, int64_t default_value) {
        if (params_.find(key) == params_.end()) {
          value = default_value;
        } else {
          value = std::atoll(params_[key].c_str());
        }
      }

      std::string first_line_;  // 对于请求来说是request_line,对于应答来说是status_line
      std::map<std::string, std::string> headers_;
      std::string body_;
      std::map<std::string, std::string> params_;  // 参数集合
    } HttpMessage;

2.2.5 http消息编解码

有了http消息的封装,接下来我们来实现一个http消息的编解码,我们采用状态机+流式解析的方式来实现http消息的解析,至于编码就非常简单。下面的代码实现了http消息编解码。


    #pragma once

    #include <string.h>

    #include <string>
    #include <unordered_map>
    #include <vector>

    #include "httpmessage.hpp"
    #include "packet.hpp"

    constexpr uint32_t FIRST_READ_LEN = 256;           //优先读取数据的大小
    constexpr uint32_t MAX_FIRST_LINE_LEN = 8 * 1024;  // http第一行的最大长度
    constexpr uint32_t MAX_HEADER_LEN = 8 * 1024;      // header最大长度
    constexpr uint32_t MAX_BODY_LEN = 1024 * 1024;     // body最大长度
    // 解码状态
    enum HttpDecodeStatus {
      FIRST_LINE = 1,  // 第一行
      HEADERS = 2,     // 消息头
      BODY = 3,        // 消息体
      FINISH = 4,      // 完成了消息解析
    };

    // 协议编解码
    class HttpCodec {
     public:
      HttpCodec() { pkt_.Alloc(FIRST_READ_LEN); }
      ~HttpCodec() {
        if (message_) delete message_;
      }
      uint8_t *Data() { return pkt_.Data(); }
      size_t Len() { return pkt_.Len(); }
      HttpMessage *GetMessage() {
        if (nullptr == message_ || decode_status_ != FINISH) return nullptr;
        HttpMessage *result = message_;
        message_ = nullptr;
        decode_status_ = FIRST_LINE;  //消息被取出之后解析状态设置为FIRST_LINE
        return result;
      }
      void SetLimit(uint32_t maxFirstLineLen, uint32_t maxHeaderLen, uint32_t maxBodyLen) {
        max_first_line_len_ = maxFirstLineLen;
        max_header_len_ = maxHeaderLen;
        max_body_len_ = maxBodyLen;
      }
      bool Encode(HttpMessage *message, Packet &pkt) {
        std::string data;
        data.append(message->first_line_ + "\r\n");
        auto iter = message->headers_.begin();
        while (iter != message->headers_.end()) {
          data.append(iter->first + ": " + iter->second + "\r\n");
          iter++;
        }
        data.append("\r\n");
        data.append(message->body_);
        pkt.Alloc(data.length());
        memmove(pkt.Data(), data.c_str(), data.length());
        pkt.UpdateUseLen(data.length());
        return true;
      }
      bool Decode(size_t len) {
        pkt_.UpdateUseLen(len);
        uint32_t decodeLen = 0;
        uint32_t needDecodeLen = pkt_.NeedParseLen();
        uint8_t *data = pkt_.DataParse();
        if (nullptr == message_) message_ = new HttpMessage;
        while (needDecodeLen > 0) {  //只要还有未解析的网络字节流,就持续解析
          bool decodeBreak = false;
          if (FIRST_LINE == decode_status_) {  //解析第一行
            if (not decodeFirstLine(&data, needDecodeLen, decodeLen, decodeBreak)) {
              return false;
            }
            if (decodeBreak) break;
          }
          if (needDecodeLen > 0 && HEADERS == decode_status_) {  //解析完第一行,解析headers
            if (not decodeHeaders(&data, needDecodeLen, decodeLen, decodeBreak)) {
              return false;
            }
            if (decodeBreak) break;
          }
          if (needDecodeLen > 0 && BODY == decode_status_) {  //解析完headers,解析body
            if (not decodeBody(&data, needDecodeLen, decodeLen, decodeBreak)) {
              return false;
            }
            if (decodeBreak) break;
          }
        }
        if (decodeLen > 0) pkt_.UpdateParseLen(decodeLen);
        if (FINISH == decode_status_) {
          pkt_.Alloc(FIRST_READ_LEN);  // 解析完一个消息及时释放空间,并申请新的空间
        }
        return true;
      }

     private:
      void ltrim(std::string &str) {
        if (str.empty()) return;
        str.erase(0, str.find_first_not_of(" "));
      }
      void rtrim(std::string &str) {
        if (str.empty()) return;
        str.erase(str.find_last_not_of(" ") + 1);
      }
      void trim(std::string &str) {
        ltrim(str);
        rtrim(str);
      }
      bool decodeFirstLine(uint8_t **data, uint32_t &needDecodeLen, uint32_t &decodeLen, bool &decodeBreak) {
        uint8_t *temp = *data;
        bool completeFirstLine = false;
        uint32_t firstLineLen = 0;
        for (uint32_t i = 0; i < needDecodeLen - 1; i++) {
          if (temp[i] == '\r' && temp[i + 1] == '\n') {
            completeFirstLine = true;
            firstLineLen = i + 2;
            break;
          }
        }
        if (not completeFirstLine) {
          if (needDecodeLen > max_first_line_len_) {
            return false;
          }
          pkt_.ReAlloc(pkt_.UseLen() * 2);  // 无法完成第一行的解析,则尝试扩大下次读取的数据量
          decodeBreak = true;
          return true;
        }
        if (firstLineLen > max_first_line_len_) {
          return false;
        }
        message_->first_line_ = std::string((char *)temp, firstLineLen - 2);
        //更新剩余待解析数据长度,已经解析的长度,缓冲区指针的位置,当前解析的状态。
        needDecodeLen -= firstLineLen;
        decodeLen += firstLineLen;
        (*data) += firstLineLen;
        decode_status_ = HEADERS;
        return true;
      }
      bool decodeHeaders(uint8_t **data, uint32_t &needDecodeLen, uint32_t &decodeLen, bool &decodeBreak) {
        uint8_t *temp = *data;
        if (needDecodeLen >= 2 && temp[0] == '\r' && temp[1] == '\n') {  //解析到空行
          needDecodeLen -= 2;
          decodeLen += 2;
          (*data) += 2;
          decode_status_ = BODY;
          return true;
        }
        bool isKey = true;
        uint32_t decodeHeadersLen = 0;
        bool getOneHeader = false;
        // 解析每个header的key,value对
        std::string key, value;
        for (uint32_t i = 0; i < needDecodeLen - 1; i++) {
          if (temp[i] == '\r' && temp[i + 1] == '\n') {  // 一个完整的key,value对
            trim(key);
            trim(value);
            if (key != "" && value != "") message_->headers_[key] = value;
            getOneHeader = true;
            decodeHeadersLen = i + 2;
            break;
          }
          if (isKey && temp[i] == ':') {  // 第一个':'才是分隔符
            isKey = false;
            continue;
          }
          if (isKey)
            key += temp[i];
          else
            value += temp[i];
        }
        if (not getOneHeader) {
          if (needDecodeLen > max_header_len_) {
            return false;
          }
          decodeBreak = true;
          pkt_.ReAlloc(pkt_.UseLen() * 2);  // 无法完成headers的解析,则尝试扩大下次读取的数据量
          return true;
        }
        if (decodeHeadersLen > max_header_len_) {
          return false;
        }
        needDecodeLen -= decodeHeadersLen;
        decodeLen += decodeHeadersLen;
        (*data) += decodeHeadersLen;
        return true;
      }
      bool decodeBody(uint8_t **data, uint32_t &needDecodeLen, uint32_t &decodeLen, bool &decodeBreak) {
        auto iter = message_->headers_.find("Content-Length");
        if (iter == message_->headers_.end()) {  // 只支持通过Content-Length来标识body的长度
          return false;
        }
        uint32_t bodyLen = (uint32_t)std::stoi(iter->second.c_str());
        if (bodyLen > max_body_len_) {
          return false;
        }
        decodeBreak = true;  // 不管是否能完成解析都跳出循环
        if (needDecodeLen < bodyLen) {
          pkt_.ReAlloc(pkt_.UseLen() + (bodyLen - needDecodeLen));  // 无法完成解析,则尝试扩大下次读取的数据量
          return true;
        }
        uint8_t *temp = *data;
        message_->body_ = std::string((char *)temp, bodyLen);
        message_->ParserBody();
        //更新剩余待解析数据长度,已经解析的长度,缓冲区指针的位置,当前解析的状态。
        needDecodeLen -= bodyLen;
        decodeLen += bodyLen;
        (*data) += bodyLen;
        decode_status_ = FINISH;  //解析状态流转,更新为完成消息解析
        return true;
      }

     private:
      HttpDecodeStatus decode_status_{FIRST_LINE};  // 当前解析状态
      HttpMessage *message_{nullptr};
      uint32_t max_first_line_len_{MAX_FIRST_LINE_LEN};
      uint32_t max_header_len_{MAX_HEADER_LEN};
      uint32_t max_body_len_{MAX_BODY_LEN};
      Packet pkt_;
    };

2.2.6 二进制缓冲区

网络通讯都需要做io buffer的处理,所以二进制缓冲区的封装也是必不可少。下面的代码实现了二进制缓冲区。


    #pragma once

    // 二进制包
    class Packet {
     public:
      ~Packet() {
        if (data_) free(data_);
      }
      void Alloc(size_t len) {
        if (data_) free(data_);
        data_ = (uint8_t *)malloc(len);
        len_ = len;
        use_len_ = 0;
        parse_len_ = 0;
      }
      void ReAlloc(size_t len) {
        if (len < len_) {
          return;
        }
        data_ = (uint8_t *)realloc(data_, len);
        len_ = len;
      }
      void CopyFrom(const Packet &pkt) {
        if (data_) free(data_);
        data_ = (uint8_t *)malloc(pkt.len_);
        memmove(data_, pkt.data_, pkt.len_);
        len_ = pkt.len_;
        use_len_ = pkt.use_len_;
        parse_len_ = pkt.parse_len_;
      }
      uint8_t *Data() { return data_ + use_len_; }             //缓冲区可以写入的开始地址
      uint8_t *DataRaw() { return data_; }                     //原始缓冲区的开始地址
      uint8_t *DataParse() { return data_ + parse_len_; }      //需要解析的开始地址
      size_t NeedParseLen() { return use_len_ - parse_len_; }  //还需要解析的长度
      size_t Len() { return len_ - use_len_; }                 //缓存区中还可以写入的数据长度
      size_t UseLen() { return use_len_; }                     //缓冲区已经使用的容量
      void UpdateUseLen(size_t add_len) { use_len_ += add_len; }
      void UpdateParseLen(size_t add_len) { parse_len_ += add_len; }

     public:
      uint8_t *data_{nullptr};  // 二进制缓冲区
      size_t len_{0};           // 缓冲区的长度
      size_t use_len_{0};       // 缓冲区使用长度
      size_t parse_len_{0};     // 完成解析的长度
    };

2.2.7 TinyWeb启动入口

最后给大家介绍的是服务启动入口程序,下面的代码实现了TinyWeb启动入口。


    #include <arpa/inet.h>
    #include <assert.h>
    #include <fcntl.h>
    #include <netinet/in.h>
    #include <stdio.h>
    #include <stdlib.h>
    #include <sys/epoll.h>
    #include <sys/socket.h>
    #include <sys/sysinfo.h>
    #include <unistd.h>

    #include <condition_variable>
    #include <functional>
    #include <iostream>
    #include <mutex>
    #include <thread>

    #include "cmdline.h"
    #include "epollctl.hpp"
    #include "handler.hpp"

    using namespace std;

    int *EpollFd;
    int EpollInitCnt = 0;
    std::mutex Mutex;
    std::condition_variable Cond;
    Handler handler;

    // 获取系统有多少个可用的cpu
    int GetNProcs() { return get_nprocs(); }

    void SetNotBlock(int fd) {
      int oldOpt = fcntl(fd, F_GETFL);
      assert(oldOpt != -1);
      assert(fcntl(fd, F_SETFL, oldOpt | O_NONBLOCK) != -1);
    }

    int CreateListenSocket(const char *ip, int port, bool isReusePort) {
      sockaddr_in addr;
      addr.sin_family = AF_INET;
      addr.sin_port = htons(port);
      addr.sin_addr.s_addr = inet_addr(ip);
      int sockFd = socket(AF_INET, SOCK_STREAM, 0);
      if (sockFd < 0) {
        perror("socket failed");
        return -1;
      }
      int reuse = 1;
      if (setsockopt(sockFd, SOL_SOCKET, SO_REUSEPORT, &reuse, sizeof(reuse)) != 0) {
        perror("setsockopt failed");
        return -1;
      }
      if (bind(sockFd, (sockaddr *)&addr, sizeof(addr)) != 0) {
        perror("bind failed");
        return -1;
      }
      if (listen(sockFd, 1024) != 0) {
        perror("listen failed");
        return -1;
      }
      return sockFd;
    }

    // 调用本函数之前需要把sockFd设置成非阻塞的
    void LoopAccept(int sockFd, int maxConn, std::function<void(int clientFd)> clientAcceptCallBack) {
      while (maxConn--) {
        int clientFd = accept(sockFd, NULL, 0);
        if (clientFd > 0) {
          clientAcceptCallBack(clientFd);
          continue;
        }
        if (errno != EAGAIN && errno != EWOULDBLOCK && errno != EINTR) {
          perror("accept failed");
        }
        break;
      }
    }

    void waitSubReactor() {
      std::unique_lock<std::mutex> locker(Mutex);
      Cond.wait(locker, []() -> bool { return EpollInitCnt >= GetNProcs(); });
      return;
    }

    void subReactorNotifyReady() {
      {
        std::unique_lock<std::mutex> locker(Mutex);
        EpollInitCnt++;
      }
      Cond.notify_all();
    }

    void addToSubHandler(int &index, int clientFd) {
      index++;
      index %= GetNProcs();
      Conn *conn = new Conn(clientFd, EpollFd[index]);  // 轮询的方式添加到子Reactor线程中
      AddReadEvent(conn);                               // 监听可读事件
    }

    void mainHandler(const char *ip, int port) {
      waitSubReactor();  // 等待所有的从Reactor线程都启动完毕
      int sockFd = CreateListenSocket(ip, port, true);
      if (sockFd < 0) {
        return;
      }
      epoll_event events[2048];
      int epollFd = epoll_create(1024);
      if (epollFd < 0) {
        perror("epoll_create failed");
        return;
      }
      int index = 0;
      Conn conn(sockFd, epollFd);
      SetNotBlock(sockFd);
      AddReadEvent(&conn);
      while (true) {
        int num = epoll_wait(epollFd, events, 2048, -1);
        if (num < 0) {
          perror("epoll_wait failed");
          continue;
        }
        // 执行到这里就是有客户端连接到来
        LoopAccept(sockFd, 100000, [&index, epollFd](int clientFd) {
          SetNotBlock(clientFd);
          addToSubHandler(index, clientFd);  // 把连接迁移到subHandler线程中管理
        });
      }
    }

    void addHandler(HttpMessage *req, HttpMessage *resp) {
      int64_t a = 0;
      int64_t b = 0;
      req->GetParam("a", a, 0);
      req->GetParam("b", b, 0);
      int64_t sum = a + b;
      resp->SetBody(std::to_string(sum));
      resp->SetStatusCode(OK);
    }

    HttpMessage *deal(HttpMessage *req) {
      HttpMessage *resp = new HttpMessage;
      handler.Deal(req, resp);
      return resp;
    }

    void subHandler(int threadId) {
      epoll_event events[2048];
      int epollFd = epoll_create(1024);
      if (epollFd < 0) {
        perror("epoll_create failed");
        return;
      }
      EpollFd[threadId] = epollFd;
      subReactorNotifyReady();
      while (true) {
        int num = epoll_wait(epollFd, events, 2048, -1);
        if (num < 0) {
          perror("epoll_wait failed");
          continue;
        }
        for (int i = 0; i < num; i++) {
          Conn *conn = (Conn *)events[i].data.ptr;
          auto releaseConn = [&conn]() {
            ClearEvent(conn);
            delete conn;
          };
          if (events[i].events & EPOLLIN) {  // 可读
            if (not conn->Read()) {          // 执行非阻塞读
              releaseConn();
              continue;
            }
            HttpMessage *req = conn->GetReq();
            if (req) {
              HttpMessage *resp = deal(req);
              conn->SetResp(resp);
              ModToWriteEvent(conn);  // 修改成只监控可写事件
              delete req;
              delete resp;
            }
          }
          if (events[i].events & EPOLLOUT) {  // 可写
            if (not conn->Write()) {          // 执行非阻塞写
              releaseConn();
              continue;
            }
            if (conn->FinishWrite()) {  // 完成了请求的应答写,则可以释放连接
              releaseConn();
            }
          }
        }
      }
    }

    void usage() {
      cout << "./TinyWeb -ip 0.0.0.0 -port 8088" << endl;
      cout << "options:" << endl;
      cout << "    -h,--help      print usage" << endl;
      cout << "    -ip,--ip       listen ip" << endl;
      cout << "    -port,--port   listen port" << endl;
      cout << endl;
    }

    int main(int argc, char *argv[]) {
      string ip;
      int64_t port;
      CmdLine::StrOptRequired(&ip, "ip");
      CmdLine::Int64OptRequired(&port, "port");
      CmdLine::SetUsage(usage);
      CmdLine::Parse(argc, argv);
      EpollFd = new int[GetNProcs()];
      handler.Register(kPost, "/add", addHandler);
      for (int i = 0; i < GetNProcs(); i++) {
        std::thread(subHandler, i).detach();  // 这里需要调用detach,让创建的线程独立运行
      }
      for (int i = 0; i < GetNProcs() - 1; i++) {
        std::thread(mainHandler, ip.c_str(), port).detach();  // 这里需要调用detach,让创建的线程独立运行
      }
      mainHandler(ip.c_str(), port);  // 主线程也陷入死循环中,监听客户端请求
      return 0;
    }

2.2.8 WebBench

编写完TinyWeb,我们需要使用性能压测工具评估服务的基准性能指标,为此春哥编写了一个多线程的压测工具WebBench。下面的代码实现了WebBench。


    #include <arpa/inet.h>
    #include <sys/socket.h>
    #include <sys/time.h>
    #include <sys/types.h>
    #include <unistd.h>

    #include <iostream>
    #include <mutex>
    #include <string>
    #include <thread>

    #include "../cmdline.h"
    #include "../httpcodec.hpp"
    #include "../httpmessage.hpp"
    #include "../packet.hpp"

    using namespace std;

    typedef struct Stat {
      int sum{0};
      int success{0};
      int failure{0};
      int spendms{0};
    } Stat;

    std::mutex Mutex;
    Stat FinalStat;

    int64_t port;
    int64_t concurrency;
    int64_t runtime;
    std::string method;
    std::string url;
    std::string body;

    bool getConnection(sockaddr_in &addr, int &sockFd) {
      sockFd = socket(AF_INET, SOCK_STREAM, 0);
      if (sockFd < 0) {
        perror("socket failed");
        return false;
      }
      int ret = connect(sockFd, (sockaddr *)&addr, sizeof(addr));
      if (ret < 0) {
        perror("connect failed");
        close(sockFd);
        return false;
      }
      struct linger lin;
      lin.l_onoff = 1;
      lin.l_linger = 0;
      // 设置调用close关闭tcp连接时,直接发送RST包,tcp连接直接复位,进入到closed状态。
      if (0 == setsockopt(sockFd, SOL_SOCKET, SO_LINGER, &lin, sizeof(lin))) {
        return true;
      }
      perror("setsockopt failed");
      close(sockFd);
      return false;
    }

    // 用于阻塞IO模式下发送应答消息
    bool SendReq(int fd, HttpMessage &req) {
      Packet pkt;
      HttpCodec codec;
      codec.Encode(&req, pkt);
      ssize_t sendLen = 0;
      while ((size_t)sendLen != pkt.UseLen()) {
        ssize_t ret = write(fd, pkt.DataRaw() + sendLen, pkt.UseLen() - sendLen);
        if (ret < 0) {
          if (errno == EINTR) continue;  // 中断的情况可以重试
          perror("write failed");
          return false;
        }
        sendLen += ret;
      }
      return true;
    }

    // 用于阻塞IO模式下接收请求消息
    bool RecvResp(int fd, HttpMessage &resp) {
      HttpCodec codec;
      HttpMessage *temp;
      while (true) {  // 只要还没获取到一个完整的消息,则一直循环
        temp = codec.GetMessage();
        if (temp) break;
        ssize_t ret = read(fd, codec.Data(), codec.Len());
        if (ret <= 0) {
          if (errno == EINTR) continue;  // 中断的情况可以重试
          perror("read failed");
          return false;
        }
        codec.Decode(ret);
      }
      resp = *temp;
      delete temp;
      return true;
    }

    int64_t getSpendMs(timeval begin, timeval end) {
      end.tv_sec -= begin.tv_sec;
      end.tv_usec -= begin.tv_usec;
      if (end.tv_usec <= 0) {
        end.tv_sec -= 1;
        end.tv_usec += 1000000;
      }
      return end.tv_sec * 1000 + end.tv_usec / 1000;  //计算运行的时间,单位ms
    }

    void client(int theadId, Stat *curStat, int port, int concurrency) {
      int sum = 0;
      int success = 0;
      int failure = 0;
      int spendms = 0;
      sockaddr_in addr;
      addr.sin_family = AF_INET;
      addr.sin_port = htons(port);
      addr.sin_addr.s_addr = inet_addr(std::string("127.0.0." + std::to_string(theadId + 1)).c_str());
      HttpMessage req;
      req.SetMethodAndUrl(method, url);
      req.SetBody(body);
      concurrency /= 10;  // 每个线程的并发数
      int *sockFd = new int[concurrency];
      timeval end;
      timeval begin;
      gettimeofday(&begin, NULL);
      for (int i = 0; i < concurrency; i++) {
        if (not getConnection(addr, sockFd[i])) {
          sockFd[i] = 0;
          failure++;
        }
      }
      auto failureDeal = [&sockFd, &failure](int i) {
        close(sockFd[i]);
        sockFd[i] = 0;
        failure++;
      };
      for (int i = 0; i < concurrency; i++) {
        if (sockFd[i]) {
          if (not SendReq(sockFd[i], req)) {
            failureDeal(i);
          }
        }
      }
      for (int i = 0; i < concurrency; i++) {
        if (sockFd[i]) {
          HttpMessage resp;
          if (RecvResp(sockFd[i], resp)) {
            failureDeal(i);
            continue;
          }
          if (resp.GetStatusCode() != OK) {
            failureDeal(i);
            continue;
          }
          close(sockFd[i]);
          success++;
        }
      }
      delete[] sockFd;
      sum = success + failure;
      gettimeofday(&end, NULL);
      spendms = getSpendMs(begin, end);
      std::lock_guard<std::mutex> guard(Mutex);
      curStat->sum += sum;
      curStat->success += success;
      curStat->failure += failure;
      curStat->spendms += spendms;
    }

    void UpdateFinalStat(Stat stat) {
      FinalStat.sum += stat.sum;
      FinalStat.success += stat.success;
      FinalStat.failure += stat.failure;
      FinalStat.spendms += stat.spendms;
    }

    void usage() {
      cout << "./WebBench -port 8088 -concurrency 10000 -runtime 60 -method POST -url /add -body 'a=10&b=90'" << endl;
      cout << "options:" << endl;
      cout << "    -h,--help                    print usage" << endl;
      cout << "    -port,--port                 listen port" << endl;
      cout << "    -concurrency,--concurrency   concurrency" << endl;
      cout << "    -runtime,--runtime           run time, unit is second" << endl;
      cout << "    -method,--method             http request's method" << endl;
      cout << "    -url,--url                   http request's url" << endl;
      cout << "    -body,--body                 http request's body" << endl;
      cout << endl;
    }

    int main(int argc, char *argv[]) {
      CmdLine::Int64OptRequired(&port, "port");
      CmdLine::Int64OptRequired(&concurrency, "concurrency");
      CmdLine::Int64OptRequired(&runtime, "runtime");
      CmdLine::StrOptRequired(&method, "method");
      CmdLine::StrOptRequired(&url, "url");
      CmdLine::StrOptRequired(&body, "body");
      CmdLine::SetUsage(usage);
      CmdLine::Parse(argc, argv);

      timeval end;
      timeval runBeginTime;
      gettimeofday(&runBeginTime, NULL);
      int runRoundCount = 0;
      while (true) {
        Stat curStat;
        std::thread threads[10];
        for (int threadId = 0; threadId < 10; threadId++) {
          threads[threadId] = std::thread(client, threadId, &curStat, port, concurrency);
        }
        for (int threadId = 0; threadId < 10; threadId++) {
          threads[threadId].join();
        }
        runRoundCount++;
        curStat.spendms /= 10;  // 取平均耗时
        UpdateFinalStat(curStat);
        gettimeofday(&end, NULL);
        std::cout << "round " << runRoundCount << " spend " << curStat.spendms << " ms. " << std::endl;
        if (getSpendMs(runBeginTime, end) >= runtime * 1000) {
          break;
        }
        sleep(2);  // 间隔2秒,再发起下一轮压测,这样压测结果更稳定
      }
      std::cout << "total spend " << FinalStat.spendms << " ms. avg spend " << FinalStat.spendms / runRoundCount
                << " ms. sum[" << FinalStat.sum << "],success[" << FinalStat.success << "],failure[" << FinalStat.failure
                << "]" << std::endl;
      return 0;
    }

我们的TinyWeb运行在32G16核的CentOS上,使用WeBench发起10万的并发,并连续压测30秒,压测结果如图2-1所示。

图2-1 压测结果

3.完整项目

春哥已将这部分代码开源并托管在GitHub上,项目页面包含了详细的使用说明和编译脚本。传送门: TinyWeb

欢迎大家积极参与,进行fork、star以及提出issue。如果因为网络问题导致无法下载,可以评论加关注,我会直接将项目完整源码发送给你。

为了方便大家阅读,「春哥特意把本文导出成pdf文件,有需要的小伙伴关注并私信春哥,留下邮箱或者其他联系方式,春哥直接将pdf发送给你」

4.写在最后

今天分享的内容就到这里,『如果通过本文你有新的认知和收获,记得关注我,下期分享不迷路,我将持续在掘金上分享技术干货』。

硬核爆肝码字,跪求一赞!!!