Redis1.0源码阅读笔记一、总体流程

828 阅读5分钟

Redis介绍

Redis 由C语言实现,是完全开源免费的,遵守BSD协议,是一个高性能的key-value数据库。Redis-1.0是第一个稳定版,并且代码量只有8k多行。虽然代码量不大,但实现的功能并不少:

  • 支持数据持久化
  • 支持master/slave 功能
  • Value支持String、List、Set数据结构

因为代码量少,结构清晰,功能丰富等特点,Redis-1.0被很多网友推荐,适合新手阅读学习。

Redis各版本的源码下载地址如下:download.redis.io/releases/

下载redis-1.0.tar.gz后,用命令tar -zxvf redis-1.0.tar.gz解压缩。在解压的文件夹中,有一个doc文件夹,里面介绍了redis的安装、每个命令的使用等。建议首先阅读README.html,按照里面的步骤,编译安装运行,并用telnet连接redis服务器,使用SETGET等指令,对Redis有一个初步了解。

Redis1.0的总体流程

main函数

阅读Redis1.0时,可以从main函数开始,在文件redis.c中。

int main(int argc, char **argv) {
    initServerConfig();
    if (argc == 2) {
        ResetServerSaveParams();
        loadServerConfig(argv[1]);
    } else if (argc > 2) {
        fprintf(stderr,"Usage: ./redis-server [/path/to/redis.conf]\n");
        exit(1);
    } else {
        redisLog(REDIS_WARNING,"Warning: no config file specified, using the default config. In order to specify a config file use 'redis-server /path/to/redis.conf'");
    }
    initServer(); 
    if (server.daemonize) daemonize();
    redisLog(REDIS_NOTICE,"Server started, Redis version " REDIS_VERSION);
#ifdef __linux__
    linuxOvercommitMemoryWarning();
#endif
    if (rdbLoad(server.dbfilename) == REDIS_OK)
        redisLog(REDIS_NOTICE,"DB loaded from disk");
    if (aeCreateFileEvent(server.el, server.fd, AE_READABLE,
        acceptHandler, NULL, NULL) == AE_ERR) oom("creating file event");
    redisLog(REDIS_NOTICE,"The server is now ready to accept connections on port %d", server.port);
    aeMain(server.el);
    aeDeleteEventLoop(server.el);
    return 0;
}

首先由函数initServerConfig用默认值初始化服务器配置,如DB数量16个、端口号6379等。当然,redis支持在启动时指定配置文件。具体可配置项见文件redis.conf。 若指定配置文件,则启动脚本修改为./redis-server ./redis.conf。指定配置文件时,由函数loadServerconfig负责读取文件,并设置信息,如端口号等。函数initServer建立socket监听,初始化时间驱动的事件。函数rdbLoad读取db数据到内存。接着,调用aeCreatFileEvent函数,向全局变量serverel字段添加socket的aceeptHandler。最后,调用aeMain函数,进行while (1)loop 循环。

在继续阅读每个函数的实现前,可以先看一下Redis中一个重要的全局变量:server, 其类型是一个名为redisServer的结构体。port保存了Redis监听的端口号, fd保存Redis打开的socket,在initServer中被赋值。clients保存连接的客户端,是一个双向链表。el保存了各类事件,包括了文件事件(aeFileEvent)和时间事件(aeTimeEvent),两类事件都是用单向链表维护。结构体redisServer还包括了其他字段,不在这里一一展开介绍,将在后面介绍到具体功能时再做说明。redisServer截取定义如下:

/* Global server state structure */
struct redisServer {
    int port;
    int fd;
    list *clients;
    aeEventLoop *el;
};

initServer

redis配置设置完成后,调用initServer初始化服务器。initServer函数负责的初始化工作,包括创建客户端连接的链表、slaves链表、监听端口、初始化时间驱动事件等。server.el = aeCreateEventLoop() 创建事件管理结构,server.fd = anetTcpServer(server.neterr, server.port, server.bindaddr) 建立socket监听。 aeCreateTimeEvent(server.el, 1000, serverCron, NULL, NULL) 创建时间事件链表。

static void initServer() {
    int j;

    signal(SIGHUP, SIG_IGN);
    signal(SIGPIPE, SIG_IGN);
    setupSigSegvAction();

    server.clients = listCreate();
    server.slaves = listCreate();
    server.monitors = listCreate();
    server.objfreelist = listCreate();
    createSharedObjects();
    server.el = aeCreateEventLoop();
    server.db = zmalloc(sizeof(redisDb)*server.dbnum);
    server.sharingpool = dictCreate(&setDictType,NULL);
    if (!server.db || !server.clients || !server.slaves || !server.monitors || !server.el || !server.objfreelist)
        oom("server initialization"); /* Fatal OOM */
    server.fd = anetTcpServer(server.neterr, server.port, server.bindaddr);
    if (server.fd == -1) {
        redisLog(REDIS_WARNING, "Opening TCP port: %s", server.neterr);
        exit(1);
    }
    for (j = 0; j < server.dbnum; j++) {
        server.db[j].dict = dictCreate(&hashDictType,NULL);
        server.db[j].expires = dictCreate(&setDictType,NULL);
        server.db[j].id = j;
    }
    server.cronloops = 0;
    server.bgsaveinprogress = 0;
    server.bgsavechildpid = -1;
    server.lastsave = time(NULL);
    server.dirty = 0;
    server.usedmemory = 0;
    server.stat_numcommands = 0;
    server.stat_numconnections = 0;
    server.stat_starttime = time(NULL);
    aeCreateTimeEvent(server.el, 1000, serverCron, NULL, NULL);
}

aeMain

aeMain是Redis处理逻辑的主循环,退出循环的标记是eventLoop->stop不等于零,具体的逻辑处理,是由函数aeProcessEvents完成的。

void aeMain(aeEventLoop *eventLoop)
{
    eventLoop->stop = 0;
    while (!eventLoop->stop)
        aeProcessEvents(eventLoop, AE_ALL_EVENTS);
}

eventLoop是全局变量serverel字段,其类型是结构体aeEventLoop

/* State of an event based program */
typedef struct aeEventLoop {
    long long timeEventNextId;
    aeFileEvent *fileEventHead;     
    aeTimeEvent *timeEventHead;     
    int stop;
} aeEventLoop;

接下来,我们要重点看一下函数aeProcessEvents具体做了哪些事。

首先,遍历链表fileEventHead,分别获得AE_READABLEAE_WRITABLEAE_EXCEPTION三类的集合。

aeFileEvent *fe = eventLoop->fileEventHead;
/* Check file events */
if (flags & AE_FILE_EVENTS) {
    while (fe != NULL) {
        if (fe->mask & AE_READABLE) FD_SET(fe->fd, &rfds);
        if (fe->mask & AE_WRITABLE) FD_SET(fe->fd, &wfds);
        if (fe->mask & AE_EXCEPTION) FD_SET(fe->fd, &efds);
        if (maxfd < fe->fd) maxfd = fe->fd;
        numfd++;
        fe = fe->next;
    }
}

接着,计算最近一个时间事件触发的剩余时间tv,用该时间当做select的timeout参数

if (flags & AE_TIME_EVENTS && !(flags & AE_DONT_WAIT))
    shortest = aeSearchNearestTimer(eventLoop);
if (shortest) {
    long now_sec, now_ms;
    /* Calculate the time missing for the nearest
     * timer to fire. */
    aeGetTime(&now_sec, &now_ms);
    tvp = &tv;
    tvp->tv_sec = shortest->when_sec - now_sec;
    if (shortest->when_ms < now_ms) {
        tvp->tv_usec = ((shortest->when_ms+1000) - now_ms)*1000;
        tvp->tv_sec --;
    } else {
        tvp->tv_usec = (shortest->when_ms - now_ms)*1000;
    }
} else {
    /* If we have to check for events but need to return
     * ASAP because of AE_DONT_WAIT we need to se the timeout
     * to zero */
    if (flags & AE_DONT_WAIT) {
        tv.tv_sec = tv.tv_usec = 0;
        tvp = &tv;
    } else {
        /* Otherwise we can block */
        tvp = NULL; /* wait forever */
    }
}

调用select,获得可读、可写的文件描述符数量。如果有可读写的文件描述符,则遍历eventLoop中的链表fileEventHead,依次调用每个节点的fielProc函数。fileEventHead链表的每个节点,类型是struct aeFileEvent, 创建节点的函数是:int aeCreateFileEvent(aeEventLoop *eventLoop, int fd, int mask, aeFileProc *proc, void *clientData, aeEventFinalizerProc *finalizerProc)。 函数参数的proc,指定了节点的fileProc实现。通过全局搜索调用函数aeCreateFileEvent的地方,我们可以知道,节点的fileProc的具体实现,包括以下四种:readQueryFromClientsendReplyToClientsendBulkToSlaveacceptHandler

retval = select(maxfd+1, &rfds, &wfds, &efds, tvp);
if (retval > 0) {
    fe = eventLoop->fileEventHead;
    while(fe != NULL) {
        int fd = (int) fe->fd;
       if ((fe->mask & AE_READABLE && FD_ISSET(fd, &rfds)) ||
            (fe->mask & AE_WRITABLE && FD_ISSET(fd, &wfds)) ||
            (fe->mask & AE_EXCEPTION && FD_ISSET(fd, &efds)))
        {
            int mask = 0;
            if (fe->mask & AE_READABLE && FD_ISSET(fd, &rfds))
                 mask |= AE_READABLE;
            if (fe->mask & AE_WRITABLE && FD_ISSET(fd, &wfds))
                mask |= AE_WRITABLE;
            if (fe->mask & AE_EXCEPTION && FD_ISSET(fd, &efds))
                mask |= AE_EXCEPTION;
            /*
             * fileProce = readQueryFromClient, sendReplyToClient, sendBulkToSlave, acceptHandler
             */
            fe->fileProc(eventLoop, fe->fd, fe->clientData, mask);
            processed++;
            /* After an event is processed our file event list
             * may no longer be the same, so what we do
             * is to clear the bit for this file descriptor and
             * restart again from the head. */
            fe = eventLoop->fileEventHead;
            FD_CLR(fd, &rfds);  // FD_CLR(fd_set *fdset);用于在文件描述符集合中删除一个文件描述符。
            FD_CLR(fd, &wfds);
            FD_CLR(fd, &efds);
        } else {
            fe = fe->next;
        }
    }
}

处理完socket事件后,就是处理时间事件。同处理socket类似,遍历eventLoop中的链表timeEventHead,依次调用每个节点的timeProc函数。timeEventHead的每个节点,类型是struct aeTimeEvent,由函数long long aeCreateTimeEvent(aeEventLoop *eventLoop, long long milliseconds, aeTimeProc *proc, void *clientData, aeEventFinalizerProc *finalizerProc)创建。搜索全局,只在函数initServer中,调用了该函数,创建aeTimeEventaeCreateTimeEvent(server.el, 1000, serverCron, NULL, NULL);

/* Check time events */
if (flags & AE_TIME_EVENTS) {
    te = eventLoop->timeEventHead;
    maxId = eventLoop->timeEventNextId-1;
    while(te) {
        long now_sec, now_ms;
        long long id;
       if (te->id > maxId) {
            te = te->next;
            continue;
        }
        aeGetTime(&now_sec, &now_ms);
        if (now_sec > te->when_sec ||
            (now_sec == te->when_sec && now_ms >= te->when_ms))
        {
            int retval;

            id = te->id;
            retval = te->timeProc(eventLoop, id, te->clientData);
            /* After an event is processed our time event list may
             * no longer be the same, so we restart from head.
             * Still we make sure to don't process events registered
             * by event handlers itself in order to don't loop forever.
             * To do so we saved the max ID we want to handle. */
            if (retval != AE_NOMORE) {
                aeAddMillisecondsToNow(retval,&te->when_sec,&te->when_ms);
            } else {
                aeDeleteTimeEvent(eventLoop, id);
            }
            te = eventLoop->timeEventHead;
        } else {
            te = te->next;
        }
    }
}

在后面章节,讲依次讲解下面的内容:

Redis1.0 与客户端通信过程

Redis的内存数据管理