Redis1.0源码阅读笔记三、内存DB管理

1,004 阅读12分钟

Redis作为内存数据库,是如何在内存中管理数据的呢?本文介绍一下Redis-1.0管理内存数据的方法。

Redis-1.0中,支持的数据结构,仅包括string、list、set。robj->ptr可能指向char *, list *, dict *。在createStringObject时,返回的是char *, 而不是结构体sdshdr *。这是为啥啊?我个人的理解是:因为用char *表示string是最常规的表示,且是一种省内存的方法。至于为啥会引入sdshr *结构体,是方便一些string的操作,如lentrim。我们在获得sds类型的对象后,可以通过如下语句获得struct sdshrd *: struct sdshdr *sh = (void*) (s-(sizeof(struct sdshdr)));

/* A redis object, that is a type able to hold a string / list / set */
typedef struct redisObject {
    void *ptr;
    int type;
    int refcount;
} robj;


typedef char *sds;
struct sdshdr {
    long len;
    long free;
    char buf[];
};

static robj *createStringObject(char *ptr, size_t len) {
    return createObject(REDIS_STRING,sdsnewlen(ptr,len));    
}

static robj *createListObject(void) {
    list *l = listCreate();

    if (!l) oom("listCreate");
    listSetFreeMethod(l,decrRefCount);
    return createObject(REDIS_LIST,l);
}

static robj *createSetObject(void) {
    dict *d = dictCreate(&setDictType,NULL);
    if (!d) oom("dictCreate");
    return createObject(REDIS_SET,d);
}

RedisServer同时管理着多个数据库,默认值是16个。在函数initServer中,被初始化:server.db = zmalloc(sizeof(redisDb)*server.dbnum)

每个数据库是一个结构体redisDb,他的具体形式如下:

typedef struct redisDb {
    dict *dict;
    dict *expires;
    int id;
} redisDb;

有三个字段组成:dict保存key-value的值(小声bb:类型名和变量名同名,不是好习惯),expires保存过期值,id记录是第几个数据库。

redisDb的每个字段,同样在initServer中初始化:

for (j = 0; j < server.dbnum; j++) {
    server.db[j].dict = dictCreate(&hashDictType,NULL);
    server.db[j].expires = dictCreate(&setDictType,NULL);
    server.db[j].id = j;
}

结构体hashDictTypesetDictType的区别,见下文的定义

结构体dict的定义如下:

typedef struct dict {
    dictEntry **table;
    dictType *type;
    unsigned long size;
    unsigned long sizemask;
    unsigned long used;
    void *privdata;
} dict;

dict的创建,由函数dictCreate完成。在redis-1.0中,dict结构在四个地方被复用:

  1. server.db[j].dict 用于存储redis的数据
  2. server.db[j].dict 用于存储键的过期时间
  3. server.sharingpool 用于存储共享对象
  4. set数据结构,是用dict实现

下面,对结构体dict的各个字段做一下说明:

  • dict.table,是一维数组,数组长度由字段size决定。每个元素是一个单向链表,链表的节点定义是dictEntry
typedef struct dictEntry {
    void *key;
    void *val;
    struct dictEntry *next;
} dictEntry;

dictEntry是通过函数dictAdd加入到table中的。

  • dict.type, 记录dict相关的操作,如hash函数、销毁函数等。有两种类型:hashDictTypesetDictType。两者的区别,除了value destructor外,是一样的。
  1. 对于储存数据的字段dict,他的dictType是hashDictType
static dictType hashDictType = {
    dictSdsHash,                /* hash function */
    NULL,                       /* key dup */
    NULL,                       /* val dup */
    dictSdsKeyCompare,          /* key compare */
    dictRedisObjectDestructor,  /* key destructor */
    dictRedisObjectDestructor   /* val destructor */
};
  1. 对于储存过期数据的字段expires、共享对象池sharingpool、存储set数据类型的dict, 他们的dictType都是setDictType
static dictType setDictType = {
    dictSdsHash,               /* hash function */
    NULL,                      /* key dup */
    NULL,                      /* val dup */
    dictSdsKeyCompare,         /* key compare */
    dictRedisObjectDestructor, /* key destructor */
    NULL                       /* val destructor */
};
  • dict.size字段,记录一维数组的长度。
  • dict.sizemask等于dict.size-1,作用是: key的hash值与dict.sizemask进行逻辑与,结果作为key插入dict.table数组的下标。我们通常的做法是,用key的hash值和dict.size进行取余。
  • dict.used记录key的数量。用于判断是否需要扩容、缩容。
  • dict.privdata 对于上述四处创建dict结构,调用dictCreate函数时,传的参数都是NULL。所以还没看懂他的作用o(╯□╰)o

根据上面的分析,我们可以知道如下结论:Redis的key-value实现,是通过dict这个数据结构实现。dict.table是一个一维数组,数据的每个元素,是一个单向链表,链表的节点是结构体dictEntry。dictEntry.key是一个redisObject结构体。

redis的key-value写入

下面,以set命令为例,说明是怎么把key-value,设置到内存数据库dict.table中的。

  1. 接收到客户端的set指令后,redis首先调用setCommand函数:
static void setCommand(redisClient *c) {
    setGenericCommand(c,0);
}
  1. setGenericCommand 函数中,首先调用dictAdd把值写入内存数据库。根据dictAdd的返回值,进行替换或者是直接返回。若成功写入,则记录引用数量、设置数据库dirty字段,移除过期信息。
static void setGenericCommand(redisClient *c, int nx) {
    int retval;

    retval = dictAdd(c->db->dict,c->argv[1],c->argv[2]);
    //key已经存在时,返回DICT_ERR
    if (retval == DICT_ERR) {
        if (!nx) {
            dictReplace(c->db->dict,c->argv[1],c->argv[2]);
            incrRefCount(c->argv[2]);
        } else {
            addReply(c,shared.czero);
            return;
        }
    } else {
        incrRefCount(c->argv[1]);
        incrRefCount(c->argv[2]);
    }
    server.dirty++;
    removeExpire(c->db,c->argv[1]);
    addReply(c, nx ? shared.cone : shared.ok);
}
  1. dictAdd写入时,首先是通过函数_dictKeyIndex获得key的hash值,若值存在,则返回DICT_ERR;若不存在,则写入dict的table字段中。由下面的代码已知,redis解决key冲突,是通过链表的形式。 dictSetHashKeydictSetHashVal,把key和value保存到dictEntry的key\value字段中
/* Add an element to the target hash table */
int dictAdd(dict *ht, void *key, void *val)
{
    int index;
    dictEntry *entry;

    /* Get the index of the new element, or -1 if
     * the element already exists. */
    if ((index = _dictKeyIndex(ht, key)) == -1)
        return DICT_ERR;

    /* Allocates the memory and stores key */
    entry = _dictAlloc(sizeof(*entry));
    entry->next = ht->table[index];
    ht->table[index] = entry;

    /* Set the hash entry fields. */
    dictSetHashKey(ht, entry, key);
    dictSetHashVal(ht, entry, val);
    ht->used++;
    return DICT_OK;
}
  1. key的hash值,是在_dictKeyIndex中计算的。在计算key之前,会先检查是否需要扩容。然后调用dictHashKey获得key的hah值,并与dict的sizemask进行逻辑与,保证hash值不超过数组边界。
static int _dictKeyIndex(dict *ht, const void *key)
{
    unsigned int h;
    dictEntry *he;

    /* Expand the hashtable if needed */
    if (_dictExpandIfNeeded(ht) == DICT_ERR)
        return -1;
    /* Compute the key hash value */
    // sizemask = ht->size - 1
    h = dictHashKey(ht, key) & ht->sizemask;
    /* Search if this slot does not already contain the given key */
    he = ht->table[h];
    while(he) {
        if (dictCompareHashKeys(ht, key, he->key))
            return -1;
        he = he->next;
    }
    return h;
}

dictHashKey是个宏定义,其定义如下:

#define dictHashKey(ht, key) (ht)->type->hashFunction(key)

那么,对于redis的内存数据库,其hashFunction具体实现就由static dictType hashDictType中的字段决定。由上文可以,他的实现名是dictSdsHash,采用的是djb2算法。具体实现如下:

static unsigned int dictSdsHash(const void *key) {
    const robj *o = key;
    return dictGenHashFunction(o->ptr, sdslen((sds)o->ptr));
}

/* Generic hash function (a popular one from Bernstein).
 * I tested a few and this was the best. */
unsigned int dictGenHashFunction(const unsigned char *buf, int len) {
    unsigned int hash = 5381;

    while (len--)
        hash = ((hash << 5) + hash) + (*buf++); /* hash * 33 + c */
    return hash;
}
  1. 内存数据库的扩容是在函数_dictExpandIfNeeded中完成的。DICT_HT_INITIAL_SIZE初始值等于4。
/* Expand the hash table if needed */
static int _dictExpandIfNeeded(dict *ht)
{
    /* If the hash table is empty expand it to the intial size,
     * if the table is "full" dobule its size. */
    if (ht->size == 0)
        return dictExpand(ht, DICT_HT_INITIAL_SIZE);
    if (ht->used == ht->size)
        return dictExpand(ht, ht->size*2);
    return DICT_OK;
}

dictExpand首先获得最接近size的2的n次幂,由函数_dictNextPower完成。然后是通过for循环,拷贝数组的每个元素。通过while循环,把每个元素对应的链表进行拷贝。通过n.sizemask = realsize-1,记录sizemask, 该值通过与key的hash值进行逻辑与,保证key的hash值小于数组的大小,从而不会越界访问。

dictExpand不仅用于扩容,也用于缩容。即dictExpand负责把dict扩展或缩小到指定size的大小。

/* Expand or create the hashtable */
int dictExpand(dict *ht, unsigned long size)
{
    dict n; /* the new hashtable */
    unsigned long realsize = _dictNextPower(size), i;

    /* the size is invalid if it is smaller than the number of
     * elements already inside the hashtable */
    if (ht->used > size)
        return DICT_ERR;

    _dictInit(&n, ht->type, ht->privdata);
    n.size = realsize;
    n.sizemask = realsize-1;
    n.table = _dictAlloc(realsize*sizeof(dictEntry*));

    /* Initialize all the pointers to NULL */
    memset(n.table, 0, realsize*sizeof(dictEntry*));

    /* Copy all the elements from the old to the new table:
     * note that if the old hash table is empty ht->size is zero,
     * so dictExpand just creates an hash table. */
    n.used = ht->used;
    for (i = 0; i < ht->size && ht->used > 0; i++) {
        dictEntry *he, *nextHe;

        if (ht->table[i] == NULL) continue;
        
        /* For each hash entry on this slot... */
        he = ht->table[i];
        while(he) {
            unsigned int h;

            nextHe = he->next;
            /* Get the new element index */
            h = dictHashKey(ht, he->key) & n.sizemask;
            he->next = n.table[h];
            n.table[h] = he;
            ht->used--;
            /* Pass to the next element */
            he = nextHe;
        }
    }
    assert(ht->used == 0);
    _dictFree(ht->table);

    /* Remap the new hashtable in the old */
    *ht = n;
    return DICT_OK;
}

删除key实现

删除指令的入口函数是delCommand

static void delCommand(redisClient *c) {
    int deleted = 0, j;

    for (j = 1; j < c->argc; j++) {
        if (deleteKey(c->db,c->argv[j])) {
            server.dirty++;
            deleted++;
        }
    }
    switch(deleted) {
    case 0:
        addReply(c,shared.czero);
        break;
    case 1:
        addReply(c,shared.cone);
        break;
    default:
        addReplySds(c,sdscatprintf(sdsempty(),":%d\r\n",deleted));
        break;
    }
}

可见,主要逻辑在deleteKey中。

static int deleteKey(redisDb *db, robj *key) {
    int retval;

    /* We need to protect key from destruction: after the first dictDelete()
     * it may happen that 'key' is no longer valid if we don't increment
     * it's count. This may happen when we get the object reference directly
     * from the hash table with dictRandomKey() or dict iterators */
    incrRefCount(key);
    if (dictSize(db->expires)) dictDelete(db->expires,key);
    retval = dictDelete(db->dict,key);
    decrRefCount(key);

    return retval == DICT_OK;
}

dictDelete调用dictGenericDelete,它是真正实现从链表删除节点的函数。 可以看到,在key被删除时,并没有缩小dict.table的大小。redis的缩容,是在定时事件中,通过tryResizeHashTables中完成。

int dictDelete(dict *ht, const void *key) {
    return dictGenericDelete(ht,key,0);
}

/* Search and remove an element */
static int dictGenericDelete(dict *ht, const void *key, int nofree)
{
    unsigned int h;
    dictEntry *he, *prevHe;

    if (ht->size == 0)
        return DICT_ERR;
    h = dictHashKey(ht, key) & ht->sizemask;
    he = ht->table[h];

    prevHe = NULL;
    while(he) {
        if (dictCompareHashKeys(ht, key, he->key)) {
            /* Unlink the element from the list */
            if (prevHe)
                prevHe->next = he->next;
            else
                ht->table[h] = he->next;
            if (!nofree) {
                dictFreeEntryKey(ht, he);
                dictFreeEntryVal(ht, he);
            }
            _dictFree(he);
            ht->used--;
            return DICT_OK;
        }
        prevHe = he;
        he = he->next;
    }
    return DICT_ERR; /* not found */
}
  • 在定时事件处理函数serverCron,将调用函数tryResizeHashTables进行缩容
/* If the percentage of used slots in the HT reaches REDIS_HT_MINFILL
 * we resize the hash table to save memory */
static void tryResizeHashTables(void) {
    int j;

    for (j = 0; j < server.dbnum; j++) {
        if (htNeedsResize(server.db[j].dict)) {
            redisLog(REDIS_DEBUG,"The hash table %d is too sparse, resize it...",j);
            dictResize(server.db[j].dict);
            redisLog(REDIS_DEBUG,"Hash table %d resized.",j);
        }
        if (htNeedsResize(server.db[j].expires))
            dictResize(server.db[j].expires);
    }
}

判断是否需要缩容的标准是:已填内存不足已申请内存的10%(REDIS_HT_MINFILL=10)

static int htNeedsResize(dict *dict) {
    long long size, used;

    size = dictSlots(dict);
    used = dictSize(dict);
    return (size && used && size > DICT_HT_INITIAL_SIZE &&
            (used*100/size < REDIS_HT_MINFILL));

redis过期键处理

把键的过期时间,写入db.expires这个dict中,是由setExpire完成的。

static int setExpire(redisDb *db, robj *key, time_t when) {
    if (dictAdd(db->expires,key,(void*)when) == DICT_ERR) {
        return 0;
    } else {
        incrRefCount(key);
        return 1;
    }
}

setExpire只会在expireCommandrdbLoad中被调用。可见,在redis-1.0中,redis还是不支持set命令加expire参数。expireCommand命令设置过期键。

static void expireCommand(redisClient *c) {
    dictEntry *de;
    int seconds = atoi(c->argv[2]->ptr);

    de = dictFind(c->db->dict,c->argv[1]);
    if (de == NULL) {
        addReply(c,shared.czero);
        return;
    }
    if (seconds <= 0) {
        addReply(c, shared.czero);
        return;
    } else {
        time_t when = time(NULL)+seconds;
        if (setExpire(c->db,c->argv[1],when)) {
            addReply(c,shared.cone);
            server.dirty++;
        } else {
            addReply(c,shared.czero);
        }
        return;
    }
}

把过期键从db.expires中删除,发生在两个地方:定期执行的serverCron函数; 在每次执行客户端请求时可能执行的freeMemoryIfNeeded函数。

  • 在定期执行serverCron函数中处理过期键。每次最多执行100次删除操作(REDIS_EXPIRELOOKUPS_PER_CRON=100)。每次随机的从db.expires获得一个设置了过期时间的key,如果该key已过期,则删除。
static int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
    /* Try to expire a few timed out keys */
    for (j = 0; j < server.dbnum; j++) {
        redisDb *db = server.db+j;
        int num = dictSize(db->expires);

        if (num) {
            time_t now = time(NULL);

            if (num > REDIS_EXPIRELOOKUPS_PER_CRON)
                num = REDIS_EXPIRELOOKUPS_PER_CRON;
            while (num--) {
                dictEntry *de;
                time_t t;

                if ((de = dictGetRandomKey(db->expires)) == NULL) break;
                t = (time_t) dictGetEntryVal(de);
                if (now > t) {
                    deleteKey(db,dictGetEntryKey(de));
                }
            }
        }
    }
}
  • freeMemoryIfNeeded中处理删除过期键。freeMemoryIfNeeded是在每次处理客户端请求时,若达到了最大内存阈值,则执行一次。
static int processCommand(redisClient *c) {
    struct redisCommand *cmd;
    long long dirty;

    /* Free some memory if needed (maxmemory setting) */
    if (server.maxmemory) freeMemoryIfNeeded();
}

freeMemoryIfNeeded可以看出,当redis申请的内存超过允许的最大内存时,首先判断server的objfreelist是否为空。若不为空,则从list删除对象,释放空间。如果从objfreelist没有空间可以释放,则从expires中, 尝试随机选择3个键,删除3个键中生存时间最短的键值对。不管是否已过期,都会删除。

static void freeMemoryIfNeeded(void) {
    while (server.maxmemory && zmalloc_used_memory() > server.maxmemory) {
        if (listLength(server.objfreelist)) {
            robj *o;

            listNode *head = listFirst(server.objfreelist);
            o = listNodeValue(head);
            listDelNode(server.objfreelist,head);
            zfree(o);
        } else {
            int j, k, freed = 0;

            for (j = 0; j < server.dbnum; j++) {
                int minttl = -1;
                robj *minkey = NULL;
                struct dictEntry *de;

                if (dictSize(server.db[j].expires)) {
                    freed = 1;
                    /* From a sample of three keys drop the one nearest to
                     * the natural expire */
                    for (k = 0; k < 3; k++) {
                        time_t t;

                        de = dictGetRandomKey(server.db[j].expires);
                        t = (time_t) dictGetEntryVal(de);
                        if (minttl == -1 || t < minttl) {
                            minkey = dictGetEntryKey(de);
                            minttl = t;
                        }
                    }
                    deleteKey(server.db+j,minkey);
                }
            }
            if (!freed) return; /* nothing to free... */
        }
    }
}

String、List、Set数据结构实现

在上文中,我们已经学习到,redis服务端,读取客户端的请求信息,是在readQueryFromClient开始的。下面的代码片段,展示了redis服务端读取请求数据,并且调用processCommand的过程。

for (j = 0; j < argc; j++) {
    if (sdslen(argv[j])) {
        c->argv[c->argc] = createObject(REDIS_STRING,argv[j]);
        c->argc++;
    } else {
        sdsfree(argv[j]);
    }
}
zfree(argv);
/* Execute the command. If the client is still valid
 * after processCommand() return and there is something
 * on the query buffer try to process the next command. */
if (c->argc && processCommand(c) && sdslen(c->querybuf)) goto again;
return;

createObject用于创建一个redisObject,保存redis的Value值,包括了string、list、set

typedef struct redisObject {
    void *ptr;
    int type;
    int refcount;
} robj;

下面这个createObject函数的实现,展示了redis的内存管理的一个优化:server.objfreelist通过一个链表的形式,管理曾经申请的,但现在空闲的redisObject结构,在需要使用的时候,重复使用,避免频繁的malloc、free操作。

static robj *createObject(int type, void *ptr) {
    robj *o;

    if (listLength(server.objfreelist)) {
        listNode *head = listFirst(server.objfreelist);
        o = listNodeValue(head);
        listDelNode(server.objfreelist,head);
    } else {
        o = zmalloc(sizeof(*o));
    }
    if (!o) oom("createObject");
    o->type = type;
    o->ptr = ptr;
    o->refcount = 1;
    return o;
}

上文中,我们已经通过set指令,介绍了string的管理。下面通过介绍命令rpush\sadd命令,介绍list和set的实现。

  • List数据结构的实现。我们通过pushGenericCommand指令,介绍Redis中,是如何管理List对象的。
static void pushGenericCommand(redisClient *c, int where) {
    robj *lobj;
    list *list;

    lobj = lookupKeyWrite(c->db,c->argv[1]);
    if (lobj == NULL) {
        lobj = createListObject();
        list = lobj->ptr;
        if (where == REDIS_HEAD) {
            if (!listAddNodeHead(list,c->argv[2])) oom("listAddNodeHead");
        } else {
            if (!listAddNodeTail(list,c->argv[2])) oom("listAddNodeTail");
        }
        dictAdd(c->db->dict,c->argv[1],lobj);
        incrRefCount(c->argv[1]);
        incrRefCount(c->argv[2]);
    } else {
        if (lobj->type != REDIS_LIST) {
            addReply(c,shared.wrongtypeerr);
            return;
        }
        list = lobj->ptr;
        if (where == REDIS_HEAD) {
            if (!listAddNodeHead(list,c->argv[2])) oom("listAddNodeHead");
        } else {
            if (!listAddNodeTail(list,c->argv[2])) oom("listAddNodeTail");
        }
        incrRefCount(c->argv[2]);
    }
    server.dirty++;
    addReply(c,shared.ok);
}

从上面的实现易见,首先通过lookupKeyWrite函数找到list对象lobj,如果不存在,则通过createListObject创建一个list结构体。如果存在,则向list中加入value。

static robj *createListObject(void) {
    list *l = listCreate();

    if (!l) oom("listCreate");
    listSetFreeMethod(l,decrRefCount);
    return createObject(REDIS_LIST,l);
}

list *listCreate(void)
{
    struct list *list;

    if ((list = zmalloc(sizeof(*list))) == NULL)
        return NULL;
    list->head = list->tail = NULL;
    list->len = 0;
    list->dup = NULL;
    list->free = NULL;
    list->match = NULL;
    return list;
}

typedef struct list {
    listNode *head;
    listNode *tail;
    void *(*dup)(void *ptr);
    void (*free)(void *ptr);
    int (*match)(void *ptr, void *key);
    unsigned int len;
    listIter iter;
} list;

向list中根据where参数,在队列头部或尾部加入。

list *listAddNodeHead(list *list, void *value)
{
    listNode *node;

    if ((node = zmalloc(sizeof(*node))) == NULL)
        return NULL;
    node->value = value;
    if (list->len == 0) {
        list->head = list->tail = node;
        node->prev = node->next = NULL;
    } else {
        node->prev = NULL;
        node->next = list->head;
        list->head->prev = node;
        list->head = node;
    }
    list->len++;
    return list;
}
  • Set数据结构的实现。我们通过saddCommand指令,介绍Redis中,是如何管理Set对象的。
static void saddCommand(redisClient *c) {
    robj *set;

    set = lookupKeyWrite(c->db,c->argv[1]);
    if (set == NULL) {
        set = createSetObject();
        dictAdd(c->db->dict,c->argv[1],set);
        incrRefCount(c->argv[1]);
    } else {
        if (set->type != REDIS_SET) {
            addReply(c,shared.wrongtypeerr);
            return;
        }
    }
    if (dictAdd(set->ptr,c->argv[2],NULL) == DICT_OK) {
        incrRefCount(c->argv[2]);
        server.dirty++;
        addReply(c,shared.cone);
    } else {
        addReply(c,shared.czero);
    }
}

同list的管理类似,先是通过lookupkeyWrite找到key对应的set对象,如果不存在,则通过createSetObject创建一个,并通过dictAdd写入db.table中。

然后,再次通过dictAdd把新增加的元素加入set中。

static robj *createSetObject(void) {
    dict *d = dictCreate(&setDictType,NULL);
    if (!d) oom("dictCreate");
    return createObject(REDIS_SET,d);
}

由上面的createSetObject可见,set的实现,是通过dict的形式实现的。

redisObject的对象引用计数

为了避免频繁的调用malloc、free, redis用全局变量objfreelist缓存了已申请但不再使用的内存。redisObject有一个字段refcount记录引用计数,每次减少引用时,调用decrRefCount,如果引用计数变为0,则加入objfreelist这个链表中。switch-case中,虽然对robj的type判断了REDIS_HASH的情况,但可以肯定的是,robj->type并不会等于REDIS_HASH。

static void decrRefCount(void *obj) {
    robj *o = obj;

#ifdef DEBUG_REFCOUNT
    if (o->type == REDIS_STRING)
        printf("Decrement '%s'(%p), now is: %d\n",o->ptr,o,o->refcount-1);
#endif
    if (--(o->refcount) == 0) {
        switch(o->type) {
        case REDIS_STRING: freeStringObject(o); break;
        case REDIS_LIST: freeListObject(o); break;
        case REDIS_SET: freeSetObject(o); break;
        case REDIS_HASH: freeHashObject(o); break;
        default: assert(0 != 0); break;
        }
        if (listLength(server.objfreelist) > REDIS_OBJFREELIST_MAX ||
            !listAddNodeHead(server.objfreelist,o))
            zfree(o);
    }
}

redis的共享内存实现

为了节约内存, redis提供了shareobjects配置选项,如果开启,则会对相同的值,用同一个对象。

static int processCommand(redisClient *c) {
    /* Let's try to share objects on the command arguments vector */
    if (server.shareobjects) {
        int j;
        for(j = 1; j < c->argc; j++)
            c->argv[j] = tryObjectSharing(c->argv[j]);
    }
    ...
    /* Exec the command */
    dirty = server.dirty;
    cmd->proc(c);
}
/* Try to share an object against the shared objects pool */
static robj *tryObjectSharing(robj *o) {
    struct dictEntry *de;
    unsigned long c;

    if (o == NULL || server.shareobjects == 0) return o;

    assert(o->type == REDIS_STRING);
    de = dictFind(server.sharingpool,o);
    if (de) {
        robj *shared = dictGetEntryKey(de);

        c = ((unsigned long) dictGetEntryVal(de))+1;
        dictGetEntryVal(de) = (void*) c;
        incrRefCount(shared);
        decrRefCount(o);
        return shared;
    } else {
        /* Here we are using a stream algorihtm: Every time an object is
         * shared we increment its count, everytime there is a miss we
         * recrement the counter of a random object. If this object reaches
         * zero we remove the object and put the current object instead. */
        if (dictSize(server.sharingpool) >=
                server.sharingpoolsize) {
            de = dictGetRandomKey(server.sharingpool);
            assert(de != NULL);
            c = ((unsigned long) dictGetEntryVal(de))-1;
            dictGetEntryVal(de) = (void*) c;
            if (c == 0) {
                dictDelete(server.sharingpool,de->key);
            }
        } else {
            c = 0; /* If the pool is empty we want to add this object */
        }
        if (c == 0) {
            int retval;

            retval = dictAdd(server.sharingpool,o,(void*)1);
            assert(retval == DICT_OK);
            incrRefCount(o);
        }
        return o;
    }
}

redis数据持久化

redis内存中的数据保存到文件,是由函数static int rdbSaveBackground(char *filename)完成。调用函数的地方,包括了以下几个地方:1. serverCron定期执行; 2. bgsave命令; 3. sync命令。 redis在数据持久化时,通过fork子进程实现。具体实现由rdbSave完成。

static int rdbSaveBackground(char *filename) {
    pid_t childpid;

    if (server.bgsaveinprogress) return REDIS_ERR;
    if ((childpid = fork()) == 0) {
        /* Child */
        close(server.fd);
        if (rdbSave(filename) == REDIS_OK) {
            exit(0);
        } else {
            exit(1);
        }
    } else {
        /* Parent */
        if (childpid == -1) {
            redisLog(REDIS_WARNING,"Can't save in background: fork: %s",
                strerror(errno));
            return REDIS_ERR;
        }
        redisLog(REDIS_NOTICE,"Background saving started by pid %d",childpid);
        server.bgsaveinprogress = 1;
        server.bgsavechildpid = childpid;
        return REDIS_OK;
    }
    return REDIS_OK; /* unreached */
}