Redis的过期机制和缓存淘汰机制

2,187 阅读15分钟

相信大家对Redis的expire命令都不陌生,日常工作中都使用过。但是大家了解expire的工作原理吗? 今天我们就来简单讲解一下expire命令的实现和工作原理。之所以和Redis的缓存淘汰机制一起讨论,是因为很多人把他们搞混淆了,之后我们会详细说明的。

过期机制

> expire mykey 1000

当我们这样操作时,就为mykey设置了1000s的过期时间。Redis将这个key保存在服务端db的expires字典项中。

/* EXPIRE key seconds */
void expireCommand(client *c) {
    expireGenericCommand(c,mstime(),UNIT_SECONDS);
}

void expireGenericCommand(client *c, long long basetime, int unit) {
	...
    /* EXPIRE with negative TTL, or EXPIREAT with a timestamp into the past
     * should never be executed as a DEL when load the AOF or in the context
     * of a slave instance.
     *
     * Instead we take the other branch of the IF statement setting an expire
     * (possibly in the past) and wait for an explicit DEL from the master. */
     // 从实例,并且已过期(如从AOF或RDB文件中初始化数据),改为删除命令,删除该key
    if (when <= mstime() && !server.loading && !server.masterhost) {
    	...
    } else {
    	setExpire(c,c->db,key,when);
        ...
    }
    ...
}

1000s后,这个key就过期了,我们将访问不到这个key对应的值。 如果1000s后,key过期了,虽然我们访问不到了,但是,数据还在内存中吗?Redis已经将这条数据从内存中释放了吗?

Redis对过期的key有俩种处理方式,主动删除和被动删除。被动删除是指用户读取这个key时,判断这个key是否过期,如果过期了,则删除这个key,返回给客户端null。但是如果这个过期的key一直没人访问,难道这个key就一直在内存中吗?不是的,Redis有个定时任务,主动删除这个过期的key。

被动删除key

对Redis中的数据进行读、写操作时(覆盖操作除外,例如set),都会调用expireIfNeeded函数,判断key是否过期,如果key过期,在master节点,则删除,并向从节点、AOF文件发送delunlink命令,然后返回null。如果是从节点,则直接返回null。

/* This function is called when we are going to perform some operation
 * in a given key, but such key may be already logically expired even if
 * it still exists in the database. The main way this function is called
 * is via lookupKey*() family of functions.
 *
 * The behavior of the function depends on the replication role of the
 * instance, because slave instances do not expire keys, they wait
 * for DELs from the master for consistency matters. However even
 * slaves will try to have a coherent return value for the function,
 * so that read commands executed in the slave side will be able to
 * behave like if the key is expired even if still present (because the
 * master has yet to propagate the DEL).
 *
 * In masters as a side effect of finding a key which is expired, such
 * key will be evicted from the database. Also this may trigger the
 * propagation of a DEL/UNLINK command in AOF / replication stream.
 *
 * The return value of the function is 0 if the key is still valid,
 * otherwise the function returns 1 if the key is expired. */
int expireIfNeeded(redisDb *db, robj *key) {
    // key是否已过期,如果未过期,返回0
    if (!keyIsExpired(db,key)) return 0;

    /* If we are running in the context of a slave, instead of
     * evicting the expired key from the database, we return ASAP:
     * the slave key expiration is controlled by the master that will
     * send us synthesized DEL operations for expired keys.
     *
     * Still we try to return the right information to the caller,
     * that is, 0 if we think the key should be still valid, 1 if
     * we think the key is expired at this time. */
    if (server.masterhost != NULL) return 1;

    /* Delete the key */
    server.stat_expiredkeys++;
    // 持久化、主从复制时,过期键的处理
    propagateExpire(db,key,server.lazyfree_lazy_expire);
    notifyKeyspaceEvent(NOTIFY_EXPIRED,
        "expired",key,db->id);
    // 删除过期的key
    return server.lazyfree_lazy_expire ? dbAsyncDelete(db,key) :
                                         dbSyncDelete(db,key);
}

代码注释写得很清楚了,逻辑也很简单,就不再赘述了。

主动删除key

Redis启动时,会初始化定时任务,

/* Create the timer callback, this is our way to process many background
 * operations incrementally, like clients timeout, eviction of unaccessed
 * expired keys and so forth. */
// 设置后台定时器事件,事件句柄为serverCron()
if (aeCreateTimeEvent(server.el, 1, serverCron, NULL, NULL) == AE_ERR) {
    serverPanic("Can't create event loop timers.");
    exit(1);
}

任务句柄为serverCron函数,每毫秒执行一次定时任务。

/* This is our timer interrupt, called server.hz times per second.
 * Here is where we do a number of things that need to be done asynchronously.
 * For instance:
 *
 * - Active expired keys collection (it is also performed in a lazy way on
 *   lookup).
 * - Software watchdog.
 * - Update some statistic.
 * - Incremental rehashing of the DBs hash tables.
 * - Triggering BGSAVE / AOF rewrite, and handling of terminated children.
 * - Clients timeout of different kinds.
 * - Replication reconnection.
 * - Many more...
 *
 * Everything directly called here will be called server.hz times per second,
 * so in order to throttle execution of things we want to do less frequently
 * a macro is used: run_with_period(milliseconds) { .... }
 */

int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
	...
    /* Handle background operations on Redis databases. */
    databasesCron();
    ...
    // redis.conf中server.hz默认值为10,每秒执行10次
    return 1000/server.hz;
}
# Redis calls an internal function to perform many background tasks, like
# closing connections of clients in timeout, purging expired keys that are
# never requested, and so forth.
#
# Not all tasks are performed with the same frequency, but Redis checks for
# tasks to perform according to the specified "hz" value.
#
# By default "hz" is set to 10. Raising the value will use more CPU when
# Redis is idle, but at the same time will make Redis more responsive when
# there are many keys expiring at the same time, and timeouts may be
# handled with more precision.
#
# The range is between 1 and 500, however a value over 100 is usually not
# a good idea. Most users should use the default of 10 and raise this up to
# 100 only in environments where very low latency is required.
hz 10
/* This function handles 'background' operations we are required to do
 * incrementally in Redis databases, such as active key expiring, resizing,
 * rehashing. */
void databasesCron(void) {
	/* Expire keys by random sampling. Not required for slaves
     * as master will synthesize DELs for us. */
    if (server.active_expire_enabled) {
        if (server.masterhost == NULL) {
            // 主实例(master)
            activeExpireCycle(ACTIVE_EXPIRE_CYCLE_SLOW);
        } else {
            // 从实例(slave)
            expireSlaveKeys();
        }
    }
    ...
}

activeExpireCycle函数执行过期缓存清理工作,

/* Try to expire a few timed out keys. The algorithm used is adaptive and
 * will use few CPU cycles if there are few expiring keys, otherwise
 * it will get more aggressive to avoid that too much memory is used by
 * keys that can be removed from the keyspace.
 *
 * No more than CRON_DBS_PER_CALL databases are tested at every
 * iteration.
 *
 * This kind of call is used when Redis detects that timelimit_exit is
 * true, so there is more work to do, and we do it more incrementally from
 * the beforeSleep() function of the event loop.
 *
 * Expire cycle type:
 *
 * If type is ACTIVE_EXPIRE_CYCLE_FAST the function will try to run a
 * "fast" expire cycle that takes no longer than EXPIRE_FAST_CYCLE_DURATION
 * microseconds, and is not repeated again before the same amount of time.
 *
 * If type is ACTIVE_EXPIRE_CYCLE_SLOW, that normal expire cycle is
 * executed, where the time limit is a percentage of the REDIS_HZ period
 * as specified by the ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC define. */

void activeExpireCycle(int type) {
	/* We can use at max ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC percentage of CPU time
     * per iteration. Since this function gets called with a frequency of
     * server.hz times per second, the following is the max amount of
     * microseconds we can spend in this function. */
     // 1000000*25/10/100=25000微秒
    timelimit = 1000000*ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC/server.hz/100;
    ...
    /* Continue to expire if at the end of the cycle more than 25%
     * of the keys were expired. */
    do {
    	...
        // num <= 20
        while (num--) {
            ...
		    // 从db->expires中获取任意一个key
            if ((de = dictGetRandomKey(db->expires)) == NULL) break;
            ttl = dictGetSignedIntegerVal(de)-now;
            // 判断这个key是否已过期,如果已过期,向AOF文件、从节点发送unlink活del命令,然后删除该key数据
            if (activeExpireCycleTryExpire(db,de,now)) expired++;
            ...
        }
        /* We don't repeat the cycle if there are less than 25% of keys
         * found expired in the current DB. */
    } while (expired > ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP/4);
}

大体逻辑是这样的:

每秒执行10次过期key清理操作:

  1. 从所有设置过期时间的key中,任意筛选20个可以(db->expires字典项中任意获取20个key)
  2. 删除过期的key
  3. 如果超过25%的key过期了,从步骤1重新开始。

这个算法基于概率实现的,并没有将所有的过期key都删除,防止线程阻塞,导致客户端响应变慢。如果主动删除漏了某个过期的key,没事,还有被动删除呢。如果被动删除也漏了,还有下面说的缓存淘汰机制呢。缓存淘汰机制也是基于概率算的,也可能有遗漏,具体下面会讲解,这是Redis为了高效、快速响应而做的牺牲。

一些小知识点

1. 系统时间不准确 比如我在主实例中:

> expire mykey 1000

然后从实例的系统时间往前调2000s,这个key同步到从实例时,Redis任务它过期了,从实例中就没有这个key了。Redis的时间都是基于服务器时间。

2. 过期时间清除

只有删除或覆盖键内容的命令(包括DEL,SET,GETSET和所有* STORE命令)才能清除过期时间。 这意味着所有在概念上更改存储在键上的值,而不用新键替换的操作都将保持过期时间不变。 例如,使用INCR递增键的值,使用LPUSH将新值推入列表或使用HSET更改哈希的字段值都是使超时保持不变的操作。

>lpush mylist q
"1"
>expire mylist 1000
"1"
>ttl mylist
"997"
>lpush mylist w
"2"
>ttl mylist
"985"
>set mykey a
"OK"
>expire mykey 1000
"1"
>ttl mykey
"990"
>set mykey b
"OK"
>ttl mykey
"-1"

缓存淘汰

Redis的数据都是保存在内存中的,内存不是无限的,总有用完的时候。当达到内存阈值时,Redis就会启动缓存淘汰策略,清理一部分内存。常用的缓存淘汰算法有LRU,在4.0版本时新增了LFU算法。

淘汰策略配置

1. maxmemory内存大小限制

redis.conf文件中配置:

maxmemory 100mb

或者Redis在运行时,使用CONFIG SET命令:

> config set maxmemory 100mb

2. 设置maxmemory-policy

Redis4.0之后有8种缓存淘汰配置:

  • volatile-lru 使用LRU算法,从设置过期时间的key中选择淘汰数据
  • allkeys-lru 使用LRU算法,从所有key中选择(推荐)
  • volatile-lfu 使用LFU算法,从设置过期时间的key中选择
  • allkeys-lfu 使用LFU算法,从所有key中选择(4.0版本以上推荐)
  • volatile-random 从设置过期时间的key中随机选择
  • allkeys-random 从所有key中随机选择
  • volatile-ttl 将要过期的,ttl最小的数据
  • noeviction 不淘汰数据,内存不够时,报错

为什么推荐使用allkeys-lru策略?

  1. LFU毕竟是4.0版本后才有的,使用LRU不用担心版本不兼容
  2. 随机选择的话,容易把热点数据给删除了,到时出现缓存击穿就GG了
  3. 如果设置过期时间的key较少,缓存淘汰的也就少了,内存很快也就不够用了

当然,如果你们公司规定了使用4.0版本以上的Redis,推荐使用allkeys-lfu

3. 随机样例数量maxmemory_samples

随机采样的精度,也就是随机取出key的数目。该值越大, 越接近于真实的LRU算法,但是数值越大,相应消耗也变高,对性能有一定影响,默认值为5。

当一个命令到达Redis时,都会调用freeMemoryIfNeededAndSafe函数判断是否需要进行缓存淘汰以及执行哪种淘汰策略。

int processCommand(client *c) {
	...
    if (server.maxmemory && !server.lua_timedout) {
        int out_of_memory = freeMemoryIfNeededAndSafe() == C_ERR;
        ...
    }
    ...
}

近似LRU算法

Redis并没有使用真正的LRU算法,那样太耗内存了,Redis使用近似LRU算法,对少量key进行采样,然后从采样的key中驱出最久未使用的key,样例个数为maxmemory-samples,默认值为5。3.0版本进一步优化了该算法,使其执行效果更接近真正的LRU算法。

Redis的键和值都是redisObject对象:

typedef struct redisObject {
    unsigned type:4;
    unsigned encoding:4;
    unsigned lru:LRU_BITS; /* LRU time (relative to global lru_clock) or
                            * LFU data (least significant 8 bits frequency
                            * and most significant 16 bits access time). */
    int refcount;
    void *ptr;
} robj;

redisObject有个24bits的空间lruLRU算法时,用来存储低位的时间戳(以秒级为单位)。key被更新或访问时,都会更新这个时间戳,即LRU_CLOCK

/* Return the LRU clock, based on the clock resolution. This is a time
 * in a reduced-bits format that can be used to set and check the
 * object->lru field of redisObject structures. */
unsigned int getLRUClock(void) {
    return (mstime()/LRU_CLOCK_RESOLUTION) & LRU_CLOCK_MAX;
}

Redis3.0后,提供了一个待淘汰候选key的pool数组,大小为16个。执行缓存淘汰时,从所有db中筛选待淘汰候选key。

/* We don't want to make local-db choices when expiring keys,
 * so to start populate the eviction pool sampling keys from
 * every DB. */
for (i = 0; i < server.dbnum; i++) {
    db = server.db+i;
    dict = (server.maxmemory_policy & MAXMEMORY_FLAG_ALLKEYS) ?
            db->dict : db->expires;
    if ((keys = dictSize(dict)) != 0) {
        // 构建待淘汰缓存池
        evictionPoolPopulate(i, dict, db->dict, pool);
        total_keys += keys;
    }
}

这时有个idle空闲时间的概念,即key多长时间未被访问了(idle约等于LRU_CLOCK-lru),按照idle升序排序。从键空间(所有key或者设置过期时间的key)中随机选择maxmemory-samples个key,分别计算它们的空闲时间idle,只有当pool有空闲空间,或者空闲时间idle大于pool里最小的idle,才会进入pool,然后从pool中选择空闲时间最大的key淘汰掉。

真实LRU算法与近似LRU的算法的对比图如下:

浅灰色带是已经被淘汰的对象,灰色带是没有被淘汰的对象,绿色带是新添加的对象。可以看出,maxmemory-samples值为5时Redis 3.0效果比Redis 2.8要好。maxmemory-samples值为10时,Redis 3.0已经非常接近理论的性能了。

LFU

LFU策略的配置: 除了上述的maxmemorymaxmemory-policymaxmemory_samples外(maxmemory-policy配置volatile-lfuallkeys-lfu),还有2个配置项:

# Redis LFU eviction (see maxmemory setting) can be tuned. However it is a good
# idea to start with the default settings and only change them after investigating
# how to improve the performances and how the keys LFU change over time, which
# is possible to inspect via the OBJECT FREQ command.
#
# There are two tunable parameters in the Redis LFU implementation: the
# counter logarithm factor and the counter decay time. It is important to
# understand what the two parameters mean before changing them.
#
# The LFU counter is just 8 bits per key, it's maximum value is 255, so Redis
# uses a probabilistic increment with logarithmic behavior. Given the value
# of the old counter, when a key is accessed, the counter is incremented in
# this way:
#
# 1. A random number R between 0 and 1 is extracted.
# 2. A probability P is calculated as 1/(old_value*lfu_log_factor+1).
# 3. The counter is incremented only if R < P.
#
# The default lfu-log-factor is 10. This is a table of how the frequency
# counter changes with a different number of accesses with different
# logarithmic factors:
#
# +--------+------------+------------+------------+------------+------------+
# | factor | 100 hits   | 1000 hits  | 100K hits  | 1M hits    | 10M hits   |
# +--------+------------+------------+------------+------------+------------+
# | 0      | 104        | 255        | 255        | 255        | 255        |
# +--------+------------+------------+------------+------------+------------+
# | 1      | 18         | 49         | 255        | 255        | 255        |
# +--------+------------+------------+------------+------------+------------+
# | 10     | 10         | 18         | 142        | 255        | 255        |
# +--------+------------+------------+------------+------------+------------+
# | 100    | 8          | 11         | 49         | 143        | 255        |
# +--------+------------+------------+------------+------------+------------+
#
# NOTE: The above table was obtained by running the following commands:
#
#   redis-benchmark -n 1000000 incr foo
#   redis-cli object freq foo
#
# NOTE 2: The counter initial value is 5 in order to give new objects a chance
# to accumulate hits.
#
# The counter decay time is the time, in minutes, that must elapse in order
# for the key counter to be divided by two (or decremented if it has a value
# less <= 10).
#
# The default value for the lfu-decay-time is 1. A Special value of 0 means to
# decay the counter every time it happens to be scanned.
#
 lfu-log-factor 10
 lfu-decay-time 1
  1. lfu-log-factor调整计数器counter的增长速度,lfu-log-factor越大,counter增长的越慢。
  2. lfu-decay-time以分钟为单位,控制counter的减少速度

Redis对象redisObject中的lru:

typedef struct redisObject {
    unsigned type:4;
    unsigned encoding:4;
    unsigned lru:LRU_BITS; /* LRU time (relative to global lru_clock) or
                            * LFU data (least significant 8 bits frequency
                            * and most significant 16 bits access time). */
    int refcount;
    void *ptr;
} robj;

当使用LFU时,lru的后8位保存计数器counter,前16位保存访问时间(分钟级)。

Redis的LFU算法中,为每个key维护一个计数器。每次key被访问的时候,计数器增大。计数器越大,可以约等于访问越频繁

当数据被访问时,更新该值:

robj *lookupKey(redisDb *db, robj *key, int flags) {
	...
    if (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {
        updateLFU(val);
    }
    ...
}

更新lru

/* Update LFU when an object is accessed.
 * Firstly, decrement the counter if the decrement time is reached.
 * Then logarithmically increment the counter, and update the access time. */
void updateLFU(robj *val) {
    unsigned long counter = LFUDecrAndReturn(val);
    counter = LFULogIncr(counter);
    val->lru = (LFUGetTimeInMinutes()<<8) | counter;
}

降低LFUDecrAndReturn 再增加计数器counter之前,先减少计数器。

/* If the object decrement time is reached decrement the LFU counter but
 * do not update LFU fields of the object, we update the access time
 * and counter in an explicit way when the object is really accessed.
 * And we will times halve the counter according to the times of
 * elapsed time than server.lfu_decay_time.
 * Return the object frequency counter.
 *
 * This function is used in order to scan the dataset for the best object
 * to fit: as we check for the candidate, we incrementally decrement the
 * counter of the scanned objects if needed. */
unsigned long LFUDecrAndReturn(robj *o) {
    unsigned long ldt = o->lru >> 8;
    unsigned long counter = o->lru & 255;
    unsigned long num_periods = server.lfu_decay_time ? LFUTimeElapsed(ldt) / server.lfu_decay_time : 0;
    if (num_periods)
        counter = (num_periods > counter) ? 0 : counter - num_periods;
    return counter;
}

函数首先取得高16 bits的最近更新时间ldt与低8 bits的计数器counter,然后根据配置的lfu_decay_time计算是否降低或降低多少。

LFUTimeElapsed用来计算当前时间与ldt的差值:

/* Return the current time in minutes, just taking the least significant
 * 16 bits. The returned time is suitable to be stored as LDT (last decrement
 * time) for the LFU implementation. */
// 当前时间(分钟级),取低16位
unsigned long LFUGetTimeInMinutes(void) {
    return (server.unixtime/60) & 65535;
}

/* Given an object last access time, compute the minimum number of minutes
 * that elapsed since the last access. Handle overflow (ldt greater than
 * the current 16 bits minutes time) considering the time as wrapping
 * exactly once. */
unsigned long LFUTimeElapsed(unsigned long ldt) {
    unsigned long now = LFUGetTimeInMinutes();
    if (now >= ldt) return now-ldt;
    return 65535-ldt+now;
}

当前时间(分钟级),取低16位,然后计算与ldt的差值now-ldt。当ldt > now时,默认为过了一个周期(16 bits,最大65535),取值65535-ldt+now

然后用差值与lfu_decay_time相除,LFUTimeElapsed(ldt) / server.lfu_decay_time,已过去n个lfu_decay_time,则将counter减少n,counter - num_periods。如果未过去lfu_decay_time,则counter不变。

这样做是为了防止一段时间内频繁访问的key,但是之后一段时间可能会很少被再访问到。只增加计数器,并不能体现计数器越大,约等于访问越频繁。

如果数据被频繁访问到,计数器就不会被减少。

**增加LFULogIncr

/* Logarithmically increment a counter. The greater is the current counter value
 * the less likely is that it gets really implemented. Saturate it at 255. */
uint8_t LFULogIncr(uint8_t counter) {
    if (counter == 255) return 255;
    double r = (double)rand()/RAND_MAX;
    double baseval = counter - LFU_INIT_VAL;
    if (baseval < 0) baseval = 0;
    double p = 1.0/(baseval*server.lfu_log_factor+1);
    if (r < p) counter++;
    return counter;
}

counter并不是简单的访问一次就+1,而是采用了一个0-1之间的控制因子pcounter最大值为255。取一个0-1之间的随机数rp比较,当r<p时,才增加counter。p取决于当前counter值与lfu_log_factor因子,counter值与lfu_log_factor因子越大,p越小,r<p的概率也越小,counter增长的概率也就越小,counter增长与访问次数呈现对数增长的趋势,随着访问次数越来越大,counter增长的越来越慢。官方给出了一个统计结果:

+--------+------------+------------+------------+------------+------------+
| factor | 100 hits   | 1000 hits  | 100K hits  | 1M hits    | 10M hits   |
+--------+------------+------------+------------+------------+------------+
| 0      | 104        | 255        | 255        | 255        | 255        |
+--------+------------+------------+------------+------------+------------+
| 1      | 18         | 49         | 255        | 255        | 255        |
+--------+------------+------------+------------+------------+------------+
| 10     | 10         | 18         | 142        | 255        | 255        |
+--------+------------+------------+------------+------------+------------+
| 100    | 8          | 11         | 49         | 143        | 255        |
+--------+------------+------------+------------+------------+------------+

新建的key问题 当创建新对象的时候,对象的counter如果为0,很容易就会被淘汰掉,所以Redis给每个新生key都设置一个初始counter

robj *createObject(int type, void *ptr) {
    robj *o = zmalloc(sizeof(*o));
    o->type = type;
    o->encoding = OBJ_ENCODING_RAW;
    o->ptr = ptr;
    o->refcount = 1;

    /* Set the LRU to the current lruclock (minutes resolution), or
     * alternatively the LFU counter. */
    if (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {
        // 初始counter为5
        o->lru = (LFUGetTimeInMinutes()<<8) | LFU_INIT_VAL;
    } else {
        o->lru = LRU_CLOCK();
    }
    return o;
}

LFU也有个待淘汰缓存池pool,它的逻辑和LRU中一模一样,只是idle的计算逻辑有点区别。

/* This is an helper function for freeMemoryIfNeeded(), it is used in order
 * to populate the evictionPool with a few entries every time we want to
 * expire a key. Keys with idle time smaller than one of the current
 * keys are added. Keys are always added if there are free entries.
 *
 * We insert keys on place in ascending order, so keys with the smaller
 * idle time are on the left, and keys with the higher idle time on the
 * right. */
void evictionPoolPopulate(int dbid, dict *sampledict, dict *keydict, struct evictionPoolEntry *pool) {
    ...
    // 随机样例
    count = dictGetSomeKeys(sampledict,samples,server.maxmemory_samples);
    for (j = 0; j < count; j++) {
        unsigned long long idle;
        ...
        /* Calculate the idle time according to the policy. This is called
         * idle just because the code initially handled LRU, but is in fact
         * just a score where an higher score means better candidate. */
        if (server.maxmemory_policy & MAXMEMORY_FLAG_LRU) {
            // LRU idle计算
            idle = estimateObjectIdleTime(o);
        } else if (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {
            /* When we use an LRU policy, we sort the keys by idle time
             * so that we expire keys starting from greater idle time.
             * However when the policy is an LFU one, we have a frequency
             * estimation, and we want to evict keys with lower frequency
             * first. So inside the pool we put objects using the inverted
             * frequency subtracting the actual frequency to the maximum
             * frequency of 255. */
             // LFU idle计算
            idle = 255-LFUDecrAndReturn(o);
        } else if (server.maxmemory_policy == MAXMEMORY_VOLATILE_TTL) {
            /* In this case the sooner the expire the better. */
            idle = ULLONG_MAX - (long)dictGetVal(de);
        } else {
            serverPanic("Unknown eviction policy in evictionPoolPopulate()");
        }

        ...
    }
}

idle = 255-LFUDecrAndReturn(o),计算idle之前,先降低counter,防止上面说的,一段时间频繁访问的key,之后长时间未被访问。

后记

相信应该有很多人以为,Redis键过期了,如果被访问到,判断是否过期,如果已过期,删除掉。如果未被访问到,则由缓存淘汰策略回收。其实这是不对的。Redis的缓存过期是通过主动或被动删除的。缓存淘汰策略只是可能刚好筛选到了过期的key,它不管这个key是否已过期了,只要筛选到了,就删除这个key。

Redis缓存淘汰策略中的LRULFU,都维护一个待淘汰缓存池pool,都是取随机maxmemory-samples个样例数据,计算它们的idle(计算方式不一样),然后判断是否可以插入到pool中,之后取poolidle最大的key淘汰掉。

PS。累~~,觉得不错的,就点个赞吧^_^