TCMalloc是谷歌开发的内存分配器。Kudu版本为 1.10.0,有 10GB 的 block cache,一直存在 top 命令看到的 kudu-tserver 进程占用内存虚高的情况。在 kudu-tserver:8050/memz 中查看内存使用详情:
------------------------------------------------
MALLOC: 13606721656 (12976.4 MiB) Bytes in use by application
MALLOC: + 4908195840 ( 4680.8 MiB) Bytes in page heap freelist
MALLOC: + 444714736 ( 424.1 MiB) Bytes in central cache freelist
MALLOC: + 286720 ( 0.3 MiB) Bytes in transfer cache freelist
MALLOC: + 28481688 ( 27.2 MiB) Bytes in thread cache freelists
MALLOC: + 52559872 ( 50.1 MiB) Bytes in malloc metadata
MALLOC: ------------
MALLOC: = 19040960512 (18158.9 MiB) Actual memory used (physical + swap)
MALLOC: + 919805952 ( 877.2 MiB) Bytes released to OS (aka unmapped)
MALLOC: ------------
MALLOC: = 19960766464 (19036.1 MiB) Virtual address space used
MALLOC:
MALLOC: 308671 Spans in use
MALLOC: 178 Thread heaps in use
MALLOC: 8192 Tcmalloc page size
------------------------------------------------
Call ReleaseFreeMemory() to release freelist memory to the OS (via madvise()).
Bytes released to the OS take up virtual address space but no physical memory.
--tcmalloc_max_free_bytes_percentage默认为10,12976 * 10% = 1297MB,小于4680MB,正常情况下应该是会调用GcTcmalloc(),释放 4680MB - 1297MB 给操作系统。但是 scan / compactions / wal 不使用memtrackers,也就无法触发MemTracker::Release,从而无法调用GcTcmalloc()。如果 scan 比较多可能会出现这种情况,1.11 通过增加了一个后台定时GC的线程解决了这个问题。