内存池概念以及设计参考

986 阅读4分钟

概念

内存池,是使用池进行内存管理,允许动态分配固定大小的内存块。 图片 将内存池和malloc和C++的new运算符相比教,后者由于块大小可变而导致碎片化,由于性能原因不建议在实时系统中使用。一种更有效的解决方案是预分配多个具有相同大小的内存块,称为内存池。应用程序可以在运行时分配,访问和释放由句柄表示的内存块。

一些系统,例如Web服务器Nginx,使用术语“内存池”来指代一组可变大小的分配,这些分配的内存以后可以一次性全部释放。

Any kind of "pool" is really just resources you've acquired/initialized in advance so that they're already ready to go, not allocated on the fly with each client request. When clients finish using them, the resource returns to the pool instead of being destroyed.

Memory pools are basically just memory you've allocated in advance (and typically in big blocks). For example, you might allocate 4 kilobytes of memory in advance. When a client requests 64 bytes of memory, you just hand them a pointer to an unused space in that memory pool for them to read and write whatever they want. When the client is done, you can just mark that section of memory as being unused again.

简单的内存池实现

一个简单的内存池模块可以在编译时分配三个池。应用程序可以通过以下接口分配,访问和释放内存:

从池中分配内存。该函数将确定所需块所在的池。如果该池的所有块都已保留,则该函数将尝试在下一个更大的池中找到一个。

获取对分配的内存的访问指针。

释放以前分配的内存块。

Memory pool vs malloc

优点:

内存池中成千上万个对象的内存释放只是一个操作,如果使用malloc为每个对象分配内存,则需要一个接一个地释放。

可以将内存池分为分层的树状结构,这适用于特殊的编程结构,例如循环和递归。

固定大小的块内存池不需要为每个分配存储元数据(诸如已分配块大小之类的)。特别是对于小块分配,这可以节省大量空间。

保证了实时系统上的确定性行为,避免出现内存不足错误。

缺点:

内存池可能需要为部署它们的应用程序进行调整。

内存池设计相关参考

默认内存管理缺点

利用默认的内存管理操作符new/delete和函数malloc()/free()在堆上分配和释放内存会有一些额外的开销。应用程序频繁地在堆上分配和释放内存,会导致性能的损失。并且会使系统中出现大量的内存碎片,降低内存的利用率。

内存分配算法

伙伴算法

简介:The buddy memory allocation technique is a memory allocation algorithm that divides memory into partitions to try to satisfy a memory request as suitably as possible. This system makes use of splitting memory into halves to try to give a best fit. According to Donald Knuth, the buddy system was invented in 1963 by Harry Markowitz, and was first described by Kenneth C. Knowlton (published 1965).

slab算法

简介:The basic idea behind the slab allocator is to have caches of commonly used objects kept in an initialised state available for use by the kernel. Without an object based allocator, the kernel will spend much of its time allocating, initialising and freeing the same object. The slab allocator aims to to cache the freed object so that the basic structure is preserved between uses

开源内存池实现

nginx的内存池

github链接:github.com/nginx/nginx

简介:nginx [engine x] is an HTTP and reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy server, originally written byIgor Sysoev. For a long time, it has been running on many heavily loaded Russian sites includingYandex,Mail.Ru,VK, andRambler. According to Netcraft, nginx served or proxied25.20% busiest sites in November 2020.

jemalloc

github链接:github.com/jemalloc/je…

简介:jemalloc is a general purpose malloc(3) implementation that emphasizesfragmentation avoidance and scalable concurrency support. jemalloc first cameinto use as the FreeBSD libc allocator in 2005, and since then it has found itsway into numerous applications that rely on its predictable behavior.

tcmalloc

github链接:github.com/google/tcma…

简介:TCMalloc is Google's customized implementation of C's malloc() and C++'s operator new used for memory allocation within our C and C++ code. TCMalloc is a fast, multi-threaded malloc implementation.

Boost Pool Library

链接:www.boost.org/doc/libs/1_…

简介:Pool allocation is a memory allocation scheme that is very fast, but limited in its usage. The Boost Pool library is a header file library. That means there is no .lib, .dll, or .so to build; just add the Boost directory to your compiler's include file path.

本文参考资料

[1] 内存池的设计

chensongpoixs.github.io/2019/01/11/…

[2] 如何设计内存池?

www.zhihu.com/question/25…

[3] What are the differences between (and reasons to choose) tcmalloc/jemalloc and memory pools?

stackoverflow.com/questions/9…

[4] Memory_pool

en.wikipedia.org/wiki/Memory…

[5] A Scalable Concurrent malloc(3) Implementation for FreeBSD

people.freebsd.org/~jasone/jem…

[6] Chapter 8 Slab Allocator

www.kernel.org/doc/gorman/…

[7] Memory pool

en.wikipedia.org/wiki/Memory…

[8] What are the usual im­ple­men­ta­tion de­tails be­hind mem­ory pools?

stackoverflow.com/questions/3…

来源: 公众号「Jacen的技术笔记」

作者: Jacen

原文链接: 内存池概念以及设计参考