名词解释

arena

go将堆地址空间分为一个个arena，在32位系统中，一个area为4MB大小，在64位系统中，一个arena为64MB大小。每个area中又分为8192个page，每个page大小为64MB/8192=8KB

span

span 是go分配内存的基本单元。
go将内存快分配为67个级别的span，0代表特殊的大对象，大小不是确定的
class: spanclass，表示该soan可存储对象的类型
bytes/obj：该span中每个元素的大小
bytes/span：该span所占的字节数
objects：该span可分配的元素的个数，也就是nelems,也是(bytes/span)/(bytes/obj),比如class为1的span，objects=(bytes/span)/(bytes/obj)=(8192)/(8) = 1024 个 tail waste：每个span产生的碎片，比如class为5的span，只能存放170个大小为48字节的元素，所以碎片大小=8192-170*48=32字节 max waste：最大浪费比，比如class=5的span,当存储刚刚满足此规格大小的元素时(33byte),最多浪费

(48−33)*170+32
-------------- = 0.31518
     8192

当需要为具体大小的对象分配内存时，并不是直接分配span，而是先找到大小最相近的span再分配

class	bytes/obj	bytes/span	objects	tail waste	max waste
1	8	8192	1024	0	87.50%
2	16	8192	512	0	43.75%
3	24	8192	341	0	29.24%
4	32	8192	256	0	46.88%
5	48	8192	170	32	31.52%
6	64	8192	128	0	23.44%
7	80	8192	102	32	19.07%
…	…	…	…	…	…
67	32768	32768	1	0	12.50%

比如要分配17字节的对象，对分配比17字节大并且最接近他的元素级别位3，最终分配了24字节，这种方式不可避免的会产生内存的浪费

area,span,page,obj的关系如上图

数据结构

mheap

mheap用于管理整个堆内存

heapArena

一个heapArena对应一个arena

mspan

一个mspan对应一个span

type mspan struct {
   next *mspan     // next span in list, or nil if none
   prev *mspan     // previous span in list, or nil if none
   list *mSpanList // For debugging. TODO: Remove.

   startAddr uintptr // address of first byte of span aka s.base()
   npages    uintptr // number of pages in span

   manualFreeList gclinkptr // list of free objects in mSpanManual spans

   nelems uintptr // number of object in the span.
   allocCache uint64
   allocBits  *gcBits
   gcmarkBits *gcBits

   sweepgen    uint32
   divMul      uint32        // for divide by elemsize
   allocCount  uint16        // number of allocated objects
   spanclass   spanClass     // size class and noscan (uint8)
   state       mSpanStateBox // mSpanInUse etc; accessed atomically (get/set methods)
   needzero    uint8         // needs to be zeroed before allocation
   elemsize    uintptr       // computed from sizeclass or from npages
   limit       uintptr       // end of data in span
   speciallock mutex         // guards specials list
   specials    *special      // linked list of special records sorted by offset.
}

next 指向下一个 span 的指针，为 nil 表示没有
prev 指向上一个 span 的指针，与 next 相反
list 指向 mSpanList，调试使用，以后会废弃
startAddr span第一个字节地址，可通过 s.base() 函数读取
npages span中page的数量
nelems span中对象数
spanclass spanClass类型
allocBits 标记span中的elem哪些是“被使用”了的，哪些是未被使用的；清除后将释放 allocBits ，并将 allocBits 的值设置为 gcmarkBits；
gcmarkBits 标记span中的elem哪些是“被标记”了的，哪些是未被标记的；
spanclass spanClass类型；
state 由于协程栈也是从堆上分配的，也在mheap管理的这些span中，mspan.spanState会记录该span是用作堆内存，还是用作栈内存；

每个 mspan 都对应两个位图标记：mspan.allocBits 和 mspan.gcmarkBits。

allocBits

allocBits中每一位用于标记一个对象存储单元是否已分配。

allocBits的大小与nelems有关，如下代码

s.allocBits = newAllocBits(s.nelems)

// newMarkBits returns a pointer to 8 byte aligned bytes
// to be used for a span's mark bits.
func newMarkBits(nelems uintptr) *gcBits {
   blocksNeeded := uintptr((nelems + 63) / 64)
   bytesNeeded := blocksNeeded * 8
   。。。
}

newMarkBits返回8字节对齐的指针，所以如果nelems大小为0~64，则allocBits指向的地址为8字节,如果nelems大小为65~128，则allocBits指向的地址为16字节。以nelems=64为例,mspan中一共有64个obj，allocBits为8字节，共64位，正好每一位都可以用来标记一个obj。

gcmarkBits

gcmarkBits中每一位用于标记一个对象是否存活。结构和allocBits类似

到GC清扫阶段，会将标记好的gcmarkBits赋值给allocBits，并重新分配一段清零的内存给gcmarkBits。gcmarkBits中为0的(未被GC的)，就能在gcmarkBits中被回收利用了。

s.allocBits = s.gcmarkBits
s.gcmarkBits = newMarkBits(s.nelems)

mcache

type mcache struct {
   // The following members are accessed on every malloc,
   // so they are grouped here for better caching.
   nextSample uintptr // trigger heap sample after allocating this many bytes
   scanAlloc  uintptr // bytes of scannable heap allocated
   tiny       uintptr //指向微小对象
   tinyoffset uintptr // tiny已经使用空间的偏移量
   tinyAllocs uintptr // tiny分配了微小对象的数量

   // The rest is not accessed on every malloc.

   alloc [numSpanClasses]*mspan // spans to allocate from, indexed by spanClass

   stackcache [_NumStackOrders]stackfreelist

   flushGen uint32
}

const (
   numSpanClasses = _NumSizeClasses << 1 = 68 << 1 = 136
   tinySpanClass  = spanClass(tinySizeClass<<1 | 1) = 5 // 所以微小对象的tinySpanClass为5,对应的spanclass位2
)

mcache是没有锁的，因为在GMP模型中，一个P只有一个mcache，并且一个P同时也只持有一个G,所以不会存在并发访问同一个mcache的情况。
每个SizeClasse对应两个mspan，一个含指针，一个不含指针。 mcache在初始化时候，是没有mspan资源的，在使用过程中动态申请，不断填充[numSpanClasses]*mspan,每个元素的mspan是一个双向链表

mcentral

每一种span都有一种对应的mcentral

type mcentral struct {
    spanclass spanClass  // 代表当前spanclass是多少 0～136
    partial [2]spanSet // list of spans with a free object
    full    [2]spanSet // list of spans with no free objects
}

spanClass 指当前规格大小
partial 存在空闲对象spans列表
full 无空闲对象spans列表

其中 partial 和 full 都包含两个 spans 集数组。一个用在扫描 spans,另一个用在未扫描spans。在每轮GC期间都扮演着不同的角色。mheap_.sweepgen 在每轮gc期间都会递增2。

再详细点如下所示：

heapArena

type heapArena struct {
   bitmap [heapArenaBitmapBytes]byte
   spans [pagesPerArena]*mspan
   pageInUse [pagesPerArena / 8]uint8
   pageMarks [pagesPerArena / 8]uint8
   pageSpecials [pagesPerArena / 8]uint8
   checkmarks *checkmarksMap

   // zeroedBase marks the first byte of the first page in this
   // arena which hasn't been used yet and is therefore already
   // zero. zeroedBase is relative to the arena base.
   // Increases monotonically until it hits heapArenaBytes.
   //
   // This field is sufficient to determine if an allocation
   // needs to be zeroed because the page allocator follows an
   // address-ordered first-fit policy.
   //
   // Read atomically and written with an atomic CAS.
   zeroedBase uintptr
}

bitmap

bitmap：在bitmap中，每1byte用来标记在arena中4个指针大小空间，比如arean为512GB，bitmap大小为 512GB/(4*8B) = 16GB.
每1byte,0~3bit用来表示这四个指针大小空间存的对象是指针还是标量，4～7bit用来表示这四个指针大小空间是否需要扫描

如上图，有一个slice，其中ptr为指针，len和cap为值类型，所以0bit为1，4bit也为1

pageInUse，pageMarks

只标记处于使用状态的span的第一个page

pageMarks和pageInUse相似

spans

spans：是一个元素为*mspan的数组，长度为8192，正好为arena中page的数量,可以用于定位一个page对应的mspan在哪

mheap

type mheap struct {

   lock  mutex
   pages pageAlloc // page allocation data structure

   sweepgen     uint32 // sweep generation, see comment in mspan; written during STW
   sweepDrained uint32 // all spans are swept or are being swept
   sweepers     uint32 // number of active sweepone calls

   allspans []*mspan // all spans out there

   _ uint32 // align uint64 fields on 32-bit for atomics

  
   pagesInUse         uint64  // pages of spans in stats mSpanInUse; updated atomically
   pagesSwept         uint64  // pages swept this cycle; updated atomically
   pagesSweptBasis    uint64  // pagesSwept to use as the origin of the sweep ratio; updated atomically
   sweepHeapLiveBasis uint64  // value of gcController.heapLive to use as the origin of sweep ratio; written with lock, read without
   sweepPagesPerByte  float64 // proportional sweep ratio; written with lock, read without
  
   scavengeGoal uint64

   // Page reclaimer state
   reclaimIndex uint64
  
   // This is accessed atomically.
   reclaimCredit uintptr

   
   arenas [1 << arenaL1Bits]*[1 << arenaL2Bits]*heapArena

   // heapArenaAlloc is pre-reserved space for allocating heapArena
   // objects. This is only used on 32-bit, where we pre-reserve
   // this space to avoid interleaving it with the heap itself.
   heapArenaAlloc linearAlloc

   // arenaHints is a list of addresses at which to attempt to
   // add more heap arenas. This is initially populated with a
   // set of general hint addresses, and grown with the bounds of
   // actual heap arena ranges.
   arenaHints *arenaHint

   // arena is a pre-reserved space for allocating heap arenas
   // (the actual arenas). This is only used on 32-bit.
   arena linearAlloc

   // allArenas is the arenaIndex of every mapped arena. This can
   // be used to iterate through the address space.
   //
   // Access is protected by mheap_.lock. However, since this is
   // append-only and old backing arrays are never freed, it is
   // safe to acquire mheap_.lock, copy the slice header, and
   // then release mheap_.lock.
   allArenas []arenaIdx

   // sweepArenas is a snapshot of allArenas taken at the
   // beginning of the sweep cycle. This can be read safely by
   // simply blocking GC (by disabling preemption).
   sweepArenas []arenaIdx

   // markArenas is a snapshot of allArenas taken at the beginning
   // of the mark cycle. Because allArenas is append-only, neither
   // this slice nor its contents will change during the mark, so
   // it can be read safely.
   markArenas []arenaIdx

   // curArena is the arena that the heap is currently growing
   // into. This should always be physPageSize-aligned.
   curArena struct {
      base, end uintptr
   }

   _ uint32 // ensure 64-bit alignment of central

   // central free lists for small size classes.
   // the padding makes sure that the mcentrals are
   // spaced CacheLinePadSize bytes apart, so that each mcentral.lock
   // gets its own cache line.
   // central is indexed by spanClass.
   central [numSpanClasses]struct {
      mcentral mcentral
      pad      [cpu.CacheLinePadSize - unsafe.Sizeof(mcentral{})%cpu.CacheLinePadSize]byte
   }

   spanalloc             fixalloc // allocator for span*
   cachealloc            fixalloc // allocator for mcache*
   specialfinalizeralloc fixalloc // allocator for specialfinalizer*
   specialprofilealloc   fixalloc // allocator for specialprofile*
   specialReachableAlloc fixalloc // allocator for specialReachable
   speciallock           mutex    // lock for special record allocators.
   arenaHintAlloc        fixalloc // allocator for arenaHints

   unused *specialfinalizer // never set, just here to force the specialfinalizer type into DWARF
}

go 内存分配

名词解释

arena

span

数据结构

mheap

heapArena

mspan

allocBits

gcmarkBits

mcache

mcentral

heapArena

bitmap

pageInUse，pageMarks

spans

mheap