go slice扩容机制

362 阅读3分钟

slice

  1. 定义 go源码之中搜索"type slice"即可找到,最好在goland类似的ide中,在libaray之中搜索
 // 24个字节(64位机器),slice运行时结构
type SliceHeader struct {
 Data uintptr //引用数组指针地址
 Len int   // 切片的目前使用长度
 Cap int   // 切片的容量
}
  1. 验证 通过更改sliceHeader的结构可以动态改变slice的内容
func main() {
   // slice也可以通过new的方式创建
   s := *new([]int)
   // 获取slice 的运行时结构
   rs1 := (*reflect.SliceHeader)(unsafe.Pointer(&s))
   // addr:0, len:0, cap:0
   fmt.Printf("addr:%d, len:%d, cap:%d\n", rs1.Data, rs1.Len,rs1.Cap)
    // make初始化
   s2:=make([]int,0)
   rs2 := (*reflect.SliceHeader)(unsafe.Pointer(&s2))
   // str:824634244816, data addr:0, len:0
   fmt.Printf("addr:%d, len:%d, cap:%d\n", rs2.Data, rs2.Len,rs2.Cap)
}
  1. 扩容 使用go tool compile -S main.go或者go build -S main.go得到汇编代码, ,我们得到append调用的是runtime.growslice,具体过程有两部分
  • 根据cap(old)和len(addelems)算出cap适应值 .append.png
// func growslice(et *_type, old slice, cap int) slice
// 扩容过程cap = len(old)+len(addelems)
newcap := old.cap
doublecap := newcap + newcap
// 1.如果cap>2*cap(old),return cap
if cap > doublecap {
   newcap = cap
} else {
// 2.如果cap(old)<1024,return cap = 2*cap(old)
//   否则 return cap = 1.25*cap(old)
   if old.cap < 1024 {
      newcap = doublecap
   } else {
      // Check 0 < newcap to detect overflow
      // and prevent an infinite loop.
      for 0 < newcap && newcap < cap {
         newcap += newcap / 4
      }
      // Set newcap to the requested cap when
      // the newcap calculation overflowed.
      if newcap <= 0 {
         newcap = cap
      }
   }
}
  • 内存对齐 go内存管理也是pageblock,datablock形式,
// runtime.growslice
switch {
case et.size == 1:
   lenmem = uintptr(old.len)
   newlenmem = uintptr(cap)
   capmem = roundupsize(uintptr(newcap))
   overflow = uintptr(newcap) > maxAlloc
   newcap = int(capmem)
case et.size == sys.PtrSize:
  ...
case isPowerOfTwo(et.size):
   var shift uintptr
   if sys.PtrSize == 8 {
      // Mask shift for better code generation.
      shift = uintptr(sys.Ctz64(uint64(et.size))) & 63
   } else {
      shift = uintptr(sys.Ctz32(uint32(et.size))) & 31
   }
   lenmem = uintptr(old.len) << shift
   newlenmem = uintptr(cap) << shift
   capmem = roundupsize(uintptr(newcap) << shift)
   overflow = uintptr(newcap) > (maxAlloc >> shift)
   newcap = int(capmem >> shift)
default:
   ...
}
// src/runtime/msize.go:13
func roundupsize(size uintptr) uintptr {
    if size < _MaxSmallSize {
        if size <= smallSizeMax-8 {
            return uintptr(class_to_size[size_to_class8[(size+smallSizeDiv-1)/smallSizeDiv]])
        } else {
            //……
        }
    }
    //……
}
// 常量定义
const _MaxSmallSiz
e = 32768
const smallSizeMax = 1024
const smallSizeDiv = 8
  • go每一页的大小是8kb,对datablock划分成不同大小的内存块,除了最小的8b,其余的大 小都是8*2n,即8,16,32,48,...32768,具体规则间隔为8,16,32,64,128...,
  • rounduptosize为向上取整函数,具体流程如下:
// runtime/mksizeclasses.go
// 1. el.size = 1, 直接调用roundup扩容
   a := []byte{1,0,1,2}   //4
   a = append(a, 1)       //8
type D struct{
   age byte
   name string
}
fmt.Println(unsafe.Sizeof(D{1,"1323"}))  // 24
d := []D{
   {1,"12323432231"},
   {2,"234"},
   {5,"567"},
   {6,"678"},
   {6,"678"},
}
d = append(d,D{4,"456"},
D{6,"678"},
D{6,"678"},
D{6,"678"},
D{6,"678"},
D{6,"678"})
fmt.Println("cap of d is ",cap(d))
e := []string{"123", "456"}
var s string
e = append(e,"123")
fmt.Println("cap of e is ",cap(e))  //12
fmt.Println(unsafe.Sizeof(s))       // 16

上述例子中struct切片 一个元素的大小是24b,首次适应得到的cap是12,占用的大小是512b,在go对象列表中恰好有这个取值,所以cap = 12,而string切片例子中,string占用16个字节大小,首次适应得到的cap是4,最终roundup(16*4)=64,所以最终cap = 64/16。 内存分配超过32768怎么办呢??roundupsize函数,如下,会按照8192b的大小增加

// _PageSize 8192
if size+_PageSize < size {
   return size
}
return alignUp(size, _PageSize)

总结:slice的扩容,首先计算首次适应的adapt(cap),内存对齐alignof(cap*el.size)得到最终mem,mem为两种情况,小于32768则查go object表对齐,大于32768则mem按照8192b的方式增加,最终cap = mem/el.size