searchusermenu
  • 发布文章
  • 消息中心
点赞
收藏
评论
分享
原创

Golang深入浅出——内存管理

2023-09-27 09:49:12
5
0

golang借鉴TCMalloc内存分配机制,实现了自己基于多级缓存的告诉内存分配机制。

分级分配

对象分级

golang内存分配器,根据申请分给的大小,选择不同的处理逻辑,运行时根据对象的大小将对象分成微对象、小对象和大对象三种。

类别 大小
微对象 (0,16B)
小对象 [16B,32KB]
大对象 (32KB, +∞)

缓存分级

golang会讲内存分成不同的级别进行管理,包括线程缓存,中心缓存,页堆三级缓存

线程缓存:属于每一个独立的线程,为一个线程独立拥有,使用过程中不需要加锁

中心缓存:当线程缓存不能满足需求时,runtime会使用中心缓存,中心缓存为全局所有,使用需要加锁;

页堆缓存:当中心缓存不足或遇到32KB以上的大对象时,会选择在页堆分配内存;

 

内存管理组件

golang的内存分配器包含内存管理单元、线程缓存、中心缓存、页堆;

内存管理单元

内存管理单元是Golang内存管理的基本单元,通过内部next和prev两个指针将内存管理单元构成一个双向链表。

数据结构

 next *mspan     // next span in list, or nil if none
 prev *mspan     // previous span in list, or nil if none
 npages    uintptr // number of pages in span
 allocBits  *gcBits
 gcmarkBits *gcBits
。。。
 // sweep generation:
 // if sweepgen == h->sweepgen - 2, the span needs sweeping
 // if sweepgen == h->sweepgen - 1, the span is currently being swept
 // if sweepgen == h->sweepgen, the span is swept and ready to use
 // if sweepgen == h->sweepgen + 1, the span was cached before sweep began and is still cached, and needs sweeping
 // if sweepgen == h->sweepgen + 3, the span was swept and then cached and is still cached
 // h->sweepgen is incremented by 2 after every GC
。。。

线程缓存

线程缓存是线程私有的缓存,会与线程上的处理器一一绑定,主要用来缓存用户程序申请的微小对象。

每一个线程缓存都持有68*2个runtime.mspan,这些内存管理单元都保存在线程缓存的alloc字段;线程缓存初始化时是不分配runtime.mspn的,当用户程序申请内存时,才会从上一级组件获取新的runtime.mspan以满足内存分配的需求。

数据结构

type mcache struct {
 nextSample uintptr // trigger heap sample after allocating this many bytes
 scanAlloc  uintptr // bytes of scannable heap allocated

 tiny       uintptr
 tinyoffset uintptr
 tinyAllocs uintptr
 alloc [numSpanClasses]*mspan // spans to allocate from, indexed by spanClass
 flushGen uint32
}

 

中心缓存

中心缓存和线程缓存不同,中心缓存是全局可见的,所以访问中心缓存的内存管理的单元需要使用互斥锁;每个中心缓存都会管理一个具体跨度类别的内存管理单元(golang根据对象的大小,将对象分为不同的跨度类别),它会持有2个不同的runtime.spanSet,分别存储包含空闲对象和不包含空闲对象的内存管理单元。

数据结构

type mcentral struct {
 spanclass spanClass
 partial [2]spanSet // list of spans with a free object
 full    [2]spanSet // list of spans with no free objects
}

页堆

页堆中包含2个中心缓存,一个是需要scan拥有68个跨度的中心缓存,另外一个是不需要scan拥有68个跨度的中心缓存。

数据结构

type mheap struct {
	lock      mutex
	pages     pageAlloc // page allocation data structure
	sweepgen  uint32    // sweep generation, see comment in mspan; written during STW
	sweepdone uint32    // all spans are swept
	sweepers  uint32    // number of active sweepone calls

	allspans []*mspan // all spans out there

	pagesInUse         uint64  // pages of spans in stats mSpanInUse; updated atomically
	pagesSwept         uint64  // pages swept this cycle; updated atomically
	pagesSweptBasis    uint64  // pagesSwept to use as the origin of the sweep ratio; updated atomically
	sweepHeapLiveBasis uint64  // value of heap_live to use as the origin of sweep ratio; written with lock, read without
	sweepPagesPerByte  float64 // proportional sweep ratio; written with lock, read without
	arenas [1 << arenaL1Bits]*[1 << arenaL2Bits]*heapArena
	heapArenaAlloc linearAlloc
	allArenas []arenaIdx

	// sweepArenas is a snapshot of allArenas taken at the
	// beginning of the sweep cycle. This can be read safely by
	// simply blocking GC (by disabling preemption).
	sweepArenas []arenaIdx

	// markArenas is a snapshot of allArenas taken at the beginning
	// of the mark cycle. Because allArenas is append-only, neither
	// this slice nor its contents will change during the mark, so
	// it can be read safely.
	markArenas []arenaIdx

	// curArena is the arena that the heap is currently growing
	// into. This should always be physPageSize-aligned.
	curArena struct {
		base, end uintptr
	}

	central [numSpanClasses]struct {
		mcentral mcentral
		pad      [cpu.CacheLinePadSize - unsafe.Sizeof(mcentral{})%cpu.CacheLinePadSize]byte
	}
}
0条评论
0 / 1000
s****n
5文章数
0粉丝数
s****n
5 文章 | 0 粉丝
原创

Golang深入浅出——内存管理

2023-09-27 09:49:12
5
0

golang借鉴TCMalloc内存分配机制,实现了自己基于多级缓存的告诉内存分配机制。

分级分配

对象分级

golang内存分配器,根据申请分给的大小,选择不同的处理逻辑,运行时根据对象的大小将对象分成微对象、小对象和大对象三种。

类别 大小
微对象 (0,16B)
小对象 [16B,32KB]
大对象 (32KB, +∞)

缓存分级

golang会讲内存分成不同的级别进行管理,包括线程缓存,中心缓存,页堆三级缓存

线程缓存:属于每一个独立的线程,为一个线程独立拥有,使用过程中不需要加锁

中心缓存:当线程缓存不能满足需求时,runtime会使用中心缓存,中心缓存为全局所有,使用需要加锁;

页堆缓存:当中心缓存不足或遇到32KB以上的大对象时,会选择在页堆分配内存;

 

内存管理组件

golang的内存分配器包含内存管理单元、线程缓存、中心缓存、页堆;

内存管理单元

内存管理单元是Golang内存管理的基本单元,通过内部next和prev两个指针将内存管理单元构成一个双向链表。

数据结构

 next *mspan     // next span in list, or nil if none
 prev *mspan     // previous span in list, or nil if none
 npages    uintptr // number of pages in span
 allocBits  *gcBits
 gcmarkBits *gcBits
。。。
 // sweep generation:
 // if sweepgen == h->sweepgen - 2, the span needs sweeping
 // if sweepgen == h->sweepgen - 1, the span is currently being swept
 // if sweepgen == h->sweepgen, the span is swept and ready to use
 // if sweepgen == h->sweepgen + 1, the span was cached before sweep began and is still cached, and needs sweeping
 // if sweepgen == h->sweepgen + 3, the span was swept and then cached and is still cached
 // h->sweepgen is incremented by 2 after every GC
。。。

线程缓存

线程缓存是线程私有的缓存,会与线程上的处理器一一绑定,主要用来缓存用户程序申请的微小对象。

每一个线程缓存都持有68*2个runtime.mspan,这些内存管理单元都保存在线程缓存的alloc字段;线程缓存初始化时是不分配runtime.mspn的,当用户程序申请内存时,才会从上一级组件获取新的runtime.mspan以满足内存分配的需求。

数据结构

type mcache struct {
 nextSample uintptr // trigger heap sample after allocating this many bytes
 scanAlloc  uintptr // bytes of scannable heap allocated

 tiny       uintptr
 tinyoffset uintptr
 tinyAllocs uintptr
 alloc [numSpanClasses]*mspan // spans to allocate from, indexed by spanClass
 flushGen uint32
}

 

中心缓存

中心缓存和线程缓存不同,中心缓存是全局可见的,所以访问中心缓存的内存管理的单元需要使用互斥锁;每个中心缓存都会管理一个具体跨度类别的内存管理单元(golang根据对象的大小,将对象分为不同的跨度类别),它会持有2个不同的runtime.spanSet,分别存储包含空闲对象和不包含空闲对象的内存管理单元。

数据结构

type mcentral struct {
 spanclass spanClass
 partial [2]spanSet // list of spans with a free object
 full    [2]spanSet // list of spans with no free objects
}

页堆

页堆中包含2个中心缓存,一个是需要scan拥有68个跨度的中心缓存,另外一个是不需要scan拥有68个跨度的中心缓存。

数据结构

type mheap struct {
	lock      mutex
	pages     pageAlloc // page allocation data structure
	sweepgen  uint32    // sweep generation, see comment in mspan; written during STW
	sweepdone uint32    // all spans are swept
	sweepers  uint32    // number of active sweepone calls

	allspans []*mspan // all spans out there

	pagesInUse         uint64  // pages of spans in stats mSpanInUse; updated atomically
	pagesSwept         uint64  // pages swept this cycle; updated atomically
	pagesSweptBasis    uint64  // pagesSwept to use as the origin of the sweep ratio; updated atomically
	sweepHeapLiveBasis uint64  // value of heap_live to use as the origin of sweep ratio; written with lock, read without
	sweepPagesPerByte  float64 // proportional sweep ratio; written with lock, read without
	arenas [1 << arenaL1Bits]*[1 << arenaL2Bits]*heapArena
	heapArenaAlloc linearAlloc
	allArenas []arenaIdx

	// sweepArenas is a snapshot of allArenas taken at the
	// beginning of the sweep cycle. This can be read safely by
	// simply blocking GC (by disabling preemption).
	sweepArenas []arenaIdx

	// markArenas is a snapshot of allArenas taken at the beginning
	// of the mark cycle. Because allArenas is append-only, neither
	// this slice nor its contents will change during the mark, so
	// it can be read safely.
	markArenas []arenaIdx

	// curArena is the arena that the heap is currently growing
	// into. This should always be physPageSize-aligned.
	curArena struct {
		base, end uintptr
	}

	central [numSpanClasses]struct {
		mcentral mcentral
		pad      [cpu.CacheLinePadSize - unsafe.Sizeof(mcentral{})%cpu.CacheLinePadSize]byte
	}
}
文章来自个人专栏
文章 | 订阅
0条评论
0 / 1000
请输入你的评论
0
0