CLR Garbage Collection Inside Out Maoni Stephens FUN

CLR: Garbage Collection Inside Out Maoni Stephens FUN 421 Software Design Engineer Microsoft Corporation

Agenda Basics Generations Segments Allocation Collection Large Object Heap Different Flavors of GC Pinning Tools and Resources

Basics Generation 1 Generation 0 New Heap Begins with New Generation Accessible References Keep Objects Alive Compacts Referenced Objects Promoted to Older Generation New Allocations Rebuild New Generation

Agenda Basics Generations Segments Allocation Collection Large Object Heap Different Flavors of GC Pinning Tools and Resources

Generational GC Three generations Most objects die in gen 0 Gen 1 holds in the in flight data Gen 2 holds the long lived data Large object heap Gen 0 and Gen 1 – ephemeral generations

Agenda Basics Generations Segments Allocation Collection Large Object Heap Different Flavors of GC Pinning Tools and Resources

GC Heap Segments Unit of Virtual. Alloc When CLR is loaded, initial segments are allocated Additional segments reserved as needed Committed/Decommitted as needed in the segment Deleted when not in use VM hoarding feature in next CLR 1. 1 SP and CLR 2. 0 – STARTUP_HOARD_GC_VM startup flag

Generations And Segments Start from the ephemeral segment Ephemeral segment becomes a gen 2 segment Newly reserved segment becomes ephemeral Many gen 2 segments, only one ephemeral

Before GC #0 Gen 0 Before GC #1 Gen 0 Before GC #2 Gen 1 Gen 0 Before GC #100 Gen 2 Gen 1 Gen 0 Before GC #500 Gen 2 Gen 1 Gen 0

Agenda Basics Generations Segments Allocation Collection Large Object Heap Different Flavors of GC Pinning Tools and Resources

Allocation Cheap lock on UP; lock free on MP Moving a pointer forward Clearing the memory for new objects Register for finalization if applicable Objects allocated together are close together

Agenda Basics Generations Segments Allocation Collection Large Object Heap Different Flavors of GC Pinning Tools and Resources

Collection When Allocation. Not enough space in gen 0. Induced GC. By calling System. GC. Collect(). System pressure.

Collection How Suspend managed threads GC Resume managed threads Two phases of GC Mark Compact

Roots Stack Handle table Statics Older generation(s) Finalizer queue

Agenda Basics Generations Segments Allocation Collection Large Object Heap Different Flavors of GC Pinning Tools and Resources

Large Object Heap For objects that are 85, 000 bytes or larger Compacting large objects costs a lot So we only sweep (freelist) Many LOH segments Segments are handled similarly to small object heap segments Collection happens during gen 2 GCs

Collection – Cost GC takes time – “% time in GC” counter If objects die in gen 0 (survival rate is 0) it’s the ideal situation The longer the object lives before being dead, the worse (with exceptions) Gen 0 and gen 1 GCs should both be relatively cheap; gen 2 GCs could cost a lot LOH – different cost model Temporary large objects could be bad Should reuse if possible

Agenda Basics Generations Segments Allocation Collection Large Object Heap Different Flavors of GC Pinning Tools and Resources

Concurrent GC Why do we have concurrent GC For interactive applications How it’s done Trade some CPU and memory for shorter pause time Only for gen 2 Done on a concurrent GC thread How do you get concurrent GC On by default Can be turned off via hosting or config

Server GC Why do we have server GC – for server apps that Have a fairly consistent number of requests Require high scalibility and high throughput How it’s done One thread for each CPU, running at highest priority One ephemeral segment per CPU How do you get server GC Only on via config or hosting Hosts like ASP. NET and SQLCLR use server GC In CLR 1. 1 it’s in mscorsvr. dll; in CLR 2. 0 it’s in mscorwks. dll

WKS GC SVR GC Where it runs On the user thread On the GC threads that triggered GC running at highest priority On a multi proc machine One small object heap + One LOH N small object heaps + N LOHs On a uni proc machine WKS GC + concurrent GC (if not turned off) WKS GC + concurrent GC OFF

Agenda Basics Generations Segments Allocation Collection Large Object Heap Different Flavors of GC Pinning Tools and Resources

Pinning Why do we need pinning Interop with unmanaged code How objects get pinned Use of GCHandle of type GCHandle. Type. Pinned Allowed by language syntax, eg. fixed in c# Args get pinned by the interop frame Fragmentation problem

Fragmentation Problem Caused By Pinning Before GC X Gen 1 P P Gen 0 start After GC X Gen 2 Gen 1 start Gen 0 start After N more GCs Gen 2 Gen 1 start P P Gen 0 start

Without demotion Gen 2 P Gen 1 start P Gen 0 start With demotion Gen 2 P Gen 1 start Gen 0 start P

Without Segment Reuse Before GC After GC Seg 0 Seg 1 Seg 2 Seg 0 Gen 2 Seg 1 Gen 2 Eph Gen 1 Seg 3 P Gen 0 G en 2 Seg 2 Gen 2 Seg 3 Gen 2 P Old Eph New Eph G en 2 Gen 1 P P

With Segment Reuse Before GC After GC Seg 0 Seg 1 Seg 2 Seg 0 Gen 2 Seg 1 Gen 2 Seg 2 Gen 2 Seg 3 Gen 2 Eph Gen 1 P Gen 0 Gen 2 P Old Eph Old Seg 3, New Eph G en 2 Gen 2 P Gen 2 Gen 1 P

What The User Code Can Do To Help Best patterns Pinning for a short time is cheap. Create pinned buffers at the beginning that will stay in regions that are very compacted Create pinned buffers that stay together instead of scattered around Techniques Allocate a pinned big buffer (byte array on the LOH), give out a chunk at a time Allocate a pool of small buffers and hand one out when needed

void M(byte[] b) { // if the buffer is already in gen 2 it’s unlikely to move. if(GC. Get. Generation(b) == GC. Max. Generation || // avoid copying if the buffer is too big b. Length > 8 * 1024) { Real. M(b); } // Get. Buffer will allocate one if none is free. byte[] Temp. Buffer = Pool. Get. Buffer(); Real. M(Temp. Buffer); Copy. Back. To. User. Buffer(Temp. Buffer, b); Pool. Release(Temp. Buffer); }

Managed Caches Design patterns Don’t do it in the finalizer Use combination of weak and strong references – could convert strong to weak after certain period Cache tuning Hit rate Cost to add an item Cost to find an item Frequency of cleanup If not implemented carefully could easily cause more gen 2 collections than needed

// sweep the cache for dead objects and compacts the list. static void sweep_Weak. Ref() { // don't bother if no new collection happened since // we swept last if (GC. Collection. Count(0) != gen 0_count) gen 0_count = GC. Collection. Count (0); else return; lock (list) { for (int i = 0; i < Weak. Ref_fill; ) { if (list[i]. Target == null) { // install the last element in the free slot. Weak. Ref_fill--; if (i != Weak. Ref_fill) { list[i] = list[Weak. Ref_fill]; list[Weak. Ref_fill] = null; } } else i++; } } }

Why We Don’t Allow Many Customized Settings Only allow very few settings specified via hosting API or config Advantages for GC to do the tuning for you GC has intimate knowledge of memory when applications run GC is tested on many platforms + many configurations

Looking Ahead Challenges on 64 -bit More common as people start using 64 -bit machines VM is practically unlimited so physical memory is the limit Servers like the SQL Server tend to not allow paging Gen 2 could take a long time

Tools And Resources http: //blogs. msdn. com/maoni/ CLRProfiler. NET CLR Memory performance counters The So. S Debugger Extension for windbg/cdb vadump Task Manager