Memory Leaks Demand Technology 1020 Eighth Avenue South

Memory Leaks Demand Technology 1020 Eighth Avenue South, Suite 6, Naples, FL 34102 phone: (941) 261 -8945 fax: (941) 261 -5456 e-mail: markf@demandtech. com http: //www. demandtech. com ã 2002 Demand Technology Software, Inc.

Outline n Topics ¨ ¨ Virtual Memory concepts Virtual Memory constraints in 32 -bit Windows Leaking processes Page faults/sec ã 2002 Demand Technology, Inc. Memory Leaks 2

Virtual Memory Manager n 4 GB virtual per process address space ¨ n n n ¨ ffff 16 Lower 2 GB - Private Page 0 reserved Code pages Heap System Upper 2 GB - System n n n System code Shared dlls System cache 7 fff ffff 16 User 0000 ã 2002 Demand Technology, Inc. Memory Leaks 3

Virtual Memory Manager ¨ ¨ ¨ All address spaces share the virtual addresses from the same upper 2 GB of the System area All virtual addresses in the lower 2 GB User area are unique Shared memory in the System range can be used for IPC ã 2002 Demand Technology, Inc. Memory Leaks Shared ffff 16 System 7 fff ffff 16 User 0000 4

Virtual Memory Manager n Boot. ini /3 GB switch ¨ Virtual storage constraint relief for some applications n n n SQL Server Exchange 2000 Etc. ã 2002 Demand Technology, Inc. Memory Leaks 5

AWE n Windows 2000 Address Windowing Extensions ¨ Address RAM > 4 GB limit -Intel Pentium III, IV n n n requires PSE 36 Driver Real addresses only Not backed on paging files – PTEs remain 32 -bit ã 2002 Demand Technology, Inc. Memory Leaks 6

Win 64 Virtual Memory ã 2002 Demand Technology, Inc. Memory Leaks 7

Virtual Memory Manager n Real Memory allocation Counters n n n ¨ Available Bytes Pool non-paged Bytes, Pool Paged Resident Bytes, System Cache Resident Bytes, System Code Resident Bytes, System Driver Resident Bytes Instantaneous Counters (all reported as Bytes) Cache Bytes is actually the pageable System Working Set = Pool Paged Resident Bytes + System Cache Resident Bytes + System Code Resident Bytes + System Driver Resident Bytes ¨ ã 2002 Demand Technology, Inc. Memory Leaks 8

Virtual Memory Manager n Memory allocation Counters n n n n Available Bytes Available KBytes Available MBytes Pool non-paged Bytes, Pool Paged Resident Bytes, System Cache Resident Bytes, System Code Resident Bytes, System Driver Resident Bytes ã 2002 Demand Technology, Inc. Memory Leaks 9

n n What is the difference between Pool Paged Bytes and Pool Nonpaged Bytes? Which applications would use which VM pool? ã 2002 Demand Technology, Inc. Memory Leaks 10

Virtual Memory map ã 2002 Demand Technology, Inc. Memory Leaks 11

Virtual Memory map n Paged ¨ ¨ ¨ File Server IIS (html, jpg, gif) ¨ Page tables ¨ n non. Paged: ¨ ¨ ã 2002 Demand Technology, Inc. Memory Leaks Memory mapped files Shared DLLs I/O buffers used by device drivers TCP Session data 12

Virtual Memory map n n n ã 2002 Demand Technology, Inc. Memory Leaks Virtual memory map is static, created once during OS initialization When any one area is full, memory allocations to that pool will fail One reserved area of virtual memory can fill up faster than the others; 13

Nonpaged pool size (ref. Q 126402): Minimum. Non. Paged. Pool. Size = 256 K Min. Addition. Non. Paged. Pool. Per. Mb = 32 K Default. Maximum. Non. Paged. Pool = 1 MB Max. Addition. Non. Paged. Pool. Per. Mb = 400 K PTE_PER_PAGE = 1024 PAGE_SIZE=4096 Non. Paged. Pool. Size = Minimum. Non. Paged. Pool. Size + ((Physical MB - 4) * Min. Addition. Non. Paged. Pool. Per. MB) If Non. Paged. Pool. Size > 128 MB THEN Non. Paged. Pool. Size = 128 ã 2002 Demand Technology, Inc. Memory Leaks 14

Paged pool size: Size = (2 * Maximum. Non. Paged. Pool. Size) / PAGESIZE Size = (Size + (PTE_PER_PAGE - 1)) / PTE_PER_PAGE Paged. Pool. Size = Size * PAGESIZE * PTE_PER_PAGE If Paged. Pool. Size > 192 MB THEN Page. Pool. Size = 192 MB Note: Paged. Pool. Size sizeof (RAM) for systems < 192 MB ã 2002 Demand Technology, Inc. Memory Leaks 15

Default Paged and Nonpaged pool size can be overridden: ã 2002 Demand Technology, Inc. Memory Leaks 16

Virtual Memory constraints n Virtual Memory constraints in 32 -bit Windows tend to appear when sizeof (RAM) 4 GB ¨ 2 GB private area is not enough virtual memory for some applications n n ¨ e. g. , SQL Server, Exchange database (store. exe) Due to fragmentation, it is typically not possible to allocate all 2 GB system area is not enough virtual memory for some applications File cache for a conventional IIS-managed web site with many static. htm, jpg, gif, etc. , files to retrieve ã 2002 Demand Technology, Inc. Memory Leaks 17 n

Virtual Memory constraints n Virtual Memory constraints in 32 -bit Windows tend to appear when sizeof (RAM) 4 GB ¨ ¨ Ample RAM exists, but it is not possible for your applications to access it due to virtual memory addressing limitations Large number of Available Bytes ã 2002 Demand Technology, Inc. Memory Leaks 18

Virtual Memory constraints n So try the /3 GB switch ¨ 1 GB system area is not enough virtual memory for some applications n n Possible shortage of Free System Page Table Entries Possible shortage of Nonpaged Pool – ¨ Where Session data from TCP connections is stored Due to fragmentation, it may not be possible to failover a 2 GB+ private address space (e. g. , SQL Server, MS Exchange database – store. exe) using Microsoft Cluster Server (MCS) n During address space recovery on the standby node, the entire virtual memory allocation is acquired at one time ã 2002 Demand Technology, Inc. Memory Leaks 19

Exchange 2000 memory tuning n Exchange default memory allocation parameters are “self-tuning, ” but may not be optimal on servers with > 1 GB RAM ¨ Adjust HKLMSYSTEMCurrent. Control. SetServicesSMTPSVCQueuing n Msg. Handle. Threshold & Msg. Handle. Async. Threshold HKLMSYSTEMCurrent. Control. SetServicesInetinfoParameters n File. Cache. Max. Handles ã 2002 Demand Technology, Inc. Memory Leaks 20

Exchange 2000 memory tuning n Store DB cache ¨ ¨ ¨ Store cache normally expands to fill RAM But, maximum cache = 900 MB This value can be adjusted using ADSI Edit tool: n n ¨ ms. Exch. ESEParam. Cache. Size. Max ms. Exch. ESEParam. Cache. Size. Min Also, consider adjusting ms. Exch. ESEParam. Log. Buffers attribute for active, back-end servers ã 2002 Demand Technology, Inc. Memory Leaks 21

Detecting memory leaks n n Processes that allocate virtual memory, but later “forget” to free it. MS says leaks won’t happen in. Net managed code due to automatic garbage collection But, meanwhile…, Where to look depends on whether process or system addresses are being allocated: ¨ ¨ Per Process: Virtual Bytes, Private Bytes, Pool Paged Bytes, Handle Count System level: Memory Pool Paged Bytes, Pool Nonpaged Bytes and the Objects Object ã 2002 Demand Technology, Inc. Memory Leaks 22

Detecting memory leaks n Look for a steady increase or a sharp spike in process Virtual Bytes, or the System’s Pool Paged Bytes. ¨ n If RAM is not full, the leak may also be manifest in the Memory allocation counters and result in increased paging, if RAM fills up. For example: ã 2002 Demand Technology, Inc. Memory Leaks 23

Detecting memory leaks (1) ã 2002 Demand Technology, Inc. Memory Leaks 24

Detecting memory leaks (2) ã 2002 Demand Technology, Inc. Memory Leaks 25

Detecting memory leaks (2) ã 2002 Demand Technology, Inc. Memory Leaks 26

Detecting memory leaks (2) ã 2002 Demand Technology, Inc. Memory Leaks 27

Detecting memory leaks (2) ã 2002 Demand Technology, Inc. Memory Leaks 28

Lab exercise. n Open n memory usage. Can you find the application that is leaking? SUPPORT. 200205080300. EXCHG 2 K. sum. smf, SUPPORT. 200205090300. EXCHG 2 K. sum. smf, and SUPPORT. 200205100300. EXCHG 2 K. sum. smf and investigate virtual ã 2002 Demand Technology, Inc. Memory Leaks 29

Detecting Memory Leaks (3) ã 2002 Demand Technology, Inc. Memory Leaks 30

Detecting Memory Leaks (3) ã 2002 Demand Technology, Inc. Memory Leaks 31

Detecting Memory Leaks (3) ã 2002 Demand Technology, Inc. Memory Leaks 32

Detecting Memory Leaks (3) ã 2002 Demand Technology, Inc. Memory Leaks 33

Page replacement n Theoretically, Page Faults/sec Cache faults/sec + Transition faults/sec + Demand Zero Faults/sec + Pages Read/sec n But ã 2002 Demand Technology, Inc. empirically… Memory Leaks 34

ã 2002 Demand Technology, Inc. Memory Leaks 35

Page replacement n Empirically, Page Faults/sec Transition faults/sec + Demand Zero Faults/sec + Pages Read/sec ã 2002 Demand Technology, Inc. Memory Leaks 36

Questions ? ã 2002 Demand Technology, Inc. Memory Leaks 37