What are the core concepts covered in Linux Virtual Memory: Page Tables, HugePages & OOM Tuning?

How the Linux Virtual Memory Subsystem translates virtual addresses to physical RAM via Page Tables; The Translation Lookaside Buffer (TLB) hardware cache and why TLB misses slow down large databases; Reducing page table memory overhead using 2MB and 1GB Transparent HugePages (THP); How the Linux OOM Killer calculates `oom_badness` and tuning `oom_score_adj` to protect mission-critical databases

Advanced 28 min readModule: Module 15: Memory Subsystem: Page Tables, TLB, HugePages & OOM Killer

Linux Virtual Memory: Page Tables, HugePages & OOM Tuning

Deep dive into Linux kernel memory management: Virtual Memory addressing, 4-level Page Tables (PGD, P4D, PUD, PMD, PTE), Translation Lookaside Buffer (TLB) shootdowns, configuring Transparent HugePages (2MB/1GB), and tuning the Out-Of-Memory (OOM) Killer (`oom_score_adj`).

What You Will Learn in This Lesson

How the Linux Virtual Memory Subsystem translates virtual addresses to physical RAM via Page Tables
The Translation Lookaside Buffer (TLB) hardware cache and why TLB misses slow down large databases
Reducing page table memory overhead using 2MB and 1GB Transparent HugePages (THP)
How the Linux OOM Killer calculates `oom_badness` and tuning `oom_score_adj` to protect mission-critical databases

Introduction & Core Concept

Every process on Linux operates inside a virtual address space (up to 128TB on 64-bit x86). The CPU's Memory Management Unit (MMU) and the Linux kernel maintain multi-level Page Tables to translate virtual addresses into physical RAM pages (default: 4KB). Understanding how page tables, TLB hardware caches, and memory overcommit interact allows you to optimize multi-terabyte database servers.

WHY DOES THIS MATTER IN THE REAL WORLD?

For a 500GB PostgreSQL or Redis instance, 4KB page tables consume ~10GB of RAM just for address mappings. Switching to 2MB HugePages slashes page table overhead to 20MB and speeds up database lookups by 20%.

Syntax & Structure

bash

echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
echo -1000 > /proc/$PID/oom_score_adj

Configuring HugePages and Protecting Critical Processes from OOM Killer

bash

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
#!/usr/bin/env bash
# Linux Kernel Memory Tuning: HugePages & OOM Killer Protection
 
set -euo pipefail
 
echo "=== Linux Virtual Memory Subsystem & OOM Hardening ==="
 
# 1. Inspect Virtual Memory & Page Table Allocation
echo "[1] System Virtual Memory Metrics:"
grep -E "MemTotal|PageTables|HugePages_Total|Hugepagesize" /proc/meminfo
 
# 2. Configure 2MB HugePages for Database Optimization
# Allocates 512 x 2MB HugePages (1GB contiguous physical RAM)
echo -e "\n[2] Allocating 1GB of 2MB HugePages..."
echo 512 | sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages > /dev/null
grep "HugePages_" /proc/meminfo
 
# 3. Protect Critical Database Process from the Linux OOM Killer
# oom_score_adj ranges from -1000 (Never Kill) to +1000 (Kill First)
DB_PID=$$ # Simulating with current process PID
echo -e "\n[3] Adjusting OOM Score Adjustment for Process (PID: $DB_PID):"
echo -1000 | sudo tee "/proc/$DB_PID/oom_score_adj" > /dev/null
 
echo "Verified Process OOM Score: $(cat /proc/$DB_PID/oom_score)"
echo "Verified Process OOM Score Adjustment: $(cat /proc/$DB_PID/oom_score_adj)"
echo -e "\n✅ Process protected from emergency kernel OOM kill sweeps!"

Line-by-Line Technical Breakdown

1Memory Overcommit & vm.overcommit_memory: Linux allows processes to allocate more virtual memory than physical RAM available (overcommit). `vm.overcommit_memory = 2` disables overcommit, strictly preventing memory allocations from exceeding `Swap + (RAM * overcommit_ratio)`, guaranteeing OOM crashes never occur.

Try It Yourself (Interactive Editor)

Modify the code in real-time and click Run to test live browser output and console logs.

Intelligent Code Runner & Live Sandbox[BASH]

BASH SOURCE EDITOR

Interactive Live Code

#!/usr/bin/env bash
# Linux Kernel Memory Tuning: HugePages & OOM Killer Protection

set -euo pipefail

echo "=== Linux Virtual Memory Subsystem & OOM Hardening ==="

# 1. Inspect Virtual Memory & Page Table Allocation
echo "[1] System Virtual Memory Metrics:"
grep -E "MemTotal|PageTables|HugePages_Total|Hugepagesize" /proc/meminfo

# 2. Configure 2MB HugePages for Database Optimization
# Allocates 512 x 2MB HugePages (1GB contiguous physical RAM)
echo -e "\n[2] Allocating 1GB of 2MB HugePages..."
echo 512 | sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages > /dev/null
grep "HugePages_" /proc/meminfo

# 3. Protect Critical Database Process from the Linux OOM Killer
# oom_score_adj ranges from -1000 (Never Kill) to +1000 (Kill First)
DB_PID=$$ # Simulating with current process PID
echo -e "\n[3] Adjusting OOM Score Adjustment for Process (PID: $DB_PID):"
echo -1000 | sudo tee "/proc/$DB_PID/oom_score_adj" > /dev/null

echo "Verified Process OOM Score: $(cat /proc/$DB_PID/oom_score)"
echo "Verified Process OOM Score Adjustment: $(cat /proc/$DB_PID/oom_score_adj)"
echo -e "\n✅ Process protected from emergency kernel OOM kill sweeps!"

>=== KWAS Production Deployment Pipeline ===

>Running set -euo pipefail checks: OK

>Containers healthy, 0 errors logged.

>Deployment completed successfully.

Common Mistakes & How to Avoid Them

#1: Enabling Transparent HugePages (`THP=always`) on Redis and MongoDB databases, causing severe latency spikes during page allocation compaction.

THP background defragmentation locks memory pages, introducing 100ms latency spikes in low-latency in-memory databases.

Incorrect / Antipattern

echo always > /sys/kernel/mm/transparent_hugepage/enabled # Bad for Redis!

Correct / Professional Solution

echo madvise > /sys/kernel/mm/transparent_hugepage/enabled # Good: Explicit allocation only

Industry Best Practices & Professional Standards

Set `echo -1000 > /proc/$PID/oom_score_adj` for primary database processes (PostgreSQL, MySQL).
Use Explicit HugePages (`hugetlbfs`) for Oracle/PostgreSQL instead of Transparent HugePages.
Monitor memory commit limits using `vm.overcommit_memory` and `/proc/meminfo`.

Lesson Summary & Core Takeaways

Page Tables translate virtual addresses to physical memory; TLBs cache translations.
HugePages (2MB/1GB) drastically reduce page table overhead for massive databases.
`oom_score_adj` controls process kill priority during system memory exhaustion.