Hugepages and KSM on Linux

Searching for Linux optimization some time ago I got two cool features: huge pages and ksm.

Huge pages

The Linux Virtual Memory subsystem (VM) provides memory to the system in blocks, called pages. By default, each page has 4 kilobytes of size. In theory, a system with 2GiB of RAM can allocate 524,288 pages with 4KiB of size. Along with the payload, pages carries some control bits. These bits are scanned by kscand so that VM subsystem can manage the page life cycle. Each page has an entry in the page table. Thus, as much pages you have, more resources will be demanded to manage all of them.

That way, Linux can allocate untill 4 MiB page size for x86 systems and 2 MiBpage size for x86_64. You can enable it defining into your .config file of kernel source:

CONFIG_HUGETLBFS=y
CONFIG_HUGETLB_PAGE=y

Once you have a kernel compiled with this two options enabled, you can allocate a number of hugepages using sysctl tool:

sysctl -w vm.nr_hugepages = 10

This will tell Linux to reserve space to 10 hugepages. You can check it out:

[root@localhost ~] grep ^Huge /proc/meminfo
HugePages_Total: 10
HugePages_Free: 10
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB

In case of x86_64 system, it means 2 MiB * 10 = 20 MiB. That pages are allocated contiguously. So, it is recommended you allocate huge pages in the boot time. Huge pages can’t be moved to swap space too.

It is commonly use in virtualization hosts and database servers.

KSM

The Kernel Samepage Merging is a mechanism which is used mainly by hypervisiors to save memory. When it is enabled, a kernel thread scans memory searching for pages with the same content but with different owner. When it occurs, Linux merges them and maps both to the same location. The new common page is marked as copy-on-write too. As soon as a process need to motify that page, Linux breaks it again into two pages. To use KSM, the kernel have to be compiled with CONFIG_KSM=y. Once the kernel is compiled with KSM, you can enable KSM to scan with:

echo 1 > /sys/kernel/mm/ksm/run

Now, your kernel will scan 100 pages each 20 millisecs by default. You can modify this writing into the files pages_to_scan and sleep_millisecs that you can find in /sys/kernel/mm/ksm folder. Monitor the KSM:

eduardo@symphony:~$ ( cd /sys/kernel/mm/ksm/ ; while true ; do clear ; for i in * ; do echo -n "${i}: " ; cat ${i} ; done ; sleep 1 ; done )

For more details, find hugetlbpage.tx and ksm.txt in Documentation folder of your Linux source.