
Image courtesy of Renjith Krishnan / FreeDigitalPhotos.net
Most computers today have support for
virtual memory. An application (i.e. process) running on such computers sees its address space as one large range of contiguous addresses, even if its memory chunks may be scattered around the physical memory (RAM). This means that when the process requests a particular memory location, the computers must figure out which physical memory location this corresponds to. The mapping of virtual memory to physical memory is stored in the
page tables.
For processes that uses lots of memory, the virtual to physical memory mapping (i.e. pagetable) will need to hold a lot of mappings (called page table entries, or PTEs), and may grow to very large sizes. Having very large pagetables claims extra resources on the system, as more memory are needed to hold the page tables, and more CPU cycles must be used to search the pagetable. The system may benefit from keeping the number of page table entries at a minimum.
This is where hugepages comes in handy. Using hugepages, we increase the memory chunks allocated by the process, thus reducing number of memory mappings needed by the process. Instead of one page table entry mapping for example 4 kB of data (which is the default size on many systems), each entry may map for example 4 MB of data.
To test how a process memory usage affects the pagetable on one of our linux servers at work, I created two tiny applications written i C. The first allocates 19 GB of memory using regular memory allocations, and the second one allocated the same amount of memory using hugepages.
Continue reading →