Virtual memory compression


Virtual memory compression is a memory management technique that utilizes data compression to reduce the size or number of paging requests to and from the auxiliary storage. In a virtual memory compression system, pages to be paged out of virtual memory are compressed and stored in physical memory, which is usually random-access memory, or sent as compressed to auxiliary storage such as a hard disk drive or solid-state drive. In both cases the virtual memory range, whose contents has been compressed, is marked inaccessible so that attempts to access compressed pages can trigger page faults and reversal of the process. The footprint of the data being paged is reduced by the compression process; in the first instance, the freed RAM is returned to the available physical memory pool, while the compressed portion is kept in RAM. In the second instance, the compressed data is sent to auxiliary storage but the resulting I/O operation is smaller and therefore takes less time.
In some implementations, including zswap, zram and Helix Software Company’s Hurricane, the entire process is implemented in software. In other systems, such as IBM's MXT, the compression process occurs in a dedicated processor that handles transfers between a local cache and RAM.
Virtual memory compression is distinct from garbage collection systems, which remove unused memory blocks and in some cases consolidate used memory regions, reducing fragmentation and improving efficiency. Virtual memory compression is also distinct from context switching systems, such as Connectix's RAM Doubler and Apple OS 7.1, in which inactive processes are suspended and then compressed as a whole.

Benefits

By reducing the I/O activity caused by paging requests, virtual memory compression can produce overall performance improvements. The degree of performance improvement depends on a variety of factors, including the availability of any compression co-processors, spare bandwidth on the CPU, speed of the I/O channel, speed of the physical memory, and the compressibility of the physical memory contents.
On multi-core, multithreaded CPUs, some benchmarks show performance improvements of over 50%.
In some situations, such as in embedded devices, auxiliary storage is limited or non-existent. In these cases, virtual memory compression can allow a virtual memory system to operate, where otherwise virtual memory would have to be disabled. This allows the system to run certain software which would otherwise be unable to operate in an environment with no virtual memory.
Flash memory has certain endurance limitations on the maximum number of erase cycles it can undergo, which can be as low as 100 erase cycles. In systems where Flash Memory is used as the only auxiliary storage system, implementing virtual memory compression can reduce the total quantity of data being written to auxiliary storage, improving system reliability.

Shortcomings

Low compression ratios

One of the primary issues is the degree to which the contents of physical memory can be compressed under real-world loads. Program code and much of the data held in physical memory is often not highly compressible, since efficient programming techniques and data architectures are designed to automatically eliminate redundancy in data sets. Various studies show typical data compression ratios ranging from 2:1 to 2.5:1 for program data, similar to typically achievable compression ratios with disk compression.

Background I/O

In order for virtual memory compression to provide measurable performance improvements, the throughput of the virtual memory system must be improved when compared to the uncompressed equivalent. Thus, the additional amount of processing introduced by the compression must not increase the overall latency. However, in I/O-bound systems or applications with highly compressible data sets, the gains can be substantial.

Increased thrashing

The physical memory used by a compression system reduces the amount of physical memory available to processes that a system runs, which may result in increased paging activity and reduced overall effectiveness of virtual memory compression. This relationship between the paging activity and available physical memory is roughly exponential, meaning that reducing the amount of physical memory available to system processes results in an exponential increase of paging activity.
In circumstances where the amount of free physical memory is low and paging is fairly prevalent, any performance gains provided by the compression system may be offset by an increased page fault rate that leads to thrashing and degraded system performance. In an opposite state, where enough physical memory is available and paging activity is low, compression may not impact performance enough to be noticeable. The middle ground between these two circumstanceslow RAM with high paging activity, and plenty of RAM with low paging activityis where virtual memory compression may be most useful. However, the more compressible the program data is, the more pronounced are the performance improvements as less physical memory is needed to hold the compressed data.
For example, in order to maximize the use of a compressed pages cache, Helix Software Company's Hurricane 2.0 provides a user-configurable compression rejection threshold. By compressing the first 256 to 512 bytes of a 4 KiB page, this virtual memory compression system determines whether the configured compression level threshold can be achieved for a particular page; if achievable, the rest of the page would be compressed and retained in a compressed cache, and otherwise the page would be sent to auxiliary storage through the normal paging system. The default setting for this threshold is an 8:1 compression ratio.

Price/performance issues

In hardware implementations the technology also relies on price differentials between the various components of the system, for example, the difference between the cost of RAM and the cost of a processor dedicated to compression. The relative price/performance differences of the various components tend to vary over time. For example, the addition of a compression co-processor may have minimal impact on the cost of a CPU.

Prioritization

In a typical virtual memory implementation, paging happens on a least recently used basis, potentially causing the compression algorithm to use up CPU cycles dealing with the lowest priority data. Furthermore, program code is usually read-only, and is therefore never paged-out. Instead code is simply discarded, and re-loaded from the program’s auxiliary storage file if needed. In this case the bar for compression is higher, since the I/O cycle it is attempting to eliminate is much shorter, particularly on flash memory devices.

Compression using quantization

Accelerator designers exploit quantization to reduce the bitwidth
of values and reduce the cost of data movement. However,
any value that does not fit in the reduced bitwidth results in
an overflow. Therefore
accelerators use quantization for applications that are tolerant
to overflows. In most applications the rate of
outliers is low and values are often within a narrow range
providing the opportunity to exploit quantization in generalpurpose processors. However, a software implementation of
quantization in general-purpose processors has three problems.
First, the programmer has to manually implement conversions
and the additional instructions that quantize and dequantize
values, imposing a programmer’s effort and performance overhead. Second, to cover outliers, the bitwidth of the quantized
values often become greater than or equal to the original
values. Third, the programmer has to use standard bitwidth;
otherwise, extracting non-standard bitwidth for representing narrow integers exacerbates the overhead
of software-based quantization. A hardware support in the memory hierarchy of
general-purpose processors for quantization can solve these problems. Tha hardware support allows representing
values by few and flexible numbers of bits and storing outliers
in their original format in a separate space, preventing any
overflow. It minimizes metadata and the overhead of locating
quantized values using a software-hardware interaction that
transfers quantization parameters and data layout to hardware.
As a result, transparent hardware-based quantization has three advantages over cache
compression techniques: less metadata, higher compression
ratio for floating-point values and cache blocks with multiple data
types, and lower overhead for locating the compressed blocks.

History

Virtual memory compression has gone in and out of favor as a technology. The price and speed of RAM and external storage have plummeted due to Moore's Law and improved RAM interfaces such as DDR3, thus reducing the need for virtual memory compression, while multi-core processors, server farms, and mobile technology together with the advent of flash based systems make virtual memory compression more attractive.

Origins

Paul R. Wilson proposed compressed caching of virtual memory pages in 1990, in a paper circulated at the ACM OOPSLA/ECOOP '90 Workshop on Garbage Collection, and appearing in ACM SIGPLAN Notices in January 1991.
Helix Software Company pioneered virtual memory compression in 1992, filing a patent application for the process in October of that year. In 1994 and 1995, Helix refined the process using test-compression and secondary memory caches on video cards and other devices. However, Helix did not release a product incorporating virtual memory compression until July 1996 and the release of Hurricane 2.0, which used the Stac Electronics Lempel–Ziv–Stac compression algorithm and also used off-screen video RAM as a compression buffer to gain performance benefits.
In 1995, RAM cost nearly $50 per megabyte, and Microsoft's Windows 95 listed a minimum requirement of 4 MB of RAM. Due to the high RAM requirement, several programs were released which claimed to use compression technology to gain “memory”. Most notorious was the SoftRAM program from Syncronys Softcorp. SoftRAM was revealed to be “placebo software” which did not include any compression technology at all. Other products, including Hurricane and MagnaRAM, included virtual memory compression, but implemented only run-length encoding, with poor results, giving the technology a negative reputation.
In its 8 April 1997 issue, PC Magazine published a comprehensive test of the performance enhancement claims of several software virtual memory compression tools. In its testing PC Magazine found a minimal performance improvement from the use of Hurricane, and none at all from any of the other packages. However the tests were run on Intel Pentium systems which had a single core and were single threaded, and thus compression directly impacted all system activity.
In 1996, IBM began experimenting with compression, and in 2000 IBM announced its Memory eXpansion Technology. MXT was a stand-alone chip which acted as a CPU cache between the CPU and memory controller. MXT had an integrated compression engine which compressed all data heading to/from physical memory. Subsequent testing of the technology by Intel showed 5-20% overall system performance improvement, similar to the results obtained by PC Magazine with Hurricane.

Recent developments