Access Official Technical PDF

Share Summary:LinkedInEmail

Accurate memory visibility helps data center operators make better decisions about virtual machine (VM) placement, recovery and memory allocation. Page Reference Logging (PRL) is a proposed extension to Intel Page Modification Logging (PML) designed to support virtual machine working set estimation by recording both read and write page references. Its central design goal is to give operators useful memory-use information without charging the VM being measured for the logging workload.

That distinction matters in industrial and other enterprise environments where virtualization supports mixed workloads and service requirements. Memory tracking can inform migration, checkpointing and memory overcommitment, but the tracking mechanism itself must not create an unacceptable impact on production workloads.

What Intel Page Modification Logging records

PML is a hardware-assisted virtualization feature that records guest physical addresses when a write causes a page’s dirty bit to be set in the extended page table (EPT). The processor places those addresses in a logging buffer. When the buffer fills, it triggers a VM exit so the hypervisor can process the recorded pages and reset the buffer.

This is useful for operations that need to identify modified pages. During live migration, for example, the hypervisor must track pages changed while the VM continues running so those changes can be transferred. Checkpointing also needs to capture memory state. In the study described here, PML reduced the time spent in the memory-saving phase of live migration by up to 10.18%. It also slightly reduced application performance degradation during migration, by up to 0.95%. The evaluation found no additional power consumption attributable to PML.

Those benefits do not make PML a complete solution for working set size (WSS) estimation. A working set includes pages actively referenced by a workload, not only pages written during a tracking interval. PML misses read-only activity, and its one-time logging of a page does not show whether that page is accessed repeatedly or only briefly.

Why PML alone cannot estimate a working set

WSS estimation aims to identify the memory a VM currently needs. This information can support memory overcommitment, in which allocated memory is adjusted to better match actual demand. It can also inform tasks such as fast VM restore and processor cache allocation.

Two limitations in PML’s records make this difficult:

  • Read accesses are not captured. A page used only for reading will not be logged as a modification. Counting only writes can therefore understate the memory a workload uses.
  • Repeated references are not distinguished. PML records a page only once, so it cannot separate frequently used, or “hot,” pages from pages touched once and then left idle. Treating both alike can overstate the working set.

There is also an operational concern. When PML is used for WSS estimation, buffer-full events generate VM exits handled on the CPU resources of the VM being measured. The study reported performance degradation of up to 34.8% in this use case. That overhead is difficult to justify for a task whose primary beneficiary is the data center operator rather than the tenant workload.

How Page Reference Logging extends PML

PRL introduces two operating modes. Its PML-compatible mode retains the existing write-tracking behavior for tasks such as migration and checkpointing. A second mode, Page Access and Modification Logging (PRLPAML), is intended for working set estimation.

In PRLPAML mode, the logging process records a guest physical address after an EPT walk regardless of whether the access modified the page. It can also record the same page more than once. Repeated entries give the estimation system evidence for identifying hot pages, while read tracking makes the record more representative of actual memory references.

PRL also changes where buffer-full work is handled. Rather than making the target VM’s CPU handle the event, the design sends an inter-processor interrupt to a CPU assigned to the privileged or controller VM. That VM is under the operator’s control and can host the logging and estimation work. The target VM can continue execution while the controller processes the buffer.

During processing, the handler transfers logged addresses into larger buffers and counts how often each address appears. The WSS estimation system uses these counts to identify hot pages over time. It checks whether the number of qualifying pages has stabilized across observation intervals; once it has, the system treats the distinct hot-page count as the working set. The estimate also accounts for the guest kernel footprint.

Evaluation findings and implementation considerations

The PRL prototype was evaluated using Gem5, a hardware simulator, with traces replayed in a virtualized environment. In the reported tests, PRL estimated the working set of synthetic workloads with an error margin below 1 MB. The evaluation also reported accurate estimates for HPL Linpack and several PARSEC applications. By comparison, the sampled-page method assessed in the study produced inaccurate estimates for the tested workloads.

The authors report that PRL caused no measurable performance difference for the target VM in their evaluation, and that missed addresses during buffer processing were negligible for the estimation results. The WSS estimation process did consume CPU resources in the privileged VM; that consumption increased with the number of VMs being monitored. This is an important deployment consideration: moving overhead away from tenant VMs does not eliminate the need to provision resources for operator-side monitoring.

The results should also be interpreted in context. PRL is a proposed hardware extension, and its evaluation relied on simulation and trace replay rather than a production processor implementing the design. The reported findings therefore describe the prototype and its tested workloads, not a guarantee of behavior across all platforms or application mixes.

Practical implications for virtualization planning

For system integrators and infrastructure teams, the key lesson is that the right page-tracking method depends on the operational objective. Write tracking can help with migration and checkpointing because those operations need to identify modified pages. Estimating a VM’s working set requires a richer view: reads, writes and the frequency of references all matter.

PRL’s design addresses that difference by separating the tracking modes and assigning WSS-related processing to operator-controlled CPU resources. If implemented in hardware and integrated with a hypervisor, this approach could support more informed memory allocation without imposing the estimation workload directly on the tenant VM. Teams evaluating memory overcommitment should still account for estimation policy, observation intervals and controller-VM capacity, as these affect how the system is operated.

Conclusion

Intel PML provides a useful hardware mechanism for tracking modified pages and can reduce live migration time, but its write-only, one-time records are insufficient for accurate working set estimation. Page Reference Logging extends the concept to repeated read and write references and redirects buffer handling away from the measured VM. The practical implication is clear: reliable memory management depends not just on collecting page data, but on capturing the right access patterns and placing the cost of analysis where it belongs.

Access Official Technical PDF

About The Author