US20090300606A1

US20090300606A1 - Virtual machine migration with direct physical access control

Info

Publication number: US20090300606A1
Application number: US12/128,511
Authority: US
Inventors: Troy Miller; Mark A. Criss; Jerry James Harrow, JR.; Thomas Turicchi; Michael Wisner
Original assignee: Individual
Current assignee: Valtrus Innovations Ltd; Hewlett Packard Enterprise Development LP
Priority date: 2008-05-28
Filing date: 2008-05-28
Publication date: 2009-12-03
Also published as: US9032397B2

Abstract

A data processing system facilitates virtual machine migration with direct physical access control. The illustrative data processing system comprises a software-programmable trap control associated with hardware registers of a computer that selectively vectors execution control of a virtual machine (VM) between a host and a guest. The data processing system further comprises a logic which is configured for execution on the computer that programs the trap control to enable the virtual machine to directly access the hardware registers when the virtual machine is not migrated and to revoke direct access of the hardware registers in preparation for virtual machine migration.

Description

BACKGROUND

Virtual machine technology is used to create and concurrently run one or more guest operating systems on a physical device. One or more virtual machines can operate on a single host computing system, each including an operating system with concurrent applications.
One aspect of virtualization can be the migration of virtual machine from one host system to another, for example to manage or balance the load across physical devices in a system, address faults or failures in a host system, address temporary unavailability of resources in the system, perform maintenance, and the like.

SUMMARY

An embodiment of a data processing system facilitates virtual machine migration with direct physical access control. The illustrative data processing system comprises a software-programmable trap control associated with hardware registers of a computer that selectively vectors execution control of a virtual machine (VM) between a host and a guest. The data processing system further comprises a logic which is configured for execution on the computer that programs the trap control to enable the virtual machine to directly access the hardware registers when the virtual machine is not migrated and to revoke direct access of the hardware registers in preparation for virtual machine migration.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention relating to both structure and method of operation may best be understood by referring to the following description and accompanying drawings:

FIGS. 1A and 1B are schematic block diagrams which depict embodiments of a data processing system that facilitate virtual machine migration with direct physical access control;

FIG. 2 is a schematic block diagram showing an embodiment of a network system that can implement virtual machine migration with direct physical access control; and

FIGS. 3A through 3G are flow charts illustrating one or more embodiments or aspects of a computer-executed method that facilitates virtual machine migration with direct physical access control.

DETAILED DESCRIPTION

In several system and method embodiments, structures and techniques enable virtual machine (VM) migration after physical access is allowed in a virtual machine.
Virtual machines can be implemented to enable migration between virtual machine hosts by disallowing direct access to physical resources. For performance reasons direct access is granted for input/output (I/O) access, translation lookaside buffer (TLB) page tables, and the like for virtual machines. Once direct access is allowed, migration is no longer allowed due to the possibility of introducing error. The illustrative systems and methods revoke the direct physical access and allow migration of the virtual machine to be performed.
By enabling direct physical access to hardware registers in a revocable manner, a virtual machine can be migrated once the direct access is revoked using various methods of virtual machine migration. While the virtual machine is not currently migrated, the virtual machine can continue to exploit direct physical access which enables much higher performance for a deployed workload in the virtual machine. If such direct physical access is unavailable, virtual machines with performance requirements that demand direct access to physical devices cannot be migrated. The structures and systems disclosed herein enable migration of such high-performance virtual machines, further enabling zero-downtime maintenance, optimization of physical resource usage, and increases in the manageability and efficiency for virtual machines that exploit direct access to physical devices.
Although typical virtualized systems can allow direct physical access, such systems have disallowed migration of a virtual machine to another virtual machine host, preventing data center operations from being optimized because the virtual machine is tied to the virtual machine host and thus cannot be moved to reallocate or free servers to be retired.
Conversely, typical virtualized systems that support migration have not allowed direct physical access, leading to diminished performance since input/output (I/O) intensive workloads can experience overheads in excess of 80% making the virtual machine operate as a platform that is unacceptable for deploying certain workload types such operations for database applications.
Referring to FIG. 1A, a schematic block diagram illustrates an embodiment of a data processing system 100 that facilitates virtual machine migration with direct physical access control. The illustrative data processing system 100 comprises a software-programmable trap control 102 associated with hardware registers 104 of a computer 106 that selectively vectors execution control of a virtual machine (VM) 108 between a host 110 and a guest 112. The data processing system 100 further comprises a logic 114 which is configured for execution on the computer 106 that programs the trap control 102 to enable the virtual machine 108 to directly access the hardware registers 104 when the virtual machine 108 is not migrated and to revoke direct access of the hardware registers 104 in preparation for virtual machine migration.
The trap control 102 can be implemented as computer hardware for functional elements such as input/output (I/O) mapping registers, translation lookaside buffer (TLB) page tables, and the like, which are enhanced to enable selectable control between full virtualization and high-performance direct access under software management. For example, in one operating mode the hardware can be set to enable a virtual machine to directly access the I/O registers for performance, but then upon software request allow the I/O card to be quiesced and the direct physical access of the I/O registers revoked so that the virtual machine is completely virtualized. After direct physical access to the I/O registers is revoked, the trap control hardware causes the processor to trap such that the host operating system intercepts input and output traffic from the guest once again.
A similar technique can be used by the processor to enable revoking of the direct physical access to the TLB page table entries, thus ensuring that only memory which is used is copied to the target virtual machine host for the virtual machine migration.
The trap control 102 can be selectively software-programmable to control enablement and disablement of direct physical access of functional blocks such as input/output I/O mapping registers 116, translation lookaside buffer (TLB) page tables 118, and others for the virtual machine 108.
The logic 114 can be further configured to operate the virtual machine 108 for direct physical access, then issue a software request to revoke the direct physical access and respond to the issued software request by quiescing hardware and revoking direct physical access so that the virtual machine 108 is completely virtualized.
The logic 114 can also be configured to disable direct physical access for the virtual machine 108 and cause a processor 120 to trap whereby a host operating system 110 intercepts input/output signals from a guest 112 and virtual machine operations are virtualized.
The logic 114 executing on the computer 106 can manages virtual machine migration by operating the virtual machine 108 during non-migration time periods by directly accessing physical hardware registers 104 and preparing the virtual machine 108 for migration by revoking direct physical access capability. The logic 114 operates the virtual machine 108 during migration time periods by maintaining full virtualization functionality.
Referring to FIG. 1B, the illustrative technique for virtual machine migration has many uses and applications. For example, a logic 124 executable on the computer 106 can perform maintenance with no virtual machine downtime by migrating the virtual machine 108 from a first host 110A to a second host 110B, performing hardware maintenance on the second host 110B with direct physical access to hardware, and migrating the virtual machine 108 back from the second host 110B to the first host 110A. Accordingly, zero-downtime maintenance is maintained by migrating the virtual machine, performing hardware maintenance, and migrating the virtual machine back without downtime of the virtual machine.
In another example operation, a logic 126 executable on the computer 106 can optimize physical resource usage by determining a predefined condition of virtual machine inactivity or lower demand, coalescing virtual machines in response to the predefined condition, operating the virtual machines full virtualized with physical devices powered-off, and moving the virtual machines back onto the physical devices and enabling direct physical access for conditions of increased demand.
In a further application, a logic 128 executable on the computer 106 enables tolerance of hardware failures. The logic 128 revokes direct physical access capability of the virtual machine 108 in preparation for migration, then migrates the virtual machine 108 from the first host 110A which executes a host operating system to a second host 110B which executes a guest operating system. The logic 128 detects failure of a hardware component at the host operating system and responds by quiescing functionality of the guest operating system. The logic 128 automatically switches to a hot-spare hardware component and restarts guest operating system functionality.
The illustrative data processing system 100 and associated operating method enable the virtual machine 108 during normal operation associated with non-migration time periods to maintain performance improvement that direct physical access enables. The virtual machine 108 during migration can have direct physical access revoked thus allowing correct migration to occur without susceptibility to logical failure resulting from non-completion of pending or outstanding input/output operations. The direct physical access capability of virtual machines 108 can be implemented to enable data center operations to be optimized because the virtual machine 108 is not tied to a virtual machine host 110 and can be moved to reallocate or free servers to be retired.
The illustrative data processing system 100 and associated operating method further enable the virtual machine guest operating system to have greater tolerance for I/O card failures. For example, the host operating system can detect when an I/O card has failed and respond by quiescing I/O to the guest. The host operating system automatically switches to a hot-spare I/O card, and then restarts the guest. Operating systems that cannot tolerate an I/O error when running natively are thus enabled to tolerate the I/O error when running as a guest.
Referring to FIG. 2, a schematic block diagram illustrates an embodiment of a data processing system such as a network system 200 that can implement virtual machine migration with direct physical access control. The network system 200 comprises a network 228 and multiple hosts 210 communicatively coupled to the network 228. A virtual machine (VM) 208 is executable on at least one of the hosts 210. Logic 214 is configured to enable migration of the virtual machine 208 from a first host 210A to a second host 210B of the multiple hosts 210 and selectively control enablement and disablement of direct physical access for the virtual machine 208 so that direct physical access for the virtual machine 208 is disabled during migration.
In the illustrative example, the hosts 210 can virtualize multiple virtual machines 208 via a virtualization platform such as a virtual machine monitor (VMM), hypervisor, or the like.
Virtualization enables resources to be shared between multiple virtual machines. The VMM is a software layer that virtualizes and multiplexes available resources of a computer system among one or more guest operating systems on the computer system. The term guest operating system refers to an operating system that the VMM can host. A domain is a running virtual machine within which a guest operating system executes. The VMM may be called a hypervisor to clarify operation at a higher privilege level than the supervisor code of the guest operating systems that are hosted. The terms VMM and hypervisor can be used interchangeably. In various configurations, VMMs can run directly on physical hardware and form an abstraction that is identical to the hardware underneath the VMM or can run as an application on a host operating system.
Referring to FIGS. 3A through 3G, flow charts illustrate one or more embodiments or aspects of a computer-executed method 300 that facilitates virtual machine migration with direct physical access control. FIG. 3A depicts a computer-executed method 300 for operating a network computing system. The method 300 comprises operating 302 a virtual machine on multiple hosts and enabling 304 migration of the virtual machine from a first host to a second host of the hosts. Enablement and disablement of direct physical access for the virtual machine is selectively controlled 306 with direct physical access for the virtual machine disabled 308 during migration.
In an illustrative implementation, enablement and disablement of direct physical access of input/output I/O mapping registers and translation lookaside buffer (TLB) page tables for the virtual machine (VM) can be selectively controlled 306.
The illustrative method 300 can be used to enable computer hardware operational control that selects between full virtualization and direct physical access under software control.
Referring to FIG. 3B, a flow chart depicts a computer-executed method 310 for virtual machine control in a data processing system. The method 310 comprises operating 312 a virtual machine for direct physical access, thus attaining substantial performance efficiency in comparison to non-direct control. In conditions for which direct physical access is inappropriate, for example for virtual machine migration, the method further comprises issuing 314 a software request to revoke the direct physical access. In response to the issued software request, hardware is quiesced 316 and direct physical access is revoked 317. The virtual machine can then be completely virtualized 318.
In a particular example, direct physical access can be revoked 317 by actuating disablement of direct physical access for the virtual machine. The method can further comprise causing a processor to trap whereby a host operating system intercepts input/output signals from a guest and VM operations are virtualized 318.
Various functions and applications can be performed based on the illustrative technique for virtual machine migration. For example, referring to FIG. 3C, a method for performing 320 maintenance with no virtual machine (VM) downtime can comprise migrating 322 the virtual machine from the first host to the second host, and performing 324 hardware maintenance on the second host. The virtual machine can be migrated 326 back from the second host to the first host.
In another example application, referring to FIG. 3D, a method for optimizing 330 physical resource usage can comprise determining 332 a predefined condition of virtual machine inactivity or lower demand and coalescing 334 virtual machines when the predefined condition occurs. The virtual machines can be operated 336 fully virtualized with physical devices powered off. The virtual machines can be moved 338 back onto the physical devices, enabling direct physical access for conditions of increased demand.
As shown in FIG. 3E, another example application which is enabled by the illustrative virtual machine control technique is shown. The illustrative method 340 comprises operating 342 a virtual machine during non-migration time periods by directly accessing 343 physical hardware registers. The virtual machine is prepared 344 for migration by revoking 345 direct physical access capability. The virtual machine is operated 346 during migration time periods by enabling 347 full virtualization functionality.
Referring to FIG. 3F, a flow chart shows an embodiment of a method for maintaining 350 a system that is enabled by the technique for controlling virtual machine migration. The illustrative method 350 comprises revoking 352 direct physical access capability of the virtual machine in preparation for migration, then migrating 354 the virtual machine from a first host which executes a host operating system to a second host executing a guest operating system. Upon detection 356 of failure of a hardware component at the host operating system, functionality of the guest operating system is quiesced 358 and the failed hardware component is replaced 360 by automatically switching 362 to a hot-spare hardware component. Guest operating system functionality is restarted 364.
Referring to FIG. 3G, a flow chart shows a more specific example embodiment of a method for maintaining 370 a system that is enabled by the technique for controlling virtual machine migration. Upon detection 376 of failure of an input/output (I/O) card at the host operating system, input/output operations to the guest operating system are quiesced 378 and the failed hardware component is replaced 380 by automatically switching 382 to a hot-spare I/O card. Guest operating system input/output operations are restarted 384.
Terms “substantially”, “essentially”, or “approximately”, that may be used herein, relate to an industry-accepted tolerance to the corresponding term. Such an industry-accepted tolerance ranges from less than one percent to twenty percent and corresponds to, but is not limited to, functionality, values, process variations, sizes, operating speeds, and the like. The term “coupled”, as may be used herein, includes direct coupling and indirect coupling via another component, element, circuit, or module where, for indirect coupling, the intervening component, element, circuit, or module does not modify the information of a signal but may adjust its current level, voltage level, and/or power level. Inferred coupling, for example where one element is coupled to another element by inference, includes direct and indirect coupling between two elements in the same manner as “coupled”.
The illustrative block diagrams and flow charts depict process steps or blocks that may represent modules, segments, or portions of code that include one or more executable instructions for implementing specific logical functions or steps in the process. Although the particular examples illustrate specific process steps or acts, many alternative implementations are possible and commonly made by simple design choice. Acts and steps may be executed in different order from the specific description herein, based on considerations of function, purpose, conformance to standard, legacy structure, and the like.
The block diagrams and flow charts further describe an article of manufacture comprising a controller-usable medium having a computer readable program code embodied in a controller for managing virtual machines (VMs) in a distributed network system.
While the present disclosure describes various embodiments, these embodiments are to be understood as illustrative and do not limit the claim scope. Many variations, modifications, additions and improvements of the described embodiments are possible. For example, those having ordinary skill in the art will readily implement the steps necessary to provide the structures and methods disclosed herein, and will understand that the process parameters, materials, and dimensions are given by way of example only. The parameters, materials, and dimensions can be varied to achieve the desired structure as well as modifications, which are within the scope of the claims. Variations and modifications of the embodiments disclosed herein may also be made while remaining within the scope of the following claims.

Claims

1. A data processing system comprising:

a software-programmable trap control associated with hardware registers of a computer that selectively vectors execution control of a virtual machine (VM) between a host and a guest; and

a logic executable on the computer that programs the trap control to enable the VM to directly access the hardware registers when the VM is not migrated and to revoke direct access of the hardware registers in preparation for VM migration.

2. The system according to claim 1 further comprising:

a network;

a plurality of hosts communicatively coupled to the network;

the virtual machine (VM) executable on at least one of the host plurality; and

the logic configured to enable migration of the VM from a first host to a second host of the host plurality and selectively control enablement and disablement of direct physical access for the VM whereby direct physical access for the VM is disabled during migration.

3. The system according to claim 1 further comprising:

the selectively software-programmable trap control configured to control enablement and disablement of direct physical access of input/output I/O mapping registers and translation lookaside buffer (TLB) page tables for the virtual machine (VM).

4. The system according to claim 1 further comprising:

the logic configured to operate the virtual machine (VM) for direct physical access, issue a software request to revoke the direct physical access, and respond to the issued software request by quiescing hardware and revoking direct physical access whereby the VM is completely virtualized.

5. The system according to claim 1 further comprising:

the logic configured to disable direct physical access for the virtual machine (VM) and cause a processor to trap whereby a host operating system intercepts input/output signals from a guest and VM operations are virtualized.

6. The system according to claim 1 further comprising:

a logic executable on the computer that performs maintenance with no virtual machine (VM) downtime by migrating the VM from the first host to the second host, performing hardware maintenance on the second host with direct physical access to hardware, and migrating the VM back from the second host to the first host.

7. The system according to claim 1 further comprising:

a logic executable on the computer that optimizes physical resource usage by determining a predefined condition of virtual machine inactivity or lower demand, coalescing virtual machines (VMs) in response to the predefined condition, operating the VMs full virtualized with physical devices powered-off, and moving the VMs back onto the physical devices and enabling direct physical access for conditions of increased demand.

8. The system according to claim 1 further comprising:

a logic executable on the computer that manages virtual machine migration by operating the virtual machine (VM) during non-migration time periods comprising directly accessing physical hardware registers, preparing the VM for migration comprising revoking direct physical access capability, and operating the VM during migration time periods comprising full virtualization functionality.

9. The system according to claim 1 further comprising:

a logic executable on the computer that enables tolerance of hardware failures by revoking direct physical access capability of the virtual machine (VM) in preparation for migration, migrating the VM from the first host executing a host operating system to a second host executing a guest operating system, detecting failure of a hardware component at the host operating system, quiescing functionality of the guest operating system, automatically switching to a hot-spare hardware component, and restarting guest operating system functionality.

10. A computer-executed method for operating a network computing system comprising:

operating a virtual machine (VM) on a plurality of hosts;

enabling migration of the VM from a first host to a second host of the host plurality;

selectively controlling enablement and disablement of direct physical access for the VM; and

disabling direct physical access for the VM during migration.

11. The method according to claim 10 further comprising:

selectively controlling enablement and disablement of direct physical access of input/output I/O mapping registers and translation lookaside buffer (TLB) page tables for the virtual machine (VM).

12. The method according to claim 10 further comprising:

providing a computer hardware operational control that selects between full virtualization and direct physical access under software control.

13. The method according to claim 10 further comprising:

operating the virtual machine (VM) for direct physical access;

issuing a software request to revoke the direct physical access;

in response to the issued software request, quiescing hardware and revoking direct physical access; and

completely virtualizing the VM.

14. The method according to claim 10 further comprising:

actuating disablement of direct physical access for the virtual machine (VM); and

causing a processor to trap whereby a host operating system intercepts input/output signals from a guest and VM operations are virtualized.

15. The method according to claim 10 further comprising:

performing maintenance with no virtual machine (VM) downtime comprising:

migrating the VM from the first host to the second host;

performing hardware maintenance on the second host; and

migrating the VM back from the second host to the first host.

16. The method according to claim 10 further comprising:

optimizing physical resource usage comprising:

determining a predefined condition of virtual machine inactivity or lower demand;

coalescing virtual machines (VMs) in response to the predefined condition;

operating the VMs full virtualized with physical devices powered-off; and

moving the VMs back onto the physical devices and enabling direct physical access for conditions of increased demand.

17. The method according to claim 10 further comprising:

operating the virtual machine (VM) during non-migration time periods comprising directly accessing physical hardware registers;

preparing the VM for migration comprising revoking direct physical access capability; and

operating the VM during migration time periods comprising full virtualization functionality.

18. The method according to claim 10 further comprising:

revoking direct physical access capability of the virtual machine (VM) in preparation for migration;

migrating the VM from the first host executing a host operating system to a second host executing a guest operating system;

detecting failure of a hardware component at the host operating system;

quiescing functionality of the guest operating system;

automatically switching to a hot-spare hardware component; and

restarting guest operating system functionality.

19. The method according to claim 10 further comprising:

detecting failure of an input/output (I/O) card at the host operating system;

quiescing input/output operations to the guest operating system;

automatically switching to a hot-spare I/O card; and

restarting guest operating system input/output operations.

20. An article of manufacture comprising:

a controller-usable medium having a computer readable program code embodied in a controller for managing virtual machines (VMs) in a distributed network system, the computer readable program code further comprising:

code causing the controller to operate a virtual machine (VM) on a plurality of hosts;

code causing the controller to enable migration of the VM from a first host to a second host of the host plurality;

code causing the controller to selectively control enablement and disablement of direct physical access for the VM; and

code causing the controller to disable direct physical access for the VM during migration.