US9244712B2 - Virtualizing performance counters - Google Patents

Virtualizing performance counters Download PDF

Info

Publication number
US9244712B2
US9244712B2 US14/060,947 US201314060947A US9244712B2 US 9244712 B2 US9244712 B2 US 9244712B2 US 201314060947 A US201314060947 A US 201314060947A US 9244712 B2 US9244712 B2 US 9244712B2
Authority
US
United States
Prior art keywords
counter
enable
guest
host
counters
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US14/060,947
Other versions
US20140053155A1 (en
Inventor
Erik C. Cola-Robles
Gilbert Neiger
Steven M. Bennett
Andrew V. Anderson
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Intel Corp
Original Assignee
Intel Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Intel Corp filed Critical Intel Corp
Priority to US14/060,947 priority Critical patent/US9244712B2/en
Publication of US20140053155A1 publication Critical patent/US20140053155A1/en
Application granted granted Critical
Publication of US9244712B2 publication Critical patent/US9244712B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3409Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466Performance evaluation by tracing or monitoring
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466Performance evaluation by tracing or monitoring
    • G06F11/348Circuit details, i.e. tracer hardware
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/461Saving or restoring of program or task context
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/461Saving or restoring of program or task context
    • G06F9/462Saving or restoring of program or task context with multiple register sets
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/468Specific access rights for resources, e.g. using capability register
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45591Monitoring or debugging support
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/815Virtual
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/86Event-based monitoring
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/88Monitoring involving counting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/885Monitoring specific for caches

Definitions

  • the present disclosure pertains to the field of information processing, and more particularly, to the field of using performance counters in a virtualization environment.
  • Virtualization in information processing systems allows multiple instances of one or more operating systems (each, an “OS”) to run on a single information processing system, even though each OS is designed to have complete, direct control over the system and its resources.
  • Virtualization is typically implemented by using software (e.g., a virtual machine monitor, or a “VMM”) to present to each OS a “virtual machine” (“VM”) having virtual resources, including one or more virtual processors, that the OS may completely and directly control, while the VMM maintains a system environment for implementing virtualization policies such as sharing and/or allocating the physical resources among the VMs (the “virtualization environment”).
  • VM virtual machine monitor
  • VM virtual machine
  • Each OS, and any other software, that runs on a VM is referred to as a “guest” or as “guest software,” while a “host” or “host software” is software, such as a VMM, that runs outside of the virtualization environment.
  • a physical processor in an information processing system may support virtualization, for example, by supporting an instruction to enter a virtualization environment to run a guest on a virtual processor (i.e., a physical processor under constraints imposed by a VMM) in a VM.
  • a virtual processor i.e., a physical processor under constraints imposed by a VMM
  • certain events, operations, and situations such as external interrupts or attempts to access privileged registers or resources, may be intercepted, i.e., cause the processor to exit the virtualization environment so that a VMM may operate, for example, to implement virtualization policies.
  • a physical processor may also support other instructions for maintaining a virtualization environment, and may include memory or register bits that indicate or control virtualization capabilities of the physical processor.
  • a physical processor supporting a virtualization environment may include performance counters for logging performance monitoring information.
  • each performance counter would be controlled by one or more control or configuration registers, or portions of a one or more control or configuration registers, associated with the counter.
  • To virtualize these performance counters each transition between a guest and a host or between two guests would typically require saving the state of all of the counters and their associated control registers and loading a new state for all of the counters and their associated control registers.
  • FIG. 1 illustrates an embodiment of the present invention in a virtualization architecture.
  • FIG. 2 illustrates an embodiment of the present invention in a method for a virtualizing performance counters.
  • FIG. 3 illustrates another embodiment of the present inventions in a virtualization architecture.
  • FIG. 4 illustrates another embodiment of the present invention in a method for a virtualizing performance counters.
  • the performance of a virtualization environment may be improved by reducing the amount of state information that must be saved and loaded on transitions between a host and a guest and between guests.
  • Embodiments of the present invention may be used to virtualize performance counters or other counters without requiring the saving and the loading of the contents of the counters and their associated control registers. Therefore, performance may be improved over a virtualization environment in which the contents of the counters and their associated control registers are saved on transitions.
  • Embodiments of the present invention provide for performance counters to be efficiently enabled or disabled for any number of hosts and guests, for performance counters to be assigned for the exclusive use of a host or a guest, and for performance counters to be shared between any number of hosts and/or guests.
  • FIG. 1 illustrates an embodiment of the present invention in virtualization architecture 100 .
  • bare platform hardware 110 may be any data processing apparatus capable of executing any OS or VMM software.
  • bare platform hardware may be that of a personal computer, mainframe computer, portable computer, handheld device, set-top box, server, or any other computing system.
  • Bare platform hardware 110 includes processor 120 and memory 130 .
  • Processor 120 may be any type of processor, including a general purpose microprocessor, such as a processor in the Intel® Pentium® Processor Family, Itanium® Processor Family, or other processor family from Intel® Corporation, or another processor from another company, or a digital signal processor or microcontroller. Although FIG. 1 shows only one such processor 120 , bare platform hardware 110 may include any number of processors, including any number of multicore processors, each with any number of execution cores, and any number of multithreaded processors, each with any number of threads.
  • Memory 130 may be static or dynamic random access memory, semiconductor-based read-only or flash memory, magnetic or optical disk memory, any other type of medium readable by processor 120 , or any combination of such mediums.
  • Processor 120 and memory 130 may be coupled to or communicate with each other according to any known approach, such as directly or indirectly through one or more buses, point-to-point, or other wired or wireless connections.
  • Bare platform hardware 110 may also include any number of additional devices or connections.
  • FIG. 1 illustrates VMM 140 , VMs 150 and 160 , guest operating systems 152 and 162 , and applications 154 , 155 , 164 , and 165 .
  • VMM 140 may be any software, firmware, or hardware host installed on or accessible to bare platform hardware 110 , to present VMs, i.e., abstractions of bare platform hardware 110 , to guests, or to otherwise create VMs, manage VMs, and implement virtualization policies.
  • a host may be any VMM, hypervisor, OS, or other software, firmware, or hardware capable of controlling bare platform hardware 110 .
  • a guest may be any OS, any VMM, including another instance of VMM 140 , any hypervisor, or any application or other software.
  • FIG. 1 shows two VMs, 150 and 160 , with guest OS 152 and guest applications 154 and 155 installed on VM 150 and guest OS 162 and guest applications 164 and 165 installed on VM 160 .
  • FIG. 1 shows only two VMs and two applications per VM, any number of VMs may be created, and any number of applications may run on each VM within the scope of the present invention.
  • a resource that can be accessed by a guest may either be classified as a “privileged” or a “non-privileged” resource.
  • VMM 140 facilitates the functionality desired by the guest while retaining ultimate control over the resource.
  • Non-privileged resources do not need to be controlled by VMM 140 and may be accessed directly by a guest.
  • each guest OS expects to handle various events such as exceptions (e.g., page faults, and general protection faults), interrupts (e.g., hardware interrupts and software interrupts), and platform events (e.g., initialization and system management interrupts).
  • exceptions e.g., page faults, and general protection faults
  • interrupts e.g., hardware interrupts and software interrupts
  • platform events e.g., initialization and system management interrupts.
  • privileged events Some of these virtualization events because they must be handled by VMM 140 to ensure proper operation of VMs 150 and 160 , protection of VMM 140 from guests, and protection of guests from each other.
  • processor 120 may be executing instructions from VMM 140 or any guest, thus VMM 140 or the guest may be running on, or in control of, processor 120 .
  • VMM 140 When a privileged event occurs or a guest attempts to access a privileged resource, control may be transferred from the guest to VMM 140 .
  • the transfer of control from a guest to VMM 140 is referred to as a “VM exit” herein.
  • VMM 140 After handling the event or facilitating the access to the resource appropriately, VMM 140 may return control to a guest.
  • the transfer of control from VMM 140 to a guest is referred to as a “VM entry” herein.
  • processor 120 controls the operation of VMs 150 and 160 according to data stored in virtual machine control structure (“VMCS”) 132 .
  • VMCS 132 is a structure that may contain state of a guest or guests, state of VMM 140 , execution control information indicating how VMM 140 is to control operation of a guest or guests, information regarding VM exits and VM entries, and any other such information.
  • Processor 120 reads information from VMCS 132 to determine the execution environment of a VM and constrain its behavior.
  • VMCS 132 is stored in memory 130 .
  • multiple VMCSs are used to support multiple VMs.
  • FIG. 1 shows VMCS 132 stored in memory 130 , storing a VMCS in a memory is not required by the present invention.
  • Processor 120 includes counters 171 , 173 , 175 , and 177 .
  • counters 171 , 173 , 175 , and 177 are performance counters that may be programmed by software running on processor 120 to log performance monitoring information; however, other embodiments may include any number of counters and/or any type or size of counter.
  • any of performance counters 171 , 173 , 175 , or 177 may be programmed to increment for each occurrence of a selected event, or to increment for each clock cycle during a selected event.
  • the events may include any of a variety of events related to execution of program code on processor 120 , such as branch mispredictions, cache hits, cache misses, translation lookaside buffer hits, translation lookaside buffer misses, etc. Therefore, performance counters 171 , 173 , 175 , and 177 may be used for tuning or profiling program code to yield the best possible performance on processor 120 .
  • Processor 120 also includes counter control storage locations 172 , 174 , 176 , and 178 , corresponding to counters 171 , 173 , 175 , and 177 , respectively.
  • Counter control storage locations 172 , 174 , 176 , and 178 may be registers or any other structures of any size, or portions of one or more registers of other structures, for storing information to control or configure counters 171 , 173 , 175 , and 177 , respectively.
  • Counter control storage locations 172 , 174 , 176 , and 178 may be programmed to store information to control or configure counters 171 , 173 , 175 , and 177 , respectively, such as information to enable the counter, to select the event to be counted, to select the method of counting (e.g., number of occurrences or duration of event), to select conditions for counting (e.g., based on privilege level of software executing when event is detected) and to set any other control, configuration, or other variables.
  • information to enable the counter to select the event to be counted, to select the method of counting (e.g., number of occurrences or duration of event), to select conditions for counting (e.g., based on privilege level of software executing when event is detected) and to set any other control, configuration, or other variables.
  • a global counter enable storage location may include one bit per performance counter, where each bit may be set to a logical one to set an individual counter enable indicator for a corresponding performance counter to an “enable” value.
  • a single bit or field of a global counter enable storage location may correspond to a group of any number of performance counters.
  • a global enable storage location may include any number of bits or fields that correspond to any number of performance counters, and may further include or be associated with one or more additional enable indicators that may be used to control any number of bits or fields that more directly control performance counters.
  • a storage location may include a field to store a “counter enable vector,” which includes one bit per counter, and an additional bit within this storage location or in a storage location or data structure elsewhere in processor 120 or virtualization architecture 100 may be used to enable or disable the counter enable vector.
  • individual counters may be individually controlled by guests according to a counter enable vector stored in a VMCS, but the default value of the additional bit may be set such that the individual counter control is automatically disabled when executing a VMM or other host software designed for a processor that does not support a counter enable vector model.
  • Processor 120 also includes global counter enable storage location 121 , which may be one or more bits in a control register, configuration register, model specific register, or any other storage location to store a global counter enable indicator.
  • global counter enable storage location 121 may be a single bit of a programmable register, where the bit may be set to a logical one to set the global counter enable indicator to an “enable” value or a logical zero to set the global counter enable indicator to a “disable” value.
  • the value of the global counter enable indicator is used, along with any other relevant information stored in counter control storage locations 172 , 174 , 176 , and 178 , by counter enable logic 170 to control the operation of counters 171 , 173 , 175 , and 177 , respectively. If the global counter enable indicator is set to an “enable” value, then each counter operates according to its individual control and configuration information. For example, counter 171 may increment on a cache hit because the contents of counter control storage location 172 include an individual counter enable bit set to an “enable” value and an event select field set to count cache hits, while at the same time counter 173 may not increment because the contents of counter control storage location 174 include an individual counter enable bit set to a “disable” value.
  • counter 171 would not increment on a cache hit even if the contents of counter control storage location 172 include an individual counter enable bit set to an “enable” value and an event select field set to count cache hits.
  • processor 120 includes control logic 180 to support virtualization, including the virtualization of counters 171 , 173 , 175 , and 177 .
  • Control logic 180 may be microcode, programmable logic, hard-coded logic, or any other form of control logic within processor 120 .
  • control logic 180 may be implemented in any form of hardware, software, or firmware, such as a processor abstraction layer, within a processor or within any component accessible or medium readable by a processor, such as memory 130 .
  • Control logic 180 causes processor 120 to execute method embodiments of the present invention, such as the method embodiments described below with reference to FIG. 2 , for example, by causing processor 120 to include the execution of one or more micro-operations, e.g., to support virtualization, in its response to virtualization instructions or other instructions from a host or guest.
  • Control logic 180 includes VM entry logic 181 to transfer control of processor 120 from a host to a guest (i.e., a VM entry) and VM exit logic 182 to transfer control of processor 120 from a guest to a host (i.e., a VM exit). In some embodiments, control may also be transferred from a guest to a guest or from a host to a host.
  • software running on a VM on processor 120 may be both a guest and a host (e.g., a VMM running on a VM is a guest to the VMM that controls that VM and a host to a guest running on a VM that it controls).
  • Control logic 180 also includes guest state load logic 183 , host state load logic 184 , and guest state store logic 185 .
  • Guest state load logic 183 is to load guest state from VMCS 132 to processor 120 on a VM entry.
  • Host state load logic 184 is to load host state from VMCS 132 to processor 120 on a VM exit.
  • Guest state store logic 185 is to store guest state from processor 120 to VMCS 132 on a VM exit.
  • control logic 180 also includes host state store logic 186 to store host state from processor 120 to VMCS 132 on a VM entry prior to guest state load logic 183 loading guest state into processor 120 .
  • control logic 180 may load and store state residing in other system components, e.g., input-output devices, memory controllers.
  • VMCS 132 may include fields, control bits, or other data structures to support virtualization. These data structures may be checked or otherwise referred to by control logic 180 to determine how to manage a VM environment.
  • guest state load enable indicator 133 may be set to cause guest state load logic 183 to load a guest value from guest state storage location 137 into global counter enable storage location 121 in connection with a VM entry
  • host state load enable indicator 134 may be set to cause host state load logic 184 to load a host value from host state storage location 138 into global counter enable storage location 121 in connection with a VM exit
  • guest state store enable indicator 135 may be set to cause guest state store logic 185 to store the contents of global enable storage location 121 in guest state storage location 137
  • host state store enable indicator 136 may be set to cause host state store logic 186 to store the contents of global enable storage location 121 host state storage location 138 , all as further described below.
  • these indicators are control bits that are set to enable or cause a desired effect, where set means
  • a VMM may enable counting of events that occur only while a guest executes, only while the VMM executes or while both the VMM and a guest execute. Additionally, a VMM may, on a guest by guest basis, enable or disable performance counters as part of the transition (i.e., the VM entry or VM exit) between the guest and the VMM. This mechanism allows the VMM to “hide” its effect on the performance counters from a guest. Alternatively, this mechanism allows counting of events in the VMM for use by a guest, or in a guest for use by the VMM.
  • FIG. 2 illustrates an embodiment of the present invention in method 200 , a method for virtualizing performance counters. Although method embodiments are not limited in this respect, reference is made to virtualization architecture 100 of FIG. 1 to describe the method embodiment of FIG. 2 .
  • a first performance counter is configured to count occurrences of a first event.
  • counter 171 may be configured, by programming counter control storage location 172 , to count cache misses.
  • a second performance counter is configured to count occurrences of a second event.
  • counter 173 may be configured, by programming counter control storage location 174 , to count cycles not halted.
  • the performance counters may be configured to count occurrences of any other events, e.g., translation lookaside buffer misses, branch mispredictions, etc.
  • VMM 140 of FIG. 1 creates a VMCS (e.g., VMCS 132 ) for a VM.
  • VMM 140 configures VMCS 132 to implement support for virtualizing counters 171 , 173 , 175 , and 177 .
  • guest state load enable indicator 133 is set to cause guest state load logic 183 to load a guest value from guest state storage location 137 into global counter enable storage location 121 in connection with a VM entry.
  • host state load enable indicator 134 is set to cause host state load logic 184 to load a host value from host state storage location 138 into global counter enable storage location 121 in connection with a VM exit.
  • guest state store enable indicator 135 is set to cause guest state store logic 185 to store the contents of global enable storage location 121 in guest state storage location 136 .
  • a transfer of control of processor 120 from the host (i.e., VMM 140 ) to a guest is initiated.
  • the VMM may initiate a VM entry.
  • the VM entry may include VM entry logic 181 causing processor 120 to execute operations or micro-operations to save the host state and load the guest state.
  • VM entry logic determines whether to load a guest value from guest state storage location 137 into global counter enable storage location 121 based on guest state load enable indicator 133 . If guest state load enable indicator 133 is set, then, in box 234 , guest state load logic 183 causes the guest value from guest state storage location 137 to be loaded into global counter enable storage location 121 ; otherwise, box 234 is not performed.
  • the VM entry is completed and control is transferred to the guest.
  • the guest begins or continues to execute.
  • counter enable logic 170 determines whether the global counter enable indicator is set. If not, method 200 continues at box 258 . If so, method 200 continues at box 240 . In box 240 , it is determined whether the first event has occurred. If so, then, in box 244 , the first performance counter operates as configured, i.e., in this embodiment, where it has been configured in box 210 to count cache misses, the first performance counter increments.
  • the second performance counter In box 250 , it is determined whether the second event has occurred. If so, then, in box 254 , the second performance counter operates as configured, i.e., in this embodiment, where it has been configured in box 212 to count occurrences of the second event, the second performance counter increments.
  • VM exit may include VM exit logic 182 causing processor 120 to execute operations or micro-operations to save the guest state and load the host state.
  • VM exit logic determines whether to store the contents of global counter enable storage location 121 in guest state storage location 137 based on guest state store enable indicator 135 . If guest state store enable indicator 135 is set, then, in box 264 , guest state store logic 185 stores the contents of global counter enable storage location 121 in guest state storage location 137 ; otherwise, box 264 is not performed. In some embodiments, there is no guest state store enable indicator 135 . In some embodiments, guest state store logic 185 always stores the contents of global counter enable storage location 121 in guest state storage location 137 . In other embodiments, guest state store logic 185 never stores the contents of global counter enable storage location 121 in guest state storage location 137 as part of VM exit processing.
  • attempts by guest software to access the global counter enable storage location 121 may cause VM exits, either by software convention (e.g., the VMM is required to appropriately set controls in the VMCS to cause VM exits on access to global counter enable storage location 121 ) or by forcing such a VM exit by the processor.
  • software convention e.g., the VMM is required to appropriately set controls in the VMCS to cause VM exits on access to global counter enable storage location 121
  • forcing such a VM exit by the processor may cause VM exits, either by software convention (e.g., the VMM is required to appropriately set controls in the VMCS to cause VM exits on access to global counter enable storage location 121 ) or by forcing such a VM exit by the processor.
  • VM exit logic determines whether to load a host value from host state storage location 138 into global counter enable storage location 121 based on host state load enable indicator 134 . If host state store enable indicator 134 is set, then, in box 268 , host state load logic 184 causes the host value from host state storage location 138 to be loaded into global counter enable storage location 121 ; otherwise, box 268 is not performed.
  • the VM exit is completed and control is transferred to the host.
  • the host begins or continues to execute.
  • counter enable logic 170 determines whether the global counter enable indicator is set. If not, method 200 continues at box 298 . If so, method 200 continues at box 280 . In box 280 , it is determined whether the first event has occurred. If so, then, in box 284 , the first performance counter operates as configured, i.e., in this embodiment, where it has been configured in box 210 to count cache misses, the first performance counter increments.
  • box 290 it is determined whether the second event has occurred. If so, then, in box 294 , the second performance counter operates as configured, i.e., in this embodiment, where it has been configured in box 212 to count occurrences of the second event, the second performance counter increments.
  • box 298 it is determined whether a VM entry is to occur. If so, method 200 continues at box 230 . If not, method 200 continues at box 299 . In box 299 , it is determined whether a halt or other such instruction is to be executed. If not, method 200 continues at box 272 . If so, method 200 ends.
  • FIG. 3 illustrates another embodiment of the invention in bare platform hardware 310 , which may be used in virtualization architecture 100 or another virtualization architecture. Except as otherwise described, the description of bare platform hardware and its elements also applies to bare platform hardware 310 , and its corresponding elements.
  • Bare platform hardware 310 includes processor 320 and memory 330 .
  • Processor 320 includes counters 371 , 373 , 375 , and 377 .
  • Processor 320 also includes counter control storage locations 372 , 374 , 376 , and 378 , corresponding to counters 371 , 373 , 375 , and 377 , respectively, which each contain a counter valid storage location to store a counter valid indicator to enable or disable the corresponding counter, along with other control or configuration information.
  • processor 320 includes counter identifier storage locations 391 , 393 , 395 , and 397 , corresponding to counters 371 , 373 , 375 , and 377 , respectively.
  • Counter identifier storage locations 391 , 393 , 395 , and 397 may be registers or any other structures of any size, or portions of one or more registers of other structures, for storing information to identify or label counters 371 , 373 , 375 , and 377 , respectively.
  • Counter identifier storage locations 391 , 393 , 395 , and 397 may be programmed to store an identifier to uniquely or redundantly (i.e., two counter identifier storage locations could be programmed with the same value) identify counters 371 , 373 , 375 , and 377 , respectively.
  • VMCS 332 includes counter identifier field 333 , which in this embodiment is the same number of bits as each of counter identifier storage location 391 , 393 , 395 , and 397 , but in other embodiments may be any number of bits. Other embodiments may include additional counter identifier fields.
  • Counter identifier field 333 may be programmed, by the VMM that creates VMCS 332 or by any other software, to store a value that may or may not match one or more of the values stored in counter identifier storage locations 391 , 393 , 395 , and 397 .
  • Processor 320 also includes comparison logic 322 , which may be implemented according to any known approach, to generate a counter match signal for each counter based on a comparison of the contents of a corresponding counter identifier storage location and the contents of counter identifier field 333 .
  • the counter match signal for a counter is asserted if the contents of the corresponding counter identifier storage location match the contents of counter identifier field 333 .
  • the counter match signal for a counter is also asserted if the contents of the corresponding counter identifier storage location equals a value of zero, regardless of the contents of counter identifier field 333 , to provide for software that does not use the counter identifier feature of the present invention to operate as expected on hardware that does include the counter identifier feature.
  • the counter match logic may determine a “match” based at least in part on other factors such as the contents of other fields within the VMCS. For example, the value of various control registers such as the CRO, CR3 or CR4 registers in the Intel Architecture may be consulted.
  • the counter match logic may include a determination of whether the software currently executing is the VMM or a guest, and enable the counters only if guest software is currently executing, based on one or more control bits in the VMCS.
  • Embodiments may include one or more “counter match enable” control bits or fields to enable or disable the counter match feature, by disabling the comparison logic or by any other desired approach. For example, if a counter match enable bit is set, then the comparison happens as described above. However, if the counter match enable bit is cleared, the counters are always enabled and no comparison is made, so that a “zero” value in a counter identifier storage location does not require the special treatment described above.
  • the counter match signal for each counter is used, along with any other relevant information stored in counter control storage locations 372 , 374 , 376 , and 378 , by counter enable logic 370 to control the operation of counters 371 , 373 , 375 , and 377 , respectively. If the counter match signal for a counter is asserted, then the counter operates according to its individual control and configuration information. For example, counter 371 may increment on a cache hit if the contents of counter control storage location 372 include an individual counter valid bit set to an “enable” value and an event select field set to count cache hits. However, if the counter match signal for a counter is not asserted, the counter is disabled and does not operate according to its individual control and configuration information. For example, counter 371 would not increment on a cache hit even if the contents of counter control storage location 372 include an individual counter valid bit set to an “enable” value and an event select field set to count cache hits.
  • processor 320 includes control logic 380 to support virtualization, including the virtualization of counters 371 , 373 , 375 , and 377 .
  • Control logic 380 causes processor 320 to execute method embodiments of the present invention, such as the method embodiments described below with reference to FIG. 4 , for example, by causing processor 320 to include the execution of one or more micro-operations, e.g., to support virtualization, in its response to virtualization instructions or other instructions from a host or guest.
  • FIG. 4 illustrates an embodiment of the present invention in method 400 , another method for virtualizing performance counters. Although method embodiments are not limited in this respect, reference is made to FIGS. 1 and 3 to describe the method embodiment of FIG. 4 .
  • a performance counter is configured to count occurrences of an event.
  • counter 371 may be configured, by programming counter control storage location 372 , to count cache misses.
  • counter identifier storage location 391 is programmed with a desired counter identification value.
  • a host creates a VMCS (e.g., VMCS 332 ) for a VM.
  • the host configures VMCS 332 to implement support for virtualizing counters 371 , 373 , 375 , and 377 , including programming counter identifier field 333 with a value equal to the counter identification value used on box 412 .
  • any number of other counter identifier fields in VMCS 332 or any other VMCSes, corresponding to any number of other guests or hosts may also be programmed with the same value to allow any number of guests or hosts to share one or more counters.
  • control of processor 320 is transferred from the host to a guest.
  • the guest begins or continues to execute.
  • a counter match signal is generated based on a comparison of counter identifier storage location 391 and counter identifier field 333 .
  • the counter match signal is asserted if the contents of the counter identifier storage location match the contents of the counter identifier field.
  • counter enable logic 370 determines whether to increment the performance counter based on the counter match signal and the contents of the counter valid storage location in the corresponding counter control storage locations (e.g., counter control storage location 372 ).
  • the performance counter operates as configured, i.e., in this embodiment, where it has been configured in box 410 to count cache misses, the performance counter increments. If the counter match signal is not asserted or the counter's valid indicator is not set, box 444 is not performed.
  • FIGS. 2 and 4 may be performed in a different order, with illustrated boxes omitted, with additional boxes added, or with a combination of reordered, omitted, or additional boxes.
  • a VMCS may configure a VMCS in any order, e.g., boxes 222 to 226 may be rearranged in any order.
  • Processor 120 may be designed in various stages, from creation to simulation to fabrication.
  • Data representing a design may represent the design in a number of manners.
  • the hardware may be represented using a hardware description language or another functional description language.
  • a circuit level model with logic and/or transistor gates may be produced at some stages of the design process.
  • most designs, at some stage reach a level where they may be modeled with data representing the physical placement of various devices.
  • the data representing the device placement model may be the data specifying the presence or absence of various features on different mask layers for masks used to produce an integrated circuit.
  • the data may be stored in any form of a machine-readable medium.
  • An optical or electrical wave modulated or otherwise generated to transmit such information, a memory, or a magnetic or optical storage medium, such as a disc, may be the machine-readable medium. Any of these media may “carry” or “indicate” the design, or other information used in an embodiment of the present invention.
  • an electrical carrier wave indicating or carrying the information is transmitted, to the extent that copying, buffering, or re-transmission of the electrical signal is performed, a new copy is made.
  • the actions of a communication provider or a network provider may constitute the making of copies of an article, e.g., a carrier wave, embodying techniques of the present invention.

Abstract

Embodiments of apparatuses, methods, and systems for virtualizing performance counters are disclosed. In one embodiment, an apparatus includes a counter, a counter enable storage location, counter enable logic, and virtual machine control logic. The counter enable storage location is to store a counter enable indicator. The counter enable logic is to enable the counter based on the counter enable indicator. The virtual machine control logic is to transfer control of the apparatus to a guest. The virtual machine control logic includes guest state load logic to cause a guest value from a virtual machine control structure to be loaded into the counter enable storage location in connection with a transfer of control of the apparatus to the guest.

Description

This application is a continuation of U.S. patent application Ser. No. 11/501,632 filed Aug. 8, 2006, the content of which is hereby incorporated by reference.
BACKGROUND
1. Field
The present disclosure pertains to the field of information processing, and more particularly, to the field of using performance counters in a virtualization environment.
2. Description of Related Art
Generally, the concept of virtualization in information processing systems allows multiple instances of one or more operating systems (each, an “OS”) to run on a single information processing system, even though each OS is designed to have complete, direct control over the system and its resources. Virtualization is typically implemented by using software (e.g., a virtual machine monitor, or a “VMM”) to present to each OS a “virtual machine” (“VM”) having virtual resources, including one or more virtual processors, that the OS may completely and directly control, while the VMM maintains a system environment for implementing virtualization policies such as sharing and/or allocating the physical resources among the VMs (the “virtualization environment”). Each OS, and any other software, that runs on a VM is referred to as a “guest” or as “guest software,” while a “host” or “host software” is software, such as a VMM, that runs outside of the virtualization environment.
A physical processor in an information processing system may support virtualization, for example, by supporting an instruction to enter a virtualization environment to run a guest on a virtual processor (i.e., a physical processor under constraints imposed by a VMM) in a VM. In the virtualization environment, certain events, operations, and situations, such as external interrupts or attempts to access privileged registers or resources, may be intercepted, i.e., cause the processor to exit the virtualization environment so that a VMM may operate, for example, to implement virtualization policies. A physical processor may also support other instructions for maintaining a virtualization environment, and may include memory or register bits that indicate or control virtualization capabilities of the physical processor.
A physical processor supporting a virtualization environment may include performance counters for logging performance monitoring information. Typically, each performance counter would be controlled by one or more control or configuration registers, or portions of a one or more control or configuration registers, associated with the counter. To virtualize these performance counters, each transition between a guest and a host or between two guests would typically require saving the state of all of the counters and their associated control registers and loading a new state for all of the counters and their associated control registers.
BRIEF DESCRIPTION OF THE FIGURES
The present invention is illustrated by way of example and not limitation in the accompanying figures.
FIG. 1 illustrates an embodiment of the present invention in a virtualization architecture.
FIG. 2 illustrates an embodiment of the present invention in a method for a virtualizing performance counters.
FIG. 3 illustrates another embodiment of the present inventions in a virtualization architecture.
FIG. 4 illustrates another embodiment of the present invention in a method for a virtualizing performance counters.
DETAILED DESCRIPTION
Embodiments of apparatuses, methods, and systems for virtualizing performance counters are described below. In this description, numerous specific details, such as component and system configurations, may be set forth in order to provide a more thorough understanding of the present invention. It will be appreciated, however, by one skilled in the art, that the invention may be practiced without such specific details. Additionally, some well known structures, circuits, and the like have not been shown in detail, to avoid unnecessarily obscuring the present invention.
The performance of a virtualization environment may be improved by reducing the amount of state information that must be saved and loaded on transitions between a host and a guest and between guests. Embodiments of the present invention may be used to virtualize performance counters or other counters without requiring the saving and the loading of the contents of the counters and their associated control registers. Therefore, performance may be improved over a virtualization environment in which the contents of the counters and their associated control registers are saved on transitions. Embodiments of the present invention provide for performance counters to be efficiently enabled or disabled for any number of hosts and guests, for performance counters to be assigned for the exclusive use of a host or a guest, and for performance counters to be shared between any number of hosts and/or guests.
FIG. 1 illustrates an embodiment of the present invention in virtualization architecture 100. In FIG. 1, bare platform hardware 110 may be any data processing apparatus capable of executing any OS or VMM software. For example, bare platform hardware may be that of a personal computer, mainframe computer, portable computer, handheld device, set-top box, server, or any other computing system. Bare platform hardware 110 includes processor 120 and memory 130.
Processor 120 may be any type of processor, including a general purpose microprocessor, such as a processor in the Intel® Pentium® Processor Family, Itanium® Processor Family, or other processor family from Intel® Corporation, or another processor from another company, or a digital signal processor or microcontroller. Although FIG. 1 shows only one such processor 120, bare platform hardware 110 may include any number of processors, including any number of multicore processors, each with any number of execution cores, and any number of multithreaded processors, each with any number of threads.
Memory 130 may be static or dynamic random access memory, semiconductor-based read-only or flash memory, magnetic or optical disk memory, any other type of medium readable by processor 120, or any combination of such mediums. Processor 120 and memory 130 may be coupled to or communicate with each other according to any known approach, such as directly or indirectly through one or more buses, point-to-point, or other wired or wireless connections. Bare platform hardware 110 may also include any number of additional devices or connections.
In addition to bare platform hardware 100, FIG. 1 illustrates VMM 140, VMs 150 and 160, guest operating systems 152 and 162, and applications 154, 155, 164, and 165.
VMM 140 may be any software, firmware, or hardware host installed on or accessible to bare platform hardware 110, to present VMs, i.e., abstractions of bare platform hardware 110, to guests, or to otherwise create VMs, manage VMs, and implement virtualization policies. In other embodiments, a host may be any VMM, hypervisor, OS, or other software, firmware, or hardware capable of controlling bare platform hardware 110. A guest may be any OS, any VMM, including another instance of VMM 140, any hypervisor, or any application or other software.
Each guest expects to access physical resources, such as processor and platform registers, memory, and input/output devices, of bare platform hardware 110, according to the architecture of the processor and the platform presented in the VM. FIG. 1 shows two VMs, 150 and 160, with guest OS 152 and guest applications 154 and 155 installed on VM 150 and guest OS 162 and guest applications 164 and 165 installed on VM 160. Although FIG. 1 shows only two VMs and two applications per VM, any number of VMs may be created, and any number of applications may run on each VM within the scope of the present invention.
A resource that can be accessed by a guest may either be classified as a “privileged” or a “non-privileged” resource. For a privileged resource, VMM 140 facilitates the functionality desired by the guest while retaining ultimate control over the resource. Non-privileged resources do not need to be controlled by VMM 140 and may be accessed directly by a guest.
Furthermore, each guest OS expects to handle various events such as exceptions (e.g., page faults, and general protection faults), interrupts (e.g., hardware interrupts and software interrupts), and platform events (e.g., initialization and system management interrupts). These exceptions, interrupts, and platform events are referred to collectively and individually as “virtualization events” herein. Some of these virtualization events are referred to as “privileged events” because they must be handled by VMM 140 to ensure proper operation of VMs 150 and 160, protection of VMM 140 from guests, and protection of guests from each other.
At any given time, processor 120 may be executing instructions from VMM 140 or any guest, thus VMM 140 or the guest may be running on, or in control of, processor 120. When a privileged event occurs or a guest attempts to access a privileged resource, control may be transferred from the guest to VMM 140. The transfer of control from a guest to VMM 140 is referred to as a “VM exit” herein. After handling the event or facilitating the access to the resource appropriately, VMM 140 may return control to a guest. The transfer of control from VMM 140 to a guest is referred to as a “VM entry” herein.
In the embodiment of FIG. 1, processor 120 controls the operation of VMs 150 and 160 according to data stored in virtual machine control structure (“VMCS”) 132. VMCS 132 is a structure that may contain state of a guest or guests, state of VMM 140, execution control information indicating how VMM 140 is to control operation of a guest or guests, information regarding VM exits and VM entries, and any other such information. Processor 120 reads information from VMCS 132 to determine the execution environment of a VM and constrain its behavior. In this embodiment, VMCS 132 is stored in memory 130. In some embodiments, multiple VMCSs are used to support multiple VMs. Although FIG. 1 shows VMCS 132 stored in memory 130, storing a VMCS in a memory is not required by the present invention.
Processor 120 includes counters 171, 173, 175, and 177. In this embodiment, counters 171, 173, 175, and 177 are performance counters that may be programmed by software running on processor 120 to log performance monitoring information; however, other embodiments may include any number of counters and/or any type or size of counter. For example, any of performance counters 171, 173, 175, or 177 may be programmed to increment for each occurrence of a selected event, or to increment for each clock cycle during a selected event. The events may include any of a variety of events related to execution of program code on processor 120, such as branch mispredictions, cache hits, cache misses, translation lookaside buffer hits, translation lookaside buffer misses, etc. Therefore, performance counters 171, 173, 175, and 177 may be used for tuning or profiling program code to yield the best possible performance on processor 120.
Processor 120 also includes counter control storage locations 172, 174, 176, and 178, corresponding to counters 171, 173, 175, and 177, respectively. Counter control storage locations 172, 174, 176, and 178 may be registers or any other structures of any size, or portions of one or more registers of other structures, for storing information to control or configure counters 171, 173, 175, and 177, respectively. Counter control storage locations 172, 174, 176, and 178 may be programmed to store information to control or configure counters 171, 173, 175, and 177, respectively, such as information to enable the counter, to select the event to be counted, to select the method of counting (e.g., number of occurrences or duration of event), to select conditions for counting (e.g., based on privilege level of software executing when event is detected) and to set any other control, configuration, or other variables.
In another embodiment, a global counter enable storage location may include one bit per performance counter, where each bit may be set to a logical one to set an individual counter enable indicator for a corresponding performance counter to an “enable” value. In other embodiments, a single bit or field of a global counter enable storage location may correspond to a group of any number of performance counters. In any of these embodiments, a global enable storage location may include any number of bits or fields that correspond to any number of performance counters, and may further include or be associated with one or more additional enable indicators that may be used to control any number of bits or fields that more directly control performance counters. For example, a storage location may include a field to store a “counter enable vector,” which includes one bit per counter, and an additional bit within this storage location or in a storage location or data structure elsewhere in processor 120 or virtualization architecture 100 may be used to enable or disable the counter enable vector. In this way, individual counters may be individually controlled by guests according to a counter enable vector stored in a VMCS, but the default value of the additional bit may be set such that the individual counter control is automatically disabled when executing a VMM or other host software designed for a processor that does not support a counter enable vector model.
Processor 120 also includes global counter enable storage location 121, which may be one or more bits in a control register, configuration register, model specific register, or any other storage location to store a global counter enable indicator. In one embodiment, global counter enable storage location 121 may be a single bit of a programmable register, where the bit may be set to a logical one to set the global counter enable indicator to an “enable” value or a logical zero to set the global counter enable indicator to a “disable” value.
The value of the global counter enable indicator is used, along with any other relevant information stored in counter control storage locations 172, 174, 176, and 178, by counter enable logic 170 to control the operation of counters 171, 173, 175, and 177, respectively. If the global counter enable indicator is set to an “enable” value, then each counter operates according to its individual control and configuration information. For example, counter 171 may increment on a cache hit because the contents of counter control storage location 172 include an individual counter enable bit set to an “enable” value and an event select field set to count cache hits, while at the same time counter 173 may not increment because the contents of counter control storage location 174 include an individual counter enable bit set to a “disable” value. However, if the global counter enable indicator is set to a “disable” value, all counters 171, 173, 175, and 177 are disabled and do not operate according to their individual control and configuration information. For example, counter 171 would not increment on a cache hit even if the contents of counter control storage location 172 include an individual counter enable bit set to an “enable” value and an event select field set to count cache hits.
Additionally, processor 120 includes control logic 180 to support virtualization, including the virtualization of counters 171, 173, 175, and 177. Control logic 180 may be microcode, programmable logic, hard-coded logic, or any other form of control logic within processor 120. In other embodiments, control logic 180 may be implemented in any form of hardware, software, or firmware, such as a processor abstraction layer, within a processor or within any component accessible or medium readable by a processor, such as memory 130.
Control logic 180 causes processor 120 to execute method embodiments of the present invention, such as the method embodiments described below with reference to FIG. 2, for example, by causing processor 120 to include the execution of one or more micro-operations, e.g., to support virtualization, in its response to virtualization instructions or other instructions from a host or guest.
Control logic 180 includes VM entry logic 181 to transfer control of processor 120 from a host to a guest (i.e., a VM entry) and VM exit logic 182 to transfer control of processor 120 from a guest to a host (i.e., a VM exit). In some embodiments, control may also be transferred from a guest to a guest or from a host to a host. For example, in an embodiment supporting layered virtualization, software running on a VM on processor 120 may be both a guest and a host (e.g., a VMM running on a VM is a guest to the VMM that controls that VM and a host to a guest running on a VM that it controls).
Control logic 180 also includes guest state load logic 183, host state load logic 184, and guest state store logic 185. Guest state load logic 183 is to load guest state from VMCS 132 to processor 120 on a VM entry. Host state load logic 184 is to load host state from VMCS 132 to processor 120 on a VM exit. Guest state store logic 185 is to store guest state from processor 120 to VMCS 132 on a VM exit. In some embodiments control logic 180 also includes host state store logic 186 to store host state from processor 120 to VMCS 132 on a VM entry prior to guest state load logic 183 loading guest state into processor 120. In some embodiments, control logic 180 may load and store state residing in other system components, e.g., input-output devices, memory controllers.
VMCS 132 may include fields, control bits, or other data structures to support virtualization. These data structures may be checked or otherwise referred to by control logic 180 to determine how to manage a VM environment. For example, guest state load enable indicator 133 may be set to cause guest state load logic 183 to load a guest value from guest state storage location 137 into global counter enable storage location 121 in connection with a VM entry, host state load enable indicator 134 may be set to cause host state load logic 184 to load a host value from host state storage location 138 into global counter enable storage location 121 in connection with a VM exit, guest state store enable indicator 135 may be set to cause guest state store logic 185 to store the contents of global enable storage location 121 in guest state storage location 137, and host state store enable indicator 136 may be set to cause host state store logic 186 to store the contents of global enable storage location 121 host state storage location 138, all as further described below. In this description of this embodiment, these indicators are control bits that are set to enable or cause a desired effect, where set means writing a logical one to the bit, but any logic convention or nomenclature may be used within the scope of the present invention.
Using the mechanism described herein, a VMM may enable counting of events that occur only while a guest executes, only while the VMM executes or while both the VMM and a guest execute. Additionally, a VMM may, on a guest by guest basis, enable or disable performance counters as part of the transition (i.e., the VM entry or VM exit) between the guest and the VMM. This mechanism allows the VMM to “hide” its effect on the performance counters from a guest. Alternatively, this mechanism allows counting of events in the VMM for use by a guest, or in a guest for use by the VMM.
FIG. 2 illustrates an embodiment of the present invention in method 200, a method for virtualizing performance counters. Although method embodiments are not limited in this respect, reference is made to virtualization architecture 100 of FIG. 1 to describe the method embodiment of FIG. 2.
In box 210 of FIG. 2, a first performance counter is configured to count occurrences of a first event. For example, counter 171 may be configured, by programming counter control storage location 172, to count cache misses. In box 212, a second performance counter is configured to count occurrences of a second event. For example, counter 173 may be configured, by programming counter control storage location 174, to count cycles not halted. In other embodiments, the performance counters may be configured to count occurrences of any other events, e.g., translation lookaside buffer misses, branch mispredictions, etc.
In box 220 of FIG. 2, VMM 140 of FIG. 1 creates a VMCS (e.g., VMCS 132) for a VM. In boxes 222 to 226, VMM 140 configures VMCS 132 to implement support for virtualizing counters 171, 173, 175, and 177. In box 222, guest state load enable indicator 133 is set to cause guest state load logic 183 to load a guest value from guest state storage location 137 into global counter enable storage location 121 in connection with a VM entry. In box 224, host state load enable indicator 134 is set to cause host state load logic 184 to load a host value from host state storage location 138 into global counter enable storage location 121 in connection with a VM exit. In box 226, guest state store enable indicator 135 is set to cause guest state store logic 185 to store the contents of global enable storage location 121 in guest state storage location 136.
In box 230, a transfer of control of processor 120 from the host (i.e., VMM 140) to a guest is initiated. For example, the VMM may initiate a VM entry. The VM entry may include VM entry logic 181 causing processor 120 to execute operations or micro-operations to save the host state and load the guest state. In box 232, VM entry logic determines whether to load a guest value from guest state storage location 137 into global counter enable storage location 121 based on guest state load enable indicator 133. If guest state load enable indicator 133 is set, then, in box 234, guest state load logic 183 causes the guest value from guest state storage location 137 to be loaded into global counter enable storage location 121; otherwise, box 234 is not performed. In box 236, the VM entry is completed and control is transferred to the guest. In box 238, the guest begins or continues to execute.
In box 239, counter enable logic 170 determines whether the global counter enable indicator is set. If not, method 200 continues at box 258. If so, method 200 continues at box 240. In box 240, it is determined whether the first event has occurred. If so, then, in box 244, the first performance counter operates as configured, i.e., in this embodiment, where it has been configured in box 210 to count cache misses, the first performance counter increments.
In box 250, it is determined whether the second event has occurred. If so, then, in box 254, the second performance counter operates as configured, i.e., in this embodiment, where it has been configured in box 212 to count occurrences of the second event, the second performance counter increments.
In box 258, it is determined whether a virtualization event has occurred. If not, method 200 continues at box 238. If so, then, in box 260, a transfer of control of processor 120 from the guest to the host is initiated. The VM exit may include VM exit logic 182 causing processor 120 to execute operations or micro-operations to save the guest state and load the host state.
In box 262, VM exit logic determines whether to store the contents of global counter enable storage location 121 in guest state storage location 137 based on guest state store enable indicator 135. If guest state store enable indicator 135 is set, then, in box 264, guest state store logic 185 stores the contents of global counter enable storage location 121 in guest state storage location 137; otherwise, box 264 is not performed. In some embodiments, there is no guest state store enable indicator 135. In some embodiments, guest state store logic 185 always stores the contents of global counter enable storage location 121 in guest state storage location 137. In other embodiments, guest state store logic 185 never stores the contents of global counter enable storage location 121 in guest state storage location 137 as part of VM exit processing. In some embodiments, attempts by guest software to access the global counter enable storage location 121 may cause VM exits, either by software convention (e.g., the VMM is required to appropriately set controls in the VMCS to cause VM exits on access to global counter enable storage location 121) or by forcing such a VM exit by the processor.
In box 266, VM exit logic determines whether to load a host value from host state storage location 138 into global counter enable storage location 121 based on host state load enable indicator 134. If host state store enable indicator 134 is set, then, in box 268, host state load logic 184 causes the host value from host state storage location 138 to be loaded into global counter enable storage location 121; otherwise, box 268 is not performed.
In box 270, the VM exit is completed and control is transferred to the host. In box 272, the host begins or continues to execute.
In box 275, counter enable logic 170 determines whether the global counter enable indicator is set. If not, method 200 continues at box 298. If so, method 200 continues at box 280. In box 280, it is determined whether the first event has occurred. If so, then, in box 284, the first performance counter operates as configured, i.e., in this embodiment, where it has been configured in box 210 to count cache misses, the first performance counter increments.
In box 290, it is determined whether the second event has occurred. If so, then, in box 294, the second performance counter operates as configured, i.e., in this embodiment, where it has been configured in box 212 to count occurrences of the second event, the second performance counter increments.
In box 298, it is determined whether a VM entry is to occur. If so, method 200 continues at box 230. If not, method 200 continues at box 299. In box 299, it is determined whether a halt or other such instruction is to be executed. If not, method 200 continues at box 272. If so, method 200 ends.
FIG. 3 illustrates another embodiment of the invention in bare platform hardware 310, which may be used in virtualization architecture 100 or another virtualization architecture. Except as otherwise described, the description of bare platform hardware and its elements also applies to bare platform hardware 310, and its corresponding elements. Bare platform hardware 310 includes processor 320 and memory 330.
Processor 320 includes counters 371, 373, 375, and 377. Processor 320 also includes counter control storage locations 372, 374, 376, and 378, corresponding to counters 371, 373, 375, and 377, respectively, which each contain a counter valid storage location to store a counter valid indicator to enable or disable the corresponding counter, along with other control or configuration information. Additionally, processor 320 includes counter identifier storage locations 391, 393, 395, and 397, corresponding to counters 371, 373, 375, and 377, respectively. Counter identifier storage locations 391, 393, 395, and 397 may be registers or any other structures of any size, or portions of one or more registers of other structures, for storing information to identify or label counters 371, 373, 375, and 377, respectively. Counter identifier storage locations 391, 393, 395, and 397 may be programmed to store an identifier to uniquely or redundantly (i.e., two counter identifier storage locations could be programmed with the same value) identify counters 371, 373, 375, and 377, respectively.
VMCS 332 includes counter identifier field 333, which in this embodiment is the same number of bits as each of counter identifier storage location 391, 393, 395, and 397, but in other embodiments may be any number of bits. Other embodiments may include additional counter identifier fields. Counter identifier field 333 may be programmed, by the VMM that creates VMCS 332 or by any other software, to store a value that may or may not match one or more of the values stored in counter identifier storage locations 391, 393, 395, and 397.
Processor 320 also includes comparison logic 322, which may be implemented according to any known approach, to generate a counter match signal for each counter based on a comparison of the contents of a corresponding counter identifier storage location and the contents of counter identifier field 333. In this embodiment, the counter match signal for a counter is asserted if the contents of the corresponding counter identifier storage location match the contents of counter identifier field 333. In this embodiment, the counter match signal for a counter is also asserted if the contents of the corresponding counter identifier storage location equals a value of zero, regardless of the contents of counter identifier field 333, to provide for software that does not use the counter identifier feature of the present invention to operate as expected on hardware that does include the counter identifier feature. In another embodiment, the counter match logic may determine a “match” based at least in part on other factors such as the contents of other fields within the VMCS. For example, the value of various control registers such as the CRO, CR3 or CR4 registers in the Intel Architecture may be consulted. Alternatively, the counter match logic may include a determination of whether the software currently executing is the VMM or a guest, and enable the counters only if guest software is currently executing, based on one or more control bits in the VMCS.
Embodiments may include one or more “counter match enable” control bits or fields to enable or disable the counter match feature, by disabling the comparison logic or by any other desired approach. For example, if a counter match enable bit is set, then the comparison happens as described above. However, if the counter match enable bit is cleared, the counters are always enabled and no comparison is made, so that a “zero” value in a counter identifier storage location does not require the special treatment described above.
The counter match signal for each counter is used, along with any other relevant information stored in counter control storage locations 372, 374, 376, and 378, by counter enable logic 370 to control the operation of counters 371, 373, 375, and 377, respectively. If the counter match signal for a counter is asserted, then the counter operates according to its individual control and configuration information. For example, counter 371 may increment on a cache hit if the contents of counter control storage location 372 include an individual counter valid bit set to an “enable” value and an event select field set to count cache hits. However, if the counter match signal for a counter is not asserted, the counter is disabled and does not operate according to its individual control and configuration information. For example, counter 371 would not increment on a cache hit even if the contents of counter control storage location 372 include an individual counter valid bit set to an “enable” value and an event select field set to count cache hits.
Additionally, processor 320 includes control logic 380 to support virtualization, including the virtualization of counters 371, 373, 375, and 377. Control logic 380 causes processor 320 to execute method embodiments of the present invention, such as the method embodiments described below with reference to FIG. 4, for example, by causing processor 320 to include the execution of one or more micro-operations, e.g., to support virtualization, in its response to virtualization instructions or other instructions from a host or guest.
FIG. 4 illustrates an embodiment of the present invention in method 400, another method for virtualizing performance counters. Although method embodiments are not limited in this respect, reference is made to FIGS. 1 and 3 to describe the method embodiment of FIG. 4.
In box 410 of FIG. 4, a performance counter is configured to count occurrences of an event. For example, counter 371 may be configured, by programming counter control storage location 372, to count cache misses. In box 412, counter identifier storage location 391 is programmed with a desired counter identification value.
In box 420, a host creates a VMCS (e.g., VMCS 332) for a VM. In box 422, the host configures VMCS 332 to implement support for virtualizing counters 371, 373, 375, and 377, including programming counter identifier field 333 with a value equal to the counter identification value used on box 412. In this embodiment, any number of other counter identifier fields in VMCS 332 or any other VMCSes, corresponding to any number of other guests or hosts, may also be programmed with the same value to allow any number of guests or hosts to share one or more counters.
In box 430, control of processor 320 is transferred from the host to a guest. In box 432, the guest begins or continues to execute. In box 434, a counter match signal is generated based on a comparison of counter identifier storage location 391 and counter identifier field 333. In this embodiment, the counter match signal is asserted if the contents of the counter identifier storage location match the contents of the counter identifier field.
In box 440, an occurrence of the event, i.e., in this embodiment, a cache miss, is recognized. In box 442, counter enable logic 370 determines whether to increment the performance counter based on the counter match signal and the contents of the counter valid storage location in the corresponding counter control storage locations (e.g., counter control storage location 372).
If the counter match signal is asserted and the counter's counter valid indicator is set, then, in box 444, the performance counter operates as configured, i.e., in this embodiment, where it has been configured in box 410 to count cache misses, the performance counter increments. If the counter match signal is not asserted or the counter's valid indicator is not set, box 444 is not performed.
Within the scope of the present invention, the methods illustrated in FIGS. 2 and 4 may be performed in a different order, with illustrated boxes omitted, with additional boxes added, or with a combination of reordered, omitted, or additional boxes. For example, a VMCS may configure a VMCS in any order, e.g., boxes 222 to 226 may be rearranged in any order.
Processor 120, or any other component or portion of a component designed according to an embodiment of the present invention, may be designed in various stages, from creation to simulation to fabrication. Data representing a design may represent the design in a number of manners. First, as is useful in simulations, the hardware may be represented using a hardware description language or another functional description language. Additionally or alternatively, a circuit level model with logic and/or transistor gates may be produced at some stages of the design process. Furthermore, most designs, at some stage, reach a level where they may be modeled with data representing the physical placement of various devices. In the case where conventional semiconductor fabrication techniques are used, the data representing the device placement model may be the data specifying the presence or absence of various features on different mask layers for masks used to produce an integrated circuit.
In any representation of the design, the data may be stored in any form of a machine-readable medium. An optical or electrical wave modulated or otherwise generated to transmit such information, a memory, or a magnetic or optical storage medium, such as a disc, may be the machine-readable medium. Any of these media may “carry” or “indicate” the design, or other information used in an embodiment of the present invention. When an electrical carrier wave indicating or carrying the information is transmitted, to the extent that copying, buffering, or re-transmission of the electrical signal is performed, a new copy is made. Thus, the actions of a communication provider or a network provider may constitute the making of copies of an article, e.g., a carrier wave, embodying techniques of the present invention.
Thus, apparatuses, methods, and systems for virtualizing performance counters have been disclosed. While certain embodiments have been described, and shown in the accompanying drawings, it is to be understood that such embodiments are merely illustrative and not restrictive of the broad invention, and that this invention not be limited to the specific constructions and arrangements shown and described, since various other modifications may occur to those ordinarily skilled in the art upon studying this disclosure. In an area of technology such as this, where growth is fast and further advancements are not easily foreseen, the disclosed embodiments may be readily modifiable in arrangement and detail as facilitated by enabling technological advancements without departing from the principles of the present disclosure or the scope of the accompanying claims.

Claims (26)

What is claimed is:
1. An apparatus comprising:
a memory to store a plurality of counters and a counter enable storage location to store:
one or more counter enable bits or fields, each of which corresponds to, and is to control, one or more of the plurality of counters; and
one or more additional enable indicators, each of which corresponds to, and is to control, a respective subset of the one or more counter enable bits or fields; and
a processor to communicate with the memory to:
enable a counter among the plurality of counters responsive to:
a counter enable bit or field, among a first subset of the one or more counter enable bits or fields and corresponding to the counter, being configured to a respective enable value, and
an additional enable indicator, among the one or more additional enable indicators and corresponding to the first subset, being configured to a respective enable value,
disable the counter responsive to a configuration to a disable value of at least one of the counter enable bit or field or the additional enable indicator,
provide virtualization for the plurality of counters,
cause a guest value or a host value to be loaded into the counter enable storage location in connection with a transfer of control between a host and a guest, and
transfer control of the apparatus from the host to the guest if the guest value is loaded and from the guest to the host if the host value is loaded.
2. The apparatus of claim 1, wherein the processor is further to disable use of the counter by the host responsive to the guest value being in the counter enable bit or field or the additional enable indicator.
3. The apparatus of claim 1, wherein the plurality of counters comprises performance counters to log performance monitor information.
4. The apparatus of claim 1, wherein the plurality of counters comprises performance counters to count a selected event.
5. The apparatus of claim 1, wherein the selected event is selected from a branch misprediction, a cache hit, a cache miss, a translation lookaside buffer hit, and a translation lookaside buffer miss.
6. The apparatus of claim 1, wherein the processor is further to:
cause the guest value to be loaded into the counter enable storage location in connection with the transfer of control from the host to the guest; and
transfer control of the apparatus from the host to the guest.
7. The apparatus of claim 1, wherein the processor is further to:
cause the host value to be loaded into the counter enable storage location in connection with the transfer of control from the guest to the host; and
transfer control of the apparatus from the guest to the host.
8. A non-transitory computer-readable storage medium having instructions stored thereon that, when executed by a processor, cause the processor to:
enable a counter among a plurality of counters responsive to:
a counter enable bit or field, corresponding to the counter, being configured to a respective enable value, and
an additional enable indicator, corresponding to the counter enable bit or field, being configured to a respective enable value; and
disable the counter, by the processor in an apparatus, responsive to configuration to a disable value of at least one of the counter enable bit or field or the additional enable indicator;
provide virtualization for the plurality of counters;
cause a guest value or a host value to be loaded into a counter enable storage location in connection with a transfer of control between a host and a guest; and
transfer control of the apparatus from the host to the guest if the guest value is loaded and from the guest to the host if the host value is loaded.
9. The computer-readable storage medium of claim 8, wherein the instructions are further to cause the processor to disable use of the counter by the host responsive to the guest value being in the counter enable bit or field or the additional enable indicator.
10. The computer-readable storage medium of claim 8, wherein the plurality of counters comprises performance counters to log performance monitor information.
11. The computer-readable storage medium of claim 8, wherein the plurality of counters comprises performance counters to count a selected event.
12. The computer-readable storage medium of claim 11, wherein the selected event is selected from a branch misprediction, a cache hit, a cache miss, a translation lookaside buffer hit, and a translation lookaside buffer miss.
13. The computer-readable storage medium of claim 8, wherein the instructions are further to cause the processor to:
cause the guest value to be loaded into the counter enable storage location in connection with the transfer of control from the host to the guest; and
transfer control of the apparatus from the host to the guest.
14. The computer-readable storage medium of claim 8, wherein the instructions are further to cause the processor to:
cause the host value to be loaded into the counter enable storage location in connection with the transfer of control from the guest to the host; and
transfer control of the apparatus from the guest to the host.
15. A method comprising:
enabling a counter among a plurality of counters responsive to:
a counter enable bit or field, corresponding to the counter, being configured to a respective enable value, and
an additional enable indicator, corresponding to the counter enable bit or field, being configured to a respective enable value; and
disabling the counter, by a processor in an apparatus, responsive to configuration to a disable value of at least one of the counter enable bit or field or the additional enable indicator;
providing virtualization for the plurality of counters;
causing a guest value or a host value to be loaded into a counter enable storage location in connection with a transfer of control between a host and a guest; and
transferring control of the apparatus from the host to the guest if the guest value is loaded and from the guest to the host if the host value is loaded.
16. The method of claim 15, further comprising disabling use of the counter by the host responsive to the guest value being in the counter enable bit or field or the additional enable indicator.
17. The method of claim 15, wherein the plurality of counters comprises performance counters to log performance monitor information.
18. The method of claim 15, wherein the plurality of counters comprises performance counters to count a selected event, wherein the selected event is selected from a branch misprediction, a cache hit, a cache miss, a translation lookaside buffer hit, and a translation lookaside buffer miss.
19. The method of claim 15, further comprising:
causing the guest value to be loaded into the counter enable storage location in connection with the transfer of control from the host to the guest; and
transferring control of the apparatus from the host to the guest.
20. The method of claim 15, further comprising:
causing the host value to be loaded into the counter enable storage location in connection with the transfer of control from the guest to the host; and
transferring control of the apparatus from the guest to the host.
21. A processor comprising:
a plurality of counters;
counter control storage locations to store a plurality of counter enable bits or fields, wherein each of the counter enable bits or fields corresponds to one of the plurality of counters;
a global counter enable storage location to store a global counter enable indicator;
counter enable logic to:
enable a counter among the plurality of counters in response to a respective enable value being set in:
a counter enable bit or field, among the plurality of counter enable bits or fields and corresponding to the counter, and
the global counter enable indicator; and
disable the counter in response to a disable value being set in at least one of the counter enable bit or field or the global counter enable indicator; and
control logic to:
support virtualization for the plurality of counters; and
enable assignment of the plurality of counters for exclusive use by a host or a guest.
22. The processor of claim 21, wherein the counter control storage locations are further to store information to configure the plurality of counters to count corresponding events.
23. The processor of claim 22, wherein the corresponding events comprise one or more of a cache miss, a cache hit, a branch misprediction, a translation lookaside buffer hit, or a translation lookaside buffer miss.
24. A handheld device comprising:
a memory to store monitor software or firmware to manage an execution environment for guest software; and
a processor to communicate with the memory to:
store a plurality of counters,
store a plurality of counter enable bits or fields in respective counter control storage locations, wherein each of the counter enable bits or fields corresponds to one of the plurality of counters,
store a global counter enable indicator in a global counter enable storage location,
enable a counter among the plurality of counters in response to a respective enable value being set in:
a counter enable bit or field, among the plurality of counter enable bits or fields and corresponding to the counter, and
the global counter enable indicator;
disable the counter in response to a disable value being set in at least one of the counter enable bit or field or the global counter enable indicator;
support virtualization for the plurality of counters; and
enable assignment of the plurality of counters for exclusive use by a host or a guest.
25. The handheld device of claim 24, wherein the processor is further to store information to configure the plurality of counters to count corresponding events.
26. The handheld device of claim 25, wherein the corresponding events comprise one or more of a cache miss, a cache hit, a branch misprediction, a translation lookaside buffer hit, or a translation lookaside buffer miss.
US14/060,947 2006-08-08 2013-10-23 Virtualizing performance counters Active 2027-01-09 US9244712B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US14/060,947 US9244712B2 (en) 2006-08-08 2013-10-23 Virtualizing performance counters

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US11/501,632 US8607228B2 (en) 2006-08-08 2006-08-08 Virtualizing performance counters
US14/060,947 US9244712B2 (en) 2006-08-08 2013-10-23 Virtualizing performance counters

Related Parent Applications (2)

Application Number Title Priority Date Filing Date
US11/501,632 Continuation US8607228B2 (en) 2006-08-08 2006-08-08 Virtualizing performance counters
US11501362 Continuation 2006-08-09

Publications (2)

Publication Number Publication Date
US20140053155A1 US20140053155A1 (en) 2014-02-20
US9244712B2 true US9244712B2 (en) 2016-01-26

Family

ID=39052292

Family Applications (2)

Application Number Title Priority Date Filing Date
US11/501,632 Active 2031-05-02 US8607228B2 (en) 2006-08-08 2006-08-08 Virtualizing performance counters
US14/060,947 Active 2027-01-09 US9244712B2 (en) 2006-08-08 2013-10-23 Virtualizing performance counters

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US11/501,632 Active 2031-05-02 US8607228B2 (en) 2006-08-08 2006-08-08 Virtualizing performance counters

Country Status (7)

Country Link
US (2) US8607228B2 (en)
JP (3) JP2010500664A (en)
CN (2) CN101490646B (en)
BR (1) BRPI0715394A2 (en)
DE (1) DE112007001714T5 (en)
TW (1) TWI366789B (en)
WO (1) WO2008021794A1 (en)

Families Citing this family (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8607228B2 (en) 2006-08-08 2013-12-10 Intel Corporation Virtualizing performance counters
WO2008117805A1 (en) * 2007-03-26 2008-10-02 Mitsui Chemicals, Inc. Mixed continuous fiber nonwoven fabric and process for manufacturing the same
US8032892B2 (en) * 2007-06-26 2011-10-04 International Business Machines Corporation Message passing with a limited number of DMA byte counters
US8595739B2 (en) * 2008-07-07 2013-11-26 International Business Machines Corporation Prioritized resource scanning
US9449314B2 (en) * 2008-10-02 2016-09-20 International Business Machines Corporation Virtualization of a central processing unit measurement facility
US8650562B2 (en) * 2009-06-12 2014-02-11 International Business Machines Corporation Method and apparatus for scalable monitoring of virtual machine environments combining base virtual machine and single monitoring agent for measuring common characteristics and individual virtual machines measuring individualized characteristics
GB2474666B (en) * 2009-10-21 2015-07-15 Advanced Risc Mach Ltd Hardware resource management within a data processing system
US20110107328A1 (en) * 2009-11-02 2011-05-05 Advanced Micro Devices, Inc. Virtual machine device and methods thereof
US8798962B2 (en) 2011-03-21 2014-08-05 International Business Machines Corporation Virtualized abstraction with built-in data alignment and simultaneous event monitoring in performance counter based application characterization and tuning
US9063762B2 (en) * 2011-08-26 2015-06-23 Vmware, Inc. Methods, apparatus, and articles of manufacture to virtualize performance counters
JP5962359B2 (en) * 2012-09-10 2016-08-03 富士通株式会社 Processor and processor evaluation method
US10558437B1 (en) 2013-01-22 2020-02-11 Altera Corporation Method and apparatus for performing profile guided optimization for high-level synthesis
US9176830B2 (en) * 2013-05-24 2015-11-03 Hyundai Motor Company Method for determining software error in virtualization based integrated control system
US9298651B2 (en) * 2013-06-24 2016-03-29 International Business Machines Corporation Continuous in-memory accumulation of hardware performance counter data
JP6194016B2 (en) * 2013-12-06 2017-09-06 株式会社日立製作所 Computer and load measuring method
US9304874B2 (en) 2014-02-03 2016-04-05 Red Hat Israel, Ltd. Virtual machine-guest driven state restoring by hypervisor
US9213569B2 (en) * 2014-03-27 2015-12-15 International Business Machines Corporation Exiting multiple threads in a computer
US9223574B2 (en) * 2014-03-27 2015-12-29 International Business Machines Corporation Start virtual execution instruction for dispatching multiple threads in a computer
US9760302B2 (en) 2014-10-03 2017-09-12 International Business Machines Corporation Servicing multiple counters based on a single access check
US9477471B1 (en) * 2015-07-07 2016-10-25 Versa Networks, Inc. Maintaining lock-free, high frequency, high performance counters in software
CN106775919B (en) * 2015-11-24 2020-03-13 龙芯中科技术有限公司 State information control method and device
US9904587B1 (en) * 2015-12-18 2018-02-27 Amazon Technologies, Inc. Detecting anomalous behavior in an electronic environment using hardware-based information
US20210026950A1 (en) * 2016-03-07 2021-01-28 Crowdstrike, Inc. Hypervisor-based redirection of system calls and interrupt-based task offloading
US10698873B2 (en) 2017-03-30 2020-06-30 Microsoft Technology Licensing, Llc Performance data storage
US11822411B2 (en) * 2021-05-06 2023-11-21 Apple Inc. Telemetry push aggregation

Citations (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US274090A (en) 1883-03-13 feldmann
US341757A (en) 1886-05-11 Compound for preventing window-frost
US4225059A (en) 1978-12-14 1980-09-30 Christopher Kappos Portable beverage cooler and dispenser
US4280330A (en) 1977-09-19 1981-07-28 Verdell Harris Vehicle heating and cooling system
US4384663A (en) 1981-07-27 1983-05-24 Smith Williams Margie M Mounting brackets for trailer tool box
US4496016A (en) 1981-11-11 1985-01-29 Derek Unsworth Land vehicles
JPS6320549A (en) 1986-07-14 1988-01-28 Hitachi Ltd Monitor information collecting system
JPS63301334A (en) 1987-06-02 1988-12-08 Fujitsu Ltd Measuring system for performance of virtual computer
US4975836A (en) 1984-12-19 1990-12-04 Hitachi, Ltd. Virtual computer system
JPH03269640A (en) 1990-03-19 1991-12-02 Nec Corp Virtual computer system
US5307645A (en) 1991-07-02 1994-05-03 Pannell Bobby L Air conditioning system for a recreational vehicle
US5584188A (en) 1995-05-25 1996-12-17 Tippmann; Joseph R. Refrigerated beverage trailer
US20030037089A1 (en) 2001-08-15 2003-02-20 Erik Cota-Robles Tracking operating system process and thread execution and virtual machine execution in hardware or in a virtual machine monitor
EP1331564A1 (en) 2002-01-24 2003-07-30 Siemens Aktiengesellschaft Fuzzy logic based intelligent load control for distributed environment
US6619065B1 (en) 2002-11-01 2003-09-16 Bruce H. Burton Heating ventilating and air conditioning worker's servicing cart
WO2003090070A2 (en) 2002-04-16 2003-10-30 Intel Corporation Control register access virtualization performance improvement in the virtual-machine architecture
US20040123288A1 (en) 2002-12-19 2004-06-24 Intel Corporation Methods and systems to manage machine state in virtual machine operations
US20040123228A1 (en) 2002-08-27 2004-06-24 Atsushi Kikuchi Coding apparatus, coding method, decoding apparatus, and decoding method
JP2004318538A (en) 2003-04-17 2004-11-11 Fujitsu Ltd Method for monitoring performance
US20050033766A1 (en) 2003-06-27 2005-02-10 Microsoft Corporation Method and framework for providing system performance information
US20050076186A1 (en) 2003-10-03 2005-04-07 Microsoft Corporation Systems and methods for improving the x86 architecture for processor virtualization, and software systems and methods for utilizing the improvements
US20050166039A1 (en) 2003-02-19 2005-07-28 Hong Wang Programmable event driven yield mechanism which may activate other threads
US20050183065A1 (en) 2004-02-13 2005-08-18 Wolczko Mario I. Performance counters in a multi-threaded processor
US20050223199A1 (en) 2004-03-31 2005-10-06 Grochowski Edward T Method and system to provide user-level multithreading
WO2006039202A2 (en) 2004-09-30 2006-04-13 Intel Corporation Providing support for a timer assocated with a virtual machine monitor
US20060136653A1 (en) 2004-12-21 2006-06-22 Microsoft Corporation Systems and methods for exposing processor topology for virtual machines
US7069413B1 (en) 2003-01-29 2006-06-27 Vmware, Inc. Method and system for performing virtual to physical address translations in a virtual machine monitor
US20070083643A1 (en) 2005-10-11 2007-04-12 International Business Machines Corporation Performance counters for virtualized network interfaces of communications networks
WO2008021794A1 (en) 2006-08-08 2008-02-21 Intel Corporation Virtualizing performance counters
US7392370B2 (en) 2004-01-14 2008-06-24 International Business Machines Corporation Method and apparatus for autonomically initiating measurement of secondary metrics based on hardware counter values for primary metrics
US7406834B2 (en) 2004-12-27 2008-08-05 Dwight Williams Self-contained mobile walk-in cooler
US20080270770A1 (en) 2005-01-24 2008-10-30 Marc Vertes Method for Optimising the Logging and Replay of Mulit-Task Applications in a Mono-Processor or Multi-Processor Computer System
US7788664B1 (en) 2005-11-08 2010-08-31 Hewlett-Packard Development Company, L.P. Method of virtualizing counter in computer system
US20110093750A1 (en) 2009-10-21 2011-04-21 Arm Limited Hardware resource management within a data processing system
US20110107328A1 (en) 2009-11-02 2011-05-05 Advanced Micro Devices, Inc. Virtual machine device and methods thereof

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0833851B2 (en) * 1986-03-15 1996-03-29 株式会社日立製作所 Virtual machine monitoring method
US7552426B2 (en) * 2003-10-14 2009-06-23 Microsoft Corporation Systems and methods for using synthetic instructions in a virtual machine

Patent Citations (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US274090A (en) 1883-03-13 feldmann
US341757A (en) 1886-05-11 Compound for preventing window-frost
US4280330A (en) 1977-09-19 1981-07-28 Verdell Harris Vehicle heating and cooling system
US4225059A (en) 1978-12-14 1980-09-30 Christopher Kappos Portable beverage cooler and dispenser
US4384663A (en) 1981-07-27 1983-05-24 Smith Williams Margie M Mounting brackets for trailer tool box
US4496016A (en) 1981-11-11 1985-01-29 Derek Unsworth Land vehicles
US4975836A (en) 1984-12-19 1990-12-04 Hitachi, Ltd. Virtual computer system
JPS6320549A (en) 1986-07-14 1988-01-28 Hitachi Ltd Monitor information collecting system
JPS63301334A (en) 1987-06-02 1988-12-08 Fujitsu Ltd Measuring system for performance of virtual computer
JPH03269640A (en) 1990-03-19 1991-12-02 Nec Corp Virtual computer system
US5307645A (en) 1991-07-02 1994-05-03 Pannell Bobby L Air conditioning system for a recreational vehicle
US5584188A (en) 1995-05-25 1996-12-17 Tippmann; Joseph R. Refrigerated beverage trailer
US20030037089A1 (en) 2001-08-15 2003-02-20 Erik Cota-Robles Tracking operating system process and thread execution and virtual machine execution in hardware or in a virtual machine monitor
EP1331564A1 (en) 2002-01-24 2003-07-30 Siemens Aktiengesellschaft Fuzzy logic based intelligent load control for distributed environment
WO2003062989A1 (en) 2002-01-24 2003-07-31 Siemens Aktiengesellschaft Fuzzy logic based intelligent load control for multimedia and telecommunication systems
WO2003090070A2 (en) 2002-04-16 2003-10-30 Intel Corporation Control register access virtualization performance improvement in the virtual-machine architecture
US20030217250A1 (en) 2002-04-16 2003-11-20 Steve Bennett Control register access virtualization performance improvement in the virtual-machine architecture
CN1659518A (en) 2002-04-16 2005-08-24 英特尔公司 Control register access virtualization performance improvement in the virtual-machine architecture
US20040123228A1 (en) 2002-08-27 2004-06-24 Atsushi Kikuchi Coding apparatus, coding method, decoding apparatus, and decoding method
US6619065B1 (en) 2002-11-01 2003-09-16 Bruce H. Burton Heating ventilating and air conditioning worker's servicing cart
US20040123288A1 (en) 2002-12-19 2004-06-24 Intel Corporation Methods and systems to manage machine state in virtual machine operations
US7069413B1 (en) 2003-01-29 2006-06-27 Vmware, Inc. Method and system for performing virtual to physical address translations in a virtual machine monitor
US20050166039A1 (en) 2003-02-19 2005-07-28 Hong Wang Programmable event driven yield mechanism which may activate other threads
JP2004318538A (en) 2003-04-17 2004-11-11 Fujitsu Ltd Method for monitoring performance
US20050033766A1 (en) 2003-06-27 2005-02-10 Microsoft Corporation Method and framework for providing system performance information
US20050076186A1 (en) 2003-10-03 2005-04-07 Microsoft Corporation Systems and methods for improving the x86 architecture for processor virtualization, and software systems and methods for utilizing the improvements
US7392370B2 (en) 2004-01-14 2008-06-24 International Business Machines Corporation Method and apparatus for autonomically initiating measurement of secondary metrics based on hardware counter values for primary metrics
US20050183065A1 (en) 2004-02-13 2005-08-18 Wolczko Mario I. Performance counters in a multi-threaded processor
US20050223199A1 (en) 2004-03-31 2005-10-06 Grochowski Edward T Method and system to provide user-level multithreading
WO2006039202A2 (en) 2004-09-30 2006-04-13 Intel Corporation Providing support for a timer assocated with a virtual machine monitor
US20060136653A1 (en) 2004-12-21 2006-06-22 Microsoft Corporation Systems and methods for exposing processor topology for virtual machines
KR20060071307A (en) 2004-12-21 2006-06-26 마이크로소프트 코포레이션 Systems and methods for exposing processor topology for virtual machines
US7406834B2 (en) 2004-12-27 2008-08-05 Dwight Williams Self-contained mobile walk-in cooler
US20080270770A1 (en) 2005-01-24 2008-10-30 Marc Vertes Method for Optimising the Logging and Replay of Mulit-Task Applications in a Mono-Processor or Multi-Processor Computer System
US20070083643A1 (en) 2005-10-11 2007-04-12 International Business Machines Corporation Performance counters for virtualized network interfaces of communications networks
US7788664B1 (en) 2005-11-08 2010-08-31 Hewlett-Packard Development Company, L.P. Method of virtualizing counter in computer system
WO2008021794A1 (en) 2006-08-08 2008-02-21 Intel Corporation Virtualizing performance counters
US20110093750A1 (en) 2009-10-21 2011-04-21 Arm Limited Hardware resource management within a data processing system
US20110107328A1 (en) 2009-11-02 2011-05-05 Advanced Micro Devices, Inc. Virtual machine device and methods thereof

Non-Patent Citations (15)

* Cited by examiner, † Cited by third party
Title
"IA-32 Intel Architecture Software Developers Manual vol. 2: System Programming Guide", Japan, Intel Corporation, 2004, pp. 15-23, 15-25 to 15-33.
Chinese Patent Office, Office Action mailed Apr. 25, 2012 in Chinese application No. 200780025942.3.
IBM and Motorola, "PowerPC 604e, RISC Microprocessor User's Manual," Mar. 1998, pp. 9-1 to 9-15.
International Preliminary Report on Patentability and Written Opinion received for PCT Application No. PCT/US2007/075286, mailed on Feb. 19, 2009 , 8 Pages.
International Search Report and Written Opinion received for PCT Patent Application No. PCT/US2007/075286, mailed on Dec. 28, 2007, 11 Pages.
Japanese Patent Office, Final Notice of Reasons for Rejection mailed Jun. 5, 2012 in Japanese application No. 2009-523936.
Notice of Allowance received for Japanese Patent Application No. 2013-022629, mailed on May 27, 2014, 1 pages of Japanese NOA only (No English Translation Available).
Office Action Received for Chinese Patent Application No. 200780025942.3, mailed on Jun. 22, 2011, 11 pages of Chinese Office Action including 5 pages of English Translation.
Office Action Received for Chinese Patent Application No. 200780025942.3, mailed on May 10, 2010, 6 pages of Office Action and 8 pages of English Translation.
Office Action received for Chinese Patent Application No. 2007800259423, mailed on Nov. 28, 2012, 3 pages of Office Action and 3 pages of English Translation.
Office Action Received for German Patent Application No. 11 2007 001 714.7-53, mailed on May 14, 2010, 8 Pages of Office Action including English Translation.
Office Action Received for Japanese Patent Application No. 2009-523936, mailed on Dec. 13, 2011, 3 pages of Japanese Office Action and 3 pages of English Translation.
Office Action received for Japanese Patent Application No. 2009-523936, mailed on Oct. 9, 2012, 1 pages of Japanese Office Action and 3 pages of English Translation.
Office Action received for Japanese Patent Application No. 2013-022629, mailed on Feb. 18, 2014, 3 pages of English Translation and 3 pages of Japanese Office Action.
Office Action Received for Taiwanese Patent Application No. 96129034, mailed on Oct. 4, 2011, 7 pages of Taiwanese Office Action and 9 pages of English Translation.

Also Published As

Publication number Publication date
US20080040715A1 (en) 2008-02-14
BRPI0715394A2 (en) 2013-06-25
JP2013109777A (en) 2013-06-06
CN103500137B (en) 2017-01-11
CN103500137A (en) 2014-01-08
CN101490646B (en) 2013-10-16
TWI366789B (en) 2012-06-21
US20140053155A1 (en) 2014-02-20
US8607228B2 (en) 2013-12-10
JP5571208B2 (en) 2014-08-13
CN101490646A (en) 2009-07-22
DE112007001714T5 (en) 2009-06-25
TW200825904A (en) 2008-06-16
JP5969550B2 (en) 2016-08-17
JP2010500664A (en) 2010-01-07
JP2014211891A (en) 2014-11-13
WO2008021794A1 (en) 2008-02-21

Similar Documents

Publication Publication Date Title
US9244712B2 (en) Virtualizing performance counters
US8230203B2 (en) Detecting spin loops in a virtual machine environment
US8819699B2 (en) Controlling virtual machines based on performance counters
US7802250B2 (en) Support for transitioning to a virtual machine monitor based upon the privilege level of guest software
US8312452B2 (en) Method and apparatus for a guest to access a privileged register
US10296366B2 (en) Virtualization exceptions
US7506121B2 (en) Method and apparatus for a guest to access a memory mapped device
US20180129619A1 (en) Virtualizing interrupt prioritization and delivery
US8151264B2 (en) Injecting virtualization events in a layered virtualization architecture
US8291410B2 (en) Controlling virtual machines based on activity state
EP1750199A1 (en) Virtualization event processing in a layered virtulization architecture
TWI620124B (en) Virtual machine control structure shadowing
US8370559B2 (en) Executing a protected device model in a virtual machine
CN110574009B (en) Apparatus and method for managing use of capabilities
US9747118B2 (en) Guest-specific microcode

Legal Events

Date Code Title Description
FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 8