(12)
`
`United States Patent
`US 6,816,809 B2
`(10) Patent No.:
`Circenis
`(45) Date of Patent:
`Nov. 9, 2004
`
`
`US006816809B2
`
`(54) HARDWARE BASED UTILIZATION
`METERING
`Inventor: Edgar Circenis, Loveland, CO (US)
`
`(75)
`
`(73) Assignee: Hewlett-Packard Development
`Company, L.P., Houston, TX (US)
`.
`.
`oo,
`Subjectto any disclaimer, the term of this
`patent is extended or adjusted under 35
`US.C. 154(b) by 64 days.
`
`.
`(*) Notice:
`
`(21) Appl. No.: 10/200,175
`
`5,654,905 A
`8/1997 Mulholland
`6,049,798 A *
`4/2000 Bishop et al... 707/10
`FOREIGN PATENT DOCUMENTS
`0320329
`6/1989
`
`EP
`
`OTHER PUBLICATIONS
`Search Reportissued on Jan. 14, 2004 in counterpart foreign
`application in GB under application No. 0316794.7.
`.
`.
`* cited by examiner
`Primary Examiner—Bryan Bui
`Assistant Examiner—Hien Vo
`
`Filed:
`
`(22)
`(65)
`
`ABSTRACT
`(57)
`Jul. 23, 2002
`A hardware based utilization metering device, and a corre-
`Prior Publication Data
`sponding method are used in a computer system having one
`US 2004/0019456 A1 Jan. 29, 2004
`or more central processor units (CPUs) to provide a measure
`,
`of CPU utilization. The device includes a state indicator
`Tint. CU? ee cceeeeeeeeeceeeeeeneeeeeenenes G04F 1/00
`(S51)
`coupled to a CPU. Thestate indicator receives an indication
`(52) U.S. Ch cece 702/178; 702/186; 710/107;
`710/260; 709/200 when the CPUis inafirst state, which may bea busystate.
`
`(58) Field of Search...0c 702/178, 186,|A counter coupled to the state indicator and coupled to a
`702/176; 710/260, 107, 241, 100; 709/104,
`system clock, receives a measure of system time from the
`102, 107, 1, 103, 106, 226, 208, 200
`system clock and receives data related to the indication
`when the CPU is in the first state, and generates a counter
`value indicative of time the CPUis in the first state. A data
`usage provider coupled to the counter maintains a non-
`volatile value of the counter value.
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`4,503,495 A *
`4,511,960 A *
`
`3/1985 Boudreau... 710/241
`4/1985 Boudreau .........c.ccc 711/219
`
`20 Claims, 6 Drawing Sheets
`
`iw
`
`WTEREACE ~~ 160
`
`USAGEDATA PROMIDER
`
`
`COUNTER
`
`|e=
`
` COUNTER
`L HEoe| haeCOUNTER
`VALUE N2
`
`
`
`
`VALUES
`
`uo)
`
`
`
`Ei
`
`1
`
`a
`
`24)
`
`210
`
`‘)
`
`dit
`
`4a
`
`
`
`
`ewefaos|| Du
`
`
`
`
`yy (EllK
`Le
`COL
`at)
`CAINS
`mace
`aa
`Oh
`COUNTER
`ae
`VUE
`rH
`| a
`
`ut NDKATOR|941 ut a wt ui NDICATORT a4
`
`
`
`(De
`IDE
`a
`
`
`
`eal i
`
`at
`
`Google Exhibit 1001
`Google Exhibit 1001
`Google v. Valtrus
`Google v. Valtrus
`
`

`

`U.S. Patent
`
`Nov.9, 2004
`
`Sheet 1 of 6
`
`US 6,816,809 B2
`
`USAGE DATA
`PROVIDER
`
`INTERFACE
`
`COUNTER
`
`COUNTER
`VALUE:
`
`3453457897345
`
`INDICATOR
`
`SYSTEM
`CLOCK
`
`|
`
`IDLE
`
`FIG. 1A
`
`

`

`U.S. Patent
`
`Nov.9, 2004
`
`Sheet 2 of 6
`
`US 6,816,809 B2
`
`USAGE DATA
`PROVIDER
`
`INTERFACE
`
`
`
`
`
`COMMUNICATION
`CHANNEL
`
`FIG. 1B
`
`

`

`U.S. Patent
`
`Nov.9, 2004
`
`Sheet 3 of 6
`
`US 6,816,809 B2
`
`PROCESSOR
`
`USAGE DATA
`PROVIDER
`
`NON
`VOLATILE
`
`INTERFACE
`
`COUNTER
`
`PROCESSOR
`
`COUNTER
`VALUE:
`
`3453457897345
`
`INDICATOR
`
`SYSTEM
`CLOCK
`
`IDLE
`
`FIG. 1C
`
`

`

`U.S. Patent
`
`Nov.9, 2004
`
`Sheet 4 of 6
`
`US 6,816,809 B2
`
`100"
`
`139
`
`SYSTEM
`CLOCK
`
`440
`
`110
`
`COUNTER1
`COUNTER]|| CPU
`VALUE
`
`150’
`
`
`
`| COUNTER 3||
`
`14
`IDLE
`190~c|INDICATOR
`
`460
`vad
`INTERFACE
`
`USAGE DATA
`PROVIDER
`
`141
`
`142
`
`
`
`
`COUNTER 2
`COUNTER
`VALUE
`
`CPU 2
`
`121
`
`IDLE
`INDICATOR
`
`12
`
`
`
`COUNTER
`VALUE
`
`CPU 3
`
`|
`.
`IDLE
`
`
`122~JINDICATOR | 444
`
`143~4
`
`COUNTER N
`
`
`
`COUNTER
`VALUE
`
`CPUN
`
`FIG. 2
`
`193-~c)
`
`IDLE
`
`INDICATOR
`
`

`

`U.S. Patent
`
`Nov.9, 2004
`
`Sheet 5 of 6
`
`US 6,816,809 B2
`
`io
`
`
`
`COUNTER
`
`COUNTER
`
`COUNTER
`
`COUNTER
`VALUE N2
`
`
`COUNTER
`
`INTERFACE. ~~ 16
`
`
`USAGE DATA PROVIDER
`
`
`
`
`COUNTER
`o>====m=
`| VALENS
`VALUES| COUNTER
`
`al VALUEf
`
`
`
`
`
`aeiCOUNTER
`P| VALUE2
`
`
`
`|| VALUEMH
`|| VALUESueVALLE4
`
`
`COUNTER
`COUNTER
`
`
`
`
`VALUE N
`VALUES
`
`
`
`yy ll on
`wg Le
` COUNTERIa
`
`bceaeCOUNTERI.(U3
`
`
`
`aes DE
`0
`i
`ta
`4
`NOICATORI™
`a
`
`
`
`ikea (Pd Ea
`ele
`wotcaToR
`mepte
`
`ay
`sei
`oepicid
`“ce CPU
`wc i
`son
`
`
`
`te[iSaCPUE
`
`Ri]
`39
`
`)e
`
`INDICATOR
`
`FIG. 3
`
`

`

`U.S. Patent
`
`Nov.9, 2004
`
`Sheet 6 of 6
`
`US 6,816,809 B2
`
`START
`
`610
`
`REPLACE CELL/
`POWER UP
`
`
`
`
`
`RECEIVE CELL/
`POWER UP
`SIGNAL, SEND
`NON-VOLATILE
`COUNTER VALUES
`
`
`
`
`
`REINITIALIZE
`COUNTERS
`
`Oo OQ =)|
`
`620
`
`630
`
`640
`
`PROVIDEIDLE
`INDICATION
`
`650
`
`660if”
`
`CPU RUNS
`
`|
`
`RECEIVE NON-
`IDLE INDICATION
`
`CPU HALT
`
`COUNTER RECEIVES
`IDLE INDICATION
`
`670
`
`680
`
`690
`
`FIG. 4
`
`

`

`US 6,816,809 B2
`
`1
`HARDWAREBASED UTILIZATION
`METERING
`
`TECHNICAL FIELD
`
`The technical field is pay-per-use systems and methods
`that use central processor metering to determine processor
`utilization for billing and other purposes.
`BACKGROUND
`
`Many computer system users are turning to a pay-per-use
`concept in which the user is billed based on some measure
`of processor utilization within the computer system. A
`measure of processorutilization may be based on a metering
`scheme that determines when a processor is in use. Such
`metering of processor utilization is currently accomplished
`by software running within the computer system’s operating
`system. In a computer system having hardware that may be
`partitioned, gathering processorutilization data from a hard-
`ware system requires communications between the metering
`application and all operating systems running within the
`hardware. The need for communication with different oper-
`ating systems posessignificant challenges because operating
`systems by their design are separated from other operating
`systems and do not have visibility to utilization data from
`other operating systems. Furthermore, the way partitioned
`systems are deployed, network connectivity may not be
`possible between different operating systems on the same
`hardware, further complicating the ability to accurately
`measure processor utilization. Current solutions involve
`communicating with each operating system independently,
`and then aggregating the information at a later time. This
`solution may be troublesome when network access is
`restricted, agent software is not
`installed, and operating
`systems are temporarily out of service, for example. This
`solution also requires the developmentof operating system-
`specific agents because each partition is capable of running
`different instances of the operating systems. In particular,
`since operating systems within a partionable computer sys-
`tem do not, by default, communicate with each other, data
`providers (software agents) are needed to run on each
`operating system to collect and transmit, or otherwise pro-
`vide utilization data from each partition. The thus-collected
`utilization data are aggregated to provide an overall value
`for processor utilization.
`SUMMARY
`
`Also disclosed is a hardware based method for measuring
`processor utilization in a computer system comprising a
`plurality of processors, the method comprising determining
`when any of the plurality of processors is busy (i.e., not
`idle); providing a busy indication to a counter associated
`with the busy processor; receiving at the counter a measure
`
`Pay-per-use systems allow computer users to acquire a
`given computing capacity that may be tailored to the user’s
`specific need by, amongother things, charging the user only
`for actual utilization of the system central processor units
`(CPUs). A measure of CPU utilization may be based on a
`Whatis disclosed is a hardware based utilization metering
`metering scheme that determines whena processoris in use
`50
`device for use in a computer system having one or more
`or busy. Such metering of CPU utilization is currently
`central processor units (CPUs), the device comprisingastate
`accomplished by software running within the computer
`indicator coupled to a CPU, wherein the state indicator
`system’s operating system. This approach of CPU metering
`receives an indication when the CPUis in a first state; a
`using software that runs within the operating system is
`counter coupled to the state indicator and coupled to a
`acceptable for CPUs running single instances (i.e., type or
`system clock, wherein the counter receives a measure of
`version) of operating systems. For example, such a scheme
`system time from the system clock and receives data related
`would be acceptable for a computer system running only
`to the indication when the CPU is in the first state, and
`UNIX or only Windows®. However, for computer systems
`generates a counter value indicative of time the CPU isin the
`that may be hardware and software partitionable, gathering
`first state; and a data usage provider coupled to the counter,
`CPU utilization data from a hardware system requires com-
`wherein the data usage provider is capable of providing the
`munications between the metering application and all oper-
`counter value.
`ating systems running within the hardware. The need for
`communication with different operating systems poses sig-
`nificant challenges because operating systems by their
`design are separated from other operating systems and do
`not have visibility to utilization data from other operating
`systems. Furthermore,
`the way partitioned systems are
`deployed, network connectivity may not be possible
`
`10
`
`15
`
`30
`
`2
`of computer system time; incrementing a counter value in
`the counter based on the received busy indication and an
`amount of computer system time that
`the processor is
`determined to be busy; and maintaining the counter value.
`Finally, what is disclosed is an apparatus that provides
`hardware based utilization metering of CPUs in a computer
`system, comprising a plurality of CPUs. Associated with the
`CPUs is means for measuring computer system time. In
`addition, for each of the plurality of CPUs, the apparatus
`includes means for determining when the processor is busy;
`meansfor providing a busy indication when the processoris
`busy; meansfor receiving the busy indication and a measure
`of computer system time; means for combining the busy
`indication and the measure of computer system time to
`generate a counter value indicative of processor utilization;
`and means for maintaining the counter value.
`
`DESCRIPTION OF THE DRAWINGS
`
`The detailed description will refer to the following
`figures, in which like numbersrefer to like elements, and in
`which:
`
`FIG. 1Ais a basic block diagram of a system that meters
`CPU utilization data in a computer system running multiple
`instances of operating systems;
`FIG. 1B is a basic block diagram of an alternate system
`that meters CPU utilization data in a computer system
`running multiple instances of operating systems;
`FIG. 1C is a more detailed block diagram of specific
`components of the system of FIG. 1A;
`FIG. 2 is a further block diagram of a system that meters
`CPU utilization data in a computer system having multiple
`CPUs running multiple instances of operating systems;
`FIG. 3 is yet a further block diagram of a system that
`meters CPU utilization data in a computer system having
`multiple CPUs running multiple instances of operating sys-
`tems; and
`FIG. 4 is a block diagram of a method for metering CPU
`utilization in a computer system having multiple CPUs
`running multiple instances of operating systems.
`
`DETAILED DESCRIPTION
`
`45
`
`55
`
`60
`
`65
`
`

`

`US 6,816,809 B2
`
`3
`4
`Using the inputs from the system clock 130 and the idle
`between different operating systems on the same hardware,
`indicator 120, the counter 140 measures CPU cycles for the
`further complicating the ability to accurately measure CPU
`CPU 110, where the CPU 110 is not in an idle state, but
`utilization. Current solutions involve communicating with
`instead is performing a service for the user of the system
`each operating system independently, and then aggregating
`100. While the CPU 110 is powered on,the counter 140 may
`the information at a later time. This solution maybetrouble-
`thus maintain a counter value as shown in FIG. 1A, with the
`some when network access is restricted, agent software is
`counter value (e.g., CPU cycles) updated based on the
`not installed, and operating systems are temporarily out of
`system time data and the idle indicator output.
`service, for example. This solution also requires the devel-
`The counter value is provided from the counter 140 to a
`opment of operating system-specific agents because each
`usage data provider 150. The counter value may be provided
`partition is capable of running different instances of the
`to the usage data provider 150 on a periodic basis or when
`operating systems. In particular, since operating systems
`the counter 140 is polled by the usage data provider 150. The
`within a partionable computer system do not, by default,
`usage data provider 150 tracks the counter value and main-
`communicate with each other, data providers (software
`tains a non-volatile master copy of the counter value. When
`agents) are neededto run on each operating system to collect
`the CPU 110 is powered on (or a hardware component
`and transmit, or otherwise provide utilization data from each
`containing the CPU 110 is powered on), the saved non-
`partition. The thus-collected utilization data are aggregated
`volatile counter value is provided from the usage data
`to provide an overall value for CPU utilization.
`provider 150 to the counter 140 to initialize the counter
`The disclosed apparatus and method for collecting CPU
`value in the counter 140. Because the usage data provider
`utilization data overcomes these problems. The apparatus
`20
`150 maintains a non-volatile copy of the counter value, even
`and method assume that a CPU maybeinafirst state or in
`if the CPU 110, or other hardware componentis removed, in
`a second state. The CPU utilization may be based on a
`addition to a loss of powersituation, an up-to-date, or nearly
`measure of time that the CPU spends in onestate or the
`up-to-date value of the counter value is always available.
`other. In an embodiment,the first state may be a busystate,
`The usage data provider 150 maintains a connection, or
`generally meaning the CPU is running processes that per-
`network interface 160 to a system or network (not shown)
`form useful work for the user of the computer system.
`that is external to the computer system 100. For example, the
`Alternatively, the first state may be an idle state wherein the
`interface 160 may be a local area network (LAN)interface
`CPU is not performing useful work. The CPU utilization
`to a LAN. The LAN may include a managementserver that
`may be based on detecting or determining when the CPU is
`in the first state. In the embodiment in whichthe first state
`receives and processes information from the various com-
`puter systems coupled to the LAN, including the counter
`values that indicate CPU utilization. The usage data provider
`150 can provide the current value of the counter valueto the
`network by way of the network interface 160. The counter
`value may be provided periodically or when polled by the
`network.
`
`
`
`As noted above, the idle indicator 120 provides an indi-
`cation that the CPU 110 is not idle. The idle indicator may
`be implemented as a hardware modification to the computer
`system 100. For example, some CPUsinclude a pin on the
`CPU chip that provides a halt (idle) indication. Some
`operating systems halt
`the CPU when the CPU is not
`processing commands(i.e., the CPUis idle), and a halt (idle)
`indication (i.e., a high or low, or 0 or 1, value) may be
`asserted at the pin. In an embodiment, the idle indicator 120
`may be coupled to the pin to read the halt (idle) indication.
`Other operating systems do not halt the CPU when the CPU
`is idle. Instead, the operating system may place the CPU into
`an idle loop, where the CPU remains until the operating
`system requires CPU processing.
`In an embodiment, a
`change to the operating system may be made such that upon
`entry into the idle loop, an externally visible register value
`is set to indicate the CPU is idle. When the CPU exits the
`idle loop, the register would be cleared. In addition to the
`just-described two embodiments of the idle indicator 120,
`other structures and methods may be used to indicate the
`CPU 110 is idle.
`FIG. 1B showsan alternative 101 to the embodimentto
`the hardware based utilization metering components shown
`in FIG. 1A. In FIG. 1B, the CPU 110 may run an operating
`system having a communications channel 140' to the usage
`data provider 150. The communication channel 140' may be
`used to communicate CPU utilization metrics for collection
`by the usage data provider 150. In this embodiment, hard-
`ware modification to meter the CPU 110 directly would not
`be required.
`FIG. 1C shows selected components of the hardware
`based utilization metering device in moredetail. In FIG. 1C,
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`the apparatus may include means for
`is the busy state,
`determining if the CPU is busy. The meansfor determining
`if the CPU is busy may include hardware means.
`Alternatively, the means for determining if the CPU is busy
`may include software means. The apparatus also includes
`means for measuring and providing system time, and means
`for combining the system time and CPU busyindication to
`provide a CPU utilization value. In an embodiment, the CPU
`utilization may be a counter value. For example, a counter
`may increment by 1 for each system clock cycle that the
`CPU is not
`in an idle state. The hardware means for
`
`determining if the CPU is busy may include hardware
`modifications to the computer system. The alternate soft-
`ware means for determining if the CPU is idle may include
`modifications to the CPU’s operating system(s). The appa-
`ratus may further include means for storing a non-volatile
`version of the CPU utilization value, meansfor initializing
`the CPU utilization value upon power up of the CPU, and
`meansfor reporting the CPU utilization value to a system or
`network external to the computer system.
`FIG. 1A is a basic block diagram of a computer system
`100 that implements hardware based utilization metering.
`The system 100 includes a CPU 110 on which multiple
`instances of operating systems may run. That is, the CPU
`110 may support different types of operating systems(e.g.,
`UNIX, Windows®), and different versions of a specific type
`of operating system. Coupled to the CPU 110 is an idle
`indicator 120. The idle indicator 120 is capable of providing
`either an “idle” indication or a “not-idle”/“busy”indication.
`In an embodiment, the idle indicator 120 provides an output
`that indicates if the CPU 110 is “idle.” The operation and
`structure of the idle indicator 120 will be describedlater. The
`
`output of the idle indicator 120 is provided to a counter 140.
`The counter 140 also receives an output from a system clock
`130. The system clock 130 may measure ticks or cycles, or
`any other measure of system time, and then provide this
`data, or a modified form of the system time data, to the
`counter 140.
`
`

`

`US 6,816,809 B2
`
`6
`fewer than four CPUs. Associated with each of the CPUs
`210-213 is a corresponding idle indicator 220-223. Each of
`the idle indicators receives an indication whenits associated
`CPU is idle, and provides an output to a corresponding
`counter 240-243 that provides a current counter value (CPU
`utilization metric). Each of the counters 240-243 receives an
`input from a system clock 230. The system clock 230
`functions in the same manneras the system clock 130 shown
`in FIG. 1A.
`
`5
`the counter 140 is shown including an optional processor
`145 that may be used to process the idle indication and
`system time data to produce a CPU utilization metric in
`addition to the counter value. For example, the processor
`145 may compute cumulative real or system time the CPU
`110 is not
`idle. The usage data provider 150 is shown
`including a non-volatile storage 155 that stores the received
`counter value from the counter 140. The non-volatile storage
`155 maybe any non-volatile storage device that is capable
`of receiving and storing the counter values. The usage data
`provider 150 may also include an optional processor 156
`that may be used to convert the received counter value into
`a CPU metric such as total CPU non-idle hours, in real time,
`for example. The processor 156 mayalso handle requests for
`the utilization data over the interface 160 or for periodically
`transmitting utilization data to an external system on the
`LAN connected to the interface 160.
`
`10
`
`15
`
`FIG. 2 is a block diagram of a computer system 100' that
`meters CPUutilization data where the computer system 100°
`includes multiple CPUs running multiple instances of oper-
`ating systems. In FIG. 2, the computer system 100' includes
`CPU 1 through CPU N (denoted as 110-113). This arrange-
`ment is meantto indicate that the computer system 100' may
`include N CPUs,whereN is an integer. Associated with each
`of the CPUS 110-113 is a corresponding idle indicator
`(120-123,
`respectively) and a corresponding counter
`(140-143, respectively). Each of the counters 140-143
`receives an input from its respective idle indicator and from
`the system clock 130. The system clock 130 shown in FIG.
`2 performs the same functions, and has the samestructure,
`as the system clock 130 shown in FIG. 1A. Each of the
`counters provides an output (counter valuefor its respective
`CPU)to a usage data provider 150'. The usage data provider
`150' is similar to the usage data provider 150 shown in FIG.
`1A, except that the usage data provider 150' maintains a
`non-volatile measure of the counter value for each of the
`CPUs 110-113 in the computer system 100'. Finally, the
`computer system 100' includes the interface 160 for com-
`municating CPU utilization data (counter values) to a system
`or network (not shown) external to the computer system
`100.
`
`Each of the counters 240-243 provides its associated
`counter value to a usage data provider 500. The usage data
`provider 500 maintains a copy of the counter value (shown
`in FIG. 3 as 501-512, respectively) for each of the CPUsin
`the cells 200, 300 and 400. The usage data provider 500
`includes the interface 160 to a system or network (not
`shown) external to the computer system 100".
`‘lhe arrangementofthe cells 200, 300, and 400 allows one
`or moreofthe cells to be removed from the computer system
`100" while maintaining the computer system in operation.
`To prevent loss of the counter values for the CPUs in a
`removed cell, the counter values are stored in the usage data
`provider 500. In an embodiment, the stored counter values
`may be maintained in non-volatile storage. In operation, the
`counter values that are closest to the CPUs(i.e., the counter
`values in the counters) are incremented, and the counter
`values in the usage data provider 500 are periodically
`updated. The counter values may be updated based on a
`specific reporting interval, or by a polling action initiated by
`the usage data provider 500.
`Should one of the cells 200, 300 or 400 be replaced with
`a new cell, or with the original cell, but with one or more
`new CPUs, the counter values may be lost whenthe cell is
`powered off. Thus, whenevera cell is powered on, the usage
`data provider 500 will reinitialize the corresponding
`counters in the cell. For example, if the cell 200 is removed,
`and the CPU 210 is replaced with a new CPU 210’, then
`whenthe original cell 200 is reinstalled and powered on,the
`usage data provider 500 will provide the stored counter
`value 501 to the counter 240. The usage data provider 500
`will also initialize the counters 241-243 by providing the
`stored counter values 502-504, respectively. Because the
`CPU 210 was replaced by the new CPU 210’, the provided
`FIG. 3 is a block diagram of a computer system 100"
`counter value 501 will reflect the CPU utilization for the
`having multiple CPUsarranged in partitions or cells. One or
`CPU 210. Subsequent non-idle time of the CPU 210' will be
`more of the CPUs may run multiple instances of operating
`counted by incrementing the counter 240.
`systems, or, certain CPUs may run specific instances of
`operating systems while other CPUsrun other instances of
`is
`In the above example,
`the replacement CPU 210'
`operating systems. As shownin FIG. 3, the computer system
`similar in all respects to the original CPU 210, so that
`100" includes hardware cells 1 though K, denoted ascells
`incrementing the counter 240 with the new CPU 210
`200, 300, and 400, respectively. This arrangement is meant
`installed should proceed the same as with the original CPU
`to indicate that the computer system 100" may include K
`210. However, the new CPU 210 may also differ in some
`cells, where K is an integer, including 1 (one), indicating the
`respects from the original CPU 210, In an embodiment, the
`computer system 100" comprises onecell. Each of the cells
`apparatus and method for metering CPU utilization may
`is a set of components that can be removed from the
`include the necessary means for accounting for differences
`computer system 100" as a group. As can be seen in FIG. 3,
`between CPUs. For example, a processorin the counter 240
`the arrangement of components within each cellis identical
`(similar to the processor 145 shown in FIG. 1C) or a
`in terms of type and number of components. However, the
`processor in the usage data provider 500 (similar to the
`cells need not include the same type or same number of
`processor 156 shown in FIG. 1C) mayadjust the cumulative
`components. In addition to division of components among
`counter value for CPU 210 should CPU 210 be replaced with
`the hardware cells 1 through K, the components may be
`a different type or model CPU, or should the system clock
`partitioned,or logically sorted. ‘The partitions may comprise
`230 change clock rate, or should an operating system
`any number of CPUsor any numberof cells. For example,
`running on the CPU 210 change such that determination of
`cell 1 (200) may comprise four separate partitions, one for
`the counter value previously determined at the counter 240
`each of the CPUs installed in cell 1. Alternatively, cell 1
`would change. For example, should the system clock 230 be
`(200) and cell 2 (300) may compriseafirst partition and the
`replaced with a system clock having a higher clock rate than
`65
`remaining cell(s) may comprise additional partition(s).
`that of the original system clock 230, the processor in the
`Taking the cell 200 as an example, there are included four
`counter 240 could simply multiply the determined counter
`CPUs 210-213. However, the cell 200 may include more or
`value by the ratio of the original clock rate to the new clock
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`

`

`US 6,816,809 B2
`
`8
`interface used for providing processorutilization data to the
`external network.
`
`10. The device of claim 1, further comprising a plurality
`of processors, each of the plurality of processors having an
`associated idle indicator and an associated counter, whereby
`the associated idle indicator and the associated counter
`combine to provide a counter value for their respective
`processor.
`11. The device of claim 10, wherein the usage data
`provider maintains a non-volatile counter value for each of
`the plurality of processors.
`12. The device of claim 10, wherein the plurality of
`processors are arranged in one or morecells.
`13. A hardware based method for measuring processor
`utilization in a computer system comprising a plurality of
`processors, the method comprising:
`determining when any of the plurality of processors is
`busy;
`providing a busy indication to a counter associated with a
`busy processor;
`receiving at the counter a measure of computer system
`time;
`incrementing a counter value in the counter based on the
`provided busy indication and an amount of computer
`system timethat the processoris determinedto be busy;
`and
`
`10
`
`15
`
`20
`
`25
`
`30
`
`40
`
`45
`
`50
`
`7
`rate to produce a consistent rate of change in the counter
`value from that determined using the original clock rate and
`that determined using the new clockrate.
`FIG. 4 is a flow chart showing a hardware based CPU
`utilization operation 600 using the system 100" of FIG. 3.
`The operation 600 starts in block 610. In block 620,the cell
`200 is replaced and the hardware components onthe cell 200
`are powered up. In block 630, the usage data provider 500
`receives an indication of the power up of the cell 200
`components, and the usage data provider 500 sends the
`stored counter values 501-504 to the counters 240-243,
`respectively. In block 640, the received counter values are
`used to reinitialize the counters 240-243. However,
`the
`CPUs 210-213 are idle (operating systems are not
`processing), and in block 650, the idle indicators 220-223
`provide an idle indication to their respective counters
`240-243, thereby preventing incrementing of the counter
`values.
`In block 660,
`the CPU 210 begins running an
`operating system, and the idle indicator 220 sends an indi-
`cation to the counter 240 that the CPU 210 is not idle. In
`block 670, the counter 240 receives the non-idle indication,
`and the system time from the system clock 230, and begins
`incrementing the counter value for the CPU 210. In block
`680, the operating system running on the CPU 210 stops
`processing, halts the CPU 210, and asserts a halt indication.
`In block 690, the counter 240 receives an idle indication and
`stops incrementing the counter value. The operation 600
`may continue with incrementing counter values for other
`CPUsin the system 100" and mayinclude routines to update
`the non-volatile counter values in the usage data provider
`maintaining the counter value.
`500.
`14. The method of claim 13, further comprising:
`Whatis claimedis:
`reinitializing the counter value in the counter, using the
`1. A hardware based utilization metering device, compris-
`maintained value of the counter value, when the pro-
`ing:
`cessor is powered on.
`an idle indicator coupled to a processor, wherein the idle
`15. The method of claim 13, wherein maintaining the
`indicator receives an indication when the processor is
`counter value comprises:
`inafirst state;
`updating in a usage data provider a current value of the
`a counter coupled to the idle indicator and coupled to a
`counter value from the counter; and
`system clock, wherein the counter receives a measure
`of system time from the system clock and receives data
`storing the updated current counter value.
`related to the indication when the processor is in the
`16. The method of claim 15, wherein updating the current
`first state, and generates a counter value indicative of
`counter value comprises periodically receiving the current
`counter value from the counter.
`time the processoris in the first state; and
`a data usage provider coupled to the counter, wherein the
`17. The method of claim 15, wherein updating the current
`data usage provider is capable of providing the counter
`counter value comprises polling the counter.
`value.
`18. An apparatus that provides hardware basedutilization
`2. The device of claim 1, wherein the first state is a busy
`metering of central processor units (CPUs) in a computer
`state.
`system, comprising:
`a plurality of CPUs, wherein the plurality of CPUs are
`arranged in one or more cells;
`for each cell, means for measuring computer system time;
`and
`
`3. The device of claim 2, wherein the processor comprises
`an operating system that halts the processor when the
`processoris not processing, and whereintheidle indicatoris
`a hardware device coupledto a pin of the processor, wherein
`the hardware device reads a signal asserted on the pin to
`receive the idle indication.
`4. The device of claim 2, wherein the processor comprises
`an operating system that places the processor in an idle loop
`when the processor is idle, and wherein the idle indicator
`comprises an externally visible register that holds a value
`indicating the processor is in the idle state.
`5. The device of claim 2, wherein the counter value
`increments based on the measure of system time when the
`processoris in the busy state.
`6. The device of claim 1, wherein when the processor is
`powered on,
`the counter receives a current value of the
`counter value.
`7. The device of claim 1, wherein the counter provides the
`counter value to the data usage provider on a periodic basis.
`8. The device of claim 1, wherein the usage data provider
`polls the counter to obtain the counter value.
`9. The device of claim 1, further comprising an interface
`coupling the data usage provider to an external network,the
`
`for each of the plurality of CPUs:
`means for determining when a CPUis busy;
`means for providing an indication when the CPU is
`busy;
`means for receiving the indication and measure of
`computer system time,
`means for combining the indication and the measure of
`computer system time to generate a counter value
`indicative of CPU utilization, and
`means for maintaining the counter value.
`19. The apparatus of claim 18, further comprising means
`for reinitializing the counter value upon powerof the CPU.
`20. The apparatus of claim 18, further comprising means
`for reporting a CPU utilization value to a network external
`to the computer system.
`*
`
`*
`
`*
`
`*
`
`*
`
`55
`
`60
`
`awn
`
`