as) United States
`a2) Patent Application Publication (10) Pub. No.: US 2005/0177755 Al
`
` Fung (43) Pub. Date: Aug. 11, 2005
`
`
`US 20050177755A1
`
`(54) MULTI-SERVER AND MULTI-CPU POWER
`MANAGEMENT SYSTEM AND METHOD
`
`(52) US. Che ecceccesssscsseeseeseessesnseneesseenesnneeneees 713/300
`
`(75)
`
`Inventor: Henry T. Fung, San Jose, CA (US)
`
`(57)
`
`ABSTRACT
`
`Correspondence Address:
`R. Michael Ananian
`FLEHR HOHBACH TEST
`ALBRITTON & HERBERT LLP
`Four Embarcadero Center, Suite 3400
`San Francisco, CA 94111 (US)
`.
`(73) Assignee: Amphus, Inc.
`;
`(21) Appl. No.:
`09/860,210
`(22)
`Filed:
`May18, 2001
`Related U.S. Application Data
`(60) Provisional application No. 60/283,375,filed on Apr.
`11, 2001. Provisional application No. 60/236,043,
`filed on Sep. 27, 2000. Provisional application No.
`60/236,062,filed on Sep. 27, 2000.
`
`Publication Classification
`
`(SL)
`
`Tint, C07eee cccceeeeesccsssssnnneeecccceceensneeesess GO6F 1/26
`
`Network architecture, computer system and/or server, cir-
`cuit, device, apparatus, method, and computer program and
`control mechanism for managing power consumption and
`workload in computer system and data and information
`servers. Further provides power and energy consumption
`and workload management and control systems and archi-
`tectures for high-density and modular multi-server computer
`systems that maintain performance while conserving energy
`and method for power management and workload manage-
`ment. Dynamic server power management and optional
`dynamic workload management for multi-server environ-
`ments is provided by aspects of the invention. Modular
`network devices and integrated server system,
`including
`modular servers, managementunits, switches and switching
`fabrics, modular power supplies and modular fans and a
`special backplane architecture are provided as well as
`dynamically reconfigurable multi-purpose modules and
`servers. Backplane architecture, structure, and method that
`has no active components and separate power supply lines
`and protection to provide high reliability in server environ-
`ment.
`
`
`
`af?
` 4-l
`P47 dli-l
`ara
`YE lea.) varTe GOVGATEform. veer #
`con
`Z
`4
`Cae CA | FRCU. CYCOatRal UAT &
`Gol gtr -f
`
`
`
`AACBus(orvtBu?
`
`5M conhel Atjrnhey
`tare Wat
`
`svar)
`
`fo
`
`
`
`
`Ethel (Actudy faprting or stuet tiring /
`Usins Spoudarl nefurrl fiche
`svhee SWAP)
`
`Google Exhibit 1024
`Google Exhibit 1024
`Google v. Valtrus
`Google v. Valtrus
`
`

`

`Patent Application Publication Aug. 11,2005 Sheet 1 of 5
`
`US 2005/0177755 Al
`
`fot7oft
`
`f-OthsrUL
`
`ZW
`
`(tee rw 0) Cod Ail
`
`
`
`aaaypayfInIsC:/Sonyparyepeuploks
`
`
`
`
`
`(SinsPvyrns
`
`
`
`

`

`Patent Application Publication Aug. 11,2005 Sheet 2 of 5
`
`US 2005/0177755 Al
`
`108
`
`a
`
`Sees
`
`/-208
`
`AONEAtOLhacens9: mameeenpemeanenensmAetOnsen se s4s8,
`
`r
`
`
`379)>NgDMYngiwend]P—97fbSESWDINPioyjntoye|_9styesoye
`Gew=12O3ptAOSE
`1SE—LMDHOTA105«|oFROA—8i"I
`awni3Limgyake|MEXpig095“wei|or’yoFEL
`munsaypusdsagat
`ToLWSi[ree
`
`
`|SToptaaeaAde;
`7Pe
`;_
`yee
`ONY9"Debeengi>dsunsypuading|nd|:——>5REE
`1djzuot“jem|1sot
`ber
`
`onsome"9oevaHOOIN—atunsay
`
`
`
`
`
`
`
`AESTARENTLEemmeannoneceescHebeEHtneet6eeueuasHebrnensemtartLinte:
`
`1-GhEAe
`
`juawedroyy)
`
`anpoyw
`
`HeatsgabbaptaeUanaadeete«
`
`SBURPOWY
`JAWS|
`
`que
`
`C‘WAS
`
`ceTjiepors
`
`RALAhoote
`
`yet
`
`
`
`

`

`Patent Application Publication Aug. 11,2005 Sheet 3 of 5
`
`US 2005/0177755 A1
`
`4rdMode
`
`FI.3
`
`IstMode
`
`0%
`
`2ndMode
`
`2ndMode CPUUtihzation
`
`¢
`a
`S
`
`as
`ao
`ss
`
`

`

`Patent Application Publication Aug. 11,2005 Sheet 4 of 5
`
`US 2005/0177755 Al
`
`43
`
`

`

`Patent Application Publication Aug. 11,2005 Sheet 5 of 5
`
`US 2005/0177755 A1
`
`Fo. BY
`
`SVE Set jee PomePmve
`
`CONTIOL SIGWALS
`
`

`

`US 2005/0177755 Al
`
`Aug. 11, 2005
`
`MULTI-SERVER AND MULTI-CPU POWER
`MANAGEMENT SYSTEM AND METHOD
`
`RELATED APPLICATIONS
`
`[0001] This application is a continuing application under
`35 U.S.C. §§ 119(e) and 120, wherein applicant and inventor
`claim the benefit of priority to U.S. Provisional Application
`Ser. No. 60/283,375 entitled System, Method And Architec-
`ture For Dynamic Server Power Management And Dynamic
`Workload Management for Multi-Server Environmentfiled
`11 Apr. 2001; U.S. Provisional Application Ser. No. 60/236,
`043 entitled System, Apparatus, and Method for Power-
`Conserving Multi-Node Server Architecture filed 27 Sep.
`2000; and U.S. Provisional Application Ser. No. 60/236,062
`entitled System, Apparatus, and Method for Power Conserv-
`ing and Disc-Drive Life Prolonging RAID Configuration
`filed 27 Sep. 2000; each of which application is hereby
`incorporated by reference.
`
`FIELD OF THE INVENTION
`
`[0002] This invention pertains generally to architecture,
`apparatus, systems, methods, and computer programs and
`control mechanisms for managing power consumption and
`work-load in data and information servers; more particularly
`to power consumption and workload management and con-
`trol systems for high-density multi-server computer system
`architectures that maintain performance while conserving
`energy and to the method for power management and
`workload managementused therein, and mostparticularly to
`system, method, architectures, and computer programs for
`dynamic server power management and dynamic workload
`management for multi-server environments.
`
`BACKGROUND
`
`[0003] Heretofore, servers generally, and multi-node net-
`workservers in particular, have paid little if any attention to
`power or energy conservation. Such servers were designed
`and constructed to run at or near maximum levels so as to
`
`serve data or other content as fast as possible, or where
`service demands were less than capacity to remain ever
`vigilant to provide fast response to service requests. Increas-
`ing processor and memory speeds have typically been
`accompanied by higher processor core voltages to support
`the faster device switching times, and faster hard disk drives
`have typically lead to faster and more energy-hungry disk
`drive motors. Larger memories and caches havealso lead to
`increased power consumption even for small single-node
`servers. Power conservationefforts have historically focused
`on the portable battery-powered notebook market where
`battery life is an important marketing and use characteristic.
`However,in the serverarea,little attention has been given to
`saving power, such servers usually not adopting or utilizing
`even the power conserving suspend, sleep, or hibernation
`states that are available with some Microsoft 95/98/2000,
`Linux, Unix, or other operating system based computers,
`personal computers, PDAs, or information appliances.
`
`[0004] Multi-node servers present a particular energy con-
`sumption problem as they have conventionally be archi-
`tected as a collection of large power hungry boxesintercon-
`nected by external interconnect cables. Little attention has
`been placed on the size or form factor of such network
`architectures, the expansability of such networks, or on the
`
`problems associated with large network configurations.
`Such conventional networks have also by-and-large paid
`little attention to the large amounts of electrical power
`consumedby such configurations or in the savings possible.
`This has been due in part because of the rapid and unex-
`pected expansion in the Internet and in servers connected
`with and serving to Internet clients. Internet service com-
`panies and entrepreneurs have been more interested in a
`short time to market and profit than onthe effect on electrical
`power consumption andelectrical powerutilities; however,
`continuing design and operation without due regard to
`power consumption in this manneris problematic.
`
`[0005] Networksservers have also by-and-large neglected
`to factor into the economics of running a network server
`system the physical plant cost associated with large rack
`mounted equipment carrying perhaps one network node per
`chassis. These physical plant and real estate costs also
`contribute to large operating costs.
`
`In the past, more attention was given to the pur-
`[0006]
`chase price of equipmentandlittle attention to the operating
`costs. It would be apparent to those making the calculation
`that operating costs may far exceed initial equipment pur-
`chase price, yet little attention has been paid to this fact.
`Morerecently, the power available in the California electri-
`cal market has been at crisis levels with available power
`reserves dropping below a few percent reserve and rolling
`blackouts occurring as electrical power requirements drop
`below available electrical power generation capacity. High
`technology companies in the heart of Silicon Valley cannot
`get enough electrical power to make or operate product, and
`server farms which consume vast quantities of electrical
`energy for the servers and for cooling equipmentand facili-
`ties in which they are housed, have stated that they may
`relocated to areas with stable supplies of low-costelectricity.
`
`[0007] Even were server manufactures motivated to adopt
`available power management techniques, such techniques
`represent only a partial solution. Conventional computer
`system power management tends to focus on power man-
`aging a single CPU,such as by monitoring certain restricted
`aspects of the single CPU operation and making a decision
`that the CPU should be run faster to provide greater perfor-
`mance or more slowly to reduce power consumption.
`
`[0008] Heretofore, computer systems generally, and server
`systems having a plurality of servers where each server
`includes at least one processor or central processing unit
`(CPU)in particular have not been power managed to main-
`tain performance and reduce power consumption. Even
`where a server system having more than one server com-
`ponent and CPU maypossibly have utilized a conventional
`personal computer architecture that provided some measure
`of localized power management separately within each
`CPU,no global power managementarchitecture or methods
`have conventionally been applied to power managethe set
`of servers and CPUsas a single entity.
`
`[0009] The commonpractice of over-provisioning a server
`system so as to be able to meet peak demands has meantthat
`during long periods of time, individual servers are consum-
`ing power and yet doing no useful work, or several servers
`are performing some tasks that could be performed by a
`single server at a fraction of the power consumption.
`
`including their
`{0010] Operating a plurality of servers,
`CPU,hard disk drive, power supply, cooling fans, and any
`
`

`

`US 2005/0177755 Al
`
`Aug. 11, 2005
`
`other circuits or peripherals that are associated with the
`server, at such minimal loading also unnecessarily shortens
`their service life. However, conventional server systems do
`not considerthe longevity of their components. To the extent
`that certain of the CPUs, hard disk drives, power supplies,
`and cooling fans may be operated at lower powerlevels or
`for mechanical systems (hard disk drive and cooling fans in
`particular) their effective service life may be extended.
`
`{0011] Therefore there remains a need for a network
`architecture and network operating method that provides
`large capacity and multiple network nodes or servers in a
`small physical footprint and that
`is power conservative
`relative to server performance and power consumed by the
`server, as well as powerconservative from the standpoint of
`power for server facility air conditioning. These and other
`problemsare solved by the inventive system, apparatus and
`method. There also remains a need for server farmsthat are
`power managed in an organized global manner sothat
`performance is maintained while reducing power consump-
`tion. There also remains a need to extend the effective
`
`lifetime of computer system components andservers so that
`the total cost of ownership is reduced.
`
`SUMMARY
`
`[0012] Aspects of the invention provide network architec-
`ture, computer system and/or server, circuit, device, appa-
`ratus, method, and computer program and control mecha-
`nism for managing power consumption and workload in
`computer system and data and information servers. Other
`embodiments of the invention further provides power and
`energy consumption and workload management and control
`systems and architectures for high-density and modular
`multi-server computer systems that maintain performance
`while conserving energy and method for power management
`and workload management. Dynamic server power manage-
`ment and optional dynamic workload management
`for
`multi-server environments is provided by aspects of the
`invention. Modular network devices and integrated server
`system,
`including modular servers, management units,
`switches and switching fabrics, modular power supplies and
`modular fans and a special backplane architecture are pro-
`vided as well as dynamically reconfigurable multi-purpose
`modules and servers.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0013] FIG. 1 is a diagrammatic illustration showing an
`embodimentof the inventive power conserving power man-
`agement between two servers and a manager.
`
`[0014] FIG. 2 is a diagrammatic illustration showing an
`alternative embodimentof a server system showingdetail as
`to how activity may be detected and operating mode and
`power consumption controlled in response.
`
`[0015] FIG. 3 is a diagrammatic illustration showing a
`graph of the CPU utilization (processor activity) as a func-
`tion of time, wherein the CPU utilization is altered by
`entering different operating modes.
`
`[0016] FIG. 4 is a diagrammatic illustration showing an
`exemplary state engine state diagram graphically illustrating
`the relationships amongst the modes and identifying some of
`the transitions between states or modes for operation of an
`embodiment of the inventive system and method.
`
`{0017] FIGS. 5-12 are diagrammatic illustrations showing
`exemplary state diagram for operating modetransitions.
`
`[0018] FIG.13 is a diagrammatic illustration showing the
`manner in which a plurality of servers may operate in
`different modes based on local detection and control of
`selected mode transitions and local detection but global
`control of other selected mode transitions.
`
`DETAILED DESCRIPTION OF EMBODIMENTS
`OF THE INVENTION
`
`[0019] The present invention pertains to computer system
`architectures and structures and methods for operating such
`computer system architectures in a compact high-pertor-
`mance low-power consumption manner. Computers, infor-
`mation appliances, data processing systems, and all manner
`of electronic systems and devices may utilize and benefit
`from the innovations described herein. Aspects of the inven-
`tion also contribute to reliability, ease of maintenance, and
`longevity of the system as a whole and operation compo-
`nents thereof. In an application that is of particular impor-
`tance and which benefits greatly from the innovations
`described here, the computer system is or includes a server
`system having at least one and more typically a plurality of
`servers. Each server will include at least one processor or
`CPU but may include multiple CPUs. In multiple server
`configurations significant power consumption reduction is
`achieved by applying the inventive power management
`scheme. These and other aspects of the invention are
`described in the sections that follow.
`
`[0020] At least some embodiments of the invention pro-
`vide a modular configuration where computers, servers,
`managers, and other devices and/or components are pro-
`vided in a modular form so that such devices or components
`may readily be placed into service, maintained, removed
`from service, and/or configured within a rack or enclosure to
`provided desired operational features. References to “modu-
`lar” devices, such as for example “modular server”, “server
`module”, “management module”, or other module are there-
`fore intended to apply to either a modular or non-modular
`device or component. For example, in the specification we
`conveniently refer to a “server”or “server module” to mean
`any server.
`
`In still another aspect the inventive structure and
`[0021]
`method provide for significant power consumption reduc-
`tion and energy savings as compared to conventional net-
`work and server architectures as only those power consum-
`ing resources that are actually needed to provide the quality
`of service required are in an active mode. Those node
`resources that are not needed may be poweredoff or placed
`in some power conserving standby mode until needed. In
`addition, operations performed by one or more nodes may be
`shifted to another node so that only the remaining active
`nodes consume power and the remaining nodes are in
`standby mode or powered off until needed. The intelligence
`within one of the nodes acting as a master node for the
`cluster or ISS may then wake up the inactive node and
`configure it for operation. A system may be woken up and
`placed in any of the available operating modes by any one
`of a plurality of events. Nodes may also be placed into an
`inactive or power conserving mode when no demandsare
`made on their resources independent of whether responsi-
`bility for their functionality has been shifted to another node
`
`

`

`US 2005/0177755 Al
`
`Aug. 11, 2005
`
`or nodes. In one embodiment of the invention the power
`consumed is reduced by a factor of about 10-times as
`compared to a standard 19-inch wide by 1.75-inch high (1U)
`rack mountable network node device. This power savingsis
`accomplishedat least in part by one or more of the following
`measures: the reduction in the number of power supplied,
`use of the mountingplate as a heat sink to assist in removing
`heat from the enclosure, providing power saving controls to
`circuits and devices within the ISS enclosure, and the above
`described ability to reconfigure and take off line unneeded
`capacity.
`
`[0022] Many different types of servers architectures are
`knownin the art. Typically, such servers have at least one
`processor with associated fast
`random access memory
`(RAM), a massstorage device that stores the data or content
`to be served by the server, a power supply that receives
`electrical power (current and voltage) from either a battery
`or line voltage from an electrical utility, a network commu-
`nication card or circuit for communicating the data to the
`outside world, and various other circuits that support the
`operation of the CPU, such as a memory (typically non-
`volatile ROM)storing a Basic Input-Output System (BIOS),
`a Real-Time Clock (RTC) circuit, voltage regulators to
`generate and maintain the required voltages in conjunction
`with the power supply, and core logic as well as optional
`micro-controller(s) that communicate with the CPU and
`with the external world to participate in the control and
`operation of the server. This core logic is sometimesreferred
`to as the Northbridge and Southbridge circuits or chipsets.
`
`[0023] From a somewhatdifferent perspective, variations
`in server architecture, reflect
`the variations in personal
`computers, mainframes, and computing systems generally.
`The vast structural, architectural, methodological, and pro-
`cedural variations inherent
`in computer systems having
`chips, chipsets, and motherboards adapted for use by Intel
`Processors (such as the Intel x86, Intel Pentium™, Intel
`Pentium™ II, Intel Pentium™ II, Intel Pentium™ IV),
`Transmeta Crusoe™ with LongRun™, AMD, Motorola, and
`others, precludes a detailed description of the manner in
`which the inventive structure and method will be applied in
`each situation. Those having ordinary skill will appreciate in
`light of the description that
`the inventive structure and
`method apply to a broad set of different processor and
`computer/server architecture types and that minorvariations
`within the ordinary skill of a practitioner in the field may be
`made to adapt
`the invention to other processor/system
`environments.
`
`[0024] Before describing particular implementations that
`relate to more or less specific CPU designs and interfaces,
`attention first directed to a simplified embodiment of the
`inventive system and method with respect to FIG. 1. In this
`embodiment, at least two (and up to n) servers or server
`modules (where servers are made in modular form or
`configuration) 402-1,
`.
`.
`.
`, 402-N are provided, each
`including a CPU 404 and a memory 408. CPU 404 includes
`an activity indicator generator 406 which generates activity
`indicators, and either (i) communicates the activity indica-
`tors to memory 408 for storage in an activity indicator(s)
`data structure 410, or not shown, (ii) communicates them
`directly to a server module control unit and algorithm 432
`within management module 430. Different types of activity
`indicators such as are described elsewhere in the specifica-
`tion, such as for example an idle thread based activity
`
`indicator may be used. Whether stored in memory or com-
`municated directly, the activity indicator(s) are used by the
`management module to determine the loading on each of the
`server modules individually and as a group. In one embodi-
`ment, activity information or indicators created on any one
`computer or device (such as a server module) is accessible
`to a manager or supervisor via standard networking proto-
`col.
`
`illustrated in FIG. 1, analogous
`[0025] Although not
`structure and signals generated and received may be used to
`control the operation of core logic circuits to thereby control
`core logic voltage and core logic clock signals in a manner
`to reduce power consumption where such core logic power
`managementis provided.
`
`[0026] Voltage and frequency are regulated locally by the
`CPU using an activity monitoring scheme, such as for
`example oneof the activity monitoring schemeillustrated in
`Table I.
`
`TABLEI
`
`Exemplary Activity Monitoring Schemes carried out in CPU or PMU
`
`Application Layer
`Network Layer
`Physical Layer
`
`Carried out by PMU
`Carried out by CPU
`NA
`Port Address
`NA
`TCP/IP
`Idle Threads, Activity Counter I/O Activities
`
`[0027] This power management scheme may be inter-
`preted in one aspect as providing a Model-to-Mode2 and
`Mode2-to-Model power management scheme, where both
`Mode 1 and Mode2 are active modes and the state of the
`
`CPUin either Mode 1 or Mode2 is controlled locally by the
`CPU,and in another aspect as providing a Mode3 (inactive
`mode or maintenance of memory contents only). Mode3
`control may also be performed locally by the CPU, but in
`one of the preferred embodiments of the invention, entry
`into a Mode 3 stage is desirably controlled globally in a
`multi-CPU system. Where the multi-CPU’s are operative
`with a plurality of servers for multi-server power manage-
`ment, the Management Module (or a Server Module acting
`as a manager on behalf of a plurality of server modules)
`determines which Server Module should enter a Mode 3
`state using the Server Module control algorithm and unit
`432. Activity monitoring of individual Server Modules 402
`is desirably based on the standard network protocol, such as
`for example SNMP. Therefore the activity indicators may be
`retrieved from the CPU 406 or memory 408 via NIC 440 as
`is known in the art. A communication link coupling micro-
`controllers (1C) 442 together, and in particular the micro-
`controller of the Management Module with the microcon-
`trollers of the several Server Modules. This permits the
`management module to communicate commandsorsignals
`to the server modules which are received by the microcon-
`trollers even when the CPUsare in a suspended state (Mode
`3). In so providing for monitoring over the first link (the
`Ethernet) and control over the second link (the AMPCbus),
`the server modules may be monitored for activity and
`controlled globally to reduce power consumption while
`providing sufficient on-line capacity. It is noted that the
`power management maybe effected by altering either or
`both of the CPU clock frequency 420 or the CPU voltage
`416.
`
`

`

`US 2005/0177755 Al
`
`Aug. 11, 2005
`
`[0028] Although a separate management module 430 is
`illustrated in FIG. 1,
`it should be understood that
`the
`managementfunctionality generally, and the server module
`control algorithm in particular may be implemented by one
`of the operating server modules. For example, the control
`algorithm would be implemented as a software or firmware
`procedure executing in the CPU and processor of a server
`module designated according to predetermined rules, poli-
`cies, or procedures to be the master.
`
`is noted that although several of the modes
`It
`[0029]
`described conserve power, they do not compromise perfor-
`mance, as the cumulative combination of server modules is
`always maintained at or above minimum targeted perfor-
`mance.
`
`In FIG.2 there is illustrated an exemplary system
`[0030]
`301 including a server (such as for example, an ISSU server
`module) 302-1, coupled to a switch (such as for example, an
`ISSU switch module) 304, and through the switch 304 and
`optionally via a micro-controller (uC) 314 within server 302
`over a separate (optional) direct bus connection 312 (such as
`for example, the AMPC bus made by Amphusof San Jose,
`Calif.) to a power management supervisor (such as for
`example, ISSU management module) 316. As described
`elsewhere herein, switch 304 is responsible for connecting
`the various server module(s) 302, management module(s)
`316, and other components that are or may be controlled to
`achieve the power conservation features of the invention.
`Recall
`that such subsystems as the power supply (not
`shown) and cooling or fan modules may also be coupled
`through the switch 304. The connectivity and signals shown
`in the diagram are intended to show significant control paths
`pertinent to the operation of the invention, and therefore
`some signals that are conventional or do not illustrate the
`operation of the invention are not shown to avoid obscura-
`tion of the invention.
`
`[0031] Attention is now focused on the internal structure
`and operation of the server module 302. During operation
`CPU 320 executes commands or instructions, or when no
`instructions are present to be executed, executes idle threads.
`The activity level of the CPU is monitored and a control
`signal Vec_CPU_control 322 is generated based on that
`sensed activity or lack of activity. The manner in which this
`activity is sensed or the manner and characteristics of the
`Vec_CPU_control signal will typically vary depending on
`the processor type, operating system, and other factors
`specific to the system architecture. By way ofillustrative
`example, an indication as to the CPU activity or lack of
`activity may be generated by monitoring by executing an
`application layer function call that returns a value indicating
`the idle thread execution based activity. This is possible in
`the Microsoft Windows 98,2000, and NT operating envi-
`ronments, for example.
`
`[0032] As the name implies, the Vec_CPU_controlsignal
`322 which is an input signal
`to voltage regulator 324
`controls or influences the CPU core voltage Vec_CPU 326.
`As described elsewhere in this description, the CPU core
`voltage 326 may be raised and lowered in conjunction with
`the CPU clock frequency to provide adequate switching
`response of CPU circuits without excessive voltage.
`Although this embodimentillustrates that the Vec_CPU-
`_control signal 322 is generated within the CPU,
`in an
`alternative embodiment, it may be generated within the core
`
`the CPU clock is
`logic block 330. In one embodiment,
`adjusted based on a signal from the core logic and the CPU
`voltage is adjusted on the basis of the CPUitself. This is due
`to the fact that the voltage change is desirably synchronized
`in time with the frequency change. In some sense,
`this
`control may be viewed as including an effective link from
`the core logic to control the voltage regulator output.
`
`[0033] Core logic 330 includes a Power Management Unit
`332 of which many types are now known; however, one
`early example of a Power Management Unit is described in
`US. Pat. Nos. 5,396,635, 5,892,959 and 6,079,025 (each of
`whichis herein incorporated by reference) by the inventorof
`the present application as well as in the other applications
`related thereto. In operation, PMU 332receives a signal over
`bus 336 and generates an output signal 338 that is commu-
`nicated overbus 340 to clock generator 342. Clock generator
`block 342 includescircuitry that generates a CPU clock 50,
`a core logic clock signal 352, a Network Interconnect Card
`(NIC) clock signal 354, and a video clock signal 356.
`
`[0034] RAM 328is coupled to core logic 330 via DRAM
`control line and hence to the CPU via bus 336. Hard disk
`
`drive 338 is similarly coupled to core logic 330 to CPU via
`bus 336. In one implementation, Redundant Array of Inde-
`pendent Disc (RAID) data storage is provided for the server
`modules. As is known,this RAID storage provides consid-
`erable data redundancy.In order to implement this RAID in
`a power managementefficient manner, two IDE controllers
`(or enhanced IDE controllers) are used to interface to two
`separate disk drives. Provision of two hard disk drives
`supports RAID Level 0, RAID Level 1, and RAID Level
`041 implementations. Aspect of the RAID power manage-
`ment disk drive longevity are described in co-pending U.S.
`Provisional Application Ser. No. 60/236,062 entitled Sys-
`tem, Apparatus, and Method for Power Conserving and
`Disc-Drive Life Prolonging RAID configuration filed 27
`Sep. 2000, hereby incorporated by reference. It is noted that
`providing RAID storage or multiple disk drives on the
`servers is advantages though not required.
`
`[0035] Clock generator 342 includes clock signal gener-
`ating and logic circuitry or other means for generating a
`CPU clock signal at the desired frequencyor for selecting a
`CPUclock signal from an available plurality of clock signal
`having different frequencies. Under the inventive power
`management scheme,the clock frequencyis adjusted down-
`ward within a permissible CPU clock frequency range to
`provide a CPU processing power that matches the present
`need, and to the extent that the present need is below the
`maximum capability of the processor when operating at full
`permissible clock frequency, to reduce the power consump-
`tion of the CPU. As the CPU core voltage may be reduced
`below a maximum voltage when the clock frequency is
`below its maximum frequency, the CPU core voltage may be
`lowered with the clock frequency or speed.
`
`[0036] A PCI bus 360 coupling NIC 362 and Video
`processor 364 is provided and interfaces with CPU 320 via
`Core logic 330. NIC 362 generates and provides a resume
`output 366 and NIC Clock input signal 368, and Video
`processor 364 is provided with a video clock signal 356 from
`the clock generator 342 and a suspend input signal 370. It is
`noted that the suspend and resume signals may come from
`multiple sources to affect the desired control and manage-
`ment.
`
`

`

`US 2005/0177755 Al
`
`Aug. 11, 2005
`
`In this illustrative embodiment, an X-bus 374 is
`[0037]
`provided to couple the Real-Time Clock (RTC) 376 and
`BIOS378 to the core logic 330 and via bus 336 to the CPU
`as required. RTC 376 may generate a resume output signal
`378. This RTC generated resume signal 378 is therefore
`operative to activate PMU 332, core logic 330, and CPU 330
`under
`a predetermined time or alarm condition. For
`example, the RTC may be set to generate a resume signal
`378 at 8:00 am local time every day to bring the server
`module 302 back online.
`
`[0038] The NIC resumesignal may be generated when a
`specific packet is received. When generated in one of these
`manners and communicated to the PMU 332it is operative
`to place the core logic 336 back into an active state and
`hence CPU 320 into any selected state of mode. One
`situation in which the NIC resume signal may be generated
`is when the server module is in a powered-on but inactive
`state, such that the CPU clock is stopped (or operating at an
`extremely low clock frequency). Under such condition, a
`simple way of waking the server module 302 is to commu-
`nicate a signal 380 from management module 316 via switch
`304. As the NIC will typically be keptactive, it will receive
`the signal 380 and generate the resume signal 366.
`
`It is noted that each of the elements, such as the
`[0039]
`hard disk drive, Video processor and other power consuming
`elements may include meansfor receiving a control signal
`that places them into a powerconservingstate or that brings
`then out of on or more power conserving states into a full
`power and performance mode.
`
`It is noted that the embodimentillustrated in FIG.
`[0040]
`2 represents a system that might utilize any of a number of
`conventional processors or CPU, and might for example
`utilize a CPU of the Intel Pentium, Pentium II, Pentium II,
`or Pentium IV types made by Intel Corporation of Santa
`Clara, Calif., various Advanced Micro Device CPUs, CPUs
`made by Transmeta, as well as other processors and CPUs
`as are knownintheart.
`
`[0041] Having now described the physical architecture
`and connectivity of an exemplary Integrated Server System,
`the structure and operation of an exemplary server module,
`management module, and switch module, aspects of how
`these modul