568
`
`IEEE TRANSACTIONS ON DEVICE AND MATERIALS RELIABILITY, VOL. 4, NO. 4, DECEMBER 2004
`
`Review of Cooling Technologies
`for Computer Products
`
`Richard C. Chu, Robert E. Simons, Michael J. Ellsworth, Roger R. Schmidt, and Vincent Cozzolino
`
`Invited Paper
`
`Abstract—This paper provides a broad review of the cooling
`technologies for computer products from desktop computers to
`large servers. For many years cooling technology has played a key
`role in enabling and facilitating the packaging and performance
`improvements in each new generation of computers. The role of
`internal and external thermal resistance in module level cooling
`is discussed in terms of heat removal from chips and module
`and examples are cited. The use of air-cooled heat sinks and
`liquid-cooled cold plates to improve module cooling is addressed.
`Immersion cooling as a scheme to accommodate high heat flux
`at the chip level is also discussed. Cooling at the system level is
`discussed in terms of air, hybrid, liquid, and refrigeration-cooled
`systems. The growing problem of data center thermal manage-
`ment is also considered. The paper concludes with a discussion of
`future challenges related to computer cooling technology.
`
`Index Terms—Air cooling, data center cooling, flow boiling, heat
`sink, immersion cooling, impingement cooling, liquid cooling, pool
`boiling, refrigeration cooling, system cooling, thermal, thermal
`management, water cooling.
`
`I. INTRODUCTION
`
`E LECTRONIC devices and equipment now permeate vir-
`
`tually every aspect of our daily life. Among the most
`ubiquitous of these is the electronic computer varying in size
`from the handheld personal digital assistant to large scale main-
`frames or servers. In many instances a computer is imbedded
`within some other device controlling its function and is not
`even recognizable as such. The applications of computers vary
`from games for entertainment to highly complex systems sup-
`porting vital health, economic, scientific, and military activities.
`In a growing number of applications computer failure results
`in a major disruption of vital services and can even have
`life-threatening consequences. As a result, efforts to improve
`the reliability of electronic computers are as important as ef-
`forts to improve their speed and storage capacity.
`Since the development of the first electronic digital computers
`in the 1940s, the effective removal of heat has played a key role
`in insuring the reliable operation of successive generations of
`computers. The Electrical Numerical Integrator and Computer
`(ENIAC), dedicated in 1946, has been described as a “30 ton,
`boxcar-sized machine requiring an array of industrial cooling
`
`Manuscript received August 30, 2004.
`The authors are with the IBM Corporation, Poughkeepsie, NY 12601 USA
`(e-mail: rcchu@us.ibm.com).
`Digital Object Identifier 10.1109/TDMR.2004.840855
`
`fans to remove the 140 kW dissipated from its 18 000 vacuum
`tubes” [1]. Following ENIAC, most early digital computers used
`vacuum-tube electronics and were cooled with forced air.
`The invention of the transistor by Bardeen, Brattain, and
`Shockley at Bell Laboratories in 1947 [2] foreshadowed the
`development of generations of computers yet to come. As a
`replacement for vacuum tubes, the miniature transistor gener-
`ated less heat, was much more reliable, and promised lower
`production costs. For a while it was thought that the use of
`transistors would greatly reduce if not totally eliminate cooling
`concerns. This thought was short-lived as packaging engineers
`worked to improve computer speed and storage capacity by
`packaging more and more transistors on printed circuit boards,
`and then on ceramic substrates.
`The trend toward higher packaging densities dramatically
`gained momentum with the invention of the integrated cir-
`cuit separately by Kilby at Texas Instruments and Noyce at
`Fairchild Semiconductor in 1959 [2]. During the 1960s, small
`scale and then medium scale integration (SSI and MSI) led
`from one device per chip to hundreds of devices per chip. The
`trend continued through the 1970s with the development of
`large scale integration (LSI) technologies offering hundreds
`to thousands of devices per chip, and then through the 1980s
`with the development of very large scale (VLSI) technologies
`offering thousands to tens of thousands of devices per chip. This
`trend continued with the introduction of the microprocessor
`and continues to this day with chip makers projecting that a
`microprocessor chip with a billion or more transistors will be a
`reality before 2010.
`In many instances the trend toward higher circuit packaging
`density has been accompanied by increased power dissipation
`per circuit to provide reductions in circuit delay (i.e., increased
`speed). The need to further increase packaging density and re-
`duce signal delay between communicating circuits led to the de-
`velopment of multichip modules beginning in the late 1970s and
`is continuing today. An example of the effect that these trends
`have had on module heat flux in high-end computers is shown in
`Fig. 1. As can be seen heat flux associated with Bipolar circuit
`technologies steadily increased from the very beginning and re-
`ally took off in the 1980s. There was a brief respite with the
`transition to CMOS circuit technologies in the 1990s; but, the
`demand for increased packaging density and performance re-
`asserted itself and heat flux is again increasing at a challenging
`rate.
`
`1530-4388/04$20.00 © 2004 IEEE
`
`Authorized licensed use limited to: Alexander Karl. Downloaded on June 16,2021 at 23:25:01 UTC from IEEE Xplore. Restrictions apply.
`
`Immersion Systems LLC – Ex. 1014
`PGR 2021-00104 (U.S. 10,820,446 B2)
`1 of 18
`
`

`

`CHU et al.: REVIEW OF COOLING TECHNOLOGIES FOR COMPUTER PRODUCTS
`
`569
`
`Fig. 2. Cross-section of a typical module denoting internal cooling region and
`external cooling region.
`
`Fig. 1. Evolution of module level heat flux in high-end computers.
`
`Throughout the past 50 years, cooling and thermal manage-
`ment have played a key role in accommodating increases in
`power while maintaining component temperatures at satisfac-
`tory levels to satisfy performance and reliability objectives.
`Sections II–V of this paper will discuss the various techniques
`that have been used to provide temperature control in com-
`puters in the past and present, as well as some of the methods
`being explored for the future.
`
`II. MODULE-LEVEL COOLING
`
`Processor module cooling is typically characterized in two
`ways: cooling internal and external to the module package and
`applies to both single and multichip modules. Fig. 2 illustrates
`the distinction between the two cooling regimes in the context
`of a single-chip module.
`
`A. Internal Module Cooling
`The primary mode of heat transfer internal to the module is by
`conduction. The internal thermal resistance is therefore dictated
`by the module’s physical construction and material properties.
`The objective is to effectively transfer the heat from the elec-
`tronics circuits to an outer surface of the module where the heat
`will be removed by external means which will be discussed in
`the following section.
`In the case of large multichip modules (MCMs) where
`variation in the location and height of chips had to be considered,
`an approach (Figs. 3 and 4) was adopted that employed a
`spring-loaded mechanical cylindrical piston touching each chip
`with point contact and minute physical gaps between the chip
`and piston and between the piston and module housing [3].
`
`Isometric cutaway view of an IBM TCM module with a water-cooled
`Fig. 3.
`cold plate.
`
`Fig. 4. Cross-sectional view of an IBM TCM module on an individual chip
`site basis.
`
`The volume within the module was filled with helium gas to
`minimize the thermal resistance across the gaps and achieve
`an acceptable internal thermal resistance. The total module
`cooling assembly was patented as a gas-encapsulated module
`[4] and later named a thermal conduction module (TCM). TCM
`cooling technology evolved through three generations of IBM
`mainframes: system 3081, ES/3090, and ES/9000, with about
`a threefold increase in cooling capability from 19 to 64 W/cm
`at the chip level and 3.7 to 11.8 W/cm at the module level
`
`Authorized licensed use limited to: Alexander Karl. Downloaded on June 16,2021 at 23:25:01 UTC from IEEE Xplore. Restrictions apply.
`
`Immersion Systems LLC – Ex. 1014
`PGR 2021-00104 (U.S. 10,820,446 B2)
`2 of 18
`
`

`

`570
`
`IEEE TRANSACTIONS ON DEVICE AND MATERIALS RELIABILITY, VOL. 4, NO. 4, DECEMBER 2004
`
`Fig. 5. Cross-sectional view of a Hitachi M-880 module on an individual chip
`site basis.
`
`[5]. The last generation TCM incorporated a copper piston (the
`original piston was aluminum) with a cylindrical center section
`and a slight taper on each end to minimize the gap between
`piston and cap while retaining intimate contact between the
`piston face and the chip [6]. Additionally, the volume inside the
`module was filled with a PAO (polyalphaolefin) oil instead of
`helium to reduce the piston-to-cap and chip-to-piston thermal
`resistances. Hitachi packaged a similar conduction scheme in
`their M-880 [7] and MP5800 [8] processors. Instead of a
`cylindrical piston Hitachi utilized an interdigitated microfin
`structure (Fig. 5).
`In the 1990s when IBM made the switch from bipolar to
`CMOS circuit technology [10] the conduction cooling approach
`was simplified and reduced in cost by adopting a “flat plate”
`conduction approach as shown in Fig. 6. The thermal path from
`chip to cap is provided by a controlled thickness (e.g., 0.10 mm
`to 0.18 mm) of a thermally conductive paste. This was possible
`largely due to improved planarity of the substrate, better control
`of dimensional tolerances and enhanced thermal conductivity of
`the paste.
`As time went on, chip power levels continued to increase. In
`addition, concentrated areas of high heat flux 2 to 3 times the
`average chip heat flux referred to as hot spots emerged. To meet
`internal thermal resistance requirements, in 2001 IBM chose to
`attach a high-grade silicon carbide (SiC) spreader to the chip
`with an adhesive thermal interface (ATI) and then use a more
`conventional thermal paste between the spreader and the cap
`[10]. This configuration is shown in Fig. 7.
`The adhesive thermal interface (ATI), while not as thermally
`conductive as the thermal paste, could be applied much thinner
`resulting in a lower thermal resistance. SiC was chosen for the
`spreader material for its unique combination of high thermal
`conductivity and low coefficient of thermal expansion (CTE).
`The CTE of the SiC closely matches that of the silicon chip thus
`avoiding stress fracturing the interface when the module heats
`up during use. The thermal resistance of this package arrange-
`ment is lower than just using thermal paste between chip and
`cap because of the use of the lower thermal resistance ATI on
`the smaller chip area. The thermal paste thermal resistance is
`mitigated by applying it over a much larger area.
`
`B. External Module Cooling
`
`Cooling external to the module serves as the primary means
`to effectively transfer the heat generated within the module to
`
`Fig. 6. Cross-sectional view of central processor module package with thermal
`paste path to module cap [9].
`
`Fig. 7. MCM cross-section showing heat spreader adhesively attached to chip
`(adapted from [10]).
`
`the system environment. This is accomplished primarily by at-
`taching a heat sink to the module. Traditionally, and prefer-
`ably, the system environment of choice has been air because
`of its ease of implementation, low cost, and transparency to
`the end user or customer. This section, therefore, will focus
`on air-cooled heat sinks. Liquid-cooled heat sinks typically re-
`ferred to as cold plates will also be discussed.
`1) Air-Cooled Heat Sinks: A typical air-cooled heat sink is
`shown in Fig. 8. The heat sink is constructed of a base region
`that is in contact with the module to be cooled. Fins protruding
`from the base serve to extend surface area for heat transfer to
`the air. Heat is conducted through the base, up into the fins and
`then transferred to the air flowing in the spaces between the fins
`by convection. The spacing between fins can run continuously
`in one direction in the case of a straight fin heat sink or they can
`run in two directions in the case of a pin fin heat sink (Fig. 9).
`Air flow can either be through the heat sink laterally (in cross
`flow) or can impinge from the top as seen in Fig. 10.
`The thermal performance of the heat sink is a function of
`many variables. Geometric variables include the thickness and
`plan area of the base plus the fin thickness, height, and spacing.
`The principal material variable is thermal conductivity. Also
`factored in is volumetric air flow and pressure drop. Many opti-
`mization studies have been conducted to minimize the external
`thermal resistance for a particular set of application conditions
`[11]–[13]. However, over time, as greater and greater thermal
`performance has been required, fin heights and fin number have
`increased while fin spacing has been decreased. Additionally,
`heat sinks have migrated in construction from all aluminum
`
`Authorized licensed use limited to: Alexander Karl. Downloaded on June 16,2021 at 23:25:01 UTC from IEEE Xplore. Restrictions apply.
`
`Immersion Systems LLC – Ex. 1014
`PGR 2021-00104 (U.S. 10,820,446 B2)
`3 of 18
`
`

`

`CHU et al.: REVIEW OF COOLING TECHNOLOGIES FOR COMPUTER PRODUCTS
`
`571
`
`Fig. 8. Typical air-cooled heat sink.
`
`Fig. 9. Typical (a) straight fin heat sink and (b) pin fin heat sink.
`
`Fig. 10. Air flow path through a heat sink: (a) cross flow or (b) impingement.
`
`(with thermal conductivity ranging from 150–200 W/mK) to
`aluminum fins on copper bases (with thermal conductivity
`ranging from 350–390 W/mK) to all copper. In certain cases
`heat pipes have been embedded into heat sinks to more effec-
`tively spread the heat [14]–[16].
`Heat sink attachment to the module also plays a role in the ex-
`ternal thermal performance of a module. The method of attach-
`ment and the material at the interface must be considered. The
`material at the interface is important because when two surfaces
`are brought together seemingly in contact with one another, sur-
`face irregularities such as surface flatness and surface roughness
`result in just a fraction of the surfaces actually contacting one
`another. The majority of the heat is therefore transferred through
`the material that fills the voids or gaps that exist between the
`two surfaces [17]. One method of heat sink attachment is by
`
`mechanical means using screws or a clamping mechanism. Air
`has traditionally existed at the interface but more recently oils
`or even phase change materials (PCMs) have been used [18] to
`reduce the thermal resistance at the interface. Another method
`of attachment has been adhesively with an elastomer or epoxy.
`This method has worked well on smaller single-chip modules
`where heat sinks do not have to be removed from the module.
`2) Water-Cooled Cold Plates: For situations where air
`cooling could not meet requirements, such as was the case in
`IBM’s 3081, ES/3090, and ES/9000 systems in the 1980s and
`early 1990s, and the case in Hitachi’s M-880 and MP5800 in the
`1990s, heat was removed from the modules via water-cooled
`cold plates. Compared to air, water cooling can provide al-
`most an order of magnitude reduction in thermal resistance
`principally due to the higher thermal conductivity of water.
`In addition, because of the higher density and specific heat of
`water, its ability to absorb heat in terms of the temperature
`rise across the coolant stream is approximately 3500 times that
`of air. Cold plates function very similarly to air-cooled heat
`sinks. For example, the ES/9000 cold plate is an internal finned
`structure made of tellurium copper [19]. As with the air-cooled
`heat sinks, changes in material properties and geometry were
`made to improve performance. A higher thermal conductivity
`tellurium copper was chosen over beryllium copper used in
`previous generation cold plates. Additionally, fin heights were
`increased and channel widths (analogous to fin spacings) were
`decreased. The ES/9000 module also marked the first time IBM
`used a PAO oil at the interface between the module cap and
`cold plate to reduce the thermal interface resistance.
`In an effort to significantly extend the cooling capability
`of liquid-cooled cold plates, researchers continue to work on
`microchannel cooling structures. The concept was originally
`demonstrated over 20 years ago by Tuckerman and Pease [20].
`They chemically etched 50 m-wide by 300- m-deep channels
`into a 1 cm 1 cm silicon chip. By directing water through
`these microchannels they were able to remove 790 W with a
`temperature difference of 71 C. More recently, aluminum ni-
`tride heat sinks fabricated using laser machining and adhesively
`attached to the die have been used to cool a high-powered
`MCM and achieve a junction to ambient unit thermal resistance
`below 0.6 K-cm /W [21]. The challenge continues to be to
`provide a practical chip or module cooling structure and flow
`interconnections in a manner which is both manufacturable
`(i.e., cost effective) and reliable.
`
`Authorized licensed use limited to: Alexander Karl. Downloaded on June 16,2021 at 23:25:01 UTC from IEEE Xplore. Restrictions apply.
`
`Immersion Systems LLC – Ex. 1014
`PGR 2021-00104 (U.S. 10,820,446 B2)
`4 of 18
`
`

`

`572
`
`IEEE TRANSACTIONS ON DEVICE AND MATERIALS RELIABILITY, VOL. 4, NO. 4, DECEMBER 2004
`
`C. Immersion Cooling
`
`Immersion cooling has been of interest as a possible method
`to cool high heat flux components for many years. Unlike the
`water-cooled cold plate approaches which utilize physical walls
`to separate the coolant from the chips, immersion cooling brings
`the coolant in direct physical contact with the chips. As a result,
`most of the contributors to internal thermal resistance are elim-
`inated, except for the thermal conduction resistance from the
`device junctions to the surface of the chip in contact with the
`liquid.
`Direct liquid immersion cooling offers a high heat transfer co-
`efficient which reduces the temperature rise of the heated chip
`surface above the liquid coolant temperature. The magnitude
`of the heat transfer coefficient depends upon the thermophys-
`ical properties of the coolant and the mode of convective heat
`transfer employed. The modes of heat transfer associated with
`liquid immersion cooling are generally classified as natural con-
`vection, forced convection, and boiling. Forced convection in-
`cludes liquid jet impingement in the single phase regime and
`boiling (including pool boiling, flow boiling, and spray cooling)
`in the two-phase regime. An example of the broad range of heat
`flux that can be accommodated with the different modes and
`forms of direct liquid immersion cooling is shown in Fig. 11
`[22].
`Selection of a liquid for direct immersion cooling cannot
`be made on the basis of heat transfer characteristics alone.
`Chemical compatibility of the coolant with the chips and
`other packaging materials exposed to the liquid is an essential
`consideration. There may be several coolants that can provide
`adequate cooling, but only a few will be chemically compatible.
`Water is an example of a liquid which has very desirable
`heat transfer properties, but which is generally undesirable for
`direct immersion cooling because of its chemical and electrical
`characteristics. Alternatively, fluorocarbon liquids (e.g., FC-72,
`FC-86, FC-77, etc.) are generally considered to be the most
`suitable liquids for direct immersion cooling, in spite of their
`poorer thermophysical properties [22], [23].
`1) Natural and Forced Liquid Convection: As in the case of
`air cooling, liquid natural convection is a heat transfer process
`in which mixing and fluid motion is induced by differences in
`coolant density caused by heat transferred to the coolant. As
`shown in Fig. 11, this mode of heat transfer offers the lowest
`heat flux or cooling capability for a given wall superheat or
`surface-to-liquid temperature difference. Nonetheless, the heat
`transfer rates attainable with liquid natural convection can ex-
`ceed those attainable with forced convection of air.
`Higher heat transfer rates may be attained by utilizing a pump
`to provide forced circulation of the liquid coolant over the chip
`or module surfaces. This process is termed forced convection
`and the allowable heat flux for a given surface-to-liquid temper-
`ature difference can be increased by increasing the velocity of
`the liquid over the heated surface. The price to be paid for the
`increased cooling performance will be a higher pressure drop.
`This can mean a larger pump and higher system operating pres-
`sures. Although forced convection requires the use of a pump
`and the associated piping, it offers the opportunity to remove
`heat from high power chips and modules in a confined space.
`The liquid coolant may then be used to transport the heat to a
`remote heat exchanger to reject the heat to air or water.
`
`Fig. 11. Heat flux ranges
`microelectronic chips [22].
`
`for direct
`
`liquid immersion cooling of
`
`Fig. 12. Forced convection thermal resistance results for simulated 12.7 mm
` 12.7 mm microelectronic chips (adapted from [24]).
`
`Experimental studies were conducted by Incropera and
`Ramadhyani
`[24] to study liquid forced convection heat
`transfer from simulated microelectronic chips. Tests were
`performed with water and dielectric liquids (FC-77 and FC-72)
`flowing over bare heat sources and heat sources with pin-fin
`and finned pin extended surface enhancement. It can be seen in
`Fig. 12 that, depending upon surface and flow conditions (i.e.,
`Reynolds number), thermal resistance values obtained for the
`fluorocarbon liquids ranged from 0.4 to 20 C W. It may be
`noted that a thermal resistance on the order of 0.5 C W could
`
`Authorized licensed use limited to: Alexander Karl. Downloaded on June 16,2021 at 23:25:01 UTC from IEEE Xplore. Restrictions apply.
`
`Immersion Systems LLC – Ex. 1014
`PGR 2021-00104 (U.S. 10,820,446 B2)
`5 of 18
`
`

`

`CHU et al.: REVIEW OF COOLING TECHNOLOGIES FOR COMPUTER PRODUCTS
`
`573
`
`support chip powers of 100 W while maintaining chip junction
`temperatures 85 C or less.
`The Cray-2 supercomputer introduced in the mid-1980s pro-
`vides an example of the application of forced convection liquid
`cooling to computer electronics [25]. As shown in Fig. 13, the
`module assembly used in the Cray-2 was three-dimensional
`in structure consisting of eight interconnected printed circuit
`boards on which were mounted arrays of single-chip carriers.
`Module power dissipation was reported to be 600 to 700 W.
`Cooling was provided by FC-77 liquid distributed vertically
`between stacks of modules and flowing horizontally between
`the printed circuit cards.
`Even higher heat transfer rates may be obtained in the forced
`convection mode by directing the liquid flow normal to the
`heated surface in the form of a liquid jet. A number of studies
`[26]–[28] have been conducted to demonstrate the cooling
`efficacy of liquid jet impingement flows. An example of the
`chip heat flux that can be accommodated using a single FC-72
`liquid jet is shown in Fig. 14. Liquid jet impingement was the
`basic cooling scheme employed in the aborted SSI SS-1 super-
`computer. The cooling design provided for a maximum chip
`power of 40 W corresponding to a chip heat flux of 95 W/cm .
`2) Pool and Flow Boiling: Boiling is a complex convec-
`tive heat transfer process depending upon liquid-to-vapor phase
`change with the formation of vapor bubbles at the heated sur-
`face. It may be characterized as either pool boiling (occurring in
`an essentially stagnant liquid) or flow boiling. The pool boiling
`heat flux,
`, usually follows a relationship of the form
`
`is a constant depending upon each fluid-surface
`where
`combination,
`is the heat transfer surface area,
`is the
`temperature of the heated surface, and
`is the saturation
`temperature (i.e., boiling point) of the liquid. The value of
`the exponent
`is typically about 3. This means that as the
`heat flux is increased at the chip surface, the heat transfer
`coefficient or cooling effectiveness increases. For example if
`and the power dissipation is doubled, the temperature
`rise will increase by only about 26% in the boiling mode
`compared to 100% in the forced convection mode.
`A problem that has been associated with pool boiling of fluo-
`rocarbon liquids is that of temperature overshoot. This behavior
`is characterized by a delay in the inception of boiling on the
`heated surface. The heated surface continues to be cooled in the
`natural convection mode, with increased surface temperatures
`until a sufficient degree of superheat is reached for boiling to
`occur. This behavior is a result of the good wetting character-
`istics of fluorocarbon liquids and the smooth nature of silicon
`chips. Although much work [29] has been done in this area, it is
`still a potential problem in pool boiling applications using fluo-
`rocarbon liquids to cool untreated silicon chips.
`The maximum chip heat flux that can be accommodated in
`pool boiling is determined by the critical heat flux. As power is
`increased more and more vapor bubbles are generated. Even-
`tually so many bubbles are generated that they form a vapor
`blanket over the surface preventing fresh liquid from reaching
`the surface and resulting in film boiling and high surface tem-
`peratures. Typical critical heat fluxes encountered in saturated
`
`Fig. 13. Forced convection liquid-cooled Cray-2 electronic module assembly.
`
`Fig. 14. Typical direct liquid jet impingement cooling performance for a
`6.5 mm  6.5 mm integrated circuit chip (adapted from [28]).
`
`saturation temperature) pool boiling
`(i.e., liquid temperature
`of fluorocarbon liquids range from 10 to 15 W/cm , depending
`upon the nature of the surface (i.e., material, finish, geometry).
`The allowable critical heat flux may be extended by subcooling
`the liquid below its saturation temperature. For example experi-
`ments have shown that it is possible to increase the critical heat
`in pool boiling to as much as 25 W/cm by subcooling the liquid
`temperature to
`25 C.
`Higher critical heat fluxes may be achieved using flow
`boiling. For example, heat fluxes from 25 to 30 W/cm have
`been reported for liquid velocities of 0.5 to 2.5 m/s over the
`heated surface [30]. In addition, it may also be noted that
`temperature overshoot has not been observed to be a problem
`with flow boiling.
`As in the case of air cooling or single phase liquid cooling,
`the heat flux that may be supported at the component level (i.e.,
`chip or module) may be increased by attaching a heat sink to
`the surface. As part of an early investigation of pool boiling
`with fluorocarbon liquids a small 3-mm-tall molybdenum stud
`with a narrow slot (0.76 mm) down the middle was attached to
`
`Authorized licensed use limited to: Alexander Karl. Downloaded on June 16,2021 at 23:25:01 UTC from IEEE Xplore. Restrictions apply.
`
`Immersion Systems LLC – Ex. 1014
`PGR 2021-00104 (U.S. 10,820,446 B2)
`6 of 18
`
`

`

`574
`
`IEEE TRANSACTIONS ON DEVICE AND MATERIALS RELIABILITY, VOL. 4, NO. 4, DECEMBER 2004
`
`a 2.16 mm 2.16 mm silicon chip. A heat flux at the chip level
`in excess of 100 W/cm was achieved [31].
`An example of a computer electronics package utilizing pool
`boiling to cool integrated circuit chips is provided by the IBM
`Liquid Encapsulated Module (LEM) developed in the 1970s
`[32]. As shown in Fig. 15, a substrate with 100 integrated
`circuit chips was mounted within a sealed module-cooling
`assembly containing a fluorocarbon coolant (FC-72). Boiling
`at the exposed chip surfaces provided a high heat transfer
`coefficient (1700 to 5700 W m -K) with which to meet chip
`cooling requirements. Either an air-cooled or water-cooled
`cold plate could be used to handle the module heat load. With
`this approach it was possible to cool 4.6 mm 4.6 mm chips
`dissipating 4 W and module powers up to 300 W.
`3) Spray Cooling: In recent years spray cooling has re-
`ceived increasing attention as a means of supporting higher
`heat flux in electronic cooling applications. Spray cooling is a
`process in which very fine droplets of liquid are sprayed on the
`heated surface. Cooling of the surface is then achieved through
`a combination of thermal conduction through the liquid in
`contact with the surface and evaporation at the liquid–vapor
`interface.
`One of the early investigations of spray cooling was con-
`ducted by Yao et al. [33] with both real and ideal sprays of
`FC-72 on a heated horizontal copper surface 3.65 cm in di-
`ameter. A peak heat flux of 32 W cm , or about 2 to 3 times
`the critical heat flux achievable with saturated pool boiling was
`reported.
`Pautsch and Bar-Cohen [34] describe two methods of spray
`cooling suitable for electronic cooling. One method is termed
`“low density spray cooling” and is defined as occurring when
`the liquid contacts and wets the surface and then boils before
`interacting with the next impinging droplet. Although a very ef-
`ficient method of heat transfer, it does not support very high
`heat fluxes. The other method is termed “high density evapo-
`rative cooling” and requires spraying the liquid on the surface
`at a rate that maintains a continuously wetted surface. In the
`paper, experiments are described demonstrating the capability
`to accommodate heat fluxes in excess of 50 W/cm while main-
`taining chip junction temperatures below 85 C with spray evap-
`orative cooling. Spray evaporative cooling is used to maintain
`junction temperatures of ASICs on MCMs in the CRAY SV2
`system between 70 C and 85 C for heat fluxes from 15 W/cm
`to 55 W/cm [35]. In addition to the CRAY cooling application,
`spray cooling has gained a foothold in the military sector pro-
`viding for improved thermal management, dense system pack-
`aging, and reduced weight [36].
`Researchers have also investigated spray cooling heat transfer
`using other liquids. Lin and Ponnappan determined that critical
`heat fluxes can reach up to 90 W/cm with fluorocarbon liquids,
`490 W/cm with methanol, and higher than 500 W/cm with
`water [37].
`
`III. SYSTEM-LEVEL COOLING
`
`for computers may be categorized
`Cooling systems
`as air-cooled, hybrid-cooled,
`liquid-cooled, or
`refrigera-
`tion-cooled. An air-cooled system is one in which air, usually
`in the forced convection mode, is used to directly cool and carry
`heat away from arrays of electronic modules and packages.
`
`Fig. 15.
`
`IBM Liquid Encapsulated Module (LEM) cooling concept.
`
`In some systems air-cooling alone may not be adequate due
`to heating of the cooling air as it passes through the machine.
`In such cases a hybrid-cooling design may be employed, with
`air used to cool the electronic packages and water-cooled
`heat exchangers used to cool the air. For even higher power
`packages it may be necessary to employ indirect liquid cooling.
`This is usually done utilizing water-cooled cold plates on
`which heat dissipating components are mounted, or which may
`be mounted to modules containing integrated circuit chips.
`Ultimately, direct liquid immersion cooling may be employed
`to accommodate high heat fluxes and a high system heat load.
`
`A. Air-Cooled Systems
`Forced air-cooled systems may be further subdivided into se-
`rial and parallel flow systems. In a serial flow system the same
`air stream passes over successive rows of modules or boards, so
`that each row is cooled by air that has been preheated by the
`previous row. Depending on the power dissipated and the air
`flow rate, serial air flow can result in a substantial air tempera-
`ture rise across the machine. The rise in cooling air temperature
`is directly reflected in increased circuit operating temperatures.
`This effect may be reduced by increasing the air flow rate. Of
`course to do this requires larger blowers to provide the higher
`flow rate and overcome the increase in air flow pressure drop.
`Parallel air flow systems have been used to reduce the temper-
`ature rise in the cooling air [38], [39]. In systems of this type, the
`printed circuit boards or modules are all supplied air in parallel
`as shown in Fig. 16. Since each board or module is delivered its
`own fresh supply of cooling air, systems of this type typically
`require a higher total volumetric flow rate of air.
`
`B. Hybrid Air–Water Cooling
`An air-to-liquid hybrid cooling system offers a method to
`manage cooling air temperature in a system without resorting
`to a parallel configuration and higher air flow rates. In a system
`of this type, a water-cooled heat exchanger is placed in the
`heated air stream to extract heat and reduce the air temperature.
`
`Authorized licensed use limited to: Alexander Karl. Downloaded on June 16,2021 at 23:25:01 UTC from IEEE Xplore. Restrictions apply.
`
`Immersion Systems LLC – Ex. 1014
`PGR 2021-00104 (U.S. 10,820,446 B2)
`7 of 18
`
`

`

`CHU et al.: REVIEW OF COOLING TECHNOLOGIES FOR COMPUTER PRODUCTS
`
`575
`
`Fig. 16. Example of a parallel air-