United States Patent [19J
`Cray, Jr.
`
`[It] Patent Number:
`[45] Date of Patent:
`
`4,590,538
`May 20, 1986
`
`[54]
`
`[75]
`
`IMMERSION COOLED HIGH DENSITY
`ELECI'RONIC ASSEMBLY
`Inventor: Seymour R. Cray, Jr., Chippewa
`Falls, Wis.
`[73] Assignee: Cray Research, Inc., Chippewa Falls,
`Wis.
`[21] Appl. No.: 442,569
`[22] Filed:
`Nov. 18, 1982
`Int. Cl,4 ............................................... H05K 7/20
`[51]
`[52] u.s. Cl •..................................... 361/385; 361/382
`[58] Field of Search ............... 361/382, 383, 384, 385,
`361/386, 412; 165/104.33; 174/15 R; 62/415,
`416, 417, 418
`
`[56]
`
`References Cited
`U.S. PATENT DOCUMENTS
`1,641,702 9/1927 Sprong .
`1,950,653 3/1934 Best .
`2,643,282 6/1953 Greene .
`2,879,455 3/1959 Seal .
`2,917,685 12/1959 Wiegand .
`2,948,518 8/1960 Kraus .
`3,065,384 11/1962 Sprude .
`3,070,729 12/1962 Heidler ................................ 361/382
`3,139,559 6/1964 Heidler .
`3,141,999 7/1964 Schneider ........................... 361/382
`3,270,250 8/1966 Davis .............................. 165/104.33
`3,334,684 8/1967 Roush et al.
`3,370,203 2/1968 Kravitz et al.
`3,417,814 12/1968 Oktay .
`3,459,998 8/1969 Focarile .
`3,504,268 3/1970 Hoffman et al ..................... 361/382
`3,512,582 5/1970 Chu et al.
`3,527,989 9/1970 Cuzner et al.
`3,529,213 9/1970 Farrand et al.
`3,537,063 10/1970 Beaulieu .
`3,586,959 6/1971 Eccles ................................. 361/385
`3,630,273 12/1971 Haye et al ........................... 165/122
`3,741,292 6/1973 Aakalu et al.
`3,812,402 5/1974 Garth .................................. 361/385
`3,833,840 9/1974 Sinden .
`3,851,221 11/1974 Beaulieu et al.
`3,904,934 9/1975 Martin .
`3,999,105 12/1976 Archey et al ..
`4,072,188 2/1978 Wilson et al. ....................... 361/385-
`4,120,021 10/1978 Roush .
`
`4,283,754 8/1981 Parks .
`4,302,793 11/1981 Rohner .
`4,417,297 11/1983 Oyama et al. ....................... 361/412
`4,420,739 12/1983 Herren .
`
`FOREIGN PATENT DOCUMENTS
`0608258 9/1956 Canada ................................ 361/412
`0839083 6/1981 U.S.S.R ............................... 361/386
`
`OTHER PUBLICATIONS
`IBM Technical Disclosure Bulletin, vol. 20, No. 9, Feb.
`1978, "Liquid Jet Cooling of Integrated Circuit Chips",
`Sachar, pp. 3727-3728.
`IBM Technical Disclosure Bulletin, vol. 10, No. 10, Mar.
`1968, "Thermal Card and Deflector System for Aug(cid:173)
`menting Emersion Cooling", Chu et al., pp. 1559-1560.
`IBM Technical Disclosure Bulletin, vol. 8, No. 10, Mar.
`1966, "Heat Dissipator Assemblies", Mandel et al., p.
`1460.
`Primary Examiner-Gregory D. Thompson
`Attorney, Agent, or Firm-Merchant, Gould, Smith,
`Edell, Welter & Schmidt
`ABSTRACf
`[57]
`An immersion cooling system for high density elec(cid:173)
`tronic assemblies such as computers includes a con(cid:173)
`tainer holding an inert cooling liquid, and stacks of
`circuit modules arranged in a generally radial pattern
`within the container. Coolant supply columns and cool(cid:173)
`ant removal columns alternate between adjacent stacks
`around the pattern. The coolant supply columns include
`distribution manifolds which distribute incoming cool(cid:173)
`ant at all levels to provide a flow of coolant to all circuit
`modules. The flow passes between adjacent boards of
`the modules and preferably along flow channels formed
`by the circuit chips aligned in rows. After passing
`across the circuit modules the heated coolant rises in
`coolant removal columns and flows over standpipes for
`removal from the container, and a pump and heat ex(cid:173)
`changer recools and recirculates the coolant. Pump up
`and pump down systems are also provided for with(cid:173)
`drawing the coolant to a reservoir for servicing the
`circuitry.
`
`17 Claims, 16 Drawing Figures
`
`43
`
`IS000257
`
`Immersion Systems LLC – Ex. 1008
`PGR 2021-00104 (U.S. 10,820,446 B2)
`1 of 16
`
`

`

`FIG.
`
`I
`
`FIG. 2
`
`I I
`I
`I
`
`44-
`
`45
`
`47
`
`~ ...
`OJ
`~ ...
`
`01
`.W
`00
`
`I
`I
`I J
`
`I
`I
`I I
`
`~t H t
`
`I t
`' I
`
`•
`
`IS000258
`
`Immersion Systems LLC – Ex. 1008
`PGR 2021-00104 (U.S. 10,820,446 B2)
`2 of 16
`
`

`

`U.S. Patent May 20, 1986
`
`Sheet2 of8
`
`4,590,538
`
`FIG. 3
`
`PUMP c----------.
`DOWN
`
`IS000259
`
`Immersion Systems LLC – Ex. 1008
`PGR 2021-00104 (U.S. 10,820,446 B2)
`3 of 16
`
`

`

`U.S. Patent May20, 1986
`
`Sheet 3 of8
`
`4,590,538
`
`80
`
`64
`
`74
`I
`
`I
`
`74-,
`
`•
`
`I
`
`I
`
`~4-
`c:=l r
`
`f
`
`74-
`'
`1 - IL I _ (_
`
`80
`
`FIG. 5
`ts·t.l /
`
`60
`
`.
`
`z f.3
`
`6Z
`
`64-
`
`79
`
`65
`FIG. 6
`
`63
`
`6Z
`
`6/
`
`IS000260
`
`Immersion Systems LLC – Ex. 1008
`PGR 2021-00104 (U.S. 10,820,446 B2)
`4 of 16
`
`

`

`U.S. Patent May20, 1986
`Sheet4 of8
`FIG. 10
`
`4,590,538
`
`84-
`
`FIG.
`
`II
`
`IS000261
`
`Immersion Systems LLC – Ex. 1008
`PGR 2021-00104 (U.S. 10,820,446 B2)
`5 of 16
`
`

`

`U.S. Patent May20, 1986
`
`Sheet 5 of8
`
`4,590,538
`
`FIG. 12
`
`I
`
`'~
`
`1
`1
`1
`
`,-3Z
`I
`r-35
`~b I, 1/0 I ~ I JI:Z,
`/14-
`121 1
`.... J/.3'1.
`, no n nn .... ;.!II,..; L. • • • • • 11
`
`\
`
`11
`
`,
`
`1
`1,
`
`.30d
`
`I . 96
`
`1
`
`il
`
`I '
`
`''
`
`I
`
`I
`
`''
`
`' I
`
`il
`
`'
`
`Jf4..
`
`/0/
`
`"' ,,J,
`
`~
`
`,4.,
`
`1 nnnnnnr
`.
`
`I ..
`I •
`
`' ' .•... _ \. \ iJ1 ,
`..
`
`••
`
`....
`
`• •• •
`
`I
`
`r
`
`I '
`
`• •
`
`I
`
`'
`
`• ,.
`
`• ..
`
`r 1
`
`ill
`''
`
`,.';'o
`
`·:;_
`
`/0/ //4-
`I"'
`
`I rrrrrr
`IZ /ZP
`
`IS000262
`
`Immersion Systems LLC – Ex. 1008
`PGR 2021-00104 (U.S. 10,820,446 B2)
`6 of 16
`
`

`

`FIG. 14
`
`10
`~
`
`DDDDDDDD
`oooqp oo
`0 ~ 0
`otr 6Z63--l-Jo
`
`/0/
`
`/0/
`
`II
`
`Q [...of/-" ,.1 _] Q
`63
`DO
`'DODD
`• DDDODDDD
`
`0
`
`80.
`
`~
`C'-l
`•
`~
`~
`
`~ a
`... -\C)
`
`~
`~
`N
`0
`
`00
`0'\
`
`Cfl
`t:r'
`(II
`~
`0'\
`~
`00
`
`.a:::.
`...
`til
`~ ...
`til w
`00
`
`IS000263
`
`Immersion Systems LLC – Ex. 1008
`PGR 2021-00104 (U.S. 10,820,446 B2)
`7 of 16
`
`

`

`U.S. Patent May20, 1986
`
`Sheet 7 of8
`Fl G. 15
`
`4,590,538
`IZ
`
`/3
`
`~ _'0.:::-=:..
`
`37 ,o
`I~
`1,0
`
`(Q
`
`0
`
`3S
`
`0
`
`0
`
`97 0
`-- ~ -
`
`-
`""' -
`-
`___ __st:; __
`
`0 0
`o o0 ogo0 o
`0
`0
`~ 0
`J
`
`.
`
`-
`- - , . - -
`___ ::ti:J __
`
`I
`
`'/
`
`/Z4
`
`II
`
`IS000264
`
`Immersion Systems LLC – Ex. 1008
`PGR 2021-00104 (U.S. 10,820,446 B2)
`8 of 16
`
`

`

`U.S. Patent May 20, 1986
`Sheet 8 of 8
`FIG. 16
`
`I.Z
`
`4,590,538
`
`/0/
`
`/0/
`
`101
`
`/01
`
`60
`/
`
`I:®
`
`JJ/G
`
`II
`
`IS000265
`
`Immersion Systems LLC – Ex. 1008
`PGR 2021-00104 (U.S. 10,820,446 B2)
`9 of 16
`
`

`

`1
`
`IMMERSION COOLED HIGH DENSITY
`ELECTRONIC ASSEMBLY
`
`4,590,538
`
`BACKGROUND OF THE INVENTION
`This invention pertains generally to the field of elec(cid:173)
`tronic assemblies and cooling systems therefor. While
`applicable to a variety of electronic fields, it is believed
`that this invention has its greatest applicability in the
`field of high speed, high capacity digital computers, 10
`sometimes referred to as supercomputers, and the de(cid:173)
`scription herein of the presently preferred embodiment
`of the invention illustrates the use of the invention in
`such a computer.
`In the development of very high speed computers, 15
`great efforts have been directed toward reducing the
`physical dimensions of the computer assembly because
`signal propagation delays due to the maximum intercon(cid:173)
`nect path length limit the maximum clock rate and
`hence speed of operation of the computer. Currently 20
`available generations of logic and memory integrated
`circuits are capable of switching at clock rates in the
`nanosecond range, but in order for such a rate to be
`used in the computer, the maximum length of the lon(cid:173)
`gest interconnecting path between circuits must be held 25
`to a very short distance, for example, about 16 inches
`length in the case of twisted wire pairs for four nanosec(cid:173)
`ond operation.
`Advances in integrated circuit technology have pro(cid:173)
`duced devices with increased numbers of logic gates 30
`and memory circuits per chip, making it theoretically
`possible to assemble the great number of logic and
`memory circuits needed for a supercomputer within an
`area permitting a 16-inch or comparably short wire
`length interconnect distance. Unfortunately, that theo- 35
`retically possible high density cannot be achieved in
`practice unless the very considerable amount of heat
`generated by such a high density assemblage of circuits
`can be successfully removed. A single emitter coupled
`logic integrated circuit can dissipate as heat energy up 40
`to one watt of power. With high density packaging it is
`possible to put enough of such integrated circuits into a
`l-inch by 4-inch by 8-inch module to generate 600 or
`700 watts. When it is considered that many dozens of
`such modules would have to be placed close together to 45
`achieve the desired result, it can be appreciated that the
`amount of heat to be dissipated far exceeds available
`cooling techniques.
`A number of techniques have been developed in the
`field of electronics for cooling electronic components 50
`and circuits. When air cooling and forced air cooling
`became inadequate, liquid or refrigerant fllled cold bar
`or cold plate chassis members were developed for sup(cid:173)
`porting circuit modules and conducting heat away from
`them. In U.S. Pat. No. 4,120,021 which is assigned to 55
`the assignee of this application, a cooling system is dis(cid:173)
`closed employing refrigerant cooled cold bars having
`slots and clamping means for receiving the edges of
`plates to which circuit boards are mounted. Heat gener(cid:173)
`ated by the circuit components is transmitted by con- 60
`vection and conduction to the cold plates and then to
`the cold bars. While this technique has been extremely
`successful for its intended purpose, it is insufficient for
`the extremely high density and heat loading described
`above.
`Cooling of electronic components by immersion in
`inert liquid has been practiced in various forms in a
`number of areas of electronics. Inert liquids suitable for
`
`2
`electronic immersion are available, for example, a fluo(cid:173)
`rocarbon product called Fluorinert produced by the 3M
`Company. These liquids can be obtained with different
`boiling points to serve different needs. A common use
`5 has been the placing of a single component in such fluid
`for isolating it for testing purposes. High-powered recti(cid:173)
`fiers have also been immersion cooled. Computer cir(cid:173)
`cuit modules have been proposed in which a number of
`circuit boards have been mounted within a sealed con(cid:173)
`tainer to form a module of a computer system. The
`modules are filled with inert liquid which removes heat
`from the circuits by nucleate boiling and recondensing
`on the walls of the module. The heat is then transferred
`to the surrounding air by fins formed on the housing of
`the module.
`Immersion cooling has great advantages over air
`cooling in terms of higher heat transfer rate and higher
`heat capacity of liquid compared with gas. However,
`immersion of circuitry in fluid alone is not sufficient to
`solve the heat problems associated with the very high
`density, large scale systems discussed above. It is neces-
`sary to also provide for mechanical and electrical con(cid:173)
`struction of the electronic assembly in a manner that
`permits very high density packaging and effective re(cid:173)
`moval of heat from the components by the liquid, while
`still providing an assembly reasonably accessible for
`service or updates.
`
`SUMMARY OF INVENTION
`The present invention provides an improved immer(cid:173)
`sion cooled high density electronic assembly which
`permits extremely high speed operation of a high capac(cid:173)
`ity computer with large immediately available random
`access memory, all within an extremely compact vol(cid:173)
`ume, coupled with a liquid immersion and circulation
`system to successfully handle the very high heat load
`produced.
`According to one aspect of the invention, circuit
`elements are provided in stacks of circuit boards, with
`means supporting the boards and arranging the stacks
`adjacent one another but spaced apart to form coolant
`flow columns therebetween, the entire structure being
`within a container or tank for total immersion in a cool-
`ant liquid. Means are provided for supplying fluid into
`alternate ones of said coolant flow columns and out the
`others, to establish coolant flow between and across the
`circuit boards of the stacks.
`In a preferred form, standpipes are placed in the alter(cid:173)
`nate coolant flow columns for removal of coolant, after
`it passes through the stacks and rises upwardly along
`the standpipe then over the top.
`According to a preferred form of the invention,
`capped standpipes serving as distribution manifolds are
`placed in the alternate coolant inlet columns, and have
`a plurality of holes distributed along their lengths, to
`provide equal distribution of coolant to all circuit
`boards of the stack.
`According to another aspect of the invention, the
`circuit boards of the stacks are arranged in a plurality of
`modules, each module comprising a plurality of circuit
`boards containing electronic circuit components
`thereon, and secured together in spaced relationship by
`a plurality of spacer or jumper pins which also provide
`65 electrical connections between adjacent boards. Circuit
`components are preferably arranged in rows thereby
`forming channels for coolant flow across each board,
`between adjacent boards. The spacers are selected in
`
`IS000266
`
`Immersion Systems LLC – Ex. 1008
`PGR 2021-00104 (U.S. 10,820,446 B2)
`10 of 16
`
`

`

`4,590,538
`
`3
`length for high density packing while providing suffi(cid:173)
`cient clearance for components and space for coolant
`flow. Connectors are provided at first ends of the
`boards for interconnection by wiring harnesses to other
`boards or modules, and power connections are pro- 5
`vided at their other ends.
`According to a preferred form of the invention, a
`number of stacks of said modules are provided, with
`support means therefor provided for positioning the
`stacks in a generally radial relationship with their con- 10
`nector ends adjacent one another and with sectorshaped
`coolant flow columns positioned therebetween to pro(cid:173)
`vide coolant circulation to the stacks. External means
`are provided for circulating the coolant through the
`columns and thereby across the circuit boards, and for 15
`cooling the coolant for recirculation.
`
`4
`ment, takes the form of a cylinder having a polygon
`cross section, being made up of a plurality of flat panels
`15, supported by a framework of vertical and horizonal
`frame members 13 and 14, respectively. The frame
`members and panels are supported around a suitably
`shaped planar base member 11, and a cover 12 is also
`provided. For convenience in servicing the computer,
`the panels 15 are removable to provide access to the
`circuitry. Preferably they are also made of transparent
`material, which aids in filling and draining coolant and
`in service access in general.
`The base 11 has a number of apertures therein as
`explained more fully hereinafter to permit the passage
`of wiring and coolant flow paths to and from the inte(cid:173)
`rior of the tank. A coolant circulation inlet line 18, a
`coolant circulation outlet line 19, and a pair of coolant
`supply lines 20, 21 used for filling and draining coolant
`from the tank during pump up and pump down opera-
`tions are schematically indicated in FIG. 1 as being
`representative of a number of such connections. Base 11
`also has a plurality of holes through which wiring
`passes for bringing both AC power and data input/out(cid:173)
`put lines to the computer, and several groups of such
`wiring are schematically represented by reference num(cid:173)
`ber 17 in FIG. 1. It will be understood that a number of
`such conduits can be provided, FIG. 1 being only sche-
`matic in nature.
`Preferably the entire circuitry for the computer is
`30 placed within container 10 including one or more cen(cid:173)
`tral processing units and all logic circuits associated
`therewith, and a large immediately accessible random
`access memory. Input/output processors and mass stor(cid:173)
`age which are external of the computer and which oper-
`ate at relatively slower rates communicate with the
`computer through input/output leads which pass
`through base 11 to the computer. In other words, all
`logic and memory needed for high speed operation are
`placed together in close proximity within the tank.
`In addition, power supplies for the logic and memory
`are also provided within the tank, because the power
`supplies also produce a large amount of heat which
`must be removed, and also because of the desirability of
`having the power supplies in close physical proximity to
`the logic and memory circuits to reduce the length and
`the resistance of power and buses.
`For convenience in servicing the computer, it is pref-
`erable that the power supplies be located lower in the
`tank or container 10 and the memory and logic circuits
`positioned higher. Preferably the horizontal frame
`members 14 are positioned at approximately the divi-
`sion line between the power and other circuits, which
`permit removal of an upper panel to gain access to a
`portion of the computer circuitry after draining the
`55 coolant level down to the level of the power supplies
`and horizontal frame member 14. This provides an effi(cid:173)
`DETAILED DESCRIPTION OF THE
`ciency in service because the logic and memory sections
`PREFERRED EMBODIMENT
`would probably need service or updates more often
`than the power supply section.
`The immersion cooled high density electronic assem-
`bly of this invention is illustrated in the accompanying 60
`Before proceeding to a detailed description of the
`drawings and described herein in terms of a very high
`construction of the preferred embodiment of the inven-
`tion, a brief overview of the electronic assembly and the
`speed, large capacity digital computer. According to
`coolant circulation will be given with reference to FIG.
`the invention, all electronic circuitry and wiring for the
`12 and FIG. 2. In FIG. 12, which is a top view of the
`computer are mounted within a suitable tank or con-
`tainer, indicated by reference number 10 in FIG. 1, 65 tank of FIG. 1 with the cover removed, it will be seen
`that there are 16 stacks of circuit modules, three of
`which holds the inert coolant liquid in which the com-
`puter circuitry is immersed. The tank or container can
`which have been given reference numbers 30a, 30b, and
`take any convenient form, and in the preferred embodi-
`30c. The stacks consist of a number of modules each
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`In the drawing
`FIG. 1 is a side elevational view of a computer ng to 20
`the present invention showing the coolant fluid tank:
`FIG. 2 is a schematic representation of the coolant
`circulation system and flow paths for a computer ac(cid:173)
`cording to the present invention;
`FIG. 3 is a schematic representation of the coolant 25
`pump 1,1p and pump down system for the computer;
`FIGS. 4 and 5 are top plan and side elevational views
`respectively of a circuit module for the computer as(cid:173)
`sembly according to one aspect of the present inven(cid:173)
`tion;
`FIG. 6 is a fragmentary elevational view as seen
`generally from the line 6-6 of FIG. 4, at an enlarged
`scale;
`FIG. 7 is an enlarged sectional view showing the
`details of interconnection and support between circuit 35
`boards of a circuit module of FIGS. 4 and 5;
`FIGS. 8 and 9 are perspective views of the connec(cid:173)
`tors/spacers of FIG. 7;
`FIGS. 10 and 11 are top plan and side elevational
`views respectively of a power module used in the com- 40
`puter:
`FIG. 12 is a horizontal sectional view as seen gener(cid:173)
`ally from the line 12-12 of FIG. 1;
`FIG. 13 is an enlarged sectional view seen generally
`along line 13-13 of FIG. 12;
`FIG. 14 is a fragmentary horizontal section as seen
`along line 14-14 of FIG. 1 illustrating the relationship
`between a circuit module stack and adjacent coolant
`supply and return columns, at an enlarged scale;
`FIG. 15 is a vertical section on an enlarged scale 50
`taken along line 15-15 of FIG. 12, through a coolant
`return standpipe; and
`FIG. 16 is a vertical section at an enlarged scale taken
`along line 16-16 of FIG. 12, showing the perforated
`module support frame.
`
`45
`
`IS000267
`
`Immersion Systems LLC – Ex. 1008
`PGR 2021-00104 (U.S. 10,820,446 B2)
`11 of 16
`
`

`

`4,590,538
`
`5
`comprising a power supply or a plurality of horizontal
`circuit boards as explained subsequently. Each stack of
`modules is supported by a pair of vertical support
`frames 31, 32, which are adjacent the stack on either
`side. For purposes of clarity in FIG. 12, reference num- 5
`bers are applied to components associated with only a
`few of the stacks to avoid clutter of the drawing, but it
`will be understood that the patterns of components
`associated with stacks 30a, b and c are repeated around
`the whole assembly for each of the stacks.
`The module support frames 31 and 32 are vertically
`oriented planar shaped structures perforated with a
`large number of holes to permit the coolant to flow
`through them. Means are provided for supporting the
`stacks of modules on the adjacent supports. The sup- 15
`ports are arranged in a pattern around the central zone
`of the computer to form a radial array of stacks of mod(cid:173)
`ules with their logic connector ends adjacent one an(cid:173)
`other and facing inwardly to the central zone of the
`assembly. Between adjacent module stacks are formed 20
`sector or triangular shaped areas, indicated by reference
`numbers 33 and 34, which form vertical coolant flow
`columns. The inlet columns are indicated by reference
`number 33, and these alternate around the computer 25
`with outlet columns 34. The outlet columns 34 have
`standpipes 35 positioned therein. Standpipes 35 are also
`sector or triangular shaped, but are smaller than col(cid:173)
`umns 34 to provide spaces 36 between standpipe 35 and
`the adjacent module supports 31, 32 to allow coolant to 30
`rising to the top to flow over the top of the standpipe
`and flow down the center thereof. Preferably, holes are
`also provided along the length of standpipes 35 for
`removal of coolant.
`In the presently preferred embodiment of the inven- 35
`tion, distribution manifolds in the form of capped stand(cid:173)
`pipes 38 are used in inlet columns 33 to help even out
`the supply of fresh coolant to modules at all level in the
`stacks. Capped standpipes 38 are generally similar in
`configuration to outlet standpipes 35 except for a seal- 40
`ing cap at the top. They are placed over fluid inlet holes
`in the base, and they have a plurality of holes provided
`in their sides along the lengths thereof. These allow
`coolant to fill the distribution zones 39 within the inlet
`columns 33, to supply coolant to the adjacent stacks of 45
`modules.
`Coolant circulation holes 22 are provided in base 11
`within the sector-shaped areas for the vertical coolant
`columns. Appropriate connections are made by pipes
`beneath base 11 to connect these circulation holes 22 as so
`coolant inlets for columns 33 and capped standpipes 38,
`or as coolant outlets for columns 34 and standpipes 35.
`Depending upon the size of the computer involved,
`all 16 stacks may not be needed. The number of stacks
`needed depends upon the size of memory to be pro- 55
`vided and whether the computer is a single processor or
`multiprocessor computer. For example, a four stack
`computer, an eight stack computer, or a 16 stack com(cid:173)
`puter could be provided. If a computer of less than the
`full 16 stack complement were used, the tank or con- 60
`tainer 10 could of course be reshaped on the open or
`unused side to close around making a straight line ap(cid:173)
`proximation of an ellipse in cross section rather than
`continuing the full regular polygon shape. Also, it will
`be appreciated that by suitable changes to the angles, 65
`etc., a radial configuration having a greater or fewer
`number than 16 stacks could be provided as desired.
`However, opening the configuration up to include more
`
`6
`than 16 stacks could increase the maximum intercon(cid:173)
`nected path length within the central zone.
`Referring now to FIG. 2, a schematic representation
`of a four stack computer, or of four stacks of a larger
`computer is given. Adjacent pairs of vertical module
`support frames 31, 32 are shown, converging at their
`inner ends to form the sector-shaped vertical coolant
`flow columns. Three supply columns 33 including
`capped standpipes 38 are indicated, alternating with a
`10 pair of return columns 34, each of which contains an
`outlet standpipe 35.
`A pair of circulation pumps 40, 41 and corresponding
`heat exchangers 42, 43 are provided for circulating the
`liquid coolant through the computer immersed within
`the tank. Pump 40 circulates coolant through heat ex(cid:173)
`changer 42, where it is cooled by tap water or other
`cooling medium circulated through the heat exchanger.
`The cooled coolant flows through line 44, which
`branches to supply coolant to the left and right supply
`columns 33 of FIG. 2. Fluid from one of the standpipes
`35 connects to return line 45 to pump 40. Similarly, fluid
`from the bottom of the other standpipe 35 connects
`through line 47 to pump 41, which sends it through heat
`exchanger 43 and line 46 to the other inlet column 33.
`It will be appreciated that the power modules and
`also the logic and memory modules which are posi(cid:173)
`tioned in stacks between opposing module support
`frames as indicated in FIG. 12 have been omitted from
`FIG. 2 for purposes of illustrating the coolant flow.
`Coolant enters the tank in the vertical coolant flow
`columns 33, then travels horizontally through multiple
`paths, as suggested by the flow arrows, across and be(cid:173)
`tween all modules and all horizontal circuit boards that
`make up the modules. In flowing across and between
`the modules the coolant picks up the heat generated by
`their circuit components. Some of the coolant then
`flows through the holes in the outlet standpipes 35 and
`some of it flows vertically upward in zones 36 in col(cid:173)
`umns 34 outside the standpipes, then flows over the tops
`of the stand-pipes 35 down through them for recircula(cid:173)
`tion through the heat exchanger. The effect of inlet
`columns 33 is to provide fresh, equally cool coolant to
`all modules, both high and low in the stack. Some of the
`coolant may be heated to form small vapor bubbles and
`these also rise, but are sucked down and recondensed
`through standpipes 35.
`Liquid-to-liquid type heat exchangers can be used for
`heat exchangers 42 and 43, and city water supply can be
`used for the cooling, providing that it remains reason(cid:173)
`ably cool, for example approximately 50 degrees Fahr(cid:173)
`enheit. Otherwise other types of heat exchangers or
`refrigeration cycle equipment can be used. Additional
`circulation pumps, heat exchangers and paths would be
`provided for the additional stacks and columns for a
`system larger than the four stack diagram of FIG. 2.
`Alternatively, larger capacity units could be used to
`circulate through the other stacks.
`FIG. 3 is a diagram similar to FIG. 2, but showing the
`coolant pump up/pump down system, rather than the
`coolant circulation system, which is shown in FIG. 2. In
`FIG. 3, the coolant pump up/pump down standpipe 24
`is shown, together with one of the coolant supply/drain
`holes 23, of which four are shown in the base of the tank
`in FIG. 12. FIG. 3 also shows in schematic form pairs of
`module support frames 31 and 32 for four stacks of the
`computer. A reservoir 50 is provided, which for conve-
`nience may consist of a plurality of smaller reservoir
`tanks 51, interconnected by lines 52 through openings in
`
`IS000268
`
`Immersion Systems LLC – Ex. 1008
`PGR 2021-00104 (U.S. 10,820,446 B2)
`12 of 16
`
`

`

`4,590,538
`
`7
`8
`Interconnection of signal traces between adjacent
`their bases, to form a single reservoir. A standpipe 53 is
`circuit boards in a module is accomplished by jumper
`provided in the reservoir with its open top near the top
`pins that also serve as mechanical spacers.
`of the reservoir, but spaced below the cover. The bot-
`As seen in greater detail in FIGS. 6-9, a plurality of
`tom of standpipe 53 is not in communication with the
`reservoir but instead connects through a line 54 to 5 jumper pins 65, 66 are provided. These fit within holes
`standpipe 24 within the tank. Supply/drain hole 23
`which are provided between opposing pairs of boards,
`connects by a line 55, which branches to an inlet of
`and they serve not only to conduct signals between
`pump down pump 56 and to an outlet of pump up pump
`boards, but also to mechanically space and secure all the
`57. The connections at the bottom of reservoir 50, at
`boards together in the module. Two types of pins 65 and
`line 52, connect to a line 58 which branches to the inlet 10 66 are used, with pin 65 being used as a "starter" pin.
`Both pins 65 and 66 have a shank portion 67, tip por-
`to pump up pump 57 and the outlet of pump down
`tions 68 having a diameter less than the shank portion at
`pump 56.
`During a pump up operation to initially fill the com-
`one end, and a shoulder portion 69 between the shank
`puter tank 10 with coolant previously stored in reser-
`and the tip. The difference is that starter pin 65 has
`voir 50, pump 57 is activated and pump 56 is off. Pump 15 another tip 68 and shoulder 69 at its opposite end, while
`57 supplies coolant from the reservoir into supply/drain
`jumper pin 66 has a recess or socket 70 designed to
`holes 23 to begin filling the reservoir. Air displaced
`receive a tip 68. Slots may be provided in the end hav-
`from within the computer tank 10 escapes through
`ing recess 70 to accommodate a secure fit. In practice,
`jumper pins 65, 66 are quite small-smaller in propor-
`standpipe 24 which equalizes pressure within reservoir
`50 filling the volume as the coolant is lowered. Stand- 20 tion to the intergrated circuits than is indicated in FIG.
`pipe 24 also prevents overfilling of the tank. After the
`7, wherein they have been somewhat exaggerated in
`tank is filled, the circulation system of FIG. 2 is started
`size for purposes of illustration.
`and then electrical power can be provided to the com-
`Rather than providing uniform patterns of jumper
`puter circuitry. The pump up operation is generally
`pins on each board, the number used and their locations
`maintained on at the same time that the circulation 25 may be varied as dictated by considerations of where
`system is on, because the pump up circuit complements
`signal path jumps to adjacent boards are required. If a
`the circulation action. Standpipe 24 is slightly higher
`signal jump between only two adjacent boards is re-
`than the circulation standpipes 35, and it helps to pull
`quired, a single starter pin 65 would be used, as indi-
`down vapor that may be formed from the operation of
`cated in the center of FIG. 7. If jump connections are
`the computer. As previously mentioned, as coolant 30 needed between three or more boards at the same loca-
`tion, a starter pin 65 is used, and where its tip projects
`circulates over the circuit elements, some vapor bubbles
`may be formed and these rise to the top of the container,
`through one of the boards, a jumper pin 66 fits over it
`so in typical operation there will be a small vapor
`and continues to the next board, and so on for as many
`pocket at the top. Standpipe 24 helps to pull down the
`boards as need to be interconnected at that location. In
`vapor, where it is eventually recondensed with the 35 each case the tip of the jumper projects through the
`other fluid.
`clearance hole 71 provided in the board, and the shoul-
`For pumping down the system to gain access for
`der portion 69, or the end around recess 70, as the case
`service, after electrical power is removed from the
`may be, butts against the board and makes contact with
`computer the pump up and circulation systems are
`signal traces surrounding the holes. The individual
`stopped, and the pump down pump 56 is started. It 40 boards are initially drilled at the proper locations for the
`transfers coolant from supply/drain hole 23 into reser-
`jumper pins then plated through. As the module is built
`voir 50, and standpipes 24 and 53 and their intercon-
`up board by board, the pins are inserted and soldered
`necting line 54 permit displacement of air or vapor to
`until the full stack of boards comprising the module is
`neutralize pressure as the liquid coolant is transferred.
`assembled. Typically, a great number of jumper pins is
`According to one aspect of the invention, logic and 45 required and provided at a variety of locations across
`memory circuits are construc