( 12 ) United States Patent
`Boyd et al .
`
`( 10 ) Patent No . : US 10 , 405 , 457 B2
`( 45 ) Date of Patent :
`Sep . 3 , 2019
`
`US010405457B2
`
`( 54 ) APPLIANCE IMMERSION COOLING
`SYSTEM
`( 71 ) Applicants : Christopher L . Boyd , Austin , TX ( US ) ;
`James P . Koen , Round Rock , TX ( US ) ;
`David Christopher Laguna , Austin ,
`TX ( US ) ; Thomas R . Turner ,
`Georgetown , TX ( US ) ; Kenneth D .
`Swinden , Hutto , TX ( US ) ; Mario
`Conti Garcia , Austin , TX ( US ) ; John
`Charles Tribou , Austin , TX ( US )
`( 72 ) Inventors : Christopher L . Boyd , Austin , TX ( US ) ;
`James P . Koen , Round Rock , TX ( US ) ;
`David Christopher Laguna , Austin ,
`TX ( US ) ; Thomas R . Turner ,
`Georgetown , TX ( US ) ; Kenneth D .
`Swinden , Hutto , TX ( US ) ; Mario
`Conti Garcia , Austin , TX ( US ) ; John
`Charles Tribou , Austin , TX ( US )
`( 73 ) Assignee : Midas Green Technologies , LLC ,
`Austin , TX ( US )
`Subject to any disclaimer , the term of this
`patent is extended or adjusted under 35
`U . S . C . 154 ( b ) by 680 days .
`14 / 355 , 533
`Dec . 13 , 2013
`PCT / US2013 / 075126
`
`( * ) Notice :
`
`( 21 ) Appl . No . :
`( 22 ) PCT Filed :
`( 86 ) PCT No . :
`$ 371 ( c ) ( 1 ) ,
`Apr . 30 , 2014
`( 2 ) Date :
`( 87 ) PCT Pub . No . : WO2014 / 109869
`PCT Pub . Date : Jul . 17 , 2014
`Prior Publication Data
`US 2015 / 0181762 A1 Jun . 25 , 2015
`Related U . S . Application Data
`( 60 ) Provisional application No . 61 / 737 , 200 , filed on Dec .
`14 , 2012 , provisional application No . 61 / 832 , 211 ,
`filed on Jun . 7 , 2013 .
`
`( 65 )
`
`( 58 )
`
`( 56 )
`
`1 )
`
`2 )
`
`Int . CI .
`( 2006 . 01 )
`HOIL 23 / 44
`H05K 7 / 20
`( 2006 . 01 )
`U . S . CI .
`CPC . . . . . . . . . H05K 7 / 20236 ( 2013 . 01 ) ; H01L 23 / 44
`( 2013 . 01 ) ; H05K 7 / 20272 ( 2013 . 01 )
`Field of Classification Search
`CPC . . . . . . . . . . . . . . . . . . . . . . HO5K 7 / 20236 ; HO5K 7 / 20272 ;
`HO1L 23 / 42 ; HO1L 23 / 44
`( Continued )
`References Cited
`U . S . PATENT DOCUMENTS
`4 , 590 , 538 A *
`5 / 1986 Cray , Ir . . . . . . . . . . . HO5K 7 / 20236
`361 / 700
`5 , 167 , 511 A * 12 / 1992 Krajewski . . . . . . . . . . . . HO1R 4 / 01
`361 / 785
`( Continued )
`FOREIGN PATENT DOCUMENTS
`. . . G06F 1 / 20
`JP
`5956100 B1 *
`7 / 2016
`2042294 Ci
`RU
`8 / 1995
`1764094 AL
`SU
`9 / 1992
`Primary Examiner — Devon Russell
`( 74 ) Attorney , Agent , or Firm — Jeffrey Van Myers
`ABSTRACT
`( 57 )
`A appliance immersion tank system comprising : a generally
`rectangular tank adapted to immerse in a dielectric fluid a
`plurality of appliances , each in a respective appliance slot
`distributed vertically along , and extending transverse to , the
`long axis of the tank ; a primary circulation facility adapted
`to circulate the dielectric fluid through the tank ; a secondary
`fluid circulation facility adapted to extract heat from the
`dielectric fluid circulating in the primary circulation facility ,
`and to dissipate to the environment the heat so extracted ; and
`a control facility adapted to coordinate the operation of the
`primary and secondary fluid circulation facilities as a func
`tion of the temperature of the dielectric fluid in the tank . A
`plenum , positioned adjacent the bottom of the tank , is
`adapted to dispense the dielectric fluid substantially uni
`( Continued )
`
`34b
`
`706
`
`Wh
`
`34a
`
`y
`
`36
`
`Il
`
`wwwwwww
`
`- 72
`70a
`
`MGT000853
`
`Immersion Systems LLC – Ex. 1023
`GR 2021-00104 (U.S. 10,820,446 B2)
`1 of 15
`
`

`

`US 10 , 405 , 457 B2
`Page 2
`
`formly upwardly through each appliance slot . A weir , inte
`grated horizontally into a long wall of the tank , is adapted to
`facilitate substantially uniform recovery of the dielectric
`fluid flowing through each appliance slot . All active and
`most passive components of both the primary and secondary
`fluid circulation facilities , and the control facility are fully
`redundant , and are adapted automatically to operate in a
`fail - soft mode .
`16 Claims , 7 Drawing Sheets
`
`( 56 )
`
`( 58 ) Field of Classification Search
`USPC
`. . . . . . . . . . . . . 361 / 699
`See application file for complete search history .
`References Cited
`U . S . PATENT DOCUMENTS
`5 , 297 , 621 A *
`3 / 1994 Taraci . . . . . . . . . . . . . . . GO1R 31 / 2891
`165 / 104 . 13
`8 , 009 , 419 B2
`8 / 2011 Attlesey et al .
`HO2M 7 / 003
`2005 / 0259402 A1 *
`11 / 2005 Yasui
`361 / 716
`2006 / 0126292 A1 *
`6 / 2006 Pfahnl . . . . . . . . . . . . H05K 7 / 20563
`361 / 695
`2006 / 0274501 A1 * 12 / 2006 Miller . . . . . . . . . . . . . . . GOIR 31 / 2863
`361 / 690
`2011 / 0075353 A1 *
`3 / 2011 Attlesey
`G06F 1 / 20
`. . . . . . . . . . . . . .
`361 / 679 . 47
`2011 / 0132579 A1 *
`6 / 2011 Best . . . . . . . . . . . . . . . . . . . HO5K 7 / 20763
`165 / 104 . 31
`2011 / 0240281 A1 * 10 / 2011 Avery . . . . . . . . . . . . . GO5D 23 / 1917
`165 / 287
`
`* cited by examiner
`
`MGT000854
`
`Immersion Systems LLC – Ex. 1023
`GR 2021-00104 (U.S. 10,820,446 B2)
`2 of 15
`
`

`

`U . S . Patent
`
`Sep . 3 , 2019
`
`Sheet 1 of 7
`
`US 10 , 405 , 457 B2
`
`34b
`
`WWWWWWWWWWWWWWWWWWWWWWWW
`
`?
`
`ir
`
`'
`
`.
`
`Yawa
`
`.
`
`wwwmmmmm
`
`Inwong
`
`34a 34a
`
`INC
`20
`
`10
`
`eur
`
`
` . l WWW .
`ittlig
`WA in
`LUB Event
`.
`. . 1 . 11
`: 21 : 55
`A
`ti
`9 : 10
`.
`u
`1
`11
` nu * * ) : 16
`
`1 . 1
`12
`t
`1 . .
`11 . 11 .
`Hurumu
`r .
`1 . 4 . 0
`
` . run .
`
`t .
`H . , 11 . Thef
`20
`.
`1 . 1 . 1 . L 1 .
`a t .
`1
`* 114 . 164 . tit
`1 .
`11 . The
`.
`1 .
`9 . 11 14
`WW . 1
`1 : 1
`11 .
`
`117 .
`.
`10 :
`13 . 11
`2 . Writi
`14 : 15 1 .
`14 .
`41 Vanuit
`DOH
`: 14 .
`10
`tinin
`11
`
`11 .
`.
`W
`
`
` 011 . . 195 .
`1
`. 1
`
` MAU . 14 . 1 .
`.
`1 - 11Tinit
`.
`1
`10 . 16
`10 : 15
`.
`!
`: 15
`yu
`MAX
`.
`4
`
`
` 1 . 1 : bit
`.
`
`.
`
`t
`
`L
`
`.
`
`.
`
`1 . 1 16 : 51
`
`1
`
`14
`Fig . 1
`
`wwwwwwwwwwwwww
`
`weer
`
`72
`70a
`
`orenwon YUYUYLYWYriririririririririri
`
`wanneer
`
`m
`
`MGT000855
`
`Immersion Systems LLC – Ex. 1023
`GR 2021-00104 (U.S. 10,820,446 B2)
`3 of 15
`
`

`

`U . S . Patent
`
`Sep . 3 , 2019
`
`Sheet 2 of 7
`
`US 10 , 405 , 457 B2
`
`34b
`
`46
`
`38
`
`Unimin
`
`
`
`WWW . KLON MANUALIDAHULUCULUKUUTA
`
`wwwwwwwwwww
`
`porcorocco COISAKCSeccuckuckOCKSMIGOS
`
`a n
`
`APUNAR
`
`AP
`
`UHA
`
`
`
`chini GAMAKANKAKOULULAUGULAKANG B
`
`44a
`
`7 .
`
`4 : 2 . W
`
`fratering
`
`' n
`
`RINNEN
`
`
`
`with !
`
`
`
`
`
`ruralement restauranten wwwmmmma
`
`wwhite
`
`$
`
`34a
`
`320
`Santa
`486286
`Fig . 2
`38
`46
`
`temu
`
`32a 33
`48a > 28a
`
`.
`
`93
`Ex
`
`*
`
`* *
`
`VV2
`
`wwwwwww
`
`21
`
`4
`
`. Wowww
`
`27 . 4
`
`in het
`
`w
`
`CALE ) :
`
`Fig . 4
`
`ht
`48a
`
`42
`
`setmanes en
`
`het licht
`
`Het
`
`Fig . 3
`
`MGT000856
`
`Immersion Systems LLC – Ex. 1023
`GR 2021-00104 (U.S. 10,820,446 B2)
`4 of 15
`
`

`

`U . S . Patent
`
`Sep . 3 , 2019
`
`Sheet 3 of 7
`
`US 10 , 405 , 457 B2
`
`.
`
`S
`*
`
`2
`
`.
`
`* * * * *
`
`*
`
`*
`
`*
`w
`* * *
`SA
`
`the
`
`SOULAN
`
`mm 22
`
`M
`* * * * * * *
`
`w
`* *
`
`* *
`
`Fig . 5
`
`WWW
`
`www
`
`NNN
`
`WWW .
`
`14
`Section C - C
`Fig . 6
`
`MGT000857
`
`Immersion Systems LLC – Ex. 1023
`GR 2021-00104 (U.S. 10,820,446 B2)
`5 of 15
`
`

`

`U . S . Patent
`
`Sep . 3 , 2019
`
`Sheet 4 of 7
`
`US 10 , 405 , 457 B2
`
`.
`
`.
`n 1
`
`VN
`
`. .
`
`H
`
`1
`
`- H
`
`.
`
`1 . 1 .
`
`1 .
`
`*
`
`O
`
`.
`4
`
`1
`
`LLA .
`2
`: 1 - . .
`PRW
`. 1
`
`8
`. . .
`
`BUWIE
`. . TUP ya
`WiiwtiN
`HR .
`TwWEMA
`WWW . NO
`1 . t u .
`4 , Wi
`. W h
`I ' I : 4 VMI
`OM .
`ie
`n . hu IV . 1 .
`11
`tu
`WWW .
`2 : 13 . AURI
`:
`.
`. 2
`bebu . .
`W
`. . Hup Wit .
`Notturn
`. 1
`m
`. tw
`VW Pattur
`WIN
`1 . 0
`WWW . . *
`21 .
`1 .
`, 1 . N ritud
`ANOPY
`
`!
`W hil
`thp , RN1
`. . .
`#
`0 . 94
`NYIM . Mh
`h2
`Wt . hr .
`# AMFOAI
`WK16
`NINNU . ht .
`WRt .
`41NAVA
`MMK3
`HEIM
`WWW .
`. * UWh 11
`
`.
`
`1
`
`N . P . W
`1
`U W
`2 . ru 4
`
`*
`*
`
`*
`*
`* *
`*
`. .
`
`Niti
`
`LUKA
`
`0 .
`
`N
`
`1 : 4 .
`
`
`* W
`
`O W
`:
`.
`1
`
`I
`
`Sport
`
`T
`
`A
`*
`*
`
`*
`
`#
`
`A
`#
`#
`
`.
`
`Fig . 7
`
`tt *
`
`Fig . 8
`
`36a
`
`ut
`
`#
`
`4 . Y
`
`OU
`
`3
`
`ts
`
`36b
`
`Fig . 9
`
`MGT000858
`
`Immersion Systems LLC – Ex. 1023
`GR 2021-00104 (U.S. 10,820,446 B2)
`6 of 15
`
`

`

`U . S . Patent
`
`Sep . 3 , 2019
`
`Sheet 5 of 7
`
`US 10 , 405 , 457 B2
`
`??
`
`Fig . 10
`
`16
`
`c 22
`
`14a
`
`68
`
`14b
`
`+
`
`"
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`. -
`
`I
`
`.
`
`2
`21 :
`
`1
`
`:
`
`O BEX
`
`M
`M
`M
`AW
`X
`XXX
`WMWWWW
`* *
`*
`* * * * * *
`*
`*
`*
`*
`* *
`*
`* * *
`X
`X X *
`*
`*
`* * *
`* WWW *
`KWA KUWA
`NWWLWWLISISSA SISSE
`
`2
`
`* * *
`
`2
`
`WW
`
`36
`
`Fig . 11
`
`66
`
`MGT000859
`
`Immersion Systems LLC – Ex. 1023
`GR 2021-00104 (U.S. 10,820,446 B2)
`7 of 15
`
`

`

`U . S . Patent
`
`Sep . 3 , 2019
`
`Sheet 6 of 7
`
`US 10 , 405 , 457 B2
`
`14
`
`* *
`
`*
`
`C
`
`36
`
`WWWXXXXXXXX
`
`
`
`
`
`
`. IKKE - WIONOXXU
`
`48a
`
`IIIIIIIIIII
`
`30
`
`28a
`
`32a
`
`44b
`
`48b
`
`38
`
`28b
`
`32b
`
`52b
`
`50b
`
`30a
`
`54a
`
`1
`
`30b
`
`546
`
`11
`
`Fig . 12
`
`MGT000860
`
`Immersion Systems LLC – Ex. 1023
`GR 2021-00104 (U.S. 10,820,446 B2)
`8 of 15
`
`

`

`U . S . Patent
`
`Sep . 3 , 2019
`
`Sheet 7 of 7
`
`US 10 , 405 , 457 B2
`
`22
`
`14
`
`*
`
`* * 86
`
`- - - - - - - - - - -
`
`- - - -
`
`Primary
`Controller
`
`58a
`
`RRRRRRRARA D
`
`IDINIEU
`
`32a
`
`NOON . JOJO ,
`
`Master
`Controller
`
`TCP / IP
`
`48a
`
`4
`
`28a 4
`. .
`
`.
`30a
`
`Secondary
`Controller
`
`60a
`
`M
`
`Tower
`
`50a i
`
`Fig . 13
`
`MGT000861
`
`Immersion Systems LLC – Ex. 1023
`GR 2021-00104 (U.S. 10,820,446 B2)
`9 of 15
`
`

`

`US 10 , 405 , 457 B2
`
`APPLIANCE IMMERSION COOLING
`SYSTEM
`
`is an essential element in the system architecture , such as the
`central processing unit ( “ CPU ” ) . One possible solution to
`this problem is to immerse circuit assemblies vertically into
`CROSS - REFERENCE TO RELATED
`a tank containing the cooling fluid such that each of the
`5 various assemblies can be withdrawn independently from
`APPLICATIONS
`the tank for servicing , replacement , upgrade , etc . One inter
`This application is related to the following Provisional
`esting example of such a system is disclosed in a web
`Applications :
`presentation entitled “ Puget Custom Computer ' s mineral
`1 . Ser . No . 61 / 737 , 200 , filed 14 Dec . 2012 ( “ First Parent
`oil - cooled PC ” , by Nilay Patel ( “ Puget ” ) ( posted 12 May
`Provisional ” ) ; and
`2007 at 11 : 57 AM ; a copy of which is submitted herewith ) .
`2 . Ser . No . 61 / 832 , 211 , filed 7 Jun . 2013 ( “ Second Parent
`As noted by the author , the lack of supplemental apparatus
`Provisional " ) ;
`in the Puget system to extract waste heat from
`the oil
`and hereby claims benefit of the filing dates thereof pursuant
`inherently limited its operating capabilities .
`to 37 CFR § 1 . 78 ( a ) ( 4 ) . ( Collectively , “ Parent Provision
`Another problem with the Cray Research systems in
`als ” ) . The subject matter of the Parent Provisionals , each in 15 particular is the nature and cost of the chosen cooling fluid :
`its entirety , is expressly incorporated herein by reference .
`fluorocarbon liquids . As is known , other dielectric fluids ,
`such as mineral oil , have better heat transfer characteristics ;
`BACKGROUND OF THE INVENTION
`of course , being an oil , the use thereof does represent a
`20 greater residue problem on modules that may be repairable .
`1 . Field of the Invention
`Notwithstanding , the Puget system implemented precisely
`The present invention relates generally to electrical appli -
`ance cooling systems , and , in particular , to an improved
`this design choice .
`appliance immersion cooling system and method of opera
`US Patent Application Publication 2011 / 0132579 , “ Liq
`uid Submerged , Horizontal Computer Appliance Rack and
`tion .
`2 . Description of the Related Art
`25 Systems and Method of Cooling such a Appliance Rack ” ,
`Best , et al . ( “ Best ” ) , discloses a appliance immersion tank
`In general , in the descriptions that follow , we will italicize
`system , include support apparatus for extracting waste heat
`the first occurrence of each special term of art which should
`be familiar to those skilled in the art of immersion cooling
`from the tank cooling fluid and dissipating to the environ
`systems . In addition , when we first introduce a term that we ment the heat so extracted . Although an improvement in
`believe to be new or that we will use in a context that we 30 several respects over the prior art discussed above , this
`believe to be new , we will bold the term and provide the
`system exhibits , inter alia , the following problems : generally
`definition that we intend to apply to that term .
`non - uniform flow patterns through the several appliance
`U . S . Pat . No . 4 , 590 , 538 , “ Immersion Cooled High Den
`slots within the tank , potentially resulting in uneven cooling
`sity Electronic Assembly ” , Cray ( filed 18 Nov . 1981 and
`across all slots ; constricted dielectric fluid supply and return
`issued 20 May 1986 ) ( “ Cray ” ) , is an early example of an 35 ports resulting in unnecessarily high fluid flow velocities at
`immersion system for cooling electronic components during
`the respective points of connection to the tank ; poor scal
`normal operation . On information and belief , the machine
`ability ; and inadequate attention to fail - soft operation .
`disclosed therein was the Cray - 2 super - computer ( “ Cray - 2 ” )
`The subject matter of all of the prior art references
`manufactured by Cray Research , Inc . ( “ Cray Research " ) , of
`discussed above , each in its entirety , is expressly incorpo
`Chippewa Falls , Wis . Of particular interest to the present 40 rated herein by reference .
`application is the description of the significant advantages
`We submit that what is needed is an improved appliance
`resulting from using an electrically non - conductive or
`tank immersion system and method of operation . In particu
`dielectric fluid to extract heat from electronic circuit assem
`lar , we submit that such a system should provide perfor
`blies during normal operation ( see , e . g . , col . 1 , line 66 - col .
`mance generally comparable to the best prior art techniques
`2 , line 29 ) .
`45 but more efficiently and effectively than known implemen
`On information and belief , Cray Research released , in
`tations of such prior art techniques .
`1985 , a marketing brochure entitled “ The CRAY - 2 Com
`puter System ” ( a copy of which is submitted herewith )
`BRIEF SUMMARY OF THE INVENTION
`describing the Cray - 2 . Of particular interest in this brochure
`In accordance with a preferred embodiment of our inven
`is the description therein of the significant advantages result - 50
`ing from using a dielectric fluid to extract heat from elec -
`tion , . . .
`tronic circuit assemblies during normal operation ( see , pages
`BRIEF DESCRIPTION OF THE SEVERAL
`10 and 13 ) .
`VIEWS OF THE DRAWINGS
`U . S . Pat . No . 5 , 167 , 511 , “ High Density Interconnect
`Apparatus ” , Krajewski , et al . ( issued 27 Nov . 1992 ) ( “ Kra - 55
`jewski ” ) , discloses another example of an immersion system
`Our invention may be more fully understood by a descrip
`for cooling electronic components during normal operation
`tion of certain preferred embodiments in conjunction with
`( see , e . g . , col . 2 , lines 43 - 51 ) . On information and belief , a
`the attached drawings in which :
`machine implementing the Krajewski system was also mar -
`FIG . 1 illustrates , in partial cut - away form , a front per
`keted by Cray Research as a follow - on super - computer to 60 spective of a tank module of an appliance immersion cooling
`the Cray - 2 .
`system constructed in accordance with our invention ;
`One particular problem in the vertical - stack - type systems
`FIG . 2 illustrates a rear perspective of the tank module
`disclosed in the above references is the necessity of draining
`shown in FIG . 1 ;
`the cooling fluid whenever physical access to the electronic
`FIG . 3 illustrates a close - up perspective of a detail A of
`modules was required . In general , such an operation , besides 65 FIG . 2 ;
`being time consuming , requires the entire system to be
`FIG . 4 illustrates a close - up perspective of a detail B of
`switched off , especially if the component requiring attention
`FIG . 2 ;
`
`MGT000862
`
`Immersion Systems LLC – Ex. 1023
`GR 2021-00104 (U.S. 10,820,446 B2)
`10 of 15
`
`

`

`US 10 , 405 , 457 B2
`
`5
`
`shown in FIG . 2 ) ; and control equipment cabinets 34a and
`FIG . 5 illustrates , in perspective view , several details of
`34b , each adapted to accommodate the module status and
`the tank shown in FIG . 1 , with special emphasis on the
`control equipment associated with a respective one of the
`dielectric fluid recovery weir integrated into the long rear
`primary circulation facilities 28a and 28b ( see , FIG . 13 ) .
`wall of the tank ;
`FIG . 6 illustrates , in cross - section view , the section C - C
`As can be best seen in FIG . 2 , the primary circulation
`facility 28 ( comprising redundant sub - facilities 28a and 28b )
`in FIG . 5 ;
`comprises both passive ( conduits , couplers , etc . ) and active
`FIG . 7 illustrates , in perspective view , the plenum facility
`( valves , pumps , sensors , etc . ) components ; a subset of the
`shown in FIG . 1 ;
`passive components are shared , whereas , in general , the
`FIG . 8 illustrates , in top plan view , the orifice plate
`portion of the plenum facility shown in FIG . 7 ;
`10 active components are duplicated and adapted to cooperate
`in operation as separate , redundant sub - facilities . Excluding
`FIG . 9 illustrates , in perspective view , the chamber por -
`tion of the plenum facility shown in FIG . 7 ;
`the tank 14 , the primary shared component is the plenum
`facility 36 ( see , FIG . 1 and FIG . 7 ) comprising an orifice
`FIG . 10 illustrates , in top plan view , a plurality of appli
`plate 36a ( see , FIG . 8 ) and a plenum chamber 36b ( see , FIG .
`ance slots distributed vertically along , and extending trans -
`15 9 ) . As can be seen in FIG . 1 , cooled dielectric fluid is
`verse to , a long axis of the tank of FIG . 1 ;
`pumped into both ends of the plenum facility 36 via a shared
`FIG . 11 illustrates , in longitudinal cross - sectional view ,
`distribution header 38 ( see , FIG . 2 and FIG . 3 ) . In general ,
`the plurality of appliance slots distributed vertically along ,
`the plenum plate 36a comprises at least one row of orifices
`and extending transverse to , the long axis of the tank of FIG .
`vertically aligned with each appliance slot 18a , with the
`1 :
`FIG . 12 illustrates , in flow schematic form , one instan - 20 dimensions and flow rates of each set being adapted to
`tiation of a flow arrangement suitable for implementing our
`provide substantially equal flow of the dielectric fluid
`upwardly into each appliance slot 18a . Preferably , each
`invention ; and
`FIG . 13 illustrates , in control schematic form , one instan
`appliance slot 18a is supplied via several rows of orifices ,
`tiation of a flow control facility suitable for implementing
`thus generally tending to reduce the volume of the dielectric
`25 fluid exiting each orifice and to make the flow of dielectric
`our invention .
`In the drawings , similar elements will be similarly num -
`fluid more uniform upwardly through the appliance slots 18 .
`bered whenever possible . However , this practice is simply
`One further shared component is the dielectric fluid recovery
`for convenience of reference and to avoid unnecessary
`facility 40 ( FIG . 2 ) comprising a dielectric fluid recovery
`proliferation of numbers , and is not intended to imply or
`reservoir 42 ( see , FIG . 3 , FIG . 4 and FIG . 13 ) positioned
`suggest that our invention requires identity in either function 30 vertically beneath the overflow lip of the weir 22 and
`adapted smoothly to receive the dielectric fluid as it flows
`or structure in the several embodiments .
`over the weir 22 ; the dielectric fluid recovery reservoir 42 is
`DETAILED DESCRIPTION OF THE
`further adapted to allow the recovered fluid to be removed
`from the reservoir 42 via redundant recovery ports 44a and
`INVENTION
`35 44b ( only port 44a can be seen in FIG . 2 as the port 44b is
`obscured by the heat exchanger 32a ; but see FIG . 12 ) . As
`Shown in FIG . 1 ( front view ) and FIG . 2 ( rear view ) is a
`can be seen in both FIG . 3 and FIG . 4 , we consider it
`tank module 10 adapted for use in an appliance immersion
`desirable to provide a vortex breaker at the input of each of
`cooling system constructed in accordance with a preferred
`the recovery ports 44 . Also , we provide a removable recov
`embodiment of our invention . For convenience of reference ,
`we have illustrated in FIG . 1 the tank facility 12 of the 40 ery reservoir cover 46 adapted to also cover a major portion
`immersion module 10 in partial cut - away to emphasize
`of the distribution header 38 ; note that , in both FIG . 2 and
`several important internal facilities ; we have shown the tank
`FIG . 3 , we have illustrated the reservoir cover 46 in
`a
`facility 12 in isolation in FIG . 5 . In general , the tank facility
`partially raised orientation so as to better depict details that
`12 comprises : a tank 14 adapted to immerse in a dielectric
`would otherwise be obscured . Note that we have constructed
`fluid a plurality of electrical appliances 16 , e . g . , contempo - 45 the reservoir 42 such that the average height of dielectric
`rary computer servers ( see , e . g . , FIG . 11 ) , each in a respec -
`fluid above the recovery ports 44 develops sufficient hydro
`tive appliance slot 18a distributed vertically along , and
`static head to meet the requirements of the pumps 48 , while
`extending transverse to , a long axis of the tank 14 ( see
`also tending to minimize the likelihood of breaking suction
`generally , FIG . 10 ) ; an appliance rack facility 20 of con -
`during normal operation .
`vention design adapted to suspend the appliances 16 ( see , 50
`At this point in the primary circulation facility 28 , we
`e . g . , FIG . 11 ) in respective appliance slots 18 ( see , FIG . 10 ) ;
`provide fully redundant sub - facilities 28a and 28b , each
`a weir 22 ( best seen in isolation in FIG . 5 and FIG . 6 ) ,
`comprising a primary circulation pump ( 48a and 48b ) and
`integrated horizontally into one long wall of the tank 14
`associated passive and active components which , collec
`adjacent all appliance slots 18 , and adapted to facilitate
`tively , provide the motive power for circulating the dielectric
`substantially uniform recovery of the dielectric fluid flowing 55
`fluid through the shared components and tank 14 . As can be
`through each of the appliance slots 18 ; an interconnect panel
`generally seen , each of these sub - facilities 28a and 28b is
`facility 24 attached to the upper rear edge of the tank 14 and
`adapted to recover the dielectric fluid exiting the tank 14 via
`adapted to mount various appliance power distribution
`the weir 22 , re - pressurize the recovered fluid , pass the
`equipment , cable interconnection panels and the like ( none
`re - pressurized fluid through a respective one of the heat
`shown ) ; and a cover 26 adapted to be opened and closed 60 exchangers 32a and 32b , and then back to the plenum
`from the front of the tank 14 ( and which may include a
`facility 36 via the header 38 .
`translucent portion to allow viewing of the interior of the
`Shown in FIG . 12 is one flow arrangement suitable for
`tank 14 when in the closed position ) . In addition to the tank
`integrating our tank module 10 into a fully redundant ,
`facility 12 , the immersion module 10 comprises : a primary
`appliance immersion cooling system , comprising the pri
`circulation facility 28 ( portions of which are shown in both 65 mary circulation facility 28 and the secondary fluid circu
`FIG . 1 and FIG . 2 ) ; a secondary fluid circulation facility 30
`lation facility 30 . In general , the secondary fluid circulation
`( of which only redundant heat exchangers 32a and 32b are
`facility 30 comprises redundant secondary circulation sub
`
`MGT000863
`
`Immersion Systems LLC – Ex. 1023
`GR 2021-00104 (U.S. 10,820,446 B2)
`11 of 15
`
`

`

`US 10 , 405 , 457 B2
`
`m
`
`5
`
`performed by the controllers 58 , 60 and 62 may be imple
`facilities 30a and 30b , each of which is adapted to circulate
`ented in the form of dedicated application - specific soft
`a cooling fluid , e . g . , treated water , through the respective
`ware executing on a conventional computer platform having
`heat exchanger 32a and 32b to extract heat from dielectric
`the appropriate resources ; indeed , it would be entirely fea
`fluid counter - circulating therethrough and to dissipate to the
`sible to implement the entire control facility 56 on a server
`environment the heat so extracted . In the illustrated embodi -
`16 installed in a tank 14 .
`ment , each of the secondary fluid sub - facilities 30a and 306
`m
`One desirable enhancement that we recommend is a
`comprise conventional cooling towers 50a ( including fan
`remote control facility , implemented , e . g . , via the master
`facility 52a ) and 50b ( including fan facility 52b ) , and
`controller 62 ( or by way of a direct , per - controller interface ) ,
`secondary circulation pumps 54a and 54b . To facilitate
`flexible operation in installations including multiple immer - 10 adapted to facilitate remote monitoring of system status
`sion modules 10 in combination with a plurality of second -
`( e . g . , temperatures , pressures , etc . ) and control over system
`ary circulation sub - facilities 30 , a common header arrange
`control parameters ( e . g . , temperature and pressure limits ,
`ment can be implemented as illustrated in the secondary
`etc . ) to the primary controllers 58 and secondary controllers
`fluid circulation loop , with flow control valves located at key
`60 . For example , using a conventional data communication
`15 hardware module 64 , e . g . , an ethernet card implementing the
`flow control points as is known .
`Shown in FIG . 13 is a control facility 56 adapted to
`TCP / IP protocol , a modern web browser can be adapted to
`monitor and control the operation of both the immersion
`provide a graphical user interface ( “ GUI ” ) with sufficient
`module 10 ( including all active components of the primary
`functionality to facilitate monitoring and control of an entire
`circulation facility 28 ) , and the secondary fluid circulation
`installation from a remote location . Such a GUI may be
`facility 30 . As will be evident to those skilled in this art , 20 implemented using any of a number of programming para
`efficient operation of our immersion module 10 requires
`digms , e . g . , PHP , . NET and the like .
`continuous monitoring and control of several essential oper -
`Operational control of redundant , continuous process flow
`ating parameters , including fluidic temperatures , pressures ,
`systems is generally well known . Preferable , each of the
`conductivity and pH at several points in the primary and
`several redundant sub - facilities are routinely activated to
`secondary circulation loops . Although the several sensory 25 assure current functionality , and to allow the inactive sub
`and control functions can be implemented using traditional
`facility to be serviced according to an established schedule .
`dedicated hardware components , we prefer to employ at
`We believe this continuous rotation of system resources to
`least one programmable logic controller ( “ PLC ” ) , commer -
`be so important that we recommend switching the sub
`cially available from any of a number of respected vendors ,
`facilities at least once , and preferably , several times , per day ;
`e . g . , the Allen - Bradley brand of PLCs from Rockwell Auto - 30 although this is possible to implement manually , we prefer
`mation , Inc . In the instantiation illustrated in FIG . 13 , we
`to enable the master controller 62 to control the sequencing
`have depicted : a primary controller 58a adapted to monitor
`of the several switch - over operations . One further aspect of
`and control the operation of the primary circulation sub -
`this sophistication in control is the ability to perform stress
`facility 28a as a function of the temperature of the dielectric
`testing of the several sub - systems under controlled condi
`fluid in the tank 14 ; a secondary controller 60a adapted to 35 tions so as to assure appropriate response to real - time
`monitor and control the operation of the secondary fluid
`emergencies .
`circulation sub - facility 30a as a function of the temperature
`in our First Parent Provisional , we have disclosed an
`of the dielectric fluid flowing through the heat exchanger
`alternate embodiment comprising an appliance immersion
`32a ; and a master controller 62 adapted to coordinate the
`tank facility wherein the function of the plenum facility 36
`activities of the primary controller 58a and secondary con - 40 is performed by a manifold facility comprising a ladder
`troller 60a . As can be seen , we have incorporated into the
`arrangement of tubular spray bars , each bar of which sup
`primary circulation sub - facility 28a : supply and return sen -
`plies dielectric fluid to a respective appliance slot . As we
`sors , including a temperature probe , T , inserted into a
`noted , one particular advantage of this arrangement is that
`thermowell ( not shown ) installed in the bottom of the
`individual spray bars may be shut off if the respective
`reservoir 42 adjacent a respective return port 44a ( note that , 45 appliance slot is not occupied and , thus , save energy . To
`in FIG . 4 , only one of the holes that receive the thermowells
`further increase energy efficiency , we have provided
`is illustrated , but both holes are illustrated in FIG . 12 ) ; a pair
`optional vertical flow barriers adapted to partition the tank
`of sensor facilities , S , which may sense temperature , pres -
`into an active portion , having active appliances , and a
`sure and conductivity , as deemed desirable ) ; and return ( and ,
`stagnant portion , having no active appliances . One further
`if desired , supply ) flow control valves and controls for the 50 enhancement we disclosed is the provision of temperature
`primary circulation pump 48a ; of course , a redundant set of
`sensors per appliance slot , such that the flow rate through
`these components exists for the primary circulation sub -
`each spray bar can be dynamically varied as a function of the
`facility 28b . In general , the goal is to maintain the tempera
`temperature of the dielectric fluid exiting the respective slot .
`ture of the dielectric fluid in the tank 14 between a prede -
`Other operative configurations will be readily perceived by
`termined minimum temperature and a predetermined 55 those skilled in this art .
`In a manner analogous to the embodiment described in
`maximum temperature .
`As noted above , we have provided separate control equip -
`our First Parent Provisional , it would be advantageous , from
`an energy point of view , to provide a plurality of flow barrier
`ment cabinets 34a and 34b , each adapted to accommodate
`the several components comprising a respective one of the
`plates 66 ( shown by way of example only in FIG . 11 ) , each
`primary controllers 58a and 58b . For convenience of access , 60 adapted to be attached to the top of the plenum facility 36 so
`we prefer to co - locate with each of the cooling towers 50 a
`as substantially to block the flow of the dielectric fluid
`protective housing ( not shown ) for the respective secondary
`through the row ( s ) of orifices in the plenum plate 36a
`controller 60 . Of course , the control facility 56 can be
`corresponding to at least a respective one of the appliance
`instantiated as a single , multi - module PLC facility , with
`slots 18a ; an elastomeric layer ( not shown ) could be pro
`similar or other combinations of monitoring devices as 65 vided on the interface surface of the plate ( s ) 66 to enhance
`deemed most appropriate for a particular installation . Alter
`the sealing effect . Such an arrangement would allow the total
`natively , one or more , and perhaps all , of the functions
`flow through the plenum facility 36 to be adjusted , in the
`
`MGT000864
`
`Immersion Systems LLC – Ex. 1023
`GR 2021-00104 (U.S. 10,820,446 B2)
`12 of 15
`
`

`

`US 10 , 405 , 457 B2
`
`undesirable foreign matter that may have been picked up by
`field , as a function of the actual number of active appliances
`the dielectric fluid on its passage though the entire primary
`16 in the tank 14 . Further , this arrangement can incorporate
`circulation facility 28 ; chemical sensors may also be pro
`a relocatable vertical baffle plate 68 ( see FIG . 11 ) adapted
`vided to detect the presence of unexpected chemicals that
`substantially to partition the tank 14 into an active portion
`14a containing the active appliances 16 and an inactive 5 may indicate failure of sub - co