(12) United States Patent
`Rothman et al.
`
`(10) Patent No.:
`45) Date of Patent:
`
`US 7,398,382 B2
`Jul. 8, 2008
`
`9
`
`USOO7398382B2
`
`J. ww.
`
`OWC (a
`
`- - -
`
`7,032,031 B2 * 4/2006 Jungcket al. ............... TO9,246
`7,085,921 B2 * 8/2006 Frye, Jr. ......................... T13/1
`7,099,927 B2* 8/2006 Cudd et al. ....
`... 709,217
`2. . R g3. h
`t it. - - - - - - - - - - - - - - 36
`7,203,735 B1 * 4/2007 Le Pennec et al. .......... 709/219
`2003/0126242 A1* 7/2003 Chang ..............
`... TO9,222
`2003/0208675 A1* 11/2003 Burokas et al. ................ T13/1
`2004/0068576 A1* 4/2004 Lindbo et al. .....
`... 709,232
`2005/0144431 A1* 6/2005 Lin et al. ....................... T13/1
`2005/0278583 A1* 12/2005 Lennert et al. ................ T14? 43
`OTHER PUBLICATIONS
`Preboot Execution Environment (PXE) Specification, Intel Corpora
`tion, Sep. 20, 1999, ver, 2.1, pp. 1-101.
`* cited by examiner
`Primary Examiner Thomas Lee
`Assistant Examiner Jaweed A Abbaszadeh
`(74) Attorney, Agent, or Firm Blakely, Sokoloff, Taylor &
`Zafman LLP
`
`(57)
`
`ABSTRACT
`
`Methods and apparatus for improving network boot effi
`ciency are disclosed. Under embodiments of the method, boot
`images that are initially sent from a boot server to various
`9.
`y
`clients are cached at network devices along communication
`paths between the boot server and the clients. In response to
`Subsequent boot image requests from the clients, boot images
`cached at the network devices are downloaded directlv from
`y
`the network devices rather than from the boot server, reducing
`network traffic to and from the boot server and domains in
`between. In addition, boot program bootstrap files may also
`be cached and downloaded in a similar manner. Techniques
`are also disclosed for intercepting boot image download and
`network bootstrap program requests at the network devices,
`and for maintaining valid boot image cache configurations
`across the network. The network devices generally include
`Switches, routers, bridges, and gateway servers.
`
`23 Claims, 17 Drawing Sheets
`
`(54) METHOD AND APPARATUS TO ENHANCE
`PLATFORM BOOT EFFICIENCY
`
`(75) Inventors: Michael A. Rothman, Puyallup, WA
`(US); Vincent J. Zimmer, Federal Way,
`WA (US)
`
`7
`
`(*) Notice:
`
`(73) Assignee: Intel Corporation, Santa Clara, CA
`(US)
`-
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 313 days.
`(21) Appl. N 11AO26,420
`ppl. No.:
`9
`Dec. 29, 2004
`Prior Publication Data
`
`(22) Filed:
`(65)
`
`US 2006/0143432 A1
`
`Jun. 29, 2006
`
`(51) Int. Cl.
`(2006.01)
`G06F 9/00
`(2006.01)
`G06F 15/177
`(2006.01)
`G06F 5/16
`52) U.S. Cl. ........................... 713/2; 709/217; 709/219;
`709/222
`(58) Field of Classification Search ................. 709/203,
`709/222; 713/2
`See application file for complete search historv.
`ry
`pp
`p
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`5,280,627 A *
`1/1994 Flaherty et al. ................ 713/2
`5,452,454. A
`9, 1995 Basu ............................. 713/2
`6,138,162 A * 10/2000 Pistriotto et al. ............ 709,229
`6,389,462 B1* 5/2002 Cohen et al. ................ TO9.218
`6,549,934 B1 * 4/2003 Peterson et al. ............. TO9,203
`6,854,009 B1* 2/2005 Hughes ...................... TO9.220
`6,965,924 B1 * 1 1/2005 Gerthe ....................... TO9.218
`6,976,058 B1* 12/2005 Brown et al. ................ 709/217
`
`
`
`
`
`130 (TYP) -128 (Typ)
`
`
`
`RETREWE
`(O(Push-
`
`PUSH
`DIRECTIV
`
`32
`:
`iD, CLIENTA
`MAC: OD-FA-20-35-D-F
`P: 24.54.60.10
`
`A
`
`126
`t
`6KRETRIEVE.
`“.
`124.53
`: PXE
`
`A. 132
`CLIENT A; 6.IMG
`CLIENTB;3.IMG
`ENTC:4.
`CENTC: 4.MG
`
`108
`
`28
`
`Docker EX1007
`Page 1 of 28
`
`

`

`U.S. Patent
`
`Jul. 8, 2008
`
`Sheet 1 of 17
`
`US 7,398,382 B2
`
`
`
`s
`
`Docker EX1007
`Page 2 of 28
`
`

`

`U.S. Patent
`
`Jul. 8, 2008
`
`Sheet 2 of 17
`
`US 7,398,382 B2
`
`
`
`N
`d ld
`co
`C
`d
`w
`c
`c
`a.
`co
`t
`r
`to 2
`co
`r
`an
`dis
`s
`a.
`d
`e
`
`Docker EX1007
`Page 3 of 28
`
`

`

`U.S. Patent
`
`Jul. 8, 2008
`
`Sheet 3 of 17
`
`US 7,398,382 B2
`
`
`
`
`
`&
`
`: i
`
`
`
`47
`
`
`
`
`
`i
`
`
`
`
`
`Docker EX1007
`Page 4 of 28
`
`

`

`U.S. Patent
`
`Jul. 8, 2008
`
`Sheet 4 of 17
`
`US 7,398,382 B2
`
`
`
`:
`
`Docker EX1007
`Page 5 of 28
`
`

`

`U.S. Patent
`
`Jul. 8, 2008
`
`Sheet 5 Of 17
`
`US 7,398,382 B2
`
`&
`
`
`
`
`
`
`
`Docker EX1007
`Page 6 of 28
`
`

`

`U.S. Patent
`
`Jul. 8, 2008
`
`Sheet 6 of 17
`
`US 7,398,382 B2
`
`
`
`S.
`
`Docker EX1007
`Page 7 of 28
`
`

`

`U.S. Patent
`
`Jul. 8, 2008
`
`Sheet 7 Of 17
`
`US 7,398,382 B2
`
`1ST SYSTEM POWER ON/RESET
`
`200
`
`INITIALIZE CLIENT OBTAIN NETWORKADDRESS
`
`2O2
`
`OBTAINBOOT (OPERATING SYSTEM)
`IMAGE OVER NETWORK FROM BOOT SERVER
`
`204
`
`1st PHASE
`
`
`
`CACHE BOOT IMAGE AT NETWORKDEVICE (E.G.,
`SWITCH, ROUTER, BRIDGEETC.) OR GATEWAY
`SERVER PROXMATE TO CLIENT
`
`206
`
`BOOT OS IMAGE
`
`208
`
`CONTINUE NORMAL RUNTIME USE
`
`210
`
`ONGOING
`
`SUBSEQUENT SYSTEM POWER ON/RESET
`
`212
`
`INITIALIZE CLIENT OBTAIN NETWORKADDRESS
`
`214
`
`OBTAIN OS BOOT IMAGE FROM CACHING DEVICE
`(NETWORKDEVICE OR GATEWAY SERVER)
`
`216
`
`Fig. 2
`
`Docker EX1007
`Page 8 of 28
`
`

`

`U.S. Patent
`
`Jul. 8, 2008
`
`Sheet 8 of 17
`
`US 7,398,382 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`INTIAL
`FIRMWARE
`
`PXE DHCP Discover
`
`yPXECLIENT TAGs/
`
`PXECLIENT TAGS
`
`PXESERVER TAGS
`3O8
`Extended DHCP Bie/PXESERVERTAGs/
`31 O
`BOOT SERVER LIST A 306r- 124.54.60.10
`
`
`
`(3) DHCP Request "APXECLENTAGs
`
`PROXY
`DHCP
`SERVICE
`
`312
`Boot Service Discover PXE CLIENT TAGS
`
`318
`Boot Service Ack
`PXESERVERTAGS
`316 - NBP NAME
`316
`
`(7) PXE Download Request (TFTP)NBP NAME
`Network Bootstrap Program Download
`
`BOOT
`SERVICE
`
`
`
`SERVICE
`
`LOAD 8.
`
`NBP
`
`LOAD 8.
`EXECUTE
`BOOT
`IMAGE
`
`132
`
`Fig. 3a
`
`:
`
`Boot IMAGE IMTFTPOR
`SERVICE
`
`i
`... TRAPREQUEST-324
`i
`:
`........l.
`CACHE NBP - 326
`i iwi. TRAPREQUEST-334
`CACHE BOOT
`IMAGE
`
`- - - - - - - - - - - - - - - - - - - - - ----- Y - - - - -
`
`336
`
`
`
`Network Device
`Gateway Server
`
`114
`
`Docker EX1007
`Page 9 of 28
`
`

`

`U.S. Patent
`
`Jul. 8, 2008
`
`Sheet 9 Of 17
`
`US 7,398,382 B2
`
`NTIAL
`FIRMWARE
`
`302 -
`PXEDHCP Discover 7PXECENTAS
`3O8
`Extended DHCP offe/PXESERVERTAGS
`3101N/ BOOT SERVER LIST W306\ 124.54.60.10
`(3) DHCP Request 302\/PxE CENT TAGs
`
`
`
`
`
`
`
`T
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`LOAD &
`EXECUTE
`NBP
`
`LOAD 8.
`EXECUTE
`BOOT
`IMAGE
`
`312
`Boot service Discover/PE9EN TAGS
`
`318
`Boot Service Ack
`
`PXESERVERTAGS
`
`316 NNBP NAME
`O PXE Download Request (TFTP)
`
`316
`
`NBP NAME
`
`| DHCP/Proxy
`IDHCP Server
`
`
`
`BOOT
`| SERVICE
`
`Network Bootstrap Prodram Download
`p
`9
`322
`
`NBP FILE
`
`SERVICE
`320
`
`
`
`
`
`126
`340
`342^DEVICE IF ADDRESS & PORT
`(9) Boot image Download Request (TFTP, TCP/IP)
`Boot IMAGE MTOR
`TCP/IP
`FILE(S)
`SERVICE
`
`:
`
`PORT
`
`NER
`
`332
`
`132
`
`Fig. 3b
`
`114
`
`Docker EX1007
`Page 10 of 28
`
`

`

`U.S. Patent
`
`Jul. 8, 2008
`
`Sheet 10 of 17
`
`US 7,398,382 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`INTIAL
`FIRMWARE
`
`3O2
`PXE DHCP Discover
`
`PXE CLIENTAGS
`
`3O8
`Extended DHCP offe/PXESERVERTAGS
`310 N/ BOOT SERVER LIST A306\ 124.54.60.10
`(3) DHCP Request 302\/PXECLIENT TAGs
`(ODHCP Ack
`
`
`
`312
`Boot Service Discover PXE CLIENT TAGS
`
`
`
`318
`Boot Service Ack
`
`PXESERVER TAGS
`
`BOOT
`SERVICE
`
`316 runBP NAME
`
`316
`
`OPXE Download Request (TFTP)NBP NAME
`
`LOAD &
`EXECUTE
`NBP
`
`load
`D
`Network Bootstrap P
`program Luownloa
`322^J NBP FILE
`(9) Boot image Download Request (TFTP, TCP/IP
`
`MITFTP
`SERVICE
`
`126
`
`6.IMG-sWITCH 114
`
`
`
`332
`
`Boot IMAGE | MTOR
`TCP/IP
`FILE(S)
`SERVICE
`
`LOAD &
`EXECUTE
`BOOT
`IMAGE
`
`Client
`
`132
`
`Fig. 3c
`
`114
`
`Docker EX1007
`Page 11 of 28
`
`

`

`U.S. Patent
`
`Jul. 8, 2008
`
`Sheet 11 of 17
`
`US 7,398,382 B2
`
`INTIAL
`FIRMWARE
`
`3O2
`PXE DHCP Discover
`
`PXE CLIENT TAGS
`
`308
`Extended DHCP offe/PXESERVERTAGS
`310
`BOOT SERVER LIST 306
`124.54.60.10
`
`
`
`(3) DHCP Request "AxeculeNTAGs
`(ODHCP Ack
`
`312
`Boot service Discover/PE9ENTAS
`318
`Boot Service Ack
`
`PXESERVERTAGS
`
`
`
`
`
`DHCP/Proxy
`DHCP Server
`
`BOOT
`SERVICE
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`316 - NBP NAME
`O PXE Download Request (TFTP)
`316 - NBP NAME
`(8) NBP Download
`INTERCEPT
`EXECUTE E"g 322 NBPFLE
`
`LOAD 8.
`
`Boot image Download
`Request (TFTP, TCPIIP)
`
`INTERCEPT
`
`X
`
`"El
`LOGIC
`
`LOAD &
`EXECUTE
`BOOT
`IMAGE
`
`BOOT IMAGE
`FILE(S)
`
`332
`
`- || MTEPOR
`TCP/IP
`SERVICE
`
`*
`
`
`
`
`
`350
`
`IPADDR CLIENTA: 6. IMG
`PADDR CLENT B: 3.IMG
`PADDR CENTC: 4.MG
`
`
`
`344
`NetWork
`Device?
`Gateway
`Sever
`
`114
`
`132
`
`Fig. 3d
`
`Docker EX1007
`Page 12 of 28
`
`

`

`U.S. Patent
`
`Jul. 8, 2008
`
`Sheet 12 of 17
`
`US 7,398,382 B2
`
`RECEIVE PACKET
`
`400
`
`INITIATE PACKET PROCESSING; EXTRACT
`HEADER METADATA
`
`402
`
`IS
`FILTER INDICAMARKED
`NHEADER2
`
`404
`
`YES
`
`REDIRECT TO SLOWPATH PROCESSING
`
`406
`
`
`
`CACHE COPY OF PACKETS; FORWARD
`ORIGINAL PACKETS; REASSEMBLY COPIED U-408
`PACKETS TO FORMFILE OR MESSAGE;
`STORE FILE OR HANDLE MESSAGE
`
`NO
`
`UPDATE IMAGE MAP
`
`EMPLOY FAST PATH PROCESSING TO
`FORWARD PACKETS
`
`410
`
`412
`
`Fig. 4
`
`Docker EX1007
`Page 13 of 28
`
`

`

`U.S. Patent
`
`Jul. 8, 2008
`
`Sheet 13 of 17
`
`US 7,398,382 B2
`
`SETUP {
`
`STORE PXESERVER ADDRESS/PORT
`VALUES ONCACHING DEVICE
`
`5OO
`
`RECEIVE PACKET
`
`5O1
`
`INITIATE PACKET PROCESSING, EXTRACT
`HEADER METADATA
`
`502
`
`
`
`DOES
`DESTINATION ADDRESS/
`PORT MATCHPXE
`ADDRESS/PORT
`
`504
`
`YES
`
`REDIRECT TO SLOWPATH PROCESSING
`
`506
`
`ONGOING
`
`
`
`CACHE COPY OF PACKETS; FORWARD
`ORIGINAL PACKETS; REASSEMBLY COPIED
`PACKETS TO FORMFILE OR MESSAGE;
`STORE FILE OR HANDLE MESSAGE
`
`NO
`
`508
`
`UPDATE IMAGE MAP
`
`EMPLOY FAST PATH PROCESSING TO
`FORWARD PACKETS
`
`510
`
`512
`
`Fig. 5
`
`Docker EX1007
`Page 14 of 28
`
`

`

`U.S. Patent
`
`Jul. 8, 2008
`
`Sheet 14 of 17
`
`US 7,398,382 B2
`
`PXESERVER PREPARES TO SEND AFILE TO
`BECACHED
`
`6OO
`
`PXESERVER SENDSA DIRECTIVE OVER THE
`MANAGEMENT CHANNEL TO THE NETWORK 6O2
`DEVICE TO CACHE THE FILE, PROVIDING
`PACKETDENTIFICATION DATA
`
`DETECTPACKETS
`
`604
`
`REDIRECT TO SLOWPATH PROCESSING
`
`606
`
`
`
`CACHE COPY OF PACKETS;
`FORWARD ORIGINAL PACKETS;
`REASSEMBLY COPIED PACKETS TO FORM
`FILE AND STORE LOCALLY (e.g., DISKDRIVE
`OR LOCAL SERVER)
`
`608
`
`UPDATE IMAGE MAP
`
`610
`
`Fig. 6
`
`Docker EX1007
`Page 15 of 28
`
`

`

`U.S. Patent
`
`Jul. 8, 2008
`
`Sheet 15 Of 17
`
`US 7,398,382 B2
`
`SYSTEM POWER ON/RESET N 700
`
`RECEIVE/INTERCEPT BOOT IMAGE
`DOWNLOAD REQUEST
`
`
`
`
`
`
`
`IDENTIFY CACHED IMAGE
`READ CHECKSUMFILE
`
`
`
`702
`
`704
`
`
`
`
`
`
`
`SEND CHECKSUMVERIFICATION
`MESSAGE TO PXESERVER
`
`706
`
`
`
`
`
`
`
`NO
`
`DO
`CHECKSUMS
`MATCH2
`
`
`
`YES
`
`708
`
`DOWNLOAD BOOT IMAGE FROM
`CACHING DEVICE
`
`710
`
`DOWNLOAD UPDATED BOOT IMAGE
`FROMPXESERVER, CACHEAT
`NETWORKDEVICEIGATEWAY SERVER
`
`COMPLETEOS BOOT
`
`Fig. 7
`
`Docker EX1007
`Page 16 of 28
`
`

`

`U.S. Patent
`
`Jul. 8, 2008
`
`Sheet 16 of 17
`
`US 7,398,382 B2
`
`MANTAIN BOOT IMAGE VERIFICATION
`TABLE ONEACH CACHING DEVICE
`AND PXESERVER
`
`800
`
`
`
`
`
`
`
`BOOT IMAGE
`UPDATED?
`
`802
`
`YES
`
`UPDATE BOOT IMAGE
`VERIFICATION TABLE ON
`THE PXESERVER
`
`804
`
`DETERMINE WHICH CACHING DEVICE(S)
`SIARE CACHING OUTDATED VERSION
`OF BOOT IMAGE
`
`806
`
`DOWNLOAD BOOT IMAGE TO THE
`APPLICABLE CACHING DEVICE(S)
`
`808
`
`UPDATE BOOT IMAGE
`VERIFICATION TABLE ON THE
`APPLICABLE CACHING DEVICE(S)
`
`810
`
`
`
`
`
`Fig. 8
`
`Docker EX1007
`Page 17 of 28
`
`

`

`U.S. Patent
`
`Jul. 8, 2008
`
`Sheet 17 Of 17
`
`US 7,398,382 B2
`
`934
`
`H-
`g
`;
`99 3.
`
`
`
`MEDIA
`INTERFACE
`
`OO 902
`924 (TYP)
`
`SRAM STORE
`
`DRAM STORE
`
`NON-VOLATLE
`STORE
`
`91
`
`FAST PATH
`PROCESSING
`
`SLOWPATH PROCESSING
`FILE SYSTEM
`918
`
`HEADER
`FILTER
`
`CACHE MESSAGE
`LOGIC
`HANDLER
`
`LOCAL FILESTORE
`
`909
`REDIRECT
`
`920
`
`922
`910
`
`908
`
`
`
`-
`-S928
`NETWORKSERVICES
`
`930
`
`911
`
`900
`
`Fig. 9
`
`Docker EX1007
`Page 18 of 28
`
`

`

`US 7,398,382 B2
`
`1.
`METHOD AND APPARATUS TO ENHANCE
`PLATFORM BOOT EFFICIENCY
`
`FIELD OF THE INVENTION
`
`The field of invention relates generally to computer plat
`forms and networks and, more specifically but not exclusively
`relates to techniques for performing network booting in an
`efficient manner.
`
`BACKGROUND INFORMATION
`
`10
`
`2
`FIG. 1a schematically depicts the first phase of one
`embodiment of the invention, wherein a network bootstrap
`program file and a boot image are cached on a network device
`in conjunction with downloading the same from a boot server
`to a client;
`FIG.1b schematically depicts a second phase of the down
`load technique, wherein the cache boot image is downloaded
`directly from the network device:
`FIG. 1c schematically depicts one embodiment under
`which the boot server sends a download directive to the net
`work device to push a copy of the cached boot image to the
`client;
`FIG. 1d schematically depicts one embodiment under
`which request messages intended for the boot server are inter
`cepted and processed at the network device;
`FIG. 1e schematically depicts one embodiment of a boot
`image configuration control scheme, wherein a checksum
`validation test is performed on a cached boot image to deter
`mine whether the cache boot image is a valid boot image:
`FIG. 2 is a flowchart illustrating operations performed
`during the first and second phases of the boot image cache and
`download scheme of FIGS. 1a and 1b,
`FIG. 3a is a message flow diagram illustrating a message
`exchange sequence corresponding to the first phase opera
`tions of FIGS. 1a and 2:
`FIG. 3b is a message flow diagram illustrating a message
`exchange sequence corresponding to the second phase opera
`tions of FIGS. 1b and 2:
`FIG. 3C is a message flow diagram illustrating a message
`exchange sequence corresponding to the second phase opera
`tions of FIG. 1C:
`FIG. 3d is a message flow diagram illustrating a message
`exchange sequence corresponding to the second phase opera
`tions of FIG. 1d.
`FIG. 4 is a flowchart illustrating operations and logic per
`formed by one embodiment of a caching and message detec
`tion scheme that employs cache and message indicia in net
`work packets;
`FIG. 5 is a flowchart illustrating operations and logic per
`formed by one embodiment of a caching and message detec
`tion scheme that employs detects files and messages to be
`trapped and/or cached based on server address and port values
`in network packet headers;
`FIG. 6 is a flowchart illustrating caching operations per
`formed in accordance with one embodiment that employs a
`managed network device;
`FIG. 7 is a flowchart illustrating operations and logic per
`formed to validate a boot image in accordance with the tech
`nique depicted in FIG. 1e,
`FIG. 8 is a flowchart illustrating operations and logic per
`formed during one embodimentofa boot image configuration
`control scheme under which boot image verification tables
`are employed; and
`FIG. 9 is a schematic diagram of a network device archi
`tecture that may be employed to practice aspects of the
`embodiments described herein.
`
`15
`
`25
`
`30
`
`35
`
`A common problem faced by IT (information technology)
`managers is to ensure that client systems in their enterprises
`can boot appropriate Software images using appropriate con
`figuration parameters. Formerly, this process typically
`entailed performing an entire operating system installation
`for each computer, which was often necessary due to the
`differences in unrestricted platform configurations within an
`enterprise. This process was improved by controlling the
`platform configuration (e.g., by purchasing significant num
`bers of platforms having the same configurations), which
`enabled an operating system to be provisioned on a platform
`by simply copying an installation image corresponding to a
`given platform configuration to each platform in the enter
`prise having that configuration.
`More recently, there is gathering momentum for imple
`menting network booting of operating systems (OS) across
`all or portions of an enterprise internet. This is driven by
`several factors, including a reduced expense (both in capital
`and maintenance costs) for diskless workstations, enterprise
`software licensing considerations, configuration manage
`ment workload for the IT department, and security consider
`ations (viruses, hackers, etc.). Under a network boot, a
`selected boot image is loaded each time a given platform is
`started or reset. The provisioning of boot images is typically
`performed in conjunction with client authentication and Secu
`rity considerations. These selected boot images and configu
`ration parameters must be acquired from selected servers in
`the enterprise as dictated by the needs of the particular envi
`ronment, the capabilities or mission of the user, the resources
`available within the client, etc. Furthermore, these clients
`should boot consistently and in an interoperable manner
`regardless of the sources or vendors of the software and the
`hardware of both client and server machines.
`While network booting provides several advantages, par
`ticularly improved configuration control and reduced soft
`ware management requirements, it introduces new problems
`with respect to network traffic and security considerations.
`For example, a large enterprise internet may serve 1000's of
`50
`users. With modern boot images approaching upwards of a
`gigabyte or more, a significant load is placed on the enterprise
`internet and boot servers, particularly during timeframes in
`which system booting is heavy, such as during the start of a
`work day.
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`40
`
`45
`
`55
`
`The foregoing aspects and many of the attendant advan
`tages of this invention will become more readily appreciated
`as the same becomes better understood by reference to the
`following detailed description, when taken in conjunction
`with the accompanying drawings, wherein like reference
`numerals refer to like parts throughout the various views
`unless otherwise specified:
`FIG. 1 is a schematic diagram of an exemplary enterprise
`network used to described embodiments of the invention
`depicted in FIGS. 1a-e,
`
`DETAILED DESCRIPTION
`
`60
`
`65
`
`Embodiments of methods and apparatus for improving
`networkboot efficiency are described herein. In the following
`description, numerous specific details are set forth to provide
`a thorough understanding of embodiments of the invention.
`One skilled in the relevant art will recognize, however, that
`the invention can be practiced without one or more of the
`specific details, or with other methods, components, materi
`als, etc. In other instances, well-known structures, materials,
`
`Docker EX1007
`Page 19 of 28
`
`

`

`3
`or operations are not shown or described in detail to avoid
`obscuring aspects of the invention.
`Reference throughout this specification to “one embodi
`ment” or “an embodiment’ means that a particular feature,
`structure, or characteristic described in connection with the
`embodiment is included in at least one embodiment of the
`present invention. Thus, the appearances of the phrases "in
`one embodiment' or “in an embodiment” in various places
`throughout this specification are not necessarily all referring
`to the same embodiment. Furthermore, the particular fea
`tures, structures, or characteristics may be combined in any
`Suitable manner in one or more embodiments.
`In accordance with aspects of the embodiments now
`described, techniques are disclosed for performing network
`booting in a manner that reduces boot-related network traffic
`while meeting the aforementioned configuration and security
`criteria required by IT managers and the like. By way of
`example, various embodiments are described in connection
`with an exemplary enterprise network that is illustrative of
`various components common to typical enterprise networks.
`The techniques may also be employed for other network
`architectures using the general principles and teachings dis
`closed herein.
`FIG. 1 shows an exemplary enterprise network 100 includ
`ing five domains 102,104,106, 108, and 110. Domains 102,
`104 and 106 are interconnected via a switch 112, while
`domains 104, 106, and 108 are interconnected via a switch
`114. Domain 110, which comprises a remote domain, is con
`nected to domain 108 via a pair of gateway server 116 and
`117, which are connected via the Internet 118. Optionally, the
`gateway servers may be lined via a leased telecommunica
`tions link or the like, or may use a virtual private network
`(VPN) link over Internet 118 or other public and/or private
`network infrastructure. Domain 102 is hosted by a domain
`server 119, while domain 106 is hosted by a domain server
`120. Each of domains 104 and 110 are hosted by a respective
`Dynamic Host Control Protocol (DHCP) server 122 and 124.
`Domain 108 includes a Preboot Execution Environment
`(PXE) boot server 126. Domain 108 also includes a host
`server that may be the same machine used for PXE boot server
`126, or may be a separate server (not shown). In some
`embodiments, a PXE boot server and a DHCP server or
`DHCP proxy server may be co-located.
`Each of domains 102, 104, 106 and 110 are depicted as
`hosting multiple clients, including exemplary desktop clients
`128 and laptop clients 130. Other types of clients may also be
`hosted by a given domain, including workstations and various
`types of servers (both not shown) in addition to the various
`domain and DHCP servers shown in the figures herein.
`Domain 108 may also host similar types of clients (not shown
`for clarity) to those shown in domains 102,104,106 and 110.
`It will be understood that each domain may include 10's,
`100's or even 1000's of clients, only a fraction of which are
`depicted in the figures herein for simplicity and clarity. Some
`of the clients may be connected to their respective domain or
`DHCP server via a wired link (e.g., Ethernet), while others
`may be connected via a wireless link (e.g., 802.11a-g) facili
`tated by an appropriate wireless access point or router. For
`simplicity, wireless access points and routers are not depicted
`in the figures herein. At least a portion of the clients in each
`domain are configured to boot their respective operating sys
`tems using a network boot process.
`FIG. 2 shows a flowchart illustrating an overview of opera
`tions that are performed to Support enhanced network boot
`efficiency, according to one embodiment. The overall process
`begins with a first system power on event for a client depicted
`in a start block 200. In response, the client will perform a set
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 7,398,382 B2
`
`10
`
`15
`
`4
`of initialization operations via load and execution of its firm
`ware (e.g., BIOS), including initializing the client to Support
`network communications. In conjunction with the set of ini
`tialization operations, the client obtains a network address in
`a block 202.
`In a block 204, the client obtains an operating system boot
`image over the network from a boot server. In conjunction
`with delivering the boot image to the client, the boot image is
`cached at a network device. Such as a Switch, router, bridge,
`etc., or a gateway server that is proximate to the client in a
`block 206. The image is then booted in a block 208 to enable
`run-time use, which is continued in a continuation block 210.
`At continuation block 212, a Subsequent system power on
`or reset event occurs. For example, the user has a problemand
`decides to reboot the client, the user decides to selectively
`shutdown the client for the day and the subsequent power on
`eventrepresents restarting the client the following day, etc. As
`before, the client is initialized, enabling the client to obtain a
`network address in a block 214. This time, however, the client
`obtains the OS boot image from the caching device (e.g.,
`Switch, router, bridge, gateway server, etc.) rather than the
`boot server. The flowchart logic then returns to block 208 to
`boot the OS image, and the operations of blocks 210, 212,
`214, and 216 are repeated on an ongoing basis.
`Booting over a network raises several management and
`performance issues related to configuration, ease of use, boot
`latency, security issues, etc. Ideally, a uniform and consistent
`set of pre-boot protocol services should be employed to
`ensure that network-based booting is accomplished through
`industry standard protocols used to communicate with the
`boot server. In addition, to ensure interoperability, the down
`loaded Network Bootstrap Program (NBP) should be pre
`sented with a uniform and consistent pre-boot operating envi
`ronment within the booting client, so it can accomplish its
`task independent of for example, the type of network adapter
`implemented in the system. This capability is useful in
`enhancing the manageability of the client machine in several
`situations; for example:
`Remote new system setup. If the client does not have an OS
`installed on its hard disk, or the client has no hard disk at
`all, downloading an NBP from a server can help auto
`mate the OS installation and other configuration steps.
`Remote emergency boot. If the client machine fails to boot
`due to a hardware or software failure, downloading an
`executable image from a server can provide the client
`with a specific executable that enables remote problem
`notification and diagnosis.
`Remote network boot. In instances where the client
`machine has no local storage, it can download its system
`Software image from the server in the course of normal
`operation.
`One industry-standard technique for performing network
`booting that is employed by embodiments described herein is
`defined by the PXE protocol (current specification entitled
`Preboot Execution Environment Specification 2.1, Sep. 20.
`1999). PXE is defined on a foundation of industry-standard
`Internet protocols and services that are widely deployed in the
`industry, namely TCP/IP (Transmission Control Protocol/
`Internet Protocol), DHCP, and TFTP (Trivial File Transfer
`Protocol). These standardize the form of the interactions
`between clients and servers. To ensure that the meaning of the
`client-server interaction is standardized as well, certain ven
`dor option fields in the DHCP protocol are used, which are
`allowed by the DHCP standard. The operations of standard
`DHCP and/or BOOTP servers (that serve up IP addresses
`and/or NBPs) are not be disrupted by the use of the extended
`protocol. Clients and servers that are aware of these exten
`
`Docker EX1007
`Page 20 of 28
`
`

`

`5
`sions will recognize and use this information, and those that
`do not recognize the extensions will ignore them.
`In brief, the PXE protocol operates as follows. The client
`initiates the protocol by broadcasting a DHCPDISCOVER
`message containing an extension that identifies the request as
`coming from a client that implements the PXE protocol.
`Assuming that a DHCP server or a Proxy DHCP server imple
`menting this extended protocol is available, after several
`intermediate steps, the server sends the client a list of appro
`priate (PXE) Boot Servers. The client then discovers a Boot
`Server of the type selected and receives the name of an execut
`able file on the chosen Boot Server. The client uses TFTP to
`download the executable from the Boot Server. Finally, the
`client initiates execution of the downloaded image. At this
`point, the client’s state must meet certain requirements that
`provide a predictable execution environment for the image.
`Important aspects of this environment include the availability
`of certain areas of the client's main memory, and the avail
`ability of basic network I/O services.
`On the server end of the client-server interaction, services
`are made available that are responsible for providing redirec
`tion of the client to an appropriate Boot Server. These redi
`rection services may be deployed in two ways:
`1. Combined standard DHCP and redirection services. The
`DHCP servers that are supplying IP addresses to clients are
`modified to become, or are replaced by servers that serve
`up IP addresses for all clients and redirect PXE-enabled
`clients to Boot Servers as requested.
`2. Separate standard DHCP and redirection services. PXE
`redirection servers (Proxy DHCP servers) are added to the
`30
`existing network environment. They respond only to PXE
`enabled clients, and provide only redirection to Boot Serv
`CS.
`To enable the interoperability of clients and downloaded
`bootstrap programs, the client PXE code provides a set of
`35
`services for use by the BIOS or a downloaded NBP. The API
`(Application Program Interface) services provided by PXE
`for use by the BIOS or NBP are:
`Preboot Services API. Contains several global control and
`information functions.
`Trivial File Transport Protocol (TFTP) API. Enables open
`ing and closing of TFTP connections, and reading pack
`ets from and writing packets to a TFTP connection.
`User Datagram Protocol (UDP) API. Enables opening and
`closing UDP connections, and reading packets from and
`writing packets to a UDP connection.
`Universal Network Driver Interface (UNDI) API. Enables
`basic control of and I/O (input/output) through the cli
`ent's network interface device. This allows the use of
`50
`universal protocol drivers such that the same universal
`driver can be used on any network interface that imple
`ments this API.
`The PXE protocol is a combination of an extension of
`DHCP (through the use of several DHCP Option tags) and the
`55
`definition of simple packet transactions that use the DHCP
`packet format and options to pass additional information
`between the client and server. In the PXE protocol, DHCP
`options fields are used to do the following:
`Distinguish between DHCPDISCOVER and DHCPRE
`60
`QUEST packets sent by a client as part of this extended
`protocol from other packets that the DHCP server or
`Boot Server might receive:
`Distinguish between DHCPOFFER and DHCPACK pack
`ets sent by a DHCP or Proxy DHCP server as part of this
`extended protocol from other packets that the client may
`receive;
`
`45
`
`25
`
`40
`
`65
`
`US 7,398,382 B2
`
`5
`
`10
`
`15
`
`6
`Convey the client system's ID to the DHCP and Boot
`Server:
`Convey the client systems architecture type to the DHCP
`and Boot Server; and
`Convey the Boot Server type from which the client is
`requesting a response.
`Based on any or all of the client network adapter type,
`system architecture type, and client system ID, the Boot
`Server returns to the client the file name (on the server) of an
`appropriate executable. The client downloads the specified
`executable into memory and executes it.
`A message exchange sequence corresponding to the initial
`(first) phase PXE operations performed under one embodi
`ment of the initial phase of FIG. 2 is schematically depicted in
`FIGS. 1a and 3a. Many of these operations pertain to a con
`ventional PXE boot sequence, as is known in the art. How
`ever, additional operations are performed to provide caching
`of boot images.
`As shown by an initial firmware block 300, the process
`begins with the client (e.g., client 132) initializing its platform
`via load and execution of a corresponding set of firmware.
`During the platform initialization, the client is configured to
`support PXE-level network communications, which employ
`packet-based messages. There is no requirement to Support
`more complex transmission protocols, such as TCP/IP, at this
`point.
`Following client platform initialization, a PXE boot pro
`cess begins at step 1, with a client broadcasting a DHCPDIS
`COVER message to the standard DHCP port (67). An option
`field in the message contains the following information,
`which is collectively depicted as PXE client tags 302 in FIG.
`3a: 1) a tag for a client identifier (UUID universal unique
`identifier); 2) a tag for the client UNDI version; 3) a tag for the
`client system architecture; and 4) a DHCP option 60, Class
`ID, set to “PXEClient:Arch:XXXXX:UNDI:yyy ZZZ”.
`Under conventional DHCPusage, a DHCP server is used to
`allocate dynamic and static (i.e., reserved) DHCP addresses
`in response to address requests from various clients. In gen
`eral, a DHCP server may operate in conjunction with a
`domain server, and may be co-located with a domain server.
`A given DHCP server may serve a single domain, or may
`serve multiple domains that are related. For example, in the
`exemplary network of FIGS. 1 and 1a-e, DHCP server 122
`serves domains 102, 104, 106 and 108, while DHCP server
`124 servers domain 110.
`In the example shown in FIG. 1a, a client 132 broadcasts a
`DHCPDISCOVER request across its local domain 106. This
`DHCPDISCOVER message is then forwarded to the domain
`104 of DHCP server 122 in accordance with