US008750155B2
`
`(12) United States Patent
`Harvell et al.
`
`(10) Patent No.:
`
`(45) Date of Patent:
`
`US 8,750,155 B2
`*Jun. 10, 2014
`
`(54)
`
`(75)
`
`CONDITIONAL PROTOCOL CONTROL
`
`Inventors: Bradley B. Harvell, Chandler, AZ (US);
`Joseph D. DePalo, Peoria, AZ (US);
`Michael M. Gordon, Paradise Valley,
`AZ (US); Jason L. Wolfe, Gilbert, AZ
`(US)
`
`(73)
`
`Assignee: Limelight Networks, Inc., Tempe, AZ
`(US)
`
`(*)
`
`Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 169 days.
`
`(52) U.S. Cl.
`CPC .................................. .. H04L 47/193 (2013.01)
`USPC .......... .. 370/252; 370/389; 370/412; 370/466
`(58) Field of Classification Search
`CPC .... .. H04L 47/193; H04L 69/326; G06F 15/16
`See application file for complete search history.
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`2002/0099844 A1 *
`2007/0067424 A1 *
`2008/0031149 Al *
`2008/0225721 A1 *
`2010/0131671 A1 *
`
`........... .. 709/232
`7/2002 Baumann et al.
`3/2007 Raciborski et al.
`.
`709/223
`.... ..
`2/2008 Hughes et al.
`370/252
`
`9/2008 Plamondon
`370/235
`................. .. 709/233
`5/2010 Kohli et al.
`
`This patent is subject to a terminal dis-
`claimer.
`
`* cited by examiner
`
`(21)
`
`Appl. No.: 13/595,904
`
`(22)
`
`Filed:
`
`Aug. 27, 2012
`
`Primary Examiner — JayY Lee
`(74) Attorney, Agent, or Firm —Kilpatrick Townsend &
`Stockton LLP
`
`Prior Publication Data
`
`(57)
`
`ABSTRACT
`
`(65)
`
`(63)
`
`US 2013/0060893 A1
`
`Mar. 7, 2013
`
`Related U.S. Application Data
`
`Continuation of application No. 12/572,981, filed on
`Oct. 2, 2009, now Pat. No. 8,274,909, which is a
`continuation-in-part
`of
`application
`No.
`PCT/US2009/038361, filed on Mar. 26, 2009.
`
`Int. Cl.
`
`(51)
`
`G01R 31/08
`G06F 11/00
`G08C 15/00
`H0411/16
`H0413/14
`H04L 1/00
`H04L 12/26
`H04L 12/56
`
`(2006.01)
`(2006.01)
`(2006.01)
`(2006.01)
`(2006.01)
`(2006.01)
`(2006.01)
`(2006.01)
`
`Techniques for modifying the performance of a transport
`layer protocol in response to a request for content are dis-
`closed. A connection can be established between a content
`distribution server and an end user computer according to
`preconfigured parameters. When a request for content is
`received over the connection, the content distribution server
`can determine one or more parameters relating to the perfor-
`mance of the connection using information from the request.
`The content distribution server can modify the connection at
`the transport layer according to the one or more parameters.
`Thereafter, the transport layer can manage delivery of the
`requested content to the end user computer in accordance
`with the modified parameters. In various embodiments, the
`content distribution server includes a modified TCP protocol
`stack which adjusts timing, pacing, and buffer allocation
`associated with a connection in response to requests from an
`application-layer data source.
`
`20 Claims, 14 Drawing Sheets
`
`‘aO0
`
`M
`
`102-1
`
`‘:08-'1
`
`110
`
`Global Internet
`
`
`End User
`Systemcs)
`
`
` End User
`
`System(s}
` AKAMAI
`EXHIBIT 1001
`
`AKAMAI
`EXHIBIT 1001
`
`

`
`U.S. Patent
`
`Jun. 10, 2014
`
`Sheet 1 of 14
`
`US 8,750,155 B2
`
`\\\\\\\\\\\\\\\\~
`
`.m;Mum..\
`
`.EEEE
`Ema_.
`
`33:5
`
`.§m>:mQ
`
`Smwmmw
`
`
`
`
`

`
`U.S. Patent
`
`02
`
`02LI.6eh
`
`41
`
`US 8,750,155 B2
`
`9amMn.wwuMJu
`
`gxoooovnooouwwxwoomoo
`
`wM.
`
`\\\\\VV\V\R3SVr§¢x3«e$$§§$3§$$$§§d$$$§§§3s¥\\§9$$¥§§333$
`
`~9->~a:«ea~-x\ee-we»
`
`5.mwwm
`
`4..F
`
`f..............................
`
`mamasWMEmfiow..
`
`wzauowwmconcobamus-xauooamnewnos»mwmoomwwwoawuwwwaswxuwmwmmxvmnm‘
`
`xx8S$§§S.S$$§§$.3$$».\§§.¥$$§§¢x.sSovs.§R.x.$$$§.$.$$S$x>a..§$$
`
`wuwuwmwumwmmmco».ooamoouamaxwnmnnuncuncnnoonoo-:eoou»ooeeao~x\-x>woo-\x~et~m<vzvin\nz-w~aac:-rm-a¢.~.~¢
`
`
`
`

`
`U.S. Patent
`
`Jun. 10, 2014
`
`Sheet 3 of 14
`
`US 8,750,155 B2
`
`..\.\..k.\
`
`\\i:..m)..V.&%$$$§..x$$$$§.s$S$8$§.x$8S§£S$§§R8$$i§§S8$§S$
`...w».3:A,.3w.n\»
`
`
`
`«»w.wLw.9.is
`Mmfiumm
`
`........:
`..z..“.wm\&
`
`mm!
`
`>S\\3\\\.)\)\V\;w\\V)v\\\)\\\\i)3:\\IIE)IIII!)D\|\r))IIii§)\)\|\)\)lll\)lI\\\\\\\V\
`
`meficnu.
`
`Eumwmm
`
`mcumami
`
`mfimmmq
`
`%eeee»¢¢->aoeeoaoc¢onIeu~II~u
`
`an
`
`$8388§i3338..2ioooolaoiiioosaoiiEfiiiiaaaot-Q.I£§aaliS8m
`
`HmzummHEmmcao.
`
`\...
`R
`
`\
`
`wwmaamounmnwumuswwutotnrounenovuouaoreoauoo-occoanwbalii
`
`
`
`
`
`
`

`
`U.S. Patent
`
`Jun. 10, 2014
`
`Sheet 4 of 14
`
`US 8,750,155 B2
`
`'."'§'S
`(I)Q:
`
`§§§§§§§
`
`1 N‘
`C‘.._
`
`0%:
`=3
`
`-.
`
`..®§
`
`'fi9§Q~Pq‘¢flé«.'O9G§v‘D£‘€\Y~iP$c6
`
`w
`
`mwmwmwwmȤwwwwumumuauwuwuuummmummmmmmwmmnmmmmmmwwww
`
`§§§$8$8$§§$S$§§3$8§§§38$§%SS$$§§338$$§§3$$§
`
`N-‘
`,-...
`
`*3§§§§S$S$§§SSSS§§$S$S§$S$$§R3$3S$§SSS$SS$9
`
`DU'I)oa!Ii)I*oBvVII‘§b\\PlB\hih\Vé\'UIJI¢\‘BINV#J96WW€O9N(O9»(fi&%#&%«&fi¢6C¢bVBfOOVBEQb§fle§VBP&%P-'2A%PIk~)&‘IRQ‘AAX‘&?~“‘9aLV‘0a¢~:)lWt&fl®
`
`
`

`
`U.S. Patent
`
`Jun. 10, 2014
`
`Sheet 5 of 14
`
`US 8,750,155 B2
`
`3.
`
`w
`
`..5wmfi§..umMMmmowohmWN...........................:w.,WMM5N
`w.....w._.N
`wowoawmmM.
`
`8S¥3§§833§§333¥§§99$¥§§9388§§t383§E$$$$§33w
`
`w..M....................................5N
`
`mwWM§§$$$§§$S$$§RS$8¥§§38$§§§§$$5§§38$§§$9$¥§§
`
`w
`
`»
`
`maaWmE$$§»5S.35:3.w:§§:zzz:z3.sz:zz.i..£.:i.:zi:§::.zww5Ww\w.mwuwa5M
`
`
`
`

`
`U.S. Patent
`
`Jun. 10, 2014
`
`Sheet 6 of 14
`
`US 8,750,155 B2
`
`"3 4”} .-\‘
`‘,0-' \?5‘\,. *3‘
`
`.~
`\¢-
`3'5‘ _.~‘
`
`
`
`
`
`{}~s:te:*m'ez*:e may
`£3~.'{?L$‘§§">i3‘?€-ES for Liiiii
`
`ii §'\’e:—:{3L:sa<s§e::§
`fmm $3:-3.rva=3r
`
` Eféeiéver Czsniem f.r0s‘r2
`
`$a:‘v::=.es*
`
`<:33i«.=:s'2t
`
`\
`
`

`
`U.S. Patent
`
`Jun. 10, 2014
`
`Sheet 7 of 14
`
`US 8,750,155 B2
`
`- --.~ z.e
`
`.-
`
`1.‘.
`
`§,iR§ Receives‘
`§’.°'t:m N 96%
`
`<
`
`i.§§"~?i§
`
`fer
`§r:fa‘:»r'm2a;:‘,§::>n
`
`_
`
`_
`
`................................................................. ..
`
`%23{:~:”:m:t«.<;€itm E£:<-at:%£:<m=.a<;§
`
`6.
`E
`
`,~<“\=:‘-(“.:{}§*a:§§§””;g; in fimy .-*' tis‘éb:s§.e3$
`
`.1.-§Ii=éZ
`
` {3s?mix<3~n'{ ~‘;}b_§ec§
`
`€3€3rme3c:’.§<:m
`
`"€?‘=:‘<.2Lsg%‘:
`
`

`
`U.S. Patent
`
`Jun. 10, 2014
`
`Sheet 8 of 14
`
`US 8,750,155 B2
`
`
`
`
`
`
`
`-?=‘=<=:~.-«%-
`
`/Wflflvfl’/fiartnhas
`
`
`
`
`
`?,,,,,..y/’,.r:§»‘/
`
`

`
`U.S. Patent
`
`1:1
`
`US 8,750,155 B2
`
`3W1W.CMn2muU
`
`4Au1................................................................................:E
`
`km...Ww.
`
`.3
`
`‘cl
`
`4:.
`
`eW6"mahanSG
`02H9x3WtmayaC
`
`Fifi. §
`
`w¥\\\\k\\\\\\\\\\\§3§\A§\k\\k\\\\§S§A\k\\§\k§\\v$«\Aa3«§ka\.\\L\k\§\\\\\§§\§§%$\\\\\§\$§§%§ \
`
`§\\\\\\\§§\\8o3o$S.\\\\\\\\\§\\\\\\§\\§8.\\\§$\ \§\\\\\§.\\§\\§§\\\\\\\§\\\33838989......
`
`.x1..
`
`
`AWM...».35.
`
`mm...».\\\x..R\\\\\\.
`..oHHS.é«\\\N
`
`...¢\\\\\\\\\\\\\\\\\.§\\\\N\\\\\\.Q\.a\.M\§\§\\\\\\\\\\\\\&&§e§J.~.H
`
`\\\\\\..
`
`
`
`
`

`
`U.S. Patent
`
`Jun. 10, 2014
`
`Sheet 10 of 14
`
`U
`
`2B5
`
`la_R“0nn.
`
`
`
`
`
`&l«Ru»...\\\\\\\\\\\sA\\§§.\\\\\\§§\.\\\§.§\ §§\§..\\\\\§\ \ .“SM.H
`
`
`
`
`
`
`
`m\
`
`:3:
`
`...\\we
`1:
`
`Nu
`
`M
`
`
`
`t»t\»wm\t\txt.\
`
`MWW
`
`.x...
`
`es...
`
`if-‘$8. 2?’
`
`
`
`
`
`
`

`
`U.S. Patent
`
`Jun. 10, 2014
`
`Sheet 11 of 14
`
`US 8,750,155 B2
`
`
`
`..‘N....‘.V....“......\.\..M...M~....m§.....~.....~.~....~.....
`
` ..v.m.“um..\....w.......\u...~......»m...
`
`w....“\.....w......w..o.w.u.v.w..-nu........
`
` mmynn-.6.~.>mr
`
`3
`
`
`
`\\\\AA\\-\\KA$ §
`
`%E,1
`
`
`.«tramT‘mm1.nus11.\...u...n......r.e.....w.....“....v.“w.~w.».m.».-w...
`
`ii25
`
`
`
`.»
`'C.§(3i3§'3ti";£
`
`
`
`
`
`i
`Tes um.
`Y”
`E 1Ef3E
`
`i
`~:1.%1s<:a-\;;<>, 31..
`
`
`
` % §
`
`

`
`U.S. Patent
`
`Jun. 10, 2014
`
`Sheet 12 of 14
`
`US 8,750,155 B2
`
`»...~.....,...v.~......\...V.w‘..\...~..“
`
`.. *‘3‘\:".
`S» <.\.-\.-
`k
`J 3
`3
`:
`\
`:
`
`"
`
`
`
`.~.....m.......m..\.......................................
`¥
`§
`V\\
`.
`E
`S
`V“
`z
`~
`\
`‘
`ta
`1
`:
`,L.»
`\
`
`..............~...‘.................,....,..§............~§\
`§
`~
`1
`1
`.
`2
`§_
`‘
`S
`S
`:
`\
`:
`t
`\
`
`-
`
`
`
`
`
`.....................w
`3
`'
`
`:.
`:
`§
`~
`S
`z
`\
`\
`vu-n\\\\-v-H-\\\u\uunm\\u>uru\w--vu\\vrs\\~\\vvvq~vvu-s
`.
`.
`. s; .-
`\
`‘ tithe
`5
`2
`m1.x.x»..“..“..«..-.\-.-....~....».».».».»-w...~»».~».~.z
`§
`~
`.
`1
`..
`:
`:'“~
`\
`::s3:<.:§- me
`3
`......................\...............................2
`
`
`
`\
`\
`
`
`
`.............................
`
`.1.
`
`.....:......
`
`‘.......,..................m\..\.......\..............\.....
`3
`
`~~
`z
`3
`;,.................~.......“..............m.....‘.....
`a
`.- _
`=
`iv":
`:
`1
`.
`~
`$w.«..~«»....,...w.«u..\~.“.~“«.«...«.~.~«~....
`~
`1
`»
`...
`V
`93> ,-
`3
`.
`...
`§.,..,.,.,..,.,.,.,.,.,.,...,.,,.,.,.,._..,.,..,..
`~
`‘
`.
`3
`
`i..~.............‘~.w.w..‘.~..‘.»..»....w.\.“.
`
`\~
`
`

`
`U.S. Patent
`
`Jun. 10, 2014
`
`Sheet 13 of 14
`
`US 8,750,155 B2
`
`«U«V..x.C.\.muww.§‘
`
`
`
`\\\\\\\\\\\\)\\\\\\\\\\\\\\\)\\\\\\\\\\\.\\V\\\\\\\\\\\\\\\\\\\\\\\\\\
`
`\3t>i‘
`wumnnunnn
`"
`E:..m...m........“,
`
`.N«N«
`
`.W.W
`.N
`
`l§.§.§\x§§.53s33z..<..>zSEE>§i».x.€
`
`am
`
`w..
`
`w
`
`iv
`3:‘. {:1
`
`;.\»RW;,mMmR._Q.m.241m8.mann».wuasmSA9m.mMmmmmmnuHugAc“.39mmPTW;n_v\(mu:rwh.
`
`Fifi, 3%
`
`w
`w.
`
`A).
`
`:53.»
`
`w<\}§.?..§3t::§.
`
`a
`«V;
`WWWW
`\
`n...
`
`mMwMx.
`
`Nm
`w
`
`\\a\\\a\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\K\I\\\K\\\\\\\\\\V\\\(\\\\\\\\\V\\\\\\\\\
`
`
`
`
`
`
`
`

`
`U.S. Patent
`
`Jun. 10, 2014
`
`Sheet 14 of 14
`
`US 8,750,155 B2
`
`_sW9M..
`
`_W
`
`3
`
`-2' 1%‘
`
`~
`
`x.
`(.539
`g‘
`4.2
`E‘:":.*‘
`.......1.....w......m..m.
`‘\ TC; ,
`
`mxmfij
`
`.x..
`
`N
`
` 4...NA:4mu.111WM\1.:WWW,151MWWWWWMMW...x.\\\\L..m
`
`
`...\\...u.m4$§s§=§§..§
`:§§.§.§§§,.as§§...s§§...§NMMWMWu\\\\\)?\\\\\
`
`xuxkt33:.lit»mM_5Wm.
`
`..x.
`
`.9.T..3»HL.nwRu.
`8.8tE.Y33.m.xW.
`
`V,ewBU%5%SWM463ym14E.Ho.9.1»EL.4cmmm..F.:2.w
`
`
`
`N..>...K
`
`F33. ‘H
`
`
`
`
`
`
`
`
`

`
`US 8,750,155 B2
`
`1
`CONDITIONAL PROTOCOL CONTROL
`
`CROSS-REFERENCES TO RELATED
`APPLICATIONS
`
`This application is a continuation of U.S. patent applica-
`tion Ser. No. 12/572,981, filed Oct. 2, 2009 which is a con-
`tinuation-in-part claiming priority benefit under 35 U.S.C.
`§365(c) of International Patent Application Serial No. PCT/
`US2009/038361, filed Mar. 26, 2009. Each of the above-
`listed applications is hereby incorporated by reference in its
`entirety for all purposes.
`
`BACKGROUND
`
`This disclosure relates in general to interoperating nodes in
`an information processing system, such as an Internet content
`delivery system or an Internet transaction acceleration sys-
`tem, and, but not by way of limitation, to control of connec-
`tion protocols.
`In an information processing system, including communi-
`cations networks such as the Internet, two or more nodes can
`work together, for example exchanging information or shar-
`ing resources, using one or more protocols that enable the
`participating nodes to interoperate. Nodes need not be physi-
`cally distinct from one another, though they may be; nor-
`mally, however, nodes are at least logically distinct from one
`another in at least some respect. Interoperating nodes may be
`operated or managed by a single common authority or by
`independent, unrelated authorities. Two or more interoperat-
`ing nodes are often independently operated or managed; the
`Internet includes many well known examples of the interop-
`eration of two or more independently managed nodes.
`A protocol can be standardized such that a node using the
`standard protocol should be able to interoperate, at least at the
`level of the protocol, with any other node using the standard
`protocol. Standard protocols that become widely adopted can
`permit a node to interoperate with many other nodes. One
`such widely adopted standard protocol on the Internet is the
`Transmission Control Protocol (TCP), which today enables
`almost every device on the Internet to interoperate with
`almost every other device. TCP operates at the connection
`layer and enables nodes to interoperate with other nodes by
`establishing communications connections.
`Standard protocols often employ the use of attributes, such
`as configurable parameters and selectable algorithms, to per-
`mit the protocol to operate effectively in various situations.
`For example, TCP controls message size, the rate at which
`messages are exchanged, and factors related to network con-
`gestion through the use ofattributes, including both by the use
`of parameters, such as the receive window field used in slid-
`ing window flow control and the retransmission timer, and by
`the use of algorithms, such as slow-start, congestion avoid-
`ance, fast retransmit, and fast recovery algorithms. It is often
`the case, in many standard protocols, that at each node the
`initial protocol attribute settings to be used for all the com-
`munication connections at the node can be independently
`specified by the operator of the node.
`A protocol can also be customized, which in general
`requires that each node have installed customized compo-
`nents to enable the custom protocol. Without the customized
`components, the node would not be able to fully interoperate
`with other nodes using the customized protocol. Although it
`therefore may limit the total number of interoperable nodes,
`or in the alternative require widespread action to install the
`protocol customized components, or possibly both, protocol
`customization is used in order to add function, improve per-
`
`2
`
`formance, increase flexibility, or modify other characteristics
`of a standard protocol, or to make available an entirely new
`customized protocol. Many customized protocols have been
`proposed for use on the Internet.
`
`SUMMARY
`
`Techniques for modifying the performance of a transport
`layer protocol in response to a request for content are dis-
`closed. A connection can be established between a content
`
`10
`
`distribution server and an end user computer according to
`preconfigured parameters. When a request for content is
`received over the connection, the content distribution server
`can determine one or more parameters relating to the perfor-
`mance of the connection using information from the request.
`The content distribution server can modify the connection at
`the transport layer according to the one or more parameters.
`Thereafter, the transport layer can manage delivery of the
`requested content to the end user computer in accordance
`with the modified parameters. In various embodiments, the
`content distribution server includes a modified TCP protocol
`stack which adjusts timing, pacing, and buffer allocation
`associated with a connection in response to requests from an
`application-layer data source.
`In one embodiment, a method for managing delivery of
`content in a system comprising a server and an end user
`computer is disclosed. The method includes establishing a
`first connection at the server for communicating with the end
`user computer and receiving a request for content from the
`end user computer over the first connection. The method also
`includes determining one or more parameters relating to the
`performance of the first connection using information from
`the request and modifying the first connection at the transport
`layer based on the one or more parameters. Modifying the
`first connection can be done without notifying the end user
`computer. The method also includes sending the requested
`content from the server to the end user computer such that the
`transport layer manages delivery of the content in accordance
`with the modified parameters.
`Optionally, the method includes retrieving metadata asso-
`ciated with a requested file and modifying the first connection
`based on the metadata. Alternatively or additionally, the
`method can include selecting a predetermined performance
`profile for the first connection using the information from the
`request and modifying the first connection based on the pre-
`determined performance profile. The method can include
`determining a connection type of the end user computer and
`a latency characteristic associated with the connection type
`and modifying the first connection at the transport layer based
`on the latency characteristic. The method can also include
`determining a data size of the requested content, measuring a
`round trip time between the server and the end user computer
`when the data size exceeds a predetermined value, and modi-
`fying the first connection at the transport layer based on the
`size of the requested content and the round trip time.
`In another embodiment, a content distribution server is
`disclosed. The server includes a network interface, a proces-
`sor, a protocol handler, and a data source. The network inter-
`face includes a plurality of ports for sending and receiving
`data over a connecting network. The processor is coupled to
`the network interface and manages a plurality of connections
`to end user computers. The protocol handler establishes the
`connections with the end user computers according to pre-
`configured transport layer parameters of the content distribu-
`tion server and manages the manner in which data is trans-
`mitted over the connections. The data source supplies the
`requested content. The data source monitors a first connection
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`

`
`US 8,750,155 B2
`
`3
`for a request, determines one or more modified transport layer
`parameters based on the request, and directs the protocol
`handler to modify the first connection independently of the
`other connections based on the one or more transport layer
`parameters.
`In still another embodiment, a content distribution server is
`disclosed. The server includes means for sending and receiv-
`ing data over a connecting network, means for managing a
`plurality of connections to end user computers, and means for
`establishing a connection with each end user computer
`according to preconfigured transport layer parameters. The
`server includes means for managing data transmission over
`the plurality of connections, means for modifying a connec-
`tion based on one or more transport layer performance param-
`eters, and means for supplying requested content to the end
`user computers over the plurality of connections. The server
`also includes means for monitoring a first connection for a
`content request, means for determining the one or more trans-
`port layer performance parameters for the first connection
`based on the request, and means for sending the requested
`content over the first connection modified by the one or more
`transport layer performance parameters.
`In yet another embodiment, a computer program product
`comprising a computer-readable medium is disclosed. The
`computer-readable medium is encoded with one or more
`sequences of one or more instructions which, when executed
`by a processor, perform steps of establishing a first connec-
`tion at the server for communicating with an end user com-
`puter and receiving a request for content from the end user
`computer over the first connection. The instructions operate
`to determine one or more parameters relating to the perfor-
`mance of the first connection based on information from the
`
`request and to modify the first connection at the transport
`layer using the one or more parameters without notifying the
`end user computer. Additionally, the instructions operate to
`send the requested content from the server to the end user
`computer such that the transport layer manages delivery of
`the content in accordance with the modified parameters.
`Further areas of applicability of the present disclosure will
`become apparent from the detailed description provided here-
`inafter. It should be understood that the detailed description
`and specific examples, while indicating various embodi-
`ments, are intended for purposes of illustration only and are
`not intended to necessarily limit the scope of the disclosure.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 depicts a block diagram of an embodiment of a
`content delivery system.
`FIGS. 2A, 2B, 2C, and 2D depict block diagrams of
`embodiments of a content download pair that sends content
`from a server to a client.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`FIG. 3 illustrates a flowchart ofan embodiment ofa process
`for modification of the TCP protocol for various connections
`to a server.
`
`55
`
`FIG. 4 illustrates a flowchart ofan embodiment ofa process
`for modifying protocol attributes potentially on a connection-
`by-connection basis.
`FIG. 5 shows aspects of a content delivery system.
`FIG. 6 is a block diagram of an embodiment of a content
`distribution server.
`
`FIG. 7 shows an exemplary content distribution server
`protocol stack.
`FIGS. 8A, 8B, and 8C show exemplary data elements such
`as can be used with a content distribution server.
`
`FIG. 9 shows exemplary performance profiles such as can
`be used with a content distribution server.
`
`60
`
`65
`
`4
`
`FIG. 10 is a flowchart of a process for modifying transport
`layer protocol attributes.
`FIG. 11 is a flowchart of process for modifying transport
`layer protocol attributes.
`In the figures, similar components and/or features may
`have the same reference label. Further, various components of
`the same type may be distinguished by following the refer-
`ence label by a dash and a second label that distinguishes
`among the similar components. If only the first reference
`label is used in the specification, the description is applicable
`to any one of the similar components having the same first
`reference label irrespective of the second reference label.
`
`DETAILED DESCRIPTION OF EMBODIMENTS
`
`The ensuing description provides preferred exemplary
`embodiment(s) only, and is not intended to limit the scope,
`applicability or configuration of the disclosure. Rather,
`the ensuing description of the preferred exemplary embodi-
`ment(s) will provide those skilled in the art with an enabling
`description for implementing a preferred exemplary embodi-
`ment. It being understood that various changes may be made
`in the function and arrangement of elements without depart-
`ing from the spirit and scope as set forth in the appended
`claims.
`
`Referring first to FIG. 1, a block diagram ofan embodiment
`of an Internet content delivery system 100 is shown. Gener-
`ally, one or more nodes request content from one or more
`other nodes. In FIG. 1, a number ofend users 108 respectively
`use their end user system or client 102 to download and view
`content objects from the global Internet 104. The content
`delivery system 110 has one or more servers that provide
`content object downloads. The content delivery system 110
`can include any number of cache servers, application servers,
`content servers, service servers, and/or database servers to
`provide content to the clients 102. Although this embodiment
`shows particular communication pairs, other embodiments
`could communicate between any pair of nodes on a network,
`including betweenpairs of clients or between pairs of servers,
`and yet other embodiments could communicate among more
`than two nodes, such as in a broadcast or multicast implemen-
`tation.
`With reference to FIGS. 2A, 2B, 2C and 2D, embodiments
`of a content download pair 200 that sends content from a
`server 206 to a client 102 are shown. A primary embodiment
`described here is the interoperation of two nodes 102, 206 on
`the Internet communicating using TCP, one node being a
`client 102 that requests information, such as web page con-
`tent, multimedia, or software downloads, and the second node
`being a server 206 that provides information in response to a
`request. TCP operates in the transport layer ofthe seven-layer
`Open Systems
`Interconnection (OSI) model.
`In other
`embodiments, nodes 1 02, 206 interoperate in ways other than
`communication in a network, such as sharing data within a
`computer or group of computers across an available system or
`intersystem interface;
`interoperate using communications
`networks other than the Internet 104, such as a private com-
`munications network; interoperate using the Internet 104 or a
`private network using protocols other than TCP, such as UDP,
`RTP, multicast protocols, and other standard protocols in the
`transport layer; interoperate using the Internet 1 04 or a private
`network using standard protocols operating in a layer that
`underlies the transport layer;
`interoperate, using standard
`protocols and the Internet or a private network, more than two
`at a time, such as in clusters or multicast groups; or interop-
`erate, using standard protocols and the Internet or a private
`network, other than as a client and server, including interop-
`
`

`
`US 8,750,155 B2
`
`5
`erating as peers, as collaborative nodes, or as a group ofnodes
`under the common control of one or more other nodes or
`under the common control of a controller.
`
`In the primary embodiment, the server 206 conditionally
`adapts the attributes of the TCP protocol for each TCP con-
`nection established by a client 102. Conditionally adapting
`the attributes ofthe TCP protocol does not require changes to
`standard TCP protocol implementations at every node, does
`not require special components be installed in the TCP pro-
`tocol implementation at every node, and therefore does not
`comprise implementing a customized protocol as previously
`described; rather, the primary embodiment utilizes the stan-
`dard TCP protocol and the attributes implemented in it. In
`other embodiments,
`the server conditionally adapts the
`attributes of other transport-layer protocols for each session
`established by a client 102; the server conditionally adapts the
`attributes of other protocols that underlie the transport layer
`for each session established by a client 102; the server 206
`conditionally adapts the attributes of the TCP protocol or
`other protocol for groups of connections or sessions estab-
`lished by clients 102; the server 206 conditionally adapts the
`attributes of the TCP protocol or other protocol for connec-
`tions or sessions established by groups or subsets of groups of
`clients 102; the client 102 conditionally adapts the attributes
`ofthe TCP protocol for each TCP connection established; the
`client 102 conditionally adapts the attributes of other proto-
`cols for each session established; the client 102 conditionally
`adapts the attributes of the TCP protocol or other protocol for
`groups of connections or sessions; a node conditionally
`adapts the attributes of the TCP protocol or other protocol for
`each connection or session; a node conditionally adapts the
`attributes of the TCP protocol or other protocol for groups of
`connections or sessions; a node conditionally adapts the
`attributes of the TCP protocol or other protocol for connec-
`tions or sessions established by groups or subsets of groups of
`nodes; a controller conditionally adapts the attributes of the
`TCP protocol or other protocol for each connection or session
`of at least one node of an interoperating group of nodes; a
`controller conditionally adapts the attributes of the TCP pro-
`tocol or other protocol for groups of connections or sessions
`of at least some nodes of an interoperating group ofnodes; or,
`a controller conditionally adapts the attributes of the TCP
`protocol or other protocol for connections or sessions estab-
`lished by groups or subsets of groups of nodes.
`Software, software modifications, or equivalent function,
`may optionally be implemented at a server, client, or node that
`sets the conditionally adapted protocol attributes of a connec-
`tion or session, but need not be implemented at servers, cli-
`ents, or nodes that passively participate in a conditionally
`adapted protocol connection or session. Such software, soft-
`ware modifications, or equivalent function will only be
`needed if existing protocol software or other software on the
`server, client, or node does not provide a facility for program-
`matically or similarly changing attributes of the protocol that
`is used; in this event, software, a software modification, or
`equivalent facilities to provide such a programmatic or simi-
`lar interface may be implemented.
`Conditionally adapting the protocol for each connection or
`session, or collection of connections or sessions, results in at
`least one node that, concurrently or over time, uses a protocol
`for multiple unrelated connections or sessions wherein the
`protocol attributes vary, at least initially and sometimes per-
`sistently, from one connection or session to another, most
`often varying differently from any ordinary protocol attribute
`variations that naturally occur from one connection or session
`to another through use of the standard protocol implementa-
`tion among heterogeneous nodes.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`In the primary embodiment, TCP connections are estab-
`lished in order to use HyperText Transfer Protocol (HTTP) to
`communicate information requests from clients 102 to serv-
`ers 206 and responses from servers 206 to clients 102. HTTP
`is a scheme that operates above, and depends on the presence
`of a functioning and reliable protocol at, the transport layer of
`the seven-layer model developed in the Open Systems Inter-
`connection (OSI) initiative. Other embodiments use applica-
`tion-layer protocols other than HTTP in conjunction with
`TCP; use TCP alone, i.e., without HTTP; use other protocols;
`or, use other application-layer protocols in conjunction with
`other protocols. HTTP utilizes Uniform Resource Locators
`(URLs), Uniform Resource Names (URNs), and Uniform
`Resource Identifiers (URIs) to identify information. URLs
`are used in the primary embodiment. Other embodiments use
`URIs, URNs, other identifiers, or other information. A URL
`begins with the scheme identifier, which identifies the
`narnespace, purpose, and syntax ofthe remainder ofthe URL.
`In the primary embodiment utilizing HTTP,
`the typical
`scheme is “http”. The scheme is followed by a host field,
`which contains the IP address or name of the host where the
`
`requested information can be found, optionally followed by a
`port number, optionally followed by a path, which is an HTTP
`selector, optionally followed by a search portion, which is a
`query string. The full URL, then, is an alphanumeric string
`containing the scheme, host field, any optional following
`strings, and special characters such as “:”, “/”, and “?” that are
`reserved for special functions such as designating a hierar-
`chical structure in the URL. Other embodiments could use
`
`different application-layer protocols such as Telnet, File
`Transfer Protocol (FTP), secure HTTP (HTTPS), and Simple
`Mail Transfer Protocol (SMTP).
`In the primary embodiment, the server 206 bases the con-
`ditional adaptation ofthe attributes ofthe TCP protocol on the
`alphanumeric URL string provided by the client 102 in its
`information request. In another embodiment, a server, client
`or other node bases the conditional adaptation ofthe attributes
`ofthe TCP protocol or other protocol on the application-layer
`protocol specified or on identifying information, equivalent
`to a URL, or other information provided in, or characteristic
`of, an information request, connection, or session. In other
`embodiments, a server, client or other node bases the condi-
`tional adaptation ofthe attributes ofthe TCP protocol or other
`protocol on the IP address of one or more servers, clients, or
`nodes; on network information associated with the IP address
`of one or more servers, clients, or nodes,
`including the
`Autonomous System (AS) number,
`identity of network
`operator, geographic location, logical or physical network
`location,
`logical
`or
`physical
`network
`segment,
`or
`network interconnection characteristics associated with the
`
`IP address(es) of one or more servers, clients, or nodes; the
`geographic location of the server, client or node; and/or, the
`logical or physical network location of the server, client or
`node; the logical or physical address of the server, client or
`node; the logical or physical name of the server, client or
`node; and/or, the network or other path from or to a server,
`client or node. In other embodiments a server, client or node
`bases the conditional adaptation of the attributes of the TCP
`protocol or other protocol on recent network performance
`measurements, including latency, jitter, packet loss, round
`trip time, and/or the measured variance in a network perfor-
`mance measurement across multiple samples; on recent mea-
`sures of utilization of a network, network segment, network
`interface, or network port; and/or, on recent measurements of
`performance or utilization of a server, group of servers, or
`server component(s) such as memory, processor, disk, bus,
`intersystem interface, and/or network interface. In still other
`
`

`
`US 8,750,155 B2
`
`7
`embodiments, a server, client or node bases the conditional
`adaptation of the attributes of the TCP protocol or other
`protocol on temporal factors, including time of day; day of
`week, month, or year; specific date; occurrence ofa holiday or
`religious observance; occurrence of a temporal event such as
`a news event or sports event; seasonal occurrence; and/or a
`scheduled event or time period.
`In the primary embodiment, the protocol attribute selector
`212 of the server 206 compares the alphanumeric URL string
`provided by the client 102 in its information request to a table
`220 containing partial or whole URLs and identifies the most
`specific match from left to right that it can find in the table
`220. In another embodiment, the server 206 compares a sub-
`set of the alphanumeric string, for example some or all of the
`characters in the query string, or the characters following the
`host field up to the first subsequent slash (i.e., “/”), to a table
`220. In another embodiment, the client or node 102 makes a
`conditional adaptation of protocol attributes, using the alpha-
`numeric URL stri