(12) Ulllted States Patent
`Price
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 8,185,611 B2
`*May 22, 2012
`
`US008l85611B2
`
`(54) STREAMING MEDIA DELIVERY SYSTEM
`
`(75)
`
`Inventor: Harold Edward Price, Bethel Park, PA
`(US)
`(73) Assignee: WAG Acquisition, LLC, Flanders, NJ
`(US)
`
`( * ) Notice‘
`'
`
`Subject to any disclaimer the term ofthis
`.
`’.
`Patent 15 extended er adlnsted nnder 35
`USC. 15403) by 0 days
`This
`atent is sub'ect to a terminal dis-
`. P
`J
`C1a1mer'
`
`5,262,875 A
`5,440,334 A
`§rg‘§;r§g‘§ :3
`5,710,970 A
`5,719,786 A
`5,734,119 A
`5,737,536 A
`5,793,980 A
`5309339 A
`5,815,662 A
`5,322,537 A
`5,910,876 A
`5’922’048 A
`5,923,655 A
`5,928,330 A *
`
`..............H 370,601
`
`11/1993 Mincer
`8/1995 Walters
`51332 figlfjggne, ,1.
`1/1998 Walters
`2/1998 N l
`3/1998 Fr::(c):
`4/1998 Herrmarm
`8/1998 Glaser et al.
`9;1998 Dan
`91998 0
`10/1993 Kgfgeget 31,
`6/1999 Sharma
`7/1999 Emma """""""""""" " 709/219
`
`7/1999 Veschi et al.
`370/394
`7/1999 Goetz et al.
`................. .. 709/231
`
`........... .. 395/200.61
`
`.......... H 395/200,51
`
`(21) Appl. No.: 12/800,177
`
`(22)
`
`Filed:
`
`May 10, 2010
`
`(65)
`
`Prior Publication Data
`
`Sep' 2’ 2010
`US 2010/0223362 A1
`Related U.S. Application Data
`
`(63) Continuation of application No. 10/893,814, filed on
`Jul. 19, 2004, now Pat. No. 7,716,358, which is a
`continuation-in-part of application No. 09/819,337,
`filed on Mar. 28, 2001, now Pat. No. 6,766,376.
`(60) Provisional application No. 60/231,997, filed on Sep.
`12, 2000.
`
`(51)
`
`Int. Cl.
`(2006.01)
`G06F 15/16
`(52) U.S. Cl.
`..................................................... .. 709/219
`(58) Field of Classification Search ................ .. 709/230,
`709/231, 217, 218, 219
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`4,963,995 A
`10/1990 Lang
`5,057,932 A
`10/1991 Lang
`5,164,839 A
`11/1992 Lang
`
`(C°mi““ed)
`
`OTHER PUBLICATIONS
`
`Salehi et a1., “Supporting Stored Video: Reducing Rate Variability
`and End-to-End Resource Requirements through Optimal Smooth-
`ing,” IEEE/ACM Transactions on Networking, vol. 6, Issue 4, pp.
`397410’ Aug’ 1998'
`
`(Continued)
`
`.
`.
`.
`PWWWV EW'”"e’ ‘ J°SePh 13 AVenm°
`Assistant Exammer T Marshall McLeod
`(74) Attorney, Agent, or Firm — Emest D. Buff; Emest D.
`Buff & Assoc. LLC
`
`(57)
`
`ABSTRACT
`.
`.
`.
`.
`.
`.
`ISt:eaminrg1r1r1ed1ai.such as audio t0I'1V1d1eO fiées, 1s sent,v1a the
`11 eme ’
`. e me 1a are 1I.nme 1a e. y P aye on a user S Com‘
`puter. Audio/video data is transmitted from the server more
`
`rapidly than it is played out by the user system. The audio/
`video data in the user buffer accumulates; and interruptions in
`playback as well as temporary modem delays are avoided.
`
`18 Claims, 3 Drawing Sheets
`
`
`
`PAGE 1 OF 15
`
`|.M.L. SLU'S EXHIBIT ‘IOO1
`
`PAGE 1 OF 15
`
`I.M.L. SLU'S EXHIBIT 1001
`
`

`
`US 8,185,611 B2
`Page 2
`
`12/2003 Chen et_a1.
`6,665,751 B1
`§;’,’,§‘S’f,‘f‘;‘;1l‘,ff.”1.'........... H 709/203
`E},
`12/2004 Meyer et al
`6829 368 B2
`12/2004 Robinett
`1
`6,831,892 B2
`2/2005 Allen
`6’850’965 B2
`5/2005 Patel
`e’889’257 Bl
`6/2005 Kovacevic
`6,907,481 B2
`5/2006 Llllevelel
`7’054’500 Bl
`9/2006 Nguyen elel
`7’lll’058 Bl
`1/2007 H0 el 31
`7,170,856 B1
`9/2007 Lang
`7’272’298 Bl
`2/2008 Allen
`7’334’044 Bl
`3/2008 Halmaway
`7,346,698 B2
`5/2008 Nguyen et al
`7,373,413 B1
`11/2001 Slncagllaetlll
`2001/6047377 A1=1<
`' """""""" "
`OTHER PUBLICATIONS
`
`'
`
`'
`
`709/1
`
`an
`in
`“Smoothing Variable-bit-rate Video
`a1.,
`Rexford et
`internetwork,”IEEE/ACMTransactions onNetworking,Vo1.7,Issue
`2, pp. 202-215, Apr. 1999.
`Bianchi, “Buffer sizing forhigh speedvideo informationretrieval on
`ATMnetworks,”Globecom-NewYork-, 1997, Vol. 2,pp. 1057-1061.
`Bing Zheng and Mohammed Atiquzzaman, Multimedia Over High
`SpeedNetworks:ReducingNetWorkRequirementsWith Fast Buffer
`Fillupl
`
`* cited by examiner
`
`370/352
`375/240
`709/219
`709/219
`709/231
`~ 709/219
`709/217
`
`U.S. PATENT DOCUMENTS
`5,963,202 A
`10/1999 Polish
`5,978,557 A
`11/ 1999 Reba“
`519955705 A
`11/ 1999 Lang
`5,999,525 A
`12/1999 Krishnaswamyet al.
`6,002,720 A
`12/1999 Yuit etal.
`6,014,693 A
`1/2000 1106131 ~~~~~ ~~
`6,014,694 A
`1/2000 Aharonietal.
`6,014,706 A
`1/2000 Cannon etal.
`5,029,194 A
`2/2000 T111
`~~~~~~~~ ~
`6,047,317 A
`4/2000 Bisdikian eta
`6,047,356 A
`4/2000 Anderson
`~~~~~~~~~~~~~~~~~~~~ ~~ 345/327
`6,057,832 A
`5/2000 LCV 5131
`6,061,731 A
`5/2000 B13-keslee ~~~~~~~~~~~~~~~~~~~ ~~ 709/231
`6,061,732 A
`5/2000 K0T5tet31~ ~~~~~~~~~~~~~~~~~ ~~ 709/231
`6,065,050 A
`5/2000 DeM0ney
`709/219
`5,085,221 A
`7/2000 Gfaf ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~ 709/202
`6,173,340 B1
`1/2001 Gready
`6,233,226 B1
`5/2001 Grjngeri
`6,279,040 B1
`8/2001 Ma
`6,292,834 B1
`9/2001 Ravi et al
`5,321,259 B1
`11/2001 W311<€f
`6,405,256 B1
`6/2002 Lin
`5,449,719 B1
`9/2002 Baker
`5,588,015 B1
`7/2003 Eyer 9131
`6,594,699 B1
`7/2003 Sahaietal
`6,598,228 B2
`7/2003 Hejna
`6,637,031 B1* 10/2003 Chou ............................ .. 725/87
`
`
`
`.
`
`PAGE 2 OF 15
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 2 OF 15
`
`I.M.L. SLU'S EXHIBIT 1001
`
`

`
`U.S. Patent
`
`May 22, 2012
`
`Sheet 1 of3
`
`US 8,185,611 B2
`
`Fig. 1
`
`PAGE 3 OF 15
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 3 OF 15
`
`I.M.L. SLU'S EXHIBIT 1001
`
`

`
`U.S. Patent
`
`May 22, 2012
`
`Sheet 2 of3
`
`US 8,185,611 B2
`
`Fig. 2
`
`llllllll
`
` ||!Hlllllll
`
`PAGE 4 OF 15
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 4 OF 15
`
`I.M.L. SLU'S EXHIBIT 1001
`
`

`
`U.S. Patent
`
`May 22, 2012
`
`Sheet 3 of3
`
`US 8,185,611 B2
`
`Fig. 3
`
`
`A.
`a.
`
`$
`
`PAGE 5 OF 15
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 5 OF 15
`
`I.M.L. SLU'S EXHIBIT 1001
`
`

`
`US 8,185,611 B2
`
`1
`STREAMING MEDIA DELIVERY SYSTEM
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application is a continuation of U.S. patent applica-
`tion Ser. No. 10/893,814, filed Jul. 19, 2004 (published on
`Dec. 9, 2004 as U.S. patent publication number 2004/
`0249969 A1, and now U.S. Pat. No. 7,716,358), which was a
`continuation-in-part of U.S. patent application Ser. No.
`09/819,337, filed Mar. 28, 2001 (now U.S. Pat. No. 6,766,
`376), which claimed the benefit under 35 U.S.C. §119(e) of
`U.S. provisional patent application Ser. No. 60/231,997, filed
`Sep. 12, 2000; claims the benefit, under 35 U.S.C. §120, of
`the respective filing dates of said applications, as well as
`benefit of the filing date of copending U.S. patent application
`Ser. No. 10/825,869, filed Apr. 16, 2004 (published on Dec.
`23, 2004 as U.S. patent publication number 2004/260828
`A1), which was a continuation of said U.S. patent application
`Ser. No. 09/819,337, filed Mar. 28, 2001 (now U.S. Pat. No.
`6,766,376), which claimed the benefit under 35 U.S.C. §119
`(e) of said U.S. provisional patent application Ser. No.
`60/231,997, filed Sep. 12, 2000; and hereby incorporates by
`reference the entire disclosure of each of said prior applica-
`tions.
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`The present invention relates to multimedia computer com-
`munication systems; and more particularly, to systems and
`methods for delivering streaming media, such as audio and
`video, on the Internet.
`2. Description of the Related Art
`Prior to the development of Internet streaming media tech-
`nologies, audio and video were formatted into files, which
`users needed to download in their entirety to their computers
`before the files could be heard or viewed. Real time, continu-
`ous media, as from a radio station, was not suitable for this
`arrangement, in that a file of finite size must be created so it
`could be downloaded. The advent of streaming media tech-
`nologies allowed users to listen to or view the files as they
`were being downloaded, and allowed users to “tune-in” to a
`continuous media broadcast, or “stream”, such as from a
`radio station.
`
`Sending audio or video files via a network is known in the
`art. U.S. Pat. No. 6,029,194 to Tilt describes a media server
`for the distribution of audio/video over networks, in which
`retrieved media frames are transferred to a FIFO buffer. A
`
`clock rate for a local clock is adjusted according to the full-
`ness of the buffer. The media frames from the buffer are sent
`
`in the form of data packets over the networks in response to
`interrupts generated by the local clock. In this manner, the
`timing for the media frames is controlled by the user to assure
`a continuous stream of video during editing. U.S. Pat. No.
`6,014,706 to Cannon, et al. discloses an apparatus and
`method for displaying streamed digital video data on a client
`computer. The client computer is configured to receive the
`streamed digital video data from a server computer via a
`computer network.
`The streamed digital video data is transmitted from the
`server computer to the client computer as a stream of video
`frames. U.S. Pat. No. 6,002,720, to Yurt, et al. discloses a
`system for distributing video and/or audio information,
`wherein digital signal processing is employed to achieve high
`rates of data compression. U.S. Pat. No. 5,923,655, to Veschi
`et al. discloses a system and method for communicating
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`
`audio/video data in a packet-based computer network,
`wherein transmission of data packets through the computer
`network requires variable periods of transmission time. U.S.
`Pat. No. 5,922,048 to Emura discloses a video server appa-
`ratus having a stream control section that determines a key-
`frame readout interval and a keyframe playback interval,
`which satisfy a playback speed designated by a terminal
`apparatus. Finally, U.S. Pat. No. 6,014,694 to Aharoni, et al.
`discloses a system and method for adaptively transporting
`video over networks,
`including the Internet, wherein the
`available bandwidth varies with time.
`
`Despite these developments, users viewing or listening to
`streaming content over Internet connections often encounter
`interruptions, due to the frequency of unanticipated transmis-
`sion delays and losses that are inherent in many Internet
`protocols. These interruptions are commonly referred to as
`“dropouts”, meaning that the data flow to the user has been
`interrupted (i.e., the audio “drops out”).
`Dropouts can be extremely annoying—for example, while
`listening to music. The current state-of-the-art solution to the
`problem uses a pre-buffering technique to store up enough
`audio or video data in the user’s computer so that it can play
`the audio or video with a minimum of dropouts. This process
`requires the user to wait until enough of the media file is
`buffered in memory before listening or viewing can begin.
`The media data is delivered by a server computer, which has
`available to it the source of the media data, such as by a
`connection to a radio station. When the user connects to the
`
`server via the Internet, audio/video output at the user’ s system
`is delayed while the user’s buffer is filled to a predetermined
`level. Typical pre-buffering wait times range from ten to
`twenty seconds or more, determined by the vendor providing
`the audio or video media. Even with this pre-buffering pro-
`cess, interruptions in playback still occur.
`In this process, the user has a software application on the
`computer commonly called a “media player”. Using the fea-
`tures built into the media player, the user starts the audio or
`video stream, typically by clicking on a “start” button, and
`waits ten to twenty seconds or so before the material starts
`playing. During this time data is being received from the
`source and filling the media player’s buffer. The audio or
`video data is delivered from the source at the rate it is to be
`
`played out. If, for example, the user is listening to an audio
`stream encoded to be played-out at 24,000 bits per second, the
`source sends the audio data at the rate of 24,000 bits per
`second. Provided that the user waits ten seconds, and the
`receipt of the buffering data has not been interrupted, there is
`enough media data stored in the buffer to play for ten seconds.
`Gaps in the receipt of audio/video data, due to Internet
`slowdowns, cause the buffer to deplete. Because transmission
`of audio/video media data to the user takes place at the rate it
`is played out, the user’s buffer level can never be increased or
`replenished while it is playing. Thus, gaps in the receipt of
`audio/video media data inexorably cause the buffer level to
`decrease from its initial level. In time, extended or repeated
`occurrences ofthese gaps empty the user’s buffer. The audio/
`video material stops playing, and the buffer must be refilled to
`its original predetermined level before playing of the media
`resumes.
`
`By way of illustration, if, in a ten second pre-buffering
`scenario, data reception stopped the instant that the media
`started playing, it would play for exactly ten seconds. Once
`the media data starts playing, it plays out of the buffer as new
`media data replenishes the buffer. The incoming data rate
`equals the rate at which the data is played out of the user’s
`buffer, assuming the receipt of data across the Internet is
`unimpeded. If there are no interruptions in the receipt of the
`
`PAGE 6 OF 15
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 6 OF 15
`
`I.M.L. SLU'S EXHIBIT 1001
`
`

`
`US 8,185,611 B2
`
`3
`media data for the duration of the time the user listens to or
`watches the material, the buffer level remains constant and
`there will still be ten seconds of data stored in the media
`
`player’s buffer when the user stops the player.
`On the other hand, ifthe media player encounters interrup-
`tions totaling six seconds while playing the material, there
`would only be four seconds of media data remaining in the
`buffer when the user stopped it. Ifdata reception interruptions
`at any time during the playing exceed ten seconds, the user’s
`media player buffer becomes exhausted. There is no media
`data to play, and the audio or video stops—a dropout has
`occurred. At this point a software mechanism in the media
`player stops attempting to play any more of the material, and
`starts the buffering process again. The media player remains
`silent until the buffer refills, at which time the media player
`will once again start playing the material. This pattern has
`brought about considerable consumer
`frustration with
`streaming media over the Internet.
`
`BRIEF SUMMARY OF THE INVENTION
`
`There is a need for improved systems and methods for
`delivering streaming content over the Internet or other com-
`munications medium, which facilitate continuous transmis-
`sion of streaming content, respond on demand without obj ec-
`tionable buffering delay, and perform without disruption or
`dropouts.
`To address these objectives, various embodiments for
`delivering streaming content are provided, which envision
`that both the server and user systems involved in the content
`delivery may have buffering capacity. The embodiments
`make varying uses of this capacity to facilitate continuous
`content transmission on demand. Nearly instantaneous play-
`back is achieved, while maintaining protection against play-
`back interruption.
`In one aspect, the server and user-sides of the transmission
`are coordinated, by (a) sending initial streaming media ele-
`ments to the user system at a sending rate more rapid than the
`playback rate, to fill the user buffer; and (b) after the user
`buffer has been filled, sending further streaming media data
`elements to the user system at about the playback rate.
`In another embodiment, the user system may be used to
`regulate transmission of streaming media to it, by a streaming
`media server. In such embodiment, the server may operate by
`(a) assigning identifiers to the sequential media data elements
`comprising the program; (b) receiving requests from the user
`system for media data elements corresponding to specified
`identifiers; and (c) sending media data elements to the user
`system responsive to said requests. A user system used in
`connection with such an embodiment may operate by (i)
`maintaining a record of the identifier of the last sequential
`media data element that has been received by said player; (ii)
`requesting transmission of the next sequential media data
`elements following said last sequential media data element,
`as said media player requires for continuous and uninter-
`rupted playback.
`Other aspects and advantages of the invention will be
`apparent from the accompanying drawings and the detailed
`description that follows.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The invention will be more fully understood and further
`advantages will become apparent when reference is had to the
`following detailed description and the accompanying draw-
`ings, in which:
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`
`FIG. 1 is a schematic/block diagram illustrating the ele-
`ments of a streaming media buffering system in accordance
`with one embodiment of the present invention;
`FIG. 2 is a schematic/block diagram of an alternative
`embodiment of the system shown by FIG. 1; and
`FIG. 3 is a flowchart illustrating a method employed in one
`embodiment of the present invention.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENTS
`
`The following is a detailed description of certain embodi-
`ments ofthe invention chosen to provide illustrative examples
`of how it may preferably be implemented.
`Audio and video media must play out over a period oftime.
`Thus, in considering the delivery of such media, it is more
`appropriate in certain respects to think of bandwidth require-
`ments than file size. The bandwidth requirement of audio or
`video media refers to the data rate in bits per second that must
`be transmitted and received in order to listen to or view the
`
`material uninterrupted.
`Transmitting the audio or video material over a connection
`slower than the bandwidth requirement results in unsatisfac-
`tory viewing or listening, if viewing or listening is possible at
`all. The connection available may, for example, be by dialup
`modem, which has a maximum receive data rate of 56,000
`bits per second. Audio and video encoded for distribution
`over the Internet may be compressed to be listenable or view-
`able within such a 56,000 bits per second bandwidth.
`Requirements for achieving adequate audio and video over
`the Internet may consume a considerable portion of the lis-
`tener’s available bandwidth.
`
`There are two types of encoding schemes used for audio
`and video material—“Variable Bit Rate” (VBR), and “Con-
`stant Bit Rate” (CBR). CBR encoding represents the encoded
`media with a constant bit rate per second, regardless of the
`complexity of the material being encoded. For example, if an
`audio source is encoded at 20 kilobits per second at a Constant
`Bit Rate, the media data being produced from the encoding is
`at 20 kilobits per second, whether the audio material is com-
`plex (e.g., symphonic) or silence. Variable Bit Rate encoding
`uses a variable number of bits to represent sounds or video,
`with more bits required for complex material (e.g., sym-
`phonic sounds or action scenes) than for simple sounds,
`silence, or still scenes. The most usual encoding scheme used
`for streaming media is CBR, because the resulting data rate is
`more predictable than for VBR. Statements in this specifica-
`tion concerning “constant” data rates and the like should be
`understood as subject to appropriate variation where VBR-
`encoded data may be involved.
`Even if a user’s Internet connection has the requisite aver-
`age bandwidth capacity to allow reception ofthe program, the
`actual rate of delivery of data to the user can fluctuate widely
`above, and more particularly, below, this average, as a func-
`tion ofthe quality ofthe user’s connectivity at any given time.
`Internet connection quality can vary rapidly over time, with
`two primary factors responsible for degradation ofthe instan-
`taneous bandwidth actually available to the user. These fac-
`tors are the quality of the user’s Internet connection, which
`can have periods of interference causing reduced available
`bandwidth, and momentary Internet congestion at various
`points along the route over which the user’s data flows. Each
`of these factors can cause delays and interruptions in the
`transmission of data to the user. Internet data communications
`
`devices such as routers are designed to drop data packets if
`they get overloaded. For material that is not time sensitive,
`these dropped packets will usually be resent, and the user will
`
`PAGE 7 OF 15
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 7 OF 15
`
`I.M.L. SLU'S EXHIBIT 1001
`
`

`
`US 8,185,611 B2
`
`5
`eventually be presented with the material. However, since
`streaming media is time sensitive, dropped packets can have
`a significant impact on the receipt and playback of an audio or
`video stream. Such degradation in the receipt of Internet data
`is very common, and prevent most users from being able to
`listen to or view streaming media without interruption unless
`some special provisions have been incorporated into the
`user’s computer software to accommodate data transmission
`interruptions.
`There are two fundamental types of streaming media,
`which affect, in some respects, the requirements for smooth
`and continuous delivery: (i) material that originates from a
`source having a realtime nature, such as a radio or TV broad-
`cast, and (ii) material that originates from a non-real-time
`source such as from a disk file. An example of non-real-time
`material might be a piece of music stored as a disk file, or a
`portion of a broadcast that originally was realtime, perhaps
`yesterday’s TV evening news, and was recorded into a disk
`file. For purposes of clarity within this document, streaming
`media of type (i) will be referred to as “real time” or “broad-
`cast” media, and streaming media of type (ii) will be referred
`to as “file based” media.
`
`In many respects, both streaming media types are handled
`similarly in conventional systems, and both are handled simi-
`larly (in a number of respects) by the streaming media deliv-
`ery system of the present invention. Nevertheless, the two
`streaming media types are readily distinguished. Broadcast
`streaming media has as its source a system or arrangement
`that by definition can only be transmitted to users as fast as the
`material is generated; for example, a disk jockey speaking
`into a microphone. File based media, on the other hand, can
`be transmitted to users at any available data rate, since in the
`context ofdata communications, the time required for reading
`a small portion of data from a file residing entirely on a locally
`accessible, random access storage device may be considered
`negligible.
`In conventional systems for streaming media over the
`Internet, media data (whether real-time or file based) is sim-
`ply transmitted from the server to the user at the rate at which
`it will be played out (the “playback rate”), regardless of the
`data rate capabilities ofthe connection between the server and
`the user.
`
`Conventional streaming media systems may incorporate
`server-side buffering systems for programmatic purposes.
`For example, the system may buffer media data at the server
`for the purpose of packet assembly/disassembly. Media data
`may also be buffered at the server to permit programming
`conveniences such as dealing with blocks of data of a specific
`size. However, conventional streaming media systems have
`not utilized server-side buffering for the purpose ofmitigating
`long term Internet performance degradation. Rather, prior art
`systems, in which data is continuously transmitted at the
`playback rate, have performed buffering for continuity pur-
`poses solely on the user side, with the consequences dis-
`cussed above of startup delays and dropouts. The present
`invention addresses such shortcomings.
`The present invention provides a system and method for
`delivering streaming media, such as audio or video media, via
`the Internet or other communications medium. Immediate
`
`playing of the media on a user’s computer is afforded, while
`reducing interruptions in playback due to Internet congestion,
`and temporary modem delays due to noisy lines. Nearly
`instantaneous playback is achieved, while maintaining pro-
`tection against playback interruption. Delayed starts, hereto-
`fore required to provide protection against interruption, are
`avoided. Data lost due to interruptions in the receipt of media
`data by the media player can be recovered while the player
`
`6
`continues to play out the audio or video material. If the
`interruptions are so severe as to deplete the user’s buffer and
`stop the play out, the media player can quickly recover as
`well, by beginning to play out again without waiting to first
`build up the buffer, as soon as the media player begins to
`receive media data elements.
`
`In one embodiment, the invention provides a system for
`distributing via the Internet streaming media composed of a
`plurality of time-sequenced data elements. As shown in FIG.
`1, the system is provided with a server 12 connected to the
`Internet 10 for transmitting the streaming media data ele-
`ments. Associated with the server 12 is a server buffer 14 for
`
`storing at least one of the data elements for transmission, and
`a buffer manager 16. Buffer 14 is a conventional computer
`storage mechanism such as a hard disk, as shown for conve-
`nience of illustration, or, preferably, an electronic storage
`arrangement such as Random Access Memory (RAM).
`The media may come from a live source, shown as 26 in
`FIG. 1, or from a stored file on the server 12, or another
`storage device, such as a hard drive.
`A number of different implementations of such a server,
`involving different ways of handling server buffer 14, will be
`discussed.
`
`10
`
`15
`
`20
`
`25
`
`In the various implementations, there is in each case at least
`one user computer 18 (or similar device) connected to the
`server 12 via the Internet 10 or other data communications
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`medium. User computer 18 is associated with media player
`software incorporating user buffer 20. The user buffer 20 is
`provided with means for storing a predetermined number of
`the data elements. User buffer 20 is a conventional computer
`storage mechanism such as a hard disk, or, preferably, an
`electronic storage arrangement such as Random Access
`Memory (RAM) as suggested by the illustration. A buffer
`manager 22 is also associated with the user computer 18. The
`buffer manager 22, having the form of software or firmware,
`is provided with means for receiving and storing a predeter-
`mined number of media data elements which are received
`
`sequentially by the media player, playing the data out sequen-
`tially as audio and/or video, and deleting media data elements
`from the buffer as they are played out (or displacing them by
`newly arrived elements). As data is played out, the next
`sequential data elements are received from the server in such
`a fashion as to approximately maintain the predetermined
`number of data elements in the user’s buffer. It should be
`
`understood that data might arrive at the media player out-of-
`sequence and that processes in the media player or the media
`player buffer manager are responsible for properly arranging
`this data.
`
`Alternatively, user computer 18 may be replaced by an
`Internet radio or Internet Appliance, which is comprised of a
`dedicated processor for receiving Internet radio or audio/
`video material. Examples of such devices might range from
`familiar computing devices such as palmtops, PDAs (Per-
`sonal Digital Assistants), and wireless phones, to devices that
`appear and operate similarly to conventional consumer elec-
`tronic devices such as radios and televisions, but with the
`additional capability of Internet access.
`FIFO Server Buffer Implementation
`There are a large number of ways of managing server
`buffer 14 in order to implement the systems and methods
`described in this specification. In one implementation, buffer
`manager 16 is adapted to effectively render server buffer 14 a
`FIFO device. In this implementation, buffer manager 16 is
`provided in the form of software or firmware that provides
`means for: receiving the media data; supplying media data in
`order to the FIFO buffer; supplying the buffer 14 with a
`predetermined number of data elements; maintaining point-
`
`PAGE 8 OF 15
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 8 OF 15
`
`I.M.L. SLU'S EXHIBIT 1001
`
`

`
`US 8,185,611 B2
`
`7
`ers 2411 through 2411 into the buffer, one for each user com-
`puter indicating the last media data element that has been sent
`to that user, thus indicating the next element or elements to be
`sent; and, once the FIFO buffer is full, deleting (or displacing)
`the oldest data element in the buffer as each new data element
`
`is received. These means are arranged to maintain the pre-
`determined number of data elements in the FIFO buffer.
`
`Buffer Manager 16 may also comprise means for digitizing,
`encoding, and packetizing the media data, and formatting
`media data according to the requirements of buffer 14.
`Data Window Buffer Implementation
`Ifthe media source is file based, such as a music clip stored
`as a disk file, and if the disk file is stored on the server or an
`associated server computer, the server’s connection to the
`source could be considered to be near instantaneous. In this
`
`case, rather than audio/video data filling and depleting the
`buffer 14, an amount of audio/video data equivalent to the
`desired buffer size may be logically constituted as a FIFO
`buffer. Such a construct is commonly called a data window.
`The data window moves on a time-sequenced basis through
`the media data file, thus defining the contents of the buffer on
`a moment-by-moment basis and performing the equivalent
`functions to receiving a new data element and deleting the
`oldest data element.
`
`Example Buffering Methods
`In an arrangement that receives media data directly or
`indirectly from a real-time source, such as a radio station,
`server buffer 14 might be set to hold (for example) 30 seconds
`ofmedia data. Because the source produces media data in real
`time, the media data is delivered to the server approximately
`at the rate it is generated.
`Of course, there can be variability in this data delivery
`process due to networking, disk accesses, and so on, causing
`the delivery rate of the media data to be variable over short
`periods of time, typically measured in seconds. But over a
`longer period of time measured in minutes or tens of minutes
`or longer, the media data is delivered from source to server at
`the rate it is generated, and the server in turn provides that
`media data to the FIFO buffer at that same rate. Since CBR
`
`encoding is typically used for streaming media, the media
`data is generated, received by the server, and provided to the
`buffer approximately at a fixed rate.
`The server buffer 14 is filled the first time the media source
`connection is established or a disk file is read. The amount is
`
`preferably adequate to bridge gaps typical of Internet and
`modem delays to the user. This buffer may, for example, hold
`enough data elements for about one minute of play.
`Once server buffer 14 is full, for each new data element
`received into the buffer the oldest data element is deleted (or
`displaced)
`from the buffer.
`In some implementations,
`requests from user computers to connect may not be accepted
`until server buffer 14 is full.
`
`Once a connection is made to a user’s computer (e.g., user
`computer 18), server 12 sends the media data to the user
`computer in the following manner. First, media data is sent to
`the user computer at a rate faster than the playback rate, which
`may be the highest rate that the data connection between the
`server and the user computer will support, or any lower rate
`that is a higher rate than the playback rate (referred to herein
`as a “higher than playback” rate), until the predetermined
`amount of data that had been stored in the server buffer has
`
`been transferred to the user’ s computer. Once the contents of
`server buffer 14 has been transferred, a steady state condition
`is reached wherein as each media data element arrives at
`
`server 12, it is immediately sent out to the user computer. In
`this steady state condition, the media data is sent at a rate that
`matches the constant fill rate of the server buffer, and is
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`received at the same rate by the user computer if there are no
`interruptions in the transmission of media data between the
`server and the user’s computer (with some variation in the
`case ofVBR content). If interruptions have interfered with the
`arrival of sent media data to the user’s computer, that data
`may have been “dropped” by routers in the Internet and needs
`to be resent. This causes data to “back up” into the server
`FIFO for that user.
`
`A data communications transport mechanism, such as the
`TCP protocol, may be used for the reliable delivery of data in
`an ordered sequence from the source of the media data to the
`server, or from the server to the media player software of the
`user computer. Resending missing data is the responsibility of
`the reliable transport mechanism. The server buffer 14
`“sends” data by delivering it to the transport mechanism. The
`transport mechanism actually manages transmission of the
`data across the communications medium, and has process