1111111111111111 IIIIII IIIII 111111111111111 11111 1111111111 111111111111111 1111111111 11111111
`US 20120084455Al
`
`c19) United States
`
`c12) Patent Application Publication
`McCue et al.
`
`c10) Pub. No.: US 2012/0084455 Al
`(43) Pub. Date:
`Apr. 5, 2012
`
`(54) SYSTEM AND METHOD FOR RENDERING
`DIGITAL CONTENT USING TIME OFFSETS
`
`(76)
`
`Inventors:
`
`John McCue, Ottawa (CA); Robert
`McCue, Ottawa (CA); Gregory
`Shostakovsky, Kanata (CA); Glenn
`McCue, Ottawa (CA)
`
`(21) Appl. No.:
`
`13/313,393
`
`(22) Filed:
`
`Dec. 7, 2011
`
`Related U.S. Application Data
`
`(63) Continuation-in-part of application No. 12/096,933,
`filed on Jun. 11, 2008.
`
`(60) Provisional application No. 61/420,352, filed on Dec.
`7, 2010.
`
`Publication Classification
`
`(51)
`
`Int. Cl.
`G06F 15116
`
`(2006.01)
`
`(52) U.S. Cl. ........................................................ 709/231
`
`(57)
`
`ABSTRACT
`
`A system and method for rendering digital content includes a
`media player having access to at least one server via a net(cid:173)
`work. In addition to storing digital data corresponding to a
`printed document, such as an eBook, the one or more servers
`also stores a descriptor file. The descriptor file includes time
`information for defining the digital data, which is determined
`relative to a timeline of an audio recording of the text of the
`printed document. The digital content to be rendered is
`selected using the time information in the descriptor file and
`a time offset external to the descriptor file.
`
`110
`
`~ 100
`
`115
`
`~Au1~ba~:~rnJ1r---------------~
`
`,---------- J ___ ----------------------------------- -___ J ______ , / / /
`ffiilll ~ [illlJ
`lliil ~ -
`IE! IBlll
`film
`ffiw Llliil
`lillill
`lill[J
`[§;]
`: lBITl
`IIBlll
`ITill rm ~ ~ ~ ~ ~ ~ C
`j Lill
`filB
`------------------
`
`O
`
`>i;
`
`•!!
`
`:
`
`-----------~
`
`124
`
`Server SidA
`
`Client Side
`r---------------
`
`~ 124
`
`122
`
`120
`
`152
`
`fililll
`i I]]]] till ffiilll
`/7): !!ill]]
`lill)I] 0HI @D
`/v :,:;Gs][illu;
`----------------
`
`120
`
`122
`
`/
`
`154
`
`150
`
`-1-
`
`Amazon v. Audio Pod
`US Patent 10,091,266
`Amazon EX-1024
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 1 of 28
`
`US 2012/0084455 Al
`
`~
`
`10
`
`Server Side
`
`Transfer UtiHIY.
`''
`
`- --·
`
`~ 2 0
`
`(Prior art)
`Fig. 1
`
`(Prior art)
`Fig.2
`
`-2-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 2 of 28
`
`US 2012/0084455 Al
`
`100
`
`115
`
`I
`~Au~~a~:~~,
`: ------ - J ___ ------------------------------------· - -- J _______ ,,-1/
`:~~~~~~~~~~~~~~ ~ -
`:
`irn~~~~~~~~~ ~ ~~~
`--- ~--- ... ·-------------
`
`124
`
`Server Side
`
`Client Side
`
`I
`I
`
`- • - - - - - - - - - ...
`:
`I
`
`152
`
`122
`
`120
`
`uillIJ :
`li?.l ~ i;rn
`: l'c;;'I
`'
`lLlill Ji2J
`: L:.iJ
`l!:::J
`~); : ~ ~ §ill
`:.::,!;
`/v :~lliilllliifilIBNI
`~---------------
`
`124
`
`120
`
`122
`
`)
`
`156
`
`150
`
`154
`
`Fig. J
`
`/
`
`-3-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 3 of 28
`
`US 2012/0084455 Al
`
`144
`
`140
`
`i--------H-----.---------.-----------r----.!OPtional Reriiote
`•
`URLs '. :
`
`122
`
`124
`
`Fig. 4
`
`ActuaF'audio
`• • . : Stream ., •••
`
`Stream Identifier
`
`Content Details
`Small Audio Files 1. 2, 3, ... , N
`
`.··•·Srnii-U.Au(ll6File
`•• 'Structurer
`
`Fig. Sa
`
`Stream Identifier
`Small audio file Identifier
`Content Details
`
`Audio Content
`
`rig.Sb
`
`-4-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 4 of 28
`
`US 2012/0084455 Al
`
`Virtual Stream Descriptor
`
`I -
`
`Internal Ide ntification
`
`I Unique Identifier
`
`Descriptlve Details
`
`Title
`
`Subtitle (optional)
`
`Author(s)
`
`Publisher
`
`Copyright
`
`Narrators
`
`Comments/Plot
`
`Duration
`
`ISBN Number
`
`VCHIP Rating
`
`Internal M
`edia Marks
`
`Table of Content
`
`Index
`
`List oi Figures
`
`Physical S tream Details
`
`Fixed Location (if not library based)
`
`Segment Count
`
`Segment Location/Directory
`
`Segment Identifier Name Base
`
`Segment Metrics List
`
`Illustration s
`
`I List of Illustrations (optional)
`
`Internal Atlvertising
`
`j List of Fixed Advertisements (optional)
`
`Fig. Sc
`
`-5-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 5 of 28
`
`US 2012/0084455 Al
`
`Stream Identifier
`Illustration Identifier
`Content Details
`
`Graphic/AudioNideo Content
`
`Fig. 5d
`
`Advertising Identifier
`
`Content Details
`
`Graphic/AudioNideo Content
`
`Fig. 5e
`
`Catalog Index
`
`Unique Identifier
`Parent Identifier
`Descriptive Details
`
`Title
`Subtitle
`Content Details
`
`Virtual Audio Stream Identifiers
`Subordinate Catalog Index ldentlfiers
`
`Fig. Sf
`
`Server List
`
`Internal Identification
`
`Unique Identifier
`Descriptive Details
`
`Name
`Port
`Root Directory
`SSL Fla~
`
`Fig. 5g
`
`-6-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 6 of 28
`
`US 2012/0084455 Al
`
`Virtual Audio
`Stream Descriptor lf-----.-----11~ ,_::t 'Ac·t~-~l·a,udio · •·
`''· stream : : •
`
`..
`
`.
`
`. .. ; .. .. ;, .. ~ . .. : ;
`.. .. ..
`.
`
`.
`
`'
`
`; ,: : \~ - '.- -
`
`: . : : : : · : : · ; : .
`
`.
`
`; _ _ : i ·
`
`:_ : · :~
`
`;
`
`I
`
`L ....
`
`i·-'Advertisements; ,-
`
`Fig. 6
`
`-7-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 7 of 28
`
`US 2012/0084455 Al
`
`Master Library
`
`Audio
`Stream
`Splitter
`
`,~
`
`Raw Audio Stream
`
`~ r row . .___ .
`
`Image
`Formatter
`
`Raw Cover Art Image
`
`- - - - - -
`
`Segme
`
`• []
`j Formatted Cov~r
`!
`Art Image
`.•
`~ -
`
`"' 'V
`. . ___ st-rea_m ~
`
`I
`
`Audio Stream
`Information
`
`Library
`Administrator
`Utility
`
`Index Information
`
`lll'lli11\;lo'I,· '11'.l!\Hi'il..-,li<'l> •
`
`"'
`
`Audio
`
`Profiler
`
`Library
`~ Verification
`& Upload
`
`Index
`Builder
`
`m
`
`Card Catalog••.
`Index Files
`
`- Audio.Pod
`Libraiy &
`Mirrors
`
`l Network
`
`Access
`
`LJ
`tJ
`
`. Update File
`,____ _________ , ..... , ............... ..
`
`Fig. 7
`
`-8-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 8 of 28
`
`US 2012/0084455 Al
`
`200
`
`Build Raw File
`list in order
`
`Open next Raw
`File
`
`204
`
`206
`
`208
`
`216
`
`segment file
`
`Close All Files
`
`Close current
`Segment File
`
`Open Next
`Segment File
`
`220
`
`222
`
`Fig. 8
`
`-9-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 9 of 28
`
`US 2012/0084455 Al
`
`Bookmark
`
`Audio Stream Type
`
`Descriptive Details
`
`Comment (Default: Title/Subtitle/Author)
`
`Content Details
`
`Virtual Audio Stream Identifier
`Absolute Time Offset
`
`Fig. 9
`
`I ritegrated Card
`Catalog
`• •
`
`Virtual Audio
`Stream Descriptor+--.----.i•>- 'A'ctDhl ·~udi~ .··••··•··
`-'···•--"•·: •Stream ... •; •
`
`Adverti$eriients ·.i
`
`Fig. 10
`
`364
`
`362
`
`368
`
`360
`
`Download. : .
`•··•·•·IV1,mage( ;••
`
`366
`
`Fig.11
`
`-10-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 10 of 28
`
`US 2012/0084455 Al
`
`'\Oen~ri9 ; :
`:+fahsfe( ,,
`' Utility
`'.
`
`Server Side
`
`... Admini$tra:tive ;
`
`I : ,SeNet List U I
`/ U,Pilate~ : < I
`
`l ·.
`
`I >: ServerLJst > I
`
`Bookmark(s)
`
`Virtual Audio
`Stream Descriptor
`
`Cover Art
`Image
`
`: ; Aciuii.l au~i;; •
`'::? l$tream : ••
`
`Fig. 12
`
`lntegr~t~d;C~rd Catalog
`. : . .
`.
`
`'
`
`,··,
`
`·:,
`
`,
`
`Cataloglndex(s) )
`
`•• Audici Stream Cb~tent
`
`Virtual Audio
`Stream Descriptors 1
`,-,-~-~--,...... .... ------~
`Actual audio: •··• I
`•· Streams :<C ·. 1
`-..,;;,;,;,;,,""""""~-'
`
`•
`
`•
`
`-11-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 11 of 28
`
`US 2012/0084455 Al
`
`Increasi ng
`Level of
`Memory
`Purging
`
`:;
`ii
`Ji
`R
`
`Bookmark
`Position
`
`Fig. 13
`
`Audio Stream LlJ
`Resident Portion D
`
`- - - - !1 ■ 11•••11 .. ••········ ►
`
`c 1.,< F.:i!,<---, -. A"""':cl "' ,n 1-1 H 1111 11 111 111111
`••••••••►
`
`D
`
`Client Player
`Position
`
`Fig.14
`
`Audio Stream [TI
`Resident Portion D
`
`-12-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 12 of 28
`
`US 2012/0084455 Al
`
`Browse
`Segment iiles
`
`NO
`
`Purge Memory
`
`Ignore Memory
`Reuqest 14------'
`
`Start Download
`Manager
`
`Fig. 15
`
`Compare
`spooler & VAS
`
`>N_o ________ -1..istop Download
`Manger
`
`Get File Size
`
`NO
`
`Download File
`
`...__ _ _ _ _ _ _ _ _ _ _ _ ---1 Delete Faulty
`File
`
`Fig. 16
`
`-13-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 13 of 28
`
`US 2012/0084455 Al
`
`Introduction
`Display
`
`Bookmark
`Display
`
`Setup
`Display
`
`Notice Board
`Display
`
`Update
`Manager
`
`Card Catalog
`• Button .:
`
`' : Setup: •
`.. Buttpf'I •'
`
`Notice Bo.ard
`• •• !3uttan
`•
`
`: Fili Up .·
`Button ··
`
`Start Memory
`Manager
`
`Fig. 17
`
`-14-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 14 of 28
`
`US 2012/0084455 Al
`
`Get Input
`Cur11r11and
`
`Load Next
`Audio Segment
`
`Issue STOP
`Command
`
`Issue STOP
`Command
`
`Display 'THE(cid:173)
`END'
`
`Delete
`Bookmark
`
`Purge stream
`from spooler
`
`Wait For
`Command
`
`Fig. 18
`
`-15-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 15 of 28
`
`US 2012/0084455 Al
`
`Get Root Index
`
`Display Index 1 - - - - -~
`
`Get Book
`Profile
`
`Bookmark
`Display
`
`Stop Player &
`Bookmark
`
`Get Audio
`Segment
`
`L__ ___ . ,
`
`Load & Position
`Player
`
`Wait For
`Command
`
`Fig. 19
`
`-16-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 16 of 28
`
`US 2012/0084455 Al
`
`226
`
`Integrated. Gard :
`· >> Ca~cllog • • •
`
`'··•Audio:strearn•\ /
`.;': ;;_.; :_ ~-.;: :. ;: :, :- : :- -: ;: :: •
`
`224
`
`\ Audio: stream 2 ••
`
`I Bookmaik . ~
`
`Virtual Media
`Descriptor
`
`.
`
`..
`
`. ... , .
`
`···· ·· .. ..
`
`{:f ~~P:9~!3~·?•'.:f ;::::·•··
`
`.
`.
`. ••'•
`.
`: mustrations ••
`. . , .
`
`Fig. 20
`
`-17-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 17 of 28
`
`US 2012/0084455 Al
`
`Originating
`Work
`
`Derivative
`Works
`
`Fig. 21
`
`-18-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 18 of 28
`
`US 2012/0084455 Al
`
`224
`
`228
`
`229
`
`► I
`
`t,. T =Duration
`
`Virtual Media Descriptor
`
`Table Of Contents
`
`List Of Illustrations
`
`List of Figures
`
`Time Offsets (6T)
`
`Fig. 22
`
`Fig. 23
`
`-19-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 19 of 28
`
`US 2012/0084455 Al
`
`241
`
`242
`
`Presentation
`Manager 1--------+
`
`I~ - - - ' - -~~
`
`~------~---------------------
`
`Time Offset
`Manager ~ 247
`
`Increasing Time Offset (t,.T)
`···········
`
`:
`ll>-1
`I
`I
`
`_Virtual Page·strcam
`
`Fig. 24a
`
`-20-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 20 of 28
`
`US 2012/0084455 Al
`
`Audio Player
`Ovorsight Timer Event
`
`Do Nothing
`
`Multiply real-time
`interval by -n
`
`Multiply real-time
`interval by n
`
`Gel current lime
`position_ reported
`
`Add product to
`Current Offset
`
`Add currant
`segme nt start time
`
`Audio Player
`End-of-Stream Event
`
`Stop Audio Player
`Oversight Timer
`
`Set Result to zero
`
`Load Ne!<t
`audloSegment
`
`Get segment start
`time from Virtual
`Media Stream
`
`(.._ __ D_ro_p_,E,_v_e_nt _ _,.)
`f
`Get Drop Buffer
`and val idate
`
`Paste Event
`
`Get Paste Buffer
`and validate
`
`Double CIiek
`Event
`
`Get Click Tag
`Buffer
`
`eText Page/Chapter
`Turn Event
`
`eText Text Hit Event
`
`Load New Book
`
`Extract requested
`ottset from buffer.
`Default is zero
`
`Get offset of new
`current page from
`Virtual Page Stream
`
`Determine Start Time
`Of character Position by f--------------------► I
`lookup or calculation
`
`Validate Result
`0 <= ,H <=
`Duralion
`
`Send Result to
`Presentation
`Manager as
`irrent Offset
`
`Fig. 24b
`
`-21-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 21 of 28
`
`US 2012/0084455 Al
`
`I segment 1 I
`
`Isegment2:1
`I Se~ment3I
`I Segment ... I
`
`lsegmentNI
`
`I Segment ti
`I Segrnent 21
`I Segment a:I
`!Segment.• .. :!
`
`lseg~eritNI
`
`Fig. 25
`
`Fig. 26
`
`-22-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 22 of 28
`
`US 2012/0084455 Al
`
`cBook Structure
`
`Internal I deutificntion
`
`---
`
`Unique Identifier
`
`eBookMe
`tries
`
`Start Time
`
`E□ d Time
`
`Duration
`
`Current Offset
`
`Current Chapter Numhcr
`
`Chapters ln Book
`
`Page Turn Djrection
`
`First Line in Book (after Pagination)
`
`Last Line In Book (after Pagination)
`
`Content
`
`eChaptcrs (pointers to)
`
`Formatted HTML Header
`
`Formatted IITML Fnotcr
`
`Fig. 27a
`
`-23-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 23 of 28
`
`US 2012/0084455 Al
`
`eBook Chapter Structure
`
`-
`
`Internal I dcntification
`
`I Unique hknLifier - Chapter Numller
`
`cChupter .Metlics
`
`Start Time
`
`End Time
`
`Duration
`
`Number of Pages in Chapter
`
`Current Page Number
`
`Chapters In Book
`
`Page Tum Direction
`
`First Line in Ch11ptcr (atlcr Pagination)
`
`Last Line In Chapler (after Pagination)
`
`Content
`
`cParas (pointers to)
`
`ePages (pointers to)
`
`Formatted HTML Header
`
`Formaned HTML footer
`
`Fig. 27b
`
`-24-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 24 of 28
`
`US 2012/0084455 Al
`
`el3ook Paragraph Structure
`
`Internal ldentific;it:ion
`
`j Unique Identifier - Paragraph Number
`
`Paragraph Metrics
`
`Anchor Time
`
`Stiirt Time
`
`End Time
`
`Durntion
`
`Charncler Count
`
`CJrnracters per Second
`
`Seconds per charnc::1er
`
`First Line numhcr ([!ftcr pagination)
`
`l.asL Line Number (after pagination)
`
`Line Count
`
`Content
`
`Raw Text
`
`Cteaned Text
`
`Formatted IJnes of Text (after p agin(ltion)
`
`Fig. 27c
`
`-25-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 25 of 28
`
`US 2012/0084455 Al
`
`Load eText
`Segment
`
`Duplicate raw tex.t
`
`Cleanup Tex.I.
`remove formating
`squeeze white space
`
`Access correct time
`tag
`
`Update Text
`Location
`Information
`
`Update word and
`character counts
`
`Update time
`metrics using tag
`deltas
`
`Fig. 28
`
`-26-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 26 of 28
`
`US 2012/0084455 Al
`
`Time Tag
`
`-
`
`Internal I dentilicalicm
`
`Unique Identifier
`
`cs
`TngMctri
`
`Anchor Time
`
`Start Time
`
`End Time
`
`Duration
`
`Text Location lnfunnation
`
`Character Count
`
`Wh1le Space Count
`
`Word Count
`
`Sentence Count
`
`Tag Delta Metrics
`
`Characters Per Second
`
`Seconds Per Character
`
`Fig. 29
`
`-27-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 27 of 28
`
`US 2012/0084455 Al
`
`Load Virtual
`Text Stream
`
`Load a Hard Time
`Tag
`
`Create Soft Time
`Tag
`
`Copy Hard Time
`Tag Delta Metrics
`
`Insert Soft Time
`Tag
`
`Insert Hard Time
`Tag
`
`c ___ □o__...n•)
`
`Fig. 30
`
`-28-
`
`

`

`Patent Application Publication
`
`Apr. 5, 2012 Sheet 28 of 28
`
`US 2012/0084455 Al
`
`Virtual Pnge Struciurt!
`
`~ Internal Id entification
`
`Unique lden1ifier- Page Number
`
`I
`
`Virluul l'a
`gc M<trics
`
`Stare Time
`
`End Time
`
`Duration
`
`First Line number (after pagination)
`
`Lasl Linc Number (a lier p11ginuiion)
`
`Line Counl
`
`Content
`
`1 XML Rendering Tag,;
`
`Fig. 31
`
`Fig. 32
`
`-29-
`
`

`

`US 2012/0084455 Al
`
`Apr. 5, 2012
`
`1
`
`SYSTEM AND METHOD FOR RENDERING
`DIGITAL CONTENT USING TIME OFFSETS
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`[0001] This application claims the benefit of priority of
`U.S. Provisional Appl. No. 61/420,352 filed Dec. 7, 2010,
`which is hereby incorporated by reference. In addition, this
`application is a continuation-in-part of U.S. patent applica(cid:173)
`tion Ser. No. 12/096,933 filed Jun. 11, 2008, which claimed
`priority from U.S. Provisional Appl. No. 60/749,632 filed
`Dec. 13, 2005, both of which are also hereby incorporated by
`reference.
`
`TECHNICAL FIELD
`
`[0002] The present application relates generally to the ren(cid:173)
`dering of digital content, and in particular, to a system,
`method, and computer-readable code for rendering digital
`content using time offsets.
`
`BACKGROUND OF THE INVENTION
`
`[0003] Traditionally,
`two different
`there have been
`approaches for delivering digital audio data. In the first
`approach, the digital audio data is mass downloaded. More
`specifically, and as shown schematically in FIG. 1, one or
`more files corresponding to an entire audio stream 10 is
`transmitted one frame 12 at a time from the server to the
`client. Once the entire audio stream 10 has been received and
`reassembled to form a continuous, contiguous audio stream,
`it is stored in storage 14 prior to being transmitted to a media
`player 16.
`[0004]
`In the second approach, streaming technology is
`used to deliver the digital audio data 'just-in-time'. More
`specifically, and as shown schematically in FIG. 2, an entire
`audio stream 20 is transmitted one frame 22 at a time from the
`server to the client, where it is received and reassembled, in
`part, to provide a continuous, contiguous audio stream (i.e., a
`small portion of audio stream continuity is preserved). Once
`each frame is played by the media player 26, it is then dis(cid:173)
`carded from the buffer.
`[0005] Small audio streams, or audio-video streams, which
`for example correspond to individual songs, very short mov(cid:173)
`ies, and music videos, are typically transmitted using the first
`approach. In terms of the delivery of these smaller streams of
`media, the delays experienced by the users are generally
`tolerated because they are relatively short in nature. Typically,
`time delays are measured as one or two minutes, and although
`possible, tend not to exceed this.
`[0006] The delivery of larger audio streams, which for
`example include books and radio shows, presents a problem
`for the user community. Whereas a single song that plays for
`4 minutes may take 1 minute to download, an audio book that
`plays for 12 hours may take 3 to 4 hours to download.
`Although the general performance is relatively the same in
`terms of throughput rate, users of this media complain about
`the hours of waiting to receive and use the media selected.
`[0007] While streaming technology obviates the waiting
`associated with mass download, any degradation experienced
`in the delivery of the content in real time introduces interrup(cid:173)
`tions in the audio stream, causing breaks and interruptions in
`the users experience of that audio stream. Moreover, since the
`digital audio data is not stored, repositioning within the audio
`stream ( e.g., using rewind or fast forward functions) inter-
`
`rupts the just-in-time nature of content delivery, and thus,
`may introduce significant delays and/or be inefficient. For
`example, in the case of rewinding a streamed audio stream,
`the content associated with the new position selected in the
`audio stream will need to be downloaded a second time and
`the future content temporarily stored in the buffer will be
`discarded.
`[0008]
`In both of these existing technologies, great effort is
`made to reassemble the audio stream into a continuous, con(cid:173)
`tiguous audio stream prior to being presented to the media
`player. In the case of the mass download approach, the entire
`audio stream is downloaded and reassembled prior to use. In
`streaming technologies, a very small portion of the audio
`stream is downloaded and reassembled prior to use with
`additional content delivered and already played content dis(cid:173)
`carded continuously, to maintain a very small portion of con(cid:173)
`tinuity in the audio stream. Notably, this reconstruction of the
`audio stream complicates the digital audio data delivery and
`increases delivery time.
`[0009]
`In addition, in both of the existing technologies, the
`user has limited tracking options. For example, 'The Godfa(cid:173)
`ther' is an audio book that, as commercially released, contains
`24 MP3 files that require 80 megabytes of storage and plays at
`normal speed for a total of almost 9 hours. In order to use
`these files with existing mass download technology, the user
`must manually keep track of which file is currently being
`listened to and where one is in that particular file.
`[001 OJ Tracking problems also develop if the users audio
`player automatically changes files, if the user is listening to
`multiple audio streams and/or if the user listens to audio
`streams on more than one client device (e.g. if a user is
`listening to the audio stream at work and wants to resume play
`at home). It can be particularly difficult and time consuming
`for the user to resume listening to an audio stream at a specific
`position.
`[0011]
`In addition, tracking problems will develop during
`cross-media switching (e.g. between audio and text). For
`example, if a user is listening to an e-book in the car and later
`wants to start reading thee-book at home, tracking problems
`will develop because the granularity provided by the prior art
`methods is insufficient to tie the audio stream to the corre(cid:173)
`sponding eText at a letter or word level. Further complica(cid:173)
`tions are expected when the cross media switching is not on a
`single platform (e.g., a single computer), but rather is on
`multiple platforms ( e.g., between a computer and a cell phone
`or c-Reader).
`
`SUMMARY OF THE INVENTION
`
`[0012] According to one embodiment of the instant inven(cid:173)
`tion, some of the above-described disadvantages are obviated
`by segmenting an audio stream into a plurality of small digital
`audio files using gaps in the natural language of the audio
`stream. These small digital audio files arc transmitted, loaded,
`and played, in a specific order, such that from the user's
`perspective, the audio stream is reproduced in an apparently
`seamless manner. Advantageously, this is done without reas(cid:173)
`sembling the audio stream, either in whole or in part. Further
`advantageously, since the small digital audio files are created
`using natural language gaps, they can be sufficiently small to
`ensure that a first small digital audio file is downloaded and
`played without significant delay, while successive small digi(cid:173)
`tal audio files are downloaded to be played in the future.
`Accordingly, the user receives the audio-on-demand in a
`timely manner.
`
`-30-
`
`

`

`US 2012/0084455 Al
`
`Apr. 5, 2012
`
`2
`
`[0013] According to another embodiment of the instant
`invention, some of the above-described disadvantages are
`obviated by providing a virtual audio stream descriptor,
`which includes a record of the position of each small digital
`audio file in the audio stream, to increase tracking options.
`More specifically, the virtual audio stream descriptor and one
`or more predetermined time offsets into the audio stream are
`used to position or reposition the audio stream at will. The
`predetermined time offsets are typically provided via internal
`media marks, external media marks, and/or rewind/fast-for(cid:173)
`ward functions.
`[0014] According to another embodiment of the instant
`invention, some of the above-described disadvantages are
`obviated by using a virtual media descriptor, which defines a
`plurality of digital media files in terms of time relative to a
`timeline of a complete audio recording, to tie the narrated text
`to the corresponding eText. Advantageously, using the con(cid:173)
`tinuous timeline of the complete audio recording allows cross
`media switching in a seamless manner.
`[0015]
`In accordance with one aspect of the instant inven(cid:173)
`tion there is provided a method of rendering digital content
`comprising: providing a media player having access to at least
`one server via a network, the at least one server having stored
`thereon a descriptor file and a plurali,y of digital data files
`derived from all originating work, the descriptor file includ(cid:173)
`ing time information for defining each digital data file, the
`time information determined relative to a timeline of an audio
`recording of the originating work; selecting the digital con(cid:173)
`tent to be rendered from the plurality of digital data files using
`the time information in the descriptor file and a time offset
`external to the descriptor file, the time offset determined
`relative to the timeline of the audio recording; rendering the
`selected digital content using the media player.
`[0016]
`In accordance with one aspect of the instant inven(cid:173)
`tion there is provided a method of rendering digital content
`comprising: providing a media player having access to at least
`one server via a network, the at least one server having stored
`thereon a descriptor file and a plurality of digital data files
`derived from an originating work, the descriptor file includ(cid:173)
`ing time information for defining each digital data file, the
`time information determined relative to a timeline of an audio
`recording of the originating work; selecting the digital con(cid:173)
`tent to be rendered from the plurality of digital data files using
`)le time information in the descriptor file and a time offset
`external to the descriptor file, the time offset determined
`relative to the timeline of the audio recording; and rendering
`the selected digital content using the media player, wherein
`the digital data includes at least one electronic text file and at
`least one digital audio file, and wherein rendering the selected
`digital content using the media player comprises providing
`automated page turns for text derived from the at least one
`electronic text file, the automated page turns determined in
`dependence upon a progressing time offset relative to the
`rendering of the at least one digital audio file.
`[0017]
`In accordance with another aspect of the instant
`invention there is provided a system for rendering digital
`content comprising: a server having stored thereon digital
`data corresponding to an originating work and a descriptor
`file, the descriptor file including time information for defining
`the digital data, the time information determined relative to a
`timeline of an audio recording of the originating work; a
`client for rendering the digital data corresponding to the origi(cid:173)
`nating work; and a network for connecting the server and the
`client and facilitating transmission of the digital data from the
`
`server to the client, wherein the client renders the digital
`content using the time information in the descriptor file and a
`time offset external to the descriptor file, the time offset
`determined relative to the timeline of the audio recording.
`[0018]
`In accordance with another aspect of the instant
`invention there is provided a computer readable storage
`medium including computer readable code, which when
`executed by a computer, causes said computer to: download
`digital content from at least one server, the at least one server
`having stored thereon a descriptor file and a plurality of
`digital data files derived from an originating work, the
`descriptor file including time information for defining each
`digital data file, the time information determined relative to a
`timeline of an audio recording of the originating work, the
`computer having access to the at least one server via a net(cid:173)
`work; select a point within the digital content from which to
`begin rendering, the point selected using the time information
`in the descriptor file and a time offset external to the descrip(cid:173)
`tor file, the time offset determined relative to the timeline of
`the audio recording; and begin rendering the digital content
`from the selected point.
`[0019]
`In accordance with another aspect of the instant
`invention there is provided a computer readable storage
`medium including computer readable code, which when
`executed by a computer, causes said computer to: render
`digital content, the computer having access to at least one
`server via a network, the at least one server having stored
`thereon digital data corresponding to an originating work and
`a descriptor file, the descriptor file including time information
`for defining the digital data, the time information determined
`relative to a timeline of an audio recording of the originating
`work, the digital data including the digital content; determine
`a current time offset while rendering the digital content; and
`create a bookmark for a selected point in the rendered digital
`content, the bookmark an electronic file including the current
`time offset and an identification of the descriptor file, the
`current time offset being a time difference from a beginning
`of the audio recording to a selected point in the audio record(cid:173)
`ing, the selected point in the audio recording corresponding to
`the selected point in the digital content.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0020] Further features and advantages of the present
`invention will become apparent from the following detailed
`description, taken in combination with the appended draw(cid:173)
`ings, in which:
`[0021] FIG. 1 is a schematic diagram showing the prior art
`mass download of an audio stream;
`[0022] FIG. 2 is a schematic diagram showing prior art
`streaming of an audio stream;
`[0023] FIG. 3 is a schematic diagram illustrating the trans(cid:173)
`mission of an audio stream in accordance with one embodi(cid:173)
`ment of the instant invention;
`[0024] FIG. 4 is a schematic diagram of one embodiment of
`a network based library;
`[0025] FIG. Sa shows an embodiment of an actual audio
`stream structure;
`[0026] FIG. Sb shows an embodiment of a small digital
`audio file structure;
`[0027] FIG. Sc shows an embodiment of a virtual audio
`stream descriptor structure;
`[0028] FIG. Sd shows an embodiment of an illustration
`structure;
`
`-31-
`
`

`

`US 2012/0084455 Al
`
`Apr. 5, 2012
`
`3
`
`[0029] FIG. Se shows an embodiment of an advertising
`structure;
`[0030] FIG. 5/ shows an embodiment of a catalog index
`structure;
`[0031] FIG. Sg shows an embodiment of a server list struc(cid:173)
`ture;
`[0032] FIG. 6 is a schematic diagram showing card catalog
`index structure relationships;
`[0033] FIG. 7 is a schematic diagram illustrating one
`embodiment of a library creation process;
`[0034] FIG. 8 is a schematic diagram illustrating one
`embodiment of an audio stream splitter process;
`[0035] FIG. 9 shows an embodiment of a bookmark struc(cid:173)
`ture;
`[0036] FIG. 10 is a schematic diagram showing virtual
`audio stream structure relationships;
`[0037] FIG. 11 is a schematic diagram illustrating one
`embodiment of a performance management process;
`[0038] FIG. 12 is a schematic diagram illustrating informa(cid:173)
`tion transfer and client memory status;
`[0039] FIG. 13 is a schematic diagram illustrating one
`embodiment of a bookmarked audio stream purge process;
`[0040] FIG. 14 is a schematic diagram illustrating one
`embodiment of an active audio stream purging process;
`[0041] FIG. 15 is a schematic diagram illustrating one
`embodiment of a memory manager process;
`[0042] FIG. 16 is a schematic diagram illustrating one
`embodiment of a download manager process;
`[0043] FIG. 17 is a flow diagram illustrating one embodi(cid:173)
`ment of a process for using the software product;
`[0044] FIG. 18 is a flow diagram illustrating one embodi(cid:173)
`ment of the player control process;
`[0045] FIG. 19 is a flow diagram illustrating one embodi(cid:173)
`ment of the general functional process;
`[0046] FIG. 20 is a schematic diagram illustrating how the
`Virtual Media Descriptor 224 links the various media
`streams;
`[0047] FIG. 21 is a schematic diagram illustrating how each
`Derivative Work is derived from the same Originating Work;
`[0048] FIG. 22 is a schematic diagram illustrating the struc(cid:173)
`ture of one embodiment of a Virtual Media Descriptor;
`[0049] FIG. 23 is a schematic diagram illustrating how
`common points corresponding to a specific point in an Origi(cid:173)
`nating Work will have different time values in different audio
`recordings;
`[0050] FIG. 24a is a schematic diagram illustrating a sys(cid:173)
`tem in accordance with one embodiment of the instant inven(cid:173)
`tion;
`[0051] FIG. 24b is a flow diagram illustrating a process of
`providing a time offset using the time offset manager;
`[0052] FIG. 25 is a schematic diagram illustrating the seg(cid:173)
`mentation of an eText according to one embodiment of the
`instant invention;
`[0053] FIG. 26 is a schematic diagram showing a plurality
`of eText segments, which when accessed in their logical
`order, form a discontinuous media stream defined by the
`Virtual eText Stream;
`[0054] FIG. 27a shows an embodiment of an eBook struc(cid:173)
`ture;
`[ 0055] FIG. 27 b shows an embodiment of an eBook chapter
`structure;
`[0056] FIG. 27c shows an embodiment of an eBook para(cid:173)
`graph structure;
`
`[0057] FIG. 28 is a schematic diagram showing the loading
`and processing of eText segments in a formatted memory
`structure;
`[0058] FIG. 29 shows an embodiment of a time tag;
`[0059] FIG. 30 is a schematic diagram illustrating the inser(cid:173)
`tion of hard time tags into formatted memory structures;
`[0060] FIG. 31 shows an embodiment of a Virtual Page
`structure; and
`[0061] FIG. 32 is a schematic diagram illustrating the cor(cid:173)
`relation between an eText Stream and two Derivative Works.
`[0062]
`It will be noted that throughout the appended draw(cid:173)
`ings, like features are identified by like reference numerals.
`
`DETAILED DESCRIPTION OF PREFERRED
`EMBODIMENTS
`
`[0063] Referring to FIG. 3, there is shown a system for
`transmitting digital audio data in accordance with one
`embodiment of the instant invention. The system includes a
`server 100, a client 150, and a network (not shown) for con(cid:173)
`necting the server 100 and the clie