(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2003/0066078A1
`Bjorgan et al.
`(43) Pub. Date:
`Apr. 3, 2003
`
`US 2003OO66078A1
`
`(54) SUBSCRIBER INTERFACE DEVICE FOR
`USE WITH AN INTELLIGENT
`CONTENT BROADCAST NETWORK AND
`METHOD OF OPERATING THE SAME
`
`Related U.S. Application Data
`(60) Provisional application No. 60/285,392, filed on Apr.
`20, 2001.
`
`Publication Classification
`(51) Int. Cl. ................................................. H04N 7/025
`(52) U.S. Cl. ................................................................ 725/34
`
`(75) Inventors: Stephen Bjorgan, San Francisco, CA
`(US); Robert Noakes, Walnut Creek,
`CA (US)
`Correspondence Address:
`DAVIS MUNCK
`900 THREE GALLERATOWER
`13155 NOEL ROAD
`DALLAS, TX 75240 (US)
`9
`
`ABSTRACT
`(57)
`An interface device and method for use with an intelligent
`content-broadcast network. User profile information is col
`lected and Stored in a memory device assessable to a
`Secondary content processor. Secondary content is informa
`tion, Such as advertisements from a television network
`Source, that is available for insertion into the primary
`(73) Assignee: FRANCE TELECOM RESEARCH broadcast Stream as appropriate. When a particular user is
`AND DEVELOPMENT L.L.C., 1000
`identified as viewing a primary-content program, a proces
`Marina Boulevard, Suite 300, Brisbane,
`Sor associates the user with a Stored profile and uses the
`CA 94005
`profile information to determine what, if any, Secondary
`content should be inserted and thus presented to the user as
`if it had been a part of the broadcast stream. The portion of
`the primary broadcast Stream where Secondary content may
`be inserted is indicated by embedded cues.
`
`(21) Appl. No.:
`
`10/126,502
`
`(22) Filed:
`
`Apr. 19, 2002
`
`300
`TUNER | -- WSBDEMODULATOR -
`
`305
`
`310
`:
`
`DEMULTIPLEXER M
`
`CONTROLS
`
`CONTROLS
`CONTROLS
`
`220
`-
`USERF-29
`um- ALTERNATIVE IMAGE
`WDEO PROCESSOR
`a VIDEO DATA
`APPLICATIONS
`- 30
`ECG MODULE
`CCMODULE
`EDING MODULE
`
`é-32
`
`CONTROLS
`
`VIDEO
`P DECODER
`
`WDEO
`DATA
`
`AUDIO
`DATA
`
`AUDIO
`DECODER
`A.
`
`33O
`
`m
`
`315
`
`AUDIO CONTROLS
`- - -->
`
`
`
`CONTROL
`SOFTWARE
`
`27
`
`SEANG 3
`
`GRAPHCS OVERLAY
`| - - - - ie
`
`coNTROLS
`
`SPLAY
`D
`
`prSEEssor
`
`AMPFER
`
`M
`
`A.
`
`/?
`INPUT
`DEVICES
`|
`365
`
`RAM ROM
`- - - a
`350
`360
`
`MODEM
`E -
`345
`
`-VOLATILE
`SSEE
`A
`355
`
`320
`
`335
`
`Y
`DSPLAY
`- A
`325
`
`M
`
`w
`340
`
`105
`
`

`

`Patent Application Publication
`
`Apr. 3, 2003 Sheet 1 of 7
`
`US 2003/0066078A1
`
`
`
`DIGITAL
`TELEVISION
`
`
`
`é-105
`RECEIVER
`
`100
`
`
`
`
`
`
`
`DIGITAL
`TRANSMTTER
`
`FIGURE 1
`
`

`

`Patent Application Publication
`
`Apr. 3, 2003 Sheet 2 of 7
`
`US 2003/0066078A1
`
`101
`Y.
`
`VCR —
`
`170 ->
`
`150
`
`O
`
`FROM CABLE
`OR SATELLITE
`
`
`
`SET-TOP BOX
`
`160-PCR)
`
`FROM
`ANTENNA
`
`FIGURE 2
`
`

`

`Patent Application Publication
`
`Apr. 3, 2003 Sheet 3 of 7
`
`US 2003/0066078A1
`
`REMOTE
`CONTROL
`
`
`
`
`
`MEMORY
`
`
`
`107
`
`INPUT
`BUFFER
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`PLAYBACK
`CONTROLLER
`
`242
`
`240
`
`COMMERCIAL
`DETECTION
`CONTROLLER
`
`OUTPUT
`BUFFER
`
`OUTPUT
`VIDEO
`STREAM
`
`WDEO
`PROCESSOR
`
`REPLACEMENT
`COMMERCIAL
`SOURCE
`
`
`
`
`
`DIGITAL
`TELEVISION
`4.
`101
`
`-
`
`FIGURE 3
`
`

`

`Patent Application Publication
`
`Apr. 3, 2003 Sheet 4 0f 7
`
`US 2003/0066078 A1
`
`
`
`
`
`
`
`
`
`ovm
`
`we
`
`
`
`
`o_a:<owns‘,
`$885$5085
`
`08EnAlli“\40528
`
`aH905on0.9.2,
`<20<29mmH:mum:
`
`
`mmmjasz><._n_w_o205200;ozExEo
`,Rmm<>>E0m
`
`
`mwxflgbgmolmoPfinoozmomw>T‘523dfl.,,
`mowwmooE555
`8m2mmom\
`_tLEEme_>vv
`
`
`
`><Em>omofnzmoMlll
`
`
`mos):m>i<zmm5<Hmommmoomm89>
`205on_,
`
`
`
`om$1wzo:<oi&<
`
`
`5:85.oonm
`
`J»‘
`
`i3.50.2mom.
`05289>305200mm£8028
`
`
`
`30520;8N205200
`
`
`
`
`
`m88...”,.nAmm
`Zima8nownSn.z_eL\5.0m2%;\K
`
`
`
`
`
`AVm.E<._o>-zozsages.
`
`
`mwéokwA
`AwAv
`
`
`
`,,mmoSmo
`
`
`
`mum
`
`
`
`mammum
`
`mom
`
`TTI-1027, Page 5
`
`
`

`

`Patent Application Publication
`
`Apr. 3, 2003 Sheet 5 of 7
`
`US 2003/0066078A1
`
`START
`
`t
`DIGITAL TELEVISION 100 CONVENTIONALLY PROCESSES RECEIVED
`DIGITAL CONTENT STREAMS, CONVERTING THE SAME TO SEQUENCES
`OF SYNCHRONIZED WIDEO AND AUDIO FRAMES FOR DISPLAY, EITHER
`CONTEMPORANEOUSLY OR ON ATME-SHIFTED BASIS.
`
`?
`
`400
`
`N RESPONSE TO A DETERMINATION THAT THE SYNCHRONIZED
`FRAMES ARE TO BESTORED FORTIME-SHIFTED DISPLAY,
`COMMERCIAL DETECTION CONTROLLER 241 IS OPERABLE TO
`DETECT A SUBSET OF VIDEO FRAMES ASSOCATED WITH A
`COMMERCIAL ASSOCATED WITH THE RECEIVED DIGITAL CONTENT
`STREAM
`
`-
`
`l 41 O
`
`- 2
`
`-
`
`COMMERCIAL DETECTION CONTROLLER 241 IS FURTHER OPERABLE
`TO ASSOCATE, IN RESPONSE TO DETECTING THE SUBSET OF
`<- 415
`VIDEO FRAMES, A CUE WITHAT LEAST ONE OF THE SUBSET OF
`VIDEO FRAMES.
`
`V
`
`FIGURE 5
`
`

`

`Patent Application Publication
`
`Apr. 3, 2003 Sheet 6 of 7
`
`US 2003/0066078A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`COMPLE
`USER
`PROFILE
`
`STORE
`USER
`PROFILE
`
`
`
`
`
`
`
`RECEIVE
`PROFILE
`MODIFICATION
`
`RE-COMPLE
`USER
`PROFILE
`
`RECEIVE
`USERD
`
`
`
`MONITOR
`FOR CLUES
`
`530
`
`535
`
`545
`
`
`
`<- 525
`
`DETERMNE
`CONTENT
`
`
`
`
`
`
`
`
`
`RECORD
`KEEPNG
`FUNCTIONS
`
`
`
`FIGURE 6
`
`

`

`Patent Application Publication
`
`Apr. 3, 2003 Sheet 7 of 7
`
`US 2003/0066078A1
`
`-
`
`-
`
`-
`USER PROFILE
`
`USERID
`
`AGE:
`SEX: M F
`
`PROGRAMMING PREFERENCES:
`
`SPORTS
`DRAMA
`ROMANCE
`ACTION
`
`RELIGIOUS
`MUSIC
`GAME SHOW
`
`COMMERCIAL PREFERENCES:
`
`TELEVISION
`MUSIC
`MOTION PICTURES
`SPORTS
`
`CONSUMER PRODUCTS
`FOOD/DINING
`PETS
`AUTOMOBILES
`
`L
`
`4.
`106
`
`FIGURE 7
`
`

`

`US 2003/0066078A1
`
`Apr. 3, 2003
`
`SUBSCRIBER INTERFACE DEVICE FOR USE
`WITH AN INTELLIGENT CONTENT-BROADCAST
`NETWORKAND METHOD OF OPERATING THE
`SAME
`
`PRIORITY CLAIM TO PROVISIONAL PATENT
`APPLICATION
`0001. The present invention claims priority to U.S. Pro
`visional Application Serial No. 60/285,392 filed on Apr. 20,
`2001. The disclosures of this related provisional patent
`application is incorporated herein by reference for all pur
`poses as if fully set forth herein.
`
`CROSS REFERENCE TO RELATED PATENT
`DOCUMENTS
`0002 The present invention is related to that disclosed in
`U.S. patent application Ser. No.
`and
`(Atty.
`Docket No. FRANO1-00003), filed concurrently herewith on
`Apr. 19, 2002, entitled “SYSTEMS FOR SELECTIVELY
`ASSOCIATING CUES WITH STORED VIDEO FRAMES
`AND METHODS OF OPERATING THE SAME”, which is
`assigned to the assignee of the present invention. The
`disclosure of this related patent application is incorporated
`herein by reference for all purposes as if fully set forth
`herein.
`
`TECHNICAL FIELD OF THE INVENTION
`0003. The present invention is directed, in general, to mix
`Internet/broadcast Systems and, more specifically, to a
`device for Subscriber use in Selectively interfacing with
`System platforms for associating cues with Stored video
`frames.
`
`BACKGROUND OF THE INVENTION
`0004. In conventional television (“TV") systems, televi
`Sion commercials may be inserted into a Video program
`Stream at a television broadcast facility. The Video program
`Stream, including the inserted commercials, is then trans
`mitted wirelessly or by a cable system to all television
`receivers within range of the Station. However, inserting
`commercials at a particular broadcast facility has certain
`drawbacks. Some very large television Stations that are
`centered in certain home markets, Such as New York,
`Chicago, or Atlanta, are also available in other cities outside
`the home markets. However, those video Streams broadcast
`from those Stations contain local advertising Specific to the
`home market. This advertising is of little value outside the
`home market.
`0005 There are well-known video processing systems
`that replace an original commercial advertisement in a
`broadcast television Signal with a Substitute commercial
`advertisement. The Substitute commercial is targeted to a
`Specific demographic group that is likely to find the Substi
`tute commercial of greater value the original commercial
`advertisement. The replacement can be done at various
`points of the delivery chain, for example, at local affiliates
`of a network.
`0006 There are other reasons for replacing commercials
`in a Video Stream. One primary reason is the rebroadcast of
`television programs containing dated commercials. Many
`commercials are of a time-limited nature. A commercial
`advertisement for the 2001 World Series that occurs during
`
`a television situation comedy (or sitcom) is of little value if
`the sitcom is rebroadcast in 2002, 2003, or later. Similarly,
`a commercial may announce that a product is offered for Sale
`at particular price in 2001. This price is unlikely to be valid
`in Subsequent years. Moreover, when the commercial is
`rebroadcast, the product may no longer in production or the
`manufacturer may no longer be in business, or both. Even in
`the short run, a commercial that is tied to a particular event
`is of little value after the event has occurred. Hence, com
`mercials for Mother's Day sales and Christmas sales are of
`no value the day after Mother's Day and Christmas, respec
`tively.
`0007. This time-limited nature of commercials is a par
`ticularly important drawback with respect to home recording
`of television programs. A viewer may use a Video-cassette
`recorder (VCR) or a video disk recorder (VDR) to record a
`broadcast television program that the viewer views at a later
`time. This time-shifted Viewing provides great convenience
`to consumers, but often renderS recorded commercials use
`less. If a viewer records a television program the week
`before Thanksgiving and watches it the week after Thanks
`giving, all recorded commercials related to Thanksgiving,
`including Sales offers that expire on Thanksgiving, are
`essentially useleSS.
`0008 Moreover, commercials are often, if not usually,
`directed to a particular type of audience. While Some may
`have universal appeal, more frequently both the type of
`product being advertised, and the way in which the message
`is conveyed, are calculated to resonate with a particular
`demographic group. Beer and wine commercials, for
`example, are targeted at an audience old enough to enjoy
`these products, while advertisements for toys and games are
`often designed to appeal to young viewers. Fast-food res
`taurants may want to appeal to both groups but use different
`messages for doing So. For example, young purchaserS may
`be attracted by exciting, fast-moving offers of free toys
`accompanied by a modern music medley, while an appeal to
`their parents might include a thoughtful demonstration of the
`healthy food choices that are available at the restaurant,
`perhaps with nature Sounds audible in the background.
`Audience targeting, of course, can currently be performed
`by matching, as closely as possible, the types of commer
`cials with the types of programs they accompany. But
`predicting the demographic profile of the viewers of a
`certain program and what ads will appeal to them can
`Sometimes be difficult, at least until a good deal of historical
`evidence has been gathered. And the time-shifting phenom
`enon, described above, may not only result in anachronous
`commercial messages, but may also result in a viewing
`audience that is different from one originally predicted.
`0009 People vary too, of course, and often have needs
`and interests that are not normally associated with the
`demographic profile into which they happen to fall. Such an
`audience may be particularly difficult to target, Since there
`are few accurate ways, if any, to predict their viewing habits
`or the type of advertising that will appeal to them.
`0010. Therefore, there exists a need in the art for
`improved Video processing Systems that enable providers of
`broadcast Video and other programming to more accurately
`target Secondary content toward the current needs of the
`Specific audience viewing their primary content program
`ming at any given time.
`
`

`

`US 2003/0066078A1
`
`Apr. 3, 2003
`
`SUMMARY OF THE INVENTION
`0.011) To address the above-discussed deficiencies of the
`prior art, it is a primary object of the present invention to
`provide, for use in a digital broadcast environment, a device
`for interfacing with Systems and System platforms for cueing
`and intelligently inserting content data into a digital content
`Stream comprising video track data, audio track data and
`content descriptor data, and methods of operating the same.
`0012. According to an advantageous embodiment of one
`invention hereof, intelligent Systems are provided for insert
`ing cues into the broadcast content Stream. One important
`aspect of this embodiment is the use of a cue to insert
`“secondary” content into a “primary' content stream. There
`is disclosed a device for interfacing with Systems and System
`platforms for Selectively associating cues with Stored Video
`frames according to a user profile compiled for this purpose.
`According to one embodiment, an exemplary Video process
`ing System comprises a Video recording System and a video
`playback System that can also Select alternate Secondary
`content based on a Stored user profile.
`0013 The video recording system is capable of (i) receiv
`ing a digital content Stream comprising Video track data,
`audio track data and content descriptor data and (ii) storing
`the Video track data as Video frames in a storage device. The
`Video recording System comprises a commercial detection
`controller operable, in response to the received content
`descriptor data, to (i) detect a Subset of the video frames
`asSociated with a commercial in the received digital content
`stream, and (ii) associate a cue with at least one of the Subset
`of the video frames.
`0.014. The video playback system is capable of playing
`back the Stored Video track data, and comprises a Video
`playback controller operable to detect cues associated with
`ones of the Subsets of the video frames and to direct the
`Video playback System in response thereto to Selectively
`replace the commercial with a Substitute commercial during
`playback based on the Stored user profile or profiles asso
`ciated with the viewers that are present at that particular
`time.
`0.015 Embedding cues in digital content, or media
`Streams facilitates the creation of applications that are
`capable of receiving and processing one or more digital
`content Streams. Controllers implementing these media pro
`cessing applications may Suitably be implemented at a video
`processing System (e.g., television, set top box, Video cas
`Sette recorder, video storage System, etc.), or at a network
`intermediary (e.g., gateway, proxy, etc.).
`0016 Exemplary “cues,” as the term is broadly used
`herein are machine-readable binary codes. Preferably, the
`cues are variable in size, format and density of information
`provided, and may Suitably be used to formed a matrix of at
`least one dimension having data contained therein. An
`important aspect of alternate embodiments hereof is that the
`format of a given cue as well as the position of insertion into
`a content Stream may Suitably be based upon the particular
`application being executed. Stated differently, a cue may be
`any Suitably arranged binary code that is in the form of a data
`packet or a portion thereof (e.g., header-based, data-based,
`etc.).
`0.017. Another important aspect of this invention is the
`use of a cue to Selectively replace a commercial within a
`
`Subset of the video frames with a Substitute commercial
`during playback. This may Suitably be to replace an
`“expired” commercial or, alternatively, to present a "tar
`geted” commercial, Such targeting possibly based, at least in
`part, upon the economic characteristics of a particular geo
`graphic location, possibly identified by Zip code, though
`other measurable characteristics concerning the viewing/
`listening audience, including demographics data (e.g., age,
`Sex, income, etc.), usage patterns, dynamic and/or static
`profiles, purchase history and the like, all of which are
`within the scope of this invention.
`0018 User profiles, that is, any system-stored data relat
`ing to the audience that is or is likely to be viewing or
`listening at a given time, may be Stored at any one or more
`of Several System components, including a remote control, a
`Set top box, a personal digital assistant (PDA), or in a control
`data base accessible through an electronic communications
`netWare.
`0019. The foregoing has outlined rather broadly the fea
`tures and technical advantages of the present inventions. So
`that those skilled in the art may better understand the
`detailed description that follows. Additional features and
`advantages of the inventions will be described hereinafter.
`Those skilled in the art should appreciate that they may
`readily use the conception and the Specific embodiment
`disclosed as a basis for modifying or designing other Struc
`tures for carrying out the same purposes of the present
`inventions. Those skilled in the art should also realize that
`Such equivalent constructions do not depart from the Spirit
`and Scope of the invention in its broadest form.
`0020. Before undertaking the detailed description, it may
`be advantageous to Set forth definitions of certain words and
`phrases used throughout this patent document: the terms
`“include” and “comprise,” as well as derivatives thereof,
`mean inclusion without limitation; the term “or,” is inclu
`Sive, meaning and/or; the phrases "asSociated with and
`“associated therewith,” as well as derivatives thereof, may
`mean to include, be included within, interconnect with,
`contain, be contained within, connect to or with, couple to
`or with, be communicable with, cooperate with, interleave,
`juxtapose, be proximate to, be bound to or with, have, have
`a property of, or the like; and the term “controller” and
`“processor” mean any device, System or part thereof that
`controls at least one operation, Such a device may be
`implemented in hardware, firmware or Software, or Some
`combination of at least two of the same. It should be noted
`that the functionality associated with any particular control
`ler/processor may be centralized or distributed, whether
`locally or remotely. In particular, a controller/processor may
`comprise one or more data processors, and associated input/
`output devices and memory, which execute one or more
`application programs or an operating System program. Defi
`nitions for certain words and phrases are provided through
`out this patent document, those of ordinary skill in the art
`should understand that in many, if not most instances, Such
`definitions apply to prior, as well as future uses of Such
`defined words and phrases.
`
`BRIEF DESCRIPTION
`For a more complete understanding of the present
`0021
`inventions, and the advantages thereof, reference is now
`made to the following descriptions taken in conjunction with
`
`

`

`US 2003/0066078A1
`
`Apr. 3, 2003
`
`the accompanying drawings, wherein like numbers desig
`nate like objects, and in which:
`0022 FIG. 1 illustrates a conceptual block diagram of a
`digital television System according to one embodiment of
`the present invention;
`0023 FIG. 2 illustrates in more detail the components of
`the exemplary digital television system of FIG. 1 that are
`normally located at the Viewing site.
`0024 FIG. 3 illustrates in greater detail selected portions
`of digital system 100 according to the embodiments of
`FIGS. 1 and 2.
`0.025
`FIG. 4 illustrates a block diagram of a digital
`television receiver according to the embodiment of FIGS. 1
`to 3;
`FIG. 5 illustrates a flow diagram of an exemplary
`0.026
`method of operating the digital television according to the
`embodiment of FIGS. 1 to 4; and
`0.027
`FIG. 6 illustrates a flow diagram of an exemplary
`method of creating and applying a user profile according to
`an embodiment of the present invention.
`0028 FIG. 7 is an example of a graphically displayed
`user profile Such as one that might be viewed on a display,
`and that can be used by a viewer to enter or modify user
`profile information.
`
`DETAILED DESCRIPTION
`0029 FIGS. 1 through 7, discussed below, and the
`various embodiments used to describe the principles of the
`present inventions in this patent document are by way of
`illustration only and should not be construed in any way to
`limit the scope of the inventions set forth herein. Those
`skilled in the art will understand that the principles of the
`present inventions may be implemented in any Suitably
`arranged content broadcast System, including for instance,
`television, digital cinema, and radio.
`0030) Note that for convenience, the term “primary con
`tent is frequently used herein to describe the programming,
`Such as Video programming, that the audience is actively
`watching or listening to, while "Secondary content” is fre
`quently used to designate broadcast information that is
`Simply included along with the primary content, Such as
`television commercials. AS is generally understood, this type
`of Secondary content is often inserted on behalf of Sponsors,
`that is, entities paying the broadcaster for this Service. It
`should be apparent to one skilled in the art, however, that
`Secondary broadcast content, that is, content that can be
`inserted into a primary broadcast Stream at places indicated
`by embedded cues, has other uses as well. Therefore, the
`terms are not intended to be limited to only the types of
`content referred to in describing exemplary embodiments.
`0.031
`FIG. 1 illustrates a conceptual block diagram
`showing Selected components of a digital television System
`(generally designated 100) according to one embodiment of
`the present invention. Note that the terms television and
`TV, as used herein, are intended to include both conven
`tional home television, which provides a convenient
`example, and the many variations thereof Such as High
`Definition TV (HDTV) and Internet-transported streaming
`Video. And while Such devices are often used for entertain
`
`ment purposes, they may also serve other functions Such as
`providing educational training material to an audience
`through either a public or a private network. Digital televi
`Sion System 100 is an exemplary Video processing System in
`accordance with the principles of the present invention.
`Digital television system 100 illustratively includes a digital
`television 101, a digital receiver 105, a transmitter 110, a
`transmission medium 115, and a remote interface device
`125.
`0032 Exemplary transmission medium 115 may suitably
`be a coaxial cable, fiber-optic cable, or the like, over which
`digital content Streams may be transmitted by transmitter
`110 to receivers such as digital receiver 105. Transmission
`medium 115 may include a radio frequency (“RF") link
`between at least portions thereof, for instance, content data
`may be transmitted between transmitter 110 and digital
`receiver 105 via an RF link, such as RF link 120. Exemplary
`digital content Streams comprise Video track data, audio
`track data and content descriptor data. Transmitter 110 may
`Suitably be located at a centralized broadcast facility, Such as
`a television Station or Studio, from which the digital content
`Streams may be transmitted to digital television receivers.
`“Broadcast”, as used herein, refers to both public and private
`broadcasts. “Content” refers to the information transmitted
`for viewing or listening, and except where explicitly Stated
`otherwise, includes both the electrical Signals used for
`transmission and the actual audio-visual information as
`perceived by the Viewing audience. An "intelligent content
`broadcast network” is one that broadcasts a plurality of
`content types, such as the primary and Secondary content
`referred to herein, and delineates them (for example, by
`embedded cues) So that a broadcast receiver can manipulate
`the content Stream, for example by Substituting alternative
`Secondary content So that current commercials are displayed
`during the playback of a pre-recorded program.
`0033 FIG. 2 illustrates in more detail the components of
`exemplary digital television system 100 that are more nor
`mally located at the viewing site, Such as one that may be
`used in practicing an embodiment of the present invention.
`Exemplary television 101 comprises television (TV)
`receiver 105, set top box 150 with infrared (IR) detector 160,
`video-cassette recorder (VCR) 170, video display screen
`106, and remote control 125.
`0034). As shown in FIG. 2, television receiver 105 com
`prises Video display Screen 106 for displaying television
`programs, infrared (IR) sensor 107, and a set of manual
`controls 108, as indicated by a surrounding dotted line.
`Manual controls 108 may include, without limitation, a
`power button, a Volume control button, Vertical and hori
`Zontal control buttons and one or more channel Selection
`buttons. Infrared (IR) sensor 107 receives IR control signals
`from hand-held remote control unit 125 that is operated by
`the television viewer. Typically, IR control signals from
`remote control unit 125 that are detected by IR sensor 107
`are processed within television receiver 105 in order to
`change the channel being viewed on Video display Screen
`106, to increase or to decrease the volume, to turn television
`receiver 105 on and off, and the like. Optionally, the IR
`control signals detected by IR sensor 107 may be relayed to
`set top box 150 or VCR 170 in order to control their
`operation. Set top box 150 performs conventional tuning and
`demodulation of incoming Signals from a cable or Satellite
`television program Service provider to produce, for example,
`
`

`

`US 2003/0066078A1
`
`Apr. 3, 2003
`
`a Stream of Moving Picture Experts Group (MPEG)
`encoded digital data from which Video signals may be
`derived.
`0035) In an illustrative embodiment, the video and audio
`track data are encoded, preferably at the centralized broad
`cast facility, prior to transmission. For instance, the audio
`track data may be encoded in AC3 format and the video
`track data may be encoded in Motion Picture Experts Group
`(“MPEG”) coding, and, in particular, MPEG-2 or higher
`format.
`0036) Exemplary video track data comprises a video
`Sequence that, for instance, may include one or more tele
`vision programs and television advertising (i.e., commer
`cials) that are associated with the television programs.
`Exemplary audio track data comprises Sound associated with
`corresponding portions of the Video track data.
`0037 Alternatively, television receiver 105 may directly
`receive an incoming television broadcast Signal from an
`external antenna (not shown). In this alternative embodi
`ment of the invention, television receiver 105 performs
`conventional tuning and demodulation of incoming RF
`Signals received from the antenna to produce, for example,
`a stream of MPEG encoded digital data from which video
`Signals may be derived.
`0.038
`FIG. 3 schematically illustrates selected portions
`of the digital Video processing System of digital television
`101, according to the embodiments of FIGS. 1 and 2.
`Digital television 101 illustratively comprises input buffer
`210, video processor 220, output buffer 230, memory 240,
`and replacement commercial source 250. Note that in an
`alternate embodiment, replacement commercial Source 250
`may actually reside in a component other than digital
`television 101.
`0.039 According to this embodiment, input buffer 210,
`video processor 220, output buffer 230, memory 240, and
`replacement commercial Source 250 cooperate to provide a
`Video recording System in accord with the principles of the
`present invention. The Video recording System is operable to
`(i) receive digital content streams comprising video track
`data, audio track data and content descriptor data, (ii)
`conventionally process the same in Video processor 220 to
`convert (e.g., decode, decompress, translate, etc.) the same
`to a sequence of Synchronized video and audio frames for
`display, and (iii) display the same to a display device using
`output buffer 230 as temporary Storage or, alternatively,
`Store Such processed Sequences of Video/audio frames in a
`Storage device, Such as memory 240, for time shift display.
`0040 Memory 240 stores commercial detection control
`ler 241, which is executable by video processor 220. Exem
`plary commercial detection controller 241 contains, among
`other things, the code for controlling the interactive coop
`eration between a main content Source and a plurality of
`Secondary content Sources. According to an advantageous
`embodiment of the present invention, memory 340 may
`comprise a removable media drive, such as a CD-ROM
`drive and commercial detection controller 241 may be
`instantiated from a removable media disk.
`0041. In the event that processed sequences of video/
`audio frames are to be Stored for time shift display, com
`mercial detection controller 241 is operable, in response to
`received content descriptor data associated there with, to (i)
`
`detect Subsets of the audio or Video frames associated with
`a commercial within the received digital content Stream, and
`(ii) Selectively associate one or more cues with at least one
`of the audio or video frames of the Subset of the same. In
`accordance here with, the cue operable is operable to cause
`a playback controller 242 (also illustratively stored in
`memory 240) to selectively replace the commercial with a
`Substitute commercial during playback of the Stored video
`track data. It is important to note that Video processor 220,
`depending upon its implementation, may Suitably convert
`the digital content Streams a Sequence of Synchronized
`analog video and audio frames, thereby forming an analog
`COntent Stream.
`0042. Again, cues may include clocking data associated
`with the commercial and, according to one advantageous
`embodiment, the clocking data includes at least one of either
`the start time of the commercial or the duration time of the
`commercial. Cues may also include at least one of either
`content indicia associated with the received digital content
`Stream or a Subscriber (user) profile.
`0043. In playback mode, exemplary playback controller
`242 directs video processor 220 to retrieve the stored
`Sequences of Video/audio frames that may Suitably include
`one or more associated cues and to Store the same in output
`buffer 230. More particularly, operating under the control of
`playback controller 242, video processor 220 retrieves each
`frame from memory 240 and parses each of the same for
`asSociated cues. Upon detection of a cue, playback control
`ler 242 processes the same to determine whether any frames
`associated with a commercial should be replaced, to thereby
`Selectively replace certain commercials with Substitute com
`mercials during playback. Selection of Substitute commer
`cials (or other Secondary content) takes into account user
`profiles (described in more detail below) defining certain
`characteristics of the current audience. The user profiles may
`be stored in memory 240, or in a user interface device such
`as remote control 125. When stored in memory 240, remote
`device 125 is used simply to indicate to playback controller
`242 the identity of current audience members, so that their
`profiles can be found or Solicited.
`0044 According to this embodiment, such substitute
`commercials are illustratively provided through replacement
`commercial Source 250. Of course, in alternate embodi
`ments, replacement commercial Source 250 may comprise
`any known Storage device, including a hard disk drive,
`CD-ROM drive, and DVD drive, a VCR tape, or a network
`feed over which another incoming digital content Stream is
`being received.
`0045 Examples of applications that may suitably be
`eased by the embedding of cues include advertising, record
`ing, insertion, Switching, adaptation and the like. Such
`applications typically require relatively exact time Synchro
`nization with arriving media packets to operate correctly.
`Failure to maintain precise time Synchronization (e.g., say
`when Switching between two or more Source streams) may
`result in undesired perceptible artifacts when the resulting
`Stream is rendered. Such time Synchronization between
`applications and media Streams is also required in imple
`mentations where relatively little media packet buffering is
`available at a stream processing point; for example, in
`wireleSS devices and Internet appliances.
`0046) Note that while it is considered advantageous to
`Substitute current Secondary content into cued portions of
`
`

`

`US 2003/0066078A1
`
`Apr. 3, 2003
`
`the primary content Stream, it is not necessary in practicing
`the present invention to do So universally. In fact, in Some
`cases it may be desirable to use “dated' cues that indicate
`Secondary content that should