(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2003/0093790 A1
`Logan et al.
`(43) Pub. Date:
`May 15, 2003
`
`US 20030093790A1
`
`(54) AUDIO AND WIDEO PROGRAM
`RECORDING, EDITING AND PLAYBACK
`SYSTEMS USING METADATA
`(76) Inventors: James D. Logan, Windham, NH (US);
`Scott A. Durgin, North Andover, MA
`(US); Brian D. Doe, Windham, NH
`(US); Vincent E. Colella, Wilmington,
`MA (US); McFarland Hale, North
`Chelmsford, MA (US); Paul M.
`Mansfield, Burlington, MA (US);
`Gregory J. Read, Newbury, MA (US);
`Jeffrey M. Santos, Newburyport, MA
`(US); Michael G. Palone, Acton, MA
`(US); Stephen Boone, Windham, NH
`(US)
`Correspondence Address:
`CHARLES G. CALL
`68 HORSE POND ROAD
`WEST YARMOUTH, MA 02673-2516 (US)
`(21) Appl. No.:
`10/165,587
`(22) Filed:
`Jun. 8, 2002
`Related U.S. Application Data
`(63) Continuation-in-part of application No. 09/536,969,
`filed on Mar. 28, 2000.
`
`(60) Provisional application No. 60/297.204, filed on Jun.
`8, 2001. Provisional application No. 60/352,788, filed
`on Nov. 28, 2001. Provisional application No. 60/304,
`570, filed on Jul. 11, 2001. Provisional application
`No. 60/336,602, filed on Dec. 3, 2001. Provisional
`application No. 60/297.204, filed on Jun. 8, 2001.
`
`Publication Classification
`
`(51) Int. Cl. ............................ G06F 3/00; H04N 5/445;
`G06F 13/00; H04N 7/173;
`
`(52) U.S. Cl. ............................ 725/38; 72.5/134; 725/142;
`725/61; 345/783; 345/845
`
`(57)
`
`ABSTRACT
`
`A System for utilizing metadata created either at a central
`location for shared use by connected users, or at each
`individual user's location, to enhance user's enjoyment of
`available broadcast programming content. A variety of
`mechanisms are employed for automatically and manually
`identifying and designating programming Segments, associ
`ating descriptive metadata which the identified Segments,
`distributing the metadata for use at client locations, and
`using the Supplied metadata to Selectively record and play
`back desired programming.
`
`101
`
`A/
`
`105
`
`111
`
`Create
`Metadata
`
`a
`
`c
`
`a
`
`107
`
`113
`
`w
`Broadcast
`Programming
`Source
`
`103
`
`V
`
`1OO
`
`115
`
`117
`
`a
`User Data
`
`-
`
`13O
`
`Communications
`
`----, ---- A
`
`create f/
`Metadata
`Storage
`
`18O
`
`V 145
`
`
`
`151
`
`161
`
`171
`
`Metadata
`
`135
`
`190
`
`143
`
`-1-
`
`Amazon v. Audio Pod
`US Patent 9,954,922
`Amazon EX-1078
`
`

`

`Patent Application Publication
`
`May 15, 2003 Sheet 1 of 4
`
`US 2003/0093790 A1
`
`/ | |
`
`
`
`| |--- - - -· T T TA
`
`
`
`
`
`
`
`-2-
`
`

`

`Patent Application Publication May 15, 2003 Sheet 2 of 4
`
`US 2003/0093790 A1
`
`Z9Z€\09ZZZZ
`
`C-C
`
`
`
`Ou!!!!!)Eº ->\jº| 0Z
`
`t
`
`-??>[ad]
`
`
`ËË? KÁZ,\boueue ulew
`
`-----------4----
`CN
`cN
`cy)
`
`v
`cy
`
`
`
`eqed uasn
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`CN
`
`N. O) /
`CN O)
`
`CN
`CN
`CN
`
`122 O.)
`i.
`
`| Z
`
`-3-
`
`

`

`Patent Application Publication May 15, 2003 Sheet 3 of 4
`
`US 2003/0093790 A1
`
`
`
`Theme Song
`
`Letter "A"
`
`Hooper's Party
`
`Bert Reads
`
`Number "9"
`
`Theme Song
`
`Letter "A"
`
`Bert Reads
`
`Number "9"
`
`AdBreak
`
`-4-
`
`

`

`Patent Application Publication May 15, 2003 Sheet 4 of 4
`
`US 2003/0093790 A1
`
`
`
`Theme Song
`
`Letter "A"
`
`Hooper's Party
`-
`(Ernie Sings
`Carly Simon
`
`Bert Reads
`
`eam rummermaernmostmas-
`
`Number "9"
`
`(Ernie annoys Bert with his singing of
`^RubberDucky."...--| AdBreak
`510
`
`III
`
`IIT IIIT
`
`III
`
`-5-
`
`

`

`US 2003/0093790 A1
`
`May 15, 2003
`
`AUDIO AND WIDEO PROGRAM RECORDING,
`EDITING AND PLAYBACK SYSTEMS USING
`METADATA
`
`CROSS REFERENCE TO RELATED
`APPLICATIONS
`0001. This application claims the benefit of the filing date
`of the following co-pending applications: U.S. Provisional
`Patent Application Serial No. 60/297.204 filed on Jun. 8,
`2001 entitled “Methods and Apparatus for Navigating Time
`Shifted Television Programming; U.S. Provisional Patent
`Application Serial No. 60/352,788 filed Nov. 28, 2001
`entitled “Methods and Apparatus for Distributing Seg
`mented Television Programming; U.S. Utility patent appli
`cation Ser. No. 09/536,969 filed on Mar. 28, 2000 entitled
`“Systems and Methods for Modifying Broadcast Program
`ming.” U.S. Provisional Application Serial No. 60/304,570
`filed on Jul. 11, 2001 entitled “Audio and Video Program
`Recording, Editing and Playback Systems using Metadata;
`U.S. Provisional Application Serial No. 60/336,602 filed on
`Dec. 3, 2001 entitled “Methods and Apparatus for Auto
`matically Bookmarking Programming Content,” and U.S.
`Utility application Ser. No. 10/060,001 filed on Jan. 29, 2002
`entitled “Audio and Video Program Recording, Editing and
`Playback Systems Using Metadata.” The disclosure of each
`of the foregoing applications is hereby incorporated herein
`by reference.
`
`REFERENCE TO COMPUTER PROGRAM
`LISTING APPENDIX
`0002. A computer program listing appendix is stored on
`each of two duplicate compact disks which accompany this
`Specification. Each disk contains computer program listings
`that illustrate implementations of the invention. The listings
`are recorded as ASCII text in IBM PC/MS DOS compatible
`files which have the names, sizes (in bytes) and creation
`dates listed below:
`
`Size
`(bytes)
`3,123 May 21, 2001
`1,141 May 21, 2001
`65,419 Jun. O5, 2001
`5,228 Jun. 04, 2001
`33,964 Jun. 04, 2001
`6,337 Jun. O1, 2001
`19,459 Jun. 04, 2001
`3,553 May 23, 200
`3,758 May 21, 200
`26,672 Jun. 04, 2001
`6,317 May 29, 200
`29,715 May 21, 200
`7,527 May 21, 200
`1,202 May 21, 200
`5,258 May 21, 200
`2,239 May 21, 200
`7,753 Jun. O1, 2001
`3.259 May 24, 200
`5,771 May 21, 200
`13,602 Jun. 04, 2001
`537 May 21, 200
`10,333 Jun. O5, 2001
`16,693 May 21, 200
`541,696 Jun. O1, 2001
`79,872 Jun. O1, 2001
`
`Date created
`
`Filename
`
`2:08 p ToolBarDig.h
`2:08 p CAnimate.h
`1:13 p CNPDlg.cpp
`O:20 a CNPDlg.h
`0:16 a ConceptDlg.cpp
`0:30 a ConceptDlg.h
`5:37 p CPLDig.cpp
`4:24 a CPLDlg.h
`2:08 p cpp
`5:39 p CSegmentDriver.cpp
`2:04 p CSegmentDriver.h.
`2:08 p dxmplayer.cpp
`2:08 p dxmplayer.h.
`DXMPLayerConstants.h
`2:08
`2:08 p DXMPlayerEventSink.cpp
`2:08 p DXMPlayerEventSink.h
`0:28 a GClient.clw.
`2:14 p GClient.cpp
`2:08 p GClient.dep
`0:21 a GClient.dsp
`2:08 p GClient.dsw
`1:09 p GClient.h.
`2:08 p GClient.mak
`0:24 a GClient.ncb
`0:23 a GClient.opt
`
`-continued
`
`Filename
`
`Date created
`
`0:26 a GClient.plg
`0:46 p GClient.rc
`2:08 p GClientDlg.cpp
`2:08 p GClientDlg.h
`2:08p gmanager.cpp
`2:08 p gmanager.h.
`4:19 a IMediaPlayer.h
`3:56 p InfoDig.cpp
`2:17 p InfoDig.h
`2:08 p ISegmentDriver.h
`6:52 a ISegmentListCtrl.h
`4:22 a MediaPlayer.cpp
`4:19 a MediaPlayer.h.
`0:13 a metadata.cpp
`1:13 p MetaDataDig.cpp
`0:21 a MetaDataDlg.h
`1:31 a NotAuthored.cpp
`6:31 a NotAuthored.h
`3:53 p playlist.cpp
`4:20 a ReadMe.txt
`0:19 a Resource.h
`2:08 p resource.h.eib
`2:08 p resource.h.old
`3:53 p segment.cpp
`1:54 p segment.h
`2:08 p segmentList.cpp
`2:08 p segmentList.h
`1:13 p SegmentListCtrl.cpp
`6:52 a SegmentListCtrl.h
`2:08 p segmentlistctrl.sav
`0:02 a SettingsDlg.cpp
`6:56 a SettingsDig.h
`2:08 p StdAfx.cpp
`2:08 p. StdAfx.h.
`1:33 a ThankYou Dig.cpp
`0:08 a ThankYou Dig.h
`0:25 a ToolBarDlg.cpp
`2:07 p CAnimate.cpp
`3:39 p Test Access.java
`5:16
`ER.TVP
`3:55 p GDBPool.java.txt
`2:16 p GGUPIServlet.java
`5:23 p GPlayListSet.java.txt
`5:26 p GPlayListSetFromCache.java.txt
`5:28 p GPlayListSetFrom Database.java.txt
`5:27 p GPlayListSetFromFile.java.txt
`12:08 p GPostServlet.java
`5:16 p GQuery.java.txt
`12:16 p GQueryServlet.java
`5:05 p GService.java.txt
`12:24 a GTest.java
`2:00 p JDBCConfig.properties
`11:51 a SnoopServlet.java
`12:10 a Copy of TestAccess.java.txt
`1:01 p tvp.php
`7:30 a frames.html
`3:25 p gspot.php
`6:37 a movie2.php
`6:37 a player.js
`4:23 p sreplace.php
`6:04 a file.ph
`
`Size
`(bytes)
`1,704 Jun. 04, 2001
`35,194 Jun. O5, 2001
`4,127 May 21, 2001
`1,353 May 21, 2001
`12,074 May 21, 2001
`3,404 May 21, 2001
`7,706 May 23, 2001
`20,350 Jun. 04, 2001
`3,187 May 24, 2001
`8,988 May 21, 2001
`3,020 May 28, 2001
`10,152 May 23, 2001
`1,894 May 23, 2001
`5,982 Jun. 04, 200
`33,088 Jun. O5, 200
`3,754 Jun. O1, 200
`13,369 Jun. O5, 200
`3.292 May 24, 200
`11,161 Jun. O5, 200
`5,676 May 24, 200
`13,465 Jun. 04, 200
`3,001 May 21, 200
`2,622 May 21, 200
`6,716 Jun. 04, 200
`3,608 May 31, 200
`1,227 May 21, 200
`1,598 May 21, 200
`42,155 Jun. O5, 200
`8,025 May 28, 200
`21,934 May 21, 200
`19,437 Jun. O5, 200
`3,086 May 28, 200
`209 May 21, 200
`1,195 May 21, 200
`8,653 Jun. O5, 200
`2,288 Jun. 04, 200
`11,679 Jun. 04, 200
`662 May 21, 200
`5,993 Apr. 19, 200
`3,060 Mar. 27, 200
`2,043 Apr. 19, 200
`7,307 Apr. 23, 200
`1,472 Apr. 27, 200
`1,189 Apr. 27, 200
`1,207 Apr. 27, 200
`1,187 Apr. 27, 200
`7,337 Apr. 23, 200
`3,592 Apr. 27, 200
`7,308 Apr. 23, 200
`1643 Apr. 19, 200
`825 Apr. 12, 200
`1,132 Apr. 20, 200
`5,604 Apr. 23, 200
`5,874 Apr. 19, 200
`935 May 29, 200
`358 May 25, 200
`12,051 Jun. O5, 2001
`919 May 22, 200
`636 May 22, 200
`4,546 Jun. O1, 2001
`983 May 31, 200
`
`COPYRIGHT STATEMENT
`0003) A portion of the disclosure of this patent document
`contains material which is Subject to copyright protection.
`The copyright owner has no objection to the facsimile
`reproduction by anyone of the patent document or the patent
`disclosure, as it appears in the Patent and Trademark Office
`patent file or records, but otherwise reserves all copyright
`rights whatsoever.
`
`-6-
`
`

`

`US 2003/0093790 A1
`
`May 15, 2003
`
`FIELD OF THE INVENTION
`0004. This invention relates to audio and video program
`reception, Storage, editing, recording and playback Systems
`and more particularly to methods and apparatus for distrib
`uting, recording, organizing and editing metadata that is
`used to Selectively distribute, record, organize, edit and play
`program content.
`BACKGROUND OF THE INVENTION
`0005. Historically, the viewing experience of TV has
`been governed by the content and programming of the
`Service providers, broadcasters and networks who decide
`when programs will be available and their duration. While
`lifestyles have become more complex and the content avail
`able to the viewer has increased, it has become more
`desirable to allow viewers to control this form of entertain
`ment on their own terms. While video cassette recorders
`(VCRs) allow viewers to capture content for future play
`back, the VCR has been plagued with limitations inherent in
`the analog tape media and the difficulty viewers commonly
`experience in programming these devices to record Selected
`future programs.
`0006 The recent advent of digital video recorders
`(DVRs), coupled with more intuitive electronic program
`guides (EPGs) used in popular DVRs, have provided new
`and Simplified recording options for viewers. In addition, as
`a useful byproduct of digital storage, DVRs provide the
`ability to pause, replay and fast-forward the playback of
`time-shifted programming. However, as the number of
`available channels and the volume and diversity of available
`content increases, currently available DVRS and program
`guides will not provide the needed ability to playback and
`scan volumes of stored video with simple controls and with
`minimal knowledge of the available content. Viewers will
`need “more information about the content to help them
`navigate between programs and within a particular program.
`SUMMARY OF THE INVENTION
`0007. In a principle aspect, the present invention takes
`the form of methods and apparatus for Selectively reproduc
`ing recorded Video program Segments retrieved from a mass
`Storage device under the control of playlist metadata which
`identifies a Selected Set of the Stored Segments ant the
`ordered Sequence in which those Segments are to be repro
`duced in the absence of an intervening control command
`from the viewer. The playlist metadata includes a text
`description of each Segment in the Sequence. In response to
`a request from the viewer, an Segment guide listing contain
`ing the text description of each Segment is displayed with the
`text description of the currently playing Segment being
`visually identified on the list. Control means operated by the
`Viewer permit the Viewer to choose a different Segment to be
`Viewed by Selecting the text description of that Segment on
`the displayed indeX listing.
`0008. In accordance with a further feature of the inven
`tion, attribute data is associated with at least Selected ones of
`the Stored Video program Segments and means are provided
`for Selecting and Sorting Stored Segments based on the
`attribute data.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`0009 FIG. 1 is a schematic block diagram that illustrates
`the functional components which are used in a preferred
`
`embodiment of invention and which operate at both a remote
`location and at one of the user locations to implement the
`invention;
`0010 FIG. 2 is a data flow diagram illustrating the
`manner in which Video program content and descriptive
`metadata is transferred between content and Service provid
`erS and a personal Video recording device operated by a
`Viewer in accordance with the invention; and
`0011
`FIGS. 3-5 illustrate screen layout displays illustrat
`ing the manner in which program Segment guides are
`displayed to enable the user to interactively control program
`Segment playback as defined by playlist metadata.
`
`DETAILED DESCRIPTION
`0012 Background
`0013 The present invention belongs to a family of related
`Systems that use metadata to control the playback of broad
`cast programming as disclosed in the previously issued
`patents and previously filed applications Summarized below.
`0014 U.S. Pat. Nos. 5,892,536 and 5,986,692, issued to
`James D. Logan et al. describe Systems which employ
`metadata to Selectively Store, manipulate and playback
`broadcast programming. Some of the novel arrangements
`and features disclosed in those two patents may be Summa
`rized as follows:
`0015 1. A remote editing station, which may be at
`the broadcast facility or at a remote location, clas
`sifies, describes or otherwise identifies individual
`Segments of broadcast programming and sends meta
`data (Sometimes referred to as "markup data”) iden
`tifying and describing those Segments to a remote
`client receiver. For example, the markup data may
`identify individual Segments by Specifying the
`Source and the time of the original broadcast, or by
`Specifying Some other unique characteristic of the
`broadcast Signal. The program Segments may be TV,
`radio, or Internet programs, or portions of programs,
`including individual Songs, advertisements, or
`SCCCS.
`
`0016 2. The communication link used to transmit
`the metadata to the client may take one of Several
`forms, including the Internet, a dialup telephone link,
`the communications pathway used to carry the
`broadcast Signals to the client, or other forms of
`communication used to transport the metadata to the
`client.
`0017 3. At the client receiver, the metadata is used
`to identify particular program Segments that may
`then be manipulated in one or more of a variety of
`ways. For example, the metadata may be used to
`Selectively play back or record particular Segments
`desired by the user; to re-Sequence the identified
`segments into a different time order; to "edit-out”
`undesired portions of identified Segments, to Splice
`new information, Such as computer text or advertis
`ing, into identified Segments for rendering with the
`program materials, or to Substitute different material
`(e.g. dubbing in acceptable audio to replace profanity
`to make programming more acceptable to minors).
`
`-7-
`
`

`

`US 2003/0093790 A1
`
`May 15, 2003
`
`0018 4. The client receives and locally stores
`incoming broadcast programming and uses the
`markup data to identify desired Segments within the
`Stored program materials. The local Storage mecha
`nism may advantageously include means for concur
`rently recording live broadcasting while replaying a
`delayed version of the previously recorded program
`ming as described in U.S. Reissue Patent 36,801
`issued to James D. Logan et al.
`0019 5. The markup data can provide a detailed
`“electronic program guide' to the broadcast pro
`gramming previously received and Stored in a per
`sonal video recorder (PVR) or an audio storage
`device, permitting the user to Selectively play back a
`desired Segment or portion of the programming
`previously recorded.
`0020. 6. The markup data may be used to create a
`recorded collection of desired Segments extracted
`from the buffered broadcast, allowing the desired
`Segments to be saved while the remainder of the
`buffered materials is discarded to conserve recording
`Space.
`0021 7. Special markup signals may be selectively
`sent to individual Subscribers based on his or her
`indicated preferences So that only preferred program
`Segments are identified and processed. For example,
`a Subscriber might request markup data only for
`Sports and news.
`0022 U.S. Pat. No. 6,088,455 issued to James D. Logan
`et al. describes related Systems that use a signal analyzer to
`extract identification Signals from broadcast program Seg
`ments. These identification Signals are then Sent as metadata
`to the client where they are compared with the received
`broadcast Signal to identify desired program Segments. For
`example, a user may specify that she likes Frank Sinatra, in
`which case she is provided with identification signals
`extracted from Sinatra's recordings which may be compared
`with the incoming broadcast programming content to iden
`tify the desired Sinatra music, which is then saved for
`playback when desired.
`0023 U.S. patent application Ser. No. 09/536,696 filed
`by James D. Logan et al. on Mar. 28, 2000 describes further
`Systems that employ metadata for Selectively recording
`and/or reproducing received broadcast programming. The
`implementations disclosed in that application employ:
`0024.
`1. A receiver connected to record incoming
`broadcast Signals and a PC connected to a web server
`via the Internet. A browser program running on the
`PC uses the web interface provided by the web
`Server, Selects Songs of interest, downloads identifi
`cation signals (e.g., extracted feature-sets or signa
`tures) which uniquely identify the content of desired
`program Segments (Songs), which are then selec
`tively saved for reproduction.
`0025 2. A signal processor that identifies character
`istics of the Stored programming (Scene changes,
`voice vs. music, voices of particular people, etc.) that
`can be used to Selectively Store desired program
`ming.
`0026 3. Identification signals derived from received
`broadcast programming at the client produce identi
`
`fication Signals which are Sent to a remote Server
`which compares the received identification Signals
`with a database at the Server and returns attribute
`information to the client to describe recognized
`information. The attribute information can include
`the title of the Segment, the name of the performing
`artist, albums that have a recording of this segment,
`etc.
`0027 4. Program segment files (e.g. songs) in a
`server library that are made available to those client
`locations which demonstrate that they are entitled to
`access the library copy by Sending an identification
`Signal to the Server that is extracted from a copy of
`the desired Segment already in the client's posses
`Sion. Thereafter, a qualified client can obtain the
`authorized copy from the Server from remote loca
`tions. Locally recorded programming can be
`uploaded from a client into the library, and Such
`uploading can be “virtual” (that is, need not actually
`take place) when an equivalent copy of the same
`program Segment is already Stored in the Server
`library.
`0028 U.S. Pat. Nos. 5,271,811, 5,732,216, and 6,199,
`076, and co-pending application Ser. No. 09/782,546 filed
`on Feb. 13, 2001, by James D. Logan et al. describe an audio
`program and message distribution System which incorpo
`rates the following features:
`0029) 1. A host system organizes and transmits pro
`gram Segments to client Subscriber locations.
`0030) 2. A scheduling file of metadata schedules the
`content and Sequence of a playback Session, which
`may then be modified by the user.
`0031 3. The content of the scheduled programming
`is varied in accordance with preferences associated
`with each Subscriber.
`0032 4. Program segments are associated with
`descriptive Subject matter Segments, and the Subject
`matter Segments may be used to generate both text
`and audio cataloging presentations to enable the user
`to more easily identify and Select desirable program
`mIng.
`0033 5. A playback unit at the subscriber location
`reproduces the program Segments received from the
`host and includes mechanisms for interactively navi
`gating among the program Segments.
`0034 6. A usage log is compiled to record the
`Subscriber's use of the available program materials,
`to return data to the host for billing, to adaptively
`modify the Subscriber's preferences based on actual
`usage, and to Send Subscriber-generated comments
`and requests to the host for processing.
`0035 7. Voice input and control mechanisms
`included in the player allow the user to perform
`hands-free navigation of the program materials and
`to dictate comments and messages, which are
`returned to the host for retransmission to other
`Subscribers.
`0036 8. The program segments sent to each Sub
`Scriber may include advertising materials, which the
`user can Selectively play to obtain credits against the
`Subscriber fee.
`
`-8-
`
`

`

`US 2003/0093790 A1
`
`May 15, 2003
`
`0037 9. Parallel audio and text transcript files for at
`least Selected programming enable Subject matter
`Searching and Synchronization of the audio and text
`files.
`0038 10. Speech synthesis may be used to convert
`transcript files into audio format.
`0039 11. Image files may also be transmitted from
`the server for synchronized playback with the audio
`programming. 12. A text transcript including embed
`ded markup flags may be used to provide a pro
`grammed multimedia presentation including spoken
`audio text created by Speech Synthesis Synchronized
`with presentation of images identified by the markup
`tags.
`0040 U.S. Utility application Ser. No. 10/060,001 filed
`on Jan. 29, 2002 entitled “Audio and Video Program
`Recording, Editing and Playback Systems. Using Metadata”
`describes means at the user's location for creating metadata
`which may be used in combination with metadata provided
`by an external Source, for editing metadata in various ways
`at the user's location, for automatically responding to user
`activity to generate new metadata which characterizes the
`user's preferences and which Serves to automatically iden
`tify and describe (or rate) programming segments, and for
`responding in novel ways to the available metadata to
`enhance the utility and enjoyment of available broadcast
`materials. Methods and apparatus are employed for Selec
`tively controlling the presentation of broadcast program
`ming in which a user viewing or listening to broadcast
`programming at a first location may take advantage of the
`insights provided by a different viewer at another location in
`order to control the manner in which Segments of the
`broadcast programming are recorded and/or replayed.
`0041. The disclosure of each of the foregoing patents and
`applications is incorporated herein by reference.
`0042 Architectural Overview
`0043. The methods and apparatus contemplated by the
`present invention facilitate the Selective Storage, organiza
`tion and reproduction (playback) of broadcast programming
`through the use of metadata that identifies and describes
`Segments of that broadcast programming. This metadata can
`be created locally or at a remote Site and transmitted to the
`user's location to enable the user to more effectively manage
`broadcast programming received at the user's location.
`0044 FIG. 1 illustrates in schematic form the manner in
`which information is processed in accordance with the
`invention. As will be described in more detail below, many
`of the structures and functions illustrated in FIG. 1 represent
`alternative or optional functionality that need not be present
`in all implementations of the invention.
`0.045. At the remote location, broadcast programming
`from a source 100 is received at 101 and may be processed
`immediately or saved in a storage unit 103 for later pro
`cessing. The incoming broadcast programming Signals may
`be received as a live public broadcast, or may take the form
`of programming content received prior to the time of its later
`public broadcast. At 105, the incoming broadcast Signals are
`parsed or Subdivided into logically Separate Segments, which
`need not be contiguous and which may be overlapping or
`nested. The individual Segments may be processed imme
`
`diately after they are identified during the parsing process, or
`they may be Stored for future processing in a storage unit
`107.
`0046. As illustrated at 111, metadata is then created
`which describes each of the identified programming Seg
`ments. The metadata describing each Segment may take the
`form of a separate data entity, or may be stored or trans
`mitted with the content of programming Segment, which it
`describes. Unless the metadata is associated with a particular
`Segment by being Stored or transmitted with that Segment, it
`includes a pointer or Some other mechanism for Specifying
`the Segment or Segments it describes. In addition, the
`metadata typically includes additional descriptive informa
`tion about the associated segment(s). The metadata created
`at 111 may be immediately processed or transmitted to the
`user after it is created, or may be Stored for later processing
`or transmission in a Storage unit illustrated at 113.
`0047 Only selected items of metadata may be transmit
`ted to the user location. The Specific metadata transmitted
`may be selected as shown at 115 in a variety of ways. Data
`describing the demographics of individual users and data
`Specifying user preferences Stored at 117 may be used to
`selectively provide the user with only that portion of the
`available metadata which is best Suited to the needs of the
`user or which a third party, Such as an advertiser, desires to
`make available to the user.
`0048. Note that metadata created by the user, or prefer
`ence data Supplied by the user or derived from an analysis
`of the user's use of the system, or from the viewer's
`demographic characteristics, may be combined with or used
`instead of metadata and preference data created at the remote
`location.
`0049. Note also that the content of broadcast program
`ming received at the remote site may be forwarded to the
`user location with or Separately from the corresponding
`metadata. This content information may take the form of the
`broadcast programming received at the remote site at 101,
`previously received programming Stored at 103, and indi
`vidual segments as parsed at 106 and stored at 107. As noted
`above, the metadata associated with these programming
`Signals may be combined with the programming content as
`transmitted to the user, or may be sent Separately over the
`Same or a different communications pathway.
`0050. The communication methods or apparatus used to
`transport metadata and/or content to the user as illustrated at
`130 may take many different forms, including: the Internet,
`a dialup telephone connection through the public Switched
`telephone network (PSTN), a wireless transmission system,
`cable, private line facilities, or data Storage media trans
`ported from the content publisher and/or the metadata cre
`ator to the user. The communications may take place over a
`combination of Such facilities and, as noted earlier, the
`content and metadata may be transmitted in one or both
`directions together or separately over the same or different
`facilities.
`0051 Metadata created at the remote location and trans
`mitted via the communications facility 130 may be stored at
`133 at the user location. The metadata stored at 133 may be
`edited at the user location as indicated at 135, and metadata
`from the user location may be returned via the communi
`cations facility 130 to the remote location for shared use by
`others.
`
`-9-
`
`

`

`US 2003/0093790 A1
`
`May 15, 2003
`
`0.052 At the user location, broadcast programming Sig
`nals are received at 141, either in the form of a live public
`broadcast from the Source 100, or as programming content
`received from the remote location via the communications
`link 130. It a leading purpose of the present invention to
`provide the user with a better and more convenient way to
`identify and reproduce that portion of the large quantity of
`programming that is broadcast for general consumption
`from many Sources via many pathways, including conven
`tional radio and television broadcasting, whether over the
`airwaves or via a cable or Satellite facility. The metadata that
`is provided from the remote location via the communica
`tions pathway(s) 130 may be used to selectively store,
`organize and/or Selectively reproduce programming
`received directly at the user location from a source 100, or
`received together or Separately with the metadata via the
`pathway 130.
`0053. The broadcast programming content received at the
`user location at 141 may be immediately processed or Stored
`for later processing and Viewing. AS described in U.S.
`Reissue Patent 36,801 issued to James D. Logan et al. by the
`invention, the incoming broadcast programming may be
`concurrently viewed or otherwise processed while it is being
`recorded in a circular buffer for possible future use. A
`reserved portion of the Storage unit Seen at 143 may imple
`ment the circular buffer. This allows the user to utilize
`VCR-type controls to pause and Selectively replay or pro
`ceSS previously broadcast programming at different forward
`and reverse playback rates. With the pause capability, the
`System is constantly recording the last 5 minutes or So of a
`live radio broadcast, or the last 30 minutes or so of a live
`television broadcast. When the user hears or views a Song or
`program that he or she likes, the user presses a "Catch'
`button, and the program will Set aside the all of a predeter
`mined part of the Stored programming in the circular buffer,
`as well as a further predetermined part of the incoming
`broadcast that continues the Saved portion, and retains both
`in temporary Storage at 103. Later metadata may then be
`applied to that Segment identifying the beginning and end of
`the program or Song being played at the time the catch
`button was activated. If the button was hit after a program
`or Song was over, but before or after another began, the
`System would assume the user was trying to capture the last
`played Song.
`0054. Unless received in already parsed form from the
`remote location, the incoming broadcasts are parsed at 145
`into Segments that correspond to the Segments created at the
`remote location at 105. As noted earlier, the available
`metadata may be used to Subdivide the incoming broadcast
`Signals into Segments. For example, the metadata may
`identify incoming Segments by Source and by Start and end
`times. Alternatively, the metadata may include "fingerprint”
`or “signature' Signal pattern that can be compared with
`incoming broadcast Signals to identify particular Segments,
`and may further include timing information, which Specifies
`the beginning and ending of each Segment relative to the
`location of the unique signature.
`0055. After individual segments have been identified in
`the incoming broadcast Stream at 145, they may be imme
`diately processed or Stored for future use in the Storage unit
`145. Not all of the segments that are identified may be of
`further use; accordingly, the available metadata may be used
`to Select or discard particular Segments as indicated at 151,
`
`and to process only the remaining Segments, or Selectively
`store them for future processing or playback at 153.
`0056. At 161, the selected segments may be modified or
`reorganized in a variety of ways in accordance with the
`metadata. For example, the Sequence in which program
`Segments are presented for playback may be modified, and
`programming materials not necessarily included in with the
`originally broadcast materials may be "spliced' into the
`presentation, or all or part of Selected Segments may be
`deleted from the presentation. The resulting program content
`which is in condition for playback may be immediately
`presented to the user, or it may be stored at 163 for selective
`playback at a more convenient time as indicated at 171 and
`190.
`0057. As illustrated in FIG. 1 at 180 and 135, the user
`may create descriptive metadata and may edit metadata
`previously received or created in a variet