United States Patent (19)
`Reimer et al.
`
`54 SYSTEMAND METHOD FOR PROVIDING
`MERCHANT INFORMATION AND
`ESTABLISHING LINKS TO MERCHANTS
`WHILE PRESENTING AMOVE
`75) Inventors: James A. Reimer, Morgan Hill; Roger
`A. Reinsch, Cupertino, both of Calif.
`73) Assignee: International Business Machines
`Corporation, Armonk, N.Y.
`
`56)
`
`(21) Appl. No.: 407,307
`22 Filed:
`Mar 20, 1995
`(51) Int. Cl. ... H04N 7/14
`52) U.S. Cl. ................................. 395/227; 348/13; 348/7;
`395/327; 395/762
`58 Field of Search .............................. 364/401; 348/13,
`348/1, 7, 10, 17; 395/227, 762,327
`References Cited
`U.S. PATENT DOCUMENTS
`4,305,131 12/1981 Best ........................................ 364/521
`4,931,950
`6/1990 Isle et al.
`... 364/513
`5,053,956 10/1991 Donald .................................... 364/40
`5,111,409 5/1992 Gasper et al
`... 395/152
`5,231,566 7/1993 Blutinger ................................ 364401
`5,307,456 4/1994 MacKay ...
`... 395/154
`5,309,355 5/1994 Lockwood .....
`... 364/401
`5,319,455 6/1994 Hoarty et al...
`... 348/7
`5,345,549 9/1994. Appel et al. .......
`395/154
`5,353,391 10/1994 Cohen et al. ......
`. 395/125
`5,367,621 11/1994 Cohen et al. ......
`395/154
`5,388,993 2/1995 McKiel et al......
`. 434/118
`5,404,316 4/1995 Klingler et al.
`395/16
`5,414,808 5/1995 Williams ................................. 395/154
`5,440,678 8/1995 Eisen et al.
`399.5/154
`5,451,998 9/1995 Hamrick .....
`. 348/13
`5,485,611
`1/1996 Astle ..............
`... 395/600
`5,515,490 5/1996 Buchanan et al.
`... 395/54
`5,539,450 7/1996 Handelman ............................... 348/12
`OTHER PUBLICATIONS
`Cowart, Robert, "Master Windows (83.1". SYBEX, 1992, p.
`31.
`Ozer, "Digital Video: Shot by Shot", PC Magazine, vol. 14,
`No. 7, Apr. 11, 1995, pp. 104-110.
`
`US005696905A
`Patent Number:
`11
`45 Date of Patent:
`
`5,696,905
`Dec. 9, 1997
`
`Sahai, "Video Thrills for Digital DeMilles: Adobe Premiere
`4.0", Windows Magazine, vol. 6, No. 1. Jan. 1, 1995, p. 140
`(abstract only).
`Simone, "Video-editing Software: Digital Master", PC
`Magazine, vol. 14, No. 7. Apr. 11, 1995, pp. 144-145.
`Multimedia Scouting IEEE Computer Graphics and Applic
`Jul. 1991 p. 26.
`Seybold vol. 7 No. 3 Nov. 2, 1992.
`The Bard of CD-ROM, Charles Boisseau Houston
`Chronicle Oct. 5, 1992.
`
`Primary Examiner-Jeffery Hofsass
`Assistant Examiner-Albert K. Wong
`Attorney, Agent, or Firm-Marilyn Smith Dawkins,
`Esquire; Sterne, Kessler, Goldstein & Fox PL.L.C.
`57
`ABSTRACT
`A system and method for providing on demand access to
`information related to a movie while the movie is being
`presented to a user, where the movie was generated from the
`movie related information, are described. The system oper
`ates by presenting the movie to the user, and then receiving
`from the user a query pertaining to the movie. The system
`determines a frame of the movie that was being presented to
`the user when the user issued the query (the system may
`extract this information from the query, or may extract this
`information from the movie itself). The system identifies, as
`specified by the query, portions of the movie related infor
`mation relating to the frame, and retrieves those portions of
`the movie related information. These retrieved portions of
`said movie related information are presented to the user.
`Also discussed is a manager to enable users to create
`personalized versions of movies, and personalized collec
`tions of items. Also discussed is a system and method for
`providing to users on demand access to merchandise infor
`mation related to a movie, and for providing on demand
`connectivity to merchants, while the movie is being pre
`sented to the users.
`
`16 Claims, 18 Drawing Sheets
`
`3.
`\ PRESENIAN
`AND
`CNIRO
`COMPENTS)
`
`OS
`
`118
`
`INDEX
`INTEFACE
`COPOENTS
`
`22
`
`Ek
`INFRATION
`CAARASES)
`
`COMNICAO
`EOui
`
`FNATION
`NFORMATON
`NERFACE
`cotts)
`
`
`
`For-ATION
`AABASES)
`
`Page 1
`
`AMAZON EX. 1005
`Amazon v. CustomPlay
`US Patent No. 9,124,950
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 1 of 18
`
`5,696,905
`
`FIG. 1
`
`106
`
`Dis
`
`102
`
`104
`
`\ PRESENTATION
`co L
`
`COMPONENT (S)
`
`108
`FOUNDATION
`INFORMATION
`INTERFACE
`COMPONENT (S)
`
`110 ;
`:
`- 12
`:
`FOUNDATION
`INFORMATION
`DAABASES)
`
`124
`
`
`
`COMMUNICATION
`MEDIUM
`
`18
`
`INDEX
`INTERFACE
`COMPONENT (S)
`
`1.
`
`122- .
`
`INDEX
`INFORMATION
`DATABASES)
`
`16
`
`CROSS
`MERCHANT
`GATEWAY
`
`114
`
`MERCHANT
`TRANSACTION
`PROCESSOR
`
`Page 2
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 2 of 18
`
`5,696,905
`
`R
`
`FIG. 2
`
`204
`
`PROCESSOR
`
`PRIMARY MEMORY
`CONTROL LOGIC
`
`20
`
`SECONDARY MEMORY
`26 STORAGEL
`DEVICE
`
`220
`
`
`
`28
`
`REMOVABLE
`STORAGE
`MEDIUM
`
`
`
`
`
`
`
`
`
`222
`
`NETWORK
`INTERFACE
`
`FIG. 3
`
`
`
`3O2
`
`SOURCE
`INFORMATION
`
`
`
`
`
`CAPTURE AND
`DIGITIZER
`COMPONENT (S)
`
`306
`
`FOUNDATION
`INFORMATION
`
`308
`
`INDEX
`GENERATION
`COMPONENT (S)
`
`30
`
`INDEX
`INFORMATION
`
`
`
`
`
`Page 3
`
`

`

`US. Patent
`
`Dec. 9, 1997
`
`Sheet 3 of 18
`
`5,696,905
`
`>m<zHIl
`
`ZOHF<=¢0¢ZH
`
`MWW
`
`xquu
`
`onp<zmong
`
`oHo=<
`
`mfiw
`
`onhusoommEwe; mfiv
`
`OMQH>~m__mH4mm<=¢
`
`onmHuwo_Howon_u=oo¢¢-pmc¢
`
`onH<zco¢zH
`
`.mHHWK
`
`mfi¢
`
`onb<zmomzH_
`>m<azouwm_
`
`mmwGov_mo.
`
`
`
`ZOHH<ZEO¢ZHmumsom.O¢_NOV
`
`onp<xmomzH"ww<xm
`
`
`
`onHusooml-wmmonhusoommam
`
`
`
`onp<=¢omzHmm<zm
`
`
`
`onhu=ao¢¢onbuzoomm
`
`a“.
`
`Page4
`
`Page 4
`
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 4 of 18
`
`5,696,905
`
`FIG. 5
`
`30
`
`
`
`
`
`
`
`502
`
`504
`
`INDEX INFORMATION
`506
`
`508
`
`WIDEO
`INOICES
`
`AUDIO
`INDICES
`
`TEXT
`INDICES
`
`PARAMETRIC
`INDICES
`
`510
`
`
`
`IMAGE
`CONTENT
`INOICES
`
`FIG. 6
`
`
`
`30
`
`INDEX
`INFORMATION
`
`GENERAL INDEX TABLE (S)
`
`602
`
`604
`
`INDEX TABLES FOR
`MOVIE A
`
`
`
`606
`
`INDEX TABLE (S) FOR
`MOVIE B
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 5
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 5 of 18
`
`5,696,905
`
`FIG. 7
`
`SCENE 708'
`
`SHOT TOS'
`-1-
`
`704
`
`SCENE 708
`
`SHOT 706
`
`to
`
`o
`
`to
`
`704
`
`F EII
`
`to on
`
`o
`
`o o o
`
`TAKE 72
`
`SHOT 106
`
`o
`
`722
`
`o o
`
`E.
`
`704
`
`SHOT 706
`704
`9 to too too to poly 1
`
`O
`
`C
`
`O C.
`
`720
`
`O Do
`to
`N--
`TAKE 710
`
`o O O. O. O. O. o O O. O. O.
`
`o
`
`O
`
`TAKE 72
`
`TAKE 714
`
`Page 6
`
`

`

`U.S. Patent
`US. Patent
`
`Dec. 9, 1997
`Dec. 9, 1997
`
`Sheet 6 of 18
`Sheet 6 of 18
`
`5,696,905
`5,696,905
`
`H
`
`ESEEmzmuw
`
`5.5
`
`n5“8:
`
`53
`
`“EEG
`
`n5m8:
`
`5.:
`
`Page?
`
`eggwas
`
`5:;wt:WEEmeE:was”£5:a:£3;E:I
`
`
`
`was.2:38mg:was
`
`
`
`m;m;.3N;ammgas/.2
`
`m.mHmmg
`
`“gem
`
`was
`
`Page 7
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 7 of 18
`
`5,696,905
`
`FIG. 9A
`
`950
`
`PRESENT USER
`WITH WIEWING
`CHOICES
`
`
`
`
`
`RECEIVE CHOICE
`FROM USER
`
`
`
`952
`
`954
`
`ACCESS SELECTION
`
`
`
`956
`
`SEND NEXT FRAME
`
`958
`
`RY
`OUE
`VED2
`RECEI
`
`YES
`
`
`
`962
`PROCESS OUERY
`
`
`
`
`
`
`
`
`
`
`
`
`
`MORE
`FRAMES
`
`
`
`9
`64
`
`966
`
`Page 8
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 8 of 18
`
`5,696,905
`
`FIG. 9B
`
`PRODUCE TIME CODE
`CORRESPONDING TO CURRENT
`POSITION OF WIDEO
`
`904
`
`EXTRACT FROM SOURCE
`FRAME
`FIRST fat." ROWS Al
`its TIME CODE & ESE
`
`90S
`
`IDENTIFY OUERYTYPE
`
`908
`
`PROCESS AccoRDING
`TO OVERY TYPE
`
`i? 910
`
`912
`
`END
`
`Page 9
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 9 of 18
`
`5,696,905
`
`
`
`
`
`
`
`FIG. 10A
`1006
`1008
`
`1004
`
`ACTOR |
`
`IAKE
`f |
`
`JOHN SMITH
`
`CLOSEUP
`F's
`
`SAB
`II
`CODE
`
`O607
`
`100
`
`END
`TIME
`CODE
`
`O658
`
`ACTOR
`IN TAKE
`TABLE
`1002
`
`FIG 10B
`CHARACTER
`INSEE
`
`1012
`
`1016
`
`1014
`
`FINAL BATTLE
`
`HERCULES
`
`FIG 100
`
`1020
`
`1022
`
`1024
`
`CHARACTER
`
`ACTOR
`
`HERCULES
`
`JOHN SMITH PRINCIPLE
`
`HERCULES
`
`JOE DAVIS
`
`ACTOR PLAYS
`
`crash
`
`Of
`
`Page 10
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 10 of 18
`
`5,696,905
`
`FIG 11
`
`1102
`
`
`
`
`
`1120
`NTIFY R
`IDENTIFY ROWS
`IN CHARACTER
`IN SCENE TABLE
`USING SCENE
`INFORMATION FROM
`TAKE ROW EXTRACTED
`FROM SOURCE TABLE
`
`
`
`
`
`22
`
`PRESENT HARACTER
`INFORMATION
`
`
`
`
`
`IDENTIFY ROWS IN
`ACTOR IN TAKE
`TABLE USING FIRST
`AND LAST TIME
`CODES FROM TAKE
`ROW EXTRACTED
`FROM SOURCE TABLE
`
`
`
`
`
`
`
`
`
`1106
`
`1104
`
`
`
`AT LEAST
`ONE ACTOR
`IN FRAME?
`
`PRESENT ACTOR INFORMATION
`
`IDENTIFY ROWS IN ACTOR PLAYS
`CHARACTER TABLE USING
`ACTOR INFORMATION
`
`PRESENT CHARACTER INFORMATION
`
`1108
`
`110
`
`12
`
`IDENTIFY ROWS IN ACTOR PLAYS
`CHARACTER TABLE USING
`CHARACTER INFORMATION
`
`1114
`
`PRESENT PRINCIPLE ACTOR
`INFORMATION
`
`1116
`
`1118
`
`Page 11
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 11 of 18
`
`5,696,905
`
`FIG. 12A
`
`1204
`
`1206
`
`ONE MOVIE
`CREDIT TABLE
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 12B
`1212
`
`1210
`
`1214
`
`1216
`
`ALL MOVIES
`CREDI TABLE
`
`Esso a N5
`
`
`
`
`
`
`
`
`
`Page 12
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 12 of 18
`
`5,696,905
`
`FIG. 13A
`
`1304
`
`1306
`
`1308
`
`1310
`
`
`
`
`
`TIME SINCE
`STORY START
`
`TIME SINCE
`LAST SCENE
`
`ABSOLUTE
`DATE TIME
`
`TIME IN
`STORY TABLE
`1302
`
`FIG. 13B
`
`'At
`1312
`
`its
`
`
`
`
`
`
`
`1314
`
`1316
`
`1318
`
`1320
`
`1322
`
`TAKE FIRST
`FRAME
`TIME CODE
`
`
`
`
`
`OCATION
`
`DATE SHOT
`(CREATED)
`
`TIME SHOT
`(CREATED)
`
`
`
`
`
`Page 13
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 13 of 18
`
`5,696,905
`
`FIG. 14A
`1408
`
`1406
`
`1410
`
`OUESTION ANSWER
`TABLE
`1404
`
`OUESTION E" | OVESTION |
`
`ANSWER
`
`
`
`FIG, 14B
`1428
`
`1426
`
`430
`
`OUESTION APPLIES
`1955
`1424
`
`
`
`OUESTION
`NUMBER
`
`FIRST SCENE
`
`LAST SCENE
`
`FIG. 15
`
`150
`
`
`
`
`
`DETERMINE
`CURRENT SCENE
`
`EXTRACT RELEVANT
`OUESTIONS
`
`PRESENT MENU OF
`"WHY" OUESTIONS
`
`
`
`
`
`
`
`RECEIVER OUESTION
`IDENTIFIER FROM USER
`
`1520
`
`1522
`
`1512
`
`1514
`
`PRESENT ANSWER
`
`
`
`
`
`1516
`
`1518
`
`Page 14
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 14 of 18
`
`5,696,905
`
`FIG. 16
`1602
`
`CAPTURE AND
`DIGITIZER
`COMPONENT (S)
`
`INDEX
`GENERATION
`COMPONENT (S)
`
`PERSONALIZED
`PRESENTATION
`MANAGER
`
`PERSONALIZED
`COLLECTION
`MANAGER
`
`FIG. 17A
`
`
`
`
`
`
`
`START
`
`
`
`1702
`
`CREATE COPY OF
`EOIT OECISION
`LIST FOR MOVIE
`
`ENABLE USER TO
`INTERACTIVELY MODIFY
`COPY OF EDIT DECISION
`LIST TO CREATE
`PERSONALIZED PRESENTATION
`
`
`
`
`
`
`
`
`
`
`
`
`
`1704
`
`1706
`
`Page 15
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 15 of 18
`
`5,696,905
`
`FIG. 17B
`
`1704
`
`1706
`
`1708
`
`1710
`
`1712
`
`1714
`
`FIRST FRAME
`TIME CODE
`
`LAST FRAME SOURCEIRST
`file'cif
`E.
`
`SOURCE FRAME
`
`SOURCE
`
`FOR
`WHICH
`
`702
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`SELECT FIRST ENTRY
`INPERSONALIZED
`EDIT DECISION LIST
`
`LOCATE UNDERLYING
`FOUNDATION INFORMATION
`CORRESPONDING TO
`ELECTED ENTRY
`
`1904
`
`1806
`
`PRESENT UNDERLYING
`FOUNDATION INFORMATION
`ENABLE USER OUERY MODE
`
`1808
`
`1814
`
`SELECT
`NEW
`ENTRY
`
`
`
`MORE
`ENTRIES
`
`1810
`
`1812
`
`Page 16
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 16 of 18
`
`5,696,905
`
`FIG. 19
`
`1902
`
`START
`
`
`
`
`
`
`
`
`
`
`
`
`
`CREATE NEW PERSONALIZED
`COLLECTION TABLE
`(MAY IMPORT FROM
`EXISTING COLLECTION)
`
`1904
`
`ENABLE USER TO
`INTERACTIVELY
`CREATEIMODIFY
`COLLECTION
`
`1906
`
`
`
`FIG. 20
`2006
`
`2004
`
`2008
`
`SOURCE FIRST
`FRAME TIME CODE
`
`SOURCE LAST
`FRAME TIME CODE
`
`
`
`
`
`
`
`
`
`
`
`
`
`PERSONALIZED
`COLLECTION
`TABLE
`2002
`
`Page 17
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 17 of 18
`
`5,696,905
`
`FIG 21
`
`2102
`
`START
`
`DETERMINE SCENE
`FROM ROWS EXTRACTED
`FROM SOURCE TABLE
`
`2104
`
`IDENTIFY MERCHANDISE IN
`SCENE BY REFERENCE TO
`MERCHANDISE TABLE USING
`SCENE INFORMATION
`
`2O6
`
`PRESENT
`MERCHANOISE
`INFORMATION
`
`208
`
`RECEIVE SUPPLEMENTAL USER
`OUERY IDENTIFYING SPECIFIC
`MERCHANOISE TYPE (E.G., COSTUMES)
`
`2110
`
`
`
`
`
`
`
`
`
`IDENTIFY MERCHANTS RELATED
`TO THE SELECTED MERCHANOISE
`TYPE BY REFERENCE TO TABLE
`CORRESPONDING TO MERCHANDISE TYPE
`
`212
`
`RECEIVE REOUEST FROM USER
`TO CONTACT A MERCHANT
`
`2114
`
`ESTABLISHINK BETEEN i? 216
`USER AND MERCHANT
`
`218
`
`Page 18
`
`

`

`U.S. Patent
`
`Dec. 9, 1997
`
`Sheet 18 of 18
`
`5,696,905
`
`
`
`
`
`2204
`
`220E
`
`FIG. 22A
`2208
`220
`
`2212
`
`2214
`
`MERCHANOISE
`
`S;
`
`SCENE
`
`FURNITURE
`
`COSTUMES
`
`LOCATION
`
`MUSIC
`
`FOOD
`
`JACKET
`SUIT
`
`GRAND
`CANYON
`
`SOUNDTRACK
`BY ABC
`
`228
`
`2220
`
`FIG. 22B
`2222
`2224
`
`2226
`
`2229
`
`COSTUMES
`MERCHANDISE
`
`5.
`
`SHOES
`
`DRESS
`
`SUIT
`
`HAT
`
`MERCHANT A
`MERCHANT C
`
`MERCHANT A
`MERCHANT B
`
`Page 19
`
`

`

`5,696,905
`
`1.
`SYSTEMAND METHOD FOR PROVIDING
`MERCHANT INFORMATION AND
`ESTABLISHING LINKS TO MERCHANTS
`WHILE PRESENTING AMOWIE
`CROSS-REFERENCE TO OTHER
`APPLICATIONS
`The following applications of common assignee contain
`some common disclosure, and are believed to have an
`effective filing date identical with that of the present appli
`cation.
`"System and Method for Linking and Presenting Movies
`With Their Underlying Source Information." application
`Ser. No. 08/407.306 issued as U.S. Pat. No. 5596.705,
`incorporated herein by reference.
`"System and Method for Enabling the Creation of Per
`sonalized Movie Presentations and Personalized Movie
`Collections,” application Ser, No. 08/407.305, incorporated
`herein by reference.
`
`DESCRIPTION
`1. Technical Field
`The present invention relates generally to data linking and
`presentation, and more particularly to linking and presenting
`movies with their underlying source information.
`2. Related Art
`The Information Age that currently exists has produced
`incredible amounts of data that are currently unavailable to
`the public. In many cases, the producers of such information
`would be more than willing to provide the information to the
`public for some reasonable fee, and the public would be
`willing to pay this fee. Typically, however, these information
`producers (also called information providers) are notable to
`take advantage of this significant business opportunity due
`to the problems inherent in providing the information to the
`public.
`One problem relates to the medium in which the infor
`mation is stored. Today, information is stored on paper, on
`film, on video, on compact disks, on magnetic tapes, on
`computer floppy disks, etc. Information related to a particu
`lar topic may be stored on any combination of these storage
`mediums. In order to provide information on that topic to an
`user, it would be necessary for an information provider to
`identify and locate all items containing information related
`to the topic, and then transport these items to the user. The
`value of these items to the user is questionable, since the user
`probably does not possess the equipment needed to access
`the information stored in all of the items. Accordingly, it is
`difficult to provide information to users because such infor
`mation is stored in many different storage mediums.
`A more significant problem results from the sheer volume
`of information relating to any particular topic. Such infor
`mation is of little value to an user unless the user is able to
`efficiently and expeditiously make use of the information.
`For example, suppose that the user is reviewing a document
`written by an author and decides that it would be interesting
`to study other works the author has had a part in creating.
`The information provided to the user from the information
`provider will be of little value unless the user is able to
`efficiently, effectively, and expeditiously traverse the infor
`mation and locate the data on the author of interest.
`These above-described commercial opportunities associ
`ated with the dissemination of information, and the prob
`lems that prevent such commercial opportunities from being
`65
`realized, shall now be described more concretely by con
`sidering a real-life example: the motion picture industry.
`
`55
`
`35
`
`45
`
`50
`
`15
`
`20
`
`25
`
`30
`
`2
`Viewing a movie via a theatrical presentation today
`involves travelling to the theater, waiting in line to buy a
`ticket, choosing a seat, and waiting for the movie to begin.
`The viewer's schedule must match that of the theater. Once
`the movie has started, the viewer must watch promotional
`previews of other films before watching the feature film.
`During the presentation, the viewer has no control over the
`sound volume or presentation flow. When the movie is over,
`the viewer travels back home. The picture quality and sound
`are excellent, and the viewer enjoys the best viewing expe
`rience possible today. However, these benefits are obtained
`only at the expense of convenience and viewing control.
`Thus, commercial opportunities exist in electronically trans
`ferring movies to viewer's homes.
`The technology to electronically transfer movies to view
`er's homes exists today. Viewing a movie over a conven
`tional network or premium channel broadcast is simpler and
`more convenient, but the viewer has fewer movie choices
`and enjoys a more limited quality experience. Like a theat
`rical viewing the viewer's schedule must match that of the
`broadcast and the viewer does not control the presentation
`flow. There is no travelling involved, and the cost is gener
`ally less than that of a theater experience. The video and
`audio quality are dependent on the viewer's television and
`sound system. Note that, with this viewing option (and also
`with the theatrical presentation option discussed above), the
`viewer is typically not provided with any supplemental
`information about the movie, such as information about the
`making of the movie, interviews with the director and cast
`members, information on the actors, information about
`special effects, etc.
`Alternatively, a person can watch a movie by renting a
`video tape or laser disk. Watching a movie in this manner
`involves two trips to the rental shop, a search of their
`available titles, a wait in line, and a rental fee. The viewing
`experience is improved over that of a broadcast presentation,
`as the viewer has some presentation controls (pause, fast
`forward, rewind, etc.). A laser disk may also offer supple
`mental information about the film. Parallel audio tracks may
`be included containing, for example, director commentary,
`Separate chapters of video may also be provided. However,
`there is no ability to play video content from different
`chapters simultaneously. The cost is generally higher than
`that of a network broadcast. Cost comparisons to that of a
`premium channel are dependent on usage patterns (flat fee
`versus fee per rental).
`A mid-90s “near video on demand' viewing experience
`represents something between that of broadcast movies and
`a VCR/laser disk. A viewer selects a movie from an
`expanded broadcast rotation (generally a much smaller
`collection than that available at a VCR rental outlet). The
`viewer must wait a few minutes until the next rotation of the
`movie broadcast. A purchase request via remote control or
`an 800 number is required to enable viewing. Once the
`presentation has begun, the viewer has a limited range of
`controls over the presentation flow. If the movie is paused,
`the viewer must wait until the next instance of the movie at
`that point is available in the rotation. This latency is depen
`dent on the number of channels available and the collection
`size configured by the viewer's cable carrier, and is typically
`aboutfive minutes. No travel is required, and the per use fee
`is typically added to the viewer's monthly cable bill. Supple
`mental information about the movie is typically not avail
`able.
`A mid-90s "true video on demand' viewing experience is
`effectively like that of a VCR rental, without two trips to the
`rental outlet. A digital movie server and ITV infrastructure
`
`Page 20
`
`

`

`5,696,905
`
`3
`are used to enable the viewer to select a movie from a large
`collection of all digitized films available on the server. The
`available selection list is not restricted by a broadcast
`rotation, and each viewer has full and immediate control
`over their movie presentation. Like "near video on demand,
`a purchase request via remote control or an 800 number is
`required to enable viewing. The per use fee is typically
`added to the viewer's monthly bill, and supplemental infor
`mation is typically not available.
`Thus, the current options for viewing a movie are limited
`because they are inconvenient (the viewer must travel to the
`theater or to the video store), and/or provide the viewer with
`little or no presentation flow. Also, these current viewing
`options are limited because they provide the viewer with
`little or no supplemental movie information. Thus, the
`motion picture industry is not taking advantage of the
`potential commercial opportunities associated with provid
`ing supplemental movie information to viewers. Even in
`those cases where supplemental information is provided, the
`viewer has little control over how such supplemental infor
`mation is presented. Accordingly, the supplemental infor
`mation is available only in a limited way to videophiles who
`are willing to pay extra for "special" or "director's cut”
`editions of the titles.
`
`10
`
`15
`
`20
`
`25
`
`4
`tation operates by retrieving an entry from the personalized
`collection table, retrieving from the foundation information
`database foundation information specified by the retrieved
`entry, and presenting the retrieved foundation information to
`the user. The items contained in the personalized collection
`include any combination of shots, takes, scenes, clips, audio
`segments, etc.
`Further, the present invention is directed to a system and
`method for providing on demand access to merchandise
`information related to a movie, and for providing on demand
`connectivity to merchants, while the movie is being pre
`sented to a user. This embodiment of the invention operates
`by presenting the movie to the user, receiving from the user
`a merchandise related query pertaining to the movie, and
`determining a scene of the movie that was being presented
`to the user when the user issued the query. Portions of movie
`related information relating to merchandise appearing in the
`scene are identified and retrieved. These portions of movie
`related information are then presented to the user. The
`invention enables the user to interact with any merchant who
`provided any merchandise appearing in the scene in
`response to a request from the user to do so.
`Further features and advantages of the present invention,
`as well as the structure and operation of various embodi
`ments of the present invention, are described in detail below
`with reference to the accompanying drawings. In the
`drawings, like reference numbers indicate identical or func
`tionally similar elements.
`The foregoing and other objects, features and advantages
`of the invention will be apparent from the following more
`particular description of (a) preferred embodiment(s) of the
`invention, as illustrated in the accompanying drawing(s).
`BRIEF DESCRIPTION OF FIGURES
`The present invention will be described with reference to
`the accompanying drawings, wherein:
`FIG. 1 is a block diagram of a preferred embodiment of
`the present invention;
`FIG. 2 is a block diagram of a computer system repre
`senting a preferred implementation of many components of
`the present invention;
`FIGS. 3 and 4 are data flow diagrams depicting the
`manner in which foundation and index information is gen
`erated according to a preferred embodiment of the present
`invention;
`FIGS. 5 and 6 illustrate example index information con
`tents;
`FIG. 7 illustrates the pedigree of a frame;
`FIG. 8 illustrates an example source table;
`FIGS. 9A, 9B. 11, 15, 17A, 18, 19, and 21 are flowcharts
`depicting the operation of the present invention;
`FIGS. 10A-10C, 12A, 12B, 13A, 13B, 14A, 14B, 17B,
`20, 22A, and 22B are example index tables; and
`FIG. 16 is a block diagram of a manager according to a
`preferred embodiment of the present invention.
`BEST MODE FOR CARRYING OUT THE
`INVENTION
`The present invention is directed to a system and method
`for organizing information, and for providing services to
`enable users to efficiently, effectively, and expeditiously
`access such information. In other words, the present inven
`tion provides services to enable users to access information
`in an intelligent, user-driven manner.
`
`DISCLOSURE OF INVENTION
`The present invention is directed to a system and method
`for providing on demand access to information related to a
`movie while the movie is being presented to a user, where
`the movie was generated from the movie related informa
`tion. The invention operates by presenting the movie to the
`user, and then receiving from the user a query pertaining to
`the movie. The invention determines a frame of the movie
`that was being presented to the user when the user issued the
`query (the invention may extract this information from the
`query, or may extract this information from the movie itself).
`The invention identifies, as specified by the query, portions
`of the movie related information relating to the frame, and
`retrieves those portions of the movie related information.
`These retrieved portions of the movie related information
`are presented to the user.
`The present invention is also directed to a system and
`method for enabling a user to create, modify, and utilize a
`personalized version of a movie. The system includes a
`foundation information database which stores movie related
`information used to generate the movie. This embodiment of
`the invention operates by enabling the user to modify a copy
`of an edit decision list associated with the movie. The edit
`decision list includes an entry for each scene, clip, and take
`in the movie. The modified edit decision list defines the
`personalized version of the movie. The personalized version
`of the movie is presented to the user in response to a request
`to do so from the user. Such presentation operates by
`retrieving an entry from the modified edit decision list,
`retrieving from the foundation information database one or
`more digitized frames specified by the retrieved entry, and
`presenting the digitized frames to the user.
`In a similar manner, the present invention is directed to a
`system and method for enabling a user to create, modify, and
`utilize a personalized collection of items. This embodiment
`of the invention operates by enabling the user to modify a
`personalized collection table, where the personalized col
`lection table defines the personalized collection and has an
`entry for each item in the personalized collection. The
`personalized collection of items is presented to the user in
`response to a request to do so from the user. Such presen
`
`30
`
`35
`
`45
`
`50
`
`55
`
`65
`
`Page 21
`
`

`

`5,696,905
`
`10
`
`5
`For illustrative purposes, the present invention is some
`times described herein in the context of information pro
`duced by the motion picture industry. That is, the present
`invention is described as a system and method for organizing
`movie and supplemental movie information, and for pre
`senting such movie and supplemental movie information to
`users. It should be understood, however, that the present
`invention is not limited to this embodiment (call the "movie
`environment” for reference purposes). Instead, the present
`invention is applicable to any environment where informa
`tion is produced, and where a commercial opportunity exists
`with the organization and presentation of such information.
`For example, the present invention is applicable to computer
`programming environment where information about the
`person who wrote a program, the company that sold the
`program, the place the program was written and tested, etc.,
`could be linked to a running program.
`The movie embodiment of the present invention is pref
`erably realized through a combination of a "true video on
`demand' infrastructure, database technology, and other spe
`cialized digital servers. By tightly coupling databases to
`specialized digital servers for video, audio, still image, text,
`etc., a novel level of control of movie presentation is
`achieved. Additional information about a movie is stored
`and managed, to permit a viewer to more effectively explore
`and understand movies. The combination of greater control
`and additional information permits the viewer to experience
`a new level of richness and entertainment from movies.
`Accordingly to the movie embodiment, text, still image,
`and audio information is directly related to specific movie
`scenes or frames. This level of association, coupled with a
`database, permits a greater level of direct access to movie
`information than previously possible. Searching and corre
`lation of supplemental information (also called side
`information) permits the examination of different side infor
`mation relationships, and permits the following sample
`actions not existing in conventional viewing technologies:
`
`6
`also managed by the database, and are used to form non-text
`searches against these data. This scheme can be used to
`search for non-text aspects such as cut boundaries and
`movies having similar soundtracks.
`The movie embodiment of the present invention also
`permits the examination of movie detail beyond that view
`able through a TV or HDTV. Feature films are today
`captured principally on high grade film, affording fidelity far
`beyond that viewable on TV or HDTV. The present inven
`tion permits a viewer to pan and zoom, to examine finer
`detail in a movie or image still.
`By using the movie embodiment of the present invention,
`filmmakers are afforded a new level of flexibility in optional
`viewing and associations from within their films. Alternate
`plot lines, cuts, or censor levels are much more easily
`constructed, because the film components are manipulated
`independently. Composition is directed through the
`database, easing the construction of alternate presentations
`and linkages.
`Structure of the Present Invention
`The present invention shall now be discussed more gen
`erally with reference to FIG. 1, which illustrates a block
`diagram of a data processing environment 102 of the present
`invention. This environment 102 includes a plurality of user
`devices 106 representing, for example, set top boxes (STB)
`in combination with television monitors, or computers (such
`as personal computers). The user devices 106 each include
`a mechanism (such as a keyboard either integrated into the
`user devices 106 or contained in a remote control unit) for
`receiving control and information messages from human
`operators.
`Each user device 106 also includes a well known transmit
`and receive component to transmit and receive control and
`information signals from other devices via a communication
`medium 124. Preferably, the communication medium 124
`represents a data communication network, such as a local
`area network or a wide area network, in which case the
`environment 102 represents a distributed computing envi
`ronment. However, the present invention is not limited to
`this connection scheme. For example, some of the devices
`shown in FIG. 1 may be positioned locally to each other,
`and/or some of the devices in FIG. 1 may be implemented
`using a single computer. In sum, any mechanism or scheme
`for connecting the devices shown in FIG. 1 is within the
`scope and spirit of the present invention.
`The environment 102 also includes a foundation infor
`mation interface component 108 that controls access to
`foundation information in a foundation information database
`112. (As shown by line 110, the foundation information
`interface component 108 is preferably directly connected to
`the foundation information database 112.) The term "foun
`dation information" is used herein to refer to information
`that has been produced by one or more parties (each party
`being a person or organization). According to the present
`invention, the foundation information represents digital
`information only. The present invention provides foundation
`information on demand to users via the user devices 106.
`With respect to the movie embodiment, the foundation
`information represents, for example, a movie and supple
`mental information pertaining to the movie (i.e., additional
`information generated during the pre-production,
`production, and post-production stages of the movie). The
`foundation information is described in greater detail below.
`Preferably, the foundation interface component 108 rep
`resents a relational database management system (RDBMS).
`The foundation information database 112 preferably rep
`resents a file server. The foundation information database
`
`15
`
`20
`
`25
`
`30
`
`35
`
`1. List the names of the actors, other credited roles, or scene
`technical detail for the current scene (e.g., director,
`producer, musical score, shot location, blue screen, matte,
`etc.).
`2. List other movies, TV shows, or other scenes in the
`current movie with one or more properties equal to those
`found above.
`3. List all scenes including a specific product tie-in (i.e.,
`product brand name used in movie).
`4. Save any of these lists for viewing selection.
`5. View scene while listening to voice overs of director or
`actor with their comments about the scene