Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 1 of 60 Page ID #:118
`Case 2:11-cv—10701-DOC-JPR Document 1-3
`Filed 12/27/11 Page 1 of 60 Page ID #2118
`
`EXHIBIT 3
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 2 of 60 Page ID #:119
`Case 2:11-cv—10701-DOC-JPR Document 1-3
`Filed 12/27/11 Page 2 of 60 Page ID #:119
`l|||||||||ll|||||||l||||||t....tn|||||||llll||||||||l|||||||||l||||H|||
`
`US005813014A
`
`United States Patent
`Gustman
`
`[19]
`
`Patent Number:
`
`[11]
`[45] ‘Date of Patent:
`
`5,313,014
`Sep. 22, 19989
`
`[54] METHOD AND APPARATUS FOR
`MANAGEMENT OF MULTIMEDIA ASSETS
`
`Prinmry Examz'ner—Wayne Amsbury
`Assistant Exa/m'ner—Diane D. Mizvahi
`
`[75]
`
`Inventor: Samuel Gustman, Santa Monica, Calif.
`
`Attorney. Agent, or Firm—IIecker & Harriman
`
`[57]
`
`ABSTRACT
`
`The invention is a generalized solution for management of
`multimedia assets. Generalized interfaces are used between
`
`a browser component, indexing server, archive server, ter-
`tiary storage manager, and method player components. The
`generalized interface defines a communication protocol that
`can be used by any browser, indexing sewer, archive server,
`tertiary storage manager, or method player component.
`Thus, system components supplied by multiple vendors can
`be interconnected to form a multimedia system that com-
`municates using the generalized interfaces of the invention.
`A browser component can accept input such as search
`requests and display multimedia data. An indexing server
`component includes a data catalogue that catalogues multi-
`media data. The catalogue can be used to identify catalogue
`elements associated with data that can satisfy a search
`request. A search mechanism is used to identify the desired
`multimedia data. A search mechanism can create additional
`
`catalogues that are a subset of the catalogue provided on the
`indexing server. An archive server identifies a tertiary stor-
`age manager that manages the store containing the multi-
`media data. The tertiary storage manager retrieves the cor-
`responding multimedia data from its store and sends it to a
`method play which plays it in a display region of the
`browser. The invention retains the content and results of a
`
`search such that it is only necessary to perform a search
`once. Search elements are used to store the content of a
`
`search, i.e., search criteria. A segment element is used to
`store the results of a search.
`
`34 Claims, 24 Drawing Sheets
`
`[73] Assignee: Survivors of the Shoah Visual History
`‘
`Foundation, Los Angcles, Calif.
`
`[21] Appl. No.: 678,727
`
`[22] Filed:
`
`Jul. 10, 1996
`
`Int. CL5 ................................................... G061?‘ 17/30
`[51]
`[52] U.S. CI.
`........................... 707/103; 707/3; 707/103;
`707/104; 707/102; 707/1; 707/4
`[58] Field of Search ..................................... 395/615, 234,
`395/285, 806, 701, 601-10, 6l0—619,'620—29;
`434/500, 308; 707/1, 3, 103, 4, 102, 104
`
`[56]
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`3/1993 Grarzyk et al.
`8/1993 Reed ...........
`2/1994 Enberg et al.
`2/1994 Glick et al.
`3/1994 Bernstein et al.
`4/1994 MaeKay ..........
`3/1995 Robison et at
`4/1995 Fujisawa
`..... ... 345/335
`.....
`6/1995 Baker etal.
`345/133
`7/1995 Dinwiddie, Jr. e a .
`370/352
`7/1995 Bowen et al.
`........ 707/9
`9/1.995 Bergen eta].
`395/200.36
`6/1996 Meske, Jr. et al.
`707/102
`3/1997 Dockter etal.
`..
`707/102
`5/1997 Braden—Harder
`707/9
`7/1997 Antognini et al.
`7/1997 Ferguson ................................... 707/10
`OTHER PUBLICATIONS
`
`
`
`........................ 348/552
`. 707/104
`348/141
`. 379/93.01
`345/356
`. 345/328
`
`5,192,999
`5,241,571
`5,283,638
`5,283,819
`5,297,249
`5,307,456
`5,402,499
`5,404,506
`5,423,730
`5,434,592
`5,436,898
`5,450,581
`5,530,852
`5,608,900
`5,630,121
`5,649,185
`5,649,186
`
`Musei Vaticani, Internet, (http://www.virtual.co.il/orgs/orgs/
`archival/bookintnhtni), pp. 1-10, Dec. 8, 1994.
`The Judaica Archival Project, Internet, (http://www.virtual—
`.co.il.orgs/archiva/bookintrhtm), pp. 1-11, Apr. 1996.
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 3 of 60 Page ID #:120
`Case 2:11-cv—10701-DOC-JPR Document 1-3
`Filed 12/27/11 Page 3 of 60 Page ID #:120
`
`U.S. Patent
`
`Sep.22,1998
`
`Sheet 1 of 24
`
`5,813,014
`
`Method
`Player
`
`102
`
`104
`
`
`
`Tertiary
`Storage
`Manager
`
`106
`
`Archive
`Server
`
`110
`
`Index
`Server
`
`108
`
`112
`
`lgackard....... 1‘
`5 """"""""""""""""""""""""""
`Hardware Platform
`
`Client and '
`Method
`
`Player
`
` 118
`
`\'uw'uV\'nI§\hbuuy
`
`5
`_
`:
`;
`:
`
`1
`
`1
`:
`,
`
` Inter-media
`Server
`. h
`
`.w”
`‘*-> Archxve Server
`
`
`and
`
`
`Tertiary Storage
`
`Manager
` 3 116
`r . . 4 . - ¢ ¢ . . . . — .. . . . . . . . . . . » . . . . . . . . . . . . . - a — — a . . o — . o - u — a — o r ¢ » — a u n . o o . p a . - p . ¢ 4:
`
`[nd
`5
`ex
`erver
`
`
`
`
`
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 4 of 60 Page ID #:121
`Case 2:11-cv—10701-DOC-JPR Document 1-3
`Filed 12/27/11 Page 4 of 60 Page ID #:121
`
`U.S. Patent
`
`Sep. 22,1998
`
`Sheet 2 of 24
`
`5,813,014
`
`-....-~
`
`I
`
`5:?
`
`3:
`
`~(
`
`«I
`
`124
`
`Media
`
`Server
`
`Index
`Server
`
`130
`
`126
`
` I
`
`« 144
`
`E
`
`E
`
`3
`
`148
`
`15°C>/
`
`15013
`
`Index
`
`. . ¢ . . . . . — . ¢ ¢ n o a — . . . . - . . ¢ ¢ . - - v . « - . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . — ..
`
`Server .q—--un~sss~\um-s\\sv§\§~tu‘suns-syssmssnvsnIwiiihhuwh
`
`.
`
`146
`
`Archwe
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 5 of 60 Page ID #:122
`Case 2:11-cv—10701-DOC-JPR Document 1-3
`Filed 12/27/11 Page 5 of 60 Page ID #:122
`
`U.S. Patent
`
`Sep. 22, 1998
`
`Sheet 3 of 24
`
`5,813,014
`
`. , . . . . . _ . . . — — . . . . _ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .. . . . . . . . . .. . . . . _ . . . . . . . . . . . . . . --
`
`162
`
`174
`
`.
`:
`-
`o
`I
`/
`I
`.
`.
`u
`
`:a
`9
`r
`.
`I
`p
`.
`a
`.
`.
`n
`n
`n
`
`,
`1
`'
`'
`'
`r
`'
`I
`.
`I
`
`»I
`'
`'
`.
`I
`I
`4
`.
`I
`'
`'
`'
`
`.v
`
`.....
`
`
`
`-
`u
`r
`u
`p
`o
`.
`p
`u
`r
`E
`:-
`p
`C
`p
`u
`.
`u
`u
`7
`.
`I
`1
`;
`a
`;
`u
`1
`I
`I
`1-
`. . . . . » — . . . . . « . . — r . . . - . » — a . . . . . . . . . — — — a —. — c.
`
`-
`Archxve
`Server
`
`. . . .. . . . .
`
`.
`
`0
`'
`'
`'
`'
`'
`I
`'
`'
`I
`'
`'
`P
`I
`I
`'
`I
`’
`I
`.
`v
`1
`'
`'
`'
`'
`'
`,.
`. . . ; . p . . , I — — . . . . . . o «— . . . . . . . . - r » o « « . .. . . . . . . . . . . . . . . . . — .
`
`.v
`
`.r
`
`.r
`
`.»
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 . . . . — » — — » v - — - - :— — o - o 4 A — - — - - - - v v - » v-
`190
`
`182
`
`3
`
`5
`1
`5
`
`Method
`Player
`
`
`
`
`
`188
`
`Tettia ry
`Storage
`.
`A h
`Sifrvlevre / Manager
`
`
`184
`3
`186
`3
`: . v . . . . . . . . . . . , . . . . . . . . . . . . . . . y . . . . 4 ¢ o . a I r - - : : u 4 » . - y - . . . . - . r . . e « ¢— ..
`
`178
`
`:
`
`E
`-
`
`3
`I
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 6 of 60 Page ID #:123
`Case 2:11-cv—10701-DOC-JPR Document 1-3
`Filed 12/27/11 Page 6 of 60 Page ID #:123
`
`U.S. Patent
`
`Sep. 22, 1998
`
`Sheet 4 of 24
`
`5,813,014
`
`219
`
`216
`
`1/0
`
`VIDEO AMP
`
`CRT
`
`213
`
`214
`
`CPU
`
`VIDEO MEMORY
`
`MAIN MEMORY
`
`210
`
`KEYBOARD
`
`MOUSE
`
`2“
`
`MASS STORAGE
`
`218
`
`217
`
`215
`
`212
`
`FIG. 2
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 7 of 60 Page ID #:124
`Case 2:11-cv—10701-DOC-JPR Document 1-3
`Filed 12/27/11 Page 7 of 60 Page ID #2124
`
`U.S. Patent
`
`Sep.22,1998
`
`Sheet 5 of 24
`
`5,813,014
`
`313
`
`.
`
`310
`
`Browser
`
`Player
`
`314 )2
`
`316
`
`Indexing
`Server
`
`312
`
`f
`
`
`
`Tertiary
`
`302
`
`Archive
`Server
`
`306
`
`FIG. 3
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 8 of 60 Page ID #:125
`Case 2:11-cv—10701-DOC-JPR Document 1-3
`Filed 12/27/11 Page 8 of 60 Page ID #2125
`
`U.S. Patent
`
`Sep. 22, 1998
`
`Sheet 6 of 24
`
`5,813,014
`
`211245.:
`input data ID
`ID
`descriptive phrase
`In/Out Tirnsmdes
`se!(keywon1's)
`sct(1'z-ruzges)
`szt(persons)
`setqxmposed keywords)
`sc¢{proposed persons)
`
`‘W9 ,
`In/Out Ttmeaades
`oamment
`
`
`
`
`403
`
`424
`
`Progosed Person
`ID
`Name
`
`sefiphrascs)
`
`Segment
`ID
`descriptive phrase
`
`
`
`ID
`semypes)
`sefflhesauml keywvrds)
`5;t(par:nI kzy-words)
`seI(chiId key-words)
`s:t(praposcd keywords)
`
`
`
`Person ID
`Person Attribute
`
`
`
`Image I Vjdgo
`ID
`1211011! Timzmdzs
`Lnmlor
`
`
`
`
`FIG. 4A
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 9 of 60 Page ID #:126
`Case 2:11-cv—10701-DOC-JPR Document 1-3
`Filed 12/27/11 Page 9 of 60 Page ID #2126
`
`U.S. Patent
`
`Sep.22,1998
`
`Sheet 7 of 24
`
`5,813,014
`
` Proposed
`
`438
`
`Keyword
`
`412
`
`
`
`Thesaural
`
`Keyword
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 10 of 60 Page ID
` #:127
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 11 of 60 Page ID
` #:128
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 12 of 60 Page ID
` #:129
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 13 of 60 Page ID
` #:130
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 14 of 60 Page ID
` #:131
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 15 of 60 Page ID
` #:132
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 16 of 60 Page ID
` #:133
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 17 of 60 Page ID
` #:134
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 18 of 60 Page ID
` #:135
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 19 of 60 Page ID
` #:136
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 20 of 60 Page ID
` #:137
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 21 of 60 Page ID
` #:138
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 22 of 60 Page ID
` #:139
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 23 of 60 Page ID
` #:140
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 24 of 60 Page ID
` #:141
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 25 of 60 Page ID
` #:142
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 26 of 60 Page ID
` #:143
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 27 of 60 Page ID
` #:144
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 28 of 60 Page ID
` #:145
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 29 of 60 Page ID
` #:146
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 30 of 60 Page ID
` #:147
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 31 of 60 Page ID
` #:148
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 32 of 60 Page ID
` #:149
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 33 of 60 Page ID
` #:150
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 34 of 60 Page ID
` #:151
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 35 of 60 Page ID
` #:152
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 36 of 60 Page ID
` #:153
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 37 of 60 Page ID
` #:154
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 38 of 60 Page ID
` #:155
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 39 of 60 Page ID
` #:156
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 40 of 60 Page ID
` #:157
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 41 of 60 Page ID
` #:158
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 42 of 60 Page ID
` #:159
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 43 of 60 Page ID
` #:160
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 44 of 60 Page ID
` #:161
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 45 of 60 Page ID
` #:162
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 46 of 60 Page ID
` #:163
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 47 of 60 Page ID
` #:164
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 48 of 60 Page ID
` #:165
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 49 of 60 Page ID
` #:166
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 50 of 60 Page ID
` #:167
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 51 of 60 Page ID
` #:168
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 52 of 60 Page ID
` #:169
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 53 of 60 Page ID
` #:170
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 54 of 60 Page ID
` #:171
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 55 of 60 Page ID
` #:172
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 56 of 60 Page ID
` #:173
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 57 of 60 Page ID
` #:174
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 58 of 60 Page ID
` #:175
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 59 of 60 Page ID
` #:176
`
`

`
`Case 2:11-cv-10701-DOC-JPR Document 1-3 Filed 12/27/11 Page 60 of 60 Page ID
` #:177