I lllll llllllll II llllll lllll lllll lllll lllll lllll lllll lllll 111111111111111111111111111111111
`US 20100082585Al
`
`c19) United States
`c12) Patent Application Publication
`Barsook et al.
`
`c10) Pub. No.: US 2010/0082585 Al
`Apr. 1, 2010
`(43) Pub. Date:
`
`(54) SYSTEM AND METHOD FOR VISUAL
`SEARCH IN A VIDEO MEDIA PLAYER
`
`(75)
`
`Inventors:
`
`Jonathan Barsook, Los Angeles,
`CA (US); Joven Matias, Pasadena,
`CA (US); Cyrus Hoomani, Studio
`City, CA (US); Arun Ramagopal,
`Studio City, CA (US)
`
`Correspondence Address:
`DISNEY ENTERPRISES
`C/O FARJAMI & FARJAMI LLP
`26522 LA ALAMEDA AVENUE, SUITE 360
`MISSION VIEJO, CA 92691 (US)
`
`(73) Assignee:
`
`DISNEY ENTERPRISES, INC.,
`Burbank, CA (US)
`
`(21) Appl. No.:
`
`12/284,633
`
`(22) Filed:
`
`Sep.23,2008
`
`Publication Classification
`
`(51)
`
`Int. Cl.
`(2006.01)
`G06F 17130
`(52) U.S. Cl. ................................. 7071706; 707/E17.014
`(57)
`ABSTRACT
`
`There is provided a system and method for visual search in a
`video media player. There is provided a computing device for
`performing a visual search in video files comprising a display,
`a memory, a video media player application in the memory,
`and a processor configured to store into the memory search
`terms by allowing a user to visually select objects as search
`terms using a user interface on the display, initiate a search
`using the search terms, obtain search results comprising time
`offsets and play durations within the video files, each play
`duration containing at least one frame corresponding to the
`search terms, and display navigational controls in the user
`interface for presenting the search results, including select(cid:173)
`able graphical thumbnail images or preview videos. A user
`can select objects via outlined, highlighted, orotherwisevisu(cid:173)
`ally identified objects in a video frame, the selecting enabled
`by associated search metadata.
`
`Video rAedla Player 1 iO
`
`Page i
`
`AMAZON EX. 1027
`Amazon v. CustomPlay
`US Patent No. 9,380,282
`
`

`

`Patent Application Publication
`
`Apr. 1, 2010 Sheet 1 of 5
`
`US 2010/0082585 Al
`
`Fig. 1a
`
`Vkieo r'/ledia Player 11 D
`
`Fig. 1 b
`
`Navigation
`'130
`
`r····
`Juliet
`
`-.
`
`·100
`JI/
`
`r Navigation
`
`1:30
`.. ·~:., . """'i).· ·······:·:···············/'-., .~,,
`..... , ~ .................... .\!:) ..... .
`(!-*}. ~ .h.ti::,:;l:
`~ ~ (*'l)
`
`Object 120
`
`Navigation ~l>ntrol 1~: r
`
`Na\nw~tion ContrG! 1.5t:::-············'
`
`~--««<«««««<«««<«««¥ ....
`
`l
`l
`l
`:;
`
`N:avigJtion Control 134
`
`Navigation Control 135
`
`Fig. 1 c
`
`r....... N:.,:ivigaHon i '.30
`
`g0;;;t;®(j
`.. ,.·:·
`: ;:'.:::h:··: :.·::::;·:~::;~
`,,,,,,.,,,,,,
`~
`1 ::;.:~ •(:·:~
`
`~~
`
`Navi~Jation Centro! 136
`
`Page ii
`
`

`

`Patent Application Publication
`
`Apr. 1, 2010 Sheet 2 of 5
`
`US 2010/0082585 Al
`
`Fig* 2a
`,~~ Search Terms 240
`
`'
`
`200
`
`/
`
`Fig. 2b - , User Interface 2·1·1
`· - - -....... ~·~···~···~····~f'~::i~ed~':r~c~h~l~"{=-~~r~n~1·s~::::~·· 4~0~· --"-i...........:i+~···~ .. ··~ .. -~\~'id~e~~z::~·i ~~v.~~di·~--~~:~_:'.: .. __ :_~-~ ...................... .-........................... .
`S::i-<ir(h (ri·t::ir-:i:'l
`a,,,,.,,~., ..
`'~
`
`··················· .......... ·:::><~:....
`
`· , ........................................................................................................................... ·.···!,,,,,,,,,,,,
`
`Thumbn;,;li!s 250
`
`------
`. f)i:fa'.\\g
`
`Sc-en::::: 9
`Seene ·12
`
`-------------- ....
`·: c:):
`1:00 :.,..
`2:CD !
`!()(}:
`
`'f"'
`[·••I
`
`I
`! l ~
`j
`~~~~-· ..... ·······························-· .......... , ................................ ··.·················································....».············································;································.!
`
`l
`
`.!
`
`l
`
`• ~_JL iDDC::JDDLJL)LJ ·l
`
`J
`_.,_. ................... •.•.•,)
`, ........... j .......................................... ········ ............................................................. ·.· .. ·.· .· ............................ ,, ..••..................... ·.·······t·
`Burton 242
`PlayHst 24 'l
`Object 220
`Thurnbnails 250
`
`Page iii
`
`

`

`Patent Application Publication
`
`Apr. 1, 2010 Sheet 3 of 5
`
`US 2010/0082585 Al
`
`Fig. 3
`
`" User lntedace 31 i
`
`Search Box 360
`
`Thuinbnalls :350
`
`Page iv
`
`

`

`Patent Application Publication
`
`Apr. 1, 2010 Sheet 4 of 5
`
`US 2010/0082585 Al
`
`Fig. 4
`
`400
`
`/
`
`Computing Device 470
`
`Memory472
`
`Processor 474
`
`Display 476
`
`Network 480
`
`Media Server 490
`
`Search Provider 494
`
`Media File
`492a
`,,,
`
`Media File
`492b
`
`Meta data
`496a
`
`Metadata
`496b
`
`I
`
`I
`
`I
`I
`I
`I
`I
`L--------------r---------~
`I
`L------------------------~
`
`Page v
`
`

`

`Patent Application Publication
`
`Apr. 1, 2010 Sheet 5 of 5
`
`US 2010/0082585 Al
`
`Fig. 5
`
`Present a user interface for a
`video media player
`application on a display
`
`:...---502
`
`,,
`Access a video file from a
`plurality of audiovisual media _,..-.. 504
`files
`
`,,
`Retrieve search metadata
`concerning the accessed
`video file
`
`_,..-.. 506
`
`,
`
`Write the search metadata
`into a memory
`
`_,..-.. 508
`
`,,
`
`Play the accessed video file
`within the user interface
`
`_,..-.. 510
`
`_.
`
`500
`
`/
`
`Initiate a search using the v-
`search terms
`
`530
`
`,.
`Obtain search results
`comprising video file time
`offsets and play durations
`
`_,..-.. 540
`
`~·
`Display navigational controls
`in the user interface for
`presenting the search results
`
`_,..-.. 550
`
`'
`Select a time offset
`and a play duration of _,..-..
`a video file
`
`560
`
`' '
`Play the video file at
`the time offset and
`play duration
`
`_,..-..
`
`570
`
`,,
`Store search terms using the
`user interface by allowing
`user to visually select objects
`
`_,..-.. 520
`
`,.
`Select and play multiple
`search results successively
`in a playlist
`
`:...--- 580
`
`Page vi
`
`

`

`US 2010/0082585 Al
`
`Apr. 1, 2010
`
`1
`
`SYSTEM AND METHOD FOR VISUAL
`SEARCH IN A VIDEO MEDIA PLAYER
`
`BACKGROUND OF THE INVENTION
`
`[0001]
`1. Field of the Invention
`[0002] The present invention relates generally to computer
`systems. More particularly, the present invention relates to
`systems for processing of computer data.
`[0003] 2. Background Art
`[0004] With the vast selection of entertainment media and
`web sites currently available, competing for the attention of
`the video media viewer is now a daunting task. Although
`traditional written means of guidance for entertainment
`media selection such as reviews, episode guides, and summa(cid:173)
`ries might suffice if the video media viewer is sufficiently
`motivated to conduct the required research, many others may
`be intimidated by the amount of effort required, even aban(cid:173)
`doning the whole enterprise of watching the video media in
`the first place. To make optimal use oflimited leisure time, the
`video media viewer demands easy, instant access to media
`content that is custom tailored to the viewer's tastes and
`interests. Traditional entertainment delivery paradigms have
`so far largely failed to satisfy this demand, putting the onus of
`content selection on the viewer, often with limited guiding
`tools or assistance.
`[0005] Even on-demand media delivery systems, such as
`Internet streaming video, suffer from a lack of tools facilitat(cid:173)
`ing easy selection of media content. For example, a television
`drama might have several episodes spanning several seasons,
`maybe even hundreds of episodes for particularly long run(cid:173)
`ning programs. Program direction, cast, storylines, and pro(cid:173)
`duction values might vary wildly from one episode to another.
`A viewer facing this inscrutable mass of media content might
`decide to ignore it completely and do something else instead.
`Since the task of content selection is rendered so difficult by
`the sheer amount of media content and the lack of helpful
`guidance available, and only a small portion of the content
`may actually interest the viewer, a quick cost-benefit analysis
`may reveal that the time and effort spent to select the media
`content outweighs the rewards ofviewership.
`[0006] Accordingly, there is a need to overcome the draw(cid:173)
`backs and deficiencies in the art by providing a system to
`allow easy selection and viewing of video media content that
`interests a viewer, without requiring undue effort from the
`viewer.
`
`SUMMARY OF THE INVENTION
`
`[0007] There are provided systems and methods for visual
`search in a video media player, substantially as shown in
`and/ or described in connection with at least one of the figures,
`as set forth more completely in the claims.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[ 0008] The features and advantages of the present invention
`will become more readily apparent to those ordinarily skilled
`in the art after reviewing the following detailed description
`and accompanying drawings, wherein:
`[0009] FIG. la presents a diagram of a video media player
`interface, according to one embodiment of the present inven(cid:173)
`tion;
`[0010] FIG. lb and FIG. le present diagrams of naviga(cid:173)
`tional controls, according to one embodiment of the present
`invention;
`
`[0011] FIG. 2a and FIG. 2b present diagrams of a user
`interface for a video media player, according to one embodi(cid:173)
`ment of the present invention;
`[0012] FIG. 3 presents a diagram of video selection navi(cid:173)
`gational controls, according to one embodiment of the present
`invention;
`[0013] FIG. 4 presents a block diagram of a media network,
`according to one embodiment of the present invention; and
`[0014] FIG. 5 shows a flowchart describing the steps,
`according to one embodiment of the present invention, by
`which a computing device can provide visual search in a
`video media player.
`
`DETAILED DESCRIPTION OF THE INVENTION
`
`[0015] The present application is directed to a system and
`method for visual search in a video media player. The follow(cid:173)
`ing description contains specific information pertaining to the
`implementation of the present invention. One skilled in the art
`will recognize that the present invention may be implemented
`in a manner different from that specifically discussed in the
`present application. Moreover, some of the specific details of
`the invention are not discussed in order not to obscure the
`invention. The specific details not described in the present
`application are within the knowledge of a person of ordinary
`skill in the art. The drawings in the present application and
`their accompanying detailed description are directed to
`merely exemplary embodiments of the invention. To maintain
`brevity, other embodiments of the invention, which use the
`principles of the present invention, are not specifically
`described in the present application and are not specifically
`illustrated by the present drawings.
`[0016] FIG. la presents a diagram of a video media player
`interface, according to one embodiment of the present inven(cid:173)
`tion. Video media player interface 100 includes video media
`player 110, object 120, and navigation 130. FIG. lb and FIG.
`le present diagrams of navigational controls, according to
`one embodiment of the present invention. FIG. lb presents a
`detailed view of navigation 130, including navigation con(cid:173)
`trols 131, 132, 133, 134, and 135. FIG. le presents an
`expanded view of navigation 130, with the additional element
`navigation control 136.
`[0017] Video media player 110 in FIG. la depicts the play(cid:173)
`back of a video with multiple distinct objects. Starting from
`the left, the leaves of a tree, an empty chair, a table, and a
`reclining character named Juliet can be visually identified.
`Although navigation 130 shows the name of the character, it
`could also be configured to display the real name of the
`actress instead, based on a user preference or some other
`setting. As shown in FIG. la, Juliet is identified as object 120,
`highlighted and outlined from the rest of the video frame
`enclosed by video media player 110. To select Juliet as an
`object, a user might hover a pointer over Juliet in the video
`frame, or click on Juliet with a pointing device, such as a
`mouse or stylus. After such a selecting action is registered,
`Juliet might be highlighted as shown in FIG. 1. The display of
`video media player 110 might also be touch-sensitive,
`whereby a finger touch or tap acts as a pointing device click(cid:173)
`ing. Clicking on Juliet or hovering the pointer for a predeter(cid:173)
`mined time might further outline Juliet and cause navigation
`130 to pop up on top of video media player 110. Although
`highlighting and outlining are used in this example, other
`methods of visually identifying the object could be utilized as
`well, such as zooming, blinking, brightening, or any other
`contrasting visual technique. The data indicating whether the
`
`Page 1
`
`

`

`US 2010/0082585 Al
`
`Apr. 1, 2010
`
`2
`
`pointer is within the bounds of an object in a particular video
`frame might be embedded within metadata that is separately
`retrieved from the video, or embedded as a data stream within
`the retrieved media file. Generating the bounds data might be
`done with an object bounds recognition algorithm, a third
`party object bounds detection service, or simply by visual
`estimation with manual or computer assisted correction.
`[0018] Examining FIG. lb, numerous navigation controls
`are shown, allowing a viewer to seek, or change the immedi(cid:173)
`ate playback position, to search results or scenes that include
`the object indicated by navigation 130, or Juliet in FIG. lb.
`Navigationcontrol 131 allows the viewer to seek to a previous
`search result featuring Juliet, navigation control 132 seeks to
`the beginning of the current search result featuring Juliet,
`navigation control 134 seeks to the end of the current search
`result featuring Juliet, and navigation control 135 seeks to the
`next search result featuring Juliet. The navigation controls
`could also be configured to seek to different media files, if the
`beginning or the end of the current media file is reached. For
`example, if the video depicted in FIG. la is the first scene
`featuring Juliet in the second episode of a series, navigation
`control 131 might seek to the final scene that features Juliet in
`the first episode of the series. Furthermore, the navigation
`controls might seek to a completely different program, such
`as, for example, a late night talk show featuring the actor
`portraying Juliet as a guest. The navigation controls can be
`flexibly configured to seek a particular identified object
`through a desired scope of media breadth.
`[0019] Not everyone might be a fan of Juliet, so navigation
`control 133 allows a viewer to select other objects. Activating
`navigation control 133 might cause navigation control 136 of
`FIG. le to appear, presenting a list of alternative choices.
`Since Juliet might belong to the "characters" category, navi(cid:173)
`gation control 136 might be configured to only display other
`"characters," which in FIG. le include Jack, John Locke,
`Hurley, and Sayid. Selecting one of these other characters
`might cause navigation 130 to change focus to the selected
`character, and immediately seek forwards or backwards to the
`nearest search result featuring that character. Since using
`navigation control 136 may change the currently selected
`object, a new search may need to be initiated with the newly
`selected object as a search term. Although navigation control
`136 presents a single level menu displaying all characters for
`simplicity, an alternative embodiment might utilize a multi(cid:173)
`level menu to represent multiple sub-categorizations. For
`example, a top-level menu might provide a selection for male
`versus female characters, while a secondary menu might pro(cid:173)
`vide for a listing of characters on the "blue team" versus a
`listing of characters on the "red team." The preferred menu
`navigation complexity might additionally be selectable as a
`user preference.
`[0020] Besides Juliet, the other identifiable objects in FIG.
`la can be similarly navigated as well. For example, clicking
`a pointer over the table in the center of the video frame might
`cause the table to be highlighted and outlined, and the text of
`navigation 130 might change to read "Table." Thus, the navi(cid:173)
`gation controls of navigation 130 would allow a viewer to
`seek to all search results featuring the particular table in the
`video frame of FIG. la. Furthermore, clicking on navigation
`control 133 might display a navigation control 136 listing all
`objects in the "Furniture" category, including the empty chair
`to the left of the video frame and a bed shown in a different
`scene. Similarly, clicking on the tree to the far left might result
`in navigation 130 displaying "Tree", and clicking navigation
`
`control 136 might list all objects in the "Nature" category,
`perhaps including an "Ocean" object featured in a beach
`scene. In this manner, a viewer can quickly and easily navi(cid:173)
`gate to objects of interest as well as objects in related, well(cid:173)
`organized categories.
`[0021] This navigation system could also be applied to
`other categories of media such as sports footage, for example.
`Video media player 110 might instead feature a soccer match,
`and a viewer might be enabled to click on a particular player
`as an object. Video media player 110 could also be configured
`to tune into a wide range of sports rather than a single sport,
`such as various Olympic events, nationwide college sports
`games, and professional sports tournaments. In this case,
`particular types or sports, particular teams or countries, or
`particular sports players might be selected as objects. Since
`these objects might not be displaying in video media player
`110, a list of thumbnails showing the objects might be used
`for object selection instead. This method of selection from a
`list of text or thumbnails might be preferable to selecting an
`object within the video frame of video media player 110,
`since following and clicking a fast moving sports player
`might prove to be difficult. If a sports player is selected as an
`object, the selected "category" might represent the selected
`player himself, rather than a broader category including mul(cid:173)
`tiple players. Thus, ifthe selected player's name is Pepe, the
`object list of navigation control 136 might include "Pepe's
`Dribbling Footage," "Pepe's Goal Shots," and "Pepe's High(cid:173)
`lights" as selectable menu objects. Thus, if a viewer is a big
`fan of Pepe, he can easily view all the scenes where Pepe is
`controlling the ball by selecting "Pepe's Dribbling Footage,"
`or skip to the exciting scenes by selecting "Pepe's Goal
`Shots" or "Pepe's Highlights." If, instead of an individual
`player, a broader category such as "Olympics Highlights" is
`selected, the offered menu objects of navigation control 136
`might include "400 m Swimming Highlights," "Soccer High(cid:173)
`lights," and "Javelin Throw Highlights," to enumerate a few.
`Another category, "Olympics Team USA Gold Medals,"
`might present all the Olympic events in which Team USA
`earned a Gold Medal. An additional navigation control simi(cid:173)
`lar to play list 241 of FIG. 2b might allow the viewer to queue
`up some or all of the matching scenes into a playlist for
`successive playback, one after another. This might be particu(cid:173)
`larly useful in a mobile context, where a viewer can catch the
`highlights of a particular match, player, team, or sporting
`event on a mobile viewing device.
`[0022] Moving to FIG. 2a and FIG. 2b, FIG. 2a and FIG. 2b
`present diagrams of a user interface for a video media player,
`according to one embodiment of the present invention. Video
`media player interface 200 includes video media player 210,
`user interface 211, search terms 240, and thumbnails 250.
`Video media player 210 corresponds to video media player
`110 of FIG. la. FIG. 2b additionally includes object 220,
`corresponding to object 110 ofFIG. la, as well as playlist241
`and button 242.
`[0023] FIG. 2a shows an expanded view of the interface
`introduced with FIG. la, with additional user interface ele(cid:173)
`ments surrounding the video frame shown by video media
`player 210. Thumbnails 250 references seek points at various
`time offsets of the currently playing video, allowing a viewer
`to quickly seek through the video by glancing at the thumb(cid:173)
`nails and clicking on a desired section of the video. Each
`thumbnail also references a particular segment of video
`defined by a playback duration, with the starting point equal
`to the time offset. Visually, the thumbnails might be repre-
`
`Page 2
`
`

`

`US 2010/0082585 Al
`
`Apr. 1, 2010
`
`3
`
`sented using traditional still frame graphical images, or alter(cid:173)
`natively with animated preview videos.
`[0024] By default, the thumbnails might be initially config(cid:173)
`ured to stagger at a periodic interval, with equal and continu(cid:173)
`ous segments. For example, if one minute is chosen as the
`periodic interval, the first thumbnail might reference time
`offset 0:00, the second thumbnail might reference time offset
`1 :00, the third thumbnail might reference time offset 2:00,
`and so on, until available video segments are exhausted. Each
`thumbnail also indicates a playback duration of one minute,
`except possibly the final thumbnail, which may be shorter
`than one minute ifthere is insufficient remaining video con(cid:173)
`tent.
`[0025] Various additional navigational controls might
`extend the functionality of thumbnails 250. For example,
`should the number of thumbnails exceed the available width
`of the user interface, scrolling navigational controls might be
`added to the sides as shown in FIG. 2a, represented by the left
`and right facing triangles. Initiating a scrolling action shifts
`the visible set of thumbnails left or right. Additionally, a seek
`bar might indicate the currently selected thumbnail and the
`seek position within the playback duration defined by the
`thumbnail. Clicking on a different thumbnail or section of a
`thumbnail causes the video position and seek bar to move
`accordingly. In FIG. 2a, the seek bar is implemented as down(cid:173)
`ward and upward facing triangles that fades away thumbnail
`segments previously played. As indicated by the position of
`the seek bar in FIG. 2a, the entire first thumbnail and slightly
`more than half of the second thumbnail have already played
`so far, as indicated by the faded gray thumbnail frames.
`[0026] User interface 211 also adds search terms 240 to the
`left of the video frame. This portion of the user interface
`allows a user to build a list of objects for a visual search.
`Rather than typing in a search term, a viewer can simply
`visually scan the video and click on an object directly within
`the video, as previously detailed in FIG. la. For example, a
`viewer might click on the "Add Object" command under
`"Search Criteria," and click on the table in the middle of video
`media player 210. As a result, "Table" is added to the Search
`Criteria object list, and the table, object 220, is outlined and
`highlighted within the video frame, as indicated in FIG. 2b.
`[0027] At this point, a search is conducted and thumbnails
`250 are updated to reflect every thumbnail that matches the
`search parameters built under the "Search Criteria" list. Since
`only "Table" is listed, each highlighted thumbnail in thumb(cid:173)
`nails 250 contains at least one frame featuring the table within
`the one-minute segment defined by each thumbnail. In FIG.
`2b, counting from the left, the second, fourth, ninth, tenth, and
`twelfth thumbnails all have at least one scene with the table
`included. Additionally, each scene is populated into playlist
`241. Selecting the "Play all" button, button 242, plays the
`contents of play list 241 successively. Play list 241 might also
`allow flexible addition and removal of scenes. In this manner,
`a viewer can automatically view all matched scenes without
`manually selecting each one, potentially saving significant
`time and hassle for searches with a large number of matched
`scenes.
`[0028] Although the structure of thumbnails 250 is kept the
`same as FIG. 2a in FIG. 2b, the addition of objects in the
`"Search Criteria" list might cause a reconfiguration of thumb(cid:173)
`nails 250 to aid in results navigation. For example, a search
`that returns only results that are outside the scope of the
`visible thumbnails might automatically scroll the thumbnails
`such that at least one of the highlighted thumbnails is visible.
`
`Alternatively, the thumbnails might be filtered such that only
`results indicated by highlighted thumbnails are shown, with
`non-highlighted thumbnails hidden from view. If a search
`returns only sparse results, or if the search extends to other
`video files besides the currently playing video, hiding thumb(cid:173)
`nails unrelated to the search may improve results navigation
`for the viewer. Once the "Search Criteria" list is reset or the
`objects are removed, the thumbnails might revert to the
`default configuration displaying all thumbnails. A context
`menu accessible from clicking an object in the list might
`allow for object removal or emptying the list.
`[0029] When adding objects to "Search Criteria," the list
`may relate objects using OR operators by default, but the list
`could be configured to allow for advanced Boolean searching,
`supporting additional operators such as the AND operator and
`the NOT operator. For example, the plus sign designation"+"
`could represent the OR operator, whereas an "X" could rep(cid:173)
`resent the NOT operator and an ampersand"&" could repre(cid:173)
`sent the AND operator. These Boolean operators might be
`selectable by clicking on the operator symbol to the left of an
`object name to reveal a drop down menu allowing selection of
`a different operator. Order of operators might also be man(cid:173)
`aged by object text indentation and drop down menus.
`[0030] Moving to FIG. 3, FIG. 3 presents a diagram of
`video selection navigational controls, according to one
`embodiment of the present invention. Video selection navi(cid:173)
`gational controls interface 300 includes user interface 311,
`which contains thumbnails 350 and search box 360. If a
`viewer has not yet decided on a particular media file to play,
`a user interface for video media playback might be premature.
`It may thus be desirable to reserve a larger amount of display
`area for the display of several thumbnails, giving the viewer a
`broader overview of available video content to facilitate
`searching and selecting a video for viewing. Alternatively, the
`user interface 311 might be launched from a video media
`player to allow a broader search that may be unrelated to the
`currently playing video.
`[0031] Search box 360 allows a viewer to type in desired
`terms for searching, such as the name of a character, an actor,
`the title of a program, or other search criteria. Clicking on an
`interface element or pressing a confirmation key such as the
`Enter key on a connected keyboard might manually initiate
`the search. Alternatively, the search might be conducted auto(cid:173)
`matically as the user types or after a period of inactivity.
`Search box 360 might be flexibly configured to allow addi(cid:173)
`tional features such as auto-complete, previous search his(cid:173)
`tory, selection of text from preconfigured lists, support for
`Boolean operators, and fuzzy searching. Search box 360
`might be replaced or supplemented with an object selector,
`such as a navigation bar featuring thumbnail images or pre(cid:173)
`view videos of current programming or popular characters
`and actors. Once a particular search is confirmed, matching
`thumbnails are highlighted while non-matching thumbnails
`are dimmed as shown in thumbnails 350.
`[0032] The example shown in FIG. 3 indicates that the
`search term of search box 360 is "Kate", so each highlighted
`thumbnail in thumbnails 350 should include at least one video
`frame featuring the "Kate" character or actress. Each thumb(cid:173)
`nail can vary in length, from a short scene to a full episode,
`depending on the parameters of the search. Additional navi(cid:173)
`gational tools might be included to allow for search customi(cid:173)
`zation and filtering, such as restricting search terms to char(cid:173)
`acter names or actor names only, or restricting searches to
`only a particular program versus all available content. Addi-
`
`Page 3
`
`

`

`US 2010/0082585 Al
`
`Apr. 1, 2010
`
`4
`
`tionally, as discussed with thumbnails 250 in FIG. 2b, thumb(cid:173)
`nails 350 in FIG. 3 might be configured to hide the dimmed
`non-matching thumbnails to further emphasize the matching
`thumbnails, and restore the display of all thumbnails after
`search box 360 is cleared. In this manner, a viewer can
`quickly access desired content by submitting a search, visu(cid:173)
`ally scanning the matching thumbnails, and selecting a
`desired video result.
`[0033] Moving to FIG. 4, FIG. 4 presents a block diagram
`of a media network, according to one embodiment of the
`present invention. Network environment 400 includes com(cid:173)
`puting device 470, display 476, network 480, media server
`490, and search provider 494. Computing device 470 includes
`memory 472 and processor 474, and can display video
`through a connected display 476. Media server 490 includes
`media files 492a and 492b. Search provider 494 includes
`metadata 496a describing media file 492a and metadata 496b
`describing media file 492b. Network 480 provides data com(cid:173)
`munications between computing device 470, media server
`490, and search provider 494.
`[0034] FIG. 4 demonstrates an example environment in
`which the previously described user interfaces might operate,
`placing the user interfaces within the context of some imple(cid:173)
`menting hardware and supporting data sources. For example,
`a content producer might maintain a web-based video stream(cid:173)
`ing service from media server 490, streaming video files to
`the public. Only two media files, media file 492a and 492b,
`are shown in FIG. 4 to simplify the discussion. In real world
`conditions, a media server might include many media files.
`Similarly, search provider 494 in FIG. 4 only contains the
`metadata required to service media files from media server
`490, but search provider 494 might also provide search meta(cid:173)
`data services for other servers and media files. Additionally,
`multiple search providers might be used to service particular
`categories of searches. For example, one search provider
`might only provide simple text metadata such as keyword
`tags, whereas another search provider might specialize in
`facial recognition for detection of people, and yet another
`search provider might specialize in detection of inanimate
`objects. Thus, the metadata might include numerous different
`data representations such as text strings for textual metadata,
`pixel bitmasks and coordinate sets for object bounding,
`images and video clips for thumbnails, and other data sets.
`Multiple search providers might also be used to load balance
`search requests across multiple servers to improve perfor(cid:173)
`mance and user concurrency.
`[0035] Computing device 470 represents the hardware the
`viewer may use to view the media files. A few examples of
`such devices might include a desktop or notebook computer,
`a mobile phone, a portable gaming system, or a smart-phone.
`Memory 472 might be a dynamic random access (DRAM)
`chip, a hard disk drive (HDD), or some other storage medium
`accessible to processor 4 7 4 for future retrieval. Processor 4 7 4
`executes program logic for operating computing device 470.
`If computing device 470 comprises a desktop computer sys(cid:173)
`tem, it might be connected to an external display 476 as
`shown in FIG. 4, but integrated solutions such as notebook
`computers might integrate display 476 within computing
`device 470. A wired or wireless connection to network 480,
`which might represent the Internet, provides data communi(cid:173)
`cations to and from supporting data sources.
`[0036] For example, the viewer at computing device 470
`might decide to access the website hosted by media server
`490. After a connection is established, computing device 470
`
`might receive a user interface program with an appearance
`similar to FIG. la into memory 472 and execute the user
`interface using processor 474, thereby displaying the user
`interface onto display 476. Peripherals such as a mouse and a
`keyboard connected to computing device 470 or a touch sen(cid:173)
`sitive display 476 might enable interaction with the user inter(cid:173)
`face. After the user interface loads, the view