PTO/SB/05 (08-08)
`Approvedfor use through 01/31/2014. OMB 0651-0032
`U.S. Patent and Trademark Office, U.S. DEPARTMENT OF COMMERCE
`Under the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unlessit displays a valid OMB contro! number.
`
`U.S. PTO
`13/506093
`03/26/2012
`
`UTILITY -
`PATENT APPLICATION
`TRANSMITTAL
`
`(Only for new nonprovisional applications under 37 CFR 1.53(b))
`
`Express Mail Label No.
`
`APPLICATION ELEMENTS
`See MPEP chapter 600 concemingutility patent application contents.
`
`ADDRESSTO:
`
`Max Apewass's
`Providing Item Information During V
`
`EG 519486680 US
`Commissionerfor Patents
`P.O. Box 1450
`Alexandria VA 22313-1450
`
`
`
`Old‘SNO8vzZO
`
`
`
`2. a eee entity status.
`[Total Pages___57
`Specification
`Both the claims and abstract muststart on a new page
`{For information on the preferred arrangement, see MPEP 608.01(a))
`4.[¥] Drawing(s) (35 U.S.C. 113)
`[Total Sheets
`10
`5. Oath or Declaration
`[Total Sheets
`3
`a.
`Newly executed (original or copy)
`b. L__| A copyfrom a prior application (37 CFR 1.63(d))
`11. C1 English Translation Document(if applicable)
`for continuation/divisional with Box 18 completed)
`DELETION OF INVENTOR(S)
`12.
`Infopmption Disclosure Statement (PTO/SB/08 or PTO-1449)
`Signed statementattacheddeleting inventor(s)
`C
`.
`itati
`tt
`namein the prior application, see 37 CFR
`opies of citations attached
`1.63(d)(2) and 1.33(b).
`13. CI Preliminary Amendment
`6.[_] Application Data Sheet. See 37 CFR 1.76
`
`70 ‘ComputerPreoram(eippendii) large table or 14.[7]Return Receipt Postcard (MPEP 503)
`ondscape Table on CD
`(Should be specifically itemized)
`8. Nucleotide and/or Amino Acid Sequence Submission
`q
`y Orr
`’
`15. [_] Certified Copy of Priority Document(s)
`(if foreign priority is claimed)
`(if applicable, items a.-c. are required)
`a.
`Computer Readable Form (CRF)
`b.
`Specification Sequence Listing on:
`i. LJ
`cp-RoM or CD-R (2 copies); or
`i. [_]
`Paper
`Cc. LJ Statements verifying identity of above copies
`18. If a CONTINUING APPLICATION, check appropriate box, and supply the requisite information below andin the first sentence of the
`specification following thetitle, or in an Application Data Sheet under 37 CFR 1.76:
`CC Continuation
`[J Divisional
`J Continuation-in-part (CIP)
`Prior application information:
`Examiner
`
`
`
`1.[¥]
`
`Fee Transmittal Form (e.g., PTO/SB/17)
`
`ACCOMPANYING APPLICATION PARTS
`
`3.
`
`9. CJ Assignment Papers (cover sheet & document(s))
`Nameof Assignee
`
`:
`
`10. (_] 37 CFR 3.73(b) Statement
`-
`(when there is an assignee)
`
`[_ ]Power of
`Attorney
`
`i
`
`16. [_] Nonpublication Request under 35 U.S.C. 122(b)(2)(B)(i).
`Applicant must attach form PTO/SB/35 or equivalent.
`17. CJ other:
`
`of prior application No.: 0.000...eee
`Art Unit:
`
`19. CORRESPONDENCE ADDRESS
`
`Correspondenceaddressbelow
`LJTheaddressassociatedwith CustomerNumber: fo OR
`RaneYuaraieenssSSSCSCSCSCSCSC“SCSCSC*d
`Address
`18457 Long Lake Drive
`
`
`feouneyusa77|Teeterossaracicos|Frat_|max@customplaycom=
`4
`(Stnawe|EK———————SYte osrz62012
`Name
`x.
`.
`Registration No. P|
`
`This collection of information is required by 37 CFR 1.53(b). The information is required to obtain or retain a benefit by the public which is tofile (and by the
`USPTOto process) an application. Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1.11 and 1.14. This collection is estimated to take 12 minutes to
`complete, including gathering, preparing, and submitting the completed application form to the USPTO.Time will vary depending upon the individual case. Any
`comments on the amountof time you require to complete this form and/or suggestions for reducing this burden, should be sent to the Chief Information Officer,
`U.S. Patent and Trademark Office, U.S. Department of Commerce, P.O. Box 1450, Alexandria, VA 22313-1450. DO NOT SEND FEES OR COMPLETED
`FORMSTO THIS ADDRESS. SEND TO: Commissionerfor Patents, P.O. Box 1450, Alexandria, VA 22313-1450.
`if you need assistance in completing the form, call 1-800-PTO-9199 and sele
`
`Page 1
`
`AMAZON EX.1010
`Amazon v. CustomPlay
`US Patent No. 9,380,282
`
`Page 1
`
`AMAZON EX. 1010
`Amazon v. CustomPlay
`US Patent No. 9,380,282
`
`

`

`Under the Paperwork Reduction Act of 1995 no
`
`PTO/SB/17 (09-11)
`Approvedfor use through 01/31/2014. OMB 0651-0032
`U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE
`persons are required to respondto a collection of information unlessit displays a valid OMB contro! number
`
`Application Number
`
`Complete if Known
`
`
`
`
`
`
`
`
`
`
`FEE TRANSMITTAL
`
`Feaninrnane[Sd
`anu
`
`
`
`(1975__[atoneydoses|
`
`
`
`
`METHOD OF PAYMENT(checkall that apply)
`Check [| Credit Card L_]Money Order [] None L_Jother (please identify):
`[| Deposit Account Deposit Account Number_______SsDeposit Account Name:
`For the above-identified deposit account, the Director is hereby authorized to: (check all that apply)
`
`
`[charge fee(s) indicated below
`[] Chargefee(s) indicated below, exceptforthefiling fee
`
`Charge any additional fee(s) or underpaymentsoffee(s) [| Credit any overpayments
`
`
`under 37 CFR 1.16 and 1.17
`
`
`WARNING: Information on this form may becomepublic. Credit card information should not be included on this form. Provide credit card
`
`information and authorization on PTO-2038.
`
`FEE CALCULATION
`
`1. BASIC FILING, SEARCH, AND EXAMINATION FEES
`
`FILINGFEES '
`SEARCH FEES
`EXAMINATION FEES
`
`
`Application Type
`Fee ($) Sed)
`Fee ($) ae is)
`Fee ($) “ee 3).
`Fees Paid ($)
`mall
`Enti
`Small Entity
`Small Entity
`
`
`Utility
`380
`190
`620
`310
`250
`125
`625
`
`
`Design
`250
`125
`120
`60
`160
`80
`Plant
`250
`125
`380
`190
`200
`100
`
`
`Reissue
`380
`190
`620
`310
`750
`375
`
`
`Provisional
`250
`125
`0
`0
`0
`0
`Small Entity
`_2. EXCESS CLAIM FEES
`
`Fee DescriptionFee($) Fee ($)
`Each claim over 20 (including Reissues)
`60
`30
`Each independentclaim over 3 (including Reissues)
`250
`125
`Multiple dependent claims
`450
`225
`Total Claims
`Extra Claims
`Multiple Dependent Claims
`x
`20
`- 20 or HP =
`0
`Fee ($)
`Fee Paid ($)
`HP = highest numberoftotal daimspaid for, if greater than 20.
`
`
`Indep.ClaimsExtraClaimsFee($)FeePaid($)
`
`
`9g
`- 3 orHP =
`6
`x
`125
`=
`750
`
`
`HP = highest numberof independent daimspaidfor,if greater than 3.
`.
`.
`3. APPLICATION SIZE FEE
`If the specification and drawings exceed 100 sheets of paper (excluding electronically filed sequence or computer
`listings under 37 CFR 1.52(e)), the application size fee due is $310 ($155 for small entity) for each additional 50
`sheets or fraction thereof. See 35 U.S.C. 41(a)(1)(G) and 37 CFR 1-16(s).
`Fee Paid ($)
`Total Sheets
`Extra Sheets
`Numberof each additional 50 or fraction thereof
`0
`- 100 =
`Q
`/50=
`(round up to a whole number)
`x
`
`Fees Paid ($)
`4. OTHER FEE(S)
`
`$130 fee (po small entity discount)
`0
`Non-English Specification,
`
`
`Other(e.g., late filing surg arge)/V
`
`
`
`fsupmitmepBYffLY
`
` Telephone 5¢1.278-1096
`Registration No.
`:
`/
`
`
`Date 03/26/2012
`Name(Print/Type)} Max ‘Abecassit~
`
`
`This collection of information is required by 37 CFR 1.136. Theinformation is required to obtain or retain a benefit by the public which is to file (and by the
`USPTOto process) an application. Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1.14. This collection is estimated to take 30 minutes to complete,
`induding gathering, preparing, and submitting the completed application form to the USPTO. Timewill vary depending upon the individual case. Any comments
`on the amount of time you require to complete this form and/or suggestions for reducing this burden, should be sent to the Chief Information Officer, U.S. Patent
`and Trademark Office, U.S. Department of Commerce, P.O. Box 1450, Alexandria, VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMSTO THIS
`ADDRESS. SEND TO: Commissionerfor Patents, P.O. Box 1450, Alexandria, VA 22313-1450.
`if you need assistance in completing the form, call 1-800-PTO-9199 andselect option 2.
`
`
`
`!P
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Fee ($)
`
`Fee Paid ($)
`0
`
`=
`
`
`
`
`
`
`
`
`
`
`
`Fee ($)
`
`=
`
`Page 2
`
`Page 2
`
`

`

`TITLE OF THE INVENTION
`
`Providing Item Information During Video Playing.
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`[0001] Systems for, and methodsof, providing, a user, during a playing of a video, an
`identification of a purchasable item that is depicted in the video. The sourceofthe identifying
`information is provided by a mapofthe videothat identifies the items that are visually depicted
`within the video. The map of the video maybe provided by the source that provides the video or
`by a sourceotherthan the source that provides the video. A source of the video may be a hard
`format such as a DVD orBlu-raydisc, or a soft formatthat is broadcasted, narrow casted,
`delivered on-demand, streamed, and/or downloaded by a variety of means such as over-the-air
`broadcast, cable,satellite, cellular, internet, and any other meansofreceiving content/data. The
`means for playing the video comprises any device capable ofplaying video suchas, for example,
`an optical disc player, a personal computer, a set box, a multimedia player, gaming device,
`and/or smart phone.
`
`2. Description of the Related Art.
`
`[0002] DVDsand Blu-rays are “optical discs” capable ofstoring one full length motion
`picture in one of two layers oneitherside of the disc. Optical discs generally provide scene
`selection and subtitles and/or closed captioning in a plurality of languages. Amongother
`features, someoptical discs include multiple camera angles for a scene, and the capability to play
`one ofa plurality of different content versions(e.g. a director's cut or unrated version and an "R"
`
`rated versions). Blu-rays provide more advanced features such as a search functionthat list
`
`keywords corresponding to noteworthy names, items,.and actions, and enable a userto select a
`
`keyword to retrieve segments matching the keyword selection.
`
`[0003] Devices capable of playing optical discs provide for, among other features,fast-
`
`forward, fast-rewind, skip-forward or skip-backwards by chapters, turn on/off subtitles, subtitle
`
`Page 3
`
`Page 3
`
`

`

`and audio language selection, camera angle selection, and optical disc menu navigationtoselect,
`for example, multiple content versions.
`
`[0004] Whilethe optical discs formats provide many capabilities and functions exceeding
`those provided by, for example, conventionallinear playing formats, manyofthe capabilities and
`functions do not fully realize the potential of a random accessvideo playing capability that is
`synergistically integrated with a mapofthe content of the video. Optical disc playing devices do
`not provide a readily accessible identification of purchasable item depicted in a segment within a
`video. These deficiencies are not addressed by the many sources that may provide an
`
`identification of the purchasable items present in a video, including, for example, a variety of
`
`websites, or various sources that may identify purchasable items presentin a video.
`
`BRIEF SUMMARYOF THE INVENTIONS
`
`[0005] The present inventionsrelate generally to system and methodsfor providing a
`
`user, during playing of a segment from within a video, an identification of a purchasable item
`
`that is depicted in that segment. By contrast to the usual passive viewing of a video, or the
`
`interactive playing ofa video,as in the playing of a computer game,a primary elementofthe
`
`"autoactive" systems and methodologies disclosed herein is the synergistic integration of a map
`
`of the content of the video with the playing of the videoto satisfy the real-time informational
`
`interests of a user and to deliver a more informed and entertaining video viewing experience.
`
`[0006] Accordingly, it is an object of the present inventions to provide, during a playing
`
`of a video, with efficient access to information that identifies the purchasable item that is
`
`depicted in a segmentwithin a video.
`
`[0007] It is also an objectof a particular on of the present inventions to provide, during a
`
`playing of a video, an indication that item informationis available for an item being currently
`
`depicted within the video.
`
`[0008] It is also an objectto associate the identification of the purchasable item with
`
`other information that may be available, internally or externally, for a selected purchasable item.
`
`Page 4
`
`Page 4
`
`

`

`[0009] It is also an object to provide for an appropriately labeled key on a remote control
`
`device for efficient access to the purchasable item information function.
`
`[0010] Briefly, these and other objects are accomplished by autoactive capabilities and
`
`functions provided by systems comprising, and methodologiesutilizing processing, random
`
`accessing, and control software functions, integrated with video segment information, such as
`
`informationthat identifies items that are depicted within the video. The creation of the segment
`information comprises the analysis of the content of a video to define segments within the video
`
`responsive to the depiction of a significant or noteworthy item. Segments maybe associated
`
`with identifying information, purchase information, additional item information, information that
`
`mayberetrieved from sources other than the video, and other descriptors relating to the
`
`depiction of that item. Each of the itemsis associated with an image ofthe item from the video,
`
`preferably a close-up shot featuring the item. In one of a numberof inventions disclosed herein,
`
`during playing of the video, whena user desires to obtain information on an item depicted during
`
`a playing of a video, the user activates item notification routines that display an indication that
`item informationis available for an item being depicted; the user activates item identification
`routines by requesting item information relating to an item depicted within the videoorin
`
`responseto the a display indicating item informationis available; the video is paused in response
`
`to the request for item information;a requestlocation is identified that is responsive to the
`request for item information, the request location may also comprise a predeterminedplaying
`period prior to the request location; a plurality of segment definitions is searched to identify
`segmentdefinitions that are responsive to the request location, a segmentdefinition defines,
`
`responsive to a depiction of an item, a beginning and an ending of a video segment within the
`
`video, a segmentdefinition is associated with item information; and item information associated
`with a responsive segment definition is displayed. If the user desires to obtain additional item
`
`information, additional item information is provided in response to the request for additional
`
`item information, the additional item information may comprises item purchase information.
`
`Following a termination of a displaying of additional item information, playing of the video is
`
`resumed at the request location.
`
`Page 5
`
`Page 5
`
`

`

`[0011] This and other inventions, embodiments, features, advantages, and objects, are
`apparentin the context of the detailed description, accompanying drawings, and appended claims
`
`that follow.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0012] The following drawingfigures, which formapart ofthis application, are
`illustrative of embodiments of the present inventions and are not meant to limit the scope ofthe
`inventions in any manner.
`
`[0013] FIG. 1A is a representation of a video frame within a motionpicture and the
`
`display of an indication that informationis available for an item being depicted;
`
`[0014] FIG. 1B is a representation of a video frame within a motion picture and the
`
`highlighting of an item for which informationis available;
`
`[0015] FIG. 1C is a representation of a video frame with a display of item information;
`
`[0016] FIG. 1D is a representation of a video frame following the video frame shown in
`
`FIG. 1C;
`
`[0017] FIG. 1E is a representation of a frame accurate display of item information;
`
`[0018] FIG. 1F is a representation of a video screen where the display of item
`
`information is not superimposed on the video;
`
`[0019] FIGS. 2A is a representation of a video frame within a video comprising a play
`
`control panel;
`
`[0020] FIG. 2B is a magnified representation of the play control panel shownin FIG. 2A;
`
`[0021] FIG.3 is an illustration of a remote control device;
`
`[0022] FIG.4A is a flow chart of a set of steps for providing a user item information for
`
`an item depicted in a video;
`
`Page 6
`
`Page 6
`
`

`

`[0023] FIG. 4B is a flow chart of a set of steps for monitoring the depictions of items
`
`within a video and providing a user an indication that item informationis available;
`
`[0024] FIG.5 is a flow chart of a method for creating a video map;
`
`[0025] FIG. 6 is a schematic diagram of a multi-featured multimedia player; and
`
`[0026] FIG. 7 is a schematic diagram of a video provider and end user network
`
`architecture.
`
`DETAILED DESCRIPTION OF THE INVENTIONS
`
`[0027] U.S. Patents 5,434,678; 6,192,340; 6,208,805; 6,408,128; 6,504,990; 6,553,178;
`
`and their respective divisions are incorporated herein by reference. Where a particular detail
`
`disclosed in an incorporated reference and a detail described here may appear inconsistent, the
`
`detail disclosed in the incorporated reference should be understoodas an alternative detail
`
`description.
`
`[0028] For purposesof the present disclosure, various terms used in the art are defined as
`
`follows:
`
`[0029] The term “associated”refers to associated, linked, related, connected, broughtinto
`
`relationship, joined, designated, ascribe, and/or attributed.
`
`[0030] The term “character”refers to a role or a part in a video, and/ora participant in a
`
`video, typically displaying human-like qualities.
`
`[0031] The term “chapter” refers to a division of a video generally defined by the author
`
`of the video. Optical discs, for example, typically have multiple chapters such as with DVDs,
`
`which often refer to a chapter as a scene.
`
`[0032] The term “clip” refers to a segment that is smaller than a chapter and usually
`
`smaller than a scene. A clip includes one or more contiguousshots, and usually depicts the same
`
`primary characters within a location. A clip’s definition is responsive to a material changein the
`
`Page 7
`
`Page 7
`
`

`

`participation ofthe principal characters, a material change in location, and/or a distinct change in
`thematic content or topic of conversation.
`
`[0033] The term "content preferences”refers, specifically and principally, although not
`
`exclusively, to a user's preferences for the form of expression, explicitness, the absence of
`
`objectionable segments, content categories and levels, length, detail, subjects, versions, thematic
`
`content, and items. A set of content preferences also includes technical preferences and
`
`presentation preferences. In the broadest sense, the term content preferences further comprises
`
`video, programming, and subject matter preferences(collectively "programmingpreferences").
`
`Programmingpreferencesrefer exclusively to a user's preferences as to specific videos(e.g.
`
`Sega's "Sherlock Holmes Consulting Detective"), types of videos(e.g. interactive video detective
`
`games), broad subject matter of videos (e.g. mysteries), and/or time and date that the user may
`
`desire to view a video. Programmingpreferencesare principally directed at the selection of a
`
`specific video.
`
`[0034] The term "content-on-demandsystem"refers to a video-on-demand system thatis
`
`responsive to a user's content preferences. A content-on-demand system not only permits a user
`
`to obtain a specific video(s) or segment(s) of videos in responseto the user's programming
`
`preferences, a content-on-demandsystem also permits the user to receive a version of the
`
`video(s) or segment(s) of videos that are responsiveto the user's other types of content
`
`preferences. A content-on-demand system and the delivery of variable content video services as
`
`described herein mayutilize a variety of types of hardware and network infrastructure and are
`
`notlimited to any particular hardware or networkinfrastructure that may be employedin a given
`
`implementation.
`
`[0035] The terms“descriptors” and “keywords”refer to words, terms, phrases, keywords,
`
`designations, codes, descriptors, labels, data, metadata, numbers,or other information that
`
`identifies, describes, categorizes, and/or provides information. Descriptors and keywords
`
`comprise, for example, wordsthat identify, describe, categorize, and/or provide information
`
`about an item depicted within a video.
`
`Page 8
`
`Page 8
`
`

`

`
`
`
`[0036] The term “dialog”refers to a dialog, conversation, monologue, lyric, utterance,
`
`and/or communication. Dialog also comprises information that may be includedin thesubtitles
`
`or the closed captioning.
`
`[0037] The term “feature-length movie”refers to a feature-length movie, a feature-length
`
`motion picture, a full-length motionpicture, full-length movie, feature movie, feature film, and
`
`the like. A feature-length movieis a video which runs for 40 minutesor longer. Theatrically-
`
`released full-length movies released by a major motion picture studio generally exceed 80
`
`minutes. A feature-length movie does not require a particular format(e.g., digital, analog,
`
`dimension) or media/transport (e.g., film).
`
`[0038] The term “item”refers to: (i) items, products, objects, properties, acts, or
`
`information, whether or not they offer a commercial or purchase opportunity; (ii) items,
`
`products, objects, properties, acts, or information that constitute actual or potential product
`
`placements, whetherprior to or after production of the video; (iii) product placements; and(iv)
`
`items, products, objects, properties, acts, or information, that offer informational or
`
`entertainment opportunities. The term “product placement”refers to: (i) items, products, objects,
`
`properties, acts, or information that offer a commercial or purchase opportunity; (ii) items,
`
`products, objects, properties, acts, or information that constitute actual or potential product
`
`placements, whetherpriorto or after production of the video; and (iii) product placements.
`
`[0039] The term “keywording”refers to associating descriptors and/or keywords.
`
`[0040] The term "network"herein refers to any private or public, wired or wireless video
`
`transmission infrastructure, such as maybeprovidedbythe utilization of one or a hybrid
`
`combination ofthe following: fiber optic, coaxial cable, twisted copper wire,cellular, radio,
`
`satellite, and/or other existing and/or forthcoming video transmission technologies. Examples
`
`include a BroadbandIntegrated Services Digital Network ("B-ISDN")utilizing fiber optic cable
`
`in its primary trunks, a Digital Subscriber Line that provides high bandwidth over twisted
`
`copper-pair phonelines, and the internet.
`
`In the broadest sense, a network also comprises the
`
`utilization of, for example, the U.S. Postal Service to deliver DVDs.
`
`Page 9
`
`Page 9
`
`

`

`[0041] The term “noteworthy” in connection with an item refers to an item that: (i) may
`
`be ofinterest to a significant audience; (ii) is noteworthy, remarkable, or compelling;(iii) is
`
`uncommon,atypical, peculiar, unusual, unique, rare, or extraordinary; (iv) is purchasable; and/or
`
`(v) is a product placement.
`
`[0042] The term “performer”refers to an individual, participant, actor, or actress,
`
`appearing in a video and/orcredited for the physical and/or the verbal performanceof a
`
`character. A performerrefers to, for example, an actor in a motionpicture, an athlete in a
`
`televised sporting event, a newscaster in a news program,and a chefin a cooking show.
`
`[0043] The term "playing", dos nor require a playing ofthe entirety of a segmentor
`
`portion of a video and does not necessarily require that a playing of every aspect of the video,
`
`audio, sub picture, and/orbit of data of a frame or segment.
`
`[0044] The term “search terms”refers to terms, words, phrases, designations, codes,
`
`descriptors, labels, data, metadata, numbers, or other information that identifies, describes, or
`
`specifies what is being searched.
`
`[0045] The terms "segment" and “video segment” refer to one or more video frames.
`
`[0046] The term "shot" refers to a sequence of video framesresulting from a single
`
`continuous run of a video camera.
`
`[0047] The term "user" as used herein is meantto include and be interchangeable with
`
`the words "player" (whenreferring to a person), "subscriber", and "viewer". Thatis, the term
`
`"user" is to be understood in the general sense of, for example, a person actively using video
`
`content, passively viewing a video,interactively playing a video game,retrieving video from a
`
`video provider, and/or actively using multimedia, internet, and/or communication services.
`
`[0048] The term "variable content video" refers to a video characterized by a nonlinear
`
`architecture facilitating a variety of possible logical sequences of segments. A variable content
`
`video comprises segmentinformation andparallel, transitional, and/or overlapping segments.
`
`Theparallel, transitional, and/or overlapping segments may support, for example, objectionable
`
`content control and focused versions. A variable content video mayalso includea userinterface,
`
`Page 10
`
`Page 10
`
`

`

`software program routines, and system control codes for controlling the playing of the
`
`video/audio.
`
`[0049] The terms "video" and "video program" are interchangeable and refer to any
`video image regardless of the source, motion, or technology implemented. Unless otherwise
`
`qualified to mean a computer software program,the term "program"is interchangeable and may
`be replaced with the word video. A video comprises images foundin full motion picture
`
`programsandfilms,in interactive electronic games, and in video produced by multi-media
`
`systems. Video comprisesstill characters, graphics, images, motion pictures,films, and
`
`multimedia productions; feature-length movies, shorts, and television programming; news,
`
`sports, cultural, entertainment, commercial, advertisements, instructional, and educational
`
`programming. Asthe context permits, a video may also comprise video, audio, sub picture
`
`information, a video map (e.g., navigation data), or other information associated with the video,
`
`such as a user interface, software program routines, and system control codes.
`
`[0050] The terms “video map”, “map”, and “segment map”, refer to a combination,
`
`arrangement, table, database,or listing of information comprising an identification of a
`
`beginning and ending of one or more segments andat least one descriptor associated with at least
`
`one segment or a sequence of segments. A video map comprises, for example, a combination,
`
`arrangement, table, listing, tables and fields, database, metadata, or xmlfile, or other data
`
`comprising segment information.
`
`[0051] The term "video-on-demand system" refers to any video delivery system thatis
`
`responsive to a user's programming preferences, i.e. provides a user-selected video at a user-
`
`selected time. Video-on-demand comprises for example movies-on-demand,video dial tone,
`
`cellular video, and digital satellite systems that are responsive to a user's time and date
`
`preferences and that provide play control functionality in the viewing of a video.
`
`[0052] Where not clearly and unambiguously inconsistent with the specific context, these
`
`and other terms defined herein are to be understood in the broadest sense.
`
`[0053] The teachings herein with respectto a video maybe applied to, for example, any
`type or kind of content that may be captured in a video format, including motionpictures,
`
`Page 11
`
`Page 11
`
`

`

`movies, television programs, news programs,sports programs, educational videos,
`advertisements, informational, commercials, and other videos that promote products and
`services. While a particular feature may be detailedwith respect to a specified type of video,it is
`intendedto apply the teachings herein broadly and harmoniously acrossall different types and
`classes of video, including, for example, and not by wayof limitation, a variable content video
`
`and a variable content game.
`
`[0054] FIG. 1A is a representation of a video frame within a motionpicture and the
`
`display of an indication that informationis available for an item being depicted. The frame 100
`
`of the video depicts three characters: a boy 110 holding balloons 111, a girl 120 wearing a dress
`
`121 and a man 130 wearing a hat 131.
`
`In this particular invention, the play location is monitored
`
`for the presence of a purchasable item. The current play location within a video is monitored and
`
`a plurality of segmentdefinitions is searchedto identify a segment definition that is responsive to
`
`the play location. In the instance that a segment-definition is responsive to the play location, and
`
`during the period defined by the segmentdefinition, an indication 140 is displayed that item
`
`information is available for an item or items being depicted. In this example, the indication is an
`
`icon of a shopping bag 140 superimposed in each frame of video during the period defined by
`
`the responsive segment.
`
`[0055] A segmentdefinition defines, responsive to a depiction of an item, a beginning
`
`and an ending of a video segmentwithin the video. A segmentdefinition is associated with item
`
`information. There are a numberofdifferent methods that may be implemented for a segment
`
`definition to define a beginning and an ending of a video segment within the video. For
`
`example, a segmentdefinition maydirectly specify a beginning pointofa portion of content
`
`(e.g., a segment) within the video, and directly specify an ending point of a portion of content
`
`within the video. In a second example, a segment definition may directly specify a beginning
`
`point of a portion of content within the video, and indirectly specify an ending point of a portion
`
`of content within the video by specifying a duration of the portion of content. In a third example,
`
`a segmentdefinition mayindirectly specify a beginning point of a portion of content within the
`
`video and indirectly specify an ending point of a portion of content within the video, by, for
`
`example, individually specifying oneor a plurality of individual units of content(e.g., a video
`
`frame or a group of pictures). Accordingly, a segmentdefinition may define, responsive to a
`
`10
`
`Page 12
`
`Page 12
`
`

`

`depiction of an item, a beginning and an ending of a video segment within thevideo by, for
`
`example, specifying a single video frame. Such as segmentconsists of a single video frame. A
`
`video frame is any imageorpartial imagein any digital or analog format comprising,for
`
`example, a frame of video, a video field, an I, B, or P frame. A video frame may comprise audio
`information. The term “specify” should be understood in the broadest sense and comprises the
`
`meaning associated with the following terms: detail, indicate, enumerate,stipulate, identify, and
`
`their respective synonyms. The terms “segment definition” and ‘“‘segmentdefinitions” should be
`understood in the broadest sense and comprises the meaning associated with the following terms:
`datum, data, information, andtheir respective synonyms.
`
`[0056] Similarly, there are a variety of methods for associating item information with a
`
`segment definition. For example, item information is associated with an item,the item is
`
`associated with a frame location where the item is depicted within the video, and the frame
`
`location is associated with a segmentdefinition. In that example, item informationis indirectly
`
`associated with a segment definition. Information associated with a segmentdefinition does not
`
`require any particular direct or indirect methodology.
`
`[0057] A beginning and ending of a segment maybe defined by a variety of means
`
`including, for example, time codes in a seconds and milliseconds format or an HH:MM:SS:FF
`
`(Hours, Minutes, Seconds, and Frames) format, bit positions, logical positionsidentifiers, or any
`
`format that can consistently identify a location or position in a video. A segmentdefinition may
`
`be individually responsive to a single depicted item or to multiple depicted items. In those
`
`instances where a segmentdefinition is associated with only a single item. Advantageously,
`
`segmentdefinitions may overlap but not necessarily congruently.
`
`[0058] An indication that item informationis available for an item can take many forms,
`
`including, for example, text, actual images of the item, representations of the items, or anything
`
`visual or audio that would signal to the user that informationis available on an item being
`depicted. For example, asis illustrated in FIG. 1B,alternatively or additionally, the item being
`depicted for which information is available maybe highlighted by an outline 141, or highlighted
`by an area of increased brightnessas if a spotlight was focused on the item. Such an
`
`embodiment may implement item frame coordinate and dimension information associated with a
`
`1]
`
`Page 13
`
`P