`
`
`
`
`
`
`
`
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`__________________________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`__________________________________
`
`AMAZON.COM, INC.,
`Petitioner,
`
`v.
`
`CUSTOMPLAY, LLC,
`Patent Owner.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Case No. TBD
`U.S. Patent No. 9,380,282
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`DECLARATION OF ALAN C. BOVIK IN SUPPORT OF
`PETITION OF AMAZON.COM, INC. FOR
`INTER PARTES REVIEW OF U.S. PATENT NO. 9,380,282
`
`
`
`AMAZON EX. 1002
`Amazon v. CustomPlay
`US Patent No. 9,380,282
`
`

`

`I.
`
`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`TABLE OF CONTENTS
`
`BACKGROUND ............................................................................................. 2 
`A. 
`Experience and Qualifications .............................................................. 2 
`B. 
`Applicable Legal Standards .................................................................. 7 
`C. 
`Person of Ordinary Skill in the Art ..................................................... 10 
`II. BACKGROUND OF THE TECHNOLOGY ................................................ 12 
`III. THE ’282 PATENT ....................................................................................... 15 
`A. 
`Summary of the ’282 Patent ................................................................ 15 
`1. 
`The Specification ...................................................................... 15 
`2. 
`The Claims ................................................................................ 16 
`The Priority Date of the ’282 Patent ................................................... 17 
`B. 
`Claim Construction ............................................................................. 17 
`C. 
`IV. CLAIMS 4, 7-9, 12, 14, 16, 18, AND 19 OF THE ’282 PATENT
`WOULD HAVE BEEN OBVIOUS .............................................................. 17 
`A. 
`Claims 4, 9, 12, 14, 16, and 19 Would Have Been Obvious
`in View of McIntire and Dey. ............................................................. 18 
`1. 
`Claim 4 ...................................................................................... 19 
`a. 
`Preamble ......................................................................... 19 
`b. 
`Receiving a Request for Information ............................. 20 
`c. 
`Identifying a Request Location ...................................... 22 
`d. 
`Retrieving a First Video Frame Identifier that is
`Responsive to the Request Location ............................... 24 
`Contemporaneously Retrieving a Second
`Video Frame Identifier ................................................... 25 
`
`e. 
`
`-i-
`
`

`

`2. 
`3. 
`
`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`Displaying Information Associated with the
`f. 
`First Video Frame Identifier ........................................... 34 
`Contemporaneously Displaying Different
`Information Associated with the Second
`Video Frame Identifier ................................................... 36 
`Claims 14 and 19....................................................................... 37 
`Claim 9 ...................................................................................... 37 
`a. 
`Receiving a User Request for Additional Information ... 38 
`b. 
`Enabling Display of the Additional Information ............ 40 
`Claim 12 .................................................................................... 40 
`4. 
`Claim 16 .................................................................................... 41 
`5. 
`Claims 7, 8, and 18 Would Have Been Obvious in View
`of McIntire, Dey, and Abecassis. ........................................................ 42 
`1. 
`Claim 7 ...................................................................................... 42 
`a. 
`Pausing in Response to the Request for Information ..... 43 
`b. 
`Resuming Playing at Beginning of Video Clip
`Responsive to Request Location .................................... 43 
`Claim 8 ...................................................................................... 45 
`2. 
`Claim 18 .................................................................................... 48 
`3. 
`Claim Chart ............................................................................... 48 
`4. 
`Claims 4, 9, 12, 14, 16, and 19 Would Have Been Obvious
`in View of Bergen and Reimer. ........................................................... 67 
`1. 
`Claim 4 ...................................................................................... 70 
`a. 
`Preamble ......................................................................... 70 
`b. 
`Receiving a Request for Information ............................. 70 
`
`B. 
`
`C. 
`
`g. 
`
`-ii-
`
`

`

`e. 
`
`f. 
`
`g. 
`
`2. 
`3. 
`
`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`Identifying a Request Location ...................................... 73 
`c. 
`d. 
`Retrieving a First Video Frame Identifier that
`is Responsive to the Request Location ........................... 76 
`Contemporaneously Retrieving a Second
`Video Frame Identifier ................................................... 79 
`Displaying Information Associated with the
`First Video Frame Identifier ........................................... 80 
`Contemporaneously Displaying Different
`Information Associated with the Second
`Video Frame Identifier ................................................... 81 
`Claims 14 and 19....................................................................... 82 
`Claim 9 ...................................................................................... 83 
`a. 
`Receiving a User Request for Additional Information ... 83 
`b. 
`Enabling Display of the Additional Information ............ 84 
`Claim 12 .................................................................................... 84 
`4. 
`Claim 16 .................................................................................... 87 
`5. 
`Claims 7, 8, and 18 Would Have Been Obvious in View
`of Bergen, Reimer, and Abecassis. ..................................................... 87 
`1. 
`Claim 7 ...................................................................................... 87 
`a. 
`Pausing in Response to the Request for Information ..... 87 
`b. 
`Resuming Playing at Beginning of Video Clip
`Responsive to Request Location .................................... 88 
`Claim 8 ...................................................................................... 90 
`Claim 18 .................................................................................... 92 
`Claim Chart ............................................................................... 93 
`
`2. 
`3. 
`4. 
`
`D. 
`
`-iii-
`
`

`

`e. 
`
`f. 
`
`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`Claims 4, 9, 12, 14, 16, and 19 Would Have Been Obvious
`E. 
`in View of Armstrong and the Knowledge of a Person of
`Ordinary Skill in the Art. ................................................................... 115 
`1. 
`Claim 4 ....................................................................................115 
`a. 
`Preamble ....................................................................... 115 
`b. 
`Receiving a Request for Information ........................... 115 
`c. 
`Identifying a Request Location .................................... 116 
`d. 
`Retrieving a First Video Frame Identifier that is
`Responsive to the Request Location ............................. 116 
`Contemporaneously Retrieving a Second Video
`Frame Identifier ............................................................ 118 
`Displaying Information Associated with the
`First Video Frame Identifier ......................................... 124 
`Contemporaneously Displaying Different
`Information Associated with the Second Video
`Frame Identifier ............................................................ 126 
`Claims 14 and 19.....................................................................127 
`2. 
`Claim 9 ....................................................................................127 
`3. 
`Claim 12 ..................................................................................128 
`4. 
`Claim 16 ..................................................................................129 
`5. 
`Claim 7 Would Have Been Obvious in View of Armstrong,
`Alone or in Combination with Abecassis. ......................................... 131 
`1. 
`Claim 7 ....................................................................................131 
`2. 
`Claim Chart .............................................................................135 
`V. CONCLUSION ............................................................................................153 
`
`F. 
`
`g. 
`
`-iv-
`
`

`

`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`I, Alan C. Bovik, do hereby declare:
`
`1.
`
`I am making this declaration at the request of Amazon.com, Inc.
`
`(“Amazon”).
`
`2.
`
`I am being compensated for my work in this matter and I am being re-
`
`imbursed at cost for my expenses. My compensation in no way depends upon the
`
`outcome of this proceeding.
`
`3.
`
`In preparing this Declaration, I considered the following materials:
`
`Exhibit No.
`1001
`
`Description
`U.S. Patent No. 9,380,282 (“the ’282 patent”)
`
`1003
`
`1004
`
`1005
`
`1010
`
`1011
`
`1014
`
`1021
`
`1022
`
`1023
`
`1024
`
`U.S. Patent Publication No. 2002/0042920 (“Thomas”)
`
`U.S. Patent Publication No. 2007/0250901 (“McIntire”)
`
`U.S. Patent No. 5,696,905 (“Reimer”)
`
`Prosecution File History for the ’282 Patent
`
`Nevenka Dimitrova et al., Media Augmentation and Personali-
`zation Through Multimedia Processing and Information Ex-
`traction, in Personalized Digital Television 203 (L.Ardissono et
`al. eds., 2004) (“Dimitrova”)
`
`U.S. Patent No. 6,154,771 (“Rangan”)
`
`U.S. Patent Publication No. 2007/0003223 (“Armstrong”)
`
`U.S. Patent Publication No. 2008/0253739 (“Livesey”)
`
`U.S. Patent No. 6,965,890 (“Dey”)
`
`U.S. Patent No. 6,038,367 (“Abecassis”)
`
`-1-
`
`

`

`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`
`Exhibit No.
`1025
`
`Description
`U.S. Patent No. 4,694,490 (“Harvey”)
`
`1026
`
`1027
`
`1028
`
`
`
`Excerpts from Interactive Video: Algorithms and Technologies
`(R. I. Hammoud ed., 2006) (“Hammoud”)
`
`U.S. Patent Publication No. 2010/0082585 (“Barsook”)
`
`U.S. Patent No. 6,956,573 (“Bergen”)
`
`I. BACKGROUND
`A. Experience and Qualifications
`
`4. My experience and qualifications are summarized in my curriculum
`
`vitae, a copy of which is provided as Exhibit 1020.
`
`5.
`
`I hold a Ph.D. in Electrical and Computer Engineering from the Uni-
`
`versity of Illinois, Urbana-Champaign (awarded in 1984). I also hold a Master’s
`
`degree in Electrical and Computer Engineering from the University of Illinois, Ur-
`
`bana-Champaign (awarded in 1982).
`
`6.
`
`I am a tenured full Professor and I hold the Cockrell Family Regents
`
`Endowed Chair at the University of Texas at Austin. My appointments are in the
`
`Department of Electrical and Computer Engineering, the Department of Computer
`
`Sciences, and the Department of Biomedical Engineering. I am also the Director
`
`of the Laboratory for Image and Video Engineering (“LIVE”).
`
`-2-
`
`

`

`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`7. My research is in the general area of digital television, digital camer-
`
`as, image and video processing, computational neuroscience, and modeling of bio-
`
`logical visual perception. I have published over 800 technical articles in these areas
`
`and hold seven U.S. patents. I am also the author of The Handbook of Image and
`
`Video Processing, Second Edition (Elsevier Academic Press, 2005); Modern Im-
`
`age Quality Assessment (Morgan & Claypool, 2006); The Essential Guide to Im-
`
`age Processing (Elsevier Academic Press, 2009); and The Essential Guide to Video
`
`Processing (Elsevier Academic Press, 2009); and numerous other publications.
`
`8.
`
`I received the 2017 Edwin H. Land Medal from the Optical Society of
`
`America in September 2017 with the following citation: For substantially shaping
`
`the direction and advancement of modern perceptual picture quality computation,
`
`and for energetically engaging industry to transform his ideas into global practice.
`
`I received a Primetime Emmy Award for Outstanding Achievement in Engineering
`
`Development, for the Academy of Television Arts and Sciences, in October 2015,
`
`for the widespread use of my video quality prediction and monitoring models and
`
`algorithms that are widely used throughout the global broadcast, cable, satellite and
`
`internet television industries.
`
`9.
`
`Among other awards and honors, I have received the 2013 Institute of
`
`Electrical and Electronics Engineers (“IEEE”) Signal Processing Society’s “Socie-
`
`ty Award,” which is the highest honor accorded by that technical society (“for fun-
`
`-3-
`
`

`

`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`damental contributions to digital image processing theory, technology, leadership
`
`and education”). In 2005, I received the Technical Achievement Award of the
`
`IEEE Signal Processing Society, which is the highest technical honor given by the
`
`Society, for “broad and lasting contributions to the field of digital image pro-
`
`cessing”; and in 2008 I received the Education Award of the IEEE Signal Pro-
`
`cessing Society, which is the highest education honor given by the Society, for
`
`“broad and lasting contributions to image processing, including popular and im-
`
`portant image processing books, innovative on-line courseware, and for the crea-
`
`tion of the leading research and educational journal and conference in the image
`
`processing field.”
`
`10. My technical articles have been widely recognized as well, including
`
`the 2009 IEEE Signal Processing Society Best Journal Paper Award for the paper
`
`“Image quality assessment: From error visibility to structural similarity,” published
`
`in IEEE Transactions on Image Processing, volume 13, number 4, April 2004; this
`
`same paper received the 2017 IEEE Signal Processing Society Sustained Impact
`
`Paper Award as the most impactful paper published over a period of at least ten
`
`years; the 2013 Best Magazine Paper Award for the paper “Mean squared error:
`
`Love it or leave it?? A new look at signal fidelity measures,” published in IEEE
`
`Transactions on Image Processing, volume 26, number 1, January 2009; the IEEE
`
`Circuits and Systems Society Best Journal Paper Prize for the paper “Video quality
`
`-4-
`
`

`

`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`assessment by reduced reference spatio-temporal entropic differencing,” published
`
`in IEEE Transactions on Circuits and Systems for Video Technology, vol. 23, no.
`
`4, pp. 684-694, April 2013.
`
`11.
`
`I received the Google Scholar Classic Paper citation twice in 2017, for
`
`the paper “Image information and visual quality,” published in the IEEE Transac-
`
`tions on Image Processing, vol. 15, no. 2, pp. 430-444, February 2006 (the main
`
`algorithm developed in the paper, called the Visual Information Fidelity (“VIF”)
`
`Index, is a core picture quality prediction engine used to quality-assess all encodes
`
`streamed globally by Netflix), and for “An evaluation of recent full reference im-
`
`age quality assessment algorithms,” published in the IEEE Transactions on Image
`
`Processing, vol. 15, no. 11, pp. 3440-3451, November 2006 (the picture quality da-
`
`tabase and human study described in the paper, the LIVE Image Quality Database,
`
`has been the standard development tool for picture quality research since its first
`
`introduction in 2003). Google Scholar Classic Papers are very highly-cited papers
`
`that have stood the test of time, and are among the ten most-cited articles in their
`
`area of research over the ten years since their publication.
`
`12.
`
`I have also been honored by other technical organizations, including
`
`the Society for Photo-optical and Instrumentation Engineers (“SPIE”), from which
`
`I received the Technology Achievement Award (2013) “For Broad and Lasting
`
`Contributions to the Field of Perception-Based Image Processing,” and the Society
`
`-5-
`
`

`

`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`for Imaging Science and Technology, which accorded me Honorary Membership,
`
`which is the highest recognition by that Society given to a single individual, “for
`
`his impact in shaping the direction and advancement of the field of perceptual im-
`
`age processing.” I was also elected as a Fellow of the IEEE “for contributions to
`
`nonlinear image processing” in 1995, a Fellow of the Optical Society of America
`
`(“OSA”) for “fundamental research contributions to and technical leadership in
`
`digital image and video processing” in 2006, and as a Fellow of SPIE for “pioneer-
`
`ing technical, leadership, and educational contributions to the field of image pro-
`
`cessing” in 2007.
`
`13.
`
`I have many years of experience in the design of digital video sys-
`
`tems. For example, among other relevant research, I have worked with the Nation-
`
`al Aeronautics and Space Administration (“NASA”) to develop high compression
`
`image sequence coding and animated vision technology, on various military pro-
`
`jects for the Air Force Office of Scientific Research, Phillips Air Force Base, the
`
`Army Research Office, and the Department of Defense. These projects have fo-
`
`cused on developing local spatio-temporal analysis in vision systems, scalable pro-
`
`cessing of multi-sensor and multi-spectral imagery, image processing and data
`
`compression tools for satellite imaging, AM-FM analysis of images and video, the
`
`scientific foundations of image representation and analysis, computer vision sys-
`
`tems for automatic target recognition and automatic recognition of human activi-
`
`-6-
`
`

`

`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`ties, vehicle structure recovery from a moving air platform, passive optical model-
`
`ing, and detection of speculated masses and architectural distortions in digitized
`
`mammograms. My research has also recently been funded by Netflix, Qualcomm,
`
`Facebook, Texas Instruments, Intel, Cisco, and the National Institute of Standards
`
`and Technology (“NIST”) for research on image and video quality assessment. I
`
`have also received numerous grants from the National Science Foundation for re-
`
`search on image and video processing and on computational vision.
`
`B.
`
`14.
`
` Applicable Legal Standards
`
` I have been asked to provide my opinion as to whether the claims of
`
`the ’282 patent would have been obvious to a person of ordinary skill in the art at
`
`the time of the alleged invention, in view of the prior art.
`
`15.
`
`I am an engineer by training and profession. The opinions I am ex-
`
`pressing in this report involve the application of my training and technical
`
`knowledge and experience to the evaluation of certain prior art with respect to the
`
`’282 patent.
`
`16. Although I have been involved as a technical expert in patent matters
`
`before, I am not an expert in patent law. Therefore, the attorneys from Knobbe,
`
`Martens, Olson & Bear, LLP have provided me with guidance as to the applicable
`
`patent law in this matter. The paragraphs below express my understanding of how
`
`I must apply current principles related to patent validity to my analysis.
`
`-7-
`
`

`

`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`17.
`It is my understanding that in determining whether a patent claim is
`
`obvious in view of the prior art, the Patent Office construes the claim by giving the
`
`claim its broadest reasonable interpretation consistent with the specification. For
`
`the purposes of this review, and to the extent necessary, I have construed each
`
`claim term in accordance with its plain and ordinary meaning under the required
`
`broadest reasonable interpretation. My opinions would not change, however, if the
`
`claims were given their ordinary and customary meaning ascribed to them by a
`
`person of ordinary skill in the art at the time of the invention.
`
`18.
`
`It is my understanding that a claim is “obvious” if the claimed subject
`
`matter as a whole would have been obvious to a person of ordinary skill in the art
`
`at the time of the alleged invention. I also understand that an obviousness analysis
`
`takes into account the scope and content of the prior art, the differences between
`
`the claimed subject matter and the prior art, and the level of ordinary skill in the art
`
`at the time of the invention.
`
`19.
`
`In determining the scope and content of the prior art, it is my under-
`
`standing that a reference is considered appropriate prior art if it falls within the
`
`field of the inventor’s endeavor. In addition, a reference is prior art if it is reason-
`
`ably pertinent to the particular problem with which the inventor was involved. A
`
`reference is reasonably pertinent if it logically would have commended itself to an
`
`inventor’s attention in considering his problem. If a reference relates to the same
`
`-8-
`
`

`

`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`problem as the claimed invention, that supports use of the reference as prior art in
`
`an obviousness analysis.
`
`20. To assess the differences between prior art and the claimed subject
`
`matter, it is my understanding that the law requires the claimed invention to be
`
`considered as a whole. This “as a whole” assessment requires showing that one of
`
`ordinary skill in the art at the time of invention, confronted by the same problems
`
`as the inventor and with no knowledge of the claimed invention, would have se-
`
`lected the elements from the prior art and combined them in the claimed manner.
`
`21.
`
`It is my understanding that something is “inherent in,” and therefore
`
`taught by, the prior art, if it necessarily flows from the explicit disclosure of the
`
`prior art. I understand that the fact that a certain result or characteristic may be
`
`present in the prior art is not sufficient to establish inherency. However, if the re-
`
`sult or characteristic is necessarily present based upon the explicit disclosure in the
`
`prior art, it is inherent in the prior art and is therefore disclosed.
`
`22.
`
`It is my further understanding that the law recognizes several ration-
`
`ales for combining references or modifying a reference to show obviousness of
`
`claimed subject matter. Some of these rationales include: combining prior art ele-
`
`ments according to known methods to yield predictable results; simple substitution
`
`of one known element for another to obtain predictable results; a predictable use of
`
`prior art elements according to their established functions; applying a known tech-
`
`-9-
`
`

`

`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`nique to a known device (method or product) ready for improvement to yield pre-
`
`dictable results; choosing from a finite number of identified, predictable solutions,
`
`with a reasonable expectation of success; and some teaching, suggestion, or moti-
`
`vation in the prior art that would have led one of ordinary skill to modify the prior
`
`art reference or to combine prior art reference teachings to arrive at the claimed in-
`
`vention.
`
`23.
`
`I also understand that an obviousness analysis must consider whether
`
`there are additional factors that would indicate that the invention would not have
`
`been obvious. These factors include whether there was: (i) a long-felt need in the
`
`industry; (ii) any unexpected results; (iii) skepticism of the invention; (iv) a teach-
`
`ing away from the invention; (v) commercial success; (vi) praise by others for the
`
`invention; and (vii) copying by other companies. I am not aware of any evidence
`
`that would suggest that the claims of the ’282 patent would have been non-
`
`obvious.
`
`C.
`
`24.
`
`Person of Ordinary Skill in the Art
`
`It is my understanding that when interpreting the claims of the ’282
`
`patent, I must do so based on the perspective of a person of ordinary skill in the art
`
`at the relevant priority date. As discussed below, I understand the relevant priority
`
`date to be March 26, 2012.
`
`-10-
`
`

`

`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`25. The ’282 patent describes the use of well-known technologies for
`
`providing supplemental information with videos. Based on my review of the spec-
`
`ification and claims of the ’282 patent, it is my opinion that a person of ordinary
`
`skill in the art would be a person having at least a bachelor’s degree in Electrical
`
`Engineering, Computer Engineering, or Computer Science. Additionally, such a
`
`person would have at least three years of experience in the design of digital video
`
`systems.
`
`26.
`
`I am able to make this assessment because in the 1990s and 2000s, I
`
`was familiar with the design and operation of numerous systems for displaying
`
`supplemental information with videos. With my graduate students, I conducted re-
`
`search on visual eye tracking, whereby the scan paths and fixation points of human
`
`viewers of videos were recorded as annotations that would later be analyzed and/or
`
`viewed as supplemental information either with or superimposed on each video. I
`
`also have conducted extensive human studies wherein human subjects would view
`
`numerous videos and would record their subject impressions of the video quality
`
`over time (or overall). This supplemental video information could then be ac-
`
`cessed for each video showing graphs, traces, or numerical representations of video
`
`quality. The engineers and scientists that I worked with during the 1990s and
`
`2000s had the requisite knowledge to make and use systems as described in the
`
`claims of the ’282 patent. Because I have worked with and supervised engineers in
`
`-11-
`
`

`

`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`the field of video metadata, I know very well what their capabilities were in the
`
`1990s and 2000s, how those engineers would interpret and understand the claims
`
`of the ’282 patent, and how they would understand the disclosures in the prior art
`
`discussed herein.
`
`27.
`
`In my opinion, as set forth in more detail below, a person having ordi-
`
`nary skill in the art at the time of the invention would have considered the devices
`
`and methods claimed in the ’282 patent to be obvious in view of the prior art.
`
`II. BACKGROUND OF THE TECHNOLOGY
`
`28.
`
`In a video context, supplemental information may be used to enhance
`
`a user’s viewing experience. Including supplemental information with, for exam-
`
`ple, a movie enhances the viewing experience by providing on-demand access to
`
`background information without requiring a user to perform an external search.
`
`An example of supplemental information is biographical information of actors ap-
`
`pearing in a video, which answers the age-old question: “who is that guy?” Sup-
`
`plemental information is also used to provide other useful information, such as mu-
`
`sic playing in a scene (“what is this song?”), filming location (“where is this
`
`street/building?”), trivia, and mistakes made during filming (such as inconsisten-
`
`cies in wardrobe or display of buildings that do not exist in the purported location
`
`of the movie).
`
`-12-
`
`

`

`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`29. Development of video systems that could display supplemental infor-
`
`mation began in the mid-1990s when the use of digital content became widespread.
`
`This time period coincided with the rise in popularity of DVDs, digital cable tele-
`
`vision, and digital satellite television. Before digital video, most content was dis-
`
`tributed through analog signals via cable (e.g., cable television) or video home sys-
`
`tem (“VHS”) tapes. The digitization of content enabled individual portions of a
`
`video to be coded with supplemental information. In addition, the use of digital
`
`transmission networks and display devices enabled the supplemental information
`
`itself to be easily sent and stored as one or more digital files.
`
`30. There are many examples of systems for providing supplemental in-
`
`formation during a video that have been known for decades. E.g., Ex. 1025 (Har-
`
`vey) (1987), 20:16-68 (disclosing how viewers of a TV program may request sup-
`
`plemental information such as a recipe being discussed during a cooking show);
`
`Ex. 1014 (Rangan), 2:20-5:26 (discussing “hypervideo” systems dating back to
`
`1991). In 1995, Reimer disclosed:
`
`presenting [a] movie to the user, and then receiving from the user a
`query pertaining to the movie. The invention determines the frame of
`the movie that was being presented to the user when the user issued
`the query …. The invention identifies, as specified by the query,
`portions of the movie related information relating to the frame, and
`retrieves those portions of the move related information. These
`
`-13-
`
`

`

`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`retrieved portions of the movie related information are presented to
`the user.
`
`Ex. 1005 (Reimer), 3:32-42.
`
`31.
`
`It was also common for digital video systems to retrieve and display
`
`information associated with multiple video frames at the same time. For example,
`
`McIntire retrieved and displayed information from multiple frames that were close
`
`to the time of the user’s request. Ex. 1004 (McIntire) ¶¶[0310]-[0313], [0272].
`
`The purpose of this feature was to provide supplemental information that a viewer
`
`was interested in but may have missed due to delay in the viewer’s reaction time.
`
`Id.
`
`32. Armstrong also disclosed retrieving and displaying information asso-
`
`ciated with multiple frames at the same time. The system in Armstrong retrieves a
`
`specific menu for supplemental content associated with the requested frame but al-
`
`so retrieves an optimized background image from another frame “close in time” to
`
`the requested frame. Ex. 1021 (Armstrong) ¶[0048]. Armstrong discloses that this
`
`“close in time” frame may be selected because it is “more efficient, superior or
`
`beneficial with respect to selection, display indexing, menu structure, encoding,
`
`decoding, and the like.” Id.
`
`33. Bergen discloses searching a video for scenes comprising a particular
`
`object and returning thumbnail images (referred to as key frames) in a storyboard
`
`-14-
`
`

`

`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`format so that the viewer can easily navigate to a desired scene. See Ex. 1028
`
`(Bergen), 14:58-65, 15:28-36, 20:28-33, Fig. 8. Searching in this manner eliminat-
`
`ed the need for a viewer to search through a video using only the play, fast for-
`
`ward, rewind, and pause features of a media player. Id., 1:25-28, 4:28-31.
`
`34.
`
`In general, many techniques for displaying relevant supplemental in-
`
`formation to a viewer of a video were well-known by March, 2012, when the ’282
`
`patent was filed.
`
`III. THE ’282 PATENT
`
`A.
`
`Summary of the ’282 Patent
`1.
`
`The Specification
`
`35. The ’282 patent describes an embodiment in which the user requests
`
`information while a video is playing. Ex. 1001, 8:33-65. The system determines
`
`the frame being played and displays information (e.g., an image) of the items in
`
`that frame for which additional information is available. Id.; see also id., 10:28-46,
`
`14:33-67. The ’282 patent notes that a “user may request item information for an
`
`item that was just depicted but is no longer currently depicted.” Id., 9:41-44. To
`
`accommodate such user delay, the system may display information for the current
`
`video frame and information “associated with the preceding six seconds of video.”
`
`Id., 10:36-43, 9:44-53. The user may select (e.g., click on) the indication to obtain
`
`more information about the item. Id., 11:16-40. The patent explains that the video
`
`-15-
`
`

`

`Amazon.com v. CustomPlay
`IPR Petition – U.S. Patent No. 9,380,282
`may be paused in response to the user’s initial request for information. Id., 13:34-
`
`39. When the viewer is finished viewing the information, the video may be re-
`
`sumed at the location where it was paused or at the beginning or end of a clip or
`
`scene. Id., 11:62-12:11.
`
`2.
`
`The Claims
`
`36. The ’282 patent includes 20 claims. I address claims 4, 7-9, 12, 14,
`
`16, 18, and 19 in this Declaration. Claim 4 is representative. It recites:
`
`An apparatus capable of processing data and instructions executable by a
`
`processor; the apparatus, when executing the instructions, performs the steps of:
`
`[a] receiving, from a user during a playing of a video, a request for infor-
`
`mati