UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`
`Samsung Electronics America, Inc.,
`Petitioner
`
`v.
`
`Prisua Engineering Corp.,
`Patent Owner.
`
`
`
`
`
`
`
`
`Patent No. 8,650,591 to Prieto
`
`IPR Case No. IPR2017-01188
`
`
`
`PETITION FOR INTER PARTES REVIEW OF CLAIMS 1-4,
`8 AND 11 OF U.S. PATENT 8,650,591 UNDER 35 U.S.C. §§
`311-319 AND 37 C.F.R. §§ 42.100 ET SEQ.
`
`
`
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
`TABLE OF CONTENTS
`
`TABLE OF EXHIBITS ........................................................................................... iv 
`MANDATORY NOTICES UNDER 37 C.F.R. § 42.8(a)(1) FOR
`I. 
`INTER PARTES REVIEW .............................................................................. 1 
`Real Party in Interest Under 37 C.F.R. § 42.8(b)(1) ............................ 1 
`A. 
`Related Matters Under 37 C.F.R. § 42.8(b)(2) .................................... 1 
`B. 
`Lead and Backup Counsel Under 37 C.F.R. § 42.8(b)(3) and
`C. 
`Service Information under 37 C.F.R. § 42.8(b)(4) ............................... 2 
`PAYMENT OF FEES UNDER 37 C.F.R. § 42.15 ........................................ 2 
`II. 
`III.  Certification of Word Count Under 37 C.F.R. § 42.24(d) ............................. 3 
`IV.  REQUIREMENTS FOR IPR UNDER 37 C.F.R. § 42.104 ........................... 3 
`Grounds for Standing Under 37 C.F.R. § 42.104(a) ............................ 3 
`A. 
`Identification of Challenge Under 3 C.F.R. § 42.104(b) and
`B. 
`Relief Requested ................................................................................... 3 
`SUMMARY OF THE ’591 PATENT ............................................................ 5 
`Overview of the Alleged Invention ...................................................... 5 
`A. 
`The Prosecution History ....................................................................... 6 
`B. 
`The Challenged Claims ........................................................................ 8 
`C. 
`VI.  RELEVANT INFORMATION CONCERNING THE CONTESTED
`PATENT ......................................................................................................... 8 
`Person of Ordinary Skill in the Art ...................................................... 8 
`A. 
`Claim Construction .............................................................................. 8 
`B. 
`1.
`“digital extraction” ..................................................................... 9 
` 
`VII.  SPECIFIC GROUNDS FOR PETITION ..................................................... 11 
`Ground 1: Claims 1, 2, 8, and 11 are anticipated by or rendered
`A. 
`obvious by Senftner ............................................................................ 11 
`1.
`Overview of Senftner ............................................................... 11 
` 
`2.
`Claims ...................................................................................... 13 
` 
`
`V. 
`
`
`
`ii
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
`B. 
`
`C. 
`
`Ground 2: Claims 3 and 4 are obvious over Senftner in view of
`Levoy .................................................................................................. 38 
`1.
`Scope and Content of the Prior Art .......................................... 38 
` 
`
`  Motivation to Combine ............................................................ 41 2.
`Claims ...................................................................................... 43 
`3.
` 
`Ground 3: Claims 1, 2, 8, and 11 are obvious under Sitrick .............. 46 
`1.
`Overview of Sitrick .................................................................. 46 
` 
`2.
`Claims ...................................................................................... 48 
` 
`D.  Ground 4: Claims 3 and 4 are rendered obvious by Sitrick in
`view of Levoy ..................................................................................... 66 
`1.
`Scope and Content of the Prior Art .......................................... 66 
` 
`
`  Motivation to Combine ............................................................ 67 2.
`Claims ...................................................................................... 68 
`3.
` 
`VIII.  CONCLUSION ............................................................................................. 70 
`CERTIFICATE OF SERVICE ............................................................................... 72 
`
`
`
`
`
`iii
`
`

`

`
`
`Petition Requesting Inter Partes Review
`
`TABLE OF EXHIBITS
`
`Patent No. 8,650,591
`
`
`
`Exhibit
`1001
`
`Description
`U.S. Patent No. 8,650,591 (“’591 patent”)
`
`1002
`
`1003
`
`1004
`
`1005
`
`1006
`
`1007
`
`1008
`
`1009
`
`1010
`
`1011
`
`1012
`
`1013
`
`1014
`
`
`
`File history of U.S. Patent No. 8,650,591
`
`Declaration of Edward Delp Regarding U.S. Patent No.
`8,650,591
`
`Prisua Engineering Corp. v. Samsung Electronics Co., Ltd., et
`al., CA No. 1:16-cv-21761-KMM, Deposition Transcript of
`Dr. Yolanda Prieto (January 17, 2017)
`
`Prisua Engineering Corp. v. Samsung Electronics Co., Ltd., et
`al., CA No. 1:16-cv-21761-KMM, Joint Claim Construction
`and Prehearing Statement (November 21, 2016)
`
`U.S. Patent No. 7,460,731 to Senftner et al. (“Senftner”)
`
`U.S. Patent Application Publication No. 2005/0151743 to
`Sitrick (“Sitrick”)
`
`U.S. Patent Application Publication No. 2009/0309990 to
`Levoy et al. (“Levoy”)
`
`Negahdaripour Decl. ISO Patent Owner’s Opening Claim
`Construction Brief (“Negahdaripour Decl.”)
`
`U.S. Patent Application Publication No. 2006/0097991
`
`U.S. Patent No. 7,307,623 to Enomoto
`
`U.S. Patent No. 4,686,332 to Greanias et al.
`
`Edward Delp Decl. ISO Petitioner’s Responsive Claim
`Construction Brief
`
`U.S. Patent Application Publication No. 2008/0148167 to
`Zeev Russak et al.
`
`iv
`
`

`

`
`
`Petition Requesting Inter Partes Review
`
`Patent No. 8,650,591
`
`
`
`1015
`
`
`
`MPEG | The Moving Picture Experts Group website, (see
`http://mpeg.chiariglione.org/)
`
`
`
`v
`
`

`

`Patent No. 8,650,591
`
` Petition Requesting Inter Partes Review
`
`Samsung Electronics America, Inc. (“Petitioner” or “Samsung”) hereby
`
`seeks inter partes review of Claims 1-4, 8, and 11 (“the Challenged Claims”) of
`
`U.S. Patent No. 8,650,591. (Ex. 1001 (the “’591 patent”).) The Challenged
`
`Claims of the ’591 patent do not claim anything inventive. The Challenged Claims
`
`disclose substituting an extracted image from one video data stream into another to
`
`create an edited video. However, prior to the ’591 patent, substitution of images
`
`extracted from one video into another video was well-known, and the Challenged
`
`Claims should be canceled for the reasons described in this Petition.
`
`I. MANDATORY NOTICES UNDER 37 C.F.R. § 42.8(a)(1) FOR INTER
`PARTES REVIEW
`A. Real Party in Interest Under 37 C.F.R. § 42.8(b)(1)
`The real parties in interest are Samsung Electronics Co., Ltd., 416 Maetan-
`
`3dong, Yeongton-gu, Suwon-City, Gyeonggi-do 443-742, South Korea; Samsung
`
`Electronics America, Inc., 85 Challenger Road, Ridgefield Park, New Jersey
`
`07660; and Samsung Electronics Latinoamerica Miami, Inc., 9850 NW 41st St
`
`#350, Doral, FL 33178.
`
`B. Related Matters Under 37 C.F.R. § 42.8(b)(2)
`A pending federal district court litigation may be affected by the decision in
`
`this proceeding: Prisua Engineering Corp. v. Samsung Electronics Co., Ltd et al.,
`
`No. 1:16-cv-21761-MOORE (S.D. Fla. May 17, 2016) (“the Litigation”). The
`
`
`
`1
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`undersigned is unaware of any other matter that would affect, or be affected by, a
`
`decision in the proceeding.
`
`C.
`Lead and Backup Counsel Under 37 C.F.R. § 42.8(b)(3) and
`Service Information under 37 C.F.R. § 42.8(b)(4)
`
`Petitioner designates the following lead and backup counsel:
`
`
`
`Lead
`Counsel:
`
`Backup
`Counsel:
`
`Heath J. Briggs (Reg. No. 54,919)
`Greenberg Traurig, LLP
`1200 17th St., Suite 2400
`Denver, CO 80202
`Telephone: 303-685-7418
`Facsimile: 720-904-6118
`BriggsH@gtlaw.com
`Patrick J. McCarthy (Reg. No. 62,762)
`Greenberg Traurig, LLP
`2101 L street, N.W., Suite 1000
`Washington, D.C. 20037
`Telephone: 202-533-2386
`Facsimile:202-331-3101
`McCarthyP@gtlaw.com
`
`
`Service on Petitioner may be made by mail or hand delivery to: Greenberg Traurig,
`
`LLP, 1200 17th St., Suite 2400, Denver, CO, 80202. Petitioner also consents to
`
`electronic service by emailing counsel of record at briggsh@gtlaw.com,
`
`mccarthyp@gtlaw.com and PrisuaGTIPR@gtlaw.com.
`
`II.
`
`PAYMENT OF FEES UNDER 37 C.F.R. § 42.15
`
`Petitioner authorizes the U.S. Patent & Trademark Office to charge Deposit
`
`Account No. 50-2638 for the fee set in 37 C.F.R. § 42.15(a) for this Petition and
`
`further authorizes for any additional fees to be charged to this Deposit Account.
`
`
`
`2
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
`III. CERTIFICATION OF WORD COUNT UNDER 37 C.F.R. § 42.24(d)
`Petitioner certifies that the word count in this Petition is XX words, as
`
`counted by the word-processing program (Microsoft Word 2010) used to generate
`
`this Petition, where such word count excludes the table of contents, table of
`
`authorities, mandatory notices, certificate of service, appendix of exhibits, and this
`
`certificate of word count. This Petition is in compliance with the 14,000 word
`
`limit set forth in 37 C.F.R. § 42.24(a)(1)(i).
`
`IV. REQUIREMENTS FOR IPR UNDER 37 C.F.R. § 42.104
`A. Grounds for Standing Under 37 C.F.R. § 42.104(a)
`Petitioner certifies that the ’591 patent is available for inter partes review,
`
`and that Petitioner is not barred or estopped from requesting an inter partes review
`
`on the grounds identified in the petition.
`
`B.
`Identification of Challenge Under 3 C.F.R. § 42.104(b) and Relief
`Requested
`
`Petitioner respectfully requests that claims 1-4, 8, and 11 of the ’591 patent
`
`be cancelled based on the following grounds of unpatentability, explained in detail
`
`below.
`
`
`
`3
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
`Ground of
`Unpatentability
`Ground 1
`
`’591 Patent
`Claim(s)
`1, 2, 8 and 11
`
`Ground 2
`
`3 and 4
`
`Ground 3
`
`1, 2, 8, and 11
`
`Ground 4
`
`3 and 4
`
`Basis for Rejection
`Anticipated by or rendered obvious by
`U.S. Patent No. 7,460,731 (“Senftner”)
`Rendered obvious by Senftner in view of
`U.S. Patent Application Publication No.
`2009/0309990 to Levoy et al. (“Levoy”)
`Rendered obvious by U.S. Patent
`Application Publication No.
`2005/0151743 (“Sitrick”)
`Rendered obvious by Sitrick in view of
`Levoy
`
`The ’591 patent issued on February 11, 2014, from Application No.
`
`13/042,955 filed on March 8, 2011. The ’591 patent claims priority to provisional
`
`application no. 61/311,892 filed on March 9, 2010.1
`
`U.S. Patent No. 7,460,731 (“Senftner”) qualifies as prior art under at least
`
`pre-AIA 35 U.S.C. §102(b). Senftner issued on December 2, 2008, more than one
`
`year before the earliest possible priority date of the ’591 patent.
`
`U.S. Patent Application Publication No. 2005/0151743 (“Sitrick”) qualifies
`
`as prior art under at least pre-AIA 35 U.S.C. §102(b). Sitrick was published on
`
`July 14, 2005, more than one year prior to the earliest possible priority date of the
`
`’591 patent.
`
`1 Petitioner does not concede the ’591 patent is entitled to the priority date of this
`
`provisional patent application.
`
`
`
`4
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
`U.S. Patent Application Publication No. 2009/0309990 (“Levoy”) qualifies
`
`as prior art under at least pre-AIA 35 U.S.C. §§102(a),(e). Levoy was filed on
`
`June 11, 2008 and published on December 17, 2009, both of which occurred prior
`
`to the earliest possible priority date of the ’591 patent.
`
`V.
`
`SUMMARY OF THE ’591 PATENT
`A. Overview of the Alleged Invention
`The ’591 patent is entitled “Video Enabled Digital Devices for Embedding
`
`User Data in Interactive Applications” and issued from an application filed on
`
`March 8, 2011.
`
`The ʼ591 patent is directed to a system that creates a new composite video
`
`by substituting a portion of an original video data stream with an image from a user
`
`input video data stream. FIG. 3 (reproduced below) shows the preferred “image
`
`substitution” described by the patent. According to the specification, “a user input
`
`150 of a photo image of the user used to replace the face of the image shown on
`
`the device 108. The user transmits the photo image 150 by wired or wireless means
`
`to the device 108. The image substitution is performed and the device 108 shows
`
`the substituted image 190.” (Ex. 1001 at 2:66-3:4 (emphasis added).)
`
`
`
`5
`
`

`

`
`
`Petition Requesting Inter Partes Review
`
`Patent No. 8,650,591
`
`
`
`
`
`(Id. at Fig. 3.)
`
`The Prosecution History
`
`B.
`The original claims of the ’591 patent were rejected by the patent examiner
`
`who found the claims unpatentable under § 101, as well as anticipated by U.S.
`
`Patent No. 8,024,768 to Berger (“Berger”). (Id. at 46.)
`
`In a response filed March 13, 2013, applicant cancelled all original claims,
`
`and added new claims 21-40 which were essentially redrafted versions of the
`
`original claims. (Id. at 58.)
`
`
`
`6
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
`On June 5, 2013, the examiner issued a final rejection, finding all claims
`
`anticipated based on Berger. (Id. at 77.)
`
`On July 16, 2013, applicant faxed a telephonic interview request to the
`
`examiner stating that applicant would like to discuss “Berger’s failure to teach
`
`video/image content substitution as required by the independent claims.” (Id. at
`
`89.) A telephone interview was held on July 22, 2013. (Id. at 97.)2
`
`In a response filed August 2, 2013, applicant cancelled claims 21-26, added
`
`new claims 41-47 (Id. at 97.)
`
`
`
`In an Advisory Action mailed August 21, 2013, the examiner rejected all
`
`claims noting that, despite applicants claim amendments, “Berger still meets the
`
`claim limitations”. (Id. at 106.)
`
`
`
`In a response filed September 5, 2013, applicant cancelled all prior pending
`
`claims, added new claims and argued the new claims were patentable over Berger
`
`because, inter alia, “Berger does not perform a substitution of one selected image
`
`for another selected image. Berger does not discuss a substitution at all. Instead,
`
`
`2 A USPTO Interview Summary form was not found in the record in PAIR. The
`
`only mention of the July 22, 2013 telephonic interview is in applicant’s response of
`
`August 2, 2013.
`
`
`
`7
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
`Berger generates a new video as a result of resizing, cropping, and pan-and-scan.”
`
`(Id. at 125.) A notice of allowance was subsequently mailed. The reasons for
`
`allowance were, in essence, that the prior art failed to disclose the features of the
`
`allowed claims. (Id. at 134-137.)
`
`
`
`The prior art cited in this Petition were not analyzed, considered, described,
`
`or cited during prosecution of the ’591 patent.
`
`The Challenged Claims
`
`C.
`Claims 1 and 11 are independent claims in the ’591 patent. Claims 2-4 and
`
`8 depend directly or indirectly from Claim 1.
`
`VI. RELEVANT INFORMATION CONCERNING THE CONTESTED
`PATENT
`A.
`A POSITA in the field of the ’591 patent at the time of the earliest possible
`
`Person of Ordinary Skill in the Art
`
`priority date (March 9, 2010) would be at least an engineer with a Bachelor of
`
`Science degree and at least three years of imaging and signal processing
`
`experience or would have earned a Master’s Degree in Electrical Engineering and
`
`at least two years of professional experience in signal, image, and video
`
`processing. (Ex. 1003 at ¶25.)
`
`B. Claim Construction
`Pursuant to 37 C.F.R. § 42.100(b), the claims in inter partes review are
`
`given the “broadest reasonable construction in light of the specification”. For the
`8
`
`
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
`purposes of this proceeding, the following claim term should be construed as set
`
`forth below. For the purposes of this proceeding, Petitioner requests that each of
`
`the remaining claim terms be given their plain meaning.3
`
`“digital extraction”
`
`
`1.
`Claims 1-4 and 11 of the ’591 patent use the terms “digitally extracted,”
`
`“digital extraction,” and like terms in the context of creating the first or second
`
`images from pixel(s) of a video data stream. (Ex. 1001 at Cl. 1 (7:14-20, 39-40,
`
`45) and Cl. 8 (8:28-34, 49-50, and 55-56).) The specification of the ’591 patent
`
`never uses the word extraction. Instead, the ’591 patent uses the word
`
`“substitution”:
`
`“Summary of the Invention: Briefly, according to an embodiment of
`the invention a method for generating an edited video data stream
`from an original video stream wherein generation of said edited video
`stream comprises a step of: substituting at least one object in a
`plurality of objects in said original video stream by at least a different
`object. According to another embodiment, an interactive broadcast TV
`system enhances the user experience by allowing said user the
`
`3 In the Litigation, Petitioner proposed construction of various terms in accordance
`
`with the Phillips standard. (Ex. 1005 at 5-22.) Petitioner maintains that those
`
`proposed constructions are correct in the Litigation under the Phillips standard.
`
`
`
`9
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
`capability of providing his/her object data for use in the substitution
`of an object data in a broadcast video by said user object data.” (Ex.
`1001 at 1:40-51 (emphasis added).)
`
`FIG. 3 illustrates this “substitution” methodology, and specifically states “Image
`
`Substitution”. In fact, the ’591 uses the word substitution twenty-seven (27) times
`
`in its specification and without explaining how pixel(s) are “digitally extracted”
`
`from a data stream to form an image.
`
`Further, during prosecution, the term “digitally extracted” was not
`
`explained. (See Ex. 1002 at 125-126.)
`
`In the Litigation, Patent Owner proposed “extracting” to mean “select and
`
`separate out”. (Ex. 1005 at 16-17.) Patent Owner’s expert, in arguing that
`
`“extraction” does not require a complete removal of that selected data from the
`
`original data, testified:
`
`“In sum, one of ordinary skill in the art attempting to practice the ‘591
`patent’s disclosure would likely simply “select” and “copy out” into
`memory the information present in the data streams, and generate a
`new third “data stream” from memory. Prisua’s proposed construction
`therefore comports with the understanding of the term as known to
`those of skill in the art.” (Ex. 1009 at ¶53; see also, id. at 48-52.)
`
`
`
`
`10
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
`Based on the foregoing and in view of Patent Owner’s assertion of its claims
`
`in the Litigation, Petitioner proposes that the broadest reasonable interpretation of
`
`“digitally extracted,” “digital extraction,” and like is “to digitally select and
`
`separate out, such as by copying.”
`
`VII. SPECIFIC GROUNDS FOR PETITION
`A. Ground 1: Claims 1, 2, 8, and 11 are anticipated by or rendered
`obvious by Senftner
` Overview of Senftner
`1.
`Senftner is entitled “Personalizing a video” and is directed to creating a
`
`personalized video through partial image replacement in the original video. (Ex.
`
`1006 at Abstract.) Like the ’591 patent, Senftner discloses computerized systems
`
`and methods for replacing images or video of an “original actor” or “target” in an
`
`original video with images or video of a “new actor” or “target replacement.” (See,
`
`e.g., id. at 2:33-54, 9:25-31.) In particular, Senftner explains:
`
`“The creation of personalized video is a combination of multiple
`fields that in totality allow for the alteration of video sequences such
`that individuals are able to replace the participants of an original
`video with themselves, their friends, their family members or any
`individuals, real or imagined, which they have images depicting. (Id.
`at 5:42-47 (emphasis added).)
`
`
`
`
`11
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
`FIG. 1 of Senftner, reproduced below, provides a high-level overview of Senftner’s
`
`image substitution process.
`
`
`
`As Senftner explains, process 100 of FIG. 1 relates to obtaining images of the new
`
`actor (called the “actor modeling process”), process 200 relates to preparing an
`
`original video having the original actor for substitution (called the “video
`
`preparation process”), and process 300 relates to making the personalized video
`
`based on steps 100 and 200 (called the “personalization process”). (Id. at 9:32-52.)
`12
`
`
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
`In order to facilitate the substitution, motion correction may be applied to one of or
`
`both of the original actor and the new actor during capture of the images or video.
`
`(Id. at 6:10-14, 17:10-23.)
`
`
`
`For the reasons provided below, Senftner anticipates or renders obvious
`
`claims 1, 2, 8 and 11 of the Challenged Claims.
`
`Claims
`
`
`2.
`for generating a
`interactive media apparatus
`[1-PREAMBLE-i] An
`displayable edited video data stream from an original video data stream,
`
`Senftner discloses this limitation. Senftner discloses “personalizing video
`
`through partial image replacement”. (Id. at Abstract.) The personalized videos
`
`may be made from “a portion of a current or classic movie or television show, an
`
`entire film or TV show, an advertisement, a music video, or a specialty clip made
`
`specifically for personalization (for example, a clip that can be personalized to
`
`show the new actor with a ‘celebrity friend’)”. (Id. at 5:20-25.) The personalized
`
`videos are made by editing the original video (e.g., the original movie or TV show)
`
`with new image information and then displaying it. (Id. at 2:41-54.) In other
`
`words, Senftner discloses “[a]n interactive media apparatus for generating a
`
`displayable edited video data stream from an original video data stream”.
`
`Thus, Senftner discloses the limitations of [1-PREAMBLE-i].
`
`
`
`13
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
`[1-PREMABLE-ii] . . . wherein at least one pixel in a frame of said original
`video data stream is digitally extracted to form a first image, said first image
`then replaced by a second image resulting from a digital extraction of at least
`one pixel in a frame of a user input video data stream, said apparatus
`comprising:
`
`As used in the claims of the ’591 patent, the limitation “a first image” refers
`
`to an image digitally extracted (e.g., copied) from a frame of the original video
`
`data stream (e.g., an image or video of the original actor extracted from a frame of
`
`the original broadcast), and the limitation “a second image” refers to an image
`
`digitally extracted (e.g., copied) from a frame of the user’s video data stream (e.g.,
`
`an image or video taken from a frame of the user’s video). Per the requirements of
`
`the preamble, the user’s image (the second image) replaces the original actor’s
`
`image (the first image) in the original broadcast (the original video data stream).
`
`This is shown in FIG. 3 of the ’591 patent, below:
`
`
`
`14
`
`

`

`
`
`Petition Requesting Inter Partes Review
`
`Patent No. 8,650,591
`
`
`
`Regarding this FIG. 3, the ’591 patent states:
`
`
`
`“Referring now to FIG. 3 we show a user input 150 of a photo image
`of the user used to replace the face of the image shown on the device
`108. The user transmits the photo image 150 by wired or wireless
`means to the device 108. The image substitution is performed and the
`device 108 shows the substituted image 190.” (Ex. 1001 at 2:66-3:4
`(emphasis added).)
`
`Senftner discloses this same arrangement. Senftner discloses that a “target”
`
`or “original actor” is selected in an original digital video (an “original video data
`
`
`
`15
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
`stream”. (Ex. 1006 at 2:41-51.) Also, a digital video of the “target replacement”
`
`or “new actor” is captured, analyzed and stored (Id. at FIG. 1, 10:3-28), such as
`
`“by means of a digital image recording device 420, such as a digital camera, a
`
`digital video recorder, or a camera-equipped cell phone”. (Id. at 17:46-49; see also,
`
`id. at 5:33-40.) Video captured with any of these devices produces “a user input
`
`video data stream”.4 Thereafter, images of the target or original actor (the first
`
`images) are “replaced” with images of a “target replacement” or “new actor” (the
`
`second images). (Id. at FIGS. 1-2, 2:51-54.) In one specific embodiment, an
`
`original actor’s face is replaced with a new actor’s face:5
`
`
`4 The ’591 patent uses the same digital equipment disclosed by Senftner to create
`
`their “user input video data stream.” (See, e.g., Ex. 1001 at 2:12 (“camera-enabled
`
`personal device 106”), 2:34-35 (“a camera input”), and 3:41-43 (“the User Data
`
`Device (UDD) 106 is an image capable digital device . . .”).
`
`5 In order to streamline the Petition, Petitioner generally refers only to “original
`
`actor” and “new actor” when explaining Senftner, as opposed to referring both
`
`“target” and “original actor,” or both and “new actor” and “target replacement.”
`
`However, Petitioner is not limiting the disclosure of Senftner via its streamlining.
`
`16
`
`
`
`
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
` “The initial description of the processes will be made using an
`example case where the video is personalized by substituting the
`image of the face of a new actor for the facial portion of the image of
`one of the video's original actors.” (Id. at 9:6-9.)
`
`To complete this “replacement” the original video having the original actor can be
`
`altered on a “pixel-by-pixel and frame-by-frame basis”. (Id. at 8:52-54.)
`
`Specifically, the “replacement” process may involve “overwriting the original data
`
`with the new data” in a single step. (Id. at 8:67-9:1.) In this regard:
`
`“The personalization process begins at step 320 where the image of
`the new actor is inserted into the video. The process for substituting
`the image of the new actor is show [sic] in additional detail in FIG. 2.
`At step 322, the 3D model of the new actor may be transformed to
`match the orientation and expression of the original actor as defined
`by data from step 210 of the video preparation process. This
`transformation may involve both rotation on several axis and
`geometric morphing of the facial expression, in either order. After the
`3D model is rotated and morphed, a 2D image of the 3D model is
`developed and scaled to the appropriate size at step 324. The
`
`
`
`Senftner refers to other targets and target replacements besides actors. (See, e.g.,
`
`Ex. 1006 at 2:44-46, 13:15-25.)
`
`
`
`17
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
`transformed scaled 2D image of the new actor is then inserted into the
`video at step 326 such that the position, orientation, and expression of
`the new actor substantially matches the position, orientation, and
`expression of the previously removed original actor. In this context, a
`"substantial match" occurs when the personalized video presents a
`convincing illusion that the new actor was actually present when the
`video was created.” (Id. at 12:27-45.)
`
`
`
`Senftner expressly discloses that the original actor is removed from the original
`
`digital video:
`
`“In order for such alteration to occur, the replacement of the face and
`portions of the head is not enough to achieve this result; in this
`situation a complete removal of the original actor is executed, their
`key motions are preserved in a secondary storage medium, and then
`referenced for the animation and insertion of the petite female's digital
`double.” (Id. at 6:8-14.)
`
`18
`
`
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
`
` (See also, id. at 5:15-25, 11:7-12, 42-59.) The replacement of the original actor by
`
`the new actor may be completed for a single image or for a series of images. (Id.
`
`at 5:19-28.)
`
`Thus, Senftner discloses the limitation “wherein at least one pixel in a frame
`
`of said original video data stream is digitally extracted to form a first image, said
`
`first image then replaced by a second image resulting from a digital extraction of at
`
`least one pixel in a frame of a user input video data stream,” and therefore
`
`discloses the limitations of [1-PREAMBLE-ii].
`
`[1a] an image capture device capturing the user input video data stream;
`
`The ’591 patent discloses various “image capture devices” for capturing the
`
`“user input video data stream”. (Ex. 1001 at 2:12, FIG. 1.) Senftner also discloses
`
`“image capture devices” and “user input video streams”: “[t]he 2D digital image
`
`425 may be created by means of a digital image recording device 420, such as a
`
`digital camera, a digital video recorder, or a camera-equipped cell phone.” (Ex.
`
`1006 at FIGS. 8-11, 17:45-48.) The disclosed cameras capture “user input video
`
`data streams”. Thus, Senftner discloses limitation [1a].
`
`[1b] an image display device displaying the original video stream;
`
`The ’591 patent discloses various display devices for displaying the original
`
`video stream, such as “a digital TV, a gaming console, a digital media
`
`
`
`19
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
`player/recorder, or set top box . . . .” (Ex. 1001 at 2:37-39.) Senftner also
`
`discloses display devices: “[t]he personalized video may then be presented to user
`
`650 by means of display device, and may be stored in memory 720 or storage
`
`medium 730”. (Ex. 1006 at 21:6-8.) FIG. 10 also shows a computer (670) with a
`
`monitor that may be used to display the original video data stream. Thus, Senftner
`
`discloses limitation [1b].
`
`[1c-i] a data entry device, operably coupled with the image capture device and
`the image display device
`
`Senftner discloses data entry devices meeting the requirements of limitation
`
`[1c-i]. Figure 10 of Senftner shows a computer (670) having a keyboard (not
`
`numbered) and a display device (a monitor) for “for creating personalized videos.”
`
`(Id. at 20:24.) As shown in FIG. 10, the computer is operably coupled to digital
`
`image sources (660) (i.e., one or more image capture devices), such as any of the
`
`digital camera, cell phone, and digital video recorder illustrated in FIG. 10. (Id. at
`
`20:35-36.) A similar arrangement is show in FIG. 11, in which the computing
`
`device (700) includes an interface to requestor (650) “such as a keyboard, mouse,
`
`or other human interface means” and “may also have an interface to a digital image
`
`device 660” which is the image capture device. After personalizing the digital
`
`video, the personalized video “may then be presented to user 650 by means of
`
`display device. . . .” (Id. at 20:62-64, 21:5-7.)
`
`
`
`20
`
`

`

`Patent No. 8,650,591
`
`
`
`Petition Requesting Inter Partes Review
`
`
`
`Thus, Senftner discloses limitation [1c-i].
`
`[1c-ii] . . . operated by a user to select the at least one pixel in the frame of the
`user input video data stream to use as the second image, and further operated
`by the user to select the at least one pixel to use as the first image;
`
`Building on the discussion of limitation [1c-i], Senftner discloses that the
`
`computer’s user (the “requestor” (650)) operates the computer through the data
`
`entry device. (Id. at 18:1-18, 20:35-38.) This “requestor” is shown in FIGS. 8-11,
`
`and meets the requirements of “a user” who selects the first and second images, as
`
`required by limitation [1c-ii]. FIGS. 8 and 9 show a flow chart of a process
`
`400/500 for creating and delivering a personalized video. (Id. at 17:23-24, 18:45-
`
`46.) FIGS. 10 and 11 describe the computing devices for creating the personalized
`
`video. (Id. at 5:5-6, 20:24-25, 20:56-57.)
`
`
`
`In replacing the original actor’s image with the new actor’s image, the
`
`requestor “selects” both the original actor’s and new actor’s images:
`
`“Apparatus, systems and techniques for providing personalized digital
`video in various appli