`
`
`Fatih M. Ozluturk
`In re Patent of:
`10,171,740 Attorney Docket No.: 39843-0080IP1
`U.S. Patent No.:
`January 1, 2019
`
`Issue Date:
`Appl. Serial No.: 15/714,558
`
`Filing Date:
`September 25, 2017
`
`Title:
`METHOD AND APPARATUS TO CORRECT BLUR IN ALL OR
`PART OF A DIGITAL IMAGE BY COMBINING PLURALITY
`OF IMAGES
`
`
`Mail Stop Patent Board
`Patent Trial and Appeal Board
`U.S. Patent and Trademark Office
`P.O. Box 1450
`Alexandria, VA 22313-1450
`
`
`PETITION FOR INTER PARTES REVIEW OF UNITED STATES PATENT
`NO. 10,171,740 PURSUANT TO 35 U.S.C. §§ 311–319, 37 C.F.R. § 42
`
`
`
`
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`TABLE OF CONTENTS
`
`I.
`
`II.
`
`THE ’740 PATENT ......................................................................................... 1
`A. Overview ................................................................................................... 1
`B. Prosecution History ................................................................................... 1
`REQUIREMENTS FOR IPR .......................................................................... 1
`A. Standing .................................................................................................... 1
`B. Challenge and Relief Requested ............................................................... 2
`C. Level of Ordinary Skill in the Art ............................................................. 3
`D. Claim Construction ................................................................................... 3
`III. APPLICATION OF PRIOR ART ................................................................... 6
`A. 1A: Kitamura and Noriyuki render obvious claims 1, 2, 4-5, 8, 10- 12,
`14-15, 18 ................................................................................................... 7
`B. 1B: Kitamura, Noriyuki, and Manabe render obvious claims 3, 7, 9, 13,
`17, and 19 ................................................................................................ 44
`C. 1C: Kitamura, Noriyuki, and Suh render obvious claims 6 and 16 ........ 51
`D. 1D: Kitamura and Suh render obvious claims 20, 24-26, and 28........... 58
`E. 1E: Kitamura, Suh, and Noriyuki render obvious claims 21 and 22 ...... 68
`F. 1F: Kitamura, Suh, and Manabe render obvious claims 23, 27, and 29 . 70
`G. 2A-2D: Kitamura, Noriyuki, and Manabe render obvious claims 1-5, 7-
`15, 17-19; Kitamura, Noriyuki, Manabe, and Suh render obvious claims
`6, 16; Kitamura, Suh, and Manabe render obvious claims 20, 23-27, 29;
`Kitamura, Suh, Manabe, and Noriyuki render obvious claims 21-22, 28 ..
`
` ....................................................................................................... 71
`IV. PTAB DISCRETION SHOULD NOT PRECLUDE INSTITUTION .......... 76
`V.
`PAYMENT OF FEES ................................................................................... 80
`VI. CONCLUSION .............................................................................................. 80
`VII. MANDATORY NOTICES UNDER 37 C.F.R § 42.8(a)(1) ......................... 80
`A. Real Party-In-Interest Under 37 C.F.R. § 42.8(b)(1) .............................. 80
`B. Related Matters Under 37 C.F.R. § 42.8(b)(2) ....................................... 80
`C. Lead And Back-Up Counsel Under 37 C.F.R. § 42.8(b)(3) ................... 80
`D. Service Information ................................................................................ 81
`
`
`
`
`
`
`i
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`
`
`
`EXHIBITS
`
`SAMSUNG-1001 U.S. Patent No. 10,171,740 to Ozluturk (“the ’740 Patent”)
`
`SAMSUNG-1002 Excerpts from the Prosecution History of the ’740 Patent (“the
`Prosecution History”)
`
`SAMSUNG-1003 Declaration of Dr. Irfan Essa
`
`SAMSUNG-1004 Reserved
`
`SAMSUNG-1005 U.S. Patent Publication No. 2001/0022860 to Kitamura
`(“Kitamura”)
`
`SAMSUNG-1006 U.S. Patent No. 6,977,687 to Suh (“Suh”)
`
`SAMSUNG-1007 U.S. Patent Publication No. 2005/0018927 to Manabe
`(“Manabe”)
`
`SAMSUNG-1008 Certified Translation of Japanese Patent Publication No.
`2002057933 (“Noriyuki”)
`
`SAMSUNG-1009 Unopposed Motion for Extension of Time to Move, Answer,
`or Otherwise Respond to Plaintiff’s Complaint (Clear Imag-
`ing Research, LLC v. Samsung Electronics Co. Ltd., 2:19-cv-
`326-JRG (E.D. Tex. Oct. 15, 2019))
`
`SAMSUNG-1010 Reserved
`
`SAMSUNG-1011 Reserved
`
`SAMSUNG-1012 Excerpt of The New Oxford American Dictionary (Second
`Edition, 2005)
`
`SAMSUNG-1013 Reserved
`
`ii
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`
`SAMSUNG-1014 Reserved
`
`SAMSUNG-1015 U.S. Patent No. 6,809,759 to Chiang
`
`SAMSUNG-1016 U.S. Patent No. 7,369,161 to Easwar
`
`SAMSUNG-1017
`
`Joseph Guzman, “Fauci says second wave of coronavirus is
`‘inevitable’”, TheHill.com (Apr. 29, 2020), available at:
`https://thehill.com/changing-america/resilience/natural-disas-
`ters/495211-fauci-says-second-wave-of-coronavirus-is
`SAMSUNG-1018 Peter Wells, et al. “Texas puts reopening on hold in face of
`new Covid-19 outbreak”, Financial Times (June 25, 2020),
`available at https://www.ft.com/content/e35f3148-a797-4e6e-
`bf7a-1a7ce3181e97
`SAMSUNG-1019 Order Granting Joint Motion to Amend the Docket Control
`Order and Time for Claim Construction Expert Disclosures
`(Clear Imaging Research, LLC v. Samsung Electronics Co.
`Ltd., 2:19-cv-326-JRG (E.D. Tex. June 12, 2020))
`SAMSUNG-1020 Email to Clear Imaging counsel dated February 10, 2020
`SAMSUNG-1021 Email from Clear Imaging counsel dated July 3, 2020
`SAMSUNG-1022 Stipulation by Samsung
`SAMSUNG-1023 Reserved
`SAMSUNG-1024 P.R. 4-3 Joint Claim Construction and Prehearing Statement
`(Clear Imaging Research, LLC v. Samsung Electronics Co.
`Ltd., 2:19-cv-326-JRG (E.D. Tex. July 23, 2020))
`SAMSUNG-1025 Plaintiff’s Disclosure of Asserted Claims and Infringement
`Contentions Pursuant to Patent Local Rules (Clear Imaging
`Research, LLC v. Samsung Electronics Co. Ltd., 2:19-cv-326-
`JRG (E.D. Tex. Jan. 14, 2020))
`
`
`
`
`iii
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`Samsung petitions for Inter Partes Review (“IPR) of claims 1-29 (“the Chal-
`
`lenged Claims”) of U.S. Patent No. 10,171,740 (“the ’740 Patent”).
`
`I.
`
`THE ’740 PATENT
`A. Overview
`The ’740 Patent relates to an approach to correcting blur in digital images by
`
`combining captured images to create a combined image such that a main subject is
`
`blur free and other areas are blurred. SAMSUNG-1001, Abstract, 12:54-62; SAM-
`
`SUNG-1003, [0010].
`
`B.
`Prosecution History
`The sole claim was rejected as “being a substantial duplicate of claim 1 of
`
`US Patent 9,774,785;” was subject to double patenting rejections over claims of
`
`US 9,774,785, US 9,392,175, and US 9,154,699; and was rejected as anticipated
`
`over US 2003/0076408. SAMSUNG-1002, 140-148. In response, claim 1 was
`
`cancelled and claims 2-30 were introduced. Id., 92-105. A Notice of Allowance
`
`followed. Id., 50.
`
`II. REQUIREMENTS FOR IPR
`A.
`Standing
`Samsung certifies that the ’740 Patent is available for IPR. The present peti-
`
`tion is being filed within one year of service of a complaint against Samsung in the
`
`Eastern District of Texas. SAMSUNG-1009. Samsung is not barred or estopped
`
`from requesting this review.
`
`1
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`
`B. Challenge and Relief Requested
`Samsung requests cancellation of the Challenged Claims on the following
`
`grounds. Additional explanation and support for each ground of rejection is set
`
`forth in Dr. Irfan Essa’s declaration (SAMSUNG-1003).
`
`Ground
`1A
`
`1B
`1C
`1D
`1E
`1F
`2A
`2B
`
`2C
`2D
`
`Claims
`1-2, 4-5, 8, 10-12, 14-15,
`18
`3, 7, 9, 13, 17, 19
`6, 16
`20, 24-26
`21-22, 28
`23, 27, 29
`1-5, 7-15, 17-19
`6, 16
`
`20, 23-27, 29
`21-22, 28
`
`§103(a) References
`Kitamura and Noriyuki
`
`Kitamura, Noriyuki, and Manabe
`Kitamura, Noriyuki, and Suh
`Kitamura and Suh
`Kitamura, Suh, and Noriyuki
`Kitamura, Suh, and Manabe
`Kitamura, Noriyuki, and Manabe
`Kitamura, Noriyuki, Manabe, and
`Suh
`Kitamura, Suh, and Manabe
`Kitamura, Suh, Manabe, and Nori-
`yuki
`
`The following table summarizes the prior art basis for each reference with
`
`respect to an assumed Critical Date of March 25, 2004.1 Each reference below
`
`qualifies as prior art:
`
`
`1 Although the ’740 Patent lists a March 25, 2004 provisional application, Patent
`
`Owner does not contend that the ’740 Patent is entitled to the provisional date.
`
`SAMSUNG-1025.
`
`2
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`
`
`
`Reference
`
`Date
`
`Basis
`
`Kitamura
`
`Suh
`
`Manabe
`
`Noriyuki
`
`
`Published 9/20/2001
`
`§102(a), §102(b), and §102(e)
`
`Filed 10/7/1998
`
`Filed 7/22/2003
`
`§102(e)
`
`§102(e)
`
`Published 2/22/2002
`
`§102(a), §102(b)
`
`C. Level of Ordinary Skill in the Art
`A person of ordinary skill in the art (“POSITA”) as of the Critical Date
`
`would have had at least a Bachelor’s Degree in an academic area emphasizing
`
`electrical engineering, computer science, or a similar discipline, and at least two
`
`years of experience related to imaging technologies. Superior education could
`
`compensate for a deficiency in work experience, and vice-versa. SAMSUNG-
`
`1003, [0006].
`
`D. Claim Construction
`Unless otherwise noted, terms should be given their plain meaning, but Peti-
`
`tioner reserves the right to respond to any constructions offered by Patent Owner or
`
`the Board. Samsung is not waiving any arguments concerning indefiniteness or
`
`claim scope.
`
`3
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`“[processor/user interface] … configured to” (claims 10, 12,
`14, 16-17, 20-22, 24, 26-27)
`Petitioner advances ordinary meaning of these phrases in this proceeding;
`
`1.
`
`however, a question exists regarding whether they should be interpreted under 35
`
`U.S.C. §112, ¶6 (“112/6”). Williamson v. Citrix Online, LLC, 792 F.3d 1339, 1346
`
`(Fed. Cir. 2015). The absence of the word “means” creates a rebuttable presumption
`
`that 112/6 does not apply. Phillips v. AWH Corp., 415 F.3d 1303, 1311 (Fed. Cir.
`
`2005). In district court, the parties dispute whether the presumption should be re-
`
`butted. Although claim construction arguments are not yet final, Petitioner may ar-
`
`gue in district court that 112/6 applies and these claims are indefinite. SAMSUNG-
`
`1024, 54-62. However, Patent Owner contends that 112/6 doesn’t apply. Id.
`
`Because the dispute in district court remains unresolved and because Peti-
`
`tioner cannot raise indefiniteness here, Petitioner relies on the presumption in this
`
`proceeding and applies prior art to these claims’ ordinary meaning consistent with
`
`Patent Owner’s litigation position. Id. Petitioner will promptly inform the Board of
`
`any district court developments related to these claims’ definiteness.
`
`Further, when determining validity, claim terms need to only be construed to
`
`“resolve the controversy.” Wellman, Inc. v. Eastman Chem. Co., 642 F.3d 1355,
`
`1361 (Fed. Cir. 2011). Because this proceeding lacks arguments/evidence rebutting
`
`the presumption, the presumption should stand and the Board should forego con-
`
`struction absent Patent Owner advocating for 112/6’s application. If Patent Owner
`
`4
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`does not endorse a 112/6 construction, no controversy exists regarding 112/6’s ap-
`
`plication. Indeed, the presumption holds when neither party presents argument/evi-
`
`dence to rebut it. HTC America, Inc. v. Virginia Innovation Sciences, Inc., IPR2017-
`
`00872, Paper 11, 9; Dick v. New York Life Ins. Co., 359 U.S. 437, 443 n.3 (1959);
`
`Am. Hoist & Derrick Co. v. Sowa & Sons, Inc., 725 F.2d 1350, 1358 (Fed. Cir. 1984).
`
`Finally, any district court claim construction ruling is reviewable by the Fed-
`
`eral Circuit, and yet, denial of institution is non-appealable. Thus, for this claim
`
`construction theory, congressional intent is best served through institution, particu-
`
`larly considering that institution would involve prior art consideration against Patent
`
`Owner’s claim construction.
`
`Therefore, in this proceeding, these claim phrases should be given their ordi-
`
`nary meaning.
`
`2.
`“designating … the first subject” (claims 1, 10, 20)
`Petitioner has asserted in the co-pending district court proceeding that limi-
`
`tations of the form “designating … the first subject” should be construed to mean
`
`“identifying an object to use as a reference point for aligning images to correct
`
`blur.” SAMSUNG-1024, 50-51. The specification of the ’740 Patent supports this
`
`construction. See id., SAMSUNG-1001, 10:55-11:6 (“the reference point for
`
`aligning the higher speed images is … the [designated] subject itself.”).
`
`5
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`The present Petition addresses this claim language according to this con-
`
`struction (see Grounds 2A-2D), as well as according to Patent Owner’s proposed
`
`interpretation that “No construction necessary” (see Grounds 1A-1F). SAM-
`
`SUNG-1024, 50-51. Previous cases have made clear that PTAB rules do not pro-
`
`hibit Petitioner from addressing the prior art under alternative claim construc-
`
`tions. See, e.g., 10X Genomics v. Bio-Rad Labs, IPR2020-00086, Paper 8, 18-22
`
`(PTAB April 27, 2020); Western Digital Corp. v. SPEX Techs., Inc., IPR2018-
`
`00084, Paper 14, 12 (PTAB Apr. 25, 2018). In fact, the Board has previously en-
`
`dorsed this approach, stating that “judicial efficiency will be enhanced by allowing
`
`Petitioner to rely upon a claim construction that Patent Owner is relying upon in
`
`the related district court litigation to assert infringement of the challenged pa-
`
`tent.” 10X Genomics, 19.
`
`III. APPLICATION OF PRIOR ART
`As detailed below, this petition shows a reasonable likelihood that Samsung
`
`will prevail with respect to claims 1-29 of the ’740 Patent.
`
`6
`
`
`
`A.
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`1A: Kitamura and Noriyuki render obvious claims 1, 2, 4-5, 8, 10-
`12, 14-15, 18
`1. Kitamura describes combining digital images2
`Kitamura, in the field of digital cameras, describes combining images into a
`
`composite image. SAMSUNG-1005, Abstract. “[I]mage information from the
`
`CCD 102 [purple in FIG. 3, infra] is stored in the primary memory 103,” dotted
`
`brown lines. Id., [0035], FIG. 1. The camera includes “image processor 104”
`
`(green) controlled by controller 305 (dotted green lines), and a slot for “recording
`
`medium 108 [brown].” Id., [0028]; see [0034], [0036]; SAMSUNG-1003, [0017]-
`
`[0022].
`
`
`2 Petitioner incorporates the entirety of the discussions of Kitamura, Noriyuki, and
`
`the combination thereof, presented in Sections III.A.1-III.A.3, into Grounds 1A-
`
`1F.
`
`7
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`
`
`
`Kitamura (SAMSUNG-1005), Detail of FIG. 3 (annotated)
`
`A user provides input (dotted blue lines) to Kitamura’s controller via the op-
`
`eration panel (blue), which has “operation buttons” with which “a photographer se-
`
`lects a … mode.” SAMSUNG-1005, [0029], [0033], [0035], FIG. 3. Images are
`
`8
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`displayed on “an image display unit 111 comprising a liquid crystal display
`
`(LCD)” (red). Id., [0029]; SAMSUNG-1003, [0022]-[0023].
`
`
`
`Kitamura (SAMSUNG-1005), Detail of FIG. 2 (annotated)
`
`Kitamura’s camera “combin[es] sensed images” into a “composite image.”
`
`SAMSUNG-1005, [0003]. “[I]mage sensing conditions” are “modified for each
`
`image sensing.” Id., [0034]. In “‘blur control’ mode, the focus position is
`
`changed,” and “images are sensed” and “combined.” Id., [0034], [0041]. The pho-
`
`tographer approves the composite image or requests generation of a new composite
`
`image. See id., [0012], [0035]-[0036], [0041]-[0043], [0062], FIG. 3. For in-
`
`stance, images 500, 501 (red in FIG. 6, infra) are “photographed at slightly shifted
`
`focus positions,” where “image 500 has a focused background ‘B’, and image 501
`
`has a focused foreground ‘A’.” SAMSUNG-1005, [0060]-[0061]. The images are
`
`combined into “composite images” 502, 503 (purple), where “the foreground ‘A’ is
`
`9
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`focused and the background ‘B’ is more strongly blurred than the image 501.” Id.,
`
`[0061]. “[I]mage 502 has a slightly blurred background ‘B,’” and in image 503,
`
`“background ‘B’ has an enhanced blur condition.” Id., [0062]. “[I]f the photogra-
`
`pher likes the image,” it “is saved on the recording medium 108 [brown].” Id,
`
`[0062]; SAMSUNG-1003, [0024].
`
`Kitamura (SAMSUNG-1005), Detail of FIG. 6 (annotated)
`
`
`
`10
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`2.
`Noriyuki describes combining digital images
`Noriyuki, in the field of digital cameras, describes “creating a bokeh-con-
`
`trolled image” by combining images of a “main imaging subject.” SAMSUNG-
`
`1008, Abstract. Noriyuki’s camera includes “a liquid crystal monitor [LCD] 103
`
`as a viewfinder.” Id., [0014], [0016], FIG. 2. SAMSUNG-1003, [0071].
`
`Noriyuki’s digital camera “record[s] … image data read-in by the CCD,”
`
`and “form[s] a bokeh-controlled image.” SAMSUNG-1008, [0017]. Bokeh is “the
`
`visual quality of the out-of-focus areas of a photographic image.” SAMSUNG-
`
`1012, 3. Noriyuki’s bokeh-controlled image, e.g., image 15, purple in FIG. 4, in-
`
`fra, is formed by combining a “plurality of images that have different focal dis-
`
`tances,” e.g., images 12, 13, red. SAMSUNG-1008, [0021]; see [0023]-[0024]. In
`
`image 12, the “foreground[] is captured clearly” and “the background[] is imaged
`
`with bokeh,” while in image 13, the foreground has bokeh and the background “is
`
`imaged clearly.” Id., [0023]; see [0004], [0017], [Claims 1 and 5]. The bokeh set-
`
`ting is user-selected, e.g., “‘foreground focused with high bokeh in background,’
`
`‘foreground focused with low bokeh in background,’ ‘high bokeh in foreground
`
`with background focused,’ [or] ‘low bokeh in foreground with background fo-
`
`cused.’” Id., [0029]; SAMSUNG-1003, [0075]-[0078].
`
`11
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`
`
`
`Noriyuki (SAMSUNG-1008), Detail of FIG. 4 (annotated)
`
`Noriyuki’s process combines “one [image] with the main imaging subject 30
`
`[foreground, green in FIG. 5A, infra] in focus, and one focused infinitely distant 31
`
`[background, dotted brown lines].” SAMSUNG-1008, [0033]. Noriyuki’s camera
`
`automatically identifies a subject, e.g., the subject “nearest to the center of the
`
`screen” or “having a prescribed color.” Id., [0037]; SAMSUNG-1003, [0078]-
`
`[0079].
`
`12
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`
`
`
`Noriyuki (SAMSUNG-1008), Detail of FIG. 5A (annotated)
`
`3.
`
`The combination yields a digital camera having an LCD
`viewfinder and automated designation of a subject
`A POSITA would have found it obvious to modify Kitamura’s LCD image
`
`display unit (see SAMSUNG-1005, [0029], [0035], [0041]) to function “as a view-
`
`finder.” SAMSUNG-1008, [0016]; see [0014], FIG. 2. Additionally, a POSITA
`
`would have found it obvious to configure Kitamura’s processor to identify a sub-
`
`ject, e.g., the subject “nearest to the center” of the LCD or the “having a prescribed
`
`color.” SAMSUNG-1008, [0037]. A composite image is formed using images in-
`
`cluding the identified subject, with the composite image focused on the identified
`
`subject. See SAMSUNG-1005, [0061]; SAMSUNG-1003, [0085]-[0092].
`
`A POSITA also would have found it obvious to allow a user to specify a
`
`bokeh setting for the composite. See SAMSUNG-1008, [0029]; SAMSUNG-1005,
`
`[0061]. Noriyuki’s bokeh settings are displayed in the LCD, and the user “select[s]
`
`13
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`the preferred state of bokeh” using the buttons of Kitamura’s operation panel.
`
`SAMSUNG-1008, [0029]; see SAMSUNG-1005, [0029], [0035]. The images are
`
`“combined in accordance with” the user’s selection. SAMSUNG-1005, [0041];
`
`see SAMSUNG-1008, [0029]; SAMSUNG-1003, [0093]-[0096].
`
`4.
`Claim 1
`[1pre] “A method for use in an imaging device, the method comprising:”
`In the combination, Kitamura describes a method for use in an imaging de-
`
`vice. See SAMSUNG-1005, [0003] (“an image sensing device such as a digital
`
`camera … and … a method for combining sensed images”), [0014], [0027]-
`
`[0031], [0035]-[0037], [0041], [0061]-[0062], [0079]-[0081], claims 1 and 8,
`
`FIGS. 1 and 2; SAMSUNG-1003, [0019]-[0038].
`
` [1.1] “displaying an image in a user interface of the device, wherein the image
`is a preview of a field of view of the device, and wherein the image includes a
`first subject and a second subject;”
`In the combination, Kitamura describes displaying an image in a user in-
`
`terface of the device. Kitamura’s digital camera includes an “operation panel …
`
`[and] an image display unit 111 comprising a liquid crystal display (LCD).” SAM-
`
`SUNG-1005, [0029]; see FIG. 2, infra; FIG. 3, infra in [1.2]; [0033]. Kitamura’s
`
`operation panel and image display unit, red in FIG. 2, together are a user interface.
`
`An image is displayed in the user interface: “The prepared composite image is dis-
`
`played on the display unit.” Id., [0035]; see [0039], [0041], [0043]; SAMSUNG-
`
`1003, [0022]-[0023].
`
`14
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`
`
`
`Kitamura (SAMSUNG-1005), Detail of FIG. 2 (annotated)
`
`Also in the combination, Noriyuki describes a user interface. Noriyuki’s
`
`“an image processing mode setting key 102 [and] a liquid crystal monitor 103
`
`[LCD] as a viewfinder” (SAMSUNG-1008, [0016]; red in FIG. 2, infra) together
`
`are a user interface. SAMSUNG-1003, [0071]-[0072].
`
`15
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`
`
`
`Noriyuki (SAMSUNG-1008), Detail of FIG. 2 (annotated)
`
`Noriyuki describes or renders obvious displaying an image in the user in-
`
`terface, wherein the image is a preview of a field of view of the device. See
`
`SAMSUNG-1008, [0016] (“liquid crystal monitor 103 as a viewfinder”), [0037]
`
`(“the imaging subject nearest to the center of the screen”). The function of a
`
`viewfinder is to display a preview of a field of view of the camera. See SAM-
`
`SUNG-1016, 12:13-29; SAMSUNG-1015, 3:52-59. Noriyuki’s description of an
`
`LCD viewfinder is a description, or otherwise renders obvious, that an image dis-
`
`played on the LCD is a preview of the field of view of Noriyuki’s camera. SAM-
`
`SUNG-1003, [0073].
`
`A POSITA would have found it obvious to modify Kitamura’s camera such
`
`that the image displayed on Kitamura’s user interface is a preview of a field of
`
`view of the camera, based on Noriyuki’s teachings of an LCD viewfinder. See
`
`16
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`SAMSUNG-1008, [0014], [0016]. A POSITA would have been motivated to dis-
`
`play a preview on Kitamura’s user interface to improve the ease of use of
`
`Kitamura’s camera and to enhance the ergonomics of taking photographs. With a
`
`preview image displayed in Kitamura’s LCD, a user does not need to hold the cam-
`
`era to his eye to look through a finder window (see SAMSUNG-1005, [0029]) to
`
`see a preview. Rather, the preview is displayed on the LCD, allowing the user to
`
`hold the camera in a more convenient position, making the camera easier and more
`
`comfortable to operate. A POSITA would have expected success given that both
`
`Kitamura and Noriyuki describe cameras having LCDs for displaying images, and
`
`that Noriyuki describes a camera having an LCD viewfinder. See SAMSUNG-
`
`1005, [0029], [0035]; SAMSUNG-1008, [0014], [0016]; SAMSUNG-1003,
`
`[0085]-[0086].
`
`Also in the combination, Kitamura discloses photographing a scene that in-
`
`cludes a first subject and a second subject. “[I]mages 500 and 501 … were pho-
`
`tographed … with the letter ‘A’ as the foreground and the letter ‘B’ as the back-
`
`ground.” SAMSUNG-1005, [0061]. ‘A,’ green in FIG. 6, infra, is the first subject
`
`and ‘B,’ dotted brown lines, is the second subject. SAMSUNG-1003, [0028]-
`
`[0029].
`
`17
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`
`
`
`Kitamura (SAMSUNG-1005), Detail of FIG. 6 (annotated)
`
`Kitamura’s modified camera displays a preview in the LCD of the user inter-
`
`face. When the modified camera is used to photograph the scene of Kitamura’s
`
`images 500, 501 (see SAMSUNG-1005, [0061]), a POSITA would have recog-
`
`nized that the displayed preview includes the first and second subjects (‘A’ and
`
`‘B’) because these subjects fall within the field of view of the camera. SAM-
`
`SUNG-1003, [0087].
`
`[1.2] “designating by a processor of the imaging device the first subject in the
`image to be kept blur free;”
`In the combination, Kitamura describes that the first subject (‘A’) in the
`
`image is to be kept blur free: “[I]mage 501 has a focused foreground ‘A,’” and
`
`in composite images, 502, 503, “the foreground ‘A’ is focused.” SAMSUNG-
`
`1005, [0061]; SAMSUNG-1003, [0028]-[0031].
`
`Also in the combination, Noriyuki describes designating a first subject in
`
`the image to be kept blur free. “[T]he main imaging subject should be detected
`
`18
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`and identified as follows …. Specifically, the imaging subject nearest to the cen-
`
`ter of the screen may be identified as the main imaging subject, or an imaging
`
`subject having a prescribed color may be identified as the main imaging subject.
`
`Whether or not the prescribed color is present should be determined by the chro-
`
`maticity (*a*b*) that indicates the hue and color saturation in, for example, an
`
`L*a*b*color system.” SAMSUNG-1008, [0037]-[0038]; see [0045]. Noriyuki’s
`
`designated subject is a first subject in the image in the user interface. Noriyuki’s
`
`designated first subject is to be kept blur free: “it is appropriate to capture a total
`
`of two images, one with the main imaging subject 30 in focus.” SAMSUNG-
`
`1008, [0033]; SAMSUNG-1003, [0079]-[0080].
`
`In the combination, Noriyuki describes a processor: “CPU 40 carries out
`
`overall control of the rest of the digital camera 1.” SAMSUNG-1008, [0019]. A
`
`POSITA would have found obvious that Noriyuki’s processor designates the sub-
`
`ject given that the CPU controls the camera (id.), and given that Noriyuki’s desig-
`
`nation of a subject based on chromaticity (SAMSUNG-1008, [0037]-[0038]) is
`
`carried out by a processor, and is not an approach that a human would take. SAM-
`
`SUNG-1003, [0074], [0080]-[0081].
`
`Also in the combination, Kitamura’s image processor 104 and controller
`
`305, green in FIG. 3, infra, together are a processor. See SAMSUNG-1005,
`
`[0028], [0035], FIG. 1; SAMSUNG-1003, [0020].
`
`19
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`
`
`
`Kitamura (SAMSUNG-1005), Detail of FIG. 3 (annotated)
`
`A POSITA would have found it obvious to configure Kitamura’s processor
`
`to designate a first subject in the image to be kept blur free in light of Nori-
`
`yuki’s description of processor designation of a subject. See SAMSUNG-1008,
`
`[0037]-[0038]. In this implementation, Kitamura’s processor designates a subject,
`
`e.g., the subject “nearest to the center of” the LCD or “having a prescribed color.”
`
`20
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`SAMSUNG-1008, [0037]. An image is taken focused on the processor-designated
`
`subject. See SAMSUNG-1008, [0033]; SAMSUNG-1005, [0061]; SAMSUNG-
`
`1003, [0088]-[0089].
`
`A POSITA would have found it obvious to apply Noriyuki’s approach to
`
`processor designation of a subject in the context of Kitamura’s blur control mode.
`
`In this implementation, Kitamura’s processor designates the ‘A’ as the first subject
`
`to be kept blur free, e.g., based on the ‘A’ being nearest to the center of the screen
`
`or having a prescribed color. See SAMSUNG-1008, [0037]-[0038]; SAMSUNG-
`
`1005, [0061]. Kitamura’s modified camera then takes multiple images, with one of
`
`the images focused on the processor-designated ‘A,’ and a composite image is gen-
`
`erated such that the ‘A’ is focused. See SAMSUNG-1005, [0061]; SAMSUNG-
`
`1008, [0033], [0037]-[0038]; SAMSUNG-1003, [0088]-[0089].
`
`A POSITA would have been motivated to configure Kitamura’s processor to
`
`designate the first subject in the image to be kept blur free to implement a process
`
`for generating composite images in which the first subject is designated automati-
`
`cally, without user input. SAMSUNG-1008, [0037]-[0038]. This automation
`
`makes Kitamura’s blur control functionality easier to use and thus accessible to in-
`
`experienced photographers. SAMSUNG-1003, [0090].
`
`21
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`A POSITA would have expected success given that Kitamura’s processor
`
`has image analysis and recognition functionality. “[T]he background region is rec-
`
`ognized by the image processor 104.” SAMSUNG-1005, [0080]. A POSITA
`
`would have expected success in implementing processor designation of subjects in
`
`the context of a blur control process given that Noriyuki’s processor designation of
`
`subjects relates to producing “a bokeh-controlled image.” SAMSUNG-1008,
`
`[0004], see [0017], [0023]-[0024], [0029], [0037]-[0038]; SAMSUNG-1005,
`
`[0061]-[0062]. A POSITA also would have expected success given the general ap-
`
`plicability of Kitamura’s blur control process to combining images for which “the
`
`focus position is changed.” SAMSUNG-1005, [0034]; SAMSUNG-1003, [0091].
`
`[1.3] “capturing a plurality of images by the imaging device, wherein the plu-
`rality of images include the first subject and the second subject;”
`In the combination, Kitamura teaches capturing an image by the imaging
`
`device: “the photographer views the desired photographic scene … then presses
`
`the shutter button 106. … [T]he object image is photoelectrically converted … and
`
`the image sensing operation is performed.” SAMSUNG-1005, [0037]; see
`
`[0011], [0014], [0028]. Also in the combination, Kitamura describes capturing a
`
`plurality of images by the imaging device. “[I]mages 500 and 501 are images
`
`… photographed at slightly shifted focus positions.” SAMSUNG-1005, [0061].
`
`Kitamura’s images 500, 501, red in FIG. 6, infra, are a plurality of images. See
`
`[0034]-[0035], [0011], [0014], [0040]-[0041]; SAMSUNG-1003, [0027]-[0029].
`
`22
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`
`
`
`Kitamura (SAMSUNG-1005), Detail of FIG. 6 (annotated)
`
`Kitamura’s plurality of images (500, 501) include the first subject and
`
`the second subject (‘A’ and ‘B,’ respectively). “The scene [of images 500, 501]
`
`was photographed with the letter ‘A’ as the foreground and the letter ‘B’ as the
`
`background.” SAMSUNG-1005, [0061]. As shown in FIG. 6, supra, the plurality
`
`of images includes the first subject (‘A,’ green) and the second subject (‘B,’ dotted
`
`brown lines). SAMSUNG-1003, [0029].
`
`[1.4] “combining the plurality of images by the processor to obtain a com-
`bined image, such that:”
`In the combination, Kitamura describes combining the plurality of images
`
`(500, 501) by the processor (image processor 104) to obtain a combined image,
`
`i.e., image 502 or 503, purple in FIG. 6, infra. “Images 502 and 503 are compo-
`
`site images formed by the image processor 104 using the images 500 and 501.”
`
`23
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`SAMSUNG-1005, [0061]; see [0035], [0041], [0011], [0014], claim 1, claim 8;
`
`SAMSUNG-1003, [0030]-[0031].
`
`Kitamura (SAMSUNG-1005), Detail of FIG. 6 (annotated)
`
`
`
`24
`
`
`
`Attorney Docket No. 39843-0080IP1
`IPR of U.S. Patent No. 10,171,740
`[1.5] “the combined image includes the first subject and the second subject,
`the first subject in the combined image is blur free, and the second subject in
`the combined image is blurred compared to3 the first subject;”
`In the combination, Kitamura describes that the combined image (502 or
`
`503) includes the first subject and the second subject (‘A’ and ‘B,’ respec-
`
`tively), and that the first subject (‘A’) in the combined image