IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`Fatih M. Ozluturk
`In re Patent of:
`10,389,944 Attorney Docket No.: 39843-0079IP2
`U.S. Patent No.:
`August 20, 2019
`
`Issue Date:
`Appl. Serial No.: 15/858,339
`
`Filing Date:
`December 29, 2017
`
`Title:
`METHOD AND APPARATUS TO CORRECT BLUR IN ALL OR
`PART OF AN IMAGE
`
`
`Mail Stop Patent Board
`Patent Trial and Appeal Board
`U.S. Patent and Trademark Office
`P.O. Box 1450
`Alexandria, VA 22313-1450
`
`
`PETITION FOR INTER PARTES REVIEW OF UNITED STATES PATENT
`NO. 10,389,944 PURSUANT TO 35 U.S.C. §§ 311–319, 37 C.F.R. § 42
`
`
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`TABLE OF CONTENTS
`
`I. 
`
`II. 
`
`THE ’944 PATENT ......................................................................................... 1 
`A.  Overview ................................................................................................... 1 
`B.  Prosecution History ................................................................................... 1 
`
`REQUIREMENTS FOR IPR .......................................................................... 1 
`A.  Standing .................................................................................................... 1 
`B.  Challenge and Relief Requested ............................................................... 2 
`C.  Level of Ordinary Skill in the Art ............................................................. 3 
`D.  Claim Construction ................................................................................... 3 
`
`III.  APPLICATION OF PRIOR ART ................................................................... 6 
`A.  1A: Aizawa, Hyodo, and Mitsufumi render obvious claims 1, 3-6, 8-11,
`13-16, 18-20 .............................................................................................. 6 
`B.  Ground 1B: Aizawa, Hyodo, Mitsufumi, and Noriyuki render obvious
`claims 2, 7, 12, and 17 ............................................................................ 57 
`
`IV.  PTAB DISCRETION SHOULD NOT PRECLUDE INSTITUTION .......... 67 
`
`V. 
`
`PAYMENT OF FEES ................................................................................... 71 
`
`VI.  CONCLUSION .............................................................................................. 71 
`
`VII.  MANDATORY NOTICES UNDER 37 C.F.R § 42.8(a)(1) ......................... 71 
`A.  Real Party-In-Interest Under 37 C.F.R. § 42.8(b)(1) .............................. 71 
`B.  Related Matters Under 37 C.F.R. § 42.8(b)(2) ....................................... 71 
`C.  Lead And Back-Up Counsel Under 37 C.F.R. § 42.8(b)(3) ................... 71 
`D.  Service Information ................................................................................ 72 
`
`
`
`
`
`
`i
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`
`
`
`EXHIBITS
`
`SAMSUNG-1001 U.S. Patent No. 10,389,944 to Ozluturk (“the ’944 Patent”)
`
`SAMSUNG-1002 Excerpts from the Prosecution History of the ’944 Patent (“the
`Prosecution History”)
`
`SAMSUNG-1003 Declaration of Dr. Irfan Essa
`
`SAMSUNG-1004 Reserved
`
`SAMSUNG-1005 Reserved
`
`SAMSUNG-1006 Reserved
`
`SAMSUNG-1007 Reserved
`
`SAMSUNG-1008 Certified Translation of Japanese Patent Publication No.
`2002057933 (“Noriyuki”)
`
`SAMSUNG-1009 Unopposed Motion for Extension of Time to Move, Answer,
`or Otherwise Respond to Plaintiff’s Complaint (Clear Imag-
`ing Research, LLC v. Samsung Electronics Co. Ltd., 2:19-cv-
`326-JRG (E.D. Tex. Oct. 15, 2019))
`
`SAMSUNG-1010 U.S. Patent Publication No. 2001/0013895 to Aizawa (“Ai-
`zawa”)
`
`SAMSUNG-1011 Certified Translation of Japanese Patent Publication No.
`2003209727 (“Mitsufumi”)
`
`SAMSUNG-1012 Excerpt of The New Oxford American Dictionary (Second
`Edition, 2005)
`
`SAMSUNG-1013 U.S. Patent No. 7,034,881 to Hyodo (“Hyodo”)
`
`ii
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`
`SAMSUNG-1014 Reserved
`
`SAMSUNG-1015 Reserved
`
`SAMSUNG-1016 Reserved
`
`SAMSUNG-1017
`
`Joseph Guzman, “Fauci says second wave of coronavirus is
`‘inevitable’”, TheHill.com (Apr. 29, 2020), available at:
`https://thehill.com/changing-america/resilience/natural-disas-
`ters/495211-fauci-says-second-wave-of-coronavirus-is
`SAMSUNG-1018 Peter Wells, et al. “Texas puts reopening on hold in face of
`new Covid-19 outbreak”, Financial Times (June 25, 2020),
`available at https://www.ft.com/content/e35f3148-a797-4e6e-
`bf7a-1a7ce3181e97
`SAMSUNG-1019 Order Granting Joint Motion to Amend the Docket Control
`Order and Time for Claim Construction Expert Disclosures
`(Clear Imaging Research, LLC v. Samsung Electronics Co.
`Ltd., 2:19-cv-326-JRG (E.D. Tex. June 12, 2020))
`SAMSUNG-1020 Email to Clear Imaging counsel dated February 10, 2020
`SAMSUNG-1021 Email from Clear Imaging counsel dated July 3, 2020
`SAMSUNG-1022 Stipulation by Samsung
`
`SAMSUNG-1023 Reserved
`
`SAMSUNG-1024 P.R. 4-3 Joint Claim Construction and Prehearing Statement
`(Clear Imaging Research, LLC v. Samsung Electronics Co.
`Ltd., 2:19-cv-326-JRG (E.D. Tex. July 23, 2020))
`
`SAMSUNG-1025 Plaintiff’s Disclosure of Asserted Claims and Infringement
`Contentions Pursuant to Patent Local Rules (Clear Imaging
`Research, LLC v. Samsung Electronics Co. Ltd., 2:19-cv-326-
`JRG (E.D. Tex. Jan. 14, 2020))
`
`
`
`
`iii
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`Samsung petitions for Inter Partes Review (“IPR) of claims 1-20 (“the Chal-
`
`lenged Claims”) of U.S. Patent No. 10,389,944 (“the ’944 Patent”).
`
`I.
`
`THE ’944 PATENT
`A. Overview
`The ’944 Patent relates to correcting image blur in digital images by com-
`
`bining a plurality of captured images to create a combined image such that a main
`
`subject is blur free and other areas are blurred. SAMSUNG-1001, Abstract, 12:66-
`
`13:2; SAMSUNG-1003, [0010].
`
`B.
`Prosecution History
`In the first office action, the claim was rejected as “patentably indistinct
`
`from claim 1 of Application No. 15/231,332,” was subject to a statutory double pa-
`
`tenting rejection over claim 1 of U.S. 9,860,450, and was rejected as anticipated by
`
`US 2003/0076408. SAMSUNG-1002, 260-267. In response, claim 1 was can-
`
`celled and new claims 2-21 were introduced. Id., 216-228. Claims 2-21 were re-
`
`jected on the ground of nonstatutory double patenting over U.S. 10,171,740 in
`
`view of U.S. 9,800,878. Id., 171-183. The applicant filed a Terminal Disclaimer,
`
`and a Notice of Allowance followed. Id., 112, 136.
`
`II. REQUIREMENTS FOR IPR
`A.
`Standing
`Samsung certifies that the ’944 Patent is available for IPR. The present peti-
`
`tion is being filed within one year of service of a complaint against Samsung in the
`
`1
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`Eastern District of Texas. SAMSUNG-1009. Samsung is not barred or estopped
`
`from requesting this review.
`
`B. Challenge and Relief Requested
`Samsung requests cancellation of the Challenged Claims on the following
`
`grounds. Additional explanation and support is set forth in Dr. Irfan Essa’s decla-
`
`ration (SAMSUNG-1003).
`
`Ground
`1A
`1B
`
`§103(a) References
`Claims
`1, 3-6, 8-11, 13-16, 18-20 Aizawa, Hyodo, and Mitsufumi
`2, 7, 12, 17
`Aizawa, Hyodo, Mitsufumi, and
`Noriyuki
`
`The following table summarizes the prior art basis for each reference with
`
`respect to an assumed Critical Date of March 25, 2004.1 Each reference below
`
`qualifies as prior art:
`
`Reference
`
`Date
`
`Basis
`
`Published 8/16/2001
`
`§102(a), §102(b), and §102(e)
`
`Aizawa
`
`Hyodo
`
`Filed 10/30/1998
`
`Mitsufumi
`
`Filed 7/25/2003
`
`§102(e)
`
`§102(a)
`
`Published 2/22/2002
`Noriyuki
`
`1 Although the ’944 Patent lists a March 25, 2004 provisional application, Patent
`
`§102(a), §102(b)
`
`Owner does not contend that the ’944 Patent is entitled to the provisional date.
`
`SAMSUNG-1025.
`
`2
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`
`
`
`C. Level of Ordinary Skill in the Art
`A person of ordinary skill in the art (“POSITA”) as of the Critical Date
`
`would have had at least a Bachelor’s Degree in an academic area emphasizing
`
`electrical engineering, computer science, or a similar discipline, and at least two
`
`years of experience related to imaging technologies. Superior education could
`
`compensate for a deficiency in work experience, and vice-versa. SAMSUNG-
`
`1003, [0006].
`
`D. Claim Construction
`Unless otherwise noted, terms should be given their plain meaning, but Peti-
`
`tioner reserves the right to respond to any constructions offered by Patent Owner or
`
`the Board. Samsung is not waiving any arguments concerning indefiniteness or
`
`claim scope.
`
`1.
`“[processor/user interface] … configured to” (claims 6-7, 9-
`10, 16-17, 19-20)
`Petitioner advances ordinary meaning of these phrases in this proceeding;
`
`however, a question exists regarding whether they should be interpreted under 35
`
`U.S.C. §112, ¶6 (“112/6”). Williamson v. Citrix Online, LLC, 792 F.3d 1339, 1346
`
`(Fed. Cir. 2015). The absence of the word “means” creates a rebuttable presumption
`
`that 112/6 does not apply. Phillips v. AWH Corp., 415 F.3d 1303, 1311 (Fed. Cir.
`
`2005). In district court, the parties dispute whether the presumption should be re-
`
`3
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`butted. Although claim construction arguments are not yet final, Petitioner may ar-
`
`gue in district court that 112/6 applies and these claims are indefinite. SAMSUNG-
`
`1024, 70-78. However, Patent Owner contends that 112/6 doesn’t apply. Id.
`
`Because the dispute in district court remains unresolved and because Peti-
`
`tioner cannot raise indefiniteness here, Petitioner relies on the presumption in this
`
`proceeding and applies prior art to these claims’ ordinary meaning consistent with
`
`Patent Owner’s litigation position. Id. Petitioner will promptly inform the Board of
`
`any district court developments related to these claims’ definiteness.
`
`Further, when determining validity, claim terms need to only be construed to
`
`“resolve the controversy.” Wellman, Inc. v. Eastman Chem. Co., 642 F.3d 1355,
`
`1361 (Fed. Cir. 2011). Because this proceeding lacks arguments/evidence rebutting
`
`the presumption, the presumption should stand and the Board should forego con-
`
`struction absent Patent Owner advocating for 112/6’s application. If Patent Owner
`
`does not endorse a 112/6 construction, no controversy exists regarding 112/6’s ap-
`
`plication. Indeed, the presumption holds when neither party presents argument/evi-
`
`dence to rebut it. HTC America, Inc. v. Virginia Innovation Sciences, Inc., IPR2017-
`
`00872, Paper 11, 9; Dick v. New York Life Ins. Co., 359 U.S. 437, 443 n.3 (1959);
`
`Am. Hoist & Derrick Co. v. Sowa & Sons, Inc., 725 F.2d 1350, 1358 (Fed. Cir. 1984).
`
`Finally, any district court claim construction ruling is reviewable by the Fed-
`
`eral Circuit, and yet, denial of institution is non-appealable. Thus, for this claim
`
`4
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`construction theory, congressional intent is best served through institution, particu-
`
`larly considering that institution would involve prior art consideration against Patent
`
`Owner’s claim construction.
`
`Therefore, in this proceeding, these claim phrases should be given their ordi-
`
`nary meaning.
`
`1.
`“designating … a … subject” (claims 1, 2, 6, 7, 11, 12, 16,
`and 17)
`Petitioner has asserted in the co-pending district court proceeding that limi-
`
`tations of the form “designating … a … subject” should be construed to mean
`
`“identifying an object to use as a reference point for aligning images to correct
`
`blur.” SAMSUNG-1024, 63-64. The specification of the ’944 Patent supports this
`
`construction. See id., SAMSUNG-1001, 10:61-11:12 (“the reference point for
`
`aligning the higher speed images is … the [designated] subject itself.”).
`
`The present Petition addresses this claim language according to this con-
`
`struction, as well as according to Patent Owner’s proposed interpretation that “No
`
`construction necessary.” SAMSUNG-1024, 63-64. Previous cases have made
`
`clear that PTAB rules do not prohibit Petitioner from addressing the prior art under
`
`alternative claim constructions. See, e.g., 10X Genomics v. Bio-Rad Labs,
`
`IPR2020-00086, Paper 8, 18-22 (PTAB April 27, 2020); Western Digital Corp. v.
`
`SPEX Techs., Inc., IPR2018-00084, Paper 14, 12 (PTAB Apr. 25, 2018). In fact,
`
`the Board has previously endorsed this approach, stating that “judicial efficiency
`
`5
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`will be enhanced by allowing Petitioner to rely upon a claim construction that Pa-
`
`tent Owner is relying upon in the related district court litigation to assert infringe-
`
`ment of the challenged patent.” 10X Genomics, 19.
`
`III. APPLICATION OF PRIOR ART
`As detailed below, this petition shows a reasonable likelihood that Samsung
`
`will prevail with respect to claims 1-20 of the ’944 Patent.
`
`A.
`1A: Aizawa, Hyodo, and Mitsufumi render obvious claims 1, 3-6,
`8-11, 13-16, 18-20
`1.
`Aizawa describes generating a digital image with desired fo-
`cus by combining multiple, differently focused images2
`Aizawa, in the field of digital cameras, describes “reconstructing, from a
`
`plurality of images that are focused differently, an arbitrarily focused image that is
`
`an image wherein the degree of blur at any depth is suppressed or intensified.”
`
`SAMSUNG-1010, Abstract; SAMSUNG-1003, [0016]-[0037].
`
`Aizawa describes an apparatus for capturing images and combining captured
`
`images into a reconstructed, arbitrarily focused image. See SAMSUNG-1010, Ab-
`
`stract, [0018], [0043], [1006], claims 15 and 17. The apparatus includes “a CCD
`
`31,” purple in FIG. 9, infra, and a processor 32, green, that converts signals from
`
`
`2 Petitioner incorporates the entirety of the discussions of Aizawa, Mitsufumi, Hy-
`
`odo, and the combination thereof, presented in Sections III.A.1-III.A.4, into
`
`Grounds 1A and 1B.
`
`6
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`the CCD “to image data.” SAMSUNG-1010, [0116]. An image of a scene to be
`
`photographed is “displayed through a viewer 33 [red].” Id. Captured image data
`
`are “stored in a memory 35,” brown. Id.; SAMSUNG-1003, [0017]-[0018].
`
`
`
`Aizawa (SAMSUNG-1010), Detail of FIG. 9 (annotated)
`
`Aizawa’s apparatus captures multiple images of a scene, e.g., the scene de-
`
`picted in FIG. 10, infra. The user provides input by “manipulat[ing] a focus desig-
`
`nator 34,” blue in FIG. 9, to “designate[] at least two regions that are to be brought
`
`into focus.” SAMSUNG-1010, [0116]. Images are focused based on the user’s in-
`
`put. In the example of FIG. 10, “a near content image g1” is focused on “the re-
`
`gion (2,2) in the middle of the image occupied by the subject T,” green, and “a far
`
`content image g2” is focused on “the region (1,1) at the upper left,” blue. Id. “The
`
`focus adjustment mechanism 36 brings a designated region into focus and takes a
`
`7
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`picture,” “[t]hen … brings the next designated region into focus, [and] takes a pic-
`
`ture.” Id.; SAMSUNG-1003, [0020]-[0024].
`
`
`
`Aizawa (SAMSUNG-1010), Detail of FIG. 10 (annotated)
`
`In Aizawa’s image reconstruction process, an “arbitrarily focused image f,”
`
`purple in FIG. 1, infra, is “reconstructed from the focused images g1 and g2,” i.e.,
`
`from the near and far content images, both in red. SAMSUNG-1010, [0057]. An
`
`arbitrarily focused image is “an image wherein the degree of blur at each depth is
`
`arbitrarily suppressed or intensified.” Id., [0042]. Image reconstruction is per-
`
`formed by filters 10a, 10b that subject the near and far content images g1, g2, re-
`
`spectively, “to prescribed processing.” Id., [0043]. “A synthesizer 11 synthesizes”
`
`the filter outputs to produce the “reconstructed image f.” Id. The filters 10a, 10b
`
`8
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`and the synthesizer 11 are in green. Image reconstruction is carried out on Ai-
`
`zawa’s camera. See claim 17 (“The plural image simultaneous capturing camera
`
`… comprising an arbitrarily focused image synthesizing apparatus”); SAMSUNG-
`
`1003, [0025]-[0027].
`
`
`
`Aizawa (SAMSUNG-1010), Detail of FIG. 1 (annotated)
`
`The degree of blur of the near and far content in the arbitrarily focused re-
`
`constructed image f is determined by filter parameters Ra and Rb, which are “near
`
`content and far content blur radiuses for the desired image.” SAMSUNG-1010,
`
`[0043]. “By adjusting” these parameters, “an arbitrarily focused image can be re-
`
`constructed.” Id. To obtain an image such that “the degree of blur in the far con-
`
`tent region” is varied “while leaving the near content region in focus” (i.e., an im-
`
`age with a focused foreground and a blurred background), “Ra is set to Ra=0 and
`
`Rb is made to vary from 0 to 4.” Id., [0072]; SAMSUNG-1003, [0030]-[0031].
`
`9
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`2. Hyodo describes a digital camera in which a subject is des-
`ignated in an image in a touch screen LCD viewfinder
`Hyodo, in the field of digital cameras, describes an “electronic camera 1”
`
`with an “imaging part 2,” purple in FIG. 3, infra. SAMSUNG-1013, 5:6-14. “An
`
`object image formed on a light receiving surface of the CCD … is photoelectrically
`
`converted and read out as a video signal.” Id., 5:41-44. “The video signal … is
`
`processed … at the signal processing part 30, … and a captured image is shown on
`
`the display part 10.” Id., 5:55-59. The display part 10 is “an LCD,” and “a touch
`
`panel 12 with light permeability is provided over the display part 10.” The LCD
`
`display part 10 and touch panel 12 are shown together in red. SAMSUNG-1013,
`
`5:23-25, see FIG. 2; SAMSUNG-1003, [0038]-[0039].
`
`10
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`
`
`
`Hyodo (SAMSUNG-1013), Detail of FIG. 3 (annotated)
`
`In Hyodo’s camera, “when the cameraman touches the principal subject 40
`
`on the screen (S100), the CPU 38 [green in FIG. 3] gets the positional information
`
`about the touched portion in accordance with an input signal from the touch panel
`
`12 (S102).” SAMSUNG-1013, 6:43-49. A “circle 42 [blue in FIG. 7, infra] of a
`
`predetermined size (a principal subject selection frame) is displayed around the de-
`
`termined touched portion … so as to indicate the touched area.” Id., 6:49-52.
`
`“Then, the principal subject is determined in accordance with the touched portion.”
`
`Id., 6:59-7:21. “After the principal subject is determined …, the principal subject
`
`is focused, and … the designated image and the positional information about the
`
`11
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`principal subject are stored in the image memory.” Id., 7:22-33; SAMSUNG-
`
`1003, [0042]-[0049].
`
`
`
`Hyodo (SAMSUNG-1013), Detail of FIG. 7 (annotated)
`
`Hyodo also describes “an example wherein a plurality of subjects (two sub-
`
`jects) are designated.” SAMSUNG-1013, 10:13-14. A “first principal subject”
`
`(green in FIG. 13, infra) is designated by “a circle (a principal subject selection
`
`frame) 52,” and “the second principal subject” (dotted brown arrow) is designated
`
`by a circle 54. Id., 10:15-21; SAMSUNG-1003, [0043]-[0044].
`
`12
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`
`
`
`Hyodo (SAMSUNG-1013), Detail of FIG. 13 (annotated)
`
`3. Mitsufumi describes a digital camera that generates a com-
`bined image having a desired amount of blur
`Mitsufumi, in the field of digital cameras, describes a digital camera that
`
`generates “an image with a desired amount of bokeh.” SAMSUNG-1011, [0006].
`
`Bokeh is “the visual quality of the out-of-focus areas of a photographic image.”
`
`SAMSUNG-1012, 3; SAMSUNG-1003, [0051].
`
`13
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`A “single image is formed through imaging the background and foreground
`
`separately and then compositing,” which “makes it possible to capture easily an
`
`image with a desire[d] amount of bokeh.” SAMSUNG-1011, [0066]. For in-
`
`stance, when photographing the scene of FIG. 6, infra, “imaging is carried out for
`
`four frames,” each focused on a different element of the scene. Id., [0060]-[0061];
`
`see FIGS. 9A-9D. The four frames are processed and “composited to form a single
`
`image,” which “is stored in the memory 60” of the digital camera and displayed on
`
`an LCD. Id., [0061], [0064], [0128]; SAMSUNG-1003, [0052]-[0053].
`
`Mitsufumi (SAMSUNG-1011), Detail of FIG. 6
`
`
`
`14
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`4.
`In the combination, a subject is designated in a preview im-
`age displayed in a touchscreen LCD display
`A POSITA would have found it obvious to modify Aizawa’s viewer to in-
`
`clude an LCD for display of preview images. See SAMSUNG-1010, [0116], FIG.
`
`9; SAMSUNG-1013, 5:23-25, FIG. 3. In this implementation, Aizawa’s camera is
`
`modified to include features such as an output memory and a D/A converter, as
`
`taught by Hyodo, that enable real time display of preview images on the LCD of
`
`the viewer. See SAMSUNG-1013, 5:41-59. Aizawa’s processor “determines the
`
`touched portion of the touch panel … in accordance with signals sent from the
`
`touch panel.” SAMSUNG-1013, 5:65-67; SAMSUNG-1003, [0054]-[0064].
`
`In addition, a POSITA would have found it obvious to incorporate a light
`
`permeable touch panel over the LCD of Aizawa’s modified viewer. See SAM-
`
`SUNG-1013, 5:23-25, FIG. 3. A user designates a subject in the image displayed
`
`in the LCD by “touch[ing] the principal subject … on the screen.” SAMSUNG-
`
`1013, 6:46. Responsive to the user’s touch input, Aizawa’s processor receives
`
`“positional information about” the touched portion of the image, in accordance
`
`with the operation of Hyodo’s CPU. See SAMSUNG-1010, [0018], [0116]; SAM-
`
`SUNG-1013, 6:43-49. A “principal subject selection frame,” such as a circle, is
`
`displayed around the touched area in the image in the LCD of the modified viewer,
`
`and the “principal subject” is “determined in accordance with” the touched area.
`
`15
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`SAMSUNG-1013, 6:42-7:21, FIG. 7. The designated “principal subject is fo-
`
`cused” and “the image is designated to be stored” in Aizawa’s memory. SAM-
`
`SUNG-1013, 7:29-32; SAMSUNG-1010, [0116]; SAMSUNG-1003, [0054]-
`
`[0064].
`
`Aizawa’s modified camera combines multiple, differently focused images to
`
`generate a reconstructed, arbitrarily focused image, such as an image having a fo-
`
`cused foreground and a blurred background. See SAMSUNG-1010, [0043],
`
`[0072], claims 15 and 17. Based on the teachings of Mitsufumi, a POSITA would
`
`have found it obvious to store the reconstructed image in the memory of Aizawa’s
`
`camera. See SAMSUNG-1011, [0128]; SAMSUNG-1010, [0018], [0116]; SAM-
`
`SUNG-1003, [0066]-[0067].
`
`5.
`Claim 1
`[1pre] “A method for use in an imaging device for capturing digital images,
`the method comprising:”
`In the combination, Aizawa describes a method for use in an imaging de-
`
`vice. See SAMSUNG-1010, [0036] (“FIG. 9 is a simplified block diagram of a
`
`digital camera”), claim 15 (“A plural image simultaneous capturing camera com-
`
`prising: a camera element; [and] a processor for receiving signals from said camera
`
`element and converting same to image data”), claim 17 (“The plural image simul-
`
`taneous capturing camera according to claim 15, further comprising an arbitrarily
`
`16
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`focused image synthesizing apparatus”), [0018], [0043], [0116], FIG. 9; SAM-
`
`SUNG-1003, [0016]-[0037].
`
` [1.1] “displaying a preview image of the scene to be captured in a user inter-
`face of the device;”
`In the combination, Aizawa describes displaying an image in a user inter-
`
`face of the device. “Light that has passed through a lens 30 enters a CCD 31 and
`
`is converted to image data by a processor. An image is displayed through a
`
`viewer 33 which the user can see.” SAMSUNG-1010, [0116]. Aizawa’s viewer
`
`33, red in FIG. 9, infra, is a user interface. See FIG. 10, showing “[t]he image
`
`displayed through the viewer” (id., [0116]), [0018] (“A plural image simultaneous
`
`capturing camera … comprises: a camera element; a processor for receiving sig-
`
`nals from the camera element and converting them to image data; a display unit
`
`for displaying image data processed by the processor.”); SAMSUNG-1003,
`
`[0017]-[0019].
`
`17
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`
`
`
`Aizawa (SAMSUNG-1010), Detail of FIG. 9 (annotated)
`
`In the combination, the image displayed in Aizawa’s viewer is a preview
`
`image of the scene to be captured: “Light that has passed through a lens 30 enters
`
`a CCD 31 and is converted to image data … An image is displayed through a
`
`viewer 33.” SAMSUNG-1010, [0116]. An example of the preview image dis-
`
`played in Aizawa’s user interface is shown in FIG. 10, infra, which shows “[t]he
`
`image displayed through the viewer 33.” Id. The displayed image is a preview im-
`
`age of the scene to be captured because the displayed image is an image of a scene
`
`to be photographed (e.g., rather than a previously photographed image). SAM-
`
`SUNG-1003, [0017]-[0019].
`
`18
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`
`
`
`Aizawa (SAMSUNG-1010), Detail of FIG. 10 (annotated)
`
`Also in the combination, Hyodo describes a user interface of the device.
`
`Hyodo’s camera includes “[a]n image display part 10” that “is an LCD … and a
`
`touch panel 12 with light permeability is provided over the display part 10.” SAM-
`
`SUNG-1013, 5:22-25. Hyodo’s image display part (e.g., LCD) and touch panel,
`
`red in FIG. 3, infra, together are a user interface. SAMSUNG-1003, [0038]-
`
`[0039].
`
`19
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`
`
`
`Hyodo (SAMSUNG-1013), Detail of FIG. 3 (annotated)
`
`Hyodo describes displaying an image in the user interface, i.e., in the
`
`LCD of the user interface: “a captured image is shown on the display part 10.”
`
`SAMSUNG-1013, 5:59; see 1:61-62, 2:7-9, 6:4-5, 13:33-34, FIG. 2, FIG. 13. The
`
`image displayed in Hyodo’s user interface is a preview of a field of view of the
`
`device: “The display part 10 can show … images … captured before the release
`
`button is touched.” SAMSUNG-1013, 5:55-63; see 13:31-34 (“an image captur-
`
`ing device for capturing an image of a field … [and] a display part for showing an
`
`image captured by the image capturing device”), 5:40-59, 6:44-7:47, explaining
`
`that user interaction with the displayed preview image occurs before a picture is
`
`taken. An example of the preview of the field of view of the device displayed in
`
`the LCD of Hyodo’s user interface is shown in Hyodo’s FIG. 13, infra, which
`
`20
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`shows “a view illustrating a state wherein a plurality of principal subjects are des-
`
`ignated on the screen.” Id, 4:57-59; SAMSUNG-1003, [0038]-[0041].
`
`
`
`Hyodo (SAMSUNG-1013), Detail of FIG. 13 (annotated)
`
`A POSITA would have found it obvious to implement Aizawa’s user inter-
`
`face (viewer 33) to include an LCD and a light permeable touch panel provided
`
`over the LCD, as taught by Hyodo, as shown in the following Figure based on Ai-
`
`zawa’s FIG. 9. See SAMSUNG-1010, [0116]; SAMSUNG-1013, 5:23-25. The
`
`LCD and touch sensitive panel together are a user interface, and a preview image
`
`21
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`is displayed on the LCD. In this implementation, Aizawa’s camera is modified to
`
`include features such as an output memory and a D/A converter, as taught by Hy-
`
`odo, that enable real time display of preview images on the LCD of the user inter-
`
`face. See SAMSUNG-1013, 5:41-59. When Aizawa’s modified camera is used
`
`for imaging the scene depicted in Hyodo’s FIG. 13, that scene is displayed as a
`
`preview image in the LCD of the user interface. See id., 10:13-34. When a user
`
`touches the touch panel to designate a subject (see id., 6:43-52, 10:13-21), the pro-
`
`cessor 32 “determines the touched portion of the touch panel … in accordance with
`
`signals sent from the touch panel,” the subject is determined and focused, and an
`
`image is taken and stored. Id., 5:65-67; see 6:43-52, 10:13-21; SAMSUNG-1003,
`
`[0054]-[0064].
`
`22
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`
`
`
`Aizawa (SAMSUNG-1010), Detail of FIG. 9, modified based on Hyodo
`
`A POSITA would have been motivated to modify Aizawa’s user interface to
`
`include an LCD for display of preview images to improve the ease of use of Ai-
`
`zawa’s camera and to enhance the ergonomics of taking photographs. SAM-
`
`SUNG-1010, [0018], [0116]; SAMSUNG-1013, 5:23-25, 6:43-52, 10:13-21. With
`
`preview images displayed in an LCD, a user does not need to hold the camera to
`
`his eye to look through a finder window to view a preview of the scene being pho-
`
`tographed. Rather, the user views the scene displayed on the LCD of the user in-
`
`terface, allowing the user to hold the camera in a more convenient position. This
`
`23
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`modification makes the camera easier and more comfortable to operate. Further-
`
`more, a POSITA would have been motivated to integrate a touch panel into Ai-
`
`zawa’s modified viewer to provide users with an avenue for direct interaction with
`
`the preview image displayed on the LCD of the user interface, , enabling an intui-
`
`tive and straightforward way for a user to provide input to the camera. See SAM-
`
`SUNG-1013, 5:23-25, 6:43-52, 10:13-21; SAMSUNG-1003, [0055].
`
`A POSITA would have expected success given that both Aizawa and Hyodo
`
`describe digital cameras having user interfaces capable of displaying preview im-
`
`ages (see SAMSUNG-1010, [0018], [0116]; SAMSUNG-1013, 5:55-63, 13:31-34,
`
`6:44-7:47), and that Hyodo confirms that an LCD of a digital camera can display
`
`preview images and can function in conjunction with a touch panels. See SAM-
`
`SUNG-1013, 5:23-25. A POSITA also would have expected success given that
`
`Aizawa’s disclosure is generic as to the specific implementation of the viewer.
`
`SAMSUNG-1003, [0056].
`
`[1.2] “receiving a user input in the user interface, wherein the user input des-
`ignates a first subject in the preview image;”
`In the combination, Aizawa describes receiving a user input designating a
`
`first subject in the preview image. “While viewing the image through the viewer
`
`33, the user manipulates a focus designator 34 and designates at least two re-
`
`gions that are to be brought into focus. … Upon receiving a signal from the focus
`
`designator 34, the processor 32 drives a focus adjustment mechanism 36. The
`
`24
`
`

`

`Attorney Docket No. 39843-0079IP2
`IPR of U.S. Patent No. 10,389,944
`focus adjustment mechanism 36 brings a designated region into focus and takes a
`
`picture.” Aizawa, [0116]; see [0018] (“a processor … [and] a focal point designa-
`
`tor for designating a plurality of subjects inside an image … wherein the proces-
`
`sor respectively and in order focuses the plurality of subjects designated.”); claim
`
`15. The designated subjects include a near content subject (foreground subject)
`
`and a far content subject (background subject), which are then photographed as “a
`
`near content image g1” in which the near content subject is focused, and “a far
`
`content image g2” in which the background is in focus. Id., [0116]. The user’s
`
`manipulation of the focus designator (blue in FIG. 9, infra) to designate one of the
`
`“plurality of subjects inside an image” or one of the “at least two regions that are to
`
`be brought into focus” is user input designating a first subject in the image.
`
`The user input (dotted blue lines) is received, e.g., by Aizawa’s processor: “Upon
`
`receiving a signal from the focus designator 34, the processor drives a focus ad-
`
`justment mechanism.” Id., [0116]. The designated subject is focused: “the pro-
`
`cessor respectively and in order focuses the plurality of subjects designated.” Id.,
`
`[0018]; see [011