IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`Fatih M. Ozluturk
`In re Patent of:
`10,171,740 Attorney Docket No.: 39843-0080IP2
`U.S. Patent No.:
`January 1, 2019
`
`Issue Date:
`Appl. Serial No.: 15/714,558
`
`Filing Date:
`September 25, 2017
`
`Title:
`METHOD AND APPARATUS TO CORRECT BLUR IN ALL OR
`PART OF A DIGITAL IMAGE BY COMBINING PLURALITY
`OF IMAGES
`
`
`Mail Stop Patent Board
`Patent Trial and Appeal Board
`U.S. Patent and Trademark Office
`P.O. Box 1450
`Alexandria, VA 22313-1450
`
`
`PETITION FOR INTER PARTES REVIEW OF UNITED STATES PATENT
`NO. 10,171,740 PURSUANT TO 35 U.S.C. §§ 311–319, 37 C.F.R. § 42
`
`
`
`
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`TABLE OF CONTENTS
`
`I. 
`
`II. 
`
`THE ’740 PATENT ......................................................................................... 1 
`A.  Overview ................................................................................................... 1 
`B.  Prosecution History ................................................................................... 1 
`
`REQUIREMENTS FOR IPR .......................................................................... 1 
`A.  Standing .................................................................................................... 1 
`B.  Challenge and Relief Requested ............................................................... 2 
`C.  Level of Ordinary Skill in the Art ............................................................. 3 
`D.  Claim Construction ................................................................................... 3 
`
`III.  APPLICATION OF PRIOR ART ................................................................... 6 
`A.  1A: Aizawa, Hyodo, and Mitsufumi render obvious claims 1, 3-11, 13-
`20, and 23-29 ............................................................................................ 6 
`B.  1B: Aizawa, Hyodo, Mitsufumi, and Noriyuki render obvious claims 2,
`12, 21, and 22 .......................................................................................... 66 
`
`IV.  PTAB DISCRETION SHOULD NOT PRECLUDE INSTITUTION .......... 73 
`
`V. 
`
`PAYMENT OF FEES ................................................................................... 77 
`
`VI.  CONCLUSION .............................................................................................. 77 
`
`VII.  MANDATORY NOTICES UNDER 37 C.F.R § 42.8(a)(1) ......................... 77 
`A.  Real Party-In-Interest Under 37 C.F.R. § 42.8(b)(1) .............................. 77 
`B.  Related Matters Under 37 C.F.R. § 42.8(b)(2) ....................................... 77 
`C.  Lead And Back-Up Counsel Under 37 C.F.R. § 42.8(b)(3) ................... 77 
`D.  Service Information ................................................................................ 78 
`
`
`
`
`
`
`i
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`
`
`
`EXHIBITS
`
`SAMSUNG-1001 U.S. Patent No. 10,171,740 to Ozluturk (“the ’740 Patent”)
`
`SAMSUNG-1002 Excerpts from the Prosecution History of the ’740 Patent (“the
`Prosecution History”)
`
`SAMSUNG-1003 Declaration of Dr. Irfan Essa
`
`SAMSUNG-1004 Reserved
`
`SAMSUNG-1005 Reserved
`
`SAMSUNG-1006 Reserved
`
`SAMSUNG-1007 Reserved
`
`SAMSUNG-1008 Certified Translation of Japanese Patent Publication No.
`2002057933 (“Noriyuki”)
`
`SAMSUNG-1009 Unopposed Motion for Extension of Time to Move, Answer,
`or Otherwise Respond to Plaintiff’s Complaint (Clear Imag-
`ing Research, LLC v. Samsung Electronics Co. Ltd., 2:19-cv-
`326-JRG (E.D. Tex. Oct. 15, 2019))
`
`SAMSUNG-1010 U.S. Patent Publication No. 2001/0013895 to Aizawa (“Ai-
`zawa”)
`
`SAMSUNG-1011 Certified translation of Japanese Patent Publication No.
`2003209727 (“Mitsufumi”)
`
`SAMSUNG-1012 Excerpt of The New Oxford American Dictionary (Second
`Edition, 2005)
`
`SAMSUNG-1013 U.S. Patent No. 7,034,881 to Hyodo (“Hyodo”)
`
`ii
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`
`SAMSUNG-1014 Reserved
`
`SAMSUNG-1015 Reserved
`
`SAMSUNG-1016 Reserved
`
`SAMSUNG-1017 Joseph Guzman, “Fauci says second wave of coronavirus is
`‘inevitable’”, TheHill.com (Apr. 29, 2020), available at:
`https://thehill.com/changing-america/resilience/natural-disas-
`ters/495211-fauci-says-second-wave-of-coronavirus-is
`SAMSUNG-1018 Peter Wells, et al. “Texas puts reopening on hold in face of
`new Covid-19 outbreak”, Financial Times (June 25, 2020),
`available at https://www.ft.com/content/e35f3148-a797-4e6e-
`bf7a-1a7ce3181e97
`SAMSUNG-1019 Order Granting Joint Motion to Amend the Docket Control
`Order and Time for Claim Construction Expert Disclosures
`(Clear Imaging Research, LLC v. Samsung Electronics Co.
`Ltd., 2:19-cv-326-JRG (E.D. Tex. June 12, 2020))
`SAMSUNG-1020 Email to Clear Imaging counsel dated February 10, 2020
`SAMSUNG-1021 Email from Clear Imaging counsel dated July 3, 2020
`SAMSUNG-1022 Stipulation by Samsung
`SAMSUNG-1023 Reserved
`SAMSUNG-1024 P.R. 4-3 Joint Claim Construction and Prehearing Statement
`(Clear Imaging Research, LLC v. Samsung Electronics Co.
`Ltd., 2:19-cv-326-JRG (E.D. Tex. July 23, 2020))
`SAMSUNG-1025 Plaintiff’s Disclosure of Asserted Claims and Infringement
`Contentions Pursuant to Patent Local Rules (Clear Imaging
`Research, LLC v. Samsung Electronics Co. Ltd., 2:19-cv-326-
`JRG (E.D. Tex. Jan. 14, 2020))
`
`
`
`
`iii
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`Samsung petitions for Inter Partes Review (“IPR) of claims 1-29 (“the Chal-
`
`lenged Claims”) of U.S. Patent No. 10,171,740 (“the ’740 Patent”).
`
`I.
`
`THE ’740 PATENT
`A. Overview
`The ’740 Patent relates to an approach to correcting blur in digital images by
`
`combining captured images to create a combined image such that a main subject is
`
`blur free and other areas are blurred. SAMSUNG-1001, Abstract, 12:54-62; SAM-
`
`SUNG-1003, [0010].
`
`B.
`Prosecution History
`The sole claim was rejected as “being a substantial duplicate of claim 1 of
`
`US Patent 9,774,785;” was subject to double patenting rejections over claims of
`
`US 9,774,785, US 9,392,175, and US 9,154,699; and was rejected as anticipated
`
`over US 2003/0076408. SAMSUNG-1002, 140-148. In response, claim 1 was
`
`cancelled and claims 2-30 were introduced. Id., 92-105. A Notice of Allowance
`
`followed. Id., 50.
`
`II. REQUIREMENTS FOR IPR
`A.
`Standing
`Samsung certifies that the ’740 Patent is available for IPR. The present peti-
`
`tion is being filed within one year of service of a complaint against Samsung in the
`
`Eastern District of Texas. SAMSUNG-1009. Samsung is not barred or estopped
`
`from requesting this review.
`
`1
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`
`B. Challenge and Relief Requested
`Samsung requests cancellation of the Challenged Claims on the following
`
`grounds. Additional explanation and support for each ground of rejection is set
`
`forth in Dr. Irfan Essa’s declaration (SAMSUNG-1003).
`
`Ground
`1A
`1B
`
`Claims
`1, 3-11, 13-20, 23-29
`2, 12, 21, 22
`
`§103(a) References
`Aizawa, Hyodo, and Mitsufumi
`Aizawa, Hyodo, Mitsufumi, and
`Noriyuki
`
`The following table summarizes the prior art basis for each reference with
`
`respect to an assumed Critical Date of March 25, 2004.1 Each reference below
`
`qualifies as prior art:
`
`Reference
`
`Date
`
`Basis
`
`Aizawa
`
`Hyodo
`
`Mitsufumi
`
`Noriyuki
`
`
`Published 8/16/2001
`
`§102(a), §102(b), and §102(e)
`
`Filed 10/30/1998
`
`Filed 7/25/2003
`
`§102(e)
`
`§102(a)
`
`Published 2/22/2002
`
`§102(a), §102(b)
`
`
`1 Although the ’740 Patent lists a March 25, 2004 provisional application, Patent
`
`Owner does not contend that the ’740 Patent is entitled to the provisional date.
`
`SAMSUNG-1025.
`
`2
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`C. Level of Ordinary Skill in the Art
`A person of ordinary skill in the art (“POSITA”) as of the Critical Date
`
`would have had at least a Bachelor’s Degree in an academic area emphasizing
`
`electrical engineering, computer science, or a similar discipline, and at least two
`
`years of experience related to imaging technologies. Superior education could
`
`compensate for a deficiency in work experience, and vice-versa. SAMSUNG-
`
`1003, [0006].
`
`D. Claim Construction
`Unless otherwise noted, terms should be given their plain meaning, but Peti-
`
`tioner reserves the right to respond to any constructions offered by Patent Owner or
`
`the Board. Samsung is not waiving any arguments concerning indefiniteness or
`
`claim scope.
`
`1.
`“[processor/user interface] … configured to” (claims 10, 12,
`14, 16-17, 20-22, 24, 26-27)
`Petitioner advances ordinary meaning of these phrases in this proceeding;
`
`however, a question exists regarding whether they should be interpreted under 35
`
`U.S.C. §112, ¶6 (“112/6”). Williamson v. Citrix Online, LLC, 792 F.3d 1339, 1346
`
`(Fed. Cir. 2015). The absence of the word “means” creates a rebuttable presumption
`
`that 112/6 does not apply. Phillips v. AWH Corp., 415 F.3d 1303, 1311 (Fed. Cir.
`
`2005). In district court, the parties dispute whether the presumption should be re-
`
`3
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`butted. Although claim construction arguments are not yet final, Petitioner may ar-
`
`gue in district court that 112/6 applies and these claims are indefinite. SAMSUNG-
`
`1024, 54-62. However, Patent Owner contends that 112/6 doesn’t apply. Id.
`
`Because the dispute in district court remains unresolved and because Peti-
`
`tioner cannot raise indefiniteness here, Petitioner relies on the presumption in this
`
`proceeding and applies prior art to these claims’ ordinary meaning consistent with
`
`Patent Owner’s litigation position. Id. Petitioner will promptly inform the Board of
`
`any district court developments related to these claims’ definiteness.
`
`Further, when determining validity, claim terms need to only be construed to
`
`“resolve the controversy.” Wellman, Inc. v. Eastman Chem. Co., 642 F.3d 1355,
`
`1361 (Fed. Cir. 2011). Because this proceeding lacks arguments/evidence rebutting
`
`the presumption, the presumption should stand and the Board should forego con-
`
`struction absent Patent Owner advocating for 112/6’s application. If Patent Owner
`
`does not endorse a 112/6 construction, no controversy exists regarding 112/6’s ap-
`
`plication. Indeed, the presumption holds when neither party presents argument/evi-
`
`dence to rebut it. HTC America, Inc. v. Virginia Innovation Sciences, Inc., IPR2017-
`
`00872, Paper 11, 9; Dick v. New York Life Ins. Co., 359 U.S. 437, 443 n.3 (1959);
`
`Am. Hoist & Derrick Co. v. Sowa & Sons, Inc., 725 F.2d 1350, 1358 (Fed. Cir. 1984).
`
`Finally, any district court claim construction ruling is reviewable by the Fed-
`
`eral Circuit, and yet, denial of institution is non-appealable. Thus, for this claim
`
`4
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`construction theory, congressional intent is best served through institution, particu-
`
`larly considering that institution would involve prior art consideration against Patent
`
`Owner’s claim construction.
`
`Therefore, in this proceeding, these claim phrases should be given their ordi-
`
`nary meaning.
`
`2.
`“designating … the first subject” (claims 1, 10, 20)
`Petitioner has asserted in the co-pending district court proceeding that limi-
`
`tations of the form “designating … the first subject” should be construed to mean
`
`“identifying an object to use as a reference point for aligning images to correct
`
`blur.” SAMSUNG-1024, 50-51. The specification of the ’740 Patent supports this
`
`construction. See id., SAMSUNG-1001, 10:55-11:6 (“the reference point for
`
`aligning the higher speed images is … the [designated] subject itself.”).
`
`The present Petition addresses this claim language according to this con-
`
`struction, as well as according to Patent Owner’s proposed interpretation that “No
`
`construction necessary.” SAMSUNG-1024, 50-51. Previous cases have made
`
`clear that PTAB rules do not prohibit Petitioner from addressing the prior art under
`
`alternative claim constructions. See, e.g., 10X Genomics v. Bio-Rad Labs,
`
`IPR2020-00086, Paper 8, 18-22 (PTAB April 27, 2020); Western Digital Corp. v.
`
`SPEX Techs., Inc., IPR2018-00084, Paper 14, 12 (PTAB Apr. 25, 2018). In fact,
`
`the Board has previously endorsed this approach, stating that “judicial efficiency
`
`5
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`will be enhanced by allowing Petitioner to rely upon a claim construction that Pa-
`
`tent Owner is relying upon in the related district court litigation to assert infringe-
`
`ment of the challenged patent.” 10X Genomics, 19.
`
`III. APPLICATION OF PRIOR ART
`As detailed below, this petition shows a reasonable likelihood that Samsung
`
`will prevail with respect to claims 1-29 of the ’740 Patent.
`
`A.
`1A: Aizawa, Hyodo, and Mitsufumi render obvious claims 1, 3-11,
`13-20, and 23-29
`1.
`Aizawa describes generating a digital image with desired fo-
`cus by combining multiple, differently focused images2
`Aizawa, in the field of digital cameras, describes “reconstructing, from a
`
`plurality of images that are focused differently, an arbitrarily focused image that is
`
`an image wherein the degree of blur at any depth is suppressed or intensified.”
`
`SAMSUNG-1010, Abstract; SAMSUNG-1003, [0016]-[0037].
`
`Aizawa describes an apparatus for capturing images and combining captured
`
`images into a reconstructed, arbitrarily focused image. See SAMSUNG-1010, Ab-
`
`stract, [0018], [0043], [1006], claims 15 and 17. The apparatus includes “a CCD
`
`31,” purple in FIG. 9, infra, and a processor 32, green, that converts signals from
`
`
`2 Petitioner expressly incorporates the entirety of the discussions of Aizawa,
`
`Mitsufumi, Hyodo, and the combination thereof, presented in Sections III.A.1-
`
`III.A.3, into Grounds 1A-1B, infra.
`
`6
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`the CCD “to image data.” SAMSUNG-1010, [0116]. An image of a scene to be
`
`photographed is “displayed through a viewer 33 [red].” Id. Captured image data
`
`are “stored in a memory 35,” brown. Id.; SAMSUNG-1003, [0017]-[0018].
`
`
`
`Aizawa (SAMSUNG-1010), Detail of FIG. 9 (annotated)
`
`Aizawa’s apparatus captures multiple images of a scene, e.g., the scene de-
`
`picted in FIG. 10, infra. The user provides input by “manipulat[ing] a focus desig-
`
`nator 34,” blue in FIG. 9, to “designate[] at least two regions that are to be brought
`
`into focus.” SAMSUNG-1010, [0116]. Images are focused based on the user’s in-
`
`put. In the example of FIG. 10, “a near content image g1” is focused on “the re-
`
`gion (2,2) in the middle of the image occupied by the subject T,” green, and “a far
`
`content image g2” is focused on “the region (1,1) at the upper left,” blue. Id. “The
`
`focus adjustment mechanism 36 brings a designated region into focus and takes a
`
`7
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`picture,” “[t]hen … brings the next designated region into focus, [and] takes a pic-
`
`ture.” Id.; SAMSUNG-1003, [0020]-[0024].
`
`
`
`Aizawa (SAMSUNG-1010), Detail of FIG. 10 (annotated)
`
`In Aizawa’s image reconstruction process, an “arbitrarily focused image f,”
`
`purple in FIG. 1, infra, is “reconstructed from the focused images g1 and g2,” i.e.,
`
`from the near and far content images, both in red. SAMSUNG-1010, [0057]. An
`
`arbitrarily focused image is “an image wherein the degree of blur at each depth is
`
`arbitrarily suppressed or intensified.” Id., [0042]. Image reconstruction is per-
`
`formed by filters 10a, 10b that subject the near and far content images g1, g2, re-
`
`spectively, “to prescribed processing.” Id., [0043]. “A synthesizer 11 synthesizes”
`
`the filter outputs to produce the “reconstructed image f.” Id. The filters 10a, 10b
`
`8
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`and the synthesizer 11 are in green. Image reconstruction is carried out on Ai-
`
`zawa’s camera. See claim 17 (“The plural image simultaneous capturing camera
`
`… comprising an arbitrarily focused image synthesizing apparatus”); SAMSUNG-
`
`1003, [0025]-[0027].
`
`
`
`Aizawa (SAMSUNG-1010), Detail of FIG. 1 (annotated)
`
`The degree of blur of the near and far content in the arbitrarily focused re-
`
`constructed image f is determined by filter parameters Ra and Rb, which are “near
`
`content and far content blur radiuses for the desired image.” SAMSUNG-1010,
`
`[0043]. “By adjusting” these parameters, “an arbitrarily focused image can be re-
`
`constructed.” Id. To obtain an image such that “the degree of blur in the far con-
`
`tent region” is varied “while leaving the near content region in focus” (i.e., an im-
`
`age with a focused foreground and a blurred background), “Ra is set to Ra=0 and
`
`Rb is made to vary from 0 to 4.” Id., [0072]; SAMSUNG-1003, [0030]-[0031].
`
`9
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`2. Hyodo describes a digital camera in which a subject is des-
`ignated in an image in a touch screen LCD viewfinder
`Hyodo, in the field of digital cameras, describes an “electronic camera 1”
`
`with an “imaging part 2,” purple in FIG. 3, infra. SAMSUNG-1013, 5:6-14. “An
`
`object image formed on a light receiving surface of the CCD … is photoelectrically
`
`converted and read out as a video signal.” Id., 5:41-44. “The video signal … is
`
`processed … at the signal processing part 30, … and a captured image is shown on
`
`the display part 10.” Id., 5:55-59. The display part 10 is “an LCD,” and “a touch
`
`panel 12 with light permeability is provided over the display part 10.” The LCD
`
`display part 10 and touch panel 12 are shown together in red. SAMSUNG-1013,
`
`5:23-25, see FIG. 2; SAMSUNG-1003, [0038]-[0039].
`
`10
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`
`
`
`Hyodo (SAMSUNG-1013), Detail of FIG. 3 (annotated)
`
`In Hyodo’s camera, “when the cameraman touches the principal subject 40
`
`on the screen (S100), the CPU 38 [green in FIG. 3] gets the positional information
`
`about the touched portion in accordance with an input signal from the touch panel
`
`12 (S102).” SAMSUNG-1013, 6:43-49. A “circle 42 [blue in FIG. 7, infra] of a
`
`predetermined size (a principal subject selection frame) is displayed around the de-
`
`termined touched portion … so as to indicate the touched area.” Id., 6:49-52.
`
`“Then, the principal subject is determined in accordance with the touched portion.”
`
`Id., 6:59-7:21. “After the principal subject is determined …, the principal subject
`
`is focused, and … the designated image and the positional information about the
`
`11
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`principal subject are stored in the image memory.” Id., 7:22-33; SAMSUNG-
`
`1003, [0042]-[0044].
`
`
`
`Hyodo (SAMSUNG-1013), Detail of FIG. 7 (annotated)
`
`Hyodo also describes “an example wherein a plurality of subjects (two sub-
`
`jects) are designated.” SAMSUNG-1013, 10:13-14. A “first principal subject”
`
`(green in FIG. 13, infra) is designated by “a circle (a principal subject selection
`
`frame) 52,” and “the second principal subject” (dotted brown arrow) is designated
`
`by a circle 54. Id., 10:15-21; SAMSUNG-1003, [0043]-[0044].
`
`12
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`
`
`
`Hyodo (SAMSUNG-1013), Detail of FIG. 13 (annotated)
`
`3. Mitsufumi describes a digital camera that generates a com-
`bined image having a desired amount of blur
`Mitsufumi, in the field of digital cameras, describes a digital camera that
`
`generates “an image with a desired amount of bokeh.” SAMSUNG-1011, [0006].
`
`Bokeh is “the visual quality of the out-of-focus areas of a photographic image.”
`
`SAMSUNG-1012, 3; SAMSUNG-1003, [0051].
`
`13
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`A “single image is formed through imaging the background and foreground
`
`separately and then compositing,” which “makes it possible to capture easily an
`
`image with a desire[d] amount of bokeh.” SAMSUNG-1011, [0066]. For in-
`
`stance, when photographing the scene of FIG. 6, infra, “imaging is carried out for
`
`four frames,” each focused on a different element of the scene. Id., [0060]-[0061];
`
`see FIGS. 9A-9D. The four frames are processed and “composited to form a single
`
`image,” which “is stored in the memory 60” of the digital camera and displayed on
`
`an LCD. Id., [0061], [0064], [0128]; SAMSUNG-1003, [0052]-[0053].
`
`Mitsufumi (SAMSUNG-1012), Detail of FIG. 6
`
`
`
`14
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`4.
`In the combination, a subject is designated in a preview im-
`age displayed in a touchscreen LCD interface
`A POSITA would have found it obvious to modify Aizawa’s viewer to in-
`
`clude an LCD for display of preview images. See SAMSUNG-1010, [0116], FIG.
`
`9; SAMSUNG-1013, 5:23-25, FIG. 3. In this implementation, Aizawa’s camera is
`
`modified to include features such as an output memory and a D/A converter, as
`
`taught by Hyodo, that enable real time display of preview images on the LCD of
`
`the viewer. See SAMSUNG-1013, 5:41-59. Aizawa’s processor “determines the
`
`touched portion of the touch panel … in accordance with signals sent from the
`
`touch panel.” SAMSUNG-1013, 5:65-67; SAMSUNG-1003, [0054]-[0064].
`
`In addition, a POSITA would have found it obvious to incorporate a light
`
`permeable touch panel over the LCD of Aizawa’s modified viewer. See SAM-
`
`SUNG-1013, 5:23-25, FIG. 3. A user designates a subject in the image displayed
`
`in the LCD by “touch[ing] the principal subject … on the screen.” SAMSUNG-
`
`1013, 6:46. Responsive to the user’s touch input, Aizawa’s processor receives
`
`“positional information about” the touched portion of the image, in accordance
`
`with the operation of Hyodo’s CPU. See SAMSUNG-1010, [0018], [0116]; SAM-
`
`SUNG-1013, 6:43-49. A “principal subject selection frame,” such as a circle, is
`
`displayed around the touched area in the image in the LCD of the modified viewer,
`
`and the “principal subject” is “determined in accordance with” the touched area.
`
`15
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`SAMSUNG-1013, 6:42-7:21, FIG. 7. The designated “principal subject is fo-
`
`cused” and “the image is designated to be stored” in Aizawa’s memory. SAM-
`
`SUNG-1013, 7:29-32; SAMSUNG-1010, [0116]; SAMSUNG-1003, [0054]-
`
`[0064].
`
`Aizawa’s modified camera combines multiple, differently focused images to
`
`generate a reconstructed, arbitrarily focused image, such as an image having a fo-
`
`cused foreground and a blurred background. See SAMSUNG-1010, [0043],
`
`[0072], claims 15 and 17. Based on the teachings of Mitsufumi, a POSITA would
`
`have found it obvious to store the reconstructed image in the memory of Aizawa’s
`
`camera. See SAMSUNG-1011, [0128]; SAMSUNG-1010, [0018], [0116]; SAM-
`
`SUNG-1003, [0066]-[0067].
`
`5.
`Claim 1
`[1pre] “A method for use in an imaging device, the method comprising:”
`In the combination, Aizawa describes a method for use in an imaging de-
`
`vice. See SAMSUNG-1010, [0036] (“FIG. 9 is a simplified block diagram of a
`
`digital camera”), claim 15 (“A plural image simultaneous capturing camera com-
`
`prising: a camera element; [and] a processor for receiving signals from said camera
`
`element and converting same to image data”), claim 17 (“The plural image simul-
`
`taneous capturing camera according to claim 15, further comprising an arbitrarily
`
`16
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`focused image synthesizing apparatus”), [0043], [0018], [0116] FIG. 9; SAM-
`
`SUNG-1003, [0016]-[0037].
`
`[1.1] “displaying an image in a user interface of the device, wherein the image
`is a preview of a field of view of the device, and wherein the image includes a
`first subject and a second subject;”
`In the combination, Aizawa describes displaying an image in a user inter-
`
`face of the device. “Light that has passed through a lens 30 enters a CCD 31 and
`
`is converted to image data by a processor. An image is displayed through a
`
`viewer 33.” SAMSUNG-1010, [0116]. Aizawa’s viewer, red in FIG. 9, infra, is a
`
`user interface. See FIG. 10, showing “[t]he image displayed through the viewer”
`
`(id., [0116]), [0018] (“A plural image simultaneous capturing camera … com-
`
`prises: a camera element; a processor for receiving signals from the camera ele-
`
`ment and converting them to image data; a display unit for displaying image data
`
`processed by the processor.”); SAMSUNG-1003, [0017]-[0019].
`
`17
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`
`
`
`Aizawa (SAMSUNG-1010), Detail of FIG. 9 (annotated)
`
`In the combination, the image displayed in Aizawa’s viewer is a preview of
`
`a field of view of the device: “Light that has passed through a lens 30 enters a
`
`CCD 31 and is converted to image data … An image is displayed through a viewer
`
`33.” SAMSUNG-1010, [0116]. An example of the preview of the field of view of
`
`the device that is displayed in Aizawa’s user interface is shown in FIG. 10, infra,
`
`which shows “[t]he image displayed through the viewer 33.” Id. The displayed
`
`image is a preview image of a field of view of the device because the displayed im-
`
`age is an image of a scene to be photographed (e.g., rather than a previously photo-
`
`graphed image). SAMSUNG-1003, [0017]-[0019].
`
`18
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`
`
`
`Aizawa (SAMSUNG-1010), Detail of FIG. 10 (annotated)
`
`In the combination, the image displayed in Aizawa’s viewer includes multi-
`
`ple subjects: “a focal point designator for designating a plurality of subjects inside
`
`an image.” SAMSUNG-1010, [0018]; see claim 15. To designate the subjects,
`
`“the user manipulates a focus designator” “[w]hile viewing the image through the
`
`viewer 33.” SAMSUNG-1010, [0116]. SAMSUNG-1003, [0020].
`
`Also in the combination, Hyodo describes a user interface of the device.
`
`Hyodo’s camera includes “[a]n image display part 10” that “is an LCD … and a
`
`19
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`touch panel 12 with light permeability is provided over the display part 10.” SAM-
`
`SUNG-1013, 5:22-25. Hyodo’s image display part (LCD) and touch panel, red in
`
`FIG. 3, infra, together are a user interface. SAMSUNG-1003, [0038]-[0039].
`
`
`
`Hyodo (SAMSUNG-1013), Detail of FIG. 3 (annotated)
`
`Hyodo describes displaying an image in the user interface: “a captured
`
`image is shown on the display part 10.” SAMSUNG-1013, 5:59; see 1:61-62,
`
`2:7-9, 6:4-5, 13:33-34, FIG. 2, FIG. 13. The displayed image is a preview of a
`
`field of view of the device: “The display part 10 can show … images … captured
`
`before the release button is touched.” SAMSUNG-1013, 5:55-63; see 13:31-34
`
`(“an image capturing device for capturing an image of a field … [and] a display
`
`part for showing an image captured by the image capturing device”), 5:40-59,
`
`6:44-7:47, explaining that user interaction with the displayed preview image occurs
`
`20
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`before a picture is taken. FIG. 13 shows an example of the preview of the field of
`
`view of the device displayed in Hyodo’s user interface: “a view illustrating a state
`
`wherein a plurality of principal subjects are designated on the screen.” Id, 4:57-59;
`
`SAMSUNG-1003, [0038]-[0041].
`
`Hyodo (SAMSUNG-1013), Detail of FIG. 13 (annotated)
`
`Hyodo’s image includes a first subject and a second subject. “FIG. 13
`
`shows an example wherein a plurality of subjects (two subjects) are designated.
`
`
`
`21
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`First, the cameraman touches the first principal subject on the screen of the dis-
`
`play part 10 …. Next, the cameraman touches the second principal subject.”
`
`SAMSUNG-1013, 10:13-21. The first subject is Hyodo’s first principal subject
`
`(the person, green arrow), and the second subject is Hyodo’s second principal sub-
`
`ject (the house, dotted brown arrow). SAMSUNG-1003, [0041]-[0042].
`
`A POSITA would have found it obvious to implement Aizawa’s user inter-
`
`face to include an LCD and a light permeable touch panel provided over the LCD,
`
`as shown in the following Figure based on Aizawa’s FIG. 9. See SAMSUNG-
`
`1010, [0116]; SAMSUNG-1013, 5:23-25. The LCD and touch sensitive panel to-
`
`gether are a user interface, and a preview image is displayed on the LCD. In this
`
`implementation, Aizawa’s camera is modified to include features such as an output
`
`memory and a D/A converter, as taught by Hyodo, that enable real time display of
`
`preview images on the LCD of the user interface. See SAMSUNG-1013, 5:41-59.
`
`When Aizawa’s modified camera is used for imaging the scene depicted in Hy-
`
`odo’s FIG. 13, that scene is displayed as a preview image in the LCD, such that the
`
`displayed preview includes a first subject and a second subject, i.e., the first and
`
`second principal subjects (the person and house). See SAMSUNG-1013, 10:13-34.
`
`When a user touches the touch panel to designate a subject (see SAMSUNG-1013,
`
`6:43-52, 10:13-21), Aizawa’s processor “determines the touched portion of the
`
`touch panel,” the subject is determined and focused, and an image is taken and
`
`22
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`stored. SAMSUNG-1013, 5:65-67; see 6:43-52, 10:13-21; SAMSUNG-1003,
`
`[0054]-[0064].
`
`
`
`Aizawa (SAMSUNG-1010), Detail of FIG. 9, modified based on Hyodo
`
`A POSITA would have been motivated to modify Aizawa’s user interface to
`
`include an LCD for display of preview images to improve the ease of use of Ai-
`
`zawa’s camera and to enhance the ergonomics of taking photographs. SAM-
`
`SUNG-1010, [0018], [0116]; SAMSUNG-1013, 5:23-25, 6:43-52, 10:13-21. With
`
`preview images displayed in an LCD, a user does not need to hold the camera to
`
`his eye to look through a finder window to view a preview of the scene being pho-
`
`tographed. Rather, the user views the scene on the LCD, allowing the user to hold
`
`23
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`the camera in a more convenient position, which makes the camera easier and
`
`more comfortable to operate. Furthermore, a POSITA would have been motivated
`
`to integrate a touch panel into Aizawa’s user interface to provide users with an av-
`
`enue for direct interaction with the preview image displayed on the LCD of the
`
`user interface, which is an intuitive and straightforward way for a user to provide
`
`input to the camera. See SAMSUNG-1013, 5:23-25, 6:43-52, 10:13-21. SAM-
`
`SUNG-1003, [0055].
`
`A POSITA would have expected success given that both Aizawa and Hyodo
`
`describe cameras having user interfaces capable of displaying preview images (see
`
`SAMSUNG-1010, [0018], [0116]; SAMSUNG-1013, 5:55-63, 13:31-34, 6:44-
`
`7:47), and that Hyodo confirms that an LCD of a camera can display preview im-
`
`ages and can function in conjunction with a touch panel. See SAMSUNG-1013,
`
`5:23-25. A POSITA also would have expected success given that Aizawa’s disclo-
`
`sure is generic as to the specific implementation of the viewer. SAMSUNG-1003,
`
`[0056].
`
`[1.2] “designating by a processor of the imaging device the first subject in the
`image to be kept blur free;”
`In the combination, Hyodo describes designating the first subject (Hyodo’s
`
`first principal subject) in the image. “[T]he cameraman touches the first principal
`
`subject on the screen of the display part 10, and a circle (principal subject selec-
`
`24
`
`

`

`Attorney Docket No. 39843-0080IP2
`IPR of U.S. Patent No. 10,171,740
`tion frame) 52 indicating the touched portion is displayed over the captured im-
`
`age.” SAMSUNG-1013, 10:14-17; see 6:13-52 (“[W]hen the cameraman touches
`
`the principal subject 40 on the screen (S100), the CPU 38 gets the positional infor-
`
`mation about the touched portion in accordance with an input signal from the touch
`
`panel 12 … [and] a circle 42 of a predetermined size (a principal subject selection
`
`frame) is displayed around the determined touched portion … so as to indicate
`
`the touched area,” where the touched portion “is touched by the cameraman to be
`
`designated as a principal subject.”), 8:3-5, 6:59-60, 7:52-58. Hyodo’s first princi-
`
`pal su