`
`
`Fatih M. Ozluturk
`In re Patent of:
`9,154,699 Attorney Docket No.: 39843-0078IP2
`U.S. Patent No.:
`October 6, 2015
`
`Issue Date:
`Appl. Serial No.: 14/586,297
`
`Filing Date:
`December 30, 2014
`
`Title:
`METHOD AND APPARATUS TO CORRECT BLUR IN ALL OR
`PART OF A DIGITAL IMAGE BY COMBINING PLURALITY
`OF IMAGES
`
`
`Mail Stop Patent Board
`Patent Trial and Appeal Board
`U.S. Patent and Trademark Office
`P.O. Box 1450
`Alexandria, VA 22313-1450
`
`
`PETITION FOR INTER PARTES REVIEW OF UNITED STATES PATENT
`NO. 9,154,699 PURSUANT TO 35 U.S.C. §§ 311–319, 37 C.F.R. § 42
`
`
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`TABLE OF CONTENTS
`
`I.
`
`II.
`
`THE ’699 PATENT ......................................................................................... 1
`A. Overview ................................................................................................... 1
`B. Prosecution History ................................................................................... 1
`
`REQUIREMENTS FOR IPR .......................................................................... 1
`A. Standing .................................................................................................... 1
`B. Challenge and Relief Requested ............................................................... 2
`D. Claim Construction ................................................................................... 3
`
`III. APPLICATION OF PRIOR ART ................................................................... 6
`A. Ground 1: Aizawa, Hyodo, and Mitsufumi render obvious claims 1-30 . 6
`
`IV. PTAB DISCRETION SHOULD NOT PRECLUDE INSTITUTION .......... 65
`
`V.
`
`PAYMENT OF FEES ................................................................................... 69
`
`VI. CONCLUSION .............................................................................................. 69
`
`VII. MANDATORY NOTICES UNDER 37 C.F.R § 42.8(a)(1) ......................... 70
`A. Real Party-In-Interest Under 37 C.F.R. § 42.8(b)(1) .............................. 70
`B. Related Matters Under 37 C.F.R. § 42.8(b)(2) ....................................... 70
`C. Lead And Back-Up Counsel Under 37 C.F.R. § 42.8(b)(3) ................... 70
`D. Service Information ................................................................................ 71
`
`
`
`
`
`
`i
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`
`
`
`EXHIBITS
`
`SAMSUNG-1001 U.S. Patent No. 9,154,699 to Ozluturk (“the ’699 Patent”)
`
`SAMSUNG-1002 Excerpts from the Prosecution History of the ’699 Patent (“the
`Prosecution History”)
`
`SAMSUNG-1003 Declaration of Dr. Irfan Essa
`
`SAMSUNG-1004 Reserved
`
`SAMSUNG-1005 Reserved
`
`SAMSUNG-1006 Reserved
`
`SAMSUNG-1007 Reserved
`
`SAMSUNG-1008 Reserved
`
`SAMSUNG-1009 Unopposed Motion for Extension of Time to Move, Answer,
`or Otherwise Respond to Plaintiff’s Complaint (Clear
`Imaging Research, LLC v. Samsung Electronics Co. Ltd.,
`2:19-cv-326-JRG (E.D. Tex. Oct. 15, 2019))
`
`SAMSUNG-1010 U.S. Patent Publication No. 2001/0013895 to Aizawa
`(“Aizawa”)
`
`SAMSUNG-1011 Certified Translation of Japanese Patent Publication No.
`2003209727 (“Mitsufumi”)
`
`SAMSUNG-1012 Excerpt of The New Oxford American Dictionary (Second
`Edition, 2005)
`
`SAMSUNG-1013 U.S. Patent No. 7,034,881 to Hyodo (“Hyodo”)
`
`SAMSUNG-1014 Reserved
`
`ii
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`SAMSUNG-1015 U.S. Patent No. 6,809,759 to Chiang
`
`SAMSUNG-1016 U.S. Patent No. 7,369,161 to Easwar
`
`SAMSUNG-1017
`
`Joseph Guzman, “Fauci says second wave of coronavirus is
`‘inevitable’”, TheHill.com (Apr. 29, 2020), available at:
`https://thehill.com/changing-america/resilience/natural-
`disasters/495211-fauci-says-second-wave-of-coronavirus-is
`SAMSUNG-1018 Peter Wells, et al. “Texas puts reopening on hold in face of
`new Covid-19 outbreak”, Financial Times (June 25, 2020),
`available at https://www.ft.com/content/e35f3148-a797-4e6e-
`bf7a-1a7ce3181e97
`SAMSUNG-1019 Order Granting Joint Motion to Amend the Docket Control
`Order and Time for Claim Construction Expert Disclosures
`(Clear Imaging Research, LLC v. Samsung Electronics Co.
`Ltd., 2:19-cv-326-JRG (E.D. Tex. June 12, 2020))
`SAMSUNG-1020 Email to Clear Imaging counsel dated February 10, 2020
`SAMSUNG-1021 Email from Clear Imaging counsel dated July 3, 2020
`SAMSUNG-1022 Stipulation by Samsung
`
`SAMSUNG-1023 Reserved
`
`SAMSUNG-1024 P.R. 4-3 Joint Claim Construction and Prehearing Statement
`(Clear Imaging Research, LLC v. Samsung Electronics Co.
`Ltd., 2:19-cv-326-JRG (E.D. Tex. July 23, 2020))
`
`SAMSUNG-1025 Plaintiff’s Disclosure of Asserted Claims and Infringement
`Contentions Pursuant to Patent Local Rules (Clear Imaging
`Research, LLC v. Samsung Electronics Co. Ltd., 2:19-cv-326-
`JRG (E.D. Tex. Jan. 14, 2020))
`
`
`
`iii
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`Samsung petitions for Inter Partes Review (“IPR”) of claims 1-30 (“the
`
`Challenged Claims”) of U.S. Patent No. 9,154,699 (“the ’699 Patent”).
`
`I.
`
`THE ’699 PATENT
`A. Overview
`The ’699 Patent relates to a method and apparatus for use in a digital
`
`imaging device for correcting image blur in digital images by combining a
`
`plurality of captured images to create a combined photographic image such that a
`
`main subject is blur free and other areas are blurred. SAMSUNG-1001, Abstract,
`
`12:48-55; SAMSUNG-1003, [0010].
`
`B.
`Prosecution History
`In the only office action, the claims were rejected on the ground of
`
`nonstatutory double patenting over claims 1-30 of U.S. Patent No. 8,630,484.
`
`SAMSUNG-1002, 77-81. The applicant filed a terminal disclaimer and the
`
`application was allowed. SAMSUNG-1002, 18, 48.
`
`II. REQUIREMENTS FOR IPR
`A.
`Standing
`Samsung certifies that the ’699 Patent is available for IPR. This petition is
`
`being filed within one year of service of a complaint against Samsung in the
`
`Eastern District of Texas. SAMSUNG-1009. Samsung is not barred or estopped
`
`from requesting this review.
`
`1
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`
`B. Challenge and Relief Requested
`Samsung requests cancellation of the Challenged Claims on the following
`
`grounds. Additional explanation and support is set forth in Dr. Irfan Essa’s
`
`declaration (SAMSUNG-1003).
`
`Ground
`1
`
`’699 Patent Claims
`1-30
`
`§103 Basis for Unpatentability
`Aizawa, Hyodo, and Mitsufumi
`
`The following table summarizes the prior art basis for each reference with
`
`respect to an assumed Critical Date of March 25, 2004.1 Each reference qualifies
`
`as prior art:
`
`Reference
`
`Date
`
`Basis
`
`Aizawa
`
`Hyodo
`
`Mitsufumi
`
`
`Published 8/16/2001
`
`§102(a), §102(b), and §102(e)
`
`Filed 10/30/1998
`
`Filed 7/25/2003
`
`§102(e)
`
`§102(a)
`
`C. Level of Ordinary Skill in the Art
`A person of ordinary skill in the art (“POSITA”) as of the Critical Date
`
`would have had at least a Bachelor’s Degree in an academic area emphasizing
`
`
`1 Although the ’699 Patent lists a March 25, 2004 provisional application, Patent
`
`Owner does not contend that the ’699 Patent is entitled to the provisional date.
`
`SAMSUNG-1025.
`
`2
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`electrical engineering, computer science, or a similar discipline, and at least two
`
`years of experience related to imaging technologies. Superior education could
`
`compensate for a deficiency in work experience, and vice-versa. SAMSUNG-
`
`1003, [0006].
`
`D. Claim Construction
`Unless otherwise noted, terms should be given their plain meaning, but
`
`Petitioner reserves the right to respond to any constructions offered by Patent
`
`Owner or the Board. Samsung is not waiving any arguments concerning
`
`indefiniteness or claim scope.
`
`
`“processor … configured to” (claims 9, 24)
`Petitioner advances ordinary meaning of these phrases in this proceeding;
`
`however, a question exists regarding whether they should be interpreted under 35
`
`U.S.C. §112, ¶6 (“112/6”). Williamson v. Citrix Online, LLC, 792 F.3d 1339, 1346
`
`(Fed. Cir. 2015). The absence of the word “means” creates a rebuttable presumption
`
`that 112/6 does not apply. Phillips v. AWH Corp., 415 F.3d 1303, 1311 (Fed. Cir.
`
`2005). In district court, the parties dispute whether the presumption should be
`
`rebutted. Although claim construction arguments are not yet final, Petitioner may
`
`argue in district court that 112/6 applies and these claims are indefinite.
`
`SAMSUNG-1024, 47-49. However, Patent Owner contends that 112/6 doesn’t
`
`apply. Id.
`
`3
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`Because the dispute in district court remains unresolved and because
`
`Petitioner cannot raise indefiniteness here, Petitioner relies on the presumption in
`
`this proceeding and applies prior art to these claims’ ordinary meaning consistent
`
`with Patent Owner’s litigation position. Id. Petitioner will promptly inform the
`
`Board of any district court developments related to these claims’ definiteness.
`
`Further, when determining validity, claim terms need to only be construed to
`
`“resolve the controversy.” Wellman, Inc. v. Eastman Chem. Co., 642 F.3d 1355,
`
`1361 (Fed. Cir. 2011). Because this proceeding lacks arguments/evidence rebutting
`
`the presumption, the presumption should stand and the Board should forego
`
`construction absent Patent Owner advocating for 112/6’s application. If Patent
`
`Owner does not endorse a 112/6 construction, no controversy exists regarding
`
`112/6’s application. Indeed, the presumption holds when neither party presents
`
`argument/evidence to rebut it. HTC America, Inc. v. Virginia Innovation Sciences,
`
`Inc., IPR2017-00872, Paper 11, 9; Dick v. New York Life Ins. Co., 359 U.S. 437, 443
`
`n.3 (1959); Am. Hoist & Derrick Co. v. Sowa & Sons, Inc., 725 F.2d 1350, 1358
`
`(Fed. Cir. 1984).
`
`Finally, any district court claim construction ruling is reviewable by the
`
`Federal Circuit, and yet, denial of institution is non-appealable. Thus, for this claim
`
`construction theory, congressional intent is best served through institution,
`
`particularly considering that institution would involve prior art consideration against
`
`4
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`
`Patent Owner’s claim construction.
`
`Therefore, in this proceeding, these claim phrases should be given their
`
`ordinary meaning.
`
`
`“designating … a main subject” (claims 1, 9, 17, and 24)
`Petitioner has asserted in the co-pending district court proceeding that
`
`limitations of the form “designating … a main subject” should be construed to
`
`mean “identifying an object to use as a reference point for aligning images to
`
`correct blur.” SAMSUNG-1024, 42-43. The specification of the ’699 Patent
`
`supports this construction. See id.; SAMSUNG-1001, 10:52-11:2 (“the reference
`
`point for aligning the higher speed images is … the [designated] subject itself.”).
`
`The present Petition addresses this claim language according to this
`
`construction, as well as according to Patent Owner’s proposed interpretation that
`
`“No construction necessary.” SAMSUNG-1024, 42-43. Previous cases have made
`
`clear that PTAB rules do not prohibit Petitioner from addressing the prior art under
`
`alternative claim constructions. See, e.g., 10X Genomics v. Bio-Rad Labs,
`
`IPR2020-00086, Paper 8, 18-22 (PTAB April 27, 2020); Western Digital Corp. v.
`
`SPEX Techs., Inc., IPR2018-00084, Paper 14, 12 (PTAB Apr. 25, 2018). In fact,
`
`the Board has previously endorsed this approach, stating that “judicial efficiency
`
`will be enhanced by allowing Petitioner to rely upon a claim construction that
`
`5
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`Patent Owner is relying upon in the related district court litigation to assert
`
`infringement of the challenged patent.” 10X Genomics, 19.
`
`III. APPLICATION OF PRIOR ART
`As detailed below, this petition shows a reasonable likelihood that Samsung
`
`will prevail with respect to claims 1-30 of the ’699 Patent.
`
`A. Ground 1: Aizawa, Hyodo, and Mitsufumi render obvious claims
`1-30
`
`
`Aizawa describes generating a digital image with desired
`focus by combining multiple, differently focused images of a
`scene2
`Aizawa, in the field of digital cameras, describes “reconstructing, from a
`
`plurality of images that are focused differently, an arbitrarily focused image that is
`
`an image wherein the degree of blur at any depth is suppressed or intensified.”
`
`SAMSUNG-1010, Abstract; SAMSUNG-1003, [0016]-[0035].
`
`Aizawa describes an apparatus for capturing images and combining captured
`
`images into a reconstructed, arbitrarily focused image. See SAMSUNG-1010,
`
`Abstract, [0018], [0043], [1006], claims 15 and 17. The apparatus includes “a
`
`CCD 31,” purple in FIG. 9, infra, and a processor 32, green, that converts signals
`
`from the CCD “to image data.” SAMSUNG-1010, [0116]. An image of a scene to
`
`
`2 Petitioner incorporates the entirety of the discussions of Aizawa, Mitsufumi, and
`
`the combination thereof, presented in Sections III.A.1-III.A.3, into Ground 1.
`
`6
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`be photographed is “displayed through a viewer 33 [red].” Id. Captured image
`
`data are “stored in a memory 35,” in brown. Id.; SAMSUNG-1003, [0017]-[0018].
`
`
`
`
`
`Aizawa (SAMSUNG-1010), Detail of FIG. 9 (annotated)
`
`Aizawa’s apparatus captures multiple images of a scene, e.g., the scene
`
`depicted in FIG. 10, infra. The user provides input by “manipulat[ing] a focus
`
`designator 34,” blue in FIG. 9, to “designate[] at least two regions that are to be
`
`brought into focus.” SAMSUNG-1010, [0116]. Images are focused based on the
`
`user’s input. In the example of FIG. 10, “a near content image g1” is focused on
`
`“the region (2,2) in the middle of the image occupied by the subject T,” green, and
`
`“a far content image g2” is focused on “the region (1,1) at the upper left,” blue. Id.
`
`“The focus adjustment mechanism 36 brings a designated region into focus and
`
`7
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`takes a picture,” “[t]hen … brings the next designated region into focus, [and]
`
`takes a picture.” Id.; SAMSUNG-1003, [0020]-[0024].
`
`
`
`
`
`Aizawa (SAMSUNG-1010), Detail of FIG. 10 (annotated)
`
`In Aizawa’s image reconstruction process, an “arbitrarily focused image f,”
`
`purple in FIG. 1, infra, is “reconstructed from the focused images g1 and g2,” i.e.,
`
`from the near and far content images, both in red. SAMSUNG-1010, [0057]. An
`
`arbitrarily focused image is “an image wherein the degree of blur at each depth is
`
`arbitrarily suppressed or intensified.” Id., [0042]. Image reconstruction is
`
`performed by filters 10a, 10b that subject the near and far content images g1, g2,
`
`respectively, “to prescribed processing.” Id., [0043]. “A synthesizer 11
`
`synthesizes” the filter outputs to produce the “reconstructed image f.” Id. The
`
`filters 10a, 10b and the synthesizer 11 are in green. Image reconstruction is carried
`
`8
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`out on Aizawa’s camera. See claim 17 (“The plural image simultaneous capturing
`
`camera … comprising an arbitrarily focused image synthesizing apparatus”);
`
`SAMSUNG-1003, [0025]-[0027].
`
`
`
`
`
`Aizawa (SAMSUNG-1010), Detail of FIG. 1 (annotated)
`
`The degree of blur of the near and far content in the arbitrarily focused
`
`reconstructed image f is determined by filter parameters Ra and Rb, which are
`
`“near content and far content blur radiuses for the desired image.” SAMSUNG-
`
`1010, [0043]. “By adjusting” these parameters, “an arbitrarily focused image can
`
`be reconstructed.” Id. To obtain an image such that “the degree of blur in the far
`
`content region” is varied “while leaving the near content region in focus” (i.e., an
`
`image with a focused foreground and a blurred background), “Ra is set to Ra=0
`
`and Rb is made to vary from 0 to 4.” Id., [0072]; SAMSUNG-1003, [0029]-
`
`[0030].
`
`9
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
` Hyodo describes a digital camera in which a subject is
`designated in an image in a touch screen LCD viewfinder
`Hyodo, in the field of digital cameras, describes an “electronic camera 1”
`
`with an “imaging part 2,” purple in FIG. 3, infra. SAMSUNG-1013, 5:6-14. “An
`
`object image formed on a light receiving surface of the CCD … is photoelectrically
`
`converted and read out as a video signal.” Id., 5:41-44. “The video signal … is
`
`processed … at the signal processing part 30, … and a captured image is shown on
`
`the display part 10.” Id., 5:55-59. The display part 10 is “an LCD,” and “a touch
`
`panel 12 with light permeability is provided over the display part 10.” The LCD
`
`display part 10 and touch panel 12 are shown together in red. SAMSUNG-1013,
`
`5:23-25, see FIG. 2; SAMSUNG-1003, [0037]-[0038].
`
`10
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`
`
`
`
`
`Hyodo (SAMSUNG-1013), Detail of FIG. 3 (annotated)
`
`In Hyodo’s camera, “when the cameraman touches the principal subject 40
`
`on the screen (S100), the CPU 38 [green in FIG. 3] gets the positional information
`
`about the touched portion in accordance with an input signal from the touch panel
`
`12 (S102).” SAMSUNG-1013, 6:43-49. A “circle 42 [blue in FIG. 7, infra] of a
`
`predetermined size (a principal subject selection frame) is displayed around the
`
`determined touched portion … so as to indicate the touched area.” Id., 6:49-52.
`
`“Then, the principal subject is determined in accordance with the touched portion.”
`
`Id., 6:59-7:21. “After the principal subject is determined …, the principal subject
`
`is focused, and … the designated image and the positional information about the
`
`11
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`principal subject are stored in the image memory.” Id., 7:22-33; SAMSUNG-
`
`1003, [0039]-[0041].
`
`
`
`Hyodo (SAMSUNG-1013), Detail of FIG. 7 (annotated)
`
` Mitsufumi describes a digital camera that generates a
`combined image
`Mitsufumi, in the field of digital cameras, describes a digital camera that
`
`generates “an image with a desired amount of bokeh.” SAMSUNG-1011, [0006].
`
`Bokeh is “the visual quality of the out-of-focus areas of a photographic image.”
`
`SAMSUNG-1012, 3; SAMSUNG-1003, [0047].
`
`12
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`A “single image is formed through imaging the background and foreground
`
`separately and then compositing,” which “makes it possible to capture easily an
`
`image with a desire[d] amount of bokeh.” SAMSUNG-1011, [0066]. For
`
`instance, when photographing the scene of FIG. 6, infra, “imaging is carried out
`
`for four frames,” each focused on a different element of the scene. Id., [0060]-
`
`[0061]; see FIGS. 9A-9D. The four frames are processed and “composited to form
`
`a single image,” which “is stored in the memory 60” of the digital camera and
`
`displayed on an LCD. Id., [0061], [0064], [0128]; SAMSUNG-1003, [0048].
`
`
`
`
`
`Mitsufumi (SAMSUNG-1011), Detail of FIG. 6
`
`
`In the combination, a subject is designated in a preview
`image displayed in a touch screen LCD interface
`A POSITA would have found it obvious to modify Aizawa’s viewer to
`
`include an LCD for display of preview images. See SAMSUNG-1010, [0116],
`
`13
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`FIG. 9; SAMSUNG-1013, 5:23-25, FIG. 3. In this implementation, Aizawa’s
`
`camera is modified to include features such as an output memory and a D/A
`
`converter, as taught by Hyodo, that enable real time display of preview images on
`
`the LCD of the viewer. See SAMSUNG-1013, 5:41-59. Aizawa’s processor
`
`“determines the touched portion of the touch panel … in accordance with signals
`
`sent from the touch panel.” SAMSUNG-1013, 5:65-67; SAMSUNG-1003,
`
`[0049]-[0059].
`
`In addition, a POSITA would have found it obvious to incorporate a light
`
`permeable touch panel over the LCD of Aizawa’s modified viewer. See
`
`SAMSUNG-1013, 5:23-25, FIG. 3. A user designates a subject in the image
`
`displayed in the LCD by “touch[ing] the principal subject … on the screen.”
`
`SAMSUNG-1013, 6:46. Responsive to the user’s touch input, Aizawa’s processor
`
`receives “positional information about” the touched portion of the image, in
`
`accordance with the operation of Hyodo’s CPU. See SAMSUNG-1010, [0018],
`
`[0116]; SAMSUNG-1013, 6:43-49. A “principal subject selection frame,” such as
`
`a circle, is displayed around the touched area in the image in the LCD of the
`
`modified viewer, and the “principal subject” is “determined in accordance with”
`
`the touched area. SAMSUNG-1013, 6:42-7:21, FIG. 7. The designated “principal
`
`subject is focused” and “the image is designated to be stored” in Aizawa’s
`
`14
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`memory. SAMSUNG-1013, 7:29-32; SAMSUNG-1010, [0116]; SAMSUNG-
`
`1003, [0049]-[0059].
`
`Aizawa’s modified camera combines multiple, differently focused images to
`
`generate a reconstructed, arbitrarily focused image, such as an image having a
`
`focused foreground and a blurred background. See SAMSUNG-1010, [0043],
`
`[0072], claims 15 and 17. Based on the teachings of Mitsufumi, a POSITA would
`
`have found it obvious to store the reconstructed image in the memory of Aizawa’s
`
`camera. See SAMSUNG-1011, [0128]; SAMSUNG-1010, [0018], [0116];
`
`SAMSUNG-1003, [0061].
`
`
`Claim 1
`[1pre] “A method comprising:”
`In the combination, Aizawa describes a method. See SAMSUNG-1010,
`
`[0042] (“an apparatus and method are described for reconstructing, from a plurality
`
`of images, … an arbitrarily focused image that is an image wherein the degree of
`
`blur at each depth is arbitrarily suppressed or intensified.”), [0043] (“a method for
`
`reconstructing a desired arbitrarily focused image”); SAMSUNG-1003, [0016]-
`
`[0036].
`
`[1.1] “displaying an image in a viewfinder;”
`In the combination, Aizawa describes displaying an image in a viewfinder.
`
`“Light that has passed through a lens 30 enters a CCD 31 and is converted to
`
`image data by a processor. An image is displayed through a viewer 33 which the
`
`15
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`user can see.” SAMSUNG-1010, [0116]. See FIG. 10, showing “[t]he image
`
`displayed through the viewer.” Id., [0116]; see [0018] (“A plural image
`
`simultaneous capturing camera relating to the present invention comprises: a
`
`camera element; a processor for receiving signals from the camera element and
`
`converting them to image data; a display unit for displaying image data processed
`
`by the processor.”). Aizawa’s viewer, red in FIG. 9, infra, is a viewfinder
`
`because it displays a preview image of a scene to be photographed with Aizawa’s
`
`camera. See SAMSUNG-1016, 12:13-29; SAMSUNG-1015, 3:52-59;
`
`SAMSUNG-1003, [0017]-[0019].
`
`Aizawa (SAMSUNG-1010), Detail of FIG. 9 (annotated)
`
`
`
`16
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`Also in the combination, Hyodo describes a viewfinder. Hyodo’s camera
`
`includes “[a]n image display part 10” that “is an LCD for example, and a touch
`
`panel 12 with light permeability is provided over the display part 10.”
`
`SAMSUNG-1013, 5:22-25. Hyodo’s image display part (e.g., an LCD) and touch
`
`panel, red in FIGS. 2 and 3, infra, together are a viewfinder. SAMSUNG-1003,
`
`[0037]-[0038].
`
`Hyodo (SAMSUNG-1013), Detail of FIG. 2 (annotated)
`
`
`
`17
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`
`
`
`Hyodo (SAMSUNG-1013), Detail of FIG. 3 (annotated)
`
`Hyodo describes displaying an image in the viewfinder, i.e., in the LCD of
`
`the viewfinder: “a captured image is shown on the display part 10.” SAMSUNG-
`
`1013, 5:59; see 5:60-63 (“The display part 10 can show … images … captured
`
`before the release button is touched.”), 13:31-34 (“an image capturing device for
`
`capturing an image of a field … [and] a display part for showing an image
`
`captured by the image capturing device”), 1:61-62, 2:7-9, 5:40-59, 6:4-5, 13:33-
`
`34, FIG. 2, FIG. 13, 6:44-7:47, explaining that user interaction with the displayed
`
`preview image occurs before a picture is taken. An example of the image
`
`displayed in the LCD of Hyodo’s viewfinder is shown in FIG. 7, infra in [1.2],
`
`18
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`which shows “a view illustrating a state wherein a principal subject is designated.”
`
`Id, 4:44-45; SAMSUNG-1003, [0037]-[0039].
`
`A POSITA would have found it obvious to implement Aizawa’s viewfinder
`
`(viewer 33) to include an LCD and a light permeable touch panel provided over the
`
`LCD, as taught by Hyodo, as shown in the following Figure based on Aizawa’s
`
`FIG. 9. See SAMSUNG-1010, [0116]; SAMSUNG-1013, 5:23-25. The LCD and
`
`touch sensitive panel together are a viewfinder, and an image is displayed on the
`
`LCD. In this implementation, Aizawa’s camera is modified to include features
`
`such as an output memory and a D/A converter, as taught by Hyodo, that enable
`
`real time display of preview images on the LCD of the viewfinder. See
`
`SAMSUNG-1013, 5:41-59. When Aizawa’s modified camera is used for imaging
`
`the scene depicted in Hyodo’s FIG. 7, that scene is displayed as an image in the
`
`viewfinder of the modified camera. “[W]hen the cameraman touches the principal
`
`subject 40 on the screen” of the viewfinder, the processor of Aizawa’s modified
`
`camera “get[s] the positional information about the touched portion” of the touch
`
`panel “in accordance with an input signal from the touch panel 12,” the principal
`
`subject is determined and focused, and an image is taken and stored. SAMSUNG-
`
`1013, 6:43-52; see 5:65-67, 10:13-21; SAMSUNG-1003, [0049].
`
`19
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`
`
`
`Aizawa (SAMSUNG-1010), Detail of FIG. 9, modified based on Hyodo
`
`A POSITA would have been motivated to modify Aizawa’s viewfinder to
`
`include an LCD to improve the ease of use of Aizawa’s camera and to enhance the
`
`ergonomics of taking photographs. SAMSUNG-1010, [0018], [0116];
`
`SAMSUNG-1013, 5:23-25, 6:43-52, 10:13-21. With an LCD-based viewfinder, a
`
`user does not need to hold the camera to his eye to look through a finder window to
`
`view the scene being photographed. Rather, the user views the scene displayed on
`
`the LCD viewfinder, allowing the user to hold the camera in a more convenient
`
`position. This modification makes the camera easier and more comfortable to
`
`operate. SAMSUNG-1003, [0050].
`
`20
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`Furthermore, a POSITA would have been motivated to integrate a touch
`
`panel into Aizawa’s modified viewfinder to provide users with an avenue for direct
`
`interaction with the image displayed on the LCD of the viewfinder, enabling an
`
`intuitive and straightforward way for a user to provide input to the camera. See
`
`SAMSUNG-1013, 5:23-25, 6:43-52, 10:13-21; see [1.2], infra. A POSITA would
`
`have expected success given that both Aizawa and Hyodo describe digital cameras
`
`capable of displaying preview images (see SAMSUNG-1010, [0018], [0116];
`
`SAMSUNG-1013, 5:55-63, 13:31-34, 6:44-7:47), and that Hyodo confirms that an
`
`LCD of a digital camera can display preview images and can function in
`
`conjunction with a touch panels. See SAMSUNG-1013, 5:23-25. A POSITA also
`
`would have expected success given that Aizawa’s disclosure is generic as to the
`
`specific implementation of the viewer. SAMSUNG-1003, [0050]-[0051].
`
`[1.2] “receiving user input, by the viewfinder, designating a main subject in
`the image;”
`In the combination, Aizawa describes receiving user input designating a
`
`main subject in the image. “While viewing the image through the viewer 33, the
`
`user manipulates a focus designator 34 and designates at least two regions that
`
`are to be brought into focus. … Upon receiving a signal from the focus
`
`designator 34, the processor 32 drives a focus adjustment mechanism.”
`
`SAMSUNG-1010, [0116]; see [0018] (“A plural image simultaneous capturing
`
`camera comprising … a focal point designator for designating a plurality of
`
`21
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`subjects inside an image.”); claim 15. The designated subjects include a near
`
`content subject (e.g., a foreground subject) and a far content subject (e.g., a
`
`background subject), which are then photographed as “a near content image g1” in
`
`which the near content subject is focused, and “a far content image g2” in which
`
`the background is in focus. Id., [0116]. The user’s manipulation of the focus
`
`designator to designate a particular one of the “plurality of subjects inside an
`
`image” or a particular one of the “at least two regions that are to be brought into
`
`focus” is user input designating a main subject in the image. Id., [0018],
`
`[0116]. Aizawa’s focus designator is shown in blue in FIG. 9, infra, with the user
`
`input depicted in dotted blue lines. SAMSUNG-1003, [0021].
`
`Aizawa (SAMSUNG-1010), Detail of FIG. 9 (annotated)
`
`
`
`22
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`Also in the combination, Hyodo describes receiving user input, by the
`
`viewfinder, i.e., by the LCD 10 and touch panel 12, designating a main subject
`
`in the image. See SAMSUNG-1013, 6:13-52 (“[W]hen the cameraman touches
`
`the principal subject 40 on the screen (S100), the CPU 38 gets the positional
`
`information about the touched portion in accordance with an input signal from the
`
`touch panel 12,” where the touched portion “is touched by the cameraman to be
`
`designated as a principal subject.”), 4:44-45 (“FIG. 7 shows “a state wherein a
`
`principal subject is designated”), 10:14-17 (“First, the cameraman touches the
`
`first principal subject on the screen of the display part 10, and a circle (principal
`
`subject selection frame) 52 indicating the touched portion is displayed over the
`
`captured image.”), 7:52-58, 8:3-5. The user touching the principal subject (in
`
`green in FIG. 7, infra) in the image displayed on the LCD of the viewfinder (in
`
`red) is user input (shown in blue), received by the viewfinder, designating the
`
`principal subject as a main subject in the image. SAMSUNG-1003, [0040]-[0041].
`
`23
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`
`
`
`Hyodo (SAMSUNG-1013), Detail of FIG. 7 (annotated)
`
`A POSITA would have found it obvious to receive user input, by Aizawa’s
`
`modified viewfinder, designating a main subject in the image displayed in the LCD
`
`of the viewfinder. See SAMSUNG-1010, [0018], [0116]; SAMSUNG-1013, 5:23-
`
`25, 6:13-52, 7:52-58, 8:3-5, 10:14-17. In this modified camera, a user touches the
`
`touch panel to designate a main subject in the image. See SAMSUNG-1013, 6:43-
`
`7:21. For instance, when Aizawa’s modified camera is used for imaging the scene
`
`depicted in Hyodo’s FIG. 7, the touch panel of the viewfinder of the modified
`
`camera receives user input designating the person (indicated with a green arrow in
`
`24
`
`
`
`Attorney Docket No. 39843-0078IP2
`IPR of U.S. Patent No. 9,154,699
`FIG. 7, supra) as the main subject in the image. See id. Aizawa’s processor 32
`
`receives positional information about the touched portion and indicates the touched
`
`portion by displaying an indicator, e.g., a circle, around the touched portion. See
`
`SAMSUNG-1013, 6:43-53. The designated main subject is focused, and an image
`
`is taken and stored. See SAMSUNG-1013, 7:29-31; SAMSUNG-1003, [0052].
`
`A POSITA would have been motivated to have Aizawa’s modified
`
`viewfinder receive user input (i.e., a touch on the touch panel) designating a main
`
`subject to implement a flexible approach to designating a subject of interest. For
`
`instance, a POSITA would have recognized that providing the flexibility for a user
`
`to designate a main subject by touching the touch screen, e.g., anywhere in the
`
`camera’s field of view, facilitates generation of a combined image that is focused
`
`on a subject of interest to a user. See SAMSUNG-1005, [0061]. Moreover, a
`
`POSITA would have been motivated