Trials@uspto.gov Paper 33
`571-272-7822 Date: December 31, 2025
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`APPLE INC.,
`Petitioner,
`v.
`SMITH INTERFACE TECHNOLOGIES, LLC,
`Patent Owner.
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`Before JAMESON LEE, JEFFREY S. SMITH, and
`MIRIAM L. QUINN, Administrative Patent Judges.
`
`LEE, Administrative Patent Judge.
`JUDGMENT
`Final Written Decision
`Determining No Challenged Claim s Unpatentable
`35 U.S.C. § 318(a)
`
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`2
`I. INTRODUCTION
`We instituted an inter partes review of claims 1–4, 6–10, 17, 19, 22–
`26, 28–30, 32, 33, 36, 41, 48, 58–62, 64, 65, 67, 69, 70, 73, 77, 82, and 83
`(“challenged claims”) of U.S. Patent No. 10, 649,580 B1 (Ex. 1001, “the
`’580 patent”) owned by Smith Technologies, LLC (“Patent Owner”). Paper
`8 (“Decision to Institute” or “Inst. Dec.”). We have authority to conduct this
`inter partes review under 35 U.S.C. § 6. This Final Written Decision is
`issued pursuant to 35 U.S.C. § 318(a) and 37 C.F.R. § 42.73.
`Oral hearing was held on November 14, a transcript of which has been
`entered as Paper 31 (“Tr.”). We determine that Apple Inc. (“Petitioner”) has
`not proved by a preponderance of the evidence that any of claims 1–4, 6–10,
`17, 19, 22–26, 28–30, 32, 33, 36, 41, 48, 58–62, 64, 65, 67, 69, 70, 73, 77,
`82, and 83 of the ’580 patent is unpatentable .
`A. Background
`Petitioner filed a Petition requesting an inter partes review of the
`challenged claims pursuant to 35 U.S.C. §§ 311–319. Paper 2 (“Pet.”).
`Patent Owner filed a Preliminary Response. Paper 6 (“Prelim. Resp.”).
`The Decision to Institute was entered on January 27, 2025. Paper 8.
`Patent Owner filed a Response to the Petition ( Paper 16, “PO Resp.”).
`Petitioner filed a Reply to the Patent Owner’s Response (Paper 22, “Reply”).
`Patent Owner filed a Sur-reply (Paper 26, “Sur-reply”).
`B. Real Parties in Interest
`Petitioner identifies itself as the real party in interest. Pet. 105. Patent
`Owner also identifies itself as the real party in interest. Paper 3, 1.
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`3
`C. Related Matters
`Each party identifies as related matter: (1)Smith Interface
`Technologies, LLC v. Apple Inc., Case No. 3-23-cv-01187 (S. D. Cal.), filed
`June 27, 2023 (pending); and (2) Smith Interface Technologies, LLC v.
`Samsung Electronics Co., Ltd. et al., 2 -22-cv-00290 (E. D. Tex.), filed
`July 29, 2022 (terminated). Pet. 105; Paper 3, 1.
`D. The ’580 Patent
`The ’580 patent issued on May 12, 2020, from Application No.
`16/664,777, filed October 25, 2019. Ex. 1001, codes (21), (22), (45). It
`claims priority through a chain of continuation applications to three
`provisional applications the earliest filed of which is Provisional Application
`No. 61/515,835, filed August 5, 2011. Id. at codes (60), (63).
`The ’580 patent is titled “Devices, Methods, and Graphical Use
`Interfaces for Manipulating User Interface Objects with Visual and/or Haptic
`Feedback.” Ex. 1001, code (54). The patent describes it generally relates
`“to touch screens.” Id. at 1:36–37. Figure 22, shown below, illustrates “a
`3D layered user interface . . . in which displayed objects may exist in one of
`a plurality of layers.” Id. at 67:33, 41–44.
`
`
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`4
`Figure 22 illustrates user interface 2200, including front layer 2202,
`back layer 2204, and objects 2206 and 2208. Ex. 1001, 67:45–50. Layers
`2202 and 2204 “are depicted in a staggered arrangement” to illustrate depth.
`Id. at 67:46–47. “Object 2206 exists in the front layer, while object 2208
`exists in the back layer.” Id. at 67:49–51.
`The patent indicates that “virtual display layers may be given the
`appearance of depth through the use of 3D depth cues,” such as “an effect,
`manipulation, transformation, animation, or operation which gives a visual
`indication of simulated depth.” Id. at 68:25–30. Various techniques
`disclosed for a 3D depth cue include “a blur operation” with “layers located
`at successively greater depths may appear blurrier than those closer to the
`user”; “a shadow element” making elements “to appear to float above th e
`next virtual display layer”; “a depth -based fog” making distant objects
`appear “‘foggier’” than objects closer to the user; and “a depth -based
`apparent rate of movement,” such that “the layers closer to the user may
`appear to move faster than those more distant.” Id. at 68:25–58.
`Of all challenged claims, claims 1, 22, and 58 are independent.
`Claim 1 is illustrative and reproduced below: 1
`[1pre] An apparatus, comprising:
`[1a] at least one non -transitory memory;
`[1b] a touch screen;
`[1c] a camera; and
`[1d] one or more processors in communication with the at least
`one non-transitory memory, the touch screen, and the
`camera, wherein the one or more processors execute
`
`1 The bracketed labels correspond to those used by Petitioner to reference
`the claim elements. See Pet. vii–vii. We use the same labels here for ease of
`reference, understanding, and consistency.
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`5
`instructions stored in the non-transitory memory to cause
`the apparatus to:
`[1e] display, via the touch screen, a first virtual display layer
`including contents ;
`[1f] display, via the touch screen, at least one user interface
`element ;
`[1g] detect, via the touch screen, at least a portion of touch on
`the least one user interface element;
`[1h] when an aspect of the touch is detected to surpass a
`threshold, display, via the touch screen, a plurality of
`markings in a second virtual display layer that appears to
`have a lesser depth than the first virtual display layer,
`[1i] where at least a portion of the second virtual display layer
`is at least partially translucent so that at least a portion of
`the contents of the first virtual display layer is visible
`through the at least portion of the second virtual display
`layer;
`[1j] detect, via the touch screen, a touch movement on at least
`one of the plurality of markings; and
`[1k] during detection of the touch movement on the at least one
`of the plurality of markings in the second virtual display
`layer that appears to have the lesser depth than the first
`virtual display layer, display, via the touch screen, a
`movement of the markings in the second virtual display
`layer as a function of the touch movement; and
`[1.l] perform a zoom operation on the at least portion of the
`contents of the first virtual display layer without
`performing the zoom operation on the plurality of
`markings in the second virtual display layer, where the
`zoom operation correlates with the movement of the
`markings in the second virtual display layer , and the at
`least portion of the second virtual display layer is at least
`partially translucent so that a result of the zoom operation
`on the at least portion of the contents of the first virtual
`display layer is visible through the at least portion of the
`second virtual display layer.
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`6
`Ex. 1001, 102:12–54.
`Claim 22 is a method claim including various steps largely
`corresponding the the functionalities recited in claim 1. Ex. 1001,
`107:15–52. Claim 58 is directed to a non -transitory computer readable
`storage medium storing one or more programs comprising instructions
`which, when executed, cause the performance of functionalities largely the
`same as the functionalities recited in claim 1 and the steps recited in claim
`22. Id. at 110:45–111:15.
`E. References and Declarations
`Petitioner relies on the following references:
`Name Patent Document/Publication Exhibit
`Ording2 US 2008/0094356 A1 1006
`Hayward3 US 2012/0176401 A1 1016
`Ramos, et al., “Zliding: Fluid Zooming and Sliding for High Precision
`Parameter Manipulation,” UIST ’05: Proceedings of the 18th Annual ACM
`Symposium on User Interface Software and Aechnology, Oct. 23, 2005, pp.
`143–152 (Ex. 1004, “Ramos Paper”).
`“Zooming & Sliding,” (available at youtube.com/watch?v=EcE3XBytN-U),
`December 28, 2007 (Ex. 1005, “Ramos Video”).
`Petitioner relies on the Declaration and Supplemental Declaration of
`Andrew Cockburn, Ph.D. (Exs. 1003, 1032) regarding technical issues, and
`the Declarations of Kathryn Talbot (Ex. 1007), June Ann Munford
`(Ex. 1009), and Gonzalo Ramos (Ex. 1022) regarding publication dates of
`
`2 Published April 24, 2008. Ex. 1006, code (43).
`3 Published July 12, 2012, from Application 13/052,994, filed March 21,
`2011. Ex. 1016, codes (21), (22), (43).
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`7
`the Ramos Paper and Ramos Video. Patent Owner relies on the Declaration
`of Mark P. Mahon , Ph.D. (Ex. 2012).
`F. Asserted Ground of Unpatentability
`Petitioner asserts that the following grounds of unpatentability:
`Claims Challenged 35 U.S.C. § 4 Reference(s)/Basis
`1–4, 6–10, 17, 19, 22–
`26, 28–30, 32, 33, 36,
`41, 58–62, 64, 65, 67,
`69, 70, 73, 77, 82, 83
`103(a) Ramos Paper, Ramos Video,
`Ording
`48 103(a) Ramos Paper, Ramos Video,
`Ording, Hayward
`Pet. 4, 94.
`II. ANALYSIS
`A. Burden of Proof
`Petitioner has the burden of proving unpatentability by a
`preponderance of the evidence. 35 U.S.C. § 316(e). That burden never
`shifts to Patent Owner except in limited circumstances not present here. In
`re Magnum Oil Tools Int’l, Ltd. , 829 F.3d 1364, 1375 (Fed. Cir. 2016).
`“‘Preponderance of the evidence’ means ‘the greater weight of evidence,
`evidence which is, more convincing than the evidence which is offered in
`opposition to it.’” United States v. C.H. Robinson Co., 760 F.3d 1376, 1383
`(Fed. Cir. 2014) (internal quotations omitted).
`
`4 The Leahy -Smith America Invents Act (“AIA”), Pub. L. No. 112-29,
`125 Stat. 284, (2011), amended 35 U.S.C. §§ 102 and 103 effective March
`16, 2013. Because the ’580 patent claims priority to an application filed
`before the AIA’s amendments to §§ 102 and 103, we apply the pre-AIA
`versions of §§ 102 and 103.
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`8
`B. Level of Ordinary Skill in the Art
`Petitioner relies on declaration testimony of Dr. Cockburn, who
`asserts that a person of ordinary skill in the art (hereinafter “POSITA”)
`“would have had at least a bachelor’s degree in computer science, computer
`engineering, electrical engineering, human-computer interaction, or a related
`field, and would have had at least two years of relevant work experience in
`the design and development of graphical user interfaces (GUIs), human -
`computer interfaces, or equivalents thereof.” Ex. 1003 ¶ 26.
`Patent Owner asserts that Dr. Cockburn’s definition of the level of
`ordinary skill in the art “is overbroad to the extent it includes work
`experience in the design and development of human-computer interfaces.”
`PO Resp. 4. Instead, Patent Owner proposes the following: “[A] POSITA
`would have had at least a bachelor’s degree in computer science, computer
`engineering, electrical engineering, or a related field, and would have had at
`least two years of relevant work experience in the design and development
`of graphical user interfaces (GUIs) or equivalents thereof.” PO Resp. 4.
` The difference between the parties’ positions is solely that Petitioner
`includes “human-computer interaction” as a relevant degree and “human
`computer interfaces” as relevant working experience. We need not resolve
`that difference, because no matter which stated level of ordinary skill is
`applied, all of our findings and conclusions would be the same and because
`neither party asserts the other party’s expert is not qualified as an expert .
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`9
`In the Decision on Institution, we adopted Petitioner’s stated level of
`ordinary skill in the art. Inst. Dec. 9. We continue to adopt Petitioner’s
`stated level of ordinary skill in the art. 5
`C. Claim Construction
`We use the same claim construction standard that would be used to
`construe a claim in a civil action under 35 U.S.C. § 282(b), including
`construing the claim in accordance with the ordinary and customary
`meaning of such claim as understood by one of ordinary skill in the art and
`the prosecution history pertaining to the patent. 37 C.F.R. § 42.100(b)
`(2022). The claim construction standard set forth in Phillips v. AWH Corp.,
`415 F.3d 1303 (Fed. Cir. 2005) (en banc) is applicable.
`Claim terms are generally given their ordinary and customary
`meaning as would be understood by one with ordinary skill in the art in the
`context of the specification, the prosecution history, other claims, and
`extrinsic evidence including expert and inventor testimony, dictionaries, and
`learned treatises, although extrinsic evidence is less significant than the
`intrinsic record. Phillips, 415 F.3d at 1312–17. Usually, the specification is
`dispositive, and it is the single best guide to the meaning of a disputed term.
`Id. at 1315.
`The specification may reveal a special definition given to a claim term
`by the patentee, or the specification or prosecution history may reveal an
`intentional disclaimer or disavowal of claim scope by the inventor. Id.
`at 1316. If an inventor acts as his or her own lexicographer, the definition
`must be set forth in the specification with reasonable clarity, deliberateness,
`
`5 Our findings and conclusions would be the same, however, had we applied
`Patent Owner’s articulation of the level of ordinary skill in the art.
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`10
`and precision. Renishaw PLC v. Marposs Societa’ per Azioni, 158 F.3d
`1243, 1249 (Fed. Cir. 1998). The disavowal, if any, can be effectuated by
`language in the specification or the prosecution history. Poly-Am., L.P. v.
`API Indus., Inc. , 839 F.3d 1131, 1136 (Fed. Cir. 2016).
`Only those claim terms that are in controversy need to be construed,
`and only to the extent necessary to resolve the controversy. Realtime Data,
`LLC v. Iancu, 912 F.3d 1368, 1375 (Fed. Cir. 2019) (“The Board is required
`to construe ‘only those terms . . . that are in controversy, and only to the
`extent necessary to resolve the controversy.’” (quoting Vivid Techs., Inc. v.
`Am. Sci. & Eng’g, Inc. , 200 F.3d 795, 803 (Fed. Cir. 1999))).
`Petitioner asserts that “no claim terms need be construed to resolve
`issues of controversy in the present Petition.” Pet. 2. Patent Owner agrees
`that “no formal constructions of any claim terms are necessary.” PO Resp.
`4. We agree with the parties that no express construction of any claim term
`is necessary.
`D. Alleged Obviousness of Claims 1–4, 6–10, 17,
`19, 22–26, 28–30, 32, 33, 36, 41, 58–62, 64,
`65, 67, 69, 70, 73, 77, 82, and 83 over
`Ramos Paper, Ramos Video, and Ording
`1. The Law on Obviousness
`A claim is unpatentable under 35 U.S.C. § 103(a) “if the differences
`between the subject matter to be patented and the prior art are such that the
`subject matter as a whole would have been obvious at the time the invention
`was made to a person having ordinary skill in the art to which said subject
`matter pertains .” 35 U.S.C. § 103(a); see KSR Int’l Co. v. Teleflex Inc. ,
`550 U.S. 398, 406 (2007). The question of obviousness is resolved on the
`basis of underlying factual determinations, including (1) the scope and
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`11
`content of the prior art; (2) any differences between the claimed subject
`matter and the prior art; (3) the level of skill in the art; and (4) where in
`evidence, so-called secondary considerations, including commercial success,
`long-felt but unsolved needs, failure of others, and unexpected results.
`Graham v. John Deere Co. of Kansas City, 383 U.S. 1, 17–18 (1966).
`2. The Ramos Paper (Ex. 1004)
`The Ramos Paper describes an “interface widget” for zooming via
`pressure input and providing a high accuracy zoom and select task.
`Ex. 1004, 143. 6 The Ramos Paper provides examples of user interface
`interactions for which such a widget would be useful. Id. Examples include
`“identifying and then picking a single pixel from a high resolution image,
`seeking a particular frame in a long video stream, or adjusting a continuous
`image color parameter.” Id.
`The Ramos Paper uses the term “zliding” for the notion of “zoom
`sliding” that uses both pressure and dragging movement of the input device
`to perform parameter manipulation. Ex. 1004, 143. For example, a user
`may use a pressure-sensitive stylus as an input device to interact with the
`“Zlider widget” described in the paper. Id. at 145. “Users use the pressure
`modality to fluidly and explicitly zoom or adjust the granularity of the
`parameter space, while sliding or dragging the input device to perform high
`precision parameter manipulation within that zoomed parameter space.” Id.
`at 143 (emphasis in original).
`Figure 1 of the Ramos Paper shows the Zlider widget and is
`reproduced below, with caption (Ex. 1004, 143):
`
`6 Citations are to page numbers in the original document at the left and right
`margins on the bottom of the pages.
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`12
`
`As shown in Figure 1, the Zlider widget includes a rectangular
`working area, Vernier, and pressure cursor. In use, the Zlider widget
`consists of a rectangular working area that the user “can scrub” to adjust a
`parameter from low to high values. Ex. 1004, 145. “To operate the Zlider
`the user taps and drags its pointer across the working area until the desired
`value is reached or effect is achieved.” Id. “At all times a red needle
`indicates the position of the value being adjusted relative to the possible
`minimum and maximum values at the extremes of the widget.” Id.
`The Zlider widget shown in Figure 1 includes a wedge-shaped
`pressure cursor that “provides users with a real-time indicator of the pressure
`they are applying.” Ex. 1004, 145. “The wedge’s area fills as the pressure
`increases, until completely filled when the pressure reaches the maximum
`
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`13
`level the device can sense.” Id. The Zlider widget “uses the release of the
`stylus from the interaction surface as an indication of selection.” Id. at 146.
` The Ramos Paper describes experimental evaluation of the Zlider
`widget. Ex. 1004, 143, 147–150. Figure 6, reproduced below, illustrates the
`experimental setup used:
`
`As shown, Ramos’s Figure 6 depicts user interface elements in an
`experimental setup of the Zlider widget used for a “serial target acquisition
`and selection task.” Ex. 1004, 148. As depicted in Figure 6, a Zlider widget
`having a rectangular working area, visible Vernier, and pressure cursor is
`adjacent to and to the right of a viewport displayed on an “interactive LCD
`graphics display tablet.” Id. A working area or workspace, thin selection
`line, and green target area are shown within the viewport. Id. In the
`experiment, a participant uses the Zlider widget to scroll through the
`workspace under the viewport to “locate and select a target in a workspace
`area” when “the target is inside the viewport, visible, and covering the
`selection line.” Id.
`
`
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`14
`3. The Ramos Video (Ex. 1005)
`The Ramos Video is an approximately four-minute video that
`illustrates the operation of the Zlider widget described in the Ramos Paper.
`See Pet. 13. The Ramos Video shows several examples of content being
`displayed on a touch screen with the Zlider widget (e.g., working area)
`displayed adjacent to the content. See Pet. 13. One example screen shot of
`the video included in the Petition is reproduced below. Pet. 14.
`
`As depicted above, an example of video content showing a cartoon monkey
`in a tree dangling a stick above a turtle (labeled as “Contents” by Petitioner)
`is displayed on a touch screen. See Pet. 14. The Zlider widget (including the
`“Working area” labeled by Petitioner) is shown adjacent to the bottom edge
`of the example content. The vernier and wedge-shaped pressure cursor are
`also shown in the Zlider widget.
`
`4. Ording (Ex. 1006)
`Ording is a published patent application that relates “to portable
`devices that adaptively determine a cursor position from a finger contact
`with a touch screen display and then perform operations according to the
`
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`15
`cursor position.” Ex. 1006 ¶ 3; see id. at code (54). Ording’s portable
`multifunction device has “a touch-sensitive display (also known as a ‘touch
`screen’) with a graphical user interface (GUI), one or more processors,
`memory and one or more modules, programs or sets of instructions stored in
`the memory for performing multiple functions,” including “digital videoing”
`and “digital video playing.” Id. ¶ 8. Figure 2, reproduced below, shows “a
`portable multifunction device having a touch screen.” Id. ¶ 16.
`
`Figure 2 of Ording shows a portable multifunction device 100 having
`touch screen 112 that displays graphics within user interface 200. Ex. 1006
`¶ 96. A user may select one or more graphics by touching the graphics with
`one or more fingers 202. Id. Device 100 also includes “home” or menu
`button 204, push button 206 for powering the device on or off, volume
`
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`16
`adjustment button 208, head set jack 212, and charging external port 124.
`Id. ¶¶ 97–98.
`Ording indicates that the device may include “a physical or virtual
`click wheel” used as an input control device to “navigate among and interact
`with one or more graphical objects (henceforth referred to as icons)
`displayed in the touch screen.” Id. ¶ 45. “The click wheel may also be used
`to select one or more of the displayed icons.” Id. “For a virtual click wheel,
`the click wheel may be either an opaque or semi transparent object that
`appears and disappears on the touch screen display in response to u ser
`interaction with the device.” Id.
`5. Dispositive Issue
`One issue is dispositive for this alleged ground of obviousness.
`Independent claims 1, 22, and 58 each include the following features:
`(1) displaying of a first virtual display layer including contents and a second
`virtual display layer that appears to have lesser depth than the first virtual
`display layer; (2) when a touch on a user interface element is detected to
`surpass a threshold, display a plurality of markings in the second virtual
`display layer; and (3) at least a portion of the second virtual display layer is
`at least partially translucent so that at least a portion of the contents of the
`first virtual layer is visible through the at least portion of the second virtual
`display layer. See, for example, limitations [1e], [1g], [1h], and [1i].
`Below, we discuss and analyze Petitioner’s application and proposed
`combination of the Ramos Paper, the Ramos Video, and Ording to meet
`these requirements, including consideration of Patent Owner’s arguments
`with respect thereto and Petitioner’s rebuttal. Thereafter, we conclude that
`Petitioner has not proved by a preponderance of the evidence that any of
`claims 1–4, 6–10, 17, 19, 22–26, 28–30, 32, 33, 36, 41, 58–62, 64, 65, 67,
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`17
`69, 70, 73, 77, 82, and 83 is unpatentable over the Ramos Paper, the Ramos
`Video, and Ording.
`6. Petitioner’s Application and Proposed Combination
`of the Ramos Paper, Ramos Video, and Ording
`Petitioner asserts:
`The Ramos Video demonstrates the use of the Zlider
`widget, showing how it can be used to zoom and pan underlying
`image content. See, generally, APPLE-1005. It further shows
`how at least a portion of the Zlider widget (e.g., pressure cursor,
`extended markings of the Vernier) is displayed in a semi -
`transparent manner such that content behind the Zlider remains
`visible through at least a portion of the Zlider. APPLE-1005,
`0:13-0:26, 3:08-3:31. The combined disclosures of the Ramos
`Paper and the Ramos Video would have informed a POSITA
`about the operation and appearance of the Zlider, including how
`its use modifies the appearance of the underlying image content
`the Zlider was being used to adjust. APPLE-1003, ¶57; see,
`generally, APPLE-1004; APPLE-1005.
`Thus, a POSITA would have considered the Ramos Paper
`and the Ramos Video together to fully understand functionality
`of the Zlider widget. APPLE-1003, ¶58. The combined
`disclosures of the Ramos Paper and the Ramos Video thus
`combine to inform various details of the Zlider widget, including
`its physical appearance (including a vernier), as shown in the
`Ramos Paper below, and its applications (e.g., allowing “users to
`use the pressure modality to fluidly and explicitly zoom or adjust
`the granularity of the parameter space, while sliding or dragging
`the input device to perform high precision parameter
`manipulation within that zoomed parameter space”), as well as
`its use in practice, including the changes in appearance of the
`underlying image content and the Zlider itself, as shown in the
`Ramos Video below. Apple-1004, 1, FIG. 1; APPLE-1005,
`3:16-3:19 (illustrating the Zlider’s zooming action and the
`change in appearance of the Vernier in response to the zooming
`and sliding action); APPLE-1003, ¶58.
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`18
`
`
`
`
`
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`19
` The combination of the Ramos Paper and the Ramos
`Video would beneficially inform provision of a widget that
`allows a user to adjust the level of zoom by modifying the
`pressure applied to a touch sensitive display, as the Ramos Paper
`teaches, while further displaying a change in appearance of
`markings, e.g., of a Vernier, in response to the zoom scale factor,
`as both Ramos Paper and Ramos Video teach, where at least a
`portion of the widget is at least semi-transparent to enable
`viewing of the underlying content, as Ramos Video shows.
`APPLE-1003, ¶59; APPLE-1004, 1, 3, FIG. 1; APPLE-1005,
`0:16-0:21 (showing a change in the appearance of the Vernier
`markings), 3:16-3:19 (showing a change in the appearance of the
`Vernier markings, the zoom effect, and at least a portion Zlider
`widget as semi-transparent), 3:28 (showing at least a portion
`Zlider widget as semitransparent). The resulting widget would
`offer several advantages. First, it would allow for “high
`precision manipulation” in “touch-sensitive platforms,” for
`applications not just limited to zooming and panning. APPL E-
`1004, 2-3; APPLE-1005, 0:07-0:12; APPLE-1003, ¶60. Second,
`the widget of the combined disclosures would offer a UI widget
`that is at least partially transparent such that content below is
`visible, resulting in a widget that is minimally intrusive and
`reduces the amount of the underlying content that is blocked or
`not visible. APPLE-1005, 3:16-3:19, 3:28; APPLE-1003, ¶60.
`Third, the widget would offer a “real-time visual indicator” of
`the pressure applied to the touchscreen (and thus the amount of
`zoom) and of the “current scale factor,” as shown by the
`changing “grid spacing” of the Vernier. APPLE-1004, 3;
`APPLE-1005, 0:16-:021, 0:28-0:41, 3:16-3:19; APPLE-1003,
`¶60.
`Pet. 26–30 (emphasis added).
`We find no problem with Petitioner’s combining the teachings of the
`Ramos Paper and the Ramos Video, except for Petitioner’s multiple
`references to “underlying image content” beneath the Zlider widget and the
`reference to “content below” the Zlider widget. Petitioner points to nothing
`in either the Ramos Paper or the Ramos Video showing that there is
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`20
`“underlying image content,” either beneath or underlying the Zlider widget.
`None of the above illustrations reproduced by Petitioner from the Ramos
`Paper and the Ramos Video shows “underlying image content” relative to
`the Zlider widget. Rather, the illustrations from the Ramos Video clearly
`show the contrary, that the Zlider widget and the image data are displayed
`side-by-side. We have also viewed the identified portions of the Ramos
`Video, i.e., Ex. 1005, 3:28, 3:16-3:19, which purportedly show at least a
`portion of the Zlider widget as semi-transparent thus allowing viewing of
`image content beneath . But we see no such translucency over image
`content. Thus, at least this aspect of petitioner’s understanding and
`application of the Ramos Paper and the Ramos Video is incorrect and
`unpersuasive.
`Petitioner similarly asserts:
`The Ramos Video further shows that the Vernier is
`displayed over the image, so a POSITA would have understood
`the Vernier is in a second virtual display layer that appears to
`have lesser depth that the first virtual display layer in which the
`image content is located. APPLE-1003, ¶89; APPLE-1005,
`0:14, 3:10-3:11. This is further evidenced, for example, by the
`appearance of extended elements of the Vernier that appear when
`the scale is greater than one, as shown below. APPLE-1005,
`3:28.
`Pet. 54. We disagree, and refer to the Ramos Video at 0:14 and 3:10-3:11,
`as identified by Petitioner, reproduced below, none of which shows the
`vernier, which Petitioner identifies as the plurality of markings within the
`Zlider widget, as being displayed over the underlying image content :
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`21
`
`[The illustration is a screen shot of the Zlider in the Ramos Video]
`
`[The illustration is another screen shot of the Zlider in the Ramos Video]
`
`[The illustration is a further screen shot of the Zlider in the Ramos Video]
`
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`22
`These illustrations clearly do not show the vernier, i.e., markings within the
`Zlider widget, which according to the claims should be in a second virtual
`display layer at a lesser depth than the first virtual display layer including
`content, to be “over” underlying image content as Petitioner asserts.
` The following screen shot from the Ramos Video, identified by
`Petitioner (Pet. 54–55), does show that sometimes, at least temporarily until
`markings are reset, the vernier has some lines exten ding slightly into area
`reserved for image content :
`
`[The illustration is a screen shot from the Ramos Video
`with Petitioner’s annotations identifying certain marking
`lines which extend slightly into area intended for image]
` But we disagree that that is sufficient to show either the vernier or the
`Zlider widget is in a second virtual display layer at a lesser depth that the
`first virtual display layer including content , i.e., over or above image
`content. The intrusion of the markings is slight and temporary, and does not
`change that the vernier and the Zlider widget are side-by-side with image
`content. The Zlider widget and vernier are not in a virtual display layer at a
`lesser depth than, i.e., above, the first virtual display layer including content.
`Furthermore, Petitioner has mapped the Zlider widget, which includes
`the cursor and the vernier markings, to be on the second virtual display
`
`
`
`
`
`
`
`
`IPR2024-01085
`Patent 10,649,580 B1
`
`23
`layer. According to the ’580 patent, a “visual display layer refers to a
`collection of display elements which have all been assigned the same
`apparent depth within the 3D layered user interface.” Ex. 1001, 68:6−9. But
`the cursor is not assigned the same depth as the other objects of the Zlider
`widget—it hovers above the vernier and over the content depending on the
`user movement of it on the screen. See illustrations above. The same goes
`for the vernier and its markings. Though they are part of the Zlider widget,
`the vernier and its markings appear as an object that is at a different depth
`than the working area, which is also part of the Zlider widget. These
`apparent different depth within that alleged “second virtual display layer” do
`not comport with the description of the ’580 patent that the “same apparent
`depth within the 3D layered user interface.” Ex. 1001, 68:6–9; see
`Tr. 30:19−31:6, 65:8−19. Petitioner alleges that the ’580 patent allows for
`the same virtual layer to have multiple levels. Tr. 63:17−64:3. But such a
`contention has not been developed sufficient ly and the Petition does not
`present such a contention. Pet. 51 (plurality of markings are displayed on a
`second virtual layer that appears to have a lesser depth than the first virtual
`layer); 54 (“a POSITA would have understood the Vernier is in a second
`virtual layer that appears to have lesser depth that the first virtual display
`layer in which the image content is located”). Additional d