Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 1 of 82 PageID 191
`
`
`
`Exhibit 5
`Direct Infringement Claim Chart – ‘771 Patent
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 2 of 82 PageID 192
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`Claim Language
`Claim 15. A computer
`system comprising:
`
`
`
`
`Where in the Accused Product(s) Each Limitation of the Asserted
`Claim(s) are Found – HoloLens 1, HoloLens 2, IVAS
`Defendant Microsoft’s accused systems (e.g., embodied in the HoloLens
`1, HoloLens 2, Integrated Visual Augmentation System (IVAS), and
`related software applications) include a computer system. Reasonable
`discovery will confirm this interpretation.
`
`The evidence below is described with regard to the HoloLens 1, which
`was introduced by Microsoft on March 30, 2016. While there are
`technical improvements and additional features associated with HoloLens
`2 (introduced February 24, 2019) and IVAS (a military variant of the
`HoloLens 2), each of the accused systems function similarly with
`reference to the ‘771 patent claim limitations as described below. Where
`needed, distinctions are made, with additional evidence and features
`described for HoloLens 2 and IVAS.
`
`In general, Microsoft has obtained patents covering its products and
`services related to the accused systems. On information and belief,
`Microsoft incorporates its patented features in the accused systems, and it
`is reasonable for D3D to rely on those Microsoft patents as evidence of
`how the HoloLens 1, HoloLens 2 and IVAS systems operate. Such
`citations to Microsoft patents are noted throughout the claim charts.
`
`The evidence below demonstrates that the accused systems operate as a
`computer system.
`
`For example, Microsoft has obtained US Patent No. 10,136,124, titled
`“Stereoscopic Display of Object.” The Abstract describes the product as
`“A computing device generates a stereoscopic display of an object by
`coordinating a first image and a second image.” The Abstract further
`describes how “the computing device may adjust at least one display
`property of the first image and/or the second image depending on one or
`more factors.” Similarly, the Summary of the Invention describes the
`product as “A computing device causes a stereoscopic display of an
`object by coordinating a display of a first image and a second image, one
`to each eye.” Claim 9 states, “A computing device, comprising: a
`processor; a memory having computer - executable instructions
`stored thereupon which, when executed by a computing device, cause
`the computing device to display a first image of an object on a display
`device, wherein the first image is aligned with a first eye of a user;
`display a second image of the object on the display device, wherein the
`second image is aligned with a second eye of the user; obtain data
`
`1 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 3 of 82 PageID 193
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`indicating a vergence distance associated with the object; adjust a display
`property of the first image based, at least in part, on the vergence
`distance, wherein adjusting the display property of the first image
`obscures the display of the object to the first eye of the user for the
`purposes of reducing eyestrain.” (emphasis added). Microsoft’s ‘124
`patent identifies April 27, 2015 as its priority date.
`
`Microsoft, and other third parties, have published information concerning
`the accused systems, which further confirm that the accused systems
`operate as a computer system.
`
`
`
`
`“As a standalone computer it doesn't need to be connected to a
`desktop or laptop computer, it is genuinely standalone.”
`
`Source: https://www.forbes.com/sites/ewanspence/2017/01/14/microsoft-hololens-review-experience-
`review/#dfd83c0eabc0 (emphasis added)
`
`Source: https://docs.microsoft.com/en-us/hololens/hololens1-
`hardware#:~:text=Microsoft%20HoloLens%20(1st%20gen)%20is,the%20real%20world%20around%20you.
`(emphasis added).
`
`For example, a Forbes January 14, 2017 article titled “Microsoft
`HoloLens Review: Winning the Reality Wars,” states:
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`2 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 4 of 82 PageID 194
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`“Opening up the HoloLens is a startling reminder that this is a self-
`contained holographic computer.”
`
`
`Further, a Verge April 6, 2016 article titled “Inside Microsoft’s
`HoloLens,” showed:
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The HoloLens 1 includes multiple, pre-installed software programs to
`provide the stereoscopic three-dimensional images.
`
`
`
`Image of components inside of the Microsoft display unit.
`
`Source: https://www.theverge.com/2016/4/6/11376442/microsoft-hololens-holograms-parts-
`teardown-photos-hands-on (emphasis added)
`
`Source: https://docs.microsoft.com/en-us/hololens/hololens1-hardware
`
`
`
`
`3 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 5 of 82 PageID 195
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`HoloLens 2
`
`The HoloLens 2 is a display unit (DU), which includes a computer. The
`below image shows partial specifications of the HoloLens 2.
`
`
`Source: https://www.microsoft.com/en-us/hololens/hardware
`
`Further, Microsoft describes the HoloLens 2 as “an untethered
`holographic computer.”
`
`
`Source: https://docs.microsoft.com/en-us/hololens/hololens2-hardware
`
`
`
`4 
`
`
`
`
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 6 of 82 PageID 196
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`The HoloLens 2 includes multiple, pre-installed software programs to
`provide three-dimensional images.
`
`
`
`
`Source: https://docs.microsoft.com/en-us/hololens/hololens2-hardware
`
`IVAS
`
`Further, the IVAS is a display unit (DU) with an on-body computer. The
`chart below shows some of the requirements of the IVAS computer
`system.
`
`
`5 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 7 of 82 PageID 197
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`
`
`Source: https://uploadvr.com/wp-content/uploads/2018/11/Army-AR-Objectives.pdf (emphasis added)
`
`For example, an excerpt from the IVAS Program Objective states:
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`"IVAS is comprised of the Heads Up Display 3.0 (HUD 3.0)
`integrated with the Synthetic Training Environment (STE) Squad
`capability. This platform will provide increased lethality,
`mobility, and situational awareness with the following features:
`Head Borne Vision System (no helmet mount assembly; low
`profile, conformal goggle/visor system providing 24/7 squad
`situational awareness in all operating environments; see thru,
`wide field of view binocular 3D display. . ."
`
`Source: https://uploadvr.com/wp-content/uploads/2018/11/Army-AR-Objectives.pdf 
`
`6 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 8 of 82 PageID 198
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`a memory;
`
`On information and belief, HoloLens 2 is the foundation for IVAS, and as
`such HoloLens 2 capabilities and functionalities are, and should be
`considered, representative of IVAS capabilities and functionalities –
`especially at the pre-discovery stage.
`
`The accused systems embody a memory. For example, the HoloLens 1,
`HoloLens 2, and IVAS include a memory. As stated above, the
`HoloLens 2 is representative of the IVAS – as the HoloLens 2 is the
`foundation of the IVAS. Reasonable discovery will confirm this
`interpretation.
`
`For example, the HoloLens 1 includes the following memory:
`
`
`
`Source: https://docs.microsoft.com/en-us/hololens/hololens1-
`hardware#:~:text=Microsoft%20HoloLens%20(1st%20gen)%20is,the%20real%20world%20around%20you.
`
`The HoloLens has 114 MB of Dedicated Video Memory, 980 MB of
`Shared System Memory, 2 GB RAM, 64 GB of storage.
`
`
`7 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 9 of 82 PageID 199
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`
`Source: https://www.windowscentral.com/microsoft-hololens-processor-storage-and-ram
`
`HoloLens 2/IVAS
`
`The HoloLens 2 is a display unit, which includes a computer and
`associated memory. The below image shows partial specifications of the
`HoloLens 2, including 4GB of RAM and 64GB of storage memory. As
`stated above, the HoloLens 2 is representative of the IVAS – as the
`HoloLens 2 is the foundation of the IVAS. On information and belief,
`Microsoft’s IVAS technology will include similar componentry,
`including memory, to the HoloLens 2.
`
`
`8 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 10 of 82 PageID 200
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`a processor;
`
`Source: https://www.microsoft.com/en-us/hololens/hardware
`
`The accused systems include a processor. For example, the HoloLens 1,
`HoloLens 2, and IVAS include one or more processors. The HoloLens 1
`includes:
`
`
`
`
`
`Source: https://docs.microsoft.com/en-us/hololens/hololens1-
`hardware#:~:text=Microsoft%20HoloLens%20(1st%20gen)%20is,the%20real%20world%20around%20you.
`
`The HoloLens has an Intel Atom x 4-Z8100 1.04 GHz Intel Airmont
`(14nm) 4 Logical Processors 64-bit capable as well as a HoloLens
`Graphics GPU/HPU.
`
`
`9 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 11 of 82 PageID 201
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`
`Source: https://www.windowscentral.com/microsoft-hololens-processor-storage-and-ram
`
`HoloLens 2
`
`For example, HoloLens 2 includes a custom Qualcomm Snapdragon 850
`processor.
`<https://pureinfotech.com/microsoft-hololens-2-tech-specs/>.
`
`
`Source: https://www.microsoft.com/en-us/hololens/hardware
`
`
`
`10 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 12 of 82 PageID 202
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`IVAS
`
`“The vendor shall use a System Integration Testbed (SIT) for
`
`integration testing of software and hardware subsystems, as well as,
`
`for evaluation of overall system-level performance. The SIT will
`
`accept actual IVAS hardware and software. The SIT shall have the
`
`capability to test hardware and software through use of simulation, as
`
`well as, breadboard equipment integration.
`
`The SIT shall include the following: All hardware and software to
`
`exercise (by stimulation or simulation) any and all HUD 3.0 sensors,
`
`processors, interfaces, networks, power sources, Soldier controls, and
`
`displays on simulated missions; test equipment and data analysis
`
`capabilities to monitor and evaluate performance and interface to the
`
`HUD 3.0 system; and a software development station allowing
`
`development, evaluation and debugging software changes. All test
`
`equipment used shall meet calibration system requirements per
`
`Section 6.”
`
`
`
`Source: https://uploadvr.com/wp-content/uploads/2018/11/Army-AR-Objectives.pdf
`
`
`The accused systems include a communications interface. For example,
`the HoloLens 1, HoloLens 2, and IVAS include communications
`interfaces, which provides numerous capabilities for communication
`between different functionalities of the device. The HoloLens 1 provides
`numerous capabilities:
`
`
`
`
`
`
`
`
`“The current HoloLens headset leans on a combination of hand
`gestures to support user interaction.”
`
`Source: https://www.windowscentral.com/eye-tracking-could-be-cards-hololens-2
`
`a communications
`interface;
`
`11 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 13 of 82 PageID 203
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`Source: https://docs.microsoft.com/en-us/hololens/hololens1-hardware
`
`
`
`
`
`
`Source: https://docs.microsoft.com/en-us/hololens/hololens1-clicker
`
`Additional information regarding the HoloLens communications interface
`can be found, by way of example, at https://docs.microsoft.com/en-
`us/hololens/hololens1-basic-usage.
`
`HoloLens 2
`
`The HoloLens 2 includes a communications interface, including
`enhanced hand recognition/gesturing capabilities
`(https://docs.microsoft.com/en-us/dynamics365/mixed-
`reality/guides/authoring-gestures-hl2) and eye-tracking capabilities
`(https://docs.microsoft.com/en-us/windows/mixed-reality/eye-gaze-
`interaction).
`
`
`
`
`
`
`
`
`
`
`
`12 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 14 of 82 PageID 204
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`
`
`“The objective of systems engineering under this project is to
`ensure the vendor successfully produces an IVAS that wirelessly
`communicates with the FWS-I, FWS-CS, STORM, and provided
`RTA and AR capability.”
`
`Source: https://uploadvr.com/wp-content/uploads/2018/11/Army-AR-Objectives.pdf 
`
`Source: https://docs.microsoft.com/en-us/hololens/hololens2-hardware
`
`
`IVAS
`
`On information and belief, Microsoft’s IVAS technology will have a
`similar communications interface to the HoloLens 2, but more advanced
`for military operation purposes.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The accused systems include an interconnection mechanism coupling the
`memory, the processor, and the communications interface. A person of
`ordinary skill in the art would understand that an interconnection
`mechanism is required in order to send a set of images to the head-
`13 
`
`“Intra-Soldier Wireless (ISW) Connectivity (multi-point wireless)
`for on body Soldier devices.”
`
`Source: https://uploadvr.com/wp-content/uploads/2018/11/Army-AR-Objectives.pdf
`
`an interconnection
`mechanism coupling
`the memory, the
`processor and the
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 15 of 82 PageID 205
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`communications
`interface; and
`
`mounted display generated by interactions of the memory, processor, and
`a communications interface.
`
`For example, the Microsoft HoloLens 1 Display Unit is shown below and
`the physical apparatus interconnects key elements including the memory,
`processor, and communications interface.
`
`
`
`Source: https://haptic.al/hat-is-hololens-mixed-reality-c01198c5bbb?gi=afe21b101d2a
`
`
`
`
`Source: https://docs.microsoft.com/en-us/hololens/hololens1-hardware
`
`
`
`
`
`
`14 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 16 of 82 PageID 206
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`HoloLens 2
`For example, the Microsoft HoloLens 2 Display Unit is shown below and
`the physical apparatus interconnects key elements including the memory,
`processor, and communications interface.
`
`
`
`Source:
`https://haptic.al/hat-is-hololens-mixed-reality-c01198c5bbb?gi=afe21b101d2a
`
`
`Source: https://docs.microsoft.com/en-us/hololens/hololens2-hardware
`
`
`
`
`
`
`15 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 17 of 82 PageID 207
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`“HUD 3.0 Interface/Interoperability with other equipment (e.g.
`FWS-I, FWS-CS, STORM, Test Maintenance and Diagnostic
`Equipment (TMDE), Conformal Wearable Battery, Tactical
`Radios, GPS systems, the Advanced Combat Helmet, the Enhanced
`Combat Helmet, the Integrated Head Protection System, and the
`Modular Scalable Vest.)”
`
`Source: https://uploadvr.com/wp-content/uploads/2018/11/Army-AR-Objectives.pdf 
`
`IVAS
`
`On information and belief, Microsoft’s IVAS technology will have a
`similar interconnection mechanism to the HoloLens 2, but may be more
`advanced for military operation purposes.
`
`
`
`
`
`
`
`
`
`
`
`On information and belief, the accused systems include a system wherein
`the memory is encoded with an application providing three-dimensional
`viewing of images by a user, that when performed on the processor,
`provides a process for processing information, the process causing the
`computer system to perform the operations of the succeeding steps.
`
`For example, the HoloLens 1 includes various pre-installed software
`(encoded memory) for three-dimensional viewing of images.
`
`
`wherein the memory is
`encoded with an
`application providing
`three-dimensional
`viewing of images by a
`user, that when
`performed on the
`processor, provides a
`process for processing
`information, the
`process causing the
`computer system to
`perform the operations
`of:
`
`
`Source: https://docs.microsoft.com/en-us/hololens/hololens1-hardware (emphasis added around “Holograms”
`software)
`
`16 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 18 of 82 PageID 208
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`
`
` 
`
`
`“The fastest path to building a mixed reality app in Unity is with
`the Mixed Reality Toolkit.”
`
`Source: https://docs.microsoft.com/en-us/windows/mixed-reality/unity-development-
`overview?tabs=mrtk,hl2 (emphasis added)
`
`“HoloLens works its magic by suspending holograms, or what
`appears to be a three-dimensional object, virtually in front of you.”
`
`Source: https://www.pcworld.com/article/3341801/microsoft-hololens-what-made-it-great-and-what-a-
`new-hololens-needs.html (emphasis added)
`
`For example, in an article from PC World dated February 22, 2019:
`
`
`
`
`
`
`
`For example, the Microsoft Development webpage:
`
`
`
`
`
`
`
`For example, on Microsoft’s “Configure a New Unity project for
`Windows Mixed Reality” webpage:
`
`
`
`
`
`
`
`
`
`Microsoft provides a number of sample applications that provide three-
`dimensional viewing of images by a user, that when performed on the
`processor, provides a process for processing information, the process
`causing the computer system to perform the succeeding operations.
`
`
`
`
`
`
`
`
`
`
`
`
`“Windows Mixed Reality (WMR) is a Microsoft platform
`introduced as part of the Windows 10 operating system. The WMR
`platform lets you build applications that render digital content on
`holographic and VR display devices.”
`
`Source: https://docs.microsoft.com/en-us/windows/mixed-reality/configure-unity-project (emphasis
`added) 
`
`17 
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 19 of 82 PageID 209
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`For example, Microsoft’s HoloTerrain
`
`
`
`
`Source: https://www.microsoft.com/en-us/p/holoterrain/9mwkd575dqq3?activetab=pivot:overviewtab
`
`HoloLens 2/IVAS
`
`In addition to the above evidence, the HoloLens 2 includes multiple, pre-
`installed software programs to provide three-dimensional images. On
`information and belief, the HoloLens 2 is representative of IVAS.
`
`
`Source: https://docs.microsoft.com/en-us/hololens/hololens2-hardware
`
`
`
`
`
`
`
`18 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 20 of 82 PageID 210
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`For example, Microsoft’s HoloTerrain
`
`
`
`selecting a volume of
`interest from a
`collection of image
`slices;
`
`“[S]ee intricate details on 3D images more easily and comfortably
`with industry-leading resolution”
`
`Source: https://www.microsoft.com/en-us/hololens/hardware (emphasis added)
`
`
`Source: https://www.microsoft.com/en-us/p/holoterrain/9mwkd575dqq3?activetab=pivot:overviewtab
`
`For example, the Microsoft HoloLens 2 webpage:
`
`
`
`
`
`
`On information and belief, the accused systems include a system for
`selecting a volume of interest from a collection of image slices. For
`example, the HoloLens 1, HoloLens 2, and IVAS embody a system for
`selecting a volume of interest from a collection of image slices.
`
`For example, in the medical field a person of ordinary skill in the art
`would know that a volume of interest can be selected from a collection of
`image slices – e.g., CT scans and MRI scans acquire images of the human
`body in the form of slices. See, e.g., Douglas et al. "Augmented reality:
`Advances in diagnostic imaging,” Multimodal Technologies and
`Interaction (2017).
`
`
`
`
`
`
`
`19 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 21 of 82 PageID 211
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`Source: https://youtu.be/SKpKlh1-en0?t=58
`
`
`
`For example, a Microsoft HoloLens promotional video demonstrates
`selecting 3D volume of a heart from a collection of CT slices.
`
`
`
`
`
`
`
`
`
`
`For example, Microsoft explains transforming CT scans into 3D models
`on Twitter.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`For example, in a Verizon video (https://vimeo.com/363910670 ), Dr.
`Osamah Choudhry is seen wearing the Microsoft HoloLens and states
`“But we realized that if we developed the technology that could take two-
`dimensional patient imaging, whether it be MRI or CAT scan and convert
`it into three-dimensional holographic renderings of that same patient, we
`could enable surgeons to dissect around a cancer. We could precisely
`remove it." (quote obtained at 55 seconds into the video).
`
`
`
`
`Source: https://twitter.com/hololens/status/880413174238924800?lang=en
`
`20 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 22 of 82 PageID 212
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`For example, Medical Doctors use HoloLens to generate three-
`dimensional images of a liver.
`
`
`
`Source: https://www.zdnet.com/article/microsofts-hololens-these-surgeons-are-using-ar-to-explore-
`organsmicrosofts-hololens-how-these/
`
`
`
`
`Source: https://www.zdnet.com/article/microsofts-hololens-these-surgeons-are-using-ar-to-explore-
`organsmicrosofts-hololens-how-these/
`
`In non-medical fields, by way of background, a person of ordinary skill in
`the art would know a volume of interest can be selected from a collection
`of radar, sonar or LiDAR slices (e.g., Kim et al, Forests, 2016 and
`Anderson et al, Landscape and Urban Planning, 2018, Llorens,
`Sensors,2011) (hereinafter Anderson article – Exhibit 9) as well as by
`large DoD contractors including Lockheed Martin. For further
`background, a 2009 article by researchers Gebhardt, Payzer, Salemann,
`Fettinger, Rotenberg and Seher at Lockheed Martin on High Fidelity
`Terrain Representations, stated that "Gridded voxel models treat a
`volumetric data set as a stack of individual voxel “slices” with each voxel
`slice, or grid, being one voxel deep. Processing and querying of this voxel
`
`21 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 23 of 82 PageID 213
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`data model can be performed slice by slice until all slices have been
`processed. This model introduces some efficiency in processing as only
`one slice is processed at a time." (hereinafter Lockheed article – Exhibit
`10)
`
`On information and belief, the HoloLens IVAS selects a synthetic
`training environment (volume of interest) from a collection of LIDAR
`data (slices).
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`For example, Microsoft provides a number of sample applications that
`provide three-dimensional viewing of images by a user, that when
`performed on the processor, provides a process for processing
`information, the process causing the computer system to perform the
`selection of a volume of interest from a collection of image slices.
`
`
`
`
`
`
`
`
`
`
`
`
`Captioned: "Functioning principle of the LIDAR sensor. The laser
`beams obtain for each crop slice a variable number of identified points
`according the distance to the sensor and angle from the horizontal."
` 
`Source: Jordi Llorens Calveras, et. al., “Ultrasonic and LIDAR Sensors for Electronic Canopy
`Characterization in Vineyards: Advances to Improve Pesticide Application Methods,” Sensors
`11(2):2177-94, Dec. 2011.
` 
`
`22 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 24 of 82 PageID 214
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`For example, Microsoft’s HoloTerrain:
`
`
`Source: https://www.microsoft.com/en-us/p/holoterrain/9mwkd575dqq3?activetab=pivot:overviewtab
`
`
`
`
`
`
`Source: https://www.microsoft.com/en-us/p/holoterrain/9mwkd575dqq3?activetab=pivot:overviewtab
`
`IVAS
`
`In accordance with the IVAS Program Objectives, by way of background,
`the volume of interest for the US Army is the 3D terrain data, which is
`acquired from airborne LiDAR (Lockheed article – Exhibit 10).
`
`
`23 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 25 of 82 PageID 215
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
` 
`Source:
`https://www.sisostds.org/DesktopModules/Bring2mind/DMX/API/Entries/Download?Command=Core_Downl
`oad&EntryId=28745&PortalId=0&TabId=105
`
`
`With regard to Microsoft’s IVAS testing, one of ordinary skill in the art
`would know that 3D terrain data is vital to military operations and
`training, and that airborne LIDAR is used as described in the Anderson
`and Lockheed articles.
`
`
`“This upgrade allows the drone to download its surveillance data to
`one of the mini-servers carried by the squad, which then turns 2D
`imagery of a given area into a 3D model. This isn’t just a map. It’s
`actually a small virtual reality that soldiers can call up on their
`goggles and explore, letting them check out potential chokepoints,
`lines of fire, and approach routes before they even see the real
`terrain.”
`
`Source: https://breakingdefense.com/2019/12/soldiers-coders-surprise-army-brass-changing-ivas-
`goggles/
`
`
`On information and belief, the IVAS program has been testing such
`terrain mapping software that converts 2D images to 3D Images.
`
`In another test, the Army used a pocket-sized drone (FLIR’s Black
`Hornet UAV) to conduct terrain mapping exercises.
`
`24 
`
`
`
` 
`
`
`

`

`Case 6:20-cv-01699-GAP-DCI Document 1-5 Filed 09/16/20 Page 26 of 82 PageID 216
`
`U.S. Patent 8,384,771
`“Method and Apparatus for Three-Dimensional Viewing of Images”
`Pre-Discovery Evidence of Use for Infringement
`
`Priority Date: December 28, 2006
`Note: Statements made herein are illustrative and not exhaustive
`
`“Army modernization officials used the pocket-sized drones, which
`are part of the Soldier Borne Sensor program, to take video footage
`of a military operations in urban terrain (MOUT) training complex.
`Then they viewed the footage in IVAS, Potts said.”
`
`"They flew [the drones] over the MOUT site; [they] downloaded its
`data; it processed an algorithm; and inside an IVAS suddenly
`you've got a 3-D holographic image of that entire MOUT site. And
`now you can do a complete battle drill without building a sand
`table," Potts recently told Military.com. "You can walk around it.
`You can look down the streets."
`
`Source: https://www.military.com/daily-news/2019/12/16/wearable-device-could-replace-sand-tables-
`soldiers.html
`
`“The National Geospatial-Intelligence Agency (NGA) has been
`deploying LIDAR in aircraft to map Afghanistan’s entire 647,500
`square kilometers. In announcing the ALIRT LIDAR project at the
`GEOINT 2010 Symposium, Air Force Lt. Gen. John Koziol,
`director of the Defense Department’s Intelligence, Surveillance and
`Reconnaissance (ISR) Task Force, heralded the technology’s
`“amazing capacities” for coverage down to inch-level fidelity."
`
`Source: https://gcn.com/articles/2011/07/18/tech‐watch‐geoint‐lidar.aspx 
`
`
`On information and belief, the accused systems include a system for
`arranging said slices corresponding to said volume of interest. For
`example, the HoloLens 1, HoloLens 2, and IVAS embody a system for
`arranging said slices corresponding to said volume of interest.
`
`One of ordinary skill in the art would understand that “slices” is a
`commonly accepted terminology for LIDAR-derived voxel datasets: For
`example, an excerpt from the Anderson article describes it as follows:
`"A straightforward way to view 3D voxel data is to change the
`perspective from the map 'top-down' view to a ‘side-on’ viewpoint,
`viewing the volume from the ground-up and visualizing the data from the
`pavements to the tree-tops. Voxels were stacked to create vertical cross
`sections through each of the cities, allowing the perspective to be changed
`from an aerial to a frontal view. In order to do this we utilized Python’s
`imaging library (PIL) to extract a single row