`
`
`
`EXHIBIT 9
`
`Application of U.S. Patent No. 7,916,147
`to the Accused Instrumentalities*,**
`
` The term “Accused Instrumentalities” refers to Microsoft’s operating systems that include RemoteFX (e.g., Windows Server, Windows 7,
`Windows 8, Windows 10, etc.) in conjunction with the features and capabilities disclosed herein, as well as all other practices of, and products made,
`used, sold, offered for sale and/or imported by Microsoft (as defined in the Complaint), or one of Microsoft’s affiliated companies, that have the
`features shown in this chart, or substantially similar features.
`** This claim chart is meant to be illustrative for purposes of meeting Plaintiff’s pleading obligations and should not be construed as limiting or
`binding.
`
`
`
`
`
`
`
`
`
`
` *
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 2 of 32
`
`
`
`1. A method of generating a
`compressed video data signal
`using at least one graphics
`processor module, comprising
`the steps of:
`
`Application of U.S. Patent No. 7,916,147 to the Accused Instrumentalities
`
`Microsoft, and other users of the Accused Instrumentalities, practice a method of generating a compressed video
`data signal using at least one graphics processor module (e.g., a graphics processing unit (“GPU”)). For example,
`Microsoft practices the claimed method via its use of the Accused Instrumentalities in conjunction with its Azure
`cloud computing services. Microsoft Windows Server (since Windows Server 2008 R2) incorporates Remote
`Desktop Services that is integrated with Microsoft RemoteFX.
`
`
`
`Source: Microsoft RemoteFX for Virtual Desktop Infrastructure: Architectural Overview, cover page,
`https://www.dabcc.com/resources/microsoft-remotefx-for-virtual-desktop-infrastructure-architectural-
`overview.pdf
`
`Microsoft Windows 7, Windows 8.1 and Windows 10 also incorporate Microsoft RemoteFX.
`
`
`
`
`U.S. Patent No. 7,916,147
`
`
`2
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 3 of 32
`
`[…]
`
`
`
`
`Source: https://support.microsoft.com/en-us/topic/update-to-disable-and-remove-the-remotefx-vgpu-component-
`in-windows-bbdf1531-7188-2bf4-0de6-641de79f09d2
`
`The Microsoft Azure platform supports Windows Server 2012 R2 and newer, Windows 7 enterprise and
`Windows 10 enterprise.
`
`
`
`
`U.S. Patent No. 7,916,147
`
`
`3
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 4 of 32
`
`
`Source: https://azure.microsoft.com/en-gb/services/virtual-desktop/#features
`
`Azure supports a variety of GPUs.
`
`
`
`
`
`Source: https://docs.microsoft.com/en-us/azure/virtual-desktop/configure-vm-gpu
`
`
`
`U.S. Patent No. 7,916,147
`
`
`4
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 5 of 32
`
`Microsoft RemoteFX provides graphics rendering capabilities for Windows applications running on Windows
`virtual machines.
`
`
`
`Source: Microsoft RemoteFX for Virtual Desktop Infrastructure: Architectural Overview, page 10,
`https://www.dabcc.com/resources/microsoft-remotefx-for-virtual-desktop-infrastructure-architectural-
`overview.pdf
`
`
`
`
`U.S. Patent No. 7,916,147
`
`
`5
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 6 of 32
`
`Microsoft RemoteFX provides host-based graphics virtualization capability that provides graphics services
`utilizing graphics processor modules.
`
`
`
`Source: Microsoft RemoteFX for Virtual Desktop Infrastructure: Architectural Overview, page 9,
`https://www.dabcc.com/resources/microsoft-remotefx-for-virtual-desktop-infrastructure-architectural-
`overview.pdf
`
`Microsoft RemoteFX comprises a Remote Desktop Virtual Graphics Manager that renders and compresses
`graphics.
`
`
`
`
`[…]
`
`
`
`U.S. Patent No. 7,916,147
`
`
`6
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 7 of 32
`
`
`Source: Microsoft RemoteFX for Virtual Desktop Infrastructure: Architectural Overview, page 9-10,
`https://www.dabcc.com/resources/microsoft-remotefx-for-virtual-desktop-infrastructure-architectural-
`overview.pdf
`
`Once generated the compressed video data signal is delivered to client devices using the Remote Desktop
`Protocol (RDP).
`
`
`
`
`
`
`
`Source: Microsoft RemoteFX for Virtual Desktop Infrastructure: Architectural Overview, page 3,
`https://www.dabcc.com/resources/microsoft-remotefx-for-virtual-desktop-infrastructure-architectural-
`overview.pdf
`
`The Accused Instrumentalities intercept a first set of instructions for the graphics processor module. The first set
`of instructions relates to how to render image frames. The first set of instructions are graphics instructions issued
`by Windows applications running on a virtual machine. Desktop Windows Manager (DWM) redirects (and thus
`intercepts) graphics instructions arriving from applications running on Windows, allowing the compositing of
`windows.
`
`DWM is a component of Microsoft’s operating systems.
`7
`
`a) intercepting a first set of
`instructions for said graphics
`processor module, said first set
`of instructions relating to how to
`render image frames;
`
`U.S. Patent No. 7,916,147
`
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 8 of 32
`
`
`
`
`Source: Desktop Window Manager, https://docs.microsoft.com/en-us/windows/win32/dwm/dwm-overview.
`
`
`
`
`
`U.S. Patent No. 7,916,147
`
`
`8
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 9 of 32
`
`
`Source: The Desktop Window Manager, https://docs.microsoft.com/en-us/windows/win32/learnwin32/the-
`desktop-window-manager
`
`
`
`
`
`
`Source: Redirecting GDI, DirectX, and WPF applications, https://docs.microsoft.com/en-
`gb/archive/blogs/greg_schechter/redirecting-gdi-directx-and-wpf-applications
`
`Microsoft RemoteFX comprises an RD Virtual Graphics Manager that contains a RemoteFX RCC (rendering,
`capturing, and compressing) engine. Microsoft RemoteFX intercepts graphics instructions arriving from at least
`9
`
`
`
`U.S. Patent No. 7,916,147
`
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 10 of 32
`
`one Windows virtual machine. The instructions are used to render a virtualized end-user displays (image
`frames).
`
`
`
`Source: Microsoft RemoteFX for Virtual Desktop Infrastructure: Architectural Overview, page 10,
`https://www.dabcc.com/resources/microsoft-remotefx-for-virtual-desktop-infrastructure-architectural-
`overview.pdf
`
`
`
`U.S. Patent No. 7,916,147
`
`
`10
`
`
`
`b) generating a second set of
`instructions for said graphics
`processor module;
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 11 of 32
`
`The Accused Instrumentalities generate a second set of instructions for the graphics processor module. The
`second set of instructions are the first set of instructions issued from Windows applications processed by DWM
`to redirect and composite windows, and then ‘optimized’ and ‘translated’ by Microsoft RemoteFX before being
`delivered to a graphics processor module.
`
`DWM redirects graphics instructions from Windows applications and composites the windows, thereby
`generating graphics instructions that will be forwarded to Microsoft RemoteFX.
`
`
`
`Source: The Desktop Window Manager, https://docs.microsoft.com/en-us/windows/win32/learnwin32/the-
`desktop-window-manager
`
`Outputs from multiple virtual machines running applications are intercepted by Microsoft RemoteFX.
`
`
`
`
`U.S. Patent No. 7,916,147
`
`
`11
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 12 of 32
`
`
`Source: Microsoft RemoteFX for Virtual Desktop Infrastructure: Architectural Overview, page 6,
`https://www.dabcc.com/resources/microsoft-remotefx-for-virtual-desktop-infrastructure-architectural-
`overview.pdf
`
`The role of RemoteFX vGPU in managing multiple, concurrent virtual machines can be replaced by a GPU such
`as the AMD Radeon Instinct MI25 GPU that is part of Microsoft Azure’s NVv4 offering.
`
`
`
`
`U.S. Patent No. 7,916,147
`
`
`12
`
`
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 13 of 32
`
`[…]
`
`[…]
`
`
`
`
`Source: Power your Azure GPU workstations with flexible GPU partitioning, https://azure.microsoft.com/en-
`gb/blog/power-your-azure-gpu-workstations-with-flexible-gpu-partitioning/
`
`RemoteFX generates instructions by ‘optimizing’ and ‘translating’.
`
`Outputs from Windows applications are intercepted, inspected, and optimized by Microsoft RemoteFX into a set
`of instructions that are delivered to a graphic processor module.
`
`
`
`
`U.S. Patent No. 7,916,147
`
`
`13
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 14 of 32
`
`
`Source: Microsoft RemoteFX for Virtual Desktop Infrastructure: Architectural Overview, page 10,
`https://www.dabcc.com/resources/microsoft-remotefx-for-virtual-desktop-infrastructure-architectural-
`overview.pdf
`
`
`The RCC engine in the RDVGM translates graphics requests (generates second instructions) from each virtual
`machine into DirectX-compliant commands on the parent partition that are then delivered a graphics processor
`module.
`
`
`
`U.S. Patent No. 7,916,147
`
`
`14
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 15 of 32
`
`
`
`c) processing said first set of
`instructions or said second set of
`instructions or a combination
`thereof in said graphics
`processor module to generate
`first graphics data, said first
`graphics data comprising image
`frames;
`
`
`
`
`Source: Microsoft RemoteFX for Virtual Desktop Infrastructure: Architectural Overview, page 9,
`https://www.dabcc.com/resources/microsoft-remotefx-for-virtual-desktop-infrastructure-architectural-
`overview.pdf
`
`The Accused Instrumentalities process the first set of instructions, the second set of instructions, or a combination
`thereof in the graphics processor module to generate first graphics data that includes image frames. The first set
`of instructions are graphics instructions issued by Windows applications running on a virtual machine. The
`second set of instructions are the first set of instructions issued from Windows applications processed by the
`Desktop Windows Manager (DWM) to redirect and composite windows, and then ‘optimized’ and ‘translated’ by
`Microsoft RemoteFX and delivered to a graphics processor module to generate an image frame (first graphics
`data).
`
`DWM redirects graphics instructions from Windows applications and composites the windows, thereby
`generating graphics instructions that will be received by Microsoft RemoteFX.
`
`
`U.S. Patent No. 7,916,147
`
`
`15
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 16 of 32
`
`
`Source: The Desktop Window Manager, https://docs.microsoft.com/en-us/windows/win32/learnwin32/the-
`desktop-window-manager
`
`Graphics instructions intercepted by Microsoft RemoteFX are inspected and optimized and are delivered to a
`graphics processor module to render a frame (first graphics data) into a single frame buffer. The frame buffer
`contains the end-user display (image frames).
`
`
`
`
`U.S. Patent No. 7,916,147
`
`
`16
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 17 of 32
`
`
`Source: Microsoft RemoteFX for Virtual Desktop Infrastructure: Architectural Overview, page 10,
`https://www.dabcc.com/resources/microsoft-remotefx-for-virtual-desktop-infrastructure-architectural-
`overview.pdf
`
`
`
`
`
`U.S. Patent No. 7,916,147
`
`
`17
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 18 of 32
`
`d) processing said second set of
`instructions in said graphics
`processor module to generate
`second graphics data;
`
`
`
`
`Source: Microsoft RemoteFX for Virtual Desktop Infrastructure: Architectural Overview, page 5,
`https://www.dabcc.com/resources/microsoft-remotefx-for-virtual-desktop-infrastructure-architectural-
`overview.pdf
`
`The Accused Instrumentalities process the second set of instructions in the graphics processor module to generate
`second graphics data. The second set of instructions are the first set of instructions issued from Windows
`applications processed by the Desktop Windows Manager (DWM) to redirect and composite windows, and then
`‘optimized’ and ‘translated’ by Microsoft RemoteFX and delivered to a graphics processor module to generate an
`image frame (first graphics data). Second graphics data are details of the parts of the image that have been
`updated or changed and the size and memory locations of the image frames corresponding to different users.
`
`DWM composites inputs from multiple Windows applications with only regions of the display that change (are
`“dirtied”) needing to be re-rendered. The dirty regions are managed with “occlusion culling support” that allows
`the composition to not render content of windows that are overlapped.
`
`
`[…]
`
`
`
`U.S. Patent No. 7,916,147
`
`
`18
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 19 of 32
`
`
`Source: How underlying WPF concepts and technology are being used in the DWM,
`https://docs.microsoft.com/en-gb/archive/blogs/greg_schechter/how-underlying-wpf-concepts-and-technology-
`are-being-used-in-the-dwm
`
`The graphics instructions that are ‘optimized’ and ‘translated’ by Microsoft RemoteFX are delivered to a
`graphics processor module to generate second graphics data comprising details of the parts of the image that have
`changed.
`
`
`
`U.S. Patent No. 7,916,147
`
`
`19
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 20 of 32
`
`
`Source: Microsoft RemoteFX for Virtual Desktop Infrastructure: Architectural Overview, page 11,
`https://www.dabcc.com/resources/microsoft-remotefx-for-virtual-desktop-infrastructure-architectural-
`overview.pdf
`
`Outputs from Windows applications are intercepted, inspected, and optimized by Microsoft RemoteFX into a set
`of instructions that are delivered to a graphic processor module for rendering of frames. Size and memory
`locations of the image frames of the corresponding to different users are second graphics data.
`
`
`
`
`U.S. Patent No. 7,916,147
`
`
`20
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 21 of 32
`
`
`Source: Microsoft RemoteFX for Virtual Desktop Infrastructure: Architectural Overview, page 6,
`https://www.dabcc.com/resources/microsoft-remotefx-for-virtual-desktop-infrastructure-architectural-
`overview.pdf
`
`The role of RemoteFX vGPU in managing multiple, concurrent virtual machines can be replaced by a GPU such
`as the AMD Radeon Instinct MI25 GPU that is part of Microsoft Azure’s NVv4 offering.
`
`
`
`
`
`
`U.S. Patent No. 7,916,147
`
`
`21
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 22 of 32
`
`[…]
`
`[…]
`
`
`
`
`
`e) processing said second
`graphics data to generate
`compression assistance data; and
`
`
`Source: Power your Azure GPU workstations with flexible GPU partitioning, https://azure.microsoft.com/en-
`gb/blog/power-your-azure-gpu-workstations-with-flexible-gpu-partitioning/
`
`The Accused Instrumentalities process the second graphics data to generate compression assistance data. Second
`graphics data are details of the parts of the image that have been updated or changed and the size and memory
`locations of the image frames corresponding to different users. Compression assistance data is details of ‘tiles’ to
`be compressed with the RemoteFX encoder or ‘macroblocks’ when H.264 compression is used.
`
`Microsoft RemoteFX only encodes areas of an image that have changed.
`
`U.S. Patent No. 7,916,147
`
`
`22
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 23 of 32
`
`
`
`
`Source: Microsoft RemoteFX for Virtual Desktop Infrastructure: Architectural Overview, page 11,
`https://www.dabcc.com/resources/microsoft-remotefx-for-virtual-desktop-infrastructure-architectural-
`overview.pdf
`
`Microsoft RemoteFX uses the mechanism of input tiling to identify areas of the image that change.
`
`
`
`U.S. Patent No. 7,916,147
`
`
`23
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 24 of 32
`
`
`Source: Remote Desktop Protocol: RemoteFX Codec Extension, page 32,
`https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-rdprfx/62495a4a-a495-46ea-b459-
`5cde04c44549
`The RDP protocol messages include details of regions and titles that have been encoded (compressed).
`
`
`
`
`
`Source: https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-rdprfx/23d2a1d6-1be0-4357-83eb-
`998b66ddd4d9
`
`[…]
`
`
`U.S. Patent No. 7,916,147
`
`
`24
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 25 of 32
`
`
`Source: https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-rdprfx/7c926114-4bea-4c69-a9a1-
`caa6e88847a6
`
`The RDP protocol transports RemoteFX codec messages to the client that include tiles of compressed data.
`
`
`
`
`
`
`[…]
`
`
`
`U.S. Patent No. 7,916,147
`
`
`25
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 26 of 32
`
`
`Source: https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-rdprfx/0f20e0d4-1cf6-476b-88a5-
`a0abd9b02f75
`
`RemoteFX compresses image frames using a tile-based transform codec.
`
`
`
`
`
`U.S. Patent No. 7,916,147
`
`
`26
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 27 of 32
`
`
`Sources: Remote Desktop Protocol: RemoteFX Codec Extension, page 32,
`https://winprotocoldoc.blob.core.windows.net/productionwindowsarchives/MS-RDPRFX/%5bMS-
`RDPRFX%5d.pdf
`
`Microsoft Remote Desktop protocol may use the H.264 codec. H.264 is an industry standard, macroblock-based
`compression technology. The compression assistance data would be details of the macroblocks to be
`compressed.
`
`
`
`
`
`
`Source: Remote Desktop Protocol (RDP) 10 AVC/H.264 improvements in Windows 10 and Windows Server 2016
`Technical Preview, https://techcommunity.microsoft.com/t5/microsoft-security-and/remote-desktop-protocol-
`rdp-10-avc-h-264-improvements-in-windows/ba-p/249588.
`
`The RDP protocol transports messages to the client that include regions of macroblocks of compressed data.
`
`
`
`U.S. Patent No. 7,916,147
`
`
`27
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 28 of 32
`
`[…]
`
`
`
`f) processing said first graphics
`data using said compression
`assistance data to generate a
`compressed video data signal;
`
`
`Source: https://winprotocoldoc.blob.core.windows.net/productionwindowsarchives/MS-RDPEGFX/%5BMS-
`RDPEGFX%5D.pdf
`
`The Accused Instrumentalities process the first graphics data using the compression assistance data to generate a
`compressed video data signal. The first graphics data is a rendered frame. Compression assistance data is details
`of ‘tiles’ to be compressed with the RemoteFX encoder or ‘macroblocks’ when H.264 compression is used. The
`compressed video data signal is the signal sent by RDP to the client.
`
`RDP provides video to client devices.
`
`
`
`
`
`
`U.S. Patent No. 7,916,147
`
`
`28
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 29 of 32
`
`
`Source: Microsoft RemoteFX for Virtual Desktop Infrastructure: Architectural Overview, page 3 (above) and
`page 6 (below), https://www.dabcc.com/resources/microsoft-remotefx-for-virtual-desktop-infrastructure-
`architectural-overview.pdf
`
`The Remote Desktop Protocol delivers compressed images to clients via messages that comprise details of the
`encoded sequence and the compressed video data itself.
`
`
`
`
`
`Source: Remote Desktop Protocol: RemoteFX Codec Extension, page 10,
`https://winprotocoldoc.blob.core.windows.net/productionwindowsarchives/MS-RDPRFX/%5bMS-
`RDPRFX%5d.pdf
`
`
`
`U.S. Patent No. 7,916,147
`
`
`29
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 30 of 32
`
`wherein the generation of said
`second set of instructions
`comprises analyzing said first set
`of instructions to determine
`which instructions of said first
`set of instructions are useful for
`the generation of compression
`assistance data.
`
`The generation of the second set of instructions includes analyzing the first set of instructions to determine which
`instructions are useful to generate compression assistance data. The second set of instructions are the first set of
`instructions issued from Windows applications generated by the Desktop Windows Manager (DWM) to redirect
`and composite windows and then ‘optimized’ and ‘translated’ by Microsoft RemoteFX and delivered to a
`graphics processor module to generate an image frame. Compression assistance data is details of ‘tiles’ to be
`compressed with the RemoteFX encoder or ‘macroblocks’ when H.264 compression is used.
`
`DWM composites inputs from multiple Windows applications. DWM analyses the first set of instructions for
`rendering frames to determine which areas of application windows overlap and that is used to generate the second
`set of instructions that will result in compression assistance data.
`
`
`
`Source: The Desktop Window Manager, https://docs.microsoft.com/en-us/windows/win32/learnwin32/the-
`desktop-window-manager
`
`Microsoft RemoteFX inspects (analyzes) the first set of instructions to distinguish between instructions of
`different users’ applications to differentiate between corresponding image frames and thereby generate second
`graphics data which in turn is used to generate compression assistance data.
`
`
`
`
`U.S. Patent No. 7,916,147
`
`
`30
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 31 of 32
`
`
`Source: Microsoft RemoteFX for Virtual Desktop Infrastructure: Architectural Overview, page 6,
`https://www.dabcc.com/resources/microsoft-remotefx-for-virtual-desktop-infrastructure-architectural-
`overview.pdf
`
`The role of RemoteFX vGPU in managing multiple, concurrent virtual machines can be replaced by a GPU such
`as the AMD Radeon Instinct MI25 GPU that is part of Microsoft Azure’s NVv4 offering.
`
`
`
`
`U.S. Patent No. 7,916,147
`
`
`31
`
`
`
`
`
`
`Case 6:21-cv-00462-ADA Document 1-9 Filed 05/04/21 Page 32 of 32
`
`[…]
`
`[…]
`
`
`
`
`Source: Power your Azure GPU workstations with flexible GPU partitioning, https://azure.microsoft.com/en-
`gb/blog/power-your-azure-gpu-workstations-with-flexible-gpu-partitioning/
`
`
`
`
`
`
`U.S. Patent No. 7,916,147
`
`
`32
`
`