‘
`
`El.-
`
`3% m
`\% ‘°
`fig
`
`Era
`E 8
`
`.f
`
`”
`
`la w w
`
`(jaw fl
`
`5,
`
`A
`
`rt/Mv‘
`
`I
`
`I“
`
`/'
`
`IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`REQUEST FOR FILING A PROVISIONAL APPLICATION FOR PATENT
`UNDER 37 CFR §1 .53 (c)
`
`INVENTOR(S)
`
`l
`1
`
`Levanon
`
`3 Nachal Besor St., Ramat Hasham, Israel
`
`o
`r E
`ES
`é‘s
`
`g I
`
`
`
`
`'
`TITLE OF THE INVENTION
`
`
`Lav:
`
`21 Bar llan St., Raanana, Israel
`
`8 "I §3
`§
`\
`
`OPTIMIZATION OF IMAGE PARCEL SIZE FOR FAST QUALITY BUILD-UP
`OF DATA PACKETS STREAMED OVER LIMITED AND NARROWBAND
`COMMUNICATION CHANNELS
`
`_)_(_ Direct all correspondence to Customer Number 23488.
`1(\J‘erald B. Rosenberg, Esq.
`(Reg No.: 30,320)
`Telephone:
`650.325.2100
`. NewTechLaw
`Facsimile:
`650.325.2107
`285 Hamilton Avenue, Suite 520
`Palo Alto, California 94301
`
`‘
`
`
`
`PATENTTRADEMARK OFFICE
`
`
`
`
`
`ENCLOSED APPLICATION PARTS (check all that apply)
`
`Specification
`No. of pages:
`6
`_ Small Entity Statement
`
`
`
`
`
`Drawings
`
`No. of sheets:
`
`__ Power ot Attorney
`
`Declaration
`
`_ Assignment and Cover Sheet
`
`Other: Return—Receipt Post Card
`
`"
`
`’
`
`METHOD OF PAYMENT OF FILING FEES FOR THIS PROVISIONAL APPLICATION FOR PATENT
`
`’ Provisional Basic Filing Fee: $ 150.00 (Small Entity: $75.00)
`
`Filing Fee Amount: $ 150.00
`
`X
`
`A check is enclosed to cover the Filing Fees.
`
`The Commissioner is hereby authorized charge Filing Fees or credit any
`overpayment to: Deposit Account Number: 50—0890.
`
`This invention was 391 made by or under contract with a US Government agency.
`
`US Governent agency on Contract:
`
`, Signature:
`
`Date: December 26 2000
`
`EL 661 534 257 US Address To:
`
`Gerald B. Rosenberg
`Reg. No.: 30,320
`
`plication Docket No:
`
`FLVT3000
`
`Express Mail Label No.2
`
`Box Provisional Alication Assistant Commissioner for Patents Washinton DC 20231
`
`gbr/flvt’3000 002 prov xmittal wpd
`
`1 0f 12
`
`Microsoft Corp; Exhibit 1070
`
`1 of 12
`
`Microsoft Corp. Exhibit 1070
`
`

`

`OPTIMIZATION OF IMAGE PARCEL SIZE FOR
`FAST QUALITY BUILD-UP OF DATA PACKETS
`STREAMED OVER LIMITED AND
`NARROWBAND COMMUNICATION
`CHANNELS
`
`Inventors:
`Isaac Levanon
`Yoni Lavi
`
`Background of the Invention
`
`The present invention is generally related to the delivery of high-resolution
`
`highly featured graphic images over limited and narrowband communications
`
`channels.
`
`Summafl of the Invention
`
`The objective is to display a two-dimensional pixel map, aI 6-Bit RGB color
`image in the preferred embodiments, of very large dimensions and permitting the
`viewing of the image from a dynamic three-dimensional viewpoint. Multiple such
`
`Attorney Docket No.: FLVTBOOO
`gbr/flvt/3000.000.provisional.wpcl
`
`I 2/26/2000
`
`2 0f 12
`
`Microsoft Corp. Exhibit 1070
`
`I
`2
`3
`4
`5
`
`6 7 8 9
`
`IO
`I I
`I2
`I3
`
`I4
`
`I5
`
`I6
`
`I7
`I8
`
`I9
`
`2O
`
`2I
`
`22
`
`23
`
`24
`25
`26
`
`
`
`
`2 of 12
`
`Microsoft Corp. Exhibit 1070
`
`

`

`-2-
`
`images are remotely hosted for on-demand selection and transfer to a client
`
`system for viewing.
`
`Images, as stored by the server, may individually range from gigabytes to
`
`multiple terabyte in total size. A correspondingly large server storage and
`
`processing system is contemplated. Conversely, client systems are contemplated
`
`to be conventional personal computer systems and, in particular, mobile, cellular,
`
`embedded, and handheld computer systems, such as personal digital assistants
`
`(PDAs) and internet-capable digital phones, with relatively limited to highly
`
`constrained network communications capabilities. For most wireless applications,
`
`conventional narrowband communications links have a bandwidth of less than
`
`approximately three kilobytes of data per second. Consequently, transmittal of
`
`entire images to a client system in reasonable time is infeasible as a practical
`
`matter.
`
`Description of the Invention
`
`For purposes of the present invention, each image (Figure l) is at least
`
`logically defined in terms of multiple grids of image parcels with various levels of
`
`resolutions (Figure 2) that are created through composition of information from
`
`all level of resolutions, and stored by the server to provide an image for transfer
`
`to a client system (Figure 3). Composed and separate static and dynamically
`
`created layers are transferred to client system in parcels in a program selectable
`
`order to optimize for fast quality build-up of the image presented to a user of the
`
`client system, particularly when the parcels are streamed over a narrowband
`
`communication link.
`
`O‘OWNOU‘I-kwM—I
`
`l 1
`
`12
`
`l3
`
`l 4
`
`15
`
`l6
`
`17
`
`18
`
`19
`
`20
`
`2i
`
`22
`
`23
`
`24
`
`
`
`
`Attorney Docket No.: FLVTBOOO
`gbr/flVl/3000.000.provisional.wpcl
`
`12/26/2000
`
`3 0f 12
`
`Microsoft Corp. Exhibit 1070
`
`3 of 12
`
`Microsoft Corp. Exhibit 1070
`
`

`

`-3-
`
`The multiple layers of an image allow the selectivity to incorporate
`
`topographical, geographical, orientational, and other terrain and mapping
`related information into the image delivered. Other layers, such as geographic
`
`grids, graphical text overlays, and hyperlink selection areas, separately provided
`
`or composed, aid in the useful presentation and navigation of the image as
`
`presented by the client system and viewed by the user.
`Compositing of layers on the server enables the data transfer burden to be
`
`reduced, particularly in analysis of the requirements and capabilities of the client
`
`system and the connecting communications link. Separate transfer of layers to the
`client system allows the client system selectivity in managing and presentation of
`
`the data to the user.
`
`The system and methods of the present invention are designed to, on
`
`demand, select, process and immediately transfer data parcels to the client
`
`system, which immediately processes and displays a low-detail representation of
`
`the image requested by the client system. The system and methods immediately
`
`continue to select, process and sequentially transfer data parcels that, in turn, are
`
`processed and displayed by the client system to augment the presented image
`
`and thereby provide a continuously improving image to the user.
`
`Selection of the sequentially transferred data is, in part, dependent on the
`
`progressive translation of the three-dimensional viewpoint as dynamically
`
`modified on the client system during the transfer process. This achieves the
`
`above-stated obiective while concurrently achieving a good rendering quality for
`
`continuous fly-over of the image as fast as possible, yet continuously building the
`
`image quality to the highest resolution of the image as stored by the server.
`
`1 2 3 4 5 6
`
`7 8 9
`
`O1
`
`1 1
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`
`
`Attorney Docket No.: FLVT3000
`gbr/flvt/SOO0.000.provisionaI.wpd
`
`12/26/2000
`
`4 0f 12
`
`Microsoft Corp. Exhibit 1070
`
`4 of 12
`
`Microsoft Corp. Exhibit 1070
`
`

`

`-4-
`
`To optimize image quality build-up over
`
`limited and narrowband
`
`communication links, the target image, as requested by the client system,
`
`is
`
`represented by multiple grids of 64x64 image pixels (Figure 4) with each grid
`
`having some corresponding level of detail. That is, each grid is treated as a
`
`sparse data array that can be progressively revised to increase the resolution of
`
`the grid and thereby the level of detail presented by the grid. The reason for
`
`choosing the 64x64 pixel dimension is that, using current image compression
`
`algorithms, 0 16-bit 64x64 pixel array image can be presented as a 2KByte data
`
`parcel.
`
`In turn, this 2KByte parcel is the optimal size, subiect to conventional
`
`protocol and overhead requirements, to be transmitted through a 3KByte per
`
`second narrowband transmission channel. Using a smaller image array,‘such as
`
`32x32, would create a 0.5KByte parcel, hence causing inefficiencies due to packet
`
`transmission overhead, given the nature of current wireless communications
`
`protocols.
`
`Image array dimensions are preferably powers of two so that they can be
`
`used in texture mapping efficiently. Each parcel, as received by the client system,
`
`is preferably immediately processed and incorporated into the presented image.
`
`To do so efficiently, according to the present invention, each data parcel
`
`is
`
`independently processable by the client system, which is enabled by the selection
`
`and server-side processing used to prepare a parcel for transmission.
`
`In addition,
`
`each data parcel is sized appropriate to fit within the level-1 cache, or equivalent,
`
`of the client system processor, thereby enable the data processing intensive
`
`operations needed to process the data parcel to be performed without extended
`
`memory access delays.
`
`In the preferred embodiment of the present invention,
`
`Attorney Docket No.: FLVT3000
`gbr/flvt/3000.000.provisional.wpd
`
`12/26/2000
`
`5 0f 12
`
`Microsoft Corp. Exhibit 1070
`
`1
`
`2 3 4 5 6
`
`7
`
`8
`
`9
`
`0
`
`12
`
`13
`
` 1 1
`
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`5 of 12
`
`Microsoft Corp. Exhibit 1070
`
`

`

`-5-
`
`data parcels are also processed for texture mapping and other image features,
`
`such as topographical detailing.
`
`Currently, with regard to conventional client systems, a larger image array,
`
`such as 128x128, is too large to be fully placed within the level-l cache of many
`
`of the smaller conventional current processors, such as used by personal digital
`
`assistants (PDAs) and cellular phones.
`
`Since access to cache memory is
`
`substantially faster than to RAM this will likely result in lower frame rate.
`
`Different and larger data parcel sizes may be optimal as transmission
`
`protocols and micro-architectures of the client computers change. For purposes
`above, the data content was a pixel array representing image data. Where the
`
`data parcel content is vector, text or other data that may subiect to different client
`
`system design factors, other parcel sizes may be used.
`In the process implemented by the present invention, data parcels maybe
`
`selected for sequential transmission based on a prioritization of the importance
`
`of the data contained. The criteria of importance maybe defined as suitable for
`
`particular applications and may directly relate to the presentation of image
`quality, provision of a textual overlay of a low-quality image to quickly provide a
`
`navigational orientation, or the addition of topography information at a rate or
`
`timing different from the rate of image quality improvement. Thus, image data
`
`text overlays, and topography can be
`layers reflecting navigational cues,
`composed into data packets for transmission subject to prioritizations set by the
`server alone, based on the nature and type of the client system, and interactively
`
`influenced by the actions and commands provided by the user of the client system
`
`(Figure 5).
`
`Attorney Docket No.: FLVT3000
`gbr/flvt/3000.000.provisional.wpd
`
`12/26/2000
`
`6 of 12
`
`Microsoft Corp. Exhibit 1070A\
`
`1 2 3 4 5
`
`o 7
`
`8
`
`9
`10
`
`l 1
`
`12
`13
`
`
` 17
`
`14
`
`15
`
`16
`
`l8
`
`19
`
`20
`21
`22
`
`23
`
`24
`
`6 of 12
`
`Microsoft Corp. Exhibit 1070
`
`

`

`-5-
`
`Progressive transmission of image parcels is performed in an iterative
`process involving selection of an image data grid within the target image of the
`client system, which is a portion of a potentially multi-layered source image stored
`by the server. The selection parameters are preferably dependent on the client
`navigation viewpoint, effective velocity, and height, and the effective level of detail
`currently presented in each grid. Once a grid is selected, the server selects the
`source data to be logically composed into the selected grid to complement the
`
`effective resolution of that grid, processing the grid data to produce the optimally
`
`sized size grid data parcels, and sequentially transmitting the parcels to the client
`
`system. Preferably, the detail of a grid array is sequentially enhanced by division
`of the grid into sub-grids related by a power of two (Figure 6). Thus, a given grid
`is preferably updated using four data parcels having twice the data resolution of
`the existing grid. Whatever number of parcels are used, each data parcel is
`
`rendered by the client system into the target image. Additional client system
`
`image data processing to providetexturing and three-dimensional representation
`
`of the data may be performed as part of the parcel rendering and integration into
`
`the target image.
`
`Attorney Docket No.: FLVTBOOO
`gbr/flvt/3000.000.provisional.wpd
`
`12/26/2000
`
`7 of 12
`
`Microsoft Corp. Exhibit 1070
`
`1
`
`2 3 4 5
`
`6
`7
`
`8
`
`9
`
`10
`1 1
`12
`13
`
`14
`
` 17
`
`15
`
`16
`
`7 of 12
`
`Microsoft Corp. Exhibit 1070
`
`

`

`51h...
`
`
`
`29%..03_E__::
`
`m5—thEzc:
`
`am?
`
`
`
`Riff..
`
`508E:
`
`mEmEmEmmccmON
`
`
`
`:8“5.,5?.53:99.0
`
` “K
`
`\Msafimomw
`
`6:0
`
`m>:own_m._mn_
`
`x_w5:5on
`
`ho32mg
`
`Eocmm:
`
`2505.:_ON
`
`25555...
`
`80f12
`
`Microsoft Corp. Exhibit 1070
`
`8 of 12
`
`Microsoft Corp. Exhibit 1070
`
`
`
`

`

`
`
`FIG.2
`
`9 0f 12
`
`Microsoft Corp. Exhibit 1070
`
`9 of 12
`
`Microsoft Corp. Exhibit 1070
`
`

`

`
`
`
`
`10 0f 12
`
`Microsoft Corp. Exhibit 1070
`
`10 of 12
`
`Microsoft Corp. Exhibit 1070
`
`

`

`FIGS
`
`11 0f 12
`
`Microsoft Corp. Exhibit 1070
`
`11 of 12
`
`Microsoft Corp. Exhibit 1070
`
`

`

`5.1g:
`
`
`
`64
`
`64
`
`64
`
`FIG.6
`
`12 0f 12
`
`Microsoft Corp. Exhibit 1070
`
`12 of 12
`
`Microsoft Corp. Exhibit 1070
`
`