Microsoft TerraServer
`
`Page 1 of 26
`
`Microsoft TerraServer 
`
`SQL Server 7.0
`
`Tom Barclay, Robert Eberl, Jim Gray,
`John Nordlinger, Guru Raghavendran,
`Don Slutz, Greg Smith, Phil Smoot
`Microsoft Research and Development
`
`John Hoffman, Natt Robb III
`Aerial Images
`
`Hedy Rossmeissl, Beth Duff, George Lee,
`Theresa Mathesmier, Randall Sunne
`United States Geological Survey
`
`Lee Ann Stivers, Ken Goodman
`Digital Equipment Corporation
`
`June 1998
`
`Summary: The Microsoft® TerraServer stores aerial and satellite images of the earth in a Microsoft SQL Server™
`database served to the public through the Internet. It is the world's largest atlas, combining five terabytes of image
`data from the United States Geodetic Survey, SOVINFORMSPUTNIK, and Encarta® Virtual Globe. Internet browsers
`provide intuitive spatial and gazetteer interfaces to the data. The TerraServer demonstrates the scalability of
`Microsoft Windows NT® Server, Enterprise Edition version 4.0 and SQL Server, Enterprise Edition running on Digital
`hardware including the AlphaServer 8400 and StorageWorks™ storage system. The TerraServer is also an
`E-Commerce application. Users can buy the right to use the imagery using Microsoft Site Servers managed by the
`USGS and Aerial Images. This paper describes the TerraServer's design and implementation.
`
`Contents
`The Microsoft TerraServer
`Database Design
`Microsoft TerraServer Hardware
`Assessment
`Summary
`Acknowledgments
`For More Information
`
`The Microsoft TerraServer
`
`https://msdn.microsoft.com/en-us/library/aa226316(v=sql.70).aspx
`
`5/3/2017
`
`Microsoft Corp. Exhibit 1036
`
`

`

`Microsoft TerraServer
`
`Page 2 of 26
`
`Figure 1. The TerraServer hardware
`
`The TerraServer has five terabytes of uncompressed satellite and aerial images of urban areas, compressed to one
`terabyte of database data. It serves these images onto the Internet with a graphical and intuitive user interface. The
`application demonstrates several things:
`
`• Information at your fingertips. This is the most comprehensive world atlas anywhere—and it is available to
`anyone with access to the Internet.
`• Windows NT Server, Enterprise Edition and SQL Server, Enterprise Edition version 7.0 scale. The
`TerraServer fills eight large cabinets: one for the Digital Alpha 8400 processors, and seven cabinets for the 324
`disks—almost three terabytes (TB) of raw disk storage and 2.3 TB of RAID5 storage.
`• Windows NT and SQL Server, Enterprise Edition are excellent for serving multimedia and spatial data
`onto the Internet.
`• Microsoft Site Server Commerce Edition can help sell images over the Internet.
`
`TerraServer is a multimedia database that stores both classical text and numeric data, as well as multimedia image
`data. In the future, most huge databases will be comprised primarily of document and image data. The relational
`metadata is a relatively small part of the total database size. TerraServer is a good example of this new breed of
`multimedia databases.
`
`The Application
`An Interesting Internet Server. TerraServer is designed to be a compelling Internet application. It tries to be
`interesting to almost everyone, everywhere, to be offensive to no one, and to be relatively inexpensive to build and
`operate. It is hard to find data like that—especially a terabyte of such data. A terabyte is nearly a billion pages of
`text—four million books. A terabyte holds 250 full-length movies. It is a lot of data.
`
`Figure 2. World population density
`
`https://msdn.microsoft.com/en-us/library/aa226316(v=sql.70).aspx
`
`5/3/2017
`
`Microsoft Corp. Exhibit 1036
`
`

`

`Microsoft TerraServer
`
`Page 3 of 26
`
`Satellite Images of the Urban World. Pictures have a universal appeal, so it was natural to pick a graphical
`application. Aerial images of the urban world seemed to be a good application. The earth's surface is about 500
`square tera-meters. 75 percent is water, 20 percent of the rest is above 70° latitude. This leaves about 100 square
`tera-meters. Most of that is desert, mountains, or farmland. Less than 4 percent of the land is urban. The TerraServer
`primarily stores images of urban areas. Right now, it has nearly five square tera-meters—and it grows as more data
`becomes available.
`
`Cooperating with the United States Geological Survey (USGS): The USGS has published aerial imagery of many
`parts of the United States. These images are approximately one-meter resolution (each pixel covers one square
`meter). We have a Cooperative Research Agreement (CRADA) with the USGS to make this data available to the
`public. We have loaded all the published USGS data (3 TB raw, 0.6 TB compressed). This is 30 percent of the United
`States. As additional data becomes available, it will be loaded into the TerraServer. This data is unencumbered and
`can be freely distributed to anyone. It is a wonderful resource for researchers, urban planners, and students. The
`picture at left shows a baseball game in progress near San Francisco. You can see the cars, but one-meter resolution
`is too coarse to show people.
`
`Figure 3. A USGS 1-meter resolution image of Candlestick Park near San Francisco
`
`Working with SOVINFORMSPUTNIK (the Russian Space Agency) and Aerial Images. To be interesting to
`everyone everywhere, TerraServer must have worldwide coverage. The USGS data covers much of the continental
`United States. There is considerable imagery of the planet, but much of it either has poor quality (10 meter to 1-km
`resolution), has not been digitized, or is encumbered. SOVINFORMSPUTNIK and their representative, Aerial Images,
`have some of the best data and were eager to cooperate. The Russians and Aerial Images contributed two square
`tera-meters of imagery (1.56-meter resolution). This data is trademarked SPIN-2, meaning satellite-2-meter imagery.
`They intend to deliver an additional 2.4 square terra-meters over the next year.
`
`Figure 4. A SPIN-2 1.6-meter image of Atlanta's Olympic stadium
`
`TerraServer is the largest world atlas. The SOVINFORMSPUTNIK SPIN-2 imagery covers Rome, Athens, Hong Kong,
`New York, Chicago, Seattle, and many other cities. TerraServer has more data in it than all the HTML pages on the
`
`https://msdn.microsoft.com/en-us/library/aa226316(v=sql.70).aspx
`
`5/3/2017
`
`Microsoft Corp. Exhibit 1036
`
`

`

`Microsoft TerraServer
`
`Page 4 of 26
`
`Internet. If printed in a paper atlas, with 500 pages per volume, the information would fill a collection of 2,000
`volumes. It grows by 10,000 pages per month. Clearly, this atlas must be stored online. The USGS data (the three
`square tera-meters) is seven times larger. This data is a world asset that will likely change the way geography is
`taught in schools, the way maps are published, and the way we think about our planet.
`
`TerraServer as a business. Slicing, dicing, and loading the SPIN-2 and USGS data is a continuing process. Today, the
`TerraServer stores a terabyte. Aerial Images, Digital, and Microsoft are operating the TerraServer on the Internet
`(http://terraserver.microsoft.com/). Microsoft views TerraServer as a demonstration of the scalability of Windows NT
`Server and Microsoft SQL Server, Enterprise Edition. Digital views it as a demonstration of their Alpha and
`StorageWorks servers. The USGS is participating as an experiment to present USGS data to a wider audience through
`the Internet. They operate an online store that allows anyone to download copies of the USGS images.
`SOVINFORMSPUTNIK and Aerial Images view TerraServer as a try-and-buy distributor for their intellectual property.
`They make coarse-resolution (8-meter, 16-meter, and 32-meter) imagery freely available. The fine-resolution data is
`viewable in small quantities, but customers must buy the right to use the good imagery. All the SPIN-2 images are
`watermarked, and the high-resolution images are lightly encrypted.
`
`Site Server Commerce Edition, a new business model for the Internet. Aerial Images' business model is likely to
`become a textbook case of Internet commerce. Because they use the Internet to sample and distribute their images,
`Aerial Images has very low distribution costs. This allows them to sell imagery in small quantities and large volumes.
`Microsoft helped USGS and Aerial Images set up Microsoft Site Servers that accept credit-card payments for the
`imagery. A Download button on the image page takes the user to these Site Servers (Microsoft has no financial
`interest in these transactions). You can buy a detailed image of your neighborhood for a few dollars.
`
`https://msdn.microsoft.com/en-us/library/aa226316(v=sql.70).aspx
`
`5/3/2017
`
`Microsoft Corp. Exhibit 1036
`
`

`

`Microsoft TerraServer
`
`Page 5 of 26
`
`Figure 5. Navigation through TerraServer
`
`User Interface to the Microsoft TerraServer
`Navigation through database searches. The TerraServer can be accessed from any Web browser (for example,
`Internet Explorer, Netscape Navigator). Full resolution SPIN-2 imagery requires the Web browser to support Java
`applets. Any Web browser that supports HTML tables and display of Jpeg data can host the TerraServer user
`interface. Navigation can be spatial through a point-and-click map control based on Microsoft's Encarta World Atlas.
`Clients only knowing the place name can navigate textually by presenting a name to the Encarta Virtual Globe
`Gazetteer. The gazetteer knows the names and locations of 1.1 million places in the world. For example, "Moscow"
`finds 28 cities, while "North Pole" finds 5 cities, a mining district, a lake, and a point-of-interest. There are 378 listings
`for San Francisco in the gazetteer. The user can select the appropriate member from the list. The map control
`displays the 40-km map of that area. The user can then pan and zoom with this map application and can select the
`USGS and SPIN-2 images for the displayed area.
`
`https://msdn.microsoft.com/en-us/library/aa226316(v=sql.70).aspx
`
`5/3/2017
`
`Microsoft Corp. Exhibit 1036
`
`

`

`Microsoft TerraServer
`
`Page 6 of 26
`
`Navigation using coverage map. The USGS gave us a shaded relief map (Mercator projection), which includes
`political (state and province) boundaries. We shaded this map green where we have some imagery. Then we built an
`image pyramid: one image for the whole planet, and two levels of zoom that cover a continent and a region on the
`continent. The bottom panels of the previous figure show an example of zooming in on New York City. We added
`this interface last, but it is the most popular way to navigate the database.
`
`Spatial navigation using the map control. A dynamic HTML page allows the client browser to talk with the
`Microsoft Expedia™ travel service map server (http://maps.expedia.com/) that provides the basic features of
`Microsoft Encarta World Atlas and Microsoft Automap® Streets as GIF images. The applet lets users pan and zoom
`over graphic images of the earth and of US street maps. The applet decides what the client wants to see and sends a
`request for that map to the Expedia map server. That server, given the corners of a rectangle and an altitude,
`generates the view of the earth inside that rectangle. It generates a GIF image that is downloaded to the applet in the
`client browser. The map server is provided by MSN™ to any Internet customer. We have wrapped it in our Java
`applet. This application works on Windows, Macintosh, and UNIX clients. Coverage map and spatial access is
`especially convenient for those who do not understand English.
`
`Zooming in and out. The map controls allow the browser to zoom out and see a larger area, or zoom in and see
`finer detail. The coarsest view shows the whole planet. The user can spin the globe to see the other side and place the
`point of interest in the center of the screen. Then the user can zoom in to see fine detail. Where we have street maps
`(Microsoft Automap Streets), the zoom can go all the way down to a neighborhood.
`
`Figure 6. Moving Around: Once you find the spot you are looking for, you can see nearby places by pushing
`navigation buttons to pan and zoom. By doing this, you can "drive" cross-country.
`
`Encarta, USGS, and SPIN-2 Themes. TerraServer has several different views of the earth: the coverage map view, the
`Encarta World Atlas view, the USGS image view, and the SOVINFORMSPUTNIK/SPIN-2 view. We call each of these
`views a theme. The user may switch from one theme to another, perhaps starting with the Encarta theme, then the
`SPIN-2 theme, and then the USGS theme of the same spot. With time, we expect to have multiple images of the
`same spot. Then the user will be able to see each image in turn. Your grandchildren will be able to see how your
`neighborhood evolved since 1990.
`
`Download. If you like the SPIN-2 imagery or USGS image, you can push the DownLoad Image button. That takes
`you to the Aerial Images or USGS Site Server. Both Aerial Images and USGS e-commerce sites run Microsoft's
`
`https://msdn.microsoft.com/en-us/library/aa226316(v=sql.70).aspx
`
`5/3/2017
`
`Microsoft Corp. Exhibit 1036
`
`

`

`Microsoft TerraServer
`
`Page 7 of 26
`
`Commerce Server, which is a component of Microsoft Site Server. That is where the similarity between Aerial Images
`and the USGS e-commerce site ends. The Aerial Images Web site allows Internet users to select one of three sizes for
`a digital image. The user can also select a choice of format (TIFF or JPEG) and can have a photograph printed and
`delivered overnight by Kodak.
`
`The USGS site allows users to download the image viewed on Microsoft TerraServer as a single digital image in JPEG
`format at no charge. The USGS site offers an easy method to purchase one or more of the original data sets used to
`form these images. The original USGS imagery is intended for use by professional geographic information system
`(GIS) users. The Site Server allows you to shop for imagery and quotes you a price. If you want to purchase a high-
`quality digital copy of these images, Site Server asks for your credit card account number, debits it, and downloads
`the images you purchased or ships them to you on the media of your choice.
`
`The USGS and SPIN-2 sites are good examples of selling soft goods over the Internet.
`
`Server Design
`The TerraServer has several components that combine to make a seamless Internet application:
`
`Internet Information Server and Active Server Pages. Clients send requests to the TerraServer's Internet
`Information Server (IIS), built into Windows NT Server 4.0. These requests are passed to Active Server Pages (ASPs)
`programmed in Visual Basic®, Scripting Edition (VBScript). These ASPs send queries to stored procedures in the SQL
`Server, Enterprise Edition database to query the Gazetteer and to fetch image tiles. The ASPs dynamically construct
`the HTML Web pages needed to mosaic the tiles together to make a complete image. The server first returns the
`HTML for the outer frame, and the HTML table referencing the two-dimensional array of tiles. The number of tiles
`displayed in the HTML table is controlled by the user and the current page's image resolution. TerraServer stores
`imagery in 32 m/p (square meters/pixel), 16 m/p, 8 m/p, and full resolution (1 m/p or 1.56 m/p). The user decides if
`they want to see a small, medium, or large view. A full resolution small view page displays 4 image tiles whereas a
`large view page displays 16 image tiles. A 32 m/p Web page ranges from 64 tiles in a small view to 256 image tiles in
`a large view. When the client pans an image, 50 percent of the images on the current frame are moved to another
`location within the frame and the other 50 percent are downloaded from the database.
`
`Figure 7. TerraServer hardware and software
`
`The VBScript program dynamically creates the necessary HTML to render that image. It sends this HTML back to the
`client's browser. The client browser then requests the images needed to fill in the picture. Depending on the image
`
`https://msdn.microsoft.com/en-us/library/aa226316(v=sql.70).aspx
`
`5/3/2017
`
`Microsoft Corp. Exhibit 1036
`
`

`

`Microsoft TerraServer
`
`Page 8 of 26
`
`size the user selects, this can be between 4 and 256 tiles. These URL requests generate between 30 and 500 database
`accesses.
`
`Tiled Image Database. The database stores both the USGS data and the SPIN-2 data as small (10 kilobytes or less)
`tiles compressed with JPEG. Larger images are created as a mosaic of these tiles. This allows quick response to users
`over slow voice-grade phone lines. It also allows the application to pan and zoom across the images.
`
`Gazetteer: The Encarta World Atlas Gazetteer has over a million entries describing most places on earth. All these
`records are stored and indexed by Microsoft SQL Server, Enterprise Edition. Stored procedures look up these names
`and produce an HTML page describing the top 10 hits, with hot links to the images if they are in the TerraServer.
`
`Site Server Commerce Edition. If a client wants to buy some imagery from Aerial Images, the client pushes the
`DownLoad Image button. Site Server uses secure HTTP to authenticate the user and quote the user a price for the
`requested data. The data providers, Aerial Images and the USGS, have built electronic "stores" tailored to their
`existing products and their markets. The sites differ in pricing philosophies and market focus. The Aerial Images site
`is designed for unsophisticated users. Imagery size and formats are designed to be attractive to nonprofessional
`users interested in a digital photo of a small area. The USGS site on the other hand, is targeted towards the USGS'
`traditional market of GIS professionals and data resellers. Both systems use credit cards as authorization for payment.
`Aerial Images' site downloads Digital images immediately over the user's Internet connection. Paper photographs are
`delivered by ground mail or overnight express services. USGS images are distributed by ground mail carriers only.
`
`Terra Server uses Microsoft SQL Server, Enterprise Edition. TerraServer uses the 1998 version of Microsoft SQL
`Server, Enterprise Edition. This version supports larger page sizes, parallel load, backup and restore utilities, has
`better support for multimedia, supports parallelism within queries, and supports much larger databases. TerraServer
`has been a good stress test for SQL Server, Enterprise Edition.
`
`Loading the Database. SOVINFORMSPUTNIK and the USGS delivered the data to us on several hundred tapes. We
`had to sort, reformat, slice, and dice this data before it could be inserted into the database. We wrote several
`programs to do this image processing. We also wrote a load manager that consumes these files and feeds the data
`into the TerraServer using the SQL Server, Enterprise Edition loader APIs (ODBC BCP). Using several parallel streams,
`it loads at approximately 2 MBps. At this rate, the load takes 6 days. The load is constrained more by the scan, slice,
`and dice process than by the SQL Server load rate. Indeed, the database load rate is 15 MBps, eight times faster than
`the load program can produce the data.
`
`Summary. TerraServer is a new world atlas—far larger than any seen before. It is a relatively simple database
`application, but it demonstrates how to build a real Internet application using Windows NT Server Enterprise Edition
`version 4.0 and SQL Server, Enterprise Edition version 7.0, running on Digital AlphaServer 8400 and StorageWorks
`servers.
`
`Database Design
`The TerraServer presents an interesting geo-spatial database design problem. It contains data from three different
`sources represented in different coordinate systems. It has to integrate all this data into a single intuitive user
`interface. This section describes how the data is represented in the database and how it is indexed.
`
`Coordinate Systems
`The earth is not flat. It is not round, either—it is a bumpy oblate spheroid. When measuring the earth at one-meter
`resolution, this becomes a very important issue.
`
`USGS DOQs. The USGS has aerial photographs of most of the United States. It has corrected these aerial
`photographs for elevation and camera optics. The USGS then maps the true image into the Universal Transverse
`
`https://msdn.microsoft.com/en-us/library/aa226316(v=sql.70).aspx
`
`5/3/2017
`
`Microsoft Corp. Exhibit 1036
`
`

`

`Microsoft TerraServer
`
`Page 9 of 26
`
`Mercator (UTM) coordinate system. The resulting digital images are mosaiced into Digital Orthorectified Quarter
`Quadrangles (DOQQs for short). A quadrangle is a one-eighth of a degree square (about 3.5 kilometers on a side).
`The USGS has published paper quadrangle maps for many decades. A DOQQ is one-quarter of a USGS Digital
`Orthorectified Quadrangle (DOQ).
`
`The UTM system. UTM divides the earth into 60 zones. Each zone is two 6º spherical triangles going from the
`equator to the poles. The continental United States occupies nine UTM zones (Alaska and Hawaii add 7 more zones).
`A UTM projection flattens each of these spherical triangles (projects them onto a plane). The meridian of the triangle
`is represented perfectly, but all the other pixel-points are slightly distorted to be trapezoids rather than squares. In
`particular, the pixels at the edges of a zone have north at 3º from the vertical. The UTM system maps latitude lines
`into curves. This is barely noticeable to the eye, but is very noticeable when images that lie on zone boundaries are
`concatenated.
`
`Figure 8. UTM
`
`USGS data uses UTM. We decided to use the USGS coordinate system for the USGS data. To be exact, the USGS
`uses UTM with the NAD83 datum. It would be too much work for us to remap the USGS data into a coordinate
`system that gives a seamless mosaic of the earth. In UTM, each point has a zone number, then a Northing (meters
`from the equator), and an Easting (meters from the west meridian of the triangle). TerraServer USGS images are a
`fixed size—1,800 x 1,200 meters. The TerraServer assigns a unique UGridID to each TerraServer image by
`concatenating the UTM zone with the image's Easting ID (Easting + 400/1800) number followed by a bit interleave of
`the Northing ID (Northing/1200). The bit interleaving causes nearby images to have a common UGridID prefix.
`
`SPIN-2 uses latitude-longitude. The SPIN-2 data is taken from 200 kilometers up. An original SPIN-2 image is a 40
`x 160 km photographic swath taken by a former Russian military satellite. (These are declassified photos. A recent
`
`https://msdn.microsoft.com/en-us/library/aa226316(v=sql.70).aspx
`
`5/3/2017
`
`Microsoft Corp. Exhibit 1036
`
`

`

`Microsoft TerraServer
`
`Page 10 of 26
`
`US-Russian treaty allows Russia to export to the United States.) Each photo has a resolution of 1.56 square meters
`per pixel. We have 2 trillion square meters of these images (0.7 TB). Each 40 x 160 km photograph is scanned into
`four separate 40 x 40 km images because it is too large to scan all at once. The digital scan is separated into four
`separate 20 x 20 km files because Adobe Photoshop cannot rotate an image larger than 30,000 pixels. Aerial Images
`personnel geo-locate five points on one 20 x 20 km image. Photoshop is used to rotate each 20 x 20 km quadrant.
`This creates the appearance of a diamond-shaped photo within a square white canvas. The upper left corner point
`and lower right corner point of the square image are computed. The upper left and lower right corner points of the
`other three images are computed relative to the first image. Our image editing program reads each SPIN-2 20 x 20
`km image and creates TerraServer SPIN-2 images that are 1/48 of a degree wide x 1/96 of a degree high. Pixels from
`adjacent 20 x 20 km images are merged, creating a single TerraServer Spin-2 image. We mapped the SPIN-2 data
`into a latitude-longitude reference system—each image is given a unique Z-Grid ID, which is the interleaving of the
`latitude and longitude of the center point of the TerraServer SPIN-2 image. On earth, there are a total of 298,598,400
`unique ZgridID values. ((360 longitude degrees x 48 "cuts per degree") x (180 latitude degrees x 96 "cuts per
`degree"))
`
`Images are 256-level gray scale JPEG. Browsers generally reduce the number of levels displayed. The images are all
`stored as JPEGs compressed to 80 percent to be faithful to the original image. This typically gives a 5:1 compression.
`
`The image tile pyramid. The images are stored in the database as image pyramids so that users can zoom in and
`out (see Figure 9). An additional constraint is that no image should be much larger than 10 KB. This constraint comes
`from the need to support clients accessing the database through 28.8 KBps modems. It takes about 3 seconds to
`download a 10-KB image. A complete Web page is made of mosaics of these small images.
`
`Tile, thumbnail, browse, and jump images. Large images are first sliced into tiles that are about 10 kilobytes each.
`Each of these tiles covers a tiny area (less than a tenth of a square kilometer). The TerraServer returns a mosaic of
`these tiles to the user on each query. Coarser tile resolutions are stored to support zooming. The data load processes
`mosaic tiles and then dithers them down to wider-panorama images. For USGS data, an 8x8 mosaic is dithered down
`to 8-meter resolution to produce a browse image from the original images. For the SPIN-2 1.56-meter data, a 5x5
`array of tiles is dithered to form the browse image. These browse images are further dithered down to 16-meter
`(thumbnail image) and 32-meter (jump image) resolution images. Because these images have lower resolution, they
`occupy 1/64, 1/256, and 1/1024 of the space of the tile images. That is, they occupy almost no space at all.
`
`Figure 9. The fine-resolution image of the Washington Monument is used to make an 8 x 8 mosaic. It is then
`dithered to an image where one pixel is 8 x 8 meters. This thumbnail image is then dithered down to a 16 x
`16 meter browse image and a 32 x 32 meter jump image. The TerraServer mosaics these small images to
`make zoomed-out images.
`
`https://msdn.microsoft.com/en-us/library/aa226316(v=sql.70).aspx
`
`5/3/2017
`
`Microsoft Corp. Exhibit 1036
`
`

`

`Microsoft TerraServer
`
`Page 11 of 26
`
`Tile 4 x 3 aspect ratio. Tile sizes were chosen to approximately match the 4 x 3 computer display aspect ratio. The
`USGS data follows the USGS coordinate system. In the USGS data set, each tile is exactly 225 x 150 meters. Based on
`visual experiments, we chose a browse image size of 1/48 of a degree wide and 1/96 of a degree high. This translates
`to an area about 1 x 2 km. Further north, the 1/48 of a degree wide shrinks to almost nothing. In the middle latitudes,
`it has the desired 3 x 4 ratio. Each thumbnail is sliced 5 x 5 to make the tile images.
`
`Once a tile size is chosen, all the other sizes are a pyramid derived from that basic unit. Table 1 gives the approximate
`sizes and cardinalities of the data sets.
`
`Table 1. Cardinalities and sizes of the USGS and SPIN-2 data sets as stored in the TerraServer. The SPIN-2 data
`tiles vary in size because the thumbnail images are 1/48 x 1/96 of a degree.
`
`Jump
`
`Browse
`
`Thumbnail
`
`Tile
`
`SPIN-2
`Resolution per Pixel
`
`32 meter
`
`16 meter
`
`8 meter
`
`1.6 meter
`
`44 x 34
`
`88 x 68
`
`176 x 134
`
`167 x 131
`
`Pixels
`
`Area
`
`Image Size (bytes)
`
`Cardinality
`
`GigaBytes
`
`USGS DOQs
`Resolution
`
`Pixels
`
`Area
`
`Image Size (bytes)
`
`Cardinality
`
`GigaBytes
`
`~ 5 km2
`
`~ 5 km2
`
`~ 5 km2
`
`to 239 x 152
`
`~ .4 KB
`
`~ 1.5 KB
`
`~ 6 KB
`
`~ .1 km2
`
`650 k
`
`.24 GB
`
`650 k
`
`1 GB
`
`650 k
`
`4 GB
`
`~ 6 KB
`
`16 m
`
`96 GB
`
`encrypted
`
`32 meter
`
`16 meter
`
`8 meter
`
`1 meter
`
`56 x 37
`
`112 x 75
`
`225 x 150
`
`225 x 150
`
`2.1 km2
`
`2.1 km2
`
`2.1 km2
`
`.03 km2
`
`~ .4 KB
`
`~ 1.7 KB
`
`~ 7 KB
`
`~ 7 KB
`
`1.5 m
`
`.6 GB
`
`1.5 m
`
`2.4 GB
`
`1.5 m
`
`10 GB
`
`96 m
`
`700 GB
`
`Database Themes
`As explained so far, there are two separate data themes: USGS DOQs and SPIN-2 satellite images. Each theme has its
`own set of SQL tables. Each image, along with its metadata, is a record in the database. The data is indexed by its
`geographic coordinates. The database size parameters are summarized in Table 2. Both USGS and SPIN-2 data
`continue to arrive—so these numbers will have increased by the time you read this.
`
`https://msdn.microsoft.com/en-us/library/aa226316(v=sql.70).aspx
`
`5/3/2017
`
`Microsoft Corp. Exhibit 1036
`
`

`

`Microsoft TerraServer
`
`Page 12 of 26
`
`Table 2. The TerraServer database has 790 GB of user data stored in 223 million records. About 200 GB of
`additional space is consumed by overhead (about 25 percent). The remaining space is used for indices,
`catalogs, recovery logs, and temporary storage for queries and utilities. The database has a formatted
`capacity of 2.2 TB.
`
`Total Disk Capacity
`
`Unprotected
`
`After RAID5
`
`324 disks x 9 GB = 2.9 TB
`
`4 x 595 GB volumes = 2.4 TB
`
`Database Size
`
`Area
`
`Gazetteer
`
`USGS
`
`SPIN-2
`
`Bytes
`
`.16 GB
`
`3 sq tera-meters @ 1m (JPEG)
`
`713 GB
`
`2 sq tera meters @ 1.6m (TIFF)
`
`77 GB
`
`Total User Data
`
`5 square tera meters
`
`Overhead Space
`
`Index, Catalog, log
`
`Temp Space
`
`Total DB size
`
`790GB
`
`200 GB
`
`94 GB
`
`100 GB
`
`1.2 TB
`
`Million Records
`
`1.1
`
`198
`
`34
`
`223 million records
`
`Figure 10: The image pyramid gives a zoom-in view of a spot
`
`SPIN-2 Theme. The raw SPIN-2 data is divided into 4 40km x 40km photographs and scanned at 1.56-meter
`resolution. One 40km x 40km photo is picked to be the anchor photograph. Five points are geo-located in the
`anchor photograph. The anchor and its 3 siblings are each quartered (a total of 16 images) and rotated the same
`angle. They rotate so that North is up, optical distortion is minimized, and geo-located pixels in the anchor image are
`accurate to 50 meters. The sibling images are geo-located such that pixels from the siblings can be aligned with the
`anchor image. The SOVINFORMSPUTNIK and Aerial Images do all this work. The data is then sent to Microsoft on 20
`
`https://msdn.microsoft.com/en-us/library/aa226316(v=sql.70).aspx
`
`5/3/2017
`
`Microsoft Corp. Exhibit 1036
`
`

`

`Microsoft TerraServer
`
`Page 13 of 26
`
`GB DLT magnetic tapes. The typical image is 300 MB. It would take three years to download such an image over a
`28.8 modem. We slice and dice these images into 10-KB tiles that can be downloaded in a few seconds. The slice-
`and-dice step produces four products:
`
`• Jumps: JPEG compressed images covering a 1 x 1.3 km area at 32-meter resolution
`• Thumbnails: JPEG compressed images covering a 1 x 1.3 km area at 16-meter resolution
`• Browse: JPEG compressed images covering a 1 x 1.3 km area at 8-meter resolution
`• Tiles: JPEG images that cover a 240 x 300 m area at 1.6-meter resolution.
`
`The key property is that these tiles can be downloaded quickly over a voice-grade telephone line. The tile images are
`lightly encrypted. Each image, along with its metadata (time, place, instrument, and so on), is stored in a database
`record. Each resolution is stored in a separate table. This data can be cross-correlated with the Gazetteer and other
`sources by using the Z-transform (see Endnote). In January 1998, we had 16 million tiles, and 650 thousand browse,
`thumbnail, and jump images. This totals 101 gigabytes of compressed user data. It is 800 GB of uncompressed data.
`Loading continues as more data arrives from the SOVINFORMSPUTNIK.
`
`USGS Theme: The USGS images are handled similarly. They arrive on DLTs from the USGS. The slice-and-dice step
`produces four products:
`
`• Jumps: JPEG compressed images covering a 1 x 1.3 km area at 32-meter resolution
`• Thumbnails: JPEG compressed images covering a 1 x 1.3 km area at 16-meter resolution
`• Browse: JPEG compressed images covering a 1 x 1.3 km area at 8-meter resolution
`• Tiles: JPEG images that cover a 150 x 225 m area at one-meter resolution.
`
`Each image, along with its metadata (time, place, instrument), is stored in a database record. Each resolution is stored
`in a separate table. Today we have 96 million tiles and 1.5 million browse, thumbnail, and jump images. This totals
`about 800 gigabytes of user data. The US is about 9.8 million square kilometers, so this is about 30 percent of the US.
`Important areas have not yet been digitized. The USGS will provide additional data as it becomes available. They plan
`to digitize the entire country by the year 2002.
`
`Logical Database Design for Image Data
`The images are stored in the SQL Server, Enterprise Edition database along with their metadata. The tiles, thumbnails,
`browse, and jump images are kept as SQL image fields as part of relational records. The schema is shown in Figure
`11.
`
`Metadata. All the original metadata for each large image is stored in the OriginalMetaData table. A user can ask the
`TerraServer for the lineage of a particular image and the TerraServer then returns the appropriate record from this
`table. This data describes the data set in detail. The original metadata table has about 100 fields