(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2002/0154214A1
`Scallie et al.
`(43) Pub. Date:
`Oct. 24, 2002
`
`US 2002O154214A1
`
`(54) VIRTUAL REALITY GAME SYSTEM USING
`PSEUDO 3D DISPLAY DRIVER
`
`(76) Inventors: Laurent Scallie, Honolulu, HI (US);
`Cedric Boutelier, Brest (FR)
`d
`ddress:
`ERRON& RONG
`OSTRAGER CHONG & FLAHERTY
`(HAWAII)
`SSESSE 1200
`9
`(US)
`10/011,027
`Nov. 2, 2001
`Related U.S. Application Data
`(60) Provisional application No. 60/244,795, filed on Nov.
`2, 2000.
`
`(21) Appl. No.:
`(22) Filed:
`
`Publication Classification
`
`(51)
`
`Int. Cl. ................................................ H04N 13/04
`
`(52) U.S. Cl. ................................................................ 348/51
`
`ABSTRACT
`
`57
`(57)
`A virtual reality game System and method uses pseudo
`drivers to generate Stereo vision outputs for a 3D Stereo
`Scopic display from game Software normally intended for
`output to a 2D display of a conventional game console or
`PC. The Pseudo Drivers can convert the game data output of
`3D video game software written in different application
`programming interface (API) formats commonly used for
`PC games to “stereo vision', thereby allowing hundreds of
`existing 3D games to be played on a virtual reality game
`system. The intercepted 3D game data can be stored in a 3D
`data recorder for later play back. The 3D game data can also
`be transmitted or downloaded to a remote player through an
`online interface. The intercepted 3D game data can be
`combined with other 3D content through a mixer and dual
`rendering System, which facilitates control of the 3D display
`before, during, and after a game, and particularly when
`Switching between different games. The Pseudo Driver for
`the 3D display can be operated in tandem with other pseudo
`driverS Such as for Stereo Sound and\or directional force
`feedback.
`
`
`
`stereo vision
`head mounted display
`
`IPR2018-01045
`Sony EX1006 Page 1
`
`

`

`Patent Application Publication Oct. 24, 2002 Sheet 1 of 7
`
`US 2002/0154214 A1
`
`FIG 1A
`
`WIDEO GAME
`(GAME ENGINE)
`
`ad-be
`
`ir
`PLAYER
`
`vsnr.
`derrinon-tv,
`PSEUDO DRIVER SYSTEM
`40
`
`
`
`
`
`API
`FUNCTION CALL,
`
`INTERCEPT
`
`
`
`
`
`
`
`GAME API
`
`DRIVERS
`DISPLAY IMAGE
`DATA
`
`14
`
`DISPLAY CARD
`
`D
`
`16
`
`2D DISPLAY MONITOR
`
`R/L STEREOSCOPIC DISPLAY (3D)
`
`IPR2018-01045
`Sony EX1006 Page 2
`
`

`

`Patent Application Publication Oct. 24, 2002 Sheet 2 of 7
`
`US 2002/0154214 A1
`
`FIG. 1B
`
`1 O
`
`VIDEO GAME
`(GAME ENGINE)
`
`AP
`FUNCTION
`CATS
`
`3D GAME
`DATA
`
`PSEUDO API DRIVERS (20)
`
`WRAPPER FOR
`PSEUDO DRIVERS
`
`AP
`FUNCTION
`CALLS
`
`
`
`3D GAME
`DATA
`s- - - - - - - - - - - - s
`(3D RECORDER)
`
`(MIXER)
`
`STEREO
`
`WIEWPOINTS
`
`L VIE
`
`R VIEW
`
`
`
`
`
`RENAMED D, 8.
`ORIGINAL DRIVERS
`(RENDER R IMAGE DATA)
`
`R IMAGE DATA
`
`RENAMED DLL &
`ORIGINAL DRIVERS
`
`L IMAGE DATA
`O
`
`IPR2018-01045
`Sony EX1006 Page 3
`
`

`

`Patent Application Publication Oct. 24, 2002
`
`Sheet 3 of 7
`
`US 2002/0154214 A1
`
`FIG 2B
`
`OPENGL GAME
`(GAME ENGINE)
`
`GAME DATA
`AP CALL
`"opengl32.dll"
`
`WINDOWS opengl32 (pseudo)
`WSYSTEM
`
`EO)
`
`
`
`
`
`
`
`
`
`FIG 2A
`(PRIOR ART)
`
`OPENGL GAME
`(GAME ENGINE)
`
`APT CAL
`"opengl32.d
`
`GAME DATA
`
`opengl32
`
`RENDERING
`FUNCTIONS
`
`. . LoadCpenGL
`Texture
`
`IMAGE DATA
`TO GRAPHICS
`CARD
`
`
`
`TO 2D DISPLAY
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`opengl32.dll
`
`
`
`
`
`RENDERING (STE
`FUNCTIONS
`(d.3d 8 dll)
`. . LoadDirectX8
`Texture
`
`
`
`R/L IMAGE VIEWS
`TO GRAPHICS
`CARDS (DUAL
`RENDERING)
`TO R/L STEREO
`WISION
`DISPLAY
`
`
`
`
`
`
`
`IPR2018-01045
`Sony EX1006 Page 4
`
`

`

`Patent Application Publication
`
`Oct. 24, 2002 Sheet 4 of 7
`
`US 2002/0154214 A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 3A
`(PRIOR ART)
`
`GT, DE GAME
`(GAME ENGINE)
`
`AP CALL
`"glide2x.dll
`
`GAME DATA
`
`RENDERING
`FUNCTIONS
`(Glide)
`. . Toadglide
`Texture
`
`FIG. 3B
`
`GLIDE GAME
`(GAME ENGINE)
`
`"glide2x.d GAME DATA
`APT CALT,
`
`WINDOWS
`VSYSTEM
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`glide2x.dll
`
`RENDERING (STEREO)
`FUNCTIONS
`(Glide)
`. . LoadClide
`Texture
`
`
`
`MAGE DATA TO
`GRAPHICS CARD
`
`
`
`
`
`TO 2D DISPLAY
`
`R/L IMAGE VIEWS
`TO GRAPHICS
`CARDS (DUAL
`RENDERING)
`
`TO R/L STEREO
`VISION
`DISPLAY
`
`
`
`
`
`
`
`IPR2018-01045
`Sony EX1006 Page 5
`
`

`

`Patent Application Publication Oct. 24, 2002 Sheet 5 of 7
`
`US 2002/0154214 A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG 4A
`(PRIOR ART)
`
`DIRECTX GAME
`(GAME ENGINE)
`
`API CALL
`"d 3d 8 d."
`'d 3 dim7 OOd
`
`GAME DATA
`
`d3 dim7 OO
`Or
`d3d 8
`
`
`
`
`
`
`
`
`
`F.G. 4B
`
`DIRECTX GAME
`(GAME ENGINE)
`
`APT CALL
`"d3 d8 dll."
`"d3 dim7 OOd
`
`GAME DATA
`
`li
`
`C
`WINDOWS
`
`d3 dim700 (pseudo)
`WSYSTEM CALT, d3d 8
`FUNCTIONS
`
`d3d 8 (pseudo)
`d3d8 d.
`d3 dim7 o.dll CALL INDEX FOR
`COM CLASS
`
`RENDERING
`FUNCTIONS ( 7 or 8)
`. . Load DirectX
`Texture
`
`
`
`
`
`
`
`RENDERING (STERE
`O)
`FUNCTIONS (d3
`. . Load DirectX
`Texture
`
`IMAGE DATA TO
`GRAPHICS CARD
`
`
`
`TO 2D DISPLAY
`
`R/L IMAGE VIEWS
`TO GRAPHICS
`CARDS (DUAL
`RENDERING)
`
`
`
`TO R/L STEREO
`VISION DISPLA
`
`IPR2018-01045
`Sony EX1006 Page 6
`
`

`

`Patent Application Publication Oct. 24, 2002 Sheet 6 of 7
`
`US 2002/0154214 A1
`
`FIG 5
`
`
`
`stereo vision
`head mounted display
`
`IPR2018-01045
`Sony EX1006 Page 7
`
`

`

`Patent Application Publication Oct. 24, 2002 Sheet 7 of 7
`
`US 2002/0154214 A1
`
`FIG 6
`
`k
`
`
`
`
`
`8:8
`
`s
`
`&
`
`i
`
`s:
`
`t
`$388
`
`$838:3
`3.
`s
`
`338883
`
`8
`
`&
`-
`
`
`
`88:
`
`s
`
`Ssss
`
`
`
`A Radeon 8500 dial Monitor
`
`s
`
`---
`
`:
`
`3
`
`stereo vision
`head mounted display
`
`IPR2018-01045
`Sony EX1006 Page 8
`
`

`

`US 2002/0154214 A1
`
`Oct. 24, 2002
`
`VIRTUAL REALITY GAME SYSTEM USING
`PSEUDO 3D DISPLAY DRIVER
`
`SPECIFICATION
`0001. This U.S. patent application claims the priority
`benefit of U.S. Provisional Application No. 60\244,795 filed
`on Nov. 2, 2000, entitled “Pseudo 3D Driver for Controlling
`3D Video Program', by the same inventors in common with
`the present application.
`
`TECHNICAL FIELD
`0002 This invention generally relates to virtual reality
`game Systems which provide a three-dimensional (3D)
`immersive experience to game players, and more particu
`larly, to a method for using pseudo drivers to create 3D game
`displays for 3D games and applications.
`
`BACKGROUND OF INVENTION
`0.003 Commercial virtual reality games are currently
`played at VR game Stations with one or more players. To
`create an immersive environment without the high cost of
`installing Surrounding wall displays in large room environ
`ments, the commonly used VR game Station typically pro
`vides a VR game that is played by a player wearing
`Stereoscopic goggles or other 3D head-mounted display
`(HMDs) and manipulating a weapon or other action equip
`ment while executing physical motions Such as turning,
`aiming, crouching,jumping, etc., on a platform or cordoned
`Space. The VR games played on conventional VR game
`Stations typically are written for the Specific, often propri
`etary, hardware and operating Systems provided by manu
`facturers for their VR game Stations. As a result, there are
`only a limited number of VR games available for play at
`current VR game Stations.
`0004 Players of VR games often want to play games that
`are popular video games they are used to playing on game
`consoles or PCs. Even though many Video games are written
`to create 3D game effects, the common video game console
`or PC hardware Supports image displays for 2D monitors or
`TV screens. While the 2D displays allow the viewer to view
`the image in Simulated 3D space, it does not provide the
`immersive depth of vision of a true 3D experience. It is as
`if the viewer is Seeing the 3D image with only one eye.
`Popular video games therefore are not used at VR game
`Stations employing Stereoscopic 3D displays unless the
`publishers of those video games have chosen to write
`versions for operation on the hardware and operating SyS
`tems used at VR game Stations of the different manufactur
`CS.
`0005. It would therefore be very desirable to have a VR
`game System in which popular 3D Video games written to be
`displayed on 2D display hardware can be operated to
`provide a 3D Stereoscopic display without having to re-write
`the video game software for the 3D display hardware. It
`would also be very useful for a new VR game system to
`enable other 3D game services for VR game players based
`upon popular video games they want to play on VR game
`Stations.
`
`tion Software intended to provide output to a two-dimen
`Sional (2D) screen display comprises:
`0007) (a) running the application software in its
`normal mode to generate 3D application data output
`which is normally to be sent to an application
`programming interface (API) driver for the 2D
`Screen display;
`0008 (b) intercepting the 3D application data output
`from the application Software and redirecting the
`data to a pseudo driver for generating a 3D Stereo
`Scopic display; and
`0009 (c) using the pseudo 3D display driver to
`generate a 3D Stereoscopic display.
`0010. In a preferred embodiment, the 3D application is a
`3D Video game, and the 3D Stereoscopic display is a Set of
`head-mounted Stereo vision goggles used in a virtual reality
`(VR) game system. The VR game system employs the
`pseudo 3D display driver to convert 3D game data from
`existing 3D video game software intended for 2D screen
`display to right and left Stereoscopic image data for the 3D
`Stereoscopic display. Conversion to Stereo vision requires
`the generation of Specific right and left image viewpoints
`which are combined by human vision to yield an immersive
`3D image. The Pseudo Driver converts the 3D game data
`output of the Video game Software in any of the application
`programming interface (APT) formats commonly used for
`popular video games to an API format that Supports the
`handling of Stereoscopic image outputs, thereby allowing
`hundreds of existing 3D Video games to be played in a
`commercial VR game System. The invention method can
`also be used to generate 3D Stereoscopic displayS for games
`played on Video game consoles or PCS for home use.
`0011. As a further aspect of the invention, the intercepted
`3D game data can be stored by a 3D data recorder for later
`play back. In this mode, a game player can replay a game or
`Scene of a game they previously played, or another player
`can re-enact the game played by another player. The 3D
`game data can also be transmitted or downloaded to a remote
`player through an online interface. This would allow the
`replay of the player's 3D visuals at home or on other
`hardware platforms without the original game Software (like
`replaying previously recorded Video).
`0012. The intercepted 3D game data being re-directed to
`the Pseudo Driver can also be overlaid, edited, or combined
`with other 2D or 3D images through a mixer for real-time
`enhancement of the resulting displayed 3D content.
`Examples include high-score rewards, promotional informa
`tion, and logo animation before, during, and after a game or
`mission.
`0013 The Pseudo Driver for the 3D stereoscopic display
`can also be operated in tandem with other pseudo drivers
`Such as drivers for Stereo Sound and\or directional force
`feedback.
`0014. Other objects, features, and advantages of the
`present invention will be explained in the following detailed
`description of the invention having reference to the accom
`panying drawings.
`
`SUMMARY OF INVENTION
`0006. In accordance with the present invention, a method
`(and System) for operating three-dimensional (3D) applica
`
`BRIEF DESCRIPTION OF DRAWINGS
`0015 FIG. 1A is a block diagram illustrating the overall
`invention method of intercepting 3D game data and using
`
`IPR2018-01045
`Sony EX1006 Page 9
`
`

`

`US 2002/0154214 A1
`
`Oct. 24, 2002
`
`pseudo 3D display drivers for generating a 3D Stereoscopic
`display, and FIG. 1B is a block diagram illustrating a
`preferred method for operation of the pseudo driver through
`the use of the “dl wrapper” method.
`0016 FIG. 2A is a diagram illustrating the conventional
`API function call for a 2D display from a first type of PC
`game (OpenGL) software, as compared to FIG. 2B illus
`trating the pseudo API call for generating a 3D Stereoscopic
`display.
`0017 FIG. 3A is a diagram illustrating the conventional
`API function call for a 2D display from a second type of PC
`game (Glide) software, as compared to FIG. 3B illustrating
`the pseudo API call for generating a 3D Stereoscopic display.
`0.018
`FIG. 4A is a diagram illustrating the conventional
`API call for a 2D display from a third type of PC game
`(DirectX) software, as compared to FIG. 4B illustrating the
`pseudo API call for generating a Stereoscopic display.
`0019 FIG. 5 is a diagram of a virtual reality (VR) game
`System using pseudo 3D display drivers to drive dual
`graphics cards for generating a 3D Stereoscopic display for
`different types of PC game software.
`0020 FIG. 6 is a diagram of a VR game system using
`pseudo 3D display drivers to drive a Single dual-head
`graphics card for generating a 3D Stereoscopic display for
`different types of PC game software.
`
`DETAILED DESCRIPTION OF INVENTION
`0021. In the following description of the invention, a 3D
`application Software generates 3D application data intended
`for rendering to a 2D display, but the 3D application data are
`intercepted and rendered by pseudo drivers for a 3D display
`instead. In a preferred implementation, the 3D application is
`a 3D Video game, and the 3D display is a Stereoscopic
`display device. The advantages of this implementation are
`described in terms of the capability of configuring a com
`mercial virtual reality (VR) game system (with multiple
`pods) to offer players their choice of many popular video
`games in an immersive VR mode with stereo vision. How
`ever, it is to be understood that the principles of the
`invention disclosed herein apply equally to other types of
`games, programs, and 3D applications, including, for
`example, CAD applications, Simulation applications, and the
`like, as well as to other use environments, Such as home use,
`Standalone PCs, networked game Stations, and online (Inter
`net) gaming.
`0022 Referring to FIG. 1A, the basic method and system
`of the present invention is illustrated for playing one of
`many popular 3D Video games that a player may want to
`play in 3D vision. The existing (previously written) 3D
`Video game Software 10 is played by a Player and generates
`a stream of 3D visuals through a game engine that outputs
`3D game data. Video games are written using one of Several
`common Application Programming Interfaces (API) for
`handling the rendering and display functions of the game. In
`a conventional mode (dashed arrows), the 3D game data
`(Series of polygons making up image objects to appear in
`Scenes, and light, shading, and color data) are output with
`API function calls to conventional API drivers 12, which
`render the 3D game data into display image data that are fed
`to a graphics display card 14 and result in a 2D image
`displayed on a 2D display monitor 16.
`
`0023. In the present invention (solid line arrows), the 3D
`game data output of the Video game Software 10 are inter
`cepted and redirected to pseudo API drivers 20 which
`generate right (R) and left (L) Stereoscopic image outputs to
`right and left Stereoscopic display cards 22, 24 that generate
`the resulting 3D Stereoscopic display on a 3D display device
`26. “Stereo vision” refers to immersive visual images which
`provide depth perception to the viewer. Depth perception is
`obtained by delivering appropriate right and left offset
`images to the user's right and left eyes.
`0024. The API function calls intercepted and re-directed
`to the Pseudo API Drivers 20 result in the intercepted 3D
`game data output being processed to RAL image data that can
`be viewed on a 3D display device, Such as VR goggles,
`helmet, or “no glasses required 3D monitor. In order to use
`any of the hundreds of existing PC games, the Pseudo
`Drivers are written to handle the common API formats used
`for PC games, such as Glide (TM), developed by 3dfx
`Interactive, Inc., of Alviso, Calif., OpenGL (TM), developed
`by Silicon Graphics, Inc., (SGI) of Mountain View, Calif., or
`DirectX (TM), distributed by Microsoft Corp., of Redmond,
`Wash.
`0025. As illustrated in FIG. 1B, the invention method
`intercepts and redirects the API function calls and 3D game
`data output from the existing 3D video game software 10 to
`Pseudo API Drivers 20. In the preferred implementation
`shown using the so-called “dll wrapper method (specific
`examples described in detail below), the Pseudo Drivers 20
`consist of a Wrapper 21 which is given the same name in the
`System directory as the dynamic link library (“dill”) for the
`original API drivers (“Original Drivers”), while the original
`dl is renamed under a different name and maintained with
`the Original Drivers. When the video game software is
`initialized, it calls the dll for the API drivers in its usual
`mode. Due to its assumption of the original dll name, the
`Wrapper 21 is called instead of the original dll and drivers
`and effectively intercepts the API function calls and 3D
`game data of the Video game Software. The Wrapper 21
`establishes a Stereo Viewpoints module 22 and sets up
`parallel R and L rendering engines from the renamed origi
`nal dll and drivers, one rendering engine 23 for rendering
`right (R) image data, and the other rendering engine 24 for
`rendering left (L) image data. The Wrapper 21 sends the 3D
`game data to the Stereo Viewpoints module 22 where right
`(R) and left (L) viewpoints are calculated or specified for the
`3D game data, resulting in RView data and LView data. The
`API function calls are directed by the Wrapper 21 to the R
`rendering module with the R view data, resulting in render
`ing the R image data, and to the L rendering module with the
`L view data, resulting in rendering the L image data. The R
`and L image data are then Sent to the R and L display cards
`for the 3D stereoscopic display (see FIG. 1A).
`0026. In the invention, the Pseudo Driver intercepts the
`3D game data between the game and the API. The 3D game
`data can thus be rendered into Stereo vision for any Specified
`Viewpoint.
`0027. In the conventional mode by contrast, the data
`Stream from the game goes to the API which is specific to the
`Video card, and undergoes rendering and transformation to
`an image fixed as 2D. The Pseudo Drivers of the invention
`method intercept the game data Stream and invoke the same
`(or comparable) rendering functions to render the 3D game
`
`IPR2018-01045
`Sony EX1006 Page 10
`
`

`

`US 2002/0154214 A1
`
`Oct. 24, 2002
`
`data into 3D Stereoscopic image data, by generating Specific
`right and left image Viewpoints. The right and left image
`data are Sent as outputs to the display cards 22 and 24, which
`then generate the respective bit-mapped image outputs to
`activate the display elements in the corresponding right and
`left eyepieces of the Stereoscopic display unit 26. In the
`preferred embodiment shown, two Separate display cards are
`used for the two Stereoscopic image feeds for greater pro
`cessing Speed and throughput.
`0028 Computational methods for generating right and
`left Stereoscopic images from given image data are well
`known, for example, as described in "3d Stereo Rendering
`Using OpenGL (and GLUT), by Paul Bourke, November
`1999,
`available
`at
`the
`Internet
`page
`http:\\astronomy. Swin.edu.au\bourkevopengl\Stereogl\. The
`method of determining the right and left eye offset and
`computing corresponding left and right eye images is
`deemed to be conventional and not described in further
`detail herein.
`0029 Referring again to FIG. 1A, an integrated Pseudo
`Driver system can also include a 3D game data recorder 30
`(3D Recorder) for storing the 3D game data for later
`playback, and a mixer 40 for enhancing the 3D content, Such
`as by overlaying, editing, or combining with other 2D or 3D
`images. The 3D Recorder 40 records the 3D game data
`Stream (vertices, polygons, textures, etc.) for Subsequent
`playback without the need to re-access the game Software,
`Such as for providing visuals while debriefing players after
`a game Session or for replaying for a player's personal use.
`The mixer 40 allows other images, 2D or 3D, to be mixed
`or interspersed with the game images. For other 3D content,
`the mixer 40 takes the form of a dual rendering module
`which renders the other 3D content and combines it with the
`game content. It is advantageous to record 3D game data
`with the 3D Recorder between the Pseudo Driver Wrapper
`and the mixer (dual rendering module), because all API
`types will have been converted into the chosen 3D image
`data format (DirectX 8, as explained below). Using data
`compression techniques, the large amount of data can be
`minimized and Stored to disk. The data Stream can be played
`back by Simply Sending it to the dual rendering module. If
`the data stream is sent to the 3D Recorder between the game
`and the Pseudo Drivers, then the game data can be played
`back Simply by Sending it to the corresponding API.
`0.030. Because of the separation between the Wrapper 21
`and the mixer (dual rendering module) 40, the mixer can
`always be running. This allows the System total control of
`the display at all times, and avoids any lapse in the display
`if, for example, control is Switched to another game. When
`the next game is run, the API Wrapper called by the new
`game re-connects with the dual rendering module.
`0031) Use of Existing Game Software in VR Systems
`0.032
`State-of-the-art first person games are composed of
`a “game engine', an object-oriented Scriptable logic, and
`game "levels. The game engine is the essential technology
`that allows for 3D graphics rendering, Sound engine, file
`management, networking Support and all other aspects of the
`core application. The content of the game sits on top of the
`rendering engine in the form of Scripts and levels basically
`Setting up the Series of Scenes and actions ("world map')
`forming the Visual environment and the logic within it.
`0.033 Tools provided by game developers are available
`for modification of the Scripted logic of the various objects
`
`in the World map, as well as generation of new environ
`ments. For PC games, this allows for new content to be
`created by game developerS and the life cycle of the game
`to be significantly greater. The current trend in game devel
`opment is to license a specific game engine and allow game
`developerS to focus on content creation, the concept and
`implementation of the levels, Sounds, models, textures and
`game logic. Using those editing tools, customized game
`environments can be produced, characters created, weapons
`and game objects designed, and Special game logic imple
`mented to create the desired game content.
`0034 Conventional 3D video games are written to be run
`on conventional hardware and operating Systems for display
`on a 2D monitor, and thus the conventional experience is
`basically 2D. The 3D game data executes function calls to
`conventional API drivers for the game that result in a 2D
`Screen image being generated. The conventional game Sys
`tem renders a 3D Scene as a centered 2D image as if the user
`were viewing it with one eye. It is desirable to use existing
`3D games for play in VR systems that engage players with
`a 3D Stereo vision display for a more immersive game
`experience. Since the existing games output 3D game data,
`the 3D game data can be converted to a 3D display.
`However, mere connection of a 3D monitor to a standard 3D
`game like Quake3 (TM), distributed by Activision,
`Inc.,
`, Calif., would not yield a Stereo vision image.
`Doubling a centered image using 3D display hardware also
`would not yield a Stereo image. Only the generation of
`Specific right and left image viewpoints for Stereo vision will
`yield a correct Stereo image on a 3D display unit.
`0035) 3D display technology includes, but is not limited
`to, HMDs, no-glasses-required monitors, LCD glasses, and
`hologram display units. Typically, all of these hardware
`types require two separate 2D input images, one for each
`eye. Each new type of 3D display technology comes with its
`own 3D format. Typically, they conform to one of the
`following standard formats (from highest quality to lowest
`quality): Separate right and left (RVL) images; frame Sequen
`tial images; Side-by-side (left\right) images; top-and-bottom
`(over\under) images; or field Sequential (row interleaved)
`image Signals.
`0036) The highest quality stereo vision signal is simply
`two separate R\L image Signals. The remaining methods use
`Some method of compression to pack both left and right
`Signals into a single Signal. Because of this compression, and
`overloading of a Single Signal, the Stereo vision image
`quality is lowered, and\or the frame rate is lowered. The
`lower quality, “single Signal' methods are typically used by
`lower-priced StereoVision hardware, like LCD glasses. Some
`hardware vendors, Such as nVidia Corp., of Santa Clara,
`Calif., have recently provided Support for Single-signal,
`Stereo vision formats. For example, the nVidia Stereo vision
`drivers are contained within the nVidia video card-specific
`driver, nvolisp.drV. The nVidia driver effectively converts a
`3D game written for DirectX or OpenGL to be viewable in
`Stereo vision using any Single-signal 3D device that is
`connected to the nVidia video card. However, these card
`Specific drivers only work if the manufacturer's Video card
`is used. Conventional hardware manufacturers do not Sup
`port card-independent high-end, Separate right and left
`image Signals.
`0037 Another important aspect of the invention is the
`interception of the data Stream at the game-API level.
`
`IPR2018-01045
`Sony EX1006 Page 11
`
`

`

`US 2002/0154214 A1
`
`Oct. 24, 2002
`
`Conventional stereoVision drivers are established between
`the API and the video card, and the code existing between
`the API and the Video card requires hardware-Specific code.
`Drivers on that level need to be made by the manufacturer
`of the Video card hardware, which is a drawback in a game
`System that offers many different games using the same
`Video card hardware. Another drawback is that the data has
`already undergone a 3D game data to 2D image data
`transformation, and is therefore fixed as 2D. Once the data
`are converted to 2D, the 2D data can be converted to
`StereoVision only with "leSS Visually accurate' mathematics.
`0.038. In the preferred embodiment of the invention, two
`Separate video cards 22 and 24 are used for the Separate right
`and left signal inputs of high-end 3D display devices.
`Doubling the number of video cards allows for the right and
`left Stereo image to be rendered Separately and Simulta
`neously. This avoids the typical 2X Slowdown required to
`display stereo rather than mono. The Pseudo Driver thus
`allows a normal 3D game to power two Video cards, which
`in turn can power high-end 3D display hardware such as V6
`or V8 (TM) StereoVision Head Mounted Displays, distrib
`uted by Virtual Research Systems, Inc., of Santa Clara,
`Calif., Visette (TM) StereoVision Head Mounted Display,
`distributed by Cyber Mind, Ltd., of Leicestershire, UK,
`Datavisor (TM) StereoVision Head Mounted Display, dis
`tributed by N-Vision, Inc., of McLean, Va., or DTI 2015XLS
`or 2018XLQ (TM) Stereovision Monitor, distributed by
`Dimension Technologies, Inc.
`0039) Pseudo 3D Display Drivers
`0040. In the present invention, the 3D game data output
`of existing game Software are intercepted and re-directed to
`Pseudo Drivers for 3D display in place of the conventional
`API drivers for 2D display. The Pseudo Drivers execute the
`Same or comparable image rendering functions but generate
`the Specific right and left image viewpoints required by 3D
`display devices. The Pseudo Drivers only convert the 3D
`game output of the game Software and do not affect or
`manipulate the game Software itself. Thus, the Pseudo
`Drivers can produce a 3D display from conventional 3D
`game Software without requiring access to or modification of
`the game Source code.
`0041 3D display technology has developed to offer very
`high resolution and wide field of view. When used with a
`head mounted display unit (HMD) which allows direct head
`tracking, VR systems can offer a very immersive Virtual
`reality experience for the player. Other 3D display devices
`that may be used include 3D monitors, such as the DTI3D
`(TM) monitor distributed by Dimension Technologies, Inc.,
`of Rochester, N.Y., which delivers a stereo vision image
`without requiring the use of Stereoscopic glasses. Most new
`3D display technology can be hooked up to games running
`on standard Intel-based PCs with the Microsoft Windows
`(TM) operating System.
`0.042
`Typical graphics API's have some 400 functions
`that the game program can call to render 3D polygonal
`Scenes. These functions, generally Speaking, have names
`like LoadTexture, SetTexture, RenderPolygons, Display
`Image, etc. All of the API's functions are held in a dynamic
`link library (dll). The API's.dll is stored in the computer's
`C:\Windows\System directory. Depending on which API
`format it is written for, a game will automatically load the
`appropriate .dll Stored in the System directory, and the
`
`functions contained within are used to render the game's 3D
`world map to the 2D screen. The API converts the data
`internally, and forwards the data to the Video card-specific
`driver. The driver optionally modifies the data further into a
`format specific to the current video card hardware. The
`video card renders the data to the screen in the form of
`textured polygons. The final image appears on the user's
`monitor as a 2D projection of the 3D world map.
`0043. The Pseudo Driver of the present invention inter
`cepts the data being sent from the game to the API. The
`Simplest method to do this borrows from a technique called
`“.dll wrapping.
`0044) In this method, a “Pseudo API” is named and
`substituted for the usual original API for which the game
`issues the display function calls. That is, the Pseudo API
`assumes the identity of the usual API's.dll that the game is
`looking for. This substitution is done at the installation level
`for the VR system by storing the Pseudo API in the System
`directory in place of the original API. When the game
`executes function calls for the API, the Pseudo API is called
`and intercepts the 3D game data. The data Stream between
`the game and the rendering API consists of thousands of
`Vertices, polygons, and texture data per frame. The Pseudo
`API then either executes calls, or issues Subcalls to the
`original APIs which are set up to be running in the back
`ground, for the usual rendering functions, then passes the
`rendered data to the Pseudo Driver matched to the type of
`3D display unit used in the VR system. The Pseudo Driver
`generates the card-independent R\L Stereoscopic image Sig
`nals which are passed as inputs to the 3D display unit.
`0045 Example: Pseudo OpenGL Driver
`0046) Many popular PC games are written for OpenGL
`API, such as Quake3. As illustrated in FIG. 2A (Prior Art),
`the game run in conventional PC-based mode initializes with
`an API call for the OpenGL dynamic link library, called
`“openg132.dll, stored in the C:\Windows\System directory.
`The game Software loads the opengl32.dll, then sends the
`Stream of game data generated by play of the game to
`opengl32.dll for linkage to the appropriate API drivers for
`rendering the game's Series of Scenes to the 2D Screen. The
`API drivers render the game data to image data and Sends the
`image data to the graphics card used by the API to drive the
`2D display.
`0047. As shown in FIG.2B, a Pseudo OpenGL Driverhas
`awrappernamed “openg132.dll” is substituted in the System
`directory in place of the OpenGL.dll formerly of that name.
`When Quake3 is run, it calls for the “openg132.dll” and
`binds with the Pseudo OpenGL Wrapper that was substi
`tuted. In this case, the OpenGL API is never actually
`initialized; in fact, it is not needed on the machine at all. The
`Pseudo OpenGL Driver linked to the pSuedo OpenGL wrap
`per pretends to be the OpenGL driver, however, all the data
`Sent to it is converted into a format that can be rendered for
`Stereo vision by a dual rendering System for the dual R\L
`Stereoscopic image outputs. DirectX 8 is used as the ren
`dered data format Since it can Support the use of multiple
`outputs to multiple graphics cards. For about 370 functions,
`Some translation and\or redirection is required. Generally
`Speaking, only about 20% of the functions are