US008206218B2
`
`(12) United States Patent
`US 8,206,218 B2
`(10) Patent N0.:
`
` Gutierrez Novelo (45) Date of Patent: Jun. 26, 2012
`
`
`(54)
`
`3D VIDEOGAME SYSTEM
`
`(56)
`
`References Cited
`
`(75)
`
`Inventor: Manuel Rafael Gutierrez Novelo,
`NueVa sama Mar1a(MX)
`
`U.S. PATENT DOCUMENTS
`4,559,555 A
`12/1985 Schoolman
`
`(73) Assignee: TDVision Corporation S.A. De C.V.,
`Col. Nueva Santa Maria (MX)
`
`(Continued)
`FOREIGN PATENT DOCUMENTS
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 168 days.
`
`(21) Appl. N0.: 12/7103191
`
`(22)
`
`Filed;
`
`Feb_ 223 2010
`
`(65)
`
`Prior Publication Data
`
`US 2010/0151944 A1
`
`Jun. 17, 2010
`
`Related US. Application Data
`
`DE
`
`198 06 547 A
`
`1 1/ 1998
`
`-
`(commued)
`
`OTHER PUBLICATIONS
`
`Akutsu, et a1., Stereoscopic DisplayWhich Shows 3D Natural Scenes
`Without Contradiction of Accommodation and Convergence, SPIE,
`2005, vol, 5664, pp. 480—487.
`
`(Continued)
`
`(63) Continuation of application No. ll/47l,280, filed on
`Jun. 19, 2006, now Pat. No. 7,666,096, which is a
`continuation of application No. PCT/MX03/00112,
`filed on Dec. 19, 2003.
`
`Primary Examiner iArthur 0- Hall
`(74) Attorney, Agent, or Firm iKnobbe Martens Olson &
`Bear LLP
`
`(51)
`
`Int- Cl-
`(2006.01)
`A63F 9/24
`(52) US. Cl.
`................ 463/30; 463/32; 463/34; 463/43;
`273/461; 345/419; 345/539; 345/653; 345/654;
`348/42; 348/47; 348/51
`(58) Field Of Classification Search .................. 463/175,
`463/779, 30734, 36739, 40413, 49757; 273/108.1,
`273/127 R, 148 R, 148 B, 309, 317.1, 340,
`273/343, 348, 3617367, 461; 345/l.lil.3,
`345/21723, 3173.4, 24, 419, 4677469,
`345/473, 539, 5437544, 625, 636, 638, 6537656,
`345/6647666, 6827683, 686, 949950, FOR. 139,
`345/FOR. 153; 348/14.15, 39, 42, 4752,
`348/115, 117, 121, 1357137, 141, 211.2,
`348/2ll.4, 211.772118, 211.14, 211.99,
`348/576, 5887589, 719, 721, 734, El3.004,
`348/El3.0647El3.067; 375/240.157240.l6,
`375/240.25; 434/37738, 43414, 69, 118,
`434/240, 2567257
`See application file for complete search history.
`
`(57)
`
`ABSTRACT
`
`A 3D videogame system capable of displaying a left-right
`sequences through a different, independent VGA or Video
`channel, with a display device sharing a memory in an
`immerse manner. The system has a videogame engine eon-
`trolling and validating the image perspectives, assigning tex-
`tures, lighting, positions, movements and aspects associated
`with each object participating in the game; creates left and
`right backbuffers, creates images and presents the informa-
`tion in the frontbuffers. The system allows handling the infor-
`mation of data associated to the xyz coordinates of the
`object’s image in real-time, increases the RAM for the left-
`right backbuffer, with the possibility to discriminate and take
`the corresponding backbuffer, whose information is sent to
`the frontbuffer or additional independent display device shar-
`ing a memory in an immerse manner.
`
`19 Claims, 13 Drawing Sheets
`
`Set lafl View
`
`410
`
`Draw in left backbutfer
`
`image representation
`in Iefi front buffer
`
`412
`
`
`.
`
`
`
`
`
`
` End
`
`
`
`
`subroutine
`417
`
`Calculate right pair
`coordinates
`
`Draw in backbuffer
`according to left camera
`
`Information display
`in right frontbuffer
`
`4’14
`
`415
`
`4‘16
`
`
`
`
`
`
`|PR2018-01045
`
`Sony EX1001 Page 1
`
`IPR2018-01045
`Sony EX1001 Page 1
`
`

`

`US 8,206,218 B2
`
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`Eichenlaub, Passive Method of Eliminating Accommodation/Con-
`vergence Disparity in Stereoscopic Head Mounted Displays, SPIE,
`2005, vol. 5665, pp. 517-529.
`Fehn, Depth-Image-Based Rendering (DIBR), Compression and
`Transmission for a New Approach on 3D-TV, SPIE, 2004, vol. 5291,
`pp. 93-104.
`Hendriks, et al., Real Time Synthesis of Digital Multi Viewpoint
`Stereoscopic Images, SPIE, 1999, vol. 3639, pp. 266-276.
`Holliman, Mapping Perceived Depth to Regions of Interest in Ste-
`reoscopic Images, SPIE, 2004, vol. 5291, pp. 117-128.
`Hur, et al., A Software-Based Minimum-Time Vergence Control
`Scheme for a Parallel-Axis Stereoscopic Camera, SPIE, 2004, vol.
`5599, pp. 32-40.
`Jin, et al., Creating a Comfortable Stereoscopic Viewing Experience:
`Effects of Viewing Distance and Field of View on Fusional Range,
`SPIE, 2005, vol. 5665, pp. 10-21.
`Kwon, et al., Automatic control of parallel stereoscopic camera by
`disparity compensation, SPIE, 2003, vol. 5006, pp. 417-423.
`Kwon, Automatic Vergence and Focus Control of Parallel Stereo-
`scopic Camera by Cepstral Filter, Journal of Electronic Imaging,
`2004, vol. 13(2), pp. 376-383.
`Lee, et al., Parallel-Axis Stereoscopic Camera with Vergence Control
`and Multiplexing Functions, SPIE, 2003, vol. 5006, pp. 389-398.
`Lee, et al., Adaptive Stereoscopic Image Conversion of 2D Image,
`SPIE, 2001, vol. 4471, pp. 66-72.
`Lemmer, et al., Enhancement of Stereoscopic Comfort by Fast Con-
`trol of Frequency Content with Wavelet Transformation, SPIE, 2003,
`pp. 283-291.
`Nam, et al., Development of the Real-Time Stereoscopic Error Cor-
`rector and Convergence Controller, SPIE, 2006, vol. 60550H-l-9.
`Park, Stereoscopic Imaging Camera with Simultaneous Vergence and
`Focus Control, Opt. Eng., 2004, 43(12) 3130-3137.
`Schowengerdt, et al., Stereoscopic Retinal Scanning Laser Display
`with Integrated Focus Cues for Ocular Accommodation, SPIE, 2004,
`vol. 5291, pp. 366-376.
`Schowengerdt, et al., True 3-D Scanned Voxel Displays Using Single
`or Multiple Light Sources, Journal of the SID, 2006, Journal of the
`SID, 2006, 14/2, pp. 135-143.
`Shibata, et al., Examination of Asthenopia Recovery Using Stereo-
`scopic 3-D display with Dynamic Optical Correction, SPIE, 2006,
`vol. 6055, pp. 60550E-1-8.
`Shibata, et al., Stereoscopic 3-D Display with Dynamic Optical Cor-
`rection for Recovering from Asthenopia, SPIE, 2005, vol. 5664, pp.
`19
`Sun, et al., Evaluating Methods for Controlling Depth Perception in
`Stereoscopic Cinematography, SPIE, 2009, vol. 7237 pp. 723701-1-
`12.
`Tam, et al., Depth Image Based Rendering for Multiview Stereo-
`scopic Displays: Role of Information at Object Boundaries, SPIE,
`2005, vol. 6016, pp. 601609-1-11.
`Tzovaras, et al., Object-Based Coding of Stereo Image Sequences
`using Joint 3-D Motion/Disparity Segmentation, 1995, SPIE, vol.
`2501, pp. 1678-1689.
`Um, et al ., Investigation ofthe Effect of Disparity-Based Asymmetri-
`cal Filtering on Stereoscopic Video, SPIE, 2004, vol. 5150, pp. 110-
`1 18.
`Ware, et al., Algorithm for Dynamic Disparity Adjustment, SPIE,
`1995, vol. 2409, pp. 150-156.
`Watt, et al., Achieving near-correct focus cues in a 3-D display using
`multiple image planes, SPIE, 2005, vol. 5666, pp. 393-401.
`Gotoh, H. “Gotoh Hiroshige’s Monthly Report,” DOS/V Power
`Report, Japan, Impress Corporation, Jul. 1, 2003, vol. 13, No. 7, pp.
`125-126.
`Scribner, et al., The Effect of Stereoscopic and Wide Field of View
`Conditions on Teleoperator Performance, Army Research Labora-
`tory, Mar. 1998, pp. l-40.
`Heath, Jenny, Virtual Reality Resource Guide, Al Expert, May 1994,
`vol. 9, pp. l-45.
`Rodriguez, et al., Full Window Stereo, J. Mol. Graphics Mod., 1999,
`vol. 17, pp. 319-314.
`Liao, et al., The Design and Application of High-Resolution 3D
`Stereoscopic Graphics Display on PC, The 8th International Confer-
`ence in Central Europe on Computer Graphics, Visualization and
`
`|PR2018-01045
`
`Sony EX1001 Page 2
`
`4/1989 Nishiya ......................... 702/152
`4,825,393 A *
`9/1989 Hiraoka
`4,870,600 A
`5/1990 Geshwind et a1.
`4,925,294 A
`10/1990 Ohtomo et al.
`4,962,422 A
`10/1994 Ashizakiet al.
`5,355,181 A
`7/1995 Hines
`5,430,474 A
`11/1995 Furness, III et al.
`5,467,104 A
`4/1996 Garcia
`5,510,832 A
`6/1996 Johnson-Williams et al.
`5,523,886 A
`1/1997 O’Neill
`5,594,843 A
`5,694,530 A * 12/1997 Goto ............................. 345/419
`5,717,415 A *
`2/1998 Iue etal.
`........................... 345/8
`5,734,807 A
`3/1998 Sumi
`5,745,126 A
`4/1998 Jain etal.
`5,751,927 A
`5/1998 Wason
`.................... 345/419
`5,801,705 A *
`9/1998 Kato etal.
`5,801,717 A
`9/1998 Engstrometal.
`5,867,210 A
`2/1999 Rod
`5,872,590 A
`2/1999 Aritake etal.
`............... 351/201
`5,877,840 A *
`3/1999 Yamadaetal.
`5,905,499 A
`5/1999 McDowallet al.
`5,929,859 A
`7/1999 Meijers
`5,973,704 A
`10/1999 Nishiumiet a1.
`5,976,017 A
`11/1999 Omoriet a1.
`5,982,375 A * 11/1999 Nelson et al.
`5,986,667 A
`11/1999 Jevans
`6,005,607 A
`12/1999 Uomoriet al.
`6,031,564 A
`2/2000 Maetal.
`6,104,402 A *
`8/2000 Goddard et al.
`6,108,029 A
`8/2000 Lo
`6,151,060 A
`11/2000 Tabata
`............... 345/630
`6,175,371 B1*
`1/2001 Schoulz et al.
`6,352,476 B2
`3/2002 Miyamoto et al.
`6,384,859 B1
`5/2002 Matsumoto et al.
`6,388,666 B1
`5/2002 Murray
`6,466,206 B1
`10/2002 Deering
`6,466,208 B1
`10/2002 Yet et al.
`6,496,183 B1
`12/2002 Bar-Nahum
`6,496,187 B1
`12/2002 Deering et al.
`6,496,598 B1
`12/2002 Harman
`6,501,468 B1* 12/2002 Kaji
`.............................. 345/419
`6,515,662 B1
`2/2003 Garland
`6,556,195 B1
`4/2003 Totsuka et al.
`6,559,844 B1
`5/2003 Alamparambil
`6,573,928 B1
`6/2003 Jones et al.
`6,583,793 B1
`6/2003 Gould et al.
`6,614,927 B1
`9/2003 Tabata
`6,753,828 B2*
`6/2004 Tuceryan et al.
`6,760,020 B1*
`7/2004 Uchiyama et al.
`6,760,034 B2
`7/2004 Prache
`6,765,568 B2
`7/2004 Swift et al.
`6,816,158 B1
`11/2004 Lemelson et al.
`6,985,162 B1
`1/2006 Schinnerer et al.
`7,219,352 B2
`5/2007 Estrop
`7,254,265 B2
`8/2007 Naske et a1.
`2002/0075286 A1
`6/2002 Yonezawa et al.
`2002/0154214 A1
`10/2002 Scallie et al.
`2003/0112327 A1
`6/2003 Jeong et al.
`8/2003 Perkins
`2003/0152264 A1
`9/2003 Uchiyama
`2003/0179198 A1
`FOREIGN PATENT DOCUMENTS
`2-109493 A
`4/1990
`07-296185
`11/1995
`8-149519 A
`6/1996
`09-139957
`5/1997
`09-237353
`9/1997
`11-509998
`8/1999
`2000-020757
`1/2000
`2002-519792
`7/2002
`2003-067784
`3/2003
`
`................. 345/419
`
`.............. 345/419
`
`................. 345/8
`
`.
`. 345/419
`
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`
`OTHER PUBLICATIONS
`
`Ames, et al., Development of a Miniaturized System for Monitoring
`Vergence During Viewing of Stereoscopic Imagery Using a Head
`Mounted Display, SPIE, 2004, vol. 5291, pp. 25-35.
`
`IPR2018-01045
`Sony EX1001 Page 2
`
`

`

`US 8,206,218 B2
`
`Page 3
`
`interactive Digital Media 2000, Plzen (Pilsen), Czech Republic, Feb.
`7-11, 2000, pp. 1-7 [retrieved from http://Wscg.zcu.cz/Wscg2000/
`PaperLZOOO/RSPdfl
`Bourke, Paul, 3D Stereo Rendering Using OpenGL (and GLUT),
`Nov. 1999 (7 pages).
`
`Microsoft DirectX 8.1 (C++), Programmers Guide, available from
`wwwsc.ehu.es/ccwgamao/docencia/Material/hoy/Direct3D,
`Jun.
`17, 2002 (406 pages).
`
`* cited by examiner
`
`|PR2018-01045
`
`Sony EX1001 Page 3
`
`IPR2018-01045
`Sony EX1001 Page 3
`
`

`

`US. Patent
`
`Jun. 26, 2012
`
`Sheet 1 of 13
`
`US 8,206,218 B2
`
`
`
`|PR2018—O1045
`
`Sony EX1001 Page 4
`
`IPR2018-01045
`Sony EX1001 Page 4
`
`

`

`US. Patent
`
`Jun. 26, 2012
`
`Sheet 2 of 13
`
`US 8,206,218 B2
`
`m
`
`game
`
`2.,
`
`graphics
`
`I graphics
`
`2D source
`code
`
`30 source
`code
`
`I
`
`Effects
`code
`
`Music
`code
`
`H
`
`23
`
`24
`
`25
`
`27
`
`|PR2018—01045
`
`Sony EX1001 Page 5
`
`IPR2018-01045
`Sony EX1001 Page 5
`
`

`

`US. Patent
`
`Jun. 26, 2012
`
`Sheet 3 of 13
`
`US 8,206,218 B2
`
`
`
`|PR2018—01045
`
`Sony EX1001 Page 6
`
`IPR2018-01045
`Sony EX1001 Page 6
`
`

`

`U.S. Patent
`
`Jun. 26, 2012
`
`Sheet 4 of 13
`
`US 8,206,218 B2
`
`Videogame
`software
`
`impiementation
`
`Programming
`language: C, C++.
`Visual Basic, others
`
`
`
`Videogame
`source code
`
`
`
`
`Game logic,
`characteristics,
`
`objects, sounds,
`
`events
`
`
`Actions and
`events handling
`
`Option 1:
`AP! calis
`OpenGL
`
`Option 2:
`API calls
`Directx
`
`
`
`40
`
`‘1
`
`42
`
`~43
`
`4‘
`
`Windows AP!
`
`
`
`Fig. 4
`
`|PR2018-01045
`
`Sony EX1001 Page 7
`
`IPR2018-01045
`Sony EX1001 Page 7
`
`

`

`US. Patent
`
`Jun. 26, 2012
`
`Sheet 5 of 13
`
`US 8,206,218 B2
`
`
`Set left bUffer
`
`
`TDVision
`
`technology?
`
`
`
`Set right buffer
`
`
`
`
`401
`
`.
`
`End Subroutine
`
`Fig. 4a
`
`|PR2018—01045
`
`Sony EX1001 Page 8
`
`IPR2018-01045
`Sony EX1001 Page 8
`
`

`

`US. Patent
`
`Jun. 26, 2012
`
`Sheet 6 of 13
`
`US 8,206,218 B2
`
`Set left View
`
`-
`
`éfln
`
`Draw in left backbuffer
`
`41 1
`
`Image repreSentation
`in left front buffer
`
`'
`
`'
`
`442
`
`
`
`,,
`
`Calculate right pair
`
`
`Draw in backbufféf'
`
`
`according to ieft camera
`
`41%
`
`Informatlon display
`in rightfrontbuffer
`
`" 4-6
`:
`1
`
`End
`
`subroutine
`
`:51?
`
`Fig. 4b
`
`|PR2018—01045
`
`Sony EX1001 Page 9
`
`IPR2018-01045
`Sony EX1001 Page 9
`
`

`

`US. Patent
`
`Jun. 26, 2012
`
`Sheet 7 of 13
`
`US 8,206,218 B2
`
`
`
`Input/Output driver
`
`Graphics driver
`
`65
`
`58
`
`,
`
`
`
`.
`
`Audio driver
`
`,
`
`54
`
`as
`
`.
`
`Gamepad
`Joystick
`
`57
`
`Fig. 5a
`
`,.
`
`_
`
`.,
`
`59
`
`68b
`
`|PR2018—01045
`
`Sony EX1001 Page 10
`
`IPR2018-01045
`Sony EX1001 Page 10
`
`

`

`US. Patent
`
`Jun. 26, 2012
`
`Sheet 8 of 13
`
`US 8,206,218 B2
`
`560
`
`‘
`
`-
`
`Application
`
`H‘WQMMWHW #muwwmmw HM“
`
`compiler needed
`
`501
`
`562
`
`.
`
`'
`
`Graphics
`parts
`
`Graphics hardware
`Change to TDVision
`
`Fig. 5b
`
`|PR2018—O1045
`
`Sony EX1001 Page 11
`
`IPR2018-01045
`Sony EX1001 Page 11
`
`

`

`US. Patent
`
`Jun. 26, 2012
`
`Sheet 9 of 13
`
`US 8,206,218 B2
`
`TDVision with DirectX 30 Algorithm
`
`€00
`
`Load surface and texture information
`
`601
`
`Load meshes information
`
`“2' Create TDVision backbuffer
`
`603 Apply initial coordinates
`
`‘3‘“ Apply game iogio
`
`5"“ Artificial intelligence validation
`
`“3 . Positions calculation
`
`3‘” Coilisions verification
`
`603 Draw information in TDVision backbuffer
`and display on screen
`
`Fig. 6
`
`|PR2018-01045
`
`Sony EX1001 Page 12
`
`IPR2018-01045
`Sony EX1001 Page 12
`
`

`

`US. Patent
`
`Jun. 26, 2012
`
`Sheet 10 of 13
`
`US 8,206,218 B2
`
`OpenGL Aigorlthm
`
`Display flow
`
`70
`
`
`
` 73
`
`
`
`
`
`Fig. 7a
`
`|PR2018—01045
`
`Sony EX1001 Page 13
`
`IPR2018-01045
`Sony EX1001 Page 13
`
`

`

`US. Patent
`
`Jun. 26, 2012
`
`Sheet 11 of 13
`
`US 8,206,218 B2
`
`Algorithm
`
`700
`
`701
`
`
`
` GLU
`OpenGL
`
`utility
`library
`
`
`
` m
`
`i Windows program
`
`m
`
`m
`
`|PR2018—O1045
`
`Sony EX1001 Page 14
`
`IPR2018-01045
`Sony EX1001 Page 14
`
`

`

`US. Patent
`
`Jun. 26, 2012
`
`Sheet 12 of 13
`
`US 8,206,218 B2
`
`81
`
`Clean backbuffer
`m..."
`......_ damn-l
`
`80
`TDVision OpenGL
`
`82
`
`Getpolntar for
`backbufier
`
`I
`
`‘
`
`algorlthm \ ‘ m "
`
` g .
`
`backbuffer
`
`as
`
`Unlock
`baokuffer'pointer
`
`90$
`
`Is TDVision
`format?
`
`I,
`
`Clean
`
`
`backbuffer Get pointefor
`m bmkffer
`
`8?"
`
`
`Display
`backbuffer to
`
`
`first surface. left
`
`exit
`
`Display
`backbuffer to
`first surface.
`right exit
`
`ens
`
`'j
`
`LOCK
`‘backbuffer
`
`m GetTDVsoin xyz
`coordinates
`
`905
`
`.
`
`803
`
`8137
`'
`
`V
`
`Display
`scene
`'
`‘
`l
`bagflfgger
`.
`‘
`_ " .'
`Unlock
`' backbuffer pointer
`
`'
`
`‘
`
`|PR2018—01045
`
`Sony EX1001 Page 15
`
`IPR2018-01045
`Sony EX1001 Page 15
`
`

`

`US. Patent
`
`Jun. 26, 2012
`
`Sheet 13 of 13
`
`US 8,206,218 B2
`
`Changes for TDVision video card compilation so
`
`Normal loft
`backbuffer
`
`
`
`
`Right
`TDVis ion
`
`backbuffer
`
`First normal left
`buffer
`
`Monitor
`
`connected to
`VGA 1 output
`
`VGA 2 output
`
`3D Visor
`
`connected to
`
`|PR2018—01045
`
`Sony EX1001 Page 16
`
`IPR2018-01045
`Sony EX1001 Page 16
`
`

`

`1
`3D VIDEOGAME SYSTEM
`
`RELATED APPLICATIONS
`
`US 8,206,218 B2
`
`2
`
`This application is a continuation of copending US. appli-
`cation Ser. No. 11/471,280, titled “3D Videogame System,”
`filed Jun. 19, 2006, which is a continuation of PCT Applica-
`tion PCT/MX2003/000112 filed on Dec. 19, 2003, published
`in the Spanish language. The disclosures of all the above-
`referenced applications, publications, and patents are consid-
`ered part of the disclosure of this application, and are incor-
`porated by reference herein in their entirety.
`
`FIELD OF THE INVENTION
`
`The present invention is related to the display of three-
`dimensional television images, more specifically to a hard-
`ware and software design for viewing three-dimensional (3D)
`images, easy to be integrated to the existing television, per-
`sonal computer and videogame system equipment.
`
`BACKGROUND OF THE INVENTION
`
`The Visual man-machine interface is constantly trying to
`improve the images for a wide range of applications: military,
`biomedical research, medical imaging, genetic manipulation,
`airport security, entertainment, videogames, computing, and
`other display systems.
`Three-dimensional (3D) information is the key for achiev-
`ing success in critical missions requiring realistic three-di-
`mensional images, which provide reliable information to the
`user.
`
`Stereoscopic vision systems are based on the human eye’s
`ability to see the same object from two different perspectives
`(left and right). The brain merges both images, resulting in a
`depth and volume perception, which is then translated by the
`brain into distance, surface and volumes.
`In the state-of-the-art, several attempts have been made in
`order to achieve 3D images, e.g., the following technologies
`have been used:
`
`Red-blue polarization
`Vertical-horizontal polarization
`Multiplexed images glasses.
`3D virtual reality systems
`Volumetric displays
`Auto-stereoscopic displays
`All of the aforementioned technologies have presentation
`incompatibilities, collateral effects and a lack of compatibil-
`ity with the current existing technology.
`in
`For example, red-blue polarization systems require,
`order to be watched, a special projector and a large-size white
`screen; after a few minutes, collateral effects start appearing,
`such as headache, dizziness, and other symptoms associated
`to images displayed using a three-dimensional effect. This
`technology was used for a long time in cinema display sys-
`tems but, due to the problems mentioned before, the system
`was eventually withdrawn from the market. Collateral symp-
`toms are caused by the considerable difference in the content
`received by the left eye and the right eye (one eye receives
`blue-polarized information and the other receives red-polar-
`ized information), causing an excessive stress on the optical
`nerve and the brain. In addition, two images are displayed
`simultaneously. In order to be watched,
`this technology
`requires an external screen and the use of polarized color
`glasses. If the user is not wearing red-blue glasses, the three-
`dimensional effect cannot be watched, but
`instead only
`double blurry images are watched.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`The horizontal-vertical polarization system merges two
`images taken by a stereoscopic camera with two lenses; the
`left and right images have a horizontal and vertical polariza-
`tion, respectively. These systems are used in some new cin-
`ema theaters, such as Disney® and IMAX®3D theaters. This
`technology requires very expensive production systems and
`is restricted to a dedicated and selected audience, thus reduc-
`ing the market and field of action. A special interest in the
`three-dimensional (3D) format has grown during the past
`three years; such is the case of Tom Hanks’ productions and
`Titanic, which have been produced with 3D content by
`IMAX3D technology. However, this technology also results
`in collateral effects for the user after a few minutes of display,
`requires an external screen and uses polarized glasses; if the
`user is not wearing these glasses, only blurred images can be
`watched.
`
`Systems using multiplexed-image shutting glasses tech-
`nology toggle left and right images by blocking one of these
`images, so it cannot get to the corresponding eye for a short
`time. This blocking is synchronized with the image’s display
`(in a monitor or TV set). Ifthe user is not wearing the glasses,
`only blurred images are seen, and collateral effects become
`apparent after a few minutes. This technology is currently
`provided by (among others), BARCO SYSTEMS for Mer-
`cedes Benz®, Ford® and Boeing® companies, by providing
`a kind of“room” to create 3D images by multiplexing (shutter
`glasses) in order to produce their prototypes before they are
`assembled in the production line.
`3D virtual reality systems (VR3D) are computer-based
`systems that create computer scenes that can interact with the
`user by means of position interfaces, such as data gloves and
`position detectors. The images are computer generated and
`use vector, polygons, and monocular depth reproduction
`based images in order to simulate depth and volume as cal-
`culated by software, but images are presented using a helmet
`as a displaying device, placed in front of the eyes; the user is
`immersed in a computer generated scene existing only in the
`computer and not in the real world. The name of this com-
`puter-generated scene is “Virtual Reality”. This system
`requires very expensive computers, such as SGI Oxygen® or
`SGI Onyx Computers®, which are out of reach of the com-
`monuser. Serious games and simulations are created with this
`technology, which generates left-right sequences through the
`same VGA or video channel, the software includes specific
`instructions for toggling video images at on-screen display
`time at a 60 Hz frequency. The videogame software or pro-
`gram interacts directly with the graphics card.
`There is a technology called I-O SYSTEMS, which dis-
`plays multiplexed images in binocular screens by means of a
`left-right multiplexion system and toggling the images at an
`80 to 100 Hz frequency, but even then the flicker is perceived.
`Only a few manufacturers, such as Perspectra Systems®,
`create volumetric display systems. They use the human eye
`capability to retain an image for a few milliseconds and the
`rotation of a display at a very high speed; then, according to
`the viewing angle, the device shows the corresponding image
`turning the pixels’ color on and off, due to the display’s high
`speed rotation the eye can receive a “floating image”. These
`systems are very expensive (the “sphere” costs approximately
`50,000 USD) and require specific and adequate software and
`hardware. This technology is currently used in military appli-
`cations.
`
`Auto-stereoscopic displays are monitors with semi-cylin-
`drical lines running from top to bottom and are applied only to
`front and back images; this is not a real third dimension, but
`only a simulation in two perspective planes. Philips® is cur-
`rently working in this three-dimension technology as well as
`
`|PR2018-01045
`
`Sony EX1001 Page 17
`
`IPR2018-01045
`Sony EX1001 Page 17
`
`

`

`US 8,206,218 B2
`
`3
`SEGA® in order to obtain a technological advantage. Results
`are very poor and there is a resolution loss of 50%. This
`technology is not compatible with the present technological
`infrastructure and requires total replacement of the user’s
`monitor. Applications not specifically created for this tech-
`nology are displayed blurred, making them totally incompat-
`ible with the inconveniences of the current infrastructure. In
`
`order to watch a 3D image, the viewer needs to be placed at an
`approximate distance of 16" (40.64 cm), which varies accord-
`ing to the monitor’s size, and the viewer must look at the
`center of the screen perpendicularly and fix his/her sight in a
`focal point beyond the real screen. With just a little deviation
`of the sight or a change in the angle of vision, the three-
`dimensional effect is lost.
`
`In the state-of-the-art, there are several patents, which are
`involved in the development of this technology, namely:
`US. Pat. No. 6,593,929, issued on Jul. 15, 2003 and US.
`Pat. No. 6,556,197, issued onApr. 29, 2003, granted to Timo-
`thy Van Hook, et al., refer to a low cost video game system
`which can model a three-dimensional world and project it on
`a two-dimensional screen. The images are based on inter-
`changeable viewpoints in real-time by the user, by means of
`game controllers.
`US. Pat. No. 6,591,019, issued on Jul. 8, 2003, granted to
`Claude Comair et al., uses the compression and decompres-
`sion technique for the transformation of a matrix into 3D
`graphical systems generated by a computer. This technique
`consists in converting real numbers matrixes into integer
`matrixes during the zeroes search within the matrix. The
`compressed matrixes occupy a much smaller space in
`memory and 3D animations can be decompressed in real-time
`in an efficient manner.
`
`US. Pat. No. 6,542,971, issued onApr. 1, 2003, granted to
`David Reed, provides a memory access system and a method
`which uses, instead of an auxiliary memory, a system with a
`memory space attached to a memory which writes and reads
`once the data input from one or more peripheral devices.
`US. Pat. No. 6,492,987, issued on Dec. 10, 2002, granted
`to Stephen Morein, describes a method and device for pro-
`ces sing the elements ofthe objects not represented. It starts by
`comparing the geometrical properties of at least one element
`of one object with representative geometric properties by a
`pixels group. During the representation ofthe elements of the
`object, a new representative geometric property is determined
`and is updated with a new value.
`US. Pat. No. 6,456,290, issued on Sep. 24, 2002, granted
`to Vimal Parikh et al., provides a graphical system interface
`for the application of a use and learning program. The char-
`acteristic includes the unique representation ofa vertex which
`allows the graphic line to retain the vertex status information,
`projection matrix and immersion buffer frame commands are
`set.
`
`Any videogame is a software program written in some
`computer language. Its objective is to simulate a non-existent
`world and take a player or user into this world. Most videoga-
`mes are focused in enhancing the visual and manual dexterity,
`pattern analysis and decision taking, in a competitive and
`improvement (difficulty level) environment, and are pre-
`sented in large scenarios with a high artistic content. As a
`game engine, most videogames are divided into the following
`structure: videogame, game library with graphics and audio
`engines associated, the graphical engine contains the 2D
`source code and the 3D source code, and the audio engine
`contains the effects and music code. Every block of the game
`engine mentioned is executed in a cyclic way called a game
`loop, and each one of these engines and libraries is in charge
`of different operations, by example:
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`
`Graphics engine: displays images in general
`2D source code: static images, “backs” and “sprites”
`appearing in a videogame screen.
`real-time vector handled
`3D source code: dynamic,
`images, processed as independent entities and with xyz coor-
`dinates within the computer-generated world.
`Audio engine: sound playback
`Effects code: when special events happen, such as explo-
`sions, crashes, jumps, etc.
`Music code: background music usually played according
`to the videogame’s ambience.
`The execution of all these blocks in a cyclic way allows the
`validation of current positions, conditions and game metrics.
`As a result of this information the elements integrating the
`videogame are affected.
`The difference between game programs created for game
`consoles and computers is that originally, the IBM PC was not
`created for playing in it. Ironically, many of the best games
`run under an IBM PC-compatible technology. If we compare
`the PCs of the past with the videogames and processing
`capabilities of the present, we could say that PCs were com-
`pletely archaic, and it was only by means of a low-level
`handling (assembly language) that the first games were cre-
`ated, making direct use of the computer’s graphics card and
`speaker. However, the situation has changed. The processing
`power and graphics capabilities of present CPUs, as well as
`the creation of cards specially designed for graphics pro-
`cesses acceleration (GPUs) have evolved to such a degree that
`they surpass by far the characteristics of the so-called super-
`computers in the 1980s.
`In 1996, a graphics acceleration system known as “hard-
`ware acceleration” was introduced which included graphics
`processors capable of making mathematical and matrix
`operations at a high speed. This reduced the main CPU’s load
`by means of card-specific communications and a program-
`ming language, located in a layer called a “Hardware Ab strac-
`tion Layer” (HAL). This layer allows the information han-
`dling of data associated to real-time xyz coordinates, by
`means of coordinate matrixes and matrix mathematical
`
`operations, such as addition, scalar multiplication and float-
`ing point matrix comparison.
`
`BRIEF DESCRIPTION OF THE INVENTION
`
`An object of the present invention is to solve the incom-
`patibility problems of the technologies for a three-dimen-
`sional image display.
`Another object of the present invention is to provide a
`multi-purpose technology which allows the final user to
`watch video images, computer graphics, videogames and
`simulations with the same device.
`
`An additional object of the present invention is to provide
`a technology which eliminates the collateral effects produced
`after watching the three-dimensional images provided by the
`present technologies, even for hours of constant use.
`It is an additional object ofthe present invention to provide
`a technologically advanced integration in software by the
`creation of a pair of buffers corresponding to the left eye and
`the right eye, and hardware with an additional, independent
`display device which shares the memory in an immerse form,
`along with digital video image processors.
`It is another object of the present invention to display the
`image physically on-screen by means of two front buffers
`created by graphics process units or GPUs.
`
`|PR2018-01045
`
`Sony EX1001 Page 18
`
`IPR2018-01045
`Sony EX1001 Page 18
`
`

`

`US 8,206,218 B2
`
`5
`Is still another object of the present invention to obtain
`brain perceptions of depth and volume with highly realistic
`images, even if they are created by computer graphics soft-
`ware.
`
`Is still other object of the present invention to provide a
`TDVision® algorithm to create highly realistic computer
`images.
`invention to make
`It
`is another object of the present
`changes in the current technological base to create a new
`digital imaging process with optical techniques in order to
`achieve a real image perception by setting the view of a right
`side camera.
`
`It is another object of the present invention to achieve
`digital media convergence, wherein a DVD-playing com-
`puter, a movie-producing laptop, the video-image transmis-
`sion capability of the internet, and PC and video game con-
`soles can be used in the intemet structure.
`
`It is another object ofthe present invention to provide a new
`assembly language algorithm, analog and digital hardware to
`obtain the best adaptation to the existing technologies’ 3D
`equipment.
`It is still another object of the present invention to provide
`three-dimensional visual computer systems for the genera-
`tion of stereoscopic images by means of animation, display
`and software modeling.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 shows the TDVision® videogame technology map.
`FIG. 2 shows the main structure for a videogame based on
`the previous art.
`FIG. 3 shows the one embodiment of a three-dimensional
`
`element for constructing an object in a certain position in
`space.
`FIG. 4 shows the development outline of a videogame
`program based on the OpenGL and DirecTX API functions
`technologies.
`FIG. 4a shows a block diagram of one embodiment of an
`algorithm for creating the left and right buffers, and addition-
`ally discriminating if TDVision technology is used.
`FIG. 4b shows a block diagram of a subroutine for setting
`the right camera view after drawing an image in the right
`backbuffer as a function of the right camera vector. The sub-
`routine also discriminates if the TDVision technology format
`is used.
`
`FIG. 5 shows a block diagram of the computing outline of
`the modifications to the graphical adapter for compiling the
`TDVision technology. It also allows the communication and
`contains the programming language and allows the informa-
`tion handling of the data associated with the images set.
`FIG. 6 represents a block diagram of an algorithm which
`allows the drawing ofinformation in the TDVi sion backbuffer
`and presenting it on-screen in DirecTX 3D format.
`FIG. 7 shows the display sequence using the OpenGL
`format.
`
`FIG. 8 shows the block diagram of the on-screen informa-
`tion display by means of the left and right backbuffers using
`the OpenGL algorithm.
`FIG. 9 shows the changes needed in the video card used for
`the TDVision technology.
`
`DETAILED DESCRIPTION OF THE INVENTION
`
`V1deogames are processes which start by providing a plu-
`rality of independently related logical states which include a
`set ofprogramming options, where each programming option
`corresponds to different image characteristics. The generic
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`program instructions can be compiled into a code by several
`computing devices, without having to independently generate
`the object codes for each device.
`The computer devices, such as personal computers, lap-
`tops, videogames, etc.,
`include central processing units,
`memory systems, video graphical processing circuits, audio
`processing circuits and peripheral ports. Typically, the central
`processing unit processes software in order to generate geo-
`metric data referring to the image to be displayed and pro-
`vides the geometric data to the video graphics circuit, which
`generates the pixel data stored in a memory frame where the
`information is sent to the display device. The aforementioned
`elements as a whole are typically called the videogame
`engine.
`Some video game engines are licensed to a third party,