Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 1 of 23
`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 1 of 23
`
`EXHIBIT A
`
`EXHIBIT A
`
`

`

`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 2 of 23
`case sizo'cv'oomAD/A Dtlflllllllllllllllllll|illliltlfllltllllllltllllllllllllllllllll3
`
`USOO6409599B1
`
`US 6,409,599 B1
`(10) Patent N0.:
`(12) United States Patent
`
`Sprout et al. Jun. 25, 2002 (45) Date of Patent:
`
`
`(54)
`
`INTERACTIVE VIRTUAL REALITY
`PERFORMANCE THEATER
`ENTERTAINMENT SYSTEM
`
`(75)
`
`.
`.
`Inventors‘ Eifidyfafié’mug 3133111,? Spmut’
`0
`0
`1 W00 ’
`(
`)
`.
`,
`(73) ASSIgnee: Hem 0“ Rye Teehneleglesa [new
`W11dWOO¢ MO (US)
`.
`.
`.
`.
`SHbJCCt to any disclaimer, the term ofthrs
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`.
`(*) Notice:
`
`.
`(21) Appl. No" 09/617’388
`(22)
`Filed:
`Jul, 17, 2000
`
`(60)
`
`Related US. Application Data
`Provisional application No. 60/144,492, filed on Jul. 19,
`1999’
`Int. Cl.7 ................................................. A63F 13/00
`(51)
`(52) US. Cl.
`.............................. 463/31; 463/42; 463/32
`(58) Field of Search ......................... 463/30—33, 40—42;
`348/13—15
`
`(56)
`
`References Cited
`
`us PATENT DOCUMENTS
`5,388,990 A
`2/1995 Beckman
`5,490,784 A
`2/1996 Carmein
`5,663,517 A
`9/1997 Oppenheim
`5,679,075 A * 10/1997 Forrest et al.
`5,682,196 A
`10/1997 Freeman
`576957406 A
`12/1997 Pad?
`57728960 A
`341998 Slmek
`g’gi’gg 2
`:33: Eggs; al
`5:795:228 A
`8/1998 Trumbull et 21].
`5,830,065 A
`11/1998 Sitrick
`
`................. 463/9
`
`12/1998 Latypov
`5,846,134 A
`5,850,352 A * 12/1998 Moczzi ct al.
`..... 364/514
`
`5,916,024 A *
`6/1999 Van Kohorn
`...... 463/40
`5,923,337 A *
`7/1999 Yamamoto
`.. 345/473
`6,056,640 A *
`5/2000 Schaaij ................ 463/4
`
`6,130,898 A * 10/2000 Kostreski et al.
`370/522
`6,154,723 A * 11/2000 Cox etal. ................... 704/240
`6,155,927 A * 12/2000 Levasseur .
`463/42
`6,159,100 A * 12/2000 Smith .......
`463/42
`6,183,364 B1 *
`2/2001 Trovato
`463/32
`6,227,974 B1 *
`5/2001 Eilat et al.
`463/40
`6,257,982 B1 *
`7/2001 Rider et al.
`................... 463/31
`.
`.
`* Cited by examiner
`
`
`
`Primary Examiner—Jessica Harrison
`Assistant Examiner—John M Hotaling, II
`(74) Attorney, Agent, or Firm—Greensfelder, Hemker &
`Gale, P.C.; Peter S. Gilster
`
`(57)
`
`ABSTRACT
`
`The present invention relates to a novel interactive Virtual
`reality (VR) performance theater and environment in which
`a plurality of participants enioy an entertainment and/or
`educational experience. The Performance is viewed by the
`participants through VR display devices such as a head
`mounted display, wherein both an immersive graphical
`environment and live and/0r pre-recorded performers are
`viewed. The participants exert
`limited control over the
`content and outcome of the performance through the use of
`input devices, such as hand held input devices having a
`plurality of buttons, and/or voice communications, while
`Viewing the live or pre-recorded performers whom are
`mixed within the immersive VR environment. The system
`uses interconnected computers, Video and audio processing
`devices, and furthermore includes a network interconnecting
`the computers, participants, and live/pre-recorded perform-
`ers for Video, audio, and graphics transmission.
`
`9 Claims, 11 Drawing Sheets
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 3 of 23
`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 3 of 23
`
`US. Patent
`
`Jun. 25, 2002
`
`Sheet 1 0f 11
`
`US 6,409,599 B1
`
`8:390
`
`o_n:<
`
`89>
`
`...E8_o_:mam€32thN232thr“$92th
`
`
`
`EoEceScmb=mmm
`
`SEEEEE
`
`
`
`moEQSOmo_o>
`
`89>.
`
`
`
`o_u:<—‘©_H_82>
`
`
`

`

`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 4 of 23
`Case 6:20-cv-00612-ADA Document 1—1 Filed 07/06/20 Page 4 of 23
`
`US. Patent
`
`Jun. 25, 2002
`
`Sheet 2 0f 11
`
`US 6,409,599 B1
`
`3
`.9
`
`u—
`
`>
`.g
`
`u.
`
`f”
`
`E
`.9 “>i
`
`Fig.2
`
`0 n
`
`oN 1
`
`
`2.5°
`
`

`

`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 5 of 23
`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 5 of 23
`
`US. Patent
`
`Jun. 25, 2002
`
`Sheet 3 0f 11
`
`US 6,409,599 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 6 of 23
`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 6 of 23
`
`US. Patent
`
`Jun. 25, 2002
`
`Sheet 4 0f 11
`
`US 6,409,599 B1
`
`
`
`

`

`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 7 of 23
`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 7 of 23
`
`US. Patent
`
`Jun. 25, 2002
`
`Sheet 5 0f 11
`
`US 6,409,599 B1
`
`Participant260
`
`Participant325
`
`D.
`
`O 0
`
`0
`
`\—
`4—:
`
`C (
`
`U
`.9-
`.2
`1:
`(U
`
`

`

`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 8 of 23
`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 8 of 23
`
`US. Patent
`
`Jun. 25, 2002
`
`Sheet 6 0f 11
`
`US 6,409,599 B1
`
`
`
`

`

`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 9 of 23
`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 9 of 23
`
`US. Patent
`
`Jun. 25, 2002
`
`Sheet 7 0f 11
`
`US 6,409,599 B1
`
`awho:
`
`
`
`Amrofigflmroéotmn.ESwEmo
`
`mace—8.0mm550x4/
`wcmmbm09
`
`@8860
`
`ErI
`
`@5252c5
`
`2m:9m65:00
`
`ewe
`
`
`
`
`
`AmyESquo6:50
`
`3m.6,wUMQEég.
`
`
`o_:Sactgmfig.u<
`$0.38N0_.H_
`
`
`
`@2chc3$5835o_u:<ELomwmooi2g<
`
`o:.we
`
`
`
`m_mcm_m
`
`o_n:<
`
`fiEme
`
`®0C®_U3<VI.
`
`
`
`25Egon.Om2>
`
`
`>m_am_cEEcotsw82>E5389.;82>m>m:229wmoosmn.
`
`mwcocaumm:82>.SE:=<
`
`€98,392:89:822%25szVIaB25598;0.2>vmv5
`
`
`£0mm?$250
`
`3:3ngmm?.mecacm
`
`
`

`

`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 10 of 23
`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 10 of 23
`
`US. Patent
`
`Jun. 25, 2002
`
`Sheet 8 0f 11
`
`US 6,409,599 B1
`
`Simple Sample Software Flow Diagram
`
`Mixer/Swnoher
`Control
`
`Rumble Seat
`Control
`
`Animation 3A
`
`Challenge 3A
`
`Performer
`Improvisation
`
`Animation 4A
`
`Branch 4A
`
`Fig.8
`
`Name Entry
`
`Interactive
`Promoting
`
`Introduction
`Animation
`
`Performer
`Introduction
`
`Branch 1
`
`ad
`Ke
`<1:)
`Control
`
`Network
`Command/Control
`
`
`Animation 18
`
`
`
`
`
`Challenge 13
`
`Performer
`Improvisation
`
`Branch 23
`
`Challenge 1A
`
`Performer
`lmprowsation
`
`Animation 2A
`
`Branch 2A
`
`Challenge BB
`
`Performer
`Improwsation
`
`Animation 48
`
`Branch 48
`
`Challenge SC
`
`Performer
`Improvisation
`
`Animation 40
`
`Branch 4C
`
`Challenge 5A
`
`Challenge SB
`
`
`Animation SB
`
`
`Performer
`Performer
`
`ImproVisation
`lmprowsation
`
`
`Animation 6A
`Animation 68
`
`
`
`Branch GB
`
`
`
`Branch 6A
`
`Performer
`Performer
`
`Improwsation
`Improvisation
`
`
`Ending
`Animation A
`Animation 5
`
`Results
`
`
`
`

`

`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 11 of 23
`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 11 of 23
`
`US. Patent
`
`Jun. 25, 2002
`
`Sheet 9 0f 11
`
`US 6,409,599 B1
`
`own
`
`
`
`new.EoEmm\
`
`$5,362
`
`E2695:
`
`Ego98:
`
`Ego98:
`
`m>EoEmm
`
`cozflm
`
`m>mZOEQK
`
`565
`
`m>206mm
`
`85w
`
`0mm
`
`own
`
`0325
`
`82.52,
`
`x5502
`
`©I
`
`@
`
`omm
`
`mam
`
`omm
`
`omm
`
`Hmmonmem
`
`(N95
`
`$8385
`
`mm95
`
`ohm
`
`$5262\ucwEcol.
`
`m>_moo._
`
`eczfiw
`
`m>.83
`
`539m
`
`m>.83
`
`.5:me
`
`m>_83
`
`8:me
`
`m>.83
`
`839m
`
`m>_moo._
`
`comma
`
`
`
`cosmeEwEE{ozzmzmaEmm
`
`oom
`
`
`
`
`
`

`

`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 12 of 23
`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 12 of 23
`
`US. Patent
`
`Jun. 25, 2002
`
`Sheet 10 0f 11
`
`US 6,409,599 B1
`
`Fig.10
`
`Enclosure
`
`Presenter
`
`75"
`
`NameEntry
`
`

`

`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 13 of 23
`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 13 of 23
`
`US. Patent
`
`Jun. 25, 2002
`
`Sheet 11 0f 11
`
`US 6,409,599 B1
`
`
`
`98SeatTheaterConfiguration
`Fig.11 75"
`
`Presenter
`
`Enclosure
`
`Height7’10”
`
`48"
`
`.———_—————i
`b
`N
`
`E
`c
`LlJ
`an
`E(U
`z
`
`

`

`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 14 of 23
`Case 6:20-cv-00612-ADA Document 1—1 Filed 07/06/20 Page 14 of 23
`
`US 6,409,599 B1
`
`1
`INTERACTIVE VIRTUAL REALITY
`PERFORMANCE THEATER
`ENTERTAINMENT SYSTEM
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application is based upon Provisional Patent Appli-
`cation entitled Interactive Virtual Reality Performance The-
`ater Entertainment System, Serial. No.: 60/144,492, Filed
`Jul. 19, 1999, the contents of which are incorporated herein
`by reference in their entirety; and continued preservation of
`which is requested.
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`The present invention relates to the field of electronic
`video and live performance entertainment. More
`specifically, the present invention relates to entertainment
`and educational systems whereby participants interact with
`an electronic or computerized environment in addition to
`live and/or pre-recorded performers. Even more specifically,
`the present invention relates to immersive virtual reality
`computer systems in which participants interact with a
`virtual reality environment and live performers using a
`variety of immersion and input devices such as a head
`mounted display and a handheld keypad.
`2. Description of the Related Art
`Virtual reality entertainment systems utilizing head
`mounted displays or other immersive devices or methods,
`whereby a participant views and interacts with a software-
`generated virtual reality environment have been used for
`training and entertainment purposes for many years. An
`immersive virtual reality environment as used and described
`herein refers to a computer generated graphical environment
`wherein a participant is “immersed” within the environment
`so as to provide to the user a sensation of being physically
`located within the graphical environment, although the
`participant is only electronically present with other objects
`within the environment. Therefore, an immersive Virtual
`reality environment creates an illusion to the participant that
`he or she is in an artificially created environment through the
`use of three-dimensional (3D) graphics and computer soft-
`ware which imitates the relationship between the participant
`and the surrounding environment.
`Traditionally, participants are represented in the software
`environment by projections of figures referred to in the art
`as avatars. Participants control their avatars through a com-
`bination of input mechanisms including, but not limited to,
`hand held input devices and data generated from electronic
`and electromagnetic tracking devices which monitor bodily
`movement. Objects within the Virtual reality environment
`which are not controlled by the motion of the participants are
`generally controlled by a computer software program and
`can also respond to the input of the participants.
`In related art virtual reality environments, figures or
`objects not controlled by participants move autonomously
`only in accordance with the software program and respond
`to the input devices of each participant (Carmein U.S. Pat.
`No. 5,490,784, Latypov U.S. Pat. No. 5,846,134). Passive
`and active objects such as trees and animals move in a
`predetermined manner within the virtual reality environ-
`ment. Furthermore, the participant has the opportunity to
`interact with such objects in the virtual reality environment,
`however,
`the objects are limited to computer generated
`images.
`
`10
`
`15
`
`20
`
`30
`
`35
`
`40
`
`45
`
`55
`
`60
`
`65
`
`2
`Generally, the object or focus of virtual reality environ-
`ments is traditionally on the participants’ interaction with
`objects generated entirely by the computer graphics program
`that do not represent, or are not substantially controlled by,
`a human participant in the simulation. Also traditionally,
`groups of participants have been entertained by graphically
`enhanced live or pre-recorded performers on stage or
`through television, cinema or other non—immersion mediums
`without the enhancements of immersion devices or methods,
`and furthermore without immersion devices or methods in
`
`combination with voice communication from the partici-
`pants to the performer and amongst and between partici-
`pants. (Trumbull et al. US. Pat. No. 5,795,228).
`Other known entertainment systems, such as that which is
`disclosed in Freeman et al. US Pat. No. 5,682,196, include
`the capacity for two-way communications between a
`computer-controlled host and a participant. These systems,
`however, do not possess the capability for voice communi-
`cation amongst and between participants, or between par-
`ticipants and a live host. The system in Trumbull allows for
`one-way audio communication from the live performer to
`the participants but does not provide for audio communica-
`tion from the participants to the live performer. Nor does the
`system of Trumbull allow for audio communication amongst
`and between the participants, furthermore enhanced by an
`immersive virtual reality environment.
`Several known art virtual reality systems have developed
`an elementary tactile interface. For example, Shaw et a1 U.S.
`Pat. No. 5,737,505 discloses an interface resembling an
`upper torso suit of armor which detects limb and body
`position. The device can provide force feedback or impart
`motion to a participant who is engaging in both active and
`passive communications with the virtual reality environ-
`ment. The virtual reality system of Shaw, however, does not
`provide communication amongst and between participants,
`nor does the system include live and/or pre-recorded per-
`formers.
`
`lack
`therefore,
`The known art entertainment systems,
`certain advantageous features. In traditional systems, the
`participant may be interacting with a host and the environ-
`ment using some input device. The host, however, is seldom
`live and is instead pre-recorded or computer-controlled,
`therefore reducing the spontaneity of any interaction with
`the participant. The host in Trumbull can be a live performer,
`but the system itself is not within a virtual reality environ-
`ment and therefore lacks the immersive effects which are
`imparted upon the participants for a more exciting experi-
`ence. Further, there is no voice communication from the
`participants to the performer, nor is there voice communi-
`cation amongst and between participants in Trumbull,
`thereby further limiting the entertainment or educational
`experience.
`Accordingly, there appears a need in the art for a virtual
`reality based entertainment and/or educational system which
`provides three-way immersive interactive communication
`amongst and between: 1). participants; 2). an immersive
`environment; and 3). live and/or pre—recorded performers.
`Further, there appears a need for a system which includes
`provisions for individual and/or mixed video, audio, and
`graphical inputs and outputs to enhance the overall enter-
`tainment and/or educational experience. Additionally, there
`is a need for the content and outcome of the experience to
`be controlled, at least in part, by the inputs of the participants
`in response to scripted and/or impromptu communications
`from the performers and/or the virtual reality environment.
`Finally, there appears to be a need for a system which is
`robust and modular in design such that a plurality of
`
`

`

`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 15 of 23
`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 15 of 23
`
`US 6,409,599 B1
`
`3
`participants, performers, and virtual reality environment
`inputs and outputs can be easily retrofitted depending on the
`desired application and to accommodate the rapid changes
`of computer technology. The synergistic combination of
`these elements can provide a hitherto-unrealized level of
`entertainment and/or educational excitement and quality for
`an audience of participants.
`SUMMARY OF THE INVENTION
`
`According to the present Virtual Reality Performance
`Theater, (VRPT),
`the inventors change the focus of the
`software program and entertainment or educational perfor-
`mance from the interaction between individual participants
`and computer generated graphical objects within a Virtual
`reality environment to a three-way interactive communica-
`tion between participants,
`live and/or pre-recorded
`performers, and an immersive virtual reality environment.
`The inclusion of three-way communication amongst and
`between participants, live and/or pre—recorded performers,
`and an immersive virtual reality environment results in a
`synergistic eifect which creates an unparalleled entertain-
`ment and/or educational experience.
`The primary object of the present invention, therefore, is
`to provide an interactive virtual reality performance theater
`wherein there exists three-way communication amongst and
`between participants(),
`live and/or pre-recorded
`performer(s), and the virtual reality environment. The three-
`way communication includes, but is not limited to, audio,
`video, and graphical input and output data. Importantly, the
`interaction of the participants and performers with each
`other and the virtual reality environment determines,
`to
`some extent, both the content and the ultimate outcome of
`the performance.
`Accordingly, a further object of the present invention is to
`create an interactive virtual reality environment in which a
`plurality of participants enjoy immersion in a software-
`generated simulation combined with live and/or pre-
`recorded performances.
`A further object of the present invention is to allow the
`participants to exert control over the content and outcome of
`the performance.
`Another object of the present invention is to provide for
`a novel immersive performance in which each participant
`and the plurality of performers are linked together for voice
`communication with a voice intercom system. The system
`should allow participants to establish a constant dialogue,
`enhancing interactivity by allowing them to share opinions
`and work together in an entertainment game or educational
`setting.
`Yet another object of the present invention is to provide a
`virtual reality environment which is easily customizable to
`the requirements of a specific application and which can be
`easily upgraded to comport with the rapid pace of computer
`technology.
`In brief summary, the present invention overcomes and
`substantially alleviates the deficiencies in the prior art by
`providing an interactive virtual reality performance platform
`and environment in which participants can interact with each
`other, with live and/or pre-recorded performers, and with an
`immersive virtual reality environment for a highly enjoyable
`entertainment and/or educational experience. Importantly,
`participants can exert limited control over the content and
`outcome of the performance through input devices and voice
`communication.
`
`The system generally utilizes input and output devices for
`each of the participants and the performers, and further
`
`10
`
`15
`
`20
`
`30
`
`35
`
`40
`
`45
`
`55
`
`60
`
`65
`
`4
`includes control computers having system data and pro-
`grams to carry out specific processing tasks. The system
`further includes a variety of audio and video components
`which mix and switch audio and video data for presentation
`to participants in the audience. Through the innovative
`mixing and switching techniques,
`the system is able to
`present to each of the participants a graphical virtual reality
`environment wherein live video of the performers is super-
`imposed within the graphical environment. The participants,
`therefore, view and interact with a novel display of both
`graphical data and live video data for an exciting entertain-
`ment and/or educational experience.
`In support of the Video mixing and switching, the virtual
`reality display is accomplished through chromakey and
`other electronically-generated costuming wherein the video
`and/or audio inputs from the performer(s) is mixed with the
`graphical environment in which the participant is immersed,
`using, for example, a head mounted display (HMD). The
`participants, therefore, experience an immersive graphical
`environment in which the performer(s) can also be seen
`and/or heard.
`
`The participants exert control over the content and out-
`come of the performance through the use of hand-held input
`devices such as a keypad or cyberglove, while viewing
`images or electronic facsimiles of the performers/
`performers, software simulations, and other video, audio,
`and graphical output of the virtual reality environment. The
`performer(s), by following a script and/or impromptu
`actions, query the audience for responses based upon, for
`example, a specific question asked or their opinions as to a
`particular subject matter. Participant responses can then
`dictate the further content and eventually the outcome of the
`performance, such as a participant being declared the “win-
`ner” based on points earned in responding to the
`performer(s) and/or system queries.
`The system further includes additional audio and motion
`devices, including but not limited to, speakers and motion-
`equipped seats for each of the participants to enhance the
`immersive experience. The immersive experience of the
`participants is supplemented by aural and physical sensa-
`tions of the motion-equipped seats, referred to hereinafter as
`“rumble seats,” in which the individual participants sit
`throughout the performance. The seat contains speakers for
`background sound effects, and seat movement effects are
`generated by software in response to the participants” voice
`communication, decisions, and physical movement.
`In addition, a network is included to connect computers
`for data transmission and to facilitate remote interconnec-
`
`tivity and participation. With implementation of the
`network, participants and performers can be physically
`present at geographically remote sites while experiencing
`the same virtual reality environment and performances.
`Accordingly, the inventors add immersive virtual reality
`technology and electronic interaction to live and/or pre-
`recorded performances by performers. By placing the par-
`ticipants in a virtual reality environment, the participants
`and performers can interact more directly while sensing a
`physical proximity to each other and the immersive virtual
`reality environment.
`An immersive environment, as more fully described
`herein below, is defined as one in which a greater than 25
`degree diagonal field of view is provided no more than 10
`feet from the viewer. To achieve an immersive environment,
`the VRPT can utilize a head mounted display (HMD) device
`that creates the sensation of immersion in a 3D graphical
`environment. In addition, specific proximity to a non-head
`
`

`

`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 16 of 23
`Case 6:20-cv-00612-ADA Document 1—1 Filed 07/06/20 Page 16 of 23
`
`US 6,409,599 B1
`
`5
`mounted display, such as a monitor at eye level, can create
`a similar immersive environment. Additional
`immersion
`
`devices and methods are more fully described herein below.
`As a result, the entire system can be classified as a blend
`or meld of virtual reality, computer graphics, and telecon-
`ferencing technology. The performers need not be at the
`local site or sites where the participants are located. Rather,
`the performers may be located as remotely as communica-
`tion channels allow and themselves may deliver their per—
`formance from different sites to a plurality of sites via the
`Internet, satellite transfer, or other channels.
`One benefit of the present system is that a human par-
`ticipant can interact with other participants and the perform-
`ers much more spontaneously than in previous forms of
`interaction. Participants can also have a far more “personal
`and unique” experience through the use of immersion
`devices and interactive communications technologies.
`Another benefit of the present system is that the virtual
`reality performance theater can now be used as a medium for
`person—to—person visual projection, as well as entertainment
`and business applications such as stand-up comedy, dra-
`matic presentations, teaching and lecturing, and other activi-
`ties related to communication of ideas and concepts to an
`audience, all within an immersive simulation medium.
`Further, the capacity to simultaneously include multiple
`performers adds to the performance a new dimension. Addi-
`tional performers are added with audio only, voice only, or
`both audio and video together. A single performer, although
`effective, can only engage in monologue and such interac-
`tion with the participating audience as the system will
`permit. By comparison, a plurality of performers in the
`electronic simulation medium allows this medium to be used
`
`for live and recorded dialogue between performers and
`participants so that true drama may be presented.
`to those
`Other objects and features will be apparent
`skilled in the art and are pointed out with more particularity
`herein below.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The above mentioned and other features and objects of
`this invention, and the manner of attaining them, will
`become more apparent and the invention itself will be better
`understood by reference to the following description of an
`embodiment of the invention taken in conjunction with the
`accompanying drawings, wherein:
`FIG. 1 is a block diagram showing the communication
`amongst and between the participants, the performers, and
`the immersive virtual reality environment.
`FIG. 2 is an orthogonal View of a computer display
`illustrating the inventor’s definition of “immersive.”
`FIG. 3 is an orthogonal view of the virtual reality perfor-
`mance theater according to the present invention.
`FIG. 4 is an orthogonal view of the presenter enclosure
`according to the present invention.
`FIG. 5 is a front view of a sample video display as viewed
`by the participants according to the present invention.
`FIG. 6 is a top view of the hand-held input device keypad
`of the present invention according to the present invention.
`FIG. 7 is a block diagram of the system components
`according to the present invention.
`FIG. 8 is a software flow diagram according to the present
`invention.
`
`FIG. 9 is a network diagram according to the present
`invention.
`
`10
`
`15
`
`20
`
`30
`
`35
`
`40
`
`45
`
`55
`
`60
`
`65
`
`6
`FIG. 10 is a top view of an alternate embodiment of the
`virtual reality performance theater system depicting a 16
`seat theater.
`
`FIG. 11 is a top view of another alternate embodiment of
`the virtual reality performance theater system depicting an 8
`seat theater.
`
`Corresponding reference characters indicate correspond-
`ing parts throughout the several views. Although the draw-
`ings represent embodiments of the present invention, the
`drawings are not necessarily to scale and certain features
`may be exaggerated in order to better illustrate and explain
`the present invention.
`DESCRIPTION OF THE PRESENT INVENTION
`
`To highlight and emphasize the novelty of the present
`invention, FIG. 1 shows the three-way communication
`amongst and between the participants, the performers, and
`the immersive virtual reality environment. The Virtual Real-
`ity Performance Theater (VRPT) system provides voice and
`video communication amongst and between participants,
`voice and video communication amongst and between the
`participants and the performers, voice and video communi-
`cation amongst and between the performers, and graphic,
`audio, and video communication amongst and between the
`participants and the performers and the immersive virtual
`reality environment. The combination of audio/voice, video,
`and graphical inputs and outputs to and from each of the
`participants, performers, and immersive environment ere-
`ates a synergistic effect which results in an unparalleled
`entertainment and/or educational experience for participants
`as described more fully herein below.
`Throughout the following description, it shall be under-
`stood that singular or multiple participants and performers
`can be used for the virtual reality performance theater
`although referred to in either the singular or multiple
`context, unless otherwise indicated.
`With regard to the following descriptions of the preferred
`and alternate embodiments of the present
`invention,
`the
`inventors define an “immersive” display as one which has
`greater than a 25 degree diagonal field of view with a
`distance of no more than 10 feet from the viewer. The 25
`
`degree diagonal field of view from a distance of 10 feet is
`illustrated more clearly in FIG. 2, where a typical display of
`a computer terminal which has a 14 inch diagonal viewing
`length would require that the viewer be approximately 31.6
`inches from the display in order for the display to qualify as
`“immersive”.
`
`Referring to FIG. 3, the preferred embodiment of the
`present invention is illustrated and generally indicated as
`virtual reality performance theater 10. Virtual reality per-
`formance theater (VRPT) 10 facilitates multiple participants
`and generally includes for each participant head mounted
`display (HMD) 12, microphone 14 mounted to HMD 12,
`headphones 15 mounted to HMD 12, and hand-held input
`device 16. HMD 12 provides a visual display of the immer-
`sive environment to each of the participants through one or
`two cathode ray tubes (CRT), one or two liquid crystal
`displays (LCD), or other vision devices. The participants
`View the virtual reality environment through HMD 12 and
`provide input to the system with hand-held input device 16
`and microphone 14. Further, the participants can hear voice
`and/or audio communications from other participants,
`performers, and the VRPT system with headphones 15.
`Audio output from VRPT 10 to the participants is also
`provided through rumble seats 18, each of which is equipped
`with a suitable speaker system or other suitable sound
`
`

`

`Case 6:20-cv-00612-ADA Document 1-1 Filed 07/06/20 Page 17 of 23
`Case 6:20-cv-00612-ADA Document 1—1 Filed 07/06/20 Page 17 of 23
`
`US 6,409,599 B1
`
`7
`transponder so that each seat may be called a “rumble seat.”
`Rumble seats 18 further include Vibration panels or other
`sound or movement transducers or devices which provide
`output to the participants which supplements the immersive
`experience within the Virtual reality environment.
`Virtual reality performance theater 10 further includes
`control center 30 where a live host interacts with participants
`and enters, for example, the name of each participant into
`registration computer 32 prior to seating the participants in
`rumble seats 18. Each participant typically passes through
`control center 30 and registers prior to participating in
`Virtual reality performance theater 10.
`Prior to and during the performance, the physical actions
`and audio inputs of the participants are captured with video
`cameras 20 and theater microphones 22. The video images
`and audio signals captured by Video cameras 22 and theater
`microphones 22, respectively, are then communicated to a
`live performer or performers whom reside in presenter
`enclosure 24 during an entertainment and/or educational
`performance. The output from Video cameras 20 and theater
`microphones 22 is also further processed by VRPT 10 as
`more fully described herein below.
`The interior of presenter enclosure 24 is more clearly
`shown in FIG. 4, where a live performer is shown viewing
`participants through Video monitor 44 and listening to
`participants through speaker 46. The outputs of video cam-
`eras 20 and theater microphones 22 are transmitted,
`respectively, to Video monitor 44 and to speaker 46 located
`within presenter enclosure 24. Presenter enclosure 24 is
`specifically designed to mask the existence of the live
`performer so that the participants are not aware of his or her
`physcial presence. Presenter enclosure 24 also includes
`stereoscopic video camera 42 to capture the live image of the
`performer and lighted backdrop 48 to facilitate the mixing of
`live video of the performer with the graphical data within
`VRPT 10 as more fully described in the following.
`Akey novel feature of the present invention includes the
`mixing of video input from the live performers with the
`graphical data or
`images within the virtual reality
`environment, which combination is then viewed by the
`participants through HMD 12. As shown in FIG. 4, stereo-
`scopic Video camera 42 captures the live Video image of the
`performer and subsequently sends the corresponding Video
`signal to the system for processing and mixing with the
`graphical data within the Virtual reality environment.
`The result is a novel Virtual reality environment illustrated
`and generally indicated as immersive display 60 as shown in
`FIG. 5. The participants view not only the graphical data
`within the Virtual reality environment, but also View live
`video of the performers superimposed or mixed with the
`graphical data in the environment. Therefore, as the partici-
`pants traverse and interact with graphical environment 62,
`live performer video image 64 is periodically superimposed
`in the environment so that the participants and the perform-
`ers can interact amongst and between each other using the
`communications devices as described herein,
`
`The inclusion of live performer Video image 64 together
`with graphical environment 62 within the Virtual reality
`environment can be controlled by the computer software of
`virtual reality performance theater 10, as described more
`fully below, or by the live performers. Immersive display 60
`may f