USOIO40643ZB2
`
`United States Patent
`(12)
`(10) Patent No.2
`US 10,406,432 B2
`
`Watanabe
`(45) Date of Patent:
`Sep. 10, 2019
`
`(54) VIRTUAL IMAGE DIS'H’LA‘Ir PROGRAM,
`VIRTUAL IMAGE DISPLAY APPARATUS,
`AND VIRTUAL IMAGE DISPLAY METHOD
`
`(58) Field of Classification Search
`A63F 13126
`CPC
`See application file for complete search history,
`
`(71) Applicant: (:RICE, lnc._. Tokyo (JP)
`
`(72)
`
`Inventor: Masashi Watanabc. Tokyo (JP)
`
`(73) Assigilcc:
`
`(BREE, Inc., Tokyo (JP)
`
`(55}
`
`Rdmm“ 0“"
`U.S. PATENT DOCUMENTS
`
`2.722.465 B2 “
`
`52010 Sumi
`
`201310214016 Al “
`
`1012013 Gagner
`
`....................... A631 [3504
`4633]
`GOTF 11'13211
`
`( '9‘ ) Notice:
`
`Subject to any disclaimer. the term of this
`patent is extended or adjusted under 35
`U.S.(T. 154(h) by 347' days.
`
`.
`(Continued)
`‘
`-
`‘
`,
`,
`7‘ ‘
`,_
`l‘ORlJJiIJN PA] 1‘.N l ])(1( UMltN 15>
`
`46334
`
`(21) Appt. Nu: 151196,:1111
`_
`(22]
`Filed:
`Jun. 29, 2016
`
`H,
`JP
`JP
`
`(”400,62 A
`2005038008 A
`2013083731 A
`
`3,1995
`2-"2005
`52013
`
`(65)
`
`(10]
`_
`
`“'1‘" “Mic-”i0" ”at“
`US 201710072305 Al
`M'
`A l( 201'}r
`ar
`1’
`Foreign Application Prioritv Data
`'
`
`OTHER PUBLICM‘IONS
`
`“Social Virtual Reality”. URL-'13 Channel. Published on Sep. 9. 2015.
`Accessed on Jun. 23. 2016. URL: 11|lps:r".'=www.youlLIbe.coin.-'walcli?
`\I"E9hqxgk3Gn0; 2 pgs.
`
`Sep. 16. 2015
`
`(JP)
`
`21015-183379
`
`(Continued)
`
`(51)
`
`Int Cl
`A63F 113196
`6'06!" 3/01
`(306.1" 19/01?
`A 63F 1.3/21]
`006}: 9/1”]
`606!" ”0;:34
`A6411” 111/428
`(52) U S (‘1‘
`CPC ' '
`
`(201431)
`(2006.01 )
`(201 1.01)
`(201431)
`[2018 01)
`(2013-01)
`(20121-111)
`I
`Ade I i/26 (2014 09)‘ A631“ 1 17211
`(2014.09); 1111531r 13/423 (2014.09); 00617
`3/11” (2013.01); G061“ 3/01} (2013.01); G06!"
`3/017 (2013.01): 6'06!“ 3/04842 (2013.01);
`606.” 9/453 (2018.02): G061" 19/006
`(2013.01); A63F 2300:0082 (2013.01)
`
`Jason Skaamp
`Primm! Exmm'ner
`[74) Attorney, Agent, or I-l‘rm
`MaiL‘r 8L Maicr. P1111
`(57)
`ABSTRACT
`_
`.
`.
`_
`.
`A game processtng apparatus displays an image 01 Virtual
`space on a display that displays an image by using a sensor
`for identifying the position and direction 013 certain body
`part of a player and the display. A controller in the game
`processing. apparatus displays the image of the virtual space
`on the display in accordance with the position and direction
`of the certain body part 01‘ the player. and. when an infor-
`mation providing condition regarding the position and direc-
`tion ol'the certain body part of the player is satisfied. outputs
`torberprovided information.
`
`9 Claims, 5 Drawing Sheets
`
`
`
`Supercell
`Exhibit 1001
`
`Page 1
`
`Supercell
`Exhibit 1001
`Page 1
`
`

`

`US 10,406,432 B2
`Page 2
`
`(56)
`
`References Cited
`
`2015."02??552
`
`201650093105
`
`2016-02 5 9405
`2017-1028523?
`201150315825
`2018-0004285
`20 1810024633
`
`A1"
`
`Al“
`
`U.S. PATENT DOCUNLENTS
`10.12015 Wilajral
`352010 Rimon
`
`1104N 2134884
`386-244
`(3001' 19-006
`345-633
`(3061' 3-012
`912016 Wilson ......
`A1"
`6061-35013
`Al“ 1052017 Khalid "
`
`
`1152017r Gordon .....
`. (3061’ 16624575
`A1“
`1.52018 Castlcman
`1104L 65.4069
`A1 “‘
`
`Al“
`1.12018
`1,0
`GOGF 330482
`3455156
`
`OTHER PUBLICATIONS
`
`Ufiicc Action dated May 29. 2018 of comesp-onding Japanese
`application No. 115015483329; 9 pgs.
`
`* cilcd by examiner
`
`Supercell
`Exhibit 1001
`
`Page 2
`
`Supercell
`Exhibit 1001
`Page 2
`
`

`

`U.S. Patent
`
`Sep. 10,2019
`
`Sheet] of 5
`
`US 10,406,432 BZ
`
`FIG. 1
`
`SENSOR
`
`DISPLAY
`
`HMD
`CONTROLLER
`
`INPUTI‘OUTPUT IIF UNIT
`
`LOUDSPEAKER
`
`
`
`INPUTI’OUTPUT UNIT
`
`CONTROLLER
`
`GAZE POSITION
`IDENTIFYING UNIT
`
`GAME MANAGER
`
`SPACE IMAGE
`OUTPUT UNIT
`
`INFORMATION
`PROVIDER
`
`GAME PROCESSING APPARATUS
`
`21
`
`22
`
`STORAGE UNIT
`
`VIRTUAL SPACE
`IMAGE INFORMATION
`
`OBJECT
`INFORMATION
`
`PROVISION COND TION
`INFORMATION
`
`GAZE POSITION
`INFORMATION
`
`GAZE TIME
`INFORMATION
`
`TOBEPROVIOEO
`INFORMATION
`
`Supercell
`Exhibit 1001
`
`Page 3
`
`Supercell
`Exhibit 1001
`Page 3
`
`

`

`US. Patent
`
`Sep. 10,2019
`
`Sheet 2 0f 5
`
`us 10,406,432 32
`
`FIG. 2
`
`.
`
`.
`
`.
`
`-
`
`—
`
`Haltttttttttttttttttttttttttttttttt.wllllllllErr...
`I—fl...;.0:-4
`rd:.II[I
`
`:1..._II...
`It:_:rJr
`:4:_11111
`
`_u--
`
`III.—luff-I
`1::_rrrrr
`fl_Ill
`......_III;
`
`t:.:r1r
`tn.JJJr
`
`FFIII|IIIIIIIIltlIllllllllltlllllllllll{tillllllllllllllllllll|l|lllIIUU
`
`‘|I|
`
`J
`
`u..u.n].—____"0M\nU"1.u.n
`uu\I.\.ma.11;uaJ.2..7:0“1.u1nPw.u.e~_..Ilu___.—-
`.u\uu.nu‘\-1—nI_.D,\_a.nu\
`nm.
`
`II\I
`
`.\I\
`
`Supercell
`Exhibit 1001
`
`Page 4
`
`Supercell
`Exhibit 1001
`Page 4
`
`

`

`U.S. Patent
`
`Sep. 10,2019
`
`Sheet 3 0f 5
`
`US 10,406,432 BZ
`
`FIG. 3
`
`
`
`Si
`
`82'
`
`SELECTiNG
`OBJECT i8
`COMPLETED?
`
`
` lDENT§FY GAEE POSiTiON
`
`
`
`
`
`
`
`MiSSiON
`ACCOMPUSHING CON81T¥0N
`IS SA?!SFiED?
`
`NQ
`
`
`
`-
`
`
`
`HS. 4
`
`
`
`
`
`
`
`
`830
`
`
`
`INFGRMANON
`PROVISEON CONDENGN ES
`-
`SATISFI ED?
`
` EXECUYE ENFGRMAWON PROViSiflN
`
`
`
`Supercell
`Exhibit 1001
`
`Page 5
`
`Supercell
`Exhibit 1001
`Page 5
`
`

`

`U.S. Patent
`
`Sep. 10,2019
`
`Sheet 4 0f 5
`
`US 10,406,432 BZ
`
`
`
`FIG. 6
`
`110
`
`
`
`Supercell
`Exhibit 1001
`
`Page 6
`
`Supercell
`Exhibit 1001
`Page 6
`
`

`

`US. Patent
`
`Sep. 10,2019
`
`Sheet 5 0f 5
`
`US 10,406,432 El
`
`Q
`::
`
`2 V
`C)
`I—
`_
`5.953
`E23I—«TEO
`«2%>_
`H
`
`0‘—
`\—
`
`ED
`[\.
`
`LL
`
`Ix.
`
`Ll_
`
`Supercell
`Exhibit 1001
`
`Page 7
`
`Supercell
`Exhibit 1001
`Page 7
`
`

`

`US 10,406,432 B2
`
`I
`VIRTUAL IMAGE DISPLAY PROGRAM,
`VIRTUAL “WAGE DISPLAY APPARATUS,
`AND VIRTUAL INIAGE DISPLAY METHOD
`
`FIlil ,l)
`
`The present invention relates to a virtttal image display
`program, a virtual image display apparatus. and a virtual
`image display method for displaying an image of virtual
`space on a display.
`
`BACKGROUND
`
`Virtual reality games played by displaying an image of
`vir1ual
`space on a head—mounted display (hereinafter
`referred to as “l 1M0“) wont on the head of a player are
`understood in the art (see Japanese Unexamined Patent
`Application Publication No. ”it—200162, for example). An
`example IlMl) may include, in an example configuration, a
`gyro sensor. with which the movement of the head of the
`player is detectable. When the player moves his/her head.
`the HMD changes the image displayed on the display in
`order to follow the movement of the head of the player.
`When the game image folloWs the movement of the head of
`the player as lllelltioned ltere, the player may stay more
`immersed in the game. and may accordingly be able to
`concentrate more on the game.
`Virtual reality games. like other games. may sometimes
`provide information to a user: forexample, they may display
`hints on the progress of the game, or may display menu
`items. However. this is most otten accomplished by display—
`ing a button. such as a menu button. together with an image
`of virtual space, such that when the button is selected,
`information may be provided. However, this results in a
`significantly reduced sense of immersion in the virtual
`space. Such a problem is not limited to virtual realin games,
`and may be present in a variety of displays of images of
`vir1ual space.
`
`SUMMARY
`
`According to an exemplary embodiment, a vir1ual image
`display program, a virtual image diSplay apparatus, and a
`virtual image display method capable of providing inl‘onna-
`tion while reducing the loss of a sense of immersion in
`virtual space may all be provided.
`invention.
`According to an embodiment of the present
`there may be provided a virtual image display program for
`displaying an image of virtual space on a display that
`displays an image by using a virtual image display appara
`tus. a detector for identifying a position and direction of a
`certain body part of a player. and the display. The virtual
`image display apparatus may include a controller and a
`storage limit where an imbrmation providing condition and
`to—be—provided information may be recorded. The program
`may cause the controller to function as a space image output
`unit and art infonnation provider. The space image output
`unit may display the image of the virtual space on the
`display in accordance with the position and direction of the
`certain body part of the player. The information provider
`may output the to—be-provided infrmnation when the infor-
`mation providing condition regarding the position and direc-
`tion of the certain body part of the player is satisfied.
`Accordingly. the information provider may execute infor—
`mation provision on the condition that
`the infomlation
`providing condition regarding the movement of the gaze
`position of the player is satisfied. According to such an
`
`to
`
`15
`
`2f]
`
`25
`
`30
`
`40
`
`50
`
`60
`
`2
`embodiment, there may he no need to display a button for
`executing information provision on an image of the virtual
`space.
`In doing so,
`infonnation can be provided while
`reducing the loss of a sense of immersion in the virtual
`space.
`According to various exemplary embodiments, the virtual
`space may include a target object selectable by a gaze ofthe
`player. an area including the target object may be recorded
`in the storage unit, the information providing condition may
`include the fact
`that a game position identified from the
`position and direction ofthe certain body part of the player
`becomes outside the area. and the controller may output the
`to—be—provided information when the gaze position of the
`player becomes outside the area, Accordingly. information
`can be provided when selection of a target object is inter-
`rupted, such as when the player is not sure about how to
`progress the game. In doing so, information provision can be
`executed with the appropriate timing.
`According to various exemplary embodiments, the virtual
`space may include a target object selectable by a gave of the
`player. the information providing condition may include the
`fact that a gaze position identified from the position and
`direction of the certain body part of the player moves outside
`the target object before the player completes selecting the
`target object. and the controller may output
`the to-be-
`provided infon'nation wlten the gaze position of the player
`moves outside the target object before the player completes
`Selecting the target object, Accordingly, information can be
`provided when the player fails in selecting a target object. in
`doing so. information provision can be executed with the
`appropriate timing.
`According to various exemplary embodiments. 3 refer—
`ence range regarding movement of the certain body part of
`the player may be recorded as the information providing
`condition in the storage unit,
`the information providing
`condition may include the fact that
`the movement of the
`certain body part ofthe player becomes outside the reference
`range, and the controller may output the to-be-provided
`information when the movement of the certain body part of
`the player becomes outside the reference range. Accord—
`ingly. the information providing condition may be satisfied
`when the movement range of the certain body part is too
`narrow or too wide, In doing so, information provision can
`be executed with the appropriate timing.
`According to various exemplary embodiments. a prede—
`termined movement of the certain body part of the player
`may he recorded in the storage unit, the inlionnation pro-
`viding condition may include the fact that movement of the
`certain body part of the player corresponds to the predeter—
`mined movement recorded in the storage unit. and the
`controller may output the to—be—provided information when
`the movement of the certain body part of the player corre-
`sponds to the predetermined movement. Accordingly,
`the
`information providing condition may be satisfied when the
`movement range of the certain body part corresponds to the
`predetermintxi movement. In doing so, information provi-
`Sion can be executed with the appropriate timing.
`According to an exemplary embodiment, the information
`provider may display the to—be—provided information on an
`object in the virtual space. Accordingly. infonnation can be
`provided to the player in a natural form.
`According to an exemplary embodiment. the information
`provider may display the to—be—provided information in a
`direction in which a moving body moves, the moving body
`being an object in the virtual space and moving in the virtual
`space. in doing so,
`information can be provided without
`reducing a sense of immersion in the virtual space.
`
`Supercell
`Exhibit 1001
`
`Page 8
`
`Supercell
`Exhibit 1001
`Page 8
`
`

`

`US 10,406,432 B2
`
`3
`image display pro-
`Therefore. according to the vinual
`gram, the virtual image display apparatus, and the virtual
`image display method according to the embodiments of the
`present invention. information can be provided while reduc—
`ing the loss of a sense of immersion in a vir1ual space.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`Exemplary FIG. 1 is a schematic diagram describing an
`exemplary embodiment of a virtual image display system.
`[Exemplary FIG. 2 is a conceptual diagram illustrating an
`exemplary embodiment ofa gameable area in virtual space.
`Exemplary FIG. 3 is a flowchart illustrating an exemplary
`embodiment of the procedure of a game progressing pro-
`LX255.
`
`[Exemplary FIG. 4 is a flowchart illustrating, an exemplary
`embodiment of the procedure of an information providing
`process.
`Exemplary [-‘IG. 5 is a schematic diagram illustrating an
`exemplary embodiment of an image that may be displayed
`on a virtual image display.
`Exemplary FIG. 6 is a schematic diagram illustrating an
`exemplary embodiment of an image that may be displayed
`on a virtual image display. wherein a hint is displayed.
`Exemplary FIG. 7A is a diagram illustrating an exemplary
`embodiment of an image that may be displayed 011 a virtual
`image display, in a state where attention is being paid to a
`target object.
`Exemplary FIG. TB is a diagram illustrating an exemplary
`embodiment ofan image that may be displayed on a virtual
`image display. in a state where a hint is displayed.
`
`DETAILED DESCRIPTION
`
`image display program. a virtual
`Hereinafier. a virtual
`image display apparatus. and a virtual image display method
`according to a
`first exemplary embodiment will be
`described. According to an exemplary embodiment,
`the
`vir1ual image display apparatus may be implemented as a
`game processing apparatus that progresses a game while
`displaying an image of virtual space. The garneplay of this
`game may progress according to the assumption that
`the
`player will wear a head-mounted display (I 1MB) on his/her
`head.
`Referring to FIG. 1, the configuration of a system includ—
`ing an HMD 10 and a game processing apparatus 20 will be
`described. At first. the configuration of the [IMD ll] con-
`nected to the game processing apparatus 20 will be
`described. The HMD 10 may include an HMD controller 11,
`a sensor 12 serving as a detector. and an inputfcutput
`interface unit
`(L’F unit) 13. The HMD controller 11 may
`output various types of information to the game processing
`apparatus 20 and may receive various types of information
`from the game processing apparatus 20 via the inputtoutput
`IIF unit 13. The sensor 12 may be or may include a detector
`for identifying the position and direction of the head of the
`player. According to an exemplary embodiment, the sensor
`12 may include at least one ofa gyro sensor, an acceleration
`sensor, and a geomagnetic sensor, and may detect
`the
`movement of the head of the player. Alternatively. a device
`that directly detects the movement of the eyeballs of the
`player, such as a line-of-sight detecting sensor that emits
`near—infrared light into the iris and detects its reflected light,
`may be used as the sensor 12. According to an exemplary
`embodiment, the head of the player may be the certain body
`part ofa player that has its position and direction identified
`and tracked.
`
`If]
`
`15
`
`2f]
`
`25
`
`30
`
`40
`
`50
`
`60
`
`4
`The IIMI) 10 may include a display 14 that displays an
`image. and a loudspeaker 15 that outputs sound. For
`example, according to an exemplary embodiment. the dis
`play 14 may display an image to be visually recognized by
`the left eye and an image to be visually recognized by the
`right eye with a parallax. This may allow the player to
`visually recognize a stereoscopic image while perceiving a
`sense of depth. The HMD controller 11 may display on the
`display 14 an image in accordance with an image signal
`received from the game processing apparatus 20 via the
`inpua’ontpnt lfl" unit 13. According to an exemplary embodi-
`ment. the loudspeaker 15 may be. for example, a headphone.
`and may be used to. for example, output sound efi'ects and
`sound that describe the progress of the game.
`Next, the configuration of the game processing apparatus
`20 will be described. The game processing apparatus 20 may
`include a controller 21. a storage unit 22. and an inputfoutput
`unit 23. The controller 21 may include hardware elements
`including a central processing unit (CPU), random-access
`memory (RAM), and read-only memory (ROM). The con-
`troller 21 may include a gaze position identifying unit 24. a
`game manager 25, a space image output unit 26. and an
`infonnation provider 27. According to an exemplary
`embodiment,
`the controller 21 may cause the hardware
`elements to execute the virtual image display program to
`function as the gaze position identifying unit 24, game
`manager 25, space image output unit 26, and infomiation
`provider 27.
`According to an exemplary embodiment, the storage unit
`22 may store virtual space image information 3|). object
`information 31, provision condition information 32, gaze
`position information 33. gaze time information 34, and
`to-bc—provided information 35.
`According to an exemplary embodiment. the virtual space
`image information 30 may be data for rendering an image of
`the virtual space background and objects. The objects may
`include target objects, which are selectable, and nonrselectr
`able objects. In an exemplary embodiment, the player may
`be able to select a target object by continuously gazing at the
`target object for a certain period oftime.
`According to an exemplary embodiment, the object infor—
`mation 31 may be information that indicates each object’s
`attributes. Specifically,
`the object
`information 31 may
`inclttde each object’s identifier. coordinates in the virtual
`space. type. and selection state. The object‘s type may be.
`for example, information that indicates whether the object is
`a target object or an object other than a target object. The
`object’s selection state may he infon‘nalion that indicates,
`when the object is a target object, whether the object is
`selected or not.
`According to an exemplary embodiment. the provision
`condition information 32 may be inlonnation that defines an
`infonnation providing condition, which may be defmed as a
`condition for providing information to the player. According
`to an exemplary embodiment. the provision condition infor—
`mation 32 may be infonnation that defines, when the time
`elapsed since the start of the game is a certain titne or longer,
`the gaze position o f the player is outside a gamaible area set
`in the virtual space. The gameable area may be an area
`where the player can select a target object by gazing at that
`object, and may include all target objects. For example, the
`provision condition information 32 may include coordinate
`information of the gameable area. When the infomiation
`providing condition is satisfied, the player may be given a
`hint on the progress ofthe game.
`According to an exemplary embodiment, the gaze posi-
`tion infemiatien 33 may be infomiation that indicates the
`
`Supercell
`Exhibit 1001
`
`Page 9
`
`Supercell
`Exhibit 1001
`Page 9
`
`

`

`US 10,406,432 B2
`
`5
`in terms of coordi—
`identified gaze position of the player,
`nates, in the virtual space. The gaze time information 34 may
`be information that indicates a period of time in which the
`player continuously gazes at a target object. The to—be—
`provided in fonnation 35 may include information that indi-
`cates the details ofa hint for accomplishing a mission of the
`game.
`According to an exemplary embodiment. the gaze posi—
`tion identifying unit 24 may receive a detection signal
`obtained by the sensor 12. from the ”MD controller 11 of
`the [IMD lll, worn on the player, via the inputr'otttput unit
`23. The gaze position identifying unit 24 may identify the
`gaze position. in terms of coordinates. in the virtual space in
`accordance with the received detection signal. and may
`update the gaze position information 33 with the identified
`gaze position.
`The game manager 25 may compare the gaze position
`information 33 with the object
`information 31 and may
`deterntine Whether the gaze position is on a target object,
`When the game manager 25 determines that the gaze posi-
`tion is on a target object. the game manager 25 may measure
`a time (gaze time) in which the player continuously gazes at
`the target object, and may update the gaze time infonnalion
`34 with the gaze time. The game manager 25 may also
`determine whether the gaze time. indicated by the gaze time
`information 34. has reached a certain predetermined time
`(such as, for example. three seconds). When the gaze time
`information 34 has reached the certain predetermined time,
`the game manager 25 may determine that selection of the
`target object is completed.
`The game manager 25 also detennines whether a mission
`ofthe game has been completed. For example. according to
`an exemplary embodiment.
`the game manager 25 may
`determine that a mission has been accomplished once there
`are a certain number of objects that have been completely
`selected within a limit time. However. when the time that
`has elapsed since the stan of the game reaches a limit time
`while the number of target objects that have been completely
`selected is less than the target score, the game manager 25
`may determine that the game is over.
`According to an exemplary embodiment, the game man—
`ager 25 may further determine whether the information
`providing condition is satisfied on the basis of the provision
`condition information 32. An example information provid-
`ing condition may be a condition where the time that has
`elapsed since the start of the game exceeds a certain prede—
`termined time. and the gaze position of the player is outside
`the garncable area. the certain predetermined time may be
`set as, for example, a time at which there is very little
`remaining time until
`the limit
`time, or an average time
`required for the player to accomplish the mission. When the
`game manager 25 determines that the information providing
`condition is satisfied. the game manager 25 may output to
`the information provider 27 a request for outputting the
`to—be—provided information 35.
`'lhe game manager 25 may also extract audio infomiation
`such as sound elfccts and sound that describes the progress
`of the game from an audio information storage tmit (not
`illustrated], and may transmit the audio information to the
`HMD ll) via the inputtoutput ttnit 23.
`The space image output unit 26 may extract the virtual
`space image information 30 within a lield-of-view range
`around the coordinates indicated by the gaze position infor—
`mation 33, and may transmit the extracted virtual space
`image information 30 as an image signal to the HMD 10.
`the space image output unit 26 may also display a character
`corresponding to the player in the virtual space. When the
`
`to
`
`15
`
`2t]
`
`25
`
`30
`
`40
`
`50
`
`60
`
`6
`game manager 25 determines that selection of a target object
`is completed. the space image output unit 26 may change the
`display form of the target object. which has been completely
`selected.
`According to an exemplary embodiment. upon receipt of
`a request for displaying a hint, which may be output from the
`game manager 25. the information provider 27 may read the
`to—be—provided infomtation 35. The information provider 2'?
`may output a display corresponding to the to-be—provided
`information 35 to the display 14. and may display the output
`display on a virtual space image.
`Referring next to FIG. 2. an object selecting function
`performed by the game processing apparatus 20 will be
`described.
`The gaze position identifying ttnit 24 may receive. from
`the HMD 10, a detection signal obtained by the sensor 12,
`and may identify the position and direction of the head of a
`player 101 in accordance with the detection signal. Specifii
`Cally, the gaze position identifying tutit 24 may calculate the
`position and direction of the head as a position in three-axis
`directions and an angle around each axis. This may include.
`for example, determining the position of the bead along the
`X-axis along a horizontal direction and a pitch Hp ofthe head
`indicating a rotation direction around the X-axis, determin-
`ing the position of the head along the Yaaxis along a vertical
`direction and a yaw 0y of the head indicating a rotation
`direction around the Y—axis, and determining the position of
`the head along the ’Z-axis along a depth direction and a roll
`Hr of the head indicating a rotation direction around the
`Z—axis.
`According to an exetnplary embodiment, the gaze posi—
`tion identifying unit 24 may identify a gaze position P1 of
`the player 101 in the virtual space, on the basis of the
`calculated position and direction of the head. A method of
`the related art may be used as a method of identifying the
`gaze position. For example.
`the gaze position identifying
`unit 24 may virtually set a straight line connecting the left
`eye position and the right eye position, and may set, as a
`line-of-sight direction D], a direction that extends in a
`direction orthogonal to the straight line and that crosses the
`midpoint of the left eye position and the right eye position.
`The game position identifying ttnit 24 may further identify
`the intersection between the line-of-sight direction DI and
`an object 102 (or background) as the gaze position P1. When
`a device that detects the eyeball movement or light reflected
`from the interior of the iris is used as the sensor 12, the
`line-of-sight direction may be identified on the basis of the
`eyeball movement or the reflected light.
`According to an exemplary embodiment, a gameable area
`105 may be provided in the virtual space. The gameable area
`105 may be set around a character corresponding to the
`player. Since the gameable area 105 may include all target
`objects 103, the gaze position P1 may be located in the
`gameable area 105 when the player gazes at one ofthe target
`objects 103 to select that target object 103.
`According to an exemplary embodiment, the space image
`output unit 26 may extract the virtual space image informa-
`tion 30 within a field-of-view range around the gaZe position
`P1, and may transmit
`the extracted virtual space image
`information 30 as an image signal to the HMD controller 11.
`The “MD controller [1 may otttpttt an image in accordance
`with the received virtttal space image information 30 to the
`display 14.
`Referring next to FIG. 3, a process by which a game may
`progress will be described. According to an exemplary
`embodiment, a condition for ending the process may be one
`of the following: the satisfaction of the mission accomplish-
`
`Supercell
`Exhibit 1001
`
`Page 10
`
`Supercell
`Exhibit 1001
`Page 10
`
`

`

`US 10,406,432 B2
`
`5
`
`2n
`
`7
`ing condition, and the elapsed time since the start of the
`game reaching the limit time. The process may be repeated
`until this ending condition is satisfied. The elapsed time may
`be measured by the game manager 25 until the game ends.
`and may be measured on the basis of a system time (for
`example, using a clock signal of the CPU).
`When the game starts, the controller 21 may identify the
`gaze position of the player (step Sl). Specifically. the gaze
`position identifying Lmit 24 in the controller 21 may identify
`the gaze position of the player using the above-mentioned to
`method, and may update the gaze position information 33
`with the identified gaze position.
`Using the gaze position information 33, the controller 2]
`may determine whether selection ofa target object is com-
`pleted (step 32). Specifically, the game manager 25 in the 15
`controller 21 tnay read the object information 3] ofa target
`object and the gaze position) infonnation 33 from the storage
`unit 22. may compare the coordinates of the target object
`with the gaze position, and may determine whether the gaze
`position is on the target object.
`When the gaze position is on the target object. the game
`manager 25 may measure the gaze time using the system
`time or the like, and may update the gaze time information
`34 with the measured gaze time. When the gaze time roaches
`a certain predetermined time interval. the game manager 25 25
`may determine that selection is completed.
`According to an exemplary embodiment, when the gaze
`position becomes outside the target object while the gaze
`time has not yet met
`the appropriate predetermined time
`interval. the game manager 25 may determine that selection 30
`of the target object has not been completed (“NO” in step
`82). and may reset the gaze time and return to step 51.
`When the game manager 25 determines that selection of
`the target object is completed (“YliS" in step 52), the game
`manager 25 may update the selection status of the object 35
`information 31 to “selection completed”. and may proceed
`to step S3. At this time. the space image output unit 26 may
`change the display form ofthe target object. to reflect that it
`has been completely selectedi
`is completed. the 40
`When selection of the target object
`controller 21 may determine whether the mission accom—
`plishing condition has been satisfied (step 53) Specifically.
`the game manager 25 in the controller 2] may refer to the
`selection status included in the object information 3.1. and
`may determine whether there are a certain number of target 45
`objects that have been completely selected within the limit
`time of the game. This ntu‘nber may be called a "target object
`target score.“
`When the number oftarget objects that have been corn—
`pletely selected is at least the target object target score. the 50
`controller 21 may detennine that the mission accomplishing
`condition has been satisfied (““18” in step SJ} This may
`satisfy the condition for ending the game progressing pro-
`cess, and the game thus may be caused to end.
`When the number of target objects that have been com— 55
`pletely selected is less than the target object target score, the
`controller 2] may determine that the mission accomplishing
`condition is not satisfied (“Nt)” in step 33), and may return
`to step S1. Specifically, when the game manager 25 in the
`controller 21 determines that
`the mission accomplishing no
`condition is not satisfied. the game manager 25 may identify
`the gaze position in step 51.
`Referring next to FIG. 4, an exemplary procedure ofan
`information providing process will be described. According
`to an exemplary embodiment. the process may be a process 65
`executed independently and in parallel with the game pro-
`gressing process.
`
`8
`According to an exemplary embodiment. a condition for
`ending the process may be one of the following: the satis—
`faction ofthe mission accomplishing condition, and the time
`elapsed since the start of the game reaching the limit time
`before the mission is accomplished. 'lhe process may be
`repeated until this ending condition is satisfied.
`The controller 21 in the game processing apparatus 20
`may determine whether the information providing condition
`has been satisfied (step 310). Specifically, the game manager
`25 in the controller 21 may detemline whether the elapsed
`time has reached a prtxletermined time interval and the gaze
`position is outside the gameable area.
`To do so. the game manager 25 may obtain the amount of
`elapsed time, and may determine Whether the elapsed time
`has reached a predetemiined time interval. When it
`is
`determined that the elapsed time has reached the predeter—
`mined tirne interval. the game manager 25 may read the gaze
`position im'ormation 33 and the provision condition infori
`mation 32 from the storage unit 22, and may determine
`whether the game position is outside the gameable area.
`When it is determined that the gaze position is outside the
`gameable area, the game manager 25 may determine that the
`infonnation providing Condition has been satisfied.
`When it
`is determined that
`the in formation providing
`condition has been satislied (“Y]".S” in step 5] 1}). the con-
`troller 21 may execute the information provision (step 811).
`Specifically, according to an exemplary embodiment, the
`game manager 25 in the controller 21 may otttpnt an
`information provision request to the information provider
`27. Upon receipt of the infonnation provision request, the
`information provider 2’? may read the to—be—provided infor—
`mation 35 from the storage unit 22. and may output the
`to-be—provided information 35 to the [IMD 10.
`In contrast. when the information providing condition has
`not yet been determined to be satisfied (“NO” in step SID).
`the controller 21 may repeatedly determine whether the
`information providing condition has been satisfied (step
`Sltl).
`According to an exemplary embodiment, a process used
`to provide infonnation may be linked with a process used to
`progress the game. For example, when selection ofa target
`object is completed in the game progressing process (“YliS"
`in step