www.uspto.gov
`
`UNITED STATES DEPARTMENT OF COMMERCE
`United States Patent and Trademark Office
`Address: COMMISSIONER FOR PATENTS
`P.O. Box 1450
`Alexandria, Virginia 22313-1450
`
`17/922,919
`
`11/02/2022
`
`Junji OTSUKA
`
`1946-1828
`
`4998
`
`Paratus Law Group, PLLC
`1765 Greensboro Station Place
`Suite 320
`Tysons Corner, VA 22102
`
`MCCULLEY, RYAN D
`
`2611
`
`MAIL DATE
`
`11/08/2024
`
`PAPER NUMBER
`
`DELIVERY MODE
`
`PAPER
`
`Please find below and/or attached an Office communication concerning this application or proceeding.
`
`The time period for reply, if any, is set in the attached communication.
`
`PTOL-90A (Rev. 04/07)
`
`

`

`Application No.
`Applicant(s)
`171922919
`OTSUKA etal.
`
`Office Action Summary Art Unit|AIA (FITF)StatusExaminer
`Ryan McCulley
`2611
`Yes
`
`
`
`-- The MAILING DATEof this communication appears on the cover sheet with the correspondence address --
`Period for Reply
`
`A SHORTENED STATUTORYPERIOD FOR REPLYIS SET TO EXPIRE 3 MONTHS FROM THE MAILING
`DATE OF THIS COMMUNICATION.
`Extensionsof time may be available underthe provisions of 37 CFR 1.136(a). In no event, however, may a reply betimely filed after SIX (6) MONTHSfrom the mailing
`date of this communication.
`If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHSfrom the mailing date of this communication.
`-
`- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133).
`Any reply received by the Office later than three months after the mailing date of this communication, evenif timely filed, may reduce any earned patent term
`adjustment. See 37 CFR 1.704(b).
`
`Status
`
`
`
`1)™) Responsive to communication(s)filed on 09 September 2024.
`C) A declaration(s)/affidavit(s) under 37 CFR 1.130(b) was/werefiled on
`2a)[¥) This action is FINAL.
`2b) (J This action is non-final.
`3) An election was madeby the applicant in responseto a restriction requirement set forth during the interview
`on
`; the restriction requirement and election have been incorporated into this action.
`4)(2) Since this application is in condition for allowance except for formal matters, prosecution as to the merits is
`closed in accordance with the practice under Exparte Quayle, 1935 C.D. 11, 453 O.G. 213.
`
`Disposition of Claims*
`1-20 is/are pending in the application.
`)
`Claim(s)
`5a) Of the above claim(s) _ is/are withdrawn from consideration.
`C} Claim(s)__ is/are allowed.
`Claim(s) 1-20 is/are rejected.
`(] Claim(s)__ is/are objectedto.
`C] Claim(s
`are subjectto restriction and/or election requirement
`)
`* If any claims have been determined allowable, you maybeeligible to benefit from the Patent Prosecution Highway program at a
`participating intellectual property office for the corresponding application. For more information, please see
`http://www.uspto.gov/patents/init_events/pph/index.jsp or send an inquiry to PPHfeedback@uspto.gov.
`
`) ) ) )
`
`Application Papers
`10) The specification is objected to by the Examiner.
`11)0) The drawing(s) filedon__ is/are: a)(J accepted or b)( objected to by the Examiner.
`Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a).
`Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d).
`
`Priority under 35 U.S.C. § 119
`12)7) Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d)or (f).
`Certified copies:
`c)Z None ofthe:
`b)() Some**
`a)C All
`1.1.) Certified copies of the priority documents have been received.
`2.2) Certified copies of the priority documents have been received in Application No.
`3.1.) Copies of the certified copies of the priority documents have been receivedin this National Stage
`application from the International Bureau (PCT Rule 17.2(a)).
`*“ See the attached detailed Office action for a list of the certified copies not received.
`
`Attachment(s)
`
`1) [[] Notice of References Cited (PTO-892)
`
`2) (J Information Disclosure Statement(s) (PTO/SB/08a and/or PTO/SB/08b)
`Paper No(s)/Mail Date
`U.S. Patent and Trademark Office
`
`3)
`
`4)
`
`(LJ Interview Summary (PTO-413)
`Paper No(s)/Mail Date
`(Qj Other:
`
`PTOL-326 (Rev. 11-13)
`
`Office Action Summary
`
`Part of Paper No./Mail Date 20241107
`
`

`

`Application/Control Number: 17/922,919
`Art Unit: 2611
`
`Page 2
`
`DETAILED ACTION
`
`Notice of Pre-AlA or AIA Status
`
`The present application, filed on or after March 16, 2013,
`
`is being examined
`
`underthefirst inventor to file provisions of the AIA.
`
`Response to Amendment
`
`This Office Action is in response to Applicant's amendment/response filed on 09
`
`September 2024, which has been entered and made of record.
`
`Abstract
`
`The objection to the abstract is withdrawn in view of the replacement abstract.
`
`Claim Interpretation
`
`The claims are no longer interpreted under 35 U.S.C. 112(f) in view of the claim
`
`amendments.
`
`Claim Rejections - 35 USC § 112
`
`The previous rejections under 35 U.S.C. 112(b) are withdrawn in view of the
`
`claim amendments.
`
`Claim Rejections - 35 USC § 101
`
`The previous rejection of claim 20 under 35 U.S.C. 101 is withdrawn in view of
`
`the claim amendments.
`
`

`

`Application/Control Number: 17/922,919
`Art Unit: 2611
`
`Page 3
`
`Response to Arguments
`
`Applicant's arguments filed 09 September 2024 have beenfully considered but
`
`they are not persuasive.
`
`Applicant argues “there is nothing in Kiemele that would fairly teach or suggest
`
`the elements presently recited” and “Kiemele clearly does not teach or suggest” the
`
`claimed features (Remarks, pg. 15). However, Applicant does not provide any
`
`reasoning as to how thecitations from Kiemele (provided in the previous Office Action
`
`and updated here) fail to teach the claim limitations. Kiemele recites “a visible-light
`
`camera configured to collect a visible-light
`
`image of a physical space” (para. 53).
`
`Kiemele further describes using a touch input, hand gestures, and even physical
`
`movementof the input device to create and update a virtual object in both displays (see
`
`Fig. 4). All of these types of input represent 3D information of areal space around a
`
`user. Therefore, Kiemele teaches acquiring three-dimensional
`
`information of a real
`
`space around a user, and dynamically changing a parameterof a virtual object based
`
`on the 3D information.
`
`Any remaining arguments are considered moot based on the foregoing.
`
`Claim Rejections - 35 USC § 102
`
`The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that
`
`form the basis for the rejections under this section madein this Office action:
`
`A person shall be entitled to a patent unless —
`
`(a)(1) the claimed invention was patented, described in a printed publication, orin public use,
`on sale, or otherwise available to the public beforethe effectivefiling date of the claimed
`invention.
`
`

`

`Application/Control Number: 17/922,919
`Art Unit: 2611
`
`Page 4
`
`Claims 1-4 and 18-20 are rejected under 35 U.S.C. 102(a)(1) as being
`
`anticipated by Kiemele et al. (US 2019/0065026; hereinafter “Kiemele’).
`
`Regarding claim 1, Kiemele discloses An information processing apparatus
`
`comprising: circuitry (“one or more processors,” para. 64) configured to acquire
`
`three-dimensional information of a real space around a user (“User input received
`
`by the touch sensor mayinclude touch, hover, and/or hand gesture input,” para. 15;
`
`“employ a visible light and/or depth camera to locate and segment image data
`
`corresponding to input device 110,” para. 17; “movementof touch-sensitive input device
`
`may effect the rendering of visual content,” para. 30), and dynamically change at
`
`least one parameter related to display of a virtual object in an image (“a user can
`
`draw 2D or 3D shapes and imagesvia the application of touch input,” para. 27; “edit a
`
`representation of visual content,” para. 33), the at least one parameter controlling
`
`display of the virtual object on each display device of a plurality of display
`
`devices according to an image expression method of expressing the image, the
`
`method being assigned for display of the image by each display device of the
`
`plurality of display devices that display the image including the virtual object (see
`
`Figs. 4-5), wherein the at least one parameter is changed according to the
`
`acquired three-dimensional information of the real space around the user (Fig. 4
`
`illustrates a user input, which reflects a real space 3D position of a user’s finger/nand or
`
`a real space 3D position of the handheld device, resulting in changesto the virtual
`
`object 404 in both the handheld device and the HMD viewpoint).
`
`Regarding claim 2, Kiemele discloses wherein the plurality of display devices
`
`includea first display device that is controlled to display scenery of a real space
`
`

`

`Application/Control Number: 17/922,919
`Art Unit: 2611
`
`Page 5
`
`in which the virtual object is virtually arranged, the scenery being viewed from a
`
`first viewpoint defined as a viewpoint of the user in the real space (“an augmented-
`
`reality experience ... to give the wearer a view of the surrounding real-world physical
`
`space ... the user will see augmented-reality objects that are not actually present in the
`
`physical space,” para. 45; ), and a second display device that is controlled to
`
`display an image of the virtual object (“a representation of visual content rendered on
`
`touch-sensitive input device,” para. 20).
`
`Regarding claim 3, Kiemele discloses wherein the circuitry dynamically
`
`changes the parameterfor controlling the first display device according to the
`
`three-dimensional information of the real space around the user acquired froma
`
`real space information acquisition device (“representation 204 [of Fig. 2] is rendered
`
`in a 3D location proximate to input device 110 where the representation appears to float
`
`over the input device,” para. 20; “varying visual content rendered in response to
`
`movementof input device,” para. 30; “any suitable input, such as a hand gesturein
`
`which the user swipes from a location proximate to the input deviceinto the virtual
`
`reality experience,” para. 32; “The one or more outward facing image sensors 812A,
`
`812B may be configured to measure physical environmentattributes of a physical
`
`space,” para. 53).
`
`Regarding claim 4, Kiemele discloses wherein the real space information
`
`acquisition device includes at least one imaging device configured to image the
`
`real space around the user or a distance measuring device configured to acquire
`
`depth information of the real space around the user (“HMD device 104 [of Fig. 1]
`
`includes a sensor system 116 for tracking touch-sensitive input device 110 ... sensor
`
`

`

`Application/Control Number: 17/922,919
`Art Unit: 2611
`
`Page 6
`
`system 116 may employ a visible light and/or depth camera to locate and segment
`
`image data corresponding to input device 110,” para. 17).
`
`Regarding claim 18, Kiemele discloses wherein thefirst display device
`
`superimposes and displays an imageof the virtual object on an imageof thereal
`
`space, projects and displays the image of the virtual object in the real space, or
`
`projects and displays the image of the virtual object on a retina of the user (“he
`
`virtual-reality experience includes an augmented-reality experience in which the near-
`
`eye display is wholly or partially transparent from the perspective of the wearer, to give
`
`the wearer a view of the surrounding real-world physical space,” para. 45).
`
`Regarding claims 19 and 20, they are rejected using the same citations and
`
`rationales described in the rejection of claim 1.
`
`Claim Rejections - 35 USC § 103
`
`The following is a quotation of 35 U.S.C. 103 which forms the basis forall
`
`obviousness rejections set forth in this Office action:
`
`Apatent fora claimed invention may notbe obtained, notwithstanding thatthe claimed
`invention is not identically disclosed as set forth in section 102, if the differences between the
`claimed invention and the prior artare suchthat the claimed invention as a whole would have
`been obvious beforethe effective filing date of the claimed inventionto a person having
`ordinary skill in the art to which the claimed invention pertains. Patentability shall notbe
`negated by the manner in whichthe invention was made.
`
`Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Kiemele in
`
`view of Panseet al. (US 2021/0398316; hereinafter “Panse’”).
`
`Regarding claim 5, Kiemele does not disclose wherein whena region in which
`
`a shielding object located betweenthevirtual object and the user is present in the
`
`real space or a region in which the three-dimensional information cannot be
`
`

`

`Application/Control Number: 17/922,919
`Art Unit: 2611
`
`Page 7
`
`acquired is detected on the basis of the three-dimensional information, the
`
`circuitry is further configured to set the region as an occlusion region, and
`
`change a display position or a display form of the virtual object or a movement
`
`amountof the virtual object in moving image display on thefirst display device
`
`so as to reduce a region in whichthevirtual object and the occlusion region are
`
`superimposed.
`
`In the same art of augmented reality, Panse teaches wherein whena region in
`
`which a shielding object located between the virtual object and the useris
`
`presentin the real space ... the circuitry is further configured to set the region as
`
`an occlusion region, and changea display position or a display form of the virtual
`
`object ... on the first display device so as to reduce a region in whichthevirtual
`
`object and the occlusion region are superimposed (“as shown in FIG. 5E, a portion
`
`or an entirety of virtual object 54 may be positioned behind physical object 20 whereby
`
`physical object 20 blocks a visualization of such portion of virtual object 54 or an entirety
`
`of virtual object 54 ... virtual object 54 may alternatively be moved within augmented
`
`reality display,” para. 92).
`
`Before the effectivefiling date of the claimed invention,
`
`it would have been
`
`obvious to one having ordinary skill
`
`in the art to apply the teachings of Panse to
`
`Kiemele. The motivation would have been “fora functional visualization by a user of the
`
`AR device’ (Panse, para. 135).
`
`

`

`Application/Control Number: 17/922,919
`Art Unit: 2611
`
`Page 8
`
`Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over the
`
`combination of Kiemele and Panse, and further in view of Dedonato et al. (US
`
`2021/0150818; hereinafter “Dedonato’).
`
`Regarding claim 6, the combination of Kiemele and Panse doesnotdisclose
`
`wherein the circuitry controls the first display device so as to display another
`
`virtual object in an indefinite region where the three-dimensional information
`
`cannot be acquired.
`
`In the same art of augmented reality, Dedonato teaches wherein thecircuitry
`
`controls the first display device so as to display anothervirtual object in an
`
`indefinite region where the three-dimensional information cannot be acquired
`
`(“update the virtual content to include a graphic associated with the unmapped location;
`
`direct the user to observe the graphic,” para. 7).
`
`Before the effectivefiling date of the claimed invention,
`
`it would have been
`
`obvious to one having ordinary skill
`
`in the art to apply the teachings of Dedonato to the
`
`combination of Kiemele and Panse. The motivation would have been “forfacilitating
`
`virtual or augmented reality interaction” (Dedonato, para. 4).
`
`Claims 7 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over
`
`Kiemele in view of Kimura et al. (US 2021/0142552; hereinafter “Kimura’).
`
`Regarding claim 7, Kiemele does not disclose acquire position information
`
`including distance information and positional relationship information between
`
`the virtual object and the user in the real space, and wherein the circuitry
`
`

`

`Application/Control Number: 17/922,919
`Art Unit: 2611
`
`Page 9
`
`dynamically changes the parameterfor controlling thefirst display device
`
`according to the position information.
`
`In the same art of augmented reality, Kimura teaches acquire position
`
`information including distance information and positional relationship
`
`information between the virtual object and the user in the real space, and wherein
`
`the circuitry dynamically changes the parameterfor controlling the first display
`
`device according to the position information (“calculates the display distance from
`
`the user to the display position of the virtual object ... when the display format
`
`determination unit has determined that the calculated display distance is equal to or
`
`larger than a... sets the display resolution of the virtual object,” paras. 114-116).
`
`Before the effectivefiling date of the claimed invention,
`
`it would have been
`
`obvious to one having ordinary skill
`
`in the art to apply the teachings of Kimura to
`
`Kiemele. The motivation would have been to improve a user’s visual understanding of
`
`the virtual object.
`
`Regarding claim 8, the combination of Kiemele and Kimura renders obvious
`
`wherein the circuitry performs control such that a display area of the virtual
`
`object to be displayed on the first display device increases as a distance between
`
`the virtual object and the user increases (“a second distance shorter than thefirst
`
`distance ... when the display distance is equal to thefirst distance, the control unit
`
`increases display area of the virtual object as comparedto a casein which the display
`
`distance is equal to the second distance,” para. 229; see claim 7 for motivation to
`
`combine).
`
`

`

`Application/Control Number: 17/922,919
`Art Unit: 2611
`
`Page 10
`
`Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over the
`
`combination of Kiemele and Kimura, and further in view of Shinohara (US
`
`2011/0261076).
`
`Regarding claim 9, the combination of Kiemele and Kimura does disclose
`
`wherein the circuitry performs control such that a display change amountin
`
`moving image display of the virtual object to be displayed on thefirst display
`
`device increases as a distance betweenthevirtual object and the userincreases.
`
`In the same art of displaying virtual objects, Shinohara teaches wherein the
`
`circuitry performs control such that a display change amount in moving image
`
`display of the virtual object to be displayed on thefirst display device increases
`
`as a distance betweenthevirtual object and the user increases (“the viewpoint
`
`movement speed determination means determines the movement speed so that the
`
`movement speed increases in value as the virtual object-to-viewpoint distance
`
`increases in value. With this configuration, the view field flows at a lower speed as the
`
`distance between the virtual object and the viewpoint becomes smaller,” para. 24).
`
`Before the effectivefiling date of the claimed invention,
`
`it would have been
`
`obvious to one having ordinary skill
`
`in the art to apply the teachings of Shinohara to the
`
`combination of Kiemele and Kimura. The motivation would have beento display
`
`movement“more precisely” (Shinohara, para. 24).
`
`Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over the
`
`combination of Kiemele and Kimura, and further in view of Mathey-Owensetal. (US
`
`2018/0005438; hereinafter “Mathey-Owens’).
`
`

`

`Application/Control Number: 17/922,919
`Art Unit: 2611
`
`Page 11
`
`Regarding claim 10, the combination of Kiemele and Kimura doesnot disclose
`
`wherein the circuitry performs control to further smooth a trajectory in moving
`
`image display of the virtual object to be displayed on thefirst display device as a
`
`distance betweenthe virtual object and the user increases.
`
`In the same art of displaying virtual objects, Mathey-Owens teaches wherein the
`
`control section performs control to further smootha trajectory in moving image
`
`display of the virtual object to be displayed on thefirst display device as a
`
`distance betweenthe virtual object and the user increases (“the smoothing
`
`algorithm is configured to ‘smooth’ the motion and/or rotation of the virtual element,”
`
`para. 27; “perform less smoothing when the user focuses on the large virtual balloon
`
`that is closer .... perform more smoothing when the user focuses on the small virtual
`
`balloon that is further away,” para. 28).
`
`Before the effectivefiling date of the claimed invention,
`
`it would have been
`
`obvious to one having ordinary skill
`
`in the art to apply the teachings of Mathey-Owensto
`
`the combination of Kiemele and Kimura. The motivation would have been to “improve
`
`the user experience” (Mathey-Owens,para. 21).
`
`Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over the
`
`combination of Kiemele and Kimura, and further in view of Chang et al. (US
`
`2016/0163110; hereinafter “Chang”).
`
`Regarding claim 11, the combination of Kiemele and Kimura doesnot disclose
`
`the circuitry dynamically changes a display change amountof the virtual object
`
`

`

`Application/Control Number: 17/922,919
`Art Unit: 2611
`
`Page 12
`
`to be displayed on thefirst display device, the display change amount being
`
`changed byan input operation of the user, according to the position information.
`
`In the same art of displaying virtual object, Chang teaches thecircuitry
`
`dynamically changes a display change amountofthevirtual object to be
`
`displayed on thefirst display device, the display change amount being changed
`
`by an input operation of the user, according to the position information (“the
`
`adjustment speedin the size of the virtual object is determined according to the
`
`movement speed of the user ... and the distance between the user andthevirtual
`
`object,” para. 38).
`
`Before the effectivefiling date of the claimed invention,
`
`it would have been
`
`obvious to one having ordinary skill
`
`in the art to apply the teachings of Chang to the
`
`combination of Kiemele and Kimura. The motivation would have beento provide
`
`additional control to a user.
`
`Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Kiemele in
`
`view of Lindeman (US 2016/0196694).
`
`Regarding claim 12, Kiemele does not disclose the circuitry controls the
`
`second display device so as to display an imageof the virtual object visually
`
`recognized from a second viewpoint different from the first viewpoint in the real
`
`space.
`
`In the same art of augmented reality, Lindeman teaches thecircuitry controls
`
`the second display device so as to display an image of the virtual object visually
`
`recognized from a second viewpoint different from the first viewpoint in the real
`
`

`

`Application/Control Number: 17/922,919
`Art Unit: 2611
`
`Page 13
`
`space(“the virtual world on the tablet can be generated from a different point of view
`
`than the virtual world of the HMD,” para. 57).
`
`Before the effectivefiling date of the claimed invention,
`
`it would have been
`
`obvious to one having ordinary skill
`
`in the art to apply the teachings of Lindeman to
`
`Kiemele. The motivation would have been to allow a user more control of the
`
`visualization.
`
`Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over the
`
`combination of Kiemele and Lindeman, and further in view of Shpigelman (US
`
`2015/0302651).
`
`Regarding claim 13, the combination of Kiemele and Lindeman does not disclose
`
`wherein the second viewpointis virtually arranged on the virtual object.
`
`In the same art of augmented reality, Shpigelman teaches wherein the second
`
`viewpointis virtually arranged on thevirtual object (“A camera position may be
`
`associated with the displaying of the first or second of the two or more scenes, and the
`
`position maybe a point of view of a character within the respective first or second of the
`
`two or more scenes. The method mayfurther include selecting a chapter by default or
`
`receiving a selection from a user of a character whose point of view is to be adopted,
`
`and the displaying may include placing the virtual camera at the location of the selected
`
`character. The method may further include displaying an avatar associated with a user
`
`in the displayed first or second of the two or more scenes,” para.8).
`
`Before the effectivefiling date of the claimed invention,
`
`it would have been
`
`obvious to one having ordinary skill
`
`in the art to apply the teachings of Shpigelman to
`
`

`

`Application/Control Number: 17/922,919
`Art Unit: 2611
`
`Page 14
`
`the combination of Kiemele and Lindeman. The motivation would have been “the user
`
`may advantageously employ the VR/AR functionality to become immersed”
`
`(Shpigelman, para. 33).
`
`Claims 14-17 are rejected under 35 U.S.C. 103 as being unpatentable over
`
`Kiemele in view of Zhu et al. (“BISHARE: Exploring Bidirectional Interactions Between
`
`Smartphones and Head-Mounted Augmented Reality”; hereinafter “Zhu’).
`
`Regarding claim 14, Kiemele does not disclose wherein the circuitry changes
`
`a display change amountofthe virtual object to be displayed on eachofthefirst
`
`and second display devices in moving image display according to the method of
`
`expressing the image assigned to each of the first and second display devices for
`
`displaying the image.
`
`In the same art of multi-device augmented reality, Zhu teaches wherein the
`
`circuitry changes a display change amountof thevirtual object to be displayed
`
`on each of the first and second display devices in moving image display
`
`according to the method of expressing the image assigned to eachofthefirst
`
`and second display devices for displaying the image (“to support content transfer in
`
`a phone-centric context (D4P), content on the phone can be dragged using a touch
`
`operation, and once reaching the edge, the movement can continue as a freehand
`
`pinch gesture [38] (D5P). In a HMD-centric setting, a user could combine spatial
`
`movementsof the phoneitself, with 2D touchscreen gestures on the phone, to perform
`
`manipulations of spatial content,” pg. 5, sec. D5).
`
`

`

`Application/Control Number: 17/922,919
`Art Unit: 2611
`
`Page 15
`
`Before the effectivefiling date of the claimed invention,
`
`it would have been
`
`obvious to one having ordinary skill
`
`in the art to apply the teachings of Zhu to Kiemele.
`
`The motivation would have been “The combination of mobile + AR as an entire system
`
`is very good, intuitive” (Zhu, pg. 9, col. 1).
`
`Regarding claim 15, Kiemele does not disclose acquire a selection result
`
`indicating whether the user has selected oneof thefirst display device and the
`
`second display device as an input device, and wherein the circuitry dynamically
`
`changes a display change amountof the virtual object changed by an input
`
`operation of the user according to the selection result.
`
`In the same art of multi-device augmented reality, Zhu teaches acquire a
`
`selection result indicating whether the user has selected oneof thefirst display
`
`device and the second display device as an input device, and wherein the
`
`circuitry dynamically changes a display change amountofthe virtual object
`
`changed byan input operation of the user according to the selection result (“In a
`
`spatial-centric task, a user may wish to use the smartphone as a 6DOF device to
`
`perform ray casting for object selection of 3D components [41] (D1H). Using the
`
`touchscreen, recasting could be enhanced with gestures, for example, to cycle between
`
`overlapping objects, or define a subsequent manipulation,” pg. 5, col. 1; “Alternatively,
`
`objects in the spatial environment can be accessed using freehand gestures. An air tap
`
`and dragging gesture can be used to select and move an object in 3D space,” pg. 6, col.
`
`2).
`
`Before the effectivefiling date of the claimed invention,
`
`it would have been
`
`obvious to one having ordinary skill
`
`in the art to apply the teachings of Zhu to Kiemele.
`
`

`

`Application/Control Number: 17/922,919
`Art Unit: 2611
`
`Page 16
`
`The motivation would have been “The combination of mobile + AR as an entire system
`
`is very good, intuitive” (Zhu, pg. 9, col. 1).
`
`Regarding claim 16, the combination of Kiemele and Zhu renders obvious
`
`wherein the circuitry acquires the selection result on a basis of a detection result
`
`of a line-of-sight of the user from a line-of-sight detection device (“Another
`
`commonform of input in AR is Gaze [35], which is commonly combined with freehand
`
`gestures [35, 44, 54], such as the air tap technique used for selection,” Zhu, pg. 2, col.
`
`2; see claim 15 for motivation to combine).
`
`Regarding claim 17, the combination of Kiemele and Zhu renders obvious
`
`wherein the circuitry acquires the selection result on a basis of a detection result
`
`of a gesture of the user from a gesture detection device (“The use of free-hand
`
`gestures,
`
`local touch events, and controller-based input events can all be combined to
`
`create new forms of cross-platform gestures that incorporate both input platforms,” Zhu,
`
`pg. 5, sec. D5; see claim 15 for motivation to combine).
`
`Conclusion
`
`Applicant's amendment necessitated any new ground(s) of rejection presented in
`
`this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a).
`
`Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
`
`A shortened statutory period for reply to this final action is set to expire THREE
`
`
`
`MONTHS from the mailing date of this action. In the eventafirst reply is filed within
`
`TWO MONTHS of the mailing date of this final action and the advisory action is not
`
`mailed until after the end of the THREE-MONTHshortened statutory period, then the
`
`

`

`Application/Control Number: 17/922,919
`Art Unit: 2611
`
`Page 17
`
`shortened statutory period will expire on the date the advisory action is mailed, and any
`
`extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of
`
`the advisory action. In no event, however, will the statutory period for reply expire later
`
`than SIX MONTHS from the date of this final action.
`
`Anyinquiry concerning this communication or earlier communications from the
`
`examiner should be directed to Ryan McCulley whose telephone numberis (571)270-
`
`3754. The examiner can normally be reached Monday through Friday, 8:00am -
`
`4:30pm.
`
`Examinerinterviews are available via telephone,
`
`in-person, and video
`
`conferencing using a USPTO supplied web-based collaboration tool. To schedule an
`
`interview, applicant is encouraged to use the USPTO AutomatedInterview Request
`
`(AIR) at http:/Awww.uspto.gov/interviewpractice.
`
`If attempts to reach the examiner by telephone are unsuccessful, the examiner's
`
`supervisor, Kee Tung can be reached on (571) 272-7794. The fax phone numberfor the
`
`organization where this application or proceeding is assigned is 571-273-8300.
`
`Information regarding the status of published or unpublished applications may be
`
`obtained from Patent Center. Unpublished application information in Patent Centeris
`
`available to registered users. To file and manage patent submissions in Patent Center,
`
`visit: httos://patentcenter.uspto.gov. Visit https:/Avww.uspto.gov/patents/apply/patent-
`
`center for more information about Patent Center and
`
`https://www.uspto.gov/patents/docx for information aboutfiling in DOCX format. For
`
`additional questions, contact the Electronic Business Center (EBC) at 866-217-9197
`
`

`

`Application/Control Number: 17/922,919
`Art Unit: 2611
`
`Page 18
`
`(toll-free).
`
`If you would like assistance from a USPTO Customer Service
`
`Representative, call 800-786-9199 (IN USA OR CANADA)or 571-272-1000.
`
`/RYAN MCCULLEY/
`Primary Examiner, Art Unit 2611
`
`