www.uspto.gov
`
`UNITED STATES DEPARTMENT OF COMMERCE
`United States Patent and Trademark Office
`Address: COMMISSIONER FOR PATENTS
`P.O. Box 1450
`Alexandria, Virginia 22313-1450
`
`APPLICATION NO.
`
`FILING DATE
`
`FIRST NAMED INVENTOR
`
`ATTORNEY DOCKETNO.
`
`CONFIRMATIONNO.
`
`17/036,913
`
`09/29/2020
`
`Varun Kompella
`
`SONI-PAUI1
`
`1036
`
`01/15/2025
`7590
`189920
`Innovation Capital Law Group, LLP
`Sony Corporation of America
`19900 MacArthur Blvd., Suite 610
`Irvine, CA 92612
`
`EXAMINER
`
`MRABI, HASSAN
`
`ART UNIT
`2147
`
`PAPER NUMBER
`
`NOTIFICATION DATE
`
`DELIVERY MODE
`
`01/15/2025
`
`ELECTRONIC
`
`Please find below and/or attached an Office communication concerning this application or proceeding.
`
`The time period for reply, if any, is set in the attached communication.
`
`Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
`following e-mail address(es):
`
`eofficeaction @appcoll.com
`processing @icaplaw.com
`vlin@icaplaw.com
`
`PTOL-90A (Rev. 04/07)
`
`

`

`M Supplemental li
`otice of
`Allowability
`
`17/036,913
`Application No.
`Examiner
`HASSAN MRABI
`
`Applicant(s)
`
`Kompella et al.
`AIA (FITF) Status
`Yes
`
`2147
`
`4.1) Acknowledgmentis made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f).
`Certified copies:
`c) () Noneofthe:
`a) DAI
`b) (] Some*
`1. (1 Certified copies of the priority documents have been received.
`2. (J Certified copies of the priority documents have been received in Application No.
`3. (J Copiesof the certified copies of the priority documents have been receivedin this national stage application from the
`International Bureau (PCT Rule 17.2(a)).
`
`Examiner, Art Unit 2147
`
`-- The MAILING DATEof this communication appears on the cover sheet with the correspondence address--
`All claims being allowable, PROSECUTION ON THE MERITS IS (OR REMAINS) CLOSED in this application. If not included
`herewith (or previously mailed), a Notice of Allowance (PTOL-85) or other appropriate communication will be mailed in due course. THIS
`NOTICE OF ALLOWABILITY IS NOT A GRANTOF PATENTRIGHTS.This application is subject to withdrawal from issue at the initiative
`of the Office or upon petition by the applicant. See 37 CFR 1.313 and MPEP 1308.
`1{¥] This communication is responsive to 01/07/2025.
`(J A declaration(s)/affidavit(s) under 37 CFR 1.130(b) was/werefiled on
`
`.
`
`2._) Anelection was madeby the applicant in responseto a restriction requirement set forth during the interview on
`restriction requirement and election have been incorporatedinto this action.
`
`; the
`
`3.{¥) The allowed claim(s) is/are 1-7,9-15 and 17-20 . As a result of the allowed claim(s), you may beeligible to benefit from the Patent
`Prosecution Highway program at a participating intellectual property office for the corresponding application. For more information
`, please see hitp:/Awww.uspto.gov/patents/init_events/pph/index.jsp or send an inquiry to PPHfeedback@uspto.gov.
`
`* Certified copies not received:
`
`Applicant has THREE MONTHS FROM THE "MAILING DATE"of this communication to file a reply complying with the requirements
`noted below.Failure to timely comply will result in ABANDONMENT ofthis application.
`THIS THREE-MONTH PERIOD IS NOT EXTENDABLE.
`
`5.) CORRECTED DRAWINGS (as "replacement sheets") must be submitted.
`(} including changes required by the attached Examiner's Amendment / Commentor in the Office action of
`Paper No./Mail Date
`.
`Identifying indicia such as the application number (see 37 CFR 1.84(c)) should be written on the drawingsin the front (not the back) of each
`sheet. Replacement sheet(s) should be labeled as such in the header according to 37 CFR 1.121(d).
`
`6.LJ DEPOSIT OFand/or INFORMATION aboutthe deposit of BIOLOGICAL MATERIAL must be submitted. Note the
`attached Examiner's comment regarding REQUIREMENT FOR THE DEPOSIT OF BIOLOGICAL MATERIAL.
`
`Attachment(s)
`1.1 Notice of References Cited (PTO-892)
`2.C Information Disclosure Statements (PTO/SB/08),
`Paper No./Mail Date
`.
`3.) Examiner's Comment Regarding Requirementfor Deposit
`of Biological Material
`4.{¥} Interview Summary (PTO-413),
`Paper No./Mail Date. 01/08/2025.
`/HASSAN MRABI/
`
`5. (2 Examiner's Amendment/Comment
`6.
`Examiner's Statement of Reasons for Allowance
`
`7. CZ Other
`
`.
`
`U.S. Patent and Trademark Office
`PTOL-37 (Rev. 08-13)
`
`Notice of Allowability
`
`.
`Part of Paper No./Mail Date 20250108
`
`

`

`Application/ Control Number: 17/036,913
`Art Unit: 2147
`
`Page 2
`
`Nottce ofPre-AIA or AIA Status
`
`1.
`
`The present application,filed on or after March 16, 2013, is being examined underthe first
`
`inventor to file provisions of the AIA.
`
`Remark
`
`2.
`
`This Office action has been issued tn response to amendmentfiled on 10/22/2024.
`
`EXAMINER’S AMENDMENT
`
`3.
`
`An examiner’s amendmentto the record appears below. Should the changes and/or
`
`additions be unacceptable to applicant, an amendment maybe filed as provided by 37 CFR 1.312.
`
`To ensure consideration of such an amendment, it MUST be submitted no later than the payment of
`
`the issue fee.
`
`4.
`
`Authorization for this examiner’s amendment was given subsequentto a telephone interview
`
`with Lyman Smith on 01/08/2025.
`
`AMENDMENTSTO THE CLAIMS
`
`5.
`
`The following is the list of clatms that were amended:
`
`

`

`Application/ Control Number: 17/036,913
`Art Unit: 2147
`
`Page 3
`
`Claim 9.
`
`(Original) The method of claim 1, wherein transitions that belong to Jare
`
`giving a priority greater than transitions that are notin J.
`
`Allowance
`
`6.
`
`Claims (1-7, 9-13), (15, 17-18) and (19-20) are allowable.
`
`Reason for Allowance
`
`7.
`
`Thecited arts of record, by over Tukiainen et al. US Patent Applicaton Publication US
`
`20200302322 Al (hereinafter Tuktainen) in view of Lillicrap. Foreign Application Publication CA
`
`2993551 C (hereinafter Lillicrap) teaches training an agent using a task prioritized experience replay
`
`algorithm.
`
`8.
`
`Claims (1-7, 9-13), (15, 17-18) and (19-20) are allowable. Independent claims 1, 15 and 19 are
`
`allowable because the prior arts of record do not teach determining a probability, of sampling the
`
`transition tuple with the index from the main buff er; updating transiting priorities for each
`
`transition tuple stored in the main buffer; sampling a mini batch of transition tuples to update the
`
`task networks based on the stored priority value.
`
`9.
`
`Thecited arts of record, over Tukiainen et al. US Patent Applicaton Publicaton US
`
`20200302322 Al (hereinafter Tukiainen) in view of Lillicrap. Foreign Application Publication CA
`
`2993551 C (hereinafter Lillicrap) do notexplicitly disclose, teach, or suggest the claimed limitations
`
`of:
`
`Claims 1 and 19.
`
`

`

`Application/ Control Number: 17/036,913
`Art Unit: 2147
`
`Page 4
`
`storing a transition tuple ina main buffer of the agent, the transition tuple including {(St,
`
`atl, St +1)}, where ris a reward vector for each task of the plurality of tasks for the agentin an
`
`environmentand S; +1is a next environmentstate after action (at);
`
`storing a priority value p(i), of the transition tuple with index 1 1n the main buffer;
`
`determining a probability, P@) of sampling the transiton tuple with the index 1 from the
`
`main buff er;
`
`updating transiting priorities for each transition tuple stored in the main buffer;
`
`sampling a mint batch of transition tuples to update the task networks based _on the stored
`
`riority
`
`value
`
`p(t)
`
`thereof;
`
`determining an action probability distribution parameter, 2(S,),of updating task policies for
`
`the observation Sy and
`
`optimizing the task policies from the updated task networks with an off-policy algorithm,
`
`wherein:
`
`data that 1s prioritized for one task is shared with one or more othertasks to transfer
`
`learning between multiple tasks.
`
`Claim 15.
`
`determining an action probability distribution parameter, ni(st), of updating task policies for
`
`the observation Sy
`
`updating transiting priorities for each transition tuple stored in the main buffer;
`
`sampling a mint batch of transition tuples to update the task networks based _on the stored
`
`priority value p() thereof; and
`
`

`

`Application/ Control Number: 17/036,913
`Art Unit: 2147
`
`Page 5
`
`optimizing task policies from the updated task networks with an off-policy algorithm,
`
`wherein
`
`transitions that belong to a set of transition indices that result in achievement of task-]
`
`during an 1 episode are given a priority greater than transitions that do not result in achievement of
`
`task-j during the 1th episode; and
`
`data that 1s prioritized for one task 1s shared with one or more othertasks to transfer
`
`learning between multiple tasks.
`
`(In combination with all other features in the claim).
`
`Conclusion
`
`Any inquiry concerning this communication or earlier communications from the examiner
`
`should be directed to HASSAN MRABI whose telephone numberis (571)272-8875. The examiner
`
`can normally be reached on M-F 7:30 am - 4:00 pm.
`
`If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor,
`
`Scott Baderman can be reached on (571)272-3644. The fax phone numberfor the organization
`
`where this application or proceedingts assigned 1s 571-273-8300.
`
`Information regarding the status of an application may be obtained from the Patent
`
`Application Information Retrieval (PAIR) system. Status information for published applications
`
`may be obtained from either Private PAIR or Public PAIR. Status information for unpublished
`
`applications is available through Private PAIR only. For more information about the PAIR system,
`
`see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system,
`
`contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like
`
`

`

`Application/ Control Number: 17/036,913
`Art Unit: 2147
`
`Page 6
`
`assistance from a USPTO CustomerService Representative or access to the automated information
`
`system,call 800-786-9199 (IN USA ORCANADA)or 571-272-1000.
`
`/HASSAN MRABI/
`
`Examiner, Art Unit 2144
`
`