www.uspto.gov
`
`UNITED STATES DEPARTMENT OF COMMERCE
`United States Patent and Trademark Office
`Address: COMMISSIONER FOR PATENTS
`P.O. Box 1450
`Alexandria, Virginia 22313-1450
`
`17/987,828
`
`11/15/2022
`
`Danillo Graziosi
`
`SONY-75200
`
`6471
`
`pues
`
`an
`HAV
`HAVERSTOCK & OWENS
`A LAW CORPORATION
`162 N. WOLFE ROAD
`SUNNYVALE,CA 94086
`
`CHEN, BIAO
`
`2611
`
`PAPER NUMBER
`
`NOTIFICATION DATE
`
`DELIVERY MODE
`
`01/13/2025
`
`ELECTRONIC
`
`Please find below and/or attached an Office communication concerning this application or proceeding.
`
`The time period for reply, if any, is set in the attached communication.
`
`Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
`following e-mail address(es):
`
`jowens @hollp.com
`us_docketing @hollp.com
`
`PTOL-90A (Rev. 04/07)
`
`

`

`Office Action Summary
`
`Application No.
`17/987 ,828
`Examiner
`BIAO CHEN
`
`Applicant(s)
`Graziosi etal.
`Art Unit
`2611
`
`AIA (FITF) Status
`Yes
`
`-- The MAILING DATEof this communication appears on the cover sheet with the correspondence address --
`Period for Reply
`
`A SHORTENED STATUTORYPERIOD FOR REPLYIS SET TO EXPIRE 3 MONTHS FROM THE MAILING
`DATE OF THIS COMMUNICATION.
`Extensionsof time may be available underthe provisions of 37 CFR 1.136(a). In no event, however, may a reply betimely filed after SIX (6) MONTHSfrom the mailing
`date of this communication.
`If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHSfrom the mailing date of this communication.
`-
`- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133).
`Any reply received by the Office later than three months after the mailing date of this communication, evenif timely filed, may reduce any earned patent term
`adjustment. See 37 CFR 1.704(b).
`
`Status
`
`
`
`1) Responsive to communication(s) filed on 11/15/2022.
`C} A declaration(s)/affidavit(s) under 37 CFR 1.130(b) was/werefiled on
`
`2a)() This action is FINAL. 2b)¥)This action is non-final.
`3) An election was madeby the applicant in responseto a restriction requirement set forth during the interview
`on
`; the restriction requirement and election have been incorporated into this action.
`4)(2) Since this application is in condition for allowance except for formal matters, prosecution as to the merits is
`closed in accordance with the practice under Exparte Quayle, 1935 C.D. 11, 453 O.G. 213.
`
`Disposition of Claims*
`1-21 is/are pending in the application.
`)
`Claim(s)
`5a) Of the above claim(s) _ is/are withdrawn from consideration.
`CL] Claim(s)__is/are allowed.
`Claim(s) 1-2,5-9,12-16 and 19-21 is/are rejected.
`Claim(s) 3-4,10-11 and 17-18 is/are objected to.
`C) Claim(s
`are subject to restriction and/or election requirement
`)
`* If any claims have been determined allowable, you maybeeligible to benefit from the Patent Prosecution Highway program at a
`participating intellectual property office for the corresponding application. For more information, please see
`http:/Awww.uspto.gov/patents/init_events/pph/index.jsp or send an inquiry to PPHfeedback@uspto.gov.
`
`) ) ) )
`
`Application Papers
`10)( The specification is objected to by the Examiner.
`11)M The drawing(s) filed on 11/15/2022 is/are: a)¥) accepted or b)() objected to by the Examiner.
`Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a).
`Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d).
`
`Priority under 35 U.S.C. § 119
`12)2) Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d)or (f).
`Certified copies:
`c)() None ofthe:
`b)( Some**
`a) All
`1.1.) Certified copies of the priority documents have been received.
`2.1.) Certified copies of the priority documents have been received in Application No.
`3.1.) Copies of the certified copies of the priority documents have been receivedin this National Stage
`application from the International Bureau (PCT Rule 17.2(a)).
`*“ See the attached detailed Office action for a list of the certified copies not received.
`
`Attachment(s)
`
`1)
`
`Notice of References Cited (PTO-892)
`
`Information Disclosure Statement(s) (PTO/SB/08a and/or PTO/SB/08b)
`2)
`Paper No(s)/Mail Date
`U.S. Patent and Trademark Office
`
`3)
`
`4)
`
`(LJ Interview Summary (PTO-413)
`Paper No(s)/Mail Date
`(Qj Other:
`
`PTOL-326 (Rev. 11-13)
`
`Office Action Summary
`
`Part of Paper No./Mail Date 20241226
`
`

`

`Application/Control Number: 17/987,828
`Art Unit: 2611
`
`Page 2
`
`DETAILED ACTION
`
`Notice of Pre-AlA or AIA Status
`
`1.
`
`The present application, filed on or after March 16, 2013,
`
`is being examined
`
`underthefirst inventor to file provisions of the AIA.
`
`Specification
`
`2.
`
`The disclosure is objected to becauseof the following informalities:
`
`e
`
`In page 3, lines 4-5, “from the pixels’ surfaces” should read “from the
`
`surface’s pixels”.
`
`e
`
`In page 5, line 13, “from the pixels' surfaces” should read “from the surface’s
`
`pixels”.
`
`Appropriate correction is required.
`
`Claim Rejections - 35 USC § 103
`
`3.
`
`In the event the determination of the status of the application as subject to AIA 35
`
`U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any
`
`correction of the statutory basis (i.e., changing from AIA to pre-AlA) for the rejection will
`
`not be considered a new groundofrejection if the prior art relied upon, and the rationale
`
`supporting the rejection, would be the same undereither status.
`
`4.
`
`The following is a quotation of 35 U.S.C. 103 which forms the basis forall
`
`obviousness rejections set forth in this Office action:
`
`

`

`Application/Control Number: 17/987,828
`Art Unit: 2611
`
`Page 3
`
`A patent for a claimed invention may not be obtained, notwithstanding that the claimed
`invention is not identically disclosed as set forth in section 102, if the differences between
`the claimed invention and the prior art are such that the claimed invention as a whole
`would have been obviousbeforethe effectivefiling date of the claimed invention to a
`person having ordinary skillin the art to which the claimed invention pertains.
`Patentability shall not be negated by the mannerin which the invention was mace.
`
`5.
`
`Claims 1-2, 4-7, 8-9, and 12-14 are rejected under 35 U.S.C. 103 as being
`
`unpatentable over Graziosi et al. ([V-PCC] [specification] Depth range adjustment,
`
`ISO/IEC JTC 1/SC 29//WG 11 m51170, Geneva, CH, October 2019, hereinafter
`
`“Graziosi_VPCC”), and in view of:
`
`e Schwalb et al. (Fast Motion Estimation on Graphics Hardware for H.264
`
`Video Encoding IEEE Transactions on Multimedia (Volume: 11, Issue: 1,
`
`2009, Page(s): 1-10), thereinafter “Schwalb”)
`
`e Weiet al. (Bi-Normal Filtering for Mesh Denoising, IEEE Transactions on
`
`Visualization and Computer, Vol. 21, No. 1, January 2015, hereinafter “Wei’).
`
`e Nakagawaet al. (Estimating Surface Normals with Depth Image Gradients for
`
`Fast and Accurate Registration, 2015 International Conference on 3D Vision
`
`(2015, Page(s): 640-647), hereinafter “Nakagawa’)
`
`Regarding claim 1, Graziosi_VPCC teaches A method of mesh geometry coding
`
`comprising: (section 1, “the luminance values carry the depth value”)
`
`mapping depth information to aluma channel using fewerthanall available
`
`bits; (section 1, “the luminance values carry the depth value. The decodedbit depth is
`
`adjusted according to the 2D nominal bit depth value (indicated by the syntax element
`
`gi~geometry_nominaf_2d_bitdepth_minusl). The adjustmentwill either only clip the
`
`values ... or scale them by a powerof two’). Note that the luma channel with the
`
`

`

`Application/Control Number: 17/987,828
`Art Unit: 2611
`
`Page 4
`
`luminance values contains depth information, and the “clipping” or “scaling” indicates
`
`that the least significant bits or most significant bits are ignored/unused.
`
`generating a depth scaling factor which is ineerperatednte-abilinear
`
`
`
`interpolation—algorithniused_duringrasterization—and
`
`(section 1, “The decoded bit
`
`depth is adjusted according to the 2D nominal bit depth value ... scale them by a power
`
`of two (in case the MSBsareto bealigned)”).
`
`However, Graziosi_VPCC does not disclose incorporated into a bilinear
`
`interpolation algorithm used during rasterization.
`
`In the same art of video coding, Schwalb discloses incorporated into a bilinear
`
`interpolation algorithm used during rasterization (Schwalb, page 3, section 3, “to
`
`generate so called “fragments” for all output pixels that the currently rendered triangle
`
`covers. A fragmentis a data structure analog to the structure of a vertex and is
`
`calculated by the Rasterizer by interpolating position coordinates’; page 5, section 4,
`
`“Supersampling to a factor of 4 with bilinear interpolation”). Note that during
`
`rasterization the bilinear interpolation gets the scaling factor involved for calculations to
`
`have the depth values scaled to the bit-depth.
`
`Graziosi_VPCC and Schwalb are in the same field of endeavor, namely
`
`computer graphics. Before the effective filing date of the claimed invention,
`
`it would
`
`have been obvious to apply the bilinear interpolation during rasterization, as taught by
`
`Schwalb into Graziosi_VPCC. The motivation would have been “calculated by the
`
`Rasterizer by interpolating position coordinates” (Schwalb, page 3, section 3). The
`
`suggestion for doing so would allow to perform rasterization with a bilinear interpolation
`
`

`

`Application/Control Number: 17/987,828
`Art Unit: 2611
`
`Page5S
`
`and the scaling factor. Therefore, it would have been obvious to combine
`
`Graziosi_VPCC and Schwalb.
`
`However, the combination of Graziosi_VPCC in view of Schwalb does not
`
`disclosing
`
`performing normalfiltering including adjusting positions of vertices
`
`according to an estimated normal from surface pixels.
`
`In the same art of computer graphics, Wei discloses performing normal
`
`filtering including adjusting positions of vertices according to an estimated
`
`normal-frem-surface-pixels-(VWei, section 1, “we take advantage of the piecewise
`
`consistent property between the two normal fields on mesh surfaces and design a novel
`
`bi-normal filtering strategy to address the inherent difficulties in estimating the two
`
`normal fields of anoisy mesh at feature regions; we formulate the vertex position
`
`update as a quadratic optimization problem”). Note the normal fields are filtered and the
`
`positions of vertices are adjusted accordingly.
`
`Graziosi_VPCC in view Schwalb, and Wei, are in the same field of endeavor,
`
`namely computer graphics. Before the effectivefiling date of the claimed invention, it
`
`would have been obvious to perform the normalfiltering and adjustment of the positions
`
`of vertices, as taught by Weiinto Graziosi_VPCC in view of Schwalb. The motivation
`
`would have been “preserve both sharp and shallow features and producevisually and
`
`quantitatively better results.” (Wei, page 52, section 8). The suggestion for doing so
`
`would allow to improve the presentation quality of 3D objects in terms of various
`
`features. Therefore, it would have been obvious to combine Graziosi_VPCC, Schwalb,
`
`and Wei.
`
`

`

`Application/Control Number: 17/987,828
`Art Unit: 2611
`
`Page 6
`
`However, the combination of Graziosi_VPCC, Schwalb, and Wei, does not
`
`disclose ... from surface pixels.
`
`In the same art of computer graphics, Nakagawa discloses ... from surface
`
`pixels. (Nakagawa, Abstract, “We present a fast registration framework with estimating
`
`surface normals from depth images. The key componentin the frameworkis to utilize
`
`adjacent pixels and compute the normal at each pixel on a depth imagebyfollowing
`
`three steps. First, image gradients on a depth image are computed with a 2D differential
`
`filtering. Next, two 3D gradient vectors are computed from horizontal and vertical depth
`
`image gradients. Finally, the normal vector is obtained from the cross product of the 3D
`
`gradient vectors”). Note that the surface normal is estimated using adjacent pixels.
`
`Graziosi_VPCC in view of Schwalb and Wei, and Nakagawa,are in the same
`
`field of endeavor, namely computer graphics. Before the effectivefiling date of the
`
`claimed invention,
`
`it would have been obvious to obtain an estimated normal from
`
`surface pixels, as taught by Nakagawainto Graziosi_VPCC in view of Schwalb and
`
`Wei. The motivation would have been “normal estimation is extremely faster becauseit
`
`needs only a few mathematical operations.” (Nakagawa, Abstract). The suggestion for
`
`doing so would allow to accelerate the norm estimation. Therefore, it would have been
`
`obvious to combine Graziosi_VPCC, Schwalb, Wei, and Nakagawa.
`
`Regarding claim 2, the combination of Graziosi_VPCC, Schwalb, Wei, and
`
`Nakagawa discloses The method of claim 1 wherein mapping the depth
`
`information to the luma channel includes using M bits of N available bits, where M
`
`is less than N. (Graziosi_VPCC, section 1, “The adjustment will either only clip the
`
`

`

`Application/Control Number: 17/987,828
`Art Unit: 2611
`
`Page 7
`
`values ... or scale them by a powerof two”). Note that the number of available bits and
`
`the numberof fewer bits can be indicated by N and M, respectively, where M is less
`
`than N, and the “clipping” or “scaling” indicates that the least significant bits or most
`
`significant bits are ignored/unused.
`
`The motivation to combine Graziosi_VPCC, Schwalb, Wei, and Nakagawa given
`
`in claim 1
`
`is incorporated here.
`
`Regarding claim_5, the combination of Graziosi_VPCC, Schwalb, Wei, and
`
`Nakagawa discloses The method of claim 1 further comprising applying the depth
`
`scaling factor to data (Graziosi_VPCC, section 1, “scale them by a powerof two”),
`
`wherein the data comprises floating point values. (Schwalb, page 3, section Ill, “the
`
`arithmetic operations are SIMD (single-instruction-multiple-data)
`
`instructions which can
`
`operate in parallel on all four floats that normally represent an XYZW-position or an
`
`RGBA-pixel’). Note that the depth scaling factor is used in the arithmetic operations,
`
`such asbilinear interpolation, and the operated data typeis offloat.
`
`The motivation to combine Graziosi_VPCC, Schwalb, Wei, and Nakagawa given
`
`in claim 1
`
`is incorporated here.
`
`Regarding claim _6, the combination of Graziosi_VPCC, Schwalb, Wei, and
`
`Nakagawa discloses The method of claim 1 wherein normalfiltering includes
`
`performing plane fitting using points (Nakagawa, Abstract, “Compared with existing
`
`normal estimation based onfitting a plane to a point cloud, our ...”; section 1, “Normal
`
`estimation is basically considered equivalentto fitting a plane to a local point cloud in
`
`

`

`Application/Control Number: 17/987,828
`Art Unit: 2611
`
`Page 8
`
`the 3D space”). Note that performing planefitting using points with a triangle or a point
`
`cloud is an existing normal estimation approach according to Nakagawa.
`
`The motivation to combine Graziosi_VPCC, Schwalb, Wei, and Nakagawa given
`
`in claim 1
`
`is incorporated here.
`
`Further, the combination of Graziosi_VPCC, Schwalb, Wei, and Nakagawa
`
`discloses ... within a triangle. (Schwalb, page 3, section Ill, “Each triangle is defined
`
`by its three “vertices’, which are data structures that hold 3-D world position
`
`coordinates’).
`
`The motivation to combine Graziosi_VPCC, Schwalb, Wei, and Nakagawa given
`
`in claim 1
`
`is incorporated here.
`
`Regarding claim 7, the combination of Graziosi_VPCC, Schwalb, Wei, and
`
`Nakagawa discloses The method of claim 1 wherein normalfiltering includes using
`
`one-ring, two-ring or three-ring face neighborhoods for minimization of total
`
`normal angles. (Wei, page 47, section 5.3, “The estimation is performed with a
`
`weighted average of all normals from its one-ring neighboring facets, where the
`
`weighting factor is selected to be the area of each facet”).
`
`The motivation to combine Graziosi_VPCC, Schwalb, Wei, and Nakagawa given
`
`in claim 1
`
`is incorporated here.
`
`Claim 8 reciting “An apparatus comprising: a non-transitory memory for storing
`
`an application, the application for:’, is corresponding to claim 1. Therefore, claim 8 is
`
`rejected for the same rationale for claim 1.
`
`

`

`Application/Control Number: 17/987,828
`Art Unit: 2611
`
`Page 9
`
`In addition, the combination of Graziosi_VPCC, Schwalb, Wei, and Nakagawa
`
`discloses An apparatus comprising: a non-transitory memory for storing an
`
`application, the application for: (Schwalb, “This implementation was evaluated on a
`
`standard Windows XP PCconsisting of the following key hardware components: AMD
`
`Athlon XP 1800+ CPU, 1 GB RAM and a GeForce 6600 GT AGP graphics card
`
`equipped with 128 MB RAM whichis connected to the GPU by a 128 bit wide bus’).
`
`Note that the PC is an apparatus equipped with non-transitory memory (PC RAM +
`
`GPU RAM) storing an application including run-time software / various data.
`
`The motivation to combine Graziosi_VPCC, Schwalb, Wei, and Nakagawa given
`
`in claim 1
`
`is incorporated here.
`
`Claims 9 and 12-14 are corresponding to claims 2 and 5-7. Therefore, claims 9
`
`and 12-14 are rejected for the same rationale for claims 2 and 5-7.
`
`6.
`
`Claims 15-16 and 19-21 are rejected under 35 U.S.C. 103 as being unpatentable
`
`over Graziosi_VPCC in view of Schwalb, Wei, and Nakagawa,and further in view of
`
`Graziosi et al. (US 2020/0204808 A1, hereinafter “Graziosi_PCCS’).
`
`Claim 15 reciting “A system comprising: one or more camerasfor acquiring three
`
`dimensional content; an encoder for encoding the three dimensional content:”,
`
`is
`
`corresponding to claim 1. Therefore, claim 15 is rejected for the same rationale for claim
`
`1.
`
`

`

`Application/Control Number: 17/987,828
`Art Unit: 2611
`
`Page 10
`
`However, the combination of Graziosi_VPCC, Schwalb, Wei, and Nakagawa
`
`does not disclose A system comprising: one or more cameras for acquiring three
`
`dimensional content; an encoder for encoding the three dimensional content:
`
`In the same art of computer graphics, Graziosi_PCCS discloses A system
`
`comprising: one or more cameras for acquiring three dimensional content; an
`
`encoder for encoding the three dimensional content: (Graziosi_PCCS, para. [0003],
`
`“Point clouds have been considered as a candidate format for transmission of 3D data,
`
`either captured by 3D scanners, LIDAR sensors”; para. [0085], “the point cloud coding
`
`structure hardware 1020 includes camera components’; para. [0118], claim 15: “A
`
`system comprising: a point cloud coding structure; and an encoderfor encoding point
`
`cloud metadata using the point cloud coding structure”). Note that it is well know that a
`
`LIDAR sensor is a kind of camera.
`
`Graziosi_VPCC in view of Schwalb, Wei, and Nakagawa, and Graziosi_PCCS,
`
`are in the same field of endeavor, namely computer graphics. Before the effectivefiling
`
`date of the claimed invention,
`
`it would have been obvious to apply camerasfor
`
`acquiring 3D content and encoder for encoding 3D content, as taught by
`
`Graziosi_VPCC in view of Schwalb, Wei, and Nakagawa. The motivation would have
`
`been “Point clouds have been considered as a candidate format for transmission of 3D
`
`data, either captured by 3D scanners, LIDAR sensors, or used in popular applications
`
`such as Virtual Reality/Augmented Reality (VR/AR)” (Graziosi_PCCS, para. [0003]).
`
`The suggestion for doing so would allow to capture and encode 3D contentthat is
`
`employed in VR/AR. Therefore, it would have been obvious to combine Graziosi_VPCC,
`
`Schwalb, Wei, and Graziosi_PCCS.
`
`

`

`Application/Control Number: 17/987,828
`Art Unit: 2611
`
`Page 11
`
`Claims 16 and 19-21 are corresponding to claims 2 and 5-7. Therefore, claim 16
`
`and 19-21 are rejected for the same rationale for claim 15 and for claims 2 and 5-7,
`
`respectively.
`
`Allowable Subject Matter
`
`7.
`
`Claim 3-4, 10-11, and 17-18 are objected to as being dependent upon a rejected
`
`base claim, but would be allowable if rewritten in independent form including all of the
`
`limitations of the base claim and anyintervening claims.
`
`Regarding dependent claims 3, 10, and 17, Graziosi_VPCC discloses unused
`
`bits due to the clipping or scaling, but does notutilize those bits for the depth scaling
`
`factor. In the context of claim as a whole, the prior art either alone or in combination
`
`does not teach or suggest the additional elements of: “generating the depth scaling
`
`factor utilizes 2 remaining bits of the N available bits’, respectively.
`
`
`Claims 4,
`11, and 18 depend from claims 3, 10, and 17, respectively.
`
`Conclusion
`
`8.
`
`Anyinquiry concerning this communication or earlier communications from the
`
`examiner should be directed to BAAO CHEN whosetelephone numberis (703)756-
`
`1199. The examiner can normally be reached M-F 8am-5pm ET.
`
`

`

`Application/Control Number: 17/987,828
`Art Unit: 2611
`
`Page 12
`
`Examinerinterviews are available via telephone,
`
`in-person, and video
`
`conferencing using a USPTO supplied web-based collaboration tool. To schedule an
`
`interview, applicant is encouraged to use the USPTO AutomatedInterview Request
`
`(AIR) at http:/Awww.uspto.gov/interviewpractice.
`
`If attempts to reach the examiner by telephone are unsuccessful, the examiner's
`
`supervisor, Kee M Tung can be reached on (571)272-7794. The fax phone numberfor
`
`the organization where this application or proceeding is assigned is 571-273-8300.
`
`Information regarding the status of published or unpublished applications may be
`
`obtained from Patent Center. Unpublished application information in Patent Centeris
`
`available to registered users. To file and manage patent submissions in Patent Center,
`
`visit: httos://patentcenter.uspto.gov. Visit https:/Avww.uspto.gov/patents/apply/patent-
`
`center for more information about Patent Center and
`
`https://www.uspto.gov/patents/docx for information about filing in DOCX format. For
`
`additional questions, contact the Electronic Business Center (EBC) at 866-217-9197
`
`(toll-free).
`
`If you would like assistance from a USPTO Customer Service
`
`Representative, call 800-786-9199 (IN USA OR CANADA)or 571-272-1000.
`
`/Biao Chen/
`
`Patent Examiner, Art Unit 2611
`
`/RYAN MCCULLEY/
`Primary Examiner, Art Unit 2611
`
`