U.S. PTO
`61/403952
`PTO/SB/16 (12-08)
`09/24/2010
`Approved for use through 09/30/2010. OMB 0651-0032
`3
`U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE
`&
`No Under the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unlessit displays a valid OMB control number.
`PROVISIONAL APPLICATION FOR PATENT COVER SHEET- Page1 of 2
`Cc
`This is a request forfiling a PROVISIONAL APPLICATION FOR PATENTunder 37 CFR 1.53(c).
`Egpress Mail Label No.
`
`Ghvén Name(first and middie [if any])
`
`INVENTOR(S
`Family Name or Surname
`
`Residence
`(City and either State or Foreign Country)
`
`Pearl River, NYi
`
`
`
`
`
`
`
`
`Microphone Array Design and Implementation for Telecommunications and Handheld Devices
`
` Direct all correspondenceto:
`CORRESPONDENCE ADDRESS
`
`
`[] TheaddresscorrespondingtoCustomerNumber: PF
`
`OR
`
`
`Individual Name i (Peter)Li
`
`
`
`*earess Li Creative Technologies, Inc., 25B Hanover Road, Suite 140
`
`
`
`
`city Florham Park
`State NJ
`Taso875-822-0048
`
`
`[_] co(s), Numberof cDs
`[_] Application Data Sheet. See 37 CFR 1.76
`
`[] Drawing(s) Numberof Sheets Other(specify) descrip w/ drawings 23 pp
`
`
`
` C] Specification (e.g. description of the invention) Number of Pages
`
`
`
`
`
`
`Additional inventors are beingnamedonthe=CC“‘(‘ONUOOCUNC#S@parately numbered sheets attachedhereto.
`TITLE OF THE INVENTION (500 characters max):
`
`
`
`Zip 07932
`[EAH W@teeaivtech.con
`
`
`
`If the specification and drawings exceed 100 sheets of paper, an application size fee is
`Fees Due: Filing Fee of $220 ($110 for small entity).
`also due, which is $270 ($135 for small entity) for each additional 50 sheetsor fraction thereof. See 35 U.S.C. 41(a)(1)(G) and 37 CFR 1.16(s).
`METHOD OF PAYMENTOFTHE FILING FEE AND APPLICATION SIZE FEE FOR THIS PROVISIONAL APPLICATION FOR PATENT
`
`Applicant claims small entity status. See 37 CFR 1.27.
`LJ is
`enclosed t
`the
`fili
`icati
`i
`i
`ficable).
`A check or money order made payable to the Director of the United States Patent and Trademark Office
`is enclosed
`e filing fee and application size fee (if applicable)
`to cover
`:
`Paymentby credit card. Form PTO-2038is attached.
`[J The Director is hereby authorized to chargethe filing fee and application size fee (if applicable) or credit any overpaymentto Deposit
`Account Number:
`
`TOTAL FEE AMOUNT($)
`
`USE ONLY FORFILING A PROVISIONAL APPLICATION FOR PATENT
`This collection of information is required by 37 CFR 1.51. The information is required to obtain or retain a benefit by the public which is to file (and by the USPTO to
`process) an application. Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1.11 and 1.14. This collection is estimated to take 10 hours to complete,
`including
`gathering, preparing, and submitting the completed application form to the USPTO. Time will vary depending upon the individual case. Any comments on the
`amountof time you require to complete this form and/or suggestions for reducing this burden, should be sent to the Chief Information Officer, U.S. Patent and
`Trademark Office, U.S. Department of Commerce, P.O. Box 1450, Alexandria, VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMSTO THIS
`ADDRESS. SEND TO: Commissionerfor Patents, P.O. Box 1450, Alexandria, VA 22313-1450.
`if you need assistance in completing the form, call 1-800-PTO-9199 and select option 2.
`SONOS EXHIBIT 1019
`
`Page1 of 28
`
`Page 1 of 28
`
`SONOS EXHIBIT 1019
`
`

`

`PROVISIONAL APPLICATION COVER SHEET
`Page 2 of 2
`
`PTO/SB/16 (12-08)
`Approved for use through 09/30/2010. OMB 0651-0032
`U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE
`Under the Paperwork Reduction Act of 1995, no persons are required to respondto a collection of information unlessit displays a valid OMB control number.
`
`The invention was made by an agencyof the United States Government or undera contract with an agencyof the United States Government.
`No.
`
`[] Yes, the nameof the U.S. Government agency and the Governmentcontract numberare:
`
`WARNING:
`Petitioner/applicant is cautioned to avoid submitting personal information in documents filed in a patent application that may
`contribute to identity theft. Persona! information such as social security numbers, bank account numbers, or credit card
`numbers(other than a check or credit card authorization form PTO-2038 submitted for payment purposes) is never required by
`the USPTO to support a petition or an application.
`If this type of personal information is included in documents submitted to the
`USPTO,petitioners/applicants should consider redacting such personalinformation from the documents before submitting them
`to the USPTO. Petitioner/applicantis advised that the record of a patent application is available to the public after publication of
`the application (unless a non-publication request in compliance with 37 CFR 1.213(a) is made in the application) or issuance of
`a patent. Furthermore, the record from an abandoned application may also be available to the public if the application is
`referenced in a published application or an issued patent (see 37 CFR 1.14). Checks and credit card authorization forms
`PTO-2038 submitted for payment purposes
`gre not retained in the application file and therefore are not publicly available.
`'
`A
`SIGNATURE—“‘“‘“‘“‘“‘“RCOCCCC#pate. 09-22-2010
`
`TYPED or PRINTED NAMEQi(Peter)Li_saREGISTRATION NO.
`(if appropriate)
`TevepHone 973-822-0048 Docket Number:
`
`Page 2 of 28
`
`SONOS EXHIBIT 1019
`
`Page 2 of 28
`
`SONOS EXHIBIT 1019
`
`

`

`PTO/SB/17 (10-08)
`Approved for use through 09/30/2010. OMB 0651-0032
`U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE
`Under the Paperwork Reduction Act of 1995 no personsare re
`quired to respondto a collection of information unless it displays a valid OMB control number
`
`
`Effective on 12/08/2004.
`
`
`Fees pursuant to the Consolidated Appropriations Act, 2005 (H.R. 4818).
`appicaionNomber
`
`FEE TRANSMITTAL[Frinnsose—Ioo202000
`
`
`For FY 2009
`
`Penaminername|CS
`
`YY] Applicant claims small entity status. See 37 CFR 1.27 fautOSC~SOSC*dS
`
`
`
`
`TOTAL AMOUNT OF PAYMENT|($) 110 [AttomeyDocketNo.JS
`
`
`
`METHOD OF PAYMENT(checkall that apply)
`[] Check
`Credit Card L] MoneyOrder [| None L_Jother (please identify):
`
`
`[] Deposit Account Deposit Account Number_______ «Deposit Account Name:
`
`
`For the above-identified deposit account, the Director is hereby authorized to: (checkall that apply)
`[] Charge fee(s) indicated below[] Charge fee(s) indicated below, except for thefiling fee
`Charge any additional fee(s) or underpayments of fee(s) [] Crediedit any overpayments
`
`
`under 37 CFR 1.16 and 1.17
`y
`overpay
`WARNING: Information on this form may becomepublic. Credit card information should not be included onthis form. Provide credit card
`information and authorization on PTO-2038.
`
`
`
`1. BASIC FILING, SEARCH, AND EXAMINATION FEES
`FILINGFete.
`SEARCH FEES
`EXAMINATION FEES
`
`
`
`
`mail
`Small Entity
`Smail Entity
`Entity
`
`ApplicationType Fee($)Fee ($) "hee is)FeesPaid($)Fee ($) "Fee is) Fee ($)
`
`
`
`
`Utility
`330
`165
`540
`270
`220
`110
`
`
`Design
`220
`110
`100
`50
`140
`70
`Plant
`220
`110
`330
`165
`170
`85
`
`
`325
`650
`270
`540
`165
`330
`Reissue
`—_—_______
`
`
`
`
`
`
`
`
`
`Provisional ——110__220 110 0 0 0 0
`2. EXCESS CLAIM FEESSmallEntity
`Fee DescriptionFee($)Fee($)
`
`
`Each claim over20 (including Reissues)
`52
`26
`Each independent claim over 3 (including Reissues)
`220
`110
`Multiple dependent claims
`390
`195
`
`
`
`
`Total Claims Fee Paid ($)MultipleDependentClaimsExtra Claims Fee ($)
`
`
`
`
`
`- 20 or HP =
`x
`=
`Fee ($)
`Fee Paid ($)
`HP = highest numberof total claims paid for, if greater than 20.
`Indep. Claims
`Extra Claims
`Fee ($)}
`=
`x
`-3orHP =
`
`
`HP = highest numberof independentclaimspaidfor, if greater than 3.
`
`
`3. APPLICATION SIZE FEE
`If the specification and drawings exceed 100 sheets of paper (excluding electronically filed sequence or computer
`listings under 37 CFR 1.52(e)), the application size fee due is $270 ($135 for small entity) for each additional 50
`sheets or fraction thereof. See 35 U.S.C. 41(a)(1(G) and 37 CFR 1 -16(s).
`Fee Paid ($)
`Fee ($)
`Total Sheets
`Extra Sheets
`Numberof each additional 50 or fraction thereof
`- 100 =
`/50= ______s (round up toa whole number)
`x =
`4. OTHER FEE(S)
`Fees Paid {$)
`$130 fee (no small entity discount)
`Non-English Specification,
`Other(e.g., late filing surcharge):
`
`FEE CALCULATION
`
`
`
`
`
`
`
`
`Fee Paid ($)
`
`
`
`
`
`
`—_—
`
`
`
`
`
`eo
`
`emedhoar
`Registration No.
`
`973-622-0048
`phone
`sinawe |
`Telephone
`
`
`
`
`Date 09-22-2010
`Name(Print/Type)| Qi (Peter) Li
`This collection of information is required by 37 CFR 1.136. The information is required to obtain or retain a benefit by the public whichis to file (and by the
`USPTOto process) an application. Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1.14. This collection is estimated to take 30 minutes to complete,
`including gathering, preparing, and submitting the completed application form to the USPTO. Timewill vary depending uponthe individual case. Any comments
`on the amountof time you require to complete this form and/or suggestionsfor reducing this burden, should be sent to the Chief Information Officer, U.S. Patent
`and Trademark Office, U.S. Department of Commerce, P.O. Box 1450, Alexandria, VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMSTO THIS
`ADDRESS. SEND TO: Commissionerfor Patents, P.O. Box 1450, Alexandria, VA 22313-1450.
`if you need assistance in completing the form, call 1-800-PTO-9199 and select option 2.
`Page3 of 28
`SONOS EXHIBIT1019
`
`Page 3 of 28
`
`SONOS EXHIBIT 1019
`
`

`

`PTO/SB/92 (07-09)
`Approved for use through 07/31/2012. OMB 0561-0031
`Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE
`
`| hereby certify that this correspondenceis being deposited with the United States Postal Service
`with sufficient postage asfirst class mail in an envelope addressedto:
`
`Under the Paperwork Reduction Act of 1995, no persons are required to respondto a collection of information untess it contains a valid OMB control number. Certificate of Mailing under 37 CFR 1.8
`
`Commissioner for Patents
`P.O. Box 1450
`Alexandria, VA 22313-1450
`
`09-22-2010
`
`Date
`
`2
`
`Qi (Peter)Li
`
`Signature
`
`Typed or printed name of person signing Certificate
`
`Registration Number,if applicable
`
`973-822-0048
`
`Telephone Number
`
`Note: Each paper musthaveits owncertificate of mailing, or this certificate must identify
`each submitted paper.
`
`This collection of information is required by 37 CFR 1.8. The information is required to obtain or retain a benefit by the public which is to file (and by the USPTO to
`process)an application. Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1.11 and 1.14. This collection is estimated to take 1.8 minutes to complete,
`including gathering, preparing, and submitting the completed application form to the USPTO. Timewill vary depending upon the individual case. Any comments
`on the amountof time you require to complete this form and/or suggestions for reducing this burden, should be sent to the Chief Information Officer, U.S. Patent
`and Trademark Office, U.S. Department of Commerce, P.O. Box 1450, Alexandria, VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMSTO THIS
`ADORESS. SEND TO: Commissionerfor Patents, P.O. Box 1450, Alexandria, VA 22313-1450.
`
`if you need assistance in completing the form, call 1-800-PTO-9199 and select option 2.
`
`Page 4 of 28
`
`SONOS EXHIBIT 1019
`
`Page 4 of 28
`
`SONOS EXHIBIT 1019
`
`

`

`e
`
`:
`.
`.
`:
`, Li Creative Technologies,Inc.
`[co 25 B Hanover Road, Suite 140, Florham Park, NJ 07932, USA
`
`-_ Tel: (973) 822-0048; Fax: (973) 822-0399; Website: www.licreativetech.com
`
`Written Assertion of Small Entity Status
`
`This documentstates that Li Creative Technologies, Inc. (LcT) is entitled to small entity
`status with regards to our filing a patent application with the USPTO and therefore should
`be permitted to pay reducedfees.
`,
`
`eeeeS
`
`Signature
`
`Qi (Peter) Li, President
`Printed Name
`
`09-22-2010
`Date
`
`Page 5 of 28
`
`SONOS EXHIBIT1019
`
`Page 5 of 28
`
`SONOS EXHIBIT 1019
`
`

`

`Microphone Array Design and Implementation for Telecommunications and
`Handheld Devices
`
`Manli Zhu and Qi Li
`
`1. Background of the Invention
`
`A microphonearray consists of a set of microphone sensors located at different positions. The array can
`achieve directional gain in any preferred spatial direction and frequency band while suppressing signals
`from other directions and bands. The array can be implemented by filtering and summing multiple
`microphone outputs. Conventional array processing techniques, typically developed for applications such
`as radar and sonar, are generally not appropriate for hands-free or handheld speech acquisition devices.
`The main reason is that the desired speech signal has an extremely wide bandwidth relative to its center
`frequency, meaning that conventional narrowband techniquesare not suitable.
`In the approaches to keep
`the constant response in the wide range of frequency, the array size is usually large;
`thus most of
`prototypes or products of microphone arrays on the market are quite large, which prevents the array
`products from having broader applications, such as for use in mobile and handheld communication
`devices.
`
`2. Summary of the Invention
`
`Ourinvention is Microphone Array Design and Implementation for Telecommunications and Handheld
`Devices. Our invention can be usedfor arbitrary directivity pattern for arbitrarily distributed microphones.
`Ourinvention can be used to design a microphonearray for small, portable communication devices, such
`as conference phones, mobile phones, or tablet computers. Toillustrate our invention, we present three
`applications of our invention: (1) a microphonearray for a conference phone conference phone device
`with of eight microphones non-uniformly distributed on around a circle with diameter of 4 inches, (2) a
`microphonearray of four microphoneslocated at the four corners of a rectangle for a wireless phone or
`handheld device; and (3) a microphone array of four microphoneslocated on the frame ofa tablet
`computer.
`
`3. Description of the Invention
`
`the drawings
`In this section, we provide a complete description of our invention together with all
`necessary to understand the invention. Our invention can be used for arbitrary numbers of microphone
`components and arbitrary locations of the microphone components. Our can be implemented in either
`software or hardware or a combination
`
`Page6 of 28
`
`SONOS EXHIBIT1019
`
`Page 6 of 28
`
`SONOS EXHIBIT 1019
`
`

`

` Microphone
`
`Noise
`Reduction
`
`Array
`
`Sound Source
`Localization
`
`Figure 1. For the best performance, the invented microphone array system mayconsistof the following
`modules: microphonearray sensors, sound source localization, beamforming, and noise reduction.
`
`The microphonearray module consists of multiple microphone components functioning as a single unit to
`pick up sound signals. The source localization module serves to find the spatial location of the principal
`sound source such that an acoustic beam can point to the sound source. The beamforming module serves
`to form acoustic beamsin the direction of the principal sound source enhancing sound from this range and
`suppressing sound from all other directions. The noise reduction module serves to further reduce
`background noise and enhance speech. Depending on applications, a real product may use someorall of
`the modules.
`
`3.1. Two-Dimensional Microphone Array Configuration
`
`180°
`
`Figure. 2 Illustration of a microphonearray configuration wherein N microphonessensorsare arbitrarily
`distributed on a circle with diameter of d (N=4).
`
`Assuming N microphonesare arbitrarily distributed on a circular with diameter of d as shown in Figure 2,
`where only four microphonesare displayed. Microphonelocations are specified a acute angles from the y-
`axis, shown as ®, (®,> 0, n=1...N). The output y of the array is the filter-and-sum of the N microphone
`outputs, L.e., y= ywMy , where x, is the output ofthe (n+1)" microphone andw,is the length-Lfilter
`applied to it as shownin Figure 3.
`
`Page7 of 28
`
`SONOS EXHIBIT1019
`
`Page 7 of 28
`
`SONOS EXHIBIT 1019
`
`

`

`Sound Source
`
`Figure 3. Illustration of filter-and-sum beamforming.
`
`The spatial directivity pattern H(@,8) for the sound source from angle @ with normalized frequency o is
`definedas:
`
`1
`
`H(c,6) = 229). - DanoaX(0,9)
`
`X(0,0)
`
`X(0,6)
`
`(1)
`
`where ¥ is the signal received at the center of the circular array and W is the frequency response of the
`real-valued FIR filter w. If the sound source is far enough away from the array, the difference between the
`signal received by the (n+1)" microphone x, and the center of the array is a pure delay 1.
`i.e.,
`X,(@,t) = X(@, Oe” . Figure 4 illustrates the distance between origin and microphone M, and microphone
`M; whenthe incoming soundis from angle of @.
`
`
`
`Figure 4. Illustration of t; and 13, the distance between origin and microphone M, and microphone M3 whenthe
`incoming sound is from angle of 0.
`
`Wederived the distance for each microphone, measured both in meters and in the number of samples, and
`summarize them in Table 1.
`
`Page 8 of 28
`
`SONOS EXHIBIT1019
`
`Page 8 of 28
`
`SONOS EXHIBIT 1019
`
`

`

`Table 1. Distance between each microphoneandorigin.
`Note: d is the radius ofthe circle, f, is the sampling frequency, and C is the sound speed.
`
`
`
`Distance (numberof samples)
`d*cos(O+@o)|d*cos(O+®y)*f,/C
`
`d*cos(0-;)|d*cos(0-@,)*f/C
`
`M
`-d*cos(0+02)*f/C
`[M3 -d*cos(8-@;
` -d*cos(8-@3)*f£,/C
`M3
`
`
`
`In general, the distance and the location have the following relationship:
`
`Table 2. Relationship of microphone position and its distance to the origin.
`
`Distance (numberof samples
`-d*cos(6)*f,/C
`d*cos(8)*f/C
`-d*sin(@)*£/C
`|d*sin(@)|d*sin(®)*fC
`® clockwise away
`from 0°
`(0<@<90°)
`-d*cos(0- ©)*f,/C
`® anticlockwise away from 0°
`(0<®<90°)
`-d*cos(6+ 0)*£,/C
`® clockwise away from 180°
`(0<®<90°
`d*cos(0- ©)*f./C
`® anticlockwise away from 180° (0<®<90°)
`
`Microphoneposition
`
`180°
`
`d*cos(0+ ®)*f£/C
`
`
`
`
`
`Now,the spatial directivity pattern H can be re-written as:
`
`H(o,0)= YW,aye?"=w"(0,8)
`
`(2)
`
`where w' = [WoW,W2',W3", ... Waa] and g(@,0) ={2'(,6)},_)y, = {ee}a
`is the steering vector, i=1...NZ, k=mod(i-/,L) and n=floor((i-/)/L).
`
`3.2. Extension to 3-Dimensional Sound Source
`
`Thecalculation in section 3.1 is for the sound source in the same plane with the array. In real applications,
`the sound can comefrom any direction in the 3-D space. We generalize the problem as shownin Figure 5.
`The sound is from the 3-dimentional (3-D) space, where 'P is the elevate angle and 6 is the azimuth. We
`have proved that when the sound is coming from the angle of (‘Y,0), the delay between each microphone
`and the center of the array is similar to Table 2 but with an extra factor sin(‘V) as shown in Table 3. When
`Y moves from 90° to 0°, sin(¥) changes from 1
`to 0, and as the result, the difference between each
`microphone gets smaller and smaller. When =0°, there is no difference between microphones, which
`means the sound reaches each microphoneat the same time. Taking into account that the sample delay
`between microphones can only be an integer, we determine the range whereall microphonesare identical.
`As shown in Figure 6, when Y<@, four microphones receive identical signals for 0°<0<360°. Our
`beamforming technique enhances sound from this range and suppresses sound from all other directions,
`treating it as background noise.
`
`Page 9 of 28
`
`SONOS EXHIBIT1019
`
`Page 9 of 28
`
`SONOS EXHIBIT 1019
`
`

`

`Figure 5. I!lustration of 3-D sound source: The soundis from the direction (‘¥,6), where '¥ is the elevation angle
`and 6 is the azimuth.
`
`
`TW=refSesin(O+D)sin(VY/C|t2= -refSesin(0-D)sin(VVC|t3= refSesin(O+@D)sin(PV/C|t4= refSesin(0-®)sin(¥)/C
`
`Table 3 The delay between each microphoneandthe array center for sound from (‘Y,6).
`
`Figure6. Illustration of the array working space: When sound comes from ‘¥< ®, four microphones receive same
`signals. Our beam-forming technique will enhance sound from this range and treating sound from other directions
`(for example S, and S,) as background noise to suppress.
`
`3.3. Least Mean SquareSolution
`
`Let the desired spatial directivity pattern be 1
`function can be defined as
`
`in pass band and 0 in stop band. The least square cost
`
`Jow)=[) | |H(@,8)-1P dodo+al,|, |H(@,0)? dodo
`-f,{| H(o,0)dadd+ahe{,! H(a,0)|? dad-2fi,{, Re(H(a,0)dad0+he[idade
`
`3)
`
`Replacing | H(@,0)|’= w’g(a, @)g” (@,@)w = w'G(a, 0)w = w' (G,(o, 0) + jG, (@,))w = w'G,(a,A)w and
`Re(H(a,)) = w’g,(@,9) , we then have
`
`Page 10 of 28
`
`SONOS EXHIBIT 1019
`
`Page 10 of 28
`
`SONOS EXHIBIT 1019
`
`

`

`J(@) =w'Qw-2w'at+d , where
`
`O=|. [, G,(a,0)dad6 +a[, {, G,(@,0)dad0
`a=||, ge(@,0)dado
`i, i,
`
`1 dadd@
`
`d=
`
`(4)
`
`where g,(@,0) = cos[@(k +7,)] and G,(@,0) =cos[@(k-1+7, —T,,)].
`
`When Q//dw=0, the cost function J is minimized. The least-square estimate of w is obtained by
`
`w=Q'a
`
`(5)
`
`3.4, Linear Constrain
`
`Applying linear constrains Cw = b, we can further constrain the spatial response to a predefined value b
`at angle 0; using following equation:
`
`Br (®etan > 6,)
`vee
`
`Doant
`wH|..
`
`Be(Pena>9/)
`
`Bend
`
`Now,the design problem becomes
`
`min wQw-2w'a+d_— subjectto Cw=b
`
`and the solution of the constrained minimization problem is equalto:
`
`w=Q'C'(CQO"'C’)'(b-CQ"'a)+Q"'a
`
`wherew isthe filter parameters for the designed beamformer.
`
`(6)
`
`(7)
`
`(8)
`
`3.5 Sound Source Localization
`
`There are two categories of techniques to estimate the sound localization: one employs time difference of
`arrival (TDOA)andanotheris based on steered response power (SRP).
`
`For an array with N microphones, a delayed, filtered and noise corrupted version of sound signals is
`presented in each of the microphone signals. The delay-and-sum beam formertime aligns and sumsall
`the microphone signal as y(t,q) = ye X,(t+A,), where A is the steering delay appropriate for
`
`n=0
`
`Page 11 of 28
`
`SONOS EXHIBIT 1019
`
`Page 11 of 28
`
`SONOS EXHIBIT 1019
`
`

`

`focusing the array to the direction of q. When the focus correspondsto the location of the sound source,
`the steered response power (SRP) should reach a global maximum.
`
`Time difference of arrival can be used to estimate the sound source location. According to the sound
`propagation theory, the sound direction is uniquely determined by the time difference for a wave to
`propagate through non-linearly distributed distant microphones. Estimating the sound direction is
`essentially identical to estimate the TDOA,which is achieved by estimating the cross correlation.
`
`Our preliminary research showed that TDOA-based localization is effective under low to moderate
`reverberation condition. The SRP approach requires shorter analysis intervals and exhibits an elevated
`insensitivity to environmental condition while not allowing for use under excessive multi-path. We
`implemented a new method called SRP-PHAT which combines the advantages of two approaches, and
`has a decreased sensitivity to noise and reverberations and more precise location estimates than the
`existing localization methods.
`
`Figure 7 shows our experimental results. The upper plot is the value of SPR-PHATat each angle. The
`minimum value correspondsto the soundlocation.
`
`
`
`
`
`Figure 7. The upper image showsthe value of SRP-PHATfor every 10°; the lower image represents the estimation
`and groundtruth.
`
`3.6 Adaptive Beamforming
`
`Section 3.2 and 3.3 introduce the algorithm to derive the fixed beamforming to form the directivity
`pattern. We further extend it
`to adaptive beamforming. Adaptive beamforming can achieve better
`interference suppression than fixed beamforming. This is because the target direction of arrival, which is
`assumedto be stable in fixed beamforming, does change with the movement of the speaker. Also, the
`sensor gains, which are assumed uniform in fixed beamforming, exhibit significant distribution. All these
`factors will reduce speech quality. On the other hand, adaptive beamforming adaptively performs beam
`
`Page 12 of 28
`
`SONOS EXHIBIT 1019
`
`Page 12 of 28
`
`SONOS EXHIBIT 1019
`
`

`

`steering and null steering; therefore, the adaptive beamforming method is more robust against steering
`error caused by the array imperfection mentioned above.
`
`The structure of our adaptive beamforming method is shown in Figure 8. It comprises of a fixed
`beamforming, a blocking matrix (BM) anda set of adaptive filters. The purpose of the blocking matrixis
`to block the target signal and let interfering noises through. The interfering noises are fed into an adaptive
`filter to minimize their influence in the output. One of the key steps in adaptive beamforming is to
`determine whenthe adaptation should be applied. Because of signal leakage, the output z of the blocking
`matrix may contain some weakspeechsignals. If the adaptation is active when speech is present, the
`speech will be cancelled out together with the noise; therefore, our invention uses a control module on the
`adaptation. This module enables adaptation according to the spectrum and energy of both noise signal and
`speechsignal.
`
`
`
`eamforming
`
`& ‘fs
`
`Speech signal
`
`Noise
`
`Blocking
`Matrix
`(BM)
`
`Spectrum and/or energy
`of b and z;
`
`
`
`Figure 8. Diagram of adaptive beamforming: It consists of a fixed beamformer, the blocking matrix, and the
`adaptivefilter. A control module is applied to enable/disable the adaptation process.
`
`In Figure 8, the dotted block represents our adaptive filtering process. We developed a sub-band adaptive
`filtering for this invention for two reasons: firstly, it leads to a higher convergence speed than when using
`a full band adaptivefilter. Secondly, our noise reduction algorithm is developed in sub-band, so applying
`sub-band adaptivefiltering here provides the same frameworkfor both beamforming and noise reduction,
`and saves on computational cost. Figure 9 shows the structure of our sub-band adaptive filtering. Both
`input signals are split into frequency sub-bandsvia an analysis filter bank. Each sub-band adaptive filter
`usually has a shorter impulse response than its full band counterpart. The step size can be adjusted
`individually for each sub-band, which leads to a higher convergence speed than when using a full band
`filter.
`
`Page 13 of 28
`
`SONOS EXHIBIT 1019
`
`Page 13 of 28
`
`SONOS EXHIBIT 1019
`
`

`

`
` Outputb offixed
`Analysis
`filter
`beamforming
`
`Analysis
`aL
`Synthesis
`
`
`filter ee NR E filter
`
` bank pS bank
`
`
`locki
`neezofblocking
`
`0
`
`Sub-band adaptivefiltering
`
`Speechsignal
`
`matrix
`
`Analysis
`
`Adaptation
`
`Synthesis
`
`Figure 9. The structure of the sub-band adaptivefilter: In the analysis step, both outputs of fixed beamforming and
`blocking matrix are split into sub-band through the analysis filter bank. In the adaptation step, the filter is adapted
`such that the output only contains speech signal. Finally, in the synthesis step, the sub-band speechsignalis
`synthesized to full-band speech throughthe synthesis filter bank. Because noise reduction and beamformingare in
`the same sub-band framework, we applied noise reduction (NR) before synthesis to save computation. The NR
`module will be introduced in the next section.
`
`To ensure the speech quality, the filter bank should not distort the sound signal by itself. We already
`implemented an efficient perfect-reconstruction filter bank, which can fully meet this requirement.In this
`implementation, all sub-band filters are factorized to operation on the prototypefilter coefficients and a
`modulation matrix is used to take advantage of FFT. This modification ensures a minimum amountof
`multiply-accumulate operations. Figure 10 shows the performance of our filter bank. The blue line
`represents the input signal to the filter bank, and the red circleis the output ofthe filter band after analysis
`and synthesis. The output perfectly matches the input, called perfect-reconstruction filter bank.
`
`Page 14 of 28
`
`SONOS EXHIBIT 1019
`
`Page 14 of 28
`
`SONOS EXHIBIT 1019
`
`

`

`\/0 for real valued GDFT OSFB
`—T
`T
`TT a
`
`7
`
`es
`
`
`input
`filterbank output
`
`©
`
`05/
`
`0.4
`
`0.3
`
`2°Oo=NR
`
`Fo)
`
`0.2
`
`
`
`input,output
`
`
`40.3 0.4)-
`
`
`Lo
`! a !
`= nt
`i
`!
`I
`t
`1
`1.0384
`1.0386
`1.0388
`1.039
`1.0392
`1.0394
`1.0396
`1.0398
`1.04
`1.0402
`1.0404
`time [fullband sampling periods]
`x 10°
`
`Figure 10. Perfect reconstruction filter bank input and output: The blue line represent input signalto filter bank and
`red circle is the output of the filter band after analysis and synthesis. The output perfectly matchesthe input.
`
`The noise reduction (NR) module as shown in Figure 9 is used to further reduce background noise after
`adaptive beamforming. It explores the short-term and long-term statistics of speech and noise, and the
`wide-band and narrow-band signal-to-noise ratio (SNR) to support a Wiener gain filtering. After the
`spectrum of noisy-speech passes through the Wienerfilter, an estimation of the clean-speech spectrum is
`generated. The filter bank synthesis module, as an inverse process of filter bank analysis module,
`reconstructs the signals of the clean speech given the estimated spectrum ofthe clean speech.
`
`3.7 Noise reduction
`
`The noise reduction module can include any kind of noise reduction algorithm, such as Wienerfilter-
`based noise reduction, spectral subtraction noise reduction, auditory (or cochlear) transform-based noise
`reduction, or model-based noise reduction algorithm.
`
`3.8 Hardware Implementation:
`
`The structure of circuit design is shown in Figure 11. The acoustic signal is picked up by four or eight
`microphone components/elements arranged as a linearor circular array. First, the microphone amplifiers
`provide 20dB gain to boost the signal level to enhance the microphone sensitivity, and then the audio
`Codec provides an adjustable gain level from -74dB to 6dB before it converts the four channels of analog
`signals into digital signals. The pre-amplifier may not need for some applications. The Codec then
`
`Page 15 of 28
`
`SONOS EXHIBIT 1019
`
`10
`
`Page 15 of 28
`
`SONOS EXHIBIT 1019
`
`

`

`transmits the digital audio signals to DSP (digital signal processing) chip for audio signal processing and
`computation. The DSPchip also transmits output signal to the Codec, and then the Codec converts it into
`analog signal, whichis then amplified by speaker amplifier to drive the internal loudspeakerif it is needed.
`
`The flash memory stores the code for the DSP chip and compressed audio signals. Once the system boots
`up, the DSP chip reads the code from flash memory into internal memory andstarts to execute the code.
`During the start up stage, we can also configure the Codec by writing to the registers of the DSP chip.
`There are switch power regulators and linear power regulators to provide appropriate voltage and current
`supply for all the components on the board.
`
`Digital domain
`
`External
`headphone
`acrcket
`
`‘
`
`: Speaker
`amplitier
`
`:
`|°
`
`:
`
`-
`
`eerove
`
`Analog domain
`
`External
`
`microphone
`socket
`
`+
`
`i.
`as
`
`‘Microphone .
`amplifier
`|:
`
`A.
`
`amplifier
`
`|:
`
`a .
`
`wee!
`
`ON
`/
`
`i icrophone :
`
`amplifier
`
`Microphone
`amplifier
`
`|.
`
`eeoeeeroereerseazres
`Flash memory
`
`C} ‘hi icrophone i
`
`Audio Codec
`
`$510 DSP chip
`
`¢*»'e*
`
`
`
`
`Rechargeableo™ Battery
`
`
`Linear power
`regulators
`
`Switch power/,
`regulators
`
`|
`
`Figure 11. Hardware implementation of the invention: It consists of 3 major chips, codec, DSP, and flash memory.
`The USBcontrolis built in the DSP chip. For 8-sensor microphone array, we can use two four-channel codec chips.
`
`Wewill use a mixed signal circuit board (6-layer PCB). The board layout will be carefully partitioned to
`isolate the analogcircuits from the digital circuits, because the noisy digital signal can easily contaminate
`the low voltage analog signal from the microphones. Although the speaker amplifier’s input and outputis
`
`11
`
`Page 16 of 28
`
`SONOS EXHIBIT 1019
`
`Page 16 of 28
`
`SONOS EXHIBIT 1019
`
`

`

`an analog signal, it will be placed in the digital region because of its high power consumption andits
`switch amplifier nature. Only linear power regulators are deployed in the analog region dueto their low
`noise property.
`
`To ensure the quality, five power regulators are designed in the microphonearray circuits. The switch
`powerregulators can achieve efficiency to 95% of input power and have high output current capacity but
`their outputs are too noisy for analog circuits. The linear regulators’ efficiency is determined bythe ratio
`of the output voltage over the input voltage, which is lower than that of switch regulators in most of the
`cases. Our experiments showed that the regulator outputs are very stable, quiet, and suitable for the low
`poweranalogcircuits.
`
`Figure 12(A) is our 4-sensor microphone array product named CrispMic™. Figure 12(B) is the PCB
`design, which is very similar to the PCB of the proposed medical recorder. In the PCB, we selected two
`new chips from the state-of-the-art semiconductor technology. The DSP chip from TI is a low power
`consumption design devised especially for portable