Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 1 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 1 of 38
`
`
`
`
`
`
`
`
`EXHIBIT 8
`EXHIBIT 8
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 2 of 38
`ee” STETAA ATTA
`
`US010779080B2
`
`a2) United States Patent
`US 10,779,080 B2
`10) Patent No.:
`
` Burnett (45) Date of Patent: *Sep. 15, 2020
`
`
`(54) DUAL OMNIDIRECTIONAL MICROPHONE
`ARRAY (DOMA)
`
`(71) Applicant: JAWB Acquisition LLC, New York,
`NY (US)
`
`(72)
`
`Inventor: Gregory C. Burnett, Dodge Center,
`MN (US)
`
`(73) Assignee:
`
`JAWB ACQUISITION LLC, New
`York, NY (US)
`
`(*) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`This patent is subject to a terminal dis-
`claimer.
`
`(21) Appl. No.: 13/948,160
`
`(22)
`
`Filed:
`
`Jul. 22, 2013
`
`(65)
`
`Prior Publication Data
`US 2014/0177860 Al
`Jun. 26, 2014
`
`Related U.S. Application Data
`(63) Continuation of application No. 12/139,355, filed on
`Jun. 13, 2008, now Pat. No. 8,494,177.
`
`(60) Provisional application No. 60/934,551, filed on Jun.
`13, 2007, provisional application No. 60/953,444,
`filed on Aug. 1, 2007, provisional application No.
`60/954,712,
`filed on Aug. 8, 2007, provisional
`application No. 61/045,377, filed on Apr. 16, 2008.
`
`(51)
`
`Int. Cl.
`HOR 3/00
`GIOL 21/0208
`HO4R 1/40
`HOAR 3/04
`GIOL 21/0216
`
`(2006.01)
`(2013.01)
`(2006.01)
`(2006.01)
`(2013.01)
`
`(52) U.S. Cl.
`CPC oe. HOAR 3/002 (2013.01); GIOL 21/0208
`(2013.01); HO4R 1/406 (2013.01); HO4R
`3/005 (2013.01); HO4R 3/04 (2013.01); GIOL
`2021/02165 (2013.01)
`(58) Field of Classification Search
`CPC ...... G10L 2021/02165; G10L 21/0208; HO4R
`3/005
`USPC weceeeneee 381/92, 94.7; 704/233, E21.004
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`5,473,701 A
`12/1995 Cezanneet al.
`6,473,733 Bl
`10/2002 McArthuret al.
`(Continued)
`
`OTHER PUBLICATIONS
`
`Weiss, Howard; Office Action mailed by U.S. Patent and Trademark
`Office dated Mar. 15, 2012 for U.S. Appl. No. 12/139,355.
`(Continued)
`Primary Examiner — Howard Weiss
`(74) Attorney, Agent, or Firm — Hard IP LLC
`
`ABSTRACT
`67)
`A dual omnidirectional microphone array noise suppression
`is described. Compared to conventional arrays and algo-
`rithms, which seek to reduce noise by nulling out noise
`sources, the array of an embodiment is used to form two
`distinct virtual directional microphones which are config-
`uredto have very similar noise responses andvery dissimilar
`speech responses. The only null formed is one used to
`remove the speech of the user from V,. The two virtual
`microphones maybe paired with an adaptivefilter algorithm
`and VAD algorithm to significantly reduce the noise without
`distorting the speech, significantly improving the SNR ofthe
`desired speech over conventional noise suppression systems.
`
`20 Claims, 17 Drawing Sheets
`
`104
`
`Voicing Information
`
`100
`
`101
`
`)
`SIGNAL
`s(n)
`
`100
`
`
`
`Cleaned Speech
`
`
`
`Noise Removal
`
`
`
`
`
`
`
`>
`
`(ce)
`NOISE
`n(n)
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 3 of 38
`Page 3 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22
`
`US 10,779,080 B2
`
`Page 2
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`7,706,549 B2
`4/2010 Zhang et al.
`8,068,619 B2* 11/2011 Zhang etal. ow. 381/92
`
`w. 381/92
`8,494,177 B2*
`7/2013 Burnett
`....
`8/2013 Burnett wo. 381/92
`8,503,691 B2*
`
`OTHER PUBLICATIONS
`
`Weiss, Howard; Office Action mailed by U.S. Patent and Trademark
`Office dated Jun. 13, 2008 for U.S. Appl. No. 12/139,355.
`
`* cited by examiner
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 4 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 4 of 38
`
`U.S. Patent
`
`Sep. 15, 2020
`
`Sheet 1 of 17
`
`US 10,779,080 B2
`
`
`
`WOTCWIOJU]SUIIIOA
`
`v0l
`
`(«*»)|001
`
`‘TVNDIS
`
`(u)s
`
`101
`
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 5 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 5 of 38
`
`U.S. Patent
`
`Sep. 15, 2020
`
`US 10,779,080 B2
`
`Sheet 2 of 17
`
`
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 6 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 6 of 38
`
`U.S. Patent
`
`Sep. 15, 2020
`
`Sheet 3 of 17
`
`US 10,779,080 B2
`
`
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 7 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 7 of 38
`
`U.S. Patent
`
`Sep. 15, 2020
`
`Sheet 4 of 17
`
`US 10,779,080 B2
`
`
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 8 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 8 of 38
`
`U.S. Patent
`
`Sep. 15, 2020
`
`Sheet 5 of 17
`
`US 10,779,080 B2
`
`
`
`FIG.6
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 9 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 9 of 38
`
`U.S. Patent
`
`Sep. 15, 2020
`
`Sheet 6 of 17
`
`US 10,779,080 B2
`
`Receive acoustic signals at a first physical
`microphone and a second physical microphone.
`
`Output first microphone signal from first physical
`microphone and second microphonesignal from
`second physical microphone.
`
`Form first virtual microphone using the first combination
`offirst microphone signal and second microphonesignal.
`
`Form second virtual microphone using second combination
`of first microphone signal and second microphonesignal.
`
`Generate denoised output signals having less
`acoustic noise than received acoustic signals.
`700—*
`FIG.7
`
`Form physical microphone array includingfirst
`physical microphone and second physical microphone.
`
`signals from physical microphonearray.
`
`Form virtual microphonearray includingfirst virtual
`microphone and secondvirtual microphone using
`
`800 —~
`
`FIG.8
`
`102
`
`704
`
`706
`
`708
`
`710
`
`802
`
`804
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 10 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 10 of 38
`
`U.S. Patent
`
`Sep. 15, 2020
`
`Sheet 7 of 17
`
`US 10,779,080 B2
`
`Linear response of V2 to a speech source at 0.10 meters
`Linear response 0
`
`f V2 to a noise source at 1 meters
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 11 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 11 of 38
`
`U.S. Patent
`
`Sep. 15, 2020
`
`Sheet 8 of 17
`
`US 10,779,080 B2
`
`Linear response of V1 to a speech source at 0.10 meters
`
`14411Vrme
`
`FIG.11
`Linear response of V1 to a noise source at | meters
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 12 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 12 of 38
`
`U.S. Patent
`
`Sep. 15, 2020
`
`Sheet 9 of 17
`
`US 10,779,080 B2
`
`Linear response of V1 to a speech source at 0.1 meters
`
`90
`
`2
`
`180 |---}---+---|----+------ 4
` V
`
`—
`
` “F300 Hz
`
`/ 4000Hz~~
`
`240
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 13 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 13 of 38
`
`U.S. Patent
`
`Sep. 15, 2020
`
`Sheet 10 of 17
`
`US 10,779,080 B2
`
`
`
`Response(dB)
`
`Frequency responseat 0 degrees
`
`Cardioid speech
`response
`
`!
`
`nooracponwnoroconebevccoeeecodenms WI. gpeethy --feevnnveeecbevcneh
`!
`|
`response
`!
`
`0
`
`1000
`
`2000
`
`3000
`
`4000
`
`5000
`
`6000
`
`7000
`
`8000
`
`Frequency (Hz)
`
`FIG.14
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 14 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 14 of 38
`
`U.S. Patent
`
`Sep. 15, 2020
`
`Sheet 11 of 17
`
`US 10,779,080 B2
`
`
`
`
`
`V1/V2 for speech versus B assuming d, = 0.1m
`
` V1 (top, dashed) and V2 speech response vs. B assuming d, = 0.1m
`
`0.9
`
`0.7
`
`0.8
`B
`FIG.15
`
`
`
`Response(dB)
`
`
`
`V1/V2forspeech(dB)
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 15 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 15 of 38
`
`U.S. Patent
`
`Sep. 15, 2020
`
`Sheet 12 of 17
`
`US 10,779,080 B2
`
`
`
`
`
`0.05
`
`O01
`
`015
`
`O38
`025
`O02
`Actual d, (meters)
`FIG.17
`
`035
`
`O04
`
`045
`
`05
`
`B factor vs. actual d, assuming d, = 0.1m and theta = 0
`
`
`B versus theta assuming d, = 0.1m
`
`
`
`80
`
`60
`
`40
`
`20
`0
`-20
`theta (degrees)
`FIG.18
`
`40
`
`60
`
`#80
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 16 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 16 of 38
`
`U.S. Patent
`
`Sep. 15, 2020
`
`Sheet 13 of 17
`
`US 10,779,080 B2
`
`(dB)
`Amplitude
`(degrees)
`Phase
`
`0
`
`1000
`
`2000
`
`3000
`
`4000
`
`5000
`
`6000
`
`7000
`
`8000
`
`0
`
`1000
`
`2000
`
`3000
`
`4000
`
`5000
`
`6000
`
`7000
`
`8000
`
`Frequency (Hz)
`
`FIG.19
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 17 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 17 of 38
`
`U.S. Patent
`
`Sep. 15, 2020
`
`Sheet 14 of 17
`
`US 10,779,080 B2
`
`(dB)
`Amplitude
` Phase
`(degrees)
`
`“40
`
`1000
`
`§=2000
`
`3000
`
`4000
`
`5000
`
`6000
`
`7000
`
`8000
`
`0
`
`1000
`
`2000
`
`3000
`
`4000
`
`5000
`
`6000
`
`7000
`
`8000
`
`Frequency (Hz)
`
`FIG.20
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 18 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 18 of 38
`
`U.S. Patent
`
`Sep. 15, 2020
`
`Sheet 15 of 17
`
`US 10,779,080 B2
`
`J 2
`
`0fn
`
`oe
`
`
`
`Amplitude(dB)
`
`
`
`
`
`
`
`
`
`Cancellation with dl = 1, thetal = 0, d2 = 1, and theta2 = 30
`40
`
`
`
`
`
`
`
`0 1000+2000 3000 4000 5000 6000 7000 8000
`
`
`
`Phase(degrees)
`
`
`
`60
`
`0
`
`1000
`
`2000
`
`3000
`
`©4000
`
`5000
`
`6000
`
`7000
`
`8000
`
`Frequency (Hz)
`
`FIG.21
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 19 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 19 of 38
`
`U.S. Patent
`
`Sep. 15, 2020
`
`Sheet 16 of 17
`
`US 10,779,080 B2
`
`
`
`Amplitude(dB)
`
`
`
`0
`
`1000
`
`2000
`
`3000
`
`4000
`
`5000
`
`6000
`
`7000
`
`8000
`
`Cancellation with dl = 1, thetal = 0, d2 = 1, and theta2 = 45
`
`
`
` Phase(degrees)
`
`
`
`0
`
`1000
`
`2000
`
`3000
`
`4000
`
`5000
`
`6000
`
`7000
`
`8000
`
`Frequency (Hz)
`
`FIG.22
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 20 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 20 of 38
`
`U.S. Patent
`
`Sep. 15, 2020
`
`Sheet 17 of 17
`
`US 10,779,080 B2
`
`Original V1 (top) and cleaned V1 (bottom) with simplified VAD (dashed) in noise
`
`Cleaned
`
`Noisy
`
`0
`
`0.5
`
`1
`1.5
`Time(samples at 8 kHz/sec)
`
`2
`
`2.5
`
`FIG.23
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 21 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 21 of 38
`
`US 10,779,080 B2
`
`1
`DUAL OMNIDIRECTIONAL MICROPHONE
`
`ARRAY (DOMA)
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application is continuation of U.S. Nonprovisional
`patent application Ser. No. 12/139,355, filed Jun. 13, 2008,
`now USS. Pat. No. 8,494,177, entitled “Dual Omnidirec-
`tional Microphone Array (DOMA),” which claims the ben-
`efit of U.S. Provisional Patent Application No. 60/934,551,
`filed Jun. 13, 2007, U.S. Provisional Patent Application No.
`60/953,444, filed Aug. 1, 2007, U.S. Provisional Patent
`Application No. 60/954,712, filed Aug. 8, 2007, and U.S.
`Provisional Patent Application No. 61/045,377, filed Apr.
`16, 2008, all of which are incorporated by reference herein
`in their entirety for all purposes.
`
`TECHNICAL FIELD
`
`The disclosure herein relates generally to noise suppres-
`sion. In particular, this disclosure relates to noise suppres-
`sion systems, devices, and methods for use in acoustic
`applications.
`
`BACKGROUND
`
`Conventional adaptive noise suppression algorithms have
`been around for some time. These conventional algorithms
`have used two or more microphones to sample both an
`(unwanted) acoustic noise field and the (desired) speech of
`a user. The noise relationship between the microphonesis
`then determined using an adaptive filter (such as Least-
`Mean-Squares
`as
`described in Haykin & Widrow,
`ISBN#0471215708, Wiley, 2002, but any adaptive or sta-
`tionary system identification algorithm may be used) and
`that relationship used to filter the noise from the desired
`signal.
`Most conventional noise suppression systems currently in
`use for speech communication systems are based on a
`single-microphone
`spectral
`subtraction technique
`first
`develop in the 1970’s and described, for example, by S. F.
`Boll in “Suppression of Acoustic Noise in Speech using
`Spectral Subtraction,” IEEE Trans. on ASSP, pp. 113-120,
`1979. These techniques have been refined overthe years, but
`the basic principles of operation have remained the same.
`See, for example, U.S. Pat. No. 5,687,243 of McLaughlin, et
`al., and U.S. Pat. No. 4,811,404 of Vilmur, et al. There have
`also been several attempts at multimicrophone noise sup-
`pression systems, such as those outlined in U.S. Pat. No.
`5,406,622 of Silverberg et al. and U.S. Pat. No. 5,463,694 of
`Bradley et al. Multi-microphone systems have not been very
`successful for a variety of reasons, the most compelling
`being poornoise cancellation performance and/or significant
`speech distortion. Primarily, conventional multi-microphone
`systems attempt to increase the SNR ofthe user’s speech by
`“steering” the nulls of the system to the strongest noise
`sources. This approach is limited in the number of noise
`sources removed by the numberof available nulls.
`The Jawbone earpiece (referred to as the “Jawbone),
`introduced in December 2006 by AliphCom of San Fran-
`cisco, Calif., was thefirst known commercial product to use
`a pair of physical directional microphones (instead of omni-
`directional microphones) to reduce environmental acoustic
`noise. The technology supporting the Jawboneis currently
`described under one or more of U.S. Pat. No. 7,246,058 by
`Burnett and/or U.S. patent application Ser. Nos. 10/400,282,
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`10/667,207, and/or 10/769,302. Generally, multi-micro-
`phone techniques make use of an acoustic-based Voice
`Activity Detector (VAD)to determine the background noise
`characteristics, where “voice” is generally understood to
`include human voiced speech, unvoiced speech, or a com-
`bination of voiced and unvoiced speech. The Jawbone
`improved on this by using a microphone-based sensor to
`construct a VAD signal using directly detected speech vibra-
`tions in the user’s cheek. This allowed the Jawbone to
`
`aggressively remove noise when the user was not producing
`speech. However, the Jawbone uses a directional micro-
`phonearray.
`
`INCORPORATION BY REFERENCE
`
`Each patent, patent application, and/or publication men-
`tioned in this specification is herein incorporated by refer-
`ence in its entirety to the same extent as if each individual
`patent, patent application, and/or publication was specifi-
`cally and individually indicated to be incorporated by ref-
`erence.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a two-microphone adaptive noise suppression
`system, under an embodiment.
`FIG. 2 is an array and speech source (S) configuration,
`under an embodiment. The microphones are separated by a
`distance approximately equal to 2d,, and the speech source
`is located a distance ds away from the midpoint of the array
`at an angle 0. The system is axially symmetric so only d, and
`8 need be specified.
`FIG. 3 is a block diagram for a first order gradient
`microphone using two omnidirectional elements O, and O,,
`under an embodiment.
`
`FIG. 4 is a block diagram for a DOMAincluding two
`physical microphones configured to form two virtual micro-
`phones V, and V,, under an embodiment.
`FIG. 5 is a block diagram for a DOMAincluding two
`physical microphones configured to form N virtual micro-
`phones V, through V,,, where N is any numbergreater than
`one, under an embodiment.
`FIG. 6 is an example of a headset or head-worn device
`that includes the DOMA, as described herein, under an
`embodiment.
`
`FIG. 7 is a flow diagram for denoising acoustic signals
`using the DOMA, under an embodiment.
`FIG. 8 is a flow diagram for forming the DOMA, under
`an embodiment.
`FIG.9 is a plot of linear response of virtual microphone
`V, to a 1 kHz speech sourceat a distance of 0.1 m, under an
`embodiment. The null is at 0 degrees, where the speech is
`normally located.
`FIG. 10 is a plot of linear response of virtual microphone
`V, toa 1 kHz noise source at a distance of 1.0 m, under an
`embodiment. There is no null and all noise sources are
`detected.
`
`FIG.11 is a plot of linear response of virtual microphone
`V, toa 1 kHz speech source at a distance of 0.1 m, under an
`embodiment. There is no null and the response for speech is
`greater than that shown in FIG. 9.
`FIG.12 is a plot of linear response of virtual microphone
`V, toa 1 kHz noise source at a distance of 1.0 m, under an
`embodiment. There is no null and the response is very
`similar to V, shown in FIG. 10.
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 22 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 22 of 38
`
`US 10,779,080 B2
`
`3
`FIG.13 is a plot of linear response of virtual microphone
`V, to a speech source at a distance of 0.1 m for frequencies
`of 100, 500, 1000, 2000, 3000, and 4000 Hz, under an
`embodiment.
`FIG. 14 is a plot showing comparison of frequency
`responses for speech for the array of an embodiment andfor
`a conventional cardioid microphone.
`FIG. 15 is a plot showing speech response for V, (top,
`dashed) and V2 (bottom, solid) versus B with ds assumed to
`be 0.1 m, under an embodiment. The spatial null in V2 is
`relatively broad.
`FIG. 16 is a plot showing a ratio of V,/V, speech
`responses shown in FIG. 10 versus B, under an embodiment.
`Theratio is above 10 dB for all 0.8<B<1.1. This meansthat
`
`the physical 6 of the system need not be exactly modeled for
`good performance.
`FIG.17 is a plot of B versusactual ds assuming that ds=10
`cm and theta=0, under an embodiment.
`FIG. 18 is a plot of B versus theta with ds=10 cm and
`assuming ds=10 cm, under an embodiment.
`FIG. 19 is a plot of amplitude (top) and phase (bottom)
`response of N(s) with B=1 and D=-7.2 usec, under an
`embodiment. The resulting phase difference clearly affects
`high frequencies more than low.
`FIG. 20 is a plot of amplitude (top) and phase (bottom)
`response of N(s) with B=1.2 and D=-7.2 usec, under an
`embodiment. Non-unity B affects the entire frequency range.
`FIG. 21 is a plot of amplitude (top) and phase (bottom)
`response of the effect on the speech cancellation in V, due
`to a mistake in the location of the speech source with q1=0
`degrees and q2=30 degrees, under an embodiment. The
`cancellation remains below -10 dB for frequencies below 6
`kHz.
`
`FIG. 22 is a plot of amplitude (top) and phase (bottom)
`response of the effect on the speech cancellation in V2 due
`to a mistake in the location of the speech source with q1=0
`degrees and q2=45 degrees, under an embodiment. The
`cancellation is below -10 dB only for frequencies below
`about 2.8 kHz and a reduction in performance is expected.
`FIG. 23 shows experimental results for a 2do=19 mm
`array using a linear B of 0.83 on a Bruel and Kjaer Head and
`Torso Simulator (HATS) in very loud (~85 dBA) music/
`speech noise environment, under an embodiment. The noise
`has been reduced by about 25 dB and the speech hardly
`affected, with no noticeable distortion.
`
`DETAILED DESCRIPTION
`
`A dual omnidirectional microphone array (DOMA) that
`provides improved noise suppression is described herein.
`Compared to conventional arrays and algorithms, which
`seek to reduce noise by nulling out noise sources, the array
`of an embodiment
`is used to form two distinct virtual
`directional microphones which are configured to have very
`similar noise
`responses
`and very dissimilar
`speech
`responses. The only null formed by the DOMAis one used
`to remove the speech of the user from V2*The twovirtual
`microphones of an embodiment can be paired with an
`adaptive filter algorithm and/or VAD algorithm to signifi-
`cantly reduce the noise without distorting the speech, sig-
`nificantly improving the SNR of the desired speech over
`conventional noise suppression systems. The embodiments
`described herein are stable in operation, flexible with respect
`to virtual microphonepattern choice, and have proven to be
`robust with respect to speech source-to-array distance and
`orientation as well as temperature and calibration tech-
`niques.
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`4
`In the following description, numerousspecific details are
`introduced to provide a thorough understanding of, and
`enabling description for, embodiments of the DOMA. One
`skilled in the relevant art, however, will recognize that these
`embodiments can be practiced without one or more of the
`specific details, or with other components, systems, etc. In
`other instances, well-known structures or operations are not
`shown, or are not described in detail, to avoid obscuring
`aspects of the disclosed embodiments.
`Unless otherwise specified, the following terms have the
`corresponding meanings in addition to any meaning or
`understanding they may convey to one skilled in theart.
`The term “bleedthrough” meansthe undesired presence of
`noise during speech.
`The term “denoising” means removing unwanted noise
`from Micl, and also refers to the amount of reduction of
`noise energy in a signal in decibels (dB).
`The term “devoicing” means removing/distorting the
`desired speech from Mic.
`The term “directional microphone (DM)” meansa physi-
`cal directional microphonethat is vented on both sides of the
`sensing diaphragm.
`The term “Mic1 (M1)” meansa general designation for an
`adaptive noise suppression system microphonethat usually
`contains more speech than noise.
`The term “Mic2 (M2)” meansa general designation for an
`adaptive noise suppression system microphonethat usually
`contains more noise than speech.
`The term “noise” means unwanted environmental acous-
`tic noise.
`
`The term “null” means a zero or minima in the spatial
`response of a physical or virtual directional microphone.
`The term “0,” means a first physical omnidirectional
`microphone used to form a microphonearray.
`The term “0,” means a second physical omnidirectional
`microphone used to form a microphonearray.
`The term “speech” means desired speech of the user.
`The term “Skin Surface Microphone (SSM)”is a micro-
`phone used in an earpiece (e.g.,
`the Jawbone earpiece
`available from Aliph of San Francisco, Calif.) to detect
`speech vibrations on the user’s skin.
`The term “V,” means the virtual directional “speech”
`microphone, which has no nulls.
`The term “V,” means the virtual directional “noise”
`microphone, which hasa null for the user’s speech.
`The term “Voice Activity Detection (VAD) signal” means
`a signal indicating when user speech is detected.
`The term “virtual microphones (VM)”or “virtual direc-
`tional microphones” means a microphone constructed using
`two or more omnidirectional microphones and associated
`signal processing.
`FIG. 1 is a two-microphone adaptive noise suppression
`system 100, under an embodiment. The two-microphone
`system 100 including the combination of physical micro-
`phones MIC 1 and MIC 2 along with the processing or
`circuitry components to which the microphones couple
`(described in detail below, but not shown in this figure) is
`referred to herein as the dual omnidirectional microphone
`array (DOMA) 110, but the embodimentis not so limited.
`Referring to FIG. 1, in analyzing the single noise source 101
`and the direct path to the microphones, the total acoustic
`information coming into MIC 1
`(102, which can be an
`physical or virtual microphone) is denoted by m,(n). The
`total acoustic information coming into MIC 2 (103, which
`can also be an physical or virtual microphone) is similarly
`labeled m2(n). In the z (digital frequency) domain, these are
`represented as M,(z) and M,(Z). Then,
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 23 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 23 of 38
`
`M,@)=S()+N(2)
`
`My(2)=N@)+S(2)
`
`with
`
`N2(Z)=N@)H|)
`
`So{Z)=S@)H2@)
`
`so that
`
`M,@)=S@)+N(@)H(2)
`
`M2(Z)-N@)+S(@)HZ)
`
`US 10,779,080 B2
`
`6
`for H,(z) are held constant (and vice versa) andit is assumed
`that the noise level is not high enoughto cause errors in the
`H,(z) calculation.
`they are used to
`After calculating H1(z) and H.(z),
`removethe noise from the signal. If Equation 1 is rewritten
`as
`
`S()=M, @)-N@)Ai)
`
`N@)-M{2)-S@)Ha(Z)
`
`S@)=M,@)-[Mo@)-S@)a2)@),
`
`S@)[1-H2@)1@)]-M@)-M,(@)H(z)
`
`Eq. 1
`
`then N(z) may be substituted as shown to solve for S(z) as
`
`This is the general case for all two microphone systems.
`Equation 1 has four unknownsand only two known rela-
`tionships and therefore cannot be solved explicitly.
`However, there is another way to solve for some of the
`unknownsin Equation 1. The analysis starts with an exami-
`nation of the case where the speech is not being generated,
`that is, where a signal from the VAD subsystem 104 (op-
`tional) equals zero. In this case, s(n)=S(z)=0, and Equation
`1 reduces to
`
`My@)-N@AME)
`
`Moy{z)=N(z)
`
`where the N subscript on the M variables indicate that only
`noise is being received.
`This leads to
`
`20
`
`25
`
`30
`
`My (@) = Moy @)M1 (2)
`
`_ Min)
`MQ) = Mon (2)
`
`Eq. 2
`
`35
`
`The function H,(z) can be calculated using any of the
`available system identification algorithms and the micro-
`phone outputs when the system is certain that only noise is
`being received. The calculation can be done adaptively, so
`that the system can react to changesin the noise.
`A solution is now available for H,(z), one of the
`unknownsin Equation 1. The final unknown, H,(z), can be
`determined by using the instances where speech is being
`produced and the VAD equals one. Whenthis is occurring,
`but the recent (perhaps less than 1 second) history of the
`microphonesindicate low levels of noise, it can be assumed
`that n(s)=N(z)~O. Then Equation 1 reduces to
`
`M,,(2)-S@)
`
`M2.(2)-S@)\H2@),
`which in turn leads to
`
`Mos(z) = Mis(2)Fa(@)
`
`Mos (2)
`Mis(Z)
`
`A(z) =
`
`whichis the inverse of the H,(z) calculation. However, it is
`noted that different inputs are being used (now only the
`speech is occurring whereas before only the noise was
`occurring). While calculating H,(z), the values calculated
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Mi(z) — Ma(2Ai(@)
`TT n@M@
`
`5) =
`
`If the transfer functions H,(z) and H,(z) can be described
`with sufficient accuracy, then the noise can be completely
`removed andthe original signal recovered. This remains true
`without respect to the amplitude or spectral characteristics of
`the noise. If there is very little or no leakage from the speech
`source into M,, then H,(z)=0 and Equation 3 reduces to
`
`S(z)&M(z)-M2(z)A(z).
`
`Eq. 4
`
`Equation 4 is much simpler to implement and is very
`stable, assuming H,(z) is stable. However,
`if significant
`speech energy is in M,(Z), devoicing can occur. In order to
`construct a well-performing system and use Equation 4,
`consideration is given to the following conditions:
`R1. Availability of a perfect (or at least very good) VAD
`in noisy conditions
`R2. Sufficiently accurate H,(z)
`R3. Very small (ideally zero) H,(z)
`R4. During speech production, H,(z) cannot change sub-
`stantially.
`R5. During noise, H(z) cannot change substantially.
`Condition R1 is easy to satisfy if the SNR of the desired
`speech to the unwanted noise is high enough. “Enough”
`meansdifferent things depending on the method of VAD
`generation. If a VAD vibration sensor is used, as in Burnett
`USS. Pat. No. 7,256,048, accurate VAD in very low SNRs
`(-10 dB orless) is possible. Acoustic-only methods using
`information from O, and O, can also return accurate VADs,
`but are limited to SNRs of ~3 dB or greater for adequate
`performance.
`Condition R5 is normally simple to satisfy because for
`most applications the microphones will not change position
`with respect to the user’s mouth very often or rapidly. In
`those applications where it may happen (such as hands-free
`conferencing systems) it can be satisfied by configuring
`Mic2 so that H,(z)=0.
`Satisfying conditions R2, R3, and R4 are more difficult
`but are possible given the right combination of V, and
`V,*Methods are examined below that have proven to be
`effective in satisfying the above, resulting in excellent noise
`suppression performance and minimal speech removal and
`distortion in an embodiment.
`The DOMA,in various embodiments, can be used with
`the Pathfinder system as the adaptive filter system or noise
`removal. The Pathfinder system, available from AliphCom,
`San Francisco, Calif, is described in detail in other patents
`and patent applications referenced herein. Alternatively, any
`
`

`

`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 24 of 38
`Case 6:21-cv-00984-ADA Document 55-8 Filed 05/25/22 Page 24 of 38
`
`US 10,779,080 B2
`
`7
`adaptive filter or noise removal algorithm can be used with
`the DOMA in one or more various alternative embodiments
`
`8
`a wide variety of virtual microphones (VMs), also referred
`to herein as virtual directional microphones, can berealized.
`While the processing component 402 described in this
`or configurations.
`example includes four processing paths generating two
`When the DOMA is used with the Pathfinder system, the
`virtual microphones or microphonesignals, the embodiment
`Pathfinder system generally provides adaptive noise cancel-
`is not so limited. For example, FIG. 5 is a block diagram for
`lation by combining the two microphonesignals (e.g., Micl,
`Mic2) by filtering and summing in the time domain. The
`a DOMA 500 including two physical microphones config-
`
`adaptive filter generally uses the signal received fromafirst ured to form N virtual microphones V, through V,,, where
`microphone of the DOMA to removenoise from the speech
`N is any number greater than one, under an embodiment.
`received from at least one other microphone of the DOMA,
`Thus, the DOMA can include a processing component 502
`which relies on a slowly varying linear transfer function
`having any numberof processing paths as appropriate to
`
`between the two microphones for sources of noise. Follow- form a numberNof virtual microphones.
`ing processing of the two channels of the DOMA,an output
`The DOMAof an embodiment can be coupled or con-
`signal is generated in which the noise content is attenuated
`nected to one or more remote devices. In a system configu-
`with respect to the speech content, as described in detail
`ration, the DOMAoutputs signals to the remote devices. The
`below.
`remote devices include, but are not limitedto, at least one of
`cellular telephones, satellite telephones, portable telephones,
`wireline telephones, Internet telephones, wireless transceiv-
`ers, wireless communication radios, personal digital assis-
`tants (PDAs), personal computers (PCs), headset devices,
`head-worn devices, and earpieces.
`Furthermore,
`the DOMA of an embodiment can be a
`component or subsystem integrated with a host device. In
`this system configuration,
`the DOMA outputs signals to
`components or subsystems of the host device. The host
`device includes, but is not limited to, at least one of cellular
`telephones, satellite telephones, portable telephones, wire-
`line telephones, Internet telephones, wireless transceivers,
`wireless communication radios, personal digital assistants
`(PDAs), personal computers (PCs), headset devices, head-
`worn devices, and earpieces.
`As an example, FIG. 6 is an example of a headset or
`head-worn device 600 that
`includes
`the DOMA,
`as
`described herein, under an embodiment. The headset 600 of
`an embodiment includes a housing having two areas or
`receptacles (not shown) that receive and hold two micro-
`phones (e.g., O, and O,), The headset 600 is generally a
`device that can be worn by a speaker 602, for example, a
`headset or earpiece that positions or holds the microphones
`in the vicinity of the speaker’s mouth. The headset 600 of an
`embodimentplaces a first physical microphone(e.g., physi-
`cal microphoneO, ) ina vicinity of a speaker’s lips. A second
`physical microphone (e.g., physical microphone O,) is
`placed a distance behindthe first physical microphone. The
`distance of an embodimentis in a range of a few centimeters
`behind the first physical microphone or as described herein
`(e.g., described with reference to FIGS. 1-5). The DOMAis
`symmetric and is used in the same configuration or manner
`output from each microphone is coupled to a processing
`as a single close-talk microphone, but is not so limited.
`component 402, or circuitry, and the processing component
`FIG. 7 is a flow diagram for denoising 700 acoustic
`outputs signals representing or corresponding to the virtual
`signals using the DOMA, under an embodiment. The
`microphones V, and V3.
`denoising 700 begins by receiving 702 acoustic signals at a
`In this example system 400, the output of physical micro-
`first physical microphone and a second physical micro-
`phone 201 is coupled to processing component 402 that
`includesa first processing path that includes application of
`phone.In responseto the acoustic signals, a first microphone
`55
`
`a first delay Z,, andafirst gain A,, and a second processing signal is output from the first physical microphone and a
`path that includes application of a second delay Z,, and a
`second microphonesignal is output from the second physi-
`second gain A,,*The output of physical microphone 202 is
`cal microphone 704. A first virtual microphone is formed
`coupled to a third processing path of the processing com-
`706 by gen