Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 1 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 1 of 38
`
`
`
`
`
`
`
`
`EXHIBIT 9
`EXHIBIT 9
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 2 of 38
`Case 6:21-cv-00984-ADA pocumeatriifl |wr
`Mr
`
`
`
`US011122357B2
`
`a2) United States Patent
`US 11,122,357 B2
`(0) Patent No.:
`*Sep. 14, 2021
`(45) Date of Patent:
`Burnett
`
`(54)
`
`FORMING VIRTUAL MICROPHONE
`ARRAYS USING DUAL OMNIDIRECTIONAL
`
`MICROPHONE ARRAY (DOMA)
`
`(71)
`
`(72)
`
`Applicant: Jawbone Innovations, LLC, Marshall,
`TX (US)
`
`Inventor: Gregory C. Burnett, Dodge Center,
`MN (US)
`
`(73)
`
`Assignee:
`
`Jawbone Innovations, LLC, Marshall,
`TX (US)
`
`(*)
`
`Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`This patent is subject to a terminal dis-
`claimer.
`
`(21)
`
`Appl. No.: 13/959,708
`
`(22)
`
`Filed:
`
`Aug. 5, 2013
`
`(65)
`
`(63)
`
`Prior Publication Data
`
`US 2014/0185825 Al
`
`Jul. 3, 2014
`
`Related U.S. Application Data
`
`Continuation of application No. 12/139,333, filed on
`Jun. 13, 2008, now Pat. No. 8,503,691.
`(Continued)
`
`(51)
`
`Int. Cl.
`
`HOAR 3/00
`HOAR 1/10
`
`(52)
`
`USS. Cl.
`
`(2006.01)
`(2006.01)
`(Continued)
`
`CPC ........ HOAR 1/1091 (2013.01); GIOL 21/0208
`(2013.01); HO4R 1/406 (2013.01);
`(Continued)
`
`(58) Field of Classification Search
`CPC .... HO4R 3/005; H0O4R 2410/05; HO4R 3/002;
`HO4R 3/04; HO4R 1/1091; HO4R 1/406;
`(Continued)
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`4,653,102 A
`4,777,649 A
`
`3/1987 Hansen
`10/1988 Carlson et al.
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`WO
`
`2009003 180
`
`12/2008
`
`OTHER PUBLICATIONS
`
`Weiss, Howard; Office Action mailed by U.S. Patent and Trademark
`Office dated Apr. 10, 2012 for U.S. Appl. No. 12/139,333.
`(Continued)
`
`Primary Examiner — Marcos D. Pizarro
`(74) Attorney, Agent, or Firm — Nutter McClennen &
`Fish LLP
`
`ABSTRACT
`(57)
`A dual omnidirectional microphone array noise suppression
`is described. Compared to conventional arrays and algo-
`rithms, which seek to reduce noise by nulling out noise
`sources, the array of an embodiment is used to form two
`distinct virtual directional microphones which are config-
`ured to have very similar noise responses and very dissimilar
`speech responses. The only null formed is one used to
`remove the speech of the user from V,. The two virtual
`microphonesmaybe paired with an adaptive filter algorithm
`and VADalgorithm to significantly reduce the noise without
`distorting the speech, significantly improving the SNR ofthe
`desired speech over conventional noise suppression systems.
`
`20 Claims, 17 Drawing Sheets
`
`
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 3 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 3 of 38
`
`US 11,122,357 B2
`
`Page 2
`
`Related U.S. Application Data
`
`(60) Provisional application No. 60/934,551, filed on Jun.
`13, 2007, provisional application No. 60/953,444,
`filed on Aug. 1, 2007, provisional application No.
`60/954,712,
`filed on Aug. 8, 2007, provisional
`application No. 61/045,377, filed on Apr. 16, 2008.
`
`2003/0044025 Al
`2003/0228023 Al
`2007/0003082 Al
`2007/0121974 Al
`2009/0010450 Al
`
`3/2003 Ouyang et al.
`12/2003 Burnett etal.
`1/2007 Pedersen
`5/2007 Nemirovski
`1/2009 Burnett
`
`OTHER PUBLICATIONS
`
`(51)
`
`(2013.01)
`(2006.01)
`(2006.01)
`(2013.01)
`
`Int. Cl.
`GOL 21/0208
`HOAR 1/40
`HOAR 3/04
`GIOL 21/0216
`(52) U.S. Cl
`CPC veeceseseeee HOAR 3/002 (2013.01); HO4R 3/005
`(2013.01); HO4R 3/04 (2013.01); GIOL
`2021/02165 (2013.01); HO4R 2460/01
`(2013.01)
`
`(58) Field of Classification Search
`CPC ...... HO4R 2460/01; G1OL 2021/02165; G10L
`21/0208
`USPC wiceceeeeeeees 381/92, 94.7; 704/233, E21.004
`See application file for complete search history.
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`5/1993 Kaneda
`5,208,864 A
`1/1994 Freeman et al.
`5,276,765 A
`10/1994 Oh et al.
`5,353,376 A
`4/1995 Silverberg et al.
`5,406,622 A
`10/1995 Bradelyet al.
`5,463,694 A
`12/1995 Cezanneet al.
`5,473,701 A
`12/1995 Yoshidaet al.
`5,473,702 A
`5/1996 Sugiyama
`5,517,435 A
`12/1996 Park etal.
`5,590,241 A
`4/1997 Matouketal.
`5,625,684 A
`5/1997 Kanamori etal.
`5,633,935 A
`9/1997 Yamaguchiet al.
`5,664,014 A
`9/1997 Nishiguchiet al.
`5,664,052 A
`3/1998 Holzrichter et al.
`5,729,694 A
`5/1998 Hosoi
`5,754,665 A
`9/1998 Claybaughet al.
`5,815,582 A
`10/1998 Andreaet al.
`5,825,897 A
`5/1999 Takada
`5,907,624 A
`12/1999 Holzrichter
`6,006,175 A
`5/2001 Choet al.
`6,233,551 Bl
`3/2004 Valveetal.
`6,707,910 Bl
`9/2004 Housni
`6,795,713 B2
`11/2005 Vaudreyet al.
`6,963,649 B2
`12/2005 Turnbull et al.
`6,980,092 B2
`4/2007 Yang et al.
`7,206,418 B2
`6/2008 Fan
`7,386,135 B2
`8,068,619 B2* 11/2011 Zhang etal. wo. 381/92
`
`.. 381/92
`8,494,177 B2*
`7/2013 Burnett
`....
`8/2013 Burnett oe 381/92
`8,503,691 B2*
`8/2015 Burnett
`9,099,094 B2
`8/2002 Watsonet al.
`2002/0110256 Al
`8/2002 Erte
`2002/0116187 Al
`
`Weiss, Howard; Office Action mailed by U.S. Patent and Trademark
`Office dated Jul. 14, 2011 for U.S. Appl. No. 12/139,333.
`Copenheaver, Blaine R; International Searching Authority; Notifi-
`cation of Transmittal of the International Search Report and Written
`Opinion of the International Searching Authority of the Declaration
`for International Patent Application No. PCT/US2008/068634, dated
`Sep. 2, 2008.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Aug. 10, 2011 for U.S. Appl. No. 12/163,592.
`Shah, Paras D.; Office Action mailed by U.S. Patent and Trademark
`Office dated Nov. 16, 2011 for U.S. Appl. No. 11/805,987.
`Shah, Paras D.; Office Action mailed by U.S. Patent and Trademark
`Office dated Jan. 16, 2009 for U.S. Appl. No. 11/805,987.
`Azad, Abul K..; Office Action mailed by U.S. Patent and Trademark
`Office dated Feb. 6, 2008 for U.S. Appl. No. 11/805,987.
`Shah, Paras D.; Office Action mailed by U.S. Patent and Trademark
`Office dated Oct. 10, 2006 for U.S. Appl. No. 10/159,770.
`Shah, Paras D.; Office Action mailed by U.S. Patent and Trademark
`Office dated Dec. 15, 2005 for U.S. Appl. No. 10/159,770.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Jul. 31, 2013 for U.S. Appl. No. 13/436,765.
`Lao, Lun S..; Office Action mailed by U.S. Patent and Trademark
`Office dated Aug. 30, 2010 for U.S. Appl. No. 10/667,207.
`Lao, Lun S..; Office Action mailed by U.S. Patent and Trademark
`Office dated Dec. 24, 2009 for U.S. Appl. No. 10/667,207.
`Lao, Lun S..; Office Action mailed by U.S. Patent and Trademark
`Office dated Mar. 11, 2009 for U.S. Appl. No. 10/667,207.
`Lao, Lun S..; Office Action mailed by U.S. Patent and Trademark
`Office dated Jul. 9, 2008 for U.S. Appl. No. 10/667,207.
`Lao, Lun S..; Office Action mailed by U.S. Patent and Trademark
`Office dated Feb. 9, 2007 for U.S. Appl. No. 10/667,207.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Mar. 3, 2016 for U.S. Appl. No. 14/224,868.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Aug. 7, 2015 for U.S. Appl. No. 14/224,868.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Dec. 21, 2014 for U.S. Appl. No. 14/224,868.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Mar. 3, 2016 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Jun. 23, 2011 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Aug. 17, 2010 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Dec. 9, 2009 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Mar. 16, 2009 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Aug. 18, 2008 for U.S. Appl. No. 10/400,282.
`
`* cited by examiner
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 4 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 4 of 38
`
`US 11,122,357 B2
`
`Sep. 14, 2021
`
`Sheet 1 of 17
`
`powesy9
`
`U.S. Patent “ypoads
`001|ON001
`OmSyaie™
`/u(u)s
`(u}sb
`
`(Gal)
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 5 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 5 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 2 of 17
`
`US 11,122,357 B2
`
`y
`
`SK S700
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 6 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 6 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 3 of 17
`
`US 11,122,357 B2
`
`
`
`
`
`Ded> pete
`
`i
`
`ped
`
`et
`
`tt ae
`
`Any
`
`Ay
`
`Ay
`
`Ay
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 7 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 7 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 4 of 17
`
`US 11,122,357 B2
`
`
`
`
`
`
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 8 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 8 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 5 of 17
`
`US 11,122,357 B2
`
`.
`
`\on
`
`ne
`
`FIG.6
`
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 9 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 9 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 6 of 17
`
`US 11,122,357 B2
`
` Receive acoustic signals at a first physical
`Q
`.
`~
`microphone and a second physical microphone.
`
`
`
`
`pe
`Ls)
`:
`“
`
`
`
`
`
`Output first microphone signal from first physical
`microphone and second microphone signal from aTOG
`second physical microphone.
`
`
`
`
`
`Formfirst virtual microphone usingthe first combination | 706
`offirst microphone signal and second microphone signal,
`
`
`
`
`
`Generate denoised output signals having less
`acoustic noise thanreceived acoustic signals.
`
`Form second virtual microphone using second combination |
`of first microphone signal and second microphone signal.
`|
`
`Form physical microphone array including first
`physical microphone and second physical microphone.
`
`signals from physical microphone array.
`
`Porm virtual microphone array including first virtual
`microphone and second virtual microphone using
`
`300"
`
`FIGS
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 10 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 10 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 7 of 17
`
`Linear response of V2 fo a speech source at 0.10 meters 240 80
`
`US 11,122,357 B2
`
`270
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 11 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 11 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 8 of 17
`
`US 11,122,357 B2
`
`Linear response of V1 to a speech source at 0.10 meters
`80.
`
`0.8
`
`iQ
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 12 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 12 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 9 of 17
`
`US 11,122,357 B2
`
`Linear response af V1 to a speech source af 0.1 meters
`
`90
`
`2
`
`190 ooBO 1<—-- SPEECH}-
`
`i
`;
`
`i
`
`\
`
`|
`/330
`
`é
`
`.fo
`\,
`24 00
`a
`
`270
`
`FIG.13
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 13 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 13 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 10 of 17
`
`US 11,122,357 B2
`
`Frequency response at 0 degrees
`|
`
`10,
`
`
`Bee Rn ey RR RN ET ee gOey A EE CA TRMRA RA AEAAe g
`
`Cardioid speech:
`
`6 Bn ene ens ea gtins enede een ene veda ona conanpanna enn ech ann tina widaw man
`
`tganee
`
`Ravenna maaves
`
`pan,
`
`S a
`
`e)
`5.@
`
`
`
`get | cncecentenanoqcoeiececeonoeroorererorooroonroioraresanorenonets ccencoteecosneororsecossnoresesssnnorstecoo:tieseseteeiesineseestfeoeeseeoe otbnasoed
`
`
`
`.
`
`response
`
`|
`AB)
`
`
`OF
`F900
`B00
`3000
`000
`BOG
`ONT
`FOG
`So00
`Frequency (Hz)
`
`2
`
`FIG.14
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 14 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 14 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 11 of 17
`
`US 11,122,357 B2
`
`age vs, B assuming d, =(Lim
`
`TE A 00 ate ae
`
`base AE I my 28 ay a
`
`nee
`
`»
`
`rote BEd De ed one oe
`
`9 eS anh ae ~
`:
`:
`Owe rea so |
`}
`3
`:
`on,
`:
`we en eeA RA HE WO Ahie a a mh et
`
`V1 (top, dashed) and V2 speechrespe wo Po a oo AS oe,
`V1/V2 for speech versus B assuming d, = 0.1m SmtCB
`
`a)eH
`0
`04
`
`3
`05
`
`:
`06
`
`3
`O7
`
`: lp
`08
`09
`1
`41
`B
`FIG.I5
`
`04
`
`05
`
`06
`
`O07
`
`08
`
`08
`
`1
`
`4
`
`
`
`Response(dB)
`
`
`
`V1/V2forspeech(dB)
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 15 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 15 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 12 of 17
`
`US 11,122,357 B2
`
`weeebee e ceeecee eeebce eee2 acee cee De ceceeeeRec cereceeeee creeReece creer cee be oe eeeeeeee earn
`i
`‘
`ny
`‘
`,
`‘
`
`$
`
`:
`
`;
`
`3
`
`‘
`:
`‘
`i
`i
`:
`i
`
`:
`‘
`:
`:
`:
`:
`:
`
`‘
`t
`‘
`b
`t
`i
`
`:
`:
`:
`:
`£
`:
`:
`:
`
`)
`t
`
`:
`
`3
`i
`.
`$
`i
`:
`i
`3
`
`3
`s
`Q
`i
`>
`
`:
`:
`‘
`:
`:
`
`'
`t
`t
`6
`z
`»
`r
`
`
`
`03
`035
`04
`048
`05
`005
`01
`O18
`02
`025
`Actnal d, (meters)
`FIG.17
`
`t
`:
`t
`>
`t
`t
`t
`*
`;
`:
`t
`‘
`‘
`t
`2
`t
`y
`‘
`:
`:
`‘
`b
`:
`a
`1
`:
`>
`t
`es
`>
`1
`‘
`Oe,
`‘
`t
`ey
`q
`‘
`i
`z
`sy
`t
`c
`4 4 bed we we wt en eee weeee ba eedee ee ete webe wen wegbe ee
`>
`t
`.
`{
`&
`e
`t
`4
`t
`‘
`t
`.
`:
`:
`#
`.
`‘
`:
`!
`i
`!
`r
`:
`i
`‘
`i
`i
`c
`i
`5
`i
`:
`c
`:
`‘
`:
`?
`3
`c
`»
`
`.
`
`3
`
`:
`
`3
`
`«
`
`B factor vs, actual d, assuming d, = 0.1m and theta =0
`B versus theta assuming d, = Q.1m
`
`¥
`
`‘,
`
`‘
`
`..
`
`%
`
`>
`i
`i
`:
`t
`*
`
`i
`3
`i
`:
`:
`i
`
`on
`
`%
`&
`ay
`2
`g
`
`‘
`:
`?
`,
`i
`
`»
`‘
`%
`§
`?
`
`i
`:
`:
`:
`+
`‘
`
`i
`Q
`3
`q
`
`#
`
`4
`e
`8
`2
`g
`
`;
`?
`i
`:
`i
`ij
`
`3
`
`é
`
`3
`
`¢
`
`$
`
`in
`t
`t
`i
`+
`t
`t
`t
`t
`t
`t
`t
`é
`c
`a ene ee
`t
`&
`e
`t
`&
`5
`c
`t
`c
`»
`
`
`6 «0 2 0 m0
`4
`6
`theta (degrees)
`FIG.18
`
`€
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 16 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 16 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 13 of 17
`
`US 11,122,357 B2
`
`
`2000
`3000
`4000
`5000
`8000
`7000
`8000
`
`. {000
`
`
`
`Amplitude(dB)
`
`
`
`Phase(degrees)
`
`N(s) for B= | and D =-7.2e-006 seconds
`
`
`6
`
`1900
`
`«2000
`
`«300040005000
`
`8060
`
`7000
`
`8000
`
`Frequency (Hz)
`
`FIG.19
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 17 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 17 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 14 of 17
`
`US 11,122,357 B2
`
`
`
`Amplitude(dB)
`
`
`
`Phase(degrees)
`
`N(s) for B = 1.2 and D = -7 2e-006 seconds
`
`
`
`
`
`
`0 T0060 SGT
`AGT
`BRET
`ROGT HG B00
`
`
`
`6
`1660
`3000
`5000
`R00
`BOG
`Frequency (Hz)
`
`
`7860
`B00
`
`BGT
`
`FIG.20
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 18 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 18 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 15 of 17
`
`US 11,122,357 B2
`
`0 Cancellation with dl =1, thetal = 0, d2 =1, and theta2 = 30
`
`c a
`3
`:
`:
`:
`:
`10
`
`
`
`-30)-
`:
`|
`:
`
`
`405——Tagg
`B00
`3000-4000
`GOGH
`GOUT
`7060
`Bo00
`
`pt
`
`S=
`
`= o
`
`u
`
`EB
`
`90
`p=
`~ BG banenenacen rnnbeees omemtn gua eatinenna tapenesenaadina ranenan y
`2
`ROL...
`253
`S18)
`ses
`
`
`a 85)
`805
`1000
`3000
`3000
`4000
`8000
`6000
`7000
`8000
`Frequency (Hz)
`
`FIG.21
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 19 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 19 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 16 of 17
`
`US 11,122,357 B2
`
`
`
`Amplitude(dB)
`
`
`
`Phase(degrees)
`
`Cancellation with di = 1, thetal = 0, d2= l, andthetaZ = 45
`
`
`
`
`{000
`2000
`30004000
`5000
`000
`7006
`8000
`
`"
`
`
`Oo
`{000
`
`2600
`
`
`5000
`4000
`S000
`BOND
`7000
`B00
`Frequency (Hz)
`
`FIG,22
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 20 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 20 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 17 of 17
`
`US 11,122,357 B2
`
`Original V1 (top) and cleaned V1 (bottom) with simplified VAD (dashed)in noise
`i4343434a
`¥&%‘4‘4a
`
`Pe11wo9M
`
`x 10°
`
`¥
`
`x10
`
`Time (samples at 8 KH
`nn,288
`)2294
`
`FIG.23
`
`YELLE
`
`¥i4343434a
`
`‘3&3k&$$&h
`
`pomeayg
`
`
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 21 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 21 of 38
`
`US 11,122,357 B2
`
`1
`FORMING VIRTUAL MICROPHONE
`ARRAYS USING DUAL OMNIDIRECTIONAL
`
`MICROPHONE ARRAY (DOMA)
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application is a continuation of U.S. Nonprovisional
`patent application Ser. No. 12/139,333, filed Jun. 13, 2008,
`now U.S. Pat. No. 8,503,691, entitled “Forming Virtual
`Microphone Arrays Using Dual Omnidirectional Micro-
`phone Array (DOMA),” which claims the benefit of U-S.
`Provisional Patent Application No. 60/934,551, filed Jun.
`13, 2007, U.S. Provisional Patent Application No. 60/953,
`444,filed Aug. 1, 2007, U.S. Provisional Patent Application
`No. 60/954,712, filed Aug. 8, 2007, and U.S. Provisional
`Patent Application No. 61/045,377, filed Apr. 16, 2008, all
`of which are incorporated by reference herein in their
`entirety for all purposes.
`
`TECHNICAL FIELD
`
`The disclosure herein relates generally to noise suppres-
`sion. In particular, this disclosure relates to noise suppres-
`sion systems, devices, and methods for use in acoustic
`applications.
`
`BACKGROUND
`
`Conventional adaptive noise suppression algorithms have
`been around for some time. These conventional algorithms
`have used two or more microphones to sample both an
`(unwanted) acoustic noise field and the (desired) speech of
`a user. 20 The noise relationship between the microphones
`is then determined using an adaptive filter (such as Least-
`Mean-Squares
`as
`described in Haykin & Widrow,
`ISBN#0471215708, Wiley, 2002, but any adaptive or sta-
`tionary system identification algorithm may be used) and
`that relationship used to filter the noise from the desired
`signal.
`Most conventional noise suppression systems currently in
`use for speech communication systems are based on a
`single-microphone
`spectral
`subtraction technique
`first
`develop in the 1970’s and described, for example, by S. F.
`Boll in “Suppression of Acoustic Noise in Speech using
`Spectral Subtraction,” IEEE Trans. on ASSP, pp. 113-120,
`1979. These techniques have been refined over the years, 30
`but the basic principles of operation have remained the
`same. See,
`for example, U.S. Pat. No. 5,687,243 of
`McLaughlin,et al., and U.S. Pat. No. 4,811,404 of Vilmur,
`et al. There have also been several attempts at multi-
`microphone noise suppression systems, such as those out-
`lined in U.S. Pat. No. 5,406,622 of Silverberg et al. and U.S.
`Pat. No. 5,463,694 of Bradley et al. Multi-microphone
`systems have not been very successful for a variety of
`reasons, the most compelling being poor noise cancellation
`performanceand/or significant speech distortion. Primarily,
`conventional multi-microphone systems attempt to increase
`the SNRof the user’s speech by “steering” the nulls of the
`system to the strongest noise sources. This approach is
`limited in the number of noise sources removed by the
`number of available nulls.
`The Jawbone earpiece (referred to as the “Jawbone),
`introduced in December 2006 by AliphCom of San Fran-
`cisco, Calif., was the first known commercial 10 product to
`use a pair of physical directional microphones (instead of
`omnidirectional microphones)
`to reduce environmental
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`acoustic noise. The technology supporting the Jawbone is
`currently described under one or more of U.S. Pat. No.
`7,246,058 by Burnett and/or U.S. patent application Ser.
`Nos. 10/400,282, 10/667,207, and/or 10/769,302. Generally,
`multi-microphone techniques makeuseof an acoustic-based
`Voice Activity Detector (VAD) to determine the background
`noise characteristics, where “voice” is generally understood
`to include human voiced speech, unvoiced speech, or a
`combination of voiced and unvoiced speech. The Jawbone
`improved on this by using a microphone-based sensor to
`construct a VAD signal using directly detected speech vibra-
`tions in the user’s cheek. This allowed the Jawbone to
`
`aggressively remove noise when the user was not producing
`speech. However, the Jawbone uses a directional micro-
`phonearray.
`
`INCORPORATION BY REFERENCE
`
`Each patent, patent application, and/or publication men-
`tioned in this specification is herein incorporated by refer-
`ence in its entirety to the same extent as if each individual
`patent, patent application, and/or publication was specifi-
`cally and individually indicated to be incorporated by ref-
`erence.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a two-microphone adaptive noise suppression
`system, under an embodiment.
`FIG. 2 is an array and speech source (S) configuration,
`under an embodiment. The microphones are separated by a
`distance approximately equal to 2d,, and the speech source
`is located a distance d, away from the midpoint of the array
`at an angle 0. The system is axially symmetric so only d, and
`8 need be specified.
`FIG. 3 is a block diagram for a first order gradient
`microphone using two omnidirectional elements O, and O,,
`under an embodiment.
`FIG. 4 is a block diagram for a DOMAincluding two
`physical microphones configured to form two virtual micro-
`phones V, and V.,, under an embodiment.
`FIG. 5 is a block diagram for a DOMAincluding two
`physical microphones configured to form N virtual micro-
`phones V, through V,,, where N is any number greater than
`one, under an embodiment.
`FIG. 6 is an example of a headset or head-worn device
`that includes the DOMA, as described herein, under an
`embodiment.
`
`FIG. 7 is a flow diagram for denoising acoustic signals
`using the DOMA, under an embodiment.
`FIG. 8 is a flow diagram for forming the DOMA, under
`an embodiment.
`FIG.9 is a plot of linear response of virtual microphone
`V, to a 1 kHz speech sourceat a distance of 0.1 m, under an
`embodiment. The null is at 0 degrees, where the speech is
`normally located.
`FIG. 10 is a plot of linear response of virtual microphone
`V, toa 1 kHz noise source at a distance of 1.0 m, under an
`embodiment. There is no null and all noise sources are
`detected.
`
`FIG.11 is a plot of linear response of virtual microphone
`V, toa 1 kHz speech source at a distance of 0.1 m, under an
`embodiment. There is no null and the response for speech is
`greater than that shown in FIG. 9.
`FIG.12 is a plot of linear response of virtual microphone
`V, toa 1 kHz noise source at a distance of 1.0 m, under an
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 22 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 22 of 38
`
`US 11,122,357 B2
`
`3
`4
`In accordance with another embodiment, a microphone
`embodiment. There is no null and the response is very
`array is formed withafirst virtual microphone formed from
`similar to V, shown in FIG. 10.
`a first combination of a first microphonesignal and a second
`FIG.13 is a plot of linear response of virtual microphone
`microphone signal, wherein the first microphone signal is
`V, to a speech source at a distance of 0.1 m for frequencies
`generated by a first omnidirectional microphone and the
`of 100, 500, 1000, 2000, 3000, and 4000 Hz, under an
`embodiment.
`second microphonesignal is generated by a second omni-
`directional microphone; and a second virtual microphone
`formed from a second combination of the first microphone
`signal and the second microphonesignal, wherein the sec-
`ond combination is different from thefirst combination. The
`
`FIG. 14 is a plot showing comparison of frequency
`responses for speech for the array of an embodiment andfor
`a conventional cardioid microphone.
`FIG. 15 is a plot showing speech response for V, (top,
`dashed) and V,, (bottom, solid) versus B with d, assumed to
`be 0.1 m, under an embodiment. The spatial null in V, is
`relatively broad.
`FIG. 16 is a plot showing a ratio of V,/V, speech
`responses shown in FIG. 10 versus B, under an embodiment.
`Theratio is above 10 dB for all 0.8<B<1.1. This meansthat
`
`first virtual microphonehasa first linear response to speech
`that has a single null oriented in a direction toward a source
`of the speech, wherein the speech is human speech.
`In accordance with another embodiment,
`a device
`includes a first microphone outputting a first microphone
`signal and a second microphoneoutputting a second micro-
`phone signal; and a processing component coupled to the
`first microphone signal and the second microphonesignal,
`the processing component generating a virtual microphone
`array comprising a first virtual microphone and a second
`virtual microphone, wherein the first virtual microphone
`comprises a first combination ofthe first microphonesignal
`and the second microphonesignal, and wherein the second
`virtual microphone comprises a second combination of the
`first microphone signal and the second microphonesignal.
`The second virtual microphone have substantially similar
`responses to noise and substantially dissimilar responses to
`speech.
`a devise
`In accordance with another embodiment,
`includes a first microphone outputting a first microphone
`signal and a second microphoneoutputting a second micro-
`phone signal, wherein the first microphone and the second
`microphone are omnidirectional microphones; and a virtual
`microphonearray comprising a first virtual microphone and
`a secondvirtual microphone, wherein thefirst virtual micro-
`phone comprises a first combination ofthe first microphone
`signal and the second microphonesignal, and the second
`virtual microphone comprises a second combination of the
`first microphone signal and the second microphonesignal.
`The second combination is different from the first combi-
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`60
`
`the physical 6 of the system need not be exactly modeled for
`good performance.
`FIG.17 is a plot of B versus actual d, assuming that d,=10
`cm and theta=0, under an embodiment.
`FIG. 18 is a plot of B versus theta with d=10 cm and
`assuming d=10 cm, under an embodiment.
`FIG. 19 is a plot of amplitude (top) and phase (bottom)
`response of N(s) with B=1 and D=-7.2 usec, under an
`embodiment. The resulting phase difference clearly affects
`high frequencies more than low.
`FIG. 20 is a plot of amplitude (top) and phase (bottom)
`response of N(s) with B=1.2 and D=-7.2 usec, under an
`embodiment. Non-unity B affects the entire frequency range.
`FIG. 21 is a plot of amplitude (top) and phase (bottom)
`response of the effect on the speech cancellation in V, due
`to a mistake in the location of the speech source with q1=0
`degrees and q2=30 degrees, under an embodiment. The
`cancellation remains below -10 dB for frequencies below 6
`kHz.
`
`FIG. 22 is a plot of amplitude (top) and phase (bottom)
`response of the effect on the speech cancellation in V, due
`to a mistake in the location of the speech source with q1=0
`degrees and q2=45 degrees, under an embodiment. The
`cancellation is below -10 dB only for frequencies below
`about 2.8 kHz and a reduction in performance is expected.
`FIG. 23 shows experimental results for a 2d,=19 mm
`array using a linear B of 0.83 on a Bruel and Kjaer Head and
`Torso Simulator (HATS) in very loud (~85 dBA) music/
`speech noise environment, under an embodiment. The noise
`has been reduced by about 25 dB and the speech hardly
`affected, with no noticeable distortion.
`
`SUMMARY OF THE INVENTION
`
`microphone and the second virtual microphone are distinct
`virtual directional microphones with substantially similar
`responses to noise and substantially dissimilar responses to
`speech.
`
`nation, and the first virtual microphone and the second
`virtual microphone are distinct virtual directional micro-
`phones.
`a device
`In accordance with another embodiment,
`includesa first physical microphone generating a first micro-
`phone signal; a second physical microphone generating a
`second microphone signal; and a processing component
`coupled to the first microphone signal and the second
`microphonesignal, the processing component generating a
`virtual microphone array comprising a first virtual micro-
`phone and a second virtual microphone. The first virtual
`microphone comprises the second microphone signal sub-
`The present invention provides for dual omnidirectional
`tracted from a delayed version ofthefirst microphonesignal,
`microphone array devices systems and methods.
`In accordance with on embodiment, a microphonearray is
`and the second virtual microphone comprises a delayed
`55
`
`formed with a first virtual microphone that includesafirst version of the first microphone signal subtracted from the
`combination of a first microphone signal and a second
`second microphonesignal.
`sensor
`a
`In accordance with another embodiment,
`microphone signal, wherein the first microphone signal is
`generated by a first physical microphone and the second
`includes a physical microphonearray includinga first physi-
`microphonesignal is generated by a second physical micro-
`cal microphone and a second physical microphone,thefirst
`phone; and a second virtual microphone that includes a
`physical microphone outputting a first microphone signal
`second combination of the first microphone signal and the
`and the second physical microphone outputting a second
`second microphonesignal, wherein the second combination
`microphonesignal; and a virtual microphonearray compris-
`is different from the first combination. The first virtual
`ing a first virtual microphone and a second virtual micro-
`phone,the first curtail microphone comprising a first com-
`bination of the first microphone signal and the second
`microphonesignal, the second virtual microphone compris-
`ing a second combination ofthe first microphone signal and
`
`

`

`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 23 of 38
`Case 6:21-cv-00984-ADA Document 55-9 Filed 05/25/22 Page 23 of 38
`
`US 11,122,357 B2
`
`5
`the second microphonesignal. The second combination is
`different from the first combination, and the virtual micro-
`phone array includes a single null oriented in a direction
`toward a source of speech of a human speaker.
`
`DETAILED DESCRIPTION
`
`A dual omnidirectional microphone array (DOMA) that
`provides improved noise suppression is described herein.
`Compared to conventional arrays and algorithms, which
`seek to reduce noise by nulling out noise sources, the array
`of an embodiment
`is used to form two distinct virtual
`
`directional microphones which are configured to have very
`similar noise
`responses
`and very dissimilar
`speech
`responses. The only null formed by the DOMAis one used
`to remove the speech of the user from V.,. The two virtual
`microphones of an embodiment can be paired with an
`adaptive filter algorithm and/or VAD algorithm to signifi-
`cantly reduce the noise without distorting the speech, sig-
`nificantly improving the SNR of the desired speech over
`conventional noise suppression systems. The embodiments
`described herein are stable in operation, flexible with respect
`to virtual microphonepattern choice, and have proven to be
`robust with respect to speech source-to-array distance and
`orientation as well as temperature and calibration tech-
`niques.
`In the following description, numerous specific
`details are introduced to provide a thorough understanding
`of, and enabling description for, embodiments of the
`DOMA. One skilled in the relevant art, however, will
`recognize that these embodiments can be practiced without
`one or more of the specific details, or with other compo-
`nents, systems, etc. In other instances, well-known struc-
`tures or operations are not shown, or are not described in
`detail, to avoid obscuring aspects of the disclosed embodi-
`ments.
`
`Unless otherwise specified, the following terms have the
`corresponding meanings in addition to any meaning or
`understanding they may convey to one skilled in theart.
`The term “bleedthrough” meansthe undesired presence of
`noise during speech.
`The term “denoising” means removing unwanted noise
`from Micl, and also refers to the amount of reduction of
`noise energy in a signal in decibels (dB).
`The term “devoicing” means removing/distorting the
`desired speech from Mic.
`The term “directional microphone (DM)” meansa physi-
`cal directional microphonethat is vented on both sides of the
`sensing diaphragm.
`The term “Mic1 (M1)” meansa general designation for an
`adaptive noise suppression system microphonethat usually
`contains more speech than noise.
`The term “Mic2 (M2)” meansa general designation for an
`adaptive noise suppression system microphonethat usually
`contains more noise than speech.
`The term “noise” means unwanted environmental acous-
`tic noise.
`
`The term “null” means a zero or minimain the spatial
`response of a physical or virtual directional microphone.
`The term “O,” means a first physical omnidirectional
`microphone used to form a microphonearray.
`The term “O,” means a second physical omnidirectional
`microphone used to form a microphonearray.
`The term “speech” means desired speech of the user.
`The term “Skin Surface Microphone (SSM)”is a micro-
`phone used in an earpiece (e.g.,
`the Jawbone earpiece
`available from Aliph of San Francisco, Calif.) to detect
`speech vibrations on the user’s skin.
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`The term “V,” means the virtual directional “speech”
`microphone, which has no nulls.
`The term “V,” means the virtual directional” noise”
`microphone, which hasa null for the user’s speech.
`The term “Voice Activit