(12) United States Patent
`Burnett et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 8.467,543 B2
`*Jun. 18, 2013
`
`USOO8467543B2
`
`(54)
`
`(75)
`
`(73)
`(*)
`
`(21)
`(22)
`(65)
`
`(60)
`
`(51)
`
`(52)
`
`(58)
`
`MCROPHONE AND VOICE ACTIVITY
`DETECTION (VAD) CONFIGURATIONS FOR
`USE WITH COMMUNICATION SYSTEMS
`
`Inventors: Gregory C. Burnett, Livermore, CA
`(US); Nicolas J. Petit, San Francisco,
`CA (US); Alexander M. Asseily, San
`Francisco, CA (US); Andrew E.
`Einaudi, San Francisco, CA (US)
`Assignee: AliphCom, San Francisco, CA (US)
`
`Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 977 days.
`This patent is Subject to a terminal dis
`claimer.
`
`Appl. No.: 10/400,282
`
`Filed:
`
`Mar. 27, 2003
`
`Prior Publication Data
`US 2003/0228O23 A1
`Dec. 11, 2003
`
`Related U.S. Application Data
`Provisional application No. 60/368,209, filed on Mar.
`27, 2002.
`
`Int. C.
`H04B I5/00
`U.S. C.
`USPC .......... 381/94.1: 381/92: 381/943: 381/94.7;
`704/226; 704/233
`
`(2006.01)
`
`Field of Classification Search
`USPC ................ 381/94.7, 71.6, 110, 71.7, 111, 11,
`381/91-92, 94.1-94.3: 704/226, 233
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`EP
`EP
`
`U.S. PATENT DOCUMENTS
`3,789,166 A
`1/1974 Sebesta
`4,006,318 A
`2f1977 Sebesta et al.
`(Continued)
`FOREIGN PATENT DOCUMENTS
`O 637 187 A
`2, 1995
`O 795 851 A2
`9, 1997
`(Continued)
`OTHER PUBLICATIONS
`Zhao Li et al: "Robust Speech Coding Using Microphone Arrays'.
`Signals Systems and Computers, 1997. Conf. record of 31st Asilomar
`Conf. Nov. 2-5, 1997, IEEE Comput. Soc. Nov. 2, 1997. USA.
`(Continued)
`Primary Examiner — Disler Paul
`(74) Attorney, Agent, or Firm — Kokka & Backus, PC
`(57)
`ABSTRACT
`Communication systems are described, including both por
`table handset and headset devices, which use a number of
`microphone configurations to receive acoustic signals of an
`environment. The microphone configurations include, for
`example, a two-microphone array including two unidirec
`tional microphones, and a two-microphone array including
`one unidirectional microphone and one omnidirectional
`microphone. The communication systems also include Voice
`Activity Detection (VAD) devices to provide information of
`human Voicing activity. Components of the communications
`systems receive the acoustic signals and Voice activity signals
`and, in response, automatically generate control signals from
`data of the Voice activity signals. Components of the commu
`nication systems use the control signals to automatically
`select a denoising method appropriate to data of frequency
`Subbands of the acoustic signals. The selected denoising
`method is applied to the acoustic signals to generate denoised
`acoustic signals when the acoustic signal includes speech and
`O1SC.
`
`26 Claims, 29 Drawing Sheets
`
`106 - ul-WAD
`
`Voicing Information
`
`s(n)
`
`H(z)
`
`H(z)
`
`n(n)
`
`s
`n(n) u-r Mic 1
`
`103
`\
`m(n)
`
`
`
`104
`
`m(n)
`s(n)
`Y. 1.
`Mic 2
`
`s
`(c)
`sign
`
`S
`
`t
`(i)
`Noise
`n(n)
`
`-100
`
`t
`Cleaned Speech
`
`
`
`105
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 1 of 44
`
`

`

`US 8,467,543 B2
`Page 2
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`. TO4,233
`
`U.S. PATENT DOCUMENTS
`5, 1986
`4,591,668
`Iwata
`2, 1990
`Gollmar et al.
`4,901,354
`3, 1992
`Baba
`5,097,515
`Ariyoshi
`5, 1993
`5,212,764
`* 10, 1994
`Oh et al.
`5,353,376
`3, 1995
`Linhard
`5,400.409
`Silverberg et al.
`4, 1995
`5,406,622
`Sims, Jr.
`5, 1995
`5,414,776
`* 12, 1995
`Yoshida et al.
`5,473,702
`5, 1996
`Scanlon et al.
`5,515,865
`Sugiyama
`5, 1996
`5,517,435
`T. 1996
`Robbe et al.
`5,539,859
`* 12, 1996
`Park et al. ..................... 704/227
`5,590,241
`4, 1997
`Matouk et al. ........... 379,392.01
`5,625,684
`ck
`5, 1997
`Kanamori et al.
`5,633,935
`Gupta et al.
`7, 1997
`5,649,055
`11, 1997
`Scanlon et al.
`5,684,460
`3, 1998
`Holzrichter et al.
`5,729,694
`5, 1998
`Hosoi et al.
`5,754,665
`11, 1998
`Warnaka et al.
`5,835,608
`12, 1998
`Scanlon
`5,853,005
`6, 1999
`Sasaki et al.
`5,917,921
`10, 1999
`5,966,090
`McEwan
`11, 1999
`5,986,600
`McEwan
`12, 1999
`Holzrichter
`6,006, 175
`Nagata
`12, 1999
`6,009,396
`5, 2000
`Martin et al.
`6,069,963
`6, 191,724
`2, 2001
`McEwan
`Ikeda
`6,266.422
`T/2001
`Handel et al.
`6,430,295
`8, 2002
`Housini ...................... 455,550.1
`9, 2004
`6,795,713
`Vaudrey et al.
`... 381 (94.7
`11/2005
`6,963,649
`12, 2005
`Turnbull et al.
`340/425.5
`6,980,092
`Yang et al. ...................... 381/92
`4, 2007
`7,206,418
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`. 381 (94.7
`
`B1
`B1
`B1
`
`
`
`4/2002 Burnett et al.
`2002fOO39425 A1
`2003/0044025 A1* 3/2003 Ouyang et al. .................. 381/92
`2003/0130839 A1* 7/2003 Beaucoup et al. ............ TO4/226
`FOREIGN PATENT DOCUMENTS
`O984 660 A2
`3, 2000
`2000 3.12395
`11, 2000
`2001 1899.87
`T 2001
`WOO2 07151
`1, 2002
`
`EP
`JP
`JP
`WO
`
`OTHER PUBLICATIONS
`L.C. Ng et al.: “Denoising of Human Speech Using Combined
`Acoustic and EM Sensor Signal Processing”, 2000 IEEE Intl Confon
`Acoustics Speech and Signal Processing. Proceedings (Cat. No.
`00CH37100), Istanbul, Turkey, Jun. 5-9, 2000 XP002 186255, ISBN
`O-78O3-6293-4.
`S. Affes et al.: “A Signal Subspace Tracking Algorithm for Micro
`phone Array Processing of Speech”. IEEE Transactions on Speech
`and Audio Processing, N.Y. USA vol. 5, No. 5, Sep. 1, 1997.
`XPOOOTT4303. ISBN 1063-6676.
`Gregory C. Burnett: “The Physiological Basis of Glottal Electromag
`netic Micropower Sensors (GEMS) and Their Use in Defining an
`Excitation Function for the Human Vocal Tract’. Dissertation. Uni
`versity of California at Davis. Jan. 1999. USA.
`Todd J. Gable et al.: "Speaker Verification Using Combined Acoustic
`and EM Sensor Signal Processing”, ICASSP-2001, Salt Lake City,
`USA.
`A. Hussain: “Intelligibility Assessment of a Multi-Band Speech
`Enhancement Scheme'. Proceedings IEEE Intl. Conf. on Acoustics,
`Speech & Signal Processing (ICASSP-2000). Istanbul, Turkey. Jun.
`2000.
`
`* cited by examiner
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 2 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 1 of 29
`
`US 8.467,543 B2
`
`
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 3 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 2 of 29
`
`US 8.467,543 B2
`
`
`
`
`
`
`
`Microphone
`Configuration
`
`140
`
`150
`
`130
`
`120
`
`160
`
`170
`
`WAD
`Algorithm
`
`
`
`Pathfinder Noise
`Suppression
`
`
`
`Denoised
`Speech
`
`Communication
`Device
`
`FIG.1A
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 4 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 3 of 29
`
`US 8,467,543 B2
`
`pouray)
`
`yosads
`
`sandepy
`
`TOY
`
`weer ee ie fee ee ee KE ee ee ne ee dee eeee
`
`
`
`
`
`J9]90087)ASTONaandepy
`
`ATewaIg
`
`JOUIOTOY
`
`dq]Old
`
`(LUVWord)
`
`(a)°u
`
`
`(u)0(«s))
`
`(2)H
`
`(u)!uYSION
`
`((*))
`
`HOS
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 5 of 44
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 5 of 44
`
`
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 4 of 29
`
`US 8.467,543 B2
`
`UNIDIRECTIONAL
`
`Omni-
`
`Super-
`
`Hyper-
`
`Bi
`
`
`
`
`
`
`
`is
`
`los I go"
`
`O
`
`126
`
`O
`
`110
`
`12 dB
`
`6 dB
`
`27%
`
`25%
`
`33%
`
`1.9
`
`2
`
`1.7
`
`Polar
`Response
`Pattern
`
`I
`
`Maxi
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`so
`
`lar
`
`O
`
`25 dB
`
`100%
`
`33%
`
`1
`
`1.7
`
`Angle of
`R 180
`(null angle)
`Rear
`Rejection
`(relative to front)
`Ambient Sound
`Sensitivity
`(relative to Omni)
`Distance
`Factor
`(relative to Omni)
`
`FIG.2
`(PRIOR ART)
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 6 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 5 Of 29
`
`US 8.467,543 B2
`
`300
`
`OMNI
`
`
`
`AWay
`from
`Speech
`
`Towards
`Speech
`
`FIG.3A
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 7 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 6 of 29
`
`US 8.467,543 B2
`
`
`
`
`
`
`
`SpJBAOL
`
`?009d?
`
`{{{TOIH
`
`19SpuBH
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 8 of 44
`
`

`

`U.S. Patent
`
`US 8.467,543 B2
`
`
`
`s
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 9 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 8 of 29
`
`US 8.467,543 B2
`
`400
`
`/
`
`N is
`
`- 1
`
`
`
`Towards
`Speech
`
`FIG.4A
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 10 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 9 Of 29
`
`US 8.467,543 B2
`
`spJEA0J,
`
`?009dS
`
`SpI8AA0J,
`
`q000dS
`
`10spupH
`
`(INH
`INWO (€)
`
`
`
`
`
`ÁBAW
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 11 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 10 of 29
`
`US 8.467,543 B2
`
`
`
`
`
`spJBAOL
`
`?009dS
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 12 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 11 of 29
`
`US 8.467,543 B2
`
`
`
`500
`
`AWay
`from
`Speech
`
`Towards
`Speech
`
`FIG.5A
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 13 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 12 of 29
`
`US 8.467,543 B2
`
`spIBMOL
`
`q009dS
`
`
`
`
`
`
`
`{{S'OIH
`
`19spueH
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 14 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 13 of 29
`
`US 8.467,543 B2
`
`HCIIS
`
`INWO
`
`|----
`
`spJBAOL
`
`?009dS
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 15 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 14 of 29
`
`US 8.467,543 B2
`
`-
`
`
`
`N
`
`600
`
`/
`
`Towards
`Speech
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 16 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 15 Of 29
`
`US 8.467,543 B2
`
`(INH
`IN[] ©
`
`
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 17 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 16 of 29
`
`US 8.467,543 B2
`
`SpJBAOL |
`?000d?
`
`SpJEMOJ,
`
`?009dS
`
`09’OIH
`
`
`
`
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 18 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 17 Of 29
`
`US 8.467,543 B2
`
`
`
`700
`
`AWay
`from
`Speech
`
`Towards
`Speech
`
`FIG.7A
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 19 of 44
`
`

`

`U.S. Patent
`
`US 8.467,543 B2
`
`
`
`
`
`{{L'OIH
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 20 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 19 Of 29
`
`US 8.467,543 B2
`
`SpIBMOL
`
`?009d?
`
`OL'OIH
`
`
`
`s ".
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 21 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 20 Of 29
`
`US 8.467,543 B2
`
`
`
`800
`
`Towards
`Speech
`
`FIG.8A
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 22 of 44
`
`

`

`U.S. Patent
`
`
`
`US 8.467,543 B2
`
`
`
`
`
`19SpuBH
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 23 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 22 of 29
`
`US 8.467,543 B2
`
`SpI8AA0J, |
`?000dS
`
`O8’OIH
`
`
`
`
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 24 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 23 Of 29
`
`US 8.467,543 B2
`
`
`
`900
`
`(Orientation Irrelevant)
`
`FIG.9A
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 25 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 24 of 29
`
`US 8.467,543 B2
`
`
`
`
`
`
`
`19SpupH
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 26 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 25 Of 29
`
`US 8.467,543 B2
`
`
`
`
`
`SpJBAOL
`
`q000dS
`
`SpJeAOL
`
`?000dS
`
`O6'OIH
`
`19Spe9H
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 27 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 26 of 29
`
`US 8.467,543 B2
`
`
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 28 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 27 Of 29
`
`US 8.467,543 B2
`
`
`
`O
`CN
`O
`y
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 29 of 44
`
`

`

`U.S. Patent
`
`US 8.467,543 B2
`
`
`
`VII’OIH
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 30 of 44
`
`

`

`U.S. Patent
`
`Jun. 18, 2013
`
`Sheet 29 Of 29
`
`US 8.467,543 B2
`
`
`
`O
`CN
`w
`va
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 31 of 44
`
`

`

`US 8,467,543 B2
`
`1.
`MCROPHONE AND VOICE ACTIVITY
`DETECTION (VAD) CONFIGURATIONS FOR
`USE WITH COMMUNICATION SYSTEMS
`
`RELATED APPLICATIONS
`
`This application claims priority from U.S. Patent Applica
`tion No. 60/368,209, entitled MICROPHONE AND VOICE
`ACTIVITY DETECTION (VAD) CONFIGURATIONS
`10
`FOR USE WITH PORTABLE COMMUNICATION SYS
`TEMS, filed Mar. 27, 2002.
`Further, this application relates to the following U.S. Patent
`Applications: Application Ser. No. 09/905,361, entitled
`METHOD AND APPARATUS FOR REMOVING NOISE
`FROM ELECTRONIC SIGNALS, filed Jul 12, 2001; appli
`cation Ser. No. 10/159,770, entitled DETECTING VOICED
`AND UNVOICED SPEECH USING BOTH ACOUSTIC
`AND NONACOUSTIC SENSORS, filed May 30, 2002;
`application Ser. No. 10/301.237, entitled METHOD AND
`APPARATUS FOR REMOVING NOISE FROM ELEC
`TRONICSIGNALS, filed Nov. 21, 2002; and application Ser.
`No. 10/383,162, entitled VOICE ACTIVITY DETECTION
`(VAD) DEVICES AND METHODS FOR USE WITH
`NOISE SUPPRESSION SYSTEMS, filed Mar. 5, 2003.
`
`15
`
`25
`
`TECHNICAL FIELD
`
`The disclosed embodiments relate to systems and methods
`for detecting and processing a desired acoustic signal in the
`presence of acoustic noise.
`
`30
`
`BACKGROUND
`
`Many noise Suppression algorithms and techniques have
`been developed over the years. Most of the noise suppression
`systems in use today for speech communication systems are
`based on a single-microphone spectral Subtraction technique
`first develop in the 1970s and described, for example, by S.
`F. Boll in “Suppression of Acoustic Noise in Speech using
`Spectral Subtraction.” IEEE Trans. on ASSP. pp. 113-120,
`1979. These techniques have been refined over the years, but
`the basic principles of operation have remained the same. See,
`for example, U.S. Pat. No. 5,687.243 of McLaughlin, et al.,
`and U.S. Pat. No. 4,811.404 of Vilmur, et al. Generally, these
`techniques make use of a single-microphone Voice Activity
`Detector (VAD) to determine the background noise charac
`teristics, where “voice' is generally understood to include
`human Voiced speech, unvoiced speech, or a combination of
`Voiced and unvoiced speech.
`The VAD has also been used in digital cellular systems. As
`an example of such a use, see U.S. Pat. No. 6,453,291 of
`Ashley, where a VAD configuration appropriate to the front
`end of a digital cellular system is described. Further, some
`Code Division Multiple Access (CDMA) systems utilize a
`VAD to minimize the effective radio spectrum used, thereby
`allowing for more system capacity. Also, Global System for
`Mobile Communication (GSM) systems can include a VAD
`to reduce co-channel interference and to reduce battery con
`sumption on the client or subscriber device.
`These typical single-microphone VAD systems are signifi
`cantly limited in capability as a result of the analysis of
`acoustic information received by the single microphone,
`wherein the analysis is performed using typical signal pro
`cessing techniques. In particular, limitations in performance
`of these single-microphone VAD systems are noted when
`processing signals having a low signal-to-noise ratio (SNR).
`and in settings where the background noise varies quickly.
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`Thus, similar limitations are found in noise Suppression sys
`tems using these single-microphone VADs.
`Many limitations of these typical single-microphone VAD
`systems were overcome with the introduction of the Path
`finder noise Suppression system by Aliph of San Francisco,
`Calif. (http://www.aliph.com), described in detail in the
`Related Applications. The Pathfinder noise suppression sys
`tem differs from typical noise cancellation systems in several
`important ways. For example, it uses an accurate Voiced activ
`ity detection (VAD) signal along with two or more micro
`phones, where the microphones detecta mix of both noise and
`speech signals. While the Pathfinder noise suppression sys
`tem can be used with and integrated in a number of commu
`nication systems and signal processing systems, so can a
`variety of devices and/or methods be used to supply the VAD
`signal. Further, a number of microphone types and configu
`rations can be used to provide acoustic signal information to
`the Pathfinder system.
`
`BRIEF DESCRIPTION OF THE FIGURES
`
`FIG. 1 is a block diagram of a signal processing system
`including the Pathfinder noise removal or Suppression system
`and a VAD system, under an embodiment.
`FIG. 1A is a block diagram of a noise Suppression/com
`munication system including hardware for use in receiving
`and processing signals relating to VAD, and utilizing specific
`microphone configurations, under the embodiment of FIG.1.
`FIG.1B is a block diagram of a conventional adaptive noise
`cancellation system of the prior art.
`FIG. 2 is a table describing different types of microphones
`and the associated spatial responses in the prior art.
`FIG. 3A shows a microphone configuration using a unidi
`rectional speech microphone and an omnidirectional noise
`microphone, under an embodiment.
`FIG. 3B shows a microphone configuration in a handset
`using a unidirectional speech microphone and an omnidirec
`tional noise microphone, under the embodiment of FIG. 3A.
`FIG. 3C shows a microphone configuration in a headset
`using a unidirectional speech microphone and an omnidirec
`tional noise microphone, under the embodiment of FIG. 3A.
`FIG. 4A shows a microphone configuration using an omni
`directional speech microphone and a unidirectional noise
`microphone, under an embodiment.
`FIG. 4B shows a microphone configuration in a handset
`using an omnidirectional speech microphone and a unidirec
`tional noise microphone, under the embodiment of FIG. 4A.
`FIG. 4C shows a microphone configuration in a headset
`using an omnidirectional speech microphone and a unidirec
`tional noise microphone, under the embodiment of FIG. 4A.
`FIG. 5A shows a microphone configuration using an omni
`directional speech microphone and a unidirectional noise
`microphone, under an alternative embodiment.
`FIG. 5B shows a microphone configuration in a handset
`using an omnidirectional speech microphone and a unidirec
`tional noise microphone, under the embodiment of FIG. 5A.
`FIG. 5C shows a microphone configuration in a headset
`using an omnidirectional speech microphone and a unidirec
`tional noise microphone, under the embodiment of FIG. 5A.
`FIG. 6A shows a microphone configuration using a unidi
`rectional speech microphone and a unidirectional noise
`microphone, under an embodiment.
`FIG. 6B shows a microphone configuration in a handset
`using a unidirectional speech microphone and a unidirec
`tional noise microphone, under the embodiment of FIG. 6A.
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 32 of 44
`
`

`

`US 8,467,543 B2
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`45
`
`3
`FIG. 6C shows a microphone configuration in a headset
`using a unidirectional speech microphone and a unidirec
`tional noise microphone, under the embodiment of FIG. 6A.
`FIG. 7A shows a microphone configuration using a unidi
`rectional speech microphone and a unidirectional noise
`microphone, under an alternative embodiment.
`FIG. 7B shows a microphone configuration in a handset
`using a unidirectional speech microphone and a unidirec
`tional noise microphone, under the embodiment of FIG. 7A.
`FIG. 7C shows a microphone configuration in a headset
`using a unidirectional speech microphone and a unidirec
`tional noise microphone, under the embodiment of FIG. 7A.
`FIG. 8A shows a microphone configuration using a unidi
`rectional speech microphone and a unidirectional noise
`microphone, under an embodiment.
`FIG. 8B shows a microphone configuration in a handset
`using a unidirectional speech microphone and a unidirec
`tional noise microphone, under the embodiment of FIG. 8A.
`FIG. 8C shows a microphone configuration in a headset
`using a unidirectional speech microphone and a unidirec
`tional noise microphone, under the embodiment of FIG. 8A.
`FIG.9A shows a microphone configuration using an omni
`directional speech microphone and an omnidirectional noise
`microphone, under an embodiment.
`FIG. 9B shows a microphone configuration in a handset
`using an omnidirectional speech microphone and an omnidi
`rectional noise microphone, under the embodiment of FIG.
`9A.
`FIG. 9C shows a microphone configuration in a headset
`using an omnidirectional speech microphone and an omnidi
`rectional noise microphone, under the embodiment of FIG.
`9A.
`FIG. 10A shows an area of sensitivity on the human head
`appropriate for receiving a GEMS sensor, under an embodi
`ment.
`FIG. 10B shows GEMS antenna placement on a generic
`handset or headset device, under an embodiment.
`FIG. 11A shows areas of sensitivity on the human head
`appropriate for placement of an accelerometer/SSM, under
`an embodiment.
`FIG. 11B shows accelerometer/SSM placement on a
`generic handset or headset device, under an embodiment.
`In the drawings, the same reference numbers identify iden
`tical or Substantially similar elements or acts. To easily iden
`tify the discussion of any particular element or act, the most
`significant digit or digits in a reference number refer to the
`Figure number in which that element is first introduced (e.g.,
`element 105 is first introduced and discussed with respect to
`FIG. 1).
`The headings provided herein are for convenience only and
`do not necessarily affect the scope or meaning of the claimed
`invention. The following description provides specific details
`for a thorough understanding of and enabling description for,
`embodiments of the invention. However, one skilled in the art
`will understand that the invention may be practiced without
`these details. In other instances, well-known structures and
`functions have not been shown or described in detail to avoid
`unnecessarily obscuring the description of the embodiments
`of the invention.
`
`50
`
`55
`
`60
`
`65
`
`DETAILED DESCRIPTION
`
`Numerous communication systems are described below,
`including both handset and headset devices, which use a
`variety of microphone configurations to receive acoustic sig
`nals of an environment. The microphone configurations
`include, for example, a two-microphone array including two
`
`4
`unidirectional microphones, and a two-microphone array
`including one unidirectional microphone and one omnidirec
`tional microphone, but are not so limited. The communication
`systems can also include Voice Activity Detection (VAD)
`devices to provide Voice activity signals that include infor
`mation of human Voicing activity. Components of the com
`munications systems receive the acoustic signals and Voice
`activity signals and, in response, automatically generate con
`trol signals from data of the Voice activity signals. Compo
`nents of the communication systems use the control signals to
`automatically select a denoising method appropriate to data
`of frequency Subbands of the acoustic signals. The selected
`denoising method is applied to the acoustic signals to gener
`ate denoised acoustic signals when the acoustic signals
`include speech and noise.
`Numerous microphone configurations are described below
`for use with the Pathfinder noise Suppression system. As such,
`each configuration is described in detail along with a method
`of use to reduce noise transmission in communication
`devices, in the context of the Pathfinder system. When the
`Pathfinder noise suppression system is referred to, it should
`be kept in mind that noise Suppression systems that estimate
`the noise waveform and Subtract it from a signal and that use
`or are capable of using the disclosed microphone configura
`tions and VAD information for reliable operation are included
`in that reference. Pathfinder is simply a convenient referenced
`implementation for a system that operates on signals com
`prising desired speech signals along with noise. Thus, the use
`of these physical microphone configurations includes but is
`not limited to applications such as communications, speech
`recognition, and Voice-feature control of applications and/or
`devices.
`The terms “speech” or “voice” as used herein generally
`refer to voiced, unvoiced, or mixed voiced and unvoiced
`human speech. Unvoiced speech or voiced speech is distin
`guished where necessary. However, the term "speech signal”
`or “speech', when used as a converse to noise, simply refers
`to any desired portion of a signal and does not necessarily
`have to be human speech. It could, as an example, be music or
`Some other type of desired acoustic information. As used in
`the Figures, “speech” is meant to mean any signal of interest,
`whether human speech, music, or anything other signal that it
`is desired to hear.
`In the same manner, “noise' refers to unwanted acoustic
`information that distorts a desired speech signal or makes it
`more difficult to comprehend. “Noise suppression' generally
`describes any method by which noise is reduced or eliminated
`in an electronic signal.
`Moreover, the term “VAD is generally defined as a vector
`or array signal, data, or information that in Some manner
`represents the occurrence of speech in the digital or analog
`domain. A common representation of VAD information is a
`one-bit digital signal sampled at the same rate as the corre
`sponding acoustic signals, with a Zero value representing that
`no speech has occurred during the corresponding time
`sample, and a unity value indicating that speech has occurred
`during the corresponding time sample. While the embodi
`ments described herein are generally described in the digital
`domain, the descriptions are also valid for the analog domain.
`The term “Pathfinder, unless otherwise specified, denotes
`any denoising system using two or more microphones, a VAD
`device and algorithm, and which estimates the noise in a
`signal and subtracts it from that signal. The Aliph Pathfinder
`system is simply a convenient reference for this type of
`denoising system, although it is more capable than the above
`definition. In some cases (such as the microphone arrays
`described in FIGS. 8 and 9), the “full capabilities” or “full
`
`Meta Platforms, Inc. - Exhibit 1001
`Page 33 of 44
`
`

`

`US 8,467,543 B2
`
`10
`
`15
`
`30
`
`35
`
`40
`
`5
`version of the Aliph Pathfinder system are used (as there is a
`significant amount of speech energy in the noise micro
`phone), and these cases will be enumerated in the text. “Full
`capabilities’ indicates the use of both H (Z) and H2(z) by the
`Pathfinder system in denoising the signal. Unless otherwise
`specified, it is assumed that only H(Z) is used to denoise the
`signal.
`The Pathfinder system is a digital signal processing—
`(DSP) based acoustic noise Suppression and echo-cancella
`tion system. The Pathfinder system, which can couple to the
`front-end of speech processing systems, uses VAD informa
`tion and received acoustic information to reduce or eliminate
`noise in desired acoustic signals by estimating the noise
`waveform and Subtracting it from a signal including both
`speech and noise. The Pathfinder system is described further
`below and in the Related Applications.
`FIG. 1 is a block diagram of a signal processing system 100
`including the Pathfinder noise removal or Suppression system
`105 and a VAD system 106, under an embodiment. The signal
`processing system 100 includes two microphones MIC 1103
`and MIC 2 104 that receive signals or information from at
`least one speech signal source 101 and at least one noise
`source 102. The path s(n) from the speech signal source 101
`to MIC 1 and the path n(n) from the noise source 102 to MIC
`25
`2 are considered to be unity. Further, H (Z) represents the path
`from the noise source 102 to MIC 1, and H2(z) represents the
`path from the speech signal source 101 to MIC 2.
`Components of the signal processing system 100, for
`example the noise removal system 105, couple to the micro
`phones MIC 1 and MIC 2 via wireless couplings, wired
`couplings, and/or a combination of wireless and wired cou
`plings. Likewise, the VAD system 106 couples to components
`of the signal processing system 100, like the noise removal
`system 105, via wireless couplings, wired couplings, and/or a
`combination of wireless and wired couplings. As an example,
`the VAD devices and microphones described below as com
`ponents of the VAD system 106 can comply with the Blue
`tooth wireless specification for wireless communication with
`other components of the signal processing system, but are not
`so limited.
`FIG. 1A is a block diagram of a noise Suppression/com
`munication system including hardware for use in receiving
`and processing signals relating to VAD, and utilizing specific
`microphone configurations, under an embodiment. Referring
`to FIG. 1A, each of the embodiments described below
`includes at least two microphones in a specific configuration
`110 and one voiced activity detection (VAD) system 130,
`which includes both a VAD device 140 and a VAD algorithm
`150, as described in the Related Applications. Note that in
`Some embodiments the microphone configuration 110 and
`the VAD device 140 incorporate the same physical hardware,
`but they are not so limited. Both the microphones 110 and the
`VAD 130 input information into the Pathfinder noise suppres
`sion system 120 which uses the received information to
`denoise the information in the microphones and output
`denoised speech 160 into a communications device 170.
`The communications device 170 includes both handset and
`headset communication devices, but is not so limited. Hand
`sets or handset communication devices include, but are not
`limited to, portable communication devices that include
`microphones, speakers, communications electronics and
`electronic transceivers, such as cellular telephones, portable
`or mobile telephones, satellite telephones, wireline tele
`phones, Internet telephones, wireless transceivers, wireless
`communication radios, personal digital assistants (PDAs),
`and personal computers (PCs).
`
`55
`
`45
`
`50
`
`60
`
`65
`
`6
`Headset or headset communication devices include, but are
`not limited to, self-contained devices including microphones
`and speakers generally attached to and/or worn on the body.
`Headsets often function with handsets via couplings with the
`handsets, where the couplings can be wired, wireless, or a
`combination of wired and wireless connections. However, the
`headsets can communicate independently with components
`of a communications network.
`The VAD device 140 includes, but is not limited to, accel
`erometers, skin surface microphones (SSMs), and electro
`magnetic devices, along with the associated Software or algo
`rithms. Further, the VAD device 140 includes acoustic
`microphones along with the associated software. The VAD
`devices and associated software are described in U.S. patent
`application Ser. No. 10/383,162, entitled VOICE ACTIVITY
`DETECTION (VAD) DEVICES AND METHODS FOR
`USE WITH NOISE SUPPRESSION SYSTEMS, filed Mar.
`5, 2003.
`The configurations described below of each handset/head
`set design include the location and orientation of the micro
`phones and the method used to obtain a reliable VAD signal.
`All other components (including the speaker and mounting
`hardware for headsets and the speaker, buttons, plugs, physi
`cal hardware, etc. for the handsets) are inconsequential for the
`operation of the Pathfinder noise Suppression algorithm and
`will not be discussed in great detail, with the exception of the
`mounting of unidirectional microphones in the handset or
`headset. The mounting is described to provide information
`for the proper ventilation of the directional microphones.
`Those familiar with the state of the art will not have difficulty
`mounting the unidirectional microphones correctly given the
`placement and orientation information in this application.
`Furthermore, the method of coupling (either physical or
`electromagnetic or otherwise) of the headsets described
`below is inconsequential. The headsets described work with
`any type of coupling, so they are not specified in this disclo
`sure. Finally, the microphone configuration 110 and the VAD
`130 are independent, so that any microphone configuration
`can work with any VAD device/method, unless it is desired to
`use the same microphones for both the VAD and the micro
`phone configuration. In this case the VAD can place certain
`requirements on the microphone configuration. These excep
`tions are noted in the text.
`Microphone Configurations
`The Pathfinder system, although using particular micro
`phone types (omnidirectional or unidirectional, including the
`amount of unidirectionality) and microphone orientations, is
`not sensitive to the typical distribution of responses of indi
`vidual microphones of a given type. Thus the microphones do
`not need to be matched in terms of frequency response nor do
`they need to be especially sensitive or expensive. In fact,
`configurations described herein have been constructed using
`inexpensive off-the-shelf microphones, which have proven to
`be very effective. As an aid to review, the Pathfinder setup is
`shown in FIG. 1 and is explained in detail below and in the
`Related Applications. The relative placement and orientation
`of the microphones in the Pathfinder system is described
`herein. Unlike classical adaptive noise cancellation (ANC),
`which specifies that there can be no speech signal in the