(12) United States Patent
`Bush et al.
`
`USOO63971.86B1
`(10) Patent No.:
`US 6,397,186 B1
`(45) Date of Patent:
`May 28, 2002
`
`(54) HANDS-FREE, VOICE-OPERATED REMOTE
`CONTROL TRANSMITTER
`
`(75) Inventors: William Stuart Bush, Manhattan
`Beach; Carlos Ferdinand Roura,
`Lomita, both of CA (US)
`(73) Assignee: Ambush Interactive, Inc., Manhattan
`Beach, CA (US)
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(*) Notice:
`
`OTHER PUBLICATIONS
`
`“24AA64/24LC64 64K I C" CMOS Serial EEPROM
`published by Microchip Technology Inc. 1999, Document
`No. DS21189C, pp. 1-12.
`“CMOS Low Voltage 2S2 SPST Switches ADG701/
`ADG702” published by Analog Devices, Inc. 1998, pp. 1-8.
`“CMOS Low Voltage 4S2 SPDT Switch ADG719" published
`by Analog Devices, Inc. 1998, pp. 1-8.
`(List continued on next page.)
`
`Primary Examiner Richemond Dorvil
`(74) Attorney, Agent, or Firm- Katten Muchin Zavis
`(21) Appl. No.: 09/469,707
`Rosenman
`(22) Filed:
`Dec. 22, 1999
`ABSTRACT
`(57)
`(51) Int. Cl." ................................................ G10L 21/00
`(52) U.S. Cl. ........................................ 704/274; 704/272 A wireless, programmable, Sound-activated and Voice
`(58) Field of "S, 373.33.5 251.
`2. 5.
`operated remote control transmitter can be used to add
`s
`s
`s
`s
`s
`s
`hands-free speech control operation to a plurality of
`(56)
`References Cited
`remotely controlled appliances manufactured by various
`manufacturers, each of which is normally controlled with
`one or more Signals from an associated remote control
`transmitter. The system may be pre-programmed with
`universal library of codes for controlling various appliance
`categories and appliances produced by various manufactur
`erS within each category. The System may also be pro
`grammed using the controlled appliances remote control
`transmitters and one or more operators spoken commands.
`Once programming is complete, there is no need for the
`operator to manually operate the System, allowing true
`hands-free voice control of the remotely controlled products.
`Voice commands are organized into a plurality of linked
`recognition vocabulary Sets, each representing a Subset of
`the complete voice command Vocabulary available. These
`Subsets are Structured in a fashion that is intuitive to the user
`because the Structure is consistent with controlled appliance
`operation. AS Such, the System allows a user to easily
`navigate Via Voice commands between recognition Sets to
`attain access to the intended Voice commands.
`
`U.S. PATENT DOCUMENTS
`4.322 718 A 3/1982 Faierstain
`4725827 A * 2/1988 Gallegos, Jr. et al. ...... 340/696
`4,771,283 A
`9/1988 Imoto .................... 340,825.71
`4,774,511 A 9/1988 Rumbolt et al.
`4.905,279 A 2/1990 Nishio
`5,142,398 A 8/1992 Heep
`5,199,080 A * 3/1993 Kimura et al. .............. 381/110
`5,226,090 A 7/1993 Kimura
`5,247,580 A 9/1993 Kimura et al.
`5,267,323 A 11/1993 Kimura
`5,444,673 A
`8/1995 Mathurin ..................... 368/63
`5,452.274 A 9/1995 Thompson
`5,471,557. A 11/1995 Chung et al.
`5,691,710 A 11/1997 Pietraszak et al.
`5,774,859 A
`6/1998 Houser et al.
`5,777,571 A 7/1998 Chuang
`5,790.754 A
`8/1998 MoZer et al.
`5,852,804. A 12/1998 Sako
`5,878,394 A
`3/1999 Muhling
`(List continued on next page.)
`
`55 Claims, 27 Drawing Sheets
`
`
`
`20
`
`
`
`
`
`MCROPHONE
`
`AUCO
`SWITCHING
`CIRCUIt
`
`SOUND
`ACTIVAON
`CIRCUIT
`
`SPEC
`recognition
`circuit
`
`NFRAREO
`INFRARE)
`ELIGHTENTING|17
`circuit
`DODE
`
`
`
`
`
`w
`
`SPEAKER
`
`NCATORS
`
`KEYPA
`
`
`
`RAD
`Write
`MMORY
`
`NFRARE
`RECEWER
`
`10: Speech RecognizNG transMitter
`
`POWERSUPPLY CRCUI
`
`Page 1
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`US 6,397,186 B1
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`9/1999 Darbee et al.
`5,959.751 A
`5,983,186 A 11/1999 Miyazawa et al.
`6,012,029 A
`1/2000 Cirino et al. ............... 704/275
`6,052,666 A * 4/2000 Diel et al.
`... 704/275
`6,070,140 A
`5/2000 Tran ........................... 704/275
`6,119,088 A 9/2000 Ciluffo
`
`
`
`OTHER PUBLICATIONS
`
`“DM1000 Component Directional Microphone” published
`by Lucent Technologies, pp. 1-2.
`“MACH 4 CPLD Family High Performance EE CMOS
`Programmable Logic.” Publication 17466 published by
`Lattice Semiconductor Corporation, May 1999 pp. 1-62.
`
`“Preliminary Product Specification Z86L81/86/98 28-Pin
`Low-Voltage Infrared Microcontroller, Document No.
`DS000701-IRX1298, published by Zilog, Inc. 1999, pp.
`1-55.
`“RSC-300/364 DataBook” published by Sensory, Inc., Aug.
`1999, pp. 1-50.
`Control”,
`“RSC-300/364 Recognition • Synthesis
`datasheet part No. 80–0111-6, published by Sensory,
`Inc.1999, pp. 1-8.
`“Build a Voice-Activated Tape Recorder Switch', by Marc
`Spivak, Popular Electronics magazine , Jan. 1996, pp.
`35-36, 79.
`“Voice DirectTM Speech Recognition IC', datasheet part No.
`80-0022-5, published by Sensory Inc., Sep. 8, 1998, pp.
`1-4.
`* cited by examiner
`
`Page 2
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`US. Patent
`
`May 28, 2002
`
`Sheet 1 0f 27
`
`US 6,397,186 B1
`
`v.OE
`
`Gum—5&2.
`
`02....._._Em_.._._._6_._
`
`moo—n—
`
`Gum—(muz—
`
`._<20_m
`
`Gz.....0¢...zOO
`
`._._DOM=U
`
`know—U
`
`._._Dom=o
`
`._._DU~=O
`
`IUmmmm
`
`QZDOm
`
`O_n_D<
`
`ZO_._._ZGOUmm
`
`ZO_._.<>_._.U<
`
`02_IU._._>>w
`
`szImomQS—
`
`Quagmi—
`
`Mm>_momm
`
`om
`
`56m.”—
`
`m..r_~_>>
`
`>mOEm=2
`
`meh<0_Dz_
`
`”Ev—«flaw
`
`D<m>m¥
`
`
`
`
`
`#5015>._n_n_:mmm>>0n_
`
`
`
`
`
`mm...t.2w2<~:.02.520005”.rum—mam”div
`
`VIK
`
`Page 3
`
`AMAZON 1001
`
`Amazon V. SpeakWare
`IPR2019-00999
`
`Page 3
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`
`
`
`
`
`
`

`

`US. Patent
`
`May 28, 2002
`
`Sheet 2 0f 27
`
`US 6,397,186 B1
`
`omul
`
`mm.OE
`
`
`
`FIIIIlllllllllllll.
`
`
`
`
`
`#5015zO_._._ZOOUmw_Tam—mum
`
`
`
`
`P5015 >105!)—E”Eamon”.MEESEmmcum—guz.
`
`02.17.015.200
`
`mnoa8wz_._..:_2m_.:._w_._
`
`ommgmz
`
`.”5:34.80mm.no
`
`._
`
`
`
`4<zmmpxmazawmoomm
`
`_:ommmm
`
`-.5.
`
`—
`
`a.
`
`
`
`.5.30528"EEK
`
`coummm
`
`
`
`02.:U._._>>wO_n=..<
`
`.EDUEO
`
`ON
`
`mZOIQOmQE
`
`DZDOw
`
`zO_._.<>_.FU<
`
`EDGE—U
`
`Page 4
`
`AMAZON 1001
`
`Amazon V. SpeakWare
`IPR2019-00999
`
`Page 4
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`
`
`
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 3 of 27
`
`US 6,397,186 B1
`
`SPEECH RECOGNITION CIRCUIT
`
`68
`GENERAL PURPOSE INPUTIOUTPUT
`
`i? 50
`
`FIG. 2b
`
`.
`
`.
`
`.
`.
`Slm
`
`.
`.
`70
`INFRARED SIGNAL CONTROLLING CIRCUIT
`
`76
`--
`
`OSCILLATOR
`
`COCK
`SOURCE
`
`EXTERNAL
`MEMORY (
`INTERFACE
`
`73
`
`READ
`) WRITE
`MEMORY
`
`MODULATOR
`
`74
`R
`DRIVER
`
`72
`
`80
`R
`
`
`
`|P9,i
`
`PO Po2 2.
`Po:
`
`1
`ROM
`
`DECODER
`
`INDICATOR
`
`52d INFRARED 71
`RECEIVER
`
`Page 5
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 4 of 27
`
`US 6,397,186 B1
`
`FIG. 2C
`
`
`
`
`
`
`
`AUDIO SWITCHING
`CIRCUIT
`
`50
`
`SPEECH RECOGNITION
`CIRCUIT
`
`INPUTIOUTPUT
`
`Page 6
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 5 of 27
`
`US 6,397,186 B1
`
`
`
`Page 7
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 6 of 27
`
`US 6,397,186 B1
`
`
`
`Page 8
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 7 of 27
`
`US 6,397,186 B1
`
`FIG. 5
`
`N
`
`
`
`
`
`
`
`S501
`
`SOUND
`ACTIVATION
`MODE
`
`DETECT SOUND
`
`
`
`SWITCH MICROPHONE
`OUTPUT TO SPEECH
`RECOGNITION CRCUIT
`
`ENTER
`RECOGNITION MODE
`
`
`
`S502
`
`C)
`
`S503
`
`S504
`
`S505
`CD
`
`S506
`
`S508
`
`LISTEN FOR WOICE
`COMMANDS
`
`
`
`MAXIMUM TIME OUT
`COUNTERVALUEP
`
`DETECT SOUNDP
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`SET TIME OUT COUNTER
`8 ERROR COUNTER - O
`
`ACTIVATE NEW
`RECOGNITION SET2
`
`FUNCTION
`COMMAND
`
`OUTPUT
`SIGNAL TOR
`SIGNAL CONTROLLING
`CIRCUIT
`
`SELECTION
`COMMAND
`
`TRANSMIT INFRARED
`SIGNAL TO APPLANCE
`
`
`
`DETECT DURATION
`MATCH2
`
`PERFORM PATTERN
`RECOGNITION
`
`SET TIME OUT COUNTER
`= O & INCREMENT
`ERROR COUNTER
`
`
`
`
`
`ACTIVATE INDICATOR
`
`REACH
`MAXIMUM ERROR
`COUNTERVALUEP
`
`Page 9
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 8 of 27
`
`US 6,397,186 B1
`
`FIG. 6
`
`S620
`
`
`
`INDEFAULT
`RECOGNITION
`MODEP
`
`S621
`
`S622
`
`S623
`
`S624
`
`ENTER SOUND
`ACTIVATION MODE
`
`SWITCH MCROPHONE
`OUTPUT TO SOUND
`ACTIVATION CIRCUIT
`
`LISTEN FOR SOUND
`
`
`
`
`
`DETECT SOUND
`2
`OVER THRESHOLD
`
`S626
`
`REMAIN IN
`SOUND ACTIVATION
`MODE
`
`
`
`Page 10
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 9 of 27
`
`US 6,397,186 B1
`
`FIG. 7
`
`PATTERN
`QUEUNG MODE
`
`
`
`
`
`
`
`DETERMINE
`RECOGNITION
`MODE
`
`S628
`
`CONTINUOUS
`LISTENING MODE
`
`
`
`RESPONSE
`PROMPTING
`MODE
`
`Page 11
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 10 Of 27
`
`US 6,397,186 B1
`
`SS30
`
`
`
`S631
`
`ENTER RESPONSE
`PROMPTING
`RECOGNITION MODE
`
`
`
`SET TIME OUT
`COUNTER = 0, ERROR
`COUNTER O & START
`TIME OUT COUNTER
`
`ACTIVATE NEW
`RECOGNITION SET2
`
`
`
`ACTIVATE A
`MACRO SECUENCE?
`
`FUNCTION
`COMMAND
`
`OUTPUT
`SIGNAL TOR
`SIGNAL CONTROLLING
`CIRCUIT
`
`SELECTION
`COMMAND
`
`FIG. 8
`
`S632
`
`ACTIVATE
`PROMPTING INDICATOR
`
`LISTEN FOR WOICE
`COMMANDS
`
`MAXIMUM TIME OUT
`COUNTER VALUEP
`
`DETECT VOICE
`COMMANDP
`
`Y
`
`GENERATE PATTERN
`
`PERFORM PATTERN
`RECOGNITION
`
`
`
`INCREMENTERROR
`COUNTER
`
`
`
`REMAIN
`N ACTIVE
`RECOGNITION
`SET2
`
`
`
`MAXIMUM ERROR
`COUNTER VALUE)
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 12
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 11 Of 27
`
`US 6,397,186 B1
`
`S660
`
`
`
`GE)
`
`ENTER
`PATTERN GUEUING
`RECOGNITION MODE
`
`N set os start TIME
`OUT COUNTER
`
`S662
`
`
`
`
`
`
`
`SET PATTERN
`COUNTER O AND
`QUEUE COUNTER = 0
`
`
`
`
`
`
`
`S663
`
`FIG. 9
`
`REACH
`TIME OUT PERIOD 2
`
`PAiRN
`COUNTERE OP
`
`Y
`
`(D)
`
`PERFORM
`PATTERN
`RECOGNITION
`
`S672
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`DETECT VOICE
`COMMAND?
`
`
`
`REACH
`MAXIMUM
`PATTERN COUNTER
`VALUE
`N
`
`INCREMENT PATTERN
`COUNTER BY 1
`
`S677
`DOES PATTERN
`COUNTER = 07
`
`Y
`G
`
`
`
`SUBMT NEXT PATTERN f$678
`FOR RECOGNITION
`
`Page 13
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 12 of 27
`
`US 6,397,186 B1
`
`
`
`
`
`
`
`S682
`
`S683
`
`FIG. 10
`
`S680
`
`IS
`QUEUE COUNTER
`= 0?
`
`N
`
`RETRIEVE INSTRUCTION
`
`(D)
`
`S681
`
`SEND SIGNAL TO
`INFRARED SIGNAL
`CONTROLLING CIRCUIT
`
`DECREMENT OUEUE
`COUNTER BY 1
`
`- - - - 4 - - - -
`S691
`
`DETERMINE
`ASSOCATED REMOTE
`CONTROL FUNCTION
`
`
`
`IDENTIFY MEMORY
`LOCATION OF
`CONTROL CODE
`
`GENERATE SIGNAL FOR
`APPLANCE
`
`SUBMIT SIGNAL TO
`INFRARED LED
`
`TRANSMIT INFRARED
`SIGNAL TO APPLIANCE
`
`S692
`
`S693
`
`S694
`
`S695
`
`- - - - - - - -
`
`Page 14
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 13 Of 27
`
`US 6,397,186 B1
`
`FIG 11
`
`CH)
`
`S700
`
`RETRIEVE INSTRUCTION
`FROMMEMORY
`
`S701
`
`
`
`
`
`
`
`IS
`INSTRUCTION A TIME
`DELAY?
`
`
`
`S703
`
`
`
`EXECUTE TIME DELAY
`
`SEND SIGNAL TO
`INFRARED SIGNAL
`CONTROLLING CIRCUIT
`
`
`
`HAVE ALL
`NSTRUCTIONS BEEN
`RETRIEVED?
`
`
`
`CATEGORY?
`
`S712
`
`
`
`Y
`IDENTIFY TARGET
`APPLIANCE
`DEVICE CATEGORY
`
`
`
`N
`
`DETERMINE
`ASSOCATED REMOTE
`CONTROL FUNCTION
`
`IDENTIFY MEMORY
`LOCATION OF
`CONTROL CODE
`
`GENERATE SIGNAL FOR
`APPLIANCE
`
`SUBMIT SIGNAL TO
`INFRARED LED
`
`TRANSMIT INFRARED
`SIGNAL TO APPLIANCE
`
`S714
`
`S715
`
`S716
`
`S717
`
`Page 15
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 14 Of 27
`
`US 6,397,186 B1
`
`F.G. 12a
`
`
`
`
`
`POWER SWITCH
`CLOSED
`
`N
`
`DISCONNECT
`POWER SOURCE
`
`S2O3
`
`EXECUTE RESET
`8, START TIME OUT
`COUNTER
`
`DEPRESS
`SETUP BUTTON
`
`S2O4
`(N)
`
`S2O5
`
`?
`
`Y
`
`DEPRESS
`CODE BUTTON
`
`NED
`SIGNAL
`
`Y
`
`DEPRESS LEARN
`SIGNAL BUTTON
`
`PERFORM
`
`TRAINING?
`
`COMMAND BUTTON
`
`S2O7
`
`S209
`
`Q)
`
`S211
`
`a
`
`AWAIT BUTTON
`SELECTION
`
`
`
`S2O6
`
`PROGRAM
`
`CODE
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`CREATE
`MACRO
`PROGRAM?
`
`Y
`
`DEPRESS
`MACRO BUTTON
`
`S213
`O
`
`MAXIMUM TIME OUT
`COUNTERVALUE
`
`
`
`Y
`
`(M)
`
`Page 16
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 15 Of 27
`
`US 6,397,186 B1
`
`F.G. 12b
`
`(S220
`APPLIANCE CATEGORY SELECTION ROUTINE
`
`
`
`
`
`O
`
`S224
`
`S225
`
`ENTER RESPONSE
`PROMPTING
`RECOGNITION MODE
`
`ACTIVATE APPLIANCE
`CATEGORY NAME
`RECOGNITION SET
`
`SET = 0 & START
`RECOGNITION
`TIME OUT COUNTER
`
`ACTIVATE
`
`LISTEN FOR WOICE
`COMMAND
`
`
`
`
`
`
`
`MAXIMUM TIME OUT
`COUNTERVALUE
`
`Y
`
`DETECT VOICE
`COMMAND
`
`ENERATE PATT
`G
`TE PATTERN
`
`PERFORM PATTERN
`RECOGNITION
`
`S230
`
`O)
`
`DENTIFY APPLIANCE
`CATEGORY NAME
`RECOGNIZED
`
`SEND INSTRUCTION TO
`IDENTIFY APPLIANCE
`CATEGORYTYPE
`
`IDENTIFY ASSOCATED
`MEMORY SECTION
`
`
`
`
`
`APPLIANCE
`NAME IDENTIFIED
`CORRECT?
`
`
`
`
`
`
`
`Page 17
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 16 of 27
`
`US 6,397,186 B1
`
`FIG. 13a
`
`SEND INSTRUCTION TO
`ACTIVATE CODE
`SELECTION PROGRAM
`
`SET = O & START
`PROGRAMMING
`TIME OUT COUNTER
`
`ENTER RESPONSE
`PROMPTING
`RECOGNITION MODE
`
`ACTIVATE APPLIANCE
`CATEGORY NAME
`RECOGNITION SET
`
`SET = 0 & START
`RECOGNITION
`TIME OUT COUNTER
`
`ACTIVATE
`PROMPTING INDICATOR
`
`LISTEN FOR WOICE
`COMMAND
`
`MAXIMUM TIME OUT
`COUNTERVALUE?
`
`
`
`S240
`
`S241
`
`S242
`
`S243
`
`S244
`
`(P)
`
`S245
`
`S246
`
`
`
`
`
`
`
`
`
`DETECT VOICE
`COMMAND?
`
`S249
`
`S250
`
`GENERATE PATTERN
`
`PERFORM PATTERN
`RECOGNITION
`
`S251
`
`
`
`
`
`
`
`
`
`
`
`
`
`IDENTIFY APPLIANCE
`CATEGORY NAME
`RECOGNIZED
`
`SEND INSTRUCTION TO
`IDENTIFY APPLIANCE
`CATEGORYTYPE
`
`S253
`
`S254
`
`DENTIFY ASSOCATED
`MEMORY SECTION
`
`
`
`APPLIANCE
`NAME IDENTIFIED
`CORRECT?
`
`SET = O & START
`PROGRAMMING
`TIME OUT COUNTER
`
`ENTER
`PATTERNOUEUING
`RECOGNITION MODE
`
`ACTIVATE DIGITS
`RECOGNITION SET
`
`S257
`
`S258
`(R)
`
`Page 18
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 17 of 27
`
`US 6,397,186 B1
`
`FIG. 13b
`
`S260
`
`SET = O & START
`RECOGNITION
`TIME OUT COUNTER
`
`S261
`
`
`
`SET PATTERN
`COUNTER - O AND
`QUEUE COUNTER = 0
`
`S262
`
`ACTIVATE
`PROMPTING INDICATOR
`
`
`
`
`
`
`
`
`
`
`
`DETECT VOICE
`COMMAND?
`
`S264
`
`Y
`
`GENERATE PATTERN
`
`QUEUE PATTERN
`IN MEMORY
`
`REACH
`MAXIMUM
`PATTERN COUNTER
`VALUE
`
`
`
`N
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`REACH
`TIME OUT PERIOD 2
`
`PERFORM
`PATTERN
`RECOGNITION
`
`S270
`
`QUEUE INSTRUCTION
`IN MEMORY
`
`INCREMENT OUEUE
`COUNTER BY 1
`
`
`
`
`
`
`
`DECREMENT PATTERN
`COUNTER BY 1
`
`S275
`DOES PATTERN
`COUNTER = 02
`
`Y
`(S)
`
`N
`
`INCREMENT PATTERN
`COUNTER BY 1
`
`SUBMIT NEXT PATTERN frS276
`FOR RECOGNITION
`
`Page 19
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 18 of 27
`
`US 6,397,186 B1
`
`
`
`
`
`
`
`FIG. 13C
`
`(S)
`
`S280
`
`IS
`QUEUE COUNTER
`= 0 2
`
`Y
`
`S281
`
`IDENTIFY APPLIANCE
`oboel
`
`O
`
`RETRIEVE INSTRUCTION
`FROMMEMORY
`
`S282
`
`SEND INSTRUCTION TO
`INFRARED SIGNAL
`CONTROLLING CIRCUIT
`
`DECREMENT OUEUE
`COUNTER BY 1
`
`
`
`
`
`
`
`S284
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`ALLOWTIME OUT
`OF PROGRAMMING
`COUNTER
`
`S285
`- - - 4 - - -
`S286
`DETERMINEDENTIFIER
`ASSOCATED WITH
`INSTRUCTION
`
`STORE IDENTIFIER
`IN MEMORY
`
`S287
`
`S288
`
`ALL
`INSTRUCTIONS
`RECEIVED2
`
`Y
`
`VALID CODE
`IDENTIFIED?
`
`ASSIGN APPLIANCE
`CODE TO APPLIANCE
`CATEGORY
`
`EXIT
`PROGRAMMING MODE
`
`S293
`
`S294
`
`Page 20
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 19 of 27
`
`US 6,397,186 B1
`
`FIG. 14a
`
`
`
`S300
`
`S301
`
`S302
`
`S303
`
`S304
`
`(P1
`
`S305
`
`S306
`
`SEND INSTRUCTION TO
`ACTIVATE SIGNAL
`LEARNING PROGRAM
`
`SET = O & START
`PROGRAMMING
`TIME OUT COUNTER
`
`ENTER RESPONSE
`PROMPTING
`RECOGNITION MODE
`
`ACTIVATE APPLIANCE
`CATEGORY NAME
`RECOGNITION SET
`
`SET = O & START
`RECOGNITION
`TIME OUT COUNTER
`
`ACTIVATE
`PROMPTENG INDICATOR
`
`LISTEN FOR WOICE
`COMMAND
`
`
`
`
`
`S307
`
`MAXIMUM TIME OUT
`COUNTERVALUEP
`
`(Q)
`
`
`
`DETECT VOICE
`COMMAND?
`
`Y
`
`S309
`
`S310
`
`GENERATE PATTERN
`
`PERFORM PATTERN
`RECOGNITION
`
`S311
`
`N
`(P)
`
`IDENTIFY APPLIANCE
`CATEGORY NAME
`RECOGNIZED
`
`SEND INSTRUCTION TO
`IDENTIFY APPLIANCE
`CATEGORY TYPE
`
`IDENTIFY ASSOCATED
`MEMORY SECTION
`
`S313
`
`S314
`
`S315
`
`SEND INSTRUCTION TO
`
`LEARNING CODE
`
`
`
`APPLIANCE
`NAME IDENTIFIED
`CORRECT?
`
`
`
`
`
`
`
`SET = O & START
`PROGRAMMING
`TIME OUT COUNTER
`
`Page 21
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 20 of 27
`
`US 6,397,186 B1
`
`F.G. 14b)
`
`S320
`
`(D
`
`S321
`
`ENTER RECOGNITION
`MODE 8 ACTIVATE
`RECOGNITION SET
`
`SET = 0 & START
`RECOGNITION TIME OUT
`COUNTER
`
`
`
`ACTIVATE INDICATOR
`
`LISTEN FOR WOICE
`COMMANDS
`
`MAXIMUM TIME OUT
`COUNTERVALUE7
`
`Q)
`
`S325
`
`DETECT SOUND?
`
`Y
`
`GENERATE PATTERN
`
`PERFORM PATTERN
`RECOGNITION
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`S329
`
`IDENTIFY ASSOCATED
`INSTRUCTION
`
`
`
`S330
`INSTRUCTION
`TO ACTIVATE
`RECOGNITION
`
`SEND INSTRUCTION
`TO DENTIFY
`FUNCTION TYPE
`
`IDENTIFY MEMORY
`LOCATION TO STORE
`LEARNED SIGNAL
`
`ACTIVATE INDICATOR
`TO PROMPT USERTO
`TRANSMIT SIGNAL
`
`RECEIVE SIGNAL
`FORMAT INTO CODE 8.
`STORE IN MEMORY
`
`S333
`
`S334
`
`S335
`
`ACTIVATE INDICATOR
`TO INDICATE RESULTS
`OF LEARNING ATTEMPT
`
`
`
`S336
`
`REPEAT
`ATTEMPT OR LEARN
`ADDITIONAL
`SIGNALS
`
`
`
`Page 22
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 21 of 27
`
`US 6,397,186 B1
`
`FIG. 15
`
`S340
`
`ACTIVATE VOICE
`TRAINING PROGRAM
`
`
`
`
`
`
`
`
`
`
`
`REACH
`TIME OUT PERIOD?
`
`(D)
`
`S350
`
`PATTERNS
`MATCH2
`
`N
`
`(V)
`
`Y
`
`S351
`
`GENERATE
`REFERENCE PATTERN
`
`
`
`S352
`SIMILAR TO
`EXISTING REFERENCE
`PATTERNSP
`
`PROMPT USER TO
`CHANGE VOICE
`COMMAND NAME
`
`SET = O & START
`TRAINING PROGRAM
`TIME OUT COUNTER
`
`
`
`
`
`
`
`ACTIVATE
`PROMPTING INDICATOR
`
`LISTEN FOR WOICE
`COMMAND
`
`DETECT VOICE
`COMMAND
`
`S344
`
`Y
`
`GENERATE PATTERN
`
`S347
`
`VOICE COMMAND
`BEEN REPEATED?
`
`Y
`
`CU)
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`(W)
`
`
`
`PROMPT USER TO
`REPEAT VOICE
`COMMAND
`
`S349
`
`ALLOWTIME OUT OF
`TRAINING PROGRAM
`COUNTER
`
`Page 23
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 22 of 27
`
`US 6,397,186 B1
`
`FIG. 16a
`
`S360
`
`S361
`
`ACTIVATE MACRO
`PROGRAMMING MODE
`
`EXECUTE APPLIANCE
`CATEGORY SELECTION
`ROUTINE
`
`
`
`
`
`
`
`
`
`
`
`S362
`
`APPLIANCE
`NAME IDENTIFIED
`CORRECT?
`
`
`
`
`
`S364
`
`S365
`
`Ox)
`
`S366
`
`S367
`(W)
`
`ENTER
`SPEAKER DEPENDENT
`RECOGNITION MODE
`
`ACTIVATE SPEAKER
`DEPENDENT
`RECOGNITION SET
`
`SET = 0 & START
`RECOGNITION
`TIME OUT COUNTER
`
`ACTIVATE
`PROMPTING INDICATOR
`
`LISTEN FOR WOICE
`COMMAND
`
`
`
`REACH
`MAXIMUM TIME OUT
`COUNTER
`VALUEP
`
`
`
`N
`
`S369 N
`DETECT VOICE
`COMMAND?
`
`(N)
`
`(W)
`
`Pigersasure
`
`GENERATE PATTERN
`
`S371
`
`PERFORM PATTERN
`RECOGNITION
`
`S372
`
`S373
`
`activate NicarorN S374
`
`ACTIVATE INDICATOR
`
`Ox)
`
`
`
`IDENTIFY ASSOCATED
`MEMORY LOCATION TO
`STORE INSTRUCTIONS
`
`ENTER DEFAULT
`RECOGNITION MODE
`
`S375
`(Z)
`
`CY)
`
`Page 24
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 23 Of 27
`
`US 6,397,186 B1
`
`FIG. 16b
`
`S391
`
`
`
`
`
`INSTRUCTION
`TO ACTIVATE
`RECOGNITION
`
`Y
`(A1)
`
`S380
`
`(A1)
`S381
`(Y)
`
`S382
`
`ENTER
`RECOGNITION MODE
`
`ACTIVATE
`RECOGNITION SET
`
`SET c O & START
`RECOGNITION TIME OUT
`COUNTER
`
`
`
`
`
`N
`
`S392
`
`STORE ASSOCATED
`INSTRUCTION
`IN MEMORY
`
`S393
`
`STORE END MARKER
`NSTRUCTION
`IN MEMORY
`
`ACTIVATE
`PROMPTING INDICATOR
`
`LISTEN FOR WOICE
`ES
`
`MAXIMUM TIME OUT
`COUNTERVALUE
`
`DETECT SOUNDP
`
`Y
`
`GENERATE PATTERN
`
`PERFORM PATTERN
`RECOGNITION
`
`
`
`N
`
`BUTTON
`PRESSED TO INSERT
`TIME DELAY?
`
`STORE INSTRUCTION
`FOR TIME DELAY IN
`MEMORY
`
`MACRO
`BUTTON PRESSED?
`
`Y
`(2)
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`S397
`
`N
`PROGRAM
`COMPLETION BUTTON
`PRESSED?
`
`
`
`N
`
`Y
`EXIT MACRO
`
`S398
`
`-5 g d ProgRAMMINGMODE (9)
`
`S389
`
`Y
`
`N
`
`ACTIVATE INDICATOR
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 25
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 24 of 27
`
`US 6,397,186 B1
`
`
`
`
`
`
`
`FIG. 17
`
`VCONNECTIONS
`
`SPEECH RECOGNITION
`CIRCUIT IC
`
`68
`
`Page 26
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`US. Patent
`
`May 28, 2002
`
`Sheet 25 0f 27
`
`US 6,397,186 B1
`
`ammfimz.
`
`Guy—guz—
`
`mm>_momm
`
`moo—oaozE_:m_-Eo_._
`02......0mhzOU
`
`.._<2mv_w
`
`._._Do~=o
`
`zO_._._ZOOUmm
`
`
`
`#5015zO_._.<>_._.o<
`
`5.5015
`
`02_Io._._>>m
`
`.EDUEU
`
`mum—guz—
`
`IUNMQw
`
`mm“GE
`
`om
`
`on
`
`O_DD<
`
`
`
`nzaommzozmomoE
`
`om
`
`54mm
`
`m.._._m>>
`
`>mOEwE
`meh<o_n_Z_—
`know—0>._n_n=..mmm>>0n_1‘
`
`>
`
`
`
`
`
`D<m>m¥
`
`
`
`
`
`amt—széhGEN—2000mmIUmwn—m"awlx
`
`Page 27
`
`AMAZON 1001
`
`Amazon V. SpeakWare
`IPR2019-00999
`
`Page 27
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`May 28, 2002
`
`Sheet 26 of 27
`
`US 6,397,186 B1
`
`EICTOICI
`
`5) NITTO?|LINOO
`
`LITTORIIS)
`
`LIT-™ORIIS)
`
`TV/NS)IS
`
`CIERIV/>|HNI
`HOEWEIGHS
`
`NOLLINSDOCERINOLLVALLOW/
`
`CINTOS
`
`„LITTORIO
`
`09
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`06
`
`
`
`LITTORIO AT ddTIS RIEINWOd
`
`/GIVER!!
`
`ELLIRINA
`
`ARHOVNE IN
`
`dÅEX
`
`Page 28
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`US. Patent
`
`May 28, 2002
`
`Sheet 27 0f 27
`
`US 6,397,186 B1
`
`him—guz—
`
`wnOE
`
`5QozEEWEe:
`023.6528:aoEo20F<>Fo<
`._<zw_mzoEzwoomm
`omméuz:owwamnz:0m
`
`
`
`
`
`:DoEo.595
`
`
`
`om
`
`
`
`om“.OE
`
`0.036.
`
`EDGE—U
`
`02_IO._._>>w
`mmZOImOEQE
`
`mow
`
`<szIn—OMOE
`
`vlak
`
`cum—guz—
`
`mm>_momm
`
`mEEs
`
`>mosms
`
`54mmo<m>mx
`
`225.2E>
`
`
`
`
`
`EDGE—0>.._n_n_:wmm>>0m
`
`
`
`
`
`mm.._..:S_wz<N_._.GEN—2000mmIOMmEm”a
`
`Page 29
`
`AMAZON 1001
`
`Amazon V. SpeakWare
`IPR2019-00999
`
`Page 29
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`
`
`
`
`
`

`

`1
`HANDS-FREE, VOICE-OPERATED REMOTE
`CONTROL TRANSMITTER
`BACKGROUND OF THE INVENTION
`1. Field of Invention
`This invention relates to devices for remotely controlling
`electronic equipment, and more particularly, to a wireless,
`user-programmable, Voice-activated and Voice-operated
`remote control System for controlling appliances.
`2. Description of Prior Art
`Historically, appliances, for example, electronic
`appliances, Such as, televisions, VCRS, digital Satellite
`Systems, audio Systems, and related accessories, have been
`remotely controlled by hand-held transmitters used to gen
`erate Signals to receivers incorporated into the electronics of
`the remotely controlled appliances. Signals for Such appli
`ances correspond to control commands, Such as channel
`Selection/tuning, power on/off, audio Volume adjustment,
`and muting controls, typically generated by the user by
`depressing buttons on a remote control transmitter keypad.
`The basic composition and operation of Such remote control
`Systems are well known in the art.
`For convenience, Such remote control transmitters are
`generally designed to be Sufficiently Small to be hand-held
`during operation. There are Several disadvantages in using
`Such hand-held transmitters. For example, the Small size and
`mobility often contribute to misplacement or loss of the
`transmitter. Also, for device operators with restricted physi
`cal mobility or Sight limitations, hand-held remote controls
`may not provide Sufficient access to the command controls
`of the remotely controlled appliances. Also, if an operator's
`hands are engaged in an activity, an interruption in the
`activity may be required to operate the hand-held remote
`control, causing inconvenience to the operator and poten
`tially having an adverse effect on productivity.
`Additional problems occur as a consumer acquires vari
`ous remotely controlled appliances, where each is typically
`Supplied with a separate remote control transmitter. AS the
`number of Separate remote control transmitters increases,
`locating, distinguishing, and locating the appropriate trans
`mitters becomes increasingly difficult. In response to this
`problem, universal remote control transmitters, pre
`programmed by the manufacturers with control commands,
`typically coded infrared signals, have been developed to
`operate many different remotely controlled electronic
`appliances, for example, as disclosed in U.S. Pat. No.
`4,774,511. In addition to universal remote control
`transmitters, learning remote control transmitters have been
`developed which receive the control command Signals,
`typically infrared codes, from the remote control transmit
`ters provided with remotely controlled appliances and gen
`erally Store the frequencies and pulses into memory, with the
`Signals becoming associated with buttons located on the
`keypad of the learning remote control transmitter. After
`programming is complete, depressing the keypad buttons
`prompts the learning remote control transmitter to
`re-transmit the codes stored within its memory. This allows
`the user to consolidate the control of Several remotely
`controllable appliances into a single hand-held remote con
`trol transmitter. An example of Such a System is disclosed in
`U.S. Pat. No. 5,142,398. For additional convenience to the
`user, the learning capability has been combined with uni
`Versal remote control transmitters which are pre
`programmed by the manufacturers with control commands
`necessary to operate many different remotely controlled
`electronic appliances, for example as disclosed in U.S. Pat.
`No. 5,691,710.
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,397,186 B1
`
`2
`All of the above-mentioned Systems require the user to
`establish physical contact, typically in the form of manually
`depressing keypad buttons, to transmit a control command
`to the remotely controlled appliance. AS Such, the above
`mentioned Systems due to their Small size are often mis
`placed causing frustration to the user.
`AS Such, Voice-operated remote control Systems have
`recently been developed which operate in response to user
`spoken commands instead of commands Selected by manu
`ally depressing keypad buttons. U.S. Pat. Nos. 5,199,080;
`5,247,580; 5,267,323 and 5,226,090 disclose various
`embodiments of Voice-operated remote control Systems
`which employ voice control commands instead of control
`commands entered through buttons on a keypad.
`Unfortunately, Such Systems are not truly hands-free, requir
`ing manual intervention by the user during use. In particular,
`Such remote control Systems as disclosed in the above
`mentioned patents, are all based upon the use of a “talk
`Switch', which must be manually depressed to enter a voice
`command when the transmission of a remote control Signal
`is desired.
`U.S. Pat. No. 5,226,090 further discloses a voice-operated
`remote control System which contains a detector for detect
`ing whether a voice command is received or not. A detected
`Signal from the detector is applied to a power Supply circuit.
`When there is no voice command received as detected by the
`detector, the power Supply control circuit cuts off electric
`energy Supplied to a speech recognition circuit. The Voice
`command detector circuit is implemented by a detecting
`program stored in the ROM in a controller. The detecting
`program determines whether the talk Switch has been
`depressed. The transmitter operates depending on whether
`the talk Switch has been depressed. If the talk Switch has
`been depressed, the transmitter is enabled to remote control
`Signals. Once the talk Switch is released, the transmitter is
`kept in a low power consumption mode, waiting for voice
`commands to be applied. AS indicated above, the means for
`generating and transmitting a remote control Signal based on
`the recognized spoken Voice command is not hands-free,
`requiring the manual intervention of pressing a talk Switch
`to accomplish these functions.
`Various other Systems are known which use speech rec
`ognition to control appliances. For example, U.S. Pat. No.
`5,852,804 discloses a system for controlling several external
`appliances connected to a speech recognition System.
`However, the system disclosed in the 804 patent requires
`physical interconnections between the control System and
`the appliance which makes it difficult for a user to add
`additional appliances or change controlled appliances.
`U.S. Pat. No. 5,878,394 discloses a system which includes
`connections to a remote control network for transmitting
`infrared codes and a graphical user interface on a personal
`computer (PC). In addition to the inconvenience of estab
`lishing the remote control network, a user must have access
`to a PC and desire to make the PC available for Such control
`functions. The system disclosed in the 394 patent has only
`limited utility for consumers who have access to a PC or can
`afford the expense of a PC and thus excludes a significant
`portion of consumers who may otherwise desire to add
`Speech recognition control to their electronic appliances if
`an inexpensive alternative is available.
`U.S. Pat. No. 5,774,859 discloses a Subscriber informa
`tion System with a speech interface, Similar to the System
`disclosed in U.S. Pat. No. 5,878,394 the system in the 859
`patent is based upon a PC-class processor. Operation of this
`System is dependent upon receiving information from an
`
`Page 30
`
`AMAZON 1001
`Amazon v. SpeakWare
`IPR2019-00999
`
`

`

`US 6,397,186 B1
`
`3
`information distribution center or head-end installation and
`therefore lacks the advantages of a Stand-alone device.
`Unfortunately, the remote control is not capable of indepen
`dently completing the Speech recognition process and trans
`mitting infrared signals indicated by Such recognition results
`to controlled appliances. Furthermore, this System also
`requires a PC-class processor.
`Inconvenience is another problem that occurs as a user
`intends to perform multiple control functions involving one
`or more controlled appliances and must recall and enter a
`Series of Voice commands to a voice-operated remote control
`system. U.S. Pat. No. 5,852,804 discloses a system, where
`one speech command may be defined which is capable of
`manipulation of more than one connected appliance.
`However, this System requires that each controlled appliance
`be physically connected to the System and that the controlled
`appliance be manually manipulated during Setup for each
`function to be executed upon recognition of the Speech
`command. Also, if a Single command is to be used to execute
`a Series of controlled appliance functions, its capability is
`limited unless time delays of variable lengths may be
`inserted within the Series of control functions to accommo
`date the initialization periods for certain controlled appli
`ances. For example, if a voice command Sends Signals to a
`television to power it on and turn to a specified channel, a
`period of time may elapse between the receipt of the power
`on command until the television warms up and is ready to
`receive additional commands. Such initialization periods
`vary from appliance to appliance and therefore means is
`needed to allow a user to adjust the time delay as necessary.
`There are various other problems with known systems
`which utilize Voice recognition to control appliances. For
`example, one problem occurs as the list of voice commands
`grows. Generally, each time a Speech recognition product
`detects a spoken word, it compares the word to those in a
`recognition vocabulary. In general, as a recognition vocabu
`lary increases in size, recognition time increases and accu
`racy decreases. The more words in a recognition vocabulary,
`the more likely the Speech recognition circuit will make a
`mistake and accuracy generally diminishes. Without the
`capability to utilize multiple Vocabulary recognition Sets, the
`commands available for selection must all be available at the
`Same time, meaning they must all reside in the same
`recognition vocabulary Set. Thus, the user may be inconve
`nienced by the limited command availability of a small
`recognition vocabulary or the diminishing speech recogni
`tion accuracy and longer recognition times resulting as the
`Single recognition vocabulary is increased in size. Another
`disadvantage of a Single recognition vocabulary is that the
`recognition of Voice command always yields the same
`response. In a multiple recognition vocabulary Set Scheme,
`the same Voice command may yield different results upon
`recognition if placed within different Vocabulary recognition
`Sets. A Still further disadvantage of a Single recognition
`Vocabulary is that the recognition vocabulary words must be
`phonetically distinctive from each other which may limit the
`use of words that may be intuitive for their intended function
`but are too phonetically similar to other words in the
`recognition vocabulary, forcing the user to remember a word
`that is leSS intuitive. In a multiple recognition vocabulary Set
`Scheme, words that are not phonetically distinctive from
`each other may still be used by placing Such words in
`different recognition vocabulary Sets.
`Another problem wi