HTUMAT MUUT
`US 20170357478A1
`( 19 ) United States
`( 12 ) Patent Application Publication ( 10 ) Pub . No . : US 2017 / 0357478 A1
`( 43 ) Pub . Date :
`Dec . 14 , 2017
`PIERSOL et al .
`
`( 54 ) INTELLIGENT DEVICE ARBITRATION AND
`CONTROL
`( 71 ) Applicant : Apple Inc . , Cupertino , CA ( US )
`Inventors : Kurt W . PIERSOL , Cupertino , CA
`( 72 )
`( US ) ; Ryan M . ORR , Cupertino , CA
`( US ) ; Daniel J . MANDEL , Tucson , AZ
`( US )
`( 21 ) Appl . No . : 15 / 268 , 338
`( 22 ) Filed :
`Sep . 16 , 2016
`Related U . S . Application Data
`( 60 ) Provisional application No . 62 / 348 , 896 , filed on Jun .
`11 , 2016 .
`Publication Classification
`
`( 51 )
`
`Int . Cl .
`G06F 3 / 16
`GIOL 15 / 30
`
`( 2006 . 01 )
`( 2013 . 01 )
`
`G10L 15 / 22
`( 2006 . 01 )
`( 2013 . 01 )
`GIOL 25 / 84
`( 52 ) U . S . CI .
`CPC . . . . . . . . . . . . . . G06F 3 / 167 ( 2013 . 01 ) ; GIOL 25 / 84
`( 2013 . 01 ) ; GIOL 15 / 30 ( 2013 . 01 ) ; GIOL 15 / 22
`( 2013 . 01 ) ; GIOL 2015 / 228 ( 2013 . 01 )
`
`ABSTRACT
`
`( 57 )
`This relates to systems and processes for using a virtual
`assistant to arbitrate among and / or control electronic
`devices . In one example process , a first electronic device
`samples an audio input using a microphone . The first elec
`tronic device broadcasts a first set of one or more values
`based on the sampled audio input . Furthermore , the first
`electronic device receives a second set of one or more
`values , which are based on the audio input , from a second
`electronic device . Based on the first set of one or more
`values and the second set of one or more values , the first
`electronic device determines whether to respond to the audio
`input or forego responding to the audio input .
`
`KKKKKKKUUUUUKKKKKK
`
`KAMARRARARAAAAAAAAAAAAAAAAKKKKKKKKKKKKKK KKKKRARRRRAAAAAAAAAARRRRRRRRRRRRRRAAAARRRRR
`
`Server System
`Digital Assistant Server
`m 112
`pre 14
`
`WAARAAN
`
`w
`
`wwwww
`
`Processing
`11 .
`
`on 108
`
`System
`100
`
`WAKKKKKKKKKKKKKKKKKKKKKKKKKKK
`
`per 106
`
`www
`
`ho 116
`
`vu . . .
`
`na
`
`Data &
`Models
`
`p118
`
`VO
`Interface
`C
`Client
`
`10 interface to External Services
`
`wwwwwww w
`
`wwwwwwwwwwwwwwwwww
`
`wan w
`
`wwwwwwwwwwwwwwww
`
`w
`
`AAAAAAAAAA
`
`104
`???
`18800 - 102
`DA
`10
`
`w
`
`vuonna from
`
`122 mm
`JOOD
`
`. . 1983 . XX . .
`
`????????????????????????????????????????????????????????????????
`- -
`
`"
`
`uuuuuURRAKKKKK UK
`
`Notworks
`
`110
`
`120
`wwwmm wwwwww
`
`
`
`www www ww
`
`w we 012 non
`
`mwana wa mm fun www
`
`on ww
`
`w
`
`Telephony
`Service
`
`are
`
`wwwwwwwww
`
`was a
`
`won man who
`
`was
`
`Calendar
`Service
`External Services
`
`a we mwa wa
`
`nne m
`
`mm non
`
`nom
`
`m
`
`moon
`
`w
`
`w
`
`wwwwwwwwwwwww
`
`Navigation
`5
`??
`
`sai
`Service
`
`wwwwwww
`
`noonnomm
`
`Information
`Service ( s )
`
`MA
`
`wwwwwwwwwwwwwwww
`
`www
`
`wanan mom an m wen mee www
`
`Page 1 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 1 of 28
`
`US 2017 / 0357478 A1
`
`
`
`
`
`
`
`
`
`wana mnamo mann mann manos
`
`
`
`
`
`
`
`
`
`
`
`External Services
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`????
`
`?????
`
`?
`
`?
`
`?
`
`?
`
`?
`
`Navigation Services
`
`?????????????????????????????????
`
`Messaging Service ( s )
`
`Information Services )
`
`w
`
`wwwwww
`
`????????????
`
`Calendar Service
`
`Telephony Service
`
`mm
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`mm mor
`20 min
`
`System 100
`
`108
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`KKKKKKKKKKKKKKKKKKKKKKKKKKK
`
`106
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`KKKKKKKKKKKKKKKKKKKKKKK
`
`
`
`Server System
`
`
`
`KATIKA KAKKAKKAKAKKAAN
`
`
`
`erano 118
`
`Data & Models untuwwon
`
`RRRAAD
`
`114
`
`mamman Processing Modules
`
`mo 1 12
`
`
`
`
`
`Digital Assistant Server
`
`
`
`
`
`110 interface to External Services
`
`wanan
`
`VO
`
`Interface to DA Client
`
`w
`
`
`
`cores 110
`
`there was
`
`Network ( s )
`
`rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
`
`
`
`
`
`
`
`
`
`nann anane was worn anna mann manne wanne wana anaona wao
`
`
`
`
`
`
`
`mann annne waar wann wanne wana anna anna
`
`
`
`
`
`
`
`
`
`w
`
`ww ww w
`
`w
`
`FIG . 1
`
`DOWNOLA
`ann an we
`0000000
`
`22 sene
`
`104 cm
`
`me mi
`
`CMDL102 00 A
`BIDA
`8 nnnnnn
`
`Page 2 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 2 of 28
`
`US 2017 / 0357478 A1
`
`SSSSSSSSSS
`
`L 244
`~ 252
`253
`218
`
`wwwwwwwwww
`
`Portable Multifunction Device 200
`sene 236
`enddessen
`Applications ( continued )
`254
`So here
`Map Module
`com 247
`Browser Module
`248
`Calendar Module
`249
`Widget Modules
`249 - 1
`Weather Widget ( s )
`249 - 2
`Stocks Widget
`249 - 3
`Calculator Widget
`249 - 4
`Alarm Clock Widget
`249 - 5
`Dictionary Widget
`
`wwwwwwwwwwwwwwwwwwww
`
`M
`
`User - Created Widget ( s )
`Widget Creator Module
`Search Module
`JOnline Video Module
`
`m
`
`Device / Global Internal State
`262
`Power System ham 2
`pe 224
`Exiemal Port
`RF Circuitry
`3
`203
`como
`208
`12031 Audio
`Circuitry
`210
`Proximity
`Sensor
`
`Peripherals
`Interface
`
`1 , 2037
`
`203
`
`249 - 6
`250
`malam 251
`255
`mm
`255
`
`257
`
`Speaker
`spremen 211
`
`Microphone
`213
`266
`
`Operating System
`Communication Module
`Contact / Motion Module
`Graphics Module
`Haptic Feedback Module
`Text input Module
`GPS Module
`Digital Assistant Client Module
`User Data and Models
`Applications
`Contacts Module
`Telephone Module
`Video Conference Module
`E - mail Client Module
`instant Messaging Module
`Workout Support Module
`Camera Module
`Image Management Module
`Video & Music Player Module
`Notes Module
`
`Memory 202
`
`wwwwwwwwwwwwwwwwwwwwww
`
`www
`
`204 mm
`
`203
`
`C?????
`
`os
`
`Processor ( s )
`
`momo
`
`mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm
`
`: :
`
`.
`
`22222222222222222222222222222222
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`: : :
`
`: : : :
`
`:
`
`:
`
`.
`
`33355555551511111111111111111111112113299999999999999999999999IIIII
`
`2067
`Display
`Controller 256
`
`Optical
`Sensor ( s )
`Controller 258
`SUR 203
`Optical
`Sensor ( s )
`264
`
`IIIIII
`
`2222222222222222222
`22222222222222222222
`
`IIIIIIIIIIIIIIIIIIIIIIIIIII
`
`203
`Touch
`Sensitive
`Display System
`212
`
`Accelerometer ( s )
`268
`
`nnnnnnnnnnn
`
`Haptic
`nnnnnnnn
`Feedback
`Controller 261
`203
`Tactile Output
`Generator ( s )
`
`wwwwwwwwwwwww
`199999999999999999999999999999
`
`+
`
`+
`
`+
`
`+
`
`+
`
`+
`
`+
`
`+
`
`+
`
`+
`
`+
`
`+
`
`+
`
`+
`
`+
`
`+
`
`+
`
`+
`
`???????????????
`
`1001111
`
`Other input
`Controllerís )
`260
`203
`Other Input
`Control Devices
`216
`
`203
`VO Subsystem
`Intensity
`Sensor ( s )
`
`11111111111111111111111111111111111111111101005555555555555
`
`MP 203
`Contact
`Intensity
`Sensor ( s )
`265
`
`FIG . 24
`
`Page 3 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 3 of 28
`
`US 2017 / 0357478 A1
`
`ggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggg po272
`
`
`
`
`
`
`
`271
`
`
`
`Evení Monitor
`
`
`
`Event Sorter 270
`women
`
`MT
`
`Hit View Determination Module
`
`pr - 273
`
`- 274
`
`uuuuuuuuuu
`
`Recognizer Determination Module
`Active Event
`
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`
`
`
`
`Event Dispatcher Module
`
`2 287 - 2
`
`ooo
`
`www
`
`283
`
`Event Delivery 288
`w wwwwwwwwwwwwwww
`
`Definitions 286 Event 1 287 - 1
`
`284 Event
`
`Comparator Event Receiver 282 Event
`Event
`non Metadata
`
`
`
`Event Recognizer
`
`ww
`MY
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`RELATUM
`
`U
`
`M
`
`HOMEM
`
`W
`
`FIG . 2B
`
`
`
`
`
`Event Recognizer 280
`
`
`
`Event Recognizer 280 Event Data 279
`
`
`
`
`
`* *
`
`n
`
`nnnnnnnn
`nnnnnnnnnnnnnnnnnn
`
`291
`
`Application View
`
`_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
`
`291
`
`Application View
`
`290
`
`
`
`Event Handler
`
`Application 236 - 1 wenn
`
`mann
`
`Application
`
`
`
`
`
`
`
`Updater 277 Data Updater 276 Object
`
`aha
`
`278
`n nnnnnnnnnnnnnnnnnn
`mannnnnnnnnnnnnnnnnn
`
`GUI Updater
`
`290
`
`
`
`Event Handler
`
`
`
`
`
`Application internal State
`
`????????????????????????????????????????????????????????????????????????????????????????
`
`Page 4 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 4 of 28
`
`US 2017 / 0357478 A1
`
`90€
`
`|
`
`Portable Multifunction Device 200
`310
`Optical Sensor
`597
`
`Speaker 211
`
`Proximity
`Sensor 266
`
`310 is SIM card slot
`312 is headphone jack
`
`ww w
`
`ww w
`
`Touch Screen 212
`Contact Intensity
`Sensor ( s ) 265
`ww www www www
`www ww www www www
`Tactile Output
`Generator ( s ) 267
`
`{
`
`w
`
`w
`
`w
`
`w
`
`w
`
`Microphone
`
`Miphone
`
`OWOH
`
`e 304
`
`External Port 224
`* OW
`
`
`Accelerometer ( s )
`997
`
`????????????????????
`
`wwwwwwwww
`
`wwwwwwwwwwwwwwwwww
`
`wwwwwwwwwwwwww
`
`URKKKKKKKKK
`UUUUUUUUUUUUUUUUUUUU
`
`KKKKKKK
`
`| 80€
`
`30€
`
`LLLLLLL 22222222222222222222222222222224222122222222222272222222222222222222222222222222222222222222222222222222222222222222222222222222222222222221222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222221222222222222222222222222
`
`Page 5 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 5 of 28
`
`US 2017 / 0357478 A1
`
`Memory 470
`
`Device 400
`
`410 w
`
`CPU ( S )
`
`420
`
`????????????????????
`
`wwwwww
`
`430
`
`VO Interface
`Display
`
`Keyboard / Mouse
`Touchpad
`Tactile Output
`Generator ( s )
`Sensor ( s )
`
`wwwwwwwwwwwwwwwwwwwww
`
`was 450
`hwn 455
`h 457
`
`h
`
`459
`
`460
`
`Network
`Communications
`Interface
`
`wwwwwwwwwww
`
`w wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`Operating System
`Communication Module
`Contact / Motion Module
`Graphics Module
`Haptic Feedback Module
`Text Input Module
`Digital Assistant Client Module
`User Data and Models
`Applications
`Contacts Module
`Telephone Module
`Video Conference Module
`E - nail Client Module
`Instant Messaging Module
`Workout Support Module
`Camera Module
`{ lmage Management Module
`Browser Module
`Calendar Module
`Vidge
`???S
`Weather Widget
`Stocks Widget
`Calculator Widget
`Alarm Clock Widget
`Dictionary Widget
`
`wwwwwwwwwwww
`
`User - Created Widget ( s )
`Widget Creator Module
`Search Module
`Video & Music Player Module
`Drawing Module
`Presentation Module
`Word Processing Module
`Website Creation Module
`Disk Authoring Module
`Spreadsheet Module
`98
`
`FIG . 4
`
`Device / Gobal Internal State
`
`mw
`
`19 LLLLLLLS
`
`L - 234
`
`have no
`
`244
`L - 247
`248
`249
`249 - 1
`249 - 2
`249 - 3
`249 - 4
`249 - 5
`
`249 - 6
`
`257
`
`Page 6 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 6 of 28
`
`US 2017 / 0357478 A1
`
`Portable Multifunction Device
`
`Speaker 211
`
`Optical Sensor
`264
`
`90€
`
`500
`
`Proximity
`Sensor 266
`wwwwwwww - 505
`
`502
`
`??????????????????????
`
`Current Time 504
`Tuesday
`
`506
`
`Messages
`524
`
`Calendar
`
`999999999999
`
`Photos
`528
`
`Camera
`530
`
`mm
`
`© - 0 . 0 . 0 ,
`Maps
`Stocks
`536
`534
`wwwwwwwwwwwww
`0 . 0 0 . 0 . 0
`Notes
`Workout
`Support 542
`544
`
`IQRAM
`Online Video
`532
`
`Clock
`540
`
`nnnnnnnnn
`
`22222222
`
`Weather
`6 . 0 . 0 . 0
`538
`nnnnnn
`
`Settings
`546
`OOO
`
`Utlities
`
`iPod 522
`
`nnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn
`
`App Store
`
`Voice Memos
`
`iTunes
`QO
`510
`trychutna
`Browser 520
`
`514
`
`©
`
`Phone 516
`
`Mail 518
`
`Touch Screen 212 mm
`Accelerometer ( s )
`
`Microphone
`213
`
`304
`FIG . SA
`
`BOHO
`LOG wwwwwwwwwwwwwwwww
`
`Page 7 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 7 of 28
`
`US 2017 / 0357478 A1
`
`pa
`
`553
`ooooooooooooooooooo
`
`
`
`aurorasoro i a uransa
`
`
`
`an
`
`v
`
`- 562
`pane Tactile Output
`Generatoris , 457 | 552
`bananco Contact Intensity
`Sensor ( s ) 459 MO
`
`FIG . 5B
`
`Page 8 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 8 of 28
`
`US 2017 / 0357478 A1
`
`DEVICE
`600 -
`
`608
`
`602
`
`+ + +
`
`+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
`
`+
`
`+ +
`
`nnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn
`
`wwwwwwwwww
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`606
`FIG . 64
`
`Page 9 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 9 of 28
`
`US 2017 / 0357478 A1
`
`624
`
`618
`
`HTTTTTTTTTTTTTTTTTTTTTTTT
`
`. TTTTTTTTTTTTTTTTTTTTTTTTTTTT
`
`219
`
`waardoor
`
`sng BUS
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`* *
`
`634
`
`*
`
`*
`
`ACCELEROMETER
`
`
`
`GPS SENSOR
`
`4444
`
`FIG . 6B
`
`> 30
`
`*
`
`* *
`
`* * *
`
`*
`
`*
`
`GYROSCOPE
`DIRECTIONAL SENSOR
`
`1111111111111111111111111
`
`V638
`MOTION SENSOR
`
`019
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`?
`
`?
`
`?
`
`?
`
`?
`
`-
`
`? »
`
`
`
`
`
`www mon now we wo man non mom
`
`??? ?? ??? ??? ??? ??? ??
`
`???
`
`??? ??? ???? ??? ??? ?? ??? ??? ??? ???
`
`219 - 4 pogogogggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggg
`wwwwwwwwwwwwwwwwwwwwwwww
`- 4 months
`
`632 www
`
`
`
`
`
`INTENSITY
`PROCESSOR ( S ) MEMORY
`YYYYYYYYYYYYYYYYYYYYYYYYYYYY
`TOUCH
`
`
`av 4 any rowen Tanum
`DISPLAY
`
`1 / 0
`
`TTTTTTTTTTTTTTTTTTTTT
`
`wwww
`
`606 pm
`
`REELLEEEEEEEEE
`
`604
`
`600
`
`606
`
`INPUT
`INPUT
`608 608
`
`034 - COMMUNICATION
`
`Page 10 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 10 of 28
`
`US 2017 / 0357478 A1
`
`S
`
`w
`
`SISESEISISSE
`
`11111111111111
`
`Operating System
`Communication Module
`User Interface Module
`Applications
`Digital Assistant Module
`No Processing Module
`STT Processing Module
`ASR System ( s )
`Natural Language Processing Module
`Ontology
`Vocabulary
`User Data
`Dialogue Flow Processing Module
`Task Flow Processing Module
`Task Fow Models
`Service Processing Module
`Service Models
`Speech Synthesis Module
`
`SSSSSSSS
`
`Wwwwwwwwwwwwwwwww
`
`wwwwwwwwwwwwwwwwwse
`
`www
`
`738
`756
`mob 740
`
`Digital Assistant System
`700
`
`Memory 702
`
`704 mm Processor ( s )
`
`wwwww
`
`710
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwww wwwwwwwwwwwwwwwwwwwwwwww
`
`DISSISSINS
`
`NISSAN IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
`
`706
`
`mmmmmmmmmmm
`
`VO Interface
`110 Devices
`
`mm
`
`walios
`
`91
`
`708
`71 ?
`114
`
`Network Communications
`erface
`Wired Communications
`Port
`Wireless Circuitry
`
`www
`
`wwwwwwwwwwwwwwwwwwwwwwww
`
`nnnnnnnnnnnn
`
`FIG . 74
`
`Page 11 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 11 of 28
`
`US 2017 / 0357478 A1
`
`User Data
`
`mau
`
`AB } fia32
`
`760
`
`
`Phonetic
`
`Alphabet Conversion Module
`
`Phonetic Representation
`
`braun
`
`44
`
`OCL
`
`
`
`Digital Assistant
`
`
`
`STT Processing Module
`
`294 roll
`
`
`
`Token Sequence
`
`Module Natural Language Processing
`
`
`
`
`Ontology
`
`Service Processing Module . .
`
`Service Models
`
`Flow Processing Module from 754
`
`Task
`
`1053
`
`yser
`
`MO
`
`Models
`
`
`
`preso 734
`
`
`
`Structured Query
`
`m
`
`pour ? 40
`
`Dialogue Processing Module
`
`Speech Synthesis Module
`
`04 Processing { { ?da
`
`FIG . 7B
`
`2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222211221222212222122221222212222127
`
`Speech
`
`mm 728
`
`Text String
`
`ma
`
`Context
`
`Metadata
`
`
`
`Speech Input
`
`
`
`Text String
`
`nnnnnnnnnnnnnnnnnnnnnnnnn
`
`726
`
`Responses
`
`Delayed Responses
`
`Follow - Up
`poona
`
`Page 12 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 12 of 28
`
`US 2017 / 0357478 A1
`
`sunt
`
`where to see the
`
`Reminder
`lieuod
`
`
`
`
`
`corso con conta com cars and may even cuen
`
`
`
`
`
`wow
`
`
`VO VW
`
`u
`
`764
`
`pot
`
`er
`
`
`
`war un
`
`
`wach schon was a
`
`
`
`Set Reminder
`
`w
`
`* Bar
`
`com won won bio
`
`arre
`
`* * *
`
`Subject
`
`
`
`Date Time
`
`UW
`
`on
`war
`www com a
`nuran
`har an
`NOW
`
`n
`
`
`
`Restaurant Reservation
`
`KIMONTHNUNCIO
`
`S
`
`. com
`
`minnan
`
`
`
`Restaurant Reservation
`
`Domain
`
`Annunci
`
`at en
`
`som
`
`760
`
`w
`
`*
`
`vom
`
`a
`
`won
`
`m
`
`
`
`WOMEN ANNONCERNUNU
`
`N
`
`CONTRACT
`
`aar
`
`
`Party Size
`znamnou
`Restaurant
`
`wenn
`
`
`
`
`
`man non or
`
`
`
`
`
`ch warm te
`
`Location
`
`T
`
`met
`open
`the
`Baleno
`Sao
`ome
`and
`share
`results on
`Brosten
`maar
`
`# euoud
`
`FIG . 7C
`
`a
`
`0 w
`
`w
`
`mos
`no
`
`w
`
`noon s
`
`a
`
`
`
`Price Range
`
`nos
`avoro
`can
`which war
`mer nur
`
`Cuisine
`
`sarann
`
`het
`
`*
`
`294 292
`
`worden met een
`
`
`
`
`listen
`
`are a
`dows
`with
`moramo www
`
`
`the
`
`wwwt
`
`om
`
`nowe
`
`werden
`ter
`
`Page 13 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 13 of 28
`
`US 2017 / 0357478 A1
`
`
`
`tttttttttttttttttt LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL
`
`DEVICE 806
`
`W wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`.
`
`. . . . . .
`
`. . . . . . 0 , 00000000000000000UUUUUUUUUUUUUUU . : . : . : . : . : . : .
`
`FIG . 8A
`
`UUUUUUUUUUUUUUUUUU
`U
`11111111111111111111111111111111111111111111111111111111111111111
`
`?
`
`.
`
`w
`
`w
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`. 834
`
`
`
`
`
`find me a TV episode " " Hey Siri
`
`JER 800
`
`DEVICE 808
`DEVICE 804
`
`3132
`
`= = = =
`
`DEVICE 802
`
`bo
`
`w
`
`Page 14 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 14 of 28
`
`US 2017 / 0357478 A1
`
`DEVICE 806
`
`WWW
`
`51HED 242Y310
`17 .
`
`E
`
`w
`
`DEVICE 808
`DEVICE 804
`
`DEVICE 802
`AY DEVICE
`
`D
`
`FIG . 8B
`
`M
`
`111111111111111111111111111111111111111111111111111111111111111111111
`Y YYYYYYYY
`MYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`0000000000
`
`om vog
`
`812
`
`www
`
`w
`
`
`
`w " Hey Siri "
`
`wwwwwwww
`USER ??
`
`Page 15 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 15 of 28
`
`US 2017 / 0357478 A1
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`*
`
`w
`
`www
`
`DEVICE 806
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`4
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwy
`43000000000000000000000000000000000000000000
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`3300311010001100100110010110010101010
`
`FIG . 8C
`
`KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK
`DEVICE 808 " BE113517ED
`DEVICE 804
`DEVICE ( RAISED )
`moto
`
`T
`
`esz
`
`- - 802 DEVICE
`
`WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`ber
`num
`WWW
`
`814
`
`
`" Hey Siri " " Hey
`Siri "
`
`USER
`800
`008
`
`Page 16 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 16 of 28
`
`US 2017 / 0357478 A1
`
`904
`
`Room Find Speakers With EntityName = Living
`
`
`
`
`
`
`
`Device Identification Storage
`
`-
`
`942
`
`u id = 123456 - -
`
`016
`
`1111111
`
`w wuu04400402122222
`wwwwwwww
`
`Anywa
`LLLLLLLLLurrr
`
`Wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`940
`
`902
`
`960
`
`
`SetMedia PlayerQueue artist : / / store / 56989 AirplayRouteuid = 12456 1980
`ht
`
`h
`
`
`
`Virtual Assistant Server
`
`962
`
`= Ramones
`
`artist : / / store / 56989
`
`900
`
`906
`
`Media Identification
`Storage
`
`Ramones in
`my living room "
`" Play The
`
`w
`
`6 : 21M
`
`-
`
`concernance
`
`908
`
`FIG . 9A
`
`Page 17 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 17 of 28
`
`US 2017 / 0357478 A1
`
`904
`
`
`
`Device Identification Storage
`
`930
`
`912
`
`
`
`Device Tracking Storage
`
`946
`
`932
`
`V
`
`Find Speakers With EnityName = appleTV
`
`
`
`
`
`940
`
`902
`
`960
`
`Ramones
`
`SetMediaPlayerQueue artist : / / store / 56989 AirplayRouteuid = 12456
`
`X980
`
`uid = 123456
`
`
`
`Virtual Assistant Server
`
`962
`
`Media Identification Storage
`
`SIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
`
`Ramones on
`my Apple TV
`
`
`" Play The
`
`wwwwwwwwwwwwwwwwwwwww 214141
`
`wywoonwoveyouuuwwwwwwwwwwwww
`
`920 -
`
`908 -
`
`artist : / / store / 56989
`
`nnnnnnnnnnnnnnnnnnnnnnnnn
`
`FIG . 9B
`
`Page 18 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 18 of 28
`
`US 2017 / 0357478 A1
`
`904
`
`
`
`Device Identification Storage
`
`Wro
`
`NANUODAMAROSA
`942 Devices
`
`
`
`Find Speakers With Entity
`Name = appleTV
`List of
`
`948
`
`
`
`Assign Device to Room
`
`940 -
`
`
`
`Virtual Assistant Server
`
`962
`
`902
`
`960
`Ramones .
`
`SetMediaPlayerQueue artist : / / store / 56989 AirplayRouteUid = 12456
`
`1980
`Disambiguate Amongst
`
`Device List
`Disambiguation Response 946
`944
`
`wwwwwwwwww
`
`Gouwscoop
`
`34 : 41
`
`Ramones on my Apple
`TV1
`" Play The
`
`????????????
`
`)
`
`920
`
`908 -
`
`artist : / / store / 56989
`
`Media Identification Storage
`
`FIG . 9C
`
`Page 19 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 19 of 28
`
`US 2017 / 0357478 A1
`
`1000
`AT A FIRST ELECTRONIC DEVICE HAVING A MCROPHONE , SAMPLING , WITH THE MICROPHONE AT THE FIRST
`ELECTRONIC DEVICE , AN AUDIO INPUT
`1002
`
`*
`
`"
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`por me mw www mm ww
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww me
`DETERMINING , WITH THE FIRST
`ELECTRONIC DEVICE , WHETHER THE AUDIO INPUT COMPRISES A SPOKEN TRIGGER
`1004
`
`hem wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`m
`
`mm mm
`
`w
`
`m
`
`m
`m
`m
`m
`m
`m
`m
`m
`m
`m
`m
`m
`m
`m
`m
`m
`m
`mmmmmmmmmmmmmmmmmmmmmmmmmm
`IN ACCORDANCE WITH A DETERMINATION THAT THE AUDIO INPUT DOES NOT COMPRISE THE SPOKEN TRIGGER
`FOREGOING SROADCASTING , WITH THE FIRST ELECTRONIC DEVICE , THE FIRST SET OF ONE OR MORE VALUES
`1006
`m wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`BROADCASTING , WITH THE FIRST ELECTRONIC DEVICE , A FIRST SET OF ONE OR MORE VALUES BASED ON THE
`SAMPLED AUDIO INPUT
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`RECEMING , WITH THE FIRST ELECTRONIC DEVICE , A SECOND SET OF ONE OR MORE VALUES FROM A SECOND
`ELECTRONIC DEVICE , WHEREIN THE SECOND SET OF ONE OR MORE VALUES ARE BASED ON THE AUDIO INPUT
`
`L
`
`THE AUDIO INPUT BASED ON THE FIRST SET OF ONE OR MORE VALUES AND THE SECOND SET OF ONE OR MORE
`VALUES
`1012
`DETERMINING . WITH THE FIRST ELECTRONIC DEVICE . WHETHER AVALUE OF THE FIRST SET OF ONE OR MORE
`VALUES IS HIGHER THAN A CORRESPONDING VALUE OF THE SECOND SET OF ONE OR MORE VALUES
`- - - -
`-
`- - -
`- - - -
`- -
`- - -
`
`U
`
`U
`
`U
`
`U
`
`U
`
`U
`
`U
`
`U
`
`LAS LL LLL LLL LL
`
`LL
`
`10
`
`mmmmmmmmmm DETERMINING , WITH THE FIRST ELECTRONIC DEVICE . WHETHER THE FIRST ELECTRONIC DEVICE IS TO RESPOND TO
`mmmmmmm
`
`IN ACCORDANCE WITH A DETERMINATION THAT THE FIRST ELECTRONIC DEVICE IS TO RESPOND TO THE
`AUDIO INPUT , RESPONDING TO THE AUDIO INPUT
`1918
`ENTERING AN INACTIVE MODE 1
`1044
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`www
`
`www www ww ww ww ww ww ww w
`
`w
`
`w
`
`wall
`
`LLLLLLLLLLLL
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`IN ACCORDANCE WITH A DETERMINATION THAT THE FIRST ELECTRONIC DEVICE IS NOT TO RESPOND TO THE
`AUDIO INPUT , FOREGOING RESPONDING TO THE AUDIO INPUT
`1014
`www
`
`FIG . 104
`
`Page 20 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 20 of 28
`
`US 2017 / 0357478 A1
`
`1000
`( cont . }
`
`pas
`I
`
`E
`
`o
`
`1
`
`1
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`m ma
`
`w
`
`
`
`ww ww ww ww www comes
`
`w
`
`w
`w
`w
`w
`w
`w
`w
`w
`w
`w
`w
`w
`w
`w
`w
`w
`w
`w
`w
`w
`w
`w w
`w
`w w
`w
`w
`w w
`w
`w
`w
`w
`w
`w
`m
`IN ACCORDANCE WITH A DETERMINATION THAT THE FIRST ELECTRONIC DEVICE IS TO RESPOND TO THE AUDIO
`INPUT . DETERMINING , WITH THE FIRST ELECTRONIC DEVICE , WHETHER A TYPE OF THE FIRST ELECTRONIC DEVICE
`MEETS A REQUIREMENT OF THE TASK
`1018
`RECEMING , WITH THE FIRST ELECTRONIC DEVICE , DATA INDICATIVE OF THE REQUIREMENT OF THE TASK !
`FROM A SERVER
`1020
`
`0
`
`0
`
`m
`
`en
`
`
`??? ??
`
`w
`???
`
`w
`w
`??
`
`w
`????
`
`w
`w
`???
`??
`
`w
`???
`
`w
`????
`
`w
`????
`
`w
`??
`
`w
`
`?
`
`w
`????
`
`w
`???
`
`w
`w
`??
`
`?
`
`?
`
`w
`
`w
`w
`??
`
`?
`
`?
`
`w
`
`?
`
`w
`
`w
`??
`
`w
`
`?
`
`w
`?
`
`?
`
`w
`w
`??
`???
`
`w
`?
`
`w
`???
`
`w
`????
`
`w
`???
`
`w
`???
`
`w
`???
`
`w
`??
`
`w
`???
`
`w
`????
`
`w
`????
`
`w
`
`w
`
`2
`
`2
`
`2
`
`KK
`
`v
`
`u
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww was
`N
`ACCORDANCE WITH A DETERMINATION THAT THE TYPE OF THE FIRST ELECTRONIC DEVICE MEETS THE
`mm
`REQUIREMENT OF THE TASK , RESPONDING TO THE AUDIO INPUT
`1022
`
`?
`
`????
`
`????
`
`???
`
`???
`
`???
`
`?
`
`???
`
`????
`
`??
`
`??
`
`????
`
`???
`
`?
`
`?
`
`???
`
`?
`
`????
`
`????
`
`??? ???
`
`????
`
`??
`
`???
`
`????
`
`?
`
`?? ????
`
`?
`
`??
`
`????
`
`???
`
`?
`
`????
`
`????
`
`???
`
`??
`
`???
`
`???
`
`???
`
`????
`
`?
`
`??? ?
`
`V
`
`w
`???
`
`w
`???
`
`w
`???
`
`?
`
`?? ?? ?
`
`m
`
`m
`
`I
`
`22 22 222
`
`
`
`ww wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`
`
`mmmmm
`mm
`
`??????
`
`0
`
`w
`. .
`H roun .
`.
`.
`. .
`.
`.
`. .
`•
`•
`•
`•
`•
`.
`w
`. . .
`.
`.
`.
`.
`.
`.
`1 .
`2 À IN ACCORDANCEWTH A DETERMINATION THAT THE TYPE OF THE FIRST ELECTRONIC DEVICE DOES NOT MEET THE
`REQUIREMENT , DETERMINING , WITH THE FIRST ELECTRONIC DEVICE ,
`WHETHER THE SECOND DEVICE IS TO RESPOND TO THE AUDIO INPUT
`1024
`
`222
`202
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`m
`
`om
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`2 IN ACCORDANCE WITH A DETERMINATION THAT THE SECOND DEVICE IS NOT TO RESPOND TO THE AUDIO INPUT ,
`PROVIDING , WITH THE FIRST ELECTRONIC DEVICE , AN OUTPUT INDICATIVE OF AN ERROR
`200
`
`-
`
`o
`
`o
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`w
`
`IN ACCORDANCE WITH A DETERMINATION THAT THE SECOND DEVICE IS TO RESPOND TO THE AUDIO INPUT ,
`FOREGOING RESPONDING TO THE AUDIO INPUT WITH THE FIRST ELECTRONIC DEVICE
`1028
`www
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`FIG . JOB
`
`Page 21 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publication
`
`Dec . 14 , 2017 Sheet 21 of 28
`
`US 2017 / 0357478 A1
`
`{ { {
`( cont . )
`
`w
`
`-
`
`*
`
`-
`
`-
`
`ww ww ww w
`
`-
`
`-
`
`»
`
`»
`
`»
`
`»
`
`»
`
`»
`
`»
`
`»
`
`»
`
`IN ACCORDANCE WITH A DETERMINATION THAT THE FIRST ELECTRONIC DEVICE IS NOT TO RESPOND TO THE I
`AUDIO INPUT , DETERMINING , WITH THE FIRST ELECTRONIC DEVICE , WHETHER A TYPE OF THE FIRST
`ELECTRONIC DEVICE MEETS A REQUIREMENT OF THE TASK
`1030
`m
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`m
`
`?? ???
`
`??? ??
`
`w my m
`RECENING WITH THE FIRST ELECTRONIC DEVICE , DATA INDICATIVE OF THE REQUIREMENT OF THE TASK .
`FROM A SERVER
`w mm 0 m
`1032
`
`??
`
`*
`
`??? *
`????
`
`w
`
`w
`
`w
`
`w
`
`?? ??
`0
`0
`
`??? ???
`0
`0
`
`2
`
`0
`
`?
`
`???
`
`???
`0
`
`2
`
`? ?? ?? ??
`113 wu ( 1
`
`?
`313
`
`???
`w
`
`??
`0
`
`?
`3
`
`??
`3
`
`???
`4
`
`??
`3
`
`??
`3
`
`?
`
`3
`
`W
`
`?
`
`??
`
`??
`
`?
`
`??
`
`???
`
`?
`
`?
`
`???
`
`?
`
`?
`
`??
`
`??
`
`??
`
`??
`??
`ALLAX 1
`
`?
`
`??
`
`1
`
`???
`
`, ??
`1
`
`1
`
`om aan mm
`
`
`
`was anna
`
`Pere
`
`•
`
`on www www ww www ww ww www ww www mowa ww www www www www www www ww ww www ww www www www www ww ww www ww www www ww ww ww ww ww www www ww www
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`IN ACCORDANCE WITH A DETERMINATION THAT THE TYPE OF THE FIRST ELECTRONIC DEVICE DOES NOT
`MEET THE REQUIREMENT OF THE TASK , FOREGOING RESPONDING TO THE AUDIO INPUT WITH THE FIRST
`ELECTRONIC DEVICE
`1040
`•
`
`*
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`*
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`?? ?? ?? ?? ?? ?? ???
`
`w
`
`?
`
`?
`
`??
`
`?? ?? ?? ??
`
`?
`
`?
`
`?
`
`?
`
`? ???
`
`????
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?? ???? ?? ??? ????
`
`w
`
`w
`
`w
`
`??? ??? Recent
`
`man want
`
`IN ACCORDANCE WITH A DETERMINATION THAT THE TYPE OF THE FIRST ELECTRONIC DEVICE DOES NOT
`MEET THE REQUIREMENT OF THE TASK , DETERMINING WHETHER TO RESPOND TO THE AUDIO
`INPUT WITH THE FIRST ELECTRONIC DEVICE BASED ON THE FIRST SET OF ONE OR MORE VALUES , THE
`SECOND SET OF ONE OR MORE VALUES , AND THE REQUIREMENT OF THE TASK
`1034
`
`w
`
`w
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`uu wa www
`
`???
`
`
`
`???? ???
`
`???
`
`????
`
`??
`
`???
`
`????
`
`???
`
`??
`
`????
`
`???
`
`???
`
`???
`
`???
`
`???
`
`???
`
`???
`
`???
`
`IN ACCORDANCE WITH A DETERMINATION TO NOT RESPOND TO THE AUDIO INPUT WITH THE FIRST
`www .
`ELECTRONIC DEVICE , FOREGOING RESPONDING TO THE AUDIO INPUT WITH THE FIRST ELECTRONIC DEVICE
`me
`1038
`????
`???
`
`???
`
`???
`
`????
`
`???
`
`????
`
`????
`
`???
`
`
`
`??? ???
`
`?
`
`???
`
`???
`
`???
`
`???
`
`????
`
`???
`
`???
`
`????
`
`???
`
`?
`
`Wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww wwwwwwwwwwwwwwwwwwwwwwwwww
`
`IN ACCORDANCE WITH A DETERMINATION TO RESPOND TO THE AUDIO INPUT WITH THE FIRST ELECTRONIC
`DEVICE RESPONDING TO THE AUDIO INPUT WITH THE FIRST ELECTRONIC DEVICE
`1030
`
`*
`
`*
`
`*
`
`wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
`
`FIG . 100
`
`Page 22 of 64
`
`SONOS EXHIBIT 1003
`
`

`

`Patent Application Publicati0ll
`
`Dec . 14 , 2017 Sheet 22 of 28
`
`US 2017 / 0357478 Al
`
`RECEIVING , WITH THE ELECTRONIC DEVICE , DATA CORRESPONDING TO AN
`AUDIO INPUT FROM THE FIRST USER DEVICE
`? ?
`2
`
`?
`
`?
`
`?
`
`?
`
`?
`|
`
`?
`
`;
`
`?
`
`?
`
`??
`???
`rix
`
`?
`
`?
`
`???
`
`?
`
`??
`
`?
`
`; fi &
`$
`% if
`; £
`
`OBTAINING , WITH THE ELECTRONIC DEVICE , AN IDENTIFICATION OF A SECOND USER DEVICE
`BASED ON THE DATA CORRESPONDING TO THE AUDIO INPUT
`_ 134
`IDENTIFYING , WITH THE FIRST ELECTRONIC DEVICE , A TASK AND A PARAMETER BASED ON THE DATA !
`CORRESPONDING TO THE AUDIO INPUT
`?
`306
`
`??? ??
`
`?? ??
`
`
`
`?? ?? ??
`
`
`
`?
`
`???
`
`??
`
`???
`
`?
`
`???
`
`??
`
`
`
`?? ??
`
`?
`
`
`
`?? ?? ??
`
`
`
`??
`
`.
`
`?
`
`?
`
`?
`
`??
`
`
`
`?? ???
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`
`
`?? ???
`
`?
`
`?
`
`? ?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`? ?
`
`? ? ?
`
`? ? ?
`
`? ? ?
`
`?
`
`?
`
`. ?
`
`??
`
`?
`
`?
`
`?
`
`?
`
`?
`
`? ? ?
`
`? ?
`
`? ? ?
`
`? ?
`
`?
`
`?
`
`?
`
`?
`
`??
`
`?? ?
`
`??
`
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`??
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`TRANSMITTING TO A SECOND ELECTRONIC DEVICE , WITH THE FIRST ELECTRONIC DEVICE , A
`REQUEST , WHEREIN THE REQUEST INCLUDES THE PARAMETER IDENTIFYING A USER DEVICE TO
`PERFORM THE TASK
`11083
`? ??? ????
`??
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`??
`??
`??
`??
`??
`?
`??
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`RECEIVING , WITH THE SECOND ELECTRONIC DEVICE . THE REQUEST INCLUDING THE PARAMETER
`. 13
`
`??
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`??
`
`????? ???? ??? ???? ???? ???? ??? ???
`
`???? ??? ???? ????
`
`??? ???? ???? ??? ??? ???? ?????? ????
`
`??? ???
`
`???? ???? ???
`
`???? ???? ??? ???
`
`???? ??? ??? ???? ??
`
`??
`
`???? ????
`
`?
`
`?
`
`??
`
`???
`
`??
`
`?
`
`????
`
`???
`
`??
`
`?
`
`?
`
`??
`
`?
`
`???
`
`??
`
`???
`
`?
`
`?
`
`?
`
`?
`
`?
`
`??
`
`? ??
`
`?
`
`?
`
`???
`
`??
`
`?
`
`?
`
`?
`
`?
`
`??
`
`?
`
`?
`
`???? ?? ???
`
`???
`
`???? ??
`
`??
`
`??
`
`??
`
`??
`
`????
`?? ??
`
`????
`
`
`???
`??
`
`???
`??
`
`???
`?
`
`????
`?
`
`?
`????
`
`????
`??
`
`???
`??
`
`???
`??
`
`????
`??
`
`???
`??
`
`?
`??
`
`???
`??? ??
`
`?? ??
`
`??
`
`???
`???
`
`????
`??
`??
`???
`
`???
`
`???
`???
`
`????
`??
`
`???
`?
`
`??
`???
`
`????
`??
`
`????
`??
`
`???
`
`???
`
`??
`
`???
`
`???
`
`???
`
`??
`???
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`IN RESPONSE TO RECEIVING THE REQUEST INCLUDING THE PARAMETER DISCOVERING , WITH THE
`SECOND ELECTRONIC DEVICE , ONE OR MORE USER DEVICES
` 10 ? 1 _
`
`? DISCOVERING WITH THE SECOND ELECTRONIC DEVICE , A USER DEVICE ASSOCIATED WITH AN
`INDICATION OF A NAMED LOCATION OF A USER
`_ _ 1 ' 13
`
`
`
`??? ?? ??
`
`
`
`???
`??
`*
`
`???
`???
`
`???
`??
`.
`
`???
`?
`
`?
`???
`
`??
`???
`
`???
`??
`
`??
`????
`
`???
`??
`
`????
`??
`
`????
`??
`
`??
`?? ??
`
`
`
`??
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`???
`????
`?? ??
`
`???
`??
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`?
`??
`????
`??
`?
`? UPOATING THE RECORD , WITH THE SECOND ELECTRONIC DEVICE , BASED ON ATTRIBUTES OF THE
`ONE OR MORE DISCOVERED USER DEVICES
` ??? . 14 . . ? ? ? ? ?
`1 14
`
`????
`
`???
`
`?
`
`?
`
`
`
`??? ??
`
`???
`
`????
`
`???
`
`???
`
`????
`
`???
`
`???
`
`???
`
`???
`
`???
`
`????
`
`?????
`
`
`
`
`
`? ????
`
`???
`
`???
`
`????
`
`?
`
`
`
`?? ??
`
`?
`
`?
`
`
`
`???? ??
`
`?
`
`???
`
`???
`
`???
`
`???
`
`????
`
`? CACHING , WITH THE SECOND ELECTRONIC DEVICE , ATTRIBUTES OF THE DISCOVERED ONE OR MORE
`USER DEVICES
`? ? ? ?
`
`??
`
`???
`
`???
`
`???
`
`???
`
`???
`
`???
`
`???
`
`???
`
`???
`
`???
`
`???
`
`???
`
`???
`
`???
`
`???
`
`???
`
`??????
`
`???
`
`????
`
`???
`
`???
`
`????
`
`???
`
`???
`
`????
`
`???
`
`??????
`
`???
`
`??
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`??
`
`??
`
`?
`
`?
`
`?
`
`?
`
`??
`
`?
`
`?
`
`?
`
`?
`
`? ?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`