Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 1 of 27 Page ID #:85
`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 1 of 27 Page ID #:85
`
`
`
`
`
`
`
`
`
`EXHIBIT D
`EXHIBIT D
`
`
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 2 of 27 Page ID #:86
`
`US010325486B2
`
`( 12 ) United States Patent
`Arling
`
`( 10 ) Patent No . : US 10 , 325 , 486 B2
`( 45 ) Date of Patent :
`* Jun . 18 , 2019
`
`( * ) Notice :
`
`( 54 ) SYSTEM AND METHOD FOR OPTIMIZED
`APPLIANCE CONTROL
`( 71 ) Applicant : Universal Electronics Inc . , Santa Ana ,
`CA ( US )
`( 72 ) Inventor : Paul D . Arling , Irvine , CA ( US )
`( 73 ) Assignee : Universal Electronics Inc . , Santa Ana ,
`CA ( US )
`Subject to any disclaimer , the term of this
`patent is extended or adjusted under 35
`U . S . C . 154 ( b ) by 0 days .
`This patent is subject to a terminal dis
`claimer .
`( 21 ) Appl . No . : 15 / 789 , 547
`( 22 ) Filed :
`Oct . 20 , 2017
`Prior Publication Data
`( 65 )
`US 2018 / 0040237 A1 Feb . 8 , 2018
`Related U . S . Application Data
`Continuation of application No . 15 / 259 , 847 , filed on
`Sep . 8 , 2016 , now Pat . No . 9 , 842 , 492 , which is a
`( Continued )
`
`( 63 )
`
`( 51 ) Int . Ci .
`( 2006 . 01 )
`GO8C 17 / 02
`HO4N 21 / 422
`( 2011 . 01 )
`( Continued )
`( 52 ) U . S . CI .
`CPC . . . . . . . . . . . . . G08C 17 / 02 ( 2013 . 01 ) ; G08C 23 / 04
`( 2013 . 01 ) ; H04N 21 / 42226 ( 2013 . 01 ) ;
`( Continued )
`Field of Classification Search
`CPC . . . . GO8C 17 / 02 ; G08C 23 / 04 ; GO8C 2201 / 92 ;
`GO8C 2201 / 70 ; G08C 2201 / 30 ;
`( Continued )
`
`( 58 )
`
`( 56 )
`
`EP
`wo
`
`References Cited
`U . S . PATENT DOCUMENTS
`9 / 1999 Adolph et al .
`5 , 959 , 539 A
`6 , 160 , 491 A
`12 / 2000 Kitao et al .
`( Continued )
`FOREIGN PATENT DOCUMENTS
`11 / 2006
`1722341 A1
`2011 / 053008 A2
`5 / 2011
`
`OTHER PUBLICATIONS
`ISA / US , Int . Search Report and Written Opinion of the Int . Search
`ing Authority issued on Int . Appin . No . PCT / US14 / 38151 , received
`Jun . 27 , 2014 , 10 pages .
`
`( Continued )
`Primary Examiner — Adnan Aziz
`( 74 ) Attorney , Agent , or Firm - Greenberg Traurig , LLP
`ABSTRACT
`( 57 )
`In response to a detected presence of an intended target
`appliance within a logical topography of controllable appli
`ances identity information associated with the intended
`target appliance is used to automatically add to a graphical
`user interface of a controlling device an icon representative
`of the intended target appliance and to create at a Universal
`Control Engine a listing of communication methods for use
`in controlling corresponding functional operations of the
`intended target appliance . When the icon is later activated ,
`the controlling device is placed into an operating state
`appropriate for controlling functional operations of the
`intended target appliance while the Universal Control
`Engine uses at least one of the communication methods to
`transmit at least one command to place the intended target
`appliance into a predetermined operating state .
`9 Claims , 14 Drawing Sheets
`
`10S ,
`- www . oe
`
`- - - -
`
`-
`
`119 :
`
`YY106
`
`- - 420
`
`! 1
`
`2
`
`-
`
`3 -
`
`. .
`
`. 29
`
`* *
`
`*
`
`406
`
`- 112
`
`- - 103
`
`. . . 204
`
`212
`
`inierist
`
`90
`
`- 202
`
`. . - 206
`
`- 2017
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 3 of 27 Page ID #:87
`
`US 10 , 325 , 486 B2
`Page 2
`
`( 51 )
`
`( 52 )
`
`( 58 )
`
`( 56 )
`
`Related U . S . Application Data
`continuation of application No . 14 / 136 , 023 , filed on
`Dec . 20 , 2013 , now Pat . No . 9 , 449 , 500 , which is a
`continuation - in - part of application No . 13 / 899 , 671 ,
`filed on May 22 , 2013 , now Pat . No . 9 , 437 , 105 , which
`is a continuation of application No . 13 / 657 , 176 , filed
`on Oct . 22 , 2012 , now Pat . No . 9 , 215 , 394 .
`( 60 ) Provisional application No . 61 / 552 , 857 , filed on Oct .
`28 , 2011 , provisional application No . 61 / 680 , 876 ,
`filed on Aug . 8 , 2012 .
`Int . CI .
`G08C 23 / 04
`( 2006 . 01 )
`( 2011 . 01 )
`H04N 21 / 4363
`U . S . CI .
`CPC . . . . . G08C 2201 / 20 ( 2013 . 01 ) ; G08C 2201 / 30
`( 2013 . 01 ) ; G08C 2201 / 30 ( 2013 . 01 ) ; G08C
`2261 / 70 ( 2013 . 01 ) ; G08C 2201 / 92 ( 2013 . 01 ) ;
`G08C 2201 / 93 ( 2013 . 01 ) ; H04N 21 / 42225
`( 2013 . 01 ) ; H04N 21 / 4363 ( 2013 . 01 )
`Field of Classification Search
`CPC . . . . . . . . . . . . G08C 2201 / 93 ; G08C 2201 / 40 ; G08C
`2201 / 20 ; H04N 21 / 42226 ; H04N
`21 / 4363 ; H04N 21 / 42225
`See application file for complete search history .
`References Cited
`U . S . PATENT DOCUMENTS
`6 , 259 , 892 B1
`7 / 2001 Helferich
`6 , 529 , 556 B13 / 2003 Perdue et al .
`6 , 968 , 399 B2 11 / 2005 Noda et al .
`7 , 379 , 778 B2
`5 / 2008 Hayes et al .
`7 , 436 , 346 B2 *
`10 / 2008 Walter
`7 , 519 , 393 B2
`4 / 2009 Bahl et al .
`7 , 589 , 642 B1
`9 / 2009 Mui
`7 , 814 , 516 B2 *
`10 / 2010 Stecyk
`8 , 040 , 888 B1 10 / 2011 MacAdam et al .
`8 , 218 , 090 B2 *
`7 / 2012 Yee . . . . . .
`8 , 269 , 892 B2
`9 / 2012 Asada et al .
`8 , 373 , 556 B2
`2 / 2013 LaLonde et al .
`8 , 477 , 179 B2
`7 / 2013 Tatsuta et al .
`8 , 633 , 986 B1
`1 / 2014 Hughes
`8 , 810 , 732 B1
`8 / 2014 Bozarth et al .
`8 , 839 , 334 B2
`9 / 2014 Lee
`8 , 881 , 205 B2 11 / 2014 Friedman
`9 , 900 , 657 B2 *
`2 / 2018 Hong
`2003 / 0095156 A15 / 2003 Klein et al .
`2004 / 0163073 AL
`8 / 2004 Krzyzanolvski et al .
`2004 / 0210933 Al 10 / 2004 Dresti et al .
`2004 / 0255329 Al 12 / 2004 Compton et al .
`2005 / 0028208 Al
`2 / 2005 Eilis
`2005 / 0195823 AL
`9 / 2005 Chen et al .
`2006 / 0146184 Al 7 / 2006 Gillard et al .
`2006 / 0168618 A1
`7 / 2006 Choi
`2006 / 0197753 A1
`9 / 2006 Hotelling
`2006 / 0227032 Al 10 / 2006 Vidal
`2007 / 0165555 Al
`7 / 2007 Deng
`2007 / 0220150 Al
`9 / 2007 Garg
`2007 / 0225828 AL 9 / 2007 Huang
`2007 / 0229465 Al 10 / 2007 Sakai
`2007 / 0292135 Al 12 / 2007 Guo et al .
`2008 / 0005764 Al
`1 / 2008 Arling et al .
`2008 / 0120673 A1 5 / 2008 Dong et al .
`2008 / 0168519 AL
`7 / 2008 Rao et al .
`2008 / 0187028 A1
`8 / 2008 Lida
`2008 / 0278567 A1 11 / 2008 Nakajima
`2009 / 0015723 AL
`1 / 2009 Doumuki
`
`. . . . . . . . . .
`
`G08C 17 / 02
`341 / 174
`
`. H04L 12 / 2805
`340 / 12 . 53
`H04N 5 / 44
`348 / 553
`
`. . HO4N 214622
`
`2009 / 0156051 A1 6 / 2009 Doyle et al .
`2009 / 0167555 Al
`7 / 2009 Kohanek
`2009 / 0207039 Al 8 / 2009 Haijima
`2009 / 0239587 A1 9 / 2009 Negron et al .
`2009 / 0248909 Al 10 / 2009 Hironaka et al .
`2010 / 0079682 A1 4 / 2010 Martch
`2010 / 0134317 A1 6 / 2010 Breuil et al .
`2010 / 0138764 A1 6 / 2010 Hatambeiki et al .
`2010 / 0157169 A1 6 / 2010 Yoshida et al .
`2010 / 0177245 A1
`7 / 2010 Ohnuma et al .
`2010 / 0271560 A1 10 / 2010 Higuchi
`2010 / 0328547 Al 12 / 2010 Mayorga
`. . . . . . . . G05B 19 / 0426
`2010 / 0332979 A1 *
`12 / 2010 Xu
`715 / 704
`2011 / 0102230 Al
`5 / 2011 Vergis et al .
`2011 / 0156944 Al 6 / 2011 Ward et al .
`2011 / 0273287 Al 11 / 2011 Lalonde et al .
`2011 / 0274008 A1 11 / 2011 Lida
`2011 / 0283129 Al 11 / 2011 Guillerm
`2011 / 0285818 A1 11 / 2011 Park et al .
`2011 / 0289113 AL 11 / 2011 Arling et al .
`2012 / 0013807 AL
`1 / 2012 Arora et al .
`2012 / 0084452 A
`4 / 2012 Pettit et al .
`2012 / 0144299 A1 *
`6 / 2012 Patel
`. . . . . . . . . . . . . . . GO6F 3 / 0488
`715 / 702
`2012 / 0173003 AL
`7 / 2012 Kim
`2012 / 0236161 A1
`9 / 2012 Kwon et al .
`2012 / 0242526 AL
`9 / 2012 Perez et al .
`2012 / 0249690 Al 10 / 2012 Chardon et al .
`2012 / 0249890 A1 * 10 / 2012 Chardon
`2012 / 0274547 AL 11 / 2012 Raeber et al .
`2012 / 0274857 AL 11 / 2012 Maxwell et al .
`2012 / 0278693 AL 11 / 2012 Black et al .
`2012 / 0297040 AL 11 / 2012 Amaro
`2012 / 0291128 A1 12 / 2012 2012 - 11 - 15
`2012 / 0330943 Al 12 / 2012 Weber et al .
`2013 / 0107131 A1 5 / 2013 Barnett et al .
`2013 / 0249679 Al
`9 / 2013 Arling
`2014 / 0085059 Al 3 / 2014 Chen et al .
`2014 / 0235265 Al 8 / 2014 Slupik
`2014 / 0235526 AL
`8 / 2014 Slupik
`
`. . . . HO4N 5 / 44
`348 / 734
`
`OTHER PUBLICATIONS
`United States Patent and Trademark Office , Final Office Action
`issued on U . S . Appl . No . 15 / 900 , 232 , Notification dated Dec . 13 , 15
`pgs . .
`United States Patent and Trademark Office , Non - Final Office Action
`issued on U . S . Appl . No . 15 / 900 , 342 , Notification dated May 9 ,
`2018 , 14 pgs .
`United States Patent and Trademark Office , Non - Final Office Action
`issued on U . S . Appl . No . 15 / 900 , 089 , Notification dated May 10 ,
`2018 , 13 pgs .
`United States Patent and Trademark Office , Non - Final Office Action
`issued on U . S . Appl . No . 15 / 899 , 971 , Notification dated Apr . 19 ,
`2018 , 11 pgs .
`ISA / US , Int . Search Report and Written Opinion of the Int . Search
`ing Authority issued on Int . Appin . No . PCT / US12 / 62161 , received
`Jan . 23 , 2013 , 12 pages .
`ISA / US , Int . Search Report and Written Opinion of the Int . Search
`ing Authority issued on Int . AppLn . No . PCT / US14 / 38151 , received
`Jun . 27 , 2014 , 10 pages .
`European Patent Office , extended European Search Report issued on
`European patent application No . 12844121 , 9 , dated Mar . 5 . 2015 , 6
`pages .
`European Patent Office , extended European Search Report issued on
`European patent application No . 14801064 . 8 , dated Apr . 18 , 2016 ,
`8 pages .
`European Patent Office , extended European Search Report issued on
`European patent application No . 14872863 . 7 , dated Nov . 25 , 2016 ,
`8 pages .
`United States Patent and Trademark Office , Final Office Action
`issued on U . S . Appl . No . 15 / 626 , 357 , Notification dated Nov . 3 ,
`2017 , 10 pages .
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 4 of 27 Page ID #:88
`
`US 10 , 325 , 486 B2
`Page 3
`
`( 56 )
`
`References Cited
`OTHER PUBLICATIONS
`United States Patent and Trademark Office , Final Office Action
`issued on U . S . Appl . No . 15 / 789 , 547 , Notification dated Jan . 25 ,
`2018 , 15 pages .
`United States Patent and Trademark Office , Final Office Action
`issued on U . S . Appl . No . 15 / 900 , 342 , Notification dated Nov . 29 ,
`2018 , 19 pgs .
`United States Patent and Trademark Office , Non - Final Office Action
`issued on U . S . Appl . No . 15 / 900 , 232 , Notification dated May 17 ,
`2018 , 14 pgs .
`* cited by examiner
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 5 of 27 Page ID #:89
`
`U . S . Patent
`
`Jun . 18 , 2019
`
`Sheet 1 of 14
`
`US 10 , 325 , 486 B2
`
`122 : 0
`
`122
`
`901
`
`- -
`
`- - -
`
`- - - - - - - -
`
`- -
`
`- -
`
`- - - -
`
`- -
`
`- -
`
`- - I - - - -
`
`- - - -
`
`-
`
`- - -
`
`- - -
`
`- - -
`
`106
`
`Q
`ole
`
`Www
`
`TTTTTTTTTTTTT
`
`09
`
`Coord
`
`108
`
`- : 09
`
`w
`
`114
`
`114 1
`
`100
`116
`
`ann
`
`104 . -
`
`. . 102
`
`Figure 1
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 6 of 27 Page ID #:90
`
`U . S . Patent
`
`Jun . 18 , 2019
`
`Sheet 2 of 14
`
`US 10 , 325 , 486 B2
`
`- - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - • - -
`
`- - -
`
`-
`
`112
`
`- - - - - - - -
`
`-
`
`-
`
`- - -
`
`- - -
`
`- - -
`
`- -
`
`- - - - -
`
`-
`
`-
`-
`-
`- - - -
`
`-
`
`-
`
`-
`-
`- - -
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`-
`
`-
`
`-
`
`108
`
`+ + +
`
`+ + + + + + + + + +
`
`deze
`+ + + + +
`
`*
`
`wwwwww
`+
`+ +
`+ + + + + +
`
`+ + + + + + +
`
`+ +
`
`210
`
`208
`
`1
`
`freni
`
`ZIZ
`
`200 .
`
`Figure 2
`
`204
`
`206
`
`- - - 207
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 7 of 27 Page ID #:91
`
`U . S . Patent
`
`Jun . 18 , 2019
`
`Sheet 3 of 14
`
`US 10 , 325 , 486 B2
`
`werererererererererer AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
`
`OOO
`
`108
`
`- -
`
`-
`
`-
`
`-
`
`-
`
`310 Amand
`
`118 1
`
`N
`
`104
`
`+
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`- -
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`- - - -
`
`- - - - - -
`
`- - - - - -
`
`- - -
`
`-
`
`901
`
`( e . g . Internet ,
`cable piant , etc . )
`
`- - - 206
`
`had 1
`
`100
`
`20071
`
`Figure 3
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 8 of 27 Page ID #:92
`
`U . S . Patent
`
`Jun . 18 , 2019
`
`Sheet 4 of 14
`
`US 10 , 325 , 486 B2
`
`108
`
`*
`
`. . . , - , - , -
`
`. . .
`
`V
`
`A M
`
`.
`
`*
`
`* *
`
`. * . . .
`
`*
`
`1
`
`420
`
`LLLLLL
`MIN
`
`TTTRICI
`C
`
`408
`
`406
`
`W
`
`- 118
`
`.
`
`212
`
`Internet
`
`402
`
`Figure 4
`
`. . .
`
`,
`
`2017
`
`III
`
`204
`
`206
`
`- 207
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 9 of 27 Page ID #:93
`
`U . S . Patent
`
`Jun . 18 , 2019
`
`Sheet 5 of 14
`
`US 10 , 325 , 486 B2
`
`W
`
`VVVVVVVVVVVyvyyyyyyyyyy
`
`External
`Ajdons jamod
`
`
`114
`
`516
`
`811
`208
`
`510
`
`Soooooooooooo
`
`WW
`
`506
`
`???????????????????? KOKKOKKOKKOND
`
`0110000000000000
`
`Processor
`
`199909000000000000000000000000000000000000000000000000
`
`W
`
`OOOOOO
`
`502
`
`OOOOOOOOOOOOOOOOOOOOOOOOOO
`
`Memory
`
`g inginiginginigisinginginiginginigisinginginiginginigisingil
`swinginginiginginigisinginginiginginigisin
`
`Ethernet
`
`Other jo
`
`512
`
`514
`
`504
`
`Figure 5
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 10 of 27 Page ID #:94
`
`atent
`
`Jun . 18 , 2019
`
`Sheet 6 of 14
`
`US 10 , 325 , 486 B2
`
`620
`
`610
`
`106
`
`- 408
`
`Volume ; mute
`622 POWERS
`624
`Savanced
`functions
`
`en tenenten
`
`ater
`
`w
`
`TAO :
`
`achBAS 02 . 01
`WW
`
`power play
`
`laadsledelse
`
`Volume ; Mote
`
`POWAC
`wo
`
`*
`
`power play
`pause , Ff , RW
`
`des
`
`wwwwwwwwwwwwwwww
`
`- 514
`
`506
`
`508
`Ethernet for
`
`when
`
`HOM
`
`* * *
`
`dan m
`
`e
`
`mang
`
`Other
`658
`Universal control engine core 650
`
`660
`
`IR recv
`
`504
`
`512
`
`1
`
`OKORONOODOOROOROOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
`
`210
`
`. . . 510
`
`* * *
`
`* 200
`
`104
`
`102
`
`Figure 6
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 11 of 27 Page ID #:95
`
`atent
`
`Jun . 18 , 2019
`
`Sheet 7 of 14
`
`US 10 , 325 , 486 B2
`
`716
`
`702
`
`704
`
`720
`
`- - 710
`
`Appliance
`STB / DVR
`
`CEC
`n / a
`
`CECC ECO
`
`AVR
`
`CEC
`CEC
`
`vedee vee vere
`
`etc .
`
`DVD
`
`CEC
`n / a
`
`I R
`n / a
`
`W
`
`WW
`
`Power on
`Power off
`Volume up
`Volume down
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`CEC
`
`1
`
`?
`
`?
`
`?
`
`?
`
`R
`
`ivivivivivivivivivivivivivivivivivivivivivivi
`
`Y YYY
`
`- -
`
`-
`
`-
`
`-
`
`-
`
`- - -
`
`-
`
`-
`
`-
`
`Function
`
`*
`
`H
`
`UHUHUHUHUHUHUHUHUHUHUHUHUHUHUHUHUHUHUHUHUHUHUHUHUHU
`
`n / a
`
`n / a
`
`CEC
`
`CEC
`
`R
`
`Pause
`
`???????????????????????????????????????????????????????????????????????
`
`Rew
`Sound field A
`Sound field B
`Input 1
`Input 2
`
`ETC . . . . .
`
`CEC
`CEC
`CEC
`
`errrr
`
`1
`?
`
`n / a
`n / a
`
`Wh
`
`Muut
`
`706
`
`712
`
`714 1
`
`718
`
`Figure 7
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 12 of 27 Page ID #:96
`
`atent
`
`Jun . 18 , 2019
`
`Sheet 8 of 14
`
`Sheet 8 of 14
`
`US 10 , 325 , 486 B2
`
`- 814
`
`816
`
`wwwwwwwwww
`
`Power
`supply
`
`USB bo0000000000000
`
`Processor
`
`818
`
`820
`
`822
`
`824
`
`IS
`
`Blue
`tooth
`
`Camera
`
`OOOOOOO
`
`Acceler .
`ometer ,
`GPS ,
`ambient
`light
`sensor ,
`NFC ,
`etc .
`
`908
`
`Display
`interface
`
`Touch
`screen
`interface
`
`LOD & touch
`sonce
`
`* * * *
`
`* * * * * * * * * *
`
`808
`
`MAN
`Hard keys
`
`Built - in memory
`
`802
`
`KIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIOIO
`
`O
`
`D
`
`External memory
`
`804
`
`- 812
`
`202
`
`Figure 8
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 13 of 27 Page ID #:97
`
`atent
`
`Jun . 18 , 2019
`
`Sheet 9 of 14
`
`US 10 , 325 , 486 B2
`
`Appliance command
`database server
`
`UCE programming
`
`906
`
`Scan for
`connected CEC
`appliances
`
`2910
`Scan for other
`AOMI attached
`appliances
`
`Set up apo
`
`Set up UCE
`
`Request user is 902
`power all
`????????????
`appliances on
`
`Determine CEC
`enabled
`appliances
`
`Determine
`HDMI ( non - CEC )
`appliances
`wwny 912
`Display list of
`located
`appliances
`
`1
`
`207
`
`- - - - - - - - -
`
`9181
`
`924
`
`Retrieve
`command sets
`for appliances
`
`Retrieve
`alternate
`comimand sets
`
`Build command
`Matrix
`
`Receive
`codesets , ISSUE
`test commands
`m7930
`
`Store final
`command
`matrix
`
`my 914 1
`User identifies
`?????
`non - CEC and non
`HDMI appliances
`Obtain
`command sets
`for appliances
`. . . 920 9201
`www
`Test and
`confirm codsets
`
`916 |
`
`www
`
`9261
`
`Build command
`matrix , transfer
`to UCE
`
`Configure
`activities
`
`Done
`
`Figure 9
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 14 of 27 Page ID #:98
`
`U . S . Patent
`
`Jun 18 , 2019
`
`Sheet 10 of 14
`
`US 10 , 325 , 486 B2
`
`Set up app
`
`Configure
`activities
`
`Present list of
`activities
`
`User selects an
`activity
`
`{ { { 35
`
`Identity content
`source
`
`{ { { }
`
`3012
`
`Identify video
`rendering
`device and
`content route
`
`Identity audio
`rendering
`device and
`content route
`
`Build activity
`matrix
`
`Test activity
`
`Works ?
`
`Download
`activity inatrix
`to UCE
`
`332 ???
`
`More ?
`
`Done
`
`AAAA
`
`* * * * *
`
`- -
`
`- -
`
`- -
`
`- - - -
`
`-
`
`- - - -
`
`* * * * *
`
`* *
`
`* *
`
`*
`
`* *
`
`*
`
`* *
`
`* * *
`
`UCE programming
`
`316
`
`issue commands
`and report
`status as
`required
`
`. ? ?
`
`Store finalized
`activity nátrix S
`
`Figure 10
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 15 of 27 Page ID #:99
`
`atent
`
`Jun . 18 , 2019
`
`Sheet 11 of 14
`
`US 10 , 325 , 486 B2
`
`- - 1101
`
`1102
`
`1104
`
`w
`
`Appliance
`Power
`Input
`
`UU
`
`Volume
`Transport
`
`State
`1106 Function
`
`1108
`
`Activity = " Listen to music " .
`Activity = " Watch a movie "
`Activity - " Watch TV "
`TV
`STB / DVR
`On
`HDMI 1
`
`-
`
`S / PDIF2
`
`-
`
`Active
`
`1 kad
`
`1112
`
`1114 1
`
`Figure 11
`
`1110 1
`youž mod 1118
`
`1200
`
`1
`
`DVD
`On
`
`AVR
`? ?
`? ?
`
`1210
`- - 1212
`
`1207 -
`
`Current appliance state
`TV
`STB / DVR
`
`by Appliance
`Power
`
`State {?? , HOMI2
`
`1206
`
`1208
`
`1204
`
`Figure 12
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 16 of 27 Page ID #:100
`
`atent
`
`Jun . 18 , 2019
`
`Sheet 12 of 14
`
`US 10 , 325 , 486 B2
`
`???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
`
`Retrieve corresponding
`data element from
`alternate command
`matrix
`
`MMM
`
`. . . 1316
`
`command request
`
`. m . . . . 1300
`Retrieve corresponding
`data element from
`preferred command
`matrix
`
`Null entry ?
`
`1302
`
`1304
`
`Transmit function
`request to appliance
`using indicated method
`
`1306
`
`Protocol supports
`confirmation ?
`
`N
`
`- - 1308
`
`Function
`confirmed ?
`
`????????????????
`
`OIETS
`
`Alternate
`method
`available ?
`
`1312
`
`wirini
`
`Generate error message 1314
`
`????????????????????????????????????????????????
`
`Done
`
`Figure 13
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 17 of 27 Page ID #:101
`
`U . S . Patent
`
`Jun . 18 , 2019
`
`Sheet 13 of 14
`
`US 10 , 325 , 486 B2
`
`" Watch IV
`activity request
`
`Access appliance
`state matrix for
`
`TV powered on ?
`
`- 1400
`
`1402
`
`Send TV " Power on "
`( CEC )
`
`Set transport
`commands = OVR
`
`VYYYYYYYYYYYYY
`
`Send TV " Mute "
`
`YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
`
`TV input =
`HDMI1 ?
`
`Send IV " inputi
`( CEC )
`
`HV
`
`VAVAVAVALAVALAVAVAVAVAVA
`
`STB / DVR
`powered on ?
`
`DVD poweredN
`on ?
`1420
`
`Send OVD " Power
`off " ( CEC )
`
`Revyenyewe
`
`Yayyy
`
`Send AVA
`" Power on "
`
`Send AVR
`input
`S / PDIF2
`
`1424
`
`1408
`
`1410
`
`1422
`
`1412
`
`~ 1414
`
`AUUUUUUUUUUUUUUUUUU .
`
`UUUUU .
`
`YUUUUUUUUUUUU .
`
`Send S7B " Power
`on " ( CEC )
`
`Set volume
`commands = AVR
`
`1416
`
`Done
`
`Figure 14
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 18 of 27 Page ID #:102
`
`atent
`
`Jun . 18 , 2019
`
`Sheet 14 of 14
`
`US 10 , 325 , 486 B2
`
`Set up app
`
`VVVANNYA
`
`Set up UCE
`
`, 1502
`1504
`
`1506
`
`1508
`
`Determine CEC
`or iP enabled
`appliances
`
`Scan for
`connected CEC
`appliances
`
`-
`
`con Metadata
`available from
`appliance ?
`
`V
`
`*
`
`1510 |
`
`1520
`
`l
`
`Retrieve icon
`metadata for
`appliances
`
`Retrieve auto
`generated
`macros
`
`Build user
`profile
`
`1524
`
`Determine
`HDMI ( non - CEC )
`appliances
`
`Display list of
`ocated
`appliances
`
`User identifies
`non - c & C and non
`HOM appliances
`
`* *
`
`*
`
`mm
`Obtain icon
`metadata for
`appliances
`
`Scan for other
`HDMI attached
`appliances
`
`1512
`
`1514
`
`1516
`
`1518
`
`1526
`
`1528
`
`Send to smart
`device
`
`Test 130TOS
`
`Macro worked ?
`
`1530
`
`Done
`
`Figure 15
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 19 of 27 Page ID #:103
`
`US 10 , 325 , 486 B2
`
`5
`
`remote control , smart phone , tablet computer , etc . , and in
`SYSTEM AND METHOD FOR OPTIMIZED
`APPLIANCE CONTROL
`particular to methods for taking advantage of improved
`appliance control communication methods and / or command
`RELATED APPLICATION INFORMATION
`formats in a reliable manner which is largely transparent to
`a user and / or seamlessly integrated with legacy appliance
`control technology .
`This application claims the benefit of and is a continuation
`To this end , the instant invention comprises a modular
`of U . S . application Ser . No . 15 / 259 , 847 , filed on Sep . 8 ,
`hardware and software solution , hereafter referred to as a
`2016 , which application claims the benefit of and is a
`Universal Control Engine ( UCE ) , which is adapted to pro
`continuation of U . S . application Ser . No . 14 / 136 , 023 , filed
`vide device control across a variety of available control
`on Dec . 20 , 2013 , which application claims the benefit of
`and is a continuation - in - part of U . S . application Ser . No . " methodologies and communication media , such as for
`13 / 899 , 671 , filed on May 22 , 2013 , which application claims
`example various infrared ( IR ) remote control protocols ;
`the benefit of and is a continuation of U . S . application Ser .
`Consumer Electronic Control ( CEC ) as may be implemented
`No . 13 / 657 , 176 , filed on Dec . 22 , 2012 , which application
`over a wired HDMI connection ; internet protocol ( IP ) , wired
`claims the benefit of U . S . Provisional Application No .
`or wireless ; RF4CE wireless , Bluetooth ( BT ) wireless per
`61 / 552 , 857 , filed Oct . 28 , 2011 , and U . S . Provisional Appli - 15 sonal area network ( s ) ; UPnP protocol utilizing wired USB
`cation No . 61 / 680 , 876 , filed Aug . 8 , 2012 , the disclosures of
`connections ; or any other available standard or proprietary
`which are incorporated herein by reference in their entirety
`appliance command methodology . Since each individual
`This application is also related to U . S . patent application
`control paradigm may have its own strengths and weak
`Ser . No . 12 / 621 , 277 , filed on Nov . 18 , 2009 and entitled
`nesses , the UCE may be adapted to combine various control
`“ System and Method for Reconfiguration of an Entertain - 20 methods in order to realize the best control option for each
`individual command for each individual device .
`ment System Controlling Device , ” which in turn is a con
`The UCE itself may be adapted to receive commands
`tinuation - in - part of U . S . patent application Ser . No . 12 / 569 ,
`from a controlling device , for example , a conventional
`121 ( now U . S . Pat . No . 8 , 243 , 207 ) , filed on Sep . 29 , 2009
`remote control or a remote control app resident on a smart
`and entitled “ System and Method for Activity Based Con
`figuration of an Entertainment System » the disclosures of 25 device such as a phone or tablet , etc . , utilizing any conve
`nient protocol and command structure ( IR , RF4CE , BT ,
`which are incorporated herein by reference in their entirety .
`proprietary RF , etc . ) As will become apparent , the control
`This application is also related to U . S . patent application
`ling device may range from a very simple unidirectional IR
`Ser . No . 13 / 198 , 072 , filed on Aug . 4 , 2011 and entitled
`device to a fully functional WiFi enabled smart phone or the
`“ System and Method for Configuring the Remote Control
`like . The UCE may receive command requests from such a
`Functionality of a Portable Device , ” the disclosure of which 30 controlling device and apply the optimum methodology to
`is incorporated herein by reference in its entirety .
`propagate the command function ( s ) to each intended target
`This application is also related to U . S . patent application
`appliance , such as for example a TV , AV receiver , DVD
`player , etc . In this manner the UCE may enable a single
`Ser . No . 13 / 240 , 604 , filed on Sep . 22 , 2011 and entitled
`“ System and Method for Configuring Controlling Device
`controlling device to command the operation of all appli
`Functionality , ” the disclosure of which is incorporated 35 ances in a home theater system while coordinating available
`herein by reference in its entirety .
`methods of controlling each particular appliance in order to
`select the best and most reliable method for issuing each
`command to each given device . By way of example without
`BACKGROUND
`limitation , a UCE may utilize IR commands to power on an
`Controlling devices , for example remote controls , for use 40 AV receiver appliance while CEC commands or another
`method may be used to select inputs or power down the
`in issuing commands to entertainment and other appliances ,
`and the features and functionality provided by such control -
`same AV receiver appliance ; or CEC commands may be
`ling devices are well known in the art . In order to facilitate
`used to power on and select inputs on a TV appliance while
`such functionality , various communication protocols , com -
`IR commands may be used to control the volume on the
`mand formats , and interface methods have been imple - 45 same TV appliance .
`mented by appliance manufacturers to enable operational
`As will become apparent , a UCE may comprise modular
`control of entertainment and other appliances , also as well
`hardware and software which may be embodied in a stand
`known in the art . In particular , the recent proliferation of
`alone device suitable for use in an existing home theater
`wireless and wired communication and / or digital intercon -
`equipment configuration , or may be incorporated into any
`nection methods such as WiFi , Bluetooth , HDMI , etc . , 50 one of the appliances such as a STB , TV , AV receiver , HDMI
`amongst and between appliances has resulted in a corre -
`switch etc . Further , when incorporated into an appliance ,
`sponding proliferation of such communication protocols and
`UCE functionality may be provisioned as a separate hard
`command formats . While many of these newer methods may
`ware module or may be incorporated together with other
`offer improved performance and / or reliability when com -
`hardware functionality , e . g . , as part of an HDMI interface IC
`pared to previous control protocols , appliance manufacturer 55 or chip set , etc .
`adoption of such newer methods remains inconsistent and
`A better understanding of the objects , advantages , fea
`fragmented . This , together with the large installed base of
`tures , properties and relationships of the invention will be
`prior generation appliances , may cause confusion , mis -
`obtained from the following detailed description and accom
`operation , or other problems when a user or manufacturer of
`panying drawings which set forth illustrative embodiments
`a controlling device , such as a remote control , attempts to
`60 and which are indicative of the various ways in which the
`take advantage of the enhanced features and functionalities
`principles of the invention may be employed .
`of these new control methods .
`BRIEF DESCRIPTION OF THE DRAWINGS
`SUMMARY OF THE INVENTION
`For a better understanding of the various aspects of the
`This invention relates generally to enhanced methods for
`invention , reference may be had to preferred embodiments
`shown in the attached drawings in which :
`appliance control via use of a controlling device , such as a
`
`65
`
`

`

`Case 8:20-cv-00700-JLS-JDE Document 1-4 Filed 04/09/20 Page 20 of 27 Page ID #:104
`
`US 10 , 325 , 486 B2
`
`10
`
`receiver 120 may not support HDMI inputs , being connected
`FIGS . 1 and 2 illustrate exemplary systems in which a
`to audio source appliances 108 , 110 via , for example S / PDIF
`standalone UEC device may be utilized to command opera
`interfaces 122 . Accordingly UCE 100 may be constrained to
`tion of several appliances ;
`transmit all commands destined for AV receiver 120 exclu
`FIGS . 3 and 4 illustrate exemplary systems in which UEC
`functionality may be incorporated into an appliance which is 5 sively as IR signals , while commands destined for the other
`appliances 106 through 110 may take the form of either CEC
`part of a home entertainment system ;
`FIG . 5 illustrates a block diagram of an exemplary UEC
`or IR signals as appropriate for each command . By way of
`example without limitation , certain TV manufacturers may
`device ;
`FIG . 6 illustrates a graphical representation of an exem
`elect not to support volume adjustment via CEC . If the
`plary UCE - based control environment ;
`illustrative TV 106 is of such manufacture , UCE 100 may
`FIG . 7 illustrates an exemplary preferred command
`relay volume adjustment requests to TV 106 as IR signals
`matrix for use in a UCE - based control environment , for
`114 , while other requests such as power on / off or input
`example as illustrated in FIG . 6 ;
`selections may be relayed in the form of CEC commands
`FIG . 8 illustrates a block diagram of an exemplary smart
`over HDMI connection 112 .
`device which may support a remote control app and a setup 15
`It will however be appreciated that while illustrated in the
`context of IR , RF , and wired CEC signal transmissions , in
`method for use in configuring a UCE ;
`FIG . 9 illustrates an exemplary series of steps which may
`general , transmissions to and from UCE device 100 may
`be performed in order to set up and configure an exemplary
`take the form of any convenient IR , RF , hardwired , point
`to - point , or networked protocol , as necessary for a particular
`UCE ;
`FIG . 10 illustrates an exemplary series of steps which may 20 embodiment . Further , while wireless communications 116 ,
`be performed in order to define to a UCE an appliance
`118 , etc . , between exemplary devices are illustrated herein
`configuration which corresponds to a user activity ;
`as direct links , it should be appreciated that in some
`FIG . 11 illustrates exemplary activity configuration matri -
`instances such communication may take place via a local
`ces such as may be defined during the steps of FIG . 10 ;
`area network or personal area network , and as such may
`FIG . 12 illustrates an exemplary current appliance state 25 involve various intermediary devices such as routers ,
`matrix which may be maintained by a UCE for use in
`bridges , access points , etc . Since these items are not neces
`determining the commands necessary to invoke one of the
`sary for an understanding of the instant invention , they are
`omitted from this and subsequent Figures for the sake of
`states defined by the matrix of FIG . 11 ;
`FIG . 13 illustrates an exemplary series of steps which may
`clarity .
`be performed by a UCE in issuing a function command to an 30
`Since smart device remote control apps such as that
`appliance ;
`contemplated in the illustrative device 104 are well known ,
`FIG . 14 illustrates an exemplary series of steps which may
`for the sake of brevity the operation , features , and functions
`be performed by a UCE in establishing appliance states
`thereof will not be described in detail herein . Nevertheless ,
`matching a desired activity defined in one of the matrices of
`if a more complete understanding of the nature of such apps
`35 is desired , the interested reader may turn to , for example , the
`FIG . 11 ; and
`FIG . 15 illustrates an exemplary series of steps which may
`before mentioned U . S . patent application Ser . No . 12 / 406 ,
`be performed by a smart device to setup command control
`601 or U . S . patent application Ser . No . 13 / 329 , 940 , ( now
`U . S . Pat . No . 8 , 243 , 207 ) .
`macros .
`Turning now to FIG . 2 , in a further illustrative embodi
`40 ment , UCE 100 may receive wireless request signals from a
`remote control 200 and / or an app resident on a tablet
`computer 202 . As before , command transmissions to appli
`With reference to FIG . 1 , there is illustrated an exemplary
`ances 106 , 108 , 110 may take the form of wired CEC com
`system in which a UCE device 100 may be used to issue
`mands or wireless IR commands . However , in this example
`commands to control various controllable appliances , such
`as a television 106 , a cable set top box combined with a 45 remote control 200 may be in bi - directional communication
`dig