`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 1 of 27 Page ID #:100
`
`
`
`
`
`
`
`
`
`EXHIBIT D
`EXHIBIT D
`
`
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 2 of 27 Page ID #:101
`Case 8200000701 ”0°“mem 1'4 Fi“11111111111111llllllllll‘llllllllllllll|||||’l|ll|lllllfilll‘flllllll
`
`US010593196B2
`
`(12) United States Patent
`US 10,593,196 B2
`(10) Patent No.:
`(45) Date of Patent:
`*Mar. 17, 2020
`Arling
`
`(54) SYSTEM AND METHOD FOR OPTIMIZED
`APPLIANCE CONTROL
`
`(56)
`
`(71) Applicant: Universal Electronics Inc., Santa Ana,
`CA (US)
`
`(72)
`
`Inventor: Paul D. Arling, Irvine, CA (US)
`
`(73) Assignee: UNIVERSAL ELECTRONICS INC.,
`Santa Ana, CA (US)
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`This patent is subject to a terminal dis-
`claimer.
`
`(21) Appl. No.: 16/197,748
`
`(22)
`
`Filed:
`
`Nov. 21, 2018
`
`(65)
`
`Prior Publication Data
`
`US 2019/0096235 A1
`
`Mar. 28, 2019
`
`Related US. Application Data
`
`(63) Continuation of application No. 15/789,547, filed on
`Oct. 20, 2017, now Pat. No. 10,325,486, which is a
`(Continued)
`
`(51)
`
`Int. Cl.
`G08C 1 7/02
`G08C 23/04
`
`(2006.01)
`(2006.01)
`(Continued)
`
`(52) US. Cl.
`CPC ............. G08C 17/02 (2013.01); G08C 23/04
`(2013.01), H04N 21/42226 (2013.01),
`(Continued)
`(58) Field of Classification Search
`CPC
`G08C 17/02; G08C 23/04; G08C 2201/20;
`G08C 23/30; G08C 23/40; G08C 23/70;
`(Continued)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`5,959,539 A
`6,160,491 A
`
`9/1999 Adolph et a1.
`12/2000 Kitao et a1.
`
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`CN
`EP
`WO
`
`102541547 A
`1722341 A1
`2011/053008 A2
`
`7/2012
`11/2006
`5/2011
`
`OTHER PUBLICATIONS
`
`United States Patent and Trademark Oflice, Final Office Action
`issued on US. Appl. No. 15/900,342, Notification Date ofNov. 29,
`2018, 19 pgs.
`
`(Continued)
`
`Primary Examiner 7 Adnan AZiZ
`(74) Attorney, Agent, or Firm 7 Greenberg Traurig, LLP
`
`(57)
`
`ABSTRACT
`
`In response to a detected presence of an intended target
`appliance within a logical topography of controllable appli-
`ances identity information associated with the intended
`target appliance is used to automatically add to a graphical
`user interface of a controlling device an icon representative
`of the intended target appliance and to create at a Universal
`Control Engine a listing of communication methods for use
`in controlling corresponding functional operations of the
`intended target appliance. When the icon is later activated,
`the controlling device is placed into an operating state
`appropriate for controlling functional operations of the
`intended target appliance while the Universal Control
`Engine uses at least one of the communication methods to
`transmit at least one command to place the intended target
`appliance into a predetermined operating state.
`
`22 Claims, 14 Drawing Sheets
`
`106
`
`122
`
`r’
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 3 of 27 Page ID #:102
`Case 8:20-cv-00701 Document 1—4 Filed 04/09/20 Page 3 of 27 Page ID #:102
`
`US 10,593,196 B2
`
`Page 2
`
`Related US. Application Data
`
`2008/0141316 A1*
`
`6/2008 Igoe .................... H04L 12/2809
`725/81
`
`continuation of application No. 15/259,847, filed on
`Sep. 8, 2016, now Pat. No. 9,842,492, which is a
`continuation of application No. 14/136,023, filed on
`Dec. 20, 2013, how Pat. No. 9,449,500, which is a
`continuation-in-part of application No. 13/899,671,
`filed on May 22, 2013,n0W Pat. No. 9,437,105,Which
`is a continuation of application No. 13/657,176, filed
`on Oct. 22, 2012, now Pat. No. 9,215,394.
`
`(60) Provisional application No. 61/552,857, filed on Oct.
`.
`.
`.
`.
`ii:(123113112?1521811131 apphcauon NO‘ 61/680’876’
`'
`a
`'
`
`(2011.01)
`(2011.01)
`
`(51)
`
`(
`
`(56)
`
`Int-Cl.
`H04N 21/422
`H04N 21/4363
`52 US Cl
`)
`-
`-
`-
`CPC ..... G08C 2201/20 (2013.01); G08C 2201/30
`(2013.01); G08C 2201/40 (2013.01); G08C
`2201/70 (2013.01), G08C 2201/92 (2013.01),
`.
`G08C 221011551 (22710221332132? 2215:3235
`(
`~
`),
`(
`~
`)
`(58) Field of Classification Search
`CPC .. G08C 23/92; G08C 23/93; H04N 21/42226;
`H04N 21/42225, H04N 21/4363
`See application file for complete search history.
`.
`References C‘ted
`
`23322; El
`6,968,399 B2
`7,379,778 B2
`7,436,346 32
`7,499,462 B2
`7,519,393 B2
`7,589,642 B1
`8,040,888 B1
`$232,222 3%
`,
`,
`8,429,713 B2*
`
`U‘S‘ PATENT DOCUMENTS
`.
`$388; E3312? 31.
`11/2005 Noda et a1.
`5/2008 Hayes et 31.
`10/2008 Walter et 31.
`3/2009 MacMullan et al.
`4/2009 Bahletal.
`9/2009 Mui
`10/2011 MacAdam et al.
`Egg; fsidad
`1
`a 0n eeta~
`4/2013 Candelore ........ H04N 21/43615
`348/558
`
`8477479 32 W20” Tatsuta et al~
`8,633,986 Bl
`“2014 Hughes
`8810 732 B1
`8/2014 B
`th
`8,839,334 B2
`”014 L02“
`8’881’205 B2
`“/2014 13;:de
`’
`’
`2002/0174270 A1* 11/2002 St
`k ................ H04L12/2805
`ecy
`710/1
`5/2003 Klein et 31.
`7/2004 Suzuki et 31
`8/2004 Krzyzanowski et a1.
`10/2004 Dresti et 31.
`12/2004 Compton et al.
`2/2005 Ellis
`9/2005 Chen etal.
`7/2006 Gillard et a1.
`7/2006 Chol
`9/2006 Hotelling
`10/2006 Vidal
`11/2006 Lim
`7/2007 Deng
`9/2007 Garg
`13388;
`Isirkznig
`11/2007 Ahlgren
`12/2007 Guo et 31.
`1/2008 Arling et 31.
`5/2008 Dong et al.
`
`2003/0095156 A1
`2004/0143847 A1
`2004/0163073 A1
`2004/0210933 A1
`2004/0255329 A1
`2005/0028208 A1
`2005/0195823 A1
`2006/0146184 A1
`2006/0168618 A1
`2006/0197753 A1
`2006/0227032 A1
`2006/0245725 A1
`2007/0165555 A1
`2007/0220150 A1
`3883;833:522 :1
`2007/0268360 A1
`2007/0292135 A1
`2008/0005764 A1
`2008/0120673 A1
`
`2008/0168519 A1
`2008/0187028 A1
`2008/0278567 A1
`3883488535? :1,
`
`7/2008 Rao et 31.
`8/2008 Lida__
`“/2008 Nakallmé}
`$4388;
`223351111“
`"""""""""
`
`G08C19/28
`341/176
`
`2/2009 Satou
`2009/0051824 A1
`6/2009 Doyle
`2009/0156051 A1
`7/2009 Kohfmek
`2009/0167555 A1
`8/2009 Hauima
`2009/0207039 A1
`9/2009 Negron et al.
`2009/0239587 A1
`18/3883 Isitlronlika et 3L
`3883/834513283 :1
`ecy
`2009/0284656 A1* 11/2009 Suzuki
`................ H04L12/2809
`348/554
`2010/0079682 A1
`4/2010 Martch
`2010/0134317 A1
`6/2010 Breuil et a1.
`2010/0138764 A1
`6/2010 Hatambeiki et a1.
`2010/0157169 A1
`6/2010 Yoshida et a1.
`2010/0177245 A1
`7/2010 Ohnuma etal.
`2010/0271560 A1
`10/2010 Higuchi et a1.
`2010/0315279 A1* 12/2010 Hamal
`................... G08C 17/02
`341/176
`
`Egg/$333431?) :1
`2011/0142059 A1
`2011/0156944 A1*
`
`2011/0273287 A1
`2011/0274008 A1
`2011/0283129 A1
`2011/0285818 A1
`2011/0289113 A1
`2012/0013449 A1
`2012/0013807 A1
`2012/0019400 A1*
`
`2012/0021684 A1*
`
`2012/0069894 A1
`2012/0082461 A1*
`
`2012/0084452 A1
`2012/0146918 A1*
`
`2012/0171958 A1
`2012/0173003 A1
`2012/0236161 A1
`2012/0242526 A1
`2012/0249890 A1
`2012/0274547 A1
`2012/0274857 A1
`2012/0278693 A1
`2012/0291128 A1
`2012/0297040 A1
`2012/0330943 A1
`2013/0005250 A1
`2013/0058522 A1
`2013/0069769 A1*
`
`2013/0107131 A1
`2013/0249679 A1
`2014/0085059 A1
`2014/0235526 A1
`
`lgggfi) gaymga 1
`ergls et a .
`6/2011 Bedingfield, Sr. et al.
`
`6/2011 Ward ............... H04L12/2832
`341/176
`
`11/2011 LaLonde et a1.
`11/2011 Lida
`11/20“ Gulllerm
`11/2011 Park
`11/2011 Arling etal.
`1/2012 Penisoara et a1.
`1/2012 Arora
`1/2012 Patel
`
`...................... G08C17/00
`340/870.15
`1/2012 Schultz ................ H04B 5/0043
`_
`455/411
`3/2012 Saklmura et a1.
`4/2012 Meyer .................... G08C17/02
`398/106
`
`4/2012 Pettitetal.
`6/2012 Kreiner ............... H04M1/7253
`345/173
`
`7/2012 Cornett et a1.
`7/2012 Kim
`9/2012 Kwon et 31.
`9/2012 Perez et al.
`10/2012 Chardon et al.
`11/2012 Raeber et a1.
`11/2012 Maxwell et a1.
`11/2012 Black et al.
`11/2012 Ja awardena et a1.
`Y
`11/2012 Amano
`12/2012 Webeietal.
`1/2013 Klm-et a1.
`3/2013 Raeslg et 81
`3/2013 Pennlngton ............ G08C17/02
`340/1228
`
`5/2013 Barnett et al.
`9/2013 Arling
`3/2014 Chen etal.
`8/2014 Slupik
`
`OTHER PUBLICATIONS
`
`United States Patent and Trademark Oflice, Final Office Action
`issued on U.S.Appl. No. 15/900,232, Notification Date ofDec. 13,
`2018’ 15 Pgs'
`_
`_
`_
`ISA/US, Int. Search Report and ertten Oplnlon ofthe Int. Search-
`ing Authority issued on Int. Appln. N0. PCT/US12/62161, dated
`Jan. 23, 2013, 12 pages.
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 4 of 27 Page ID #:103
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 4 of 27 Page ID #:103
`
`US 10,593,196 B2
`Page 3
`
`(56)
`
`References Cited
`OTHER PUBLICATIONS
`
`ISA/US, Int. Search Report and Written Opinion of the Int. Search-
`ing Authority issued on Int. Appln. N0. PCT/US 14/38151, dated
`Jun. 27, 2014, 10 pages.
`European Patent Oflice, extended European Search Report issued on
`European patent application No. 128441219, dated Mar. 5, 2015, 6
`pages.
`European Patent Oflice, extended European Search Report issued on
`European patent application No. 148010648, dated Apr. 18, 2016,
`8 pages.
`European Patent Oflice, extended European Search Report issued on
`European patent application No. 14872863.7, dated NOV. 25, 2016,
`8 pages.
`EPO, examination report issued on European patent application No.
`148010648, dated Jul. 11, 2019, 4 pages.
`CNIPA, 2nd Office Action issued on Chinese patent application No.
`2014800579440, dated Jul. 16, 2019, 13 pages.
`High-Definition Multimedia Interface, HDMI Licensing, LLC,
`Specificatien Version 1.3a, NOV. 10, 2006, pp. 1-276.
`United States Patent and Trademark Oflice, Non-Final Oifice Action
`issued on US. Appl. No. 16/197,552, Notification Date of Sep. 19,
`2019, 10 pgs.
`United States Patent and Trademark Oflice, Non-Final Oifice Action
`issued on US. Appl. No. 16/196,756, Notification Date of Sep. 6,
`2019, 9 pgs.
`United States Patent and Trademark Oflice, Non-Final Oifice Action
`issued on US. Appl. No. 16/156,766, Notification date of Oct. 4,
`2019, 10 pgs.
`United States Patent and Trademark Oflice, Non-Final Oifice Action
`issued on US. Appl. No. 16/199,463, dated NOV. 22, 14 pgs.
`
`* cited by examiner
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 5 of 27 Page ID #:104
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 5 of 27 Page ID #:104
`
`U.S. Patent
`
`Mar. 17, 2020
`
`Sheet 1 0f 14
`
`US 10,593,196 B2
`
`
`
`Figure 1
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 6 of 27 Page ID #:105
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 6 of 27 Page ID #:105
`
`U.S. Patent
`
`Mar. 17, 2020
`
`Sheet 2 0f 14
`
`US 10,593,196 B2
`
`
`
`‘ 207
`
`Figure 2
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 7 of 27 Page ID #:106
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 7 of 27 Page ID #:106
`
`U.S. Patent
`
`Mar. 17, 2020
`
`Sheet 3 0f 14
`
`US 10,593,196 B2
`
`118
`
`
`
`/ {e4}. Entemet,
`/
`came pianteic.)
`
`\
`
`
`
`Figure 3
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 8 of 27 Page ID #:107
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 8 of 27 Page ID #:107
`
`U.S. Patent
`
`Mar. 17, 2020
`
`Sheet 4 0f 14
`
`US 10,593,196 B2
`
`
`
`Figure 4
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 9 of 27 Page ID #:108
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 9 of 27 Page ID #:108
`
`U.S. Patent
`
`Mar. 17, 2020
`
`Sheet 5 0f 14
`
`US 10,593,196 B2
`
`
`
`
`pawer suppfiy
`
` Externai 5
`
`5 16
`
`Processor
`
`.........
`
`
`
`g
`504
`
`0
`I
`
`Figure 5
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 10 of 27 Page ID #:109
`Case 8:20-cv-00701 Document 1—4 Filed 04/09/20 Page 10 of 27 Page ID #:109
`
`U.S. Patent
`
`Mar. 17, 2020
`
`Sheet 6 0f 14
`
`US 10,593,196 B2
`
`52c.
`
`610
`
` K
`
`7‘
`74/
`1'14
`
`.
`
`-.
`
`. 50¢
`1
`50:3
`\
`_
`\
`<3",
`.1
`... .._ _..
`
`514
`.
`
`Ethernet I HQME l- Other
`
`-
`
`290
`
`“
`
`104
`
`102 ’
`
`Figure 6
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 11 of 27 Page ID #:110
`Case 8:20-cv-00701 Document 1—4 Filed 04/09/20 Page 11 of 27 Page ID #:110
`
`U.S. Patent
`
`Mar. 17, 2020
`
`Sheet 7 0f 14
`
`US 10,593,196 B2
`
`715,,
`
`7100wa
`
`792
`704,
`u,
`
`.....\IK....................
`
`y
`
`720
`
`73.0
`708
`..........................I...............
`VRI Appéiance
`
`.................................................................... .
`
`U---“
`Votumeup
`.n---
`
`Functaon
`
`”wwww
`-—nn-
`U“---
`
`705
`
`712
`
`714
`
`718
`
`figure 7
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 12 of 27 Page ID #:111
`Case 8:20-cv-00701 Document 1—4 Filed 04/09/20 Page 12 of 27 Page ID #:111
`
`U.S. Patent
`
`Mar. 17, 2020
`
`Sheet 8 0f 14
`
`US 10,593,196 B2
`
`
`
`820
`
`8:2
`
`,.
`
`85.4
`
`:
`
`I”
`
`i
`
`I
`
`Externai memory
`
`804
`
`Figure 8
`
` ; Dismay
`;
`interface
`
`
`Processor
`
`
`818
`We"
`...............
`
`
`
`
`
`tooth
`Touch
`:
`.................
`SCi‘een
`_______
`I
`
`interface
`
`I
`I
`I
`g Camera .........:
`
`'
`....................................................
`g
`
` Bum-in memory
`...........
`
`
`
`; Acceier- ;
`ometer,
`g
`{395,
`g
`i ambient i
`Right
`sensor,
`:
`NFC,
`
`EEC.
`
`
`........................
`
`"812
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 13 of 27 Page ID #:112
`Case 8:20-cv-00701 Document 1—4
`Filed 04/09/20 Page 13 of 27 Page ID #:112
`
`U.S. Patent
`
`Mar. 17, 2020
`
`Sheet 9 of 14
`
`US 10,593,196 B2
`
`AppEiance command
`database server
`
`UCE programming
`
`Set up app
`
`I
`
`Set up UCE
`
`E
`
`E
`
`E
`
`................. 902
`Request user to
`,2
`
`
`E
`power aEl
`
`E
`appEiarmes on
`5-
`904
`Scan for
`E
`Determine CEC "
`
`enabied " connected CEC
`appiiahces
`appiiances
`
`
`
`
`
`
`
`,9/06
`
`910
`\
`Scan for other
`
`
`
`_ HUME attached
`appéiamzes
`
`
`922
`/
`
`930
`../
`
`Receive
`codesets, issue
`“test command};
`
`9.
`
`Store Ema!
`command
`matrix
`
`E E E E E E E E E E E E E E E E E
`
`................
`
`
`
`Determine
`HDME (non-CEO
`
`
`
`appéiarsces
`
`Retrieve
`
`
`.., command sets
`
`
`for appiiances
`
`
`command gets; 93,8
`
`Retrieve
`
`
`aitemate
`
`
`
`.1; Build command
`matrix
`
`912
`.
`Dispan “St of W,»
`located
`
`appiiances
`
`................................... 914
`,1
`User identifies
`
`
`non-(EEC and non-
`E-iDM! appiiences
`
`
`
`
`916
`W,»
`
`.
`.
`Obtain
`
`
`mmmand sets
`
`
`Em appiiances
`
`
`
`920
`
`~~~~~~
`
`...................................
`Test ans?
`confirm (zodse‘ts;
`
`_ matrix, transfer
`1:0 UCE
`
`actEvities
`
`Coeffigure
`
`Figure 9
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 14 of 27 Page ID #:113
`Case 8:20-cv-00701 Document 1—4 Filed 04/09/20 Page 14 of 27 Page ID #:113
`
`U.S. Patent
`
`Mar. 17, 2020
`
`Sheet 10 0f 14
`
`US 10,593,196 B2
`
`Set up app
`
`UCE programming
`
`.'
`
`Configure
`activities
`
`-
`
`
`1002
`.................................... j
`Pre5ent iist of
`
`activities
`
`111006
`identifycontem: ,
`somce
`
`
`1,004
`;
`
`User se§ects an /
`activity
`
`
`
`
`
`identify video
`rendering
`device and
`
`content route
`
`
`
`1008
`’
`
`101C?
`’1
`
`{dentéfy audéo
`rendering
`device and
`content route
`
`Build activity
`matrix
`
`1912
`’
`
`1914
`
`.................................... 1015
`issue commands ”
`and report
`status as
`
`required
`
`TESt activity M
`
`activity matrix
`
`
`Downioad
`
`
`activity matrix
`to UCE
`
`1922
`
`Store finatézed
`
`
`
`'
`
`"
`
`’
`
`Figure 3.0
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 15 of 27 Page ID #:114
`Case 8:20-cv-00701 Document 1—4 Filed 04/09/20 Page 15 of 27 Page ID #:114
`
`U.S. Patent
`
`Mar. 17, 2020
`
`Sheet 11 0f 14
`
`US 10,593,196 B2
`
`1100
`
`1101
`
`1102
`
`1 104
`
`2“
`
`
`Activit = ”L2sten to mus2c”
`
`Activfi I a “Watch a movie”
`
`
`
`02222222- - 5mm F- I----------
`
`Activity- "Watch TV”
`
`I
`
`
`
`\1116
`
`1112
`
`*— 1114
`
`figure 11
`
`Current appéiance state
`
`Figure 1.2
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 16 of 27 Page ID #:115
`Case 8:20-cv-00701 Document 1—4 Filed 04/09/20 Page 16 of 27 Page ID #:115
`
`U.S. Patent
`
`Mar. 17, 2020
`
`Sheet 12 0f 14
`
`US 10,593,196 B2
`
`Command request
`
`'
`
`\‘*~~~~~ 1300
`
`
`
`
`Retrieve corresponding
`data element from
`
`
`
`
`
`preferred command
`matrix
`
`
`
`Transmit function
`
`
`request to appliance
`using Endicated method
`
`
`
`Retrieve correspnnding
`data eiement from
`zzéternate command
`matrix
`
`
`
`retrace? supports -
`.
`confirmarion? .
`-
`'
`
`
`
`
`
`
`
`
` I Function
`
`-
`. “confirmed? .
`a.
`
`
`
`
`' Aitemate
`methc-d
`
`'
`
`- evaiiabie?
`
`
`
`
`Generate error message
`
`
`
`Figure 13
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 17 of 27 Page ID #:116
`Case 8:20-cv-00701 Document 1—4 Filed 04/09/20 Page 17 of 27 Page ID #:116
`
`U.S. Patent
`
`Mar. 17, 2020
`
`Sheet 13 0f 14
`
`US 10,593,196 B2
`
`“Watch TV”
`
`
`activityrequesi
`'
`
`
`"\\~1400
`
`Access appfiance
`state matrix for
`”Watch TV”
`
`Send TV ”Power on”
`(CEC)
`
`Set transport
`commands 2 DVR
`
`Send TV "Mute”
`
`(1R)
`
`DVD powered '
`on?
`
`-
`
`_
`
`=3
`
`
`
`
` STE/DVR
`'
`
`Send DVD “1’ower
`
`'
`off” {EEC}
`
`
`Send TV ”énpu’cl"
`(CEC)
`
`Send AVR
`“Power on”
`
`
`
`
`powered on? -
`
`'
`
`N
`
`'
`
`\w 1425
`
`Send 878 ”Power
`
`on” {CEC}
`
`commands === AVR
`
`;
`
`Set volume
`
`1416/
`
`-
`
`Figure 14
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 18 of 27 Page ID #:117
`Case 8:20-cv-00701 Document 1—4 Filed 04/09/20 Page 18 of 27 Page ID #:117
`
`U.S. Patent
`
`Mar. 17, 2020
`
`Sheet 14 0f 14
`
`US 10,593,196 B2
`
`1510
`
`\\\
`
`
`
`1520
`I
`,x/
`
`207\
`..
`--
`
`--
`
`.
`
`Set up app
`
`// 1502
`
`/
`
`/
`
`/
`
`Determine CEC
`or 3P enat-éed
`appfiances
`
`
`
`Scan for
`-- connected CEC
`appiiances
`
`Scan for other
`Determine
`\ HDM: {noneCEC} - HUM: attached
`appiiances
`'i'
`appiiances
`
`
`
`
`
`l
`1506
`0
`I//
`
`
`
`/
`V"
`/
`
`//
`Y
`Cr, Metac'a
`
`avaiiabte from
`appliance?
`
`
`N I;
`
`\\\., .1512
`
`..
`Dismay list of
`heated
`appiiances
`
`
`
`/
`
`15 14
`///
`f
`1510
`//t"
`
`1518
`/x
`
`1525
`
`/,/
`
`Send to smart
`device
`
`
`
`Figure 15
`
`User identifies /
`
`
`
`‘—
`.
`~C :C
`‘
`(‘00 ‘L 30d non
`HUME appliances
`
`
`I
`.
`.
`.
`- /‘/
`
`
`Reineve econ
`__:
`Obtain won
`
`
`
`
`
`metadata for -— metadata for
`
`
`
`
`appiiances
`appiénces
`
`1922
`Retfieve auto
`
`
`gene rated
`‘4----------1““
`macros
`
`
`
`Buiic‘ user
`
`profiie
`
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 19 of 27 Page ID #:118
`Case 8:20-cv-00701 Document 1—4 Filed 04/09/20 Page 19 of 27 Page ID #:118
`
`US 10,593,196 B2
`
`1
`SYSTEM AND METHOD FOR OPTIMIZED
`APPLIANCE CONTROL
`
`RELATED APPLICATION INFORMATION
`
`This application claims the benefit of and is a continuation
`of U.S. application Ser. No. 15/789,547, filed on Oct. 20,
`2017, which application claims the benefit of and is a
`continuation of U.S. application Ser. No. 15/259,847, filed
`on Sep. 8, 2016, which application claims the benefit of and
`is a continuation of U.S. application Ser. No. 14/136,023,
`filed on Dec. 20, 2013, which application claims the benefit
`of and is a continuation-in-part of U.S. application Ser. No.
`13/899,671, filed on May 22, 2013, which application claims
`the benefit of and is a continuation of U.S. application Ser.
`No. 13/657,176, filed on Dec. 22, 2012, which application
`claims the benefit of U.S. Provisional Application No.
`61/552,857, filed Oct. 28, 2011, and U.S. Provisional Appli-
`cation No. 61/680,876, filed Aug. 8, 2012, the disclosures of
`which are incorporated herein by reference in their entirety.
`This application is also related to U.S. patent application
`Ser. No. 12/621,277, filed on Nov. 18, 2009 and entitled
`“System and Method for Reconfiguration of an Entertain-
`ment System Controlling Device,” which in turn is a con-
`tinuation-in-part of U.S. patent application Ser. No. 12/569,
`121 (now U.S. Pat. No. 8,243,207), filed on Sep. 29, 2009
`and entitled “System and Method for Activity Based Con-
`figuration of an Entertainment System,” the disclosures of
`which are incorporated herein by reference in their entirety.
`This application is also related to U.S. patent application
`Ser. No. 13/198,072, filed on Aug. 4, 2011 and entitled
`“System and Method for Configuring the Remote Control
`Functionality of a Portable Device,” the disclosure of which
`is incorporated herein by reference in its entirety.
`This application is also related to U.S. patent application
`Ser. No. 13/240,604, filed on Sep. 22, 2011 and entitled
`“System and Method for Configuring Controlling Device
`Functionality,” the disclosure of which is
`incorporated
`herein by reference in its entirety.
`
`BACKGROUND
`
`Controlling devices, for example remote controls, for use
`in issuing commands to entertainment and other appliances,
`and the features and functionality provided by such control-
`ling devices are well known in the art. In order to facilitate
`such functionality, various communication protocols, com-
`mand formats, and interface methods have been imple-
`mented by appliance manufacturers to enable operational
`control of entertainment and other appliances, also as well
`known in the art. In particular, the recent proliferation of
`wireless and wired communication and/or digital intercon-
`nection methods such as WiFi, Bluetooth, HDMI, etc.,
`amongst and between appliances has resulted in a corre-
`sponding proliferation of such communication protocols and
`command formats. While many of these newer methods may
`offer improved performance and/or reliability when com-
`pared to previous control protocols, appliance manufacturer
`adoption of such newer methods remains inconsistent and
`fragmented. This, together with the large installed base of
`prior generation appliances, may cause confusion, mis-
`operation, or other problems when a user or manufacturer of
`a controlling device, such as a remote control, attempts to
`take advantage of the enhanced features and functionalities
`of these new control methods.
`
`SUMMARY OF THE INVENTION
`
`This invention relates generally to enhanced methods for
`appliance control via use of a controlling device, such as a
`
`2
`
`remote control, smart phone, tablet computer, etc., and in
`particular to methods for taking advantage of improved
`appliance control communication methods and/or command
`formats in a reliable manner which is largely transparent to
`a user and/or seamlessly integrated with legacy appliance
`control technology.
`To this end, the instant invention comprises a modular
`hardware and software solution, hereafter referred to as a
`Universal Control Engine (UCE), which is adapted to pro-
`vide device control across a variety of available control
`methodologies and communication media,
`such as
`for
`example various infrared (IR) remote control protocols;
`Consumer Electronic Control (CEC) as may be implemented
`over a wired HDMI connection; intemet protocol (IP), wired
`or wireless; RF4CE wireless; Bluetooth (BT) wireless per-
`sonal area network(s); UPnP protocol utilizing wired USB
`connections; or any other available standard or proprietary
`appliance command methodology. Since each individual
`control paradigm may have its own strengths and weak-
`nesses, the UCE may be adapted to combine various control
`methods in order to realize the best control option for each
`individual command for each individual device.
`The UCE itself may be adapted to receive commands
`from a controlling device, for example, a conventional
`remote control or a remote control app resident on a smart
`device such as a phone or tablet, etc., utilizing any conve-
`nient protocol and command structure (IR, RF4CE, BT,
`proprietary RF, etc.) As will become apparent, the control-
`ling device may range from a very simple unidirectional IR
`device to a fully functional WiFi enabled smart phone or the
`like. The UCE may receive command requests from such a
`controlling device and apply the optimum methodology to
`propagate the command function(s) to each intended target
`appliance, such as for example a TV, AV receiver, DVD
`player, etc. In this manner the UCE may enable a single
`controlling device to command the operation of all appli-
`ances in a home theater system while coordinating available
`methods of controlling each particular appliance in order to
`select the best and most reliable method for issuing each
`command to each given device. By way of example without
`limitation, a UCE may utilize IR commands to power on an
`AV receiver appliance while CEC commands or another
`method may be used to select inputs or power down the
`same AV receiver appliance; or CEC commands may be
`used to power on and select inputs on a TV appliance while
`IR commands may be used to control the volume on the
`same TV appliance.
`As will become apparent, a UCE may comprise modular
`hardware and software which may be embodied in a stand-
`alone device suitable for use in an existing home theater
`equipment configuration, or may be incorporated into any
`one of the appliances such as a STB, TV, AV receiver, HDMI
`switch etc. Further, when incorporated into an appliance,
`UCE functionality may be provisioned as a separate hard-
`ware module or may be incorporated together with other
`hardware functionality, e. g., as part of an HDMI interface IC
`or chip set, etc.
`A better understanding of the objects, advantages, fea-
`tures, properties and relationships of the invention will be
`obtained from the following detailed description and accom-
`panying drawings which set forth illustrative embodiments
`and which are indicative of the various ways in which the
`principles of the invention may be employed.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`For a better understanding of the various aspects of the
`invention, reference may be had to preferred embodiments
`shown in the attached drawings in which:
`
`5
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 20 of 27 Page ID #:119
`Case 8:20-cv-00701 Document 1—4 Filed 04/09/20 Page 20 of 27 Page ID #:119
`
`US 10,593,196 B2
`
`3
`FIGS. 1 and 2 illustrate exemplary systems in which a
`standalone UEC device may be utilized to command opera-
`tion of several appliances;
`FIGS. 3 and 4 illustrate exemplary systems in which UEC
`functionality may be incorporated into an appliance which is
`part of a home entertainment system;
`FIG. 5 illustrates a block diagram of an exemplary UEC
`device;
`FIG. 6 illustrates a graphical representation of an exem-
`plary UCE-based control environment;
`FIG. 7 illustrates an exemplary preferred command
`matrix for use in a UCE-based control environment, for
`example as illustrated in FIG. 6;
`FIG. 8 illustrates a block diagram of an exemplary smart
`device which may support a remote control app and a setup
`method for use in configuring a UCE;
`FIG. 9 illustrates an exemplary series of steps which may
`be performed in order to set up and configure an exemplary
`UCE;
`FIG. 10 illustrates an exemplary series of steps which may
`be performed in order to define to a UCE an appliance
`configuration which corresponds to a user activity;
`FIG. 11 illustrates exemplary activity configuration matri-
`ces such as may be defined during the steps of FIG. 10;
`FIG. 12 illustrates an exemplary current appliance state
`matrix which may be maintained by a UCE for use in
`determining the commands necessary to invoke one of the
`states defined by the matrix of FIG. 11;
`FIG. 13 illustrates an exemplary series of steps which may
`be performed by a UCE in issuing a function command to an
`appliance;
`FIG. 14 illustrates an exemplary series of steps which may
`be performed by a UCE in establishing appliance states
`matching a desired activity defined in one of the matrices of
`FIG. 11; and
`FIG. 15 illustrates an exemplary series of steps which may
`be performed by a smart device to setup command control
`macros.
`
`DETAILED DESCRIPTION
`
`With reference to FIG. 1, there is illustrated an exemplary
`system in which a UCE device 100 may be used to issue
`commands to control various controllable appliances, such
`as a television 106, a cable set top box combined with a
`digital video recorder (“STB/DVR”) 110, a DVD player
`108, and an AV receiver 120. While illustrated in the context
`ofa television 106, STB/DVR 110, a DVD player 108, and
`an AV receiver 120, it is to be understood that controllable
`appliances may include, but need not be limited to, televi-
`sions, VCRs, DVRs, DVD players, cable or satellite con-
`verter set-top boxes (“STBs”), amplifiers, CD players, game
`consoles, home lighting, drapery,
`fans, HVAC systems,
`thermostats, personal computers, etc.
`In the illustrative
`example of FIG. 1, appliance commands may be issued by
`UCE 100 in response to infrared (“IR”) request signals 116
`received from a remote control device 102, radio frequency
`(“RF”) request signals 118 received from an app 124 resi-
`dent on a smart device 104, or any other device from which
`UCE 100 may be adapted to receive requests, using any
`appropriate communication method. As illustrated, trans-
`mission of the requested appliance commands from the UCE
`to appliances 106,108,112,120 may take the form ofwireless
`IR signals 114 or CEC commands issued over a wired HDMI
`interface 112, as appropriate to the capabilities of the par-
`ticular appliance to which each command may be directed.
`In particular,
`in the exemplary system illustrated, AV
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`
`receiver 120 may not support HDMI inputs, being connected
`to audio source appliances 108,110 via, for example S/PDIF
`interfaces 122. Accordingly UCE 100 may be constrained to
`transmit all commands destined for AV receiver 120 exclu-
`
`sively as IR signals, while commands destined for the other
`appliances 106 through 110 may take the form of either CEC
`or IR signals as appropriate for each command. By way of
`example without limitation, certain TV manufacturers may
`elect not to support volume adjustment via CEC. If the
`illustrative TV 106 is of such manufacture, UCE 100 may
`relay volume adjustment requests to TV 106 as IR signals
`114, while other requests such as power on/olf or input
`selections may be relayed in the form of CEC commands
`over HDMI connection 112.
`
`It will however be appreciated that while illustrated in the
`context of IR, RF, and wired CEC signal transmissions, in
`general, transmissions to and from UCE device 100 may
`take the form of any convenient IR, RF, hardwired, point-
`to-point, or networked protocol, as necessary for a particular
`embodiment. Further, while wireless communications 116,
`118, etc., between exemplary devices are illustrated herein
`as direct
`links,
`it should be appreciated that
`in some
`instances such communication may take place via a local
`area network or personal area network, and as such may
`involve various
`intermediary devices
`such as
`routers,
`bridges, access points, etc. Since these items are not neces-
`sary for an understanding of the instant invention, they are
`omitted from this and subsequent Figures for the sake of
`clarity.
`Since smart device remote control apps such as that
`contemplated in the illustrative device 104 are well known,
`for the sake of brevity the operation, features, and functions
`thereof will not be described in detail herein. Nevertheless,
`if a more complete understanding of the nature of such apps
`is desired, the interested reader may turn to, for example, the
`before mentioned US. patent application Ser. No. 12/406,
`601 or US. patent application Ser. No. 13/329,940, (now
`US. Pat. No. 8,243,207).
`Turning now to FIG. 2, in a further illustrative embodi-
`ment, UCE 100 may receive wireless request signals from a
`remote control 200 and/or an app resident on a tablet
`computer 202. As before, command transmissions to appli-
`ances 106,108,110 may take the form of wired CEC com-
`mands or wireless IR commands. However, in this example
`remote control 200 may be in bi-directional communication
`208 with UCE 100 and accordingly the UCE may delegate
`the transmission of IR commands 210 to the remote control
`
`device 200, i.e., use remote control 200 as a relay device for
`those commands determined to be best executed via IR
`
`transmissions. As also generally illustrated in FIG. 2, a setup
`app 214 executing on a smart device such as tablet computer
`202 may be utilized in conjunction with an Internet (212,
`204) accessible or cloud based server 206 and associated
`database 207 to initially configure UCE 100 for operation
`with the specific group of appliances to be controlled, i.e., to
`communicate to UCE 100 a matching command code set
`and capability profile for each particular appliance to be
`controlled, for example based on type, manufacture, model
`number, etc., as will be described in greater detail hereafter.
`With reference to FIG. 3, in a further illustrative embodi-
`ment UCE functionality 100' may be embedded in an
`appliance, for example STB/DVR 310. In this example,
`remote control 102 and/or smart device 104 may transmit
`wireless request signals directly to STB/DVR 310 for action
`by the built-in UCE function 100', which actions may, as
`before, comprise CEC command transmissions via HDMI
`connection 112 or IR command transmissions 114, originat-
`
`
`
`Case 8:20-cv-00701 Document 1-4 Filed 04/09/20 Page 21 of 27 Page ID #:120
`Case 8:20-cv-00701 Document 1—4 Filed 04/09/20 Page 21 of 27 Page ID #:120
`
`US 10,593,196 B2
`
`5
`ing in this instance from an IR blaster provisioned to the
`STB/DVR appliance 310. In this configuration, a set up
`application resident in STB/DVR 310 may be utilized to
`configure UEC 100', using for example an Internet connec-
`tion 304 accessible through a cable modem and/or cable
`distribution system headend.
`In the further illustrative embodiment of FIG. 4, UCE
`functionality 100' may be embedded in an AV receiver 420
`which may serve as an HDMI switch between various
`content sources such as a STB/DVR 110 or a DVD player
`108 and a rendering device such as TV 106. In addition to
`HDMI inputs, AV receiver 420 may also support vario