USOO7043532B1
`
`(12)
`
`United States Patent
`Humpleman et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,043,532 B1
`*May 9, 2006
`
`(54) METHOD AND APPARATUS FOR
`UNIVERSALLY ACCESSIBLE COMMAND
`AND CONTROL INFORMATION IN A
`NETWORK
`
`(75) Inventors: Richard Humpleman, Fremont, CA
`(US); Dongyan Wang, Santa Clara, CA
`(US)
`(73) Assignee: Samsung Electronics Co., Ltd., Suwon
`(KR)
`
`(*) Notice:
`
`0
`-
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`spent is Subject to a terminal dis
`
`(21) Appl. No.: 09/307,004
`
`(22) Filed:
`
`May 7, 1999
`Related U.S. Application Data
`(60) Provisional application No. 60/084,578, filed on May
`7, 1998.
`
`(51) Int. Cl.
`(2006.01)
`G06F 5/73
`(2006.01)
`G06F 5/16
`(52) U.S. Cl. ....................................... 709/208; 709/223
`(58) Field of Classification Search ................ 709/223,
`709/208,328, 227; 345/329, 733, 734; 707/102;
`715/733, 734; 71.9/317,328
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`6, 1977 Holz
`4,031,543 A
`8, 1989 Barall
`4,860,006 A
`9/1993 Grandmougin
`5,249,043 A
`5,257,366 A * 10/1993 Adair et al. ................... 707/4
`5,293,635 A
`3/1994 Faulk, Jr. et al.
`
`
`
`5,347,304 A
`5,387,927 A
`5,389,963 A
`
`9, 1994 Moura et al.
`2f1995 Look et al.
`2/1995 Lepley et al.
`(Continued)
`
`EP
`
`FOREIGN PATENT DOCUMENTS
`841 10755.0
`9, 1984
`
`(Continued)
`OTHER PUBLICATIONS
`Evans , G. "Solving home automation problems using
`artificial intelligence techniques, IEEE Trans. on Consumer
`Electronics, pp. 395-400, Aug. 1991.*
`(Continued)
`Primary Examiner Patrice Winder
`(74) Attorney, Agent, or Firm—Michael Zarrabian, Esq.;
`Kenneth L. Sherman, Esq.; Myers, Dawes, Andras &
`Sherman, LLP
`
`(57)
`
`ABSTRACT
`
`A method and system for performing a service on a home
`network, by: connecting a first and a second home device to
`the home network; providing a database including a plurality
`of application interface description data objects, where each
`application interface description data object includes infor
`mation in a structured format for commanding and control
`ling of a home device by one or more other home devices
`connected to the network; the second home device accessing
`a first application interface description object for the first
`home device in the database; the first home device accessing
`a second application interface description object for the
`second home device in the database; sending control and
`command data from the first home device to the second
`home device utilizing the second application interface
`description object over the network; and sending control and
`command data from the second home device to the first
`home device utilizing the first application interface descrip
`tion object over the network. Whereby, the first and second
`home devices perform said service.
`
`27 Claims, 14 Drawing Sheets
`
`Case 1. XML application CNTERACEA) for femata query by E.
`Cass 2. MovskMLApplication-cNierfAGA) to controller forcesqueryby (E)
`case 3, MoveXML. Application-cINTRFACEA) to 3rd party device a collecting
`place (Library) for a frterfacss for remote query by .earary would have to
`have the address Uri of the associated application available for direct control
`actisinancestorises.
`Case 4 is case3but the indirect corolaction and response) is ass directed at
`the library devica. in this case the gray is butwits arowarder,
`
`VMware - Exhibit 1015
`VMware v. IV I - IPR2020-00470
`Page 1 of 35
`
`

`

`US 7,043,532 B1
`Page 2
`
`6,466,971 B1 * 10/2002 Humpleman et al. ....... TO9.220
`6,539,422 B1* 3/2003 Hunt et al. ...........
`709/217
`6,546,419 B1 * 4/2003 Humpleman et al. ....... 709/223
`6,560,639 B1* 5/2003 Dan et al. .......
`TO9.218
`6,618,764 B1* 9, 2003 Sht
`TO9,249
`wkw
`CyIl . . . . . . . . . . . . . . . . . . . . . . .
`FOREIGN PATENT DOCUMENTS
`
`
`
`903052132
`EP
`914O1245.5
`EP
`963047O6.3
`EP
`96306507.3
`EP
`963072004
`EP
`961 16873.9
`EP
`971.00356.1
`EP
`97117812.4
`EP
`9-261355
`JP
`9-282263
`JP
`10-145773
`JP
`11-884.06
`JP
`11-194987
`JP
`11-317756
`JP
`11-355.294
`JP
`11-355.357
`JP
`WO PCT/EP95/OO.191
`WO PCT/US95.00354
`WO PCT/US95/17108
`WO PCT/US96, 18798
`WO PCT/US97/08490
`
`U.S. PATENT DOCUMENTS
`5,392,033 A
`2, 1995 Oman et al.
`5,452.291 A
`9, 1995 Eisenhandler et al.
`5.488.412 A
`1/1996 Majeti et al.
`5,495,561 A
`2/1996 Holt .......................... 358,115
`5,526,483. A
`6, 1996 French et al. .................. 714.f4
`5,546,484 A
`8/1996 Fling et al.
`5,561,709 A 10, 1996 Remillard et al.
`5,570,085 A * 10/1996 Bertsch ..................... 340/3.54
`5,572,643 A 11/1996 Judson
`5,579.308 A 11/1996 Humpleman
`5,596,702 A *
`1/1997 Stucka et al. ............... 345,746
`5,612,730 A
`3, 1997 Lewis
`5,636.211 A * 6, 1997 Newlin et al. .............. 370/465
`5,657,221 A * 8/1997 Warman et al. ............... TOO/83
`5,740,362 A * 4, 1998 Buickel et al. ............. TO9,201
`5,778,226 A * 7/1998 Adams et al. .
`... 709,311
`5,790,789 A * 8/1998 Suarez ....................... TO9,202
`5,826,000 A 10, 1998 Hamilton
`5,860,010 A
`1, 1999 Attal .......................... 717,137
`5,864,669 A *
`1/1999 Osterman et al. ........... 709/203
`5,886,732 A
`3/1999 Humpleman
`5,909,183 A * 6/1999 Borgstahl et al. ...... 340,825.22
`5,940,072 A
`8/1999 Jahanghir et al.
`5,940,387 A
`8/1999 Humpleman
`5,953,526 A
`9/1999 Day et al. ................... 717/108
`5,956.487 A
`9, 1999 Venkatraman et al. ...... TO9.218
`6,005,861 A 12/1999 Humpleman
`6,020,924 A
`2/2000 Jahanghir
`6,032,202 A
`2/2000 Lea et al. ...................... T10/8
`OTHER PUBLICATIONS
`6,037.933 A
`3, 2000 Blonstein et al.
`Deng, Shuang "Capture effect in residential Ethernet LAN”.
`2.87.
`3. kia et al.
`IEEE GLOBEC, ISBN: 0-7803-2509-5, pp. 1678-1682,
`6,085,236 A
`7, 2000 Lea
`Nov. 1995.*
`6, 101499 A
`8, 2000 Ford et al.
`Kokubun, T. et al. “Object-oriented database system with
`6,128,619 A * 10/2000 Fogarasi et al. ............ 707/102
`GIS for optical cable network operation”, IEEE GLOBEC,
`6,134,594 A 10/2000 Helland et al. ............. 709,229
`6,151,624 A * 1 1/2000 Teare et al. ................. '7 ISBN: 0-7803-3336-5, pp. 1521-1527, Nov. 1996.
`6,175.362 B1
`1/2001 Harms et al.
`& G
`6,181,333 B1
`1/2001 Chaney et al.
`Corcoran, P.M. et al. “Browser-style interfaces to a home
`6,182,094 B1
`1/2001 Humpleman et al.
`automation network, IEEE Trans. on Consumer Electron
`6,188,397 B1
`2/2001 Humpleman
`ics, pp. 1063-1069, Jun. 1997.*
`6,189,019 B1* 2/2001 Blumer et al. .............. 707/513
`Corcoran, P.M. “Mapping home-network appliances to TCP/
`6, 191,781 B1
`2/2001 Chaney et al.
`IP sockets using hree-tiered home architecture', IEEE Trans
`6, 198479 B1* 3/2001 Humpleman et al. ....... 345,733
`on Consumer Electronics, pp. 729-736, Jun 1998.
`6.288,716 B1* 9/2001 Humpleman et al.
`345,733
`6,349,352 B1
`2/2002 Lea ............................. 71 Of72
`
`5, 1990
`5, 1991
`6, 1996
`9, 1996
`9, 1996
`10, 1996
`1, 1997
`10, 1997
`10, 1997
`10, 1997
`5, 1998
`3, 1999
`7, 1999
`11, 1999
`12/1999
`12/1999
`1, 1995
`11, 1995
`12/1995
`11, 1996
`5, 1997
`
`* cited by examiner
`
`VMware - Exhibit 1015
`VMware v. IV I - IPR2020-00470
`Page 2 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 1 of 14
`
`US 7,043,532 B1
`
`1 3
`
`?2.
`
`Building Blocks or Modules
`
`Aft 6.
`
`-se
`
`12
`
`Multiple Client Multiple Server one Network Cantrol Model
`SERVERS
`CLINTS SERVERS
`
`
`
`A. 6, 3
`
`Access Network
`
`
`
`Aecess Network
`
`2.
`
`vi
`w
`
`I c.
`
`ry as
`
`VMware - Exhibit 1015
`VMware v. IV I - IPR2020-00470
`Page 3 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 2 of 14
`
`US 7,043,532 B1
`
`f
`
`Z.
`
`Control System
`
`HUMAN
`
`e - Commutanications sis
`
`t
`
`CLENT
`
`SRVER
`
`FC. 3.
`
`
`
`
`
`Commands
`-- Communications - -
`
`W
`
`A76, ed
`
`S - gco Transfer Communication
`
`-
`
`20
`
`Module Elements for serverserver CL
`C
`
`2O
`
`Control
`Program
`
`Language
`interface
`
`
`
`
`
`SYER
`
`SERVER
`
`2 a.
`
`A, 61.
`
`VMware - Exhibit 1015
`VMware v. IV I - IPR2020-00470
`Page 4 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 3 of 14
`
`US 7,043,532 B1
`
`PDA (Remotec)
`
`- - - -
`Harran
`Control
`GUI
`- - -
`
`CLIENT
`
`2.
`
`DVDP
`Smart Card
`
`Service
`
`SOURCE
`SERVER
`hu
`
`oxes with Clients
`DTV (STB)
`- - -
`Euraan
`Control
`
`CLENT
`
`
`
`- SERVER
`
`at 4.6
`
`--a
`
`Boxes with Server(s)
`AUDIO
`AMPLETER
`
`-
`SNK
`SERVER
`
`Service
`
`DVCR
`
`Service
`
`SO
`SE OR SINK
`
`DBS Access Device
`s TV Access Device
`ATSC Access Device
`Internet Access Device
`
`BOXMANAGEMENT
`SERVER
`
`r - - -
`Service
`VERS
`- - REMOTE SER
`service
`r Service - h
`-
`
`l
`
`- - - -
`
`A. C. (b.
`
`VMware - Exhibit 1015
`VMware v. IV I - IPR2020-00470
`Page 5 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 4 of 14
`
`US 7,043,532 B1
`
`Basic AW, ClientServer/Server, Model
`
`P
`
`(,
`
`6, 8
`
`Control
`
`12
`
`in cntrol CENT
`GUI
`r
`r
`
`control
`
`
`
`fetch control
`object (GUI) / /
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`SERVER 1
`(data source
`eg DVCR)
`
`
`
`4Control
`
`Control
`
`SERVER 2
`(data sink
`eg /2 DTV)
`state
`clock
`
`s
`S
`
`g
`
`2.
`
`VMware - Exhibit 1015
`VMware v. IV I - IPR2020-00470
`Page 6 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 5 of 14
`
`US 7.043,532 B1
`
`Table 1: Capabilities Table
`
`
`
`Table 2: Attributes Table
`
`So
`
`also
`
`S.
`Defaultsink
`
`is a Big
`
`15 chars
`
`Deal
`
`data type,
`
`e
`
`Soce
`
`P
`
`ce P addres
`
`VMware - Exhibit 1015
`VMware v. IV I - IPR2020-00470
`Page 7 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 6 of 14
`
`US 7,043,532 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Message construction used by CAL and AVIC
`All-
`lt
`
`App layer a
`
`Message
`Compression
`
`Message
`String
`Construction
`
`Message construction of new XCEXML
`
`e -
`
`App layer <
`
`l
`
`Industry
`Standard
`orasat
`(vocabulary)
`
`
`
`
`
`SERVCE
`APPLICAtcN
`
`
`
`Servica
`APPLICAON
`
`VMware - Exhibit 1015
`VMware v. IV I - IPR2020-00470
`Page 8 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 7 of 14
`
`US 7,043,532 B1
`
`l
`
`XML AP block diagram
`
`"
`
`SERVICE (or device) B
`
`
`
`SERVICE (or device) A
`
`sub-set of
`
`
`
`NERFACED
`DoomentType definition
`to race XML Receive ite
`face for Methods Calls
`
`instance of service or device
`NRFACE-AM
`(or vulidity checking)
`(also Capabilities file)
`
`so
`
`XML TO C Message
`OOS-PABLE
`RUNME
`T
`
`APP
`A
`
`R
`2- O
`
`XML or NWORK
`
`fe. /5
`-a-
`
`ye Vua 3
`
`
`
`Transfer and Learn foreig NTERFACE-NSTANCEXML
`Deau ts. A
`
`TRANSFER XML OBECT
`
`
`
`Cueries of foreign
`App A interface by App 8
`cML CUSRY
`
`
`
`a lu-
`
`APP
`B
`
`CMessage TO XML
`LOCK-FTABE for
`RNME
`
`W
`
`29
`
`
`
`
`
`
`
`VMware - Exhibit 1015
`VMware v. IV I - IPR2020-00470
`Page 9 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 8 of 14
`
`US 7,043,532 B1
`
`as -- a--sus- a -wevwr is ow
`
`library (cases 3,4)
`
`suu Puss Movses so a pre-stropos are is season o was a soppaeans roo or
`
`surrous o
`
`... --------s massesso a sesaoud
`so--esurussessopodseer wompso wo M
`-----wasses soooossessess-s-s-a-s-s-s-s-s-s-srssooncorpoosas-a-oesous-soena ooo--see
`
`:
`(A2-moved)
`(A-moved)
`{A1-moved)
`; :
`; : APP. INTERFACE APP. INTERFACE APP. INTERFACE:
`; :
`Description(s)
`Description(s)
`Description(s)
`:
`of Application(C1):
`of Application(C)
`of Application(C2):
`1 guary
`
`Interface
`Case-3
`
`Transfer
`CaSO-3
`
`CONTROLLER
`CONTROLLED
`NETWORK
`TGuary T. iT
`AP
`(A)
`:
`Transfer case-2 :
`APP. INTERFACE
`Description(s)
`:- - - - - - - - - ... --- Description(s)
`:
`captation)
`of Application(C)
`
`
`
`
`
`
`
`:
`
`zo
`
`I r
`
`:
`
`HARDWARE
`OR
`SERVICE
`
`:
`.
`
`.
`
`Case 1. XML Application-C INTERFACE (A) for remote query by (E)
`Case 2. Move XML Application-C INTERFACE (A) to controller for local query by (E)
`Case 3. Move XML Application-C INTERFACE (A) to 3rd party device-a collecting
`place (library) for all Interfaces for remote query by (E). The library would have to
`have the address (URI) of the associated application available for direct control
`action and responses.
`Case 4. is case 3 but the "indirect control action (and response) is also directed at
`the library device. In this case the library is built with a "forwarder.
`
`Arch. / 7.
`
`VMware - Exhibit 1015
`VMware v. IV I - IPR2020-00470
`Page 10 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 9 of 14
`
`US 7,043,532 B1
`
`
`
`XCEXML Protocol Stack
`
`at Co. 18
`
`
`
`Connaud Translation Process.
`
`... or fulfil translation over the Internet
`
`VMware - Exhibit 1015
`VMware v. IV I - IPR2020-00470
`Page 11 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 10 of 14
`
`US 7,043,532 B1
`
`A
`/
`HN Device A: Controller Module
`
`
`
`a 2-
`
`Native interface (e.g. C)
`
`t
`
`HN Device B: Controlled Module
`
`4 2.
`
`- 4
`
`ad
`23
`
`aft
`232
`73
`
`176e
`
`7 -
`
`172
`
`1 3-
`
`- ld
`
`XMLRPC Action
`
`XML RPC Response
`
`X
`
`3
`23.
`8
`Interface Fetche
`f
`2- ce.
`
`Device A XML Interface
`
`Registerlupdate
`
`:
`
`Request Interface
`B from L.
`
`-
`Fetch Interface
`B from Device
`Fetch interface
`
`Register/Update
`Interface B
`
`es
`odec
`
`as:
`
`Controller App
`
`Controlled App
`
`-
`Broker
`
`i:
`
`xix:
`ii
`
`8x
`
`& 4
`62.
`
`hird Device or Central hub.
`with INMiddlewarc layer
`To internet
`CALL.DTD describes the message format of XMLRPC output. It can be used by receiver of
`XMLRPC message for validity check.
`
`& O
`
`o
`--
`
`A c. f
`
`VMware - Exhibit 1015
`VMware v. IV I - IPR2020-00470
`Page 12 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 11 of 14
`
`US 7,043,532 B1
`
`Relation between Interface Library & XCE Database
`
`
`
`
`
`
`
`
`
`Standardized
`Definition of
`Objects, Methods,
`
`Standardized CE
`Interface Document
`Type Definition
`(DTD)
`
`a on Pabh bands one be as sea as a see
`
`wo
`
`Home
`Network
`
`Y
`
`-
`
`2.
`
`2.
`
`n
`
`wn - - as a
`
`- - - - - - - -
`
`- - - - - - - - - - - - - - - - - ---------------- t
`-
`HNDL (Home Network Interface Library)
`"stige of
`/ o te
`- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
`Relation between Interface Library & XCE Database
`
`A d. 2 o
`
`4
`
`as a
`
`m r
`
`s
`
`72
`
`Register? pdatc interface
`Interface Library
`//? (1 Generating softwalesetti
`
`VMware - Exhibit 1015
`VMware v. IV I - IPR2020-00470
`Page 13 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 12 of 14
`
`US 7.043,532 B1
`
`HN Middleware
`Layer
`
`HNORB&L
`
`Connected to Internet
`
`Device A Interface
`
`ill
`
`:
`
`Record Category
`
`Play Category
`
`:
`
`:
`
`Device
`2
`
`Device B Enterface
`
`:
`
`Links, e
`XLink
`XPointer
`
`Interface
`
`Example of Hierarchical IL & Device Interfaces
`
`VMware - Exhibit 1015
`VMware v. IV I - IPR2020-00470
`Page 14 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 13 of 14
`
`US 7,043,532 B1
`
`layer
`
`2
`
`Hierarchical Interface Structure. Four layers.
`Interface Layer
`2.
`XML interface for each HN listing current available devices.
`7, 1 IG
`General XML interfaces for each device, listing function
`-
`4
`Specific XML interface for each function in a function category. I VCR record tdr function
`
`
`
`Interface Example
`MIN (VCRs, TV...)
`General VCR Interface.
`
`A. C. 2-2
`
`VMware - Exhibit 1015
`VMware v. IV I - IPR2020-00470
`Page 15 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 14 of 14
`
`US 7,043,532 B1
`
`Table 3: Example Formats and Types
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`image
`
`language
`
`jpeg, gif, bitmap,
`tiff
`English, French,
`etc.
`
`video clib
`text
`
`application
`
`avi, quicktime,
`npeg
`html, plain
`
`nSword, pdf,
`postscript, gzip
`
`A. G. 2.--
`
`VMware - Exhibit 1015
`VMware v. IV I - IPR2020-00470
`Page 16 of 35
`
`

`

`US 7,043,532 B1
`
`1.
`METHOD AND APPARATUS FOR
`UNIVERSALLY ACCESSIBLE COMMAND
`AND CONTROL INFORMATION IN A
`NETWORK
`
`CROSS-REFERENCES TO RELATED
`APPLICATIONS
`
`Applicant claims the benefit of U.S. Provisional Applica
`tion No. 60/084.578 entitled “Command and Control Using
`XML, filed on May 7, 1998 which is incorporated herein by
`reference. The present application is related to the following
`copending applications that are commonly assigned and
`which are incorporated herein by reference: U.S. patent
`application Ser. No. 09/104,299, entitled “Browser Based
`Command and Control Home Network” now U.S. Pat. No.
`6.288,716; U.S. patent application Ser. No. 09/104,297,
`entitled “Method and Apparatus for A Home Network Auto
`Tree Builder”; U.S. patent application Ser. No. 09/104,298,
`entitled “Improved Home Network, Browser Based, Com
`mand and Control” now U.S. Pat. No. 6,198.479; U.S. patent
`application Ser. No. 9/103,469, entitled “Method and Appa
`ratus for Creating Home Network Macros' now U.S. Pat.
`No. 6.243,707; and U.S. patent application Ser. No. 09/104,
`606, entitled “Programming Tool For Home Networks now
`U.S. Pat. No. 6,182,904.
`
`10
`
`15
`
`25
`
`FIELD OF THE INVENTION
`
`The present invention relates to the field of network
`systems, and more particularly, to home network having
`multiple devices connected thereto.
`
`30
`
`BACKGROUND OF THE INVENTION
`
`A network generally includes a communication link and
`various devices with communication capability connected to
`the communication link. The devices include computers,
`peripheral devices, routers, storage devices, and appliances
`with processors and communication interfaces. An example
`of a network is a home network for a household in which
`various devices are interconnected. A usual household can
`contain several devices including personal computers and
`home devices that are typically found in the home. As Such
`the term “device' typically includes logical devices or other
`units having functionality and an ability to exchange data,
`and can include not only all home devices but also general
`purpose computers. Home devices include Such electronic
`devices as security systems, theater equipment, TVS, VCRs,
`Stereo equipment, and direct broadcast satellite services or
`(DBSS), also known as digital satellite services (DSS),
`sprinkler systems, lighting systems, micro waves, dish
`washer, ovens/stoves, washers/dryers, and a processing sys
`tem in an automobile.
`In general, home devices are used to perform tasks that
`enhance a homeowner's life style and standard of living. For
`example, a dishwasher performs the task of washing dirty
`dishes and relieves the homeowner of having to wash the
`dishes by hand. A VCR can record a TV program to allow
`a homeowner to watch a particular program at a later time.
`Security systems protect the homeowner's valuables and can
`reduce the homeowner's fear of unwanted entry.
`Home devices, such as home theater equipment, are often
`controlled using a single common control unit, namely a
`remote control device. This single common control unit
`allows a homeowner to control and command several dif
`ferent home devices using a single interface. Thus, may
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`manufacturers have developed control units for controlling
`and commanding their home devices from a single interface.
`One drawback associated with using the remote control
`unit to command and control home devices is that it provides
`static and command logic for controlling and commanding
`each home device. Another drawback associated with using
`remote control units is that known remote control units
`cannot control a plurality of diverse devices, and more
`particularly, cannot control a plurality of devices having
`different capabilities to communicate with each other in
`order to accomplish tasks or provide a service.
`In conventional network systems a user provides com
`mands using a remote control unit or device control panel.
`Once the user ceases, there is no controller unit or device in
`the network to provide commands for automatic operation.
`After a user initially controls and commands a first set of
`devices, conventional systems do not provide a mechanism
`for the first set of devices to automatically communicate
`with a second set of devices in the network as necessary in
`order to accomplish tasks without direct user control and
`command of the second set of devices. Further, conventional
`systems do not provide an efficient method for various
`network devices to obtain information about other network
`devices in the network for command and control.
`There is, therefore, a need for a method and a system
`which provides dynamic control and command devices in a
`home network. There is also a need for such a method and
`system to provide the ability to control a plurality of diverse
`devices having different capabilities to communicate with
`each other in order to accomplish tasks or provide a service.
`There is also a need for such a method and system to provide
`the ability for various network devices to automatically
`command and control other various network devices. There
`is also a need for Such a method and system to provide
`universally accessible command and control information for
`inter-device communication.
`
`SUMMARY OF THE INVENTION
`
`The present invention satisfies these needs. In one
`embodiment the present invention provides a method and
`system for performing a service on a home network, by:
`connecting a first and a second home device to the home
`network; providing a database including a plurality of appli
`cation interface description data objects, where each appli
`cation interface description data object includes information
`in a structured format for commanding and controlling of a
`home device by one or more other home devices connected
`to the network; the second home device accessing a first
`application interface description object for the first home
`device in the database; the first home device accessing a
`second application interface description object for the sec
`ond home device in the database; sending control and
`command data from the first home device to the second
`home device utilizing the second application interface
`description object over the network; and sending control and
`command data from the second home device to the first
`home device utilizing the first application interface descrip
`tion object over the network. Whereby, the first and second
`home devices perform said service.
`In one version of the invention, the first home device
`stores first application interface data therein, and the second
`home device stores second application interface data therein.
`The database is formed by querying the first and second
`home devices to transfer said application interface data for
`the first and second home devices to the database device.
`
`VMware - Exhibit 1015
`VMware v. IV I - IPR2020-00470
`Page 17 of 35
`
`

`

`US 7,043,532 B1
`
`3
`The database can be stored in a database device and con
`nected to the network for universal access by network
`devices. In that case, the first application interface descrip
`tion object for the first home device can be provided from
`the database to the second home device over the network.
`Further, the second application interface description object
`can be provided from the database to the first home device
`over the network.
`Further, three or more home devices can be connected to
`the network, wherein at least one home device accesses the
`database to query the application interface description
`objects of a plurality of home devices for sending command
`and control data to the plurality of home devices over the
`network. Each application interface description object can
`include data in a structured format. The structured format
`can include XML format.
`
`10
`
`15
`
`4
`FIG. 19 shows another embodiment of server device to
`server device command and control architecture;
`FIG. 20 shows the relationship between a device interface
`library and consumer electronics definition data base for
`home devices;
`FIG. 21 shows hierarchal form of an embodiment of a
`device interface definition;
`FIG. 22 shows an example of layers in device interface
`definition of FIG. 21;
`FIG. 23 shows a command transmission and interpreta
`tion process between a sender and receiver device; and
`FIG. 24 shows an example table of a partial list of packet
`types and formats for providing translation services accord
`ing to an aspect of the present invention.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`In one aspect, the present invention provides inter-device
`communication in a network Such as a home network. As
`home devices become more intelligent and can share infor
`mation, inter-device communication allows a user to inter
`connect devices in a network to take advantage of the
`information sharing capabilities of those devices. As such,
`inter-device communication plays a crucial role in affording
`a user with the ability to fully and flexibly utilize the
`networked devices.
`Referring to FIG. 1, in an embodiment of the present
`invention, a network 10 includes at least one client device 12
`and at least one server device 14 interconnected via a
`communication link 16. The communication link 16 can
`include a 1394 serial bus providing a physical layer (me
`dium) for sending and receiving data between the various
`connected home devices. The 1394 serial bus supports both
`time-multiplexed audio/video (A/V) streams and standard IP
`(Internet Protocol) communications. In certain embodi
`ments, a home network uses an IP network layer as the
`communication layer for the home network. However, other
`communication protocols could be used to provide commu
`nication for the home network.
`Each client device 12 may communicate with one or more
`server devices 14 in the network 10. Further, each server
`device 14 may communicate with one or more other server
`devices 14, and one or more client devices 12, in the network
`10. Each client device 12 can include a user communication
`interface including input devices such as a mouse and
`keyboard for receiving user input, and a display for provid
`ing a control user interface for a user to interact with the
`networked devices. The user interface can include a graphi
`cal user interface (GUI) display 18 for providing informa
`tion to the user. Referring to FIG. 2, as defined herein, each
`server device 14 provides a service for the user, except
`control user interface, and each client device 12 provides
`control user interface for user interaction with the network
`10. As such, only client devices 12 interact directly with
`users, and server devices 14 interact only with client devices
`12 and other server devices 14. Example services can
`include MPEG sourcing/sinking and display services.
`FIG. 3 shows a block diagram of an example home
`network 10 that includes a plurality of client devices 12 and
`a plurality of server devices 14. Each server device 14 may
`include hardware as a resource in the network for providing
`services to the user. Further, each server device 14 may store
`a server or service control program 20 for controlling the
`server hardware, and include a graphical control object
`(GCO) user interface description 22 for user interface with
`the server control program 20 as shown in FIG. 4.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`25
`
`30
`
`35
`
`40
`
`45
`
`These and other features, aspects and advantages of the
`present invention will become better understood with regard
`to the following description, appended claims and accom
`panying drawings where:
`FIG. 1 shows a block diagram on an embodiment of a
`network according to one aspect of the present invention;
`FIG. 2 shows the block diagram of FIG. 1 in an example
`device control and communication scenario;
`FIG. 3 shows a block diagram of an example home
`network system according to the present invention which
`includes a plurality of client and server devices;
`FIG. 4 shows a block diagram of example embodiments
`of a client device and a server device of FIG. 3;
`FIG. 5 shows example embodiments of client devices;
`FIG. 6 shows example embodiments of server devices:
`FIG. 7 shows a block diagram of two example networked
`server devices capable of communication with, and control
`of one another;
`FIG. 8 shows a block diagram of an example architecture
`of an audio/video (A/V) model including examples of a
`Source server device, a sink server device and a client device
`in a network;
`FIG. 9 shows another example audio/video (A/V) model;
`FIG. 10 shows an example capabilities data table for a
`network device;
`FIG. 11 shows an example attribute data table for a
`network device;
`FIG. 12 shows an example configuration of building
`blocks for generating command messages among networked
`devices;
`50
`FIG. 13 shows another example configuration of the
`building blocks of FIG. 12 for generating command mes
`SageS.
`FIG. 14 shows three examples of interaction among
`networked client and server devices;
`FIG. 15 shows an example block diagram for definitions
`of API extensions of networked device interfaces;
`FIG.16 shows an example architecture for a server device
`application accessing the interface description document of
`another server device;
`FIG. 17 shows another example inter-device control
`architecture between a controller server device and a con
`trolled server device;
`FIG. 18 shows an embodiment of an XML protocol
`providing a Web standard common middleware layer in a
`communication stack at the API level between networked
`devices;
`
`55
`
`60
`
`65
`
`VMware - Exhibit 1015
`VMware v. IV I - IPR2020-00470
`Page 18 of 35
`
`

`

`US 7,043,532 B1
`
`5
`For control between a controlling client device 12 and a
`controlled server device 14, the client device 12 accesses the
`GCO 22 of the server device 14 by, for example, transferring
`the GCO 22 from the server device 14 to the client device
`12 over the network. The client device 12 then uses the
`transferred GCO 22 to create a control user interface GUI 18
`for the user to communicate with the control program 20 of
`the server device 14 from the client device 12 over the
`network. The user provides command and control to at least
`the control program 20 of the server device 14 from the
`client device 12.
`Storing the GCO 22 of each server device 14 in the server
`device itself may reduce the processing and storage require
`ments of the client devices 12 in networks with several
`server devices 14. Further, storing the GCOS 22 in the server
`devices 14 may allow each server device 14 to provide its
`own GUI look and feel to the user, and allows for modifi
`cation or updating of the GCOS 22 without modifications to
`client devices 12.
`Referring to FIG. 4, to provide command and control
`between a client device 12 and the server device 14, in one
`embodiment, the client device 12 can include a renderer 24
`for displaying a GUI 18 using a GCO 22 stored in the client
`device 12 or transferred to the client device 12 over the
`network from a desired server device 14. For example, in an
`initial device selection phase, the client device 12 can fetch
`the GCO 22 of at least one server device 14 over the
`network, and the renderer 24 displays a GUI 18 using the
`GCO 22 for controlling the server device 14. Preferably, the
`GUI 18 is customized to the server device 14 and can
`include a built-in command set for controlling the server
`device 14.
`In addition, the GUIs 18 of various server devices 14 may
`include commonalities such as: (1) a common GCO model
`type for the client device renderer 24 to display GUIs 18, (2)
`common communication protocols for transferring the
`GCOs 22 from various server devices 14 to the client device
`12, and (3) common communication protocols for GUI
`interaction from the client device 12 to the control program
`20 of the corresponding server device 14, wherein the client
`device 12 does not require a built-in knowledge of a par
`ticular server device 14 being controlled.
`Referring still to FIG. 4, a server device 14 may include
`one or more server control programs 20 to control the server
`hardware for providing a service. The GUI interface 18 from
`the GCO 22 of the server device 14 provides interface to the
`server device control programs 20. The server device 14
`may also include control state data 26 indicating the control
`status of the server device 14 and server device hardware in
`providing a requested service.
`For example, the control state data 26 can include the
`status of control information in the GUI 18 for the server
`device 14, Such as timer setup for a recording action in a
`VCR server device. The control state data 26 is stored in the
`controlled server device 14, and displayed to a user through
`the GUI 18 of the server device 14 at the controlling client
`device 12, for user control of the server device 14. Prefer
`ably, the controlling client device 12 for displaying the GUI
`18 of the server device 14 does not retain knowledge of the
`control state data 26 for the controlled server device 14.
`Each server device 14 can be controlled by one or more
`client devices 12. As such, the control state data 2