United States Patent 15
`Warman et al.
`
`[54] METHOD AND APPARATUS FOR
`CONTROLLING NON-COMPUTER SYSTEM
`DEVICES BY MANIPULATING A
`GRAPHICAL REPRESENTATION
`
`[75]
`SS
`
`Inventors: David J. Warman, Bainbridge Island;
`MarkA. Lacas; Geoffrey P. Coco,
`both of Seattle, all of Wash.
`
`[73] Assignee: MediaLink Technologies Corporation,
`Seattle, Wash.
`
`[21] Appl. No.: 334,416
`
`[22] Filed:
`
`Nov. 4, 1994
`
`Related U.S. Application Data
`
`[63] Continuation-in-part of Ser. No. 307,864, Sep. 16, 1994,
`abandoned.
`
`Taat, C16 cccccccccccssssssesscsecsccssesssseesesssseenene GOSB 15/02
`[52]
`[52] U.S. Ch wesc
`364/188; 364/146; 395/348
`
`
`[58] Field of Search...
`csscsessesssesees 364/140, 141,
`364/145, 146, 188, 189, 191; 395/155,
`161, 169
`
`[56]
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`7/1981 Beckedorff et al...eeescee 364/146
`4,279,012
`7/1989 Bhaskar et al.
`.
`4,849,880
`6/1991 Wexelblat et al...csecscnsene 364/146
`5,021,976
`5,062,060 10/1991 Kolnick............
`.
`364/188
`5,086,385
`2/1992 Launey etal.
`364/147
`5,321,829
`6/1994 Zifferen ........
`8/1994 Kolnick........
`.- 395/164
`9,335,323
`
`3/1995 Willson et al. oo...ssescccescseees 364/146
`5,400,246
`
`US005657221A
`
`(11] Patent Number:
`
`[45] Date of Patent:
`
`5,657,221
`Aug. 12, 1997
`
`FOREIGN PATENT DOCUMENTS
`
`0596594
`
`5/1994 European Pat. Off. .
`
`Primary Examiner—Paul P. Gordon
`Attorney, Agent, or Firm—Christensen, O’ Connor, Johnson
`& Kindness PLLC
`
`[57]
`
`ABSTRACT
`
`invention
`The graphical control system of the present
`includes a computer (20), a device interface 5) for a
`non-computer system device (21) having at least one feature
`control (29a) (or display (29b)), a bus network (28) con-
`necting the computer (20) to the device interface (35), and
`a visual network operating system (78) based on an object-
`oriented programming paradigm. The device interface (35)
`comnects the non-computer system device (21) to the bus
`network (28) and provides the mechanism for converting
`computer-generated commandsinto signals for controlling
`the operation of the feature control (29a) of the non-
`computer system device (21). The visual network operating
`system (78)is a distributed operating system thatis partially
`stored on the computer (20) andpartially stored in the device
`interface (35). A computer portion causes the computer (20)
`to generate or create a visual device control (40) that
`graphically replicates the feature control (29a) normally
`associated with the non-computer system device (21). The
`visual device control (40) is operated by a conventional
`graphical control device, such as a mouse (32), track ball,
`touch screen, joystick, etc. As the replicated controls are
`operated, the computer (20) sends messages to the device
`interfaces (35), which cause the feature control (29a) of the
`non-computer system device (21) to respond in the same
`way it would have responded had the feature control been
`operated manually, electrically or effected by another com-
`puter on the bus network (28).
`
`74 Claims, 20 Drawing Sheets
`
`¢‘Ormitoam
`
`\Cer A
`
`Google Exhibit 1047
`Google v. VirtaMove
`
`Google Exhibit 1047
`Google v. VirtaMove
`
`

`

`U.S. Patent
`
`Aug. 12, 1997
`
`Sheet 1 of 20
`
`5,657,221
`
`
`24Ns
`ae
`
`™~
`
`&&
`
`29a
`
`©“
`
`

`

`U.S. Patent
`
`Aug. 12, 1997
`
`Sheet 2 of 20
`
`5,657,221
`
`\
`© 5
`%
`oo /
`
`a
`
`a
`rs v8
`
`LLPeay
`5s|41
`
`BRIDCE
`
`4
`
`22
`
`x
`Na
`
`20
`
`ee
`
`45
`
`rs
`
`ee
`
`42
`
`44
`
`

`

`Aug.12, 1997
`
`a2
`
`294
`
`296
`
`290
`
`XY
`
`S=3
`
`CHANNEL1
`
`US. Patent
`
`Sheet 3 of 20
`
`5,657,221
`
`

`

`US. Patent
`
`Aug. 12, 1997
`
`Sheet 4 of 20
`
`5,657,221
`
`,
`
`510
`
`30 530 540 550
`
`PAPATATATAAToeVVV
`
`EEaaa!
`POAeeooeeooeeeeoe
`PEVONVAee
`
`

`

`USS. Patent
`
`Aug. 12, 1997
`
`Sheet 5 of 20
`
`5,657,221
`
`78
`
`VISUAL NETWORK
`OPERATING
`SYSTEM
`
`
`
`
`DATABASE
`STORAGE
`
`66
`
`67
`
`68
`
`

`

`U.S. Patent
`
`Wy
`
`Sheet 6 of 20
`
`5,657,221
`
`Aug. 12, 1997
`
`LLL
`
`

`

`U.S. Patent
`
`
`
`GEVALITIAT
`
`40LNIWLSALavy
`
`
`
`AVIdSI@0TOULNODdFItAda
`
`Aug. 12, 1997
`
`Sheet 7 of 20
`
`5,657,221
`
`+9A¥IdSI@YOTOYLNOD
`
`HOIATGAG
`
`
`
`(HINISTadTdINVA
`
`PEECERARY
`
`“ILVINdGINVW
`
`FAISITY/TLVIINNAKO)
`
`JOQTVANISONVHO
`
`WOud/OL
`
`
`SHOAITYTHLOGNV
`
`
`
`
`beYALNdNO)TVNOSHId
`
`
`
`TOULNODTILTHAVYDVdiVYdNnd)
`
`
`
`INIGNTINIAVIdSIC
`
`OTAINOLSVATLV08
`
`
`
`LTdddaINI
`
`FIVINdINVW
`
`
`
`QIINISTUdTYANTVA
`
`JOGAAG18
`
`
`
`
`
`TAITITH/TLVIINQWWOD
`
`SQIVANISONVHO
`
`
`
`JOIATGWOUA/OL
`
`ce
`
`BL
`
`
`
`
`
`

`

`US. Patent
`
`Aug. 12, 1997
`
`Sheet 8 of 20
`
`5,657,221
`
`98
`
`96
`
`92
`
`94
`
`90
`
`

`

`U.S. Patent
`
`Aug. 12, 1997
`
`Sheet 9 of 20
`
`5,657,221
`
`TINVESNI
`
`LANIVd
`
`YINVISNI
`
`SSVTI
`
`LIXIVE
`
`LITHO
`
`HOTAII
`
`LIE
`
`(FIA)INIWITYA
`
`FINIEIATS
`
`LITLEO
`
`ANT¥A
`
`TOULNOD
`
`TVASIA
`
`LITE
`
`ANTVA
`
`TOULNOD
`
`JONVISNI
`
`(FIA)LNIWITI
`
`AFINTUAATS
`FOINVESNI
`
`TVASIA
`
`MOCUNIM
`
`LIGTLEO
`
`MOCNIM
`
`FINVISNI
`
`
`
`
`
`
`

`

`oh3<
`
`ccR=ef
`
`CB)wohe
`
`
`
`
`
`SS¥T9
`
`PIG
`
`

`

`Aug. 12, 1997
`
`Sheet 11 of 20
`
`5,657,221
`
`
`
`TOINVISNIMOGNIM
`
`e&h
`
`T¥ASIA
`
`YINVISNI
`
`crt
`
`TVASIA
`
`TINVISNI
`
`U.S. Patent YeOLPus
`
`
`
`AYOLNYANINUOMLAIN
`
`col
`
`TVASIA
`
`HONTATALY
`
`FINVESNI
`
`9&1}
`
`oF}
`
`9¢]
`
`Bel
`
`INIWOOINT
`
`LINIVd
`
`GINVLISNI
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Aug. 12, 1997
`
`Sheet 12 of 20
`
`3,657,221
`
`T¥ASIA
`
`INGTTIATE
`
`
`
`YINVESNIMOGNIM
`
`TVASIA
`
`8A}
`
`

`

`U.S. Patent
`
`Aug. 12, 1997
`
`Sheet 13 of 20
`
`5,657,221
`
`
`
`TINVISNTMOGNIM
`
`TINVdYdIAITANVY
`
`TVASIA
`
`
`
`|—| TWASIA
`
`
`
`
`

`

`US. Patent
`
`Aug. 12, 1997
`
`Sheet 14 of 20
`
`5,657,221
`
`nat
`
`eeNINyyNIN
`
`Ld
`
`
`
`FINVISNIMOGNIM
`
`MHL
`
`

`

`Aug. 12, 1997
`
`Sheet 15 of 20
`
`5,657,221
`
` HINVESNI
`[orToor
`
`U.S. Patent
`
`=
`
`MOUNIM
`
`
`
`TINIETATA
`
`FINVLISNI
`
`TYASTA
`
`
`
`TONIYTATA
`
`JONVISNI
`
`TVASIA
`
`
`
`
`

`

`Sheet 16 of 20
`
`5,657,221
`
`MOGNIM|MIAsn|—|
`FINVLSNI
`
`U.S. Patent
`
`Aug. 12, 1997
`
`LIMIVE
` INVISNI
`
`

`

`U.S. Patent
`
`Aug. 12, 1997
`
`Sheet 17 of 20
`
`5,657,221
`
`MOCUNIM|AIAUS|
`SONVISNI
`
`
`INVISNI TVNSIA
`
`TVOSTA
`
`ve
`
`TOINVISN
`
`TWASIA
`bee/
`
`ote/b0¢
`
`bie
`
`9é¢
`
`
`
`
`
`
`
`
`

`

`5,657,221
`
`109
`
`119
`
`CLASS
`
`INSTANCE
`
`U.S. Patent
`
`Aug. 12, 1997
`
`Sheet 18 of 20
`
`INTERFACE
`PACKET
`OBJECT
`
`DEVICE CONTROL
`ELEMENT (DCE)
`OBJECT
`
`INSTANCE
`
`INTERFACE
`PACKET
`INSTANCE
`
`DEVICE CONTROL
`ELEMENT (DCE)
`
`Fig. 17.4
`
`

`

`U.S. Patent
`
`5,657,221
`
`Aug. 12, 1997
`
`Sheet 19 of 20
`
`INSTANCE
`
`CLASS
`
`

`

`US. Patent
`
`Aug. 12, 1997
`
`Sheet 20 of 20
`
`5,657,221
`
`66
`
`D06¢
`
`6IéAlé
`
`INIOILNO
`
`LAMIVd
`
`FINVISNI
`
`£e¢@
`
`ININOINI
`
`LINIVd
`
`FINVISNI
`
`
`

`

`5,657,221
`
`1
`METHOD AND APPARATUS FOR
`CONTROLLING NON-COMPUTER SYSTEM
`DEVICES BY MANIPULATING A
`GRAPHICAL REPRESENTATION
`
`RELATIONSHIP TO OTHER APPLICATIONS
`
`This application is a continuation-in-part of U.S. patent
`application Ser. No. 08/307,864, filed Sep. 16, 1994, now
`abandoned as of Dec. 26, 1995 and entitled A Method and
`Apparatus for Controlling Non-Computer System Devices
`By Manipulating a Graphical Representation. The subject
`matter of application Ser. No. 08/307,864 is incorporated
`herein by reference.
`FIELD OF THE INVENTION
`
`This invention generally relates to a method and apparatus
`for using a computer to control non-computer system
`devices connected to the computer.
`BACKGROUND OF THE INVENTION
`
`Object-oriented programming paradigms have become an
`increasingly commontool in computer programming. Such
`paradigms are often employed by graphical user interfaces,
`where computer system elements are visually represented
`and manipulated by visible screen entities such as icons on
`a computer screen or other display device. For programming
`purposes, “objects” are used to represent the manipulatable
`computer system. elements by containing methods and data
`that define those elements. By representing such computer
`system elements as objects, it is unnecessary for the pro-
`grammer to generate a specific set of code for each computer
`system element. Rather, the programmer can define classes
`of objects and assign certain universal behaviors to each
`class. Computer system. elements that can be represented by
`objects include computer peripherals and computer applica-
`tion programs. Examples of application programs are
`spreadsheets, word processing programs, database
`programs, etc.
`In graphical user interfaces employing an object-oriented
`programming paradigm, application programs are typically
`represented to a user by an icon displayed within a window
`on a computer screen, one icon for each application program
`that can be run. Execution of an application program is
`initiated by selecting its corresponding icon, most often
`using a pointing device such as a mouse. When an applica-
`tion program is selected, a message is sent to the corre-
`sponding application program object, indicating that the
`application program object is to invoke certain ofits meth-
`ods. For example, if a word processing program is selected,
`the methods contained within the application program object
`may includestarting the word processing program. The user
`mayalso “drag” icons from one area ofthe screen to another,
`or from one window to another using the mouse. The user
`may even “drag” one icon representative of an application
`object and “drop”that icon on top of another. This “object-
`object” interaction will result in a combination of applica-
`tion objects. For example, if a word processing document
`icon is dropped upon a word processing program icon, the
`object-object interaction results in starting the word process-
`ing program and causing that program to open the word
`processing document. This is possible because both the word
`processing program and the word processing document have
`been represented as compatible application program objects.
`Hence, the icons in the object-oriented programming para-
`digm allow the user to graphically control various computer
`system elements and the interrelationships between com-
`puter system elements.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`2
`While the conventional object-oriented, graphical user
`interface described above has been used to allow a user to
`initiate execution of such computer system elements as
`applications programs, use of object-oriented programming
`5 paradigms to graphically control and monitor non-computer
`system devices has been severely limited. Non-computer
`system devices include virtually any electronic device
`equipped with the necessary hardware to be connected to a
`personal computer either directly or via a network. For
`example, non-computer system devices may include lamps,
`television sets, VCRs, video cameras,
`telephones,
`amplifiers, CD players, equalizers, etc. Such devices typi-
`cally come equipped with a variety of feature controls and
`displays for operation including volume controls, power
`switches, input and output meters, channel selectors, etc. In
`order to control a non-computer system device using a
`graphical user interface, in the past, each feature control
`(e.g., a volume control) or feature display (e.g., an output
`meter) that is to be manipulated by the user has been coupled
`to the graphical user interface computer by specially
`designed software, resulting in a visible screen entity for
`each feature control of the device being displayed on the
`computer screen.
`Conventionally, non-computer system devices have been
`graphically represented and controlled via a special program
`designed specifically for each device, where the specially
`designed program is either built into the computer operating
`system or loaded into the computer operating system as an
`add-on software product. For example, the CD Remote
`program, Version 1.3 for the Macintosh computer provides
`the computer user with a graphic interface for controlling a
`Macintosh CD player hardwired to the computer. The useris
`provided with a graphical display of the CD player’s control
`panel, complete with graphical stop, play, pause, skip, etc.
`controls. To initiate execution of any one of these commands
`upon the CD player, the user merely selects the correspond-
`ing graphical control using the mouse. Unlike graphical user
`interface programs for controlling applications programs,
`the graphical controls are not visible screen entities that the
`user can “drag and drop” or “cut and paste” into other areas
`of the screen, into other windows, or on top of one another.
`Instead, the placement, position and execution of each of the
`graphical controls is predefined by the specially designed
`CD Remote program for the CD player.
`In contrast to the Macintosh CD player, non-computer
`system devices are normally not used alone. Instead, non-
`computer system devices are used in combination with other
`non-computer system devices. For example, a stereo system
`often comprises at least a CD player, an equalizer and an
`amplifier, each of which may be made by the same or a
`different manufacturer. In order to use a personal computer
`to operate an entire stereo system using the approach
`employed by the CD Remote program referred to above,
`each stereo system device would be necessarily be hard-
`wired to the personal computer and a specially designed
`program comprising specific sets of code would be neces-
`sary for controlling each device, each feature control, and
`each feature display on each device, and for controlling the
`relationships between the devices and the feature controls
`and displays. In addition, the code would have to be spe-
`cially designed for each manufacturer’s device. Evenif the
`devices were connected to the personal computer via a
`network as opposedto directly hardwired, specialized code
`would still be required to control each device because, prior
`to the present invention, a graphic user interface employing
`an object-oriented programming paradigm for controlling
`stereo devices produced by varying manufacturers did not
`exist.
`
`35
`
`40
`
`45
`
`50
`
`35
`
`60
`
`65
`
`

`

`5,657,221
`
`3
`Accordingly, there is a needfor a graphical control system
`for controlling non-computer system devices and the rela-
`tionships between those devices. In order to eliminate the
`need for specially designed code for each device, such a
`graphical control system should employ a common para-
`digm for representing the non-computer system devices to
`be controlled. In addition,
`the graphical control system
`should provide for dynamic visual device controls that
`represent each feature control of a device and allow a user
`to graphically control and monitor each device without
`having any specific knowledge about the device and without
`making any physical contact with the device. The present
`invention is directed to providing such a graphical control
`system.
`
`SUMMARY OF THE INVENTION
`
`15
`
`In accordance with this invention, a graphical control
`system for controlling non-computer system devices is pro-
`vided. Non-computer system devices are devices that can be
`electronically controlled such as musical amplifiers and
`equalizers, television sets, VCRs, video cameras, etc. Vir-
`tually any electronic device equipped with the necessary
`hardware to be either directly or indirectly controlled by a
`computer,
`including a simple on/off switch, can form a
`non-computer system device. The graphical control system
`includes a computer, an interface(s) for a non-computer
`system device(s) having at
`least one feature control, a
`coupling medium for coupling the computerto the interface
`(s) and a visual network operating system (VNOS). The
`coupling medium can be a bus network or simply a point-
`to-point connection between a computer and a single non-
`computer system device. Various typed of bus networks can
`be used including fiber optics, co-axial cable and twisted
`wire pair. The coupling medium could also include a radio
`frequency coupling medium, such as a radio frequency
`coupling network. The interface(s) connects the non-
`computer system device(s) to the bus network and provides
`the mechanism for converting computer-generated com-
`mands into signals for controlling the operation of the
`non-computer system devices.
`The visual network operating system (VNOS) is a dis-
`tributed operating system that is partially stored in a com-
`puter and partially stored in the interface(s). The major
`portion is stored in the computer. The computer portion
`causes the computer to generate or create graphical controls
`that represent the type of feature controls normally associ-
`ated with the non-computer system devices connected to the
`network. For example, in the case of a musical amplifier,
`such controls may include an on/off switch, an input meter
`and a volumecontrol. An equalizer would include boost and
`cut controls for frequency bands. The graphical controls are
`operated by a conventional graphical control device, such as
`a mouse, track ball, touch screen, joystick, etc. As the
`graphical controls are operated, the computer sends mes-
`sages to the interface(s), which cause the non-computer
`system devices to respond in the same way the devices
`would have responded to the manual or electrical operation
`of the equivalent feature controls.
`In accordance with other aspects of this invention, the
`visual network operating system (VNOS)is based on an
`object-oriented programming paradigm in which objects are
`interoperable and are organized into classes in a hierarchical
`fashion.
`
`In accordance with further aspects of this invention, five
`different types of objects are employed by the invention.
`They are: window objects; visual reference objects; value
`
`60
`
`4
`control element (VCE) objects; device objects; and packet
`objects. Window objects contain the data and methods
`necessary for displaying a window on the screen of the
`computer display. Visual reference objects contain the meth-
`ods and data necessary for displaying and associating icons
`and visual device controls (VDCs) with a window object. A
`VCE object contains a value represented by the VDC and
`controlled by setting the VDC, and methods for manipulat-
`ing the value. A device object contains the methods and data
`for communicating with particular types of non-computer
`system devices and for managing a graphical representation
`of a non-computer system device. Packet objects contain the
`methods and data for communicating data between the
`computer and non-computer system devices via the network.
`In accordance with yet other aspects of this invention, the
`window, visual reference, VCE, device, and packet objects
`are subdivided into lower order subclasses.
`
`As can be readily appreciated from the foregoing
`summary, the invention provides a graphical control system
`for controlling a wide variety of non-computer system
`devices. Virtually any electronic device equipped with the
`necessary hardware can be controlled. The graphical control
`system creates visual device controls that represent the
`feature controls and feature displays normally associated
`with the non-computer system devicethatis to be controlled.
`The visual device controls are operated by a conventional
`user manipulation device, such as a mouse, track ball, touch
`screen, joystick, etc. The invention eliminates the need for
`specially designed code for each non-computer system
`device by employing an object-oriented paradigm to repre-
`sent the non-computer system devices. While the invention
`is designed for use, in part, in the forward direction wherein
`the computer controls non-computer system devices, in
`some versions of the invention it is desirable for the non-
`computer system devices to be subject to manual or elec-
`trical control, monitoring, or control by another computer. In
`such cases data representing control changes are sent from
`the non-computer system devices and/or other computers to
`all other non-computer devices and personal computers on
`the network which store object data related to the control
`being changed. As a result, any associated visual device
`control is updated and monitored.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The foregoing aspects and many of the attendant advan-
`tages of this invention will become morereadily appreciated
`as the same becomes better understood by reference to the
`following detailed description, when taken in conjunction
`with the accompanying drawings, wherein:
`FIG.1 is a pictorial diagram of a preferred embodiment
`of a graphical control system formed in accordance with the
`present invention that includes a personal computer, a bus
`network and a plurality of devices controlled by a graphical
`control program incorporated in the personal computer;
`FIG. 2 is a block diagram of the personal computer,
`devices and bus network shown in FIG. 1;
`FIG. 3 is an isometric view of an amplifier;
`FIG.4 is an isometric view of a typical personal computer
`suitable for use in the graphical control system shown in
`FIG.1;
`FIG. 5 is a block diagram of several components of the
`personal computer shown in FIG. 4 that are used by the
`graphical control system of the present invention;
`FIG.6 illustrates a typical Amplifier Panel window pro-
`duced by the graphical control system shown in FIG.1;
`
`

`

`5,657,221
`
`5
`FIG..7 is a flow chart illustrating the functions provided
`by the graphical control system shown in FIG. 1,
`FIG. 8 is a graphical representation of an object imple-
`mented in the graphical control system shown in FIG. 1;
`FIG. 9A is a hierarchical diagram of an object-oriented
`programming paradigm implemented in the graphical con-
`trol system shown in FIG. 1;
`FIG.9B illustrates the methods and data provided by the
`object-oriented programming paradigm depicted in FIG. 8;
`FIG. 10illustrates the flow of messages between instances
`of objects when the graphical control system shownin FIG.
`1 first encounters a device on the network;
`FIG.11 illustrates the flow of messages between instances
`of objects when the graphical control system shownin FIG.
`1 enables a user to control a device;
`FIG.12 illustrates the flow of messages between instances
`of objects when the graphical control system shownin FIG.
`1 enables a user to create a cloned visual device control;
`FIG. 13illustrates the flow of messages between instances
`of objects when the graphical control system shown in FIG.
`1 automatically regenerates operable dimensionsof a visual
`device control;
`FIGS. 14, 15 and 16 illustrate the flow of messages
`between instances of objects when the graphical control
`system shown in FIG. 1 enables a user to control a plurality
`of devices with a single visual device control;
`FIG. 17A is a hierarchical diagram of an object-oriented
`programming paradigm implemented in a device as part of
`the graphical control system shownin FIG. 1 provided by a
`device;
`FIG. 17B illustrates the methods and data provided by the
`object-oriented programming paradigm depicted in FIG.
`17A; and
`FIG. 18illustrates the flow of messages between instances
`of objects when the graphical control system shownin FIG.
`1 effectuates a device control.
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENT
`
`As will be better understood from the following
`description, the present invention is directed to a graphical
`control system for controlling non-computer system devices,
`wherein virtually any electronically controllable device
`equipped with the necessary interface hardware can be
`controlled. While the following description explains the
`invention in connection with musical system components,
`such as amplifiers and equalizers, it is to be understood that
`the invention can be used with other non-computer system
`devices, including a simple on/off switch for controlling a
`lamp. In addition, the invention easily accommodates non-
`computer system devices produced by various manufactur-
`ers. The invention employs a personal computer, a network,
`interfaces for connecting the computer and the non-
`computer system devices to the network, and a visual
`network operating system. While the preferred network is a
`bus network, such as a fiber optic bus, other types of
`networks, including non-wired networks can be employed in
`actual embodiments of the invention. Further, the network
`can simply be a point-to-point connection for connecting a
`computer to the interface of a single non-computer system
`device. The visual network operating system, which controls
`the operation of the computer and the interface is based on
`an object-oriented programming paradigm. In orderto better
`understand the preferred embodiment of the invention
`described below, certain aspects of object-oriented program-
`
`5
`
`10
`
`15
`
`20
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`65
`
`6
`ming paradigms that are relevant to the following discussion
`are first described.
`
`The fundamental aspects of object-oriented programming
`paradigms are that objects can be organized into classes in
`a hierarchical fashion and that objects are interoperable.
`Classes are abstract generic descriptions of objects and their
`behaviors. A class defines a certain category or grouping of
`methods and data within an object. Methods comprise,
`procedures or code that operate upon data. Methods as
`applied to data define the behavior of the object. Refinement
`of the methods of the class is achieved by the creation of
`“sub-classes.” In other words, a class can be thoughtof as a
`genus, and its subclass as the species. Subclasses allow the
`introduction of a new class into the class hierarchy, and
`inherit the behaviors of its superclass while adding new
`behaviors to the subclass.
`
`An instance of a class is a specific individual entity,
`something concrete having observable behavior. An instance
`is a specific object with the behaviors defined by its class.
`Instances are created and deleted dynamically in an object-
`oriented programming paradigm. The class, however, is the
`broad, yet abstract, concept under which the instance
`belongs. The instance inherits all the methods ofits class,
`but has particular individual values associated with it that
`are unique. There is only one location in the memory of a
`computer for the class. There may, however, be numerous
`instances of the class, each of which has different values and
`different physical locations in memory.
`FIG. 1 illustrates a bus network 28 interconnecting a
`personal computer 20 and a plurality of non-computer
`system devices 21. The graphical control system of the
`present invention employing the object-oriented program-
`ming paradigm as described above allows a user to use the
`personal computer 20 to control the operation of the non-
`computer system devices 21 via bus network 28. For ease of
`illustration only three non-computer system devices 21 are
`shown in FIG. 1. Theillustrated devices are a two-channel
`amplifier 22, a tuner 24 and a graphic equalizer 26. As will
`be appreciated by those of ordinary skill in the art from the
`following description, additional non-computer system
`devices can be connected to and controlled by personal
`computer 20 via the bus network 28, if equipped with the
`necessary interface hardware. Non-computer system devices
`may include video cameras, speakers,
`television sets,
`telephones, lamps, etc. Even simple light switches can form
`non-computer system devices. For illustrative purposes
`only, the devices shown in FIG. 1 are depicted as audio
`sound system devices, i.e., amplifiers, equalizers, tuners, etc.
`In addition, those of ordinary skill in the art will recognize
`that additional personal computers equipped with the nec-
`essary hardware interfaces may be connected to bus network
`28 and simultaneously used to the control non-computer
`system devices 21. In fact, any computer system including
`but not limited to portable computers, powerbooks, personal
`device assistants that is equipped with the necessary hard-
`ware interface to be connected to bus network 28 or any
`other coupling medium may be used control
`the non-
`computer system devices 21.
`
`Inter-Device Communication
`
`As shown in FIG. 2, the personal computer 20 is con-
`nected to the network 28 by way of an RS232 interface 42
`and a bridge 45. Standard personal computers typically
`include an RS232 card 43 and an RS232 cable 44 for
`communicating with external devices such as printers.
`RS232 is an interface standard that specifies signal voltage
`
`

`

`5,657,221
`
`7
`levels, etc. RS232 is also a byte-level communication pro-
`tocol that specifies start bits, stop bits, etc. for sending bytes
`of data. Generally, a higher level communication protocolis
`used on top of the RS232 byte-level communication
`protocol, i.e., the personal computer 20 includes software
`defining a point-to-point communication protocol that is
`used on top of the R$232 byte-level protocol. The bridge 45
`provides the interface between the RS232 cables 38 and the
`bus network 28, on which a network communication pro-
`tocol is used.
`
`Also depicted in FIG. 2 are the same non-computer
`system devices 21 shown in FIG. 1. Each non-computer
`system device 21 includes an interface 35 comprising an I/O
`board 47 and a processor board 46 which allow the devices
`to be directly connected to the network 28. The 1/O board 47
`is specifically constructed for use with the bus network
`configuration and a particular type of transmission medium,
`whereas the processor board 46 can be used with different
`communication configurations and transmission media.
`The processor board 46 of interface 35 controls the
`communication of the devices 21 over the bus network 28.
`The processor board 46 includes a processor 41, an elec-
`tronically erasable and programmable read-only memory
`(EEPROM)58, and a random access memory (RAM)59.
`Oneof ordinary skill in the art will recognize that interface
`35 includes many more components than those shown in
`FIG. 2. Such other components are not described because
`they are conventional, and an understanding of them is not
`believed to be necessary to an understanding of the present
`invention.
`
`When a non-computer system device 21 sends data, a
`command, or program code (collectively referred to herein
`as data), the processor board 46ofinterface 35 housedin the
`device constructs a packet of data representing the relevant
`information. The processor board 46 determines the time at
`which the bus network 28 is available for sending the packet.
`The processor board 46 sends the packetto the I/O board 47
`at the appropriate time, and upon receipt of the packet, the
`TV/O board transmits the packet via the bus network 28 to
`personal computer 20. When a non-computer system device
`21 receives data from a data source, such as the personal
`computer 20, the /O board 47 receives the packet over the
`bus network 28, and sends the packet to the processor board
`46. Upon receipt of the packet, the processor board 46 of
`interface 35 processes the packet and performs any appro-
`priate function, possibly including sending a response
`packet back to the personal computer 20.
`Typically, non-computer system devices come equipped
`with a variety of internal and external feature controls 29a
`and feature displays 29b that are used to control and monitor
`the operation of such devices, including but not limited to
`volume controls, channel selectors, power switches, input
`and output meters, etc. (in FIG. 1, internal controls and
`displays are represented by phantom lines, while external
`controls and displays are represented by solid lines). The
`processor boards 46 in the non-computer system devices 21
`are further connected to these controls and displays to either
`control the devices directly or to control the devices in
`conjunction with other processors or controllers within the
`devices based upon packets received over the bus network
`28.
`The bus network 28 shown in FIGS. 1 and 2 can be
`formed of various coupling media such as glass or plastic
`fiber optic cables, coaxial cables, twisted wire pair cables,
`ribbon cables, etc. In the professional audio industry it is
`generally preferable to use fiber optic cables, as fiber optic
`
`10
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`cables are highly immune to electromagnetic interference
`and are capable of carrying signals over long distances
`without significant attenuation. As described herein, the bus
`network 28 represents fiber optic cables. Accordingly, the
`V/O board 47 of interface 35 shown in FIG. 2 is constructed
`for use with fiber optic cables.
`Various network communication protocols can be used to
`communicate over the bus network 28.
`In one actual
`embodimentof the present invention, the network commu-
`nication protocol used to communicate over the bus network
`28is of the type disclosed in commonly assigned U.S. Pat.
`No. 5,245,604, entitled “Communication System,” the dis-
`closure and drawings of which are specifically incorporated
`herein by reference. The network communication protocol
`described by U.S. Pat. No. 5,245,604 is referred to herein as
`the MediaLink protocol. The advantage of the MediaLink
`protocolis that it provides an upper limit on the amount of
`time it takes to communicate over the bus network 28. This
`is important in real-time environments such as a musical
`performance stage, where unpredictable delay would result
`in unacceptable distortion. As all network communication
`protocols must, the MediaLink protocol includes a network
`resource sharing and managementalgorithm such that only
`one device communicates over the bus network 28 at any
`one given time and such that each device has sufficient
`access to the bus network 28. While the MediaLink protocol
`and the bus network are presently preferred, it is