VWGoA - Ex. 1003
`Volkswagen Group of America, Inc., Petitioner
`
`1
`
`

`
`U.S. Patent
`
`Dec. 5,2000
`
`Sheet 1 of 12
`
`6,157,705
`
`2
`
`

`
`U.S. Patent
`
`Dec. 5,2000
`
`Sheet 2 of 12
`
`6,157,705
`
`
`
`60
`
`Resources
`
`
`
`1\
`
`I
`
`
`
`Web App E
`
`HTTP Server Q
`
`
`
`Network Dvr 2;
`
`03$
`
`
`
`
`
`IVR App Q
`
`os Q
`
`
`
`
`
`12
`
`Identifier/U RL
`
`Table Q
`
`Speech
`Recognizer
`
`16
`
`fly /Q9’
`
`3
`
`

`
`m,%\$5
`
`3U
`
`tH3%HnHEozmzHmmzomz_<_2MP_r..................................................-L_
`
`mMI_mmm.mmHNHHmoE%:z_5moEommamMH<z_s_mEm>z<%_oo__DHH
`
`p.QmNHMéogmzHW3HHmEmamm3HHmHoEmEz_Mméoamz
`
`4
`
`
`

`
`U.S. Patent
`
`Dec. 5,2000
`
`Sheet 4 of 12
`
`6,157,705
`
`
`
`5
`
`

`
`U.S. Patent
`
`Dec. 5,2000
`
`Sheet 5 0f 12
`
`6,157,705
`
`Voice File 3
`
`
`
`Resources
`
`
`
`OS 5_(_)
`
`IVR App Q
`
`11
`
`
`
`
`
`
`
`
`Identifier/URL
`
`Table Q
`
`Speech
`Recognizer
`
`1 6
`
`9;;/. 2%’
`
`6
`
`

`
`HwaP3U
`
`m
`
`000
`
`Sheet 6 of 12
`
`6,157,705
`
`
`
`tma<E.m_O._..n.__>_oo._m_>>
`
`
`
`
`
`.=,...,m_m__u=._.zmo_zoammamm._.2mm_m<m_._n_
`
`
`
`
`
`“*0
`
`7
`
`

`
`U.S. Patent
`
`Dec. 5,2000
`
`Sheet 7 of 12
`
`6,157,705
`
`ETRADE
`Customer log on
`'7
`
`®
`
`202
`
`smarter way to inv
`
`211
`
`-
`
`206
`
`est
`
`K
`E*TRADE .
`wr
`
`I” Canadian?
`'
`Click Here
`t%'ifi]keh‘3nr1eaI?r acceis
`investsin Caiafigf °
`218
`
`Welcome to the
`
`_
`,
`21213
`V
`" w ~~'
`F
`5 Oln
`_
`. Today
`.
`Need Hel
`212
`E*TRADE
`0 en an E*TRADE accogt. If ygu make
`a Van age
`your own ll'iVe% erri] E§C|Sl0l'lS: it's time to
` jfl
`m ?r§l?nngp:n“deggmg||ito maTr1§3EnEeSn\tA§I[stem
`9_9_”l'I15~°fl§
`We've set the standard for _unpara||e|e_d
`Margin Rates
`igggvfgifig, control, convenience, service I
`.
`geimx S
`_
`_
`A
`lnjustafew clicks,
`Customer ervice
`{ouénlelarn hon, easy
`
`FAQS
`I
`is o pace or ers,
`208
`“
`track the markets and
`mana eyour
`portfoio online.
`Do your homework
`
`'
`
`2143
`2
`“
`-ER
`”
`_
`m the news
`
`E *
`
`V
`
`-
`
`’
`
`b
`
`’
`
`‘
`
`accoun
`
`Online Agplication
`Re Uest b Mail
`QPLWE
`Visitor
`The E*TRADE Demo
`Free Quotes
`Pla the Stock Game
`The Company
`
`Press Releases
`
`Job Ogportunities
`
`k:,,_J
`204
`
`214b
`Find out whée we're making headlines.
`1997
`Current NewsR
`R‘ E
`M
` ., squire,
`ay
`.
`E*TRADE is profiled as one of the companies
`"bringing about nothing less than the
`democratization of Wall Street."
`E*TR¢«l%E'fCiE1<(§)>. Christos ggtgaklfs. is named
`oneo
`e 09
`visionary
`s y
`Communications Week.
`
`_
`
`E once you
`Egmlgfe
`w ywe've ecomea
`Online
`-
`
`get the treal sto .
`urcus orn_erss are
`their ex 9"e"°e5
`with E* RADE.
`
`/1200
`
`220
`
`222
`
`224
`
`226
`
`228
`
`210
`
`quotes |demo commissions |contact us | help
`
`©Co ri ht 1996 E ' TRADE Securities, Inc. All Rights Reserved.
`
`
`fly 3%’
`
`8
`
`

`
`U.S. Patent
`
`Dec. 5,2000
`
`Sheet 8 0f 12
`
`6,157,705
`
`E *TRADE
`
`®
`
`visitor
`center
`
`/ 230
`
`Customer log on
`
`E*TRADE's Free Quotes
`
`E *TRADE
`
`Quotes are delayed 20 minutes during market hours.
`
`Account Benefits
`investment Tools
`
`Quotes
`For a free quote, please enter the symbol:
`
`Security
`Customer Service
`
`FAQs
`
`232
`
`234
`
`236
`
`Symbols
` To find a symbol, please enter the name and type below:
`
`Onllne A lication
`Reguest by Mail
`
`Open an IRA
`
`Qn
`238
`234
`
`239
`
`Learn more about E'TRADE by trying our Trading Demo.
`
`210
`
`‘home
` uotes demo commissions contact us hel
`p -
`Free Quotes
`q
`|
`l
`|
`l
`Play the Stock Game
`©Co ri ht 1996 E ' TRADE Securities, Inc. All Rights Reserved.
`
`The Company
`Press Releases
`
`Job Opportunities
`
`204
`
`36
`
`9
`
`

`
`U.S. Patent
`
`Dec. 5,2000
`
`Sheet 9 0f 12
`
`6,157,705
`
`E * TRADE
`
`®
`
`.
`
`.
`
`“L$'%;%%-@-—
`
`240
`
`/
`
`Take advanta e of E*TRADE'S low commissions and simple fee structure.
`For more in ormation, see our full commission schedule. You may also
`want our tra Ing emo.
`
`242
`
`
`
`
`
`
`
`
`7115/16
`713/16
`
`
`
`
`All quotes are delayed 20 minutes during market hours.
`
`52-Week High: 07/25/97 72 3/4
`52-Week Low: 09/03/96 37 7/8
`
`Return to the Free Quotes gage.
`
`
`quotes |demo [commissions |contact us|he|p
`
`210
`
`©Co ri ht1996E'TRADE Securities, Inc. All Rights Reserved.
`
`9?;/. 3@
`
`10
`
`
`
`BAN KAMERICA CORP
`
`244
`
`246
`
`248
`
`250
`
`10
`
`

`
`U.S. Patent
`
`Dec. 5,2000
`
`Sheet 10 of 12
`
`6,157,705
`
`E *
`
`®
`
`oraer
`
`Switch Accounts
`Account: 262
`V Transaction: V Number
`V Stock
`of shares:
`symbol:
`
`@ Buy
`0 39"
`0 Sell Short
`0 Buy t° Cover
`
`':
`264
`
`lrindsymbol
`
`256
`
`’/ 260
`
`For help, click on any V
`V Price:
`
`(9 Market
`0 Um": ':I
`0 stop: l:
`0 Stop Limit: :|
`
`VTerm:
`
`Optional: VA|lornone |'_"l (300)shares and
`
`OVBI"
`
`Trading password: l:| Review the order carefully
`
`|canceI
`
`To place mutual fund orders, use our new E*TRADE Mutual Fund Center.
`
`main
`
`*
`
`the markets
`
`trading
`
`your account
`
`mutual fund center
`messa es
`9
`
`help
`
`Investor Tools
`Markets at a Glance
`Quotes/News/Charts
`Stock Watch
`
`Enter Stocks Order
`Open Stocks Order
`Options Trading
`Transaction Histom
`
`Accounts Balances
`Accounts Records
`Portfolio Summary
`Switch Accounts
`
`
`
`(9 Co ri ht 1997 E*TRADE Securities Inc. All Ri hts Reserved.
`
`94% 36
`
`11
`
`

`
`U.S. Patent
`
`Dec. 5, 2000
`
`Sheet 11 of 12
`
`6,157,705
`
`400
`
`416
`
`Establish Data
`
`Communication
`
`Channel
`
`Establish V0iC€
`Communication
`
`Channel
`
`404
`
`Synchronize Data and
`Voice Communication
`Channels
`
`Transmit Document over
`Data Communication
`
`414
`
`Channel
`
`Transmit Response over
`Voice Communication
`
`Channel
`
`418
`
`406
`
`Receive Voice
`
`Command
`
`Associate Voice
`
`Command with Natural
`
`408
`
`Language Phrase
`
`Associate Natural
`
`Language Phrase
`with Network Address
`
`Address
`
`Retrieve Document
`
`Identified By Network
`
`12
`
`12
`
`

`
`U.S. Patent
`
`Dec. 5, 2000
`
`Sheet 12 of 12
`
`6,157,705
`
`mum
`
`mm>mmm
`
`am
`
`._<oO._
`
`Qvaoamz
`
`Hmoz
`
`fl
`
`8.,
`
`éozmz
`
`v_z_._
`
`
`
`zo_._.<o_z3_>_s_Oomommmooma
`
`mam
`
`
`m_o_>mn_Eos_m__>_
`onmama
`
`
`
`
`
`moéoaz_<_2><.Em_o
`
`§
`
`mo<¢mmHz_mommao
`
`._oEz8
`
`ma
`
`13
`
`13
`
`
`
`
`
`

`
`6,157,705
`
`1
`VOICE CONTROL OF A SERVER
`
`FIELD OF THE INVENTION
`
`invention generally relates to computer
`The present
`systems, and relates in particular to mechanisms that provide
`voice control of a server.
`
`BACKGROUND OF THE INVENTION
`
`Avast amount of information is available using computer
`servers. Servers, mainframe computers, and other computer
`storage devices on networks provide a warehouse of infor-
`mation and services. However, accessing information and
`initiating processes or services on such servers is difficult
`using presently available mechanisms. In particular, mul-
`tiple commands using a keyboard or mouse are usually
`required for a user to navigate through the file directory
`structure of a server to locate desired information. The data
`
`processing field has failed to develop systems that provide
`voice control of a remote server from a local point, so that
`a local user can command the remote server by voice to
`display visual information at the local point or carry out a
`desired process.Past systems provide incomplete solutions
`to this problem. For example, interactive voice response
`(IVR) systems are used to deliver stored information over a
`telephone line to an end user. IVR systems are specialized
`computer systems that have a processor, a storage device
`such as a hard disk, hardware for interfacing the processor
`to the public switched telephone network (PSTN) and an
`IVR application program that runs on the processor.
`Generally, the end user connects to the IVR system using a
`telephone. The end user takes the telephone handset off hook
`and dials a pre-determined telephone number that identifies
`the IVR system. The telephone call is delivered over the
`PSTN to one of several trunk lines connected to the IVR
`
`system. The IVR system answers the call by seizing one of
`the trunk lines and playing a pre-recorded greeting to the
`caller. Typically the greeting is a voice recording stored
`digitally on a storage device that provides the end user with
`a menu of processing options that can be selected using
`telephone dial pad keys. Some IVR systems include voice
`recognition software or processors, so that an end user can
`select system options by speaking a short word or phrase
`such as a number.
`
`Example IVR applications include automated receptionist
`services, various customer service or “help desk”
`applications, and airline reservations systems.
`IVR systems can be configured to carry out a voice
`dialogue with the end user. The dialogue comprises a series
`of questions from the IVR system and answers from the end
`user until the desired service is provided to the end user by
`the IVR system. However,
`IVR systems are unable to
`display visual information, such as pre-formatted text or
`graphics, or dynamically created custom graphic
`information, in conjunction with the dialogue. The absence
`of visual information from present IVR systems is a major
`limitation, and represents the loss of a powerful medium for
`conveying information.
`Client-server computer systems also provide an incom-
`plete solution. In a client-server system, a client at a local
`point is connected by a data connection to a server at a
`remote point. The client can be a computer or a combination
`of a computer and software running on the computer. The
`data connection can be a cable, a local area network (LAN),
`a wide area network, or another type of network. The data
`connection can be the global network, operating according
`to standard protocols, known as the Internet. The server can
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`be a file server of the LAN, or a server not affiliated with the
`client. For example,
`the server can be a server that
`is
`publicly accessible using anonymous file transfer protocol
`(FTP) over the Internet. Using the Internet and certain wide
`area network technologies, a client can connect to, “log on”
`to, request and use a distant server.
`One popular technology enjoying wide use with the
`Internet is known as the World Wide Web. The World Wide
`
`Web enables a computer to locate a remote server using a
`server name in an agreed-upon format that is indexed at a
`central Domain Name Server (DNS). The local computer or
`client runs a browser program. Using the browser, the client
`locates the remote server using the DNS, and connects to the
`remote server. The client requests information from the
`server using a communication protocol called the Hypertext
`Transfer Protocol
`(HTTP), by providing a Uniform
`Resource Locator (URL) that uniquely identifies a page of
`information stored on the server. AURL is a form of network
`address that identifies the location of information stored in
`
`a network and represents a reference to a remote World Wide
`Web server, known as a website. The pages of information
`are files prepared in the Hypertext Markup Language
`(HTML). Thus, a Web client-server system can be used to
`request and display information stored on a remote server.
`URLs generally are formatted according to the following
`syntax:
`<protocol id>://{<server>} <second level domain><top
`level domain>{<directory>} {<file>}
`The <protocol id> identifies the transmission protocol to be
`used. For example, in the case of the Web it is <http>, and
`in the case of an anonymous file transfer protocol transaction
`it is <ftp>. The <server> element is an optional server name
`such as <www.>. The <server> element may also identify a
`directory on a storage device of the Web server that contains
`HTML documents. The <second level domain> element is
`the name of the server domain as found in the DNS table,
`such as <etrade>. The <top level domain> element identifies
`the type of the second level domain, and must be an item
`selected from a finite set of globally recognized top level
`domains, such as “com,” “org,” “mil,” “edu,” “gov,” and
`others. The <directory> element is an optional name of a
`directory within the <server>, such as <DocumentRoot/>.
`The <file> element is an optional name of a file, document
`or image to be retrieved, such as <Index.html>. Thus, a URL
`serves as one type of a network address to locate a document
`anywhere in a network.
`However, client-server systems and World Wide Web
`systems cannot respond to voice commands and cannot
`deliver visual or graphic information that is coordinated with
`a voice dialogue. These systems also do not enable a user to
`select or initiate computation processes in coordination with
`a voice dialogue. World Wide Web systems can include
`panels or pages that are dynamically generated by the
`systems, and can include internal or external computation
`processes rather than static documents or pages. However,
`there is no way to select such processes or locate them using
`voice interaction.
`
`In addition, when a user wishes to obtain specific infor-
`mation or activate a specific process using a Web server, but
`the URL of the information or process is unknown, the user
`must follow the hypertext links of numerous irrelevant pages
`until the desired information is located. Navigation through
`this control structure is clumsy. It is especially inconvenient
`at the slow data transfer speeds that presently characterize
`most use of the Web.
`
`One approach to these issues is to integrate speech
`recognition software in a computer program or computer
`
`14
`
`14
`
`

`
`6,157,705
`
`3
`remote from a server. The computer is connected through an
`interface to a microphone. The computer runs a speech
`recognition program that converts voice commands received
`by the microphone into keystrokes or commands under-
`standable by the program. This is known as “local control”
`or client-side speech recognition because recognition of
`voice commands is carried out in a local computer. The local
`computer is separated from a server that stores Web pages
`and runs application programs that serve the client. The
`speech recognition controls only the program of the local
`computer, not
`the server or applications running in the
`server. However, such an approach has several disadvan-
`tages.
`First, high-quality speech recognition is expensive both in
`the commercial sense and in terms of computing resources
`and power needed to provide acceptable results. Currently,
`high-quality speech recognition software is in very limited
`use and is not generally available at a reasonable price to the
`vast majority of home or business computer users.
`Also, local control allows an end user to access only the
`information structure presented by the program currently
`running in the local computer. The voice commands are
`limited to the command set of the current program. Local
`control cannot provide flexible shortcuts through a Web site,
`and cannot enable the Web site to identify, during a voice
`dialogue with the end user, suitable Web pages to present to
`the end user.
`
`Thus, there is a need for a system that enables a local
`client to rapidly retrieve information from a remote server
`using voice commands.
`There is also a need for a system that enables a local client
`to carry out a voice dialogue with a remote server and
`receive or retrieve visual and graphic information that is
`coordinated with the voice dialogue.
`There is also a need for an arrangement with which voice
`commands or a voice dialogue can be used to locate, select,
`activate or initiate a computing process or service that is
`available at the server; to locate information in a database,
`and to execute trades in a securities trading system.
`SUMMARY OF THE INVENTION
`
`These and other needs are fulfilled by the present
`invention, which comprises, in one embodiment, a method
`of controlling a remote server by a voice command issued
`from a location local to a client, comprising the steps of
`establishing a voice communication channel between the
`location local to said client and the remote server; estab-
`lishing a data communication channel associated with the
`voice communication channel between the client and the
`
`remote server; receiving the voice command by the voice
`communication channel; associating the voice command
`with a resource identifier; selecting a server resource based
`on the resource identifier; and delivering the resource from
`the remote server to the client by the data communication
`channel.
`
`One feature of the invention is establishing the voice
`communication channel integrated with the data communi-
`cation channel. Another feature is delivering a voice mes-
`sage over the voice communication channel in coordination
`with delivery of the server resource. Yet another feature is
`processing the voice command using a speech recognition
`process to recognize the natural language phrase.
`One aspect of this embodiment is associating the natural
`language phrase with a network address by performing the
`steps of: identifying the natural language phrase in a table of
`the remote server that maps natural language phrases to
`network addresses; and looking up the network address in
`
`4
`the table. A feature of this aspect is loading said document
`identified by the network address from a storage device
`coupled to the remote server.
`Another aspect of the invention is establishing a data
`communication channel between the client and the remote
`
`server configured to communicate data between the client
`and the remote server; and establishing a voice communi-
`cation channel between the client and the remote server
`
`10
`
`configured to communicate sound information including the
`voice command between the client and the remote server.
`
`One feature of this aspect is establishing a telephone con-
`nection from the client to a voice recognizer that is coupled
`to the remote server. Another feature is establishing a
`telephone connection from the client to an interactive voice
`response (IVR) system coupled to the remote server. Still
`another feature is establishing an Internet telephony con-
`nection from the client to a voice recognizer that is coupled
`to the remote server.
`
`Still another aspect of the invention is recognizing the
`natural language phrase in the voice command at the IVR
`system;
`transmitting the natural
`language phrase to the
`remote server; and transmitting a voice response from the
`IVR system to the client.
`The invention also encompasses a computer system and a
`computer program product configured in accordance with
`the foregoing aspects and f,eatures.
`Additional objects, advantages and novel features of the
`invention will be set forth in part in the description which
`follows, and in part will become apparent to those skilled in
`the art upon examination of the following or may be learned
`by practice of the invention. The objects and advantages of
`the invention may be realized and attained by means of the
`instrumentalities and combinations particularly pointed out
`in the appended claims.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The present invention is illustrated by way of example,
`and not by way of limitation, in the figures of the accom-
`panying drawings and in which like reference numerals refer
`to similar elements and in which:
`
`FIG. 1A is a block diagram showing an embodiment of
`the invention.
`
`FIG. 1B is a block diagram of a software architecture of
`the embodiment of FIG. 1A.
`
`FIG. 1C is a block diagram of hardware elements of an
`interactive voice response (IVR) system.
`FIG. 2A is a block diagram showing an alternate embodi-
`ment of the invention.
`
`FIG. 2B is a block diagram of software architecture of the
`embodiment of FIG. 2A.
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`FIG. 3A is a diagram of an exemplary screen display
`involved in coordinating data and voice communication
`channels.
`
`55
`
`FIG. 3B is a diagram of an exemplary screen display of
`a Web securities brokerage application that can be used with
`the invention.
`
`60
`
`65
`
`FIG. 3C is a diagram of an exemplary screen display that
`provides a securities quote entry facility.
`FIG. 3D is a diagram of an exemplary screen display
`showing a securities quote.
`FIG. 3E is a diagram of an order page in the brokerage
`application of FIGS. 3B—3D.
`FIG. 4 is a flow diagram of an embodiment of a process
`for voice control of a server.
`
`15
`
`15
`
`

`
`6,157,705
`
`5
`FIG. 5 is a block diagram of a computer system that can
`be used to operate the present invention.
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENT
`
`A method and apparatus for voice control of a server is
`described. In the following description, for the purposes of
`explanation, numerous specific details are set forth in order
`to provide a thorough understanding of the present inven-
`tion. It will be apparent, however, to one skilled in the art
`that the present invention may be practiced without these
`specific details. In other instances, well-known structures
`and devices are shown in block diagram form in order to
`avoid unnecessarily obscuring the present invention.
`VOICE CONTROL OF A SERVER
`
`FIG. 1A is a block diagram of a system that provides voice
`control of a server. An end user 2 has a computer 4 and a
`telephone 10 located at a local point 1. The computer 4 is
`coupled through a network 6 to a server 8 at a remote point
`3. In one embodiment, the network 6 is a local area network,
`and the server 8 is separated from the end user 2. In the
`preferred embodiment,
`the network 6 is the Internet,
`the
`server 8 is configured as a World Wide Web server, and the
`remote point 3 is geographically remote from the local point
`1. The connection from the computer 4 to the server 8 is a
`data communication channel.
`
`The telephone 10 is coupled through the public switched
`telephone network (PSTN) 12 to an interactive voice
`response (IVR) system 14. The connection between the
`telephone 10 and the IVR system 14 is a voice communi-
`cation channel. The IVR system 14 is coupled to a speech
`recognizer 16 and to the server 8.
`In the preferred
`embodiment, the IVR system 14, the speech recognizer 16,
`and the server 8 are in the same location, although this is not
`required. The speech recognizer 16, IVR system 14, and the
`server 8 can be connected over a network.
`
`FIG. 1B is a block diagram of the software architecture of
`the system of FIG. 1A. The client computer 4 runs an
`operating system (OS) 20. Exemplary operating systems
`include Windows, MacOS, and Solaris. In known manner,
`the OS 20 provides an interface to the hardware elements of
`the computer 4 that can be used by application programs;
`controls and supervises the use of system resources; and
`loads, supervises and controls the operation of application
`programs. A network driver program 22 runs on top of the
`OS 20 and under its control. The network driver program 22
`functions as an interface between the OS 20 and the network
`
`6. An example of a network driver program 22 is Novell
`NetWare. An example of a client computer 4 is a Compaq
`personal computer, a workstation such as a SPARCstation
`made by Sun Microsystems, Inc., or a network computer
`made by Network Computer, Inc.
`The client computer 4 also runs a browser 24. The
`browser 24 is an application program that can read HTML
`documents, interpret HTML code stored in the documents,
`and display visual information based on the interpretation of
`the HTML code on a display of the client computer 4. The
`user 2 can use computer 4 to command browser 24 to control
`its display. Examples of suitable programs that can be used
`for the browser 24 are Netscape Navigator® and Microsoft®
`Explorer.
`The server 8 runs an OS 30 that generally performs the
`same functions described above in connection with OS 20.
`
`The OS 30 may be the same as OS 20 or may be a different
`operating system. In the preferred embodiment, the OS 30 is
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`an operating system that is optimized to perform server
`functions, such as Microsoft Windows NT.
`Under control of OS 30, the server 8 runs an HTTP server
`32. The HTTP server 32 is an application program that
`enables the server 8 to send and receive information using
`the HTTP protocol. Examples of available programs that can
`be used as HTTP server 32 include the Apache HTTP server
`and Microsoft Internet Information Server. The server 8 may
`be a computer, or the combination of a computer and a
`suitable operating environment.
`The server 8 also runs a Web application 34. Resources 60
`are available in association with the server 8. “Resources”
`that are “available” on server 8 in this context means
`documents stored on server 8 or on a network device or
`
`system that is accessible through server 8, or computation
`functions or processes provided by or through server 8.
`“Resources” also is intended to encompass HTML pages or
`other electronic information generated dynamically or as
`output from a program. “Resources” is also intended to
`include databases accessible through a server; documents,
`programs, and data accessible using one or more hyperlinks
`from a first document; and client-server application
`programs, such as a securities trading system. Such
`resources may be physically stored on the server 8, or may
`be physically stored elsewhere and accessed by or using the
`server 8. For example, the Resources 60 are a set of HTML
`files stored on a hard disk of the server 8. Such documents
`
`are merely an example of the type of files that can be stored
`in association with the server 8 and delivered to the com-
`
`puter 4 using the invention; other stored information also
`can be delivered. Alternatively, resources 60 can be pro-
`grams such as Java applets, ActiveX controls, or other
`processes, facilities or functions provided by server 8.
`FIG. 1C is a block diagram of the IVR system 14. Abus
`102 acts as an internal data connection for all the elements
`
`of IVR system 14 and provides a central pathway for data
`and instructions. Processor 104 executes instructions of an
`
`operating system and application programs and controls
`overall operation of the system.
`Instructions, data, and
`stored programs are stored in one or more storage devices
`110, for example, in a hard disk drive or floppy disk drive.
`A non-volatile memory 108 such as a ROM is used to store
`bootstrap instructions and other constant
`information.
`Instructions and data for execution and use by processor 104
`are loaded from storage device 110 into a volatile local
`high-speed memory 106, such as RAM.
`A terminal interface 114 coupled to the bus 102 provides
`communications between processor 104 and an external
`display terminal, which may be another computer. A net-
`work interface 116 couples bus 102 to an external data
`network such as LAN. The processor 104 can load instruc-
`tions or data from devices coupled to the external data
`network.
`
`One or more port interfaces 112 connect bus 102 and
`processor 104 to telephone trunks 120 that are connected to
`the public telephone network 122. The port interfaces 112
`can seize a call on a trunk, answer telephone calls arriving
`on the trunk 120, place outbound calls, and manage multiple
`calls on a single line or trunk. The port interfaces 112 can
`also send and receive call placement and setup control
`information in the voice band or out-of-band. Each port
`interface 112 is uniquely identified within IVR system 14 by
`a port number that is used to select and address the port
`interface. “Port” refers to a logical connection between IVR
`systems 14 and one or more telephone lines of a trunk 120.
`A telephony port supervisor 118 provides electronics for
`controlling the port interfaces 112. The port supervisor 118
`
`16
`
`16
`
`

`
`6,157,705
`
`7
`can select an available port interface and route commands or
`data from processor 104 to an available port interface 112.
`The public telephone network 122 connects calls to or
`from trunks 120 through one or more telephone company
`offices 130 and to or from remote telephones or other
`equipment.
`The network interface 116 or terminal interface 114 also
`
`can connect an external computer, such as server 8, to IVR
`system 14.
`FIG. 1B also shows the software architecture of the IVR
`
`system 14. The IVR system 14 is a computer that runs an OS
`40. The OS 40 can be the same operating system as OS 20,
`30, or a different operating system. Because the IVR system
`14 is expected to respond to telephone calls arriving from the
`PSTN 12 on a live, real-time basis, it is advantageous for OS
`40 to be optimized to handle real-time tasks. An example of
`such a real-time optimized OS is the QNX operating system,
`which can be used as OS 40. IVR systems are commercially
`available from InterVoice, Octel Corporation, and others.
`The IVR system 14 also runs an IVR application program
`42 under control of OS 40. In cooperation with the OS 40,
`the IVR application program 42 is responsible for detecting
`incoming calls, answering incoming calls, presenting pre-
`recorded greetings and messages to the caller, receiving and
`interpreting caller inputs, and responding to caller inputs.
`The IVR application program 42 defines features and func-
`tions available to the end user 2 for controlling the IVR
`system 14. The IVR application 42 also provides an inter-
`face to an IVR system operator, such as a command line
`interface, graphical user interface, or terminal interface. An
`operator can connect a computer or “dumb terminal” to
`program the IVR system 14 or monitor its operation.
`The IVR application program 42 can also control com-
`munication between the IVR system 14 and the server 8. In
`one embodiment, the IVR system 14 and the server 8 each
`have serial data ports that are connected by an RS-232 serial
`data connection. The IVR application program 42 can read
`from or write to the serial data port of IVR system 14.
`Accordingly, the IVR application program 42 can command
`the server 8 to generate, load, and/or send Resources 60
`through the network 6 to browser 24.
`The IVR system 14 is coupled to a speech recognizer 16
`and an Identifier/URL table 50. In an alternate embodiment,
`the speech recognizer 16 is integrated into the IVR system
`14.
`
`Afunction of the IVR application program 42 is to receive
`a voice phrase from the voice communication channel and
`forward the voice phrase to the speech recognizer 16 for
`recognition. The speech recognizer 16 receives a voice
`phrase or utterance in the form of a digitized waveform. The
`speech recognizer 16 attempts to match the digitized wave-
`form with one of a set of stored waveform images that
`represent “known” utterances or words, according to math-
`ematical rules and transformations programmed into the
`speech recognizer. In this way, the speech recognizer essen-
`tially scans the voice phrase to identify words in it, and
`provides a digital text string or code as output. Each digital
`text string or code is uniquely associated with a pre-defined
`natural language phrase. For example, although every indi-
`vidual speaker may say the natural language phrase “yes” in
`a different way, once the speech recognizer 16 recognizes an
`input utterance as the phrase “yes,” the speech recognizer
`provides a single code or text string as output. Hardware and
`software usable for speech recognition is well known in the
`art and is generally described in G. Pelton, “Voice Process-
`ing” (New York: McGraw-Hill, Inc., 1993). Manufacturers
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`of commercially available speech recognition equipment
`and software include Dragon Systems, IBM, Applied Lan-
`guage Technologies and Nuance.
`In the preferred embodiment, the speech recognizer 16
`carries out continuous, speaker-independent speech
`recognition, meaning that the speech recognizer will recog-
`nize words that are spoken continuously without pauses that
`separate the words, and will recognize any speaker without
`a training session. Also, in the preferred embodiment, the
`speech recognizer 16 associates an utterance or voice phrase
`spoken by the user 2 and received over the voice commu-
`nication channel with a resource identifier. In this context,
`“resource identifier” means a name, label or number that
`non-uniquely identifies a resource, or provides a mapping of
`an utterance to a resource or class of resources. For example,
`when the utterance is “stocks,” the resource identifier could
`be “stock-functions.” The same resource identifier could be
`used for an utterance of “bonds,” so that utterances of
`“stocks” or “bonds” lead to the same resource or group of
`resources.
`
`Another function of the IVR application program 42 is to
`associate a natural language phrase or resource identifier
`detected by the speech recognizer 16 with a URL of a Web
`document 60 stored in association with the server 8. The
`Identifier/URL table 50 stores associations of resource iden-
`tifiers and URLs of Resources 60. In one embodiment, the
`Identifier/URL table 50 is a table of a relational database
`system having columns for an index value, a phrase code, a
`natural language phrase or resource identifier, and a URL.
`For example, the Identifier/URL table 50 has a record that
`has the values 101, 100, “home page”, and a URL value that
`identifies a home page of the resource. This record indicates
`that the natural language phrase “home page” is associated
`with a Web document 60 having the specified address.
`Alternatively, the resource identifier “home-page” is stored
`and associated with the same resource.
`
`In operation, the user 2 connects the client computer 4 to
`the server 8 over the network 6 using the browser 24. For
`example, the user 2 enters a URL that identifies a known
`home page of the serv