Paper No. 1
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`
`
`
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`
`
`
`
`
`MICROSOFT CORPORATION,
`Petitioner,
`v.
`IPA TECHNOLOGIES, INC.,
`Patent Owner.
`
`Patent No. 6,757,718 B1
`Issued: June 29, 2004
`Filed: June 30, 2000
`Inventors: Christine Halverson, Luc Julia, Dimitris Voutsas, Adam Cheyer
`Title: SYSTEM, METHOD, AND ARTICLE OF MANUFACTURE FOR
`AGENT-BASED NAVIGATION IN A SPEECH-BASED DATA
`NAVIGATION SYSTEM
`
`
`
`
`
`
`Inter Partes Review No. IPR2018-01440
`
`
`
`
`
`
`
`
`PETITION FOR INTER PARTES REVIEW OF U.S. PATENT NO. 6,757,718
`
`
`
`
`
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`Table of Contents
`
`I.
`II.
`
`INTRODUCTION ........................................................................................... 1
`COMPLIANCE WITH REQUIREMENTS FOR A PETITION FOR INTER
`PARTES REVIEW .......................................................................................... 1
`A. Grounds for Standing (§42.104(a)) ....................................................... 1
`B.
`Fee for Inter Partes Review (§42.15(a)) ................................................ 2
`C. Mandatory Notices (37 CFR §42.8(b)) ................................................. 2
`1.
`Real Party in Interest (§42.8(b)(1)) ............................................. 2
`2.
`Related Proceedings .................................................................... 2
`3.
`Lead and Backup Counsel (§42.8(b)(3)) .................................... 3
`4.
`Service Information (§42.8(b)(4)) .............................................. 3
`Proof of Service Notice (§§42.6(e) and 42.105(a)) .............................. 3
`D.
`III. PRECISE RELIEF REQUESTED (CLAIMS CHALLENGED)
`(§42.104(B)) .................................................................................................... 4
`IV. THE CONTESTED PATENT ......................................................................... 5
`A.
`Effective Filing Date ............................................................................. 5
`B.
`Level of Ordinary Skill in the Art ......................................................... 5
`C. Overview of the 718 Patent ................................................................... 5
`D.
`Construction of Claim Terms ................................................................ 9
`1.
`“navigation query” .................................................................... 10
`2.
`“electronic data source” ............................................................ 10
`3.
`“rendering an interpretation of the spoken request” ................. 11
`4.
`“constructing [at least part of] a navigation query based upon
`the interpretation” ..................................................................... 11
`V. OVERVIEW OF THE PRINCIPLE REFERENCE ...................................... 11
`A. Moran et al., Multimodal User Interfaces in the Open Agent
`Architecture (1997) (“Moran”) (Ex.1003) .......................................... 11
`1.
`Public Availability .................................................................... 11
`2.
`Overview of Moran ................................................................... 13
`VI. DETAILED REASONS FOR RELIEF REQUESTED ................................ 16
`i
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`b.
`
`e.
`
`f.
`
`2.
`3.
`
`c.
`d.
`
`A. Moran Anticipates Claims 1-4, 6, 8-13, 15, 17-22, 24, 26-27 ............ 16
`1.
`Independent Claim 1 ................................................................. 16
`a.
`Preamble ......................................................................... 16
`i.
`Speech-based Navigation of a Remote Electronic
`Data Source .......................................................... 16
`ii. Mobile Information Appliance ............................. 17
`“receiving a spoken request for desired information from
`the user utilizing the mobile information appliance of the
`user, wherein said mobile information appliance
`comprises a portable remote control device or a set-top
`box for a television” ....................................................... 18
`“rendering an interpretation of the spoken request” ....... 20
`“constructing a navigation query based upon the
`interpretation” ................................................................. 21
`“utilizing the navigation query to select a portion of the
`electronic data source” .................................................... 22
`“transmitting the selected portion of the electronic data
`source from the network server to the mobile information
`appliance of the user.” .................................................... 23
`Independent Claim 10 ............................................................... 24
`Independent Claim 19 ............................................................... 25
`a.
`“A system for speech-based navigation of an electronic
`data source located at one or more network servers
`located remotely from a user” ........................................ 25
`“a mobile information appliance operable to receive a
`spoken request for desired information from the user,
`wherein said mobile information appliance comprises a
`portable remote control device or a set-top box for a
`television” ....................................................................... 25
`“spoken language processing logic, operable to render an
`interpretation of the spoken request” ............................. 26
`“electronic communications infrastructure for
`transmitting the selected portion of the electronic data
`
`b.
`
`c.
`
`d.
`
`ii
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`4.
`5.
`
`source from the network server to the mobile information
`appliance of the user” ..................................................... 26
`Claims 2, 3, 12, and 21 ............................................................. 27
`Claims 4, 13, and 22 ................................................................. 29
`a.
`Soliciting Additional Input ............................................. 29
`b.
`Refining the Navigation Query....................................... 30
`c.
`Selecting a Portion of the Data Source ........................... 30
`Claims 6, 15, and 24 ................................................................. 31
`6.
`Claims 8, 9, 17, 18, 26 and 27 .................................................. 32
`7.
`Claims 11 and 20....................................................................... 33
`8.
`B. Moran Renders Obvious Claims 1-27 ................................................. 35
`1.
`Independent Claims 1, 10, and 19 ............................................. 35
`a.
`“constructing a navigation query based upon the
`interpretation” ................................................................. 35
`Claims 2, 3, 12, and 21 ............................................................. 36
`Claims 4, 13, and 22 ................................................................. 38
`a.
`“solicit[ing] additional input from the user, including
`user interaction in a modality different than the original
`request” ........................................................................... 38
`“refining [logic operable to refine] the navigation query
`based upon the additional input” .................................... 40
`Claims 5, 14, and 23 ................................................................. 41
`4.
`Claims 6, 15, and 24 ................................................................. 44
`5.
`Claims 7, 16, and 25 ................................................................. 45
`6.
`Claims 8, 9, 11, 17, 18, 20, 26 and 27 ...................................... 46
`7.
`C. Moran in View of Burns with or without Thrift Renders Obvious
`Claims 1-27 ......................................................................................... 46
`1.
`Claims 1, 10, and 19 ................................................................. 46
`a.
`Constructing a Navigation Query ................................... 47
`b.
`Refining the Navigation Query and Using the Refined
`Navigation Query to Select ............................................. 50
`
`2.
`3.
`
`b.
`
`iii
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`c. Mobile Information Appliance ....................................... 52
`Claims 2, 3, 12, and 21 ............................................................. 58
`2.
`Claims 4-9, 11, 13-18, 20, and 22-27 ....................................... 62
`3.
`D. Moran in View of Ditzik, with or without Burns or Thrift, Renders
`Obvious Claims 5, 7, 14, 16, 23, and 25 ............................................. 62
`1.
`Claims 5, 7, 14, 16, 23, 25 ........................................................ 62
`VII. CONCLUSION .............................................................................................. 68
`
`
`iv
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`I.
`
`INTRODUCTION
`Petitioner Microsoft Corporation respectfully requests the Board institute
`
`inter partes review of claims 1-27 of U.S. Patent No. 6,757,718 (“718 Patent”)
`
`(Ex.1001) based principally on Multimodal User Interfaces in the Open Agent
`
`Architecture (1997) by Moran et al., whose authors include two of the listed
`
`inventors of the 718 Patent. Moran describes the claimed subject matter of the 718
`
`Patent in a printed publication made available to the public more than three years
`
`before the application underlying the 718 Patent was filed—the 35 U.S.C. §102(b)
`
`bar applies precisely in these circumstances.
`
`II. COMPLIANCE WITH REQUIREMENTS FOR A PETITION FOR
`INTER PARTES REVIEW
`A. Grounds for Standing (§42.104(a))
`Petitioner certifies that it is not barred or estopped from requesting inter
`
`partes review of the 718 Patent and that this petition is filed within one year of the
`
`service date of a complaint alleging infringement. Neither Petitioner, nor any party
`
`in privity with it, has filed a civil action challenging the validity of any of the 718
`
`Patent’s claims. The 718 Patent has not been the subject of a prior inter partes
`
`review by Petitioner or its privies. Petitioner therefore certifies this patent is
`
`available for inter partes review according to 37 C.F.R. §42.104(a).
`
`1
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`Fee for Inter Partes Review (§42.15(a))
`B.
`The director is authorized to charge the fee specified by 37 C.F.R. §42.15(a)
`
`to Deposit Account No. 50-1597.
`
`C. Mandatory Notices (37 CFR §42.8(b))
`Real Party in Interest (§42.8(b)(1))
`1.
`The real party-in-interest of this petition pursuant to §42.8(b)(1) is
`
`Microsoft Corporation.
`
`Related Proceedings
`2.
`The 718 Patent is involved in the following proceedings:
`
`Google LLC v. IPA Technologies Inc. et al, IPR2018-00476 (PTAB Jan. 12,
`
`2018; dismissed July 31, 2018); DISH Network Corporation et al v. IPA
`
`Technologies Inc., IPR2018-00351 (PTAB Dec. 20, 2017; dismissed May 4, 2018);
`
`IPA Technologies Inc. v. Google LLC, 1:18-cv-00318 (D. Del. Feb. 26, 2018); IPA
`
`Technologies Inc. v. Microsoft Corporation, 1:18-cv-00001 (D. Del. Jan. 2, 2018);
`
`IPA Technologies Inc. v. NVIDIA Corporation, No. 1:17-cv-00287 (D. Del. Mar.
`
`20, 2017; dismissed Mar. 22, 2018); IPA Technologies Inc. v. Sony Electronics
`
`Inc., et al., No. 1:17-cv-00055 (D. Del. Jan. 19, 2017; dismissed Apr. 16, 2018);
`
`IPA Technologies Inc. v. Amazon.com, Inc. et al., No. 1:16-cv-01266 (D. Del. Dec.
`
`19, 2016); and IPA Technologies Inc. v. DISH Network Corporation et al., No.
`
`1:16-cv-01170 (D. Del. Dec. 9, 2016; dismissed Apr. 10, 2018).
`
`2
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`Additionally, the following other related proceedings may be affected by or
`
`have an affect upon the outcome of an inter partes review involving the 718
`
`Patent: Microsoft Corporation v. IPA Technologies Inc., IPR2018-00734 (PTAB
`
`Mar. 6, 2018); Microsoft Corporation v. IPA Technologies Inc., IPR2018-00791
`
`(PTAB Mar. 21, 2018); Microsoft Corporation v. IPA Technologies Inc., IPR2018-
`
`00792 (PTAB Mar. 21, 2018); Microsoft Corporation v. IPA Technologies Inc.,
`
`IPR2018-00793 (PTAB Mar. 21, 2018); Microsoft Corporation v. IPA
`
`Technologies Inc., IPR2018-00794 (PTAB Mar. 21, 2018).
`
`Lead and Backup Counsel (§42.8(b)(3))
`3.
`Lead Counsel is Joseph A. Micallef (Reg. No. 39,772),
`
`jmicallef@sidley.com, (202) 736-8492. Backup Lead Counsel is Scott M. Border
`
`(pro hac vice to be requested), sborder@sidley.com, (202) 736-8818.
`
`Service Information (§42.8(b)(4))
`4.
`Service on Petitioner may be made by email (sidley-ipatech-
`
`ipr@sidley.com), mail, or hand delivery to: Sidley Austin LLP, 1501 K Street,
`
`N.W., Washington, D.C. 20005. The fax number for lead and backup counsel is
`
`(202) 736-8711.
`
`Proof of Service Notice (§§42.6(e) and 42.105(a))
`D.
`Proof of service is provided in Attachment A.
`
`3
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`III. PRECISE RELIEF REQUESTED (CLAIMS CHALLENGED)
`(§42.104(B))
`Petitioner proposes the below grounds for trial, and addresses potential
`
`arguments by proposing additional grounds that may more closely satisfy certain
`
`claim limitations. Petitioner respectfully requests that Trial be instituted on the
`
`following grounds:
`
`(1) Claims 1-4, 6, 8-13, 15, 17-22, 24, 26-27 are
`unpatentable under 35 U.S.C. §102 by Moran et al., Multimodal User
`Interfaces in the Open Agent Architecture (1997) (“Moran”)
`(Ex.1003).
`
`(2) Claims 1-27 are unpatentable under 35 U.S.C. §103 over
`Moran.
`
`(3) Claims 1-27 are unpatentable under 35 U.S.C. §103 over
`Moran in view of U.S. Patent No. 5,454,106 to Burns (“Burns”)
`(Ex.1004), with or without U.S. Patent No. 6,188,985 to Thrift
`(“Thrift”) (Ex.1005).
`
`(4) Claims 5, 7, 14, 16, 23 and 25 are unpatentable under 35
`U.S.C. §103 over Moran in view of U.S. Patent No. 5,983,073 to
`Ditzik (“Ditzik”) (Ex.1006), with or without Burns or Thrift.
`
`
`Neither Moran, nor combinations involving Moran, Burns, Thrift or Ditzek,
`
`formed the basis of a rejection during prosecution of the 718 Patent. Juniper
`
`Networks v. Mobile Telecommc’ns Techs., LLC, IPR2017-00642, Paper 24, 8 (July
`
`27, 2017).
`
`4
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`IV. THE CONTESTED PATENT
`A. Effective Filing Date
`The 718 Patent was filed on June 30, 2000, as Application No. 09/608,872,
`
`and is a continuation of an application filed on March 13, 2000. Although
`
`Petitioner applies March 13, 2000 as the earliest effective filing date for the
`
`purposes of this proceeding, the date is not relevant to the patentability analysis
`
`here because the principle prior art at issue in this Petition was publicly available
`
`by January 1997 and is prior art under both 35 U.S.C. §102 (a) and (b) (pre-AIA).
`
`Level of Ordinary Skill in the Art
`B.
`A person of ordinary skill in the field of the 718 Patent (“a Skilled Artisan”)
`
`would have had a Bachelor’s degree in Computer Science or equivalent field and
`
`one to two years of work experience in user interfaces for computer systems,
`
`database information retrieval, or a related area. Such a person would have had a
`
`working knowledge of various techniques relevant to human-computer interactions
`
`and information retrieval systems, including over a network such as the Internet,
`
`and multimodal user input, such as error correction, speech recognition, and
`
`natural language interpretation. Ex.1002,¶72.
`
`C. Overview of the 718 Patent
`The 718 Patent generally describes a system and method for processing a
`
`query based on spoken natural language input using a platform of application
`
`5
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`agents. Ex.1001, Abstract. The query is used to search and retrieve information
`
`from electronic data sources. Id., 7:12-14.
`
`The 718 Patent explains that a benefit of “navigation of electronic data by
`
`means of spoken natural language” is that it “allows relatively naive users to
`
`navigate and access desired data by means of natural language input.” Id., 1:20-25,
`
`29-32. It notes “the reality that a stream of naive spoken natural language input
`
`will, over time, typically present a variety of errors and/or ambiguities” in
`
`interpretations of the spoken input and attempts to account for those to construct a
`
`“complete, valid navigational template.” Id., 2:5-16,54-56.
`
`
`
`6
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`Figure 1a shows a “voice input device 102” that captures voice input data,
`
`and transmits data to a “communications box 104.” Id., 3:19-28, 50-62. The voice
`
`data is retransmitted over a network to a remote server(s), where the voice data is
`
`processed and used to construct a navigation query. E.g., id., 3:11-16, 61-4:24.
`
`The server comprises request processing logic 300, including a speech recognition
`
`engine, natural language parser, query construction logic, and query refinement
`
`logic. Id., Fig.3. The server is connected to a remote data source that may
`
`comprise databases, websites, or other electronic information repositories, and may
`
`reside on the server. Id. Information retrieved from that data source is transmitted
`
`via the network to the communications box for display using a display device. Id.,
`
`4:25-37.
`
`The 718 Patent also describes the use of the prior art Open Agent
`
`Architecture “OAA®” software platform. Id., 13:15-14:67. The OAA is described
`
`as “provid[ing] a useful software platform for building systems that integrate
`
`spoken natural language as well as other user input modalities.” Id., 13:52-54.
`
`The specification explains that the OAA enables collaboration among distributed
`
`software agents (id., 13:18-19) a “facilitator” in “Interagent Communication
`
`Language” (id., 13:22-40). When a task is presented to the facilitator, it
`
`determines whether the registered capabilities “will help satisfy a current goal or
`
`sub-goal thereof” and delegates sub-goals accordingly. Id., 13:40-45. The
`
`7
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`facilitator “coordinates and integrates the results received from different client
`
`agents on various sub-goals, in order to satisfy the overall goal.” Id., 13:49-51.
`
`Figure 6 discloses the use of the OAA for the purposes of the 718 Patent,
`
`showing in a representative embodiment that the core of the system is the facilitator:
`
`
`
`The facilitator can receive voice data from a user, id., 14:27-32, which is sent to a
`
`speech recognition agent and natural language agent to interpret the user query and
`
`then return it in an interagent communication language. Id., 14:31-37. The results
`
`are routed through the facilitator in order to retrieve requested content, id., 14:34-
`
`46, and then routed back through the facilitator, to the user interface agents, to be
`
`displayed to the user. Id., 14:44-50.
`
`8
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`The 718 Patent states that problems arise when the user input cannot be used
`
`to successfully retrieve the requested information. Id., 10:50-11:6. The 718 Patent
`
`states that deficiencies can be detected during query construction as well as after
`
`navigation of the data source. Id., 10:50–11:6. In either scenario, the 718 Patent
`
`states that “a preferred technique” to handle such errors and deficiencies involves
`
`“soliciting additional input from the user in a manner taking advantage of the
`
`partial construction already performed and via user interface modalities in addition
`
`to spoken natural language.” Id., 11:14-20. The 718 Patent states that the
`
`additional input is used to refine the query, and these steps are repeated until no
`
`remaining problems or deficiencies are identified. Id., 11:25-32.
`
`D. Construction of Claim Terms
`Given the near identity in the prior art addressed below and the 718 Patent,
`
`the Board need not adopt specific constructions for any claim term because, under
`
`any reasonable construction, those features were taught by the prior art. Vivid
`
`Techs., Inc. v. Am. Sci. & Eng’g, Inc., 200 F.3d 795, 803 (Fed. Cir. 1999) (claim
`
`terms need only be construed to the extent necessary to resolve the case).
`
`Nevertheless, if the Board believes any term needs construction, Petitioner
`
`respectfully requests that the construction of those terms include Patent Owner’s
`
`below interpretations proposed in District Court.
`
`9
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`“navigation query”
`1.
`Patent Owner proposed that “navigation query” be interpreted as “an
`
`electronic query, form, series of menu selections, or the like; being structured
`
`appropriately so as to navigate a particular data source of interest in search of
`
`desired information.” Letter (Ex.1007), 2.
`
`The 718 Patent also defines the term “navigated” to mean “the contents [of a
`
`data source] are accessed and searched.” Ex.1001, 5:24-25. This appears
`
`consistent with what a Skilled Artisan would conclude is the ordinary meaning of
`
`that term (and related terms, such as “navigate” or “navigation”) in the context of
`
`the 718 Patent. Ex.1002,¶57.
`
`“electronic data source”
`2.
`Patent Owner proposed that “electronic data source” be interpreted as
`
`“source of information in numerical form that can be digitally transmitted or
`
`processed and that is implemented on or by means of a computing device.”
`
`Ex.1007, 2. Patent Owner asserted that its construction “encompasses the range of
`
`electronic data sources discussed in the specification, including ‘database(s),
`
`Internet/web site(s),… multimedia content, such as movies or other digital video
`
`and audio content, other various forms of entertainment data, or other electronic
`
`information.’” Id.
`
`10
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`“rendering an interpretation of the spoken request”
`3.
`Patent Owner proposed that “rendering an interpretation of the spoken
`
`request” be interpreted as “determining a meaning of the spoken request using a
`
`computing device, such as that provided by extracting speech data from acoustic
`
`voice signals or data and linguistically parsing the speech data.” Ex.1007, 2.
`
`4.
`
`“constructing [at least part of] a navigation query based
`upon the interpretation”
`Patent Owner proposed that “constructing a navigation query based upon
`
`the interpretation” and “constructing at least part of a navigation query based
`
`upon the interpretation” be interpreted as “combining or arranging elements of (at
`
`least part of) the navigation query based upon the interpretation.” Ex.1007, 2.
`
`V. OVERVIEW OF THE PRINCIPLE REFERENCE
`A. Moran et al., Multimodal User Interfaces in the Open Agent
`Architecture (1997) (“Moran”) (Ex.1003)
`Public Availability
`1.
`Moran was published in 1997 in conjunction with the International
`
`Conference on Intelligent User Interfaces that took place on January 6–9, 1997.
`
`Ex.1003, cover, i. Ex.1002,¶68. Moran was presented and distributed at the
`
`January 1997 conference to all attendees without any obligation of secrecy or
`
`confidentiality. Ex.1002,¶68. Copies of the IUI 97 Publication that includes Moran
`
`were available from the Association for Computing Machinery to interested
`
`persons in the field and were received by libraries, such as M.I.T (where it was
`
`11
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`catalogued and indexed), at least by March 14, 1997. Ex.1003, i; Ex.1002,¶69.
`
`Moran was thus publicly available to interested members of the public no later than
`
`March 1997 and as early as January 6, 1997, Ex.1002,¶¶68-69, and is thus prior art
`
`to the 718 Patent at least under 35 U.S.C. §§102(a) and (b).
`
`In addition, Moran was published by the Association for Computing
`
`Machinery (“ACM”) in 1997. Ex.1003, cover, i, 61. For a similar organization,
`
`the Board has that the “IEEE is a well-known, reputable compiler and publisher of
`
`scientific and technical publications,” and in such circumstances, the Board has
`
`taken “Official Notice that members in the scientific and technical communities
`
`who both publish and engage in research rely on the information published on the
`
`copyright line of IEEE publications.” Power Integrations, Inc., v. Semiconductor
`
`Components Industries, LLC, IPR2018-00377, Paper No. 10, *10 (July 17, 2018);
`
`see also id. (“The Board has determined that ‘[a]llowing IPR petitioners to rely on
`
`the IEEE publication date in an IPR proceeding, which is an administrative
`
`proceeding designed and intended to afford expedited and efficient relief, serves
`
`the interests of justice.’”) The ACM is also a “well-known, reputable compiler and
`
`publisher of scientific and technical publications,” see, e.g.,
`
`https://www.acm.org/about-acm/about-the-acm-organization, so the same
`
`reasoning should apply to publications from the ACM, such as a Moran, see also
`
`Coriant (USA) Inc. et al. v. Oyster Optics, IPR2018-00258, Paper No. 13, *11
`
`12
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`(June 6, 2018) (“For established publishers, demonstrating a date of publication is
`
`alone sufficient for showing accessibility to the public.”)
`
`2. Overview of Moran
`Moran describes “agent-based multimodal user interfaces” and their
`
`operation in conjunction with other agents implemented under the Open Agent
`
`Architecture (OAA) system that was designed and developed with a focus on
`
`“providing access to agent-based applications through an intelligent, cooperative,
`
`distributed, and multimodal agent-based user interfaces.” Ex.1003, 61, Abstract.
`
`Moran notes that “[o]ne of the major advantages of having an agent-based
`
`interface to a multiagent application is that it greatly simplifies the interactions
`
`between the user and the application: application agents may interact with a human
`
`in the same way they interact with any other agent.” Id., 65. The use of the OAA
`
`in Moran appears to be the same as in the 718 Patent as the five authors of Moran
`
`include two of the four inventors listed on the face of the 718 Patent. Ex.1003, 61;
`
`Ex.1001, face.
`
`Moran explains that “[i]n a number of domains, access to information can
`
`very naturally be organized around a map-based interface,” and “the agent-based
`
`approach to multimodality [is] extremely useful” in designing such interfaces.
`
`Ex.1003, 66. Moran provides exemplary multiagent applications with multimodal
`
`input for information retrieval in several domains. Ex.1003, 66-67 (describing
`
`13
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`office assistant application, map-based tourism information system, military
`
`simulator “CommandTalk” system, “Air Travel Information System,” and disaster
`
`response system).
`
`Moran describes coordinating various input modalities, including speech-
`
`based natural language, for implementing the user interface. Ex.1003, 62, 66.
`
`Users can “enter commands with a mix of modalities, for example, a spoken
`
`command in which the object to be acted on is identified by a pen gesture (or other
`
`graphical pointing operation,” because the system “supports speech, handwriting
`
`and pen-based gestures in addition to the conventional keyboard and mouse
`
`inputs.” Id., 62. Moran discloses that existing speech recognition systems can be
`
`used with its applications, once agents have been created to interface with those
`
`systems. Id. Moran also states that “[a] major advantage of using an agent-based
`
`architecture is that it provides simple mix-and-match for the components,” and
`
`describes existing natural language systems that can be used interchangeably. Id.
`
`Moran discloses a User Interface (UI) Agent, which “manages the various
`
`modalities and applies additional interpretation to those inputs as needed,” along
`
`with a subset of agents including a speech recognition agent and a natural language
`
`understanding agent. Id. Moran also discloses a Modality Coordination agent,
`
`responsible “for combining the inputs in the different modalities to produce a
`
`single meaning that matches the user’s intention” as well as “for resolving
`
`14
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`references, for filling in missing information for an incoming request, and for
`
`resolving ambiguities by using contexts, equivalence or redundancy.” Id., 63.
`
`Moran further discloses a Facilitator agent that receives requests and delegates
`
`them to an appropriate agent. Id., 64.
`
`Moran explains the interactions between agents in processing user queries.
`
`Id., 62 (“The UI agent manages the interpretation of the individual modalities and
`
`passes the results to a Modality Coordination agent, which returns the composite
`
`query, which is then passed to the Facilitator agent for delegation to the
`
`appropriate application agents ....”). Moran explains that agents communicate via
`
`“Interagent Communication Language” or “ICL,” a “high-level logical language”
`
`that was “designed to be compatible with the output of our natural language
`
`understanding systems, thereby simplifying transforming a user’s query or
`
`command into one that can be handled by the automated agents.” Id., 64.
`
`In addition, Moran discloses using agent-based interfaces for accessing
`
`electronic data sources like distributed applications and computer databases. E.g.,
`
`id., 61 (“OAA-based applications can be run from a lightweight portable computer
`
`(or PDA) because only the user interface agents need run on the portable. They
`
`provide the user with access to a range of agents running on other platforms.”); id.,
`
`65 (“This natural language request installed a trigger on an agent ... [that] instructs
`
`a Web retrieval agent to scan data from three on-line newspaper databases.”).
`
`15
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`VI. DETAILED REASONS FOR RELIEF REQUESTED
`A. Moran Anticipates Claims 1-4, 6, 8-13, 15, 17-22, 24, 26-27
`Independent Claim 1
`1.
`Preamble
`a.
`The preamble recites “[a] method for speech-based navigation of an
`
`electronic data source located at one or more network servers located remotely
`
`from a user, wherein a data link is established between a mobile information
`
`appliance of the user and the one or more network servers.”
`
`i.
`
`Speech-based Navigation of a Remote Electronic
`Data Source
`Moran describes the same “Open Agent Architecture (OAA) system” as
`
`described in the 718 Patent and which implements a user interface with multiple
`
`input modalities, including support for spoken natural language (“speech-based”)
`
`as input to the multimodal interface, to access and search distributed computer-
`
`based data sources, as described below. Ex.1003, 62 (interface “is based on natural
`
`language (for example, English), and is entered with either speech or
`
`handwriting”); Ex.1002,¶¶98-99.
`
` “[N]avigation” would be understood to encompass “accessing and
`
`searching the contents of a data source.” §IV.D.1; Ex.1002,¶57.
`
`“[E]lectronic data source” would be understood to encompass a “source of
`
`information in numerical form that can be digitally transmitted or processed and
`
`16
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`that is implemented on or by means of a computing device.” §IV.D.2;
`
`Ex.1002,¶59.
`
`The interface agents in Moran are used for “accessing and searching”
`
`computer-based data sources such as databases, websites, or other electronic
`
`information repositories (“navigation of an electronic data source”).
`
`Ex.1002.,¶102. For example, Moran explains that for a tourist application “the
`
`user can ask for the distance between a hotel and sightseeing destination,” and
`
`“[t]he locations of the two places are in different databases . . . .” Ex.1003, 62
`
`(describing providing “access [to] conventional applications such as email,
`
`calendar, and databases” using “spoken or written natural language commands”);
`
`id., 65-66 (describing different distributed electronic databases such as an “office
`
`assistant” or “map-based tourist information”).
`
`The “electronic data source[s]” described in Moran include online
`
`databases and other network-based sources located at “distributed” or “remote”
`
`network servers (i.e., located at one or more network servers located remotely
`
`from a user.) Ex.1002,¶105 (citing, e.g. Ex.1003, 65-66 (describing “WWW
`
`sources,”