Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`In Re:
`
`U.S. Patent 6,757,718
`
`
`
`: Attorney Docket No. 081841.0113
`
`Inventor: Christine Halverson et al.
`
`Filed:
`
`Issued:
`
`June 30, 2000
`
`June 10, 2004
`
`
`
`
`
`
`
`
`
`Assignee:
`
`IPA Technologies Inc.
`
`:
`
`:
`
`: IPR No.: Unassigned
`
`Title:
`
`Mobile Navigation of Network-Based Electronic Information using
`Spoken Input
`
`Mail Stop PATENT BOARD
`Patent Trial and Appeal Board
`U.S. Patent and Trademark Office
`P.O. Box 1450
`Alexandria, Virginia 22313-1450
`
`Submitted Electronically via the Patent Trial and Appeal Board End to End System
`
`PETITION FOR INTER PARTES REVIEW OF CLAIMS 1-4, 6, 8-9, 10-13,
`15, 17-18, 19-22, 24, AND 26-27 OF U.S. PATENT NO. 6,757,718 UNDER 35
`U.S.C. §§ 311-319 AND 37 C.F.R. §§ 42.100 ET SEQ.
`
`
`
`
`
`1
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`TABLE OF CONTENTS
`
`
`
`I. MANDATORY NOTICES, STANDING, AND FEES ..................................... 7
`II. OVERVIEW OF CHALLENGE AND RELIEF REQUESTED ....................... 9
`A. Publications Relied Upon ................................................................................ 9
`B. Grounds for Challenge ..................................................................................15
`III. OVERVIEW OF THE ’718 PATENT ..........................................................15
`A. Summary of the Claimed Subject Matter .......................................................15
`B. Prosecution History of the ’718 Patent .........................................................18
`C. Priority ...........................................................................................................19
`IV. SUMMARY OF PRIOR ART AND REFERENCES RELIED ON.............19
`A. Brief Summary of Kupiec (Ex. 1013) .............................................................19
`B. Brief Summary of Cheyer (Ex. 1019).............................................................23
`C. Brief Summary of Kimura (Ex. 1015) ............................................................24
`D. Brief Summary of Freeman (Ex. 1014) ..........................................................25
`V. CLAIM CONSTRUCTION ..............................................................................26
`A. Level of Ordinary Skill in the Art ..................................................................26
`B. Preambles of independent claims 1, 10, and 19 are limiting. .......................26
`C. “navigation query” (Claims 1, 4, 10, 13, 19, and 22) ...................................27
`D. “mobile information appliance” (Claims 1-3, 8, 10, 12, 17, 19, 21, and 26) 28
`E. “mobile information appliance comprises a portable remote control device
`or a set-top box for a television” (Claims 1, 10, and 19) .....................................29
`VI. A REASONABLE LIKELIHOOD EXISTS THAT THE CHALLENGED
`CLAIMS ARE UNPATENTABLE .........................................................................30
`A. Ground 1: The ‘718 Patent Claims 1-4, 6, 8-9, 10-13, 15, 17-18, 19-22, 24,
`and 26-27 are obvious over Kupiec (Ex. 1013) in view of Cheyer (Ex. 1019). ...30
`1.
`Independent Claim 1. .................................................................................30
`2. Dependent Claims 2-4, 6, and 8-9. .............................................................38
`3.
`Independent Claim 10. ...............................................................................49
`4. Dependent Claims 11-13, 15, and 17-18. ...................................................51
`5.
`Independent Claim 19. ...............................................................................54
`
`2
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`6. Dependent Claims 20-22, 24, and 26-27. ...................................................57
`B. Ground 2: The ‘718 Patent Claims 1-4, 6, 8-9, 10-13, 15, 17-18, 19-22, 24,
`and 26-27 are obvious over Kupiec (Ex. 1013) in view of Cheyer (Ex. 1019) and
`in further view of Kimura (Ex. 1015). ..................................................................59
`1.
`Independent Claim 1. .................................................................................60
`2. Dependent Claims 2 & 3. ...........................................................................62
`3. Dependent Claims 4, 6, 8-9 ........................................................................64
`4.
`Independent Claim 10. ...............................................................................64
`5. Dependent Claim 12. ..................................................................................64
`6. Dependent Claims 11, 13, 15, and 17-18. ..................................................64
`7.
`Independent Claim 19. ...............................................................................65
`8. Dependent Claim 21. ..................................................................................65
`9. Dependent Claims 20, 22, 24, and 26-27. ..................................................65
`C. Ground 3: Claims 6, 15, and 24 are obvious over Kupiec in view of Cheyer
`in further view of Freeman. ..................................................................................65
`1. Dependent Claim 6 .....................................................................................66
`2. Dependent Claim 15 ...................................................................................68
`3. Dependent Claim 24 ...................................................................................69
`VII. CONCLUSION ..............................................................................................69
`
`
`
`
`
`
`
`
`3
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`LIST OF EXHIBITS
`
`U.S. Patent No. 6,742,021 by Christine Halverson, Luc Julia, Dimitris
`Voutsas, and Aden J. Cheyer, entitled “Navigating Network-Based
`Electronic Information Using Spoken Input with Multimodal Error
`Feedback”
`
`File History for U.S. Patent No. 6,742,021
`
`U.S. Patent No. 6,757,718 by Christine Halverson, Luc Julia, Dimitris
`Voutsas, and Adam Cheyer, entitled “Mobile Navigation of Network-
`Based Electronic Information Using Spoken Input”
`
`File History for U.S. Patent No. 6,757,718
`
`U.S. Patent No. 6,523,061 by Christine Halverson, Luc Julia, Dimitris
`Voutsas, and Adam Cheyer, entitled “System, Method, and Article of
`Manufacture for Agent-Based Navigation in a Speech-Based Data
`Navigation System”
`
`File History for U.S. Patent No. 6,523,061
`
`U.S. Patent No. 6,851,115 by Christine Halverson, Luc Julia, Dimitris
`Voutsas, and Adam Cheyer, entitled “Software-Based Architecture for
`Communication and Cooperation Among Distributed Electronic
`Agents”
`
`File History for U.S. Patent No. 6,851,115
`
`File History for U.S. Patent Application No. 60/124,720
`
`File History for U.S. Patent Application No. 60/124,719
`
`File History for U.S. Patent Application No. 60/124,718
`
`Declaration of Dr. Kevin Negus
`
`U.S. Patent No. 5,500,920 by Julian M. Kupiec, entitled “Semantic
`Co-ocurrence Filtering
`for Speech Recognition and Signal
`Transcription Applications” (“Kupiec”)
`
`U.S. Patent No. 6,006,227 by Eric Freeman et al., entitled “Document
`Stream Operating System” (“Freeman”)
`
`1001
`
`1002
`
`1003
`
`1004
`
`1005
`
`1006
`
`1007
`
`1008
`
`1009
`
`1010
`
`1011
`
`1012
`
`1013
`
`1014
`
`4
`
`

`

`1015
`
`1016
`
`1017
`
`1018
`
`1019
`
`1020
`
`1021
`
`1022
`
`1023
`
`1024
`
`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`U.S. Patent No. 5,247,580 by Toshiyuki Kimura et al., entitled
`“Voice-operated remote control system” (“Kimura”)
`
`Complaint, IPA Technologies Inc. v. DISH Network Corp. et al., No.
`1:16-cv-01170 (D. Del.) (“District Court Litigation”)
`
`Proof of Service of Complaint on DISH Network L.L.C. and DISH
`Network Corporation
`
`Source Code Appendix to U.S. Patent No. 5,500,920 by Julian M.
`Kupiec
`
`Non-patent literature publication by Adam J. Cheyer and Luc Julia,
`two of the named inventors on the ’061 Patent, entitled “Multimodal
`Maps: An Agent-based Approach” (“Cheyer”), first published on May
`26, 1995.
`
`Non-patent literature publication by Sankyu Park et. al, citing Cheyer
`article, entitled “A Framework for Multi-Agent Systems with Multi-
`modal User Interfaces in Distributed Computing Environments,”
`published in 1997.
`
`Non-patent literature publication by Yi Han and Ingrid Zukerman,
`citing Cheyer article, entitled “A Mechanism for Multimodal
`Presentation Planning Based on Agent Cooperation and Negotiation,”
`published in 1997.
`
`Non-patent literature publication by Andrew Kehler et al., citing
`Cheyer article, entitled “On Representing Salience and Reference in
`Multimodal Human-Computer Interaction,” published in 1998.
`
`Non-patent literature publication by Philip Cohen et al., citing Cheyer
`article, entitled “QuickSet: Multimodal Interaction for Distributed
`Applications,” published in 1997.
`
`Non-patent literature publication by Jean-Claude Martin, citing
`Cheyer article, entitled “Towards ‘intelligent’ cooperation between
`modalities. The example of a system enabling multimodal interaction
`with a map,” published in 1997.
`
`5
`
`

`

`1025
`
`1026
`
`1027
`
`1028
`
`1029
`
`1030
`
`1031
`
`1032
`
`1033
`
`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`Non-patent literature publication by Michael Johnston et al., citing
`Cheyer article, entitled “Unification-based Multimodal Integration,”
`published in 1997.
`
`Declaration of Harry Bunt Concerning the International Conference
`on Cooperative Multimodal Communication (CMC /95) in Eindhoven,
`May 24-26, 1995 and the Publication of Papers Presented at the
`Conference.
`
`the International
`Declaration of Michael McTear Concerning
`Conference on Cooperative Multimodal Communication (CMC /95) in
`Eindhoven, May 24-26, 1995.
`
`Declaration of Gert-Jan van Velzen Concerning the “Proceedings of
`the
`International Conference
`on Cooperative Multimodal
`communications: CMC /95, Eindhoven, May 24-26, 1995” Reference.
`
`Affidavit of Christopher Butler from the Internet Archive.
`
`Declaration of Scott Bennett, Ph.D.
`
`Declaration of Ted Baldwin Concerning the “PAAM 96: Proceedings
`of the First International Conference on the Practical Application of
`Intelligent Agents and Multi-Agent Technology, 22nd-24th April 1996”
`Reference.
`
`Redline Comparison between article text of Cheyer in Exhibit 1019
`(published in 1995 Proceedings Publication) and article text in Ex.
`1030, Attachment 1d (republished in 1998 Proceedings Publication).
`
`Redline Comparison between article text of Cheyer in Exhibit 1019
`(published in 1995 Proceedings Publication) and article text in Ex.
`1029, Exhibit A (Cheyer Article on SRI Website no later than August
`1997 and preserved by Internet Archive).
`
`1034
`
`
`
`Certified Translations of Exhibits C and D to the Declaration of Gert-
`Jan van Velzen (Exhibit 1028).
`
`
`6
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`I. MANDATORY NOTICES, STANDING, AND FEES
`
`Real Party in Interest: DISH Network Corporation and DISH Network
`
`L.L.C. (collectively, "Petitioner" or "DISH") are the Petitioner. DISH is a provider
`
`of direct broadcast satellite services. Non-party EchoStar Technologies L.L.C. is a
`
`real party in interest. EchoStar Technologies L.L.C. is now a subsidiary of DISH
`
`Network Corporation and provides set top boxes to DISH that are used to provide
`
`direct broadcast satellite services to customers. In the past, DISH has also listed
`
`EchoStar Corporation (the former corporate parent of EchoStar Technologies
`
`L.L.C.) as a real party in interest in PTAB proceedings involving DISH. However,
`
`due to the change in EchoStar Technologies L.L.C.’s ownership, EchoStar
`
`Corporation is no longer a real party in interest for this proceeding.
`
`Related Matters: The ’718 Patent is currently involved in a pending lawsuit
`
`involving Petitioner entitled IPA Technologies, Inc. v. DISH Network Corporation
`
`et al., United States District Court for the District of Delaware, Case No. 1:16-CV-
`
`01170 (RGA) (the “District Court Action”). See Ex. 1016. Patent Owner asserts
`
`U.S. Patent No. 6,757,718 against Petitioner in the District Court Action. Ex.
`
`1016, pp. 13-17. The ’718 Patent is also involved in the following related
`
`proceedings: IPA Technologies Inc. v. NVIDIA Corporation., No. 1-17-cv-00287
`
`(D. Del.); IPA Technologies Inc. v. Sony Electronics Inc., et al., No. 1-17-cv-00055
`
`(D. Del.); IPA Technologies Inc. v. Amazon.com, Inc. et al., No. 1-16-cv-01266 (D.
`
`Del.).
`
`7
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`Lead Counsel and Request for Authorization: Pursuant to 37 C.F.R.
`
`§§ 42.8(b)(3) and 42.10(a), Petitioner designates the following: Lead Counsel is
`
`Eliot D. Williams (Reg. No. 50,822) of Baker Botts L.L.P.; Back-up Counsel are
`
`G. Hopkins Guy (Reg. No. 35,886) and Ali Dhanani (Reg. No. 66,233) of Baker
`
`Botts L.L.P.
`
`Service Information: Service information is as follows: Baker Botts L.L.P.,
`
`1001 Page Mill Rd., Palo Alto, CA 94304-1007 Tel. 650 739 7500; Fax 650-736-
`
`7699.
`
`
`
`Petitioner
`
`consents
`
`to
`
`service
`
`by
`
`electronic mail
`
`at
`
`eliot.williams@bakerbotts.com,
`
`hop.guy@bakerbotts.com,
`
`and
`
`ali.dhanani@bakerbotts.com. A Power of Attorney is filed concurrently herewith
`
`under 37 C.F.R. § 42.10(b).
`
`Certification of Grounds: Petitioners certify that the ’718 Patent is eligible
`
`for inter partes review. Petitioners were served with the complaint in the District
`
`Court Action on December 20, 2016. Ex. 1017. Therefore, Petitioners are not
`
`barred or estopped from requesting inter partes review on the grounds set forth
`
`herein.
`
`Fees: The Office is authorized to charge the fee set forth in 37 C.F.R.
`
`§ 42.15(b) to Deposit Account No. 02-0384 as well as any additional fees that
`
`might be due in connection with this Petition.
`
`8
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`II. OVERVIEW OF CHALLENGE AND RELIEF REQUESTED
`
`Petitioner challenges claims 1-4, 6, 8-9, 10-13, 15, 17-18, 19-22, 24, and 26-
`
`27 of U. S. Patent No. 6, 757,718 (the “’718 Patent), titled “Mobile Navigation of
`
`Network-Based Electronic Information Using Spoken Input.” See Ex. 1001.
`
`A.
`
`Publications Relied Upon
`
`As discussed infra, the ’718 Patent is not entitled to a priority date before
`
`March 13, 2000. Petitioner relies upon the following patents and publications:
`
`Exhibit 1013 — U.S. Patent No. 5,500,920 by Julian M. Kupiec, entitled
`
`“Semantic co-occurrence filtering for speech recognition and signal transcription
`
`applications” (“Kupiec”), filed on September 30, 1994 and issued on March 19,
`
`1996. Kupiec is available as prior art under at least 35 U.S.C. §§ 102(a), 102(b),
`
`and 102(e).
`
`Exhibit 1019 — Adam J. Cheyer and Luc Julia of SRI International ("SRI"),
`
`two of the named inventors on the ’061 Patent, entitled “Multimodal Maps: An
`
`Agent-based Approach” (“Cheyer”). Ex. 1019, p.2. As detailed below, Cheyer
`
`was first distributed and made available to members of the public having ordinary
`
`skill in the art no later than May 24, 1995. Cheyer was thereafter indexed and
`
`catalogued and publicly available in libraries around the world and on SRI's public
`
`website. Cheyer is a publication under 102(b) because it has been “disseminated
`
`or otherwise made available to the extent that persons interested and ordinarily
`
`9
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`skilled in the subject matter or art exercising reasonable diligence, can locate it.”
`
`SRI Int'l, Inc. v. Internet Sec. Sys., Inc., 511 F.3d 1186, 1194 (Fed. Cir. 2008).
`
`May 24, 1995 Presentation at CMC/95 Conference
`
`The First
`
`International Conference on Cooperative Multimodal
`
`Communication was held in Eindhoven, The Netherlands from May 24-26, 1995
`
`(“CMC/95”). Ex. 1019; Ex. 1026, ¶¶ 5,10-11; Ex. 1027, ¶¶5,8. Certain papers
`
`presented at the conference, including Cheyer, were collected and published in the
`
`“Proceedings of the International Conference on Cooperative Multimodal
`
`Communication CMC/95: Eindhoven, May 24-26, 1995” by H.C. Bunt and R.J.
`
`Beun (“1995 Proceedings Publication”). Ex. 1026, ¶10. The 1995 Proceedings
`
`Publication, including Cheyer, was distributed to all CMC/95 attendees at the
`
`conference and thus publicly available no later than May 24, 1995. Ex. 1026, ¶11.
`
`CMC/95 was attended by individuals active in the area of multimodal
`
`communications and spoken language technologies, including at least 50 people.
`
`Ex. 1026, ¶¶6,13; Ex. 1027, ¶6. Non-attendees working in the field would have
`
`known of the conference because the number in the field was not very large and
`
`the conference was well publicized. Ex. 1026, ¶¶13-14; Ex. 1027, ¶¶6-7. Thus,
`
`Cheyer was publicly available, and specifically was distributed to researchers in
`
`the field of natural language processing and multimodal communication, no later
`
`than May 24, 1995. Ex. 1026, ¶¶11,15; Ex. 1027, ¶8. See MIT v. AB Fortia, 774
`
`10
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`F.2d 1104, 1108-09 (Fed. Cir. 1985) (paper presented orally at conference and
`
`subsequently distributed to six recipients constitutes “printed publication”).
`
`In addition to dissemination at CMC/95, Cheyer was also publicly available
`
`in libraries before the critical date.
`
`WorldCat
`
`The 1995 Proceedings Publication was entered into the WorldCat library
`
`catalog on August 4, 1995, indicating that the publication was available in at least
`
`one library by September 1995. Ex. 1030, ¶¶19,35-36; Attachment1f. The catalog
`
`entry was searchable at least by title and conference name (which was descriptive
`
`of the content), or the organizer’s name (Harry Bunt). Ex. 1030, ¶35;
`
`Attachment1f.
`
`Netherlands Royal Library
`
`The 1995 Proceeding Publication, including Cheyer is publicly available
`
`from the Netherlands Royal Library (“NRL”). Ex. 1028. NRL received the 1995
`
`Proceedings Publication on August 8, 1996 and cataloged it on September 13,
`
`1996. Id., ¶¶8-9. Beginning on that date, Cheyer was available to the public and
`
`could be found by searching via author, descriptive title, or keywords. Id., ¶¶6,9.
`
`Thus, Cheyer was also cataloged, searchable, and accessible to the interested
`
`public at NRL at least by September 13, 1996. Id.
`
`Institute for Perceptual Research
`
`11
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`Cheyer has also been publicly available at the library at the Institute for
`
`Perceptual Research (or Instituut Voor Perceptie Onderzoek) – whose holdings
`
`were subsequently transferred to the library of Eindhoven University of
`
`Technology – since at least 1996. Ex. 1026, ¶¶12. The copy of the 1995
`
`Proceedings Publication held at the main library of Eindhoven University of
`
`Technology bears markings indicating that it was received, indexed, and cataloged
`
`by the Institute for Perceptual Research at Eindhoven at least by 1996. Id. The
`
`library at the Institute for Perceptual Research at Eindhoven was open to the
`
`public, including those of ordinary skill in the art and maintained a catalog of
`
`publications that allowed searching on title, author, or keyword. Id. Accordingly,
`
`Cheyer was cataloged, searchable, and accessible to the interested public at the
`
`Institute for Perceptual Research library at least by 1996. Id.
`
`SRI Website
`
`Cheyer was
`
`also
`
`publicly
`
`available
`
`on
`
`the
`
`SRI website
`
`(http:/www.ai.sri.com:80/~cheyer/papers/mmap/mmap.html), bearing a date of
`
`August 12, 1996. Ex. 1029. The Internet Archive captured and made an archival
`
`copy publicly available, starting with the abstract and hyperlinked table of
`
`contents, with each page of the article navigable via linked pages. Id. Each of the
`
`pages was captured and made publicly available no later than August 8, 1997 Id.
`
`SRI was known in the field of natural language processing and multimodal
`
`communication and it was common for those of ordinary skill to review and
`
`12
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`reference SRI publications and other documents (Ex. 1026, ¶¶ 16-17; Ex. 1027,
`
`¶¶9-10). A person of ordinary skill in the art (“POSA”) exercising reasonable
`
`diligence, would have been able to find the Cheyer paper on the SRI website.
`
`Republication
`
`In 1998, Cheyer was republished in Multimodal Human Computer
`
`Communication: Systems, Techniques, and Experiments, Harry C. Bunt et al. eds.,
`
`in Lecture Notes in Artificial Intelligence 1374 (Berlin: Springer, 1998),
`
`(hereinafter, "1998 Proceedings Publication"). Ex. 1030, Attachment 1d. The
`
`1998 Proceedings Publication is held in 10 libraries world-wide, and library
`
`records for it were created on April 29, 1998. Id., ¶37. Accordingly, it was
`
`publicly available in at least one library by at least May 1998. Id., ¶38. One
`
`library’s copy is stamped June 25, 1998. Id., Attachment1d, p.5. The 1998
`
`Proceedings Publication could be found by searching for: (1) the descriptive title;
`
`(2) the descriptive conference name; and (3) the name of the Springer series
`
`(Lecture notes in computer science). Id., ¶37. Thus, the 1998 Proceedings
`
`Publication was publicly available no later than May 1998. Id., ¶¶37-38.
`
`Highly-Cited
`
`Public availability of Cheyer is corroborated by the numerous researchers
`
`spanning companies and timeframe that cited to Cheyer in their own publications
`
`from 1997-1998. See Exhibits 1020 – 1025 (published articles citing Cheyer). One
`
`such article was entitled “Development Tools for the Open Agent Architecture”
`
`13
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`published in the Proceedings of the First International Conference on the Practical
`
`Application of Intelligent Agents and Multi-Agent Technology (PAAM 96), which
`
`was publicly available in 1997. See Exhibit 1031 (declaration of Ted Baldwin,
`
`establishing public availability of PAAM 96 at Exhibit A by November 17, 1997).
`
`Updates to Cheyer
`
`The authors of Cheyer made formatting changes between May 1995 and
`
`May 1998, with no relevant substantive changes to the disclosure. Petitioners
`
`include a redline comparison between the text of the 1995 Proceedings Publication
`
`and 1998 Proceedings Publication at Exhibit 1032. Petitioners are also including a
`
`redline comparison between the text of the 1995 Proceedings Publication and the
`
`text hosted on the SRI website. Ex. 1033. The SRI website version included one
`
`additional paragraph (see pp.3-4) and an acknowledgements paragraph (see p.9)
`
`not included in the 1995 and 1998 versions, but the Petition relies on neither
`
`paragraph. For convenience, throughout the Petition and the Negus Declaration,
`
`citations are to the version in the 1995 Proceedings Publication.
`
`Cheyer is available as prior art under at least 35 U.S.C. § 102(b) because it was
`
`disseminated on May 24, 1995 at the 1995 Proceedings and was publicly available:
`
`(1) as the 1995 Proceedings Publication from at least one library since September
`
`1995, from the NRL since September 13, 1996, and from the Institute for
`
`Perceptual Research since 1996; (2) on SRI's Website no later than August 8,
`
`1997; and (3) as the 1998 Proceedings Publication from libraries since May 1998.
`
`14
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`Exhibit 1015 — U.S. Patent No. 5,247,580 by Toshiyuki Kimura et al.,
`
`entitled “Voice-operated remote control system” (“Kimura”), filed on July 22,
`
`1992, and issued on September 21, 1993. Kimura is available as prior art under at
`
`least 35 U.S.C. §§ 102(a), (b), and (e).
`
`Exhibit 1014 — U.S. Patent No. 6,006,227 by Eric Freeman et al., entitled
`
`“Document Stream Operating System” (“Freeman”), filed on June 28, 1996 and
`
`issued on December 21, 1999. Freeman is available as prior art under at least 35
`
`U.S.C. §§ 102(a), 102(e).
`
`B. Grounds for Challenge
`
`Petitioner requests cancellation of the claims on the following grounds:
`
`1.
`
`Claims 1-4, 6, 8-9, 10-13, 15, 17-18, 19-22, 24, and 26-27 are obvious
`
`over Kupiec with Cheyer.
`
`2.
`
`Claims 1-4, 6, 8-9, 10-13, 15, 17-18, 19-22, 24, and 26-27 are obvious
`
`over Kupiec and Cheyer with Kimura.
`
`3.
`
`Claims 6, 15, and 24 are obvious over Kupiec and Cheyer with
`
`Freeman.
`
`III. OVERVIEW OF THE ’718 PATENT
`
`A.
`
`Summary of the Claimed Subject Matter
`
`The ’718 Patent describes navigating an electronic data source by means of
`
`spoken language using one or more “agents.” Ex. 1003, at 2:35-38. An object of
`
`the invention is to provide “information navigation technology that allows
`
`15
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`relatively naïve users to navigate and access desired data by means of natural
`
`language input.” Id. at 1:29-32. The system as described is shown below in FIG.
`
`1a.
`
`Id. at FIG. 1a. A “voice input device” receives voice input data. Id. at. 3:56-
`
`58. The voice input data is transmitted across a network to be interpreted by
`
`“request processing logic.” Id. at. 4:11-15.
`
`
`
`16
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`
`
`A “speech recognition engine processes acoustic voice data and attempts to
`
`generate a text stream of recognized words.” Id. at 7:39-41. A “navigation query”
`
`is constructed based upon the interpretation of the spoken request. Id. at 8:62-64.
`
`The “navigation query” is used to “navigate” a remote “data source 110. ” Id. at
`
`4:30-32.
`
`The ’718 Patent discloses a “mobile computing embodiment” of the system,
`
`wherein the “voice input device 102, communications box 104, and client display
`
`device 112” are replaced by an “integrated, mobile, information appliance 202
`
`such as a cellular telephone or wireless personal digital assistant (wireless PDA).”
`
`Id. at. 6:2-6.
`
`17
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`
`
`B.
`
`Prosecution History of the ’718 Patent
`
`The ’718 Patent was filed on June 30, 2000, as a continuation of U.S. Patent
`
`Application No. 09/524,095, filed on March 13, 2000, which later issued as
`
`asserted patent 6,742,021 (“the ’021 Patent”). Ex. 1001. The ’021 Patent was a
`
`continuation-in-part application of application no. 09/225,198, filed on January 5,
`
`1999. Ex. 1007.
`
`During prosecution, the claims faced rejections under 35 U.S.C. § 101 for
`
`double patenting rejection over the ’021 Patent application. Ex. 1004, p.78.
`
`Furthermore, the claims were amended to overcome rejections under 35 U.S.C. §§
`
`102 and 103, including rejections for both anticipation and obviousness over a
`
`18
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`prior art reference known as “Levin.” Id. at 79. Applicants amended the claims to
`
`require that the “mobile information appliance comprises a portable remote control
`
`device or a set-top box for a television.” Id. at 155-156.
`
`C.
`
`Priority
`
`The ’718 Patent was filed on June 30, 2000, as a continuation of U.S. Patent
`
`Application No. 09/524,095, filed on March 13, 2000, which later issued as
`
`the ’021 Patent. Ex. 1001. The ’021 Patent was a continuation-in-part application
`
`of application no. 09/225,198, filed on January 5, 1999. Ex. 1007. The ’021
`
`Patent also claims priority to three provisional applications each filed March 17,
`
`1999. Exs. 1009, 1010, 1011. Petitioner disagrees that the ’718 Patent is entitled
`
`to a priority date any earlier than March 13, 2000, as matter disclosed by the
`
`application that issued as the ’021 Patent was not present in either the parent
`
`application or any of the provisionals. In any event, the claims of the ’718 Patent
`
`are nevertheless invalid under a priority date of January 5, 1999.
`
`IV. SUMMARY OF PRIOR ART AND REFERENCES RELIED ON
`
`The ’718 Patent claims what was well-known in the prior art. None of the
`
`prior art discussed below was considered by the Patent Office during prosecution
`
`of the ’718 Patent.
`
`A. Brief Summary of Kupiec (Ex. 1013)
`
`U.S. Patent No. 5,500,920 by Julian M. Kupiec entitled “Semantic co-
`
`occurrence filtering for speech recognition and signal transcription applications”
`
`19
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`(“Kupiec”) was filed on Sep. 30, 1994, and issued on Mar. 19, 1996. Ex. 1013.
`
`Kupiec describes transcribing words from a form convenient for input by a human
`
`user, e.g., spoken or handwritten words, into a form easily understood by an
`
`applications program executed by a computer” including “transcription systems
`
`and methods appropriate for use in conjunction with computerized information-
`
`retrieval (IR) systems and methods.” Id. at 1:36-45.
`
`Kupiec discloses a “transducer 20 converts a user's spoken utterance into a
`
`signal that can be processed by processor 10.” Id. at 5:43-6:7. Kupiec discloses
`
`various functional components implemented as software modules executed by the
`
`processor. Id. at 6:44-50.
`
`20
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`
`
`The processor 10 can be coupled to an IR subsystem 40 including “a
`
`processor that can process queries to search for documents in corpus 41.” Id. at
`
`5:44-48; 6:44-50, FIG. 4. The corpus “comprises a database of documents that can
`
`be searched” via “query operations.” Id. at 6:29-33; 6:53-7:48.
`
`21
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`
`
`Kupiec’s Fig. 2 shows a user “inputs a question 201 into system 1 by
`
`speaking ... where it is converted into a phonetic transcription 250.” Id. at 9:18-23.
`
`Thereafter, the “phonetic transcription 250 is provided to hypothesis generator 60
`
`where it is matched using phonetic index 62 to generate a set of hypotheses 260.”
`
`Id. at 9:38-61. “Query constructor 70 uses the hypotheses 260 to construct one or
`
`more queries 270 that will be sent to IR subsystem 40 for execution.” Id. at 11:10-
`
`13. “The execution of the initial and any additional queries causes a set of
`
`documents 240 to be retrieved from corpus 41.” Id. at 11:53-60. Finally, “the
`
`22
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`results 280 can be presented to the user using processor 10 in conjunction with
`
`visual display 30.” Id. at 12:34-42.
`
`B.
`
`Brief Summary of Cheyer (Ex. 1019)
`
`Cheyer describes a “prototype map-based application for a travel planning
`
`domain.” Ex. 1019, p.2. A user can utilize a “synergistic combination of several
`
`input modalities” to search the map application on a mobile device, such as a “pen-
`
`equipped PC’s or a Dauphin handheld PDA.” Id. at 1,5. Furthermore, the system
`
`is “connected either by modem or ethernet to a server machine which will manage
`
`database access, natural language processing and speech recognition for the
`
`application.” Id. at 5-6.
`
`Cheyer discusses
`
`the modification of
`
`the existing “Open Agent
`
`Architecture” (“OAA”). Id. at 6. The OAA uses a “a hierarchical configuration
`
`where client agents connect to a ‘facilitator’ server.” Id. at 7. The ‘facilitator’
`
`“records the published functionality of their sub-agents, and when queries arrive in
`
`Interagent Communication Language form, they are responsible for breaking apart
`
`any complex queries and for distributing goals to the appropriate agents.” Id.
`
`23
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 6,757,718
`
`
`
`As seen in Fig. 3, Cheyer describes several specific OAA agents used by the
`
`system. Id. at 8-9. The “Interface Agent” manages “what is currently being
`
`displayed to the user,” and accepts the user’s multimodal input. Id. at 9.
`
`In an example use case, Cheyer describes: a “user speaks: ‘How far is the
`
`restaurant from this hotel?’” Id. Then, the request is sent to the “natural language
`
`agent” and “translated into ICL form.” Id. at 10. The “interface agent uses
`
`contextual structures to find what ‘the restaurant’ refers to, and waits for the user to
`
`make a gesture indicating ‘the hotel’, issuing prompts if necessary.” Id. Finally,
`
`the “domain agent (RR) sends database requests asking for the coordinates of the
`
`items in question, ...calculates the distance according to the scale of the currently
`
`displayed map, and requests the user interface to produce output displaying the
`
`result of the calculation.” Id.
`
`C. Brief Summary of Kimura (Ex. 1015)
`
`Kimura teaches a “