`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`
`
`
`
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`
`
`
`
`
`MICROSOFT CORPORATION
`Petitioner,
`v.
`IPA TECHNOLOGIES, INC.,
`Patent Owner.
`Patent No. 6,757,718 B1
`Issued: June 29, 2004
`Filed: June 30, 2000
`Inventors: Christine Halverson, Luc Julia, Dimitris Voutsas, Adam Cheyer
`Title: SYSTEM, METHOD, AND ARTICLE OF MANUFACTURE FOR
`AGENT-BASED NAVIGATION IN A SPEECH-BASED DATA
`NAVIGATION SYSTEM
`
`
`
`
`
`
`Inter Partes Review No. IPR2018-01440
`
`
`
`
`
`
`
`
`DECLARATION OF DR. HENRY LIEBERMAN, HDR
`REGARDING U.S. PATENT NO. 6,757,718
`
`Petitioner Microsoft Corporation - Ex.1002, Cover
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`
`
`I, Henry Lieberman, do hereby declare and state, that all statements made
`
`herein of my own knowledge are true and that all statements made on information
`
`and belief are believed to be true; and further that these statements were made with
`
`the knowledge that willful false statements and the like so made are punishable by
`
`fine or imprisonment, under Section 1001 of Title 18 of the United States Code.
`
`
`
`
`
`
`
`Executed on this 9th day of August, 2018.
`
`_______________________________
`
`Henry Lieberman, HDR
`
`Petitioner Microsoft Corporation - Ex.1002, p. i
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`Table of Contents
`
`I.
`
`II.
`
`INTRODUCTION ........................................................................................... 1
`A.
`Engagement ........................................................................................... 1
`B.
`Background and Qualifications ............................................................. 1
`C.
`Compensation ........................................................................................ 3
`D.
`Information Considered ......................................................................... 3
`LEGAL STANDARDS FOR PATENTABILITY .......................................... 4
`A. Anticipation ........................................................................................... 5
`B. Obviousness ........................................................................................... 7
`III. The 718 Patent ............................................................................................... 12
`A.
`Effective Filing Date of the 718 Patent ............................................... 12
`B. Overview of the 718 Patent ................................................................. 13
`C.
`The Prosecution History of the 718 Patent ......................................... 19
`D.
`Claims of the 718 Patent ..................................................................... 20
`E.
`Construction of Terms Used in the 718 Patent Claims ....................... 25
`1.
`“navigation query” .................................................................... 26
`2.
`“electronic data source” ........................................................... 27
`3.
`“rendering an interpretation of the spoken request” ................ 27
`4.
`“constructing [at least part of] a navigation query based upon
`the interpretation” ..................................................................... 27
`IV. OVERVIEW OF THE pRINCIPLE PRIOR ART ........................................ 28
`A.
`Level of Ordinary Skill in the Art ....................................................... 28
`B. Moran et al., Multimodal User Interfaces in the Open Agent
`Architecture (1997) (“Moran”) (Ex.1003) .......................................... 29
`C. U.S. Patent No. 5,454,106 to Burns et al. (“Burns”) (Ex.1004) ......... 37
`D. U.S. Patent No. 6,188,985 to Thrift et al. (“Thrift”) (Ex.1005) .......... 44
`COMPARISON OF PRIOR ART TO THE 718 PATENT CLAIMS .......... 46
`A. Moran, or Moran in View of Burns, with or without Thrift Renders
`Claims 1-18 Unpatentable ................................................................... 46
`
`V.
`
`Petitioner Microsoft Corporation - Ex.1002, p. ii
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`1.
`
`2.
`3.
`
`4.
`5.
`6.
`
`Claim 1 Is Unpatentable ............................................................ 46
`a.
`Preamble ......................................................................... 46
`b.
`Receiving a Spoken Request .......................................... 59
`c.
`Rendering an Interpretation ............................................ 62
`d.
`Constructing a Navigation Query ................................... 63
`e.
`Utilizing the Navigation Query ...................................... 76
`f.
`Transmitting the Electronic Data Source ....................... 79
`Claims 2 and 3 are Unpatentable .............................................. 86
`Claim 4 Is Unpatentable ............................................................ 93
`a.
`Soliciting Additional Input ............................................. 93
`b.
`Refining the Navigation Query....................................... 97
`c.
`Selecting a Portion of the Data Source ........................... 99
`Claim 5 Is Unpatentable .......................................................... 105
`Claim 6 Is Unpatentable .......................................................... 112
`Claim 7 Is Unpatentable .......................................................... 113
`a.
`Claim 7 recites “[t]he method of claim 1, wherein the
`mobile information appliance is a wireless
`telephone.” .................................................................... 113
`Claim 8 Is Unpatentable .......................................................... 119
`Claim 9 Is Unpatentable .......................................................... 120
`Claim 10 Is Unpatentable ........................................................ 120
`a.
`Preamble ....................................................................... 120
`b.
`Establishing a Data Link ............................................... 121
`c.
`Code Segment Receiving a Spoken Request ................ 121
`d.
`Devices Comprising a Portable Remote Control ......... 122
`e.
`Rendering an Interpretation of the Spoken Request ..... 122
`f.
`Constructing a Navigation Query ................................. 122
`g.
`Utilizing the Navigation Query .................................... 123
`h.
`Transmitting the Electronic Data Source ..................... 123
`10. Claim 11 Is Unpatentable ........................................................ 124
`Petitioner Microsoft Corporation - Ex.1002, p. iii
`
`7.
`8.
`9.
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`11. Claim 12 Is Unpatentable ........................................................ 126
`12. Claim 13 Is Unpatentable ........................................................ 126
`a.
`Preamble ....................................................................... 126
`b.
`Refining the Navigation Query..................................... 127
`c.
`Selecting a Portion of the Data Source ......................... 127
`13. Claim 14 Is Unpatentable ........................................................ 127
`14. Claim 15 Is Unpatentable ........................................................ 128
`15. Claim 16 Is Unpatentable ........................................................ 128
`16. Claim 17 Is Unpatentable ........................................................ 128
`17. Claim 18 Is Unpatentable ........................................................ 129
`18. Claim 19 Is Unpatentable ........................................................ 129
`a.
`Preamble ....................................................................... 129
`b. Mobile Information Appliance ..................................... 129
`c.
`Comprising a Portable Remote Control Device ........... 130
`d.
`Spoken Language Processing Logic ............................. 130
`e.
`Query Construction Logic ............................................ 130
`f.
`Navigation Logic .......................................................... 131
`g.
`Electronic Communications Infrastructure................... 132
`19. Claim 20 is Unpatentable ........................................................ 132
`20. Claim 21 is Unpatentable ........................................................ 133
`21. Claim 22 is Unpatentable ........................................................ 133
`a.
`Soliciting Additional Input ........................................... 133
`b.
`Query Refining Logic ................................................... 134
`c.
`Selecting a Portion of the Data Source ......................... 134
`22. Claim 23 is Unpatentable ........................................................ 135
`23. Claim 24 is Unpatentable ........................................................ 135
`24. Claim 25 is Unpatentable ........................................................ 135
`25. Claim 26 is Unpatentable ........................................................ 136
`26. Claim 27 is Unpatentable ........................................................ 136
`
`Petitioner Microsoft Corporation - Ex.1002, p. iv
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`I.
`
`INTRODUCTION
`
`A. Engagement
`I have been retained by counsel for Petitioner as an expert witness in
`1.
`
`the above-captioned proceeding. I have been asked to provide my opinion about
`
`the state of the art of the technology described in U.S. Patent No. 6,757,718 (“the
`
`718 Patent”) (Ex.1001) and on the patentability of the claims of this patent,
`
`particularly in view of Multimodal User Interfaces in the Open Agent Architecture,
`
`by Moran et al. (“Moran”) (Ex.1003), U.S. Patent No. 5,454,106 to Burns et al.
`
`(“Burns”) (Ex.1004), U.S. Patent No. 6,188,985 to Thrift (“Thrift”) (Ex.1005), and
`
`U.S. Patent No. 5,983,073 to Ditzik (“Ditzik”) (Ex.1006).13
`
`2.
`
`B.
`3.
`
`The following is my written testimony on these topics.
`
`Background and Qualifications
`I earned a Bachelor of Science degree in Mathematics from the
`
`Massachusetts Institute of Technology, where I was hired to work in research at
`
`the MIT Artificial Intelligence Laboratory (now the Computer Science and
`
`Artificial Intelligence Lab (CSAIL)). I then earned a “Habilitation à Diriger des
`
`
`
`1 My citations to non-patent publications are to the original page numbers of the
`
`publication, and my citations to U.S. Patents are to the column:line number or
`
`paragraph number of the patents or published patent applications, as applicable.
`
`Petitioner Microsoft Corporation - Ex.1002, p. 1
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`Recherches en Informatique,” a Ph.D. equivalent in Computer Science, from the
`
`University of Paris VI (Sorbonne) Pierre and Marie Curie. I have been a Research
`
`Scientist at MIT in the Media Lab or CSAIL since 1973. At the Media Lab, I led
`
`the Software Agents Group. I have also been a Visiting Professor at the University
`
`of Paris VI and taught a graduate course entitled “Artificial Intelligence and
`
`Human Interface.”
`
`4.
`
`I have served as conference chair or program committee member for
`
`major conferences in my field of specialty: ACM (Association for Computing
`
`Machinery) International Conference on Intelligent User Interfaces (IUI), ACM
`
`Transactions on Interactive Intelligence Systems (ACM TiiS), IFIP Conference on
`
`Human-Computer Interaction (INTERACT), International Conference on
`
`Autonomous Agents (AA), and Autonomous Agents and Multi-Agent Systems
`
`(AAMAS), as well as others.
`
`5.
`
`For IUI in particular, I have been a member of the Program
`
`Committee since 2000 and was the Program Chair for the 2000 and 2008
`
`conferences; I have been appointed to the Oversight Committee; and I received a
`
`Best Paper award in 2003. I served as a panelist at the first IUI, the 1997
`
`International Conference on Intelligent User Interfaces, held on January 6-9, 1997,
`
`in Orlando, Florida. IUI97 was attended by dozens of people. All of the attendees
`
`of IUI97 were active participants in the area of user interfaces.
`
`Petitioner Microsoft Corporation - Ex.1002, p. 2
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`6.
`
`I have also served on the editorial board of several journals, including
`
`the Journal of Autonomous Agents and Multi-Agent Systems. And I was elected
`
`to the Executive Council of the AAAI (the Board of Directors of the Association
`
`for the Advancement of Artificial Intelligence) for a 2012-2015 term.
`
`7.
`
`Based at least on my education and experience, I consider myself an
`
`expert in computer science with a focus on intelligent user interfaces for
`
`applications in various domains (for computer systems), including multimodal
`
`input and data information retrieval. My qualifications and experience are set forth
`
`in more detail in my curriculum vitae.
`
`C. Compensation
`I am being compensated at a rate of $400 per hour for my study and
`8.
`
`testimony in this matter. I am also being reimbursed for reasonable and customary
`
`expenses associated with my work and testimony in this investigation. My
`
`compensation is not contingent on the outcome of this matter or the specifics of my
`
`testimony.
`
`Information Considered
`D.
`9. My expert opinions are based on my years of education, research and
`
`experience, as well as my investigation and study of relevant materials. In forming
`
`my opinions, I have considered the materials I identify in this report and those
`
`listed in the Petition’s List of Evidence and Exhibits (Attachment B).
`
`Petitioner Microsoft Corporation - Ex.1002, p. 3
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`10.
`
`I may rely upon these materials and/or additional materials to respond
`
`to arguments raised by the Patent Owner. I may also consider additional
`
`documents and information in forming any necessary opinions, including
`
`documents that may not yet have been provided to me.
`
`11. My analysis of the materials produced in this investigation is ongoing
`
`and I will continue to review any new material as it is provided. This report
`
`represents only those opinions I have formed to date. I reserve the right to revise,
`
`supplement, and/or amend my opinions stated herein based on new information
`
`and on my continuing analysis of the materials already provided.
`
`II. LEGAL STANDARDS FOR PATENTABILITY
`
`12.
`
`In expressing my opinions and considering the subject matter of the
`
`claims of the 718 Patent, I am relying upon certain basic legal principles that have
`
`been explained to me.
`
`13. First, I understand that for an invention claimed in a patent to be
`
`found patentable, it must be, among other things, new and not obvious from what
`
`was known before the invention was made.
`
`14.
`
`I understand the information that is used to evaluate whether an
`
`invention is new and not obvious is generally referred to as “prior art” and
`
`generally includes patents and printed publications (e.g., books, journal
`
`Petitioner Microsoft Corporation - Ex.1002, p. 4
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`publications, articles on websites, product manuals, etc.).
`
`15.
`
`I understand that in this proceeding Petitioner has the burden of
`
`proving that the claims of the 718 Patent are anticipated by or obvious from the
`
`prior art by a preponderance of the evidence. I understand that “a preponderance
`
`of the evidence” is evidence sufficient to show that a fact is more likely true than it
`
`is not.
`
`16.
`
`I understand that in this proceeding, the information that may be
`
`evaluated is limited to patents and printed publications. My analysis below
`
`compares the claims to patents and printed publications that are prior art to the
`
`claims.
`
`17.
`
`I understand that there are two ways in which prior art may render a
`
`patent claim unpatentable. First, the prior art can be shown to “anticipate” the
`
`claim. Second, the prior art can be shown to have made the claim “obvious” to a
`
`person of ordinary skill in the art. My understanding of the two legal standards is
`
`set forth below.
`
`A. Anticipation
`I understand that the following standards govern the determination of
`18.
`
`whether a patent claim is “anticipated” by the prior art.
`
`19.
`
`I have applied these standards in my evaluation of whether the claims
`
`Petitioner Microsoft Corporation - Ex.1002, p. 5
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`of the 718 Patent would have been anticipated by the prior art.
`
`20.
`
`I understand that the “prior art” includes patents and printed
`
`publications that existed before the earliest filing date (the “effective filing date”)
`
`of the claim in the patent. I also understand that a patent will be prior art if it was
`
`filed before the effective filing date of the claimed invention, while a printed
`
`publication will be prior art if it was publicly available before that date.
`
`21.
`
`I understand that, for a patent claim to be “anticipated” by the prior
`
`art, each and every requirement of the claim must be found, expressly or
`
`inherently, in a single prior art reference as recited in the claim. I understand that
`
`claim limitations that are not expressly described in a prior art reference may still
`
`be there if they are “inherent” to the thing or process being described in the prior
`
`art. For example, an indication in a prior art reference that a particular process
`
`complies with a published standard would indicate that the process must inherently
`
`perform certain steps or use certain data structures that are necessary to comply
`
`with the published standard.
`
`22.
`
`I understand that if a reference incorporates other documents by
`
`reference, the incorporating reference and the incorporated reference(s) should be
`
`treated as a single prior art reference for purposes of analyzing anticipation.
`
`23.
`
`I understand that it is acceptable to consider evidence other than the
`
`Petitioner Microsoft Corporation - Ex.1002, p. 6
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`information in a particular prior art document to determine if a feature is
`
`necessarily present in or inherently described by that reference.
`
`B. Obviousness
`I understand that a claimed invention is not patentable if it would have
`24.
`
`been obvious to a person of ordinary skill in the field of the invention at the time
`
`the invention was made.
`
`25.
`
`I understand that the obviousness standard is defined in the patent
`
`statute (35 U.S.C. § 103(a)) as follows:
`
`A patent may not be obtained though the invention is not identically
`disclosed or described as set forth in section 102 of this title, if the
`differences between the subject matter sought to be patented and the
`prior art are such that the subject matter as a whole would have been
`obvious at the time the invention was made to a person having
`ordinary skill in the art to which said subject matter pertains.
`Patentability shall not be negated by the manner in which the
`invention was made.
`
`26.
`
`I understand that the following standards govern the determination of
`
`whether a claim in a patent is obvious. I have applied these standards in my
`
`evaluation of whether the asserted claims of the 718 Patent would have been
`
`considered obvious in June 2000.
`
`27.
`
`I understand that to find a claim in a patent obvious, one must make
`
`Petitioner Microsoft Corporation - Ex.1002, p. 7
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`certain findings regarding the claimed invention and the prior art. Specifically, I
`
`understand that the obviousness question requires consideration of four factors
`
`(although not necessarily in the following order):
`
` The scope and content of the prior art;
` The differences between the prior art and the claims at issue;
` The knowledge of a person of ordinary skill in the pertinent art; and
` Whatever objective factors indicating obviousness or non-obviousness
`may be present in any particular case.
`In addition, I understand that the obviousness inquiry should not be
`
`28.
`
`done in hindsight, but must be done using the perspective of a person of ordinary
`
`skill in the relevant art as of the effective filing date of the patent claim.
`
`29.
`
`I understand the objective factors indicating obviousness or non-
`
`obviousness may include: commercial success of products covered by the patent
`
`claims; a long-felt need for the invention; failed attempts by others to make the
`
`invention; copying of the invention by others in the field; unexpected results
`
`achieved by the invention; praise of the invention by those in the field; the taking
`
`of licenses under the patent by others; expressions of surprise by experts and those
`
`skilled in the art at the making of the invention; and the patentee proceeded
`
`contrary to the accepted wisdom of the prior art. I also understand that any of this
`
`evidence must be specifically connected to the invention rather than being
`
`associated with the prior art or with marketing or other efforts to promote an
`
`Petitioner Microsoft Corporation - Ex.1002, p. 8
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`invention. I am not presently aware of any evidence of “objective factors”
`
`suggesting the claimed methods are not obvious, and reserve my right to address
`
`any such evidence if it is identified in the future.
`
`30.
`
`I understand the combination of familiar elements according to known
`
`methods is likely to be obvious when it does no more than yield predictable results.
`
`I also understand that an example of a solution in one field of endeavor may make
`
`that solution obvious in another related field. I also understand that market
`
`demands or design considerations may prompt variations of a prior art system or
`
`process, either in the same field or a different one, and that these variations will
`
`ordinarily be considered obvious variations of what has been described in the prior
`
`art.
`
`31.
`
`I also understand that if a person of ordinary skill can implement a
`
`predictable variation, that variation would have been considered obvious. I
`
`understand that for similar reasons, if a technique has been used to improve one
`
`device, and a person of ordinary skill in the art would recognize that it would
`
`improve similar devices in the same way, using that technique to improve the other
`
`device would have been obvious unless its actual application yields unexpected
`
`results or challenges in implementation.
`
`32.
`
`I understand that the obviousness analysis need not seek out precise
`
`teachings directed to the specific subject matter of the challenged claim, but
`
`Petitioner Microsoft Corporation - Ex.1002, p. 9
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`instead can take account of the “ordinary innovation” and experimentation that
`
`does no more than yield predictable results, which are inferences and creative steps
`
`that a person of ordinary skill in the art would employ.
`
`33.
`
`I understand that sometimes it will be necessary to look to interrelated
`
`teachings of multiple patents; the effects of demands known to the design
`
`community or present in the marketplace; and the background knowledge
`
`possessed by a person having ordinary skill in the art. I understand that all these
`
`issues may be considered to determine whether there was an apparent reason to
`
`combine the known elements in the fashion claimed by the patent at issue.
`
`34.
`
`I understand that the obviousness analysis cannot be confined by a
`
`formalistic conception of the words “teaching, suggestion, and motivation.” I
`
`understand that in 2007, the Supreme Court issued its decision in KSR Int’l Co. v.
`
`Teleflex, Inc. where the Court rejected the previous requirement of a “teaching,
`
`suggestion, or motivation to combine” known elements of prior art for purposes of
`
`an obviousness analysis as a precondition for finding obviousness. It is my
`
`understanding that KSR confirms that any motivation that would have been known
`
`to a person of ordinary skill in the art or derived from the nature of the problem to
`
`be solved, is sufficient to explain why references would have been combined.
`
`35.
`
`I understand that a person of ordinary skill attempting to solve a
`
`problem will not be led only to those elements of prior art designed to solve the
`
`Petitioner Microsoft Corporation - Ex.1002, p. 10
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`same problem. I understand that under the KSR standard, steps suggested by
`
`common sense are important and should be considered if supported by reasoned
`
`analysis and evidentiary support. Common sense teaches that familiar items may
`
`have obvious uses beyond the particular application being described in a reference,
`
`that if something can be done once it is obvious to do it multiple times, and in
`
`many cases, a person of ordinary skill will be able to fit the teachings of multiple
`
`patents together like pieces of a puzzle. As such, the prior art considered can be
`
`directed to any need or problem known in the field of endeavor as of the priority
`
`date of the 718 Patent and can provide a reason for combining the elements of the
`
`prior art in the manner claimed. In other words, the prior art does not need to be
`
`directed towards solving the same problem that is addressed in the patent. Further,
`
`the individual prior art references themselves need not all be directed towards
`
`solving the same problem.
`
`36.
`
`I understand that an invention that might be considered an obvious
`
`variation or modification of the prior art may be considered non-obvious if one or
`
`more prior art references discourages or lead away from the line of inquiry
`
`disclosed in the reference(s). A reference does not “teach away” from an invention
`
`simply because the reference suggests that another embodiment of the invention is
`
`better or preferred. My understanding of the doctrine of teaching away requires a
`
`clear indication that the combination should not be attempted (e.g., because it
`
`Petitioner Microsoft Corporation - Ex.1002, p. 11
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`would not work or explicit statements saying the combination should not be made).
`
`37.
`
`I understand that a person of ordinary skill is also a person of ordinary
`
`creativity.
`
`38.
`
`I further understand that in many fields, it may be that there is little
`
`discussion of obvious techniques or combination, and it often may be the case that
`
`market demand, rather than scientific literature or knowledge, will drive design
`
`trends. When there is such a design need or market pressure to solve a problem
`
`and there are a finite number of identified, predictable solutions, a person of
`
`ordinary skill has good reason to pursue the known options within their technical
`
`grasp. If this leads to the anticipated success, it is likely the product not of
`
`innovation but of ordinary skill and common sense. In that instance the fact that a
`
`combination was obvious to try might show that it was obvious. The fact that a
`
`particular combination of prior art elements was “obvious to try” may indicate that
`
`the combination was obvious even if no one attempted the combination. If the
`
`combination was obvious to try (regardless of whether it was actually tried) or
`
`leads to anticipated success, then it is likely the result of ordinary skill and
`
`common sense rather than innovation.
`
`III. THE 718 PATENT
`
`A. Effective Filing Date of the 718 Patent
`39. The 718 Patent was filed on June 30, 2000 as Application
`
`Petitioner Microsoft Corporation - Ex.1002, p. 12
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`No. 09/608,872, claiming priority to two utility applications and three provisional
`
`applications. The 718 Patent claims to be a continuation of Application
`
`No. 09/524,095, filed on March 13, 2000, which claims to be a continuation-in-part
`
`of Application No. 09/225,198, filed on January 5, 1999. The cited provisional
`
`applications were filed on March 17, 1999. I am not offering an opinion on
`
`whether priority should be awarded, but the prior art I analyze herein predates the
`
`earliest effective filing date of January 5, 1999, in any event. I have used this
`
`priority date to underscore my analysis as to what a person of ordinary skill in the
`
`art at the time would have known and understood about the 718 Patent, its claims,
`
`and the prior art.
`
`B. Overview of the 718 Patent
`40. The 718 Patent is entitled “MOBILE NAVIGATION OF
`
`NETWORK-BASED ELECTRONIC INFORMATION USING SPOKEN
`
`INPUT.” Ex.1001, Face. The 718 Patent generally describes a system and method
`
`for processing a spoken query input using a platform of application agents. Id. at
`
`Abstract. The query is used to search and retrieve information from electronic data
`
`sources. Id.
`
`41. The 718 Patent is purportedly and specifically directed to addressing
`
`what the patent describes as a purported need for a “methodology and apparatus for
`
`rapidly constructing a voice-driven front-end atop an existing, non-voice data
`
`Petitioner Microsoft Corporation - Ex.1002, p. 13
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`navigation system, whereby users can interact by means of intuitive natural
`
`language input not strictly conforming to the step-by-step browsing architecture of
`
`the existing navigation system, and wherein any errors or ambiguities in user input
`
`are rapidly and conveniently resolved.” Id. at 2:19-26. The Patent explains that a
`
`purported benefit of “navigation via spoken natural language” is that it “allows
`
`relatively naive users to navigate and access desired data by means of natural
`
`language input.” Id. at 1:30-32. The Patent purportedly attempts to account for
`
`errors or ambiguities in interpretations of the spoken input, in order to construct a
`
`“complete, valid navigational template.” Id. at 2:54-56. The Patent explains the
`
`purported significance of its spoken language navigation:
`
`For example, the front-end should not require learning a highly
`specialized command language or format. More fundamentally, the
`front-end must allow users to speak directly in terms of what the user
`ultimately wants . . . as opposed to speaking in terms of arbitrary
`navigation structures (e.g., hierarchical layers of menus, commands,
`etc.) that are essentially artifacts reflecting constraints of the pre-
`existing text/click navigation system. At the same time, the front-end
`must recognize and accommodate the reality that a stream of naive
`spoken natural language input will, over time, typically present a
`variety of errors and/or ambiguities: e.g., garbled/unrecognized
`words . . . and under-constrained requests . . . . An approach is needed
`for handling and resolving such errors and ambiguities in a rapid,
`user-friendly, non-frustrating manner.
`
`Petitioner Microsoft Corporation - Ex.1002, p. 14
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`Id. at 2:1-18. Accordingly, the 718 Patent states that “[t]he present invention
`
`provides a spoken natural language interface for interrogation of remote electronic
`
`databases and retrieval of desired information.” Id. at 7:12-14.
`
`42. Figure 1a of the 718 Patent, Ex.1001, Fig. 1a, provides a general
`
`overview of the disclosed system:
`
`
`
`43. Figure 1a shows that a “voice input device 102” captures voice input
`
`data. Id. at 3:56–57. The voice input device in a preferred embodiment is a
`
`“portable remote control device with an integrated microphone,” which transmits
`
`Petitioner Microsoft Corporation - Ex.1002, p. 15
`
`

`

`Declaration of Dr. Henry Lieberman Regarding U.S. Patent No. 6,757,718
`
`data to a “communications box 104.” Id. at 3:62–67. The communications box
`
`then retransmits the voice data over a network 106 to a remote server(s) 108(n),
`
`where the voice data is processed and used to construct a navigation query. Id. at
`
`3:66–4:5, 4:11–14. The server is comprised of request processing logic 300,
`
`including a speech recognition engine, natural language parser, query construction
`
`logic, and query refinement logic. Id. at 4:17–20; see also id. at fig. 3. The server
`
`108 is connected to a remote data source 110 that may comprise databases,
`
`websit