COhlPUTiNG PRACTICES
`
`Edgar H. Sibley
`Panel Editor
`
`Based on a review of some actual expert-system projects, guidelines are
`proposed for choosing appropriate applications and managing the
`development process.
`
`EXPERT SYSTEMS: PERILS AUD PROMISE
`
`DANIEL G. BOBROW, SANJAY MITTAL, and MARK J. STEFIK
`
`the expres-
`By virtue of their flamboyant quality,
`sions artificial
`intelligence (Al) and expert systems have
`helped contribute
`to an expanding wave of activity
`and unrealistic expectations about the state of the
`art. Taking a long, hard look at those expectations,
`this article contrasts them with
`the results of some
`actual case studies and proposes both a more realis-
`tic view of the practice of building expert systems
`and some guidelines
`for choosing appropriate appli-
`cations.
`The term expert system refers to computer pro-
`grams that apply substantial knowledge of specific
`areas of expertise
`to the problem-solving process.
`The term expert is intended
`to imply both narrow
`specialization and competence, where success of the
`system is often due to this narrow
`focus. Just as
`human experts have varying
`levels of expertise, so
`too do computer systems. Although,
`in general, ex-
`pert systems have less breadth of scope and flexibil-
`ity than human experts, and for this reason are
`sometimes criticized
`for creating unrealistic expecta-
`tions, we find it more productive
`to ask about the
`level and range of expertise of a given program (i.e.,
`how well does it do on a specific set of tasks), rather
`than struggling with
`the imprecise boundary of what
`constitutes
`“expert.”
`
`01966 ACM OOOl-0782/86/0900-0880
`
`759
`
`In some circles, the terms knowledge-based systems
`or knowledge systems are used instead of expert system
`[lo]
`to focus attention on the knowledge
`the systems
`carry, rather than the question of whether or not
`such knowledge constitutes expertise. These terms
`also imply
`the use of technology
`for explicit
`repre-
`sentation of knowledge. But, once again, the bound-
`ary between explicit and implicit
`representation
`is
`imprecise
`in that knowledge can be represented ex-
`plicitly
`to different degrees and can take different
`forms.
`High-level performance can be achieved without
`explicit
`representation of knowledge as in an auto-
`pilot [%I; one might even ask whether a C-Compiler
`or a payroll program constitutes an expert system.
`Clearly
`they both embody knowledge:
`in the one
`case, of a language and computer and, in the other,
`of accounting and taxes. When constructed with
`conventional programming
`techniques, both would
`have a very limited
`range of capabilities. However,
`is also possible to build either as a knowledge-based
`system. One might then ask the AI payroll system
`hypothetical questions, such as the net difference
`taxes if you added two more deductions.
`In their
`usual embodiments, systems like these are generally
`designed with built-in commitments as to how the
`knowledge embedded in them is to be used, that is,
`to have “compiled out” [built
`into the programs) lim-
`
`in
`
`it
`
`000
`
`Communications of the ACM
`
`September 1986 Volume 29 Number 9
`
`IBM-1015
`Page 1 of 15
`
`

`
`Computing Practices
`
`behavior. To use this same knowl-
`ited input/output
`edge to generate tax advice, in the case of the pay-
`roll system, would require recoding the system for
`that purpose. For this reason, the term knowledge-
`bused is generally
`reserved for systems that have ex-
`plicit knowledge bases and some flexibility
`in the
`use of that knowledge.
`In this article, we examine primarily knowledge-
`based expert systems: systems that achieve expert-
`level performance using explicit
`representations of
`knowledge. For purposes of brevity, we will
`refer to
`them simply as expert systems.
`
`TO BUILDING
`APPROACHES
`EXPERT SYSTEMS
`Depending on the extent and depth of the explicit
`representation of knowledge, we can delineate
`three
`different approaches to expert-system development:
`the low road, the middle road, and the high road [6].
`The low road involves direct symbolic program-
`ming, usually
`in the Lisp programming
`language. It
`takes advantage of new low-cost AI machines with
`flexible programming environments enhanced by
`user interfaces that exploit window systems on large
`displays. These environments support a style of pro-
`gram development called “exploratory program-
`ming” [28] in which
`there is incremental, parallel
`development of program specification and imple-
`mentation-an
`appropriate style for applications
`where the primary concern is efficiency and the re-
`quired knowledge base is small and does not need to
`be changed very frequently.
`The low road was used for the early expert sys-
`tems (e.g., Dendral
`[19]), which combined AI tech-
`niques for heuristic search with Lisp capabilities
`for
`symbolic manipulation. Dendral generated and
`tested hypotheses about chemical structures and
`spectroscopic data. It needed to be efficient because
`the search space of possible solutions
`is very large;
`moreover, programming
`it directly
`into procedures
`was practical since the knowledge used for inter-
`preting spectral data is fairly static.
`The high road, on the other hand, involves build-
`ing a system that contains explicit
`representation of
`fairly complete knowledge of some subject matter,
`and can use that knowledge
`for more than one pur-
`pose. A system is called “deep” when its knowledge
`represents the principles and theories underlying
`the subject; a consequence of this depth is that such
`systems often require
`long chains of reasoning from
`first principles
`to practical
`results.
`Sophie [ll]
`is a high-road system that performs
`diagnostic reasoning and qualitative simulation, and
`can reason from first principles about how physical
`devices work. For many classes of devices, Sophie
`
`the behavioral states that the devices
`can determine
`will
`traverse for a given set of inputs. When the
`actual output of a device does not agree with
`the
`predicted output,
`the program uses the same funda-
`mental knowledge
`to generate hypotheses about
`which parts of the device may be broken.
`The ultimate goal of high-road systems is a knowl-
`edge processing capability
`that is general enough to
`span domains-an
`ability
`to apply general principles
`and commonsense reasoning to the articulation and
`testing of general facts and principles, and to experi-
`ment with processing and reasoning about these
`facts. Given current computers and compilation
`techniques, high-road systems are usually
`too slow
`for real-world
`(large-scale) applications, since they
`take only very small steps toward the solution of big
`problems; currently
`they are used only for research.
`Middle-road systems, not surprisingly,
`fit squarely
`between these two extremes. They also involve ex-
`plicit representation of knowledge, but though some
`direct programming may be used, most of the inter-
`esting behavior of the system is governed by knowl-
`edge that is articulated by experts and represented
`explicitly
`in a knowledge base. Canned problem-
`solving tactics rather than first principles are most
`often the rule.
`Knowledge engineers working with experts often
`use knowledge-engineering
`tools and hybrid
`lan-
`guages [15, 171 for this purpose. The technology
`fa-
`cilitates
`the representation,
`reorganization, and de-
`bugging of programs expressed as interacting pieces
`of knowledge. A key characteristic of middle-road
`systems is that they are sharply
`focused on a single
`task and incorporate knowledge specialized
`for the
`task, but the explicit
`representations often do not
`specify the limitations of that knowledge.
`A well-known
`example of a middle-road expert
`system is the Mycin system for medical diagnosis
`and prescription
`[7], Mycin contains rules that asso-
`ciate symptoms with diseases, for which
`it can also
`evaluate and prescribe treatment. Mycin,
`like most
`expert systems, is called a shallow system because
`most of its reasoning chains are short. It has no phys-
`iological model of disease or health, no model of
`how diseases cause symptoms, and no model of how
`treatment can help cure diseases. For most applica-
`tions, the middle road is the most effective approach
`now available
`for building expert systems.
`the
`Expert systems are no panacea for achieving
`impossible or even the very difficult. A mere incli-
`nation to have an expert system is no guarantee that
`one can be built.
`Identifying a need-“We
`need an
`expert who can make money on the stock market”
`or “It would be great to have a program to transform
`a functional circuit specification
`to an optimal
`inte-
`
`September 1986 Volume 29 Number 9
`
`Communications of the ACM
`
`aa1
`
`IBM-1015
`Page 2 of 15
`
`

`
`Computing Practices
`
`not suffice to determine
`layout”-does
`grated circuit
`task for an expert system.
`an appropriate
`issues
`Instead, there are a number of fundamental
`and requirements
`that must be considered.
`In the
`balance of this article, we present three successful
`projects as case studies. We then generalize
`from
`these examples to present guidelines
`for choosing
`appropriate applications and developing successful
`systems.
`
`CASE STUDIES
`The Rl System for Configuring Vaxen
`In 1978, the Digital Equipment Corporation and a
`group headed by John McDermott at Carnegie-
`Mellon University started a joint project to build an
`expert system that could aid in the configuration of
`VAX computers. As McDermott
`[Zl] describes it, the
`rule-based configurer Rl received a customer’s pur-
`chase order and then determined what if any substi-
`tutions and additions were needed to make the or-
`der consistent and complete. It produced a number
`of diagrams showing
`the spatial and logical relation-
`ships among the 90 or so components
`that typically
`constitute a VAX-11 computer system.
`the con-
`Rl was the fourth attempt at automating
`struction of complete configuration descriptions
`for
`DEC computers
`[2O]. At least part of the reason the
`three earlier attempts failed was that their imple-
`mentation
`technology did not allow knowledge
`to be
`expressed explicitly. They were built with stan-
`dard programming
`technology, which made it hard
`to understand
`the interactions between pieces of the
`system and to change and augment the system as
`additional knowledge was gained.
`
`Knowledge Representation.
`is a rule-based system
`Rl
`that uses the OPS-5 programming
`language 1141 built
`in Lisp. In Rl, a typical rule (rendered here in
`pseudo-English) might look like this:
`
`active
`current
`The most
`IF:
`putting
`the unibus modules
`backplane
`in
`some box
`
`context
`in
`the
`
`is
`
`AND it
`modu.le
`
`which
`has been determined
`to
`try
`to put
`in a backplane
`
`AND
`
`. . .
`
`THEN: Enter
`panel
`space
`
`the
`for
`
`of verifying
`context
`a multiplexer
`
`to be
`The rule consists of a series of conditions
`tested (the IF parts), followed by actions to be taken
`provided
`the IF part is true. The conditions
`test
`such things as the state of the problem-solving pro-
`cess, availability
`of parts, and connectivity
`of the
`configuration.
`
`tested in early 1980, contained
`The first prototype,
`roughly 750 such rules, which
`interacted with a
`database describing some 4000 parts that could be
`used in configuring a VAX system. This prototype
`was far from perfect, and over 50 person-years of
`effort were invested in developing Rl
`into the cur-
`rent DEC product known as XCON [2]. During
`that
`time, the system grew from 700 rules to about 3500.
`The completed system (which
`is still being changed
`and updated today) configures not only the VAX sys-
`tems, which were originally chosen for the configu-
`ration problem, but the much more complicated
`PDP-11 family.
`
`In light of its unchallenged success to-
`Evolution.
`day, it is perhaps hard to believe that Rl was almost
`canceled three times. The uncertainty among people
`within DEC was based on a misconception about the
`nature of expert-system
`technology. Basically,
`the
`term expert systetn encouraged very high expecta-
`tions, while
`the Rl prototype continued
`to make
`mistakes: It was not an “instant expert.” However,
`such an expectation
`is not appropriate
`for any sys-
`tem, or even a person new at a job. No matter how
`well trained a new employee is, new knowledge
`must be acquired and older knowledge
`restructured,
`and this takes time.
`Ensuring proper development of an expert system
`after the prototype stage requires building
`the pro-
`cess in the organization
`that is to use it. In the case
`of Rl, a group monitored
`the problems, which were
`categorized as follows:
`
`l
`
`l
`
`in the database,
`incorrect component description
`incorrect configuration
`knowledge,
`incomplete configuration
`knowledge,
`l an error in the data input
`to the system, and
`l a confusion by the person reporting
`the problem
`(essentially, a nonproblem).
`
`l
`
`For each of the first four problems, a specific person
`was assigned responsibility
`for obtaining
`the appro-
`priate data and modifying
`the system. This might
`mean redesigning a set of rules when required by
`interactions with previous rules, or rewriting
`in-
`structions, or changing
`the user interface
`in response
`to errors in the input data.
`Of the 2500 rules added to the original system,
`somewhat less than 40 percent were used to make
`major refinements
`in the knowledge. Another
`10 percent made minor refinements
`to improve out-
`put, while about 35 percent provided additional
`functionality
`to deal with new system types. A large
`core of configuration
`knowledge
`in the original sys-
`tem was specific to the VAX 780, while
`the current
`system (XCON) deals as well with
`the much more
`family. Finally, about 15 per-
`complicated PDP-11
`
`082
`
`Communications of the ACM
`
`September 1986 Volume 29 Number 9
`
`IBM-1015
`Page 3 of 15
`
`

`
`Comprrtillg Practires
`
`cent of the new rules were added to extend the
`definition of the configuration
`task to include
`things
`like laying out cables or floor layout of racks. (Lay-
`out was a new capability added to the system.) The
`moral of this story is, of course, that expert systems
`must be extended and changed over time-a
`task that is
`made simpler by using explicit knowledge program-
`ming.
`The RI team also had to define the delivery vehi-
`cle for the Rl system. Since the original OPS-5 im-
`plementation
`in Lisp was considered
`too slow to be
`used for production
`jobs, DEC reimplemented OPS-5
`in Bliss, making it much faster. Nonetheless,
`the Lisp
`environment still provided much better tools for de-
`bugging, so that now changes to the system are re-
`portedly done in the Lisp environment and then
`transferred
`to the Bliss production system. Today,
`with
`the high-performance,
`low-cost, personal Lisp
`machines currently available, such reengineering of
`the delivery engine might not have been necessary.
`The current Rl/XCON
`system configures over
`97 percent of the orders received for VAX-11 sys-
`tems. It has significantly
`reduced the time needed to
`configure an order and has produced more accurate
`configurations, both of which benefits have brought
`significant monetary advantage since customers pay
`for machines after they start working.
`Incorrectly
`configured orders cause significant delay if it is nec-
`essary to wait for additional parts. A further unex-
`pected benefit for most orders is that configurations
`are created in a much more consistent style using Rl
`than when configured by the group technical edi-
`tors. This has allowed
`fine-tuning
`the factory to han-
`dle a more stereotypical order.
`In terms of the human side of this equation,
`Rl/XCON
`has meant that the technical editors em-
`ployed at DEC have a higher productivity. Given
`this tool to amplify
`their capabilities,
`they not only
`produce more, but are also more satisfied because
`they now concentrate on the interesting problems
`rather than the dreary, day-to-day repetitive
`tasks
`that consumed their time before. Rl/XCON
`repre-
`sents a real success story for expert-system
`technol-
`ogy, for the company, for its employees, and for cus-
`tomers.
`the
`the system? Although
`What about maintaining
`encoding of XCON’s knowledge
`in production
`rules
`was supposed to make it easier to manage the
`is
`knowledge base, the cost of XCON’s maintenance
`still high, but very worthwhile
`given the economic
`benefits derived from it. Furthermore,
`the notion of
`declarative knowledge
`representations
`is open-
`ended, in the sense that it will accommodate
`changes to the knowledge base. Current research at
`Carnegie-Mellon
`is aimed at reducing maintenance
`
`the creation of higher
`through
`costs still further
`level “shells”
`that actively support the process of
`knowledge acquisition and testing. New prototype
`versions of the configuration
`system are now being
`reengineered using these shells.
`
`The Pride System for Mechanical Design
`Pride [25] is an expert system developed at Xerox to
`assist engineers designing paper transports inside
`copying and duplicating machines. It is a joint effort
`begun in early 1984 between the Xerox Palo Alto
`Research Center (PARC) and the Xerox Reprograph-
`ics Business Group (RBG). The first prototype was
`ready for testing in early 1985. Since then the sys-
`tem has been tested extensively on past and current
`design cases, and a support group is being created
`within
`the RBG to gradually
`take over further devel-
`opment and testing.
`
`Problem Domain. The first stage of the Pride project
`consisted of analyzing
`the problem-solving behavior
`of a group of experts working on the design of paper
`transports [z]. From this exercise, it became clear
`that knowledge was distributed among many differ-
`ent experts, and that these experts showed varying
`degrees of specialization based on the technology
`used (e.g., pinch roll versus belt versus vacuum
`transport
`technologies).
`Other kinds of specializations were based on vari-
`ations in design specifications
`(e.g., precisely regis-
`tering paper or building a recirculating document
`handler). Depending on the volume of copies to be
`produced per month, there are differences based on
`trade-offs and cost considerations. Other specializa-
`tions relate to different parts of the design task: ma-
`terial selection, jam clearance, cost analysis, or de-
`sign and analysis. Some engineers generate perfectly
`plausible designs, but are unable to analyze the de-
`signs to determine whether
`the designs meet the
`requirements, or predict performance problems;
`others primarily analyze other engineers’ designs.
`Finally,
`there is a difference
`in the depth of
`knowledge: Some engineers know only standard ap-
`proaches, whereas others can provide reasoned argu-
`ments about the trade-offs involved
`in different de-
`sign decisions. The differences
`in expertise become
`more apparent in nonstandard designs or nonstan-
`dard problems.
`It became apparent to technical managers that the
`existing engineering expertise
`from the engineering
`research division was often not used by the design
`engineers in the applied divisions. Moreover, design
`engineers often did not have adequate access to
`techniques
`for analyzing
`the designs they created,
`even though the techniques were often available
`either in research publications or as computer pro-
`
`September 1986 Volume 29 Number 9
`
`Communications of fhe ACM
`
`883
`
`IBM-1015
`Page 4 of 15
`
`

`
`Computing Practices
`
`two chief purposes of Pride were
`grams. Therefore,
`to facilitate
`the necessary knowledge transfer and pro-
`vide a single framework
`for creating and testing a
`design. Toward
`this latter end, the project has al-
`ways had two consulting domain experts: one in the
`design of paper transports and the other in the anal-
`ysis and simulation of mechanical systems.
`
`Representation of Knowledge i12 Pride. A designer’s as-
`sistant rather than an “automated designer,” Pride
`reflects the reality
`that, at every stage of its develop-
`ment, some design knowledge and criteria will be
`missing from the knowledge base. On the other
`hand, the task of exploring a large design space-
`maintaining
`the dependencies between different
`parts of the design, systematically
`checking con-
`straints and applying analyses, and maintaining al-
`ternate designs-is
`overwhelming
`in its detail. The
`design engineer and the Pride system are expected
`to work as a team: Together, they can explore a
`larger design space in a shorter time than either
`could working alone.
`Pride’s knowledge of design is organized in a ge-
`neric design plan. The plan includes goals-what
`part of the design is being done (e.g., “design the
`paper path”); methods-how
`to make some design
`decision (e.g., “the width of a drive roll is typically
`25 mm and can go up in increments of 1”); con-
`straints-checks
`and requirements on the decisions
`made at a certain design goal (e.g., “the width of the
`idler must be 2 1.2 times the width of the driver”);
`and calculations-derivation
`of values from other de-
`sign parameters. These primitives are represented as
`objects in Loops [4, 331. For example, a simple con-
`straint might be represented as the following:
`
`type
`description
`applywhen
`
`mustsatisfy
`paraconstrained
`
`predicate
`testExpression
`
`advice
`
`SingleConstraint
`“Idler width >1.2 times driver width”
`(>(defRollerPair
`idler width) 50 mm)
`;when to apply
`YES
`;could be an optimizer
`(defRollerPair
`idler width)
`;focus on idler width
`GreaterThan
`(TIMES 1.2
`(defRollerPair driver width))
`(Increase(defRollerPair
`idler width))
`(Decrease(defRollerPair driver width)
`
`This simplified presentation of a single constraint
`indicates how the object groups different aspects of
`the constraint and makes them accessible. In addi-
`tion to the textual comment and the parsed expres-
`sion for the constraint,
`it provides advice about what
`to do if the constraint
`is not satisfied (e.g., try in-
`creasing the idler width,
`if possible, or if not, then
`
`try decreasing the driver width). The first advice
`might not be possible if rollers of that size are not
`available, or if a larger width would cause parts to
`rub together. The system applies advice in explora-
`tion of a large design space using a set of goal objects
`that represent a design plan for the transport. When
`constraints are violated and the advice does not lead
`to an acceptable modification of a design parameter,
`the system gets additional guidance from the
`designer.
`
`Testing and Current Status. To operate successfully
`as a design assistant, Pride must support the process
`of exploration by an engineer: It must be able to
`both generate design alternatives and to verify
`that
`designs satisfy constraints. The system’s ability
`to
`check constraints systematically
`is especially valua-
`ble when the design space is complex and when
`designs are optimized by different groups of
`engineers.
`ability has been tested
`Pride’s design verification
`on designs from ongoing projects where the system
`applies a set of analyses and constraints
`in the
`knowledge base to designs created by engineers. In
`one test, Pride found flaws in a design, helped pin-
`point the source of the problems, and proposed a
`new design that avoided those problems. The knowl-
`edge base has now been successfully
`tested on many
`paper transport problems from past and current
`copier projects.
`Although
`the current knowledge base contains
`only some of the copier technologies used by engi-
`neers, it has most of the knowledge
`for designing
`paper transports using those technologies. A devel-
`opment group has been created to acquire the
`knowledge
`for other technologies during
`the coming
`months. It is expected that Pride will begin to see
`operational use in 1986.
`
`The Dipmeter Advisor
`The history of the dipmeter advisor for petroleum
`exploration
`[31] is similar
`to that of the VAX con-
`figurer Rl. A feasibility study in the Schlumberger
`research laboratories was started in 1978, and the
`initial prototype was ready for testing in 1980.
`It was
`built on the DEC 2020
`in 245 kilobytes of Lisp code
`interacting with another program of 450 kilobytes
`written
`in VAX Fortran. This initial
`implementation
`proved too slow and unwieldy-a
`system that slows
`the expert doing his or her job has only negative
`value. A reimplementation was completed
`in 1983
`using Interlisp-D on a Xerox 1100, a personal com-
`puter whose high-speed display allowed develop-
`ment of a user interface
`that was much more at-
`tuned to rapid interaction with
`the expert.
`
`884
`
`Communications of the ACM
`
`September 1986 Volume 29 Number 9
`
`IBM-1015
`Page 5 of 15
`
`

`
`Conlputiq Practices
`
`Problem Domain. To determine as early as possible
`whether an oil well will contain oil or be a dry hole,
`oil explorers
`lower specialized
`logging instruments
`into the borehole; these instruments provide
`infor-
`mation about the geology of the subsurface forma-
`tions being pierced.
`As consultants
`to the oil industry, Schlumberger
`provides instrumentation
`and data analysis (e.g., of
`the data logs taken at different depths). One such
`instrument,
`the “dipmeter,” measures the inclina-
`tion or tilt of the rock strata penetrated by a bore-
`hole. The dipmeter advisor developed by Schlum-
`berger mimics the analysis of an expert using these
`data: Analysis programs process the signal, and iden-
`tification and classification
`rules use this analysis in
`an expert-system environment.
`
`interpreta-
`Knowledge Representation. The dipmeter
`tion problem requires alternating stages of signal
`analysis and expert interpretation. After the signal
`analysis creates objects describing parts of the bore-
`hole, rules similar
`in structure
`to those for Rl are
`used to extend the description;
`the rules use both
`the results of signal analysis and other information,
`such as the geology of the undersea formation.
`
`from
`User Interface. Well-log analysts look at output
`strip recorders showing
`the data progression for var-
`ious depths in the well. The interface
`to the dipme-
`ter system preserves the visual
`features of such dis-
`plays, augmented by summary
`information, as
`shown in Figure 1. The multiple columns provide an
`overview of the data that is comparable, and proba-
`
`through a stratigraphic
`the user midway
`This figure shows
`analysis. On the far right side is a log of the dipmeter datti.
`The black “elevator box”
`to the left of that column shows
`the
`
`to the left. Radical
`two columns
`portion of the log expanded
`shifts in focus are obtained by moving
`the elevator. Other
`logs and summary
`information are shown on the left.
`
`FIGURE 1. Dipmeter lntetface
`
`September 1986 Volume 29 Number 9
`
`Communications of the ACM
`
`005
`
`IBM-1015
`Page 6 of 15
`
`

`
`Compufiq Practices
`
`bly superior to, the long rolls of paper tacked up on
`the walls of experts’ offices. A critical
`feature of the
`interface
`is the ease in moving from one portion of
`the data to another with a smooth scrolling opera-
`tion: In fact, Schlumberger claims they can scroll the
`paper on the screen at 57 miles an hour-an
`un-
`usual metric for a display system.
`Much care was devoted to the user interface, as
`can be seen in the distribution
`of code given in
`is.
`Table I, which shows how important
`the interface
`For a knowledge-based system, one might have ex-
`pected that most of the space would be devoted to
`the explicit knowledge and inference engine re-
`quired
`to manipulate
`it. However,
`together, explicit
`knowledge and inference engine consume only
`about 30 percent of the total memory. Implicit
`knowledge
`for feature detection makes up another
`13 percent, while by far the largest identifiable por-
`tion, 4.2 percent, is devoted to the interface. This is
`not atypical:
`In other systems we know, such as
`Pride, one-third
`to one-half of the code is devoted to
`the user interface.
`
`CRITERIA AND STAGES
`FOR EXPERT SYSTEMS
`The success of expert-system development depends
`to a very great measure on an appropriate selection
`of applications
`[27]. In this section, we consider
`some of the most important criteria
`for choosing pro-
`jects and outline
`the major developmental stages
`once a project is under way.
`
`Criteria
`for Problem Selection
`Value of Solving the Problem. Choose the application
`carefully. The problem that is being approached
`must be worth solving. One way of confirming
`this is
`to determine whether management
`is willing
`to
`commit the necessary human and material re-
`to
`sources. Since it may take tens of person-years
`make a system real, as in the Rl example, the value
`of the solution must be substantial. We are told that
`Rl/XCON
`now repays its development cost every
`few months.
`
`to the same
`Alternative Solutions. Non-AI solutions
`problem are a real possibility. A vivid
`illustration
`of
`the value of considering alternative solutions comes
`from the Darn project at Xerox [241-a prototype
`
`TABLE I. Distribution of Memory in the Dipmeter Advisor
`
`inference engine
`Knowledge base
`Feature detection
`User Interface
`Support environment
`
`22
`x3%
`42%
`1fP-h
`
`to aid in the diagnosis and repair
`expert system built
`of a computer disk subsystem. The disk diagnosis
`and repair process was a complicated one, requiring
`a large number of tests to take into account the
`many possible failure modes, including electronic
`failures of several controller boards. According
`to
`technicians’
`records, diagnosing and repairing disk
`problems for this unit were consuming 30-50 per-
`cent of the resources of the repair group.
`After working
`three months on the prototype, ex-
`ternal factors caused a three-month hiatus in the
`project. When it was resumed, the Darn system pro-
`totype was no longer of value because this particular
`disk and controller had been phased out-as a result
`of all the problems-and
`replaced by a new disk
`with a single circuit board. In the case of a suspected
`disk failure,
`the single board was easily replaced; if
`the replacement did not solve the problem, the disk
`itself would be sent back to the manufacturer. This
`new disk strategy had so reduced the time required
`to fix disk problems that aid from an expert system
`was no longer of significant value, making the
`knowledge base specific to the original disk obsolete.
`
`resource in the success-
`Test Cases. Another critical
`ful development of an expert system is a good set of
`test cases by which
`to extract knowledge
`from the
`experts. By watching experts actually solve prob-
`lems, rather than just having them describe how they
`do it, it is possible to understand
`the real process
`that goes on and the actual knowledge
`that is used.
`A suite of test cases can also be used for testing
`implementations
`as they reach different stages in
`development.
`Asking potential users early about test cases also
`provides valuable clues as to the real feasibility and
`value of a proposed system. One does not easily get
`test cases for problems that occur only once every
`six months, and there probably would not be enough
`commonality between cases. Moreover, where solu-
`tion methods are radically different
`for each prob-
`lem, too much knowledge
`is probably required
`in
`the system. For example, after we were asked to
`consider producing an expert system to design de-
`vice controllers and briefly exploring
`the domain, it
`became apparent to us that each device controller
`was unique
`in its requirements, and therefore
`the
`project was rejected. One might say that expert-
`system technology
`is suitable
`for automating
`tasks
`that are fairly routine and mundane, not exotic and
`rare.
`
`There are other rules of thumb
`Task Difficulty.
`should be applied to tasks being considered as
`expert-system domains. A suitable
`task is probably
`one that would
`take an expert an hour or two, not
`counting
`the time spent on mechanical
`tasks like
`
`that
`
`888
`
`Communications of the ACM
`
`September 1986 Volume 29 Number 9
`
`IBM-1015
`Page 7 of 15
`
`

`
`Computing Practices
`
`in forms. Tasks that take
`making sketches or filling
`only a few minutes of expert time can probably be
`solved by simpler
`technology, or may not be worth
`automating at all; tasks that require more than a few
`hours of expertise are probably
`too difficult and un-
`bounded in terms of the knowledge
`they require.
`Unless the longer tasks are iterations of a much
`shorter one, done several times in the same session,
`the longer tasks are generally
`too complex.
`Another warning sign is a