The role of the knowledge engineer Knowledge acquisition

The role of the knowledge engineer. Knowledge acquisition.

Software development: conventional systems and KBS You are probably familiar with a standard model of the software development life cycle. It is likely to be something like this: Feasibility study ® Analysis ® Requirements definition ® Design ® Implementation ® Testing ® Maintenance & review l

Software development: conventional systems and KBS Knowledge-based systems require special approaches to systems analysis, especially to the collection of the data (or rather knowledge) on which the system is based. ¨ We will discuss the ways in which this model needs to be modified to take account of these special features in lecture 8. ¨

Knowledge Engineering ¨ The term "knowledge engineering" is often used to mean the process of ¨ designing ¨ building ¨ installing an expert system or other knowledgebased system. In other words, the whole process of making a KBS, from beginning to end.

Knowledge Engineering ¨ Some authors use the term to mean just the phase in which the knowledge base is built.

Building the knowledge base Five processes can be identified: l 1. Knowledge acquisition l 2. Knowledge analysis & representation l 3. Knowledge validation l 4. Inference design l 5. Explanation and justification l These are not stages that have to follow each other - some of them will run concurrently. l

Knowledge Acquisition Knowledge acquisition is: The process of gathering the knowledge to stock the expert system's knowledge base. l

Knowledge Acquisition This has proved to be the most difficult component of the knowledge engineering process. It's become known as the 'knowledge acquisition bottleneck', and expert system projects are more likely to fail at this stage than any other. l This is the principle reason why expert systems have not become more widespread. l

Knowledge Acquisition l Sources of knowledge: ¨ Documents: textbooks, journal articles, technical reports, records containing case histories, etc. This will almost never be sufficient to provide the knowledge base for a real-world expert system. ¨ The range of problems which a textbook examines and solves is always smaller than the range of problems that a human expert is master of. l

Knowledge Acquisition l Sources of knowledge: n Human experts

Knowledge Elicitation The most important part of knowledge acquisition is knowledge elicitation obtaining knowledge from a human expert (or human experts) for use in an expert system. ¨ Knowledge elicitation is difficult. Hence the knowledge acquisition bottleneck mentioned above. ¨ It is necessary to find out what the expert(s) know, and how they use their knowledge. ¨

Knowledge Elicitation ¨ Expert knowledge includes: ¨ domain-related facts & principles; ¨ problem-solving strategies; ¨ meta-knowledge - for instance, knowledge about when to use a particular piece of knowledge; ¨ explanations and justifications.

Knowledge Elicitation ¨ The knowledge elicitation/analysis task involves ¨ finding at least one expert in the domain who: ¨ is willing to provide his/her knowledge; ¨ has the time to provide his/her knowledge; ¨ is able to provide his/her knowledge. - any or all of these are liable to prove difficult.

Knowledge Elicitation ¨ The knowledge elicitation/analysis task involves ¨ repeated interviews with the expert(s), probably combined with other, non-interview, techniques.

Knowledge Elicitation - the compiled knowledge problem One major obstacle to knowledge elicitation: experts cannot easily describe all they know about their subject. ¨ They do not necessarily have much insight into the methods they use to solve problems. ¨ Their knowledge is "compiled" (like a compiled computer program - fast & efficient, but unreadable). ¨

Knowledge Elicitation - interview techniques ¨ Some of the interview techniques used in knowledge elicitation: ¨ Unstructured interview. A general discussion of the domain, designed to provide a list of topics and concepts. ¨ Structured interview. Concerned with a particular concept within the domain - a particular problem-solving skill or small group of skills.

Knowledge Elicitation - interview techniques ¨ interview techniques : ¨ Problem-solving interview. The DE is provided with a real-life problem, of a kind that they deal with during their working life, and asked to solve it. As they do so, they are required to describe each step, and their reasons for doing what they do. The transcript of their verbal account is called a protocol.

Knowledge Elicitation - interview techniques ¨ interview techniques : ¨ Think-aloud interview. As above, but the DE merely imagines that they are solving the problem presented to them, rather than actually doing it. Once again, they describe the steps involved in solving the problem.

Knowledge Elicitation - interview techniques ¨ interview techniques : ¨ Critical incident analysis. The DE is asked to provide details of cases which were particularly difficult, or of special interest for some other reason. He/she describes how they were solved, and the lessons that were learnt.

Knowledge Elicitation - interview techniques ¨ interview techniques : ¨ Dialogue. The DE interacts with a client, in the way that they would normally do during their normal work routine.

Knowledge Elicitation - interview techniques ¨ interview techniques : ¨ Review. The KE and DE examine the record of an interview session together.

Knowledge Elicitation non-interview techniques ¨ Some of the non-interview techniques used in knowledge elicitation: n Sample lecture preparation. The DE prepares a lecture, and the KE analyses its content.

Knowledge Elicitation non-interview techniques ¨ non-interview techniques: ¨ Concept sorting ("card sort"). The DE is presented with a series of cards, with the names of domain concepts written on them, spread out on a table top, and asked to arrange them into clusters, in such a way that the cards in each cluster have something important in common. Then the DE is asked to name the principles that he/she has used to form these clusters. This process can be repeated to produce a hierarchy of concepts.

Knowledge Elicitation non-interview techniques ¨ non-interview techniques: ¨ Repertory grid (particularly the "laddered grid" technique). ¨ Questionnaires. Especially useful when the knowledge is to be elicited from several different experts.

Knowledge Elicitation - interview techniques It is standard practice to tape-record KE sessions. ¨ For something like a problem-solving interview, one would wish to videotape it as well. ¨ However, KEs should be aware of the costs this involves, in time and money - it can take as much as 15 hours of secretarial time to transcribe and edit a one-hour interview. ¨

Knowledge analysis & representation • Simultaneously with the knowledge acquisition process, a knowledge analysis process takes place. The KE uses the data the transcripts and protocols, etc - from the knowledge acquisition sessions to build a good model of the expertise that the DE is using to solve problems in the domain.

Knowledge analysis & representation • • The raw data (taken from the DE) is converted into intermediate representations. These are structured representations of the knowledge, but not yet the sort of coded knowledge that can be put into the knowledge base. This will improve the knowledge engineer's understanding of the subject;

Knowledge analysis & representation • • • This will probably provide knowledge in a form that can be shown to the DE, for criticism and correction; This provides easily-accessible knowledge for future KEs to work from (knowledge archiving). The intermediate representation is then converted into the knowledge representation formalism which is to be used in the KBS software.

Knowledge validation It is necessary to verify the knowledge against the knowledge source (the expert or document). ¨ It is also necessary to validate the knowledge against known outcomes. ¨ The objective is to produce knowledge of high integrity. ¨

Inference design l It may be necessary to design the software which will comprise the inference engine; or a particular shell may already have been specified.

Explanation and justification l An explanation facility, capable of explaining/justifying any of the reasoning and conclusions that the system produces, needs to be designed and programmed.

Computer-assisted knowledge elicitation Since knowledge engineering skills, and hence knowledge engineers, are rare (see appendix), it would be desirable to automate the job. l i. e. to write an expert system to do knowledge engineering. l

Computer-assisted knowledge elicitation • The state of the art in AI (especially in natural language processing) is not sufficiently advanced to permit fullyautomated knowledge elicitation.

Computer-assisted knowledge elicitation • However, 'knowledge elicitation workbenches', or 'knowledge engineering environments', are commercially available • • e. g. KEE, Kn. Acq. Tools, ETS, KRITON, AQUINAS; their principle use is to simplify the task of converting a protocol into frames, rules, etc. , and inserting these structures into an expert system shell as soon as they are formulated.

Fully computerised knowledge acquisition It might be thought that one could avoid using a domain expert altogether, by building a system that could extract knowledge, given facts about the domain. ¨ This is the approach taken by machine learning systems: l ¨ "classic" machine learning systems such as ID 3 (Quinlan, 1979) & AQ 11 (Michalski & Chilauski, 1980);

Fully computerised knowledge acquisition n systems designed to provide knowledge for a particular system's knowledge base, e. g. METADENDRAL, designed to discover rules for the rule-base in DENDRAL; n data mining systems; these do a similar job to classic machine learning systems, but work on a very large database of information. n sub-symbolic systems, i. e. neural nets and genetic algorithms. More about these in the last lecture in this course.

Fully computerised knowledge acquisition l There are plenty of examples of machine learning systems producing formerlyunknown knowledge, and knowledge that was better than that of a domain expert

Knowledge discovery e. g. (1) META-DENDRAL l produced rules about the behaviour of molecules in a mass spectroscope that were published in a chemistry journal as original contributions to the field; l

Knowledge discovery e. g. (2) AQ 11 l produced rules about how to diagnose diseases in Soya bean plants. l

l AQ 11’s rules were correct 97% of the time. The domain expert's rules were correct 83% of the time; he abandoned his rules, and adopted AQ 11's rules instead.

Knowledge discovery e. g. (1) META-DENDRAL produced rules about the behaviour of molecules in a mass spectroscope that were published in a chemistry journal as original contributions to the field; l e. g. (2) AQ 11 produced rules about how to diagnose diseases in Soya bean plants. They were correct 97% of the time. The domain expert's rules were correct 83% of the time; he abandoned his rules, and adopted AQ 11's rules instead. l

Fully computerised knowledge acquisition This approach is particularly fruitful in 'knowledge-poor' domains, i. e. domains where not much expert knowledge is available. ¨ However, it is a mistake to believe that one can do machine learning without a domain expert - at the very least, you need an expert to select the training examples, and to explain the domain terminology. Probably also to identify the features of the examples which are likely to be relevant. ¨