Ethical and Legal issues From Building Expert Systems

Ethical and Legal issues • From: Building Expert Systems: Principles, Procedures and Applications, E. M. Awad, West Pub. Co. , 1996, ISBN 0 -314 -06626 -8 • • Ethics deals with: moral vs immoral, legal vs illegal As a software engineer or programmer, you are obliged to make ethical decisions: – nature of actions: are they fair, reasonable, conscionable? – consequences of actions or inactions: who gains, who loses? – consequences of actions on others: society issues threats to ethics: – telecommuncations, networks, Internet – massive distributed databases – ease of access to information – view that stored information/knowledge are competitive weapons • B. Ross Cosc 4 f 79 1

Some ethics guidelines • William Allen (Boeing’s CEO for 14 years) – Do not be afraid to admit that you don’t know. – Be definite - tell it like it is. – Try to promote honest feelings toward the company. – Don’t talk too much. . . let others talk. – Be considerate of your associates’ views. – Above all, be human - keep your sense of humour and learn to relax B. Ross Cosc 4 f 79 2

ACM Code of Ethics (extracts) • • • General Moral Imperatives – honour property rights including copyrights and patents – respect the privacy of others – honor confidentiality Professional responsiblities – give comprehensive evaluations of computer systems and their impact, with emphasis on possible risks – improve public understanding of computing and its consequences Organizational Leadership Imperatives – acknowledge and support ethical uses of computing systems – support policies that protect the dignity of users and others affected by computing systems B. Ross Cosc 4 f 79 3

Ethics and Expert systems • • approach system building using principles of integrity and ethics don’t oversell expert systems – be honest with weaknesses, possible problems – be honest with impact caused by inevitable errors “The zeal with which expert systems are built should be tempered with the realization that they are based on human knowledge and ideas, which are less than perfect. ” B. Ross Cosc 4 f 79 4

Legal issues • • • Who owns knowledge? – If expert is hired on a contract basis to make an expert system, and perhaps receives royalties, s/he is more liable than if they are an employee of the firm – if company “purchases” expert’s knowledge, s/he may escape future liability What if an expert employee is unwilling to release knowledge gained during years of employment? – does the company have rights to that knowledge? Scenario: an employee gains expertise via using an expert system. Does the company have the exclusive rights to that knowledge? – no, unless the employee signed an intellectual property rights agreement withthe company when hired B. Ross Cosc 4 f 79 5

Legal liability • • Case: A doctor consults a medical expert system to diagnose a patient. The patient is treated but soon dies due to misdiagnosis and improper treatment. Who is liable? – knowledge engineer? – expert? – company who bought the system (hospital? ) – shell programmer? – user (doctor)? – the expert system software itself? – all of the above? B. Ross Cosc 4 f 79 6

Expert System Liability • a) knowledge engineer: responsible for choosing expert; ensuring system’s accuracy and reliability; knowledge is properly acquired and represented – KE is personally liable, unless an employee of the organization • b) domain expert: his or her knowledge is encapsulated in KB – again, contract vs employee relationship affects liability • c) shell developer: bugs in inference, other utilities – could be wide-ranging: lots of expert systems may be written! • d) user: user knowledge when inputting into the system; relying on faulty diagnoses B. Ross Cosc 4 f 79 7

Expert System Liability • Product: – off-the-shelf software – mass-marketed – custom-designed, but affects a lot of customers – → proving negligence is unnecesary to hold developer liable – → can have “express warranty” disclaimer to limit liability • Service: – custom-designed software – → must prove negligence – → law of state applies B. Ross Cosc 4 f 79 8

Expert Systems and Liability • • Knowledge engineering and computer programming is not yet standardized or certified. If/when it does, then KEs are subject to malpractice: negligence or professional liability applied to developers for design defects in systems tailored specifically for professional use. May be an issue in the future, as KBS’s and programming matures. a user can be liable for failing to use an expert system when one is available. B. Ross Cosc 4 f 79 9

Future AI systems • • What if an expert system autonomously evolves and learns while used in the field? how can expert or KE be held liable? System may grow in ways totally unexpected from initial developers. Can the software itself be held liable? • Stay tuned. • • B. Ross Cosc 4 f 79 10

Summary of Legal Issues 1. expert owns knowledge if no prior agreement established (eg. preemployment contract). 2. KE is subject to personal liability if problems arise with system. If KE is an employee, then employer is liable. 3. If expert system is a product, no proof of negligence is necessary. A disclaimer of warranties can be used. Loss to developer is “a cost of doing business”. 4. If expert system is a service, then contract law of state applies. 5. US courts are reluctant towards “exclude warranty disclaimers”. 6. Cases involving warranties require user show who is at fault and why. B. Ross Cosc 4 f 79 11