Introduction to Machine Learning Prof D Spears COSC

Introduction to Machine Learning* Prof. D. Spears COSC 4010/5010, Section 1 Spring 2004 * This material is taken from the textbook, Machine Learning, Volume I, Eds. Michalski, Carbonell, and Mitchell, Tioga, 1983, and from Artificial Intelligence by Russell and Norvig.

Definition of Machine Learning n n Informal definition: Any computer program that improves its performance at some task through experience and/or data. Formal definition: A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E. Wow! Look at how much it learned!

Other Disciplines From Which Machine Learning Draws Ideas and Techniques decision theory AI probability & statistics biological evolution control theory information theory machine learning statistical mechanics computational complexity theory ethology game theory philosophy optimization psychology neurophysiology

Some Learning Strategies/Techniques n n n n Rote learning Inductive inference e b Stochastic/Bayesian inference n a gc n i e n v Deductive inference i r t a c e L ra o e Reinforcement learning v assi p Neural network learning Evolutionary learning Clustering Analogical learning Learning from human instruction (being told) Learning by discovery Case-based reasoning Speed-up learning Multi-strategy learning is very popular

Examples of Types of Knowledge Acquired Via Learning n Declarative Knowledge n n n Concepts Preferred values of parameters Grammars Taxonomies Procedural Knowledge n n n Rules Rule strengths Graphs/networks Computer programs Plans Example strategies for acquisition: Inductive inference Evolutionary learning Clustering Analogy Induction Reinforcement learning Evolutionary learning Stochastic learning

Example Data Structures Used for Learned Knowledge Type of knowledge: n n n n Decision trees Logical expressions Neural networks Condition-action rules Rule sets Finite-state automata Lisp code Concepts Behavioral rules Plans Computer programs

History of Machine Learning n 1950’s: Neural modeling n n 1960’s: Pattern recognition and decision-theoretic learning n n n E. g. , perceptrons (Rosenblatt, 1958) Groundwork for this work was laid by researchers in mathematical biophysics (Rashevsky, 1948) (Mc. Culloch and Pitts, 1943). Major thrust was on learning tabula rasa. Focus on self-organization and neuron-like learning elements. Acquire linear, polynomial, or related forms of a discriminant function from a given set of training examples, e. g. , (Nilsson, 1965). Samuel’s checker’s program (Samuel, 1959, 1963). Acquired a master level of performance. Statistical decision theory for pattern recognition, e. g. , (Watanabe, 1960) (Duda & Hart, 1973). 1969: Minsky & Papert on theoretical limitations of perceptron learning. 1970 s: Adaptive control n Self-adjust parameters to maintain stability in spite of disturbances, e. g. , (Davies, 1970) (Fu, 1971).

History of Machine Learning (cont’d) n 1960’s and 70’s: Models of human learning n n 1970’s: Genetic algorithms n n High-level symbolic descriptions of knowledge, e. g. , logical expressions or graphs/networks, e. g. , (Karpinski & Michalski, 1966) (Simon & Lea, 1974). META-DENDRAL (Buchanan, 1978), for example, acquired taskspecific expertise (for mass spectrometry) in the context of an expert system. Winston’s (1975) structural learning system learned logic-based structural descriptions from examples. Developed by Holland (1975) 1970’s - present: Knowledge-intensive learning n A tabula rasa approach typically fares poorly. “To acquire new knowledge a system must already possess a great deal of initial knowledge. ” Lenat’s CYC project is a good example.

History of Machine Learning (cont’d) n 1970’s - present: Alternative modes of learning (besides examples) n n n Learning from instruction, e. g. , (Mostow, 1983) (Gordon & Subramanian, 1993) Learning by analogy, e. g. , (Veloso, 1990) Learning from cases, e. g. , (Aha, 1991) Discovery (Lenat, 1977) 1991: The first of a series of workshops on Multistrategy Learning (Michalski) 1970’s – present: Meta-learning n n n Heuristics for focusing attention, e. g. , (Gordon & Subramanian, 1996) Active selection of examples for learning, e. g. , (Angluin, 1987), (Gasarch & Smith, 1988), (Gordon, 1991) Learning how to learn, e. g. , (Schmidhuber, 1996)

History of Machine Learning (cont’d) n n n n 1980 – The First Machine Learning Workshop was held at Carnegie-Mellon University in Pittsburgh. 1980 – Three consecutive issues of the International Journal of Policy Analysis and Information Systems were specially devoted to machine learning. 1981 – A special issue of SIGART Newsletter reviewed current projects in the field of machine learning. 1983 – The Second International Workshop on Machine Learning, in Monticello at the University of Illinois. 1986 – The establishment of the Machine Learning journal. 1987 – The beginning of annual international conferences on machine learning (ICML). 1988 – The beginning of regular workshops on computational learning theory (COLT). 1990’s – Explosive growth in the field of data mining, which involves the application of machine learning techniques.

A general model of external learning agents performance standard critic sensors feedback learning element changes performance knowledge element learning goals problem generator effectors AGENT environment

Evaluating Learners on unseen data Learning curves A C C U R A C Y AMOUNT OF TRAINING DATA SEEN

Some Ideas for Projects n n n n n Multi-agent / swarm reinforcement learning Concept learning using logical, stochastic, neural, or evolutionary representations or hybrids Learning a good representation for learning concepts (meta-learning) Data mining: Discovering patterns in large data sets (medical? consumer? ) Modeling the process of scientific discovery Evolving a simple artificial brain Cognitive models of human learning “Safe” learning Learning in artificial life/worlds Learning in soccer-playing agents Unsupervised learning (clustering) to develop taxonomies Learning to predict temporal sequences Training a neural network to recognize objects, faces, etc. Multi-agent learning to cooperate or compete Learning to improve game playing strategies Evolving computer programs (genetic programming) Comparative studies of different learning methods A variant of a study found in a machine learning conference paper Analogical learning (e. g. , applying knowledge of one case to a new case) Learning a model of a student for intelligent tutoring