Data Mining Books 1 2 3 Data Mining

Data Mining Books: 1. 2. 3. Data Mining, 1996 Pieter Adriaans and Dolf Zantinge Addison-Wesley Discovering Data Mining, 1997 From Concept to Implementation Cabena and et al. Prentice Hall Data Mining, 2000 Concept and Techniques Jiawei Han and Micheline Kamber Morgan Kaufmann 1

- Proceedings 1. 2. 3. 4. 5. 6. 7. Proceedings of the International Conference on Data Mining (ICDM) Proceedings of the International Conference on Data Engineering (ICDE) Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining Proceedings of the International Conference on Very Large Data Bases (VLDB) Proceedings of ACM SIGMOD International Conference on Management of Data Proceedings of the International Conference on Database Systems for Advanced Applications (DASFAA) Proceedings of the International Conference on Database and Expert Systems Applications (DEXA) 2

8. Proceedings of the International Conference on Data Warehousing and Knowledge Discovery (Da. Wak) 9. Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD) 10. European Conference on Principles of Data Mining and Knowledge Discovery (PKDD) - Journals 1. 2. 3. 4. 5. 6. 7. 8. IEEE Transactions on Knowledge and Data Engineering (TKDE) Journal of Intelligent Information Systems Data Mining and Knowledge Discovery ACM SIGMOD Record The International Journal on Very Large Database Knowledge and Information Systems Data & Knowledge Engineering International Journal of Cooperative Information Systems 3

Outline - Introduction - Knowledge Discovery in Databases (KDD) - Data Mining and Query Tools - Basic Data Mining Techniques - Data Mining and Data Warehouse - Association Rules 4

- A short story • The library of Babel (infinite) ØBooks must be somewhere in the library ØPeople wander round this library until they die ØThe library contains an infinite amount of data but no information • Today’s environment ØToo much data but too little information - Challenge • Find the required information from huge amounts of data • The amount of data is growing increasingly difficult to find the meaningful information 5

- Knowledge Discovery in Database (KDD) • The whole process of extraction of implicit, previously unknown and potentially useful knowledge as a production factor from a large data sets • Include data selection, cleaning, coding, data mining, and reporting - Data Mining • The key stage of Knowledge Discovery in Database (KDD) • The process of finding the desired information from large database 6

- KDD is not a new technique but rather a multidisciplinary field of research 7

- AI, machine learning (1950) - It is extremely difficult to create computer that has an intelligent close to that of human beings • Lack of creativity and self-learning - 1960: stop researching about learning • Neural network fail (XOR) - 1980 ~: neural network changes architecture, new machine learning algorithm (decision tree, genetic algorithm, etc. ), powerful computer, focus on simple and practical problem 8

- Why learning • Even for simple problem, such as timetable planning extremely hard to solve with a computer but easily solved by experienced human - Using expert system to solve problem • Even for simple systems, a great many rules existed. It is difficult to find the right rules. • Need to interview relevant experts many times and integrate them to obtain the expert knowledge ØKnowledge acquisition: using learning algorithms to generate rules automatically 9

- Why interest in data mining • In the 1980 s, all organizations begin to build database. Until now, they contain gigabytes of data with much ‘hidden’ information that cannot easily be traced using SQL ØSQL is just a query language under the constraints that you already know • As the use of networks, it will become increasingly easy to connect database ØDiscover more information • Machine learning techniques have been improved ØEasier to find interesting information • Client/server environment ØElectronic commerce 10

- Data mining tool & Query tool • Suppose a large database containing millions of records that describe customers’ purchases ØWho bought which product on what date? ØWhat is the average turnover in July? ØWhat is an optimal segmentation of clients ØWhat are the most important trends in customer behavior • If you know exactly what you are looking for, use query tool • If you know only vaguely what you are looking for, use data mining tool 11

- Data mining in electronic commerce • The success of KDD come primarily from marketing • Prediction ØCustomer buying baby clothes today may buy computer games in ten years, and fifteen years later a motorcycle 12

• Suppose a company keeps the data about what products they bought ØMail to everyone only 3% ~ 4% interest ØAnalyze user behavior, and cluster customers according to their interests can save 50% of mailing costs 13

- The problems of data mining • Lack of long-term vision ØWhat do we want to get from the database in the future? • Not all files are up to date ØExample: the price of computer • • Struggle between departments Poor cooperation between users and EDP dept. Legal and privacy restrictions Data model need to be transformed for different data mining technique • Timing problems: integrate data from different sources • Interpretation problems 14