IBMs Watson Deep QA IBMs Watson represents an

IBM’s Watson

Deep QA • IBM’s Watson represents an innovation in Data Analysis Computing called Deep QA (Question Answering) • Their project is a hybrid technology that incorporates information retrieval with artificial Intelligence called Question‐‐‐Answering or QA technology. • QA is a form of information retrieval where by the aim is to respond to natural language (NL) questions about some topic by providing natural language answers rather than a list of relevant documents (keyword searching) or isolated data (databases). • It means Watson can parse natural language better than anything out there. • In the system we use today, the user formulates a question, • pulls out 2 -3 keywords, • sends them to a search engine which gets documents with the keywords present within and sends them back to the user, • who reads the documents and analyzes the results.

Deep QA • All of the real "work" is done by the user. • But a better system is one in which the user asks a natural language question, • the "expert" understands the question, • produces answer and evidence, • analyzes the results, • produces a statement about how confidant it is in the answer, • and returns its findings to the user. • This "expert" role used to be filled by librarians. • Watson is capable of doing this in less time and with a much larger set of resources than any human librarian could.

Deep QA • Given a natural language question and resources, it should return • precise answers, • accurate confidence, • justifications, • and a fast response. • Before Watson, the best a program could do would either be the first two of these, or the last, but rarely all four. • Watson can do all four, and this is how:

How Does Watson Do It? • As soon as the prompt is chosen and clue revealed, Watson receives the text clue. • While Alex is reading the clue aloud (approximately 3 seconds) Watson parses the clue, • looking for type of answer, • things to search for, • and relationships to former clues, especially in that category. • It searches through its reference materials • retrieves candidate answers, • analyzes those answers, • synthesizes a final list of answers that are the same type hinted at in the question, • calculates the confidence in the answer based on keyword matching, • temporal reasoning (If the king of Spain lived for so many years and died on such a date, he was born on such a date), • statistical paraphrasing (fluid and liquid are technically different, but are commonly interchangeable terms), • and geospatial reasoning (Bordeaux is in France), • and possibly repeats this process if it broke the question into smaller pieces, decides on the most statistically probable answer, • and buzzes in as the buzzer is turned on, moments after Alex finishes speaking.

How Does Watson Do it So Fast? • How does Watson accomplish this feat in just three seconds? • For a single processor, this task might take anywhere between 20 minutes and 3 hours, • IBM again has found a solution. • Using a method called asynchronous scale‐out • These jobs are doled out to an entire computing cluster comprised of 90 power 750 series servers with 2880 POWER 7 processor cores, and 15 terabytes of RAM • can operate at 80 tera. FLOPs. (floating-point operations per second) • Using this computing cluster, the same process takes less than 3 seconds and is done in plenty of time for Watson to activate his buzzer. • Between the software and hardware of Watson, it's easy to see how advanced this technology has become.

How Can We Use Watson? • What do we do with this new technology? • It's nice that we have a computer that can understand natural language questions, • But how do we use it?

Watson in Health. Care • • • One such use of Watson is in the health care system. Medicine is the ultimate information-based profession. Med students are required to memorize thousands of facts and terms. Especially in the field of differential diagnosis, the ability to quickly access any of those thousands of facts can make or break a case. IBM is currently working on a program to assist doctors in the differential diagnosis process. Not only does it take input of symptoms from the doctor, • it looks through the patient's file for important details like • age, • trips out of the country, • family history, • or critical things like pregnancy, • and prompts further questions to ask the patient. Then it can suggest a list of possible diagnoses with their relative confidences and justifications. This is especially important in differential diagnosis because Watson would not return one answer, but a group with varying statistical probability. After all, the most statistically likely disease is not necessarily the one the patient has.

Watson in Higher Education? • • • This is from IBM: Watson can be used in higher education. Higher education is essential for a flourishing economy People with less schooling put a drag on the workforce. Higher education produces people who are capable of being the "expert" Thorough education takes years to create a talent base like that utilized by companies similar to IBM, A single person with a high school diploma working a system like Watson could do the work of a college intern. By using Watson, one could expand the effective talent base, improving the economy and thus the government With a stronger and well-funded government there would be more support for utilities, healthcare, and public education, starting the cycle again. Do you agree with this assessment by IBM?