Journey to the Mysterious World of Chatbots Deepti

Journey to the Mysterious World of Chatbots Deepti Tiwari & Manju Joseph February 2019

• Agenda © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Advancements in AI & Chatbots • Next. Gen Interactions • Taking Stock • Scope of Pilot • Chatbot Architecture • Key Design Considerations • Pilot Results

Advancements in AI & Chatbots © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential • • Buzzword of the decade Equipped with NLP and ML technologies • Human-like interactions • AI-powered chatbots • Disruption potential

Next. Gen Interactions © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential

Taking Stock • Accessing large volumes • Content everywhere – Position Complication Situation Use AI to derive transformative insights from a wealth of enterprise content Huge volumes of structured and unstructured content that had to be curated Implication Accuracy and value of response is critical Engage with internal and external technology experts to build, test, and train an AI-infused chatbot of data Benefit user docs, marketing, support Action Maximize customer satisfaction by providing personalized and value-added services • Chatbot Mostpilot chatbots are built to implementation few of our • for flagship products do simple things Content overlap across different products • Analyzing context to provide intelligent • Content not easily answers is still not a searchable reality © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential

• • Scope of Pilot • Start small and scale later Host the chatbot on Cisco’s doc portal Key chatbot features: • Accuracy • Conversational maturity • © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Autonomous reasoning

Chatbot Workflow NLU/NLP Layer Machine Learning Hi, how may I help you today? Where will the future Olympics be held? Conversational System Dialogue Manager Did you mean Summer Olympics? Yes Tokyo will host 2020 Summer Olympics. © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Response Generation Information Extraction Deep Learning Feature Preprocessin Po. S Tagging g. Engineering

NLP Context Awareness Key Design Considerations Marketin g PSIRTS Tech. Docs FAQs TAC Training the Bot CDETS Knowledge Ecosystem Personalization © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential

Understanding the Context Taxonomy Metadata © 2019 Cisco and/or its affiliates. All rights reserved.

Natural Language Processing How do I configure DHCP server for zero-touch provisioning in Cat 9 K? Entity “How” “do” Intent “I” “configure” “DHCP” “server” “for” “zero-touch” Type: Product “provisioning” “in” “Cat” “ 9 K”? Technology/Feature Pre-processing: Tokenization | Po. S Tagging | Stopwords Challenges Entity Tagging: Identifying concept • • Intent Identification: Action to be taken by chatbot Context: Word Embedding Model © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Acronyms Domain Terminology Special Characters Rich Text

• Training the Chatbot • • • © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Knowledge Base Creation Ø Ingest raw data Ø Generate domain vocabulary Reinforcement Learning Ø Commonly used acronyms Ø Greeting statements Ø Answering weird questions Approach for unresolved queries Ø Pre-defined set of answers Ø Establish dialogues Capturing user experience Ø Feedback logs (Thumbs up/down) Ø Chat closure analysis

Personalization • Understanding an individual over a period of time Ø Ø • • © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential User’s profile information Area of interests (products, technologies, doc types, tasks, install base) Improve first-contact resolution Ø Chat history Ø Current context Ability to predict the right options Ø Minimal conversational interaction Ø Meaningful predictions

Knowledge Ecosystem • • Marketing PSIRTS Generate new knowledge to handle most common queries Leverage data from variety of content sources Ø Tech. Docs FAQs TAC CDETS • Handle both structured and non-structured content Ø Ø © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Tech. Docs, Marketing, Bugs, Security Incidents, Technical Support, FAQs Tech. Docs content vs other content Extract, Refine, Process non-structured content

Chatbot Pilot Results Iteration 1 Accuracy Data Input 28% 52% Conversation 20% 2 Platform Series 9500 & 9300 7000 XML Topics and Contents Correct XML Data Format Partial Wrong Iteration 2 Accuracy 44% 42% Conversation 14% Reasoning © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Correct Partial Wrong

“Moving from pilot to large-scale implementations of AI-chatbot is a daunting task that requires

Demo