Developing A Cognitive Assistant For the Audit Plan

Developing A Cognitive Assistant For the Audit Plan Brainstorming Session Qiao Li Miklos Vasarhelyi

Introduction 2 Audit Plan Brainstorming Session: Mandated in two auditing standards: SAS No. 99 and SAS No. 109 Purpose: Discuss the susceptibility of the entity’s financial statements to material misstatement due to fraud (Landis 2008, Hammersley et al. 2010, Beasley and Jenkins 2003) Common procedure: Checklist in open-ended form Current Issues: Limitations with Checklist: structurally-restrictive tool (Seow 2011, Asare and Wright 2004, Dowling and Leech 2007, Landis 2008) – limits new ideas generation & induces biases Uncollected experience and expertise Unreached additional external information What we can do? ? Audit Cognitive Assistant

Cognitive Assistant (Intelligent Personal Assistant) 3 Cognitive Assistant (Intelligent Personal Assistant) Speech-enabled technologies that uses inputs such as the user’s voice, vision (images), and contextual information to provide assistance by answering questions in natural language, making recommendations, and performing actions (Canbek and Mutlu 2016, Hauswald et al. 2015, Myers et al. 2007, Garrido et al. 2010, Chen, 2015) Commercial personal assistants (Mehrez, 2013): Apple Siri, Google Now, Microsoft’s Cortana. Amazon’s Echo/Alexa IBM Watson

Research Motivation 4/35 Advantages Of Cognitive Assistant In Audit Decision Plan Brainstorming: Main Features of Cognitive Assistants Potential Support in Audit Plan Brainstorming Information Retrieval Provide audit related information for risk assessment Recommendation Support Recommend discussion topics and potential risks Adaptive Learning Collect knowledge and experience from senior auditors Service Delegation Audit applications / program connection

Proposed Method: Proposed Audit Cognitive Assistant - Luca Open an application Luca industry Client Recommended Topics: General understanding, new events, business risks… Position Architecture Interface 5 Show Answer Query You may also interested in: … Answer Automatic Speech Recognition Query Classifier Question or Action QA DBMS Knowl edge Base Web Search Open (ACL, IDEA…) Calculator Open standards Open templates Audit workpaper Calendar ……. Recognition (ASR) • Language Understanding • Dialogue Management • Natural Language Generation Action Audit Related Applications It Can Access Modules: • Automatic Speech text Execute Action backstage supporter Luca Processing… • Text-to-Speech synthesis Knowledge Database Domain Knowledge Unstructu red data Knowledge about users

Architecture of the Proposed Audit Cognitive Assistant Luca Open an application Luca industry Client Show Answer text 1 2 3 4 5 6 7 Execute Action backstage supporter Audit Related Applications It Can Access Web Search Open tool Calculator Open standards Open templates Audit workpaper Calendar ……. Query You may also interested in: … Ap p Automatic Speech Recognition Luca Processing… Recommended Topics: General understanding, new events, business risks… Position Architecture Interface 6 Capability Domain: judgement or experience Answer Unstructured: financial Search engine such as google statements, accounting policies, Query ACL, IDEA QA analytical procedures, litigation, Classifier Ratios; others like Benford’s claims, recent news information, law Knowl Question Regulations related to the audits workpapers, prior year audit DBMS edge audit area oror account Action and adjustments… deficiencies Base Required procedures, User interaction: queries and guidance or programs search Action Open prior audit Audit plan schedule Modules: • Automatic Speech Recognition (ASR) • Language Understanding • Dialogue Management • Natural Language Generation • Text-to-Speech synthesis Knowledge Database Domain Knowledge Unstructu red data Knowledge about users

Question Answering System of the Proposed Cognitive Assistant 7 Question Analysis Machine Learning Models Normalized Question Keyword extraction; Punctuations, abbreviations, nouns and verbs Answer Selection Knowledge base Query Generation Generated Query Answer Matching Doc. With Candidates ② ① Paired QA DBMS NL Answer Generating Answer with highest score Answer Extraction ① +② IR-based (Domain) QA + Knowledge-based (Cognitive) QA

8 Thank You !