The Pros and Cons of Using Big Data

The Pros and Cons of Using Big Data In Auditing: A Synthesis of the Literature and a Research Agenda M. Alles and G. Grey Discussant’s Comments Severin Grabski

Overview • Objectives of Manuscript: – Discuss Pros & Cons of Incorporating Big Data in Financial Statement Audits • “Temper the Unbounded Optimism” – Present Research Agenda to Identify Specific Aspects of Big Data that could Benefit Auditors • Discussion will try to match these Objectives – Discuss Pros & Cons of Assertions made in the Manuscript 2

Assertions • Inevitability of Big Data in Auditing (p. 3) – AGREE • Temper optimism (p. 4) – GOOD, AVOID HYPE CYCLE & TROUGH OF DISILLUSIONMENT • Compares “Big Data” to AI, Web Trust, Continuous Auditing (p. 4) – ARE THE DRIVERS FOR BIG DATA THE SAME AS THEY ARE FOR CA? WEB TRUST? AI? 3

Assertions • Auditors are successfully completing audits without the use of Big Data (p. 5). – DO WE REALLY KNOW THAT ALL OF THESE AUDITS WERE SUCCESSFUL? • Cost/Benefit of using Big Data – do more (same) audit work for lower cost (p. 5). – WILL THE AUDIT OF THE FUTURE (in 2030) LOOK THE SAME AS THE AUDIT OF THE 1930 s? REPLACE RATHER THAN ENHANCE --- IS THIS WHAT IS NEEDED? 4

Assertions • Term Big Data Not Always Consistent with its Definition in Wider Data Analytics Community (p. 5) – BIG DATA IS NOT JUST ABOUT VOLUME – IT IS MORE ABOUT THE VARIETY OF DATA – IT IS ABOUT THE VERACITY OF THE DATA – SOCIAL MEDIA, Io. T, ENVIRONMENTAL SENSOR DATA, ETC. COMBINED WITH STRUCTURED DATA 5

Assertions • Term Big Data Not Always Consistent with its Definition in Wider Data Analytics Community (p. 5) 6

Examples of big data…. . Walmart handles more than 1 million customer transactions every hour, which is imported into databases estimated to contain more than 2. 5 petabytes * of data — the equivalent of 167 times the information contained in all the books in the US Library of Congress. FICO Credit Card Fraud Detection System protects 2. 1 B active accounts world-wide. The volume of business data worldwide, across all companies, doubles every 1. 2 years, according to estimates

Assertions • Big Data may be outside of the comfort zone and technical capability of the current audit profession (p. 5). – BIG DATA IS OUTSIDE OF THE COMFORT ZONE FOR MOST ANY BUSINESS, YET THEY EMBRACE IT – WHY SHOULD AUDIT FIRMS BE ANY DIFFERENT? 8

Assertions Using a Client’s Big Data as Part of the Financial Audit would be a Paradigm Shift (p. 6): 1. Access to proprietary and sensitive client data heretofore unused 2. Significantly higher reliance on nonfinancial data 3. New technical skills needed 4. Increased business acumen needed – AND THE POINT IS? BUSINESSES 9

Assertions • Big data would enable managers to run the business more proactively rather than look for explanations with a rear-view perspective. This does not imply that auditors will have to adopt such a proactive perspective. They would unlikely want to do so in the absence of a rigorous analysis of what Big Data brings to auditing. (p. 12) – UNLESS REQUIRED BY LAWS/REGULATIONS – WOULD AUDIT FIRMS NOT WANT TO RUN THE 10

Assertions • Only when the limits of traditional data have been reached are auditors likely to turn to other types of Big Data (p. 13) – AUDIT FAILURES EXIST. THE TRADITIONAL DATA AND RELATED TECHNIQUES HAVE NOT BEEN SUFFICIENT. – THIS SEEMS TO INDICATE THAT THE AUDIT MODEL MUST CHANGE TO INCLUDE BIG DATA. 11

Assertions • Auditors have “Big Data” on all previous audits. Hoitash et al. (2006) demonstrated using peer information can significantly improve audit analytics. – Can auditors use data from other audited firms in their examination of an audit client? (p. 13) – Confidentiality? • AICPA • ICAEW 12

Confidentiality - AICPA • A member has received a request from a third party (for example, a trade association, member of academia, or surveying or benchmarking organization) to disclose client information or intends to use such information for the member's own purposes (for example, publication of benchmarking data or studies) in a manner that may result in the client's information being disclosed to others without the client being specifically identified. May the member comply with such a request…? 13

Confidentiality - ICAEW • “to respect the confidentiality of information acquired as a result of professional and business relationships and, therefore, not disclose any such information to third parties without proper and specific authority, unless there is a legal or professional right or duty to disclose, nor use the information for the personal advantage of the professional accountant or third parties. ” 14

Assertions Table 1 • Pros and Comments are very good Table 2 • Liked Inhibitors – Comment – Amelioration presentation • Not sure if all inhibitors are valid, e. g. , direct access to client’s Big Data – MUCH OF THAT DATA SHOULD BE AVAILABLE VIA OTHER MEANS – TRANSACTIONAL DATA IS ALREADY IN PLAY VIA ERP SYSTEMS 15

Assertions • The manuscript discusses an audit as a binary decision (p 18), however, in the aggregate (like the drug regime statement)… – A 1% IMPROVEMENT IN ALL THE AUDITS WOULD RESULT IN A SIGNIFICANT AGGREGATED BENEFIT (REDUCED CLASS ACTION LAWSUITS, MORE EFFICIENT CAPITAL MARKETS, ETC. ) 16

Assertions Increasing levels of difficulty for acceptance by auditors (p. 21) 1. More Data – COLLECT, STORE, SECURE (STORAGE IS CHEAP, AUDIT FIRMS MUST HAVE SECURE SYSTEMS) 2. Messy Data – “PRISTINE” ERP DATA WILL LIKELY HAVE NOISE, ESPECIALLY WHEN MULTIPLE ERP SYSTEMS ARE USED 3. Correlation rather than Causation – DEPENDS UPON THE SETTING – PREDICTIVE ANALYTICS – IS THIS WHAT AUDITORS NEED? 17

Assertions • Audit Data Standard on Data Integrity (p. 22) – DATA PROPERLY RECORDED BUT IS IT TRUE? 18

Assertions • Verifiability and Credibility in Auditing are Vital (p. 24) – AGREE • Use of correlation to suggest areas for further investigation – AGREE • Implied not instituting formal controls base on that information – AUDITORS DON’T IMPLEMENT CONTROLS 19

Assertions • If Auditors Find A Correlation Between Newly Divorced Employees And Expense Overbilling What Should They Do? (p. 24) – EXTERNAL AUDITORS WOULD PROVIDE THIS INFORMATION TO MANAGEMENT – INTERNAL AUDITORS WOULD LIKELY PERFORM 100% AUDIT – CONTROLS WOULD BE PUT IN PLACE FOR ALL EMPLOYEES TO ELIMINATE OVERBILLING POTENTIAL 20

Assertions • Correlation versus Causality/Role of Theory (p. 24 -27) – AGREE WITH THE IMPORTANCE OF HAVING DOMAIN KNOWLEDGE – SHORTCOMINGS IN ALL THE STUDIES WERE BASED ON LACK OF DOMAIN KNOWLEDGE – EMPHASIS SHOULD BE ON THE DEEP BUSINESS KNOWLEDGE THAT AUDITORS PROVIDE 21

Assertions • External Auditor Research Opportunities List – LIKED THE LIST – AGREE WITH “AT THE EARLY STAGES” – MISSING: COSTS OF NOT INCORPORATING BIG DATA INTO THE AUDIT • How will the audit profession respond to the accusation that “it was not in the sample” is inadequate given the common use of Big Data by businesses? And it the audit firm had used 100% 22 sample/Big Data they would have found the fraud

Assertions • Research Opportunities (p. 29 -31) – Outside of External Audit Domain – Internal Auditor – Accounting Firms, Not in the Audit Area – SEEMED MORE LIKE AN ACKNOWLEDGEMENT THAT THESE RESEARCH AREAS EXIST 23

Other Thoughts • Conclusion about Tweets & Hurricane Sandy (p. 32) – IDENTIFIES PRECISELY THE PROBLEM OF HAVING A LACK OF DOMAIN KNOWLEDGE AND THE APPLICATION OF STATISTICAL TECHNIQUES – “MAGICAL THINKING” SEEMS TO BE ASSUMING THAT BIG DATA INDEPENDENT OF CONTEXT AND DOMAIN KNOWLEDGE 24 WILL SOLVE ALL PROBLEMS

Other Thoughts • At time, it seemed as if I was reading an anthology rather than getting the authors’ thoughts (and not sure why the quotes/authors were selected) • Many assertions were made, but minimal data was provided to substantiate the assertions • Many assertions were made assuming an all or nothing approach • The manuscript did not “think outside the 25

Extensions • Big Data is not just about Volume – It is more about the variety of data – Social Media, Io. T, Environmental Sensor Data, etc. combined with structured data • 80% Time Spent Cleaning Data – Based upon Relationship to Outcome 26

Extensions • How can External Audits Use “Big Data” – Social Media --- Going Concern – Structured Data --- Operations • • Compare Production to Shipping Data; Compare Purchases to Production Data; Compare Shipping Data to Cash Receipts; Compare Purchasing Data to Cash Disbursements 27

Extensions • Audit Outcome is Binary, BUT – Decision is Based Upon Many Multiple Continuous Decisions • • • Cash Balance Purchases Payables Sales Receivables Inventory • Need to Consider ALL of These & ALL Data Types! 28

Extensions • Audit Procedures Developed Based upon – Nature of Risks Present • HOW WOULD BIG DATA REDUCE RISK? – Timing of Tests • Change in Confirmations, etc. – Extent of Testing • Covered (100% audits) • Impact on Substantive Tests (not likely on Tests of Controls) 29

Extensions • Big Data to Determine More Efficient and Effective Resource Allocation for the Audit – Right People for the Right Task – Based Upon What Will Happen Next 30

Extensions • Audit Applications – Environmental Sensors & Factory Production Levels to Verify Production – Social Media Tweets – • Generally are not restricted, • Need hashtag, • Can access employee, vendor, customer tweets 31

VW &Twitter 32

Extensions • • Need to Find the Right Signal Sensible Model Establish Tasks Time Period • MUST HAVE DEEP DOMAIN KNOWLEDGE • THIS IS THE VALUE EXTERNAL AUDITORS PROVIDE! 33

PWC Audit Quality • Complying with accounting and auditing standards; • Applying a deep and broad understanding of our clients’ businesses and financial environments in which they operate; • Using our expertise to raise and resolve issues early; and • Exercising professional skepticism in all aspects of our work. 34

Conclusions • Manuscript accomplished goals of getting me to think about Big Data in Auditing • There is more to Big Data and Auditing than presented in the manuscript • Would have liked the manuscript to go further outside of the “comfort zone” to identify more potential uses of Big Data 35

THANK YOU! 36