Audit FollowUp Survey Ken Lish CPA Audit Manager

Audit Follow-Up Survey Ken Lish, CPA Audit Manager, NSF OIG

Presentation Goals Why are audit follow-up procedures important to the OIG community? Why did we conduct the survey and how was it developed? What did we learn from the survey? Where do we go from here?

OMB Circular A-50 Provides policies & procedures for considering audit reports “Audit followup is an integral part of good management, and is a shared responsibility of agency management officials and auditors. Corrective action taken by management on resolved findings and recommendations is essential to improving the effectiveness and efficiency of Government operations. Each agency shall establish systems to assure the prompt and proper resolution and implementation of audit recommendations. ” Key Agency Roles: 1) Agency Head, 2) Management Officials, 3) Audit Followup Official OMB Circular A-50: https: //www. whitehouse. gov/omb/circulars_a 050

Congressional Interest Senate Hearing: Implementing Solutions: The Importance of Following Through on GAO and OIG Recommendations, December 10, 2015 “Our hearing today is about learning more about how to improve agency follow-up and make sure that everyone is doing their part in assessing recommendation follow-up. The work that the GAO and inspectors general perform in providing recommendations to agencies is important and the recommendations shouldn’t go unimplemented or unnecessarily delayed. ”- Senator and Ranking Member Heidi Heitkamp “…an OIG’s oversight work is not complete once its report and recommendations are issued. An equally important responsibility of Inspectors General is to ensure that agencies are implementing OIG recommendations. By establishing and diligently following procedures to track an agency’s progress, Inspectors General can hold the agency accountable for implementing these important recommendations. ” - Hon. Michael Horowitz, DOJ Inspector General and CIGIE Chair Hearing: http: //www. hsgac. senate. gov/hearings/implementing-solutions-the-importance-of-following-through-on-gao-and-oig-recommendations Senator and Ranking Member Heidi Heitkamp’s Opening Remarks: http: //www. hsgac. senate. gov/download/? id=245768 BE-18 A 0 -436 F-B 2 A 19601 FA 54 BB 30 Honorable Michael E. Horowitz’s Written Testimony: https: //oig. justice. gov/testimony/t 151210. pdf

Congressional Interest Proposed Senate Bill S. 3109 Inspector General Recommendation Transparency Act of 2016 Introduced to the Committee on Homeland Security and Governmental Affairs on June 29, 2016 “Not later than 1 year after the first semiannual reporting deadline described in subsection (a) that occurs after the date of enactment of this subsection, each Inspector General shall make available on the website of the Office of the Inspector General a list of any open recommendations for the establishment that have been open for more than 1 year. ” - S. 3109 Proposed Senate Bill S. 3109: https: //www. congress. gov/114/bills/s 3109/BILLS-114 s 3109 is. pdf

CIGIE and IG Level Interest CIGIE Committee Level Discussions & Discussions Among Individual OIGs Large Variability What Are Best Practices? Can The Process be Improved?

The Survey Where to Start? How do we tackle this project? What are the first steps? What do we want to get out of this effort? Baseline Data Snapshot of Themes/Issues An Answer to “Where do We Go From Here? ” “The purpose of the survey is to identify the different processes used throughout the IG community for following-up on audit recommendations. It is our hope that the results of this survey will be a valuable tool for the members of the community to evaluate and reflect on their own follow-up processes for recommendations. ”

The Survey Design and Methodology Who designed it? Dr. Brett Baker, Deputy Inspector General for Audit, DOD OIG Honorable Hubert Sparks, Inspector General, ARC 7 Person Team from NSF OIG Methodology Combination of Broad-Based and Targeted Questions Open Ended Brief Goal Oriented

The Survey The Questions 1. Please briefly describe your office’s follow-up process for ensuring audit recommendations are properly addressed and corrective actions are implemented. Also, please note any differences in this process for internal audits of the agency (ex. performance audits, the financial statement audit, etc) versus external audits (ex. grantee audits, contractor audits, etc), if applicable. 2. How does your office handle situations where the agency’s corrective actions aren’t fully responsive to an audit’s recommendations? 3. If the agency continues to be non-responsive to a recommendation, what additional steps would your office take (ex. 7 -day letter, media notification, write-up in Semi. Annual Reports to Congress, etc. )? 4. Please describe who makes the final decision (ie. agency, OIG, joint process, other) as to when a recommendation has been fully addressed and final action has occurred? 5. Any additional feedback or general comments/observations?

The Survey The Responses 67% Rate of Response Who responded? IGs: 10 (20% of respondents) DIGs/AIGs/DAIGs: 20 (39% of respondents) Other: 21 (41% of respondents) 100+ Pages

What Did We Learn? Addressing Non-Responsiveness How do OIGs handle situations where the agency’s corrective actions aren’t fully responsive to the recommendations? Remedies to Address Non-Responsiveness 100% 90% 80% 70% 7 Day Letter 60% SAR Write-up 50% Agency Head Other Section 5(a)(12) of the IG Act Suspension and Debarment Withholding Grant Funds 40% 30% 20% 10% 0% 7 Day Letter Media Notification SAR Write-Up/ Agency Other Head/Board Staqndard Congresssional Communication Other

What Did We Learn? Final Action Who Decides When Final Action Has Occurred? 70% Wide Variance 60% Different for Internal vs External 50% 40% Systems Can Diverge 30% 20% OMB Circular A-50 10% 0% Agency OIG Joint Other Special Directive

What Did We Learn? Final Action Agenc y Who Decides When Final Action Has Occurred? “The Agency Audit Follow-up Official is responsible for ensuring and certifying that agreed-to actions are completed. This is a management function as the OIG has no authority to compel anyone to do anything. ” “Its the office's program and they have to decide what is best for their objectives and goals given budgetary and other constraints. ” OIG “The OIG has the authority and indeed the responsibility for ensuring final action has occurred. ”

What Did We Learn? Final Action “The question is does OIG accept the Audit Resolution Official's decision or include significant disagreements in the SAR using section 5(12) at the IG Act or other means such as noted above. Also, should OIGs contest the OMB position that the agency Audit Resolution Official has the final decision authority or at least make it clear the OIG has the right to pursue recommendations further? ” [sic]

What Did We Learn? Resolution During Reporting 13 OIGs indicated they start the resolution process prior to the issuance of the final report. Resolve Recommendations Before Report Issuance Corrective Actions Start Immediately Corrective Actions Must be Provided at Draft Response with Target Associated Dates If Corrective Action Plan is Sufficient During Draft, Another is not Needed Require Alternative Action if Not Concurring Appendix in Report: Summary of Corrective Actions Classify Recommendations as Unresolved, Resolved, or Closed at Report Issuance

What Did We Learn? Centralized vs Decentralized Process 7 OIGs indicated they use a centralized resolution process. 22 OIGs indicated they us a decentralized resolution process. Process Variations Fully Centralized Fully Decentralized Hybrid Model

What Did We Learn? Tracking Systems 8 OIGs indicated they use Team. Central for tracking recommendations. Cost/Labor Efficient Shared Data Shared Systems OIG Managed System Agency Managed System Separate Systems

What Did We Learn? Follow-Up Audits 20 OIGs indicated they conduct follow-up audits. Audit Planning Process Quality Control Ensure Achievement of Intended Results Disagreements with the Agency “We do not close audit recommendations until a follow-up audit is done. ”

What Did We Learn? Auditing the Resolution Process 2 OIGs indicated they are auditing or have audited the agency’s follow-up process. Audit Oversight of Final Action Objectives: 1) Controls, and 2) Resources Periodically Audit “Follow-Up System as a Whole”

What Did We Learn? Strong Working Relationship 7 OIGS indicated that a close working relationship with their agency is critical to the audit follow-up process. Ongoing Dialogue During Audit Frequent Communication During Follow-Up Create Buy-In Create Ownership Trust Tone at the Top

What Did We Learn? Multiple Levels of Recommendations 6 OIGs indicated that they designate recommendations as either significant or not significant. Significance is Based on a Monetary Threshold Significance is Based on the Level of Impact and the Area of Operations Follow-Up Audits for Unimplemented Significant Recommendations Limit Corrective Action Review to Significant Recommendations

What Did We Learn? Closing Unaddressed Recommendations Agency Formally Accepts Risk Disagree with Agency Follow-up Official Close Recommendations Deemed Not Significant; Keep Significant Recommendations Open Indefinitely

What Did We Learn? Recommendations Open Indefinitely Keep Recommendations Open Indefinitely Additional Action Needed Additional Documentation Needed 7 Year Sunset Date Issue an Updated Report

Audit Follow-Up Models The Cooperative Audit Resolution and Oversight Initiative (CAROI) Department of Education Principles Communication Collaboration Trust Understanding Resolution AGA Workgroup Guide to Improving Program Performance and Accountability Through Cooperative Audit Resolution and Oversight Expands the CAROI process to make it applicable to all federal and state agencies and programs. Sharing CAROI Information: http: //www 2. ed. gov/policy/gen/guid/caroi/index. html Guide to Improving Program Performance and Accountability Through Cooperative Audit Resolution and Oversight: https: //www. agacgfm. org/AGA/Tools. Resources/documents/CAROI. pdf

Audit Follow-Up Models NSF Stewardship Collaborative Comprised of OIG and Agency Senior Managers Overall Goal: Improve Stewardship of Taxpayers’ Investment in Science, Engineering, and Education Monitor and Improve the Audit Resolution Process Address Emerging and Outstanding Issues Address Issues Impacting OIG and Agency Relationship Membership: Balances Expertise with Expeditious Decision Making

What’s Next? Half- or Full-Day Roundtable/Conference Organize Date/Time Solicit Topic Ideas Identify Speakers Questions? Topic Ideas?