Auditing data management system Bruno Deprez Data Audit

Auditing data management system Bruno Deprez Data Audit officer SPC, Oceanic Fisheries Program

What is data management audit? • Definition: “…a formal, often periodic examination and checking of data or procedures to verify their correctness…” • “Audit” term is often used in financial and accounting sectors – To evaluate organization and people – To validate the official financial results of an organization before publication • Fishing data management audit purpose is different. The final objective is to improve the quality level of data entered in country in order to make the studies based on this data easier and better • To achieve this objective, we need to ensure the correctness and the coverage of the data, throughout systemic and non-systemic audit • This audit is only applied to logsheet data

Objectives of Data Audit • To ensure that data are standardized – Almost guaranteed by the use of TUFMAN system to enter data – Basic checks on the database to make sure it is used properly • To improve the quality and coverage of data available at national level and to the WCPFC – Data quality audit is applied to position/time, catch and effort – Coverage audit tries to identify missing logsheets – Both use extensively the VMS data and samples of paper logsheets • To improve data collection in country – Existence of data registration system – Timeliness and frequency of provision of logsheet data from companies – Level of liaison/cooperation with fishing companies to retrieve logsheets – Existence of a filing system to easily retrieve hard-copy or scanned logsheets

The Data Quality Audit Tool • Stand-alone application to be linked to your Tufman database • Comes with a set of tests separated in three categories : – Position/Time • The tests will verify the accuracy of logsheets times and positions • E. g. Average speed between sets, Missing sets, Overlapping trips, etc – Catch • The tests will verify the accuracy of the catch part of logsheets • E. g. By Catch – Effort • The tests will verify the accuracy of the effort part of logsheets • Discrepancies in number of hooks • Generates pre-formatted Excel reports with test status and list of warning and error database entries • Comes with an embedded GIS option in order to visualize any trip on the map • The list of tests is not static and new ones can be easily added throughout the audit process • Some basic automated correction could be also applied on demand

Example : The average speed between sets

Example of a test: Use of limits The speed limits for a longliner in the tool are set as follows : – Raise a warning if the speed is above 11 kts – Raise an error if the speed is above 15 kts

Example : The average speed between sets

Example of a test: The detail part • 324 knots is obviously wrong • The error is on the longitude entered for Feb 7 th – 01733 E is wrong, it should be something around 173 E (this points ends up in Namibia, Africa) – Need to get back to the paper logsheet and see what was the right value

Use of GIS feature on error • This trip error is obvious • It is not always the case, and viewing the trip itself can help finding some answers • Let’s take a look at this example

Use of GIS feature on error

Use of GIS feature on error

Use of GIS feature on warning • Previous example was from the error report. Let’s take one from the warning report (much less obvious) • The SOLANDER 6 has been steaming at 11 -12 knots between 3 positions, it is above the warning threshold • Here the use of the GIS feature can help to ensure there is a position error on the logsheet

Use of GIS feature on warning

Use of GIS feature with VMS • In order to help the decision to modify or not a point, the VMS reconciliation can be used (when applicable) • This process is explained in details next on this presentation • Its main purpose is to compare the logsheets with the trips generated from VMS and find reconciliations

Use of GIS feature with VMS

Data Collection Audit • During in-country visit, the following aspect of data management will be checked – Data Registration • E. g any potential hard-copy logsheet going missing prior to data entry or submission to regional agency – Hard-copy data management • E. g does the responsibility matrix for hard-copy data management cover all work and match actual responsibilities – Database System • E. g Are there any database post data-entry checking • TUFMAN logsheets will be checked against randomly selected hard-copy logsheets

In-Country Data Audit Output • The output of an audit consists in – The audit conclusion containing the audit results • For each part of the audit (quality, coverage and data collection), the detailed status : FAILED, WARNING or SUCCESS • The global audit status • The summing up of the actions to be undertaken – The details of quality audit • For each quality test (average speed between sets, inland positions, etc) • The warnings and errors to be modified in Tufman – The detail of data collection • Comments about each logsheet manually checked • Comments about how to improve data collection – The detail of logsheet coverages per year, gear and flag • To be generated in Tufman using the new report

Systemic Data Audit • Tufman Reconciliation report is analysed during in-country visit • The same tool is used internally at SPC to compare the same VMS data with the logsheets sent by the member countries to SPC • Every quarter, an export will be sent to countries to highlight gaps in the logsheets received at SPC • Three main reasons for a gap – Data received in-country from fishing company but not sent to SPC – Data not received in-country from fishing company – Wrong VMS trip