Implementation of the European Statistics Code of Practice

Implementation of the European Statistics Code of Practice in Statistics Portugal « Maria João Zilhão Planning and Quality Control Unit High Level Seminar “Quality Matters in Statistics” 21 -23 June, Athens, Greece «

Code of Practice To guaranty: The trustworthiness of a Statistical organization Working methods and its results Establish the conduct of the organization

How important is the Code of practice European Statistics Code of Practice 15 principles: Institutional environment Statistical Processes Statisticians Statistical Output Managers Code of Practice Users «

Integrity Confidentiali ty Trust To transmit …. Objectivity in production processes impartiality (…)

European Statistics Code of Practice How important are the Operational aspects? Systematization of a code Basic guidelines of a Statistical organization Clarification to the users Clear message of integrity Deployment into in-house documents / Policies / Procedures «

Statistics Portugal example (i) Levels of documentation Mission, Vision and Values of the Organization ESS Code of Practice Policies Operational procedures and tasks Internal level

Statistics Portugal example (ii) 1. Principles for the dissemination policy of Official Statistics Dissemination Policy 2. Release of official statistics 3. Accessibility of official statistics 4. Monitoring and assessment of the dissemination policy Revision Policy 1. Key factors underlying a revision 2. Typology of revisions 3. Dimensions of revisions analisys 4. General and operational principles of the revisions policy

Quality Management System at Statistics Portugal Quality Documentation System ISO 19011 ISO 10013 Departmen ts Customer Satisfaction Surveys Performan ce indicators Departmen ts Documentin g Processes Internal Audits External Audits Quality Management Unit Departmen ts Suggestions / Complaints Quatily reports / Method. Doc. P D A C

European Statistics Code of Practice – Preparation of the Peer Review Opportunity to reflect upon the principles, and Statistics Portugal Performance Systematize the related documentation and relevant information § Internal team – different areas § Involvement of external stakeholders (other producers; respondents; users); § Identification of good practices; § Faced as an external audit – improved in-house process. «

References for auditing ISO Norms: • 9000: 2000…Implementation of Quality Management Systems • 19011 -. . Auditing… • 10013 - …Documentation systems

Preconditions Top Management support A Clear communication and involvement of the staff Proper preparation! Documentation System The results of auditing and self-assessments must be consequent

Documentation System Documenting processes ISO 10013 Procedures handbooks Documentation on survey processes Process design Definition of different phases of the process Definition of responsibilities Definition of minimum documentation Handbooks are produced by pluridisciplinary teams

Statistical Production Handbook Procedures Phases I. Conception II. Operation Processes I. 1. Viability Study II. 1. Planing and preparation of data collection I. 2. Methodological Study II. 2. Data collection I. 3. Technical approval II. 3. Data treatment and analysis III. Dissemination IV. Evaluation Process III. 1. Dissemination Note: Each Process is composed by sub-processes and tasks Process IV. 1. Evaluation Audits

Internal Quality Audits at Statistics Portugal (I) Objectives To verify the degree of procedures implementation (handbooks procedures) To Identify improvement opportunities To check the internal articulation of surveys To Improve survey documentation P D A C

Internal Quality Audits at Statistics Portugal (II) Statistical Production Handbook Procedures II. Operation I. 1. Viability Study II. 1. Planing and preparation of data collection I. 2. Methodological Study II. 2. Data collection I. 3. Technical approval II. 3. Data treatment and analysis III. Dissemination III. 1. Dissemination IV. Evaluation IV. 1. Evaluation « I. Conception

Internal Quality Audits at Statistics Portugal – Preparation (III) Training of 14 Auditors (Statisticians) (ISO 19011) Discussion Top Management Support Internal Procedure on auditing (Clear rules!) Exchange of Information / Strong Communication about auditing, through Top Management, and the Quality Unit

Internal Quality Audits at Statistics Portugal –(IV) Principal rules Top Management defines the Quality/Statistical Audits’ annual plan No one should audit its own activity One Audit team needs 2 or 3 auditors Auditors produce Audit Plans and reports according to Procedures Improvement actions must be carried out by the audited departments Improvement actions must be published and followed by the audit teams

External Quality Audits at Statistics Portugal § § § § Auditors are Peers (from other statistical Offices) Basic rules are the same as for internal auditing Two visits (+/- one week each) Contacts between auditors and audited teams in the mid period Auditors work recommendations with audited teams Results are presented to the board Action plan is made by audited teams and presented to the board

Benefits for Quality Improvement Some NSI’s have experienced Quality Auditing! The identification of improvement opportunities in the audited process The implementation of corrective / preventive actions A Powerfull Tool! The redefinition of procedures Auditing involves people in the quality program (auditors and audited teams) BUT!!!! Good Recommendations - A look from the outside! -May face resistances Identification of good practices that can be spread in the organization Self-assessment and auditing can be combined together A time to re-think and improve! -Time consuming activity - May be dificult to start - Better results in the long run if systematically implemented

Statistics Portugal experience: External Customer Satisfaction Surveys

External Customers Preparation of Matrix Quality dimensions were discussed and defined accordingly to the following structure: Attributes that compose “Quality perception, concerning statistical information: 1. Quality perception, concerning… 1. 1. Statistical information 1. 2. Products Data pertinence 1. 3. Services Credibility 2. Value for money Actuality 3. Global image Data accessibility 4. Expectations Metadata accessibility 5. Loyalty/fidelity Coherence and Comparability Geographical disaggregation Synthetic data Data punctuality «

External Customers A Matrix / Table Based on the concepts related to these 5 dimensions the matrix includes questions accordingly to several target groups: We can use also complementary information from database of customers: Examples: ÜLibraries Ü Typology of information requested ÜResearchers Ü Customers typology ÜCentral Public Administration Ü Customers level of experience ÜMedia etc ÜSchools ÜSite «

External Customers Questionnaire structure and scales questionnaires are built in an identical way, having the same structure for different target groups taking into account the different target groups that are to be inquired, questions may vary (although they are linked to the dimensions and its concepts) «

« External Customers Variables 31 qualitative variables divided into 4 groups: § recognised quality of the statistical information (7 queries) - based on the Eurostat quality dimensions § recognised quality of the product (6 queries) § recognised quality of the service (7 queries) § Statistics Portugal perceived image (11 queries) « Response scale «

Data analysis extreme balance responses (EBR) The analysis of the results was displayed by using the “extreme balance responses” (EBR). This aimed to value more the extreme answers and not the middle scores which tend to represent a less expressive dissatisfaction/disagreement or satisfaction/agreement. The attributed ponderations were as follows: EBR = F 1*(-1)+ F 2*(-0, 5)+ F 3*(-0, 25)+ F 4*(0, 25)+ F 5*(0, 5)+ F 6*(1) where FI = Relative frequency of each observed value for each one of the categories I =(1, …, 6). Data became to be referenced in a metric scale: between -1 and +1, where values next to -1 mean full dissatisfaction/disagreement and values near +1 mean full satisfaction/agreement. «

External Customers Permanent: Targets: Ü Libraries (Statistics Portugal and Universities) Ü Website Ü Study visits / Events Ü Post- Service Ü Webinq Occasional: Media (2006) Ü Researchers (2007) Ü Business Associations (2009) Ü Schools (2009 and 2010) Ü To assess quality perceived by external Statistics Portugal costumers Ü Identification and implementation of improvement actions « Aims: Ü

THANK YOU! Maria João Zilhão Statistics Portugal Planing and Quality Control Unit « « mjoao. zilhao@ine. pt