The IAEA Integrated Regulatory Review Service IRRS IRRS

The IAEA Integrated Regulatory Review Service (IRRS) IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective Part 1: Initial IRRS Missions Moscow, 09 - 11 December 2014 Ahmad Al Khatibeh Section Head, Regulatory Infrastructure and Transport Safety Section, Division of Radiation, Transport and Waste Safety (NSRW) IAEA International Atomic Energy Agency

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Introduction • IRRS programme first implemented in 2006 and by December 2013, more than 50 missions completed. • Given this milestone, a comprehensive analysis of completed IRRS mission reports from the radiation safety perspective was commissioned to identify opportunities for continued improvement of the IRRS peer review programme. • The analysis report; “IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective” has been distributed at this workshop and this presentation provides a summary of the report’s key findings and proposals. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 4 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Introduction • The consultancy comprised experienced international experts from Canada, Finland, Nigeria, Sweden, Ukraine and USA. • The outputs of the consultancy would be: • an analysis report drawing conclusions in two main areas: 1. The IRRS process; analysing strengths and potential weaknesses, and 2. Safety issues notably common to the majority of Member States. • and with regards to follow up missions: 1. The impact of the IRRS on improving regulatory effectiveness in MS 2. IRRS report recommendations implemented and those not, with possible reasons. 3. Possible impact on the safety standards. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 5 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Introduction The IRRS programme has been very successful, with more than 50 missions and follow-up missions at the end of 2013 and a heavy programme into the future. However, as the next slide illustrates, demand for IRRS varies significantly globally, with an ongoing high demand in Europe, low in Africa and very low in Asia, the Middle East and Latin America. The reasons for this significant regional variability may be a topic for discussion at this workshop, since it would appear the IRRS programme is not equally attractive to all Member States. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 7 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END June 2006 to November 2014: Global Distribution of IRRS Missions and Follow-Ups Implementation is uneven regionally: • High in Europe • Low in Africa • Very low in Asia and Latin America IAEA MS: 162 (February 2014) WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 8 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Assessment of IRRS Programme Performance The Analysis of IRRS reports highlights many programme strengths: • Provides Host with an objective, evidence-based status of the legal, governmental and regulatory framework and regulatory infrastructure relative to applicable IAEA requirements. • Provides the basis for an improvement programme. • Promotes and strengthens implementation of internationally agreed standards for safety and provides input to the review and revision of IAEA Safety Standards. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 9 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Assessment of IRRS Programme Performance • Provides opportunity to exchange national experiences and practices. • Promotes harmonisation, recognises regional and national differences and the unique approaches that contribute to safety. • Relative to 2006 -2010, IRRS reports between 2011 and 2013 reveal improved objective reporting and continuous improvement in evidence -gathering, report drafting and validation of host country selfassessment findings. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 10 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Assessment of IRRS Programme Performance Despite the foregoing evidence of IRRS effectiveness, the mission reports analysed reveal numerous examples of inconsistencies in recommendations and variable interpretation of the requirements, from which lessons can be learned. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 11 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Recurring safety issues The analysis reveals certain IRRS recommendations and suggestions frequently recur: • Some issues recur throughout the analysis period of 2006 to 2013 • Some issues are no longer seen after 2010, but conversely, in the period since 2010 there are new recurring issues. IRRS Guidelines have been revised and GSR Parts 1 and 3 were published, superseding earlier editions. These changes may have influenced the above. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 12 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Recurring safety issues The existence of recurring issues suggests either: • that countries put insufficient priority on their improvement and / or; • the applicable IAEA requirements are not realistically achievable in the timescales set. Significantly, it is possible too, that issues of understanding and interpretation of the requirements (by both reviewers and hosts) plays a part. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 13 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Recurring safety issues The IRRS process encourages reviewers to develop observations (recommendations, suggestions and good practice) specific to circumstances in the host country. This is fundamental to the concept of peer review, but leads to differing perspectives, even where addressing common and recurring issues. The consequence of this approach is variability in emphasis and interpretation of compliance with specific requirements IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 14 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Recurring safety issues The foregoing suggests the need for discipline in drafting IRRS reports, such that: • conclusions reached on recurring issues are consistent across every mission; • similar observations in each country use the same bases; and • IRRS observations directly contribute to the continuous improvement of IAEA safety standards and the global exchange of regulatory experience IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 15 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS as an integrated service: Addressing the full scope of the national legal, governmental and regulatory framework for nuclear and radiation safety. IRRS offers an integrated approach to the review of national frameworks for safety, thereby including in its scope: • every organisation having regulatory responsibilities and functions; • all facilities and activities involving ionising radiation above exemption levels. This analysis finds that since the inception of the IRRS programme in 2006, only a small minority of missions have fully met this objective. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 16 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS as an integrated service: Addressing the full scope of the national legal, governmental and regulatory framework for nuclear and radiation safety. To illustrate the issue behind this important finding of the analysis: • An IAEA NSTC joint working group found that 42% of Member States do not have adequate patient protection programmes. • More than 50% have yet to implement adequate waste management and environmental protection programmes. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 17 Contents

Average Status of TSAs 1 - 5 for MS in all Regions IAEA International Atomic Energy Agency

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS as an integrated service: The host country commonly excludes certain facilities and activities and usually one regulatory body is involved. Commonly, other national agencies assigned regulatory functions and responsibilities are not consulted. IRRS Guidelines state; “If the state wishes to exclude specific facilities and activities, a detailed explanation for the exclusion of these facilities or activities should be provided. The mission report will reflect this in both the title and content of the relevant report sections, including the Executive Summary”. The analysis indicates this Guidelines requirement has rarely been met (0 nly Switzerland to date) IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 19 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS as an integrated service: Thus, the analysis team’s first ‘lesson-learned’ … Lesson Learned LL 5 Lessons-Learned: IRRS should include every organisation having significant relevant regulatory responsibilities and functions and review the oversight of all facilities and activities involving ionising radiation above exemption levels. LL 5 Limiting the scope of IRRS missions or excluding national agencies assigned regulatory responsibilities and functions as defined by GSR Part-1, introduces potential to miss essential findings associated with nuclear and radiation safety. IRRS Guidelines emphasis on national framework for safety is not being met. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 20 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson Learned LL 5 Lessons-Learned: IRRS should include every organisation having significant relevant regulatory responsibilities and functions and review the oversight of all facilities and LL 5 activities involving ionising radiation above exemption levels. LL 5: 1 st Proposal for Consideration Proposal: IRRS should review full extent of national, legal and governmental framework and regulatory infrastructure for safety, directly involving all organisations having oversight of all radiation facilities and activities in the country. P 7 IAEA should consider revising IRRS modular structure making optional Module 11 a ‘core’ module or explicitly incorporating all elements of Module 11 into core modules to ensure full extent of Host’s national framework for safety is addressed. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 21 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson Learned LL 5 Lessons-Learned: IRRS should include every organisation having significant relevant regulatory responsibilities and functions and review the oversight of all facilities and LL 5 activities involving ionising radiation above exemption levels. LL 5: 2 nd Proposal for Consideration Proposal: Where the Host Country requests an integrated review as proposed above, then all agencies collectively comprising the national regulatory body and assigned responsibility and providing the full scope of P 8 regulatory responsibilities and functions (as defined in GSR Part-1) should be invited to participate in the review, with their various identities and roles confirmed at the preparatory meeting. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 22 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS and the Safety Standards: Analysis suggests IRRS Team Members and Host counterparts may not always share common understanding / interpretation of the safety requirements and their application. This is a complex issue that calls for a training programme to build better understanding of IAEA safety requirements, with expectations of what must be evidentially in place to demonstrate compliance. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 24 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS and the Safety Standards: ‘Lesson-learned LL 6’ IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 25 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson Learned LL 6 Lessons Learned: IRRS Team Members and Host Country counterparts may not always have a common understanding of the relevant IAEA requirements and their proper application. LL 6 In a few instances, recommendations and suggestions are not based on the IAEA Safety Standards. LL 6: 1 st Proposal for Consideration Proposal: IAEA should develop mechanisms to facilitate common understanding of IAEA Safety Standards and relevant guidance throughout IRRS process. This could include training programmes, improved templates P 10 for self-assessment (including much improved ‘expectations’) and for drafting IRRS reports and developing improved guidance for preparation of advanced reference material (ARM). IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 26 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson Learned LL 6 Lessons Learned: IRRS Team Members and Host Country counterparts may not always have a common understanding of the relevant IAEA requirements and their proper application. LL 6 In a few instances, recommendations and suggestions are not based on the IAEA Safety Standards. LL 6: 2 nd Proposal for Consideration Proposal: IAEA should consider on-line training for both IRRS reviewers and Host Country addressing IRRS process, including drafting technical notes, formulating observations and selecting bases. P 11 Before embarking on a pre-IRRS self-assessment and preparation of ARM, Host Country should have access to IAEA online training tools to facilitate training on interpretation and application of the IAEA Safety Standards. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 27 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS and the Safety Standards: ‘Lesson-learned LL 7’ IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 28 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson Learned LL 7 Lessons Learned: Analysis reveals wide-ranging inconsistencies in application and interpretation of standards and some recommendations and suggestions LL 7 have no basis in the Standards. LL 7: Proposal for Consideration Proposal: The IAEA Deputy Coordinator’s role in ensuring correct use of IAEA requirements should be reinforced. P 15 IRRS Guidelines should assign to the Deputy Coordinator, responsibility for alerting IRRS reviewers and counterparts where misunderstanding / misinterpretation of a requirement is evident. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 29 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS and the Safety Standards: ‘Lesson-learned LL 9’ IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 30 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson-Learned LL 9 Lessons Learned: No procedure for feedback from IRRS to support development and revision of IAEA safety standards. IRRS Team members accumulate considerable experience of application of IAEA safety standards. LL 9 Post-IRRS, this asset may not be optimally utilised in providing feedback to improve standards, developing guidance and setting expectations with regard to their effective application. LL 9: 1 st Proposal for Consideration P 17 Proposal: The IAEA Deputy Coordinator (DC) should be assigned responsibility for gathering feedback on the application of IAEA safety standards and formally reporting it to IAEA. Special attention should be paid to requirements that appear to be interpreted variably. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 31 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS as peer review: IRRS Guidelines state a peer review should be a validation of the host country’s self-assessment. . . … however, the analysis reveals little evidence that the Host selfassessment is actively referenced by reviewers during the mission. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 33 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS as peer review: ‘Lesson-learned LL 10’ IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 34 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson-Learned LL 10 Lessons Learned: Analysis of IRRS reports reveals little evidence that Host self-assessment is actively referenced by reviewers and few references to LL 10 non-compliances identified during self-assessment already being addressed in Host’s action plan. LL 10: 1 st Proposal for Consideration Proposal: The IRRS Team’s initial impressions of selfassessment and action plan should be circulated to reviewers P 19 and the host immediately pre-mission and form an annex to the IRRS report. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 35 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson-Learned LL 10 Lessons Learned: Analysis of IRRS reports reveals little evidence that Host self-assessment is actively referenced by reviewers and few references to LL 10 non-compliances identified during self-assessment already being addressed in Host’s action plan. LL 10: 2 nd Proposal for Consideration Proposal: IRRS report should clearly document validation (or otherwise) of host’s self-assessment as a preface to the reviewer’s findings. P 20 Where country has identified issues through self-assessment, this should be acknowledged in the IRRS report just as issues not revealed in the selfassessment are addressed in the IRRS report. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 36 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS as peer review: IRRS Guidelines: Peer review should be an exchange of professional knowledge and experience. Analysis reveals that mission schedules typically assign only 2 -4 days for interviews and observations (which include valuable professional discussions) whereas 8 -10 days are allotted to clarification of findings and report writing. Hence, little time for mutual learning and exchange of regulatory experience. Strategies and processes to reduce report writing hours would benefit all parties - releasing mission time for exchange of experience, in the spirit of peer review. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 37 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS as peer review: ‘Lesson-learned LL 11’ IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 38 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson-Learned LL 11 Lessons Learned: Time available for interviews and observation is increasingly constrained, resulting in less time for mutual learning and the LL 11 exchange of regulatory experience. LL 11: Proposal for Consideration Proposal: Having read Host ARM, IRRS Team members should pre-draft their sections of the IRRS report, using an IRRS report template. These ‘initial impressions’ should be shared with other Team members (and host) P 22 at least one week before the mission, accompanied by a list of issues of concern, where specific interviews and observations appear necessary. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 39 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS as peer review: Comprehensive, accurate, but more succinct and consistently structured Host Country Advance Reference Material (ARM) including the selfassessment, may free up time for the exchange of experience during an IRRS mission. This goal may be facilitated by offering training, guidance and tools (such as ARM templates) so that both Host counterparts and IRRS reviewers can make more effective use of this comprehensive evidence of the current status of the Host’s infrastructure for safety. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 40 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS as peer review: ‘Lesson-learned LL 12’ IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 41 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson-Learned LL 12 Lessons Learned: More succinct and consistently structured Host Country Advance Reference Material (ARM) including self-assessment may improve LL 12 IRRS mission effectiveness by facilitating discussions and freeing up mission time for exchange of experience. LL 12: Proposal for Consideration Proposal: IAEA should develop a template for consistent and straightforward compilation of ARM, possibly based on SARIS self. P 23 assessment report. At the same time, SARIS question-sets should be reviewed and abbreviated. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 42 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS recommendations and suggestions: Analysis indicates that IRRS recommendations and suggestions are not always sufficiently specific or realistic to result in tangible improvements correlated with the Host’s action plan. The expectation is that the Host action plan will be revised to accommodate IRRS recommendations or suggestions. For this to be effective, recommendations and suggestions must: • follow the SMART principle (i. e. Specific, Measurable, Achievable, Realistic and Timely); and • recognise the circumstances of the host country and realism therefore, of its action plan for improvement. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 43 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS recommendations and suggestions: ‘Lesson-learned LL 13’ IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 44 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson-Learned LL 13 Lessons Learned: Recommendations and suggestions in IRRS reports are not always specific, realistic or designed to result in tangible improvements LL 13 correlated with the Host Country’s action plan. LL 13: 1 st Proposal for Consideration Proposal: The “SMART” test should be formally applied when developing recommendations and suggestions (i. e. IRRS Team Leader working with P 24 Team Coordinator, should ensure they are Specific, Measurable, Achievable, Realistic and Timely). IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 45 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson-Learned LL 13 Lessons Learned: Recommendations and suggestions in IRRS reports are not always specific, realistic or designed to result in tangible improvements LL 13 correlated with the Host Country’s action plan. LL 13: 2 nd Proposal for Consideration Proposal: To the extent possible, recommendations and suggestions should be correlated with the Host Country’s existing action plan and refined, as P 25 appropriate to be realistically in accordance with the action plan. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 46 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson-Learned LL 13 Lessons Learned: Recommendations and suggestions in IRRS reports are not always specific, realistic or designed to result in tangible improvements LL 13 correlated with the Host Country’s action plan. LL 13: 3 rd Proposal for Consideration Proposal: Recommendations that may take a long time to be implemented or have a high priority, should include interim goals set out in the IRRS P 26 report and in due course, by agreement, transferred to the Host Country’s updated action plan. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 47 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS reporting of good practice: Although the term ‘good practice’ is unambiguously defined in the IRRS Guidelines, the analysis suggests the concept is over-used and frequently misapplied. The analysis also finds value in reporting notable practice ‘graded’ according to the circumstances of the Host Country. Thus, a two-level approach to reporting interesting, unique or otherwise worthy practices may be beneficial: • first level of ‘notable practice’ reserved for practices which are effective in the circumstances of the Host Country but do not fully meet the criteria for an IRRS good practice; and • second level of ‘good practice’ remaining as currently defined in the IRRS Guidelines. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 49 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS reporting of good practice: The analysis finds little evidence that good practice identified during IRRS is effectively distributed to the global community in manner that encourages its adoption. A good practice is notable only as long as it is worthy of sharing with the wider community. Once a good practice is established or widely implemented, it represents the status quo. Analysis reveals that essentially the same (or similar) ‘good practices’ are repeatedly reported over a long period. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 50 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS reporting of good practice: ‘Lesson-learned LL 15’ IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 51 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson-Learned LL 15 Lessons Learned: The concept and definition of a good practice is not LL 15 applied rigorously. LL 15: Proposal for Consideration Proposal: IAEA should consider the following: P 29 • Require all elements of a good practice be comprehensively addressed in the IRRS report. Unless all four criteria clearly met, concept of good practice should not be applicable; • Define and encourage reporting of a second level of practice (e. g. ‘Notable Practice’) which acknowledges actions and processes notable in the circumstances of the Host but do not meet IRRS Guidelines criteria for good practice. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 52 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS reporting of good practice: ‘Lesson-learned LL 16’ IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 53 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson-Learned LL 16 Lessons Learned: Dissemination of good practices occurs, but there is little LL 16 evidence of their effective implementation worldwide. LL 16: Proposal for Consideration Proposal: IAEA should consider the following: P 30 • A good practice should be documented only where it has not been seen before. This suggests development of an IAEA searchable database of good practice, easily accessible to IRRS reviewers and the global community. MS should be encouraged to report to the database if (and how) they adopt the good practices they find there. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 54 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS review completeness: Analysis shows that partial application of an IAEA requirement together with inconsistent or variable interpretation of the requirement occasionally leads to an incomplete review with recommendations and/or suggestions that are not comprehensive. In some cases, a requirement is addressed in various separate parts of the IRRS report without due cross-referencing, so that the extent of compliance is not necessarily clear to the reader. Without a structured, explicit reporting of compliance with every element of a requirement, it is difficult to determine whether the unreported parts of the requirement are in fact, implemented by the Host Country. Not reporting (i. e. keeping silent on) elements of a requirement cannot be accepted as tacit validation of compliance with the requirement as a whole. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 55 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS review completeness: ‘Lesson-learned LL 17’ IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 56 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson-Learned LL 17 Lessons Learned: Partial application of IAEA requirements occasionally leads LL 17 to an incomplete review because only the implemented elements are addressed during the mission. P 31 LL 17: 1 st Proposal for Consideration Proposal: IRRS report template should explicitly ensure that every element of each requirement is addressed in full. For each element of each IAEA requirement clear statements of compliance should be included for all regulated facilities and activities of the national framework for safety. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 57 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson-Learned LL 17 Lessons Learned: Partial application of IAEA requirements occasionally leads LL 17 to an incomplete review because only the implemented elements are addressed during the mission. LL 17: 2 nd Proposal for Consideration Proposal: Where it is appropriate to use only part of a requirement as the basis for a recommendation or suggestion care must be taken to maintain the integrity and original intent of the requirement as a whole. P 32 This may be achieved if the over-arching requirement (printed in bold font in the more recent IAEA safety standards) is reproduced in full in IRRS report, together with relevant part of requirement (i. e. ‘shall’ statement) being used as the basis. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 58 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS review completeness: Analysis shows that non-compliances noted in the body text of IRRS reports, occasionally do not result in a subsequent recommendation. The issue is often reflected in the text with terms such as; ‘concern’ or ‘advice’. The IRRS Guidelines clearly require that all identified non-compliances must lead to a recommendation. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 59 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS review completeness: ‘Lesson-learned LL 18’ IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 60 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson-Learned LL 18 Lessons Learned: Non-compliances are occasionally noted in the body text of IRRS reports which do not result in a subsequent recommendation. LL 18: Proposal for Consideration P 33 Proposal: IRRS reviewers should be reminded of the IRRS Guidelines statement that all identified non-compliances must lead to a recommendation. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 61 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS review consistency: Analysis reveals that observations in any one IRRS report are occasionally inconsistent with each other. In some cases there is a statement of compliance in the findings and elsewhere in the same report, a recommendation or suggestion indicating the opposite. Observations do not always appear in the most appropriate part of the report. This may be the consequence of the current modular structure of the IRRS. All modules of the IRRS are interdependent. As such, it is important that findings or recommendations and suggestions in any one section of the report are drafted fully taking into account other areas under review. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 62 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS review consistency: ‘Lesson-learned LL 19’ IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 63 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson-Learned LL 19 P 36 Lessons Learned: Observations in the same IRRS report are occasionally inconsistent with each other, particularly where they’re made in another module. Additionally, in some IRRS reports analysed, observations do not appear in the most appropriate part of the report. LL 19: 1 st Proposal for Consideration Proposal: Guidance (in both the self-assessment and IRRS report template) should highlight those requirements that typically have an impact across several modules and indicate how they are linked. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 64 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Lesson-Learned LL 19 P 37 Lessons Learned: Observations in the same IRRS report are occasionally inconsistent with each other, particularly where they’re made in another module. Additionally, in some IRRS reports analysed, observations do not appear in the most appropriate part of the report. LL 19: 2 nd Proposal for Consideration Proposal: Observations should be explicitly directed to the organisation responsible for their implementation (e. g. government, regulatory body, other agency etc. ). IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 65 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS Analysis - Conclusions: Based on this analysis of 50 IRRS missions (including 7 follow-up missions) during the period 2006 to 2013, the following conclusions may be drawn: • The IRRS is established as the prime peer review of national frameworks for nuclear and radiation safety. • IRRS provides Member States with the opportunity to identify and prioritise necessary improvements to their national regulatory infrastructures for nuclear, radiation, radioactive waste and transport safety. • The IRRS has given regulators subject to IRRS and government officers of participating States access to international advice and the exchange of regulatory knowledge and experience. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 67 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS Analysis - Conclusions: • The IRRS is an integrated service. The term ‘integrated’ does not refer to the integration of IAEA services but specifically to the integration of all State organisations, functions and activities that comprise the national, legal and governmental framework and infrastructure for safety. • Thus, IRRS should address the Host Country’s regulatory framework for protecting the public, patients, workers and the environment from radiation risks associated with all regulated facilities and activities in the country. • In consequence, all national agencies that comprise the national regulatory body for all facilities and activities in the country should expect to participate in IRRS. By the IRRS Guidelines definition, an IAEA review of limited scope not addressing all elements of the State’s national, legal and governmental framework and infrastructure for safety should not be considered as an IRRS. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 68 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS Analysis - Conclusions: • Where the State has good cause to ask for a limited scope the current IRRS Guidelines call for a written justification. • The IRRS is under-utilised as a source of objective feedback for the improvement of IAEA safety standards. • The self-assessment and experience exchange processes would benefit from improvement. • Since the IRRS follow-up process appears not to be functioning in accordance with the Guidelines, alternative progress monitoring methodologies may be needed with greater flexibility according to Host needs and circumstances. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 69 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END IRRS Analysis - Conclusions: • An ongoing programme of IRRS lessons-learned workshops with dissemination of lessons-learned, reporting IRRS results at international undertakings and conducting periodical analysis of the IRRS process would benefit the global safety regime. • Since IRRS is a dynamic process, its efficiency and effectiveness can be further improved by using experience gained from previous missions, particularly in the following areas: • Drafting and global dissemination of recommendations, suggestions and good practices. • IRRS report completeness in terms of comprehensive review of all relevant IAEA standards and guidance. • Consistency in the structure, style, content and emphasis of IRRS reports. IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 70 Contents

IRRS Missions 2006 -2013: Analysis from a Radiation Safety Perspective END Thank you IAEA WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09 -11 December 2014 71 Contents