Quality Assurance and Peer Review Professor Alan Paterson

Quality Assurance and Peer Review Professor Alan Paterson Centre for Professional Legal Studies, University of Strathclyde, Scotland

International Developments UN Principles and Guidelines on Legal Aid (2012) Principle 13 “All States to put in place mechanisms to ensure that all legal aid providers possess the education, training, skills and experience that are commensurate with the nature of their work “ • Criteria for accreditation of the legal aid providers (Guidelines 15); • Quality standards for legal aid providers and paralegal (Guidelines 16 and 14); • Adequate training, and supervision by qualified lawyers (Guideline 14 – mentioned in the context of paralegals only, though this would probably be appropriate for lawyers as well); • Monitoring and evaluation mechanisms to assess the quality of the services provided by lawyers and paralegals (Guidelines 16 and 14)

International Developments II EU Commission Recommendation of 27 November 2013 on the right to legal aid for suspects or accused persons in criminal proceedings Section 3 on effectiveness and quality of legal aid establishes that “Legal assistance provided under legal aid schemes should be of high quality in order to ensure the fairness of proceedings. To this end, systems to ensure the quality of legal aid lawyers should be in place in all Member States. ” This includes accreditation of legal aid providers and appropriate training; appointment of lawyers while taking into account the wishes of the accused and allowing accused to choose a provider from a list; and replacement of lawyers who fail to provide adequate legal assistance. [ Input & Structure ] EU Directive 2016/1919 of the European Parliament and of the Council of 26 October 2016 on legal aid for suspects and accused persons in criminal proceedings and for requested persons in European arrest warrant proceedings, establishes in Article 7 ‘Quality of legal aid services and training’: 1. Member States shall take necessary measures, including with regard to funding, to ensure that: (a) there is an effective legal aid system that is of an adequate quality; and (b) legal aid services are of a quality adequate to safeguard the fairness of the proceedings, with due respect for the independence of the legal profession. [ Input, Structure, Process and Outcome ]

International Developments III UN Global Study findings Improving quality of legal aid services was identified as the number one priority for Member States in the Global Study on Legal Aid (UNODC, 2016) Monitoring/ Evaluation mechanism of Quality of legal aid? 57% of responding states indicated that the mechanism used to assess quality was client complaints. Only 26% reported taking any proactive steps to assess quality – the most frequent measure being client satisfaction surveys. Publication of the UNODC Handbook on Ensuring Quality of Legal Aid Services in Criminal Justice Processes (2019 )

Ø Global Study: Most pressing technical assistance need cited was “development of quality criteria for legal aid providers” Ø To respond to this need, as a first step UNODC aided by Miri Sharon and Alan Paterson with 50 other experts from all regions developed a new Handbook to provide an overview and guide on measures to constantly improve and monitor the quality of criminal legal aid services

What we know about what is out there ? Nikartas and Limante: Tools and Criteria for Measuring Legal Aid Quality: Guidelines for EU Member States (2018) Accreditation and Training Model clients Complaints Client satisfaction Peer Review

Peer Review “The evaluation of the service provided against specified criteria and levels of performance by an independent lawyer with significant current or recent practical experience in the areas of law being reviewed”

The Spread of Peer Review England Wales, Scotland, Canada (Ontario), Finland, Georgia, Moldova, South Africa, Chile, China, New Zealand, The Netherlands, Australia and Ukraine? ? 9

The aim of Peer Review in Scotland • To assess the quality of the outcomes achieved by legal aid providers? • To assess the overall quality of performance by legal aid providers? • To exclude poor performers from the list of legal aid providers? • To attain continuous improvement by providers? • To change the culture of legal aid providers?

Scottish Peer Reviewers • Appointed by panel of laypersons and lawyers after an open recruitment process. • Peer reviewers are experienced lawyers. • Peer reviewers receive training for 3 to 4 days on reviewing and marking files according to the criteria. • Training fosters consistent application and interpretation of the criteria. • Once accepted as peer reviewers, they have to go through a peer review themselves. • Cannot review a competitor firm or one they have links to.

Peer Review Cases and Criteria - CRIMINAL • 8 random files for each lawyer are assessed against published peer review criteria for less serious criminal cases (e. g. assault), serious criminal cases (robbery) and appeals. Criteria include: • Whether the lawyer made an effective application for bail, • Whether special defences were considered, • Adequate preparation for trial, • Advice given to the client on how to plea, • Advice given to client about sentence, • Advice give to client about appealing, Criteria are client centred

The Peer Review process Ø Establish Quality Assurance Committee ( QAC) – partnership of legal profession and legal aid authority. Ø Administrator on instructions of QAC selects firms to be reviewed at start of cycle of reviews Ø Administrator using database identifies 8 closed criminal files chosen at random for each lawyer stratified to reflect the areas of work in which the lawyer practises Ø Administrator writes to firm asking for files and sends to peer reviewer. (In 25% of cases, the administrator sends files to a second reviewer for an additional review without seeing the first reviewer’s report. Ø Peer reviewer (s) send detailed report to QAC

Quality Assurance Scheme Criminal Peer Reviews

Results from peer reviews • Typical causes of fails: • Delays in taking action or applying for legal aid • Poor communication with clients e. g. not informed of the evidence against them, poor initial instructions, no discussions of early guilty plea, no record of meetings with client, poor file notes of phone call or interviews • Experts not instructed, failure to obtain disclosure of evidence from the Prosecution, possible abuses of the curt process • Failure to comply with legal aid procedures.

Costs and Benefits • Scots cost of QA in 2013/14 was 220, 000 Euros (2. 3% of Scots expenditure on administration) ( 123 Euros per file) • (Complaints 2016/17 2, 582 Euros per case).

Driving up Standards? • Evidence that peer review is driving up standards. • Fail rate on routine reviews has reduced in criminal cases. • High subsequent pass rate of original failed practitioners • Numbers of Final Reviews and Special Reviews has fallen compared with civil. • Numbers of solicitors achieving the highest marking has increased. • Survey of solicitors in 2013 showed that 84% of respondents had a positive or neutral opinion on whether the QA scheme was an effective way of ensuring quality. • A few solicitors who were doing very small amounts of legal aid, or had failed reviews, withdrew from our register of suppliers. The great majority of lawyers/firms that fail their initial review pass the next time round

Case Report Analysis Total No per criteria 1 2 7(a) 1 43 2(a) 2(b) 58 290 3(a) 3(b) 3(c) 3(d) 237 638 25 10 4 21 196 5 53 6 32 7(b) 6 66 8 9(a) 9(b) 25 199 10 11 98 227 167 12 13(a) 13(b) 13 83 14 14 134 5920 6004 5646 4547 4680 6385 5647 4215 5359 5174 3431 1649 3697 2326 4490 5068 5514 5301 2153 4092 1433 3702 3 548 298 14 C 170 308 494 N/A 2(c) 19 31 33 122 53 759 1107 133 256 1125 147 68 156 415 515 85 72 105 73 63 57 139 44 22 196 221 45 41 226 99 890 2237 626 901 3023 4939 2695 4113 1741 369 275 415 96 8 22 10 188 79 445 85 583 94 201 973 591 616 4360 2070 5146 2266

Running a pilot programme 1) Appoint a body to oversee the pilot programme partnership 2) Identify the standards of criminal practice 3) From these derive the peer review criteria for assessing process and outcome 4) Identify the assessment system 5) Run a selection competition for peer reviewers 6) Decide on what should be assessed – performance in court, files 7) Decide on whether additional factors should be taken into account e. g. client satisfaction; complaints