Quality Assurance QA For Digital Library Projects Brian

Quality Assurance (QA) For Digital Library Projects Brian Kelly UKOLN B. Kelly@ukoln. ac. uk http: //www. ukoln. ac. uk/qa-focus/ QA Focus – Supporting JISC's Digital Library Programmes 1 This talk will describe the work of the JISCfunded QA Focus team and outlines approaches to QA which can be deployed in Digital Library projects

Contents • Introduction • Background: • The need for QA Focus • Work activities: • Web Accessibility • Providing Advice • Future Work • Metadata Software Development • Service Deployment Toolkit • A QA Methodology • Relevance To You • Conclusions QA Focus – Supporting JISC's Digital Library Programmes 2

About UK Web Focus / UKOLN UK Web Focus: • Funded by JISC since Nov 1996 to advice UK HE (and the FE) on standards and best practices for the Web • Provided by Brian Kelly • Based at UKOLN, a national centre of expertise in digital information management • Located at the University of Bath • Project manager for JISC's QA Focus project Note: Since 1 August 2003 UK Web Focus has been jointly funded by JISC and Resource and now also advises the cultural heritage sector in England Wales QA Focus – Supporting JISC's Digital Library Programmes 3

Introduction Aims Of Today’s Talk • To describe the role of quality assurance in digital library programmes • To describe the approach taken by the QA Focus project • To summarise QA Focus activities to date • To describe future activities • To explore potential for collaboration with projects & services represented here Note that although this talk is based on a JISC-funded project the content is applicable to everyone involved in Web activities – including Webmasters QA Focus development – Supporting JISC's Digital Library Programmes 4

Introduction An Introduction to QA What is Quality? “Quality is the ability of your product to be able to satisfy your users” What is Quality Assurance? “Quality assurance is the process that demonstrates your product is able to satisfy your users” QA Focus – Supporting JISC's Digital Library Programmes 5

Introduction An Introduction to QA Why Is QA Needed Now? In the past JISC had standards-based philosophy but: • Approach of "let a 1, 000 flowers bloom" • Diversification as no killer application then (it could have been Gopher!) Things are different now: • The Web is the killer application • There is now a need for interoperability (not just self-contained independent Web sites) • There is a need for publicly-funded services to be seen to provide value for money QA Focus – Supporting JISC's Digital Library Programmes 6

Introduction Approach Taken Two possible approaches to ensuring compliance with standards and best practices: Enforce • Inspect all project’s work • Strict auditing, with penalties for no-compliance Encourage • Training for project staff • Developmental, explaining reasons for compliance, documenting examples of best practices and providing advice on implementation and monitoring The latter approach is preferable, especially in a HE context QA Focus – Supporting JISC's Digital Library Programmes 7

QA Focus: • Funded by JISC • Currently provided by UKOLN and AHDS • Supports JISC's 5/99, FAIR & X 4 L programmes • Staff: Brian Kelly, Project manager (Marieke Guy, QA Focus officer, UKOLN) Hamish James, AHDS Gareth Knight, QA Focus officer, AHDS • Areas covered: Web Digitisation Metadata Software development Service deployment QA Focus – Supporting JISC's Digital Library Programmes 8

QA For Web 9 QA For Web Sites The issues: • The Web is the main delivery mechanism for projects and services • An increasing awareness of the importance of: Accessibility Use of new devices (PDAs, WAP, e-books, …) Repurposing of Web content (e. g. archiving) • Technologies such as XSLT will support repurposing of valid XML resources But: • Invalid HTML is the norm • Many authoring tools produce poor HTML QA • Focus – Supporting JISC's aware Digital Library Authors aren’t of. Programmes the problems

QA For Web Guidelines We often say: • Open standards are important • HTML, XHTML, CSS, … are important but fail to explain why and how JISC’s QA Focus is addressing such concerns by: • Documenting example of best practices in which projects can share their implementation successes (and difficulties they experienced) • Provide brief advice in specific aspects of the standards and best practices • Surveying its communities to highlight best practices and areas in which improvements can be made • Demonstrating use of testing tools and procedures QA Focus – Supporting JISC's Digital Library Programmes 10

QA For Web Standards & Best Practices Standards For Web: • Use compliant HTML / XHTML • Use CSS Best Practices For Web: • Implement WAI accessibility guidelines • Ensure Web resources are suitable for reuse and repurposing • Where proprietary formats need to be used, flag them and use in most open way QA Focus – Supporting JISC's Digital Library Programmes 11

QA For Web Surveying The Community Surveys of JISC 5/99 (and FAIR & X 4 L) project Web sites have been carried out in order to: • Obtain a profile for the community • Identify examples of best practices • Identify areas in which further advice is needed Surveys included: • HTML & CSS compliance Accessibility • 404 error pages HTTP headers • Repurposing resources QA Focus – Supporting JISC's Digital Library Programmes 12

QA For Web Survey Philosophy The surveys made use of freely-available Web-based tools: • Methodology is open • No software needs to be installed locally (apart from Web browser) • Findings can be reproduced • Latest results can be obtained by clicking on link to testing service The surveys typically examined project entry points and not entire Web site as: • This page has the highest profile • The aim is to validate a methodology which can be deployed by projects themselves, not to test every page on behalf of the projects QA Focus – Supporting JISC's Digital Library Programmes 13

Advice Documentation: Case Studies are being commissioned These are: • Written by projects themselves • Describe the solution adopted to a particular problem • Include details of lessons learnt – not just a press release! QA Focus – Supporting JISC's Digital Library Programmes 14

QA For Web Survey Findings Initial set of findings available from <http: //www. ukoln. ac. uk/qa-focus/surveys/web-10 -2002/> QA Focus – Supporting JISC's Digital Library Programmes 15

QA For Web Providing Motivation We have found evidence of failure to comply with HTML standards There is a need to explain why compliance is important (and avoid the “it’s OK in my browser” argument) and to provide motivation for projects to update their tools, authoring procedures, etc. A further set of surveys look at repurposing of the project Web sites: • Availability of Web sites in the Internet Archive • Ease of making Web sites available on a PDA • Transformation of embedded metadata QA Focus – Supporting JISC's Digital Library Programmes 16

QA For Web Repurposing Resources We examined the Web sites to see if they were available in the Internet Archive and could be transformed into a format for viewing on a PDA small number AAsmall numberof sites could not of. Web sites be transformed. were not in the Analysis of HTTP Internet headers Archive indicated due that to thisthe was due to robots. txt incorrect HTTP file. headers. We will need to in provide advice in this area. QA Focus – Supporting JISC's Digital Library Programmes 17

QA For Web Surveys For FAIR And X 4 L Programmes QA Focus now supports JISC's FAIR and X 4 L programmes The Web site surveys now provides a selfassessment interface http: //www. ukoln. ac. uk/qa-focus/ surveys/fair/web-self-assessment/ We hope that projects will (a) check and, if necessary, fix their home page; (b) use such tools to check/fix other key pages (c) JISC's address underlying QA Focus –and Supporting Digital Library Programmescauses of problems 18

QA For Web A Caveat There a number of limitations to the work we have carried out so far: • Project Web sites have different purposes (information about the project; communications with project partners; project deliverables themselves; etc. ) • Projects have different levels of funding, resources, expertise, etc. • Projects are at different stages of development (and some have finished) The surveys are intended to demonstrate a methodology which projects can use for themselves QA Focus – Supporting JISC's Digital Library Programmes 19

Advice Providing Advice We have: • Surveyed project Web sites and identified areas of lack of compliance with standards & best practices • Shown examples of the potential importance of compliance for repurposing resources In addition we need to provide: • Brief focussed advice on the standards • Information on how to monitor compliance • Case studies on solutions deployed by projects themselves • Guidance on dealing with implementation difficulties and what to do when strict compliance is difficult to achieve QA Focus – Supporting JISC's Digital Library Programmes 20

Advice Documentation: Advice Advisory briefing documents are being produced These are: • Brief, focussed documents • Informed by findings of the surveys QA Focus – Supporting JISC's Digital Library Programmes 21

Tools Putting it Together Based on surveys we find project Web sites which have invalid HTML and CSS or other examples of failure to implement best practices So we: • Explain why projects should follow best practices • Provide case studies on approaches by projects But how do we ensure projects : • Check Web sites for themselves • Have systematic checking procedures There is a need to describe: • Tools & architectures for projects for check compliance with standards & best practices • Systematic QA procedures QA Focus – Supporting JISC's Digital Library Programmes 22

Tools Example - Link Checking Tools We should all be familiar with link checking tools. But: • Do such tools check all types of links (e. g. What about links to external style sheet files, Java. Script files, etc. )? • Are such tools integrated with normal working practices? QA Focus – Supporting JISC's Digital Library Programmes 23

Describing Tools and Approaches Approach is documented at <http: //www. ukoln. ac. uk/qa-focus/ documents/papers/iadis-2003/> As an example of approach which provides easier access to testing tools, see the , tools approach deployed on UKOLN Web site. This approach: • Applies to all resources on Web site • Covers HTML and CSS validation and various other tests • Some recursive apps (, rvalidate) • Easily implemented with single line redirect QA Focus – Supporting JISC's Digital Library Programmes 24

Next Steps Extended Coverage We are addressing other areas besides the Web (and digitisation) including: Metadata Software development Deployment into service Moving On From Automated Testing The initial work made use of automated testing tools: Can be used remotely Objective Applicable across all projects We have started work on QA procedures in areas which are not suitable for automated checking QA Focus – Supporting JISC's Digital Library Programmes 25

Next Steps Self Assessment Toolkit Further Deliverables We will be developing a self-assessment toolkit for projects to use, by individual projects or across project clusters The toolkit will consist of: • Examples of QA procedures • Documented examples of use of testing tools • Self-assessment questionnaires • Advice on standards and best practices • Case studies • FAQs • … QA Focus – Supporting JISC's Digital Library Programmes 26

Next Steps Self Assessment Toolkit QA Focus – Supporting JISC's Digital Library Programmes 27

Next Steps – Metadata Thoughts on QA for metadata: Syntax Check syntax on embedded DC Cataloguing Rules Ensure projects have appropriate cataloguing rules Interoperability Ensure metadata can interoperate with third parties Fitness for purpose Ensure metadata is appropriate for its purpose Change control Ensure architecture for managing metadata can cope with change control (cf Exploit Case Study) QA Focus – Supporting JISC's Digital Library Programmes 28

Next Steps – Software Thoughts on QA for software: Specification Process for developing systems spec, agreeing spec with stakeholders, etc. Methodology Documentation of systems development methodology (cf UML case study) Testing Documentation of testing methodology Deployment Awareness of deployment challenges See for example <http: //www. ukoln. ac. uk/qafocus/documents/briefings/#software> QA Focus – Supporting JISC's Digital Library Programmes 29

Next Steps – Service Deployment Thoughts on QA for service deployment: Development Architecture Early public documentation of development architecture Service Architecture Awareness of likely service architecture and liaison with service Security Awareness of security concerns Licensing issues Scenarios 30 Software developed using cool open software environment – but JISC Service has no knowledge of environment. Resources digitised, but copyright clearance not documented. Focus – Supporting Digital Library Service. QArefuses to take. JISC's delivery due Programmes to liability concerns.

Next Steps QA Focus And You What relevance has QA Focus for you? • You can write case studies and advisory documents: Seen to be working with JISC We can promote your approaches (and therefore your work) This will help instigate best practices within your group QA Focus – Supporting JISC's Digital Library Programmes 31

Next Steps QA Focus And You What relevance has QA Focus for you? • You can deploy QA practices for your projects, etc: You should benefit You can provide feedback to QA Focus • You can develop QA procedures in innovative areas (e. g. OAI, thesaurii, e. Prints, etc. ) which QA Focus can use: You gain kudos, publications, etc. We can help embed your practices across the community e. g. see joint UKOLN/CDLR paper on "Deployment Of Quality Assurance Procedures For Digital Library Programmes" at <http: //www. ukoln. ac. uk/qa-focus/documents/papers/iadis-2003/> QA Focus – Supporting JISC's Digital Library Programmes 32

Next Steps Beyond Advice – Implementing QA QA Focus is providing a wide range of advice for digital library programmes However there is also a need for projects themselves to be proactive in implementing QA procedures What approaches can projects take, bearing in mind issues such as resources, existing procedures, etc. ? QA Focus – Supporting JISC's Digital Library Programmes 33

QA Focus Methodology The QA Focus methodology which we are advising projects (and JISC) to deploy: • Based on well-established QA principles • Seeks to be lightweight and achievable • Seeks to provide benefits to the projects themselves, as well as (downstream) interoperability benefits to JISC programmes Consists of: • • Documented policies Procedures for monitoring compliance with policies Audit trails Sharing and collaboration QA Focus – Supporting JISC's Digital Library Programmes 34

QA Focus Methodology Applying QA – Standards Policy: Open standards used where possible. Proprietary formats with published spec and cross-platforms viewers and authoring tools may be acceptable Monitoring: Project manager responsible Documentation: Document on standards (cf QA Focus doc) Exceptions: If open standards not available, immature or costly to implement proprietary standards may be used. The project manager should document such decisions: why proprietary format was used, migration strategies to open formats in future, architecture for migration and indication of costs QA Focus – Supporting JISC's Digital Library Programmes 35

QA Focus Methodology Applying QA – Web Standards Policy: The Web site will use XHTML 1. 0 and CSS 2. 0 standards Architecture: The Web site will be based on XHTML templates and SSI Monitoring: New and updated pages validated using , validate and , cssvalidate. Every month , rvalidate will be used Exceptions: HTML derived automatically (e. g. Save As HTML in Power. Point) need not comply with standards. The files will be stored in a standard directory to enable such files to be excluded from checks QA Focus – Supporting JISC's Digital Library Programmes 36

QA Focus Methodology Applying QA – Proprietary Formats Policy: Monitoring: Proprietary formats may be deployed on Web site in certain documented circumstances including MS Office files, PDF and Flash MS Office files should also be stored in most open HTML format. PDF files should include link to online PDF converter Flash can be used for specific applications in which it is needed. Project manager QA Focus – Supporting JISC's Digital Library Programmes 37

QA Focus Methodology Applying QA – Accessibility (1) Policy: The Web site will strive to attain WAI A guidelines. Consistent accessibility shortcuts will be used. An accessibility policy will be published. Architecture: The Web site will be based on XHTML templates which comply with WAI A. Monitoring: New and updated pages will be validated using , bobby. A monthly batch checker will be used and audit reports published (to enable any trends to be spotted). Exceptions: A list of permitted exceptions will be provided. Note that University of Edinburgh Library have a useful Web accessibility policy QA Focus – Supporting JISC's Digital Library Programmes 38

Applying QA – Accessibility (2) Policy: The organisation has no accessibility policy Authors are free to implement their own accessibility shortcuts (if at all) Architecture: No centralised policy covering authoring tools or architecture will be provided Monitoring: No monitoring will be carried out QA Focus – Supporting JISC's Digital Library Programmes 39

QA Focus Methodology Applying QA – Usability (1) Policy: A structured usability test will be applied using at least 5 users for new Web sites Tailored 404 error pages will be provided Use of <link rel="home" …> to support navigation across Web site Use of <link rel="next" …> to support navigation in structured areas of Web site Architecture: The <link rel="next" …> elements should be implemented by automated scripts Monitoring: Need to check the next, previous goes to correct page. Exceptions: … QA Focus – Supporting JISC's Digital Library Programmes 40

QA Focus Methodology Applying QA – Usability (2) Policy: Monitoring: Exceptions: Broken links to be kept to a minimum The , checklink command applied to new and updated resources and , checklink run monthly The Xenu link checker will be run monthly and a report published, in order for any trends to be monitored Don't fix broken links in published documents or derived files. Need to formulate policy on large numbers of broken links (cf JISC Web site) QA Focus – Supporting JISC's Digital Library Programmes 41

Recommendations QA Focus recommendations are likely to include: • Project proposals should describe approaches to selection of standards, technical architecture & QA • Once approved initial project report should document: Standards chosen (e. g. see "ESDS Web Standards Policy" case study and "Matrix for Selection of Standards" briefing document) Justification for any deviations from open standards (e. g. see NOF-TAS FAQ) Technical architecture Intended service deployment environment QA Focus – Supporting JISC's Digital Library Programmes 42

Reality Check Is this really needed? The pages are fine in my browser. But: • What about repurposing resources (XHTML RSS)? • What about new devices (using wireless PDA in student's union)? • What about accessibility? • What about tomorrow's maintenance nightmare? • What about the exploitation of XML resources (quicker, richer, …)? • What about the vision of universal access, and removal of platform and application dependencies? QA Focus – Supporting JISC's Digital Library Programmes 43

Getting There I want to improve the quality of my Web services. But I've got a large Web site, investment in current systems, inertia, … So: • Deploy QA procedures in new developments • Use tools such as W 3 C's Log Validator to prioritise key areas - <http: //www. w 3. org/QA/Tools/> • • • A CMS can help Consider use of Tidy on server (cf. LSE approach) Ensure output from backend scripts is compliance Revisit MS Office Web tools Plan migration strategy to XHTML 2 world (compliant XHTML 1. 0 XSLT XHTML 2. 0 = ) Remember you already have QA procedures QA • Focus – Supporting JISC's Digital Library Programmes 44

Conclusions We have described: • The approaches to QA which have been taken by the QA Focus project • Findings of a number of surveys which have highlighted common problems • An example of an approach to providing access to testing tools using a URI-interface • A systematic methodology for QA • Recommendations for future programmes QA Focus – Supporting JISC's Digital Library Programmes 45

Questions Any questions? QA Focus – Supporting JISC's Digital Library Programmes 46