Issues in Human Rights Web Archiving Robert Wolven

Issues in Human Rights Web Archiving Robert Wolven Columbia University Libraries Human Rights Archives and Documentation, CHRDR Conference 46 October 2007

Libraries have a mission to build, organize, and preserve coherent collections for research n There’s a great deal of human rights-related content on the web n Much of it is not currently collected by libraries n Something should be done about that n

A great deal of content exists only online n There’s a high risk that some will disappear n Libraries and archives are custodians of our cultural heritage n n Libraries and archives should lead in preserving “at risk” content

Web Archiving as Preservation n Small footprint in organization n The Hoover effect n Haphazard library collections n Ineffective access

A Lot of Content

Much of it is not … collected n Refugees International n 40 documents on web site n 0 in Columbia collections n 10 listed in OCLC n 1 held by more than 2 libraries n No library holds more than 3

Web Archiving Issues Ways and Means n Selection policies n Permissions – and Obligations n Organization and Integration n Presentation and Uses n Sharing the Costs and Benefits n Organizational Transformation n

n Center for Research Libraries’ Political Communications Web Archive Project website: http: //www. crl. edu/content/Polit. Web. htm Final report: http: //www. crl. edu/PDF/PCWAFinal. Report. pdf

Web Archiving Tools n Archive-It (Internet Archive) n n PANDAS (National Library of Australia) n n http: //www. archive-it. org http: //pandora. nla. gov. au/pandas. html OCLC Digital Archive n http: //www. oclc. org/digitalarchive/default. htm

“I only want to download text/html and nothing else. Can I do it? ” n n You can … add a filter that excludes all filters that end in other than 'html|htm', etc. , or, if you want to instead look at document mimetypes, you can Add a Content. Type. Reg. Exp. Filter filter as a midfetch filter to the http fetcher. [From Heritrix FAQ http: //crawler. archive. org/faq. html#user-heritrix]

Policy Technology n Full web site or selected content? n Preserve relationships, “look and feel”? n All file types? n How often?

“Corrigendum Changes have been made to the report on "Muslims in the European Union – Discrimination and Islamophobia" after it was printed. Following pages are replacing the Annex page in the EN and FR version of the report. PDF” From the European Union Agency for Fundamental Rights website

Selection by Type of Agency n Governmental n International n Academic n Educational

Selection by Focus n Global, regional, local n Ethnicity, religion, gender, age n Legal, medical, economic n Crisis-driven

Selection by Content n Fixed documents: Case studies n Position papers n Topical reports n Press releases n Bulletins, newsletters n Activity reports n

Selecting by Content n Non-textual (image, sound, video) n Ephemeral, dynamic content n Redundant (? ) content: Languages, formats n Republished or unique? n

Rights and Obligations Permissions: ask or assume n Rights: n Dark archive n Closed archive n Conditional exposure n n Obligations: Parallel (mirror) access n Free, reliable access n Perpetual access n

Organization and Integration … or, now that we have it, how do we know what we’ve got? How do other agencies know what’s been done? How do researchers find it?

“From 1 March the European Monitoring Centre on Racism and Xenophobia (EUMC) became the EU Agency for Fundamental Rights (FRA). The content on the website is being gradually transformed to reflect the scope, activities and products of the new Agency. ”

Integrating Access n Through Authority Control n Through controlled vocabulary n Through series

Integrating Collections n With print – in the catalog n With archives – in finding aids n With digital collections – in …

Use n Internal organization and navigation n Indexing n Analytical tools n Citation: pedigree and persistent links

Sharing Costs and Benefits n Centralized Collaboration n Distributed Collaboration n Disclosure (at what level of detail) n Exposure (to the web; OAI-PMH)

Transformative Action n Concept of “collecting” n Modes of selection n Bridging communities of practice

“Where do you stop? ”