University Electronic Publishing through Digital Libraries Courseware Theses

University Electronic Publishing through Digital Libraries: Courseware, Theses and Dissertations Singapore - Dec. 2002 Edward A. Fox fox@vt. edu http: //fox. cs. vt. edu CS DLRL Internet TIC NDLTD CITIDEL NSDL … Virginia Tech, Blacksburg, VA, USA

Acknowledgements (Selected) • Sponsors: ACM, Adobe, IBM, Microsoft, NSF (Grants CDA-9312611; DUE-0121741, 0136690, 0121679; IIS 0080748, 0086227, 0002935, and 9986089), OCLC, SOLINET, UNESCO, US Dept. Ed. (FIPSE), VTLS, … • Faculty/Staff (now): Boots Cassel, Debra Dudley, Lee Giles, Rex Hartson, John Impagliazzo, Deborah Knox, JAN Lee, Kurt Maly, Gail Mc. Millan, Manuel Perez, Muhammad Zubair, … • Students: Fernando Das Neves, Marcos Goncalves, Paul Mather, Ryan Richardson, Priya Shivakumar, Hussein Suleman, Wensi Xi, … • UNESCO Analytical Survey: Leonid Kalinichenko

Outline • Case Study: NDLTD • • Case Study: CSTC Case Study: CITIDEL Interoperability: OAI, ODL Conclusions

A Digital Library Case Study • Domain: graduate Project: education, research Networked Digital • Genre: ETDs=electronic Library of Theses & theses & dissertations Dissertations • Submission: (NDLTD) http: //etd. vt. edu http: //www. ndltd. org • Collection: http: //www. theses. org

The Networked Digital Library of Theses and Dissertations www. NDLTD. org Training Authors Expanding Access Preserving Knowledge Improving Graduate Education Enhancing Scholarly Communication Empowering Students & Universities Leader of the Worldwide ETD (Electronic Thesis and Dissertation) Initiative

NDLTD Grad Program IT Library Ed. (Tech)

Key Ideas: Scalability Networked infrastructure University collaboration Workflow, automation Education is the rationale Maximal Access 8 th graders vs. grads Authors must submit Standards PDF, SGML, MM, MARC, DC, URNs, Federated search

What led to today’s meeting? • 1987 mtg in Ann Arbor: UMI, VT, … • 1992 mtg in Washington: CNI, CGS, UMI, VT and 10 universities with 3 reps each • 1993 mtg in Atlanta to start Monticello Electronic Library (regional, US Southeast): SURA, SOLINET • 1994 mtg at VT: std: PDF + SGML + multimedia objects • 1996 funding by SURA, US Dept. of Education (FIPSE) • 1997 meetings in UK, Germany, . . . • 1998 – 1 st symposium – Memphis (20) • 1999 – 2 nd symposium – Blacksburg (70) • 2000 – 3 rd symposium – St. Petersburg (225) • 2001 – 4 th symposium – Caltech (200) • 2002 – 5 th symposium–BYU; 2003–Berlin; 2004–Kentucky

What are the long term goals? • 400 K US students / year getting grad degrees are exposed / involved • 200 K/yr rich hypermedia ETDs that may turn into electronic portfolios (images, video, audio, …) • Dramatic increase in knowledge sharing: literature reviews, bibliographies, … • Services providing lifelong access for students: browse, search, prior searches, citation links • Hundreds/thousands of downloads / year / work

Convene Local Planning Group ETD

Build Local ETD Site ETD Workshop/Training Digital Library Policies Inspection/Approval

Student Prepares Thesis/Dissertation NDLTD Literature Computer Resources Research

Student Defends & Finalizes ETD My Thesis ETD

Student Gets Committee Signatures and Submits ETD Signed Grad School

Graduate School Approves ETD, Student is Graduated Ph. D.

Library Catalogs ETD, Access is Opened to the New Research WWW NDLTD

National / Regional Projects • Australia • • U. New South Wales (lead) U. of Melbourne U. of Queensland U. of Sydney Australian National U. Curtin U. of Technology Griffith U. • Germany • Humboldt University (lead) • 3 other universities • 5 learned societies: Math, Physics, Chemistry, Sociology, Education • 1 computing center • 2 major libraries • Ohio. LINK: 79 colleges/univs • Consorci de Biblioteques Universitàries de Catalunya, as group, www. cbuc. es: 9 sites • India • Korea • Brazil • UK (British Library, JISC, Edinburgh) • UNESCO (especially Latin America, Eastern Europe, Africa)

Some Countries • • • • Australia Belgium Brazil Canada China, Hong Kong Columbia Finland France Germany India (Hyderabad) Italy Korea Mexico • • • Netherland Norway Russia Singapore S. Africa (Rhodes U. ) S. Korea Spain Sudan Sweden Taiwan UK USA

Institutional Members • • • • • British Library Cinemedia Coalition for Networked Information (CNI) Committee on Institutional Cooperation (CIC) Consorci de Biblioteques Universitàries de Catalunya Diplomica. com Dissertationen Online (Germany) ETDweb, a Division of Answer 4. com Ibero-American Science & Technology Education Consortium (ISTEC) National Documentation Centre (NDC), Greece National Library of Portugal (for all universities) OCLC Online Computer Library Center Ohio. LINK Organization of American States (SEDI/OAS) Southeastern Library Network (SOLINET) UNESCO (www. unesco. org/webworld/etd)

Access Possibilities Web search engines www. theses. org Virginia MIT National Tech Library of Portugal www. library openarchives. catalog org clients CBUC (Spain) Ohio Link 3 rd Party Services (e. g. , UMI) National Projects: AU, GE, …

ETD-MS • ETD Metadata Standard • XML-encoded metadata standard (content and encoding) for Electronic Theses and Dissertations (ETDs) • in part conforming to Dublin Core (DC) • using UNICODE • (optionally / later using RDF) • Well specified relationship with MARC

NDLTD Members and ETD-MS • NDLTD members will • Share metadata for their ETDs • Providing that in either ETD-MS • Or if they use a version of MARC locally, work to have that eventually shared in either MARC 21 or UNIMARC • Run OAI, either locally or in consortia, so their metadata can be harvested, according to necessary terms and conditions

Some recent additions • ETD individuals support • http: //etdindividuals. dlib. vt. edu: 9090 • ETD discussion (e-prints) • http: //ndltdpapers. dlib. vt. edu: 9090 • Conference papers and presentations • http: //www. ndltd. org/WVUproc. htm • Marcel Dekker book in publication

What are plans at VT? • LOCKSS welcomed us • Lots of Copies Keeps Stuff Safe • • MARIAN: harvest, crawl/scrape, fed search Metadata crosswalks and format converters XML schema for ETDs Open Digital Libraries: easy to add services! • http: //oai. dlib. vt. edu/odl

Union catalog (OCLC) • OCLC will expand the OAI data provider on TDs • Will get data from World. Cat • Will harvest from all who contact them • Need DC and either ETD-MS or MARC • Will have a set for ETDs

Union catalog (VTLS, VT) • VTLS will enhance search/browse service for ETDs • Will harvest from OCLC’s set of ETD records • Will receive through other mechanisms, too • Will work with MARC-21 and ETD-MS • VT will continue to offer experimental services

NUDL (www. nudl. org) Int’l Research Support • Networked University Digital Library • Partners: Germany, Mexico (Puebla and Monterrey), Brazil • Problems: Multilingual search, high performance DLs, requirements/usability, … • Start with ETDs, then expand to other student works, portfolios, data sets, (CS) courseware, . . .

Outline • Case Study: NDLTD • • Case Study: CSTC Case Study: CITIDEL Interoperability: OAI, ODL Conclusions

CS Teaching Center (CSTC) • Instead of building large, expensive multimedia packages, that become obsolete and are difficult to re-use, concentrate on small knowledge units. • Learners benefit from having well-crafted modules that have been reviewed and tested. • Use digital libraries to build a powerful base of support for learners, upon which a variety of courses, self-study tutorials & reference resources can be built.

Browsing (2)

JERIC • Journal of Educational Resources in Computing • Accessible from www. cstc. org and www. acm. org • ACM and SIGCSE support • Refereed and interactive • Part of ACM Digital Library

Outline • Case Study: NDLTD • • Case Study: CSTC Case Study: CITIDEL Interoperability: OAI, ODL Conclusions

www. CITIDEL. org • Computing and Information Technology Interactive Digital Education Library, an NSDL Collection Track project • Led by Virginia Tech, with co-PIs: • Fox (director, DL systems) • Lee (history) • Perez (user interface, Spanish support) • Partners • College of New Jersey (Knox) • Hofstra (Impagliazzo) • Villanova (Cassel) • Penn State (Giles)

Summary of Spring 2001 Survey of CITIDEL-related Collections and their Sizes Size of Collection 1 -5 items 6 -100 items 101 -999 items +1000 items Number of Collections Identified 100 -300 50 20 -35 10 -25

Multi-dimensional Categorization

CITIDEL Collection Sources include ACM include CSTC NCSTRL Research Index IEEE-CS … include metadata include ACM DL fulltext include SIGCSE proceedings NEC’s data JERIC Experts’ finding aids include data processed w. R. I. Borner’s info viz software repository

CITIDEL Collection Building thru Submitting include after Creating include after Searching, Composing using VIADUCT Nominating after Crawling Browsing thru Get. Smart or thru aided by Classifying using Crawlifier

Overview of CITIDEL architecture

Distributed repository structure

Digital library architecture for local and interoperable CITIDEL services

Outline • Case Study: NDLTD • • Case Study: CSTC Case Study: CITIDEL Interoperability: OAI, ODL Conclusions

Open Archives Initiative OAI www. openarchives. org openarchives@openarchives. org

The World According to OAI Service Providers Discovery Current Awareness Metadata harvesting Data Providers Preservation

Technical Umbrella for Practical Interoperability… Reference Libraries Museums Publishers E-Print Archives …that can be exploited by different communities

Tiered Model of Interoperability Mediator services Metadata harvesting Document models

OAI – Black Box Perspective Services: Search Browse Metadata: Summarize Visualize OA 7 OA 4 OA 2 OA 3 OA 1 OA 6 OA 5 Docs: DO DO

Aggregation through OAI Harvesting CITIDEL NCSTRL Lite Sites Archive Eprints Active Own: History, Research. Index, CSTC, … IEEE-CS, ACM, …

Approaches to Open Archives Build By Institution Build By Discipline Author Category Interdisciplinary Year Language Query …

OAI Perspective • Rethink your efforts in terms of providers of • Data, Services • Reduced work for data providers • Tools available • Don’t need to offer services • Reduced work for service providers • Others provide the data • Can use tools and systems for OAI, XOAI • Results • More data becoming available • To more people • Supported by improved services

repository support data harvesting data h a r v e s te r OAI protocol r e p o s i t o r y items

selective harvesting - datestamps harvest within date range record r e p o s i t o r y

selective harvesting - sets harvest within set record r e p o s i t o r y S 1 S 2

What is an Open Archive ? • Any WWW-based system that can be accessed through the well-defined interface of the Open Archives Protocol for Metadata Harvesting • … aka OAI-Compliant Repository • No implications for: • • Physical storage of data Cost of data Metadata and data formats Access control to server

Sample OAI Record <record> <header> <identifier>oai: sigir: ws 3</identifier> <datestamp>2001 -08 -13</datestamp> </header> <metadata> <dc> <title>OAI Workshop at SIGIR</title> <creator>Hussein Suleman</creator> <language>English</language> </dc> </metadata> <about> <metadata. ID>oai: sigir: ws 3 md</metadata. ID> </about> </record>

Sets • Protocol mechanism to allow for harvesting of sub-collections • No well-defined semantics – depends completely on local data providers • May be defined by arrangement between data providers and service providers • E. g. , Subject areas, years, author names, search queries

Protocol for Metadata Harvesting • Service Requests • Identify • List. Metadata. Formats • List. Sets • Get. Record • List. Identifiers • List. Records • Metadata Multiplicity • Date Ranges • Resumption Tokens

Example: Union Collection of ETDs (Electronic Theses and Dissertations, for Networked Digital Library of Theses and Dissertations, NDLTD)

Example: Details

Open Digital Library (ODL) Hypothesis (Hussein Suleman) • Can we leverage the successful model of the OAI Protocol for Metadata Harvesting to alleviate our architectural problems ? Maybe … if Digital Libraries can be modeled as • networks of extended Open Archives, where • each extended Open Archive is a • source of data and/or a provider of services.

Example Architecture (NDLTD) Virginia Tech User Interface Phys. Net Humboldt Search Browse Recent Duisburg Cal. Tech Union Catalog MIT Filter MIT legend Dresden User Interface OAI/ODL archive OAI/ODL protocol

ODL Demonstration - Front. Page

ODL Demonstration - Search

ODL Demonstration - Browse

Outline • Case Study: NDLTD • • Case Study: CSTC Case Study: CITIDEL Interoperability: OAI, ODL Conclusions

Conclusions • Digital libraries can help advance education. • Singapore is invited to engage in NSDL, CITIDEL, NDLTD, and other ventures. • UNESCO Analytical Survey on Digital Libraries in Education is recommending DLE in each nation. • Local and national support can • • stimulate activities, including collaboration promote a sharing culture, especially in research and teaching leverage others’ investments (networking, computing, …) encourage / facilitate learning, innovation and problem solving

Selected Links • CITIDEL • www. citidel. org • NCSTRL • www. ncstrl. org • NDLTD • www. ndltd. org • NSDL • www. nsdl. org • Virginia Tech Digital Library Courseware • http: //ei. cs. vt. edu/~dlib • Virginia Tech Digital Library Research Laboratory (DLRL) • http: //www. dlib. vt. edu • (5 S, 5 SL, American. South. Org, CSTC, ENVISION, MARIAN, NSDL, OAI, ODL) • Repository Explorer • http: //purl. org/net/oai_explorer NDLTD,

More Links • ARC Cross-Archive Search Service • http: //arc. cs. odu. edu/ • Dublin Core Metadata Initiative • www. dublincore. org • E-Prints DL-in-a-box • www. eprints. org • Open Archives Initiative • http: //www. openarchives. org/OAI/openarchivesprotocol. htm • http: //www. dlib. vt. edu/projects/OAI/ • XML Schema Validator • http: //www. w 3. org/2001/03/webdata/xsv • XML Tools at W 3 C • http: //www. w 3. org/XML/#software