Using Intranet Log Analysis to Improve Retrieval Performance

Using Intranet Log Analysis to Improve Retrieval Performance Udo Kruschwitz, University of Essex Objectives Nick Webb, SUNY Albany • Study Intranet usage through query logs • Hence build enhanced Information Retrieval and Question Answering systems Richard Sutcliffe, University of Limerick Progress • Query Analysis • Create a set of topics based on a study of the queries together with a detailed knowledge of the intranet domain • Establish the most important topics and the availability of relevant answers UKSearch Retrieval Engine • Initial UKSearch IR system has been built An Interactive Information system • System is in long term use by campus community • Guides users through a document collection • Several detailed log analyses have been carried out • Presents query modification suggestions which can either refine or relax the search Architecture • Keeps a detailed interaction log for later analysis Screenshot Query Category Examples Percent Academic or Other Unit data archive 13. 15 Computer Use web mail, printing credit 13. 10 Administration of Studies registration 11. 04 Person Name Udo Kruschwitz, udo 9. 53 Structure/Regulations corporate plan 8. 08 Calendar / Timetable TIMETABLES 6. 91 Map / Campus / Room map of teaching room 5. 57 Other second hand bicycle 32. 62 Next Steps • Determine an efficient means of reaching the answers • Improve the indexing and query components of the system • Extend the search engine to Question Answering based on our TREC, CLEF and NTCIR systems References Usage Statistics Kruschwitz, U. (2005). Intelligent Document Retrieval: Exploiting Markup Structure. Volume 17 of The Information Retrieval Series, Springer, 2005. Kruschwitz, U. and H. Al-Bakour, (2005). Users Want More Sophisticated Search Assistants - Results of a Task-Based Evaluation. Journal of the American Society for Information Science and Technology (JASIST), 56(13): 1377 -1393. Kruschwitz, U. & R. F. E. Sutcliffe (2007). Analysis of an Academic Intranet Search Log: A Justification for System-Guided Search. Submitted to JASIST. Average Query Length 1. 97 Length of Longest Query 17 Queries with Spelling Errors ~6% Kruschwitz, U. , N. Webb & R. F. E. Sutcliffe (Forthcoming). Query Log Analysis for Adaptive Dialogue-Driven Search. In J. Jansen, A. Spink & I. Taksa (Eds. ): Handbook of Web Log Analysis. New York, NY: IGI Publishing.