Webbased searching Understanding student experiences to enhance the
Web-based searching Understanding student experiences to enhance the development of this critical graduate capability Sylvia Lauretta Edwards Dip. Lib. (RMIT), GCEd(HE), MIT(Res. ), AALIA, MACS Abstract: Web-based information searching is a critical graduate capability. As teachers we can impressed by the students’ ability to "talk the talk" and make it sound like they understand the process of searching. Unfortunately, later, some of these same students do not "walk the walk". Ongoing research to understand the variation in students’ searching experiences has been the first step. On the basis of student perceptions we are challenged to find activities that may bring about the ways of experiencing searching that we want them to engage in. This workshop will outline the research findings in brief and then allow participants to explore the issues the research has revealed in designing assessment instruments to bring about desirable learning outcomes in their own teaching. Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 1
Workshop Outline • You will be given information on a phenomenographic study of students’ experiences of web-based information searching, aiming to explore student conceptions of the searching experience. – This will include: • • An outline of the research inspiration The brief methodology The main findings from the research, and Provide examples of using the research findings in assessment & curriculum design • Workshop will then allow you time to explore the issues and apply this your own work, or © Edwards, S. L. (2003)to Web-based Searching Workshop 2 Workshop presented October 1 st, 2003
Inspiration Background to the study: Where did idea come from? • Teaching in an IT Unit - Information Resources (ITB 322/ITN 322) – Student’s learn about a variety of information resources and their uses, independent of the format • Information retrieval techniques for all of them, how to find and keep current on information resources in future career, try to improve information searching skills so that students learn to retrieve a lower recall of resources more relevant to their needs. – Their information explosion awareness increases • Over 600 library/dialog databases • Noticed that students often display little understanding of the tools they are using, and seem to show little evidence of reflection upon how to improve their results • The study began as Ph. D research with 3 rd yr. and PG IT students and has now been expanded into 1 st yr students © Edwards, S. L. (2003) Web-based Searching Workshop 3 in various faculties using the Teaching Fellowship Workshop presented October 1 st, 2003
Research Aims 1. 2. To determine variation in the ways IT students approach information searching when using web-based databases. To determine variation in IT students’ ways of learning how to search for information when using web-based databases. 3. To recommend teaching and learning strategies for curriculum design that are based on managing student’s experiences. 4. To determine if there are different levels of sophistication in information searching, or other differences in student information searching behaviour approaches. 5. If levels do exist, to identify any triggers which allow students’ to move from one level of searching sophistication to the next level. Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 4
Participants and Method • Phenomenographic Study - with transcript analysis, aiming to uncover the variation (32 students and 44 Interviews) • Different cultures, ages and genders represented – 1 st Round of Interviews • Twenty ITB 322/ITN 322 students – 1 st Interviews conducted Wk 3 -6 of semesters, Dual audio taped and video recorded. (Transcripts of both) – Weekly diary entries also analysed weekly looking for any changes in the way the respondents appear to be reflecting upon the process. • Twelve 1 st Yr. students from IT, Science and Creative Industries – 2 nd Round of Interviews • 12 of the previous students re-Interviewed at the end of the Teaching period Audio Taped (Transcripts) – Key Questions • Has information searching experience changed, How, why, reasons for the change? Information searching is … Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 5
Refer to EARLI 2002 • For the students undertaking the unit – All students believed they were searching differently as a result of unit – students identified reasons for change as • assignments designed to encourage reflection • the ability of the teaching staff • NB: No student identified the OLT as one of the reasons for change FOR MORE INFO. . . Edwards, S. L. , & Bruce, C. (2002). The assignment that triggered …change: Assessment and the relational learning model for generic capabilities. Paper presented at the EARLI/Northumbria Assessment Conference Learning communities and assessment cultures: connecting research with practice, 28 -30 August 2002, Longhirst Hall, University of Northumbria, Newcastle upon Tyne, UK. Available full-text on Educati. On-line and AEHE (special edition) (in press) Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 6
1 and 2 interviews used to identify … The Outcome Space Teaching Fellowship findings Confirm framework with the 1 st. MORE year students FOR INFO. . . Edwards, S. L. & Bruce, C. S. (2002) Needles, haystacks, filters and me: the IT confidence dilemma. Refereed Conference Paper presented at Lifelong Learning Conference 2 nd: Yeppoon, Central Queensland, Australia, 16 -19 June 2002. [Lifelong Learning Conference: refereed papers from the 2 nd International Lifelong Learning Conference, Yeppoon, Qld. ] pp. 165 -171. ISBN: 187 6780 19 3 Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 7
Category 1: Information searching is seen as “looking for a needle in a “Once I understand thehaystack” topic I am looking for then I can find it. • Student’s focus is on the topic. • Aware of info. environment but not the details of the tool or the environment structure • Often confusion b/t different tools and tool searching options. • Comfortable with IT & moderate confidence – Not confident to push further into the environment • Planning is poor, or possibly nonexistent • Little, if any, reflection occurring – Eg: searcher time. Int 1: 5(p. 6) truncation is more like, let’s say you have the ‘or’ ‘not’ ‘and’ and you try to use these words. Lets say you want both (topic 1) and (topic 2), you put the ‘and’ in it so you come up with switches tools & terms at same both of them. That means it has to have both of these terms in it. And ‘or’ will be either this or either this, so any of them © Edwards, S. L. (2003) Web-based Searching Workshop 8 will do. • Likely the search process will be Workshop presented October 1 st, 2003
Category 1: Information Searching is finding a needle in a haystack Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 9
• Students focus on the topic Information • Strong emphasis on terms/synonyms, DBsearching is selection, &“finding retrieving results seen as a way through a maze” • Planning process commenced I must understand the topic, & plan to retrieve results in a useable format • Aware of Info. Environment structure & tool search options. • Begins to use advanced search features • Aware of Info. Quality (not a major focus) • Talk about tools interchangeably • IT confidence good but still miss own mistakes • Comfortable with online searching Int 1: 3 (p. 4) …I try and find a search Category 2: engine that has an advanced search – Show signs of reflection, see results and option and do it that way and specify changing terms whereabouts I want the web site to be. – Likely to persist, consider alternatives, & persevere • Still blame the tool rather than question their own abilities. Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 10
Category 2: Information Searching is finding a way through the maze Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 11
Category 3: Information searching is asuse“using the tools • seen Students tools to understand topic as & a filter” to find not theunderstand required the information. I may/may topic, but I can use tools to help me find it • Focus is the tools, the topic is second • Strong awareness of Info. Environment structure & search tool options (adapt searching based on the tool) • • Info. Quality not important in search Tools refine topic & filter results size IT confidence high BUT Students aware of mistakes – take steps to correct mistakes • Planning evident – term analysis & pronounced attempt at synonyms Int 1: 6 (p. 1) … I had to search Internet • Reflection evident – During search, terms & strategies change • Students search is successful Workshop presented October 1 st, 2003 definitely, but I also had to search some library to get a general information about (topic), and I also had to find the recent updates because the issue had to be current. … Then I would probably go to the library and find books about (topic) and (sub-topic), because it is needed for general definitions. © Edwards, S. L. (2003) Web-based Searching Workshop 12
Category 3: Information Searching is using the tools as a filter Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 13
• Use tools to understand topic, as a filter, & to limit to high quality Information searching is (retrieve primary sources) seen as “panning • Focus on tool structure, then topic, for & gold” then qualitytools to find primary resources of information I can useon theinfo. appropriate Category 4: – Tools used to both refine topic & search to filter out poor quality items – ie: domain name searching or scanning • IT confidence very high • Aware of mistakes, check & correct them • More inclined to ask for help from peers. • Strong planning and reflection are evident – reflection on information character/quality – More inclined to stop a search, reflect upon improvement, then reattempt later a previously failed search. • Successful in searching. Workshop presented October 1 st, 2003 Int 1: 4 (p. 5)… but if you do a search on say (topic) or something, and you’ll end up with thousands of pages. But if you put lecture notes or tutorial on the end of it, it narrows it down quite a bit. © Edwards, S. L. (2003) Web-based Searching Workshop 14
Category 4: Information Searching is panning for gold Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 15
Structure of Awareness … Using the categories of description showing the variations • Introduced changes to assessment materials to lead students into the structure of the experience considered desirable – – • The next step is on online tool to encourage reflection across the structure of awareness. – • Success of this is not yet known Structure of awareness section of each category has revealed elements that need to be attended to, to allow the student to be aware of what is happening in many areas of the experience. There will be trial with 3 rd yr. IT “crash test dummies” in Semester 1, 2004. Decided to existing model already used in teaching a reflective internet searching approach © Edwards, S. L. (2003) Web-based Searching Workshop 16 Workshop presented October 1 st, 2003
An Action Research Model for Reflective Internet Searching Edwards, S. L. , & Bruce, C. S. (2002). Reflective Internet searching: an action research model. The Learning Organization: an international journal, 9(3/4), 180 -188. Plan the Net Search Reflect Critical Evaluation Ways of thinking about the Internet Act Conduct Search Ways of learning about the Internet Constantly Changing Internet Environment Record Results Print, Save, Bookmark Workshop presented October 1 st, 2003 Edwards& Bruce © 2002 17 © Edwards, S. L. (2003) Web-based Searching Workshop
Assignments used in the unit • 3 Items of Assessment originally developed – Reflective Journal (20%) – Information Consultants Search (30%) – Information Resource Guide (50%) • pedagogical underpinnings should be apparent as we go through Discuss these through looking at the design and development of each as we go Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 18
Reflective Journal • Submitted weekly, marked & within 1 week. • Briefly this includes: – Write a summary of weeks materials, readings completed, general comments on unit and current studies – Gives them the chance to reflect on what was covered and develop own ideas – Must reflect upon their current searching, what was new or different this week in their searching approach, or what did they learn this week about searching – includes a section for questions to the lecturer or suggestions for improvement – NB: They get weekly feedback! Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 19
Reflective Journal • Students attributing change in searching to this assignment stated that the effort of having to reflect upon the content each week had caused them to change their experience. … if I wasn’t doing the journals this semester I don’t think I would have rethought a lot of the things that I’d learned straight away, and I think that a lot of the good things that I’d learned may not have, you know, may have just disappeared. Male student, UG 21 -25 yrs (Int 2001: 2. 2 p. 8) Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 20
Information Search • Given a Topic, which they must search using The Library databases, Internet Search Engines and commercial database vendors • Interview the lecturer as if they are the “client” – think about end users needs, what are they really looking for when they search, matching information needs to end results – Must produce a search report explaining their search strategies, tool selection, and reflect on approaches – learning about changing their strategies across different search tools and databases, and evaluating resources Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 21
Search Report • Students attributing change in searching to this assignment stated that being forced to reflect and report on assumptions, keyword selection, or other aspects of the search process, made them realise that they should change the way they approached searching. What if I just blame it on assignment two … That – big time change for me. Well you see it had to be comprised of all these different components. Like ok, your client …synopsis, your assumptions, your keywords and synonyms, your … there were so many different areas. …. So you had to make sure you that you know all these in order to put down in your report if you really wanted to do well in this report. … So I came up with all these assumptions S. L. (2003) Web-based Searching Workshop and was like, © Edwards, oh ok I’ve never thought about these 22 Workshop presented October 1 st, 2003
Resource Guide • Encouraged to select a topic that will be in the area they want to work in when they are finished at QUT – Team project (working in real-life) – Find all resources that would be necessary to help them keep up to date in their field over their working life • What books would they want on the office bookshelf? What journals? What electronic conferences or Usenet Newsgroups should they look at, what Internet sites should they bookmark (add to “favorites”); what newspapers and databases should be scanned regularly, etc. • Must select the “Desert Island Top 5” best © Edwards, S. L. (2003) Web-based Searching Workshop 23 resources found and give justification for their Workshop presented October 1 st, 2003
Reviewing Assignments against the Categories • Category 3 is built into Assignments – It appears Assign’t 1 & 2 intuitively mapped with Category 3 • Both assignments require students to reflect • Assignment 2 in particular asks students to refine their topics, devise alternative search terms, and adapt their searching based on the tool in use. • But Category 4 ? – Assign’t 2 & 3 - limited extent, no process reflection • Assignment 2 requires a limit to high quality resources only. • Assignment 3 asks for “Desert Island Top 5” and reasons – But assessment marks did not reflect the significance of this, so little attention/weight given to the exercise by the © Edwards, S. L. (2003) Web-based Searching Workshop 24 students. Workshop presented October 1 st, 2003
Where to from here … • On the basis of student perceptions, assessment is now being carefully re-constructed to bring about desired learning outcomes – to encourage reflection to level 4 – need to design the assessment to make the assignments work harder, to lead students into the structure of the experience that is considered desirable. In that way helping them to move into a higher level of information searching experience. • How ? – A series of questions designed to encourage reflection about their choices and report on the reasons for the choice, could be added, with appropriate weighting in assessment criteria. Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 25
Assessment in my other Units in IT • In the information searching expected in a first year unit, students are asked to undertake one or two of the QUT Pilot project modules, – Expected to complete the exercises and write a 300 word discussion of what they have learnt from the exercise, reflecting in particular on their searching experiences and how they search. Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 26
Learning Theories • Learning theories state that successful learning includes (Marton, Dall'Alba & Beaty, 1993). – – – the students ability to increase their knowledge, to memorise and reproduce that knowledge, to apply it and understand what was done, to see something in a new way, and finally to change as a person. • Furthermore information literacy, which includes the ability to search for information, is a major part of the lifelong learning process that© Edwards, all graduates need in order to 27 S. L. (2003) Web-based Searching Workshop presented October 1 st, 2003
Lessons Learnt While embedding generic capabilities into assessment is vital to trigger changes in conceptions or experiences, the ability and quality of the teaching staff should not be ignored. Their ability, to arouse curiosity and make the learning environment both stimulating and enjoyable, helps promote interest in the topic, which can also lead to changes in learning outcomes. • Based on the experiences in this FIT unit, a word of caution needs to be sounded. – – Students will not automatically engage in interaction via the OLT. Students will engage when the assessment marks require it. The OLT itself does not create a community of learners. The unit coordinator needs to design the elements of the unit to enable the OLT to be used to its maximum potential. The whole learning experience must be © Edwards, S. L. (2003) Web-based Searching Workshop presented October 1 st, 2003 crafted. 28
Let’s list a few of your ideas 1. Can you suggest a few classic assessment items, or even a few different forms of assessment? 2. Can you suggest how information searching skills can be used within a unit or item of assessment? – – – to increase their searching skills? to apply it and understand what was done? to come to see searching in a new way? Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 29
Your Turn – the “work”shop begins Alone or in Pairs - 20 mins • How can you apply this in a unit you teach? – Select one of your units and/or choose one item of assessment to either redesign or to develop from scratch – Develop a list of basic criteria for this assessment item – what do you want them to learn? • What else can you do as a teacher to craft the learning experience in your unit? – ie: remember that the learning design can be used to develop a learning community • If you consider your unit design as a holistic process, what role does information searching play? – Include both present and any future ideas. Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 30
Your Turn in Small Teams Small Groups 20 mins 1. Select one unit/assessment that someone in the team is currently developing or teaching. 2. Reflect upon the ideas so far. Decide how searching skills used in the unit could help the students develop their skills further. 3. Decide now an alternative set of assessment items and/or teaching strategies to encourage the development of searching skills. 4. Using a spokesperson, share with the © Edwards, S. L. (2003) Web-based Searching Workshop group your final ideas. Workshop presented October 1 st, 2003 31
Questions anyone? Get more Information on assessment from TALSS http: //www. talss. qut. edu. au/ or try 1. TEDI at UQ http: //www. tedi. uq. edu. au/Teaching/ 2. Alternative modes of teaching and learning at UWA http: //www. acs. uwa. edu. au/csd/altmodes/ Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 32
Further References 1. Candy, R. C. , Crebert, R. G. (1991). Ivory tower to concrete jungle: The transfer of learning skills from the academy to the workplace. Journal of Higher Education, 62(5), 570 -592. 2. Candy, P. , Crebert, G. & O'Leary, J. (1994). Developing lifelong learners through undergraduate education. Commissioned Report No. 28. NBEET. Canberra: AGPS. 3. Carlson, T. , & Spegel, N. (1996). Adventure into education: Innovation in teaching in tertiary settings. ERA/AARE Joint Conference Papers, Singapore. 4. Clanchy, J. & Ballard, B. (1995). Generic skills in the context of higher education. Higher Education Research and Development, 14(2) pp. 155 -166. 5. QUT MOPP Policy C/1. 3 Generic attributes of QUT graduates http: //www. qut. edu. au/admin/mopp/C/C_01_03. html last accessed 1 st October 2003 6. Raths, D. (1999). Next-century skills. (Industry trend or event) Info. World, 21(16), pp. 97 -99. Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 33
Concluding Thoughts What is happening to our individual/group view of information searching this morning? • Remember … Use materials already available – Library Pilot Modules – Edwards & Bruce “Reflective Searching Model” – Website/library database searching tips and “hands-on” – Most important … © Edwards, S. L. (2003) Web-based Searching Workshop 34 Build desired experiences into your Workshop presented October 1 st, 2003
Thank you for participating Sylvia Edwards QUT Teaching Fellow and Lecturer, School of Information Systems, FIT s. edwards@qut. edu. au http: //www. fit. qut. edu. au/~edwardss/ Workshop presented October 1 st, 2003 © Edwards, S. L. (2003) Web-based Searching Workshop 35
- Slides: 35