VCE IT Theory Slideshows by Mark Kelly 2016
VCE IT Theory Slideshows by Mark Kelly 2016 -2019 study design Data Collection Techniques Begin By Mark Kelly, vceit. com, mark@vceit. com
Contents ITI U 3 O 1 KK 2 -techniques for efficient and effective data collection • Interviews • Questionnaires, surveys • Observation • Others • Querying of resources 2
Remember… • “Efficient” = saving or not wasting – TIME – MONEY – LABOUR or EFFORT • “Effective” = every other evaluation criterion, usually related to quality, or how well the solution does its job, especially: – – – – Accuracy Readability Ease of use Security Portability Robustness etc VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 3
Interviews DISADVANTAGES • Costly in terms of time, labour • Data obtained is qualitative (opinion-based) that must be interpreted and encoded to be able to manipulate statistically • May be subject to bias from the interviewer • The presence of the interviewer may make subjects be less responsive or truthful about sensitive, embarrassing or incriminating issues. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 4
Interviews ADVANTAGES • Valuable as a starting point for research into poorly-understood subjects. • Uncover issues and topics that are later used in broader data collection using quantitative techniques (e. g. surveys) • Significant information from non-verbal signals can be observed, e. g. hesitation, passion, evasiveness, nervousness, anger, happiness, boredom. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 5
Interviews ADVANTAGES • Subjects speak, not write, so answers tend to be longer and more detailed. • The interviewer’s presence may encourage subjects to be more cooperative. Most people want to be helpful. • Unlike surveys/questionnaires, questioning is not pre -fixed. The order and content of questions may change dynamically to pursue interesting or unexpected answers. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 6
Surveys & Questionnaires DISADVANTAGES • Anonymity of subjects may encourage some to lie or boast. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 7
Surveys & Questionnaires DISADVANTAGES • Difficult or sensitive questions may be left unanswered. – Using data from surveys/questionnaires that have missing answers may introduce bias into stats. – The reason that the answers were not given to certain questions may in itself be significant and need further investigation. – E. g. if 90% of females answer Q 8, but only 15% of males answer it, something is clearly interesting and important about the issue in Q 8! VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 8
Surveys & Questionnaires DISADVANTAGES • If data collectors select subjects who are passing by (e. g. in shopping centres), an unconscious bias may make the collector tend to select people of a certain age, sex, ethnicity, status or appearance which may bias the data. – E. g. young female data collectors may be nervous to stop large young men to ask for their participation. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 9
Surveys & Questionnaires DISADVANTAGES • If subjects are volunteers: – it may be harder to to enforce cooperation. – may introduce self-selection bias – the fact that people volunteer to answer questions may mean • they have a vested interest in the subject • they are of a certain personality type that makes them want to help other people VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 10
Surveys & Questionnaires DISADVANTAGES If subjects are not volunteers: • They may resent being forced to respond, and give misleading answers VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 11
Interviews vs Surveys & Questionnaires • Interviews slowly collect small amounts of deep but narrow insights person by person. – Lots of “Why…” questions. e. g. “Why do you drink alcohol? ” • Surveys and questionnaires quickly collect lots of less personal and more shallow data from a broad range of people. – Lots of “What…” questions. E. g. “What types of alcohol do you drink? How much? How often? ” – Free-form extended textual answers are rarely used. – Most answers are numerical or chosen from pre-made limited lists of options. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 12
Interviews vs Surveys & Questionnaires • Interviews often ask different questions of different people, so answers can be hard to compile and compare. • Surveys and questionnaires ask the same questions of all respondents, so answers can be easily compiled, averaged, sorted, graphed, compared. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 13
Open and Closed Questions • Interviews usually ask open-ended questions. – No fixed format, length or type of answer is enforced. – Questions usually seek opinions or reasons. • Surveys and questionnaires ask close-ended (or closed) questions. – Answers have a fixed type and length of response (usually from a limited list) making all answers consistent and comparable. – Questions often seek factual information. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 14
Surveys & Questionnaires • Closed questions rely on researchers already knowing what the relevant answers are… VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 15
Open and Closed Questions Note how this openended question allowed an answer that the restrictive closed question did not. Acquiring this knowledge through an earlier interview process may have improved the quality of the closed question in the questionnaire. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 16
Surveys & Questionnaires • So questionnaires/surveys often rely on previous exploratory research undertaken with interviews using open-ended questioning. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 17
Surveys & Questionnaires ADVANTAGES • A lot of data can be quickly collected. • Quantitative data need not be encoded, so it’s quick to process. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 18
Surveys & Questionnaires ADVANTAGES • Online data collection – Can be electronically validated for existence, type and range. – Can collect data 24/7. – Can reach respondents in remote places. – Can deliver data immediately after its collection. – Can be modified or amended instantly, if necessary. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 19
Surveys & Questionnaires Data input techniques • Radio buttons. One, and only one answer must be chosen from a limited list of options. – Answer cannot be left blank – No more than one item may be selected. Named after old-style car radios with a series of buttons that are pushed in to select a pre-tuned station. Pressing any button automatically de-selects whichever button is already pushed in. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 20
Surveys & Questionnaires Data input techniques • Check boxes (tickboxes). None or any number of items may be chosen from a limited list of options. – Answer can be left blank – Several options may be selected. Named after the American term “check”. We say “tick” VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 21
Surveys & Questionnaires Data input techniques • Likert scale. A response is chosen from a list of discrete options on a scale. – To make encoding easier, cannot choose “mid-way” values. Named after its inventor, psychologist Rensis Likert VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 22
Surveys & Questionnaires Data input techniques • Sliding scale. A response is chosen from a continuous range between pre-set limits. Named after inventor, Mario Sliding* VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com. *This is a lie 23
Surveys & Questionnaires Data input techniques • Textbox. Allows a free-form, variable-length response. • Very hard to validate much apart from existence. Irrelevant sidebar, ‘cos I’m getting bored The conventional “Save” icon uses an image of the extinct 3. 5” floppy disk. You will probably never have seen an actual floppy disk, but the icon is still used in MS Office in 2016. The next generation will look at the lightbulb icon for “Good idea” and never ask what thing is, even though they’ve never seen a real one. But the icon will be used for another few decades. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com. 24
Surveys & Questionnaires Tips • Answers should allow the entry of any valid data • Give TWO reasons why the following “Incorrect” example is incorrect. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 25
Surveys & Questionnaires • It does not allow ages <18 or >55 to be entered. • Ranges overlap – does a 25 year old choose option 1 or 2? VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 26
Surveys & Questionnaires • Also notice the “No answer” option for questions that are potentially sensitive. Without the option, some people may lie, and data loses its integrity. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 27
Surveys & Questionnaires • Also notice that a range of ages may be more comfortable for people who don’t want to reveal their exact age. Right, ladies? VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 28
Surveys & Questionnaires • Also notice that a range of ages may be more comfortable for people who don’t want to reveal their exact age. • Right, ladies? VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 29
Surveys & Questionnaires • Sorry, ladies… • Errr, ‘women’? • Be sure to visit my other great slideshow for the SAT: ‘How To be Gender Inclusive when Writing for a Global Audience’. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 30
Observation • Involves watching people behave naturally in their normal environment. • Can reveal facts that questions may never have anticipated. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 31
Observation • Observation reveals what really happens, rather than what subjects say happens. • Subjects often mis-report their behavior out of embarrassment, vanity or fear of punishment. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 32
Observation • E. g. doctors and nurses in a hospital are surveyed about how rigorously they wash their hands after handling patients. • The survey indicates 100% compliance. • Observation reveals 30% compliance. • Survey respondents may not deliberately lie, but can misremember, or not realise what they actually do. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 33
Observation • E. g. teachers are surveyed about whether they treat male and female students fairly. • The survey indicates 100% compliance. • Observation reveals 50% compliance, according to what researchers think of as ‘fair’. • Survey respondents may not deliberately lie, but may interpret “fairness” very differently. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 34
Observation • Observer must not interfere or affect subject’s behaviour. • Observer should remain objective and not become emotionally-involved with subjects (“go native”) • Observers need a checklist of behaviours to look for and record, rather than just record what they want. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 35
Observation • Observation may be covert (hidden) e. g. via CCTV cameras. • Observation may be overt (undisguised) via the physical presence of the observer. • Subjects may be made aware or left unaware of the observation, and may have or have not given permission to be observed. – Legal and/or ethical issues of privacy may arise. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 36
Querying of resources • Refers to searching datasets to extract useful information from the masses of data. • May query primary data (original data collected personally by the researchers) or secondary data (data collected by other people, e. g. government departments like CSIRO) • Many providers of secondary data provide an API (application programming interface) to allow outsiders controlled access to their datasets. – E. g. the Australian Bureau of Meteorology lets developers create apps that extract weather forecast data from its databases. VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 37
Querying of resources Queries may use SQL or QBE* • SQL = Structured Query Language. A standardised format for selecting data from databases. E. g. SELECT * FROM Book WHERE price > 100. 00 ORDER BY title; • QBE = Query By Example. The user types what they want to find in the field they want to find it. *These terms and techniques are not examinable VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 38
Querying of resources Queries may use SQL or QBE* • QBE = Query By Example. Users type what they want to find in the field they want to find it. *These terms and techniques are not examinable VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 39
Other data collection techniques • • Hosting a web discussion forum Providing online chat Supplying a web form to request information Creating a webpage, blog etc to get information Allowing visitors to upload files Publishing an online database Giving access to an intranet or extranet VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 40
VCE IT THEORY SLIDESHOWS 2016 -2019 study design Mark Kelly mark@vceit. com These slideshows may be freely used, modified or distributed by teachers and students anywhere but they may NOT be sold. they must NOT be redistributed if you modify them. This is not a VCAA publication and does not speak for VCAA. Portions (e. g. exam questions, study design extracts, glossary terms) may be copyright Victorian Curriculum and Assessment Authority and are used with permission for educational purposes. Thanks, guys! VCE IT slideshows © 2015 -2019 Mark Kelly, vceit. com 41
THANKS! Because you’ve been so good, here’s a picture you can look at while your teacher works out what to do next Visit vceit. com for more goodies 42
- Slides: 42