Survey Research Designs Parlindunga Pardede English Education Department

Survey Research Designs Parlindunga Pardede English Education Department Universitas Kristen Indonesia

Vision Mission Becoming a leading English Education Department through the three missions of higher education based on Pancasila, Christian values, and the application of information and communication technology in Jabodetabek in 2025. 1. Organizing creative English learning by applying information and communication technology. 2. Increasing the quantity and quality of research and publication in the field of English Education in both the national and international levels 3. Organizing quality scientific activities in the field English Education at the national and international levels. 4. Continuously increasing the quantity and quality of community service in the field of English Education. 5. Increasing networking with related national and international institutions in the three missions of higher education.

Definition • Survey research designs are quantitative research procedures in which investigators administer a survey to a sample or to the entire population to describe its attitudes, opinions, behaviors, or characteristics. • Unlike experimental research, survey designs do not involve a treatment to manipulate the conditions of the participants. Unlike correlational designs which relate variables or predict outcomes, survey designs merely describe trends in the data. • Surveys help identify important beliefs and attitudes of individuals, such as “Students and Teachers’ Perception of ICT Use in English learning”, “Students’ Interest in English Speech Contest”.

When to Use Survey Research? • to describe trends (e. g. parents interest in the inclusion of English in primary schools curriculum) • to identify and determine individual opinions, beliefs and attitudes about policy issues • to evaluate programs (e. g. English speaking days in a secondary school)

Types of Survey Designs • Cross-sectional designs study a single population or sample size during a single specified timeframe and provide a “snapshot” of opinion data. • Longitudinal designs study trends over time to see changes occurred, and are further classified into trend, cohort, and panel studies.

Types of Survey Designs

Types of Longitudinal Survey 2015 2016 2017 TREND • Identifies and examines changes or trends within a population over time. • Each time, Information is collected from different samples of a single population • E. g. Freshmen’s perceptions of blended learning in 2015, 2016 and 2017 2016 2015 2017 COHORT • Identifies and examines changes or trends within different subpopulations having a common defining characteristic over time. • E. g. Grade 10 of a school in three successive years. 2015 2017 PANEL • Identifies and examines the “growth” of attitudes of the sample of the same populations due to changing conditions over time. • E. g. The perception of the same group of students towards blended learning in 2015, 2016, and 2017

Survey Characteristics Sampling from a population Collecting data using questionnaires or interviews Designing instruments for data collection Obtaining a high response rate

Sampling from a population

Questionnaires and Interviews

Steps in Developing a Questionnaire 1. Decide the information required (Check latest relevant publication; and put the information to develop the framework. 2. Define the target respondents 3. Choose the method(s) of reaching your target respondents. 4. Decide on question content. 5. Develop the question wording. 6. Put questions into a meaningful order and format. 7. Check the length of the questionnaire. 8. Pre-test (pilot-test) the questionnaire. 9. Develop the final survey form

Questionnaire Framework No Aspects Indicators Item Number Question and Rensponse Types 1 Demographics • • Gender Age Semester Etc. 1 2 3 4… Neutral ; Multiple Choice 2 Access to ICT • • Ownership of PC Accessibility to internet at school Accessibility to internet at home Duration appropriateness to access internet 7 8 9 10 Neutral ; Multiple Choice 3 Interest to ICT Use in Learning • Preference to F 2 F or On Line class • Preference to read printed or digital texts for getting new information • ……. . 11 12 … Positive ; Degree of agreement in 1 to 4 scale 4 …………………………………………………. ……………………

Students Perception of Indonesian Use in English Classes Aspect: Preference

Students Perception of Indonesian Use in English Classes Aspect: Belief

Students Perception of Indonesian Use in English Classes Aspect: Role of Indonesian in Language Components Learning Aspect: Role of Indonesian for Understanding Learning Materials

Potential Question Construction Problems and Solutions to them (1) 1. The question is unclear. Identify the unclear or vague words and replace them with words understood by participants in the study. 2. There are multiple questions. Reduce the multiple questions to a single question. 3. The question is wordy. Cut out unnecessary words to simplify and shorten the question 4. The question is negatively worded or wordy. Restate or reword the question to eliminate negative connotations or leading words. 5. The question includes jargon. Jargon may not be familiar to all participants in a study. Eliminate the jargon and use words familiar to all participants.

Potential Question Construction Problems and Solutions to them (1) 6. There are overlapping responses. To avoid respondents’ confusion when answering a question, make sure that the response options do not overlap by creating distinct options. 7. There are unbalanced response options. Decide on a single response option and use it consistently for all response categories for a question. Prevent the use of unbalanced expressions, e. g. important, higly important, very important. 8. There is a mismatch between the question and the answers. The responses may not match the “action” word used in the question. Identify the verb or adjective in the question that will be the basis for the response options and create options using this word. (E. g. , if the question says “to what extent, ” the answer will say “a great extent. ” 9. The question includes overly technical language. Simplify the question so that all individuals will know the meaning of the words and can respond to the question. 10. Not all questions are applicable to all participants. If some participants cannot answer the question, include “branching” or “contingency questions. ” These questions follow the original question and provide options to include all participants

Questionnaire’s Validity & Reliability • Validity (1) internal validity; whether or not the questionnaire is in fact measuring what it claims to measure (2) external validity; whether or not those persons taking the survey by answering the questionnaire did so under similar conditions (3) construct validity; “degree to which the survey measures whatever construct is tried to measure. (Examined using factor analysis) • Reliability Concerns with whether the instrument measure what it purports to measure in a consistent manner at different time. Tested with Cronbach's alpha coefficient of reliability.

Steps in Doing Survey Research Identify the Research Problem Report and Evaluate Research Review the Literature Analyze and Interpret Data Specify the Research Purpose Collect Data Research Question Research Design (Quan, Qual, Mixed • • Sampling Instruments Protocol Analysis

Steps in Doing Survey Research 1. Identify research problem 2. Review the literature 3. Specify research purpose • Formulate research question • Design the research method 4. Collect data 5. Analyze and interpret the data 6. Report and evaluate research • Sample: ≥ 30 participants • Instruments: (Quantitative Data without treatment) • Analysis Technique: Ø Descriptive Statistics Ø t-test (for comparing 2 groups)

Example