Evaluation and Quality Assurance m EQUITY Dr Flix

Evaluation and Quality Assurance m. EQUITY Dr. Félix García-Loro, UNED, Spain Improving Higher Education Quality in Jordan using Mobile Technologies for Better Integration of Disadvantaged Groups to Socio-Economic Diversity m. EQUITY – ERASMUS+ PROGRAMME Project Number: 561527 -EPP-1 -2015 -1 -BG-EPPKA 2 -CBHE-JP

1. 1. Global Objective The design and development of an adapted curriculum in the degree of engineering Ø which responds to the requirements of modernization and accessibility of Jordan higher education based on digital learning resources for mobile devices Ø for improving the educational integration of disadvantaged students or groups at risk whose needs or socioeconomic status significantly constrain their possibilities for receiving a suitable education 2

1. 2. Specific Objectives 1. To analyze users’ needs in different contexts and curricula in the three partner universities 2. To evaluate the amount of use of mobile technology by specific groups of students, creating new chances for their inclusion in the learning process 3. To design a Mobile Digital Resources (MDR) Model to support the educational features of mobile technology in order to adapt it to the learning conditions of disadvantaged people 4. To encourage the use of digital mobile educational resources to contribute and favor equal access to education and training programs by fostering motivation of students in the learning process 5. To broaden the chances of students with special needs (disabled and of low socioeconomic status) to get a better education 6. To increase the influence of the Jordan Higher Education Quality Assurance System with the promotion and adoption of the MDR model as the paradigm of a broader remodeling in the curricula of the degree in engineering for students with special needs 7. To train teachers in the use of digital and mobile educational resources 3

University of Jordan (UJ), Jordan University of Science and Technology (JUST), Jordan. 2. Partners Princess Sumaya University for Technology (PSUT), Jordan. Plovdiv University “Paisii Hilendarski” (PU), Bulgaria. Spanish University for Distance Education (UNED), Spain. Ravensbourne Higher Education Institution (RAVE), United Kingdom. 4

3. WPs WP Name WP 1. Analysis of needs WP 2. Development of a conceptual model of mobile digital resources in the context of training groups of disadvantaged people (MDR Model) WP 3. Development and adaptation of mobile applications and digital educational resources for the needs disadvantaged groups of learners WP 4. Implementation of the package MDR in training –pilot experiment and analysis WP 5. Evaluation and quality assurance WP 6. Dissemination and Exploitation WP 7. Management of the project 5

4. WP 5. Evaluation and quality assurance Specific Objectives: Global Objective: OBJECTIVES OF WP 5 To carry out quality assurance and evaluation of the project QUALITY ASSURANCE PLAN § To design the Quality Assurance and Evaluation plans § To develop the Quality Assurance and Evaluation Plans § To create the tools needed to measure evaluation and quality § To write the final evaluation report EVALUATION PLAN 6

5. The Quality Assurance Plan A set of criteria and measures to achieve the highest possible quality standards during all phases of the project The Quality Assurance Plan is based on the international standards of the ISO/9001: 2008 certificate The Quality Assurance Plan is divided into a series of interrelated components 7

5. 1. Components of the Quality Assurance Plan Product quality: Efficiency, effectiveness, functionality Dissemination and promotion Check-ups Quality planning: Management and deadlines Quality control Planning Quality process: Evaluation of content Measurement, analysis and improvement Easiness Evaluation Implementation Quality testing phase: Outcome (impact) and satisfaction 8 Reliability and validity 8

5. 2. The Evaluation Plan is one of the components of the Quality Assurance Plan Through the evaluation of each phase of the project It describes a set of criteria and measures to achieve the highest possible quality standards Identify the weaknesses of the project trying to overcome them And of each WP of the project Reach conclusions in order to get better results Highlight the strengths and develop them to the highest level 9

5. 3. Evaluation of each WP’s Ø WP : 1 to 7 Ø Global objective Ø Variables Ø Tools Ø Results 10

5. 3. 1. WP 1 - Analysis of Needs GLOBAL OBJECTIVE To ensure the highest possible quality in the analysis of the educational needs of the disadvantaged groups of people that have taken part in the m. EQUITY project, and how the pedagogical features of the mobile technology designed with the MDR Model used in this project have been adapted to their specific needs. Addressee Partners Tools Questionnaire on the assessment of the analysis of needs detected Results obtained in the evaluation are very positive as all questions have scored above 3 in a scale of 4. 11

5. 3. 2. WP 2 -MDR Model Development GLOBAL OBJECTIVE To assess the development of the conceptual model of educational digital resources for training of disadvantaged groups of people with special needs (MDR model). Addressee Partners Tools Questionnaire on the MDR conceptual model Results Partners are satisfied and have granted a mean of 3. 64 out of 4 12

5. 3. 3. WP 3 -Development and adaptation of mobile applications and digital educational resources for the needs disadvantaged groups of learners GLOBAL OBJECTIVE To evaluate the development and suitability of the mobile application system with augmented reality and the other technological resources of the MDR model to the educational purposes in the different scenarios proposed in the project Efficiency/Usefulness/Utility Variables Tools • Questionnaire for students on the evaluation of efficiency, usefulness and utility of mobile applications and educational digital resources • Questionnaire for teachers on the evaluation of efficiency, usefulness and utility of mobile applications and educational digital resources Results Recipients are satisfied with the MDR system, obtaining in all cases, (variables efficiency, usefulness, utility) a mean above 3 out of 4.

5. 3. 4. WP 4 - Implementation experiment and analysis of the package MDR in training –pilot GLOBAL OBJECTIVE To measure the satisfaction of teachers and students taking part in the courses that use the MDR model, according to the different scenarios on which the project is based, thus ensuring the highest possible quality in the implementation and analysis of this model. • • Scenarios Main Scenario: Jordan Universities Other Scenarios: • • Tool Scenario 1: 2: 4: 5: General Secondary Schools for Children with Special Needs Higher Council for Affairs of persons with Disabilities Gaza refugees camp Our Lady of Peace Center The "Pre" phase, prior to using the mobile devices of the MDR model (teacher and students) The "Post" phase, after using the mobile devices of the MDR model (teacher and students) 17 Questionnaires 14

5. 3. 4. WP 4 - Global Hypothesis The use of educational mobile digital resources, designed with mobile technology, within the teaching-learning process increases the motivation of students; promotes socioeducational inclusion of students with special needs; and favors their performance thus improving the quality of the educational system. Hyphotesis 1 Hypothesis 2 Hypothesis 3 15

5. 3. 4. WP 4 - Hypothesis, Variables and Results Hypothesis 1 The use of educational digital resources, mobile devices, and augmented reality et al. in the teaching-learning process increases the motivation of students with special educational needs and socioeconomic and labor disadvantaged • Variables Results • Dependent Variable (DV): Motivation Independent Variable (IV): educational digital resources Variable Motivation –results above 3 out of 4. 16

5. 3. 4. WP 4 - Hypothesis, Variables and Results Hypothesis 2 The use of educational digital resources, mobile devices, and augmented reality et al. in the learning process foster the socio-educational integration of students with special educational needs and socioeconomic and labor disadvantaged Dependent Variable (DV): Autonomy, self-esteem, socioeducational and labor integration • Independent Variable (IV): educational digital resources • Variables Results Autonomy - Both teachers and students have given very high scores in the pre and post phases, above 3 out of 4. Self-esteem - with mean very close to 3 out of 4, both in the pre and in the post phases. Sociocultural and labor integration - Students and the teachers think that the MDR model is very beneficial, with means in both phases close to 3 out of 4. 17

5. 3. 4. WP 4 - Hypothesis, Variables and Results Hypothesis 3 The use of educational digital resources, mobile devices, and augmented reality et al. in the learning process increases performance of students with special educational disabilities as well as socioeconomic and labor disadvantaged. Variables Results • Dependent Variable (DV): Performance • Independent Variable (IV): educational digital resources Variable Performance: In all cases the means given to courses by teachers and students are close to 3 out of 4 18

5. 3. 5. WP 6 - Dissemination and Exploitation GLOBAL OBJECTIVE To check the level of dissemination of the outcome of the project. Population Partners Tools Questionnaire on the evaluation of dissemination and exploitation of the m-EQUITY project Results Total mean of 3. 44 out of 4, the highest score that can be obtained 19

5. 3. 6. WP 7 - Management of the project GLOBAL OBJECTIVE To evaluate management and planning of the project Variables Co-ordination and management Communication among partners PMC tools Tools Questionnaire on evaluation of management and coordination, communication and tools (pre and post phase) Results Management and coordination: In the pre phase the total mean was 3. 53 out of 4, while the results of this variable increased in the post phase to 3. 6 out of 4. Communication: The mean was above 3 on a scale of 4. PMC tools: In the pre phase the total mean was 3. 39 out of 4, while the results increased in the post phase and amounted to 3. 45 out of 4. 20

Results : motivation & Students by university STUDENTS - Universities Variable: Motivation 4 3. 47 3. 5 3 2. 86 3. 46 2. 87 2. 86 2. 5 2 1. 5 1 0. 5 0 JUST UJ Pre Post 21 PSUT

Results : motivation & teachers by scenario Variable: Motivation 4 3. 5 3 2. 5 2 1. 5 1 0. 5 0 3. 33 2. 52 3. 27 3. 5 3. 2075 2. 8475 3. 29 3. 25 Teachers - Scenarios 3. 67 2. 56 3. 54 2. 62 2. 82 3. 02 4 3. 5 3 3. 43 2. 85 3. 07 3. 5 3. 36 2. 78 2. 76 3. 07 2. 5 1. Generally 2. The group 3. Students 4. It is 5. Generally speaking, in class looks prefer to necessary to speaking, this students do for new carry out reward group of not fear learning more complex students for students is failure challenges tasks in order them to finish quite to learn the motivated for further. assignments learning 6. Generally speaking, students do not worry about the outcome of their work 7. Students have attention difficulties and need constant assistance 2 1. 5 1 0. 5 0 Scenario 1: General Scenario 2: Higher Secondary Schools for Council for Affairs of Children with Special persons with Needs Disabilities Scenario 4: Gaza refugees camp 22 Scenario 5: Nazik Al Hariri welfare center for special education

Results: autonomy & students by university STUDENTS - Universities Variable: Autonomy 4 3. 5 3. 44 3. 08 3. 07 3. 5 3. 08 3 2. 5 2 1. 5 1 0. 5 0 JUST UJ Pre Post 23 PSUT

Results: autonomy & students by scenarios Variable: Autonomy STUDENTS - Scenarios 4 3. 5 3 3. 64 3. 16 3. 49 2. 91 3. 21 3. 6 3. 4 3. 07 2. 5 2 1. 5 1 0. 5 0 Scenario 1: General Scenario 2: Higher Secondary Schools Council for Affairs for Children with of persons with Special Needs Disabilities Pre Scenario 4: Gaza Scenario 5: Nazik Al refugees camp Hariri welfare center for special education Post 24

Pre 26. I to to be ily rt . as. . cl . . o. as . . . in e . . e. . . n. ie on re nd te at m pa th . . un . as. . at fr m to al of ke ta fa ot nn ca ue e lik to lo t ss rc la a er ef ve ha pr . D . I 25 24 . I 23 . I he n io nt ng be i cl 2. 23 ot te at of 3. 26 22 y y pa se 3. 31 . M . I . . . m 3. 07 21 20 ith w nd te at io n la t ot nn ca re ec au . B . I he . T 4 3. 5 3 2. 5 2 1. 5 1 0. 5 0 19 18 17 Results : sociocultural and labor integration & Students by university STUDENTS - Universities 3. 43 3. 34 3. 33 3. 28 3. 25 3. 18 3. 13 3. 11 2. 99 2. 73 2. 93 2. 88 2. 57 2. 48 2. 28 2. 16 Post 25

Results: sociocultural and labor integration & students by scenario STUDENTS - Scenarios Variable: Sociocultural and labor integration 4 3. 48 3. 5 3 2. 88 3. 35 3. 37 3. 31 2. 78 2. 71 2. 78 2. 5 2 1. 5 1 0. 5 0 Scenario 1: General Scenario 2: Higher Secondary Schools for Council for Affairs of Children with Special persons with Disabilities Needs Pre 26 Scenario 4: Gaza refugees camp Post Scenario 5: Nazik Al Hariri welfare center for special education

Conclusion In conclusion, it can be affirmed that the m. EQUITY project has been a success from its planning, during its development and in its complete implementation, with all the working packages obtaining very satisfactory results 27

THANKS!! clarapm@dieec. uned. es mcastro@ieec. uned. es mjalbert@edu. uned. es fgarcialoro@ieec. uned. es 28