Subjective Approach to EAssessment Through Similarity Measure RAJILAWAL

Subjective Approach to E-Assessment, Through Similarity Measure. RAJI-LAWAL HANAT Y. AND AKINWALE ADIO T. LAGOS STATE UNIVERSITY OJO, LAGOS, FEDERAL UNIVERSITY OF AGRICULTURE ABEOKUTA Presented By: Raji-Lawal H. Y.

Presentation Outline � Introduction � Research question � Literature Review � Similarity Measure � Modified Jaccard Coefficient SM � Similarity Measure computation � Discussion of Result � Conclusion � References

Introduction � E-assessment is a branch of e-learning, it breaks all barriers associated with location and physical distance in education. � E-assessment will serve as an aid to examiners in terms of stress relieve. � Subjective approach to e-assessment relies on intelligent assessment technologies. This has to do with encoding the system in such a way that it can act as an assessor.

Introduction � The traditional method of marking is based on preparing a marking guide, and matching the marking guide with the student’s answer. The student is graded based on the similarity of answers supplied by him/her and the marking guide. � This research explored approaches that can be involved in matching the marking guide and answers supplied by students, specifically similarity measures.

Introduction Similarity means fidelity of property conservation between an object and its reference � It is also defined as sharing of features of ideal objects � Similarity measure methods are mathematical formulas for evaluating how similar two entities are. �

Research Questions � How can subjective exams be marked automatically, using similarity measure method? � How can string positioning be tracked in cases of matching strings of the same character in length n-gram with different positioning?

Literature Review � Mr. A. A Shinde and Ms sumediachkhandre( 2013) researched on fast implementation, conduction and assessment of questions in the exam, both objective and subjective. � In the objective section students were provided with questions with four options out of which they have to pick an option. � Short answer is provided for subjective questions by students through FCK editor. Ms Paint is used for depicting diagrams by students for illustrative explanation.

Literature Review � The correct answers stored in data base are matched with answers supplied by students; and assessment is done based on this. � In the subjective section, matching is done by searching for keywords in the students’ answer, if the keyword is present in the student’s answer the mark is 1 otherwise 0. Thus 0 is assigned for irrelevant keyword, spelling mistakes, and keyword transposition.

Literature Review � Tom Mitchell(2013) researched on intelligent assessment technologies commercially available assessment engine. � The engine was designed and developed for Dunde university medical school. It marks answers presented by students in short phrases. The system marking scheme states acceptable and unacceptable answers for each question. � The system marks based on the syntactic – semantic, marking guide supplies and saves the mark appropriately.

Literature Review � This assessment is based on natural language processing. The advanced objective computer based assessment of fill in the gaps questions, a better approach of fill in the gap objective exams has the following features: � It allows professional judgment to contribute to playing a key role in the marking process. � It changes the emphasis on the way the marking technology is perceived, and thus the system serves as a tool that supports examiners during making. � It allows examining bodies to employ free-text items in computer based assessments; it thus makes marking and marking moderation efficient and costeffective.

Literature Review � The system can thus serve the following purposes: � Act as a second marker for comparison with human marker. � Targeting borderline candidates for remarking by human marker. � Highlighting specific responses requiring a human ‘second opinion’. � Improving the skills of item writers and more markers.

Literature Review � Aahul b. Diwate et al(2013) did a systemic review on pattern matching, from which it was deduced that internet search is best done by using Boyer Morris and Knuth Morris Pratt algorithm � Rami H Mansi and Jehad Q. Odeh(2009), researched on improving Naïve string matching. It is actually to improve the slow processing of Naïve string matching by varying its algorithm to reduce its processing time. The variation is based on the matching position.

Literature Review � Vidya Saikrisha et al, researched on string matching and its application in diversified field. Fields like intrusion detection, information extraction, question answering, text clustering e. t. c.

Literature Review � A. A Shinde, M. S Sumedha Chokhandre(2013), researched on the draw backs of objective EExam, and subjective E-exam is seen to posses the strengths of the former. They evaluated students answer in objective by matching student’s answer with answers in the data base, which must be an exact match for the answers to be correct. The subjective evaluation is done by comparing student’s answer with the correct answer in the data base. The comparism is done based on extraction and matching of key words.

E-assessment and Similarity Measure � E-assessment is a tool for evaluating student’s answer electronically, it can only achieve this by automating the traditional marking method, which entails comparing the marking guide with the student’s answer. � In this case, key words are extracted from students answer, and compared with the extracted key words from correct answer. � Comparison can be achieved using similarity measure.

Similarity Measure � Similarity measure has to do with how alike or different two entities are. � Similarity between two elements can be viewed in three different perspectives: ◦ Their degree of commonality ◦ Their degree of disparity ◦ Are they perfect match

Similarity Measure � The research explored similarity measures such as: � Jaccard � jaccard coefficient � overlapping � These measures are simple measure for comparing strings, they compare strings without taking cognisance of position of characters within a string. � This research proposed a modified similarity measure, which is Modified Jaccard Coefficient similarity measure.

Selected Similarity Measures � Overlapping Similarity Measure � Cosine Similarity Measure � Jaccard Similarity Measure � Dice Similarity Measure � Jaccard Coefficient

Similarity Measure Method � Jaccard Similarity � � � 1 2 �A L � P R 3 G O 4 O G 5 R R 6 I A 7 T M 8 H. 9 M.

Similarity Measure Method � Overlapping Similarity

Similarity Measure Method � Cosine similarity

Similarity Measure Method � Dice method AL LG � � PR RO � GO OR RI IT OG GR RA AM TH HM

Similarity Measure Method(Jaccard coefficient) � P- Number of variable positive for both object � q- Number of variable that are positive for the ith object and negative for the jth object � r- Number of variables that are negative for the ith object and positive for the jth object � s- Number of variables that are negative for both objects. � t = p + q + r + s = total number of variable

Modified Jaccard Coefficient SIMILARITY MEASURES � ∑* ={C| C contains set of words over an alphabet} � Given sets A anb B, �A = � B = � Modified Jaccard Similarity = Jaccard Similarity Coeficient - Pn

Modified Jaccard Coefficient SIMILARITY MEASURES � Cb - number of characters common to both words � Ua - number of characters unique to A � Ub - number of characters unique to B � Pn – Number of mis-positioned characters

Similarity Measure Computation � This research explored five similarity measures with complexity big theta(n) � Including the modified jaccard coefficient similarity measure. � Patterns and texts were matched using the similarity measures listed above. The table below depicts the similarity values of the patterns with respect to the similarity measure methods.

Similarity Measure Computation (Table 1: Comparism of similarity measure methods) Patter/text Jaccard Similarity Measure Board 1 Jaccard Coeficient Similarity Overlapping Similarity Measure 1 1 Cosine Similarity Measure Dice Similarity Measure 0. 94 0. 2 Modified Jaccard Coefficient Similarity Measure 0. 4 Broad Eat 1 1 1 0. 78 0. 5 0 1 1 1 0. 71 0 0. 333 1 1 1 0. 71 0. 50 0. 333 1 1 1 0. 81 0. 60 0. 66 1 1 1 0. 87 0 0. 40 Ate Was Saw War Raw Latent Talent Wives

Similarity Measure Computation(Table 1: Comparism of similarity measure methods) Best 1 1 1 0. 97 0. 67 0. 50 1 1 1 0. 78 0 0 1 1 1 0. 99 0. 66 0. 571 1 0. 99 0. 7 0. 818 1 1 1 0. 99 0. 71 0. 75 Bets Grape Pager Euglena Euglean Tracheophyta Trachoephyta Protozoa Protozao

Discussion of Result � Table 1 shows that Jaccard and Overlapping similarity measure does not consider absolute position N-gram of character, but rather deals with character composition in length of the pattern and text. � Cosine and dice considers absolute position N-gram but not as explicit as modified Jaccard coefficient similarity measure.

Discussion of Result � Modified jaccard Coefficient similarity measure considers character composition in length , and absolute position N-gram of strings. � It gives a better similarity value compared to others when it comes to dealing with strings with mis-positioned characters. � It enhances detection of spelling errors. It gives an insight of the distance between the pattern and text, based on the absolute position N-gram of the character that they encompass.

Discussion of Result Modified jaccard coefficient can serve as a standard measure for matching student’s answer with the marking guide. � Its evaluation approach is better than exact matching, which is popularly embraced in question answering. �

Conclusion � Most similarity measures doesn’t consider absolute position N-gram of characters in strings. The few that does it runs with a greater complexity. � This modified Jaccard coefficient detects this easily, and runs with a smaller complexity. � This method is efficient and handles problems of this kinds absolutely. � Compared to jaccard similarity measure, it is an improved and modified version of Modified jaccard Coefficient similarity measure.

Conclusion � Due to the strength of modified Jaccard similarity measure in detecting mis position of characters in string with length N-gram, it stands as a better approach to comparing the student’s answer and correct answer, in an electronic assessment engine.

References Aahul B. Diwate, Prof Statish J Alaspurkar 2013), Systemic review on pattern matching, international journal of advanced research in computer science and software engineering. � Iftikhar Hussain etal(2013), Improved approach for exact pattern matching, International journal of computer society. � Pattern Matching Concept, Wikipedia. org � Rami H Mansi and Jehad Q Odeh(2009), Improving Naïve string matching , Asian Journal of Information Technology. � Vidya Saikrishna etal( 2012), String matching and its application, international journal of computer society. � Lovic c, Baud R. H(2000), Fast exact string pattern matching algorithm adapted to the characteristic of the medical language, J AM medical information association. �

References Adegbija M. V(2012), The new technologies and the conduct of E-Examinations: A case study of National Open University of Nigeria. � Mr A. A Shinde, Ms. Sumedha Chokhandre(2013), A novel approach with subjective assessment for E-Examination, International Journal of Engineering research. � Alyona Medelyan, Natural Language Processing keyword extraction tutorial with RAKE and Maui, Air pair writing competition(2015). � Adio Akinwale, Adam Niewiadomski(2012), Effective similarity measure in electronic testing at programming languages, Journal of applied computer science. � Adio Akinwale, Adam Niewiadomski(), New similarity measure in electronic tests at programming languages, Tewi Piziferna Internalyzane �