QUALITY ASSURANCE IN ASSESSMENTS: DO MCQS SCORES CORRELATE WITH SEQS SCORES IN INTERNAL ASSESSMENT MBBS EXAMINATIONS?
Byline: JALEEL A. and KHANUM Z.Key Words: Reliability, Validity, Psychometric analysis, Correlation SEQs and MCQs.
INTRODUCTION
Quality Assurance is of utmost importance in medical education to maintain standards of not only teaching but also assessment (Sadaf et al., 2012). Assessment is meant to measure progress of training of the students towards the achievement of outcomes (Palmer and Devitt, 2006). Assessment is a mean of communicating between students and the teachers and provides feedback to the stakeholders (Case and Swanson, 2003). Assessment can be formative or summative.
It includes various formats like Multiple Choice Questions (MCQs), Short Essay Questions (SEQs), viva voce and practical examinations. Each format has its advantages and disadvantages. MCQs are mostly used worldwide in undergraduate examinations because, they are not only cost effective, feasible and requires less human resource but also can test higher cognitive level of learning according to blooms taxonomy i.e. interpretation and problem solving (Patel and Mahajan, 2013). MCQ assess the wider aspects of subject, SEQs may test their expression and depth of knowledge and viva can test their confidence and concept about subject (Singh, 2006). MCQs and SEQs are mostly used as tool for assessment of cognitive domain in most of the medical schools. Correlation of one assessment tool with the outcome of the other measuring the same skill is also an important aspect in quality assurance of the assessment (Pande et al., 2013; Collins, 2006; Tarant and Ware, 2008).
Study done by Mahmood (2015), correlates the MCQs and SEQs scores in written undergraduate ophthalmology assessment. Scores of MCQs and SEQs of fourth year MBBS in send-up examination were correlated for validity and reliability. Moderate correlation was seen in the MCQs and SEQs scores, split half reliability of MCQs showed moderate correlation while inter-rater reliability of SEQs scores showed high correlation (Mahmood, 2015).
Similar study was done to evaluate modified essay questions (MEQs) and MCQs in undergraduate medical students in department of Medicine, Qassim KSA.
40% MEQs and 60% MCQs assessed problem solving skills while rest were of recall and comprehension level. Item writing flaws were identified in 16%MEQs and 12% MCQS (Moeen-Uz-Zafar and Ajarralah, 2011).
Study by Sharma and Mutalik, 2014 correlated the scores of MCQs and SEQs in undergraduate third year examination in otorhinolaryngology in medical school at Martitus. Significant correlation exists between SEQs and MCQs scores, which indicates the performance to be independent of the type of questioning. Analysis of the scores shows that the top-20% scorers and bottom-20% scorers did not correlate significantly (Sharma and Mutalik, 2014).
MCQs and SEQs scores were correlated in the assessment in semesters by 121 nursing students in courses of mother and child health examinations at Shahrekord Medical Sciences Iran. Positive correlation was obtained between the two, however students score better in the MCQs (Delaram and Sharifi, 2014).
A retrospective study was performed by Mujeeb et al., (2010) in which performance of 533 students were analyzed in pharmacology. Statistically significant correlation exists between MCQs and SEQs in 433 students who passed, while poor correlation exists for students who failed the examination.
The objective of the study was to assess the correlation of scores of MCQs and SEQs in MBBS examinations at medical colleges/universities.
METHODOLOGY
ERIC, Medline, Pakmedinet, PubMed, Pubget were used as databases for review of literature. CINAHL and Web of science were not considered. Original peer reviewed full text research articles were included, while those which were non peer reviewed, or abstracts only were excluded from the study. MCQ, SEQ and correlation were used as search terms. The relevant studies were quoted as reference. Twelve studies reported correlation of MCQs and SEQs and dominant role of MCQs on outcome i.e. the performance of the students.
Table 1: Review of Literature Studies on Correlation between MCQs and SEQs.
###Concept/Theoretical###Type of
Author/Date###Topic/###Method###Findings
###model###study/
Anila and###Correlation of###Scores of MCQs and SEQs###Pearson's###Correlational###Moderate correlation
Zohra###scores of MCQs and###are correlated in###correlation###study###was found in SEQs
###SEqs in###biochemistry and###and MCQs in both
###biochemistry and###physiology###subjects
###physiology
###undergraduate
###Mock examination
###in Pakistan
Kumar et al###Comparison of###Higher scores in MCQs###Pearson's###Cross###Students scored
2017###scores of MCQs and###compared with SEQs###correlation###sectional###higher in MCQs
###SEQs in physiology###study###compared with SEQS
Mahmood,###Correlation of MCQ###Scores of MCQs and SEQs###Pearson's###Correlational###Moderate correlation
2015###and SEQs scores in###were correlated in###correlation###study###Found in scores of
###ophthalmology###undergraduate###MCQs. Moderate
###undergraduate###Ophthalmology sent up###reliability was seen in
###examination###examination for validity###MCQs
###and reliability
Sharma and###Correlation of###Scores of MCQs and SEQs###Pearson's###Descriptive###Strong significant
Mutalik,###MCQs and SEQs in###were correlated###correlation###study###correlation was found
2014###undergraduate###between scores of
###Otolaryngology###MCQs and SEQS
###examination
Delaram and###Correlation of###Nursing courses in###Pearson's###Descriptive###Positive significant
Sharifi, 2014###MCQs and SEQs in###semester were evaluated.###correlation###study###correlation was
###Nursing###Correlation was done###obtained. Students
###examination of###between scores of MCQs###perform better in
###mother and child###and SEQS at Shahrekord###MCQs
###examinations###Medical Sciences Iran.
Mujeeb, 2010###Correlation of###Scores of MCQs and SEQs###Pearson's###Cross###Positive correlation
###MCQs and SEQs in###were correlated in 533###correlation###sectional###between students
###Pharmacology###students in pharmacology###study###who passed and no
###correlation between
###students who failed
Mehta and###Item analysis of###Difficulty index,###Item analysis###Descriptive###62% items were in
Mokhasi,###multiple choice###discriminatory index and###study###acceptable range of
2014###questions- An###distractor effectiveness###Difficulty index, 52%
###assessment of###was determined in 100###items were excellent
###assessment tool###first year MBBS students###in terms of
###discrimination index
###and 35.3 % were
###nonfunctional
###distractors.
Bodkha, 2012###Effectiveness of###Scores of SAQ, MCQs and###Scores of###Descriptive###Performance in Short
###MCQ, SAQ and###MEQs were compared###Twenty high###study###answer questions
###MEQ in high and###achievers###(SAQs) is better than
###low achievers###were###MCQs and MEQs
###compared
###with twenty
###low achievers
DISCUSSION
Written examination is mostly based on SEQs and MCQs in most of the medical schools in Pakistan. The scores obtained by the students reflect their performance. Various studies have been done to correlate the scores of SEQs and MCQs to obtain the meaningful interpretation in terms of validity and reliability (Mahmood, 2015). It also provides the information about the consistency of the students' performance in MCQs and SEQS. It provides answer to certain questions as if both SEQs and MCQs assess same level of cognition or not (Kumar et al., 2017).
Study done at Fatima memorial Medical and Dental College shows significant positive correlation on MCQ and SEQ scores in second year MBBS mock examinations held in 2016. The written examination was held based on table of specifications provided by the University of Health Sciences Lahore. 143 students took Biochemistry and 149 Physiology mock examination. 45 MCQs and 9 SEQs were given to the students in each subject. Correlation of scores were significant in both subjects i.e. r = 0.39, P = 0.04 in Physiology and r = 0.59, P = 0.000 in Biochemistry (in press).
Study by Moqattash et al., (1995) showed better scores in MCQs than SEQs and concluded that students can recall the facts in MCQs and scored better, while the students who have critical thinking, interpretation and problem solving capabilities scored higher in essays.
Study by Pai et al., (2010) showed significant correlation of performance in SEQs with MCQs in the subject of pharmacology with students scored better in MCQs than SEQs at Kasturba medical college, Manipal University, Karnataka, India.
Study by Sharma and Mutalik, (2014) showed significant correlation between SEQs and MCQs in Otolayrngology examination and concludes the performance to be independent of the type of questions.
Moderate correlation (r = 0.5 P < 0.01) was seen in the study by Mahmood, (2015) in the send-up undergra-duate examination of Ophthalmology, which suggested that the two methods of assessment i.e. MCQs and SEQs are different but correlate with each other.
Another study by Mujeeb et al., (2010) reported strong correlation between MCQs and SEQs (r = 0.6, p < 0.01). Several other studies also reported correlation between MCQ and SEQ, which points out that students who perform well in MCQ also perform well in SEQ (Oyebola et al., 2000).
Study by Kumar et al., 2017 compared the scores of first year MBBS students in SEQs and MCQs in the subject of physiology.
The review concluded that scores of MCQs correlated with SEQs in most of the studies, which shows that students who perform well in MCQs also perform well in SEQS. However two modalities though correlated but act independently to assess student's performance.
Authors' Contribution
AJ: Manuscript writing. ZK: Review of Literature.
ACKNOWLEDGEMENTS
We would like to acknowledge the HEC to access the 3 online data bases, help to write this review paper.
Conflict of Interest
Author decline no conflict of interest.
REFERENCES
1. Case, S., and Swanson, D. Constructing Written Test Questions For the Basic and Clinical Sciences. National Board of Examiners, 2003.
2. Collins, J. Education techniques for lifelong learning: writing multiple-choice questions for continuing medical education activities and self-assessment modules. Radiographics: a review publication of the Radiological Society of North America, Inc, 2006; 26: 543-551.
3. Kumar,, P., Ghildiyal, J.P., Singh, R.R., and Kumar, S. Comparison of performance in physiology subject assessed by short essay type questions and multiple choice questions of first year medical students. AJMS. 2017; 8 (1): 82-84.
4. Mahmood, H. Correlation of MCQ and SEQ scores in written undergraduate ophthalmology assessment. JouJCPSP and Surg. 2015; 25 (3): 185-88.
5. Moeen-uz-Zafar, K and Aljarallah, B.M. Evaluation of modified essay questions (MEQ) and multiple choice question (MCQ) as a tool for assessment f the cognitive skills of undergraduate medical students. Int J Health Sci (Qassim), 2011; 5 (1): 39-43.
6. Moqattash, S., Harris, P.F., Gumaa, K.A,, Abu-Hijleh, M.F. Assessment of basic medical sciences in an integrated systems-based curriculum. Clin Anat. 1995; 8: 139-147.
7. Mujeeb AM, Pardeshi ML, Ghongane BB. Comparative assessment of Multiple choice questions versus short essay questions in Pharmacology examinations. The Ind J Med Sc. 2010; 64 (3): 118-24.
8. Pai, M.R.S.M., Sanji, N., Kotian, S. Comparative assessment in pharmacology multiple choice questions versus essay with focus on gender differences. J of Clin and Diag Res. 2010; 4: 2515-20.
9. Palmer, E.J., and Devitt, P. Constructing multiple choice questions as a method for learning. Ann Acad Med Singap. 2006; 35: 604-8.
10. Pande, S.S., Pande, S.R., Parate, V.R., Nikam, A.P. and Agrekar, S.H. Correlation between difficulty and discrimination indices of MCQs in formative exam in Physiology. South East Asian J Med Edu. 2013: 45-50.
11. Patel, K.A. Mahajan, N.R. Itemized Analysis of Questions of Multiple Choice Question (MCQ) Exam. Int J of Sci Res. 2013; 2: 279-280.
12. Sadaf, S., Khan, S., and Ali, S.K. Tips for developing valid and reliable bank of multiple choice questions (MCQs). Edu. Health, 2012; 25: 195-7.
13. Singh, A.K., Validity. Singh, A.K., ed. In: Tests, measure0ments and research methods in behavioral sciences, Patna, India: Bharati Bhawan Publishers, 2006: pg.93.
14. Sharma, H.S., and Mutalik, M.M. Comparison between student scores in multiple choice questions and structured essay questions at III MBBS examination in Otorhinolaryngology in a medical college in Mauritius. Int J Pharm Bio Sci. 2014; 5 (4): 1195-1202.
15. Tarrant, M., and Ware, J. Impact of item-writing flaws in multiple-choice questions on student achievement in high-stakes nursing assessments. Med. Edu. 2008; 42 (2): 198-206.
 Printer friendly
 Cite/link
 Email
 Feedback
| |
| Publication: | Biomedica |
|---|---|
| Date: | Sep 30, 2018 |
| Words: | 2171 |
| Previous Article: | REPRODUCTIVE HORMONE PROFILES OF WOMEN WITH INFERTILITY AND MENSTRUAL DISORDERS: A RETROSPECTIVE STUDY. |
| Next Article: | COMPARATIVE ANATOMY AND MOLECULAR MECHANISMS INVOLVED IN RODENT SALIVARY GLAND DEVELOPMENT. |
| Topics: | |