Year :
2022
| Month :
December
| Volume :
16
| Issue :
12
| Page :
JC01 - JC04
Full Version
Utility of Mini-Clinical Evaluation Exercise in the Assessment of Medical Interns in Simulation based Medical Education: A Pilot Study
Published: December 1, 2022 | DOI: https://doi.org/10.7860/JCDR/2022/57091.17215
Gajanan Chavan, Tejavenu Yedla, Shradha Patel
1. Head, Department of Emergency Medicine, Jawaharlal Nehru Medical College, Wardha, Maharashtra, India.
2. Tutor, Department of Virtual Lab, Jawaharlal Nehru Medical College, Wardha, Maharashtra, India.
3. Tutor, Department of Virtual Lab, Jawaharlal Nehru Medical College, Wardha, Maharashtra, India.
Correspondence Address :
Dr. Tejavenu Yedla,
School of Virtual Learning, Sawngi Meghe, Maharashtra, India.
E-mail: teju.yedla@gmail.com
Abstract 
Introduction: Medical graduates skill assessment in simulation is underdeveloped and needs due attention in making them proficient. The mini-Clinical Evaluation Exercise (mini-CEX) could be an important tool in assessing and making them perfect, through Competency-Based Medical Education (CBME).
Aim: To determine utility of mini-CEX as a definitive and reliable assessment tool in high fidelity medical simulation.
Materials and Methods: This pilot study was conducted at School of Virtual Learning (SVL) affiliated with JNMC, Datta Meghe Institute of Medical Sciences (Deemed to be University) Sawangi (M), Wardha, Maharashtra, India between 1st November 2021-31st December 2021, on 50 medical interns. The learners were taught of basic clinical examination on SimMan simulator along with training in mini-CEX. The case scenarios on Hypertensive, Valvular, Acute Coronary artery disorders and dysrhythmias cardiac medical emergencies were allotted and the performance of administrators and learners were assessed by mini-CEX. The mini-CEX rating form was used to evaluate the competence and student satisfaction score form was sent to learners for their feedback. The data was collected on online Google form and was tabulated and analysed, descriptive statistics, mean with standard deviation was calculated.
Results: The mean age of participants was 22.5±0.53 years. There were 15 males (30%) and 35 females (70%). Average satisfaction score for the assessors was 7.4±1.2 and for the interns was 8.2±0.2.
Conclusion: Excellent level of satisfaction was observed in the learners, as well as, the administrators by use of mini-CEX format. Hence, this exercise can be applied in accessing high fidelity simulation.
Keywords
Assessment tools, Cardiac emergency scenarios, Competency based medical education, High fidelity manikin
Introduction
Medical students are increasingly getting educated with profound emphasis on a curriculum based on cognitive, psychomotor, and affective domains of learning. There is a worldwide change in methods of medical education and simulation is providing an advantage as it is providing hands on medical practices in safe environment in repetitive fashion. As simulation offers a learning opportunity through immersion, reflection, feedback and practice without any risk of real life, which in turn offers the potential advantages in comprehensive and practical training with better and safer patient care (1). Simulation Based Medical Education (SBME) is a prerequisite for training the future clinicians in CBME, which is a paradigm shift in Indian Medical Education (IME) (1). SBME provides a real-life situation to the Healthcare Providers (HCP) to cope up competently in ethical and legal ways by imparting it in prescribed and planned manner (2).
To implement this CBME, the assessment methods associated with higher level of ‘shows how’ and ‘does’ of Miller Pyramid has received increased attention and hence recommended. Kogan J et al., emphasised the importance of direct observation of medical trainees with actual patients for performance-based clinical skills assessment (3). In our medical education system, the internship posting is limited to filling and signing of log-books and there are no means for formally assessing clinical skills (4). Internship is, thus, a critical period to practice the required clinical skills; this is very important since a large part of our Indian population resides in rural areas and primary health practitioners need to be efficient in diagnosing common disorders and procedural skills (4).
In 1995 mini-CEX was adopted by American Board of Internal Medicine (ABIM) for learner’s assessment after necessity to assess the foreign medical graduates’ skill (5). It is one of the most frequently used assessment tools to measure the competency in clinical set-up. In its original form, the mini-CEX is a 9-point rating scale organised in three levels of unsatisfactory (1-3), satisfactory (4-6), and high satisfactory (7-9) (6). The mini-CEX assessment tool uses seven different criteria for skills like medical interviewing, physical examination, clinical judgment, professionalism, counselling, efficiency and overall clinical competence (7).
Several studies have been conducted and the efficacy of mini-CEX was established in evaluating the students’ clinical skills (8),(9),(10). However, mini-CEX can be deficient and biased by certain external factors such as quality of teaching, professional competence, learners attitude and their education background (11),(12). Superior training and teaching outcome of nursing staff is being achieved by simulation and was assessed by mini-CEX at different centres (13),(14). Mini-CEX is also used in clinical competence assessment of medical graduates and was found out to be an effective tool (6),(15),(16).
Mini-CEX doesn’t need additional manpower, instruments, equipments, and patients. Moreover mini-CEX is easy and user friendly to implement (7). Hence, the present study was undertaken to introduce mini-CEX not only as an assessment tool in SBME, but also, to find out its utility as an assessment tool and also provide safe and competent medical graduates to our society.
Since, there is no standard uniform assessment protocols and tools in use for Indian medical graduates (interns) apart from logbook, the present pilot study was undertaken as an attempt to find out, if, mini-CEX could be implemented as a work place-based assessment not only in SBME, but in other clinical postings.
Material and Methods
This pilot study was conducted at SVL affiliated with Jawaharlal Nehru Medical College, Sawangi (M), Wardha, Maharashtra, India during 1st November-31st December 2021. Ethics Committee of (DMIMS (DU)/IEC/2020-21/58) Jawaharlal Nehru Medical College permitted the present study without any limitations and reservations as simulation-based study. A total of 50 MBBS interns, who were posted at SVL were enrolled and sensitised to assessment modalities of mini-CEX beforehand. The group was taught according to multiple-station mini-CEX evaluation combined with scenario simulation assessment.
Inclusion criteria: Interns who were posted in SVL for atleast two months and attending the classes regularly, were included in the study.
Exclusion criteria: Learners with fixed ideas against simulation based medical education with absenteeism and refusal of participation, were excluded from the study.
For the precise implementation of mini-CEX conduction, thorough training of assessors is mandatory. A total of five assessors working full time at skill lab, trained in SimMan scenarios and mini-CEX administration participate in the present study. Five trained standardised patients were selected to play a role of patients to give history, consent etc.
SimMan 3G PLUS (Laerdal) was used for simulation of cardiovascular system emergency scenarios like hypertensive, valvular, acute coronary disease and dysrhythmias. Mini-CEX with seven components was used as an assessment tool. After completion of mini-CEX, interns were given feedback form, probing various outcomes of mini-CEX. The module was developed and tested in three phases.
Phase 1 (Tool Development)
The mini-CEX tool was intended to facilitate formative assessment of core clinical skills. The observations were documented according to mini-CEX direct observation tool designed by the core faculties of SVL based on competencies determined by National Medical Council (NMC) India and American Board of Internal Medicine (ABIM) (17). The core competencies taken into considerations were communication skills, psychomotor skills, patient safety behaviour, professionalism, clinical decision making, ethical approach and overall clinical competence. So, the skills evaluated were medical interviewing, physical examinations, informed decision making/counselling, professionalism, organisation, overall competence and clinical judgment/reasoning. This tool was validated by the subject experts.
Mini-CEX (Appendix 1)
The seven domains were assessed by 9-point, (1-2-3: Unsatisfactory, 4-5-6: Satisfactory, 7-8-9: Excellent). Students’ and evaluators feedback was also taken by a 9-point Likert scale. The face validity was confirmed by the experienced faculties of SVL and the content validity was approved by the educational experts and specialists. But, as interns are not evaluated apart from logbook, the criteria validity could not be calculated.
The reliability was determined by the Cronbach’s alpha, which was calculated with an emphasis on internal consistency. A Cronbach’s alpha calculated with the overall scores was 0.85 indicating good internal consistency. The educational impact was measured according to levels of Kirkpatrick model (18) which consists of four levels- Reaction, Learning, Behaviour and Result.
Phase-2 (Simulation Preparation)
Five trained standardised patients (SPs-these are the trained paramedical persons utilised for teaching purpose in simulation) were used to give the history of assigned cardiac condition such as hypertensive emergencies, dysrhythmias, etc., and SimMan was used for clinical examinations, moulage of pacemaker fixed to SimMan was used in few cardiac emergency scenarios.
Phase-3 (Implementation)
The session was started with briefing (10 minutes) and the learners were instructed not to disclose the common scenarios to their peers.
The performance and evaluation lasted for 20 minutes followed by debriefing (30 minutes). The feedback on their satisfaction was taken in prescribed forms. The utility of the mini-CEX was estimated using the Vleuten V utility formula and introduced the utility formula by combining utility elements: Validity (v), Reliability (R), Educational Impact (EI), Acceptability (A), Cost (c)
Utility=R*V*EI*A*C.
Vleuten V emphasised that, this formula was purely a conceptual model and not meant as an actual algorithm, as most of the elements can never be quantified (19).
Statistical Analysis
The data were analysed with Statistical Package for the Social Sciences (SPSS) version 22.0 statistical software. The continuous variables were expressed as mean±standard deviation (SD), and categorical variables in percentage.
Results
The mean age of participants was 22.5±0.53 years. There were 15 males (30%) and 35 females (70%). The mean score of the participating students was 5.69±1.20 on whole mini-CEX. The mean scores of the assessment in seven domains of mini-CEX are shown in (Table/Fig 1).
Mean satisfaction score for the evaluators was 7.4±1.2 and for the students was 8.2±0.2. Present study showed high educational impact on level I as per Kirkpatrick model (High satisfaction perceived by the interns on learning and improvement on learning and acquisition of skills as helpful or very helpful) (18).
So, as per Van der Vleuten formula (19), calculated utility of mini-CEX was-
Validity: As per the subject experts, face validity and content validity was good. Further studies needed to calculate criterion validity.
Reliability: The calculated Cronbach’s alpha score was 0.85 which indicated good reliability.
Educational impact: from the scores of mini-CEX evaluation tools, satisfaction scores and feedback scores, educational impact was found to be high as per Kirkpatrick Model (18). For level I (reaction), learners were highly satisfied by the mini-CEX. Mini-CEX positively affected interns behaviour and made them more confident to perform in real life scenarios due to improved knowledge and skills (Kirkpatrick level II).
Cost: For mini-CEX no additional faculties were hired. Apart from printed mini-CEX checklist and feedback forms no additional cost was required. This cost was minimal and acceptable by the Institute. So, it can be concluded that conducting mini-CEX was cost effective.
Acceptability: Both the learners and five assessors were highly satisfied by mini-CEX and wanted to include it in their curriculum, as it improved learners competencies and anticipated to perform better in real life scenarios. Assessors felt that, it was easy to carry out and could timely point out the mistakes and rectify on the spot by the constructive feedback. The high satisfaction scores of both assessors and trainees projected high acceptability of mini-CEX.
Discussion
As per newer guidelines and recommendations of NMC of India, the reforms in medical education are underway. It is challenging to improve combined theoretical knowledge and clinical competence (16). But apart from training, authentic and legitimate assessment is the most important part of medical education. CBME greatly emphasises on the assessment of clinical competency associated with the methods involving higher levels of ‘shows how’ and ‘does’ of Miller Pyramid (4). Various studies showed that interns were not observed while performing clinical examination and procedures. As they were not observed and corrected, the skills were not improved and this may lead to the problems in future (4),(20).
Certain assessment tools like Direct Observational Procedural Skills (DOPS) can be very much useful to improvise theoretical knowledge as well as clinical skills. Hence, it is emphasised to use these tools in assessing the learners which plays major role in medical education. Mini-CEX is the frequently used tools to evaluate the performance at work place (6). However, there are very few studies on use of mini-CEX in simulation based medical education for medical graduates (7),(20).
Many researchers through their individual studies and systematic reviews have emphasised on the utility of mini-CEX as an assessment tool [3-6]. In 1996, Vleuten V proposed the formula of utility (19). The criterion validity could not be calculated due to the paucity of data for the statistical calculations. Ansari A et al., concluded that the construct and criterion validity of the mini-CEX indicated that it is an important instrument for the direct observation of trainees clinical performance (21). Durning S et al., also confirmed the validity of mini-CEX as assessment tool (22). Kogan J et al., in their systematic review established the strongest validity evidence for mini-CEX (3). Present study showed good reliability of mini-CEX as an assessment tool which was consistent with the studies of Kogan J et al., (3), Mortaz Hejri S et al., (6), Durning S et al., (22).
Lörwald AC et al., in their systematic review and meta-analysis concluded that 11 studies reported high educational impact (high satisfaction with mini-CEX; trainees perceived mini-CEX as helpful for learning) of mini-CEX (23). The present study was consistent with their observation. Yusuf L et al., showed statistically significant improvement in scores of learners due to training about assessment patterns, duration and criteria along with positive behaviour and educational impact (24).
The satisfaction scores of both the students and the evaluators were excellent in the present study. For acceptability, present study findings were similar with those of Yousuf N, who in his systematic review concluded that studies have shown high satisfaction rates of faculty (mean rating ranging from 6.1 to 8.06 out of 9) and trainees (mean rating ranging from 6.6 to 8.0 out of 9) for mini-CEX (20). According to Charokar K et al., mini-CEX was acceptable to the postgraduates and faculty. It was found to be an effective and formative assessment tool for learning clinical skills in a supportive workplace-based environment for clinical skills improvements (25).
For optimal patient care medical graduates must be assessed for clinical competence in simulation and mini-CEX is an efficient tool in doing so. The present study shows significant correlation between scores achieved in mini-CEX assessment by learners with various researchers’ findings (20),(26),(27). Mini-CEX is particularly helpful in timely feedback and problem resolution which in turn is useful to have in-depth understanding and operation of skills. In accordance with study conducted by Yusuf L et al., the outcome of this study also shows that learning by mini-CEX tool had excellent student satisfaction (24).
Limitation(s)
The limitations of the present study are that authors could not calculate the criterion and construct validity due to the paucity of statistical values for comparison. Level III and level IV as per Kirkpatrick model could not be evaluated as they are based on clinical scenario.
Conclusion
The utility of mini-CEX was established considering its validity, reliability, educational impact, cost-effectiveness and acceptability. Moreover, the evaluators and the inters were highly satisfied with the evaluation system. Based on the results of the present study, it can be concluded that mini-CEX is an effective assessment tool for evaluating the learners in simulation based medical education. It can make medical graduate competent by improving their knowledge and procedural skills. Further studies can be done to evaluate level III and level IV as per Kirkpatrick model by observing interns performing in real life scenarios like emergency and wards.
Acknowledgement
Authors sincerely thank to the School of Virtual Learning staff and faculties and study participants.
Reference
|
| | 1. | Datta R, Upadhyay K, Jaideep C. Simulation and its role in medical education, MJAFI. 2012;68(2):167-72.
[ CrossRef] [ PubMed] | | 2. | Visser H, Waston M, Salvado O, Passenger J. Progress in virtual reality simulators for surgical training and certification. Med J Aust. 2011;194(4):S38-40.
[ CrossRef] | | 3. | Kogan J, Holmboe E, Hauer K. Tools for direct observation and assessment of clinical skills of medical trainees: A systematic review. JAMA. 2009;302(12):1316-26.
[ CrossRef] [ PubMed] | | 4. | Sethi S, Badyal D. Clinical procedural skills assessment during internship in ophthalmology. J Adv Med Educ Prof. 2019;7(2).56-61.
| | 5. | Norcini J, Blank L, Arnold G, Kimball H. The mini-CEX (clinical evaluation exercise): a preliminary investigation. Ann Intern Med. 1995;123(10):795-99.
[ CrossRef] [ PubMed] | | 6. | Mortaz Hejri S, Jalili M, Shirazi M, Masoomi R, Nedjat S, Norcini J, et al. The utility of mini-Clinical Evaluation Exercise (mini-CEX) in undergraduate and postgraduate medical education: Protocol for a systematic review. Syst Rev. 2017;6(1):146. Doi: 10.1186/s13643-017-0539-y. PMID: 28720128; PMCID: PMC5516345.
[ CrossRef] [ PubMed] | | 7. | Behere R. Introduction of Mini-CEX in undergraduate dental education in India. Education for Health. 2014;27(3): 262-68.
[ CrossRef] [ PubMed] | | 8. | Lörwald AC, Lahner FM, Mooser B, Perrig M, Widmer MK, Greif R, et al. Influences on the implementation of Mini-CEX and DOPS for postgraduate medical trainees’ learning: A grounded theory study. Med Teach. 2019;41(4):448-56. Doi: 10.1080/0142159X.2018.1497784. Epub 2018 Oct 28. PMID: 30369283.
[ CrossRef] [ PubMed] | | 9. | Kumarage S, Fernando R, Gunasekara L. A survey of knowledge and practices of transfusion medicine among post intern doctors in specialized hospital in Sri Lanka. Lab Med. 2017;48(1):46-50.
[ CrossRef] [ PubMed] | | 10. | Khalil S, Aggarwal A, Mishra D. Implementation of a mini-clinical evaluation exercise (Mini-CEX) program to assess the clinical competence of postgraduate trainees in pediatrics. Indian Pediatric. 2017;54(4):284-87.
[ CrossRef] [ PubMed] | | 11. | Lee V, Brain K, Martin J. Factor’s influencing mini-CEX rater judgments and their practical implications: A systematic literature review. Acad Med. 2017;92(6):880-87.
[ CrossRef] [ PubMed] | | 12. | Chang YC, Lee CH, Chen CK, Liao CH, Ng CJ, Chen JC, et al. Exploring the influence of gender, seniority and specialty on paper and computer-based feedback provision during mini-CEX assessments in a busy emergency department. Adv Health Sci Educ Theory Pract. 2017;22(1):57-67.
[ CrossRef] [ PubMed] | | 13. | Korthagen F. Situated learning theory and the pedagogy of teacher education: Towards an integrative view of teacher behavior and teacher learning. Teaching and Teacher Education. 2010;26(1):100-06.
[ CrossRef] | | 14. | Brown RA, Guinea S, Crookes PA, McAllister M, Levett-Jones T, Kelly M, et al. Clinical simulation in Australia and New Zealand: Through the lens of an advisory group. Collegian. 2012;19(3):177-86. Doi: 10.1016/j.colegn.2012.05.002. PMID: 23101352.
[ CrossRef] [ PubMed] | | 15. | Koyun A, Ocalan D. Evaluation of pregnant examination simulation with Mini-CEX in nursing education: An experience of Turkey. CEMS. 2016;1(4):13-23.
[ CrossRef] | | 16. | Medical Council of India. Competency Based Undergraduate Curriculum for the Indian Medical Graduate, 2018;1-3.
| | 17. | https://www.abim.org/about/research/abstracts/abim-competency-based-medical-education-pilots. Accreditation Council for Graduate Medical Education: Educational Conference, February 2015.
| | 18. | Heydari M, Taghva F, Amini M, Delavari S. Using Kirkpatrick’s model to measure the effect of a new teaching and learning methods workshop for healthcare staff. BMC Res Notes. 2019;12(1):388.
[ CrossRef] [ PubMed] | | 19. | Vleuten V. The assessment of professional competence: Developments, research and practical implications. Adv Health Sci Educ Theory Pract. 1996;1(1):41-67.
[ CrossRef] [ PubMed] | | 20. | Yousuf, N. Mini clinical evaluation exercise: Validity and feasibility evidences in literature. Education in Medicine Journal. 2012;4(1):100-07.
[ CrossRef] | | 21. | Ansari A, Ali S, Tyrone D. The construct and criterion validity of the mini-CEX. Acad Med. 2013;88(3):413-20.
[ CrossRef] [ PubMed] | | 22. | Durning S, Cation L, Markert R, Pangaro L. Assessing the reliability and validity of the mini-clinical evaluation exercise for internal medicine residency training. Acad Med. 2002; 77(9):900-04.
[ CrossRef] [ PubMed] | | 23. | Lörwald AC, Lahner FM, Nouns ZM, Berendonk C, Norcini J, Greif R, et al. The educational impact of Mini-Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) and its association with implementation: A systematic review and meta-analysis. PLoS One. 2018;13(6):e0198009.
[ CrossRef] [ PubMed] | | 24. | Yusuf L, Ahmed A, Yasmin R. Education impact of mini-clinical evaluation exercise: A game changer. Pak j Med Sci. 2018;34(2):405-11.
[ CrossRef] [ PubMed] | | 25. | Charokar K, Kapoor A. Introduction of Mini-CEX for the formative assessment of postgraduates in the general surgery discipline. J Clin Diag Res. 2020;1(9):PC01-PC05.
[ CrossRef] | | 26. | Deering SH, Hodor JG, Wylen M, Poggi S, Nielsen PE, Satin AJ, et al. Additional training with an obstetric simulator improves medical student comfort with basic procedures. Simul Healthc. 2006;Spring;1(1):32-34.
[ CrossRef] [ PubMed] | | 27. | Pteani L. Enhancing clinical practice and education with high fidelity human patient simulators, Nurse Educ. 2004;29(1):25-30. [ CrossRef] [ PubMed] |
DOI: 10.7860/JCDR/2022/57091.17215
Date of Submission: Apr 13, 2022
Date of Peer Review: May 14, 2022
Date of Acceptance: Oct 19, 2022
Date of Publishing: Dec 01, 2022
AUTHOR DECLARATION:
• Financial or Other Competing Interests: None
• Was Ethics Committee Approval obtained for this study? Yes
• Was informed consent obtained from the subjects involved in the study? No
• For any images presented appropriate consent has been obtained from the subjects. NA
PLAGIARISM CHECKING METHODS:
• Plagiarism X-checker: Apr 15, 2022
• Manual Googling: Sep 16, 2022
• iThenticate Software: Oct 15, 2022 (15%)
ETYMOLOGY: Author Origin
|