Assessment of Critical Thinking Skills of Postgraduate Students by Using the Critical Self-Thinking Inventory for Clinical Examination

Table of contents

1. Introduction

ritical Thinking is a human cognitive process which is characterized by a purposeful selfregulatory judgement. As a result of this a person forms judgement about what to believe or what to do in a given situation. For doing this a person uses critical thinking skills which include a set of core skills likeinference, analysis, interpretation, evaluation and selfregulation. These skills are necessary to form a judgment and to improve the quality of judgment. The aim of critical thinking is that learners should develop skills that are lasting and transferable. Acquiring critical thinking is a question of practice. Clinical competence depends upon critical thinking skills and problem solving abilities.

Traditionally case presentation has always been a time-tested and important tool of medical education. It consists of presenting challenging medical cases to under-graduate medical students, postgraduate students, and treating physicians. The assessment of clinical competence with the help of traditional long case has received a lot of criticism in recent years. The weaknesses of the traditional long cases are the lack of objectivity and low validity and reliability [2]. In order to make the long examination more objective, valid and reliable, many modifications in the original format were suggested [3,4,5].

Structuring of long case, like in Objective Structured Long Examination Record (OSLER) may improve the reliability of the long case. Structuring may make it more time efficient and also it may provide an opportunity to impart valuable feedback. In an attempt to improve the long case, Gleeson in 1997 introduced the OSLER objective structured long case examination record as a more valid, reliable and objective tool to assess clinical competence [10]. The OSLER is a 10item analytical record of the traditional long case which attempts to improve the objectivity, validity and reliability of existing practices. [9] [10]. In our study we have used OSLER -'Objective Structured Long case Examination Record' as a tool to find out whether it helps to develop critical thinking skills of post-graduate students. For assessment of critical thinking skills we have used the Critical Self-thinking inventory for clinical evaluation CSTI-CE which was designed and developed by researchers at our institute.

We undertook the study with the following rationale -In the current health care environment it is the need of the hour that educational programs are directed towards preparation of quality Doctors. This will be possible when postgraduate students are taught to use critical thinking skills in order to make sound clinical judgments. It is necessary that students use critical thinking skills in clinical practice and patient care.

2. II.

3. Aim

To study the utility of "Critical Self Thinking Inventory for Clinical Examination" (CSTI-CE) for assessment of critical thinking skills during OSLER in post-graduate students of Department of Obstetrics & Gynaecology.

4. III.

5. Materials and Methods

6. Observations and Results

The present study was conducted in the Department of Obstetrics & Gynaecology and SHPER-School of Health Professional Education and Research, Datta Meghe Institute of Medical Sciences (Deemed to be University), Sawangi (Meghe), Wardha. We used the Critical Self-Thinking Inventory for Clinical Examination, for assessing the critical thinking skills of post-graduate students. This inventory was designed and developed by researchers at our institute. [Annexure A] The Critical Self-Thinking Inventory for Clinical Examination [Annexure A] was used for self-assessment of students in three encounters. Once after traditional long case and thereafter with first and second encounters of OSLER. Students were allowed 15 minutes to complete the task and hand in their sheets after each encounter. This inventory contains 20 items concerning the patient diagnostic thinking. Each item contains a stem and a rating scale. This inventory contains 20 items inclusive of 5 main domains of clinical skills. For History Taking and Clinical Judgment performance based scale was used. For Examination and Localising signs Perception based scale was used. The last Critical reflection was used as a measure of self-assessment of the student. The results of the marking done by the students in the first, second and third encounters, were compared pertaining to all domains of clinical skills.

For the Domain 1 of -History Taking, the mean scores obtained by the students in the three encounters were compared and also the standard deviation were calculated as depicted in the following table. It was observed that the scores of the students were improving form the first to third encounters. The total mean scores improved from 2.87 in first encounter to 4.19 in the third encounter. [Table I] At the end of our study an attempt was made to find the reliability and consistency of the CSTI-CE. The internal consistency was tested using coefficient alpha or Cronbach's alpha which was 0.932. Because the Cronbach's alpha was found to be > 0.75, the CSTI-CE was found to be reliable and valid. [Table IX

7. Discussion

Critical thinking is self-directed, self-disciplined, self-monitored and self-corrective thinking. Critical thinking is often referred to as a disposition, to describe a person's inclination to use critical thinking when faced with problems to solve, ideas to evaluate, or decisions to make. APA Delphi panel of international experts defined "critical thinking" for purposes of training and measurement as follows: "Critical thinking is the process of purposeful, self-regulatory judgment. This process Researches are of the opinion that critical thinking skills can be positively correlated with the consistent internal motivation to think. Also specific critical thinking skills can be matched with specific critical thinking dispositions. These assumptions therefore suggest that a skill-focused curriculum will enable a person to think critically. Solving problems and making decisions using critical thinking involves both skills and habits of mind. A person strongly disposed toward critical thinking is habitually truth-seeking, open-minded, analytical, systematic, inquisitive, confident in reasoning, and judicious.

Newer approaches to learning and assessment have been introduced in modern education. In an attempt to improve the long case, Gleeson introduced the OSLER objective structured long case examination record as a more valid, reliable and objective tool to assess clinical competence [10]. An attempt was made to evaluate the mini-clinical evaluation exercise (mini-CEX), to assess the clinical skills of residents [11]. Also methods like DOPS [12] and WBPA [13] have been newly introduced as methods of assessment of postgraduate students. The innovative modalities of modern medical education provoke the student for clinical reasoning, analysing, evaluating and also for problem solving abilities and decision making. Similar newer modalities for measuring the critical thinking skills of the students are also necessary.

Much criticism has been directed at the assessment of critical thinking skills and clinical competence with the traditional long case particularly in the recent years. The long case is a traditional clinical examination that assesses the student's competence at the 'shows how' level in Miller's pyramid [14]. In the traditional long case the students spend more than an hour with the patient and they take the history and then they examine the patient in detail. This process is not observed by the examiner. The student is then examined by the examiners over a 20-30 minute period. There are problems associated with the traditional long case in terms of objectivity, validity and reliability. Hence we have use the innovative modality of OSLER -Objective Structured Long-Case Examination Record along with the traditional long case, in an attempt to find out whether it will help to improve the critical thinking skills of postgraduate students.

In the present study the critical thinking skills of the students were assessed after traditional long case and then after one encounter of OSLER and then after second encounter of OSLER, thus giving three encounters. It was found that there was a significant improvement in scores in the third encounter over and above the second and first encounters. In the present study, the mean scores of students after 1st encounter were 3.06±0.46, after 2nd encounter they were 3.64±0.19 and after third encounter they were 3.64±0.19. Students paired 't' test was used to compare scores of students after 1st, 2nd and 3rd encounters and was found to be statically significant with P=0.0001. The levels of significance was taken as < 0.05.

A detailed analysis of all the 20 questions was across the five domains was done. The scores obtained by the postgraduate students in the five different Domains were compared, after first encounter of traditional long case and after the first and second OSLER intervention. We found there was a significant improvement in score of the students. Also the total scores obtained by the individual students in the three encounters were compared. The mean scores in the first encounter were 61.37±9.37, in the second encounter the mean scores were 72.87±3.82, and in the third encounter the scores were 86±4.01.

The scores obtained were divide into four groups <20, 21-40, 41-70 and 71-100. In our study none of the students scored < 20 in any of the encounters. In the range of 21-40 there was only one student in the first encounter and none in second and third encounters. In the group 41-70 there were 79% students in 1st encounter, 42 % in 2nd encounter and none in 3rd encounter. In the 71-100 score group, there were 17% students in 1st encounter, 58% in 2nd encounter and 100% in third encounter.

This indicates that at the end of our study there was development of adequate knowledge and skills of the students. It was 100% in the 3rd encounter which shows that OSLER helped to develop the critical thinking skills of the postgraduate students.

In the literature, no similar study was found which was conducted for postgraduate students in faculty of Medicine.

However, similar study was conducted by Profetto-McGrath, J in 2003, to study the critical thinking skills in nursing students [15]. A study of 228 nursing students across all four years of the baccalaureate program was done. Out of a maximum score of 420, the mean scores for the CCTDI ranged from 304.24 to 315.36, with an overall sample mean score of 312.30, which reflects a positive score. Most participants (85.50%) scored between 280 and 350 (positive scores). An important finding was the significant relationship between the students' critical thinking dispositions and their critical thinking skills (x2 = 9.37, p = 0.014, power > 80).

Zettergren et al in 2004 tried to evaluate criticalthinking skills in a group of professional physical therapist students and to determine if changes occurred over time [16]. Two hundred students enrolled in the 5year, professional physical therapist education program at a private New England university were included in the study. The participants completed one standardized test of critical thinking: the California Critical Thinking Skills Test (CCTST). A one-way analysis of variance with post hoc analysis was used to compare mean scores on the CCTST among the three groups. The results showed statistically significant differences between the scores of the third-year and fifth-year students (M=2.59, P=0.0001) and the scores of the fourth-year and fifthyear students (M=1.81, P=0.05). The results indicated a statistically significant increase in student's criticalthinking skills from the third year to the fifth year in physical therapist students at the target university. The difference occurred because of the additional years of formal classes, didactic education and an 8-week clinical internship in the summer after the fourth year.

Practical clinical assessment forms the corner stone of assessing clinical competence of a student. The essential components of clinical competence includes taking a proper history, physical examination, calling for relevant investigations, diagnosing the patients problem and formulating a treatment plan. In the traditional pattern of education, clinical assessment is mainly in the form of long case, short case and vivavoce. In our study we have used OSLER "objective structured long case examination record" an innovative method and found that OSLER helped to develop the critical thinking skills of the postgraduate students in the Department of Obstetrics & Gynaecology. Dr Rita Sood, in an editorial has made an attempt to examine whether OSLER was better than traditional case [17]. It was found that OSLER helped with direct observation of history taking and communication process, observation of physical examination and establishment of facts, laboratory investagions in proper order and also with the ability to identify and solve patient problems and formulate overall management. A study was conducted in Brazil by Luiz E.A. Troncon et al aiming at improving the assessment of senior medical students. A standardized and structured modification to the traditional long-case examination was proposed [18]. It was found that modifying the format of the long-case examination increased its value in the assessment of student clinical competence. In our study the OSLER assessments were conducted at the interval of one month each and it was found that the time period was optimum. Ton Peng et al from Malaysia, implemented fortnightly clinical assessments using modified OSLER (Objective Structured Long Examination Record) over a 6-weekperiod of clinical rotation. And they concluded that regular objective assessment associated with giving feedback would improve clinical outcome [19].

Hence we conclude that although the category of students were different, the results obtained by the various studies were similar.

8. VI.

9. Conclusion

At the end of our study we found that using innovative methods like OSLER helped to develop the critical thinking skills of postgraduate students in the Department of Obstetrics & Gynaecology in a significant manner. For the postgraduate students the residency training program of 2-3 years, is the best time for young Doctors to hone their skills. But for this to happen, developing critical thinking skills must be the focus of postgraduate medical education. The ability to think critically and innovate is necessary to improve the health of our patients, our communities, and our profession.

10. Recommendations

In the literature, no similar study conducted for postgraduate students in faculty of Medicine was found. Hence there is a scope that our inventory CSTI-CE, be used by other researchers so that findings can be corroborated. The scope of the study can be broadened to other faculties like Dental, Physiotherapy, Nursing and Ayurveda also.

Figure 1. Table I :
I
S. NO. History Taking 1 ST encounter Mean SD 2 ND encounter Mean SD 3 RD encounter Mean SD
1 I made the patient comfortable and introduced myself. 2.75 0.60 3.58 0.58 3.79 0.41
I discussed every symptom in detail with the
2 patient giving due 2.83 0.76 3.62 0.71 4.33 0.63
Importance to every symptom.
3 I feel, I initiated systematically and in chronological sequence 2.95 0.46 3.41 0.71 4.12 0.61
4 I analysed each symptom and correlate to a specific condition with justification. 2.95 0.62 3.54 0.50 4.45 0.72
Total 2.87 3.53 4.19
Figure 2. Table II :
II
NO. S. Clinical Judgement 1 ST encounter 2 ND encounter 3 RD encounter
Mean SD Mean SD Mean SD
Symptoms the patient described and the
5 leading questions I asked, led the history to 3.08 0.82 3.50 0.51 4.00 0.65
a definite provisional diagnosis.
The diagnosis I made is commoner and not
6 rare one and I could remember the 2.87 0.61 3.70 0.46 4.04 0.55
symptomatology of the disease
I analyse my findings at each step to justify
7 the diagnosis and other conditions related to 2.87 0.67 3.50 0.58 4.41 0.58
that condition
8 I could justify my provisional diagnosis and I was confident about the diagnosis 3.20 0.58 3.70 0.55 4.50 0.51
Total 3.00 3.6 4.23
In the Domain 3 of Examination, the total mean students improved form the 3.03 in the first to 4.23 in the
scores were calculated and compared for the three third encounter. [Table III]
different encounters. It was found that the scores of the
Figure 3. Table III :
III
S. NO Examination 1 ST encounter Mean SD 2 ND encounter Mean SD 3 RD encounter Mean SD
Based on my provisional diagnosis I could
9 identify the system involved in the disease 2.91 0.40 3.41 0.50 4.16 0.63
process
10 Physical examination I performed was not challenging for me 3.08 0.65 3.79 0.50 4.08 0.71
I felt that history and examination findings
11 are complementary and not contradictory to 3.12 0.67 3.50 0.58 4.45 0.58
each other
12 I think I examined the patient thoroughly and completely to justify my diagnosis 3.04 0.95 3.62 0.64 4.25 0.60
Total 3.03 3.58 4.23
Figure 4. Table IV :
IV
S. NO. Localizing Signs 1 ST encounter Mean SD 2 ND encounter Mean SD 3 RD encounter Mean SD
I could guess at least two signs that I
13 thought would be present in this patient after 2.91 0.71 3.54 0.50 4.41 0.50
taking history
14 I followed each step correctly in eliciting the signs 3.00 0.88 3.66 0.56 4.33 0.56
15 I could interpret the signs supporting my diagnosis and its relevance in this case 3.04 0.35 3.62 0.64 4.37 0.64
16 I think none of the sign had confused me or challenged my diagnosis 3.20 0.88 3.62 0.57 4.04 0.62
Total 3.03 3.61 4.28
Figure 5. Table V :
V
S. NO. Critical Reflection 1 ST encounter Mean SD 2 ND encounter Mean SD 3 RD encounter Mean SD
17 I identified my strengths and weaknesses in this case 3.29 0.69 3.91 0.58 4.54 0.58
18 Based on weaknesses I could identify the areas of improvement 3.45 0.65 3.87 0.33 4.45 0.58
I realise that every case is unique and
19 different and hence it has to be examined in the light of previous knowledge and 3.37 0.82 3.79 0.65 4.62 0.57
experience
20 I have analysed this case with my full efficiency 3.37 0.92 3.91 0.58 4.58 0.50
Total 3.37 3.87 4.54
Figure 6. Table VI :
VI
13 50 69 89
14 72 78 86
15 67 77 85
16 61 68 89
17 61 71 89
18 69 70 91
19 64 68 84
20 69 75 90
21 71 77 85
22 70 76 88
23 73 76 92
24 72 78 86
Mean 61.37 72.87 86
SD 9.37 3.82 4.01
The scores obtained were interpreted as follows - 41-70 Adequate knowledge & skills; Needs
Score Interpretation [Table VII] reinforcement of skills
<20 Inadequate knowledge 71-100 Adequate competence
21-40 Adequate knowledge; inability to correlate
knowledge with demonstrable clinical skills
Student No 1 st encounter 2 nd encounter 3 rd encounter
1 68 72 84
2 54 77 79
3 54 70 87
4 51 70 87
5 68 81 79
6 56 71 79
7 58 70 78
8 60 70 84
9 56 71 87
10 35 69 89
11 64 70 91
12 50 75 86
Figure 7. Table VII :
VII
In our study none of the students scored < 20 encounter and none in 3rd encounter. In the 71-100
in any of the encounters. In the range of 21-40 there was score group, there were 17% students in 1st encounter,
only one student in the first encounter and none in 58% in 2nd encounter and 100% in third encounter
second and third encounters. In the group 41-70 there [Table VIII].
were 79% students in 1st encounter, 42 % in 2nd
Figure 8. Table VIII :
VIII
Score Range 1 st encounter 2 nd encounter 3 rd encounter
<20 0(0%) 0(0%) 0(0%)
21-40 1(4.17%) 0(0%) 0(0%)
41-70 19(79.17%) 10(41.67%) 0(0%)
71-100 4(16.67%) 14(58.33%) 24(100%)
Total 24(100%) 24(100%) 24(100%)
Figure 9. Table IX :
IX
Cronbach's Alpha N of Items F-value p-value 95% confidence interval Lower Upper Bound Bound
0.932 20 3.14 0.0001,S 0.88 0.96
V.
Figure 10.
gives reasoned consideration to evidence, context,
conceptualizations, methods, and criteria."
Year 2020
31
Score <20 21-40 41-70 71-100 Interpretation Adequate knowledge; inability to correlate knowledge with demonstrable clinical Inadequate knowledge skills Adequate knowledge & skills; Needs reinforcement of skills Adequate competence Volume XX Issue II Version I
D D D D ) E
(
Medical Research
Global Journal of
© 2020 Global Journals
1

Appendix A

Appendix A.1 Acknowledgement

My sincere thanks to all the faculty and postgraduate students of the Department of Obstetrics and Gynaecology for their whole hearted and enthusiastic participation.

Appendix B

Appendix B.1 Conflict of Interest -None

Bibliography

Appendix C

  1. Direct Observation of Procedural Skills) in undergraduate skills-lab: Does it work? Analysis of skillsperformance and curricular side effects. C Profanter , A Perathoner , Dops . 10.3205/zma000987. GMS Z Med Ausbild 2015. 2015 Oct 15. 32 (4) .
  2. The OSCE progress test-Measuring clinical skill development over residency training. Debra Pugh , Claire Touchie , Susan Humphrey-Murto , & Timothy , J Wood . Medical Teacher 2016. 38 p. .
  3. Critical reasoning: The art and science of critical and creative thinking, D J Pesut , Herman J Albany . 1999. (NY: Delmar)
  4. A standardized, structured long-case examination of clinical competence of senior medical students. E A Luiz , Roberto O Troncon , Jose´ Dantas , C Fernando . Medical Teacher 2000. 22 (4) .
  5. Effectiveness of Problem Based Learning as an Instructional Tool for Acquisition of Content Knowledge and Promotion of Critical Thinking Among Medical Students. Journal of the College of Physicians and Surgeons Pakistan 2013. 23 (1) p. . (Rakhshanda tayyeb)
  6. Assessment of clinical competence using the Objective Structured Long Examination Record (OSLER). Fergus Gleeson . 10.3109/01421599709019339. AMEE Medical Education Guide 1997. 19 (9) p. . (Medical Teacher)
  7. Defects in postgraduate clinical skills as revealed by objective structured long examination record (OSLER). F Gleeson . Irish Medical Journal 1992. 85 p. .
  8. The assessment of clinical skills/ competence/performance. G E Miller . Medical Education 1990. 40 (11) p. . (Academic Medicine.. Supplement 9)
  9. The Mini-CEX: A Method for Assessing Clinical Skills. J J Norcini , L L Blank , F D Duffy , G S Fortna . 10.7326/0003-4819-138-6-200303180-00012. Ann Intern Med 138 p. .
  10. Workplacebased assessment as an educational tool: AMEE Guide No. 31. John Norcini , & Vanessa Burch . 10.1080/01421590701775453. Medical Teacher 2007. 29 p. .
  11. The relationship of critical thinking skills and critical thinking dispositions of baccalaureate nursing students. J Profetto-Mcgrath . Journal of Advanced Nursing 2003. 43 (6) p. .
  12. Addressing the ''Two Disciplines'' problem: linking theories of cognition and learning with assessment and instructional practice. J W Pellegrino , G P Baxter , R Glaser . Review of Research in Education 1999. 24 p. .
  13. Changes in Critical Thinking Scores: An Examination of One Group of Physical Therapist Students. Kathleen K Zettergren , Ronald Beckett . Journal of Physical Therapy Education Fall2004. 18 p. .
  14. Defining critical thinking, M Scriven , R Paul . from//www.critical-thinking.org/University/univclass/Defining.Html February 23 2013.
  15. The integration of testing with teaching: applications of cognitive psychology in instruction. N Frederiksen . American Journal of Education 1994. 102 p. .
  16. Long case examination-can it be improved?. R Sood . J Indian Acad Clin Med 2001. 2 p. .
  17. The higher order cognitive skills: what they are and how they might be transmitted. S Chipmen . The Intergenerational Transfer of Cognitive Skills, T G Sticht, B A Mcdonald, M J Beeler (ed.) (Toronto, Canada
    ) 1989. Ablex Publishing.
  18. The clinical case report: a review of its merits and limitations. T Nissen , R Wynn . BMC Res Notes 2014. 7 p. 264.
  19. Successive objective long-case assessment as a driver of clerkship learning-Evaluation through perception questionnaire. Toh Peng Yeow , Toh Peng Yeow , Wai Sun Choo . 10.3109/01421591003717065. Medical Teacher March 2012. 34 (3) p. 257.
Notes
1
© 2020 Global Journals
Date: 2020 2020-01-15