gms | German Medical Science

Clinical thinking is a central component of physician competence. Optimal patient care depends on thorough analysis of the information provided by the patient and on the risk-benefit assessment of diagnostic tests and therapies.

It follows that every university-level programme of medical education must have the objective of forming good clinical reasoning skills in its students. Conclusive evidence of the particular advantages of specific types of teaching in fostering these skills, however, has yet to be provided.

Clinical problems present essential differences in comparison to well-structured tasks. Necessary information is not available in its entirety at the beginning of the process, and the nature of a problem can change its dynamics during the diagnostic-therapeutic process (case history and examination). The task is a matter of complex problem solving because there are no standardised procedures for arriving at a solution; instead, each problem is unique, and the physician can never be fully certain that the solution found is actually correct [1], [2].

This subject has been intensively worked on within the field of educational research since the 1970s. The objective is to better qualify which processes are at work during clinical reasoning and to identify differences between experts and novices. From this information, it should be possible to draw conclusions that will aid better instruction in clinical reasoning.

Over the course of this time, many concepts explaining the acquisition of expertise in clinical reasoning alternated and were further developed [3]. One concept, for example, is that of so-called “illness scripts” [4]. These are representations of problems (diseases, for example), syndromes or groups of diseases along with the conditions in which they occur, their manifestations, diagnoses and therapy concepts, as well as their pathophysiological bases.

Other models emphasise the particular benefit of practical experience in achieving medical expertise [5], [6]. Empirical evidence shows that skills associated with clinical problem solving were more developed among experts than among novices [7]. It was shown, for example, that the expert’s knowledge in memory is primarily linked together by probabilities [4], or that it is characterised by reference models which are spontaneously recognised and then confirmed with hypothetico-deductive methods [2]. Furthermore, experts are better at using so-called “semantic qualifiers” that classify a symptom on a bipolar scale or in the quality of key features recognised in a patient history [8].

Clinical reasoning, however, is also dependent on content and context [9], and expertise in one area does not necessarily mean that comparable skills are available in a different specialised discipline or medical case. In fact, it is not even possible to generalise expertise within a field [10], [11].

None of these approaches are seen as mutually exclusive today [12]. As a result, experts make parallel use of analytical (deductive, controlled) and non-analytical (unconscious, spontaneous) processes in resolving patient cases. This is because effective clinical problem solving is based on clear procedures of information gathering, hypothesis formation and hypothesis testing on the one hand, while a diagnosis depends on knowledge of the underlying mechanisms and the problem-related integration and mental organisation of this information on the other. Recognition of patterns accelerates recollection [13]. The two processes complement each other and a bidirectional exchange of information takes place.

The question of how to optimally teach students such complex abilities has, for good reason, yet to be definitively answered. Students who had been instructed in the heuristic method of concurrent hypotheses (Bayes’ Theorem), for example, were able to implement the method satisfactorily following the teaching unit but later failed in the clinic when it came to transferring this abstract model into practice [14]. Other studies showed that work on didactically chosen patient cases with errors and elaborated feedback or on case studies with instruction had a positive effect on clinical reasoning in the respective trial setting [15], [16], [17].

To date, two teaching and learning environments have almost unanimously proved effective in facilitating clinical reasoning: problem-based learning (PBL) and clinical praxis/clinical experience [18], [19], [20].

Because it is particularly well-suited to learning and practicing the problem solving process [21], PBL should contribute to students’ mastering of clinical reasoning [18] [20]. PBL combines content with context, and the application of memorised knowledge to clinical problems fosters the development of coherent, pathophysiological concepts [21], [22]. Meta-analysis [23] of the question of which types of instruction promote clinical reasoning revealed two studies showing that students improved their critical thinking with PBL [24], [25] and that they were able to deliver more accurate, more coherent and more comprehensive explanations for medical problems than other students [22]. Critical thinking and clinical reasoning are not necessarily comparable, however.

Concrete clinical experience with patients is a further indispensable factor in the development of clinical reasoning. Firstly, students must also practice their acquired abilities [26], and secondly, clinical reasoning is strongly influenced by experience [12] and is a consequence of resulting multidimensional knowledge. It was shown that the students’ stages of development often corresponded to their clinical experience, for example, rather than to their academic year [27], [28].

On the basis of these findings, traditional curricula, in which instruction is limited to the subject at hand, little PBL is implemented and clinical experience comes later and often to a lesser extent, presumably offers students less opportunity to learn and practice clinical reasoning.

In 1990, in order to measure clinical reasoning by means of self-assessment, Bordage and Marsden developed a corresponding inventory [29] - the Diagnostic Thinking Inventory (DTI). Until then, inventories were used that measured critical instead of clinical reasoning, for instance. Subjects were asked to outline their thinking processes either orally or in written form, or the solving of a concrete clinical problem was tested. The DTI, on the other hand, is an instrument that measures both self-evaluated flexibility in thinking and structure of knowledge in memory independently of context and can also differentiate between medical experts of varying degrees of training. It is comprised of 41 items in which subjects use a scale to evaluate themselves in predefined situations. The answers, in turn, represent a specific degree of clinical reasoning. The original research was conducted among 270 test subjects with different degrees of training – from first-semester students to experienced physicians. Significant differences between students and physicians were revealed, whereas the differences between physicians of varying degrees of experience were insignificant.

The DTI inventory was implemented and validated in various studies in the following years [30] and was also analysed in conjunction with other cognitive or psychometric tests [31].

In total, the present study results reached with the DTI are not consistent. In the majority of studies, the students’ DTI results improved significantly with instruction in clinical problem solving, instruction in diagnostic procedure errors, through the handling of patient cases, through participation in diagnostic case discussions or with increasing semesters of study [32], [33], [34], [35], [36]. Two other studies, however, showed that the conducted interventions did not lead to any improvements in the DTI [26], [37].

The inventory measures the self-assessment of the participants on the type and structure of their clinical reasoning, but it does not measure their actual diagnostic ability. For this reason, the correlation between DTI results and resolved cases or determined diagnoses is often weak.6,32 In addition to clinical reasoning skills, a corresponding knowledge base in the respective specialist field and clinical experience are required to arrive at the correct diagnosis.

The medical curricula of different schools vary significantly in their objectives and focal points. Despite their differing paths, their common goal is an education that will produce good physicians. In the long term therefore it is necessary to provide evidence as to the effect that different curricula have on acquiring clinical reasoning as a core competence of physicians. As a first step, this pilot study, conducted in three different medical schools in North Rhine-Westphalia, examined the measurability of differences between students of varying curricula, on the one hand, and between differing stages of study, on the other.

Answers to the following questions were sought:

1.

Can an increase in the students’ self-assessed clinical reasoning competence over the course of their studies be evidenced using DTI?

2.

Are there differences regarding clinical-reasoning-related self-assessment between students from varying courses of study and curricula?

Presumably, clinical reasoning competence would increase with the advance of study (i.e., advancing academic semesters). Differences between students of varying curricula would be apparent when, for example, the amount of elements such as PBL or clinical training differed. Students at W/HU and in the model course of study at RUB undergo model curricula that are characterised in part by problem-oriented learning (PBL) from their start and – particularly at W/HU – by long and numerous clinical traineeships. PBL is the central element of the first four semesters at W/HU. In addition, the amount of clinical experience prescribed by the curricula of W/HU is the highest among all of the courses of study. Starting from the second half of the fourth semester until the practical year at the end of their studies, 46 weeks are devoted to blocks of practical traineeships, while six weeks are spent in on-site observation in general medicine. The two courses of study at RUB differ in regard to their curriculum. The traditional course of study has a subject-related traditional curriculum with a six-week PBL block in the fourth semester. The model course of study is theme-based and praxis-related. TBL is a structuring element in the first four semesters and is offered concurrently with 5^th, 8^th and 9^th semester studies. The two courses of study also differ in numbers of students. While 42 candidates are accepted to the model course of study programme per academic year, the traditional course of study admits 200 students per year. The survey included all students present in the 7^th and 9^th semesters of the model course of study as well as two seminar groups from the general medicine course in the 7^th semester of the traditional course of study.

In order to measure the self-assessment of memory-unrelated components of clinical reasoning, the DTI (Diagnostic Thinking Inventory) was implemented. To this end, the German-language translation of the questionnaire (courtesy of Dr. Stieger [36]) was administered in the model courses of study at three schools in North Rhine-Westphalia (Bochum, Cologne and Witten/Herdecke) and in one traditional course of study (Bochum) in machine-readable form.

In order to answer the question of whether clinical reasoning improved with the duration of study, different semesters within a course were included. This was the case for the model course at W/HU and the model course at RUB. As a measurement of differences in self-assessed clinical reasoning between students of varying courses and curricula, advanced students from all three locations were surveyed prior to their practical (final) year. At the time of our study, the 8^th semester was the most advanced pre-practical-year semester available at W/HU because these students had enrolled in a period when registration was only possible for the summer semester (beginning in spring at German universities). It was possible to include 9^th semester students from RUB and UoC, as their studies had commenced in the winter semester (beginning in autumn at German universities).

The questionnaires were distributed for completion and immediately collected again at the participating universities in a period between October and December 2010 at a specific time, in each case, when face-to-face instruction was taking place, at a point of greatest possible attendance of that semester’s students.

The students were briefed on the purpose of the project directly preceding the distribution of questionnaires. Participation was voluntary and the survey was anonymous; separate declarations of consent were not given.

Following evaluation, students’ scores and respective commentary were made available to them upon request and presentation of the correlating questionnaire number. The results were then made available under that number’s reference.

Instrument

The DTI questionnaire measures the self-assessment of non-knowledge-related components of clinical reasoning and comprises 41 questions (see sample questions, table 1 [Tab. 1]). Twenty of these can be allocated to the subscale “flexibility in thinking” and 21 to the subscale “structure of memory” (see examples below). “Flexibility in thinking” illustrates the participants’ ability during the diagnostic process to arrive at the correct diagnosis and to flexibly incorporate new information. “Structure in memory” reflects the degree of organisation and accessibility of the memorised knowledge from which the participants’ draw. Every question is composed of a stem (usually a statement) and a response scale. At each end, the response scale offers two opposing answers/statements in response to the initial statement, with six boxes between to select from. Participants are asked to check the box in the scale that best reflects their position between the response options. The options are randomly placed either at the left or the right end of the scale which means that the response reflecting more advanced clinical reasoning could be at either end.

Sample question from the flexibility subscale (see figure 1 [Fig. 1]).

Sample question from the structure subscale (see figure 2 [Fig. 2]).

All of the completed questionnaires were scanned and were then read using analysis software (FormPro 2.5). Only questionnaires in which all questions had been clearly answered were included. Score results were calculated by assigning values of 1 to 6 to the selection boxes. The response showing the most pronounced clinical reasoning was given the highest score. The total score achieved (a maximum of 126 points possible) as well as the points in the subscales “flexibility” (max. 120 points) and “structure” (max. 126 points) were calculated for each participant. Cronbach’s alpha was used in determining the consistency of the DTI. Effect size was calculated as Cohen’s d.

All data were statistically analysed using SPSS 19.0. Since the data were not normally distributed (checked with the Kolmogorov-Smirnov goodness-of-fit test), the individual groups were checked for differences using the Mann-Whitney test. A significance level of 5% was chosen. Considering the number of comparisons performed, a Bonferroni correction of alpha errors was not necessary.

Between 48% and 78% of the students solicited took part in the survey. Between 71% and 95% of the returned questionnaires were complete and evaluable.

The test cohorts’ evaluable questionnaires were divided as follows (see table 1 [Tab. 1] and 2 [Tab. 2]).

An initial evaluation of the three represented academic years from Witten/Herdecke University and the three cohorts from Ruhr University Bochum was targeted at revealing the value of DTI in measuring significant differences in the self-assessment of clinical reasoning between varying academic years of one curriculum or, respectively, differences between academic years of different curricula.

Ideally, abilities in clinical reasoning would increase with the number of semesters attended. Among students at W/HU (see figure 3 [Fig. 3]), a distinct corresponding semester-related increase in self-assessed clinical reasoning was evidenced using DTI; the increase from the 5^th to the 8^th semester is significant and has a large effect size. An increase from the 5^th to the 6^th semester, of medium effect size, is also evidenced, while no significant difference could be detected between the 6^th and 8^th semesters.

If model courses of study compared are more conducive to the facilitation of clinical reasoning in comparison with traditional courses, then differences between cohorts with differing curricula but the same length of study should be detectable. In a comparison of 7^th semester RUB students, however, no difference could be evidenced between those from traditional studies and those from model studies (see figure 4 [Fig. 4]).

In a further study, students from the three participating model courses were compared; namely 9^th semester students from both the University of Cologne and Ruhr University Bochum, and 8^th semester students from Witten/Herdecke University. The three cohorts’ results from the self-assessment of clinical reasoning using DTI differed significantly. The students from W/HU achieved higher scores than the students from both of the other schools, and the students from RUB achieved higher overall scores than the students from UoC (lesser effect size in this comparison) – see figure 5 [Fig. 5].

The objective of the project was to examine the effectiveness of measuring differences in self-assessment with the aid of the Diagnostic Thinking Inventory not only between students from different faculties but also between students in different semesters within one faculty.

The internal consistency of the questionnaire was acceptable to good; that of the subscales, however, was meagre. As a result, only the respective total scores were compared in all groups.

Significant differences could be measured both between individual academic years at a university (increasing scores in the self-assessed clinical reasoning of students from Witten/Herdecke University) as well as between comparable semesters at different universities. The results from students at W/HU corresponded to the initial hypothesis that clinical reasoning skills increase with the advance of course studies. Both of the elements that, according to references, can aid this development (PBL [24], [25] and clinical experience in the form of blocks of practical training or clinical traineeships [27], [28]), are soundly represented in the curriculum of W/HU. If PBL alone had an effect on clinical reasoning, then higher scores would be expected from students in the model courses of study at RUB than from their fellow students from traditional courses. This, however, was not evidenced in this study.

A comparison of all students from the most advanced academic years of the schools surveyed also shows that courses of study with the greatest amount of PBL and practical clinical experience achieve the highest scores, whereas those students with curricula displaying lesser amounts of these elements attain lower results.

In order not to limit the resulting data to use within the survey groups and to screen the attained scores for plausibility, they were compared with the results measured by Bordage [29]. The lowest measured total score of a group in the present study (students from the UoC, 9^th semester with 150.5 points) are below the results recorded by Bordage for students in their 3^rd academic year (158.3 points), while results from the group with the highest achieved scores in our study (W/HU, 8th semester with 178.2 points) are above the results measured for senior house officers (168.4 points) and general practitioners (172.3 points). This suggests that DTI is particularly well-suited to intra-faculty comparisons due to the difficulty of calibration for varying systems. Whereas the Bordage study showed evidence of a continual score increase from first-year students (153.9 points) to registrars (180.2 points), the differences found between the surveyed groups in our study was greater.

The questionnaire was implemented in a non-standardised setting among three different medical schools in the German State of North Rhine-Westphalia. The results are therefore not necessarily representative for the respective cohorts. The variance of results in the data collected for the traditional course of study (7^th semester, RUB) is particularly great, and the participants (40) represent only a small, randomly chosen group from the whole of the cohort. Considering this, it is questionable if the data are representative for students of this academic year on the whole.

In Cologne and at WH/U, the questionnaires were handed out following a progress test, which could explain the low response rate, the poor reliability and the lower overall results. The reliability and overall scores were higher at W/HU. Greater peer pressure may have had a motivating effect in the smaller cohort in this case. The percentage of nonevaluable questionnaires from W/HU was also lower. Being that approximately one third of the questionnaires from the UoC were not evaluable, comparisons using their results should be viewed critically. Questionnaires without clear responses to all of the questions were not factored in. Despite the instruction to check one of the boxes between the proposed statements, many students chose to mark the dividing line between two boxes, evidently unable to decide on a value offered in the continuum. This was also observed in the questionnaires filled out by students from RUB and is the reason for the low percentage of evaluable returns from this location. Furthermore, we must bear in mind when interpreting the data that although the DTI is an established instrument, it does not objectively gauge clinical reasoning as such but serves as a measure of the self-assessment of students in this discipline. An (indeed desirable) overall critical stance in respect to their own competence could have a negative influence on the students’ self-assessment concerning clinical reasoning. Undoubtedly, the surveyed cohorts can also differ in the extent of their self-criticism. No further instrument was implemented that could have served to standardise this aspect. One can assume, for instance, that male students tend to overassess their competence [38]. The male/female distribution did not differ significantly in the surveyed groups, however. For this reason, the differences discovered in this study should not be attributable to a gender-related overassessment among the students at W/HU and an underassessment among those in Cologne.

In summary, the present pilot study indicates that the DTI is an appropriate instrument for a comparative survey of self-assessed clinical reasoning among students with various curricula.

The study did, however, also show that further steps must be taken if inferences are to be drawn from the measured differences about the effect of apportionment of different curricular elements such as PBL or clinical-practical experiences.

Principally, it must be ensured that the survey is conducted under comparable circumstances, whereby a higher rate of return must be reached. Further, it would seem necessary to implement an additional inventory in order to investigate possible self-overassessments or self-underassessments by the students. The DTI itself should be further validated with other specialised, objective tests for clinical reasoning and conduct [39], for example key feature [40], script concordance [41] or situational judgement [42] tests.

The authors declare that they have no competing interests.