gms | German Medical Science

Introduction: Students need adequate feedback about their performances to benefit from educational programmes and to improve their learning. In nursing education, Switzerland Simulated Patients (SP) are employed to train communication and procedural skills. After a Simulated Clinical Encounter (SCE) the SP directly offers oral feedback to the students. Measuring the quality of SP feedback, however, requires validated and reliable instruments. Win May from the Keck School of Medicine of the University of Southern California, Los Angeles, (USA), and Dixie Fisher from the University of Southern California, School of Medicine, Los Angeles, (USA), together developed the “Quality of Simulated Patient Feedback Form (QSF)”, which assesses the immediate oral SP feedback after an encounter. With the friendly permission we could validate the QSF Instrument at our Institution.

Research questions:

The following research questions were developed:

• How good is content validity
• as evaluated by expert judgements
• How high is reliability
• as measured by Cronbach α, which is a measure of internal consistency.-
• as measured by test re test ( as a measure of stability)
• by generalizability over rater, time and items

Method: Since the QSF existed in English only, forward-backward translation was completed. Content Validity: All 18 items of the QSF were deployed to 25 medical and nursing educators in German countries. The tool “Survey Monkey” was used to develop an online survey. Test/retest: To establish reliability, test / retest of QSF a generalizability analysis was conducted.

Results: Mean / standard deviation: The Cronbach α was calculated at 0.783 for the whole scale indicating high homogeneity. Internal Consistency: An encouraging internal consistency was obtained. In addition, 54.8% of the experts rated items as being very important.

Test retest and Interrater Reliability: To obtain a measure of generalizability, it is crucial to estimate variance components for all sources of variation, including higher order interactions. ANOVA procedures were carried out with three repeated measurement factors (SP) of time (2 levels, pre and post), CD (3 levels or 3 recorded SP encounters), and item (18 levels) gave us the following sums of squares (SS) and estimated variance components. If we wanted to generalize to a universe of raters, items, and times, the ratio of “wanted” universe score variance to observed variance is, 0.61. If, on the other hand, we regard the items as fixed, because it would be unrealistic to view the items as a random sample from a universe of many more items of SP feedback, so that we want to generalize only to a universe of raters and times, the generalizability rises to 0.88.

Conclusion: The quality of the oral SP feedback needs to be measured adequately. The QSF has been found to assess oral SP feedback in a valid and reliabel way.