Clinical simulation: how to train patient assessment

In a study with 421 internal medicine residents, observed in 1,228 real encounters, the lowest score among the skills assessed was for the physical examination (Norcini et al., 2003). The highest score was for professionalism (Norcini et al., 2003). Assessing a patient brings together several different skills, and each one improves with observed practice and with feedback. Encounter simulation is one of the safest places to start that practice.

NEXO is a daily medical diagnosis game: one clinical case a day, six progressive clues, and ICD-10 proximity feedback across five levels.

This text shows how to use simulation with people, with a simulated patient, colleagues or a mannequin, to train patient assessment in parts: interview, physical examination, communication and organization. It covers what simulation is, what studies show, how to divide the encounter into skills, debriefing and how to practice with colleagues. The virtual patient on the screen has its own text about virtual patients, and case study, the text about case-based learning.

Key points

• Encounter simulation is practicing the meeting with the patient in a controlled environment, with a simulated patient, a colleague or a mannequin.
• With deliberate practice, simulation outperformed traditional clinical teaching for acquiring specific skills in a meta-analysis (McGaghie et al., 2011).
• Feedback and repeated practice appear as the most cited features of effective simulation.
• Dividing the encounter into domains, such as interview, physical examination and organization, helps to train and observe each part.
• Debriefing, the conversation after the scene, is where simulation becomes learning.
• Role-play among colleagues also trains communication and can be done with little structure.

Summary

• What encounter simulation is
• What the evidence shows
• Assessing the patient in parts
• Debriefing: the conversation after the scene
• How to practice with colleagues
• Where NEXO comes in
• Frequently asked questions
• Encounter simulation checklist

What encounter simulation is

Encounter simulation is reproducing the meeting with the patient to practice, make mistakes and correct them before the real encounter. The format changes according to what you want to train.

Simulated patient and standardized patient

The simulated patient is, in general, a lay person trained to portray a patient with a specific condition realistically (Cleland et al., 2009). When the presentation is the same for all students, without varying from one to another, it is called a standardized patient (Cleland et al., 2009). The AMEE guide on the topic describes its use for teaching and assessing the consultation and the clinical examination, both in simulation environments and in the service itself (Cleland et al., 2009).

The same guide notes that the simulated patient can also give feedback to the student and assess their performance (Cleland et al., 2009). Whoever plays the role of the patient takes part in teaching, including with feedback on what it was like to be cared for.

When the school has a simulated patient

The AMEE guide details how to recruit, train and follow up simulated patients, including how to monitor the validity and reliability of the performance and the impact of the role on whoever portrays it (Cleland et al., 2009). For the student, the practical message is to make the most of what this patient knows how to do. As a practical suggestion, at the end of the encounter, ask the simulated patient what it was like to be cared for by you: at what moment they felt heard, at what moment they were confused, what was missing in the explanation.

Role-play among colleagues

In role-play, one colleague plays the patient, another conducts the encounter and, often, a third observes. It is the easiest format to set up: it requires a script, a quiet place and time to talk afterwards. Studies comparing role-play and simulated patient appear in the next section.

Mannequins and simulators

Mannequins and simulators of body parts are used to train procedures and the physical examination, such as auscultation. The use of simulators in medical education and the difference between the formats appear in the text about case-based learning. This text focuses on the encounter with the patient, with people on both sides.

Where each format works best

Format	What it trains best	What it requires
Simulated or standardized patient	Interview, physical examination, communication and feedback from the person who was cared for	Trained person and a case script
Role-play among colleagues	Communication and structure of the interview	Colleagues, script and time for debriefing
Mannequin or simulator	Procedures and parts of the physical examination	Skills lab
Virtual patient	Data collection and decisions throughout the case	A case program, covered in the text about virtual patients

What the evidence shows

Studies on simulation are many and varied. Three questions help organize what they show: does simulation work, what makes it work, and do a simulated patient and a peer give the same result?

Simulation with deliberate practice

A meta-analysis compared education based on simulation with deliberate practice with traditional clinical teaching for acquiring specific clinical skills (McGaghie et al., 2011). The search covered 20 years of publications, from 1990 to 2010 (McGaghie et al., 2011). Of the 3,742 articles found, 14 entered the analysis, and the overall effect size was 0.71 in favor of simulation (McGaghie et al., 2011). The authors acknowledge that the number of studies is small and recommend that simulation be introduced carefully and evaluated rigorously (McGaghie et al., 2011).

Deliberate practice means practice with a defined goal, repetition and feedback, not just "doing it many times". The role of practice with feedback in reasoning appears in the text about clinical reasoning in the clinical cycle. The broader meta-analysis on simulation with technology is summarized in the text about virtual patients.

What makes simulation work

A BEME systematic review analyzed 109 studies on high-fidelity simulation and looked for the features linked to effective learning (Issenberg et al., 2005). Feedback was named the most important feature in 47% of the articles, and repeated practice appeared as a key feature in 39% (Issenberg et al., 2005). Integration into the curriculum appeared in 25%, and variation in the degree of difficulty, in 14% (Issenberg et al., 2005).

Other features mentioned were a controlled environment, in which the student can make a mistake, notice it and correct the error without consequences, cited in 9% of the articles, and clear objectives, in 6% (Issenberg et al., 2005). The authors rated the overall quality of the research as weak, but concluded that high-fidelity simulation is effective under the right conditions and complements teaching with patients (Issenberg et al., 2005).

Feature	In how many articles it was highlighted
Feedback	47% (Issenberg et al., 2005)
Repeated practice	39% (Issenberg et al., 2005)
Integration into the curriculum	25% (Issenberg et al., 2005)
Variation of difficulty	14% (Issenberg et al., 2005)
Controlled environment to make mistakes and correct them	9% (Issenberg et al., 2005)
Clear objectives	6% (Issenberg et al., 2005)

Simulated patient or peer?

A review on simulated patient and role-play in communication training concluded that most studies indicate better results when the program includes practice of the skills (Lane and Rollnick, 2007). The review found only one study that directly compared role-play and simulated patient, with no significant difference between the two (Lane and Rollnick, 2007). The authors point out methodological weaknesses that make firm conclusions difficult (Lane and Rollnick, 2007).

A randomized trial after that review, with 103 fifth-year students in Germany, trained the conversation with parents of sick children using role-play among peers, with a standardized patient or with no training (Bosse et al., 2012). Both trainings improved self-efficacy and the score in a six-station OSCE, compared with the group with no training (Bosse et al., 2012). The gain was greater with role-play, because of a higher score in understanding the parents' perspective (Bosse et al., 2012).

In the students' own assessment, both formats were well accepted and considered very realistic (Bosse et al., 2010). The students found the standardized patient more useful and more applicable to future encounters (Bosse et al., 2010).

Student assessment, scale from 1 to 6	Role-play	Standardized patient
Acceptance	5.32 (Bosse et al., 2010)	5.51 (Bosse et al., 2010)
Realism	5.60 (Bosse et al., 2010)	5.53 (Bosse et al., 2010)
Usefulness for training communication	5.01 (Bosse et al., 2010)	5.34 (Bosse et al., 2010)
Applicability to future encounters	4.29 (Bosse et al., 2010)	5.00 (Bosse et al., 2010)

Acceptance and realism did not differ significantly; usefulness and applicability were higher with the standardized patient (Bosse et al., 2010). Both studies dealt with communication, not physical examination.

Assess the patient in parts

Assessing a patient is not a single skill. The mini-CEX, a method of observing real encounters, separately assesses interview, physical examination, professionalism, clinical judgment, patient counseling, organization and efficiency, and overall competence (Norcini et al., 2003). Separating the encounter this way helps you train one part at a time in simulation.

The domains, one by one

As a practical suggestion, use the mini-CEX domains to choose the focus of each simulation round:

Domain assessed	What to train in simulation	What the observer notes
Interview	Open the consultation, let the patient speak, ask what is missing	Questions that were left out and interruptions
Physical examination	Sequence, technique and explanation to the patient during the examination	Parts skipped and maneuvers with questionable technique
Professionalism	Introduce yourself, ask permission, respect modesty	Moments of patient discomfort
Clinical judgment	Link findings to hypotheses	Findings that did not enter the reasoning
Counseling	Explain what comes next in plain language	Technical terms without explanation
Organization and efficiency	Order of the encounter and use of time	Back and forth and time spent on each part

The domains come from the mini-CEX (Norcini et al., 2003); the columns on what to train and note are a suggestion from the Editorial.

Why the physical examination deserves attention

In the mini-CEX study, residents had the lowest scores in physical examination and improved in all domains over the first year (Norcini et al., 2003). The data came from 21 residency programs and 316 assessors (Norcini et al., 2003). The physical examination depends on technique, sequence and knowing what to look for, and simulation allows you to repeat each part with someone observing. As a practical suggestion, set aside rounds just for the physical examination of one system, with a colleague observing the technique.

Assess without being assessed

The mini-CEX was designed as an assessment, but the domains also work for training among colleagues. The authors observed that assessors somewhat compensate for patient difficulty and that each resident sees several patients, which reduces the effect of a more difficult case (Norcini et al., 2003). As a practical suggestion, in simulation with colleagues, vary the cases and the observers across rounds, so that feedback does not depend on a single scene.

From finding to hypothesis

The clinical judgment domain is the bridge to reasoning: what to do with what you found in the interview and the examination. How to summarize the problem and prioritize hypotheses is in the text on clinical reasoning in the clinical cycle. How to present the case after the encounter is in the text on clinical reasoning in the internship.

Simulation and real encounters

The mini-CEX was created for real encounters, and the authors note that it has more fidelity than formats such as the standardized patient and covers a greater variety of situations and problems (Norcini et al., 2003). Simulation and supervised encounters complement each other: in simulation, you repeat and make mistakes safely; in the service, you encounter the variety of patients.

Debriefing: the conversation after the scene

Debriefing is the structured conversation after the simulation, in which those who provided care and those who observed review what happened and why. It is where the scene becomes learning.

Debriefing with good judgment

An influential debriefing proposal starts from an idea: the student's actions come from their "frames", such as knowledge, assumptions and feelings (Rudolph et al., 2006). The instructor acts as a "cognitive detective", with genuine curiosity about these frames, and treats errors as puzzles to solve (Rudolph et al., 2006). The technique combines statement and question: an objective observation with the instructor's judgment, followed by a curious question about the student's reasoning (Rudolph et al., 2006).

An example of how this can sound: "I saw that you examined the abdomen before asking about the pain, and I was worried that the pain got in the way of the exam. What were you thinking at that moment?". The authors report that the approach helps the instructor give critical judgments while keeping a trusting relationship (Rudolph et al., 2006).

What the studies say about debriefing

A systematic review identified 177 studies, with 11,511 participants, that used debriefing in technology-enhanced simulation (Cheng et al., 2014). Compared with no intervention, simulation with debriefing had favorable effects on all outcomes, with effect sizes between 0.28 and 2.16 (Cheng et al., 2014). Video debriefing had a negligible and non-significant effect, of 0.10, compared with debriefing without video (Cheng et al., 2014). Demonstration by an expert followed by short debriefing had non-significant effects in favor, between 0.21 and 0.74, compared with long debriefing (Cheng et al., 2014). Debriefing at the end of the scene or during the scene had variable results, depending on the outcome and the type of training (Cheng et al., 2014).

The authors note that the characteristics of debriefing, such as duration, presence of the instructor, content and structure, are often described incompletely in studies (Cheng et al., 2014). Other debriefing design features had mixed or non-significant results (Cheng et al., 2014). If you research this type of question, NEXO Researcher is the program for proposing studies with NEXO.

A simple debriefing script

As a practical suggestion, use three moments:

1. Reaction: how was it for the person who attended and for the person who played the patient?
2. Analysis: what happened in each domain, with concrete observations from the observer, followed by a question about the reasoning.
3. Summary: one or two things to keep and one to change in the next round.

How to train with colleagues

Simulation labs and standardized patients depend on the institution. Role-play among colleagues you organize with your class, and it also trained communication in the rehearsal with students (Bosse et al., 2012).

The trio: patient, doctor and observer

As a practical suggestion, train in a trio:

Role	What they do
Patient	Reads the case script beforehand and answers only what is asked, with the character's emotions
The one who attends	Leads the interview and, when it makes sense, the physical examination of one system
Observer	Writes down concrete observations by domain and leads the debriefing

After each round, the roles switch. This way, each one trains attending, observing and feeling the care from the patient's side. In the rehearsal with students, role-play had a greater gain precisely in understanding the perspective of the person being cared for (Bosse et al., 2012).

The patient script

As a practical suggestion, the script fits on one page: who the patient is, the complaint in their own words, what they tell if asked, what they only tell if the question is good, what worries them and the examination findings that the observer reveals when the examination is done. Cases from books, from questions and from clinical sessions serve as a basis.

A simulation round

As a practical suggestion:

1. Choose the focus: one or two domains, such as interview and organization.
2. Attend: with time agreed beforehand.
3. Do the debriefing: reaction, analysis by domain and summary.
4. Repeat: the same situation or a similar one, with what changed.
5. Record: what to keep and what to change, for the next round.

Common mistakes

• Skipping the debriefing: feedback is the most cited resource of effective simulation (Issenberg et al., 2005).
• Training everything at once: separating domains allows observing each part (Norcini et al., 2003).
• Doing only one round: repeated practice appears among the most cited resources (Issenberg et al., 2005).
• An observer who only praises or only criticizes: concrete observation followed by a question about the reasoning yields more conversation (Rudolph et al., 2006).
• A patient who gives everything at once: the script should answer only what is asked.

Where NEXO comes in

Simulation trains the encounter with the patient; NEXO trains the hypothesis from the data, every day. The two meet at the moment when findings become a diagnosis.

The case of the day as a warm-up

The Daily brings one case per day, the same for everyone, and renews at midnight. Each case arrives in clues, one at a time, and each guess shows the proximity in ICD-10. It is a short training of clinical judgment mastery: linking findings to hypotheses.

From the case to the scene

As a practical suggestion, after playing the Daily case with the class, turn the case explanation into a patient script for role-play. Whoever played already knows the disease; the challenge now is to take the history and explain the exam as if it were the first time. The Archive gathers previous daily cases, and Multimedia brings cases with image, audio and other exams, good for training the description of findings.

Streak, ranking and achievements

The streak records the days you played, the ranking lets you compare the case of the day with the class, and achievements mark the path. To take to a card what was missing in the case, the Anki button is described in the guide on spaced repetition.

To research, bring to the class and create

If you research simulation and learning and want to propose studies with NEXO, meet NEXO Researcher. To bring NEXO to the medical school, the student league or the residency, there is NEXO Closer. Doctors who review the cases in their specialty take part through NEXO Curator, and those who create health content on social media can take part through NEXO Creators.

Frequently asked questions

What is clinical simulation in Medicine?

It is the practice of care in a controlled environment, with a simulated patient, colleagues, mannequins or computer programs. The simulated patient is, in general, a lay person trained to portray a patient realistically (Cleland et al., 2009).

Does simulation work?

With deliberate practice, simulation surpassed traditional clinical teaching for acquiring specific skills, with an effect size of 0.71 in a meta-analysis of 14 studies (McGaghie et al., 2011). The overall quality of the research is still considered weak (Issenberg et al., 2005).

What is the difference between a simulated patient and a standardized patient?

The standardized patient is a simulated patient who gives the same presentation to all students (Cleland et al., 2009). This allows comparing the performance of different students, as in an OSCE.

Is role-play among colleagues worth it?

It is worth it for communication. In a trial with fifth-year students, role-play and standardized patient improved the score on the communication OSCE, and role-play had a greater gain in understanding the parents' perspective (Bosse et al., 2012). The students found the standardized patient more useful (Bosse et al., 2010).

What is debriefing?

It is the structured conversation after the simulation. A well-known approach combines an objective observation with a curious question about the reasoning of the person who provided care (Rudolph et al., 2006).

Does recording the simulation on video help?

In a meta-analysis, debriefing with video had a negligible effect compared with debriefing without video (Cheng et al., 2014).

How to train physical examination outside the lab?

In a trio, with a colleague as the patient, focus on one system and an observer noting the technique and the sequence. Physical examination was the domain with the lowest scores among residents in the mini-CEX study (Norcini et al., 2003).

Does simulation help in the OSCE?

The OSCE assesses care in stations with a defined time, and the standardized patient is a common part of it. The comparison between OSCE, virtual patient and diagnostic game is in the text about virtual patients. As a practical suggestion, train with the same time as the station and with an observer using the domains of care.

How long should a round last?

It depends on the focus. As a practical suggestion, agree on the time beforehand, set aside for debriefing a time similar to that of the care, and prefer several short rounds to one long one.

How does NEXO help in care simulation?

The Daily case trains linking findings to hypotheses, and the case explanation can become a patient script for role-play with the class. Multimedia brings cases with image, audio and other exams to train the description of findings.

Care simulation checklist

1. You chose the focus of the round, one or two domains?
2. Is there a patient script that answers only what is asked?
3. Is there an observer whose role is to take notes by domain?
4. Did you agree on the time of the consultation beforehand?
5. Did you hold the debriefing with reaction, analysis and summary?
6. Did the observer use concrete observation followed by a question?
7. Did you repeat the situation with what changed?
8. Did you swap roles so everyone could interview and observe?
9. Did you set aside rounds for the physical examination of one system?
10. Did you record what to keep and what to change for next time?

References

1. Norcini JJ, Blank LL, Duffy FD, Fortna GS. The mini-CEX: a method for assessing clinical skills. Ann Intern Med. 2003. https://pubmed.ncbi.nlm.nih.gov/12639081/
2. Cleland JA, Abe K, Rethans JJ. The use of simulated patients in medical education: AMEE Guide No 42. Med Teach. 2009. https://pubmed.ncbi.nlm.nih.gov/19811162/
3. McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011. https://pubmed.ncbi.nlm.nih.gov/21512370/
4. Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005. https://pubmed.ncbi.nlm.nih.gov/16147767/
5. Lane C, Rollnick S. The use of simulated patients and role-play in communication skills training: a review of the literature to August 2005. Patient Educ Couns. 2007. https://pubmed.ncbi.nlm.nih.gov/17493780/
6. Bosse HM, Schultz JH, Nickel M, et al. The effect of using standardized patients or peer role play on ratings of undergraduate communication training: a randomized controlled trial. Patient Educ Couns. 2012. https://pubmed.ncbi.nlm.nih.gov/22137189/
7. Bosse HM, Nickel M, Huwendiek S, et al. Peer role-play and standardised patients in communication training: a comparative study on the student perspective on acceptability, realism, and perceived effect. BMC Med Educ. 2010. https://pubmed.ncbi.nlm.nih.gov/20353612/
8. Rudolph JW, Simon R, Dufresne RL, Raemer DB. There's no such thing as "nonjudgmental" debriefing: a theory and method for debriefing with good judgment. Simul Healthc. 2006. https://pubmed.ncbi.nlm.nih.gov/19088574/
9. Cheng A, Eppich W, Grant V, et al. Debriefing for technology-enhanced simulation: a systematic review and meta-analysis. Med Educ. 2014. https://pubmed.ncbi.nlm.nih.gov/24909527/