Virtual patients: what works in case study
On the rotation, the preceptor's question comes in front of everyone: did you hear the murmur, feel the liver, think of the other hypothesis? The answer comes out fast, sometimes before certainty, because making a mistake in public weighs on you. That is why practicing cases without an audience makes sense. The virtual patient was born for this: a place to collect data, make mistakes and decide without anyone watching.
NEXO is a daily medical diagnosis game: one clinical case a day, six progressive clues, and ICD-10 proximity feedback across five levels.
This text goes deeper into what studies say about the virtual patient in Medicine: the types, what the meta-analyses show, which design features make a difference, how it compares with a diagnostic game and with the OSCE, and where the evidence is still limited. If you want an overview of the types of study game, start with the guide to games for medical students.
Main points
- A virtual patient is a digital, interactive simulation of a clinical scenario; the most common format in the literature is the interactive patient scenario.
- Compared with not studying, the virtual patient had large effects on knowledge, reasoning and skills; compared with teaching without a computer, it was on average similar.
- Repeating until mastery, receiving detailed feedback and comparing contrasting cases were design features associated with more learning.
- Virtual patient, diagnostic game and OSCE train or assess different parts of care and complement each other.
- Many of the studies are of low to moderate quality and measure reasoning in different ways, which limits the conclusions.
Summary
- Why practice cases without an audience
- The types of virtual patient
- What the evidence shows
- What makes a virtual patient work
- Virtual patient, diagnostic game and OSCE compared
- Limits of the studies
- Frequently asked questions
- Checklist for studying with a virtual patient
Why practice cases without an audience
The clinical reasoning of experienced people depends largely on recognizing patterns, and that recognition matures with deliberate practice on many varied cases (Cook and Triola, 2009). The rotation offers cases, but not in the quantity or variety each student needs, and almost always with someone watching.
What a virtual patient is
A virtual patient is a digital, interactive simulation of a clinical scenario, made to teach health professionals (Kononowicz et al., 2019). A group of educators defined the term as interactive, multimedia, screen-based patient scenarios (Berman et al., 2016). In general, you collect the history, order tests, interpret the results and decide what to do, and the case responds to your choices.
What it is best for
A critical review proposed that the most characteristic function of the virtual patient is to develop and assess clinical reasoning (Cook and Triola, 2009). The same authors recommend that, in assessment, the case reward reasoning, not exhaustive data collection or following an algorithm (Cook and Triola, 2009). This has a consequence for those who study: ordering every possible test is not the goal; the goal is to order what changes the hypothesis.
Making mistakes without an audience
In a virtual case, the mistake happens far from the patient and the preceptor. You can go back, redo and compare. This freedom does not replace the rotation, but it prepares you for it: whoever has already made a mistake in the virtual case arrives at the bedside with a better question.
The types of virtual patient
The term "virtual patient" is used for quite different things, and this confuses readers and researchers. A review of 536 articles that used the term proposed a classification to organize these meanings (Kononowicz et al., 2015).
Interactive patient scenarios
These are on-screen cases in which you move forward in stages, choosing questions, tests and management. Among the educational articles in the review, 37% used this format, the most common of all (Kononowicz et al., 2015). Over time, it established itself as the predominant form of virtual patient in teaching, despite technical advances that would allow more complex formats (Kononowicz et al., 2015).
High-fidelity simulations
These are programs that simulate the patient's physiology in real time: blood pressure drops if you take too long, saturation improves if you give oxygen. They accounted for 19% of the educational articles in the review (Kononowicz et al., 2015). They are usually used for emergency, intensive care and anesthesia.
Virtual standardized patients
These are characters you talk to, by text or voice, to practice history taking and communication. They represented 16% of the educational articles (Kononowicz et al., 2015).
Games with a virtual patient
Formats with game rules, scoring or competition appeared less often in the review (Kononowicz et al., 2015). To learn what the evidence says about points, rankings and achievements in teaching, see the text on gamification in medical education.
Which type for which goal
The choice of type follows what you want to practice. For diagnostic reasoning and the sequence of investigation, interactive scenarios are the most studied. For quick decisions with the patient changing on screen, high-fidelity simulations come closest to the emergency setting. For history taking and communication, virtual standardized patients let you repeat the conversation as many times as needed. And, to keep practice short and daily, formats with game rules help you return to the case every day.
What the evidence shows
The guide to games for medical students summarizes the main numbers. Here it is worth looking in more detail at what was compared with what, because the answer changes depending on the comparison.
Compared with not studying
In a meta-analysis of computer-based virtual patients, the effect compared with no intervention was large: 0.94 for knowledge, 0.80 for clinical reasoning and 0.90 for other skills, in standardized mean difference (Cook et al., 2010). This type of comparison shows that the virtual patient teaches, but it does not say whether it teaches more than other forms of study.
The result matches that of a broader meta-analysis of technology-enhanced simulation in the health professions, with 609 studies, which found large effects on knowledge, skills and behaviors and moderate effects on patient-related outcomes, always compared with no intervention (Cook et al., 2011).
Compared with other forms of teaching
Against teaching without a computer, the effects were close to zero: 0.06 for knowledge and practically 0 for reasoning (Cook et al., 2010). In the most recent synthesis, using only randomized trials, the virtual patient tied with traditional teaching in knowledge and was better in skills, among them clinical reasoning, procedural skills and the combination of procedure with teamwork (Kononowicz et al., 2019). The authors found evidence of effect both in high-income countries and in low- and middle-income countries (Kononowicz et al., 2019).
The practical reading is that the virtual patient works at least as well as a good class for knowledge, and tends to deliver more when the goal is to practice reasoning and skills.
What improves in reasoning
A systematic review focused on undergraduate medical students found a positive effect of virtual patients on clinical reasoning in 11 of 19 studies, the equivalent of 58% (Plackett et al., 2022). The most useful detail is in the domains: data collection, ideas about the diagnosis and patient management improved in 72% of the analyses, against 43% for knowledge application, flexibility of thinking and problem solving (Plackett et al., 2022).
In other words, the virtual patient more clearly improves the skills linked to the case that was practiced. General thinking skills change less, which matches the idea that clinical reasoning depends on the content of each case.
An example in the classroom
In a study in Japan with fourth-year students, a classroom virtual patient session, with answers via electronic voting, was followed by an increase in clinical reasoning scores and, to a lesser degree, in knowledge scores, between the pre-test and the post-test (Watari et al., 2020). The before-and-after design, without a comparison group, shows feasibility and an improvement, but does not allow saying how much of it came from the virtual patient.
What makes a virtual patient work
How much a virtual patient teaches depends on how it is made. Studies that compared different versions of the same case, and those that listened to students, point to some features that make a difference.
Design features supported by studies
In the meta-analysis of virtual patients, comparisons between designs suggested that four features improve learning: repeating until mastery is shown, offering an advance organizer of the content, giving detailed feedback and contrasting cases explicitly (Cook et al., 2010). Explicit contrast is especially useful for the differential: seeing two similar diseases side by side helps fix the finding that separates them.
What students ask for
In a study with focus groups, students described ten principles of a good virtual patient (Huwendiek et al., 2009). Among them:
- being relevant and having a difficulty suited to the student's level;
- being highly interactive and giving specific feedback;
- using image, sound and video well;
- helping focus on the important points and recap what was learned;
- bringing questions and explanations tied to the clinical reasoning process.
These are student perceptions, and the authors themselves call for controlled studies to test these principles (Huwendiek et al., 2009).
How the case fits into the course
The way of integrating the virtual patient into the course also matters. In a study in pediatric clerkships at six schools, satisfaction and perceived learning were higher when the use of the virtual patient came with the removal of other tasks, instead of simply adding work (Berman et al., 2009). In a survey with internal medicine clerkship directors in the United States, 55% merely added the virtual patients to the curriculum, without integrating them into other activities (Lang et al., 2013).
Discussing the case afterward also helps. In a Swedish introductory clerkship course, students perceived more benefit from the virtual patients in the hospitals where the case was worked on more intensively in the follow-up seminars (Edelbring et al., 2012).
What to look at in the feedback
Detailed feedback was one of the design features associated with more learning (Cook et al., 2010), but it only helps if you read it carefully. After each case, look for three things: which data point should have changed your hypothesis, which test or question was unnecessary and which finding separates the right answer from your second option. Writing down these three answers takes little time and leaves the case ready for review.
What this means for those who study alone
Even outside the curriculum, you can apply these findings. Repeat the case until you get it right without looking anything up, read the whole feedback, compare it with a similar case and take your doubts to the discussion with colleagues or with the preceptor. The text on case-based learning in clinical practice shows how to study cases alone and in groups.
Virtual patient, diagnostic game and OSCE compared
Virtual patient, diagnostic game and OSCE often appear together in conversations about teaching with cases, but each one covers a different part of the encounter.
The OSCE
The objective structured clinical examination was described in 1975 as a series of stations that students rotate through: at one, they take the history, examine one aspect or interpret tests; at the next, they answer questions about what they found (Harden et al., 1975). The proposal was to better control the variables of the examination, define objectives clearly and decide the scoring in advance (Harden et al., 1975).
An evidence review concluded that the OSCE is feasible in different cultural and geographic contexts, to assess a wide range of objectives, at different stages of the course (Patrício et al., 2013). Reliability depends on the format: in a systematic review, the mean alpha between stations was 0.66, and more stations and more examiners per station were associated with more reliable scores (Brannick et al., 2011).
The diagnostic clue game
In a diagnostic clue game, the case arrives in stages, in a defined order, and the task is to reach the name of the condition with the least data. The focus is on the hypothesis: with each clue, you decide whether you can already take a guess or whether you wait for one more piece of data.
In NEXO, the case arrives in up to six clues, and each guess shows the proximity in ICD-10, which indicates whether you are in the right group of diseases. On the third clue, the area of the case appears, and on the fifth, the ICD range. The Daily mode brings the same case to everyone each day, the Archive gathers the previous cases and Multimedia includes cases with image, audio and other exams. The Anki button creates the card straight from NEXO and opens it in the Anki app, searches for the cards you already have and, when the case has an image, takes the image to the card. To understand how the limits of each mode work, see the NEXO economy page.
Side by side
| Virtual patient | Diagnostic clue game | OSCE | |
|---|---|---|---|
| What it trains or assesses | Data collection, reasoning and decisions throughout the case | Formulation and review of the diagnostic hypothesis with each piece of data | Observed clinical skills: history, examination, interpretation, communication |
| Format | Case on screen that responds to your choices | Case in stages, in a defined order, with a guess at each stage | Rotating stations, with time and a scoring script |
| Feedback | Depends on the design; detailed feedback improves learning (Cook et al., 2010) | Immediate, with each guess | From the examiner, according to the script (Harden et al., 1975) |
| Typical use | Individual study and the clerkship curriculum | Short daily practice, alone or in a group | Formative and summative assessment |
| What the studies show | Better than traditional teaching in skills, equal in knowledge (Kononowicz et al., 2019) | Testing memory with feedback improves retention in the health professions (Green et al., 2018) | Feasible and reliable when there are enough stations and examiners (Brannick et al., 2011) |
How to combine the three
The three complement each other. The diagnostic game trains the hypothesis in a short time and every day. The virtual patient adds data collection and the decisions of the whole case. The OSCE shows, before an examiner, whether you can do all of this with a simulated patient and a clock running. For training the hypothesis with clues, the text on differential diagnosis brings a step by step script.
If you coordinate a medical school, league or residency program and want to use clue cases together with virtual patients and OSCE, get to know NEXO Closer. Researchers who want to compare case formats can get to know NEXO Researcher, and physicians who want to review cases, NEXO Curator. Those who produce medical content and want to bring cases like these to their audience can get to know NEXO Creators.
Limits of the studies
The evidence on virtual patients is extensive, but it has limits that apply to the field as a whole. Knowing them helps you read each new study with the right degree of confidence.
Quality and heterogeneity
In the synthesis of 51 randomized trials, methodological limitations and high heterogeneity led to an overall low quality of evidence (Kononowicz et al., 2019). The trials that compared virtual patients with other forms of digital teaching, or different versions of a virtual patient, were too few to generate clear recommendations (Kononowicz et al., 2019). In the previous meta-analysis, heterogeneity was also large in most of the analyses (Cook et al., 2010).
How reasoning is measured
In the review on clinical reasoning, the mean quality of the studies was moderate, and 8 of 19 studies, or 42%, did not report any measure of validity or reliability of the instrument used to measure reasoning (Plackett et al., 2022). Without validated and consistent measures, the authors could not perform a meta-analysis (Plackett et al., 2022).
Few studies on design
The possible design variations are almost unlimited, but few studies have tested them rigorously, and research on instructional design and integration into the curriculum is still lacking (Cook and Triola, 2009). The principles cited by students also await confirmation in controlled studies (Huwendiek et al., 2009).
The effect of the digital format itself
Online teaching in the health professions, in general, had large effects against no intervention and small effects against non-online formats, such as 0.12 for knowledge (Cook et al., 2008). The same pattern appears with virtual patients: the digital medium alone does not explain the gain. What matters is the case design and how it is used.
How to read a new study on virtual patients
Three questions help evaluate any study you find. First: what was the virtual patient compared against? Beating no intervention is expected; tying with a good class is already a useful result. Second: how was reasoning measured, and was the instrument validated? Third: was the gain measured right after or weeks later, and in cases identical to the trained ones or in new cases? With these three answers, you know how much weight to give the result.
If you research medical education and want to answer questions like these with cases in clues, check out NEXO Researcher.
Frequently asked questions
What is a virtual patient in Medicine?
It is a digital and interactive simulation of a clinical scenario, used to teach health professionals (Kononowicz et al., 2019). In general, you collect the history, order tests and decide on management, and the case responds to your choices.
Does a virtual patient work better than a class?
For knowledge, on average, the results were similar. For skills, including clinical reasoning, the virtual patient was better than traditional teaching in the synthesis of randomized trials, with low to moderate quality evidence (Kononowicz et al., 2019).
Which features make a virtual patient more effective?
Repeating until mastery, having an advance organizer, receiving detailed feedback and explicitly contrasting cases were associated with more learning (Cook et al., 2010). Students also value adequate difficulty, lots of interaction and explanations tied to reasoning (Huwendiek et al., 2009).
What is the difference between a virtual patient and an OSCE?
The virtual patient is an on-screen case used mainly for training. The OSCE is an exam in stations, with timed stations and an examiner, used to assess observable clinical skills (Harden et al., 1975). One prepares you for the other.
How does NEXO fit into studying with cases?
NEXO trains the diagnostic hypothesis: the case arrives in up to six clues and each guess shows the proximity in ICD-10. It combines with the virtual patient, which trains data collection and decisions, and with the OSCE, which assesses skills in front of an examiner. To learn about the game, read what NEXO is.
How to study with a virtual patient alone?
Repeat the case until you get it right without looking anything up, read the entire feedback, compare it with a similar case and note the finding that separated the hypotheses. Then discuss your doubts with colleagues: the perception of benefit was greater where cases were worked on more intensively in seminars (Edelbring et al., 2012).
Does simulation help prepare for the OSCE?
In the health professions, technology-supported simulation training was associated with large effects on knowledge, skills and behaviors when compared with no intervention (Cook et al., 2011). For the OSCE, the most useful thing is to train the format: timed stations, one task per station and someone observing. The virtual patient covers well the reasoning and test interpretation part.
Is a virtual patient useful for the internship?
It is, especially to vary the cases and train reasoning before encountering the problem on the ward. Studies show clearer gains in skills tied to the trained case, such as data collection and ideas about the diagnosis (Plackett et al., 2022).
Checklist for studying with a virtual patient
- Define the objective: data collection, hypothesis, management or communication.
- Choose the format that trains that objective.
- Ask only for what changes the hypothesis, not every possible test.
- Read the entire feedback, even when you get it right.
- Repeat the case until you get it right without looking anything up.
- Compare with a similar case and note the finding that separates the two.
- Take your questions to the discussion with colleagues or a preceptor.
- Combine formats: a diagnostic game for the hypothesis, a virtual patient for the whole case and an OSCE for observed practice.
- Read each new study looking at the comparison group and the way reasoning is measured.
Want to practice the hypothesis today? Play the case of the day on NEXO.
References
- Kononowicz AA, Zary N, Edelbring S, Corral J, Hege I. Virtual patients--what are we talking about? A framework to classify the meanings of the term in healthcare education. BMC Med Educ. 2015. https://pubmed.ncbi.nlm.nih.gov/25638167/
- Cook DA, Triola MM. Virtual patients: a critical literature review and proposed next steps. Med Educ. 2009. https://pubmed.ncbi.nlm.nih.gov/19335571/
- Cook DA, Erwin PJ, Triola MM. Computerized virtual patients in health professions education: a systematic review and meta-analysis. Acad Med. 2010. https://pubmed.ncbi.nlm.nih.gov/20703150/
- Kononowicz AA, Woodham LA, Edelbring S, et al. Virtual Patient Simulations in Health Professions Education: Systematic Review and Meta-Analysis by the Digital Health Education Collaboration. J Med Internet Res. 2019. https://pubmed.ncbi.nlm.nih.gov/31267981/
- Plackett R, Kassianos AP, Mylan S, et al. The effectiveness of using virtual patient educational tools to improve medical students' clinical reasoning skills: a systematic review. BMC Med Educ. 2022. https://pubmed.ncbi.nlm.nih.gov/35550085/
- Harden RM, Stevenson M, Downie WW, Wilson GM. Assessment of clinical competence using objective structured examination. Br Med J. 1975. https://pubmed.ncbi.nlm.nih.gov/1115966/
- Cook DA, Levinson AJ, Garside S, et al. Internet-based learning in the health professions: a meta-analysis. JAMA. 2008. https://pubmed.ncbi.nlm.nih.gov/18780847/
- Cook DA, Hatala R, Brydges R, et al. Technology-enhanced simulation for health professions education: a systematic review and meta-analysis. JAMA. 2011. https://pubmed.ncbi.nlm.nih.gov/21900138/
- Berman NB, Durning SJ, Fischer MR, Huwendiek S, Triola MM. The Role for Virtual Patients in the Future of Medical Education. Acad Med. 2016. https://pubmed.ncbi.nlm.nih.gov/26959224/
- Huwendiek S, Reichert F, Bosse HM, et al. Design principles for virtual patients: a focus group study among students. Med Educ. 2009. https://pubmed.ncbi.nlm.nih.gov/19493183/
- Berman N, Fall LH, Smith S, et al. Integration strategies for using virtual patients in clinical clerkships. Acad Med. 2009. https://pubmed.ncbi.nlm.nih.gov/19550193/
- Lang VJ, Kogan J, Berman N, Torre D. The evolving role of online virtual patients in internal medicine clerkship education nationally. Acad Med. 2013. https://pubmed.ncbi.nlm.nih.gov/24072116/
- Edelbring S, Broström O, Henriksson P, et al. Integrating virtual patients into courses: follow-up seminars and perceived benefit. Med Educ. 2012. https://pubmed.ncbi.nlm.nih.gov/22429178/
- Watari T, Tokuda Y, Owada M, Onigata K. The Utility of Virtual Patient Simulations for Clinical Reasoning Education. Int J Environ Res Public Health. 2020. https://pubmed.ncbi.nlm.nih.gov/32722097/
- Patrício MF, Julião M, Fareleira F, Carneiro AV. Is the OSCE a feasible tool to assess competencies in undergraduate medical education? Med Teach. 2013. https://pubmed.ncbi.nlm.nih.gov/23521582/
- Brannick MT, Erol-Korkmaz HT, Prewett M. A systematic review of the reliability of objective structured clinical examination scores. Med Educ. 2011. https://pubmed.ncbi.nlm.nih.gov/21988659/
- Green ML, Moeller JJ, Spak JM. Test-enhanced learning in health professions education: A systematic review: BEME Guide No. 48. Med Teach. 2018. https://pubmed.ncbi.nlm.nih.gov/29390949/