Why Adaptive Learning

Clinical practice and lifelong learning

According to a 2017 report by the American Medical Association (AMA), the new focus of medical education is on “creating physicians who are self-directed, critically thinking, expert workplace learners. These learners learn how to know what they don’t know and appropriately use just-in-time knowledge resources and decision support systems to address identified gaps1.”

With the ever-changing health-care environment, physicians must adapt and learn new ways to solve different and more complex clinical problems. Use of the metacognitive approach — the ability to think about learning based on understanding what you know and what you don’t know — is gaining traction in many areas of education including health care education.

The AMA, as part of its Accelerating Change in Medical Education Initiative, has developed a model to create “master adaptive learners.” This model uses a metacognitive approach to encourage physicians to develop adaptive expertise that will allow them to learn and create innovative solutions in response to practice challenges2.

JBJS Clinical Classroom on NEJM Knowledge+ supports this educational paradigm shift by applying metacognition — by assessing both learners’ knowledge and their awareness of what they know (consciously competent) and what they don’t know (unconsciously incompetent). Even well-trained experienced surgeons must update their knowledge and skills to diagnose and treat increasingly complex patient conditions and improve patient outcomes. Clinical Classroom provides regularly updated, evidence-based, peer-reviewed content to support the maintenance and updating of knowledge and skills required by the changing health-care environment.

Unlike any other study tool

JBJS Clinical Classroom is a learning system that uses a biological model of adaptive learning technology. It observes what and how a person learns and individualizes the person’s experience with precise focus on knowledge, skill, and confidence deficits3.

Users of JBJS Clinical Classroom may spend up to 50% less time than users of traditional e-learning methods to learn the same material3.

When answering questions in Clinical Classroom, learners must choose: “I know it,” “Think so,” “Unsure,” or “No idea” to describe their confidence in the answer they are selecting. The adaptive algorithm looks at how a question is answered (right or wrong), the self-reported confidence level, as well as the time taken to answer each question. It then uses evidence-based algorithms to continually direct users away from subjects in which they are proficient and confident and toward weaker areas until all content is mastered. It adapts as the user learns.

Clinical Classroom identifies learners who are “unconsciously incompetent” — that is, they don’t know what they don’t know. This directs individual learning and provides focus for faculty and residency directors when planning learning experiences.

Study tools that ask hundreds of questions provide experience aimed at passing exams versus mastery and reinforcement of information. Once a learner has answered 200 questions, all they have is 200 answered questions.

By contrast, Clinical Classroom quickly identifies learners’ strengths and weaknesses during the test-taking process so that they can continually focus on areas they have not yet mastered or about which they are uncertain.

Clinical Classroom allows learners to see not only what questions they have missed but also which learning objectives they are finding most challenging to master. The platform also contains an automated “recharge” function to help learners retain previously learned content and to relearn things they may have forgotten over time.

Built by experts

JBJS recruited experts across orthopaedic subspecialties to develop learning objectives that require higher-order thinking, such as an ability to evaluate and diagnose a patient’s condition and to select an appropriate treatment. Writers then developed questions to test each learning objective, provide learning resources with supporting information for the questions, and a list of references for additional information.

All questions and learning resources are peer-reviewed by several subspecialty content experts and the JBJS Editor-in-Chief and then revised as needed before being accepted for use in Clinical Classroom. The platform is updated regularly to provide new learning objectives and questions and to revise or remove any that are no longer current.

With the ever-changing health-care environment, physicians must adapt and learn new ways to solve different and more complex clinical problems.

Better exam preparation

When preparing for an exam, a learner’s time is limited. Because users of Clinical Classroom avoid spending time on content they have already mastered and can focus on areas where they need reinforcement, they may spend up to 50% less time than users of traditional e-learning methods to learn the same materials3.

Clinical Classroom uses the same learning technology as NEJM Knowledge+, which prepares learners for the internal medicine and family practice board exams. Healy et al. reported on a small group of learners who used NEJM Knowledge+ to prepare for the American Board of Internal Medicine certification examination (ABIM-CE) and found that, between 2014 and 2016, a significantly higher proportion passed on their first attempt compared to the national average (95% vs. 89%, z = 2.6397, p = 0.0083)4.

Another study, by Wagner et al., showed that performance on weekly quizzes may be strongly predictive of performance on resident in-service examinations and tracking data from periodic quizzes may help direct educational interventions5. With Clinical Classroom, learners can develop their own quizzes and the system allows residency faculty to develop and assign quizzes to their residents.

  1. American Medical Association. Creating a community of innovation. Chicago: American Medical Association; 2017.
  2. Cutrer WB, Miller B, Pusic MV, Mejicano G, Mangrulkar RS, Gruppen LD, Hawkins RE, Skochelak SE, Moore DE Jr. Foster ing the development of master adaptive learners: a conceptual model to guide skill acquisition in medical education. Acad Med. 2017 Jan;92(1):70-5.
  3. Area9 Learning. Adaptive learning: eliminating corporate e-learning fatigue. 2017.
  4. Healy M, Petrusa E, Axelsson CG, Wongsirimeteeku P, Hamn- vik O, O’Rourke M, Feinstein R, Steeves R, Phitayakorn R. An exploratory study of a novel adaptive e-learning board review product helping candidates prepare for certi cation examina- tions. AMEE MedEdPublish. 2018.
  5. Wagner BJ, Ashurst JV, Simunich T, Cooney R. Correlation between scores on weekly quizzes and performance on the an-nual resident in-service examination. J Am Osteopath Assoc. 2016 Aug 1;116(8):530-4.