More than 100 years after Abraham Flexner reenvisioned medical education in the United States, another great shift is occurring. Rapidly evolving technologies, such as massive open online courses (MOOCs), are disrupting medical education and distance education and are transforming the way we envision lecture halls and classrooms.1 These asynchronous online technologies allow educators to reach and teach thousands of learners without regard to geography or time zones.
There are two different types of MOOC. The xMOOC focuses on the unidirectional transmission of knowledge from teacher to learners and may be a course promoted by a company or university (e.g., Coursera, Udacity, edX). Critics of xMOOCs often argue that these courses are rebranded forms of lecturing and passive learning. In medical education, the use of video-recorded lectures is similar to these “sage on a stage”–type resources.2
The connectivist MOOC (cMOOC), in contrast, emphasizes connections among participants and the generation of new knowledge. Learners and peer-teachers create content on digital platforms to enhance the learning experience for all. The constructivist origins2 of cMOOCs encourage “crowdsourcing” (i.e., obtaining content or ideas by inviting contributions from large groups of people, often online communities of practice) and active educational engagement. This makes the cMOOC model well suited to engaging learners in discussions of ill-defined problems that require complex solutions.
It is difficult to engage practicing clinicians in continuing medical education (CME) that does not focus on their clinical expertise. As a result, few online CME resources have been developed to target professional development in areas such as medical education. In this report, we describe our innovative cMOOC-type approach to creating novel online educational resources that engage clinicians in nonclinical professional development focused on medical education.
Academic Life in Emergency Medicine (ALiEM; www.aliem.com) is an open access medical education Web site that was founded in 2009 by one of this report’s authors (M.L.). ALiEM has grown into a multiauthor educational site that is part of the expanding Free Open Access Meducation movement.3 In 2014, ALiEM received over 100,000 pageviews and 36,000 unique visitors per month on average.
In August 2013, the ALiEM site launched the Medical Education in Cases (MEdIC) series (http://www.aliem.com/medic/) as a strategy to engage practicing health care professionals with online CME exercises. The MEdIC series was conceptualized by one of this report’s authors (T.C.), and then created and implemented by our international, multisite team of authors (M.L., B.T., T.C.). Each month, the ALiEM site publishes a MEdIC scenario that raises nonclinical educational dilemmas and is designed to have layers of complexity so that it bears strong resemblance to a real-life situation. Each of the first six cases (published August 2013 through January 2014) emphasized different CanMEDS and/or Accreditation Council on Graduate Medical Education competencies.
We use online technologies with connectivist and constructivist approaches to engage learners in problem-based learning exercises. The complex medical education scenarios are posted on an open forum with the intention of crowdsourcing solutions to the educational dilemmas by drawing on the wisdom of ALiEM’s readers.
Case development and publication process
Most of the cases in the MEdIC series are based on real scenarios that have been fictionalized to ensure anonymity. Cases are written by an associate editor (T.C., B.T.) or in conjunction with a reader. Each case is sent to two medical education content experts who are given four to six weeks to write expert responses that cite relevant literature and resources.
Each month, one case discussion post, which includes the case and questions for discussion, is published on the ALiEM Web site. The associate editors cofacilitate a Twitter- and Web site–based discussion during the subsequent week. The case is promoted using various social media platforms (Twitter, Rich Site Summary, Facebook, Google Plus). Each participant comment receives a timely response from an associate editor or another participant. No comments are erased or censored.
At the end of the first week, an associate editor thematically analyzes the tweets and comments to create a curated community commentary. This commentary undergoes a member check for authenticity by the other associate editor and/or discussion participants. The curated community commentary and the previously solicited expert responses are then published on the site in the case’s wrap-up post. A free, downloadable PDF file that contains the case and its objectives, the relevant elements from the Accreditation Council for Graduate Medical Education and CanMEDS competency frameworks, the two expert responses, the curated community commentary, and suggested references is also made available. This resource is licensed under the following Creative Commons license: Attribution-NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0).
Readers and discussion participants are encouraged to download and use the PDF resource for their own local teaching and learning. We ask them to report usage of these resources and provide us with feedback.
Figure 1 illustrates our three-phase case development and publication workflow. An annotated illustration of a case discussion post is available as Supplemental Digital Appendix 1 at https://links.lww.com/ACADMED/A269.
In addition to the associate editors, 14 experts and community members from three countries (Canada, the United States, and South Africa) contributed to the first six cases by writing cases and expert responses.
Because of the anonymous nature of our Web-based interface, we cannot confirm the identities of all discussion participants. Approximately 85% of the participants in our Web site–based discussions have used easily confirmable identities (i.e., their full names and credentials could be found by following links in their comments). Of the 194 comments posted in the first six months, just 29 (15%) were anonymous. Anonymous commenters are accorded the same treatment as those who identify themselves.
Measures of reader engagement
Statistics and methodology.
As measures of reader engagement for the first six cases of the MEdIC series, we used data from Google Analytics, the ALiEM Web site, and Topsy (a Twitter aggregator) for the period August 1, 2013 to February 14, 2014, as described below:
- Using Google Analytics, we gathered the following data seven days after each case was published: total pageviews, average time on page, unique users, and location (city and country) of viewers. We chose the seven-day window to allow comparison across cases.
- From the ALiEM Web site, we derived data from the comments section of the posts. For each case, these data included the total number of comments, the average number of words per comment, and the total number of words in the comments section.
- We used Topsy to count the number of tweets made about each case.
- We used the ALiEM Web site’s embedded social media plug-in to record the number of Facebook Likes and Google Plus +1s for each case.
All data analysis—descriptive statistics, Spearman correlations, Fisher exact test, and one-way t test—was conducted using IBM SPSS Statistics for Mac, Version 21.0 (IBM Corp., Armonk, New York). For both parametric and nonparametric tests, we selected a P value of .01 (Bonferroni correction) because we planned four different correlations and comparisons: comparing the number of words in comments for the case discussion posts versus wrap-up posts; correlating the number of moderator comments with participant (i.e., nonmoderator) comments; correlating the number of moderator tweets with participant tweets; and comparing the average time on page for MEdIC cases to the grand mean of time on page within the ALiEM Web site.
The first six cases of the MEdIC series collectively received 6,607 pageviews in just over six months. The median (interquartile range [IQR] 25%–75%) metrics for these six cases in the first week following posting were as follows: 861 (634–1,114) pageviews; 767 (518–953) unique visitors from 326 (218–405) cities in 45 (32–50) countries; 30 (24–39) total comments; 52 (40–56) total tweets of the page’s URL link; 17 (13–30) Likes on Facebook; and 5 (5–7) +1s on Google Plus. Table 1 provides more detailed descriptive analytic data for each of the first six case discussion posts.
The case discussion and wrap-up posts mirror cMOOC-like and xMOOC-like environments, respectively. Case discussion posts, compared with wrap-up posts, had a significantly higher number of comments (median [IQR]: 30 [24–39] versus 4 [2–6]; P < .001) and total words used by participants in the comments section (median: 4,313 [2,909–7,019] versus 333 [44–492] words; Fisher exact test, P = .002). This demonstrates the range of engagement that is possible using the same digital platform. With our sample of six cases, there was no significant correlation between the number of moderator comments and number of nonmoderator comments on the Web site. There was a correlation between the number of moderator tweets and nonmoderator tweets (Spearman rho = 0.710, P = .01), suggesting that the moderators may have had more of an effect on the Twitter-based discussion than on the Web site–based discussion.
The average time that readers spent visiting individual Web site pages (between August 1, 2013, and February 14, 2014) was 3 minutes 38 seconds for the MEdIC posts versus 1 minute 48 seconds for all other (> 1,000) ALiEM posts. Assuming that the time on page for the 6,607 MEdIC pageviews and 568,392 ALiEM pageviews during that same period had a parametric distribution, we used a one-sample t test statistic to compare the average time on page for MEdIC posts and average time for all ALiEM posts. The mean difference between the MEdIC posts and the grand mean for all ALiEM posts was statistically significant at 110 seconds (95% CI 86–134, P < .001). This suggests that, on average, the ALiEM readers of the MEdIC cases spend a longer period of time on the MEdIC pages than ALiEM readers spend on other ALiEM pages.
As of February 2014, six readers had informed us that they used the MEdIC PDF resources to run local professional development or teaching sessions.
This preliminary analysis has several limitations. First, at the time of this report, we had a small number of cases with which to draw correlations and comparisons. However, even with this small sample size, we found significant differences in metrics such as time on page and comment word count.
Second, we used only surrogate measures of reader engagement. These metrics do not necessarily equate to learning, but they do demonstrate a degree of active engagement in the case discussions that suggest a broader audience, something that was not seen with the curated community commentaries and expert responses in the wrap-up post, which represent passive content covering similar material.4 That said, industry standards for reporting online metrics and engagement are the pageview and time-on-page statistics that we reported.
Third, the number of pageviews and unique visitors in the first week are metrics we selected to make it possible to compare the relative reach and popularity of the six cases. We are not aware of any best practices in the medical education literature for reporting normalized statistics for relative comparisons of online resources that are launched asynchronously.
Fourth, our educational innovation requires significant volunteer contributions to be sustainable, which may limit its reproducibility. Substantial work is required to create, promote, and curate each case in the MEdIC series. As noted above, 14 volunteers, under the leadership of two volunteer associate editors, contributed to the first six cases in the MEdIC series. In the subsequent year (through January 2015), more than 30 contributors, editors, and case experts contributed to the MEdIC series.
Fifth, when the MEdIC series launched in August 2013, the ALiEM Web site was already successful (i.e., it was receiving more than 100,000 pageviews per month). The prominence of the ALiEM platform increased the visibility of and interest in the MEdIC series. Initiatives beginning de novo may find it more difficult to engage an audience.
Traditionally, most CME and faculty development activities have occurred at the institutional level using workshop-based formats. Online professional development is a format that has yet to be truly harnessed. A 2013 systematic review5 indicated that only 20 studies had been published describing the use of online technologies to enhance health professions faculty development. Of those studies, just 14 focused on the professional development of health professions educators. Nine of the 20 studies reported a connectivist, discussion-based component, but they all occurred in closed courses or defined groups as opposed to MEdIC’s MOOC-like environment.
As we suggest above, this cMOOC-type teaching strategy may be particularly useful for discussing topics with more nebulous content. The connectivist nature of our online teaching methodology allows us to crowdsource solutions to our problem-based medical education cases via the wisdom of our readers in order to create electronic teaching resources that our readers can then use to teach the topic themselves. For example, the downloadable PDF resource created for each case using this process can be used to facilitate local, live faculty development sessions or teaching workshops.
Future research in this area should focus on developing a better understanding of the materials best suited for this connectivist/constructivist approach; developing reporting metrics for digital scholarship that can serve as proxy measures of learning and determining their validity; and testing learning outcomes to evaluate the effectiveness of such teaching modalities on knowledge uptake and behavioral change in participants.
Our early experience with the MEdIC series demonstrates that a global medical community can be engaged in an open, active, online learning process focused on health professions education. Unlike more passive learning initiatives, such as xMOOCs and vodcasts, this type of teaching might allow online communities to deal with messy, real-life problems that do not have clear answers. Our endeavor is proof of concept that it is possible to engage practicing clinicians in active online discussions around nonclinical and medical education topics.
Acknowledgments: The authors would like to thank all the case contributors, discussion participants, and expert discussants for the Academic Life in Emergency Medicine (ALiEM) Medical Education in Cases (MEdIC) series. They would also like to thank MEdIC assistant editors Eve Purdy, Sarah Luckett-Gatopoulos, and Tamara McColl for their ongoing work with the project.