Abstract
Little is known about the acceptance of different teaching/learning modalities by students and faculty in the preclinical semesters of medical school. We report the results of an anonymous survey at Ross University School of Medicine, where most of the currently popular instructional methods are used. Study subjects included 327 students and 30 faculty members. 5 questions each were asked about lectures, handouts, textbooks, mediasite (video-recorded lectures), simulation, PBL (problem based learning), TBL (team-based learning), and ICM (introduction to clinical medicine, physical examination) practicals, scored on a 5-step Likert scale. Response rates were approximately 80% for students and more than 50% for faculty. Students gave the highest scores to mediasite followed by simulation, handouts, and ICM practicals. Lowest student scores were for PBL followed by TBL and textbooks. Faculty gave highest scores for lectures, followed by ICM practicals and textbooks. They gave the lowest scores for TBL followed by mediasite and PBL. Differences between students and faculty were statistically significant for lectures (), mediasite (), textbooks (), and PBL ().
1. Introduction
Over the last two decades a paradigm shift has occurred in the way medicine is taught and learnt. Traditionally, medical schools have attempted to include an enormous amount of factual information into their preclinical curricula.This has resulted in a perception that students are overwhelmed with information that is scarcely related to medical practice and in this sense irrelevant, and that they respond to this challenge by adopting a strategy of superficial fact learning rather than deep understanding. The solution to this problem was thought to be training medical students as self-learners [1].
Three trends are discernible in the developments of the last three to four decades. One consists of efforts to introduce more clinical content in the first two years of medical school while minimizing the teaching of those basic science elements that have no obvious clinical relevance. The second trend is the development of active learning modalities in which the emphasis is not on fact knowledge and routine clinical skills, but on the development of reasoning and communication skills. This emphasis produced small-group based learning featuring students’ active participation and self-learning, including problem-based learning (PBL) and later team-based learning (TBL) [2]. The third development is driven by innovations in simulation technology and web-based methods. It brought forth video-recorded lectures that students watch at home, and simulation using patient manikins.
In order to be used successfully, these new methods need to be fully accepted by faculty and students. However, surprisingly few studies have investigated the attitudes and opinions of faculty and students about these rapidly diversifying teaching modalities. For example, Alimoglu et al. [3] report that in the first and second years of medical school in Turkey the students gave lowest ratings for lectures. They were more satisfied with PBL, and most of all with practicals. Henning et al. [4] reported that students expressed needs for: more clinical exposure early in the curriculum; fewer lectures; greater consistency of assessment; and more constructive relationships. Jelsing et al. [5] studied attitudes of first year students at Mayo Medical School. Student ratings for satisfaction and perceived learning were low for clinical integration sessions, mentor interaction and shadowing relative to didactic sessions, TBL, and independent study. Other studies have been done on attitudes to individual teaching modalities such as TBL, but without comparing them with attitudes to other modalities used at the same school [6, 7]. Given the ubiquity of acquiescence bias in survey research [8], high ratings for one modality in isolation must not be interpreted as evidence that students or professors prefer this modality to the alternatives.
The current study assesses differences in the opinions of students and faculty about 8 different teaching/learning modalities in the first 2 years of medical school. The modalities include lectures, PBL (problem-based learning), a form of TBL (team-based learning), mediasite (video-recorded lectures on the Internet), textbooks, “handouts” (mainly printouts of the PowerPoints presented in the lectures), ICM (Introduction to Clinical Medicine) practicals (mainly physical examination), and simulation. The study further explored whether Ph.D. and MD faculty members differ in their opinions. For students, we examined the effects of demographics and MCAT (Medical College Admission Test) scores on preferences for learning modalities.
2. Methods
2.1. Setting
Ross University is a medical school on the Caribbean island of Dominica. Nearly all students are United States citizens or residents who intend to practice medicine in the US. The school operates on a trimester system. At the time of the study (2009-2010), combined enrolment in the 4 trimesters of the basic science program was 1400 students, who were trained by approximately 60 full time and 20 part time faculty members. On average, about 100 out of the total of nearly 400 students per trimester were enrolled in a Progressive Academic Education (PAcE) program that differs from the regular program mainly by inclusion of weekly “TBL” sessions. Students volunteer for the PAcE program. At the end of their basic science program, all students have to pass the first part of the externally administered United States Medical Licensing Examination (USMLE), before being promoted to the clinical clerkships.
2.2. Teaching/Learning Modalities
The following modalities are used at the school.
Lectures
Attendance of lectures (average of 20 hours per week) is optional for all students. Typical student lecture attendance is 20% to 25%.
PBL (Problem-Based Learning)
4 PBL cases per trimester are handed out piecemeal, with mandatory weekly 2-hour sessions over 9 weeks. Group size is 8 students, and a faculty facilitator is present in each session.
TBL (Team-Based Learning)
The main activity consists of large-room quiz sessions in which multiple choice questions are answered first individually, followed by discussion in groups of approximately 6 students, and a group test. In the later part of each session, expert faculty members are available for explanations and to answer student questions. Application exercises in the form of case studies are included, but the peer evaluations that are part of “canonical” TBL are not.
ICM (Introduction to Clinical Medicine) Practicals
Small-group training in physical examination and related clinical skills, mandatory for all students. Most sessions are in trimester 4, but there are some sessions in trimesters 1 to 3.
Simulation
Small-group sessions with patient simulators, mandatory in all 4 trimesters.
Mediasite
Videotaped lectures, accessible online.
Textbooks
Each course has recommended textbooks.
Handouts
These are mainly printouts of the PowerPoints presented in the lectures.
2.3. Study Design
The opinions of students and faculty were solicited with a questionnaire that was designed in two forms: one for students, and the other for faculty. Question wording is reproduced as follows. Five questions were asked about each of the 8 teaching modalities. Answers were on a 5-step Likert scale.
Questionnaires used for students and faculty. Note that 5 questions are asked (each on a 1-to-5 Likert scale) about each of the 8 teaching/learning modalities.
Student Version of the Questionnaire: Age: Gender: MCAT Score:(1) How much do you personally like to learn by (modality X) and why?(2) How effective is (modality X) for you to acquire clinical knowledge, and why?(3) How effective is (modality X) for you to acquire basic science knowledge, and why? (4) How effective is (modality X) for you to acquire clinical judgment, and why?(5) How effective is (modality X) for you to prepare for step 1 USMLE, and why? Faculty Version of the Questionnaire: Age: Gender: Teaching experience at medical school: Highest academic degree obtained: (1) How much do you personally like to teach by (modality X) and why? (2) How effective is (modality X) for students to acquire clinical knowledge, and why? (3) How effective is (modality X) for students to acquire basic science knowledge, and why? (4) How effective is (modality X) for students to acquire clinical judgement, and why? (5) How effective is (modality X) for students to prepare for step 1 USMLE, and why? The answer choices for all the above questions were as under: (1) Not at all (2) Somewhat (3) Moderately (4) Highly (5) Most.
All student participants were in the PAcE program. PAcE students were used because of their exposure to TBL (in addition to all other learning modalities), and easy accessibility during their mandatory weekly sessions, in which the student questionnaires were distributed. Students were asked to return their filled questionnaire to the researchers during the TBL session, or leave it on the work table at the end. Participation in the study was voluntary. The protocol had been approved by the Institutional Review Board of Ross University.
2.4. Responses
30 faculty members (17 MD and 13 Ph.D.) and 327 students returned completed questionnaires. The students included 68 from trimester 1, 87 from trimester 2, 115 from trimester 3, and 57 from trimester 4. Response rates were approximately 50% for faculty and 80% for students.
2.5. Statistical Analysis
SPSS 16.0 software was used for all statistical procedures. The reported significance values are for two-tailed t tests throughout, without assumption of equal variance between groups.
3. Quantitative Results
3.1. Differences between Faculty and Students
The judgments of students and faculty were similar in many respects. Across all 40 questions, the correlation between the average scores awarded by faculty and students was (). Table 1 shows, for example, that students and faculty alike believe that ICM physical examination sessions are useful for the acquisition of clinical knowledge; and both students and faculty believe that PBL is less effective than other learning modalities in preparing students for their licensing exam.
At the individual level, responses to the five questions that were asked about each learning modality were always highly correlated. For each of the 8 modalities, the unrotated first principal component of a principal component analysis explained between 59.6% and 68.4% of the variance for students, and Cronbach’s for these 5-item scales ranged from .794 to .879. Therefore we added the responses to the 5 questions for each modality separately for students and faculty to obtain an overall score for the strength of positive and negative attitudes. The score could vary between 5 and 25.
Figure 1 shows that all modalities get high scores, with averages not far from 20 on the 5-to-25 scale. There are nevertheless striking differences between students and faculty. One difference is that faculty members give high scores to live lectures and low scores to mediasite. For students, these preferences are reversed. Another difference is seen for textbooks, which are appreciated more by faculty than by students. After a Bonferroni correction for multiple testing, differences between students and teachers remained statistically significant at for lecture and mediasite, and at for textbooks. A nominally significant difference for PBL, with higher scores by teachers than students (), became nonsignificant. However, the Bonferroni correction is considered a stringent test and is often criticized for producing too many type 2 errors (false negatives) [9].
Of the small-group learning modalities, simulation and ICM practicals are highly appreciated by both students and faculty. However, problem-based learning (PBL) and team-based learning (TBL) get relatively low scores from both. The low student scores for PBL are especially striking. As shown in Table 1, many students report not liking PBL. They doubt that PBL is useful for acquiring basic science knowledge and clinical knowledge, and they do not believe that it prepares them for the external licensing exam.
3.2. Differences between Ph.D. and MD Faculty
Figure 2 compares the opinions of Ph.D. and MD faculty. Substantial differences are seen only for PBL () and ICM practicals (), which are appreciated more by MDs than Ph.D. degrees, and lectures (), which are more popular with Ph.D. degrees. These significance values did not survive a Bonferroni correction. The low statistical significance values are not due to small effect sizes but to small sample sizes, which range from 2 to 8 for Ph.D. degrees and from 10 to 17 for MDs.
3.3. Differences between Students in Different Trimesters
There were few systematic differences between students in different trimesters. Only two results stand out in Figure 3. When “first-year” (trimester 1 and 2) students are compared with “second-year” (trimester 3 and 4) students, PBL is evaluated more favorably by the second-year students (, after Bonferroni correction). When trimester 4 is compared with the combined trimesters 1, 2, and 3, we observe lower ratings for ICM practicals by 4th trimester students compared to the lower trimesters (, after Bonferroni correction). However, TBL is rated somewhat higher by trimester 4 students (, after Bonferroni correction).
(a)
(b)
3.4. Effects of Student Sex, Age, and MCAT Score
None of the 8 learning modalities was preferred specifically by males or females, but females rated all modalities more favorably than did males. This difference suggests greater acquiescence bias of females, meaning a tendency to agree with statements of all kinds. When an acquiescence score was formed as the average of all 40 scores (8 modalities, 5 questions/modality), the gender difference was highly significant for both students (, , ; after Bonferroni correction) and faculty (, , ; after Bonferroni correction). Maximum-likelihood factor analysis showed that for students, 19.5% of the total variance in responses to the 40 items was explained by acquiescence alone. These 40 questions produced an excellent acquiescence scale, with Cronbach’s of .889.
The effects of gender, age, MCAT scores, and trimester were further investigated with regression models predicting the summed scores for each of the teaching modalities. Acquiescence was included in the models, but had only minor effects on the significance levels of the other predictors. Few significant results were obtained. MCAT score was related positively with preference for lectures () and negatively with preference for simulation (). Handouts became more popular in the higher trimesters (). Older age was associated with somewhat lower scores for mediasite (). There was also a trend for mediasite to be less popular with the higher trimesters (). None of these results remained statistically significant after Bonferroni corrections for multiple testing (4 predictors in 8 regressions = 32 measurements).
3.5. Factor Structure of Student Preferences
Preferences for individual learning modalities are likely to depend on broader dispositions, which can be investigated by factor analysis. In an exploratory factor analysis using the maximum-likelihood method, the scree plot suggested 4 factors. Table 2 shows the factor loadings after a 4-factor varimax rotation. Factor 1 reveals that students who like ICM physical examination practicals also tend to like simulation and vice versa. Factor 2 describes the lecture-learning students who rely on handouts (printouts of the PowerPoints presented in the lectures), mediasite (video-recorded lectures), and live lectures. Book enthusiasts get high scores on factor 3, and factor 4 defines small-group learners who like TBL and to some extent PBL.
In regression models predicting these four factors with sex, age, trimester, MCAT (medical college admission test) score, and acquiescence bias, acquiescence was always a strong predictor, but results for the other predictors were similar when acquiescence was omitted from the models. MCAT was a negative predictor of factor 1 (), meaning that those with lower MCAT scores tend to prefer physical examination practicals and simulation. Factor 2 was related positively to trimester () and negatively to age (). There was virtually no collinearity between trimester and age, which had a correlation of only +.029. However, these significance values did not survive a Bonferroni correction.
4. Qualitative Results and Discussion
This study produced several important results. The first is that most people like most learning modalities. On the 5-to-25 scale used in Figures 1 to 4, average scores hover near 20. Even problem-based learning (PBL), which got the lowest student marks in Figure 1, still scored above the midpoint of the scale. If we had studied PBL in isolation, we might have concluded that PBL is well appreciated by students. Although similar experiences have been made by others [5], the literature contains reports of high acceptance rates for single teaching/learning modalities in the absence of comparative data for other modalities used at the school [6, 7]. Our results suggest that the interpretation of studies that are limited to a single teaching/learning modality needs to be reconsidered based on the ubiquity of either high general student satisfaction or acquiescent response style or both.
It is tempting to interpret the generally high marks as acquiescent response style [10], and attribute the generally more favorable ratings by females than males (Figure 4) to greater female acquiescence. However, the largest available study of acquiescent response style in a worldwide sample had found no significant gender effect in most countries [8]. Therefore it is possible that the higher female scores that we observe in our study reflect genuine sex differences in enthusiasm for learning or teaching, independent of the specific modalities.
Differences between the opinions of students and faculty are far more specific than those between males and females. Students’ distaste for didactic lectures is not surprising, since it has been reported by others [3]. In the open-ended “why” questions of the questionnaire, students frequently mentioned ill-defined objectives, inconsistency of lectures, repetitive teaching by faculty, and gaps in teaching attributed to lack of communication between faculty member.
Revealing is that although live lectures are generally unpopular, the same lectures are more highly appreciated when delivered through the computer terminal. Students frequently commented on their questionnaires that recorded lectures give them the freedom to study anytime day and night, to double-speed less important sections and repeat the more important ones. Students’ ready acceptance of canned lectures is not unexpected. Experience at other schools has shown that students rate e-learning as highly as more traditional methods of clinical skills teaching and appreciate its integration in a blended approach [11, 12].
Nevertheless, students who view live lectures favorably also tend to give high scores to mediasite (video-recorded lectures) and handouts (printouts of the PowerPoints presented in the lectures). This lecture-focused study style is captured by the 2nd factor in Table 2. Our data suggests that despite frequent criticisms, students do not perceive the lecture content as useless. They rather object to the need to be in the classroom at the appointed times. Teachers, conversely, feel frustrated lecturing in a half-empty lecture hall. Several faculty members remarked on their questionnaires that videotaped lectures are a passive way of learning without teacher-student interaction and feedback.
Another unsurprising difference is students’ dislike for textbooks. Most professors recommend textbooks in an attempt to teach in-depth understanding of their field. They seem to agree with Sir William Osler, who wrote: “To study medicine without books is to sail an uncharted sea, while to study medicine only from books is not to go to sea at all” [13]. Students seem to value focused learning of those concepts and facts that they consider important for medical practice or for passing high-stakes exams. In our experience, most students shun traditional textbooks and instead prefer “high-yield” board-review books that condense the basic medical sciences to those elements that are most likely to be tested. Printouts of the PowerPoints presented in the lectures are more popular than books because they are perceived as more focused on the most important concepts and facts. Nevertheless, there are students who prefer studying from textbooks and other written materials, as indicated by factor 3 in Table 2. We also need to view students’ distaste of books on the background of a secular rise in extraversion that has been recorded among American students in the late 20th century [14]. Extraverts do not usually enjoy books.
However, extraverts should enjoy small-group activities such as PBL and TBL. Much of the spread of these forms of instruction in recent decades can possibly be attributed to rising extraversion among students and faculty. The low appreciation of PBL and also TBL (Figures 1 and 3) is therefore surprising.
This observation contrasts with findings by others who reported favorable student views of TBL in medical anatomy [7]. Others have reported that TBL enhances mastery of course content, especially for weaker students [15], and that it is more effective than the usual small group lecture in achievement of knowledge objectives for clinical clerkship students [16]. The guarded reception of PBL and TBL by our respondents is even more surprising considering that they have volunteered for a TBL-based program. Presumably, they did so because of a pre-existing inclination to small-group work and/or self-directed learning.
Comments by students and faculty show that the main perceived disadvantage of PBL is that it is time-consuming and that its assessment is subjective. Other studies have reported similar perceptions [17, 18]. In addition, Table 1 shows that PBL is not perceived as an effective way of learning core basic science material. Some faculty members nevertheless view PBL favorably because they consider it useful for learning skills such as communication and “professionalism.”
We find significantly higher PBL marks in trimesters 3 and 4 compared to trimesters 1 and 2 (uncorrected ). One reason might be that a main function of PBL is the integration of different facets of basic science knowledge in their application to a clinical case. Integration of basic science knowledge is inevitably difficult in trimesters 1 and 2 because the students have little knowledge that they could apply to a clinical case. Possibly, PBL is more effective with more advanced students who already have some basic and clinical knowledge.
Comparison of Figures 2 and 3 shows that ratings of PBL by Ph.D. faculty are no better than those by 1st and 2nd trimester students. MD faculty view PBL far more favorably. One possible reason for the relatively high marks by MD faculty is that some MD faculty members are fresh medical graduates in search of a residency, who are on temporary PBL instructor positions. Their vested interest in PBL may have affected their ratings.
Not all small group teaching receives low marks. ICM practicals (physical examination) and simulation are well appreciated by students and faculty. Student ratings for ICM practicals were high in the first three trimesters but declined significantly in trimester 4. The likely reason is that there are only a few ICM practical sessions in the first three trimesters, and their grading is described as “lenient” by the involved faculty. These sessions become a major part of the curriculum only in trimester 4, when students have to pass an important practical exam. Frequent sessions and/or a serious examination seem to reduce the glamour of these exercises. As expected from their educational background, ICM practicals were appreciated more by MD faculty than Ph.D. faculty (Figure 2).
Why are these modalities viewed more favorably than PBL and TBL by both students and faculty? One possibly important difference is that ICM practicals and simulation are content-driven, whereas PBL and TBL are process-driven. Medical students have to learn physical examination, and they have to learn it in small groups. The procedures used in these small-group sessions are determined pragmatically by the objective of learning physical examination. There is also agreement that medical students have to learn the basic routines of emergency medicine, and simulation is an obvious method. Unlike the other forms of small-group teaching, simulation has the added advantage of engaging students’ fascination with high-tech virtual reality.
PBL and TBL, however, are defined not by essential content but by somewhat stereotyped procedures. In PBL, cases are handed out piecemeal. Students have to write data, analysis, hypothesis, and learning objectives on the board, and research their learning objectives in preparation for the next group session. In TBL, students are exposed to an individual quiz, group discussion, and group quiz. These procedures do not conform to students’ principle of “high yield” learning, which means learning as much useful knowledge as possible as fast as possible.
We must not forget that these results were obtained at a very large school that admits approximately 1200 students per year: 400 each trimester. For PBL, each class is split up into about 50 groups. With an average of 2 PBL groups per faculty member, there will be 25 facilitators who have very different approaches to group work and grading, resulting in frequent complaints of inconsistencies. Even in TBL there are approximately 100 students in the classroom, divided into 15 or more groups. Possibly, PBL and TBL can be used more effectively with smaller classes.
The major limitation of this study is that its findings are from a single medical school. To some extent, the results reflect the ways in which teaching methods are used in that medical school, rather than the appeal of each teaching modality in any general sense. Therefore a main conclusion is that similar surveys need to be conducted at other medical schools, to show whether the patterns we observed are frequent or are specific to the time and place of this study. Such studies should not be limited to a single teaching modality because both students and teachers tend to give high scores to all modalities. To get meaningful results, different modalities need to be compared.
Practice Points
(i)There is moderately good agreement between faculty and students in their judgments about 8 different teaching/learning modalities ().(ii)Faculty members prefer live lectures, whereas students prefer the same lectures delivered electronically.(iii)Textbooks are appreciated by faculty members but not students.(iv)PBL and TBL are less popular than other modalities, with students as well as faculty.(v)Clinical skills sessions in physical examination and simulation are well received by both faculty and students.
Conflicts of Interests
The authors report no conflict of interests.
Acknowledgments
The authors acknowledge all the faculty and students of the Ross University School of Medicine, Dominica for their participation in this study.