Background. McMaster University first introduced Problem Based Learning (PBL) in the mid 1960s. However, measuring the relationship between PBL for undergraduate nursing programs and students test performance has not yet been assessed in the USA. Purpose. The main purpose of this paper is to describe the effectiveness of PBL on senior student test performance on content related to PBL in a BSN program. Diabetes mellitus and renal insufficiency were taught by traditional lecture format in the previous years. This was the first year we taught this content by the problem based learning method. Method. Historical control group was used to compare the test performances between the PBL groups and the traditional group using Student’s t-test. Result. The mean of diabetes mellitus related questions missed by the PBL group was less than the traditional group (, and ). The mean of renal insufficiency related questions missed by the PBL group was more than the traditional group (, and ). Discussion. This study produced inconclusive findings. Factors that could be attributed to their performance will be discussed.

1. Introduction

McMaster University Medical School conceptualized Problem Based Learning (PBL) which is “the learning that results from the process of working towards the understanding of, or resolution of, a problem” [1]. The core value of PBL is to use a contextualized problem to motivate learners to actively seek relevant knowledge using all possible resources. PBL is intended to equip students with hands-on learning strategies to help them meet their future responsibilities and establish a lifelong knowledge-seeking habit which is self-directed learning. PBL has since been adopted by other medical schools and adapted by other disciplines leading to an assortment of learning and teaching models. Studies indicate that students prefer PBL to traditional lecture formats [25]. Numerous studies indicate that the process of learning is different in PBL and PBL challenges students to become self-directed life learners [2, 3, 6]. Few studies have demonstrated the effectiveness of PBL in an undergraduate nursing program on student test performance. This paper reports the effectiveness of PBL by comparing the test performances from the PBL group and the traditional group in an undergraduate nursing program in a private university in the Southeastern United States.

2. Background

2.1. PBL and Learning Styles

Lecture based formats tend to promote “surface” learning, where the student is able to reiterate what was covered in the subject-centered material. On the other hand, PBL promotes “deep understanding” where students study more for meaning and less for reproduction [7]. Those students who are using PBL tend to access resources more frequently with a more intentional style of learning because PBL reshapes the learning style and developing patterns that define the proactive lifelong learning when the student transitions from a novice to an expert learner [8]. PBL also provides a more challenging, motivating, and enjoyable approach to enhance education [9]. While PBL fosters more in-depth learning, this in-depth learning may come at a cost to the breadth of learning with less focus on required essential information delivery [1].

2.2. PBL and Learning Outcomes/Performance

PBL’s impact on the attendants’ learning objectives is largely positive but not entirely so. Blake et al. [10] found that PBL contributes to the higher performances of medical students on the licensing examinations compared with those who received traditional lecture teaching. Licensing examination is a typical example of standardized test which contains consistent question formats, conditions for administering, and resulting scoring procedures. However, the findings on the student’s performance after PBL education on standardized examinations were not consistent. Some studies indicated that the PBL group did not perform as well as the students in traditional curricula on standard examinations that measure basic science knowledge [2, 11]. PBL had no positive effect on the final term exam of medical students; however, it enhanced their knowledge, understanding, and retention of the subject course [12]. Recent studies have found that PBL does not compromise performance on standardized tests, particularly when examinations moved from “measuring factual knowledge to assessing the application of knowledge” [10]. PBL was significantly superior with respect to students’ program evaluation, their attitudes toward learning, and their PBL class attendance [11]. In addition, PBL students have consistently outperformed those students from traditional programs on measures of diagnostic skills and clinical reasoning [9, 13].

2.3. PBL and Nursing

PBL is based on or situated in healthcare and activates prior knowledge that is then elaborated in small group discussion. The process facilitates learning and knowledge retrieving [9]. Retrieving knowledge and applying it to a clinical setting is an important step to help a novice nurse to become an expert [8]. Rapidly advancing medical technology and science requires nursing education to become “hybrid,” combining old teaching methods with innovative methodologies [14]. Being an active learner and keeping abreast with current knowledge is a necessary trait for being a nurse. Nursing education has begun to shift from teacher-centered learning to student-centered learning where students are required to actively seek knowledge. Instead of training for the nurses who only can see one problem, PBL offers a more holistic perspective of the problem in its own contextual nursing environment. PBL fits well with the concepts of the nursing metaparadigm (person, environment, health, and nurse (verb)) [15]. The PBL process addresses each problem with a focus on person (problem embedded in that person), the specific environment that affects the person’s health, particular health related issues, and the nursing strategies/management executed to promote a person’s health. In addition, the PBL process is a learner-centered process that triggers free requisition for knowledge by the learner. PBL has been incorporated into some undergraduate nursing programs but its effectiveness has not yet been reported [1619]. One study showed that PBL will produce proactive learning and suggested that nursing education should incorporate PBL [20]. Beers and Bowden reported that those nursing students taught with PBL had significant improvement in long-term knowledge retention compared with their counterpart [21]. One study indicated that Korean nursing students who were taught by PBL had better test performance than the traditional group [22]. The purpose of this study was to evaluate the educational benefit on those who were taught in traditional didactic methods versus PBL.

3. Method

3.1. Conducting a PBL

In its purest form, PBL begins with a complex or “messy” problem that precedes and motivates learning among students. The discussion that emanates from the problem activates prior knowledge that students have accumulated through previous nursing courses, formal education, life experience, or even media exposure. DM and renal insufficiency were chosen because they are diseases that can happen at any stage of a life span and the complexity of each has broad association with basic science and clinical experience. The translational natures of the diseases will provide students with a rich experience with learning in nursing management. These two PBLs were conducted in the same manner and each PBL was progressively unveiled in three sessions. Beginning with considering the contextual setting of one problem, students then identified their knowledge gaps. Knowledge gaps are evolved into learning objectives that students work on during the week. Students, then, developed their learning objectives to fill the gaps and divided these learning objectives amongst themselves as information seeking assignments for the following week. They worked on the problem and its progression or evolvement for three weeks with unveiling different aspects of the problem related to the personal developmental, psychological, and family dynamics. During their one-hour session in the second and third weeks, students shared findings with the group and tried to integrate the newly acquired knowledge into a comprehensive explanation for the problem and nursing management. These sessions are facilitated by a tutor; however, it is the students who identify knowledge deficits and learning issues for the subsequent session.

Eight faculty members were the PBL “tutors” for the sixteen PBL groups with each tutor facilitating two PBL groups. The tutors who all have clinical nursing experience with DM and renal insufficiency, however, were not to act as content experts but facilitators of the group process. It is inevitable that each faculty member possessed her/his own facilitating style. Throughout the PBL process, tutors were instructed not to dominate the group discussion but to provide information about the patient from the problem related subjective issues or objective findings via role play. If information was not available but a question was posed that would promote more in-depth learning, the tutors would challenge students about the importance of this piece of information and encourage them to seek additional relevant information.

3.2. Sample

This study used a historical control group with comparative descriptive design with a convenience sample. The nursing students in this institution have at least completed two years undergraduate program and fulfilled the prerequisites for the two additional years of nursing school to complete the nursing baccalaureate degree. The two groups used for this study included (1) traditional group comprised of seventy-eight nursing senior students from the previous year who received the traditional teaching with a didactic or lecturer-centered learning process and (2) PBL group which consists of ninety-one senior year students from the same institution where they studied the same content, however, but used student-centered learning process with PBL in contrast to the previous year’s senior group. Due to the high demand on nursing students’ enrollment, the decision was made on implementing PBL in their senior group to avoid big classroom teaching and provide a flavor of small classroom learning experience which is the draw for many students attending the private university. All students were surveyed by email regarding previous PBL experience from previous colleges or programs. One student had direct PBL experience from another university; three students had observed PBL in other countries during their exchange student experience. To the rest, PBL was the first time in their high education.

3.3. Purpose and Procedure

The purpose of the comparison analysis is to evaluate the educational benefit on students’ test performance in those who were taught in a traditional didactic method versus PBL. We followed the steps for the PBL by the guidelines developed at McMaster University. These steps are as follows: (1) clarify unknown terms and concepts in the problem description; (2) define the problem; (3) analyze the problem and try to produce as many different explanations for the problem using prior knowledge and common sense; (4) critique the explanations proposed and try to produce a coherent description of the processes that underlie the problem; (5) formulate learning issues for self-directed learning; (6) fill the gaps in one’s knowledge through self-study; and (7) share findings with the group and try to integrate the knowledge acquired into a comprehensive explanation for the problem [23]. These steps were incorporated into the PBL curriculum for nursing management for diabetes mellitus (DM) and renal insufficiency patients.

Ninety-one senior students were enrolled in a nursing core course offered to all undergraduate senior nursing students addressing caring for clients with diseases across life span. These students were assigned to eighteen PBL groups with five students in each group (one group had 6 students due to student total number). All students had library orientation to prepare them to actively conduct effective literature search. Except for the content regarding DM and renal insufficiency, all other materials in this course were taught in a traditional didactic method. PBL was used for DM and renal insufficiency subjects. To avoid narrowing their exploration of the PBL problems, we did not give any clues or assigned reading information related to either DM or renal insufficiency.

After each PBL session, students evaluated their own and their peer’s performance. They also evaluated their tutor’s ability after each PBL session on tutor’s ability to inspire, motivate, and encourage acquisition of new knowledge during the process. The tutors also evaluated students’ performance at the end of the third PBL session. During the last session of PBL, the intended learning objectives developed by the course instructors were shared among the students to ensure that the basic subject related knowledge has been learned by the students. If there is any knowledge gap between the content sought in the PBL and intended learning objectives, the students have to actively seek knowledge to fill the gap. A test was given a week after the intended learning objectives were given. These intended learning objectives are the core requirements on nursing management on specific disease.

3.4. Test Item Analysis

All test items were written in multiple choice question formats. Course test results were used to compare the PBL and the traditional group from the previous year. Thirty DM related questions and seventeen renal insufficiency related questions were given to these two groups during their senior year in two consecutive years. The questions addressed nursing process (assessment, planning, implementation, and evaluation) and tested students’ cognitive level. The difficulty of those test questions was based on Bloom’s taxonomy [24] ranged from knowledge (remembering information), and comprehension (understanding information), application (applying information to a situation) to analysis (comparing and contrasting a situation). Nearly 50% of the DM test questions were measuring students’ ability to apply their knowledge to a situation. About 12% test questions on renal insufficiency was testing students’ ability to retrieve or recall the knowledge they learned. With intention of comparing test results, all questions were not modified from the previous year in order to ensure the comparability. To compare the different test results from the PBL group and the traditional group, a Student’s t-test was used for data analysis. Before the data analysis, descriptive statistics were examined and this showed that the two groups were comparable. Both group’s test performances were examined for outliers and assessed for the assumption of normality using skewness and kurtosis. The two sets of data have normal distribution. All statistical analyses were performed using the Statistical Package for Social Sciences (SPSS) software.

4. Results

4.1. Demographic Characteristics

The demographic characteristics are comparable between the two groups (Table 1). The mean age for traditional group was 28 years old and the mean age for PBL group was 27. Most students are female in both groups. The average grade point average (GPA, usually A equals to 4 and D is 1) was 3.46 for traditional group and 3.44 for PBL group. Sixty-nine percent of students were Caucasians in traditional group and 55.2% in PBL group. The PBL group had more students from either Black or other minority groups. Sixty-four percent of students had a bachelor degree in a health related field in traditional group while only 49.45% of students in PBL group had a bachelor degree in a health related field prior to coming to the nursing school.

Table 2 indicated that, of all thirty questions related to DM, students in the PBL group answered 3.19 questions incorrectly (SD 1.97) and students in the traditional group answered 4.76 (SD 2.53) questions incorrectly. The difference in means between the two groups was statistically significant ( and ). In the renal insufficiency test result comparison, the mean of questions missed by the PBL group was 4.54 (SD 1.74) compared with the traditional group 2.94 (SD 1.46). The difference in means between the two groups was also statistically significant ( and ).

5. Discussion

The findings from this study are not conclusive on student test performance. These inconclusive findings could be caused by students having differences in prior knowledge in diseases and nursing management in these two subjects. Although both PBLs began with a “messy” statement, the students expressed that they were more familiar with DM than renal insufficiency. We also acknowledge that the prevalence of DM is much greater than renal insufficiency and even general public has more knowledge on DM than renal insufficiency. During three weeks of unveiling of the progress of renal PBL, information on patients’ lab work demonstrated a severe case of fluid, electrolyte, and acid-base imbalances which presented more challenge to the students.

Given the fact that PBL was designed for higher education and for students who have already learned some self-directed learning skills, it is not difficult to conclude that those students would do better in a test with questions related to applied knowledge and analysis component (high level) than in test questions that only require memorization or simple retrieving and comprehensive knowledge. The test item analysis indicated that most of questions in DM are high level questions. These findings support the aforementioned literature review [10]. Renal insufficiency test questions required more retrieving of information such as information on fluid, electrolyte, and acid-base imbalances versus test questions on DM that were heavy on application and analysis. In addition, mastery knowledge on fluid, electrolyte, and acid-base imbalance can be challenging even to physicians [25, 26]. It is possible to conclude that the test for the contents covered in the PBL should rather be focused on application and analysis, which require a breadth of knowledge, while tests for measuring the subject focused in-depth knowledge should be complimented with at least some lecture format to ensure that the basic knowledge has been mastered.

Other than test performance, we observed that students had made progress in their knowledge acquisition process. During PBL consequential sessions, students generated more learning objectives addressing broader and holistic nursing care during the following weeks and the progression of these learning objectives reflected that deeper learning had occurred. For example, some learning issues identified in the second week were as follows: the genetic role played in DM case needed to be explored after they found out that the patient’s father has noninsulin dependent diabetes mellitus, the relationship between stress and disease progression, and the social support system that could be helpful for an adolescent teenager coping with DM.

6. Conclusion

As research indicated that PBL fosters more in-depth learning; this in-depth learning may come at a cost to the breadth of learning [1]. Blake et al. also suggested that PBL especially improved students’ performance on knowledge application [10]. This could offer some recommendations for future improvement. Difficult subjects which require more fundamental knowledge retrieving and comprehension should be taught in different formats which include but are not limited to didactic lecture, seminar, and case study to ensure mastery of the content knowledge by the students. We should begin with incorporating the most common and less complicated diseases such as peptic ulcer, pneumonia, and congestive heart failure during their junior year and move into more complicated problems. As they are working through PBL problems from their junior year, instead of senior year, students could gain more confidence in working with PBL problems. Further, the inconsistency of different styles of different tutors could affect the outcome of students’ learning experience. One recent study suggests that the effective utilization of digital PBL with incorporated electronic health records into programmed simulations to provide consistent information as the case is unveiling would enhance the students’ learning experience [27, 28].

Conflict of Interests

The author declares that there is no conflict of interests regarding the publication of this paper.