Abstract
Evidence-based practice is the foundation of nutrition and dietetics. To effectively apply evidence-based practice, health professionals must understand the basis of research. Previous work has identified the lack of involvement of dietitians in research. As part of a curriculum redevelopment in undergraduate nutrition and dietetics courses, research skill teaching was enhanced. This study evaluated the effect of a new, year two level nutrition research methods unit on the perceived research skills of students. The unit consisted of two key components: a student-led class research project and a small group systematic literature review. Prior to commencement and on completion of the course, students completed a modified version of the Research Skills Questionnaire. Results demonstrated that self-perceived competence increased by a small degree in a set of specific research skills as well as in broader skills such as information gathering and handling, information evaluation, ability to work independently, and critical thinking. The new research unit was also evaluated highly on a student satisfaction survey. Despite these positive findings, students indicated that their general feelings towards research or a career in research were unchanged. In summary, this unit enhanced students’ perceived research skills. Further exploration of students’ attitude towards research is warranted.
1. Introduction
Health professionals are increasingly working within the evidence-based practice paradigm to support the provision of safe and quality care for their patients. Evidence-based practice is defined as the careful and considered use of up to date best evidence in making plans to improve the health of individuals and populations [1]. To be able to successfully implement evidence-based practice, health professionals must first understand research methodology to enable informed critique of relevant evidence. Despite evidence-based practice being a core standard for all health professionals, few health practitioners are engaged in research [2].
Nutrition and dietetics are a health profession concerned with the treatment and prevention of nutrition and lifestyle related diseases of individuals, groups, and populations, by influencing eating behaviors and the wider food environment affecting sustainable and nutritious, food supply, policy, and intake. Previous work has identified the lack of involvement of dietitians in research [3] and a range of barriers reported as factors influencing this level of commitment [4]. Together with perceptions and attitudes towards evidence-based practice, knowledge of research, years of experience, and mandate for research within job role have been shown to be the greatest predictors for dietitians involvement in and capacity for research [5–7]. A recent randomized controlled trial demonstrated that involvement in tailored education, regardless of format, increased research self-efficacy of dietitians [8]. For dietitians to be able to more effectively contribute to improvements in health, there is a need for greater investment in research capabilities [9]. Little is known about what predicts research outcomes for nutrition professionals not involved in patient care.
The development of research skills for many health professions commences in undergraduate education yet there is limited evidence regarding effective research teaching and learning approaches. In nutrition and dietetics, the literature suggests that hands on, real life, independent research experiences are valued by students [10] and that personal interest, leadership from role models, and supervisors influence attitudes towards research [11]. It has been suggested that research skill development for dietitians should commence early in undergraduate training, be contextualized, and involve real experiences [11]. With growing student numbers, providing “real” experiences is challenging and so unique teaching and learning approaches must be developed.
This research aimed to evaluate the effect of a new, undergraduate year two level, nutrition research methods unit on research skills ability of students. The unit aimed to increase students’ ability to design, conduct, and communicate a research study in nutrition and develop and implement a systematic literature review.
2. Methods
2.1. Teaching and Learning Approach
As part of a curriculum redevelopment at the study university in an undergraduate nutrition and dietetics course, research skill teaching was enhanced. Based on a review of the evidence, research skills teaching and learning were integrated across all years of the curricula focusing on early yet contextualized research methods learning and hands on experiences (Table 4). This study is focused on one of the subjects implemented in year two of the integrated curriculum: Applied Research Methods in Nutrition. The unit is compulsory for all students enrolled in the Bachelor Nutrition and Dietetics and Bachelor Nutrition Science.
The detailed unit blueprint is illustrated in Figure 1. The course consisted of two key components: (i) students leading a class research project whereby they developed a set of research questions and subsequently an appropriate study design, participated in the study as subjects, collected and collated class data, analyzed data, and prepared a scientific paper; (ii) a group systematic review of the literature. The theoretical underpinnings of the teaching and learning approach were based on Kolb’s experiential learning theory [12]. Through the purposive design, students were invited to partake in the whole research process, with their peers, in a safe and supportive classroom setting, yet solving a realistic priority nutrition issue. Two academics, both early career researchers themselves (Zoe Davidson and Claire Palermo), coordinated the unit in tandem.
The class research project was designed and implemented by the students. Academic staff provided the broad topic area (low FODMAP (fermentable oligosaccharides, disaccharides, monosaccharides, and polyols) diets) from which research questions will be developed. The students chose a randomized, crossover controlled study, with qualitative semistructured interviews to answer their research questions. Teachers guided the students through the research process in a two-hour whole of class group discussion each week. For example, one of the teachers would remind students of content learnt in the previous year, such as developing a research question, and then students would work in small groups to brainstorm elements for their study, feedback to the whole group, and then the teacher would facilitate discussion to achieve consensus on how the whole group wished to proceed to the next step of the research process. Students were briefed in the introductory lecture that this unit required them to show initiative and independence and that without their input in class the study would not be a success. The learning from this experience was summatively assessed via an individual student scientific paper submission, whereby students chose one of the class research questions to write up as a journal article following style guides common of a nutrition and dietetics journal.
Students also completed a systematic review of the literature in groups of five. The systematic review process was completely independent of the class research project, with students completing the review in an area decided with their tutor. The review process was facilitated by department academics in a series of student led tutorials incorporating peer based learning principles [13]. Academics identified a broad research question/topic whith the students and then developed into a more structured question before beginning the process of searching, identifying, extracting, synthesizing, and communicating evidence to answer the question. Student groups participated in a single one-hour tutorial once per week for the 12 week semester with their tutor. Academics and students were provided with a tutorial manual which outlined key tasks to achieve each week in order to ensure completion of the review process within the semester. Students were provided with formative feedback in week four regarding their question and search strategy. The learning from the systematic review was summatively assessed via a group paper submitted in accordance with the PRISMA guidelines on reporting the systematic literature reviews [14].
As a part of the systematic review component of the unit, students participated in a peer assessment process, using the principles of effective peer assessment [15] whereby students were empowered to take responsibility for their performance and learning [16]. In the first tutorial of semester, each small group set criteria to which they would assess their peers on. At the end of semester, students completed an assessment of each member of their group and provided this to their tutor. The collated peer assessment for each student was used to adjust the group’s mark to provide an individual systematic review mark. For example, if a group received 42/50 for their systematic review paper and as student received a collated peer assessment of 8/10, their individual mark was determined as 80% of 42/50 = 34/50.
2.2. Evaluation
All students enrolled in the unit in 2013 () were invited to participate in this research and informed consent was obtained. Ethics approval was granted by the relevant university human research ethics committee (approval number: CF13/703-2013000306).
A pre-post evaluation was used to measure change in self-perceived research knowledge and skills. Prior to commencement of the course, all students completed a modified version of the Research Skills Questionnaire (RSQ) developed by Centre for Excellence in Applied Undergraduate Research Skills at the University of Reading [17]. Students were then asked to complete the same survey on completion of the unit. The survey required students to rate their self-perceived competency in both broader/transferable (11 items) and specific (7 items) research skills on a scale of 1 to 10, with 1 representing the lowest level of competence and 10 the highest. Attitudes toward research were assessed on a scale of 1 to 7. Students were also asked to qualitatively describe their understanding of research and their general feelings towards research such as motivation, interest, involvement, and stimulation and if they would consider a career in research after graduation.
In addition to the questionnaire, students’ satisfaction with the unit was evaluated using the university’s student evaluation questionnaire. This 10-item questionnaire asked students to rate their satisfaction with elements of teaching and learning on a scale of one to five where one corresponds to strongly disagree and five equates strongly agree. The evaluation explores the ability of the teaching and learning to allow students to achieve learning objectives, its intellectual value, and appropriateness of resources, feedback, and effectiveness of staff (Table 3).
Only students who completed both pre- and postsurveys were included in the analysis to assess change in perceived research skills. Wilcoxon Signed Ranks Test was used to assess if there was a difference between self-perceived competence in research skills and attitudes towards research prior to and following the completion of the unit. In order to establish if those who completed the pre- and postsurvey were different from those who completed the presurvey only, we also statistically evaluated if there was a difference in the above variables of interest between these two groups using Mann Whitney- tests. Significance was considered at . Qualitative text responses were analyzed using a content analysis approach [18] by Claire Palermo, guided by qualitative description methodology [19], whereby text was coded and the most common codes developed into the main ideas from the data which was later verified by the other authors (Zoe Davidson). All students who responded to unit evaluation questionnaire were included in the analysis and descriptive statistics reported for each item on the questionnaire.
3. Results
There were 46 respondents to the survey at the commencement of the unit. Twenty-two students responded to the post-RSQ; however, only 17 students completed the survey in its entirety. Forty students responded to the unit evaluation questionnaire. In the 46 initial respondents, the average age of the group was 22 years which included 43 (93%) females, 35 (76%) students of Australian nationality, and seven students (15%) who had completed a degree prior to their current program of study.
There was no significant difference between the students who completed both the pre- and postsurvey () compared to those who completed the presurvey only () with the exception of two areas. Students who completed both surveys had a higher perceived competence in time management (median (IQR): 8 (7, 8) versus 7 (6, 7); 0.013) and from an attitude perspective felt more involved with research (3 (1, 4) versus 4 (3, 5); 0.006).
The analysis revealed that the teaching and learning strategies increased self-perceived competence in all of the specific research skills assessed (Table 1); however, these increases were small often representing an increase of 1 point in the 10-point scale. The unit also resulted in small improvements in self-perceived competence in several broader skills including information gathering and handling, information evaluation, ability to work independently, and critical thinking. There were no reported changes in self-perceived communication, literacy, numeracy, project management, teamwork, or problem-solving skills (Table 1). There were also no changes in attitudes towards research following completion of the unit (Table 2).
Similarly, qualitative analysis revealed that there was no change in any of the general feelings towards research or a career in research after participation in the unit. However, the student’s definition and understanding of the discourse of research changed marginally from a broad understanding of research with a focus on using existing evidence and collection of data to a more specific understanding that included the collection of data by conducting research rather than relying on existing evidence. Prior to commencement of the unit, students described research as a systematic process used to test hypothesis and gather information on a topic for which there is a need to know more about. At the conclusion of the semester, students described research as a process whereby they can discover something new by undertaking a study or using existing evidence. They also saw the value of research in contributing to answering the questions that arise out of practice more after completing the unit:
gathering [existing] evidence in a systematic way in order to address a question. (student number 19, prequestionnaire)
… [research] can either be undertaking new studies or compiling and extracting data from previous studies to answer a new question. (student number 9, postquestionnaire)
The unit was rated highly by students with median scores for all assessed items ranging from 4.3 to 4.8 out of a possible five (Table 3). Of particular note was the students’ satisfaction with how this unit built upon previous units in the course (4.78 out of 5) which fulfills the university criteria of “outstanding” for a unit evaluation result.
An unintended evaluation was the number of groups progressing their systematic literature review to publication in a peer-reviewed journal. At the time of submission, one of the ten groups has had their manuscript accepted for publication; one is under review; and two are in the final stages of drafting their manuscript for of publication.
4. Discussion
This research aimed to evaluate the effect of a year two undergraduate research methods unit on research skills of students. We found that the unit increased self-perceived competence in designing, sampling, recruiting, collecting, analysis, and communicating research. The unit also increased information gathering, handling, and evaluation and the students self-perceived ability to think critically and work independently. Despite the statistical significance of these changes, the effect size was relatively small with self-perceived competence in research skills increasing in most variables by 1 point on the 10-point scale. Of note, however, is the shift in the range of scores, with the minimum increasing across many variables by up to 5 points indicating that those with the lowest perceived competence in research skills may benefit more from this teaching and learning strategy. There is also inherent difficulties with using ordinal scales, in that it is difficult to fully understand the impact of a change in 1 point in perceived confidence. More tangible and “real world” outcomes such as publications arising from the unit are perhaps more concrete indicators of skill development.
The sample demographics, while drawn from one institution, are representative of nutrition and dietetics students across Australia, young and predominately female [20], and therefore the findings may be generalizable to another undergraduate nutrition education. The findings show that this simulated, classroom based, nonresource intensive learning experience had the capacity to develop the research skills of students. It adds to the minimal literature in nutrition on teaching and learning strategies to promote research skill development and may also be of benefit to other health professions with similar student demographics.
The findings are congruent with previous research in nutrition students that has shown the enjoyment and development of research skills through experiencing research itself [21, 22]. The approach presented in this study offers a scaffold from which to create strong foundation for applied research prior to when students are in practice. This has the potential to enhance their real world research experience by preparing them more effectively.
Comparison of RSQ measures to the other literatures demonstrates that medical students self-perceived broader/transferable research skills were similar at baseline to our cohort but their specific self-perceived research skills were lower than the nutrition and dietetics students at baseline [17]. This is perhaps further evidence of the successful scaffolding of research teaching and learning between years one and two of the curriculum. The limitations of self-assessment of capability are acknowledged [23]. In this study, self-assessment of research skills was used to facilitate reflection and learning rather than summative assessment. When used for these purposes, self-perception is an important component of learning [24]. Measuring actual participation in research is needed. Future research may follow this cohort longitudinally and use validated measures of involvement in research [25] to assess the impact of this undergraduate learning experience on their work practice.
The finding that this learning experience did not alter students’ self-perceived skills in communication, teamwork, and project management may relate to the fact that these learning outcomes were not made explicit to students. Other evidences from dietetics students’ experiences of research suggest that it improved project management skills [22]. There is an opportunity to build in formative assessment into Applied Research Methods in Nutrition to highlight the learning that may be occurring around communication, teamwork, and project management as a result of this experience.
This study is limited in that it reports outcomes from only a single small sample of students from the one university. This was due to a poor response rate to the postquestionnaire. This research was conducted during semester, so competing demands of students such as assignments and exams as well as completion of unit evaluations most definitely hindered our response rate. Also the method of administration likely influenced the follow-up response rate. There were 46 respondents to the survey at the commencement of the unit. This presurvey was administered in class time. Only 17 of these respondents submitted completed surveys at the follow-up which was administered outside of class time.
The comparison of self-perceived research skills and attitudes towards research between those students who completed both surveys and those who completed the presurvey only highlighted some important differences between these students. Those who completed both surveys perceived that they were slightly more confident in time management. Considering the competing demands on the students at the time of the postsurvey, these students may have felt they had more time to complete the survey. The other difference between these two groups of students was that those who completed both surveys reported that they felt more involved in research at the pre survey. This could indicate that we have captured those students who were more engaged and who potentially had the best learning outcomes. However, there was no difference in the multiple other research skills or attitudes assessed at the presurvey.
There is an opportunity to follow these students longitudinally to measure the further development of their research skills through practical placement and other learning activities as well as their involvement with research following their degree. This longitudinal evaluation will assist with providing tangible outcomes regarding the development of research skills. In addition, the answers to the qualitative component of the RSQ lacked appropriate depth to sufficiently describe changes in students’ attitudes towards research. Students’ answers may have also been influenced by the timing of the survey (end of semester). Future research should incorporate focus groups to enable in-depth exploration of this topic and also account for discourses among student views. Ultimately, we seek to understand not just how to train evidence-based practitioners but also how to motivate future researchers.
5. Conclusion
Our new, year two level research methods unit appeared successful in enhancing students’ research skills. Further exploration of learner attitudes towards research is needed with consideration of how these might be addressed in future research skills teaching.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
The authors acknowledge the fifty-five Bachelor of Nutrition and Dietetics and Bachelor of Nutrition Science second students of 2013 who had faith in them and immersed themselves in this new unit. The authors would also like to acknowledge the support of the tutors, Lisa Ryan, Judi Porter, and Kate Huggins whose enthusiasm and commitment to immersing undergraduate students in a systematic review process did not go unnoticed.