Abstract
Purpose. While some aspects of what makes for an effective portfolio program are known, little is published about what students value in the faculty-student-portfolio relationship. Lack of student buy-in and faculty engagement can be significant challenges. The purpose of this study was to identify behaviors and types of engagement that students value in their relationships with portfolio-associated faculty. Methods. Medical students (174) participating in the Ohio State University College of Medicine Portfolio Program described behaviors observed in their portfolio-associated faculty in a survey completed at the end of the first year of their four-year program. Narrative responses were coded and categorized into themes, followed by member checking. Results. A total of 324 comments from 169 students were analyzed. Four themes were identified: (1) creating a supportive environment; (2) inspiring academic and professional growth; (3) investing time in students; and (4) providing advice and direction. Conclusions. The themes identified suggest that students value certain types of coaching and mentoring behaviors from their portfolio-associated faculty. The themes and their specific subcategories may be useful in making decisions regarding program development and guiding recruitment and training of these faculty coaches.
1. Introduction
The educational portfolio is a tool which is often used for developing reflection and self-directed learning skills in the health professions. Many medical schools have developed portfolio programs for their students and have provided some form of faculty support intended to maximize the portfolio’s benefits. Little is known about the relative importance of the particular behaviors and types of engagement demonstrated by faculty in guiding a student’s portfolio work.
Portfolios are often used to foster reflective practice. Indeed, evidence indicates that portfolios can improve students’ self-awareness and reflection and can provide a means of support for emotional situations [1]. Portfolios are also utilized to promote self-directed and deep learning, defined as learning that impacts behavior, intuition, and performance [2]. In the context of competency-based educational paradigms, portfolios have provided a means of incorporating the requisite large variety of assessment tools and promoting practice-based learning principles [3]. However, for portfolios to be effective in supporting and assessing competency development, the program must have robust integration into the curriculum and faculty support [4].
There has been wide variation in practices related to the use of such portfolios [5]. In a review of 141 US medical schools, 72% of respondents stated that their portfolios allowed for assessment of competencies, while 28% stated that their portfolios did not assess competencies, 45% reported portfolio use for formative evaluation only, 48% for both formative and summative evaluation, and 3% for summative evaluation alone [6]. In another review of thirty portfolio programs across undergraduate and postgraduate medical education in The Netherlands, providing feedback and promoting reflection were identified as the primary educational aims. 87% of the programs used one-on-one mentoring and 53% employed small group sessions to support portfolio use [7]. Based on interviews with portfolio mentors, four factors have been identified as important to program design: (1) portfolio structure, (2) an appropriate assessment structure, (3) provision of new experiences and materials, and (4) sufficient teacher capacity for “adequate coaching and assessment” [8]. Little has been published in regard to the last and arguably the most important factor, from the perspective of the student.
The purpose of this study was to explore what students perceive as important in optimizing these faculty-student relationships. We were also curious as to whether faculty would be viewed and valued more as coaches, as mentors, or as serving some other role. From a practical perspective, we hoped to identify ways to improve student buy-in and to effectively engage faculty in a portfolio program.
2. Methods
Ohio State’s new competency-based Lead.Serve.Inspire (LSI) medical school curriculum includes a portfolio program designed to help medical students develop reflective, self-directed learning skills. The program consists of a personal, secure, online portfolio for each student structured around the framework of competency domains defined by the core educational objectives of our MD curriculum. An additional portfolio section is used for career explorations. Students are given reflective assignments which provide them with opportunities to learn from and respond to new experiences and performance data (Appendix A). Performance reports are posted in student portfolios within 24 hours after completion of curricular units and include data from a variety of assessment types tailored to capture the student’s performance in each competency (Appendix B).
Portfolio-associated faculty or portfolio coaches serve as learning guides, working with students in a one-to-one setting throughout the student’s medical training. A student’s initial meeting with his or her coach occurs during the first week of medical school. Students meet with their coach approximately every 8 weeks for 30 minutes. Timing of meetings is aligned with curricular units. These meetings are one-on-one and in-person during the first year of the curriculum. Coaches are expected to accomplish two primary tasks: (1) review of performance metrics (e.g., performance on tests, quizzes, and clinical evaluations) and (2) promotion of reflection. The former is accomplished face-to-face with a discussion of performance and a focus on establishing new personal/professional goals and plans. Coaches are also expected to help direct students to appropriate resources that might help them achieve their goals. In addition, portfolio coaches read and comment on the students’ reflective writings, providing affirmation or encouragement, further questions to deepen the student’s reflection on the topic, or additional resources from which the student might benefit.
Faculty members apply for the coaching positions. Applicants are interviewed by the portfolio program director and are selected based on level of experience in mentoring or coaching and demonstration of self-reflection and communication skills during the interview. Efforts are made to recruit a diverse pool of faculty coaches based on specialty, gender, and ethnicity. Our cohort of 25 coaches included faculty from eight clinical departments with the majority coming from Internal Medicine or Pediatrics. Sixty percent were female, 40% male, and 8% underrepresented minorities. Once selected, coaches undergo a formal training session that includes learning how to provide feedback on student reflective writing, interpreting performance assessments, and a simulated coaching exercise utilizing standardized students. The cohort of faculty coaches who work with students within the same student class participate together in ongoing coach development approximately every 8 weeks throughout the academic year. These coach development meetings take place just before their scheduled meetings with students and typically last 90 minutes. Topics include interpretation of student performance reports, understanding curricular and extracurricular activities and resources within the college, methods of effective coaching, and general collegial exchange and support.
This study was conducted within the context of our annual portfolio and coaching program evaluation. As a whole, student evaluation of the program indicated that students were generally satisfied with the portfolio program. In response to “my work with the portfolio and my coach enhanced my personal and professional development” 35 (20%) strongly agreed, 77 (44%) agreed, 42 (24%) neither agreed or disagreed, 15 (8%) disagreed, and 6 (3%) strongly disagreed. When asked to rate each component of the program on a scale of 1 to 6 with 1 designated as not important and 6 as very important, students rated the individual face-to-face meeting with one’s coach as the most important component (mean 4.72), followed by access to coach outside scheduled times (4.43), coach comments in portfolio (3.38), and reflective assignments (2.73).
We captured information about the student’s experience with a coach through narrative responses to an electronic survey collected from all students at the end of their first year of medical school. The narratives were intended to gather information about the behaviors and types of coach-student engagement that students found helpful, as well as those that could lead to improvement in the coach-student relationship. These narrative comments were submitted in response to two open-ended questions:(1)Describe one or two things that your coach has done in your coaching relationship that you have found to be particularly helpful to you. What would you like for him/her to continue to do?(2)Describe one or two things that your coach might do to improve your coaching relationship.
Two researchers (Jack Kopechek and Sorabh Khandelwal) performed a content analysis of the comments from 169 first-year medical students. A comment consisted of any phrase, sentence, or group of sentences that expressed a single thought from the student. Some students shared several comments while others had only one. Responses were coded, using an iterative method of constant comparative analysis associated with grounded theory. On secondary analysis, codes were further categorized into themes [9].
To confirm our findings, member checking was performed by Sorabh Khandelwal and Jack Kopechek using a focus group of 10 students who had completed the original portfolio program evaluations. This was a convenience sample that included several elected representatives of the student council.
This study received approval from the Human Behavioral and Social Sciences Institutional Review Board at the Ohio State University.
3. Results
We received comments from 169 of the 174 students (97%). A total of 324 comments were analyzed from these 169 students. The four primary themes identified, and their associated categories/codes, are represented in Table 1.
3.1. Creating a Supportive Environment
The largest portion of comments was centered on the portfolio coach creating a supportive environment, to promote positive emotion. Of the four categories under this theme, many comments involved the creation of safety within the relationship. Students appreciated a relationship built on honesty, understanding, and trust, which allowed for open dialogue.
He has done a fantastic job of creating a mentoring relationship in which I feel comfortable and willing to speak freely and completely. He is supportive without being coddling or appearing insincere.
Another category under this theme centered on the portfolio coach offering emotional encouragement.
I also like that she is encouraging and positive and affirms that I am doing well.
Other comments centered on certain portfolio coach behaviors such as listening, being nice, being energetic, and being friendly.
She is just the best! She does a great job really listening to what I have to say. She gets where I’m coming from.
3.2. Inspiring Academic and Professional Growth
Approximately a quarter of the comments were centered on the portfolio coach inspiring academic and professional growth. Narratives within this theme focused on helping students develop the skills associated with self-reflection and self-directed learning. Students appreciated faculty helping them develop reflective skills.
Does more asking than telling- he asks me very good, thought-stimulating questions that often help me realize something about myself I hadn’t before.
Students also appreciated some guidance with setting goals, both personal and professional.
She has encouraged me to think about concrete action plans and goals to set for the next time we meet.
Students clearly valued feedback on their academic work.
Dr. xxx does a good job of exploring strong and weak points and giving good feedback and advice on how to improve going forward from our meetings.
In fact, there were many comments from students wanting more feedback from their portfolio-associated faculty member. Of the 18 comments on feedback, about half of them dealt with wanting more feedback. One student remarked, “She could give me a little more specific feedback on what my weaknesses are academically.”
3.3. Investing Additional Time in Students
About one-fifth of the comments were centered on investment of time. There were three categories under this theme. Students wanted coaches to take the time to thoroughly review their academic performance.
I would like to go into more detail about the OSCE performances. Possibly watch an OSCE video, and identify specific items to improve for next patient interactions.
Students appreciated the accessibility exhibited by many coaches.
He also maintains an atmosphere of availability, making it clear that he is always a phone call away if we need to talk.
Students also seemed to value the effort placed by coaches to be prepared for the meetings.
She always promptly responds to my reflections and keeps up-to-date on my performance in school. This makes our meetings more productive.
3.4. Providing Direction and Advice
The remaining comments were centered on the portfolio coach acting in an advisory role or directing students to helpful resources. Many of the comments were focused on the portfolio coach providing referrals to other health care professionals to help with professional development.
My coach has offered to connect me with physicians who can help me explore my interests.
She is also the one who directed me to the academic counselor when I was having trouble adapting to the pace of medical school.
Other comments were focused on obtaining advice for future success in medical school.
He has been good at giving me professional advice and offering insight into studying in medical school.
The focus group confirmed the themes mentioned above.
4. Discussion
One of the driving forces behind Ohio State University College of Medicine’s curriculum redesign was the goal to produce physicians who can reflect on their experiences as a means of directing their own lifelong learning. An educational portfolio program was established within the LSI curriculum to promote students’ development of these critical skills with the support and guidance of a faculty coach. We investigated the student perspective on what was of value in this faculty-student-portfolio relationship, as well as aspects that could be improved.
Our data suggest that, from a student’s perspective, the most important skill for portfolio-associated faculty is the ability to create a supportive environment. Students valued coaches who were friendly, receptive, and skilled listeners. The other themes were “inspiring academic and professional growth,” “investing time in students,” and “providing advice and direction.” Interestingly, only one-fourth of attributes valued by students were related to self-reflection and self-directed learning, despite the fact that these skills were explicitly stated as aims of the program. This observation is consistent with the low student rating of the importance of reflective assignments in comparison to other portfolio program components.
From these observations, we tried to discern whether students were looking for a coach or a mentor in their relationship with portfolio-associated faculty. Coaching and mentoring share a common foundation in their use of encouragement and aim to promote success. Mentoring often focuses on career counseling, development of one’s professional identity, advancement in research and academic careers, and the support of personal growth [10]. Positive mentor behaviors include sincerity, encouragement, empowerment, the building of professional networks, and assisting in personal development [10, 11]. In comparison, coaching emphasizes monitoring and improvement of performance, increasing motivation, and provision of opportunities for practice in areas of weakness or difficulty [12]. Framed by these definitions, our results indicated that students placed a high value on behaviors that are shared in common by coaches and mentors. Students valued portfolio-associated faculty who were effective coaches, supporting performance improvement, as well as skillful mentors, assisting with career advancement. Comments related to career advising and making connections with others are examples of descriptions that speak to more of a mentor-mentee relationship.
While many published reports of portfolio programs have not included descriptions of the student-faculty relationships, where these are reported, some of the same or closely related themes have been identified. One study explored the perspectives of portfolio mentors. One of the four conditions for program success that was identified was “coaching,” defined more specifically as the aid provided to learners to help them identify their learning needs and develop plans to address those needs, as well as supporting motivation to learn [8]. Another study that explored the relationship of trainer to general practice registrant found that creation of a supportive environment was so critical to the success of the program that tension in this relationship resulted in abandonment of the portfolio as a learning tool [2]. Our portfolio program offers further evidence that faculty associated with portfolios serves key functions related to creation of a supportive learning environment, as well as more specific coaching functions related to inspiring professional and academic growth and providing advice and direction.
These data have implications for faculty recruitment and training for programs that use portfolios associated with a longitudinal relationship with a faculty member. Both coaching and mentoring skills are needed to support the relationship and distinctions between those roles are less important [13–21]. Indeed, the effective portfolio coach/mentor, from the perspective of the student, appears to be a unique hybrid of these two roles (Figure 1). Specific coaching skills that we discovered included offering advice on tactics students can use to help them succeed, assisting students in identifying opportunities to practice skills, and, finally, connecting students with appropriate supplemental resources. Mentoring skills are also needed related to provision of career advice and making professional connections.
Interestingly while improved self-awareness and reflection is an acknowledged benefit of portfolios, students were less apt to describe behaviors related to supporting these aspects of the program. Further exploration is needed to understand how faculty might reinforce the importance of self-reflection in the hope that students will value this skill as they progress in medical school. We do not yet know how to best measure self-awareness and reflective ability as direct outcomes of the use of portfolios or the impact of the student-faculty relationship on these learning outcomes.
5. Limitations
Our study is limited, in that it captures the student’s perspective at a single point in time early in one medical school’s curriculum. Narratives were obtained from the first class of students in a new medical school curriculum. Student anxiety associated with a new curriculum, the timing of the meetings (after examinations), and the evaluation itself (end of the first year) may have increased the students’ perceived value of support. It is possible that students will change their expectations as they progress in medical school. Students may value different aspects of the relationship at different times in the curriculum. Their needs or insights may evolve. For example, students may seek less emotional support and more career advice as they begin residency applications and subsequently enter the match process. Reflection, goal-setting and requesting feedback may be valued more highly later in training. In addition, data from this study were obtained from students who attended a US medical school located in the Midwest and may not be generalizable to medical schools in other regions or countries.
While the study was intended to address behaviors and types of faculty-student engagement that were valued by students, the program design may have influenced student responses. Our program used the term coach and had competency performance and career components as part of its structure. Furthermore, the faculty conducting the analysis used a descriptive coding approach directed toward behaviors that might typify a coach, mentor, or advisor.
6. Conclusion
First-year medical students valued their portfolio-associated faculty for behaviors and types of engagement associated with both mentorship and coaching. The behaviors and types of engagement discussed in this article may help inform faculty recruitment and development efforts in similar portfolio programs. Competencies for portfolio-associated faculty include the ability to create a safe environment within the relationship, to provide feedback that stimulates reflection resulting in academic and professional growth, to demonstrate a willingness to invest time in students, and to share knowledge of available resources that can help students improve performance and develop careers. Ongoing research on students’ perspectives throughout medical school will be necessary to fully understand their developmental needs and how to best equip faculty coach/mentors to assist students with these needs.
Appendix
A. Example of Reflective Prompt
A.1. Strengths and Weaknesses of Reflection
After reviewing the core educational objectives and the six associated competencies for the OSU College of Medicine, write a reflective essay that includes the following: (i)Descriptions of your personal strengths and examples of how these are supporting your progress toward meeting one or two of the competencies (ii)Descriptions of your personal areas for improvement in regard to one or two of the competencies (iii)Insights into what it will take to make progress in areas where improvement is needed (iv)Goals and a specific plan for strengthening your areas for improvement during the next six weeks
B. Example of Block Performance Report
See Table 2.
Competing Interests
The authors report no competing interests.