Research Article | Open Access
Assessing Origins of Quality Gaps in Discharge Summaries: A Survey of Resident Physician Attitudes
Because little is known about attitudes of primary authors of discharge summaries in academic institutions, namely, trainees and physician assistants (PAs), we sought to explore values, possible areas for improvement, and interest in formal discharge summary education. A survey composed of Likert scale analyses, dichotomous relationships, and open-ended questions was designed using focus groups and validated via expert committee review. Of 135 total residents (PGY 1–3), 79 residents and 10 PAs in a large academic hospital in New York City completed it. Of surveyed trainees, 77.2% reported that quality discharge summaries are useful in primary care. Interns had less outpatient experience with discharge summaries compared to PGY 2 and PGY 3 (23.7% versus 63.4%, ) and were less comfortable authoring discharge summaries for patients who were not as familiar to them (47.4% versus 24.4%, ). The majority (54.8%) of interns as well as all PAs reported never receiving feedback on discharge summaries. Finally, 63.2% of interns and 90% of PAs responded that formal teaching would be helpful. Interns’ greater discomfort may speak to their poor understanding of core components of a useful discharge summary, which teaching sessions may improve. Alternatively, shifting the authorship responsibility from interns to seniors could be explored as a quality improvement initiative.
Effective handoffs during transitions of care are essential to preserve patient safety in the era of hospitalist-based medicine. Discharge summaries are a prime example. Research has identified several flaws with current discharge summaries; outpatient practitioners are dissatisfied with the quality , and medication reconciliation errors may lead to readmission . Although there have been many efforts to improve authorship of discharge summaries [3–10], little is known about the attitudes of the primary authors of these documents at academic institutions, namely, interns, residents, and physician assistants (PAs).
Direct resident interviews have revealed factors that interfere with producing quality discharge summaries , yet these, lacking anonymity, are subject to bias. It has been the authors’ observation that senior house staff tend to value discharge summaries more than interns do. A hypothesis of the present study is that house staff closer to the beginning of their training may devalue discharge summaries because they lack sufficient outpatient experience to appreciate their worth from a transitional care perspective.
With advancement in electronic health record technology, automated tools are increasingly utilized in discharge summaries. Studies have demonstrated that autopopulating certain fields decreases accidental omission rates and increases primary care physicians’ ratings of quality [12, 13]. However, there is also data to suggest that there were more errors in medication lists that were automatically populated rather than manually populated, highlighting the fact that despite their ease of use, these tools need to be further fine-tuned and always used with careful oversight of the discharge summary’s author .
This study had several objectives. The first was to capture the attitudes of house staff to determine if experience affects perception of discharge summary usefulness. An additional objective was to isolate sections of the discharge summary that house staff felt were more or less important, including many of the automated sections, some of which included hospital quality-core measurements. A final objective of this study was to measure house staff exposure to prior teaching and feedback on discharge summary authorship and their willingness to have such formal discharge summary education. This information was solicited to gauge baseline stakeholder buy-in and guide plans for potential future quality improvement initiatives.
A survey was developed using a resident focus group composed of PGY 3 in the spring of their final year of training, ensuring they would not be surveyed the following fall. Open-ended questions stimulated discussion of how their perceptions of the discharge summary had changed throughout their training and how they perceived their interns’ opinions of the discharge summary. The survey was then created targeting themes identified in the discussion and internally validated via review by an expert committee.
Using Likert scale analysis, dichotomous relationships, and open-ended questions, the survey sought to measure the attitudes of internal medicine residents and PAs (on nonteaching services) toward authoring discharge summaries. Surveys were voluntarily completed and were anonymous to avoid potential bias. Paper surveys were administered to residents (PGY 1–3) and PAs at a large academic medical center in New York City during an outpatient morning conference with PGY 1–3 and during morning sign-out, respectively. Interns, residents, and PAs present were told only that it was an anonymous survey studying discharge summaries and that completion was voluntary. The PGY 1–3 had a protected time of 15 minutes to complete the survey, if they chose to do so, during the morning conference. The PAs completed the survey before or after they gave or received morning sign-out. Both groups of respondents placed the completed surveys in designated pile and after all were completed they were collected by one of the study authors. Because the total number of interns, residents, and PAs was known to be small, a random sampling was not undertaken given a random sample would likely not have represented the viewpoints of the entire group sufficiently. The survey was conducted roughly 3 months into the academic year over a 6-week period.
Results were calculated and stratified by level of training. Comparisons of interns (PGY 1) versus residents (PGY 2 and PGY 3) were also conducted for relevant questions and assessed using Fisher’s exact test to assess the hypothesis that greater overall experience and outpatient experience with discharge summaries would correlate with higher reported value of discharge summaries. Because of small numbers of trainees in each group and because most discharge summary writing experience was developed in the intern year at our institution, PGY 2 and 3 were consolidated for analysis as one group and compared against the least experienced group, the interns. Outcomes included Likert scale ratings of house staff opinion on the utility of discharge summaries for outpatient providers, their effort spent on discharge summaries as compared to admission notes, their comfort level in authoring discharge summaries on cross-coverage patients (i.e., on the weekend if there was an unanticipated discharge or if the primary team member had not completed the discharge summary prior to rotating off service), and their opinion regarding utility of formal training. Additional demographic data were collected for correlation study including frequency of completion of discharge summaries, frequency of use in the outpatient setting, and exposure to formal training on writing discharge summaries. All data were self-reported.
Study data were collected and managed using REDCap electronic data capture tools hosted at Weill Cornell Medical College. REDCap (Research Electronic Data Capture) is a secure, web-based application designed to support data capture for research studies, providing (1) an intuitive interface for validated data entry; (2) audit trails for tracking data manipulation and export procedures; (3) automated export procedures for seamless data downloads to common statistical packages; and (4) procedures for importing data from external sources. Its use was supported in part by funds from the Clinical Translational Science Center (CTSC), New York, NY, USA, and National Center for Advancing Translational Sciences (NCATS; Rockville, MD, USA) Grant no. UL1-RR024996 . This study was approved by the Weill Cornell Medical College-New York Presbyterian Institutional Review Board.
Once collected in REDCap, data were analyzed using SAS software . Given the small sample size and the use of categorical data for classification of responses, Fisher’s exact test was conducted to measure associations between groups. Comparisons of interns (PGY 1) versus residents (PGY 2 and PGY 3) and of MDs (PGY 1, PGY 2, and PGY 3) versus PA were also conducted for relevant questions and assessed using Fisher’s exact test. Two tailed -tests with statistical significance set at of <0.05 were used for testing the null hypothesis such that no difference exists by training level. Data followed the exact distribution. Additional correlations between Likert scales (ordinal variables) were calculated using Spearman rho correlation coefficients with statistical significance set at of <0.05.
There were very few open-ended responses and so these were not formally analyzed.
3.1. Study Participants and Experience
There were 89 participants in the study, including 79 of 135 total internal medicine residents (58.5%). Of the 89 participants, 38 (42.7%), 21 (23.6%), 20 (22.5%), and 10 (11.2%) were PGY 1, PGY 2, PGY 3, and PA, respectively. Because only 10 PAs participated, their responses were used for qualitative purposes only.
Interns and residents differed with respect to clinical experience in both inpatient and outpatient settings. Compared to interns, 76.3% of whom reported working in the inpatient setting for 3 months or less, 70.1% of all residents reported working in the inpatient setting for more than 12 months. Also, 92.1% of interns reported fewer than 6 weeks of outpatient medicine experience, while the majority of residents reported outpatient experience as greater than 12 weeks (Table 1).
Use of discharge summaries in the outpatient setting differed significantly by training level. More than 3/4 of interns had used a discharge summary in the outpatient setting 5 times or fewer (26.3% never had used one) compared to just 36.6% of residents (19.5% had never used one) . Similarly, 61.0% of residents reported using discharge summaries in the outpatient setting more than 10 times compared to only 7.9% of interns . Use of discharge summaries in the inpatient setting, such as obtaining prior hospitalization history when a patient is readmitted, also differed significantly by training level (, Table 2).
Although house staff assume that the trainee who had primary responsibility for the hospitalized patient will author the discharge summary, this is not always the case. Residents said more frequently than interns that they had the experience of authoring the hospital course in a discharge summary when they were not the team member most involved in the care of the patient . While 60.5% of interns reported that they had ever been the principal author of the hospital course section without being the primary team member involved most closely in the case, almost all residents (95.1%) reported doing this. When asked about comfort level, nearly half (47.4%) of interns report being “a little” to “not at all” comfortable with authoring the majority of the hospital course when they were not the primary house officer. Residents less frequently report these feelings, with less than a quarter feeling this level of discomfort. Differences in terms of comfort level by training level reached statistical significance when broad categories were used ( (Table 3)).
3.2. Implicit and Explicit Attitudes
Although 75% of all house officers reported that a quality discharge summary is “very” or “extremely” useful to inpatient and outpatient practitioners as a historical document, the majority of house staff (62.0%) reported that they put less effort into authoring the discharge summary than to authoring an admission note; 34.0% devoted “equal effort” to the discharge summary compared to the admission note. Six residents reported spending more effort authoring the discharge summary than the admission note, while no intern reported doing such. Effort did not differ significantly by training level overall (Table 3, Figure 1).
3.3. Sections of Discharge Summaries
Fully 88.6% of the house staff reported that the hospital course portion is a “very important” component of the discharge summary. When asked about the effort placed on this portion of the discharge summary, 76.3% of interns reported that they put in “a great deal of effort” and 21.1% of interns put in “much” effort. Similarly, 53.7% of residents put in “a great deal of effort” and 39.0% of residents put in “much effort” when writing the hospital course of the discharge summary. There was no association between the explicit value that house staff place on the discharge summary as a transfer of care document and the effort reportedly spent authoring them (Spearman rho = 0.1, , ).
Core measures (e.g., assessing ejection fraction for congestive heart failure, discharge with beta blocker for acute myocardial infarction) are mandatory fields within the discharge summaries. House staff acknowledged possibly completing these measures inaccurately in order to save time: occasionally (, 19.1%), rarely (, 29.2%), and very rarely (, 23.6%). The following are some examples of answers to open-ended questions that complement the quantitative data: for example,
If you could re-write the template for the discharge summary how would you change the content involved and why?
“Put less emphasis on performance measures and diet/activity and more on hospital course, medication changes, upcoming appointments, and follow up issues.”
“Take out core measures, not helpful.”
When asked about satisfaction with discharge summary quality, 55.0% of interns and 73.1% of residents reported feeling “often” or “always” satisfied with the discharge summaries that they authored. When asked about satisfaction with the quality of the discharge summaries used authored by others, 39.5% of interns and 24.4% of residents responded that they were “often” or “always” satisfied. These differences were not statistically significant by training level, however ( and , resp.).
3.5. Education and Teaching of Discharge Summaries
Formal education on discharge summary authoring did not differ by training level . More than one-third reported that they had never received education, and approximately half reported receiving education one or two times. Interns and residents did not differ with respect to receiving feedback on their discharge summaries . When asked about formal feedback from attending or residents on discharge summaries they had authored, 44.7% of interns and 34.1% of residents reported never receiving feedback. The majority (53.2%) of house staff would welcome a teaching or training session on discharge summaries; 32.9% report they are not sure if they would want a training session and 13.9% reported that they would not (Table 4).
As expected, upper level house staff had more experience utilizing discharge summaries in both the inpatient and outpatient settings. This extra experience did not correlate significantly with how they valued the discharge summary, however. It is possible that the small sample size, single site, or low utilization limited the ability to detect significant differences. It is also possible that experience with discharge summaries in an outpatient follow-up setting does not influence house officers’ perception of their value.
Upper level house staff have more experience authoring discharge summaries on patients who are unfamiliar to them, and they are more comfortable in doing this. Composing a hospital summary for the unfamiliar patient parallels the observation that patients experience an increase in preventable adverse events when there is lack of continuity of care, a known risk in cross-coverage medicine and on the discharge process . However, just as a solid fund of medical knowledge and improvements in sign-out help guide treatment decisions when cross-covering (e.g., the work-up of angina at midnight), composing a quality discharge summary about an unfamiliar patient also requires the author to have a solid understanding of what is important to report and the means of obtaining the data. Rather than reading every single progress note to determine the medical story, an experienced house officer utilizes efficient chart sleuthing (e.g., to find the type and duration of antibiotics provided and whether or not hypoxia was corrected by discharge for a patient admitted with bacterial pneumonia). Thus, this question also may have been assessing a house officer’s comfort in knowing what is needed for completing quality discharge summaries, knowledge that interns may lack.
These differences highlight a safety risk: those who are least comfortable, the interns, are the ones charged with authoring the document. While this is not uncommon in medical training, the lack of supervision and feedback is. Future quality efforts could focus on teaching the “essential components” of a discharge summary based on a given diagnosis and utilize residents who are more comfortable to both teach and supervise intern authorship. Studies have demonstrated that focused resident curricula in this area have improved the quality of discharge summaries [3, 5, 6, 8], and those skills persisted for years beyond the training . In addition, with both interns and residents, confidence levels rose after further discharge summary education . A future study could examine if attitudes and confidence improve together after education, suggesting a link between low confidence and poor attitudes.
This study also identifies a discrepancy between explicit attitudes and actual practice that very likely impacts quality. Although more than 75% of house staff reported that the discharge summary was important for transfer of care, they admitted that they put less effort into it than an admission note. Assessing true attitudes, especially if they are incongruous with expectations placed on trainees, can be difficult. Response bias and the Hawthorne effect likely exist here. Additionally, future studies may uncover barriers that prevent the authors’ professed values from reaching fruition. However, by identifying a true and meaningful difference in effort by using the admission note as a gold standard, this study points out that reforming discharge summary teaching may require mirroring admission note teaching such as including formal presentations.
We were concerned that even though they do not take much time to complete, core quality measures may be distracting or annoying when included in discharge summaries, leading to lower implicit attitudes of discharge summaries and the allocation of less effort. While interns and residents did not report that these areas are “distracted” from the other sections of the discharge summary, they did admit to answering these questions incorrectly in order to save time, suggesting that their reported value is inflated as compared to their actual value. To the best of our knowledge, this is the first study of its kind to capture confession of falsification of records of quality measures demonstrating, likely, that these records lack accuracy. In addition, while there were too few open-ended comments left to study quantitatively, many responders focused on the awkwardness of including the core measure data within the discharge summary and suggested it be removed. Future studies may seek to determine if separate documentation of these measures distinct from the discharge summary improves their accuracy as well as improving attitudes and quality of the discharge summary.
Our study highlights discordance between the house staff perceptions of their own discharge summaries versus the quality of those written by their colleagues. Our house officers often reported satisfaction with their own discharge summaries yet dissatisfaction with the discharge summaries that they read. This discrepancy suggests that authors may be inflating the quality of their own summaries. The perceived lack of quality in others’ discharge summaries may provide stakeholder (i.e., the house officers) “buy-in” for quality improvement projects. House staff did demonstrate interest in further teaching on how to complete quality discharge summaries.
There were limitations to this study in addition to those cited previously. One large limitation was that house staff may report inflated values of discharge summaries, feeling they are expected to recognize the discharge summary as an important document, even though their genuine attitudes may differ. Another limitation in this attitude survey was the assumption that a discharge summary should take as much effort as an admission note, when these two documents, while similar, may not be equivalent. Another potential bias is that the study was completely self-reported, which for reasons including recall bias is inherently flawed. A future study could measure actual documented time spent in various rotations and actual authorship of discharge summaries and correlate these data with resident attitudes. Another limitation is that MDs were on an outpatient rotation when they may have been more primed to report highly regarding discharge summaries as they need to utilize these documents in follow-up visits. Efforts to survey the residents in the early part of their outpatient rotation were made to avoid this bias. In addition, the MDs surveyed were those who were present for morning teaching conferences. Those residents who were not captured were those who did not attend the outpatient educational conferences, and their opinions may be quite different from those residents who were in attendance.
This is the first study of its kind to measure house staff attitudes toward discharge summaries. This study highlights how attitudes can be incongruous with action and demonstrated key differences between the least experienced house officers and the more experienced. These areas all demonstrate avenues for future quality improvement that can be targeted more successfully toward the stakeholders, that is, the house staff, to create more effective and meaningful improvements in these critical transfer of care documents.
Conflict of Interests
Drs. Mallory Otto, Madeline Sterling, and Brian Eiss have no potential conflict of interests to report. Dr. Eugenia Siegler has received royalties from Springer Publishing Company.
There were no contributors to the paper other than the authors. While there was no direct funding for the study, use of REDCap was provided via funds from the Clinical Translational Science Center (CTSC), New York, NY, USA, and National Center for Advancing Translational Sciences (NCATS; Rockville, MD, USA) Grant no. UL1-RR024996 . Findings have been previously presented at the Mid-Atlantic Regional Society of General Internal Medicine (SGIM) 2014 conference as well as the SGIM 2014 National Conference.
- L. Kalanithi, C. E. Coffey, M. Mourad, A. R. Vidyarthi, H. Hollander, and S. R. Ranji, “The effect of a resident-led quality improvement project on improving communication between hospital-based and outpatient physicians,” American Journal of Medical Quality, vol. 28, no. 6, pp. 472–479, 2013.
- K. Legault, J. Ostro, Z. Khalid, P. Wasi, and J. J. You, “Quality of discharge summaries prepared by first year internal medicine residents,” BMC Medical Education, vol. 12, article 77, 2012.
- J. S. Talwalkar, J. R. Ouellette, S. Alston et al., “A structured workshop to improve the quality of resident discharge summaries,” Journal of Graduate Medical Education, vol. 4, no. 1, pp. 87–91, 2012.
- K. M. Finn, R. Heffner, Y. Chang et al., “Improving the discharge process by embedding a discharge facilitator in a resident team,” Journal of Hospital Medicine, vol. 6, no. 9, pp. 494–500, 2011.
- A. Dinescu, H. Fernandez, J. S. Ross, and R. Karani, “Audit and feedback: an intervention to improve discharge summary completion,” Journal of Hospital Medicine, vol. 6, no. 1, pp. 28–32, 2011.
- M. Key-Solle, E. Paulk, K. Bradford, A. C. Skinner, M. C. Lewis, and K. Shomaker, “Improving the quality of discharge communication with an educational intervention,” Pediatrics, vol. 126, no. 4, pp. 734–739, 2010.
- M. H. Coit, J. T. Katz, and G. T. McMahon, “The effect of workload reduction on the quality of residents' discharge summaries,” Journal of General Internal Medicine, vol. 26, no. 1, pp. 28–32, 2011.
- J. S. Myers, C. K. Jaipaul, J. R. Kogan, S. Krekun, L. M. Bellini, and J. A. Shea, “Are discharge summaries teachable? The effects of a discharge summary curriculum on the quality of discharge summaries in an internal medicine residency program,” Academic Medicine, vol. 81, no. 10, pp. S5–S8, 2006.
- D. S. Weisman, L. Bashir, A. Mehta et al., “A medical resident post-discharge phone call study,” Hospital Practice, vol. 40, no. 2, pp. 138–146, 2012.
- R. B. Balaban, J. S. Weissman, P. A. Samuel, and S. Woolhandler, “Redefining and redesigning hospital discharge to enhance patient care: a randomized controlled study,” Journal of General Internal Medicine, vol. 23, no. 8, pp. 1228–1233, 2008.
- S. R. Greysen, D. Schiliro, L. I. Horwitz, L. Curry, and E. H. Bradley, “‘Out of sight, out of mind’: housestaff perceptions of quality-limiting factors in discharge care at teaching hospitals,” Journal of Hospital Medicine, vol. 7, no. 5, pp. 376–381, 2012.
- C. van Walraven, S. M. Duke, A. L. Weinberg, and P. S. Wells, “Standardized or narrative discharge summaries. Which do family physicians prefer?” Canadian Family Physician, vol. 44, pp. 62–69, 1998.
- C. van Walraven, A. Laupacis, R. Seth, and G. Wells, “Dictated versus database-generated discharge summaries: a randomized clinical trial,” CMAJ, vol. 160, no. 3, pp. 319–326, 1999.
- J. Callen, J. McIntosh, and J. Li, “Accuracy of medication documentation in hospital discharge summaries: a retrospective analysis of medication transcription errors in manual and electronic discharge summaries,” International Journal of Medical Informatics, vol. 79, no. 1, pp. 58–64, 2010.
- P. A. Harris, R. Taylor, R. Thielke, J. Payne, N. Gonzalez, and J. G. Conde, “Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support,” Journal of Biomedical Informatics, vol. 42, no. 2, pp. 377–381, 2009.
- SAS Institute, Base SAS 9.3 Procedures Guide, SAS Institute, Cary, NC, USA, 2011.
- L. A. Petersen, T. A. Brennan, A. C. O'Neil, E. F. Cook, and T. H. Lee, “Does housestaff discontinuity of care increase the risk for preventable adverse events?” Annals of Internal Medicine, vol. 121, no. 11, pp. 866–872, 1994.
- R. N. Axon, F. T. Penney, T. R. Kyle et al., “A hospital discharge summary quality improvement program featuring individual and team-based feedback and academic detailing,” The American Journal of the Medical Sciences, vol. 347, no. 6, pp. 472–477, 2014.
Copyright © 2015 Mallory Otto et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.