- About this Journal ·
- Abstracting and Indexing ·
- Aims and Scope ·
- Annual Issues ·
- Article Processing Charges ·
- Author Guidelines ·
- Bibliographic Information ·
- Citations to this Journal ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Recently Accepted Articles ·
- Reviewers Acknowledgment ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents
Gastroenterology Research and Practice
Volume 2013 (2013), Article ID 730829, 6 pages
Negative Pressure Wound Therapy for the Treatment of the Open Abdomen and Incidence of Enteral Fistulas: A Retrospective Bicentre Analysis
1Department of General, Visceral and Vascular Surgery, Fürst-Stirum-Klinik, 76646 Bruchsal, Germany
2Department of General, Visceral, Vascular and Pediatric Surgery, University Hospital of Saarland, 66421 Homburg, Germany
3Department of General and Visceral Surgery, Barmherzige Brüder Hospital, 80639 Munich, Germany
Received 5 June 2013; Accepted 6 September 2013
Academic Editor: Rao R. Ivatury
Copyright © 2013 Sven Richter et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Introduction. The open abdomen (OA) is often associated with complications. It has been hypothesized that negative pressure wound therapy (NPWT) in the treatment of OA may provoke enteral fistulas. Therefore, we analyzed patients with OA and NPWT with special regard to the occurrence of intestinal fistulas. Methods. The present study included all consecutive patients with OA treated with NWPT from April 2010 to August 2011 in two hospitals. Patients’ demographics, indications for OA, risk factors, complications, outcome and incidence of fistulas before, during and after NPWT were recorded. Results. Of 81 patients with OA, 26 had pre-existing fistulas and 55 were free from a fistula at the beginning of NPWT. Nine of the 55 patients developed fistulas during () or after NPWT (). Seventy-five patients received ABThera therapy, 6 patients other temporary abdominal closure devices. Only diverticulitis seemed to be a significant predisposing factor for fistulas. Mortality was slightly lower for patients without fistulas. Conclusion. The present study revealed no correlation between occurrence of fistulas before, during, and after NWPT, with diverticulitis being the only risk factor. Fistula formation during NPWT was comparable to reports from literature. Prospective studies are mandatory to clarify the impact of NPWT on fistula formation.
The open abdomen (OA) generally represents a sequel either of trauma or of complications following visceral surgery. Its management is challenging per se and can be further aggravated by complications [1–3]. Of those, intestinal fistulas are the most serious because of their association with significant morbidity and mortality [4–6]. Development of enteral fistulas is estimated up to 25% during the therapy of OA [3, 7–9], which may be caused by dryness of the small or large bowel resulting from exposition to the ambient air as well as by mechanical irritation of wound dressings . In addition, important predisposing factors that contribute to higher fistula rates have been identified, such as inflammatory bowel disease, pancreatitis, diverticulitis, renal insufficiency, poor nutritional state, and ischemic conditions [5, 11, 12]. When fistulas develop, considerable amounts of stool fill both the abdominal wound and abdominal cavity, thereby aggravating the dilemma and leading to a vicious circle [3, 13]. Thus, in the management of OA, the prevention of any enteral fistula is mandatory to achieve appropriate abdominal wall closure, as this in turn represents the best way to avoid further complications [14, 15]. In the treatment of OA, manifold temporary abdominal closure (TAC) methods have been described, including skin-approximation closure , absorbable synthetic mesh products , Bogota bags , Barker’s vacuum packing technique, and negative pressure wound therapy (NPWT) [2, 3, 19]. In recent studies, higher fascial closure rates were realized using NPWT [19–25] compared to Barker’s vacuum packing technique [26, 27], skin towel clips, Bogota bag, and absorbable mesh [2, 17, 28, 29]. Although NPWT is widely used [30, 31], it is still a matter of debate whether fistulas develop due to the negative pressure applied [28, 32, 33]. According to the guidelines of the National Institute for Health and Clinical Excellence (NICE)  data of all patients with OA treated between January 1, 2010, to June 30, 2011 (treated with NPWT or alternatively), were submitted to the NICE audit. Herein, and supported by further literature , concerns have been expressed that NPWT may cause intestinal fistulas. The purpose of the present study was therefore to examine the incidence of fistulas in patients with OA undergoing NPWT, occurring either during ABThera or other TAC therapy.
Data was obtained retrospectively from patients’ records of 2 German hospitals (University Hospital of Saarland, Homburg, Saar; Barmherzige Brüder Hospital, Munich). All patients with OA treated by NPWT between April 2010 and August 2011 were included into the study. Because NWPT of the open abdomen was representing the standard in the therapy of the OA in both hospitals, all patients were potentially eligible for the data analysis and were therefore included. Primary data acquisition was performed by standardized questionnaires from the patients records, partially available as electronic records from the hospital information systems (SAP, St. Leon Roth) and transferred to spreadsheet software (Microsoft Excel, Redmond, USA). The whole study design was approved by the local ethical committees of both hospitals. In most patients, the ABThera OA NPWT system (KCI Europe, Amstelveen, Netherlands) was used, consisting of a visceral protective layer (polyurethane foam enveloped in a polyethylene sheet with small fenestrations), polyurethane foams (for placement on top of the visceral protective layer), and self-adhesive polyethylene drapes (for air-tight coverage of the wound). Only a small minority of patients received other TAC devices based on the surgeon’s individual decision. These latter devices consisted of a sterile and perforated plastic foil for visceral protection, directly covered with standard polyurethane foam (V.A.C. GranuFoam, KCI) and sealed with self-adhesive polyethylene drapes. NPWT in all patients, including negative pressure settings of −100 mmHg to −125 mmHg , and regular dressing changes were performed as recommended by the manufacturer and described elsewhere . Patients received standard of care, which included intensive care unit therapy (with mechanical ventilation if necessary), oral fluids, and adequate nutrition. Patients were grouped based on presence/absence of a fistula as well as occurrence of the fistula: no fistula, preexisting fistula, fistula developed during NPWT, and fistula after termination of NPWT. Patients’ demographics (e.g., gender, age, height, BMI, APACHE, and SOFA scores), indications for OA, and preexisting risk factors were recorded. Complications (including fistula formation) were analyzed and adverse events as well as patients’ outcome (e.g., primary/secondary fascial closure and death) were analyzed. The occurrence of any fistula during or after NPWT was defined as dependent variable. Statistical analyses were performed using the Statistical Package for Social Sciences (SPSS statistics, Version 15.0, Chicago, SPSS Inc.). All demographic variables (e.g., patient age, height, BMI, APACHE, and SOFA scores) were expressed as mean ± standard deviation. Kruskal-Wallis rank test for quantitative data was applied. Fisher’s exact test or Chi-square test was used for the comparison of no fistula group to any fistula group, for OA indications, predisposing factors, outcomes, and adverse events.
Data was obtained from a total of 81 consecutive patients undergoing NPWT. Statistical analysis of patient demographics revealed no significant differences (Table 1). The majority of OA was due to surgical/nontraumatic indications. Bowel perforation, anastomotic leakage, or abdominal compartment syndrome showed a significant () relation to the occurrence of fistulas (Table 2). All patients had at least one preexisting risk factor for fistula formation with a high percentage of patients having acute renal insufficiency or malignancy (Table 3). However, the only significant predisposing factor for fistula formation was diverticulitis, being observed only in patients with fistulas (0.0% versus 17.1%; ) (Table 3).
Seventy-five patients (92.6%) received ABThera therapy, whereas in 6 patients (7.4%) other TAC devices were used. There were 55 patients with no preexisting fistulas and 26 patients with preexisting fistulas. Out of the 55 patients with no preexisting fistulas, 5 developed fistulas during NPWT. These fistulas have become clinically evident between day 3 and day 10 after the beginning of NPWT. Three of patients with fistulas during NPWT received standard ABThera therapy, whereas the other 2 patients were treated by other TAC devices. Another 4 patients with no preexisting fistula and no fistula during NPWT developed a secondary fistula between 3 and 46 days after the end of therapy. Out of those four patients, two received other TAC devices whereas the other 2 patients received ABThera therapy. The all-in-all fistula rate during both acute NPWT and short time interval after the operation was 16.4%. 5 of the fistulas (9.1%) occurred under ABThera therapy whereas 4 fistulas (7.3%) occurred when using other TAC devices. The number of days with OA was significantly lower for patients with no fistulas compared to patients who developed fistulas during NPWT ( days versus days; ). This correlated significantly with fewer days of NPWT for patients with no fistulas compared to patients who developed fistulas during NPWT ( days versus days; ). The majority of OA could be closed within 30 days (84.8%) in patients with no fistula. In contrast, closure rate in the same time interval for patients with fistulas was 74.3% () (Table 4). Overall mortality was 30.9% and slightly lower for patients without fistulas compared to those with fistulas (28.6% versus 34.3%); however, this did not reveal any significance () (Table 4). As well, no significant difference in number of adverse events between patients with fistulas and patients without fistulas could be shown (Table 5).
The treatment of the open abdomen represents a surgical challenge per se but can be further aggravated by a variety of complications, resulting in high mortality [7, 26, 31]. It is known from the literature that NPWT results in a higher rate in secondary abdominal wall closure compared to standard care. Secondary closure rate was 84.8% in patients with no fistula overall and 74.3% in patients with any fistula. These closure rates are comparable to what is already known in the NPWT of the open abdomen [14, 19, 24, 28, 31, 32].
However, fistulas may lead to a vicious cycle as the fistula itself may impede fascial closure and thereby perpetuate and aggravate the difficult situation of the OA. Although there may be a multifactorial etiology of fistula development, dessication of the bowel is regarded as a major factor for the development of fistulas [10, 37]. Therefore, one strategy to prevent fistulas is to cover all exposed bowel with omentum, avoid hyperresuscitation and the resulting bowel edema, and minimize serosal injury to the exposed bowel. Furthermore a definitive closure of the skin or fascia therefore represents the best dressing and prevents enteroatmospheric fistulas . The recently introduced ABThera device was also suspicious for causing enteral fistulas . It can be hypothesized that the vacuum effect may be harmful, as it may impede enteral microcirculation, and it has been proposed hitherto never to use the vacuum sponge in direct contact to bowel serosa. Within the data presented herein, we report 9 new fistulas (16.4%) during or after NPWT, which is higher than the fistula rate reported in two recently published reviews with predominantly included trauma patients [1, 31]. 55 of our patients had no preexisting fistula at the beginning of the NPWT. During or after NPWT 9 patients developed a fistula. The fistula rate of 16.4% as shown herein is comparable to the most studies including mainly patients with peritonitis and OA (14.8–21%) [28, 38–40]. Although our patients’ collective was bigger than those studies included in the above mentioned reviews, we could not determine any statistical significant parameter for fistula formation—except for colon diverticulitis. This may be explained by a higher vulnerability of the intestinal. Nevertheless, our study is representing a collective of patients with secondary peritonitis as the indicating reason for the use of NPWT.
However, some shortcomings of the present study have to be mentioned. In general, the crunch question of NPWT in the treatment of the open abdomen is, Does NPWT helps to treat fistulas or does it cause new fistulas? One may argue that the NPWT itself is causing fistulas. Most of the patients included suffered from severe peritonitis and were critically ill being evident as high APACHE and SOFA scores. To clarify a possible adverse effect of NPWT in fistula formation, bigger homogenous collectives of patients have to be evaluated prospectively. Due to the retrospective character, an adequate power and sample size calculation has not been done. In addition, the OA treatment with NPWT represents a single arm study without any control group, so there is no way to determine if the rate of fistula would have been different with another treatment. Furthermore in one of both clinics including the majority of patients, single high volume lavage with primary closure of the fascia was preferred in the treatment of secondary peritonitis as published by our group in 2009 .
One finding of our study is that more than 40% of the fistulas developing under NPWT occurred when self-made abdominal vacuum devices were used. All four patients with self-made vacuum TACs were from one hospital but at least treated by different surgeons, what is representing a confounder variable. Although the difference in new fistula incidence seems to be evident, it is not possible to draw a final conclusion as to whether other TAC devices using NPWT are an independent factor influencing fistula rate because of the small number of patients treated with homemade TAC devices. Our study surely consists of some bias, and it remains quite difficult to hypothesize about the pathophysiology of fistula development during self-made vacuum devices. Explanations may be that with self-made vacuum devices the sponge may get in contact with the bowel or that irregular perforation holes in the foil create inhomogeneous suction levels at the intestinal site.
Another reason for the development of fistulas may consist in the changing intervals of the abdominal dressing. So far, no study exists defining the optimal time point for changing the vacuum dressing. In our experience clinical routine shows that many dressings have been redone on demand, when patients show clinical worsening such as septic complications or when the consistence of the suction fluid is changing quality. According to our results longer NPWT results in a high rate of fistulas during or after the therapy. Maybe one reason to avoid fistulas is to change the NPWT system in shorter intervals with an assessment for earlier closure of the abdominal wall or at least skin closure. As well, we cannot answer the question whether the development of fistulas correlates with the extent of vacuum pressure applied, as the vacuum settings in our retrospective data collection were not standardized and thus heterogeneous but mostly over 100 mmHg of negative pressure. However, this point may not be of too great importance, as the suction measured within the abdominal cavity seems to be quite similar at any negative vacuum pressure applied . In contrast to that, recent data indicates that lower suction intensity [35, 39] is more preferred than higher intensity as we did it in our study. Overall mortality was comparable to previous published studies [1, 7, 19, 26]. Interestingly, the occurrence of fistula did not affect overall mortality in our study.
All in all, it has to be kept in mind that not all surgeons have unlimited access to industrially standardized NPWT products as it is represented by the ABThera or similar products. In addition, most of the patients suffering from open abdomen—caused by surgical complications initially—are treated in an emergency situation. In those cases there may be often no other way to achieve a temporary closure of the abdomen other than using other TAC devices because the availability of NPWT products in the treatment of the open abdomen is not the standard in all hospitals. In addition many hospitals facing economical challenges cannot afford such cost intensive therapies.
The development of fistulas during NPWT is representing a challenging and common problem in the context of peritonitis in critically ill patients. When using standardized industrially NPWT products, intestinal fistulas cannot be avoided. It can be assumed that peritonitis itself and other patient related factors may contribute to fistula development. To draw a final conclusion, if NPWT products in the therapy of the open abdomen are causing or preventing fistulas, larger databases included in prospective multicentre studies are mandatory.
Conflict of Interests
None of the authors has any conflict of interests related to the data presented in the current work.
- P. Boele van Hensbroek, J. Wind, M. G. W. Dijkgraaf, O. R. C. Busch, and J. Carel Goslings, “Temporary closure of the open abdomen: a systematic review on delayed primary fascial closure in patients with an open abdomen,” World Journal of Surgery, vol. 33, no. 2, pp. 199–207, 2009.
- A. Campbell, M. Chang, T. Fabian et al., “Management of the open abdomen: from initial operation to definitive closure,” The American Surgeon, vol. 75, no. 11, supplement, pp. S1–S22, 2009.
- M. Kaplan, P. Banwell, D. P. Orgill et al., “Guidelines for the management of the open abdomen,” Wounds, vol. 17, supplement 1, pp. 1–24, 2005.
- M. Falconi and P. Pederzoli, “The relevance of gastrointestinal fistulae in clinical practice: a review,” Gut, vol. 49, supplement 4, pp. iv2–iv10, 2001.
- B. Hahler, D. Schassberger, R. Novakovic, and S. Lang, “Managing complex, high-output, enterocutaneous fistulas: a case study,” Ostomy Wound Management, vol. 55, no. 10, pp. 30–42, 2009.
- S. Sriussadaporn, R. Pak-Art, and S. Bunjongsat, “Immediate closure of the open abdomen with bilateral bipedicle anterior abdominal skin flaps and subsequent retrorectus prosthetic mesh repair of the late giant ventral hernias,” Journal of Trauma, vol. 54, no. 6, pp. 1083–1089, 2003.
- N. Goussous, B. D. Kim, D. H. Jenkins, and M. D. Zielinski, “Factors affecting primary fascial closure of the open abdomen in the nontrauma patient,” Surgery, vol. 152, no. 4, pp. 777–784, 2012.
- M. Kaplan, “Managing the open abdomen,” Ostomy Wound Management, vol. 50, no. 1A, supplement, pp. C2–C8, 2004.
- M. B. Shapiro, D. H. Jenkins, C. W. Schwab, and M. F. Rotondo, “Damage control: collective review,” Journal of Trauma, vol. 49, no. 5, pp. 969–978, 2000.
- R. R. Martin and M. Byrne, “Postoperative care and complications of damage control surgery,” Surgical Clinics of North America, vol. 77, no. 4, pp. 929–942, 1997.
- J. M. Llop, S. Cobo, A. Padulles, L. Farran, R. Jódar, and M. B. Badia, “Nutritional support and risk factors of appearance of enterocutaneous fistulas,” Nutrición Hospitalaria, vol. 27, no. 1, pp. 213–218, 2012.
- A. C. Lynch, C. P. Delaney, A. J. Senagore, J. T. Connor, F. H. Remzi, and V. W. Fazio, “Clinical outcome and factors predictive of recurrence after enterocutaneous fistula surgery,” Annals of Surgery, vol. 240, no. 5, pp. 825–831, 2004.
- D. Demetriades, “Total management of the open abdomen,” International Wound Journal, vol. 9, supplement 1, pp. 17–24, 2012.
- M. Schmelzle, I. Alldinger, H. Matthaei et al., “Long-term vacuum-assisted closure in open abdomen due to secondary peritonitis: a retrospective evaluation of a selected group of patients,” Digestive Surgery, vol. 27, no. 4, pp. 272–278, 2010.
- R. R. Ivatury, “Update on open abdomen management: achievements and challenges,” World Journal of Surgery, vol. 33, no. 6, pp. 1150–1153, 2009.
- R. J. Hannon, J. M. Hood, and R. C. Curry, “Temporary abdominal closure: a new product,” British Journal of Surgery, vol. 79, no. 8, pp. 820–821, 1992.
- T. W. Jernigan, T. C. Fabian, M. A. Croce et al., “Staged management of giant abdominal wall defects: acute and long-term results,” Annals of Surgery, vol. 238, no. 3, pp. 349–357, 2003.
- L. Fernandez, S. Norwood, R. Roettger, and H. E. Wilkins III, “Temporary intravenous bag silo closure in severe abdominal trauma,” Journal of Trauma, vol. 40, no. 2, pp. 258–260, 1996.
- M. E. Franklin, A. Alvarez, and K. Russek, “Negative pressure therapy: a viable option for general surgical management of the open abdomen,” Surgical Innovation, vol. 19, no. 4, pp. 353–363, 2012.
- C. C. Cothren, E. E. Moore, J. L. Johnson, J. B. Moore, and J. M. Burch, “One hundred percent fascial approximation with sequential abdominal closure of the open abdomen,” American Journal of Surgery, vol. 192, no. 2, pp. 238–242, 2006.
- G. B. Garner, D. N. Ware, C. S. Cocanour et al., “Vacuum-assisted wound closure provides early fascial reapproximation in trauma patients with open abdomens,” American Journal of Surgery, vol. 182, no. 6, pp. 630–638, 2001.
- P. R. Miller, J. W. Meredith, J. C. Johnson, and M. C. Chang, “Prospective evaluation of vacuum-assisted fascial closure after open abdomen: planned ventral hernia rate is substantially reduced,” Annals of Surgery, vol. 239, no. 5, pp. 608–614, 2004.
- P. R. Miller, J. T. Thompson, B. J. Faler, J. W. Meredith, and M. C. Chang, “Late fascial closure in lieu of ventral hernia: the next step in open abdomen management,” Journal of Trauma, vol. 53, no. 5, pp. 843–849, 2002.
- D. Perez, S. Wildi, N. Demartines, M. Bramkamp, C. Koehler, and P.-A. Clavien, “Prospective evaluation of vacuum-assisted closure in abdominal compartment syndrome and severe abdominal sepsis,” Journal of the American College of Surgeons, vol. 205, no. 4, pp. 586–592, 2007.
- C. E. Stonerock, R. P. Bynoe, M. J. Yost, and J. M. Nottingham, “Use of a vacuum-assisted device to facilitate abdominal closure,” American Surgeon, vol. 69, no. 12, pp. 1030–1034, 2003.
- D. E. Barker, J. M. Green, R. A. Maxwell et al., “Experience with vacuum-pack temporary abdominal wound closure in 258 trauma and general and vascular surgical patients,” Journal of the American College of Surgeons, vol. 204, no. 5, pp. 784–792, 2007.
- D. E. Barker, H. J. Kaufman, L. A. Smith, D. L. Ciraulo, C. L. Richart, and R. P. Burns, “Vacuum pack technique of temporary abdominal closure: a 7-year experience with 112 patients,” Journal of Trauma, vol. 48, no. 2, pp. 201–207, 2000.
- T. K. Bee, M. A. Croce, L. J. Magnotti et al., “Temporary abdominal closure techniques: a prospective randomized trial comparing polyglactin 910 mesh and vacuum-assisted closure,” Journal of Trauma, vol. 65, no. 2, pp. 337–342, 2008.
- L. N. Tremblay, D. V. Feliciano, J. Schmidt et al., “Skin only or silo closure in the critically ill patient with an open abdomen,” American Journal of Surgery, vol. 182, no. 6, pp. 670–675, 2001.
- F. Herrle, T. Hasenberg, B. Fini et al., “Open abdomen 2009: a national survey of open abdomen treatment in Germany,” Chirurg, vol. 82, no. 8, pp. 684–690, 2011.
- A. J. Quyn, C. Johnston, D. Hall, et al., “The open abdomen and temporary abdominal closure systems—historical evolution and systematic review,” Colorectal Disease, vol. 14, no. 8, pp. e429–e438, 2011.
- S. Acosta, T. Bjarnason, U. Petersson et al., “Multicentre prospective study of fascial closure rate after open abdomen with vacuum and mesh-mediated fascial traction,” British Journal of Surgery, vol. 98, no. 5, pp. 735–743, 2011.
- M. Rao, D. Burke, P. J. Finan, and P. M. Sagar, “The use of vacuum-assisted closure of abdominal wounds: a word of caution,” Colorectal Disease, vol. 9, no. 3, pp. 266–268, 2007.
- Excellence NIfHaC: Clinical guideline IPG322, “Negative pressure wound therapy for the open abdomen,” 2009, http://guidance.nice.org.uk/IPG322/Guidance/pdf/English.
- G. E. Glass and J. Nanchahal, “The methodology of negative pressure wound therapy: separating fact from fiction,” Journal of Plastic, Reconstructive and Aesthetic Surgery, vol. 65, no. 8, pp. 989–1001, 2012.
- J. E. Fitzgerald, S. Gupta, S. Masterson, and H. H. Sigurdsson, “Laparostomy management using the ABThera open abdomen negative pressure therapy system in a grade IV open abdomen secondary to acute pancreatitis,” International Wound Journal, vol. 10, no. 2, pp. 138–144, 2013.
- P. H. Navsaria, M. Bunting, J. Omoshoro-Jones, A. J. Nicol, and D. Kahn, “Temporary closure of open abdominal wounds by the modified sandwich-vacuum pack technique,” British Journal of Surgery, vol. 90, no. 6, pp. 718–722, 2003.
- A. L. Adkins, J. Robbins, M. Villalba, P. Bendick, and C. J. Shanley, “Open abdomen management of intra-abdominal sepsis,” American Surgeon, vol. 70, no. 2, pp. 137–140, 2004.
- A. J. Fieger, F. Schwatlo, D. F.-X. Mündel et al., “Abdominal vacuum therapy for the open abdomen—a retrospective analysis of 82 consecutive patients,” Zentralblatt für Chirurgie, vol. 136, no. 1, pp. 56–60, 2011.
- W. Kneist, V. Juhre, and H. Lang, “Intestinale Fisteln bei Therapieverfahren am offenen Abdomen—Ergebnisse einer retrospektiven Analyse von 50 konsekutiven Patienten,” Zeitschrift für Gastroenterologie, vol. 50, no. 8, p. K335, 2012.
- M. R. Moussavian, S. Richter, O. Kollmar, J. Schuld, and M. K. Schilling, “Staged lavage versus single high-volume lavage in the treatment of feculent/purulent peritonitis: a matched pair analysis,” Langenbeck's Archives of Surgery, vol. 394, no. 2, pp. 215–220, 2009.
- T. Bjarnason, A. Montgomery, J. Hlebowicz, S. Lindstedt, and U. Petersson, “Pressure at the bowel surface during topical negative pressure therapy of the open abdomen: an experimental study in a porcine model,” World Journal of Surgery, vol. 35, no. 4, pp. 917–923, 2011.