Abstract
Objective. To assess quality of care of women with severe maternal morbidity and to identify associated factors. Method. This is a national multicenter cross-sectional study performing surveillance for severe maternal morbidity, using the World Health Organization criteria. The expected number of maternal deaths was calculated with the maternal severity index (MSI) based on the severity of complication, and the standardized mortality ratio (SMR) for each center was estimated. Analyses on the adequacy of care were performed. Results. 17 hospitals were classified as providing adequate and 10 as nonadequate care. Besides almost twofold increase in maternal mortality ratio, the main factors associated with nonadequate performance were geographic difficulty in accessing health services (), delays related to quality of medical care (), absence of blood derivatives (), difficulties of communication between health services (), and any delay during the whole process (). Conclusions. This is an example of how evaluation of the performance of health services is possible, using a benchmarking tool specific to Obstetrics. In this study the MSI was a useful tool for identifying differences in maternal mortality ratios and factors associated with nonadequate performance of care.
1. Introduction
The outcome of a critically ill patient is a result of clinical and individual factors, including previous health status, physiologic reserve, disease diagnosis, and also adequacy of care provided during the disease. Thus, it is difficult to individually analyze and predict morbidity and mortality outcomes in critically ill patients [1]. Stratification of patient groups according to clinical severity may facilitate interpretation of these results by comparing similar groups [2].
Some scoring systems are capable of quantifying severity, for example, the APACHE (Acute Physiology and Chronic Health Evaluation), SAPS (Simplified Acute Physiology Score), SOFA (Sequential Organ Failure Assessment), and MPM (Mortality Prediction Model) [1]. However, they were developed using general populations of critically ill patients in high income countries. Considering severe pregnancy complications, several factors seem to reduce the capacity to classify severity and predict mortality among pregnant women with these models.
The different physiologic parameters, diseases unique to pregnancy, and a population largely composed of young women who were previously healthy contribute to the little applicability of these tools in Obstetrics [3]. As a result, traditional risk stratification models usually overestimate mortality among pregnant women, which may hinder analysis of the performance of care provided and interpretation of morbidity and mortality outcomes [3].
“Benchmarking” may be understood as a reference point against which comparisons can be made, regarding the performance between facilities and/or best practice. The demand for this type of data is growing, not only due to initiatives to pay for performance but also because of clinical, administrative, and research applications. Performance feedback at an institutional or individual level may lead to an improvement in overall performance [2].
Several initiatives for maternal and infant health have been implemented worldwide, aimed at achieving the millennium development goals (MDG) [4–8]. Nevertheless, advances made over the years are below those required for effective morbidity and mortality reduction. Structured health systems are identified as fundamental to obtain better results and accelerate progress for achieving these goals [9, 10].
The World Health Organization (WHO) used organ dysfunction criteria and parameters of extreme severity specific to Obstetrics to define life-threatening conditions associated with pregnancy, standardizing the maternal near miss criteria [8]. A maternal near miss is an event in which a woman nearly died, but survived a severe complication occurring during pregnancy, childbirth, or within 42 days of its termination. It represents the extreme degree of organ dysfunction/failure in the wide spectrum of morbidity and differs from death only by the final outcome [8].
Until this definition, several studies used different parameters for severe morbidity, such as admission to intensive care units (ICU) or clinical diagnoses [11–14]. The first retrospective validation of these criteria was performed in a population of obstetric patients admitted to ICU, using the total maximum SOFA score as the gold standard and showing that the WHO near miss criteria obtained a sensitivity and specificity of 99.2 and 86.0%, respectively, for the identification of organ failure in at least one organ system [15].
The Brazilian Network for Surveillance of Severe Maternal Morbidity was a prospective study aimed at identifying potentially life-threatening and maternal near miss cases [16, 17]. Assuming that a woman suffering from a near miss event is exactly like one who has died, except for the outcome, criteria would be validated if all maternal deaths were identified and if the false-positive cases represented exactly the near miss cases. The performance of the WHO near miss criteria was confirmed, with a sensitivity and specificity of 100 and 92%, respectively [18]. In this study a tool called maternal severity index (MSI) was also developed specifically to predict mortality for the obstetric population. This appears to be a first step in making a case-mix analysis and a comparison between obstetric services by matching similar populations [18].
Strategies aimed at strengthening health systems are necessary. However, many systems still do not have the capacity to measure and understand their own weaknesses, making it difficult for healthcare policy managers to incorporate scientific strategies towards strengthening systems [10]. The maternal near miss approach may be a tool for assessment of quality of maternal care provided. As a result, standardization and comparison can be made between maternal morbidity groups from different locations and over time, identifying weaknesses [7]. Thus, the aim of this study was to simulate the evaluation of an obstetric health system, through analysis of the performance of care in the Brazilian Network for the Surveillance of Severe Maternal Morbidity, using the maternal near miss criteria approach.
2. Materials and Methods
2.1. Ethics Statement
Research protocol was approved by the Institutional Review Board of the coordinating institution (University of Campinas) on 5 May 2009 (Document CEP 027/2009). The study was approved by the local Institutional Review Board of each participating center and also nationally. Each center was previously consulted regarding this analysis of performance and data publication. Approval was unanimous. To ensure confidentiality, each center is not identified and received confidential information on the category it was classified in order to be able to adopt procedures to improve quality of care provided.
2.2. Study Population
The Brazilian Network for Surveillance of Severe Maternal Morbidity was a cross-sectional multicenter study aimed at identifying severe maternal morbidity cases, using the new WHO definition [8, 16]. From July 2009 to June 2010, 27 referral hospitals, representing a purposeful sample of the Brazilian health facilities caring for women’s deliveries, made a prospective surveillance to identify severe maternal morbidity/near miss cases.
The study was planned in detail, with preparatory meetings to discuss methods and procedures with participants from all centers. In addition to personal and clinical information of each case of severe maternal morbidity identified, a rigorous system of screening for any of the three delays in obstetrical care was also implemented [19]. After prospective data collection was completed, a rigorous checking system for data consistency was developed. Additional details on method and procedures are in other publications [16, 17].
2.3. Development of a Model for Mortality Prediction
Previously, it was possible to build a model for mortality prediction, named maternal severity index (MSI) [18]. Briefly, two models for mortality prediction were developed. First, it was confirmed that the number of near miss markers could be related to mortality and this correlation was called maternal severity score (MSS). The WHO near miss markers are shown in Table 1, distributed as Group A (organ dysfunction) and Group B (severe dysfunction/failure). Two models of bivariate logistic regression were then developed and tested to describe the relationship between severe morbidity and mortality. For this, the total study population was divided into two subpopulations “A” and “B” to develop and test the prediction model, respectively. The sample size of population “B” was obtained considering a probability of 0.05 for type I error, 0.20 for type II error, and a minimum area under the ROC (receiver operating characteristic) curve of 0.80.
The first model was a univariate analysis including only MSS or the number of severity markers. The second used univariate analyses considering MSS, distal predictors of mortality (such as demographic and obstetric characteristics), and near miss criteria as independent variables and the outcome maternal death as the dependent variable. Variables significantly correlated with mortality were selected for multivariate analysis. Positive or negative correlation coefficients (β) were attributed to each variable included in the model. Calibration and discrimination of models were performed. Model 2 showed the best performance for mortality prediction. Therefore, it was chosen to be the maternal severity index (MSI) [18]. Briefly, the SMI is estimated through an equation which takes into account the MSS (number of life-threatening conditions) and the presence of some associated specific conditions (life-threatening condition identified in the first 24 hours of hospital stay, severe preeclampsia, cancer, any marker of cardiovascular failure, any marker of respiratory failure, and hysterectomy), which are the variables significantly correlated with mortality mentioned above. With the SMI thus estimated for each case and the mean SMI for any specified group, the number of expected deaths can be determined. When the number of observed deaths is compared with the number of expected deaths, the concept of standardized mortality ratio (SMR) is used. To simplify the estimation of SMR, a calculator was developed (Figure 1).
2.4. Analysis of Performance of Network Centers
The mean MSI for each center was obtained and the standardized mortality ratio (SMR) was calculated for each one (Figure 1). SMR is the ratio between observed mortality in the population and expected mortality by mortality prediction based on severity of the case expressed by MSI. To allow calculation of SMR for all centers, a value of 0.1 was attributed, when no death was observed or there was no expected death due to the small sample size and/or low complexity of cases in that center.
In the original model, the categorization of performance was based on cutoff points selected assuming the normal distribution into five classes of care and the understanding that ; that is, the occurrence of half or less than the expected number of deaths could correspond to excellent care. Thus, the five categories of performance were defined as very high, high, intermediate, very low, and low [18].
In this analysis, SMR was calculated from the mean MSI for each of the 27 study centers. Due to the relatively small number of centers and to make analyses more consistent as an exercise to evaluate large systems, the original classification was modified, and groups were relocated to two new categories. Categories “very high,” “high,” and “intermediate,” including to 1.24, were reclassified as “adequate” care. Categories “low” and “very low,” with SMR of 1.25 to 2, were recoded as “nonadequate.” Thus, this performance was analyzed as a dependent variable. Using both groups of adequacy of care, variables related to structure, process, management, and delays were correlated as independent variables. Although SMR was calculated for each health facility, it should not be understood as an evaluation only of the care the hospital provided but the general care those women received. This includes not only the hospital activities/responsibilities but also those from the woman herself, her family, community, and access to health services.
Furthermore, patient outcomes, their respective indicators, and main causes of complication were evaluated for both groups of level of performance. Finally, Poisson multiple regression analysis was performed, using the level of performance as a dependent variable, estimating the prevalence ratio and its respective 95% CI to identify variables independently associated with performance. A 5% statistical significance level was used. All measures of effect in the study design and their respective values were calculated after adjusting for cluster effect.
3. Results
In this period, when these facilities took care of 82,388 deliveries with 82,144 live births (representing a fraction of around 2.8% of all annual live births in the country), 9555 women had severe pregnancy complications, with 770 near miss cases and 140 maternal deaths. In Table 2, the mean MSI, number of observed deaths, sample size (potentially life-threatening and near miss conditions), expected number of deaths, SMR, and, finally, the recategorized level of performance of care are presented for each of the 27 centers, 17 being classified as having “adequate” and 10 as “nonadequate” performance. Generally, the overall performance of Network was adequate, since there were only two more maternal deaths than expected due to severity of cases ().
Table 3 shows the distribution of outcomes of severity and of main causes of morbidity, according to the level of performance of the centers. Although differences were not significant, there was proportionally almost twice the proportion of deaths among centers classified as “nonadequate” care. Maternal health indicators also show that the occurrence of maternal near miss was similar between performance groups, but there was an almost twofold increase in the maternal mortality ratio (MMR) in the group with nonadequate care.
Structure indicators according to level of performance are shown in Table 4. Centers with “adequate” performance were located with an almost 2.5 higher prevalence in the southeast and south of the country, but this difference was not significant. When process and management indicators (Table 5) were analyzed, there was twofold increase in ICU admissions in adequate care centers, but this difference was also not significant.
Association between delay in care and the level of performance is shown in Table 6. Generally, both detection of any type of delay and some specific categories of delays, particularly those concerning the third delay (related to quality of care), were significantly more common in the nonadequate performance group. Considering all the predictive variables included in the multiple regression analysis (Table 7), the use of magnesium sulfate and location of the health center in the south or southeast were the main variables independently associated with adequate performance of care.
4. Discussion
With the use of the MSI and SMR, it was possible to assess the performance of centers from the Brazilian Network for Surveillance of Severe Maternal Morbidity. This was a first initiative to use the near miss concept as a tool for assessment of case-mix and adequacy of care received in Obstetrics. However, as already stated, this assessment of the quality of obstetrical care is probably more appropriate for a specific population that “receives” the care than for a specific health facility that “provides” the care. This is because the global assessment does not imply only the activities performed at the hospital but also the characteristics of women and their community services and access to health services. If a woman arrives late at the hospital in a very severe almost dying condition, she is more likely to die even if the hospital is tertiary, well equipped, and with a good trained staff. The responsibility for such a death cannot be attributed solely to this hospital. In other words, the care provided by the hospital can be adequate while the care received by the woman can be not adequate.
In the classification of performance, sample size per center corresponds to the number of cases with potentially life-threatening conditions and threatening life conditions (near miss and maternal death) and not the total number of live births, because only women presenting with some severity indicators were included in the study. Although calculation of SMR for all centers was possible after attributing a value of 0.1 to those that did not present any observed or expected death, these estimates had low accuracy. Therefore, the MSI seems limited and less precise for use in populations with a small number of cases, with lower clinical severity (low MSI) and those with no observed death. The lower accuracy of estimates in these cases should increase attention and care for interpreting the performance of these health services.
Centers participating in this Brazilian Network were selected considering their availability to participate in the study, the total number of annual deliveries, and geographic location. Thus, the estimated sample size could be achieved, ensuring a broad distribution in national territory. Nevertheless, most of these hospitals were linked to large university institutions or had teaching activities, and this implies that they are mostly referral for healthcare of severe pregnancy complications, with evidence-based protocols and similar standards of care. The relative homogeneity of these facilities may have contributed to the lack of significant differences between levels of performance of care and most variables related to the profile of severity of illness, cause of morbidity, and structure and management indicators, in addition to the limited number of centers participating in the study.
The greater proportion of infectious causes among centers with adequate performance may be perceived as better adequacy to international protocols for the management of sepsis [20], already widely known among academic medical services. Referral of these cases to hospitals equipped with a high complexity arsenal is a determining factor for the survival of these patients.
As observed, there was an almost twofold increase in proportion of deaths in centers with nonadequate performance. However, the prevalence of maternal near miss was practically the same in both groups of performance. This is in agreement with recent knowledge that severe pregnancy complications occur practically in the same frequency in all countries and regions, regardless of the level of development and availability of resources. In fact, the varying factor is mortality, which is always higher in contexts of lower development and scarce resources, as currently demonstrated [21, 22].
In this study, information on potential delays in obtaining care among women suffering severe complications was collected. In addition to objective information from medical charts, local researchers made a subjective assessment and searched for the three types of delay in obtaining care [19]. The presence of any delay in care was significantly related to a worse performance of service. These results are in agreement with the concept that the main preventive factors in decreasing maternal mortality are delays in the care process, from symptom identification by the patient to the provision of adequate treatment by healthcare professionals [19]. Globally there was a greater delay in providing services in nonadequate care centers, mainly due to the absence of blood products and difficulty in establishing communication between services and/or regulatory centers.
These findings seem to follow a presumptive logic. Subjects living in geographic regions with difficult access have the greatest difficulty in seeking medical care, including antenatal followup. Healthcare facilities close to these homes are also probably on the outskirts of large cities and high-complexity hospitals are usually in the center of these cities. In general, this peripheral healthcare equipment does not have an adequate structure to provide immediate care and monitor complications. There is also difficulty in regulating complex cases to large referral centers. Finally, successive delays in providing care are related to a higher number of severe outcomes and deaths.
This hypothesis may corroborate information that health system strengthening may actually have the greatest impact on improvement of clinical care [10]. For instance, prompt action of the whole health system for hierarchization of care, according to demand of severity, could ensure a reduction in deaths by improving adequacy of care. These strategies would go beyond the sole responsibility of the healthcare managers. Most probably, social development actions, civil, and transport infrastructure are necessary to correct determining factors for the health of these more vulnerable populations.
The MSI for mortality prediction follows the same indications and limitations of similar models, such as the APACHE and MPM. Use for individual evaluation is limited, since the greatest outcome predictor is individual response to therapy administered [2]. In addition, waiting to provide specific measures only when certain markers of severity emerge is not recommended. Individual clinical care is dynamic and the use of severity scores for decision making may delay healthcare, with poor provision of adequate resources. Most existing scores consider the clinical parameters obtained in the first 24 hours after ICU admission and do not assess time, care, or alterations present before it [2]. MSI was developed using data from the prehospital phase until discharge, using several sources distributed all over the country, which may increase the accuracy of its prediction capacity. However, if the development of the model and the use in the same population could be a limitation of the method, tests in other samples are needed for external validation and in fact it was already performed in a huge sample from the WHO study [22].
The SMR may be perceived as the evaluation of performance of a system rather than of a health service alone. In this study, SMR was recategorized as “adequate” and “nonadequate,” the cutoff point being the limit between intermediate and low care from the original classification [18]. Thus a group of centers that actually had SMR above 1 was classified as providing “adequate” care. Centers that could not prevent any expected death and in some cases had a slight increase in the number of deaths in relation to the expected due to severity of their cases were categorized as having adequate performance. Although this methodological option may have reduced the identification of variables related to actual excellent care (SMR between 0 and 0.8), it was a strategy adopted as a form of simplifying data analysis and prioritizing identification of centers with nonadequate performance. In the future, with a larger number of participating hospitals and subjects, it is likely that analysis with three categories (e.g., high, intermediate, and low performance) may be a valuable strategy for evaluating all the components of performance individually.
5. Conclusions
In the Brazilian Network for Surveillance of Severe Maternal Morbidity, the near miss approach was used to simulate analysis of an obstetric health system. After applying the MSI and SMR, analysis of the performance of services received was possible and its associated factors were assessed. Problems arising from the health system organization were identified as significant, especially those related to accessibility to health services and quality of medical care provided.
The use of this specific tool for mortality prediction may contribute to the analysis of obstetric health systems and identification of weaknesses. Furthermore, it may help to strengthen these systems, with an effective reduction in deaths. Nevertheless, new studies in different populations should be conducted for external calibration of tools developed in the Brazilian Network.
Ethical Approval
Research protocol was approved by the Institutional Review Board of the coordinating institution on 5 May 2009 (Document CEP 027/2009).
Conflict of Interests
All authors declare that they have no conflict of interests.
Author’s Contribution
The idea for the study arose in a group discussion among all the authors. After the end of data collection, Samira Maerrawi Haddad, Jose Guilherme Cecatti, Joao Paulo Souza, Mary Angela Parpinelli, and Maria Laura Costa prepared a detailed plan of analysis which was then performed by Maria Helena Sousa. The first version of the paper was drafted by Samira Maerrawi Haddad under supervision of Jose Guilherme Cecatti and then complemented with suggestions made by the other authors. All authors contributed to the development of the study protocol and approved the final version of the paper.
Acknowledgments
This study was funded by CNPq/DECIT (The National Research Council and the Department of Science and Technology of the Brazilian Ministry of Health), Grant number 402702/2008-5. The content is solely the responsibility of the authors and does not necessarily represent the official views of CNPq. It did not influence the content of the paper. Brazilian Network for the Surveillance of Severe Maternal Morbidity Group consisted of Joao L. Pinto e. Silva, Fernanda G. Surita, Rodolfo C. Pacagnella, Rodrigo S. Camargo, Vilma Zotareli, Lúcio T. Gurgel, Lale Say, Robert C. Pattinson, Marilza V. Rudge, Iracema M. Calderon, Maria V. Bahamondes, Danielly S. Santana, Simone P. Gonçalves, Olímpio B. Moraes Filho, Simone A. Carvalho, Francisco E. Feitosa, George N. Chaves, Ione R. Brum, Gloria C. Saint’Ynes, Carlos A. Menezes, Patricia N. Santos, Everardo M. Guanabara, Elson J. Almeida Jr, Joaquim L. Moreira, Maria R. Sousa, Frederico A. Peret, Liv B. Paula, Luiza E. Schmaltz, Cleire Pessoni, Leila Katz, Adriana Bione, Antonio C. Barbosa Lima, Edilberto A. Rocha Filho, Melania M. Amorim, Debora Leite, Ivelyne Radaci, Marilia G. Martins, Frederico Barroso, Fernando C. Oliveira Jr, Denis J. Nascimento, Cláudio S. Paiva, Moises D. Lima, Djacyr M. Freire, Roger D. Rohloff, Simone M. Rodrigues, Sergio M. Costa, Lucia C. Pfitscher, Adriana G. Luz, Daniela Guimaraes, Gustavo Lobato, Marcos Nakamura-Pereira, Eduardo Cordioli, Alessandra Peterossi, Cynthia D. Perez, Jose C. Peraçoli, Roberto A. Costa, Nelson L. Maia Filho, Jacinta P. Matias, Silvana M. Quintana, Elaine C. Moises, Fátima A. Lotufo, Luiz E. Carvalho, Carla B. Andreucci, Márcia M. Aquino, Maria H. Ohnuma, Rosiane Mattar, and Felipe F. Campanharo.