About this Journal Submit a Manuscript Table of Contents
Epidemiology Research International
Volume 2013 (2013), Article ID 490906, 6 pages
http://dx.doi.org/10.1155/2013/490906
Research Article

Validation of the ISAAC Standardized Questionnaire Used by Schoolchildren from Mexicali, Baja California, Mexico

1Instituto Nacional de Salud Pública, Avenida Universidad No. 655, Colonia Santa María Ahuacatitlán, 62100 Cuernavaca, MOR, Mexico
2Universidad Autónoma de Chihuahua, Avenida Escorza No. 900, Zona Centro, 31000 Chihuahua, CHIH, Mexico
3Universidad Autónoma Metropolitana—Iztapalapa, Avenida San Rafael Atlixco 186, Colonia Vicentina, Delegación Iztapalapa, 09340 Mexico City, DF, Mexico

Received 11 July 2013; Revised 18 October 2013; Accepted 19 October 2013

Academic Editor: David Christiani

Copyright © 2013 Karla Gorozave-Car et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background. In Mexico, several studies have been conducted under the ISAAC methodology; nevertheless, no validation studies of the ISAAC questionnaire based on objective clinical testing in our country have been published. Aims of the Study. To validate the ISAAC questionnaire, used in a study of prevalence of allergic diseases, based on medical, respiratory, and allergic evaluations of schoolchildren being 11 to 16 years old in Mexicali, Mexico. Material and Methods. We conducted a cross-sectional study to validate the ISAAC questionnaire through the generation of an index (considered as gold standard) using pre- and postbronchodilator spirometry test, exhaled nitric oxide measurements, and atopic evaluations. 114 schoolchildren were included (23 asthmatics with respiratory symptoms and 91 nonasthmatics without respiratory symptoms) and we evaluated the sensitivity, specificity, and positive and negative predictive value of the questionnaire using discriminant analysis. Results. We observed sensitivity of 35.2% and specificity of 93.3% and the positive and negative predictive values were 82.6% and 61.5%, respectively. Conclusions. Our findings indicate that the ISAAC Mexican version questionnaire was less sensitive and more specific when compared to the gold standard; however, it is adequate and able to discriminate children with and without asthma and a useful tool to use in epidemiological studies.

1. Introduction

Asthma, allergic rhinitis, and atopic dermatitis are three of the allergic diseases that most affect children worldwide [13]. Asthma is a multifactor disease characterized by chronic inflammation of the airways and a reversible obstruction of air flow. Concern about these diseases has arisen because of their increase over the years, with prevalence and mortality rates having increased worldwide [4].

The International Study of Asthma and Allergies in Childhood (ISAAC) methodology was established in 1990s [4]. This methodology is a useful and practical strategy to most effectively identify these types of affections in children. It has been used in many countries where different languages are spoken, as well as in countries where the language does not have a way to describe asthma symptoms.

The ISAAC is designed with three phases. Phase I is designed to assess the basic prevalence and severity of asthma and other allergic diseases by administering target questionnaires. Phase II investigates possible risk factors related to these diseases and those suggested in phase I. Phase III repeats phase I to assess the prevalence tendency of these diseases and to examine the hypothesis established by conclusions from phase I [5].

The ISAAC protocol maximizes the value of epidemiological research involving transversal studies by providing a standard tool to enable comparisons with other populations. Therefore, it is advisable to establish a mechanism to validate the tool in order to assure the quality of information and support the findings. Though ISAAC validation studies have been conducted worldwide, some of these studies have been performed without considering objective clinical tests, contrasting only the responses of parents with those of children or using other tests for validation [69]. On the other hand, different studies using this methodology have been conducted in Mexico [1012]; however, there are no published studies that validate the ISAAC questionnaire using objective clinical tests. Thus, the purpose of the present study is to validate the ISAAC questionnaire used in a study of prevalence and risk factors in Mexicali, Baja California, Mexico, conducted in 2004, measuring the tool’s sensitivity, specificity, and concordance by contrasting previous results with clinical tests for lung function and other markers in children.

2. Material and Methods

2.1. Design of Study Population

To validate the questionnaire using objective clinical tests, a convenience sample of 114 schoolchildren from 11 to 16 years old was chosen from a total of 5287 schoolchildren who participated in the transversal study of allergic diseases during 2004 in Mexicali, Baja California, Mexico.

2.2. Data Collection

An ISAAC questionnaire was used to collect data on sociodemographic variables, past health history, and respiratory health. Each participant was assigned a personal number and was classified as asthmatic or nonasthmatic, according to information reported on the questionnaires, which were self-administered by the parents. All procedures were explained to the parents, who signed an informed consent form. The children also gave their informed assent. The study protocol was reviewed and approved by ethics committees of the National Institute of Public Health. Of 114 children selected, 23 reported a medical diagnosis of asthma and have respiratory symptoms in the last 12 months and 91 did not report symptoms in the last 12 months or medical diagnosis of asthma.

2.3. Mechanisms Used to Validate the Questionnaire

The questionnaires asked two questions. Has a doctor ever diagnosed your child as asthmatic? And have you had wheezing or whistling in the chest in the last 12 months? After children were identified as asthmatic or non-asthmatic based on the answers, different clinical tests were conducted to obtain a gold standard index to validate both questions. The medical diagnosis of asthma reported in the ISAAC questionnaire was validated based on the following tests.

2.3.1. Reversibility Test for Lung Function

A spirometry test (Easy One, Zurich, Switzerland) was administered to each participant in accordance with American Thoracic Society 1994 criteria [13]. After an acceptable and reproducible test was obtained, a 200 μg dose of albutherol (a short-acting B2 adrenergic receptor agonist) was administered. After 15 minutes, a second spirometry was conducted to assess the change in lung function. To determine the bronchodilator response we use the degree of reversibility between prebronchodilator and postbronchodilator of FEV1 (≥12% and ≥200 mL) according to the GINA criteria [14]; however, we also calculated the percentage of change between prebronchodilator and post bronchodilator test, quantified as a percent relative change value and derived from the number of schoolchildren who had a positive reversibility test using both criteria was almost similar, we decided to use this percent change value for the generation of the gold standard index.

2.3.2. Exhaled Nitric Oxide (eNO) Concentrations

eNO concentrations were measured for each child during visits to the health center and according to American Thoracic Society recommendations [15]. As recommended, the child was asked to be seated for five minutes before and during the test. All tests were conducted indoors to reduce possible contamination from environmental nitric oxide. The exhaled nitric oxide was instantly measured by chemiluminescence using a portable NIOX MINO, with a measurement range from 5 to 300 ppb and a detection limit of 5 ppb [16]. For each participant, the two best tests were selected and the average concentration for those tests was considered.

2.3.3. Skin Test

A skin test was conducted using multitest testing [17, 18]. This enabled identifying a specific allergen response to standard allergens (Dermatophagoides farinae, grass, trees, dogs, cats, and cockroaches) glycerinated in a concentration of 1 : 10 (weight/volume) with negative control and positive to histamine (1 : 100). The test was performed by trained medical personnel. A diameter of ≤3 mm for the allergens tested was considered to be a null or negative result. A positive skin test was determined by a wheel greater or equal to 3 mm or equal to the same diameter of the wheel produced by the histamine control, after 15 minutes of placing the skin test. We determined atopy status as having at least one reaction to an antigen tested.

2.4. Statistical Analysis

First, the questionnaire’s reliability was determined by assessing the inner consistency and concordance of each answer using kappa statistics. Second, the quality and validity of the medical diagnoses of asthma were assessed by comparing answers with the index generated according the clinical tests, with the latter being considered the gold standard. The rate was obtained by discriminant analysis [19], which is a multivariate statistical technique that analyzes significant differences among groups of individuals versus variables measured in those groups. If there are any differences, it tries to explain them and provides systematic classification procedures for new observations with unknown origins in one of the groups studied. In that respect, we considered 3 classification variables: (1) percentage change in FEV1 or reversibility test, (2) average concentration of FeNO levels with logarithmic units, and (3) atopic condition, with 2 comparison groups (asthmatic individuals and non-asthmatic individuals) in a sample size of 114 children (23 asthmatic individuals and 91 nonasthmatics individuals). Additionally and with the aim of comparing the results obtained, we also analyzed the information considering only the reversibility test according to GINA criteria as the gold standard and tested again the sensitivity and specificity of the ISAAC questionnaire. Finally, a new classification was obtained based on a constructed discriminant function, a canonic correlation, and the transformed lambda value. ANOVA tests were used as validation parameters for the classification obtained.

3. Results

Twenty-three (20.2%) of the 114 children participating in the research reported in the questionnaire having a medical diagnosis of asthma, with a higher percentage (60.9%) being female, 12 to 14 years old. A total of 79.8% of schoolchildren participating in the research did not report symptoms and were considered as non-asthmatics, 59.1% of which were female, from 11 to 16 years old.

Table 1 presents the results of the descriptive statistics and main outcomes, where it can be seen that the parameters for lung function do not differ between the two groups of individuals studied (asthmatic and non-asthmatic participants), although the FEV1 reversibility percentage was slightly greater for the asthmatic group ( ). Average FeNO concentrations were higher in the group of individuals that did not report a clinical diagnosis of asthma. Differences were not statistically significant.

tab1
Table 1: Descriptive statistics and main outcomes of the study population according to the information obtained by ISAAC questionnaire and clinical tests.

Positive skin tests were observed in 48% of the total sample, with a higher prevalence in the asthmatic group versus non-asthmatic schoolchildren (82.6% versus 39.6%, resp., ) (Table 1). The allergens that provoked a positive response with greater frequency were different between the groups. The most frequent allergens for asthmatic children were trees and dermatophagoides farinae (14% and 9%, resp.) whereas for non-asthmatic children they were cockroaches and cat and grass allergens (23% and 16%, resp.) (Table 2).

tab2
Table 2: Frequency of positive skin prick tests according to most common sensitivities allergens tested in the study population.

A new classification of participants was obtained as a result of the gold standard. Fifty-four of the 114 children were classified as asthmatic and 60 as non-asthmatic, as shown in Table 3. Nineteen of the 23 children who reported a diagnosis of asthma on the questionnaires were classified by the gold standard (sensibility = 35.2% and positive predictive value = 82.6%) and 56 of the 60 schoolchildren classified as non-asthmatic by the gold standard were identified correctly as non-asthmatic individuals by the questionnaire (specificity = 93.3% and negative predictive value = 61.5%). On the other hand, when we use the reversibility test considering GINA guidelines only as gold standard, the sensitivity and specificity were 22% and 81%, respectively (Table 4).

tab3
Table 3: Classification of the study population according to gold standard (percentage change in FEV1, average concentration of FeNO levels, and atopic condition).
tab4
Table 4: Classification of the study population considering the reversibility test as gold standard.

Based on the data obtained, we can observe that the probability that the questionnaire correctly classifies a child as asthmatic (sensibility) was lower than the probability of correctly classifying him or her as non-asthmatic (specificity). Table 5 shows the characteristics of individuals according to the gold standard classification.

tab5
Table 5: Descriptive statistics and main outcomes of the study population according to the classification obtained by gold standard index.

With the ROC analysis, a probability of 64% was observed for asthmatic and non-asthmatic individuals and both groups were correctly classified by the discriminant analysis. Concordance (kappa test) for both tests was 65.8.

4. Discussion

The ISAAC study and its phases have been developed in our country [1012]; the prevalence and risk factors associated with allergic diseases among schoolchildren are known. Nevertheless, to our knowledge the present study is the first conducted in our country to validate the ISAAC questionnaire through objective clinical testing.

Our results indicate that 54 of the 114 children participating in the research were classified as asthmatic according to the gold standard (percentage change in FEV1, average of FeNO concentrations, and skin test positive). Meanwhile, according to the questionnaires, 23 individuals reported having an asthmatic condition, which represents a very low sensibility (35.2%) of the instrument compared to other studies. This could indicate that interpreting all recent symptoms in the questionnaire was not adequate and may result in subestimating the prevalence of childhood asthma; that is, the 35 schoolchildren classified as negative false by the questionnaire are probably those who are not familiar with the term of wheezing and did not correctly report the respiratory noises and sensations, and the 4 schoolchildren who were found as positive false had some respiratory disease or respiratory symptoms when the questionnaire was applied. On the other hand, we observed that our gold standard classified 60 children as non-asthmatics versus 91 who were reported as healthy by the questionnaire, resulting in specificity of 93.3%. Similarly, the positive predictive value of the questionnaire was high (82.6%). These results are very important in public health because this tool can be used with confidence to identify the health situation in communities.

With regard to validation studies conducted worldwide, specifically among the Latino school population, it is important to point out that, unlike the majority of studies published, our research is one of the first studies to use several clinical tests (spirometry with bronchodilatation, measurement of FeNO levels, and skin tests). Other studies validate questionnaires mainly by contrasting responses of parents with those of children or by exclusively using spirometry.

Research conducted in a Latino population in Los Angeles, CA, USA, [20] evaluated the validation of responses to a questionnaire using spirometry and children’s responses. The study compared parents’ responses with children’s responses regarding asthma diagnosis. The more accurate report was obtained from children, for which 47% of children reported as asthmatic in the questionnaire were observed to be positive by at least one of four spirometries conducted.

Another validation study was conducted with Spanish speakers in Spain [21]. This study demonstrated how much variability an event can present, such as bronchial hyperreactivity, when compared with other studies performed in the same country.

In addition, a multicentric validation study conducted in 40 countries and 90 centers was performed by comparing responses before and after children were shown a video to identify respiratory diseases. This study found that teenagers interpreted asthmatic symptoms differently. In 20 centers there was concordance of ≥0.4, indicating that the questionnaires should obtain the same results when administered using different modalities, at different times, and by different observers [22]. Thus, total validation of the questionnaire is a strategy that more greatly guarantees obtaining reliable data. In the present research, the same questionnaire was administered to an additional subsample at two different but consecutive moments. According to kappa statistics, that verified the reliability and quality of the instrument, we observed a questionnaire reproducibility of 66%.

With regard to the classification of asthma in this study using the ISAAC questionnaire, it is important to mention that this was obtained mainly with two questions. Has a doctor ever diagnosed your child as asthmatic? And during the last 12 months has your child observed wheezing? Therefore, we are probably excluding children with wheezing during another period of life, which could indicate an asthmatic condition.

A limitation of this study may be that the gold standard was based on three different tests; therefore the sensitivity of the questionnaire could be affected. For this reason, additionally we used only the reversibility test considering the GINA guidelines [14] as gold standard and the sensitivity of the ISAAC questionnaire was low; Jenkins MA and collaborators found sensitivity of 0.39 (0.21–0.61) in adults and 0.54 (0.48–0.67) in children using the bronchial hyperresponsiveness (BHR) as gold standard for asthma, noting that the BHR was highly specific but not sensitive [23]. Nevertheless, the literature has reported that the tests that we used can be used for the diagnosis of asthma. Therefore, we believe that, in this case, lack of affirmative criteria to diagnose asthma would underestimate the sensibility of questionnaire. Finally, our sample size is small which may have limited the power of our study.

5. Conclusions

In summary, our findings indicate that the ISAAC Mexican version questionnaire was less sensitive and more specific when compared to the gold standard; however it is adequate and able to discriminate children with and without asthma and a useful tool to use in epidemiological studies. Nevertheless, to improve the consistency and quality of information, measurement instruments need to be continuously validated because, though they are the standardized ones, it is important to remember that language, culture, and accessibility can affect the results. This will enable verifying the authenticity of the data and assuring that the efforts of governmental, social, and scientific institutions benefit high-risk populations.

Conflict of Interests

The authors declare they have no competing conflict of interests.

Acknowledgments

We want to extend our gratitude to the schoolchildren that participated in the study. This research was supported by a research grant from the National Center for Environmental Health, Center for Disease Control and Prevention, Atlanta, GA.

References

  1. A. Arnedo-Penaa, L. García-Marcosb, G. García Hernándezc et al., “Tendencia temporal y variaciones geográficas de la prevalencia de síntomas de rinitis alérgica en escolares de 6-7 años de ocho áreas españolas, según ISAAC,” Anales de Pediatría, vol. 62, no. 3, pp. 229–236, 2005. View at Publisher · View at Google Scholar
  2. S. T. Holgate, “The epidemic of allergy and asthma,” Nature, vol. 402, no. 6760, pp. B2–B4, 1999. View at Scopus
  3. R. L. Smyth, “Asthma: a major pediatric health issue,” Respiratory Research, vol. 3, supplement 1, pp. S3–S7, 2002. View at Publisher · View at Google Scholar
  4. The International Study of Asthma and Allergies in Childhood Steering Committee, “Worldwide variations in the prevalence of asthma symptoms: the International Study of Asthma and Allergies in Childhood (ISAAC),” European Respiratory Journal, vol. 12, no. 2, pp. 315–335, 1998. View at Scopus
  5. Coordination Committee for the International Study of Asthma and Allergies in Childhood (ISAAC), “Manual for the International Study for Asthma and Allergies in Childhood (ISAAC),” InBochum, FRG, 1992, http://isaac.auckland.ac.nz/phases/phaseone/phaseonemanual.pdf.
  6. E. Yamada, A. T. Vanna, C. K. Naspitz, and D. Solé, “International study of asthma and allergies in childhood (ISAAC): validation of the written questionnaire (eczema component) and prevalence of atopic eczema among Brazilian children,” Journal of Investigational Allergology and Clinical Immunology, vol. 12, no. 1, pp. 34–41, 2002. View at Scopus
  7. A. T. Vanna, E. Yamada, L. K. Arruda, C. K. Naspitz, and D. Solé, “International study of asthma and allergies in childhood: validation of the rhinitis symptom questionnaire and prevalence of rhinitis in schoolchildren in São Paulo, Brazil,” Pediatric Allergy and Immunology, vol. 12, no. 2, pp. 95–101, 2001. View at Publisher · View at Google Scholar · View at Scopus
  8. P. G. Gibson, R. Henry, S. Shah et al., “Validation of the ISAAC video questionnaire (AVQ3.0) in adolescents from a mixed ethnic background,” Clinical and Experimental Allergy, vol. 30, no. 8, pp. 1181–1187, 2000. View at Publisher · View at Google Scholar · View at Scopus
  9. H. H. Chan, A. Pei, C. Van Krevel, G. W. K. Wong, and C. K. W. Lai, “Validation of the Chinese translated version of ISAAC core questions for atopic eczema,” Clinical and Experimental Allergy, vol. 31, no. 6, pp. 903–907, 2001. View at Publisher · View at Google Scholar · View at Scopus
  10. A. Barraza-Villarreal, L. H. Sanín-Aguirre, M. M. Téllez-Rojo, M. Lacasaña-Navarro, and I. Romieu, “Prevalencia de asma y otras enfermedades alérgicas en niños escolares de Ciudad Juárez, Chihuahua,” Salud Pública de México, vol. 43, no. 5, pp. 433–443, 2001. View at Publisher · View at Google Scholar
  11. B. Del-Rio-Navarro, A. Berber, V. Blandón-Vijil et al., “Identification of asthma risk factors in Mexico City in an International Study of Asthma and Allergy in Childhood survey,” Allergy and Asthma Proceedings, vol. 27, no. 4, pp. 325–333, 2006. View at Publisher · View at Google Scholar · View at Scopus
  12. M. I. Tatto-Cano, L. H. Sanín-Aguirre, V. González, S. Ruiz-Velasco, and I. Romieu, “Prevalencia del asma, rinitis, y ezcema en niños escolares de Cuernavaca, Morelos,” Salud Pública de México, vol. 39, pp. 497–506, 1997.
  13. “Standardization of spirometry, 1994 update. American Thoracic Society,” American Journal of Respiratory and Critical Care Medicine, vol. 152, no. 3, pp. 1107–1136, 1995.
  14. “Global Initiative for Asthma: global strategies for asthma management and prevention,” Global Inicitative for Asthma, 2011, http://www.ginasthma.org/guidelines-gina-report-global-strategy-for-asthma.html.
  15. P. E. Silkoff, “Recommendations for standardized procedures for the online and offline measurement of exhaled lower respiratory nitric oxide and nasal nitric oxide in adults and children-1999,” American Journal of Respiratory and Critical Care Medicine, vol. 160, no. 6, pp. 2104–2117, 1999. View at Scopus
  16. K. Alving, C. Janson, and L. Nordvall, “Performance of a new hand-held device for exhaled nitric oxide measurement in adults and children,” Respiratory Research, vol. 7, article 67, 2006. View at Publisher · View at Google Scholar · View at Scopus
  17. K. Julge, S. Meriste, A. Kemp, and B. Björkstén, “Atopic allergy and delayed type hypersensitivity in Estonian children,” Clinical and Experimental Allergy, vol. 32, no. 10, pp. 1420–1423, 2002. View at Publisher · View at Google Scholar · View at Scopus
  18. J. P. Laaban, B. Kouchakji, M. F. Dore, E. Orvoen-Frija, P. David, and J. Rochemaure, “Nutritional status of patients with chronic obstructive pulmonary disease and acute respiratory failure,” Chest, vol. 103, no. 5, pp. 1362–1368, 1993. View at Scopus
  19. J. E. Dallas, Métodos multivariados aplicados al análisis de datos, International Thomson Editores, 2002.
  20. M. Lara, N. Duan, C. Sherbourne et al., “Differences between child and parent reports of symptoms among Latino children with asthma,” Pediatrics, vol. 102, no. 6, article E68, 1998. View at Scopus
  21. R. M. Busquets, J. M. Antó, J. Sunyer, N. Sancho, and O. Vall, “Prevalence of asthma-related symptoms and bronchial responsiveness to exercise in children aged 13-14 yrs in Barcelona, Spain,” European Respiratory Journal, vol. 9, no. 10, pp. 2094–2098, 1996. View at Scopus
  22. S. J. Hong, S. W. Kim, J. W. Oh et al., “The validity of the ISAAC written questionnaire and the ISAAC Video questionnaire (AVQ 3.0) for predicting asthma associated with bronchial hyperreactivity in a group of 13-14 year old Korean schoolchildren,” Journal of Korean Medical Science, vol. 18, no. 1, pp. 48–52, 2003. View at Scopus
  23. M. A. Jenkins, J. R. Clarke, J. B. Carlin et al., “Validation of questionnaire and bronchial hyperresponsiveness against respiratory physician assessment in the diagnosis of asthma,” International Journal of Epidemiology, vol. 25, no. 3, pp. 609–616, 1996. View at Publisher · View at Google Scholar · View at Scopus