Use of Cognitive Interviews to Adapt PROMIS Measurement Items for Spanish Speakers Living with HIV

Solorio, R.; Ayala, N. C.; Paez, E.; Skalicky, A. M.; Morales, L. S.

doi:https://doi.org/10.1155/2016/8340863

AIDS Research and Treatment

On this page

Abstract Introduction Methods Results Discussion Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2016 | Article ID 8340863 | https://doi.org/10.1155/2016/8340863

Use of Cognitive Interviews to Adapt PROMIS Measurement Items for Spanish Speakers Living with HIV

R. Solorio,^1,2N. C. Ayala,¹E. Paez,³A. M. Skalicky,¹and L. S. Morales^1,4

Academic Editor: Otoniel Martinez-Maza

Received01 Dec 2015

Accepted28 Jan 2016

Published28 Feb 2016

Abstract

Purpose. To use cognitive interviewing techniques to assess comprehension of existing Patient-Reported Outcomes Measurement Information System (PROMIS) items among Latinos living with HIV and then refine items based on participant feedback. Methods. Latino monolingual Spanish speakers living with HIV () participated in cognitive interviews. Items from four PROMIS domains, including depression, anxiety, fatigue, and alcohol use, were assessed for comprehension. Audiotaped interviews and handwritten notes were subjected to content analysis to identify problems specific to each instrument for each domain. Results. The assessments from the cognitive interviews identified areas for improvement in each domain. We present data on the type of items that were difficult to comprehend and provide examples for how items were refined based on participants’ and PROMIS Statistical Coordinating Center (PSCC) feedback. Six out of 48 depression items, 7 out of the 61 anxiety items, 18 out of 42 fatigue items, and 7 out of 44 alcohol use items were found to have poor comprehension. These items were refined based on participant feedback; the items were then submitted to the PSCC for additional guidance on linguistics and grammar to improve comprehension. Conclusions. Cognitive interviews may be used to enhance comprehension of PROMIS items among Latinos.

1. Introduction

There are 51.9 million Latinos in the United States making up 17% of the total population [1]. Among all Latinos in the US, 60% are US-born and 40% are foreign-born [1]. Among all Latinos in the US, 38% are monolingual Spanish speakers [1]; given this large number, it is critical that survey measures used in research and in clinical care be available in Spanish and be understood by populations with low levels of literacy. This is especially true of measures that assess conditions that often affect people living with HIV, such as depression, anxiety, fatigue, and alcohol use.

Spanish-speaking Latinos in the US may come from Mexico, Central America, South America, and the Caribbean; among these, Mexicans are the largest subgroup, accounting for 65% of all US Latinos [2]. Because there are significant phonological, grammatical, and lexical variations in the Spanish spoken by Latino subgroups in the US, cultural adaptation, including the exclusion and substitution of words or phrases, may be needed [3]. Use of a survey instrument or item bank without careful consideration of the target population’s culture(s) and language use can result in poor item comprehension and its sequelae, including diminished responses rates and misleading results [4].

The translation of survey instruments for research is a complex process often involving multiple translators conducting independent forward and backward translations [5]. Conceptual and semantic difficulties can arise when translating idiomatic phrases to convey concepts that are unique to a particular region, country, or society. Even when a translation is created following rigorous methods, additional translations or refinements to existing translations may be necessary due to subgroup differences in culture and language [3, 6].

The purpose of this paper is to illustrate how cognitive interviewing techniques can be used to improve comprehension of PROMIS I and PROMIS II depression, anxiety, fatigue, and alcohol use items with Spanish speakers living with HIV. PROMIS is a multiyear effort funded by the National Institutes of Health to create item banks for assessing patient-reported health areas across diseases and chronic conditions (http://www.nihpromis.org). We present data on the type of items that were difficult to comprehend and present examples for how we went about refining and rewording items based on participants’ and PSCC feedback (Figures 1 and 2).

2. Cognitive Interviewing

Cognitive interview techniques were developed to improve the quality of survey data by reducing response error that can occur if questions are not interpreted in the manner they were intended [7, 8]. Grounded in cognitive psychology and information processing theory, cognitive interviewing employs the verbalization of thoughts, feelings, interpretations, and ideas that come to mind while examining survey questions [9]. In addition, respondents are asked to suggest alternate wording to increase comprehension. The use of cognitive interviewing is increasingly recognized as an important part of the formative evaluation process for questionnaire development [7, 8, 10].

Multiple approaches have been described for conducting cognitive interviews. Strategies that have been used for questionnaire design and development include the think-aloud interviews, respondent debriefing, probing techniques, and paraphrasing [9, 11]. In the think-aloud process, participants are asked to respond aloud to a specific set of questions that address the instructions provided with the questionnaire, items within the questionnaire, and response options [12]. In respondent debriefing, after a participant completes a questionnaire, an interviewer probes for specific information about what made some items difficult for the participant to comprehend [11]. With cognitive probing, participants are asked to provide information on the clarity and comprehensibility of instructions, the meaning of individual items and response choices, and the relevance of each item [11, 13]. In paraphrasing, respondents are asked to repeat a survey item in his/her own words. Often, a combination of these approaches is used [14].

3. Methods

3.1. Setting

This project constituted part of larger study that seeks to assess content validity of PROMIS depression, anxiety, fatigue, and alcohol use items in the context of clinical care for persons living with HIV who are English and Spanish speakers [15]. For this study, cognitive interviewing was used to assess comprehension of PROMIS I and PROMIS II depression, anxiety, fatigue, and alcohol use items and then refine these based on participant feedback. The adaptation of the measures described in this paper is the first step to improve comprehension of items among Latinos living with HIV who are monolingual Spanish speakers with low levels of literacy; in a future step and with a larger sample, the psychometric characteristics of the adapted instruments will need to be conducted. All procedure and consent forms were approved by the University of Washington and University of California at San Diego Institutional Review Boards.

3.2. Participants

Fifty-six Latinos who were monolingual Spanish-speaking adult men and women living with HIV were recruited to participate in this study (see Table 1). The participants were recruited in Seattle, WA, and San Diego, CA. The participants were recruited from two Centers for AIDS Research (CFAR) Network of Integrated Clinical Systems (CNICS) sites, including the University of Washington, Madison Clinic (Seattle), and the University of California at San Diego, Owen Clinic, and from Entre Hermanos (EH), a community-based organization serving the Latino LGBT community in Seattle, WA.

To be eligible, participants had to meet the following criteria: (a) be of Latino heritage; (b) speak Spanish only; (c) be 18 years of age or older; (d) live in King County, WA, or San Diego County, CA; and (e) meet criteria for diagnosis of depression, anxiety, fatigue, or alcohol use. The Patient Health Questionnaire-9 (PHQ-9) was used to determine depression severity, including suicidal ideation [16, 17]. Anxiety severity was measured with the PHQ-5 [18]. Fatigue severity was measured with the HIV symptom index [19]. Severity of alcohol use was measured using the Alcohol Use Disorders Identification Test [20].

Eligible participants were approached by study recruiters after their clinic/agency appointments or via telephone and invited to learn more about the study. Those willing had study procedures explained to them and informed written consent was obtained. Participants received a $25 incentive payment for completing the 90-minute interview. All interviews with study participants were conducted in a private meeting room at the clinic site or at the EH site. A trained, bilingual research assistant conducted the interviews in Spanish and each interview was recorded.

3.3. Instruments

We used PROMIS I and PROMIS II items for this project. Patient-reported outcome (PRO) measures use answers that patients provide to questions to produce numeric values which indicate patients’ state of wellbeing or suffering as well as their ability or lack of ability to function (these are also referred to as “items”). PROMIS measures have been developed for a wide range of chronic diseases, including HIV.

For this study, we are interested in assessing PROMIS I and PROMIS II items for comprehension among Latinos living with HIV who are monolingual Spanish speakers with low levels of literacy and then refining items, based on participant feedback. We are primarily interested in using PROMIS I and PROMIS II items from three domains, depression, anxiety, and fatigue, and one subdomain, alcohol use. We examined 195 items from PROMIS I and II. Within PROMIS I, we selected 152 existing items; these items had all been previously translated into Spanish in accordance with PROMIS network standards [3, 5, 21]. These existing items had been previously tested among Latino adults [5].

We also selected 43 items from PROMIS II using the item generation methodology recommended by the PROMIS network [21, 22]. The item set that we selected contained 10 items for depression, 16 for anxiety, 10 for fatigue, and 7 for alcohol use. We translated each of these items from English into Spanish with the help of the PROMIS Statistical Center (PSC) [3], following the Functional Assessment of Chronic Illness Therapy (FACIT) translation methodology [5].

The PROMIS I and PROMIS II item banks have been developed in part through cognitive interviewing [13], a technique that provides researchers with participant input for each PROMIS item tested [11, 12, 21]. For the PROMIS I item bank, however, some items were drawn from existing questionnaires and had not undergone cognitive interview testing and some were legacy measures that were validated using other methods [23].

3.4. Study Design and Procedures

The described work was conducted in multiple steps. All steps were completed in serial order for the depression, anxiety, fatigue, and alcohol use items.

A semistructured protocol was developed for each scale (Spanish versions). The protocol consisted primarily of cognitive probing and paraphrasing.

For all PROMIS items, testing consisted of administering the scale items and subsequently asking individuals for the meaning of selected words or expressions. When participants did not understand a question, the intended meaning of the question was explained. Participants were also asked to suggest alternative wording or phrasing to improve the clarity of questions that were difficult to understand.

Step 1 (definition of patient groupings by domain). Table 1 describes the study sample. To ensure broad representation of Latino persons living with HIV across severity levels for each domain, we defined severity levels for each domain as assessed by the clinical assessment and then selected Latinos living with HIV for each severity level to test for comprehension of PROMIS items (Table 2).

Step 2 (cognitive interviews assessed PROMIS item comprehension for anxiety, depression, fatigue, and alcohol use). Prior to the cognitive interview, participants completed a domain-specific item set (e.g., only the depression items) through paper and pencil administration. We followed this with the cognitive interview, wherein we asked open-ended questions regarding response categories, time frame, item interpretation, and domain content.
A trained RA reviewed each item stem and its response with the participant. The RA began by using the standardized question (see Table 3) for each item. The RA recorded the participant’s interpretation of the item and opinion on preferences in an Excel spreadsheet. We also asked participants about the extent to which each item contained information they considered most important to communicate with their provider regarding their experiences, whether the questions included all the issues they deemed important for their provider to know regarding their experience with depression/anxiety/fatigue/alcohol use, and what was missing from each list of items.
The cognitive interviews evaluated participants’ comprehension of the translated item. Each PROMIS English item was taken through three main steps necessary to meet FACIT methodology requirements [5]. The interviewers first took the participants through the think-aloud process and then performed the respondent debriefing. In the respondent debriefing, participants reviewed the problem areas and identified common themes and possible solutions.
Cognitive interviews were conducted with 56 Latinos living with HIV for the first round of this project, then with 28 for the second round, and then with five for the third round. Each participant was tested with PROMIS items from one domain, using one set of instructions, and one response scale.
For the first round of item testing, we tested 195 PROMIS items. This was followed by a second round of cognitive interviews on 54 items, and a third round on 21 items. All interviews were audio-recorded.

3.5. Qualitative Analysis

The audiotaped interviews were reviewed by three of the coauthors. Handwritten notes taken during the interviews were also examined to provide additional clarity and detail. The cognitive protocol that was developed to guide the interview process by identifying in advance the specific words and phrases in the surveys to be examined also facilitated data analysis by providing a structured framework to systematically review the data and to analyze the interview content. Content analysis was conducted to identify, code, and categorize primary patterns of data. Data were coded by three coauthors, using Atlas.ti. Review of the data revealed trends in participants’ level of understanding about the wording of specific items in each domain. In addition, participants’ suggestions regarding alternative wording and phrasing were examined. The same procedures were used to analyze all domains.

We extracted participant quotes from cognitive interview transcripts and sorted them with each corresponding PROMIS item to assess the level of comprehension of the concept being portrayed in each existing item. If more than one respondent had difficulty understanding the concept of an item or had an understanding that was different from what was intended to be conveyed, the researchers reviewed the item for potential wording changes. PROMIS I items had already gone through one round of cognitive interviewing with five-to-ten Spanish-speaking participants. We did not anticipate finding many PROMIS I items that produced difficulty in comprehension. Nevertheless, this step was an important one in assessing the validity of the domains for this specific population.

The interview guide we used for the cognitive interview of the PROMIS items is shown in Table 3. We documented difficulties in comprehension, misunderstanding, or uncertainty in the meaning of items in a computer spreadsheet. We also captured information extracted from the audio recordings and participants’ suggestions for translation changes for difficult-to-understand items. The full research team reviewed the final spreadsheet summary of participant feedback for each item.

We submitted items to the PROMIS Statistical Coordinating Center (PSCC) for grammatical, linguistic, and FACIT methodology review. If we proposed wording changes, we explained the reasoning for the changes, taking into account the need for the item to be comprehended by Spanish speakers from a diverse group of countries. The PSCC team reviewed our proposed modifications and evaluated their translatability beyond Spanish. Modified items required at least five additional cognitive interviews per item to be conducted. The purpose of additional cognitive interviews was to ensure the comprehension and understanding of Spanish-speaking participants of the finalized items.

In cases where problems were encountered, the participant was asked to suggest alternative wording to help clarify the question or phrasing of the instrument to better convey the intended meaning. Subsequent interviews began by using the original wording of the question. Once the first round of interviews was completed, respondent comprehension of items was assessed for each domain. We conducted a total of three rounds of cognitive interviews for each domain; after each round, item modification was conducted based on participants’ feedback; we used participants’ own words or suggestions; we then sought the advice of PROMIS PSCC and reworded items. The reworded items were then tested with the next round of interviewees. This process was meant to enhance the flow, comprehension, and overall clarity of each item in each domain. If necessary, we revised the English version of the item to maintain conceptual equivalence with the Spanish version.

4. Results

The participant sociodemographic characteristics for the first round of cognitive interviews are described in Table 1. Table 2 illustrates severity of depression, anxiety, fatigue, and alcohol use among the participants. Although participants understood most of the PROMIS items in Spanish, we did find several items that were difficult to comprehend (Table 4): seven out of 61 anxiety items, 6 out of 48 depression items, 18 out of 42 fatigue items, and 7 out of 44 alcohol use items were found to have poor comprehension.

The anxiety domain (Table 4) had several items that were difficult to comprehend. The words “atemorizado,” “súbitas,” “espasmos,” “indecisos,” and “sobresalté” are not often used in everyday speaking. Double negatives also caused confusion. In the fatigue domain, we had the same problems with difficult words used to explain the concept of fatigue. The population better understood the concept of fatigue by using the word “agotado,” rather than “exhausto.” Also, when making questions to ask “on average” it was best understood when each term was explained in detail rather than using the word “promedio” meaning on average in arithmetic terms. Most participants preferred active voice, detail, and explanation of the concept. The alcohol use subdomain showed the same difficulties with complex words such as “atareado” and “tome,” and no definite translation for the concept of “high.”

The fatigue domain had the most items that were difficult to comprehend. The main problem with comprehension for most items was use of uncommon words, sentence structure, passive voice, and in maintaining consistency of terms. In the depression domain, we found the words “estrecha,” “abrumador,” and “pesimista” to be uncommonly used and unfamiliar to participants. Some participants were confused about the difference between the words “desesperanzado” and “desesperado.” Although similar in sound, they have different meanings: “desesperado,” meaning desperate, is the word most frequently used; “desesperanzado,” meaning hopeless, is a polysyllabic word that confused participants because of its infrequent use. The concept of “emotional exhaustion” was lost when placed in a complex sentence. The use of describing the main concept helped improve comprehension of the item, but it did not help participants to respond to the item.

After we identified these changes, we tested the revised versions once more via cognitive interviewing of five additional participants. These items were refined based on participant feedback; the items were then submitted to the PSCC. After further refinement of items from the PSCC, we tested the items with five additional cognitive interviews. The PSCC translation review team accepted the recommended final changes for difficult to comprehend anxiety, depression, fatigue, and alcohol use items identified in Table 4.

5. Discussion

This study explains a process used in conducting cognitive interviews to test and define PROMIS items among Spanish speakers living with HIV. This process allowed for an improvement in comprehension of items and, thus, an improvement in measurement skills. We used an extensive translation process known as the Functional Assessment of Chronic Illness Therapy (FACIT) translation measurement system [5] and afterwards conducted cognitive interviews to identify items that may still cause problems in comprehension. The cognitive interviewing step after the translation of items ensures that items are culturally and linguistically appropriate for the target population. Using this process, we identified items participants had difficulty comprehending (Table 4). Problems with comprehension related mainly to the complexity of the words used. Words containing more than three syllables were difficult to comprehend. Words that sounded like other more commonly used words, such as the case of “desesperado” and “desesperanzado,” were difficult to understand. The term preferred for fatigue is “agotado” rather than “cansado” or “agotado.” Our study population tended to have low levels of formal education and literacy and these factors often impeded comprehension of some of the items; other studies have found that low level of literacy among Spanish speakers is associated with low levels of comprehension in other types of measures [13, 24–26].

Currently, there are resources such as the Lexile Analyzer available in Spanish to determine the reading comprehension level of each item. However, this does not take into account the need to use simple words or ensure that the developer’s concept is represented properly. Therefore, it appears that cognitive interviewing may be a necessary step in the translation of items. Although the process is time-intensive, it produces effective results. This study’s findings support previous research in which cognitive interviews allowed for the identification of language that was easier to comprehend and identified the need for the inclusion of explanatory phrases to enhance item comprehension [27]. The use of cognitive interviewing in addition to the FACIT translation process provides easier-to-comprehend items and may minimize the language barriers between providers and patients.

Translating health care instruments to other languages is not a simple process. It is often necessary to conduct cognitive interviews with participants from a target community and with a particular condition in order to achieve adequate item comprehension among study participants. Based on the results of cognitive testing, an item may have to be worded differently, use words that are not direct translations, or include phrases that may clarify the content of the item.

Limitations of this study need to be considered. The study is limited by a modest sample size consisting of mostly middle-aged men recruited in two west coast cities: Seattle, Washington, and San Diego, California. Future studies should seek to (1) have more gender diversity (include more women), (2) have age diversity, and (3) include participants from a wider array of literacy levels (include more native Spanish speakers with a higher literacy level). Due to the location of this study, most of the Spanish-speaking participants were of Mexican descent; future studies would benefit from including participants from other Latin American countries.

This study used cognitive interviews to evaluate PROMIS item comprehension for depression, anxiety, fatigue, and alcohol use among Spanish speakers living with HIV. Using this process allowed for improvements in comprehension of items for these domains. For the next step of this research, a need exists to use quantitative methods to ensure measurement equivalence between the Spanish and English versions of the PROMIS item banks, as done in previous studies [28, 29]. Such research will require the collection of a large number of completed PROMIS items in Spanish and English.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This research was funded by a cooperative agreement awarded to the University of Washington (Principal Investigators: D. Patrick, H. Crane, and P. Crane) from the National Institute of Allergy and Infectious Diseases (NIAID) and National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS) (Grant no. U01 AR 057954). Support was also provided by the National Institute of Allergy and Infectious Diseases (NIAID), University of Washington Center for AIDS Research (Grant no. P30 AI027757), and CNICS (R24 AI067039) and National Institute of Alcohol Abuse and Alcoholism (NIAAA) (ARCH Grants U01 AA020802, U01 AA020793, and U24 AA020801).

References

Pew Research Center, A statistical portrait of U.S. Hispanics, 2013, http://www.pewhispanic.org/2013/02/15/hispanic-population-trends/ph_13-01-23_ss_hispanics1/.
Pew Research Center, “A Demographic Portrait of Mexican-Origin Hispanics in the United States,” 2013, http://www.pewhispanic.org/2013/05/01/a-demographic-portrait-of-mexican-origin-hispanics-in-the-united-states/.
View at: Google Scholar
C. Acquadro, K. Conway, A. Hareendran, and N. Aaronson, “Literature review of methods to translate health-related quality of life questionnaires for use in multinational clinical trials,” Value in Health, vol. 11, no. 3, pp. 509–521, 2008.
View at: Publisher Site | Google Scholar
G. Martinez, B. V. Marín, and A. Schoua-Glusberg, “Translating from English to Spanish: the 2002 National Survey of Family Growth,” Hispanic Journal of Behavioral Sciences, vol. 28, no. 4, pp. 531–545, 2006.
View at: Publisher Site | Google Scholar
S. L. Eremenco, D. Cella, and B. J. Arnold, “A comprehensive method for the translation and cross-cultural validation of health status questionnaires,” Evaluation & the Health Professions, vol. 28, no. 2, pp. 212–232, 2005.
View at: Publisher Site | Google Scholar
D. Wild, A. Grove, M. Martin et al., “Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation,” Value in Health, vol. 8, no. 2, pp. 94–104, 2005.
View at: Publisher Site | Google Scholar
J. B. Jobe and D. J. Mingay, “Cognitive research improves questionnaires,” American Journal of Public Health, vol. 79, no. 8, pp. 1053–1055, 1989.
View at: Publisher Site | Google Scholar
K. Alaimo, C. M. Olson, and E. A. Frongillo, “Importance of cognitive testing for survey items: an example from food security questionnaires,” Journal of Nutrition Education, vol. 31, no. 5, pp. 269–275, 1999.
View at: Publisher Site | Google Scholar
G. B. Willis, Cognitive Interviewing and Questionnaire Design: A Training Manual, vol. 7 of Working Paper Series, Office of Research and Methodology, National Center for Health Statistics, Hyattsville, Md, USA, 1994.
S. Sudman, N. M. Bradburn, and N. Schwartz, Eds., Thinking about Answers: The Application of Cognitive Processes to Survey Methodology, Jossey-Bass, San Francisco, Calif, USA, 1996.
D. E. Irwin, J. W. Varni, K. Yeatts, and D. A. DeWalt, “Cognitive interviewing methodology in the development of a pediatric item bank: a patient reported outcomes measurement information system (PROMIS) study,” Health and Quality of Life Outcomes, vol. 7, no. 3, pp. 1–10, 2009.
View at: Publisher Site | Google Scholar
G. B. Willis, Cognitive Interviewing: A Tool for Improving Questionnaire Design, Sage Publications, 2005.
A. A. García, “Cognitive interviews to test and refine questionnaires,” Public Health Nursing, vol. 28, no. 5, pp. 444–450, 2011.
View at: Publisher Site | Google Scholar
E. T. Carbone, M. K. Campbell, and L. Honess-Morreale, “Use of cognitive interview techniques in the development of nutrition surveys and interactive nutrition messages for low-income populations,” Journal of the American Dietetic Association, vol. 102, no. 5, pp. 690–696, 2002.
View at: Publisher Site | Google Scholar
T. C. Edwards, R. J. Fredericksen, H. M. Crane et al., “Content validity of patient-reported outcomes measurement information system (PROMIS) items in the context of HIV clinical care,” Quality of Life Research, vol. 25, no. 2, pp. 293–302, 2016.
View at: Publisher Site | Google Scholar
K. Kroenke, R. L. Spitzer, and J. B. W. Williams, “The PHQ-9: validity of a brief depression severity measure,” Journal of General Internal Medicine, vol. 16, no. 9, pp. 606–613, 2001.
View at: Publisher Site | Google Scholar
S. T. Lawrence, J. H. Willig, H. M. Crane et al., “Routine, self-administered, touch-screen, computer-based suicidal ideation assessment linked to automated response team notification in an HIV primary care setting,” Clinical Infectious Diseases, vol. 50, no. 8, pp. 1165–1173, 2010.
View at: Publisher Site | Google Scholar
R. L. Spitzer, K. Kroenke, and J. B. Williams, “Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire,” Journal of the American Medical Association, vol. 282, no. 18, pp. 1737–1744, 1999.
View at: Publisher Site | Google Scholar
C. J. Portillo, W. L. Holzemer, and F.-Y. Chou, “HIV symptoms,” Annual Review of Nursing Research, vol. 25, pp. 259–291, 2007.
View at: Google Scholar
K. Bush, D. R. Kivlahan, M. B. McDonell, S. D. Fihn, and K. A. Bradley, “The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking,” Archives of Internal Medicine, vol. 158, no. 16, pp. 1789–1795, 1998.
View at: Publisher Site | Google Scholar
D. A. DeWalt, N. Rothrock, S. Yount, and A. A. Stone, “Evaluation of item candidates: the PROMIS qualitative item review,” Medical Care, vol. 45, no. 5, supplement 1, pp. S12–S21, 2007.
View at: Publisher Site | Google Scholar
P. A. Pilkonis, S. W. Choi, S. P. Reise, A. M. Stover, W. T. Riley, and D. Cella, “Item banks for measuring emotional distress from the patient-reported outcomes measurement information system (PROMIS): depression, anxiety, and anger,” Assessment, vol. 18, no. 3, pp. 263–283, 2011.
View at: Publisher Site | Google Scholar
C. Christodoulou, D. U. Junghaenel, D. A. DeWalt, N. Rothrock, and A. A. Stone, “Cognitive interviewing in the evaluation of fatigue items: results from the patient-reported outcomes measurement information system (PROMIS),” Quality of Life Research, vol. 17, no. 10, pp. 1239–1246, 2008.
View at: Publisher Site | Google Scholar
M. George, R. Pinilla, S. Abboud, J. A. Shea, and C. Rand, “Innovative use of a standardized debriefing guide to assist in the development of a research questionnaire with low literacy demands,” Applied Nursing Research, vol. 26, no. 3, pp. 139–142, 2013.
View at: Publisher Site | Google Scholar
J. G. Grzywacz, T. Alterman, C. Muntaner, S. Gabbard, J. Nakamoto, and D. J. Carroll, “Measuring job characteristics and mental health among latino farmworkers: results from cognitive testing,” Journal of Immigrant and Minority Health, vol. 11, no. 2, pp. 131–138, 2009.
View at: Publisher Site | Google Scholar
M. C. Rosal, E. T. Carbone, and K. V. Goins, “Use of cognitive interviewing to adapt measurement instruments for low-literate Hispanics,” The Diabetes Educator, vol. 29, no. 6, pp. 1006–1017, 2003.
View at: Publisher Site | Google Scholar
G. I. Lopez, M. Figueroa, S. E. Connor, and S. L. Maliski, “Translation barriers in conducting qualitative research with Spanish speakers,” Qualitative Health Research, vol. 18, no. 12, pp. 1729–1737, 2008.
View at: Publisher Site | Google Scholar
K. A. Hepner, L. S. Morales, R. D. Hays, M. O. Edelen, and J. Miranda, “Evaluating differential item functioning of the PRIME-MD mood module among impoverished black and white women in primary care,” Women's Health Issues, vol. 18, no. 1, pp. 53–61, 2008.
View at: Publisher Site | Google Scholar
J. A. Teresi, A. L. Stewart, L. S. Morales, and S. M. Stahl, “Measurement in a multi-ethnic society: overview to the special issue,” Medical Care, vol. 44, no. 11, supplement 3, pp. S3–S4, 2006.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2016 R. Solorio et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

1769

Downloads

1141

Citations