Abstract

The aim of this study was to examine the feasibility of problem cannabis use screening instruments administration within wide school surveys, their psychometric properties, overlaps, and relationships with other variables. Students from 7 Spanish regions, aged 14–18, who attended secondary schools were sampled by two-stage cluster sampling (net sample 14,589). Standardized, anonymous questionnaire including DSM-IV cannabis abuse criteria, Cannabis Abuse Screening Test (CAST), and Severity of Dependence Scale (SDS) was self-completed with paper and pencil in the selected classrooms. Data was analysed using classical psychometric theory, bivariate tests, and multinomial logistic regression analysis. Not responding to instruments’ items (10.5–12.3%) was associated with reporting less frequent cannabis use. The instruments overlapped partially, with 16.1% of positives being positive on all three. SDS was more likely to identify younger users with lower frequency of use who thought habitual cannabis use posed a considerable problem. CAST positivity was associated with frequent cannabis use and related problems. It is feasible to use short psychometric scales in wide school surveys, but one must carefully choose the screening instrument, as different instruments identify different groups of users. These may correspond to different types of problematic cannabis use; however, measurement bias seems to play a role too.

1. Introduction

Cannabis is the most widely used illicit drug worldwide [1]. Spain is currently the country with the highest last month prevalence of cannabis use among those aged 15–24 in Europe, with consistently high figures since 2005 (17.2–18.6%, [2]) and with the highest prevalence of regular (3–39 times within the past 12 months) and heavy (40 times or more within the past 12 months) use according to 2009/2010 HBSC (Health Behaviour in School-Aged Children, [3]) survey of 15-16-year olds [4].

A range of negative effects of regular cannabis use on adolescent health and psychosocial status have been identified, including adverse effects on psychosocial development and mental health (including developing psychotic symptoms), decrease in academic performance (which can lead to academic failure), and other negative outcomes later in life [5–9]. Since 2009, cannabis as a primary drug has overtaken heroin among those requesting drug treatment for the first time in their lives in the European Union. Its mention as a primary drug further increased in 2010 (accounting for more than 100 000 treatment demands). In this year, 76% of reported treatment entrants aged 15–19 years cited cannabis as their primary drug as did 86% of those younger than 15 years [10].

Chiefly from the perspective of planning public health interventions, like secondary/targeted drug prevention or drug treatment, there is considerable value in implementing screening instruments that are capable of detecting (probable) cannabis dependence or problematic use at the population level. Population or youth surveys in several European countries have recently started to incorporate screening instruments to measure cannabis dependence or problematic cannabis use [11–17]. In order to better understand these instruments, the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) has recommended methodological studies [18].

In the context of general population or school surveys, there is often shortage of space for inclusion of new items, as this may considerably increase the cost of the survey. Moreover, studies confirm that well-constructed short screening scales can be strong predictors of the same results found in more lengthy instruments or interviews [19, 20].

There are few epidemiological studies conducted in Europe to date applying psychometric scales to assess problematic forms of cannabis use in the general population of adolescents [13, 15–17]. Moreover, the relationship between different instruments has not been systematically studied.

The aims of the present study were (1) to explore the feasibility of administration of short cannabis instruments within a wide school survey of Spanish adolescents, (2) to obtain insight into the psychometric performance of CAST, SDS, and DSM-IV cannabis abuse criteria, (3) to find out whether the instruments identify the same groups of cannabis users by exploring their overlaps and associations of positivity on them with key variables.

2. Methods

Data collection was carried out on a sample of Spanish students within a biennial national school survey conducted by the Spanish National Programme on Drugs (DGPNSD) in 2006.

2.1. Reference Population and Sampling Frame

The reference population for this survey was students from 7 Spanish regions, selected by convenience, between the ages of 14 and 18 years who attended secondary schools. It was estimated that 82% of all Spanish youth of this age was schooled.

2.2. Sample

Two-stage cluster sampling was used, by randomly selecting 322 schools as first stage units and 644 classrooms as second stage units. In order to select the schools, the sampling frame was firstly stratified by autonomous region (seven strata) and school type (two strata, public and private schools). 9.6% of the registered students did not attend class on the date and time of the survey. 23.6% of schools and 3.5% of the classrooms were replaced because of refusal. 0.2% of students refused to participate in the study. The final sample consisted of 14,589 school attendees born between 1987 and 1992. Mean age was 15.6 years, and 46% were boys. The numbers of cannabis users in the studied sample were: lifetime use n = 5002, last 12 months use n = 4089, and last 30 days use n = 2735.

2.3. Consent to Participate in the Study

The participation of students in the survey was based on a passive parental consent. Parents’ associations of schools, school administrations, and regional educational authorities were informed about the nature, objectives, and characteristics of the study. All selected students were informed that participation in the survey was voluntary. To ensure confidentiality, questionnaires were anonymous.

2.4. Data Collection Tool

A standardized questionnaire was self-completed with paper and pencil by all of the students in the selected classrooms during a normal class (45–60 minutes), in the presence of the teacher who remained at the lectern throughout. The questionnaire included questions related to sociodemographic characteristics, drug use, perception of risk of different drug use behaviours, leisure time, level of perceived availability of different psychoactive drugs, social and health-related problems, sources of information on drugs, drug use by friends and classmates, and the attitudes of parents toward drug use. Frequency of cannabis use in the last 12 months and in the last 30 days was measured in categories of 0, 1, 2, 3, 4-5, 6–9, 10–19, 20–39, and 40 or more days (in the case of last 12 months use). Alcohol bingeing in the past 30 days was defined as drinking 5 or more glasses of alcohol in one single occasion.

Short cannabis disorders screening instruments were sought, which have been tested in similar settings [21]. Three, the shortest, were included in the questionnaire. All three were previously validated for self-administration. (1)DSM-IV cannabis abuse criteria: a 5-item instrument (corresponding to four abuse criteria), which has been incorporated in the US national survey on drugs since the year 2000 and administered in those who used the respective substance on 6 or more days in the past year [22]. By fulfilling at least one of the four abuse criteria, a person meets the criteria for abuse, which is diagnosed in the absence of dependence (DSM-IV, [23]). For the purpose of the present epidemiological survey, abuse criteria are used without excluding dependence cases. (2)The Severity of Dependence Scale (SDS): a 5-item scale that measures psychological components of dependence [24]. It has been successfully used to assess cannabis dependence in Germany, Australia, and Brazil [11, 25, 26], including adolescent populations of cannabis users, where a cut-off point of 4 for dependence was established [27].(3)The Cannabis Abuse Screening Test (CAST) is a 6-item scale screening for problematic forms of cannabis use, validated using DSM-IV cannabis dependence and cannabis use disorders criteria as a gold standard. It has been used successfully with teenagers, and a cutoff of 2 has recently been proposed to screen for either cannabis dependence or cannabis use disorders in this population [15]. Binary scoring procedure was used, as the resulting total score has a more intuitive interpretation, while the full scoring offers only modest measurement advantages [15].

See Table 1 for all instruments’ items, answer options, and the respective scoring.

The instruments were adapted into Spanish and Catalan in a process including translation, back-translation, cognitive debriefing with young cannabis users, and discussion at an expert meeting. They were placed at the end of the questionnaire and addressed only to those adolescents who had indicated that they had used cannabis in the past 12 months. The assessment frame of all three instruments was “last 12 months.”

In the case of one missing item on a particular scale, in all analyses that included positivity on the instrument, its value was replaced by mean score of the rest of the scale items. This strategy was not applied to DSM-IV abuse criteria.

2.5. Data Analysis

All analyses were carried out using SPSS for Windows 16.0.1.

Analyses of missing items and their relationship with gender, age, and frequency of use in the past year and past month were performed using the Chi-Square test.

An analysis of psychometric properties of the three cannabis screening instruments was carried out including item analysis. Unidimensionality was explored by principal components analysis and internal consistency measured by Cronbach alpha. Internal consistency was assessed by age, to examine possible age-related lack in response consistency.

Overlap between the different instruments was examined. Instruments’ total scores were tested for their relationship with frequency of cannabis use in the past 12 months. In this correlation analysis, averages of the above-mentioned categories were used to compute the Pearson coefficient and order of these categories to compute Spearman’s rho.

A series of bivariate analyses (Chi-Square Test and One-Way ANOVA) and a multinomial logistic regression analysis was run to explore the associations of being positive/negative on the different combinations of instruments (in mutually exclusive categories) with key demographic, substance use, and drug-related problems variables. Students who were negative on all instruments were referent category.

3. Results

3.1. Missing Items Analysis

Of the 4089 cannabis users in the past 12 months, 3569 fully completed the CAST scale, 3546 the SDS scale, and 3573 the DSM-IV abuse criteria. Between 10.5% and 12.3% of answers per item were missing (in most of these cases the entire scale was left blank). Leaving blank a substantial number of items was associated with reporting lower frequency of cannabis use in the past 12 months and in the past 30 days (Chi-Square Test, P < 0.001 in both cases). There were, however, 136 students who reported that they had used cannabis on at least 20 days during the past year, and their total scores of at least one scale could not be computed due to missing items. They represented 3% of all past year users. No association between the number of missing items and gender or age was found.

3.2. Psychometric Properties of the Instruments

Table 1 shows results of item analysis as well as some results for the composite scores of the instruments. Some items had a higher share in the total score than others: the first two items in the case of DSM-IV abuse, the third item of CAST, and item 4 of SDS. The importance of these items was generally confirmed by analysis of proportion positive if the item was deleted.

Principal components analysis revealed unidimensionality in each of the instruments with relatively high factor loadings. Less than 50% of variance was explained by the first component in all cases (46-47%). Cronbach alphas of the three instruments were moderate to satisfactory (0.613–0.762). The only item which would provide (small) improvement of alpha when deleted was item 4 of the SDS scale.

Analysis of internal consistency by age group did not reveal a clear age-related pattern with the exception of CAST, where those aged 14 showed a somewhat lower internal consistency (results not shown).

3.3. Positivity on the Instruments according to Published Cut-Off Points

At least one DSM-IV criterion for abuse was fulfilled by 28.6% (1023) of the instrument completers. 34.3% (1273) of students who completed at least five items of CAST had 2 or more points and would thus be assigned possible cannabis dependence/cannabis use disorders. SDS positive cases (possible dependence on cannabis) at a cutoff of 4 points or more were 16.3% (592). If only those, who used cannabis on six or more days in the past twelve months were considered for positivity, the figures would be 21.6%, 28.7%, and 11.7%, respectively. Students positive on the respective instruments constituted 7.0%, 8.7%, and 4.1% of the entire sample, or 5.3, 7.2, and 2.9%, if six or more days users only were considered.

Sensitivity analysis was performed taking into account the 136 daily or near-daily users whose total score could not be computed due to missing items. Assuming that all of these frequent users would cross the cut-off point on the respective instruments, the original range of prevalence estimates based on crossing the thresholds for positivity of the different instruments (see above) would change slightly to 2.9–7.9%.

3.4. Relationships between Instruments

Only 16.1% of all positives were positive on all three instruments. Out of the 1708 positive on at least one instrument, most cases, 70.9%, were positive on CAST criteria with 26.1% only on CAST; the second largest group, 59%, was positive on the DSM-IV abuse with 18.6% only on DSM-IV abuse. A smaller proportion, 33.4%, was positive on the SDS scale with 8.2% only on SDS (see Figure 1).

Figure 1 

Overlap between the three studied instruments. Total percentages out of those positive on at least one scale (n = 1708).

Pearson correlations between the instruments’ total scores were moderate: 0.363 between DSM-IV abuse and SDS, 0.456 between CAST and SDS, and 0.500 between CAST and DSM-IV abuse (between 0.13 and 0.25).

3.5. Relationships of Scores and Positivity on Instruments with Frequency of Cannabis Use in the Past Twelve Months and Other Variables

CAST total score had the highest correlation of all the instruments with frequency of cannabis use in the past twelve months (0.567 and 0.588, Spearman and Pearson correlation coefficient, resp.). Number of fulfilled DSM-IV abuse criteria correlated with frequency of use in the past year only moderately (0.326 and 0.339, correlations in the same order), and SDS’s total score correlations were the lowest (0.283 and 0.251).

In bivariate analyses, there were significant associations between the mutually exclusive categories of all combinations of positivity/lack of positivity on the tested instruments and all of the following variables: years of cannabis use, gender, frequency of cannabis use in the past 30 days and in the past year, friends’ consumption of cannabis, having an accident requiring medical help in the past 12 months within 6 hours after cannabis use, being under police arrest in the past 12 months within 6 hours after cannabis use, thinking that habitual cannabis smoking causes “quite some problems” or “a lot of problems,” smoking tobacco on some days a week or more during the past month, age at survey, not going to school for two or more days in the past month, because of not feeling like to, and two or more days of alcohol bingeing in the past month and going out one or more nights per week (see Table 2).

In a multinomial logistic regression analysis (see Table 3), those being positive only on SDS were not significantly different from the referent category (negative on all instruments), with the exception of a higher proportion of those thinking that habitual cannabis use poses “quite some problems” or “a lot of problems” and a significantly higher proportion of those belonging to the youngest age group (14-15 years old).

Out of the remaining categories of instrument positivity, those categories, which contained positivity on CAST (positive only on CAST or in combination with SDS, DSM-IV abuse or both) had the highest average number of years of cannabis use. They also had a higher prevalence of truancy, a higher prevalence of frequent cannabis use in the past month (10 or more days of use), higher, but mostly not statistically significant, prevalence of alcohol bingeing in the past month, a higher proportion of the majority (or all) friends consuming cannabis, and the highest prevalence of regular tobacco smoking in the past month (at least some days a week). Students who were positive on CAST also had a higher prevalence of suffering an accident requiring medical help and/or detention by police within six hours after cannabis use in the past year, but this was further strengthened if positivity on CAST was combined with positivity on DSM-IV abuse.

4. Discussion

The present study confirmed that it was feasible to use short psychometric instruments for problematic forms of cannabis use in a large national probabilistic survey on substance use in Spanish adolescents. Even though the instruments were placed at the end of the questionnaire, 88–90% of items were filled in. Missing answers occurred more often in individuals who reported low frequency of cannabis use. Sensitivity analysis was performed which did not find a significant influence of missing scale results in those reporting frequent cannabis use on the overall prevalence estimate.

We have studied overlap between the three instruments and found that it was relatively small (16.1%) with more than half of the respondents positive on only one instrument. The instruments’ total scores did not correlate highly either. This is partially due to the different concepts of problem cannabis use, which are behind the three instruments. The expected relationships in this respect would be a high overlap between DSM-IV abuse and CAST (the latter designed to screen for cannabis use disorders, that is, cannabis abuse and/or cannabis dependence) and a subgroup of abuse and/or CAST positive persons being identified as cannabis dependent by SDS. However, this was clearly not the case.

Moreover, focusing only on cannabis dependence gives a result which is similar to comparing all three instruments. If we were to explore the overlap between CAST and SDS, both validated against gold standard representing cannabis dependence, in the present study, we would still find only 28.1% of cases identified by both instruments, while almost 60% would be positive only according to CAST and 12.9% only according to SDS. Swift et al. [28] studied three measures of cannabis dependence, among them the SDS and found 56% of cases identified by all three measures.

One possible explanation of low overlap between screening instruments in the present study might be the existence of different types of problematic cannabis users, with different characteristics and different kinds of problems experienced by them [29]. Another possibility is the presence of measurement bias, with varying extent in the three studied instruments. The multinomial logistic regression analysis discussed sheds some light on this possibility.

According to the results of this analysis, students positive only on SDS scale were similar to those who were negative on all instruments with the exception of being a younger age and more likely to think that habitual cannabis use causes “quite some problems” or “a lot of problems”. On the other hand, those groups which included positivity on CAST had the longest history of cannabis use, higher prevalence of different problems, some of which were directly related to cannabis use. In this group, there was a lower proportion of girls, higher tobacco smoking, markedly higher prevalence of high frequency cannabis use in the past month, more of their friends smoked cannabis, while they thought less that habitual cannabis use caused substantial problems. DSM-IV abuse positivity has further strengthened CAST’s association with such problems as being detained by the police or having an accident shortly after cannabis use. This is consistent with two abuse criteria—legal problems and using the substance in physically hazardous situations—and confirms concurrent validity of the instrument. Fulfilling DSM-IV abuse criteria was associated less strongly than CAST with history and frequency of cannabis use, and other problems potentially related to cannabis use, but more strongly than SDS positivity.

This suggests that the usefulness of SDS in screening for problematic cannabis use in the general population of adolescents is limited, as it may identify a considerable proportion of “wrong people.” This finding from the multinomial logistic regression analysis is further corroborated by the weak association of the scale with frequency of use in the past year—an important indicator within the diagnostic approach [30, 31] and by the fact that item 4 of the scale (wish to stop) contributed to the total score considerably while its removal would improve the internal consistency of the instrument. This weaker performance might be associated with the concept behind the scale, namely psychological components of dependence. These might be difficult to assess or confusing for adolescents. For example, other authors have found a response bias in adolescents in the direction of overreporting dependence, mainly its symptoms “being unable to cut down” and “tolerance” [32]. Another study suggested that substance use disorders diagnostic criteria might have different meaning for adolescents, who may experience rapid development of tolerance after initiation of use and may be confused about a question on “using more or longer than intended” as they typically do not foresee a limit of consumption but intend to become intoxicated [33].

Although conducted on a large probabilistic sample, the present study has several limitations. The scales administration was based on self-assessment, moreover of relatively complex concepts. The sampling frame of the present study was adolescents aged 14–18 who were schooled; 18% of this age group were estimated to be unschooled and thus outside of the sampling frame. Moreover, almost 10% of the students of the selected classrooms were not attending school on the day of survey. There are associations of dropping out of school and truancy with substance use and related problems shown by numerous studies (e.g., [34] and [16, page 172]). However, the analysis of psychometric properties of the explored instruments may be less biased by these factors than, for example, prevalence estimation.

The placing of the screening instruments (almost at the end of the questionnaire), and, possibly, the order of their presentation may have affected the number of missing responses and/or the reliability of responses due to such factors as running out of time, or tiredness of the respondent and possibly the measures of psychometric properties of the instruments. A study comparing the effects of different orders of presentation would be necessary for clarification of this question.

Furthermore, as establishing clinical diagnoses was not part of this study, its findings have to be interpreted cautiously in relation to these concepts. The reported psychometric properties of the applied instruments might not be transferable to clinical or targeted samples of heavy cannabis users, owing to “spectrum bias” [35] in those samples. Nevertheless, it is important to know how instruments to detect problematic cannabis use perform in the general adolescent population.

In conclusion, SDS, CAST, and DSM-IV abuse criteria have performed moderately well in the present sample, but identified largely different groups of users with only modest overlaps, even between scales validated against the same gold standard. Given the fact that a researcher planning an epidemiological survey may decide to choose any of these instruments to represent a measure of “problematic cannabis use,” this finding may be highly relevant. While one possible explanation for these differences might be the existence of different subtypes of problematic cannabis use, measurement bias of varying extent may also play a role.

The present analyses put in question the validity of the SDS scale in the general adolescent population. The concept of psychological components of dependence represented by the scale may be difficult to understand for adolescent cannabis users. This may be the reason for high scoring on SDS items operationalizing this concept in the youngest lower frequency users who have few cannabis-related problems and who think that habitual cannabis use causes “quite some problems” or “a lot of problems”.

On the other hand, CAST scale had generally strong relationship with frequency of cannabis use in the last year and other variables suggesting its concurrent validity. DSM-IV abuse was less strongly associated with intensive and long-term use, but its association with legal problems and an item indicating possible use in physically hazardous situations corroborated the validity of the instrument in relation to the concept behind it.

The findings of the present study are relevant also from the broader perspective of using short, self-report-based screeners of complex, more abstract concepts in the population of adolescents in psychiatry, general practice, and clinical psychology.

Further research, especially qualitative, is needed to shed light on adolescents’ understanding of psychometric instruments’ items related to cannabis dependence symptoms in order to improve the ascertainment of problematic cannabis use in this population.

Abbreviations

Acknowledgments

This study was conducted with financial support of Delegación del Gobierno para el Plan Nacional sobre Drogas (DGPNSD), Madrid, Spain, and European Monitoring Centre for Drugs and Drug Addiction (EMCDDA), Lisbon, Portugal. The authors would like to acknowledge Colin Taylor for providing useful advice and comments throughout the paper preparation.