Treatment Outcomes in Patients with Internet Addiction: A Clinical Pilot Study on the Effects of a Cognitive-Behavioral Therapy Program
Internet addiction is regarded as a growing health concern in many parts of the world with prevalence rates of 1-2% in Europe and up to 7% in some Asian countries. Clinical research has demonstrated that Internet addiction is accompanied with loss of interests, decreased psychosocial functioning, social retreat, and heightened psychosocial distress. Specialized treatment programs are needed to face this problem that has recently been added to the appendix of the DSM-5. While there are numerous studies assessing clinical characteristics of patients with Internet addiction, the knowledge about the effectiveness of treatment programs is limited. Although a recent meta-analysis indicates that those programs show effects, more clinical studies are needed here. To add knowledge, we conducted a pilot study on the effects of a standardized cognitive-behavioral therapy program for IA. 42 male adults meeting criteria for Internet addiction were enrolled. Their IA-status, psychopathological symptoms, and perceived self-efficacy expectancy were assessed before and after the treatment. The results show that 70.3% of the patients finished the therapy regularly. After treatment symptoms of IA had decreased significantly. Psychopathological symptoms were reduced as well as associated psychosocial problems. The results of this pilot study emphasize findings from the only meta-analysis conducted so far.
Numerous studies of the past decade point to Internet addictive behavior as a growing health issue in different parts of the population. Prevalence estimations range up to 6.7% within adolescents and young adults in southeast Asia , 0.6% in the United States , and between 1 and 2.1% in European countries [3, 4] with adolescents showing even increased prevalence rates (e.g., ). Based on these observations, the APA has decided to include Internet Gaming Disorder—one common subtype of Internet addiction (IA)—into section III of the DSM-5 “as a condition warranting more clinical research and experience before it might be considered for inclusion in the main book as a formal disorder” .
People affected by IA report symptoms resembling those known from substance-related and other nonsubstance-related (e.g., gambling disorder) addiction disorders. They display a strong preoccupation with Internet activities, feel an irresistible urge to go online, show increasing hours spent online (tolerance), feel irritated and dysphoric when their online access is restricted or denied (withdrawal), keep on going online despite negative consequences in different areas of life (e.g., conflicts with family members and decreasing achievements in school, college, or job), and are not able to cut back from their behavior (loss of control). Since further parallels have been reported regarding shared neurobiological features (e.g., ; for a review see ) and similarities in underlying personality traits (e.g., [8, 9]), it has been proposed to perceive IA as another type of nonsubstance-related addiction disorder. Furthermore, increased rates of comorbid IA within patients suffering from other forms of addiction that have been reported solidify this assumption [6, 10].
Clinical studies underpin increased psychopathological symptoms and decreased levels of functioning in patients , deteriorating quality of life , social retreat, and isolation, respectively , as well as high levels of psychosocial and psychopathological symptoms [14, 15]. For example, Morrison and Gore  reported high levels of depression within a sample of 1319 study participants. Likewise, Jang and colleagues  documented increased psychosocial strain, especially concerning obsessive-compulsive and depressive symptoms in adolescents suffering from IA.
Since IA is more and more recognized as a serious mental disorder causing distress and decreased levels of functioning in those affected by it, increasing efforts to develop and document different treatment strategies have emerged, including psychotherapeutic and psychopharmacological interventions for IA . Although one has to admit that current clinical investigations are lacking in methodological quality or are based on comparably small patient samples (for a review of treatment outcome studies on IA see King et al. ), the first findings regarding response and remission after treatment in IA are promising.
One study that met several quality standards of clinical outcome studies according to the analytic review by King et al.  investigated effects of a multimodal cognitive-behavioral program in adolescents with IA . 32 patients treated due to IA were statistically compared to a wait-list control group receiving no treatment (24 subjects). Primary endpoints of this study included a self-report measure for IA (Internet Overuse Self-Rating Scale by Cao and Su ) as well as self-report measures assessing time management skills and psychosocial symptoms. Changes in these outcome measures were assessed prior to, immediately after, and at the end of the treatment. A followup was performed six months after the treatment. The results showed that, in both groups, a significant decrease of IA-symptoms was observable and also stable over the period of six months. However, only the treatment group was displaying significant improvement in time management skills and decreasing psychosocial problems regarding lower anxiety and social problems.
Likewise, studies applying psychopharmacological treatment have demonstrated promising results indicating that patients with IA benefit from SSRI and methylphenidate [21, 22], matching findings from clinical evidence in the treatment of patients with gambling disorder .
Moreover, a recently published meta-analytic study by Winkler and colleagues  that included 16 clinical trials with different therapeutic approaches based on 670 patients indicates high effectiveness of treatment of IA: the detailed results suggest that there were significant differences depending on the type of therapeutic treatment with cognitive-behavioral programs displaying higher effect sizes () regarding decreased symptoms of IA than other psychotherapeutic approaches (). However, the general results indicate that every treatment approach analyzed yielded significant effects.
However, the literature on treatment outcomes in IA is still both underdeveloped and heterogeneous in many ways, as it is also stated by the authors of the above-mentioned meta-analysis [24, page 327]: “However this study illustrates the lack of methodological sound treatment studies, offers insight into the current state of internet addiction treatment research, bridges research investigations from “East” and “West” and is a first step in the development of an evidence-based treatment recommendation.” This stresses the need for more clinical trials relying on accurately defined therapy programs. In the light of these circumstances, we will introduce a short-term psychotherapeutic treatment program for IA and provide first data from a pilot study regarding its usefulness and its effects. Although this pilot study may be based on a comparably small sample size and lacks inclusion of a wait-list control group, we regard it as helpful to publish these preliminary data.
1.1. Short-Term Treatment for Internet and Computer Game Addiction (STICA)
Since 2008, the work group of the Outpatient Clinic for Behavioral Addiction in Germany offered counseling for patients suffering from different kinds of IA. In the meantime, about 650 patients—mostly males aged between 16 and 35 years—introduced themselves as treatment seekers. In the light of increasing patient contacts, a standardized psychotherapeutic program for IA was developed and a therapy manual was developed (STICA)  which is based on cognitive-behavioral techniques known from treatment programs of other forms of addictive behavior. STICA is meant to be used for an outpatient treatment and consists of 15 group sessions plus additional eight sessions of individual therapy.
While the individual sessions are dealing with individual contents, the group sessions are following a clear thematic structure. In the first third of the program, the main themes approach the development of individual therapy aims, the identification of the Internet application that is associated with symptoms of IA, and the conduct of a holistic diagnostic investigation of psychopathological symptoms, deficits, resources, and comorbid disorders. Motivational techniques are also applied to enhance the patients’ intention to cut down the dysfunctional behavior. In the second third, psychoeducative elements are introduced and deepened analyses of the Internet use behavior, focusing on its triggers and the reactions of the patient on cognitive, emotional, psychophysiological, and behavioral levels in that situation (SORKC-scheme, ), are performed. One crucial aim at this stage is the development of a personalized model of IA for each patient, based on the interaction of the Internet application used, predisposing and maintaining factors of the patient (e.g., personality traits) and the patients’ social environment. At the last stage of the therapy, situations with heightened craving for getting online are further specified and strategies to prevent relapse are developed. A detailed overview on the structure of STICA is presented in Table 1.
1.2. Research Questions
In this study, we aimed at gathering first data on the effectiveness of STICA. We also intended to characterize the patients included regarding psychosocial symptoms, comorbidity, and personality features that can play a role in therapeutic treatment regarding building up a therapeutic alliance and differences in the treatment response . Additionally, effects of psychosocial strain at the beginning of the therapy and personality traits on treatment outcome are reported. Lastly, we want to provide a comparison between patients regularly finishing the therapy (completers) and those who dropped out of the program (dropouts).
2. Materials and Methods
2.1. Data Acquisition and Statistical Analyses Plan
In this trial, data were collected from 42 patients consecutively introducing themselves to the Outpatient Clinic for Behavioral Addictions in Germany due to IA (clinical convenience sample). These patients were included out of an initial clinical sample of 218 treatment seekers. From these, 74 (33.9%) had to be excluded because of not meeting the criteria of IA. 29 (13.3%) more subjects had to be excluded because of being under the age of 17. 73 further exclusions (33.5%) were due to severe comorbid disorders, refusing to receive psychotherapeutic treatment, or severity of IA making an inpatient treatment necessary. The patients were asked to provide personal data for scientific processing and gave written informed consent. The investigation was in line with the declaration of Helsinki. Due to missing or incomplete data in the primary endpoints at T1, 5 subjects had to be excluded from the final data analyses.
Inclusion criteria were the presence of IA according to AICA-S (Scale for the Assessment of Internet and Computer Game Addiction, AICA-S ; see paragraph 2.2) and a standardized clinical interview of IA (AICA-C, Checklist for the Assessment of Internet and Computer Game Addiction, ). Moreover, male gender and age above 16 years were further requirements.
Exclusion criteria referred to severe comorbid disorders (other addictive disorders, psychotic disorders, major depression, borderline personality disorder, and antisocial personality disorder). Also, patients reporting current medication because of psychiatric disorders and those reporting being in psychotherapeutic treatment were excluded from data analyses.
As primary endpoints, the remission of IA according to a standardized self-report questionnaire (AICA-S) was defined. As secondary endpoints, changes in the following dimensional variables were assessed: severity of psychosocial symptoms, time spent online, negative consequences due to Internet use, and self-efficacy expectancy.
Data were assessed at the very beginning of the therapy (T0) and immediately after termination of the therapy (T1). Data analyses are reported for both conditions, intent to treat (including patients dropping out of treatment) and completers. For the intent-to-treat analyses, the last observation carried forward (LOCF) method was applied. The LOCF advises to use the last data available in those subjects not terminating a treatment condition regularly. In the present study, data from T0 were used for those subjects dropping out of the treatment program before T1 was assessed.
For statistical analyses, chi-square tests were used for the comparison of dichotomous variables with cramer-v as a measure of effect size. Changes in the primary and secondary endpoints were measured using paired -tests for pre- and postcomparison for one sample, with as a measure of effect size for dependent samples. According to the proposal by Dunlap et al. , adapted was calculated if the correlation between the pre- and postscores of the dependent variables was bigger than 0.50. All analyses were performed using SPSS 21.
For the classification of IA, two measures were applied at T0. For the Scale for the Assessment of Internet and Computer Game Addiction (AICA-S, ), a standardized self-report measure was applied assessing IA according to adapted criteria for gambling disorder and substance-related disorders (e.g., preoccupation, tolerance, withdrawal, and loss of control). Each criterion indicating IA is assessed either on a five-point Likert scale (never to very often) or in a dichotomous format (yes/no) and a weighted sum score can be derived from the accumulation of the diagnostic items. A cutoff of 7 points (that is corresponding to a total of 4 criteria which are met) has been found to have the best diagnostic accuracy in detecting IA (sensitivity = 80.5%; specificity = 82.4%) in an investigation of patients entering our outpatient clinic. According to previous investigations, AICA-S can be considered as showing good psychometric properties (Cronbach’s ), construct validity, and clinical sensitivity . Since AICA-S was also the primary endpoint, it was also assessed at T1.
To further assure diagnosis of IA, a clinical expert rating was administered as well. The Checklist for Internet and Computer Game Addiction (AICA-C, ) was used for that purpose. AICA-C includes six core criteria for IA (preoccupation, loss of control, withdrawal, negative consequences, tolerance, and craving) that have to be rated by a trained expert on a six-point scale ranging from 0 = criterion not met to 5 = criterion fully met. According to analyses on its diagnostic accuracy, a cutoff of 13 points has yielded the best values (sensitivity = 85.1%; specificity = 87.5%). It has successfully been checked for its psychometric properties (Cronbach’s ) and its clinical accuracy .
The General Self-Efficacy Scale (GSE; ) was used in order to assess the construct of generalized self-efficacy expectancy by ten items. GES is understood as the quantity of subjective judgments of the amount of personal abilities to surmount problems and daily challenges. Numerous studies have reported that GSE has to be regarded as an important resilience factor, with high GSE predicting functional behavioral changes and motivating individuals to actively face engrossing situations . GSE was administered at T0 and T1.
The NEO Five-Factor Inventory  was conceptualized to measure the five domains of the Five-Factor Model. It consists of 60 items answered on 5-point Likert scales and is one of the most used self-report measures in personality research. Numerous studies have stressed its good psychometric quality and validity . The NEO-FFI was used only at T0 to examine predictive power of the five factors on therapy outcome and compliance.
At the measure points, T0 and T1, psychopathological symptoms were assessed using the Symptom Checklist 90R , a widely used clinical questionnaire with sound psychometric properties . Psychopathological distress is assessed by 90 items (0 = no symptoms to 4 = strong symptoms) loading on nine subscales. The SCL-90R is referring to the degree to which the subject has experienced the symptoms during the last week. The global severity index (GSI)—a global sum score across the nine subscales—represents the overall distress.
3.1. Description of the Sample
The sociodemographic statistics of the treatment seekers can be found in Table 2.
As can be derived from Table 2, most patients were not in a partnership with nearly the half of them still living at home with their parents. Most of the treatment seekers were not employed yet but owned a high school education.
Most of the patients were displaying addictive use of online-computer games (78.4%). 10.8% were using different Internet applications addictively, 8.1% used social networking sites, and 2.7% were doing excessive research in information data bases.
Regarding subclinical features, the following indices were found for NEO-FFI: () for neuroticism, () for extraversion, () for openness, () for agreeableness, and () for conscientiousness.
3.2. Changes in Primary and Secondary Endpoints
70.3% (26) finished the therapy regularly (completers), 29.7% (11) patients dropped out during the course (dropouts). The results show that the completers had significant improvements in the primary and most of the secondary endpoints. The pre- and postscores of the primary and secondary endpoints for the completers can be derived from Table 3.
As can be seen in Table 3, significant decrease in the score of AICA-S is observable after the treatment. Moreover, significant decreases in hours spent online per weekend day and decreasing conflicts because of the Internet use in five of the six areas assessed were observable. Likewise, a significant decrease in the GSI was found, with completers displaying significantly diminished scores after treatment in seven of the nine subscales of the SCL-90R.
As expected, therapy effects were to some extent smaller when adding the dropouts to the analyses. However, the intent-to-treat analyses also reveal that after the treatment, the score in AICA-S decreased significantly (, ; ). The same was observable for the average amount of time spent online at one day of the weekend (, ; ) and overall negative consequences associated with the Internet use (, ; ). Also, in psychopathological symptoms, significant pre- and postchanges were observable, regarding GSI (, ; ) and the SCL-subscales obsessive-compulsive (, ; ), social insecurity (, ; ), depression (, ; ), anxiety (, ; ), aggression (, ; ), phobic anxiety (, ; ), and psychoticism (, ; ). Also, the self-efficacy expectancy increased significantly after treatment (, ; ).
3.3. Influence on Treatment Response
The analyses of sociodemographic differences between completers and dropouts showed no significant results regarding age, partnership, family status, living situation, or employment status. The only difference showing a trend significance (; ; cramer-v = .438) was found in education with completers displaying higher school education (76.9%) than dropouts (63.7%).
Regarding influence of personality traits on therapy termination, no significant group differences were found either, with the exception of the factor openness. A trend significance turned out indicating that completers (; ) were displaying higher scores than the dropouts (; ; , ). Likewise, no group differences were found regarding psychosocial symptoms in T0 (SCL-90R) or the degree of self-efficacy expectancy (GSE). Also, severity of IA-symptoms did not discriminate between completers and dropouts nor did the amount of hours spent online (assessed by AICA-S).
In this pilot study, we investigated effects of a standardized short-term psychotherapy on a sample of outpatient clients suffering from IA. To that purpose, a total of initially 42 patients were treated according to the therapy program with their psychological health status being assessed when entering the therapy and immediately after its termination. As a primary endpoint, we assessed symptoms of IA according to a reliable and valid self-report measure (AICA-S; ). Additionally, time spent online, negative consequences arising from online activities, self-efficacy expectancy, and psychosocial symptoms were defined as secondary endpoints.
About 70% of the treatment seekers passed the complete therapy program (completers), and about one-third dropped out during the course of the therapy. Thus, the dropout rate is well within the outpatient dropout rates within mental health care (see ; 19–51%) but exceeds those reported by Winkler and colleagues (see ; 18.6%). The further results indicate that the treatment program has promising effects. After the therapy, a significant decrease of IA-symptoms could be observed. The effects sizes found here amounted to for the completers and for the total sample including the dropouts. According to the definition of Cohen , this can be regarded as an indication of large effects. Moreover, it corresponds to the effect sizes on IA-status after psychotherapy (; with confidence intervals between .84 and 2.13) reported in the meta-analyses by Winkler et al. . Likewise, time spent online on the weekends was significantly reduced after the therapy with a comparably large effect size () that is nevertheless smaller compared to the data provided by the latest meta-analysis on that topic (see ; ).
It is important to explain that the aim of this therapy approach is not to keep the patients away from any use of the Internet per se. Instead, specific therapy aims are developed based on the results of an extensive probatory in which Internet use habits of the patient are elucidated and problematically used Internet contents are identified. The therapy aims at motivating the patient to initiate abstinence from the Internet activity identified as being related to core symptoms of IA, like the loss of control and craving. Thus, a mean value of zero hours spent online was not expected. Indeed, the mean online time of 2.6 hours per day is well within the range of the German population average. In a representative survey on approximately 2500 German subjects, Müller et al.  reported that the average time spent online on a day of the weekend was 2.2 hours within regular Internet users.
Moreover, also most of the secondary endpoints changed significantly during the therapy. First of all, problems arising from addictive Internet use decreased in several areas, concerning frequency of family conflicts, denial of other recreational activities, frequency of health problems, struggles with friends, and negative effects on school or job performance. Self-efficacy expectancy increased with a medium effect size of and the mean score in the GSE after treatment is comparable to the one derived from the general German population . This indicates that optimistic expectancy towards the individual’s ability to surmount appearing difficulties and challenges reaches an acceptable level after the treatment. If differences in self-efficacy expectancy among patients after treatment can be perceived as a predictor for middle- and long-lasting therapy, effects should be investigated in follow-up studies.
Lastly, psychosocial symptoms associated with IA decreased significantly after the treatment. This was the case for the global severity index as well as for seven of nine subscales of the SCL-90R. Large effect sizes were accomplished for the global severity Index and obsessive-compulsive and depressive symptoms, as well as for social insecurity.
Surprisingly, we did not find any variables distinguishing between patients passing the complete therapy and those dropping out of the program that could have served as valuable markers for the therapy success. There was a statistical trend indicating that patients with higher levels of educations were more likely to finish the therapy regularly. Also, we found—again as a trend—that patients completing the therapy display higher scores in the personality traits openness. In the personality literature, high openness is described as being interested in alternatives to traditional thinking and acting and showing curiosity towards new aspects and ways of thinking . One might conclude from this that patients scoring high on this factor might have a more favorable attitude regarding psychotherapy and therefore are more likely to get themselves into the changes of psychotherapy. However, the relationships reported here were only marginally significant. This might be explained by the small sample size, especially concerning the patients dropping out of treatment. Clearly, more research is needed to identify predictors of therapy completion in patients with IA.
This study has a number of limitations that need to be addressed. A major shortcoming has to be seen in the lack of a control group, whether a wait-list control (WLC) or a therapy as usual group (TAU). Since there was only the single condition of a treatment group, statistical (by intraindividual comparisons) and interpretative limitations are obvious. It is not possible to finally determine if the effects of decreasing symptoms of IA and psychopathological strain are due to the psychotherapeutic intervention or origin from variables that were not controlled for. Secondly, a convenience sample of treatment seekers was examined without a randomization procedure. This raises the question if the participants of this study have to be regarded as selective. Moreover, the clinical sample under investigation was made up by only 42 male patients. This is quite a small sample size that did not allow for any deepened statistical analyses (e.g., the influence of different types of IA on therapy outcome). Since the sample was consisting only of male patients, the findings cannot be generalized to female patients. Lastly, the study design did not include a followup, so it is not possible to draw conclusions on the stability of the therapy effects that were observed immediately after the treatment. To correct these shortcomings, the authors are conducting a follow-up clinical trial at present . This project that aims at the inclusion of 193 patients suffering from IA consists of a multicenter randomized and controlled trial with a follow-up assessment 12 months after the termination of the therapy.
Based on the data provided in this pilot study, it is reasonable to suppose that psychotherapeutic treatment of patients suffering from IA is effective. After application of a standardized cognitive-behavioral treatment, we found significant changes in symptoms of IA, time spent online, negative repercussions following the Internet use, and associated psychopathological symptoms, with the largest effects on depressive and obsessive-compulsive symptoms. This pilot study, which was conducted to herald the start of a larger, randomized, and controlled clinical trial, confirms the conclusions that Winkler and colleagues  have drawn from the data of their meta-analyses: IA appears to be a mental disorder that can be effectively treated by psychotherapeutic strategies—at least when referring to the immediate therapy effects.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
E. Aboujaoude, L. M. Koran, N. Gamel, M. D. Large, and R. T. Serpe, “Potential markers for problematic internet use: a telephone survey of 2,513 adults,” CNS Spectrums, vol. 11, no. 10, pp. 750–755, 2006.View at: Google Scholar
G. Floros and K. Siomos, “Excessive Internet use and personality traits,” Current Behavioral Neuroscience Reports, vol. 1, pp. 19–26, 2014.View at: Google Scholar
G. Murray, D. Rawlings, N. B. Allen, and J. Trinder, “Neo five-factor inventory scores: psychometric properties in a community sample,” Measurement and Evaluation in Counseling and Development, vol. 36, no. 3, pp. 140–149, 2003.View at: Google Scholar
American Psychiatric Association, Diagnostic and Statistical Manual of Mental Disorders, (DSM-5), American Psychiatric Publishing, 5th edition, 2013.
C. H. Ko, G. C. Liu, J. Y. Yen, C. F. Yen, C. S. Chen, and W. C. Lin, “The brain activations for both cue-induced gaming urge and smoking craving among subjects comorbid with Internet gaming addiction and nicotine dependence,” Journal of Psychiatric Research, vol. 47, no. 4, pp. 486–493, 2013.View at: Publisher Site | Google Scholar
K. W. Müller, M. E. Beutel, B. Egloff, and K. Wölfling, “Investigating risk factors for internet gaming disorder: a comparison of patients with addictive gaming, pathological gamblers and healthy controls regarding the big five personality traits,” European Addiction Research, vol. 20, no. 3, pp. 129–136, 2014.View at: Publisher Site | Google Scholar
K. W. Müller, A. Koch, U. Dickenhorst, M. E. Beutel, E. Duven, and K. Wölfling, “Addressing the question of disorder-specific risk factors of internet addiction: a comparison of personality traits in patients with addictive behaviors and comorbid internet addiction,” BioMed Research International, vol. 2013, Article ID 546342, 7 pages, 2013.View at: Publisher Site | Google Scholar
K. W. Müller, M. E. Beutel, and K. Wölfling, “A contribution to the clinical characterization of Internet Addiction in a sample of treatment seekers: validity of assessment, severity of psychopathology and type of co-morbidity,” Comprehensive Psychiatry, vol. 55, no. 4, pp. 770–777, 2014.View at: Publisher Site | Google Scholar
T. R. Miller, “The psychotherapeutic utility of the five-factor model of personality: a clinician's experience,” Journal of Personality Assessment, vol. 57, no. 3, pp. 415–433, 1991.View at: Google Scholar
F. H. Kanfer and J. S. Phillips, Learning Foundations of Behavior Therapy, John Wiley & Sons, New York, NY, USA, 1970.
F. Cao and L. Su, “The factors related to Internet overuse in middle school students,” Chinese Journal of Psychiatry, vol. 39, pp. 141–144, 2006.View at: Google Scholar
B. Dell'Osso, S. Hadley, A. Allen, B. Baker, W. F. Chaplin, and E. Hollander, “Escitalopram in the treatment of impulsive-compulsive internet usage disorder: an open-label trial followed by a double-blind discontinuation phase,” Journal of Clinical Psychiatry, vol. 69, no. 3, pp. 452–456, 2008.View at: Google Scholar
K. Wölfling, C. Jo, I. Bengesser, M. E. Beutel, and K. W. Müller, Computerspiel-und Internetsucht—Ein kognitiv-behaviorales Behandlungsmanual, Kohlhammer, Stuttgart, Germany, 2013.
K. Wölfling, K. W. Müller, and M. E. Beutel, “Diagnostische Testverfahren: Skala zum Onlinesuchtverhalten bei Erwachsenen (OSVe-S),” in Prävention, Diagnostik und Therapie von Computerspielabhängigkeit, D. Mücken, A. Teske, F. Rehbein, and B. te Wildt, Eds., pp. 212–215, Pabst Science Publishers, Lengerich, Germany, 2010.View at: Google Scholar
W. P. Dunlap, J. M. Cortina, J. B. Vaslow, and M. J. Burke, “Meta-analysis of experiments with matched groups or repeated measures designs,” Psychological Methods, vol. 1, no. 2, pp. 170–177, 1996.View at: Google Scholar
R. Schwarzer and M. Jerusalem, “Generalized Self-Efficacy scale,” in Measures in Health Psychology: A User's Portfolio. Causal and Control Beliefs, J. Weinman, S. Wright, and M. Johnston, Eds., pp. 35–37, NFER-NELSON, Windsor, UK, 1995.View at: Google Scholar
P. T. Costa Jr. and R. R. McCrae, Revised NEO Personality Inventory (NEO-PI-R) and NEO Five-Factor Inventory (NEO-FFI) Professional Manual, Psychological Assessment Resources, Odessa, Fla, USA, 1992.
L. R. Derogatis, SCL-90: Administration, Scoring and Procedures Manual—I for the R, (Revised) Version and Other Instruments of the Psychopathology Rating Scales Series, Johns Hopkins University School of Medicine, Chicago, Ill, USA, 1977.
C. J. Brophy, N. K. Norvell, and D. J. Kiluk, “An examination of the factor structure and convergent and discriminant validity of the SCL-90R in an outpatient clinic population,” Journal of Personality Assessment, vol. 52, no. 2, pp. 334–340, 1988.View at: Google Scholar
J. Cohen, Statistical Power Analysis for the Behavioral Sciences, Lawrence Erlbaum Associates, Hillsdale, NJ, USA, 2nd edition, 1988.
R. R. McCrae and P. T. Costa Jr., Personality in Adulthood: A Five-Factor Theory Perspective, Guilford Press, New York, NY, USA, 2003.