Long-Term Outcomes and Cost-Effectiveness of an Internet-Based Self-Help Intervention for Social Anxiety Disorder in University Students: Results of a Randomized Controlled Trial

Kählke, Fanny; Buntrock, Claudia; Smit, Filip; Berger, Thomas; Baumeister, Harald; Ebert, David Daniel

doi:https://doi.org/10.1155/2023/7912017

Depression and Anxiety

On this page

Abstract Introduction Methods Results Data Availability Additional Points Disclosure Conflicts of Interest Authors’ Contributions Acknowledgments Supplementary Materials References Copyright Related Articles

Research Article | Open Access

Volume 2023 | Article ID 7912017 | https://doi.org/10.1155/2023/7912017

Long-Term Outcomes and Cost-Effectiveness of an Internet-Based Self-Help Intervention for Social Anxiety Disorder in University Students: Results of a Randomized Controlled Trial

Fanny Kählke,¹Claudia Buntrock,²Filip Smit,³Thomas Berger,⁴Harald Baumeister,⁵and David Daniel Ebert¹

Academic Editor: Giulia Landi

Received24 May 2023

Revised31 Jul 2023

Accepted21 Sept 2023

Published17 Nov 2023

Abstract

Social anxiety disorder (SAD) is widespread among university students and is associated with high costs for the society. While unguided internet- and mobile-based interventions (IMIs) may have short-term effects in reducing SAD symptoms, evidence for their long-term efficacy and cost-effectiveness is still limited. The aim of this study is to examine the 6-month outcomes of an IMI for university students with SAD. Participants were recruited via mass mails sent to enrolled students and included if they were at least 18 years old, met the diagnostic criteria of SAD in a structured clinical interview for DSM-IV axis I disorders (SCID-I), and provided written informed consent. In a prospective study designed as a two-armed randomized-controlled trial, 200 students (mean age 26.7 years) diagnosed with SAD were randomly assigned to an IMI or a waitlist control (WLC) condition. The IMI consisted of nine weekly sessions based on the cognitive-behavioral treatment model for social phobia by Clark and Wells. The primary outcome was SAD symptom severity assessed via the Social Phobia Scale (SPS) and the Social Interaction Anxiety Scale (SIAS). A health economic evaluation from a societal and healthcare perspective examined the costs related to the symptom-free status and quality-adjusted life years (QALYs) gained. Statistically significant differences in SAD symptom severity previously found at posttreatment favoring the IMI were maintained at a 6-month follow-up [SIAS (Cohen’s ; 95% CI, 0.30, 0.87) and SPS (Cohen’s ; 95% CI, 0.54, 1.1)]. From a societal perspective, at a willingness to pay (WTP) of €0, the intervention was found to have a 92% and 93% probability of cost-effectiveness compared with the WLC per symptom-free status and QALY gained, respectively. From a healthcare perspective, the likelihood of cost-effectiveness of the intervention was 97% per symptom-free status at a WTP of €1000 (US$1326) and 96% per QALY gained at a WTP of €6000 (US$7956). This IMI is effective in treating university students with SAD and has an acceptable likelihood of cost-effectiveness compared with WLC from a societal perspective. This intervention can be integrated into university healthcare to reach students with SAD as it is scalable, shows a high probability of cost-effectiveness, and overcomes known treatment barriers. This trial is registered with DRKS00011424.

1. Introduction

Social anxiety disorder (SAD) is a prevalent and impairing disorder and is considered a public health concern. In particular, university students fall within the age range when common mental health problems reach their developmental peak [1]. Being at a university is associated with many stressors and transitional events [2]. In Germany, persons aged 18–35 show a 4.8% [3] 12-month prevalence of SAD, while 12.2% of university students show clinically relevant social phobic symptoms [4]. Several adverse effects on quality of life (QoL) and identity formation [5], alcohol consumption [6], and suicidal ideation [7] are reported in university students with SAD. These may lead to premature dropout and academic underachievement [8]. Moreover, SAD is associated with substantial impairment across several domains, such as relationship, daily and social life, and work [9]. Altogether, it generates direct (treatment), indirect (productivity losses, absence of work, and low qualification [10, 11]), and intangible costs (lower QoL and social impairment). In Germany, the mean total 6-month costs for SAD in a clinical sample were estimated at €4802 per person, which are mainly attributed to indirect cost [12].

While the university setting enables comprehensive approaches for prevention, early intervention, and treatment of students with SAD, students in general are averse to face-to-face counseling centers preferring to solve their problems themselves [13]. In particular, students with SAD face attitudinal barriers (fear of stigmatization and negative evaluation) to help-seeking [14] that may leave them untreated with a chronic condition [15].

Internet- and mobile-based interventions (IMIs), which are flexible, accessible, and anonymous [16–18], present a promising approach to reach those affected individuals. Unguided IMIs have received adequate attention owing to their potential for high scalability and relatively low marginal costs. The efficacy of unguided IMIs based on cognitive-behavioral approaches targeting SAD has been shown with medium effects at posttreatment compared with that of passive controls (, 95% CI [0.50–1.05], , , ). However, in contrast to guided IMIs, the evidence for the long-term efficacy of unguided IMIs is still limited [19–22].

Moreover, the value of IMIs for SAD in university students has not been sufficiently investigated. Existing evidence suggests that IMIs targeting SAD in university students might have beneficial effects up to one year with or without guidance [23–25]. However, the evidence base is weak because studies mainly focused on fear of public speaking [23, 25], had substantial dropout rates [24, 25], and were underpowered [23].

Apart from the efficacy of IMIs for SAD, evidence to support their cost-effectiveness is still insufficient. While the assessment of the efficacy takes the benefits for patients into account, the assessment of economic consequences also considers a wider perspective by providing insights into societal costs and benefits. Only three studies investigated the economic merits of IMIs for SAD, indicating that guided [26–28] and unguided [29] IMIs may be a cost-effective treatment option in the long term (≥6 months) in the general population. However, it should be considered that evidence is still limited due to high dropout rates [29], few statistical analyses, and the absence of a full health economic evaluation [28].

Previously published results of this study (10 weeks after randomization) showed that IMI was superior to a waitlist control (WLC) group in reducing SAD symptom severity in university students with moderate to large effect sizes [30]. In this study, we report its clinical effects at 6-month follow-up and the health economic outcomes from a societal and public healthcare perspective over a 6-month period.

2. Methods

2.1. Study Design and Participants

A two-arm randomized controlled design was used to allocate 200 participants (block size of 8, varying ratio) to either the intervention (IMI) or control (WLC) group [31]. Alongside this trial, a cost-effectiveness analysis was conducted. An independent investigator randomized participants applying the RandList program [32]. Self-reported measurements were collected over three measurement points using a secured web-based assessment system (UNIPARK [33], 256-bit encrypted): at baseline (T0), posttreatment (T1; 10 weeks after randomization), and 6-month follow-up (T2) (Figure 1). The participants were recruited via e-mails sent to enrolled students at universities in Switzerland (), Austria (), and Germany () from January 2017 to February 2018. The applicants met the following inclusion criteria: they were 18 years or older, scored >21 on the Social Phobia Scale (SPS) and/or>32 on the Social Interaction Anxiety Scale (SIAS), met the diagnostic criteria of SAD according to the structured clinical interview for DSM-IV axis I disorders (SCID-I), and provided written informed consent. The interviews were conducted by trained interviewers via telephone [34]. The exclusion criteria included individuals at risk of suicide, showing dissociative symptoms, being diagnosed with psychosis, or currently undergoing psychotherapy. The Ethics Committee of the Friedrich-Alexander-University Erlangen-Nuremberg, Germany, approved the study. The trial was registered in the German clinical trial registry (DRKS00011424) on December 14^th, 2016. More details on the study design, diagnostic interview, and participants can be found in the study protocol [31].

2.2. Intervention

The intervention consisted of nine weekly sessions (approx. 60 minutes each) based on the cognitive-behavioral treatment of Clark and Wells [35, 36]. It contained text-based information material, various exercises (e.g., attention training), and diaries (e.g., identifying and questioning negative thoughts). The sessions were based on motivational enhancement (Session 1); psychoeducation (Session 2); identification and modification of negative thoughts through a thought diary (Sessions 3) and cognitions related to fear of positive evaluation (FPE) (Session 4); exercises to reduce self-focused attention, including behavioral experiments, such as in vivo exposures (Sessions 5–7); healthy lifestyle and problem-solving skills (Session 8); and relapse prevention (Session 9). The focus on FPE reflected a new treatment element that included the core components of SAD: fear of negative and positive evaluation [37]. A detailed description of the intervention can be found elsewhere [31].

3. Outcome Measures

3.1. Primary Outcome

The level of social anxiety was measured by the SPS and the SIAS [38]. These two self-report questionnaires complement each other and are usually administered together. The SIAS assesses more general fears of social interaction, whereas the SPS focuses on fears of being judged by others during daily activities. Each scale consists of 20 items rated on a 5-point Likert scale (0 = “not at all” to 4 = “extremely”). These two scales have been found to be valid, reliable, and useful for clinical and research purposes [39]. Cronbach’s α for the SIAS and SPS ranged from 0.90 to 0.94 [40] (current sample, ). Symptom-free status was operationalized as on the SPS and ≤26 on the SIAS [41]. Stangier et al. [38] proposed these cut-off values to differentiate between social phobic cases and non-social phobic cases. Diagnostic status was assessed via a diagnostic interview (SCID-I) by an interviewer blinded to the treatment condition after 6 months. Interrater reliability was evaluated in 20% of randomly selected participants.

3.2. Secondary Outcomes

The secondary outcomes are as follows: (i)Symptoms of social anxiety (the Liebowitz Social Anxiety Scale [LSAS-SR] assesses fear and avoidance in 24 different situations) [42] (Cronbach’s in the current sample)(ii)Fear of positive evaluation (Fear of Positive Evaluation Scale [FPES] showing good psychometric properties in clinical and healthy samples [10 items] [43, 44] [], Disqualification of Positive Social Outcomes Scale [DPSOS] [13 items] [45] [])(iii)Depressive symptoms (Beck Depression Inventory II [BDI-II] [21 items] has shown high reliability and validity in SAD clients) [36, 46] ()(iv)General psychopathology (Brief Symptom Inventory [BSI] [53 items, 9 dimensions] and Global Severity Index [GSI] as overall mean score reported and robust psychometric properties) [47, 48] ()(v)Interpersonal problems (Inventory of Interpersonal Problems [IIP-64] [8 dimensions] has shown adequate psychometric properties) [49, 50] ()

4. Health Economic Evaluation

4.1. Quality-Adjusted Life Years (QALYs)

QALYs were computed using the Assessment of Quality of Life (AQoL-8D) and the EuroQol (EQ-5D-5L) instruments. The AQoL-8D assesses eight dimensions (independent living, pain, senses, mental health, happiness, coping, relationships, and self-worth) and is a reliable and valid instrument [51] with Cronbach’s α of 0.96. The EQ-5D-5L is a widely applied, valid, and reliable measurement of QoL [52]. It consists of five items on a 5-point Likert scale related to mobility, self-care, common activities, pain/discomfort, and anxiety/depression. Utility values were derived using instrument-specific utility weights (EQ-5D [53] and AQoL [54]). QALYs were calculated using the area-under-the-curve (AUC) method of linearly interpolated utilities between measurements to cover the whole 6-month follow-up period. The EQ-5D was only used for sensitivity analyses.

4.2. Costs

We retrospectively assessed the 3-month healthcare costs, productivity losses, and patient and family costs using the “Trimbos Institute and Institute of Medical Technology Questionnaire for Costs Associated with Psychiatric Illness” (TiC-P) adapted to the German healthcare system [55]. The TiC-P is a frequently used and reliable self-report instrument for healthcare utilization and productivity losses among (German) [56, 57] patients with mental illnesses [58]. For each participant, the units of resource use were multiplied by the standard unit cost prices [59]. The intervention’s market price was estimated at €150 (US$198.89) per participant, reflecting costs due to maintenance and hosting. In addition to the German value added tax of 19%, the interventions costs were €178.50 (US$236.68). It was presumed that each participant had a computer and internet access. The costs of therapeutic appliances and medication were obtained from the Lauer-Taxe [60] and calculated according to the method of Bock et al. [59]. German healthcare costs were last published in 2011 and therefore indexed from 2011 to 2017 (index factor of 1.09) according to the German consumer price index [61].

Costs stemming from productivity losses through absenteeism and presenteeism were only assessed in students who had a paid job. Using the human capital approach [62], absenteeism costs were calculated as the number of working days lost due to illness multiplied by the average gross daily wage of the student’s monthly salary. Presenteeism was determined based on an inefficiency score (Osterhaus method [63]) multiplied by the number of working days affected. Then, costs of presenteeism were calculated based on the student’s gross daily wages. Productivity losses generated by unpaid work, such as daily chores, were valued using a shadow price of €19.63 (US$26.03) per hour for domestic help. Table S1 shows additional costing information.

The AUC method was applied to estimate cumulated costs by linearly interpolating costs over a 3-month period measured at baseline, post, and follow-up to cover the entire 6-month follow-up period [64]. All costs were expressed in Euros (€) for 2017 (December), the year in which the study was conducted. Purchasing power parities based on the Organization for Economic Cooperation and Development were used to convert costs to US dollars [65] (reference year: 2017; €1 was equated to US$1.33). The resource utilization in Austria and Switzerland was valued using German standard unit cost prices to increase consistency in costs and minimize confounding.

4.3. Evaluation of Clinical Outcomes

This study was conducted to detect a mean standardized difference of in the primary outcomes (SPS/SIAS) between the groups at post-measurement [31]. The results were reported according to the Consolidated Standards of Reporting Trials statement [66] using intention-to-treat (ITT) procedures. Missing clinical outcome data were imputed by applying a Markov Chain Monte Carlo multivariate imputation algorithm with 10 estimations per missing value [67].

The evaluation of the clinical outcomes was performed using the SPSS software [68]. The IMI and WLC were compared six months after randomization (T2) using analysis of covariance (ANCOVA) with baseline levels as covariates. Due to the violation of normally distributed error terms of many outcomes, robust ANCOVA was used [69]. These analyses were adjusted for multiple testing. Hence, α was set at a level of <0.025 [70] for testing the primary outcomes and <0.05 for all other tests. Cohen’s with 95% CIs was calculated.

Treatment response and clinically significant deterioration were defined by the Reliable Change Index [38]. The participants were defined as reliably improved if their SPS (SIAS) score declined from baseline to 6-month follow-up with more than 1.96 standard units, while also considering the reliability of the measurement instruments to compensate for random measurement error. The participants met the criteria for reliable change when they had improved (deteriorated) at least 7.03 points on the SPS and 9.53 points on the SIAS. Moreover, the participants were rated as symptom-free if they scored 17 or below on the SPS and 26 or below on the SIAS [38]. To further guide the clinical interpretation, the numbers needed to treat (NNT) were calculated [71, 72]. Differences in symptom-free status, reliable change, and diagnostic status as assessed by SCID interviews between the groups were assessed at follow-up using the chi-squared test.

4.4. Health Economic Evaluation

The evaluation adhered to standards set by the Consolidated Health Economic Evaluation Reporting Standard (CHEERS) and International Society for Pharmacoeconomics and Outcomes Research (ISPOR RCT-CEA Task Force Report) [73, 74].

All data was analyzed based on the ITT principle. Thus, missing cost data was imputed using the regression imputation procedure using the predictors of the outcome (e.g., baseline costs, annual gross salary, and status of employment) and dropout rate (e.g., sex and age) that were identified via logistic regression analysis.

The ordinary least square (OLS) regression was used to estimate QALYs while controlling for baseline utility values. From a societal and public healthcare perspective, cost categories and cost per study arm were evaluated by OLS regression models. One cost outlier was identified by calculating the Mahalanobis distance based on the total cost (, in the WLC) and handled using winsorization where it was substituted by the value at the 99th percentile [75]. No discounting of costs and effects was applied because the follow-up period did not exceed one year. The incremental cost-effectiveness ratio (ICER) displays the incremental costs per unit of the effect (QALY, symptom-free status). The ICER was calculated as , where the are cumulated over a period of 6 months and are reflected by the symptom-free status or QALY gains.

For our economic analyses, we applied a probabilistic decision-making approach [76] that accounts for stochastic uncertainty [77] in the study data. It provides the decision-maker with information about probabilities rather than statistical significance. A 5000-fold bootstrapped seemingly unrelated regression equation model on costs and effects was used to generate the incremental costs and effects, while adjusting for baseline utilities, age, and prior psychotherapy [78]. The 5000 bootstrap replications of cost-and-effect pairs were used to obtain 95% confidence intervals and plotted in a cost-effectiveness plane. The plane depicts the incremental effects between the intervention and control group on the -axis and the incremental costs between the groups on the -axis. The intervention “dominates” the control groups if better effects are obtained for lower costs. Hence, the majority of simulated ICER falls in the southeast quadrant. In contrast, in the northwest quadrant, the intervention is “inferior” to the control group as higher costs are associated with worse health outcomes. Thus, it is not considered cost-effective [62]. In the southwest quadrant, an intervention is less effective and less costly than the control group. On the other hand, in the northeast quadrant, an intervention is more effective and more costly than the control condition. Here, the amount of money a decision-maker is willing to pay for one additional positive outcome determines the adoption of a new intervention. A cost-effectiveness acceptability curve was displayed that indicates the probability of cost-effectiveness of an IMI at varying WTP ceilings, given that there is no common threshold for gaining one unit of health (e.g., symptom-free status). Analyses were carried out with Stata version 16.1 [79].

4.5. Sensitivity Analyses

The sensitivity analyses were conducted to inspect the robustness of our results. First, the data of the participants who completed the 6-month follow-up assessment were analyzed. Second, changing market prices can lead to varying intervention costs, which explains why the increased intervention costs were examined (+50%, 100%). Third, Swiss students (, 20%) were excluded to validate the robustness of the findings. Their number and employment rate were balanced across groups, but differences in healthcare settings and salaries could have biased the results. Fourth, to facilitate comparability across studies, the widely applied EQ-5D instrument was used to generate QALYs. Lastly, the diagnostic status as a meaningful effect outcome for policymakers was used for the cost-effectiveness analysis.

5. Results

5.1. Sample

The sample predominately consisted of female (, 62%) German university students (, 75%) aged 27 (SD 6.34) (Table S2). A comprehensive description of the study sample and the participant flow can be found elsewhere [31]. We did not observe any clinically relevant baseline differences between the study conditions. The dropout rates between IMI (, 32%) and WLC (, 13%) differed significantly (; ; ), yet the dropout rate was not associated with the sociodemographic factors or the baseline SAD symptoms. The attrition rate was 22.5% (45/200) at the 6-month follow-up.

5.2. Outcome Measures

As shown in Table 1, the IMI was associated with lower scores on both primary outcome measures than WLC. These between-group differences were statistically significant: SPS, , ; SIAS, , . The corresponding standardized effect sizes were moderate for SIAS (, 95% CI [0.30, 0.87]) and large for SPS (, 95% CI [0.54, 1.10]). Fewer participants in the IMI () than in the WLC () presented with a clinical diagnosis of social phobia assessed through a SCID interview (; ; ). The inter-rater reliability showed substantial agreement (Cohen’s kappa, [80]).

5.3. Treatment Response, Symptom-Free Status, and Symptom Deterioration

After 6 months, significantly more participants in the IMI showed a reliable improvement and achieved a symptom-free status compared with those in the WLC based on the SPS and the SIAS. Likewise, the clinically significant deterioration was lower in the IMI compared with the WLC for both outcomes (Table S3).

5.4. Secondary Outcome Analyses

Table 2 shows the results for the secondary outcomes, interpersonal problems, depression, somatic symptoms, FPE, and QoL. Significant between-group differences for all outcomes, except the EQ-5D, with effect sizes ranging from (95% CI [0.05, 0.50]) for the AQoL to (95% CI [0.47, 1.05]) for the LSAS-SR were observed.

6. Health Economic Evaluation

6.1. Health Outcomes

Regarding symptom-free status, the IMI significantly differed from WLC on the SPS [incremental effect (; 95% CI, 0.15–0.37)] and on the SIAS [incremental effect (; 95% CI, 0.14–0.34)]. On average, the participants in the IMI gained 0.66 QALYs (95% CI, 0.64–0.67) during follow-up, whereas the participants in the WLC gained 0.61 QALYs (95% CI, 0.59–0.62). Statistically significant differences in the adjusted incremental QALYs were observed (; 95% CI, 0.02–0.07).

6.2. Costs

At baseline, the mean total costs only differed (€138; US$183) moderately between the IMI (€464, US$615) and the WLC (€603; US$800). Table 3 displays the average accumulated costs over a 6-month period per participant by study arm. After 6 months, the total incremental costs were -€211 in favor of the intervention group (IMI, €850; WLC, €1061). The average healthcare costs were higher in the IMI (€345) compared with the WLC (€240). The patient and family costs were similar in both groups slightly favoring the IMI. Productivity losses especially presenteeism at work produced the highest cost differences of -€227 (IMI, €391; WLC, €618) exceeding the intervention costs.

6.3. Societal Perspective

Table 4 shows the incremental costs, effects, and ICERs based on the 5000 bootstraps. The IMI dominated the WLC related to the symptom-free status with larger effects on the SPS and SIAS and less costs (SPS, -€321, 95% CI [−862, 66]; SIAS, -€324, 95% CI [−774, 125]). In the cost-effectiveness plane, the majority of ICERs fell under the southeast quadrant (Figure 2), reflecting a 92% probability that the intervention generates greater health effects at lower costs than WLC (Figure 3).

The IMI generated small QALY gains at lower costs (-€319, 95% CI, −831–64) compared with the WLC. From a societal perspective, 93% of the simulated ICERs fell under the southeast quadrant reflecting the intervention’s probability of dominating WLC (Figure 4). Assuming a WTP of €1000 for QALY gains, the probability rose to 95% (Figure 5).

6.4. Healthcare Perspective

The bootstrapped ICER related to the symptom-free status on the SPS and SIAS yielded an ICER of €348 (95% CI, SPS [−284,1069]; SIAS [304, 1043]), indicating that the IMI generated larger effects at higher costs (SPS, €81; SIAS, €79) compared with the WLC. Hence, the majority of ICERs fell under the northeast quadrant (86%), while the probability of cost-effectiveness of the intervention compared to WLC rose from 70/69% at a WTP of €500 to 97% at a WTP of €1000 (Figures 6 and 7). Regarding the cost utility, the IMI generated higher effects per QALY gained at higher costs compared with WLC (€81; 95% CI, 105–200). Thus, 86% of the simulated ICERs fell in the northeast quadrant, showing higher QALY gains and costs (Figure 8). At a WTP of €0, €2000, and €6000 for gaining one QALY, the probability rose from 14% to 54% to 96% (Figure 9).

6.5. Sensitivity Analyses

First, the study completers generated significant effects on all assessed outcomes and effect sizes at least as large as in the ITT analysis (). Second, regarding the QALY gains, even increasing the invention costs by 50% and 100% did not alter the interpretation of results either from a societal or healthcare perspective, respectively (Table 2). Third, excluding the Swiss students did not affect the results of the CEA or the CUA analyses. Fourth, using the EQ-5D-5L resulted in a slightly nonsignificant (, ) incremental QALY gain of the IMI (0.939 QALY, SD 0.070) compared with the WLC (0.932 QALY, SD 0.865). The greater sensitivity of the AQoL instrument and the potential ceiling effect of the EQ-5D instrument may have led to the differences between the AQoL QALYs (>0.55) and the EQ-5D QALYs (>0.9). Nevertheless, for gaining a QALY at a WTP of €0/€10,000, the probability of cost-effectiveness was similar (99%/89%) from a societal perspective and lower (15%/26%) from a healthcare perspective compared with the AQoL QALYs. Fifth, using the diagnostic status for the health economic evaluation yielded similar results to the symptom-free status (Table 4).

7. Discussion

7.1. Principal Findings

This study is the first to evaluate the long-term efficacy and the cost-effectiveness of an unguided IMI for university students with SAD compared with a waitlist control condition (WLC) with unrestricted access to treatment as usual over 6 months from a societal and public healthcare perspective. The IMI maintained a significant and favorable effect on social phobia symptoms with moderate to large effect sizes between groups at follow-up assessment (6 months; SPS, ; SIAS, ) compared with the WLC. The IMI generated slightly lower costs (-€321; 95% CI, −862–66), more QALYs (0.046; 95% CI, 0.024–0.68), and symptom-free status (; 95% CI, 0.15–0.37) compared with the WLC in the long term. From a societal perspective, the IMI dominated the WLC, while from a healthcare perspective, the probability for cost-effectiveness was 96% at a WTP of €6000 (US$7956) per symptom-free status and QALY. Our findings were robust to sensitivity analyses.

7.2. Comparison with Prior Work

Our findings are consistent with the evidence on the efficacy of unguided IMIs in the general population suffering from SAD in the long term [19]. A study () compared an unguided IMI with two types of guided self-help after 6 months [36] and found within-group effects () similar to our study, but no significant effect was observed between the groups. Likewise, smaller but also persistent effects () of an unguided IMI were found when compared with WLC after one year [29].

Regarding the university students, our findings support the existing evidence for internet-based interventions targeting SAD showing similar results as the previous studies [24] (some focusing on fear of public speaking [23, 25]). Furthermore, these studies are characterized by a substantial dropout rate at posttreatment [24] (40%) or follow-up [25] (67%) and did not assess the long-term efficacy of the intervention.

Likewise, further research is needed to confirm the economic benefits of IMIs for SAD. Results from our trial add to the converging evidence pointing to their cost-effectiveness. Four studies reported on health economic outcomes of IMIs in SAD [26–29]. Three studies found that guided IMIs might be a cost-effective approach compared with an active control over a period of 6 months and 4 years from a societal and provider perspective, respectively. Showing similar results to our study, one guided IMI generated less costs and better treatment outcomes. At a WTP of 0 Euro, this IMI showed an 81% probability of cost-effectiveness at 6 months [25] and a 61% probability at a 4-year follow-up [27]. Another guided IMI compared with face-to-face treatment was judged as cost-effective, only including cost for therapist time [27].

To our knowledge, only one health economic evaluation has compared unguided IMI and passive control [29], which indicated that the intervention is likely to be cost-effective. The study differs from the present study in terms of the general population, higher average age, different inclusion criteria (also subclinical participants), and instruments used (SF-6D versus AQoL). The IMI generated higher costs and better effects at 6 months and dominated the control condition after 12 months. Compared with our study, this trial focused on a nonclinical sample and had a substantial dropout rate (50%) which could lead to selection bias by attrition.

7.3. Strengths and Limitations

This study has several strengths and limitations. The data in our trial was based on self-report measures and diagnostic interviews, thereby increasing the robustness of the results while minimizing bias accompanying self-reported data due to recall period or selective recall. Evidence-based assessment consisting of more than one assessment technique is recommended for accurate assessment and effective treatment of anxiety disorders [81]. Additionally, our study only showed a relatively small number of dropouts (22.5%) at 6-month follow-up compared with most unguided studies [82]. This could be due to our structured research process (e.g., diagnostic interviews) and the highly educated and technologically sophisticated group of participants. Moreover, the characteristics of the study groups were well balanced at baseline, and the proportion of female participants (62%) was relatively low compared with most IMIs also reflecting the distribution of SAD in the population. A further strength of our study is the inclusion of a full economic evaluation. Notably, the economic findings were consistent across various sensitivity analyses.

However, several limitations were noted. First, the time horizon of our study was limited to 6 months. Costs due to present and future underachievement, prolonged studies, and study dropouts resulting in lower academic qualifications could not be captured nor included in our evaluations and may even increase the impact of the intervention. Second, most instruments used self-report measures, which might have led to “social desirability” and “recall bias.” Third, there are several possible reasons why our study generated relatively high effect sizes compared to other IMIs targeting SAD. Significant structuring and research attention, e.g., before and after the intervention via diagnostic interviews, could have led to the overestimation of effects compared with interventions in routine care due to the Hawthorne effect [83]. Additionally, participants were characterized by high symptom severity at baseline assessment giving them more room for improvement compared to samples with subclinical symptoms. Adherence and motivation were increased by automated reminders and telephone reminders for questionnaire completion. Fourth, our findings can be generalized to similar settings. Thus, our results may be applicable to university students in Western countries with similar study characteristics. Finally, in health economic evaluations [84], a standard care comparator (e.g., face-to-face CBT) is recommended rather than a waitlist control group to avoid potential nocebo effects [85].

7.4. Clinical Implications and Future Research

The study results support the idea that IMIs could be a low-threshold, effective, and affordable way to reduce the adverse effects of social anxiety disorder. The advantage of an IMI is that compared with standard therapies, the marginal cost decreases when extending coverage or increasing the uptake of the intervention [86]. Low marginal costs and the scalability of IMIs enable them to be used as public health tools to generate effects at the population level compared with guided or face-to-face treatments that are not as scalable.

Moreover, based on the nature of social anxiety disorder, patients tend to avoid face-to-face contact with the therapist for fear being negatively evaluated. IMIs offer a low-threshold approach to help those affected without using face-to-face (F2F) treatments. This preference may also increase the use of unguided approaches compared with standard therapies. Therefore, future research should evaluate head-to-head comparisons of self-help IMIs with guided IMIs and F2F treatments.

Furthermore, in our study, only students who met the diagnostic criteria were included. Nevertheless, the students who were interested in participating but showing subclinical symptoms of SAD increased threefold. Thus, further research is needed to investigate the preventive effect of this IMI in students at risk of developing SAD. The application of this IMI across the range of mild to severe symptoms of SAD may better fit the requirements of a student mental health service at a university. However, under uncontrolled naturalistic conditions, the lack of research attention and reminders may decrease the effectiveness of unguided IMIs. Implementation studies could further examine the uptake and effectiveness of unguided IMIs under routine care conditions.

The German SAD treatment guidelines [87] recommend IMIs based on cognitive-behavioral therapy to bridge the waiting times or accompanying face-to-face treatment. Moreover, the results of our study add to the emerging evidence base in support of recommending IMIs as a viable treatment option in clinical guidelines. An official recognition of IMIs as a treatment option for SAD would help bridge the current treatment gap. During the COVID-19 pandemic, the barriers of treatment utilization increased and SAD symptoms in students were maintained due to minimal social contact and isolation [88]. Untreated persons generate long-term costs caused by persistent social anxiety symptoms, such as low academic performance, subsequent study dropouts, and worse job prospects, which are not included in our health economic evaluation. For the future, larger studies with longer follow-up periods are needed to investigate the full extent of SAD from a cost viewpoint.

8. Conclusion

This study strengthens the existing evidence confirming that internet-based self-help interventions for SAD can generate and sustain a significant and favorable effect in reducing social anxiety symptoms in a cost-effective way. Given the positive effects of the intervention, the implementation of this IMI as part of a student’s healthcare management at the university would be essential.

Data Availability

The datasets generated and/or analyzed during the current study are available from the corresponding author on reasonable request.

Additional Points

Key Points. (i) Unguided internet- and mobile-based interventions can induce and maintain favorable effects in students with social anxiety disorder over a period of 6 months. (ii) The mobile interventions have an acceptable likelihood of cost-effectiveness. (iii) It would be helpful to integrate such mobile interventions into routine mental healthcare at universities.

Disclosure

The funders did not have a role in the study design, data collection, analysis and interpretation of the results, or the decision to publicize the study results.

Conflicts of Interest

DDE has served as a consultant to/on the scientific advisory boards of Sanofi, Novartis, Minddistrict, Lantern, Schoen Kliniken, Ideamed, German health insurance companies (BARMER and Techniker Krankenkasse), and a number of federal chambers for psychotherapy. He is also a stakeholder in the Institute for Health Training Online (formerly GET.ON, now HelloBetter), which is aimed at implementing scientific findings related to digital health interventions into routine care. HB received consultancy fees, reimbursement of congress attendance, and travel costs as well as payments for lectures from psychotherapy, psychiatry, and further medical associations, institutes, clinics, and companies in the context of e-mental-health topics. He sublicensed a digital intervention to a company providing digital health interventions. He has been the beneficiary of study support (third-party funding) from several public funding organizations. The other authors, CB, FK, FS, and TB, declare no competing interest.

Authors’ Contributions

FK, DDE, and TB have contributed to the study design. FK drafted the manuscript. CB and FS contributed to the analysis and interpretation of the data. CB, DDE, FS, HB, and TB critically revised the content. All authors read and approved the final manuscript.

Acknowledgments

The authors would like to thank all participants, research assistants such as Julia Burger, Mirjam Thomas, therapists, and all others who contributed to our study. Funding was received from BARMER GEK (German statutory health insurance company). DDE and HB obtained the funding for this trial. Open access funding was enabled and organized by Projekt DEAL.

Supplementary Materials

Table S1: unit costs for the type of health service utilized by the participants. Table S2: baseline sample characteristics. Table S3: treatment response, symptom-free status, and symptom deterioration at 6-month follow-up. (Supplementary Materials)

References

R. P. Auerbach, P. Mortier, R. Bruffaerts et al., “WHO world mental health surveys international college student project: prevalence and distribution of mental disorders,” Journal of Abnormal Psychology, vol. 127, no. 7, pp. 623–638, 2018.
View at: Publisher Site | Google Scholar
L. Farrer, A. Gulliver, J. K. Y. Chan et al., “Technology-based interventions for mental health in tertiary students: systematic review,” Journal of Medical Internet Research, vol. 15, no. 5, article e101, 2013.
View at: Publisher Site | Google Scholar
F. Jacobi, M. Höfler, J. Strehle et al., “Twelve-months prevalence of mental disorders in the German health interview and examination survey for adults–mental health module (DEGS1-MH): a methodological addendum and correction,” International Journal of Methods in Psychiatric Research, vol. 24, no. 4, pp. 305–313, 2015.
View at: Publisher Site | Google Scholar
M. Baldauf, A. Thomas, and B. Strau, “Symptoms of social phobia and their relationship to interpersonal characteristics in a sample of German medical students,” PPmP - Psychotherapie Psychosomatik Medizinische Psychologie, vol. 64, no. 2, pp. 76–81, 2014.
View at: Publisher Site | Google Scholar
B. K. Gültekin, I. F. Dereboy, and F. Dereboy, “The prevalence of social phobia, and its impact on quality of life, academic achievement, and identity formation in university students,” Turk Psikiyatri Dergisi, vol. 22, pp. 150–158, 2011.
View at: Google Scholar
D. M. Gilles, C. L. Turk, and D. M. Fresco, “Social anxiety, alcohol expectancies, and self-efficacy as predictors of heavy drinking in college students,” Addictive Behaviors, vol. 31, no. 3, pp. 388–398, 2006.
View at: Publisher Site | Google Scholar
M. Olfson, M. Guardino, E. Struening, F. R. Schneier, F. Hellman, and D. F. Klein, “Barriers to the treatment of social anxiety,” The American Journal of Psychiatry, vol. 157, no. 4, pp. 521–527, 2000.
View at: Publisher Site | Google Scholar
M. Tillfors and T. Furmark, “Social phobia in Swedish university students: prevalence, subgroups and avoidant behavior,” Social Psychiatry and Psychiatric Epidemiology, vol. 42, no. 1, pp. 79–86, 2007.
View at: Publisher Site | Google Scholar
R. C. Kessler, “The impairments caused by social phobia in the general population: implications for intervention,” Acta Psychiatrica Scandinavica, vol. 108, no. s417, pp. 19–27, 2003.
View at: Publisher Site | Google Scholar
C. Acarturk, F. Smit, R. de Graaf, A. van Straten, M. Ten Have, and P. Cuijpers, “Economic costs of social phobia: a population-based study,” Journal of Affective Disorders, vol. 115, no. 3, pp. 421–429, 2009.
View at: Publisher Site | Google Scholar
W. J. Magee, W. W. Eaton, H.-U. Wittchen, K. A. McGonagle, and R. C. Kessler, “Agoraphobia, simple phobia, and social phobia in the National Comorbidity Survey,” Archives of General Psychiatry, vol. 53, no. 2, pp. 159–168, 1996.
View at: Publisher Site | Google Scholar
N. Stuhldreher, E. Leibing, F. Leichsenring et al., “The costs of social anxiety disorder: the role of symptom severity and comorbidities,” Journal of Affective Disorders, vol. 165, pp. 87–94, 2014.
View at: Publisher Site | Google Scholar
D. D. Ebert, P. Mortier, F. Kaehlke et al., “Barriers of mental health treatment utilization among first‐year college students: First cross-national results from the WHO World Mental Health International College Student Initiative,” International Journal of Methods in Psychiatric Research, vol. 28, no. 2, article e1782, 2019.
View at: Publisher Site | Google Scholar
Y. Zhang, J. Chen, W. Gao et al., “From fears of evaluation to social anxiety: the longitudinal relationships and neural basis in healthy young adults,” International Journal of Clinical and Health Psychology, vol. 23, no. 2, article 100345, 2023.
View at: Publisher Site | Google Scholar
M. J. Chartier, A. L. Hazen, and M. B. Stein, “Lifetime patterns of social phobia: a retrospective study of the course of social phobia in a nonclinical population,” Depression and Anxiety, vol. 7, no. 3, pp. 113–121, 1998.
View at: Publisher Site | Google Scholar
T. Berger and G. Andersson, “Internetbasierte psychotherapien: besonderheiten und empirische evidenz,” PPmP - Psychotherapie Psychosomatik Medizinische Psychologie, vol. 59, no. 3/4, pp. 159–170, 2009.
View at: Publisher Site | Google Scholar
D. D. Ebert, T. Van Daele, T. Nordgreen et al., “Internet- and mobile-based psychological interventions: applications, efficacy, and potential for improving mental health,” European Psychologist, vol. 23, pp. 167–187, 2018.
View at: Publisher Site | Google Scholar
D. D. Ebert, P. Cuijpers, R. F. Muñoz, and H. Baumeister, “Prevention of mental health disorders using internet- and mobile-based interventions: a narrative review and recommendations for future research,” Frontiers in Psychiatry, vol. 8, 2017.
View at: Publisher Site | Google Scholar
I. L. Kampmann, P. M. G. Emmelkamp, and N. Morina, “Meta-analysis of technology-assisted interventions for social anxiety disorder,” Journal of Anxiety Disorders, vol. 42, pp. 71–84, 2016.
View at: Publisher Site | Google Scholar
S. El Alaoui, E. Hedman, B. Ljótsson, and N. Lindefors, “Long-term effectiveness and outcome predictors of therapist-guided internet-based cognitive-behavioural therapy for social anxiety disorder in routine psychiatric care,” BMJ Open, vol. 5, no. 6, article e007902, 2015.
View at: Publisher Site | Google Scholar
E. Hedman, T. Furmark, P. Carlbring et al., “A 5-year follow-up of internet-based cognitive behavior therapy for social anxiety disorder,” Journal of Medical Internet Research, vol. 13, no. 2, article e39, 2011.
View at: Publisher Site | Google Scholar
T. Berger, E. Hohl, and F. Caspar, “Internetbasierte therapie der sozialen phobie: ergebnisse einer 6-monate-katamnese,” Zeitschrift für Klinische Psychologie und Psychotherapie, vol. 39, no. 4, pp. 217–221, 2010.
View at: Publisher Site | Google Scholar
M. Tillfors, P. Carlbring, T. Furmark et al., “Treating university students with social phobia and public speaking fears: internet delivered self-help with or without live group exposure sessions,” Depression and Anxiety, vol. 25, no. 8, pp. 708–717, 2008.
View at: Publisher Site | Google Scholar
H. C. McCall, C. G. Richardson, F. D. Helgadottir, and F. S. Chen, “Evaluating a web-based social anxiety intervention among university students: randomized controlled trial,” Journal of Medical Internet Research, vol. 20, no. 3, article e8630, 2018.
View at: Publisher Site | Google Scholar
C. Botella, M. J. Gallego, A. Garcia-Palacios et al., “An internet-based self-help treatment for fear of public speaking: a controlled trial,” Cyberpsychology, Behavior, and Social Networking, vol. 13, 2010.
View at: Publisher Site | Google Scholar
E. Hedman, E. Andersson, B. Ljotsson et al., “Cost-effectiveness of internet-based cognitive behavior therapy vs. cognitive behavioral group therapy for social anxiety disorder: results from a randomized controlled trial,” Behaviour Research and Therapy, vol. 49, no. 11, pp. 729–736, 2011.
View at: Publisher Site | Google Scholar
E. Hedman, S. El Alaoui, N. Lindefors et al., “Clinical effectiveness and cost-effectiveness of Internet- vs. group-based cognitive behavior therapy for social anxiety disorder: 4-Year follow-up of a randomized trial,” Behaviour Research and Therapy, vol. 59, pp. 20–29, 2014.
View at: Publisher Site | Google Scholar
N. Titov, G. Andrews, L. Johnston, G. Schwencke, and I. Choi, “Shyness programme: longer term benefits, cost-effectiveness, and acceptability,” The Australian and New Zealand Journal of Psychiatry, vol. 43, no. 1, pp. 36–44, 2009.
View at: Publisher Site | Google Scholar
J. Powell, V. Williams, H. Atherton et al., “Effectiveness and cost-effectiveness of a self-guided internet intervention for social anxiety symptoms in a general population sample: randomized controlled trial,” Journal of Medical Internet Research, vol. 22, no. 1, article e16804, 2020.
View at: Publisher Site | Google Scholar
F. Kählke, T. Berger, A. Schulz et al., “Efficacy of an unguided internet-based self-help intervention for social anxiety disorder in university students: a randomized controlled trial,” International Journal of Methods in Psychiatric Research, vol. 28, no. 2, article e1766, 2019.
View at: Publisher Site | Google Scholar
F. Kählke, T. Berger, A. Schulz et al., “Efficacy and cost-effectiveness of an unguided, internet-based self-help intervention for social anxiety disorder in university students: protocol of a randomized controlled trial,” BMC Psychiatry, vol. 19, no. 1, article e1766, 2019.
View at: Publisher Site | Google Scholar
Datinf GmbH, “Randlist,” Software to Create Randomisations Lists, 2018.
View at: Google Scholar
Questback GmbH, EFS Survey, Version Summer 2017, Datinf GmbH Tübingen, 2017.
P. Rohde, P. M. Lewinsohn, and J. R. Seeley, “Comparability of telephone and face-to-face interviews in assessing axis I and II disorders,” The American Journal of Psychiatry, vol. 154, no. 11, pp. 1593–1598, 1997.
View at: Publisher Site | Google Scholar
D. Clark and A. Wells, A Cognitive Model of Social Phobia, Guilford Press, 1995.
T. Berger, F. Caspar, R. Richardson, B. Kneubühler, D. Sutter, and G. Andersson, “Internet-based treatment of social phobia: a randomized controlled trial comparing unguided with two types of guided self-help,” Behaviour Research and Therapy, vol. 49, no. 3, pp. 158–169, 2011.
View at: Publisher Site | Google Scholar
J. W. Weeks and A. N. Howell, “The bivalent fear of evaluation model of social anxiety: further integrating findings on fears of positive and negative evaluation,” Cognitive Behaviour Therapy, vol. 41, no. 2, pp. 83–95, 2012.
View at: Publisher Site | Google Scholar
U. Stangier, T. Heidenreich, A. Berardi, U. Golbs, and J. Hoyer, “Die erfassung sozialer phobie durch die social interaction anxiety scale (SIAS) und die social phobia scale (SPS),” Zeitschrift für Klinische Psychologie und Psychotherapie, vol. 28, no. 1, pp. 28–36, 1999.
View at: Publisher Site | Google Scholar
R. P. Mattick and J. C. Clarke, “Development and validation of measures of social phobia scrutiny fear and social interaction anxiety,” Behaviour Research and Therapy, vol. 36, no. 4, pp. 455–470, 1998.
View at: Publisher Site | Google Scholar
N. Heinrichs, K. Hahlweg, W. Fiegenbaum, M. Frank, B. Schroeder, and I. von Witzleben, “Validität und reliabilität der social interaction anxiety scale (SIAS) und der social phobia scale (SPS),” Verhaltenstherapie, vol. 12, no. 1, pp. 26–35, 2002.
View at: Publisher Site | Google Scholar
N. S. Jacobson and P. Truax, “Clinical significance: a statistical approach to defining meaningful change in psychotherapy research,” Journal of Consulting and Clinical Psychology, vol. 59, no. 1, pp. 12–19, 1991.
View at: Publisher Site | Google Scholar
U. Stangier and T. Heidenreich, Die Liebowitz Soziale Angst-Skala (LSAS). Internationale Skalen für Psychiatrie, Hogrefe, 2003.
T. A. Fergus, D. P. Valentiner, P. B. McGrath, K. Stephenson, S. Gier, and S. Jencius, “The fear of positive evaluation scale: psychometric properties in a clinical sample,” Journal of Anxiety Disorders, vol. 23, no. 8, pp. 1177–1183, 2009.
View at: Publisher Site | Google Scholar
J. W. Weeks, R. G. Heimberg, T. L. Rodebaugh, and P. J. Norton, “Exploring the relationship between fear of positive evaluation and social anxiety,” Journal of Anxiety Disorders, vol. 22, no. 3, pp. 386–400, 2008.
View at: Publisher Site | Google Scholar
J. Weeks and W. The, “The disqualification of positive social outcomes scale: a novel assessment of a long-recognized cognitive tendency in social anxiety disorder,” Journal of Anxiety Disorders, vol. 24, no. 8, pp. 856–865, 2010.
View at: Publisher Site | Google Scholar
M. Hautzinger, F. Keller, and C. Kühner, Beck Depressions-Inventar (BDI-II), Harcourt Test Services, 2006.
A. Schlarb and M. Hautzinger, Klinisch-psychiatrische Ratingskalen für das Kindes-und Jugendalter, Hogrefe, Göttingen, Germany, 2011.
G. Franke, BSI, Brief Symptom Inventory von L. R. Derogatis (Kurzform der SCL-90-R) - deutsches Manual, Beltz Test GmbH, 2000.
L. Horowitz, S. E. Rosenberg, B. A. Baer, G. Ureño, and E. Al, “Inventory of interpersonal problems: psychometric properties and clinical applications,” Journal of Consulting and Clinical Psychology, vol. 56, no. 6, pp. 885–892, 1988.
View at: Publisher Site | Google Scholar
L. Horowitz, B. Strauß, and H. Kordy, Inventar zur Erfassung interpersonaler Probleme: IIP-D; deutsche Version, Beltz, 2000.
J. Richardson, A. Iezzi, M. A. Khan, and A. Maxwell, “Validity and reliability of the assessment of quality of life (AQoL)-8D multi-attribute utility instrument,” The Patient - Patient-Centered Outcomes Research, vol. 7, no. 1, pp. 85–96, 2014.
View at: Publisher Site | Google Scholar
M. Sonntag, A. Konnopka, F. Leichsenring et al., “Reliability, validity and responsiveness of the EQ-5D in assessing and valuing health status in patients with social phobia,” Health and Quality of Life Outcomes, vol. 11, no. 1, p. 215, 2013.
View at: Publisher Site | Google Scholar
K. Ludwig, J. M. Graf von der Schulenburg, and W. Greiner, “German value set for the EQ-5D-5L,” PharmacoEconomics, vol. 36, no. 6, pp. 663–674, 2018.
View at: Publisher Site | Google Scholar
J. Richardson, K. Sinha, A. Iezzi, and M. A. Khan, “Modelling utility weights for the assessment of quality of life (AQoL)-8D,” Quality of Life Research, vol. 23, no. 8, pp. 2395–2404, 2014.
View at: Publisher Site | Google Scholar
L. Hakkaart-van Roijen, A. Van Straten, M. Donker, and B. Tiemens, “Trimbos/iMTA questionnaire for costs associated with psychiatric illness (TIC-P),” Rotterdam Institure for Medical Technology Assessment (iMTA), 2002.
View at: Google Scholar
S. Nobis, D. Lehr, D. D. Ebert et al., “Efficacy and cost-effectiveness of a web-based intervention with mobile phone support to treat depressive symptoms in adults with diabetes mellitus type 1 and type 2: design of a randomised controlled trial,” BMC Psychiatry, vol. 13, no. 1, 2013.
View at: Publisher Site | Google Scholar
C. Buntrock, M. Berking, F. Smit et al., “Preventing depression in adults with subthreshold depression: health-economic evaluation alongside a pragmatic randomized controlled trial of a web-based intervention,” Journal of Medical Internet Research, vol. 19, pp. 1–16, 2017.
View at: Publisher Site | Google Scholar
C. Bouwmans, K. De Jong, R. Timman et al., “Feasibility, reliability and validity of a questionnaire on healthcare consumption and productivity loss in patients with a psychiatric disorder (TiC-P),” BMC Health Services Research, vol. 13, no. 1, p. 217, 2013.
View at: Publisher Site | Google Scholar
J.-O. Bock, D. Bowles, C. Brettschneider et al., Standardisierte Bewertungssätze aus gesellschaftlicher Perspektive für die gesundheitsökonomische Evaluation, vol. 55, Nomos Verlagsgesellschaft mbH & Co. KG, 2015.
L-F GmbH, Die LAUER-Taxe, CGM LAUER, 2016.
Destatis, Consumer Price Indices, Federal Statistical Office of Germany, 2015.
M. Drummond, M. J. Sculpher, K. Claxton, G. L. Stoddart, and G. W. Torrance, Methods for the Economic Evaluation of Health Care Programmes, Oxford University Press, 2015.
J. T. Osterhaus, D. L. Gutterman, and J. R. Plachetka, “Healthcare resource and lost labour costs of migraine headache in the US,” PharmacoEconomics, vol. 2, no. 1, pp. 67–76, 1992.
View at: Publisher Site | Google Scholar
J. N. Matthews, D. G. Altman, M. J. Campbell, and P. Royston, “Analysis of serial measurements in medical research,” BMJ, vol. 300, no. 6719, pp. 230–235, 1990.
View at: Publisher Site | Google Scholar
I. Shemilt, J. Thomas, and M. Morciano, “A web-based tool for adjusting costs to a specific target currency and price year,” Evidence & Policy, vol. 6, no. 1, pp. 51–59, 2010.
View at: Publisher Site | Google Scholar
K. F. Schulz, D. G. Altman, D. Moher, and for the CONSORT Group, “CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials,” BMJ, vol. 340, p. c332, 2010.
View at: Publisher Site | Google Scholar
R. J. A. Little and D. B. Rubin, “Statistical analysis with missing data,” in Wiley Series in Probability and Statistics, pp. 1–381, Wiley, 2014.
View at: Publisher Site | Google Scholar
IBM Corporate, IBM SPSS Statistics for Windows, Version 25.0, IBM Corporate, 2017.
A. Field, Andy Field - Discovering Statistics Using IBM SPSS Statistics, Sage, 2013.
Q. Shi, E. S. Pavey, and R. E. Carter, “Bonferroni-based correction factor for multiple, correlated endpoints,” Pharmaceutical Statistics, vol. 11, no. 4, pp. 300–309, 2012.
View at: Publisher Site | Google Scholar
D. G. Altman, “Confidence intervals for the number needed to treat,” BMJ, vol. 317, no. 7168, pp. 1309–1312, 1998.
View at: Publisher Site | Google Scholar
R. J. Cook and D. L. Sackett, “The number needed to treat: a clinically useful measure of treatment effect,” BMJ, vol. 310, no. 6977, pp. 452–454, 1995.
View at: Publisher Site | Google Scholar
D. Husereau, M. Drummond, S. Petrou et al., “Consolidated health economic evaluation reporting standards (CHEERS) statement,” BMC Medicine, vol. 11, no. 1, 2013.
View at: Publisher Site | Google Scholar
S. D. Ramsey, R. J. Willke, H. Glick et al., “Cost-effectiveness analysis alongside clinical trials II--an ISPOR good research practices task force report,” Value in Health, vol. 18, no. 2, pp. 161–172, 2015.
View at: Publisher Site | Google Scholar
W. J. Dixon and K. K. Yuen, “Trimming and winsorization: a review,” Statistische Hefte, vol. 15, no. 2-3, pp. 157–170, 1974.
View at: Publisher Site | Google Scholar
B. A. Hout, M. J. Al, G. S. Gordon, and F. F. H. Rutten, “Costs, effects and C/E-ratios alongside a clinical trial,” Health Economics, vol. 3, no. 5, pp. 309–319, 1994.
View at: Publisher Site | Google Scholar
A. H. Briggs and A. M. Gray, “Handling uncertainty in economic evaluation,” in The Advanced Handbook of Methods in Evidence Based Healthcare, pp. 314–337.
View at: Publisher Site | Google Scholar
A. R. Willan, A. H. Briggs, and J. S. Hoch, “Regression methods for covariate adjustment and subgroup analysis for non-censored cost-effectiveness data,” Health Economics, vol. 13, no. 5, pp. 461–475, 2004.
View at: Publisher Site | Google Scholar
Stata Corp, Stata Statistical Software: Release 16, 2019, https://www.scirp.org/%28S%28lz5mqp453edsnp55rrgjct55%29%29/reference/referencespapers.aspx?referenceid=2757660.
J. R. Landis and G. G. Koch, “The measurement of observer agreement for categorical data,” Biometrics, vol. 33, no. 1, pp. 159–174, 1977.
View at: Publisher Site | Google Scholar
K. Moses, M. Gayed, S. Chuah, and B. M. Wootton, “The use of evidence-based assessment for anxiety disorders in an Australian sample,” Journal of Anxiety Disorders, vol. 75, article 102279, 2020.
View at: Publisher Site | Google Scholar
W. Van Ballegooijen, P. Cuijpers, A. Van Straten et al., “Adherence to internet-based and face-to-face cognitive behavioural therapy for depression: a meta-analysis,” PLoS One, vol. 9, no. 7, article e100674, 2014.
View at: Publisher Site | Google Scholar
M. B. Abraham, J. A. Nicholas, M. Crone, T. T. Ly, E. A. Davis, and T. W. Jones, “The importance of the Hawthorne effect on psychological outcomes unveiled in a randomized controlled trial of diabetes technology,” Journal of Diabetes Science and Technology, vol. 12, no. 3, pp. 735-736, 2018.
View at: Publisher Site | Google Scholar
J. A. Sacristán, J. M. Abellán-Perpiñán, T. Dilla, J. Soto, and J. Oliva, “Some reflections on the use of inappropriate comparators in CEA,” Cost Effectiveness and Resource Allocation, vol. 18, pp. 1–6, 2020.
View at: Google Scholar
T. A. Furukawa, H. Noma, D. M. Caldwell et al., “Waiting list may be a nocebo condition in psychotherapy trials: a contribution from network meta-analysis,” Acta Psychiatrica Scandinavica, vol. 130, no. 3, pp. 181–192, 2014.
View at: Publisher Site | Google Scholar
M. Gomes, E. Murray, and J. Raftery, “Economic evaluation of digital health interventions: methodological issues and recommendations for practice,” PharmacoEconomics, vol. 40, no. 4, pp. 367–378, 2022.
View at: Publisher Site | Google Scholar
B. Bandelow, I. Aden, G. W. Alpers et al., S3-Leitlinie Behandlung von Angststörungen: Version 2, AWMF, 2021.
J. Xiong, O. Lipsitz, F. Nasri et al., “Impact of COVID-19 pandemic on mental health in the general population: a systematic review,” Journal of Affective Disorders, vol. 277, pp. 55–64, 2020.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2023 Fanny Kählke et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

433

Downloads

267

Citations