Comparison of results when basing variable selection on the FARMS algorithm or common strategies. The number of covariates included in the final model excludes the “forced in” covariates. FARMS parameters used in this case are number of adding covariates = 8, number of starting covariates = 10, and the selecting criteria for both best subset and best model was the BIC. (All runs were executed on an Intel Xeon x5680 machine with 6 CPU cores and 95 GB RAM memory under a Linux Suse 11.0 OS).
Time (seconds)* Mean (IQR)
Number of vars.**
BIC
AIC
Adj.
HLA
FARMS
1.27 (1.00; 1.40)
9
2235.4
2183.03
11.67%
10.74%
All subsets
>1 month
—
—
—
—
—
Forward selection1
3.84 (3.13; 3.52)
17
2259.4
2168.86
14.69%
12.98%
Forward stepwise1
4.32 (3.51; 3.90)
17
2259.4
2168.86
14.69%
12.98%
Forward selection2
2.01 (1.62; 1.88)
10
2235.45
2178.27
12.36%
11.33%
Forward stepwise2
2.35 (1.89; 2.18)
9
2235.44
2183.03
11.67%
10.74%
Forward selection3
1.99 (1.61; 1.88)
10
2235.45
2178.27
12.36%
11.33%
Forward stepwise3
2.38 (1.92; 2.22)
9
2235.44
2183.03
11.67%
10.74%
Backward stepwise3
13 s
10
2236.52
2174.58
12.93%
11.81%
OLP
FARMS
33.4(19.47; 37.95)
12
2224.8
2158.11
14.77%
13.57%
All subsets
>1 month
—
—
—
—
—
Forward selection1
393.2 (324.40; 336.60)
79
2396.53
2010.56
38.40%
32.22%
Forward stepwise1
545.9 (451.70; 469.70)
83
2415.46
2010.53
38.97%
32.51%
Forward selection2
401.7 (329.50; 343.40)
80
2403.3
2012.56
38.40%
32.13%
Forward stepwise2
462.4 (382.50; 401.50)
76
2385.18
2013.51
37.76%
31.77%
Forward selection3
38.09 (31.34; 33.11)
12
2224.82
2158.11
14.77%
13.57%
Forward stepwise3
38.31 (31.34; 33.43)
12
2224.82
2158.11
14.77%
13.57%
Backward stepwise3
>12 hours
23
2232.63
2108.63
21.68%
19.45%
1Selection by AIC and base model with intercept-only. 2Selection by AIC and base model with forced-in variables. 3Selection by BIC and base model with forced-in variables. *Time obtained after 100 executions for each scenario. **Including forced-in variables.
