Table 1: Results of the 11 evaluated algorithms presented at the workshop and the evaluated freeware packages on the 15 test datasets. The algorithms are ranked () based on their overall score () by using (5). This score is based on the ranks of the gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) segmentation and the three evaluated measures: Dice coefficient ( in %), 95th-percentile Hausdorff distance ( in mm), and the absolute volume difference (AVD in %). The rank denotes the rank based on the mean () over all 15 test datasets for each measure (0: , 1: , and 2: AVD) and component (0: GM, 1: WM, 2: CSF, 3: brain (WM + GM), and 4: intracranial volume (ICV = WM + GM + CSF)). Teams BIGR2 and UofL BioImaging, and FreeSurfer and Jedi Mind Meld have equal scores based on the mean (); therefore the ranking based on the standard deviation () is taken into account to determine the final rank (BIGR2: rank 4, UofL BioImaging: rank 8, FreeSurfer: rank 13, and Jedi Mind Meld: rank 17). Columns 2 and 3 present the average runtime per scan in seconds (s), minutes (m), or hours (h) and the scans (T1: T1-weighted scan, 3D T1: 3D T1-weighted scan, IR: T1-weighted inversion recovery (IR), and F: T1-weighted FLAIR) that are used for processing.

TeamScansGMWMCSFBrainICV
AVDAVDAVDAVDAVD
() () () () () () () () () () () () () () ()

1BIGR235 mT1, IR, F12422445143812128412
84.71.96.188.42.46.078.33.22395.12.73.296.03.95.2
(1.3)(0.4)(3.3)(1.2)(0.5)(5.1)(5.0)(0.8)(17)(0.5)(0.8)(1.6)(1.3)(1.1)(3.0)

2UofL BioImaging6 sT15194113221382112525
83.01.78.687.92.28.778.92.79.794.92.43.996.73.41.8
(1.5)(0.3)(5.4)(2.0)(0.6)(6.6)(4.2)(0.5)(10)(0.6)(0.5)(2.0)(0.8)(0.6)(2.0)

3CMIV3 mT1, F6755310338505724311
82.42.76.887.72.47.378.63.01494.53.82.696.83.84.9
(1.4)(0.4)(4.0)(1.6)(0.4)(3.8)(3.1)(0.4)(5.9)(0.5)(1.1)(2.2)(0.8)(1.3)(2.3)

4UB VPML Med30 mT1, IR, F4421578131761441101115
83.32.15.988.62.77.174.84.33194.62.82.494.86.67.7
(1.3)(0.3)(5.3)(1.7)(0.4)(3.8)(7.1)(1.7)(19)(0.6)(0.4)(1.8)(2.0)(2.0)(4.1)

5Bigr_neuro2 hT1, F713366964106471010758
81.53.75.987.33.07.378.23.21694.04.63.696.33.93.5
(1.7)(0.9)(4.2)(1.4)(0.4)(3.8)(4.7)(0.6)(14)(0.8)(1.4)(2.4)(1.2)(0.9)(2.7)

6Robarts16 m3D T1, IR1131588611136616318111
79.72.09.886.23.17.180.32.72093.12.87.997.92.60.9
(2.4)(0.1)(7.3)(1.3)(0.4)(6.2)(4.1)(0.5)(13)(1.6)(0.5)(3.6)(0.3)(0.4)(0.7)

7Narsil2 mT1, F3517112171877135316189
83.52.35.587.13.35.866.613.31494.82.92.992.5243.7
(1.8)(0.4)(4.4)(1.3)(0.9)(5.3)(2.4)(5.4)(9.5)(0.5)(0.5)(2.0)(0.5)(8.9)(1.7)

8SPM_T1_F3 mT1, F8916971914275914146144
81.22.91086.03.05.274.14.61093.95.85.396.68.21.5
(2.2)(0.3)(8.5)(1.5)(0.1)(3.8)(3.4)(0.6)(4.7)(1.0)(2.2)(3.8)(0.2)(3.0)(1.0)

9SPM_T1_IR3 mT1, IR12117161255103818117282
79.43.07.283.53.66.378.34.01093.94.63.497.76.51.0
(2.1)(0.4)(6.3)(2.1)(0.3)(4.6)(3.8)(0.6)(5.7)(0.8)(1.2)(2.8)(0.2)(1.3)(0.8)

10MNAB15 mT1, IR, F28123411151516866911151217
83.92.89.188.02.77.868.14.92994.54.53.892.57.19.7
(2.1)(0.9)(6.5)(1.2)(0.8)(4.0)(4.0)(2.2)(21)(1.0)(2.0)(3.2)(1.1)(4.2)(4.7)

11SPM_T13 mT191061110311161591108691313
80.33.06.985.63.16.070.75.32393.94.43.295.38.15.5
(2.4)(0.5)(6.8)(1.7)(0.1)(4.1)(3.8)(1.5)(15.7)(0.9)(1.6)(2.9)(0.9)(3.7)(3.7)

12FSL_Seg10 mT1131610101314126599131341266
78.74.38.686.03.711.569.93.41293.35.53.094.25.33.4
(2.2)(1.2)(6.3)(2.6)(0.8)(6.3)(2.8)(0.2)(10.3)(0.8)(1.4)(1.5)(0.8)(1.1)(1.5)

13SPM_T1_IR_F4 mT1, IR, F10121815141271161051215133153
80.13.013.983.63.88.476.94.11293.65.95.197.78.21.2
(2.4)(0.2)(9.6)(2.1)(0.5)(5.2)(3.1)(0.5)(6.0)(1.1)(1.8)(3.6)(0.2)(1.8)(0.9)

14FSL _PVSeg10 mT1151581315171374107111691377
77.74.38.484.83.819.769.53.41193.66.13.594.25.33.4
(2.6)(1.3)(6.5)(3.2)(0.9)(10)(2.2)(0.3)(5.8)(0.9)(1.6)(3.0)(0.8)(1.1)(1.5)

15FreeSurfer1 h3D T116617129818121811617616171018
77.42.312.185.23.17.265.84.35092.33.66.392.46.610
(2.0)(0.6)(6.0)(2.2)(0.5)(4.6)(3.7)(0.6)(19.6)(0.8)(0.6)(3.4)(0.9)(0.4)(4.5)

16Jedi Mind Meld27 sT1, IR, F1414111716151081111615128111614
77.83.98.980.64.513.773.33.71893.25.43.594.3206.8
(5.9)(2.8)(7.6)(9.6)(3.6)(12)(3.5)(0.8)(9.0)(1.7)(4.8)(3.0)(1.7)(3.7)(3.1)

17S2_QM1.5 hT1, IR, F171713141718169913014171514910
76.45.59.383.94.924.867.93.81493.37.05.593.76.54.0
(3.4)(3.0)(6.4)(3.4)(3.2)(10)(2.3)(0.6)(9.9)(1.4)(3.5)(4.5)(1.1)(1.4)(2.2)

18LNMBrains5 mT1, IR, F181814181816141712145181817181716
72.86.89.578.36.815.368.87.72088.58.67.489.220.47.9
(5.3)(2.3)(7.6)(6.4)(3.5)(13)(6.6)(2.4)(13)(4.5)(2.8)(7.7)(4.8)(3.7)(8.3)

Semiautomatic due to per scan parameter tuning.