small mindboggle logo Mindboggle
Software Data Papers People
back to Papers


Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration.
Arno Klein, Jesper Andersson, Babak A. Ardekani, John Ashburner, Brian Avants, Ming-Chang Chiang,
Gary E. Christensen, D. Louis Collins, James Gee, Pierre Hellier, Joo Hyun Song, Mark Jenkinson, Claude Lepage,
Daniel Rueckert, Paul Thompson, Tom Vercauteren, Roger P. Woods, J. John Mann, Ramin V. Parsey.
2009. NeuroImage. 46(3): 786-802.    PMCID: PMC2747506

[PubMedCentral]    [PDF]    [doi:10.1016/j.neuroimage.2008.12.037]

Supplementary Material (12.4 MB PDF)

Corrigenda:

Equation 2 was not used to compute the box plots in Figure 5. The caption for Figure 5 should replace the sentence:
"These box and whisker plots show the target overlap measures between deformed source and target label volumes averaged first across all of the regions in each label set (LPBA40, IBSR18, CUMC12, and MGH10) then across brain pairs."
with:
"These box and whisker plots show mean target overlap measures averaged over all subjects in each label set, where mean target overlap is an average (taken over all labeled regions) of the fraction of a target region overlapping the co-registered source region (Eq.2 was not used to compute these values):
sum_r(|T_r intersection S_r| / |T_r|) / N_regions "

Table 2 header should read "Volume (voxels)" instead of "Volume (mm)."

Note: In this article, registration methods are ranked with respect to the top-ranking method, not with each other. Be careful not to conclude that one method performed more accurately than another if their rank test means are within a standard deviation of one another.



Code (Matlab and Python):
Compare (and plot) evaluations of the registration methods: evaluate_compare.m, which calls plot3k2.m
Warp evaluations: evaluate_warps_main.m, evaluate_warps.m
Distance error evaluations: evaluate_distances_main.m, evaluate_distances.m
Label boundaries (for distance errors): make_boundaries_main.m, make_boundaries.m
"Paired indifference-zone" ranking: rank_indifferencezone.m     2010/07/06: standalone python algorithm: rank_paired_indifferencezone.py
Permutation tests for ranking: permutation_test.m
ANTS shell script (created after the study): ants.sh (see Klein, et al, 2010 for subsequent study)
IRTK parameter file: IRTK_parameters_2.5mm.txt
FNIRT parameter file: FNIRT_schedule.cnf
ANIMAL parameter file: ANIMAL_parameters.pl
Matlab code for composing SPM Normalize and DARTEL transforms: SPM_code.tar.gz
Call preprocessing scripts and registration software packages (Python): run_tests.txt

Data:
CUMC12 data, IBSR18 data, and MGH10 data and information
Rigidly registered pairs of LPBA40 images (3.1 GB) and label volumes (263 MB) and label list
Freesurfer-derived LPBA40 surfaces (6.3 MB)
All pairwise measures (.mat Matlab files) (over 3 GB)

6/2011: CUMC12 ANTs template (all 12 brains) (5.3 MB)



Results:
Tables of UNION overlap values (intersection of automated and manual labels over their union)
for each region by registration method, averaged across all brain pairs per label set:
Column-sortable HTML: LPBA40, IBSR18, CUMC12, MGH10
Tables of TARGET overlap values (intersection of automated and manual labels over manual labels)
for each region by registration method, averaged across all brain pairs per label set:
Comma-delimited (.csv): LPBA40, IBSR18, CUMC12, MGH10
Column-sortable HTML: LPBA40, IBSR18, CUMC12, MGH10
Example: LPBA40 set, lighter boxes denote higher accuracy (click on column headers to sort):

RegionFLIRTAIRANIMALARTD.DemonsFNIRTIRTKJRD-fluidROMEOSICLESyNSPM_N*SPM_NSPM_USSPM_D
L sup. frontal gyrus747873828081808381718179767978
R sup. frontal gyrus747875818080798279708178767876
L middle frontal gyrus717575797779797878717977767776
R middle frontal gyrus717475787774787877727877767774
L inf. frontal gyrus647173737272737472657472737270
R inf. frontal gyrus656969737270737472677372727269
L precentral gyrus606371736871736967557468517072
R precentral gyrus606269726970727068587366537071
L middle orbitofrontal gyrus566665696768686967636966666664
R middle orbitofrontal gyrus586667706764686967646966666664
L lateral orbitofrontal gyrus475761615859606359526159595757
R lateral orbitofrontal gyrus465557595555576058515956565554
L gyrus rectus566558706770646866566964616562
R gyrus rectus596761716873657068587166646664
L postcentral gyrus515663676166666260506861456566
R postcentral gyrus545661676366666562546861476568
L sup. parietal gyrus656670727171717371627369607069
R sup. parietal gyrus656671727170717371647370627169
L supramarginal gyrus546065666466656563486864606665
R supramarginal gyrus545959656365656463526764616563
L angular gyrus565860646363626562496462586362
R angular gyrus566159666564656665526664626563
L precuneus626360676667666765586763586663
R precuneus636564696769676966596965606764
L sup. occipital gyrus475456625859596058446157495958
R sup. occipital gyrus465255595557575856455955455656
L middle occipital gyrus586370696970667068576968576865
R middle occipital gyrus596470696870667069606867566765
L inf. occipital gyrus526072696768637067616867416563
R inf. occipital gyrus536372696670637168636866406664
L cuneus526256676369636565556764566464
R cuneus536159656267636462546663546362
L sup. temporal gyrus607072767274777571667771707171
R sup. temporal gyrus617272777374777673687872717272
L middle temporal gyrus556264686668666764626865606664
R middle temporal gyrus576465706870697068667167646866
L inf. temporal gyrus536067686670666865606764526462
R inf. temporal gyrus566167686671677067636865566564
L parahippocampal gyrus546665736869726967567264356865
R parahippocampal gyrus556564716768726865557164356764
L lingual gyrus606667727070707069667367416869
R lingual gyrus636971747172737270677569417070
L fusiform gyrus556668736971696867627166456866
R fusiform gyrus586668736872697067627167456865
L insular cortex647370817673817774637970697675
R insular cortex617166807473807473637869697472
L cingulate gyrus626760736869716968587262646766
R cingulate gyrus616660726870706866567162646766
L caudate606660787368787173587569686969
R caudate606858787370777172587568707072
L putamen647364817677787173677867677772
R putamen647364807675777173697869687771
L hippocampus597071777375757171617569407470
R hippocampus627072777375757271617568487370
cerebellum816685888586828386828884288680
brainstem786979818182787981778077377976



Registration evaluation studies conducted since this article was published:

Klein, et al, 2010

EMPIRE: Evaluation of Methods for Pulmonary Image Registration 2010 (MICCAI 2010 challenge workshop)