Reproducibility in the absence of selective reporting : An illustration from large-scale brain asymmetry research
Journal article
Kong, Xiang-Zhen, ENIGMA Laterality Working Group, Francks, Clyde and Lorenzetti, Valentina*. (2022). Reproducibility in the absence of selective reporting : An illustration from large-scale brain asymmetry research. Human Brain Mapping. 43(1), pp. 244-254. https://doi.org/10.1002/hbm.25154
Authors | Kong, Xiang-Zhen, ENIGMA Laterality Working Group, Francks, Clyde and Lorenzetti, Valentina* |
---|---|
Abstract | The problem of poor reproducibility of scientific findings has received much attention over recent years, in a variety of fields including psychology and neuroscience. The problem has been partly attributed to publication bias and unwanted practices such as p-hacking. Low statistical power in individual studies is also understood to be an important factor. In a recent multisite collaborative study, we mapped brain anatomical left–right asymmetries for regional measures of surface area and cortical thickness, in 99 MRI datasets from around the world, for a total of over 17,000 participants. In the present study, we revisited these hemispheric effects from the perspective of reproducibility. Within each dataset, we considered that an effect had been reproduced when it matched the meta-analytic effect from the 98 other datasets, in terms of effect direction and significance threshold. In this sense, the results within each dataset were viewed as coming from separate studies in an “ideal publishing environment,” that is, free from selective reporting and p hacking. We found an average reproducibility rate of 63.2% (SD = 22.9%, min = 22.2%, max = 97.0%). As expected, reproducibility was higher for larger effects and in larger datasets. Reproducibility was not obviously related to the age of participants, scanner field strength, FreeSurfer software version, cortical regional measurement reliability, or regional size. These findings constitute an empirical illustration of reproducibility in the absence of publication bias or p hacking, when assessing realistic biological effects in heterogeneous neuroscience data, and given typically-used sample sizes. |
Keywords | multisite collaboration; P-hacking; publication bias; reproducibility; team science |
Year | 2022 |
Journal | Human Brain Mapping |
Journal citation | 43 (1), pp. 244-254 |
Publisher | John Wiley & Sons |
ISSN | 1065-9471 |
Digital Object Identifier (DOI) | https://doi.org/10.1002/hbm.25154 |
PubMed ID | 32841457 |
Scopus EID | 2-s2.0-85089785260 |
PubMed Central ID | PMC8675427 |
Research or scholarly | Research |
Page range | 244-254 |
Publisher's version | License File Access Level Open |
Output status | Published |
Publication dates | |
Online | 25 Aug 2020 |
Publication process dates | |
Accepted | 19 Jul 2020 |
Deposited | 10 Jan 2022 |
Additional information | *Valentina Lorenzetti is a member of the ENIGMA Laterality Working Group. |
https://acuresearchbank.acu.edu.au/item/8x43y/reproducibility-in-the-absence-of-selective-reporting-an-illustration-from-large-scale-brain-asymmetry-research
Download files
Publisher's version
OA_Kong_2022_Reproducibility_in_the_absence_of_selective.pdf | |
License: CC BY 4.0 | |
File access level: Open |
108
total views33
total downloads0
views this month0
downloads this month