Abstract
The multiple hypothesis testing problem occurs when a number of individual hypothesis tests are considered simultaneously. The larger the number of inferences (tests) made, the more likely erroneous inferences become. Several classical statistical techniques have been developed to address this problem. Unfortunately, very often, in studies that aim to examine the significance of multiple effects (for example, correlations), the effect of multiple testing is ignored. This leads to an inflated number of significant findings (e.g., correlation coefficients). On the other hand, in the case of a large number of tests about relatively small effects, and thus a large family of inferences, the power of individual tests decreases rapidly when classical procedures for controlling multiplicity are applied, often resulting in too few significant findings. In this paper, we propose an alternative and powerful approach to face the problem of testing multiple hypotheses and develop a new MCPerm method for testing the significance of multiple correlations. The proposed solution is definitely more effective than Holm’s, which proved to be more conservative in the presence of several dependencies and tended to indicate fewer of those.