SciCombinator

Discover the most talked about and latest scientific content & concepts.

Concept: Jacob Cohen

2

More than 100 comparative outcome trials, directly comparing 2 or more psychotherapies for adult depression, have been published. We first examined whether these comparative trials had sufficient statistical power to detect clinically relevant differences between therapies of d=0.24. In order to detect such an effect size, power calculations showed that a trial would need to include 548 patients. We selected 3 recent meta-analyses of psychotherapies for adult depression (cognitive behaviour therapy (CBT), interpersonal psychotherapy and non-directive counselling) and examined the number of patients included in the trials directly comparing other psychotherapies. The largest trial comparing CBT with another therapy included 178 patients, and had enough power to detect a differential effect size of only d=0.42. None of the trials in the 3 meta-analyses had enough power to detect effect sizes smaller than d=0.34, but some came close to the threshold for detecting a clinically relevant effect size of d=0.24. Meta-analyses may be able to solve the problem of the low power of individual trials. However, many of these studies have considerable risk of bias, and if we only focused on trials with low risk of bias, there would no longer be enough studies to detect clinically relevant effects. We conclude that individual trials are heavily underpowered and do not even come close to having sufficient power for detecting clinically relevant effect sizes. Despite this large number of trials, it is still not clear whether there are clinically relevant differences between these therapies.

Concepts: Psychology, Statistical significance, Effect size, Meta-analysis, Cognitive behavioral therapy, Psychotherapy, Statistical power, Jacob Cohen

2

This article presents results from a systematic review and two meta-analyses that examine whether prison yoga and meditation programs are significantly related to increased psychological well-being and improvements in the behavioural functioning of prisoners. Comprehensive searches of the empirical literature were conducted up to December 2014. Participants who completed yoga or meditation program in prison experienced a small increase in their psychological well-being (Cohen’s d = 0.46, 95% confidence interval [CI] = [0.39, 0.54]) and a small improvement in their behavioural functioning (Cohen’s d = 0.30, 95% CI = [0.20, 0.40]). Moderator analyses suggested that there was a significant difference in effect sizes for programs of longer duration and less intensity, compared with those that were shorter and more intensive, for psychological well-being. Programs of longer duration had a slightly larger positive effect on behavioural functioning (d = 0.424), compared with more intensive programs (d = 0.418). Overall, the evidence suggests that yoga and meditation have favourable effects on prisoners.

Concepts: Systematic review, Statistical significance, Effect size, Meta-analysis, Meditation, Statistical power, Jacob Cohen, Gene V. Glass

2

Genotyping chips for rare and low-frequent variants have recently gained popularity with the introduction of exome chips, but the utility of these chips remains unclear. These chips were designed using exome sequencing data from mainly American-European individuals, enriched for a narrow set of common diseases. In addition, it is well-known that the statistical power of detecting associations with rare and low-frequent variants is much lower compared to studies exclusively involving common variants. We developed a simulation program adaptable to any exome chip design to empirically evaluate the power of the exome chips. We implemented the main properties of the Illumina HumanExome BeadChip array. The simulated data sets were used to assess the power of exome chip based studies for varying effect sizes and causal variant scenarios. We applied two widely-used statistical approaches for rare and low-frequency variants, which collapse the variants into genetic regions or genes. Under optimal conditions, we found that a sample size between 20,000 to 30,000 individuals were needed in order to detect modest effect sizes (0.5% < PAR > 1%) with 80% power. For small effect sizes (PAR <0.5%), 60,000-100,000 individuals were needed in the presence of non-causal variants. In conclusion, we found that at least tens of thousands of individuals are necessary to detect modest effects under optimal conditions. In addition, when using rare variant chips on cohorts or diseases they were not originally designed for, the identification of associated variants or genes will be even more challenging.

Concepts: Sample size, Statistical significance, Operations research, Statistical hypothesis testing, Effect size, Statistical power, Debut albums, Jacob Cohen

2

Maize with the insecticidal properties of the entomopathogenic bacterium Bacillus thuringiensis Berliner, known as Bt maize, has been sown in Europe since 1998. For several years, EU and Spanish regulations have required laboratory and field trials to assess risks of genetically modified crops for nontarget organisms prior to their authorization. Thirteen field trials were conducted in Spain to measure the effects of Bt maize on a broad range of arthropod taxa; no effects were found in accordance with most literature records. However, statistical analyses of single trials rarely have the statistical power to detect low effect sizes if they do not have a sufficient sample size. When sample size is low, meta-analysis may improve statistical power by combining several trials and assuming a common measure of effect size. Here we perform a meta-analysis of the results of 13 independent field trials conducted in Spain in which effects of single or stacked Bt traits on several arthropod taxa were measured with no significant results. Since the taxa included in each single trial were not the same for all trials, for the meta-analysis we selected only those taxa recorded in a minimum of six trials, resulting finally in 7, 7, and 12 taxa analyzed in visual counts, pitfall traps and yellow sticky traps, respectively. In comparison with single trial analysis, meta-analysis dramatically increased the detectability of treatment effects for most of the taxa regardless of the sampling technique; of the 26 taxa analyzed, only three showed poorer detectability in the meta-analysis than the best recorded in the 13 single trials. This finding reinforces the conclusion that Bt maize has no effect on the most common herbivore, predatory and parasitoid arthropods found in the maize ecosystems of southern Europe.

Concepts: Sample size, Statistical significance, Effect size, Meta-analysis, Bacillus thuringiensis, Statistical power, Jacob Cohen, Transgenic maize

1

We carried out an evaluation of a large-scale New Zealand retrofit programme using administrative data that provided the statistical power to assess the effect of insulation and/or heating retrofits on cardiovascular and respiratory-related mortality in people aged 65 and over with prior respiratory or circulatory hospitalisations.

Concepts: Blood, Statistical significance, Actuarial science, Type I and type II errors, Evaluation methods, Statistical hypothesis testing, Effect size, Jacob Cohen

1

Connectivity studies using resting-state functional magnetic resonance imaging are increasingly pooling data acquired at multiple sites. While this may allow investigators to speed up recruitment or increase sample size, multisite studies also potentially introduce systematic biases in connectivity measures across sites. In this work, we measure the inter-site effect in connectivity and its impact on our ability to detect individual and group differences. Our study was based on real, as opposed to simulated, multisite fMRI datasets collected in N=345 young, healthy subjects across 8 scanning sites with 3T scanners and heterogeneous scanning protocols, drawn from the 1000 functional connectome project. We first empirically show that typical functional networks were reliably found at the group level in all sites, and that the amplitude of the inter-site effects was small to moderate, with a Cohen’s effect size below 0.5 on average across brain connections. We then implemented a series of Monte-Carlo simulations, based on real data, to evaluate the impact of the multisite effects on detection power in statistical tests comparing two groups (with and without the effect) using a general linear model, as well as on the prediction of group labels with a support-vector machine. As a reference, we also implemented the same simulations with fMRI data collected at a single site using an identical sample size. Simulations revealed that using data from heterogeneous sites only slightly decreased our ability to detect changes compared to a monosite study with the GLM, and had a greater impact on prediction accuracy. However, the deleterious effect of multisite data pooling tended to decrease as the total sample size increased, to a point where differences between monosite and multisite simulations were small with N=120 subjects. Taken together, our results support the feasibility of multisite studies in rs-fMRI provided the sample size is large enough.

Concepts: Statistics, Sample size, Statistical significance, Magnetic resonance imaging, Statistical hypothesis testing, Effect size, Statistical power, Jacob Cohen

1

The calculation of heart rate variability (HRV) is a popular tool used to investigate differences in cardiac autonomic control between population samples. When interpreting effect sizes to quantify the magnitude of group differences, researchers typically use Cohen’s guidelines of small (0.2), medium (0.5), and large (0.8) effects. However, these guidelines were originally proposed as a fallback for when the effect size distribution (ESD) was unknown. Despite the availability of effect sizes from hundreds of HRV studies, researchers still largely rely on Cohen’s guidelines to interpret effect sizes and to perform power analyses to calculate required sample sizes for future research. This article describes an ESD analysis of 297 HRV effect sizes from between-group/case-control studies, revealing that the 25th, 50th, and 75th effect size percentiles correspond with effect sizes of 0.26, 0.51, and 0.88, respectively. The analyses suggest that Cohen’s guidelines may underestimate the magnitude of small and large effect sizes and that HRV studies are generally underpowered. Therefore, to better reflect the observed ESD, effect sizes of 0.25, 0.5, and 0.9 should be interpreted as small, medium, and large effects (after rounding to the closest 0.05). Based on power calculations using the ESD, suggested sample sizes are also provided for planning suitably powered studies that are more likely to replicate. Researchers are encouraged to use the ESD data set or their own collected data sets in tandem with the provided analysis script to perform custom ESD and power analyses relevant to their specific research area.

Concepts: Scientific method, Mathematics, Sample size, Statistical significance, Effect size, Statistical power, Calculation, Jacob Cohen

1

Equitable access to programs and health services is essential to achieving national and international health goals, but it is rarely assessed because of perceived measurement challenges. One of these challenges concerns the complexities of collecting the data needed to construct asset or wealth indices, which can involve asking as many as 40 survey questions, many with multiple responses. To determine whether the number of variables and questions could be reduced to a level low enough for more routine inclusion in evaluations and research without compromising programmatic conclusions, we used data from a program evaluation in Honduras that compared a pro-poor intervention with government clinic performance as well as data from a results-based financing project in Senegal. In both, the full Demographic and Health Survey (DHS) asset questionnaires had been used as part of the evaluations. Using the full DHS results as the “gold standard,” we examined the effect of retaining successively smaller numbers of variables on the classification of the program clients in wealth quintiles. Principal components analysis was used to identify those variables in each country that demonstrated minimal absolute factor loading values for 8 different thresholds, ranging from 0.05 to 0.70. Cohen’s kappa statistic was used to assess correlation. We found that the 111 asset variables and 41 questions in the Honduras DHS could be reduced to 9 variables, captured by only 8 survey questions (kappa statistic, 0.634), without substantially altering the wealth quintile distributions for either the pro-poor program or the government clinics or changing the resulting policy conclusions. In Senegal, the 103 asset variables and 36 questions could be reduced to 32 variables and 20 questions (kappa statistic, 0.882) while maintaining a consistent mix of users in each of the 2 lowest quintiles. Less than 60% of the asset variables in the 2 countries' full DHS asset indices overlapped, and in none of the 8 simplified asset index iterations did this proportion exceed 50%. We conclude that substantially reducing the number of variables and questions used to assess equity is feasible, producing valid results and providing a less burdensome way for program implementers or researchers to evaluate whether their interventions are pro-poor. Developing a standardized, simplified asset questionnaire that could be used across countries may prove difficult, however, given that the variables that contribute the most to the asset index are largely country-specific.

Concepts: Evaluation, Non-parametric statistics, Cohen's kappa, Inter-rater reliability, Jacob Cohen, Fleiss' kappa, Scott's Pi, Joseph L. Fleiss

1

This study determined whether a post-activation potentiation (PAP) effect could be elicited across multiple sets of a contrast PAP protocol. Fourteen rugby league players performed a contrast PAP protocol comprising four sets of two paused box squats accommodated with bands alternated with two standing broad jumps. The rest period between the squats and the jumps and between the sets was 90 s. A control protocol with standing broad jumps only was performed on a separate session. A standing broad jump was performed ∼2 min before each protocol and served as a baseline measurement. Standing broad jump distance was significantly greater (4.0 ± 3.4% to 5.7 ± 4.7%) than baseline during the four sets of the contrast PAP protocol with the changes being medium in the 1, 2 and 4 sets (effect size [ES]: 0.58, 0.67 and 0.69, respectively) and large for the 3 set (ES: 0.81). Conversely, no PAP effect was observed in the control protocol. Additionally, the stronger players displayed a larger PAP effect during each of the four sets of the contrast PAP protocol (Cohen’s d: 0.28-1.68) and a larger mean effect across these four sets (Cohen’s d: 1.29). Horizontal jump performance is potentiated after only 90 s of rest following an accommodating exercise and this PAP effect can be elicited across four sets. Additionally, the PAP response is largely mediated by the individual’s strength level. These results are of great importance for coaches seeking to incorporate PAP complexes involving horizontal jumps in their training programs.

Concepts: Effect size, Jacob Cohen

1

To review and synthesize the existing literature on the effects of yoga on cognitive function by determining effect sizes that could serve as a platform to design, calculate statistical power, and implement future studies.

Concepts: Statistical significance, Cognition, Educational psychology, Effect size, Meta-analysis, Statistical power, Jacob Cohen