Concept: Effect size
This randomized controlled trial was performed to investigate whether placebo effects in chronic low back pain could be harnessed ethically by adding open-label placebo (OLP) treatment to treatment as usual (TAU) for 3 weeks. Pain severity was assessed on three 0- to 10-point Numeric Rating Scales, scoring maximum pain, minimum pain, and usual pain, and a composite, primary outcome, total pain score. Our other primary outcome was back-related dysfunction, assessed on the Roland-Morris Disability Questionnaire. In an exploratory follow-up, participants on TAU received placebo pills for 3 additional weeks. We randomized 97 adults reporting persistent low back pain for more than 3 months' duration and diagnosed by a board-certified pain specialist. Eighty-three adults completed the trial. Compared to TAU, OLP elicited greater pain reduction on each of the three 0- to 10-point Numeric Rating Scales and on the 0- to 10-point composite pain scale (P < 0.001), with moderate to large effect sizes. Pain reduction on the composite Numeric Rating Scales was 1.5 (95% confidence interval: 1.0-2.0) in the OLP group and 0.2 (-0.3 to 0.8) in the TAU group. Open-label placebo treatment also reduced disability compared to TAU (P < 0.001), with a large effect size. Improvement in disability scores was 2.9 (1.7-4.0) in the OLP group and 0.0 (-1.1 to 1.2) in the TAU group. After being switched to OLP, the TAU group showed significant reductions in both pain (1.5, 0.8-2.3) and disability (3.4, 2.2-4.5). Our findings suggest that OLP pills presented in a positive context may be helpful in chronic low back pain.This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal.
A focus on novel, confirmatory, and statistically significant results leads to substantial bias in the scientific literature. One type of bias, known as “p-hacking,” occurs when researchers collect or select data or statistical analyses until nonsignificant results become significant. Here, we use text-mining to demonstrate that p-hacking is widespread throughout science. We then illustrate how one can test for p-hacking when performing a meta-analysis and show that, while p-hacking is probably common, its effect seems to be weak relative to the real effect sizes being measured. This result suggests that p-hacking probably does not drastically alter scientific consensuses drawn from meta-analyses.
Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. We conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. Replication effects were half the magnitude of original effects, representing a substantial decline. Ninety-seven percent of original studies had statistically significant results. Thirty-six percent of replications had statistically significant results; 47% of original effect sizes were in the 95% confidence interval of the replication effect size; 39% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68% with statistically significant effects. Correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.
What are the statistical practices of articles published in journals with a high impact factor? Are there differences compared with articles published in journals with a somewhat lower impact factor that have adopted editorial policies to reduce the impact of limitations of Null Hypothesis Significance Testing? To investigate these questions, the current study analyzed all articles related to psychological, neuropsychological and medical issues, published in 2011 in four journals with high impact factors: Science, Nature, The New England Journal of Medicine and The Lancet, and three journals with relatively lower impact factors: Neuropsychology, Journal of Experimental Psychology-Applied and the American Journal of Public Health. Results show that Null Hypothesis Significance Testing without any use of confidence intervals, effect size, prospective power and model estimation, is the prevalent statistical practice used in articles published in Nature, 89%, followed by articles published in Science, 42%. By contrast, in all other journals, both with high and lower impact factors, most articles report confidence intervals and/or effect size measures. We interpreted these differences as consequences of the editorial policies adopted by the journal editors, which are probably the most effective means to improve the statistical practices in journals with high or low impact factors.
Much has been written regarding p-values below certain thresholds (most notably 0.05) denoting statistical significance and the tendency of such p-values to be more readily publishable in peer-reviewed journals. Intuition suggests that there may be a tendency to manipulate statistical analyses to push a “near significant p-value” to a level that is considered significant. This article presents a method for detecting the presence of such manipulation (herein called “fiddling”) in a distribution of p-values from independent studies. Simulations are used to illustrate the properties of the method. The results suggest that the method has low type I error and that power approaches acceptable levels as the number of p-values being studied approaches 1000.
The hypothesis that the S allele of the 5-HTTLPR serotonin transporter promoter region is associated with increased risk of depression, but only in individuals exposed to stressful situations, has generated much interest, research and controversy since first proposed in 2003. Multiple meta-analyses combining results from heterogeneous analyses have not settled the issue. To determine the magnitude of the interaction and the conditions under which it might be observed, we performed new analyses on 31 data sets containing 38 802 European ancestry subjects genotyped for 5-HTTLPR and assessed for depression and childhood maltreatment or other stressful life events, and meta-analysed the results. Analyses targeted two stressors (narrow, broad) and two depression outcomes (current, lifetime). All groups that published on this topic prior to the initiation of our study and met the assessment and sample size criteria were invited to participate. Additional groups, identified by consortium members or self-identified in response to our protocol (published prior to the start of analysis) with qualifying unpublished data, were also invited to participate. A uniform data analysis script implementing the protocol was executed by each of the consortium members. Our findings do not support the interaction hypothesis. We found no subgroups or variable definitions for which an interaction between stress and 5-HTTLPR genotype was statistically significant. In contrast, our findings for the main effects of life stressors (strong risk factor) and 5-HTTLPR genotype (no impact on risk) are strikingly consistent across our contributing studies, the original study reporting the interaction and subsequent meta-analyses. Our conclusion is that if an interaction exists in which the S allele of 5-HTTLPR increases risk of depression only in stressed individuals, then it is not broadly generalisable, but must be of modest effect size and only observable in limited situations.Molecular Psychiatry advance online publication, 4 April 2017; doi:10.1038/mp.2017.44.
Here we show that constructal-law physics unifies the design of animate and inanimate movement by requiring that larger bodies move farther, and their movement on the landscape last longer. The life span of mammals must scale as the body mass (M) raised to the power ¼, and the distance traveled during the lifetime must increase with body size. The same size effect on life span and distance traveled holds for the other flows that move mass on earth: atmospheric and oceanic jets and plumes, river basins, animals and human operated vehicles. The physics is the same for all flow systems on the landscape: the scaling rules of “design” are expressions of the natural tendency of all flow systems to generate designs that facilitate flow access. This natural tendency is the constructal law of design and evolution in nature. Larger bodies are more efficient movers of mass on the landscape.
There is increasing evidence that gardening provides substantial human health benefits. However, no formal statistical assessment has been conducted to test this assertion. Here, we present the results of a meta-analysis of research examining the effects of gardening, including horticultural therapy, on health. We performed a literature search to collect studies that compared health outcomes in control (before participating in gardening or non-gardeners) and treatment groups (after participating in gardening or gardeners) in January 2016. The mean difference in health outcomes between the two groups was calculated for each study, and then the weighted effect size determined both across all and sets of subgroup studies. Twenty-two case studies (published after 2001) were included in the meta-analysis, which comprised 76 comparisons between control and treatment groups. Most studies came from the United States, followed by Europe, Asia, and the Middle East. Studies reported a wide range of health outcomes, such as reductions in depression, anxiety, and body mass index, as well as increases in life satisfaction, quality of life, and sense of community. Meta-analytic estimates showed a significant positive effect of gardening on the health outcomes both for all and sets of subgroup studies, whilst effect sizes differed among eight subgroups. Although Egger’s test indicated the presence of publication bias, significant positive effects of gardening remained after adjusting for this using trim and fill analysis. This study has provided robust evidence for the positive effects of gardening on health. A regular dose of gardening can improve public health.
We sought to determine whether high-dose folinic acid improves verbal communication in children with non-syndromic autism spectrum disorder (ASD) and language impairment in a double-blind placebo control setting. Forty-eight children (mean age 7 years 4 months; 82% male) with ASD and language impairment were randomized to receive 12 weeks of high-dose folinic acid (2 mg kg(-1) per day, maximum 50 mg per day; n=23) or placebo (n=25). Children were subtyped by glutathione and folate receptor-α autoantibody (FRAA) status. Improvement in verbal communication, as measured by a ability-appropriate standardized instrument, was significantly greater in participants receiving folinic acid as compared with those receiving placebo, resulting in an effect of 5.7 (1.0,10.4) standardized points with a medium-to-large effect size (Cohen’s d=0.70). FRAA status was predictive of response to treatment. For FRAA-positive participants, improvement in verbal communication was significantly greater in those receiving folinic acid as compared with those receiving placebo, resulting in an effect of 7.3 (1.4,13.2) standardized points with a large effect size (Cohen’s d=0.91), indicating that folinic acid treatment may be more efficacious in children with ASD who are FRAA positive. Improvements in subscales of the Vineland Adaptive Behavior Scale, the Aberrant Behavior Checklist, the Autism Symptom Questionnaire and the Behavioral Assessment System for Children were significantly greater in the folinic acid group as compared with the placebo group. There was no significant difference in adverse effects between treatment groups. Thus, in this small trial of children with non-syndromic ASD and language impairment, treatment with high-dose folinic acid for 12 weeks resulted in improvement in verbal communication as compared with placebo, particularly in those participants who were positive for FRAAs.Molecular Psychiatry advance online publication, 18 October 2016; doi:10.1038/mp.2016.168.
Objectives To examine the risk of relapse and time to relapse after discontinuation of antidepressants in patients with anxiety disorder who responded to antidepressants, and to explore whether relapse risk is related to type of anxiety disorder, type of antidepressant, mode of discontinuation, duration of treatment and follow-up, comorbidity, and allowance of psychotherapy.Design Systematic review and meta-analyses of relapse prevention trials.Data sources PubMed, Cochrane, Embase, and clinical trial registers (from inception to July 2016).Study selection Eligible studies included patients with anxiety disorder who responded to antidepressants, randomised patients double blind to either continuing antidepressants or switching to placebo, and compared relapse rates or time to relapse.Data extraction Two independent raters selected studies and extracted data. Random effect models were used to estimate odds ratios for relapse, hazard ratios for time to relapse, and relapse prevalence per group. The effect of various categorical and continuous variables was explored with subgroup analyses and meta-regression analyses respectively. Bias was assessed using the Cochrane tool.Results The meta-analysis included 28 studies (n=5233) examining relapse with a maximum follow-up of one year. Across studies, risk of bias was considered low. Discontinuation increased the odds of relapse compared with continuing antidepressants (summary odds ratio 3.11, 95% confidence interval 2.48 to 3.89). Subgroup analyses and meta-regression analyses showed no statistical significance. Time to relapse (n=3002) was shorter when antidepressants were discontinued (summary hazard ratio 3.63, 2.58 to 5.10; n=11 studies). Summary relapse prevalences were 36.4% (30.8% to 42.1%; n=28 studies) for the placebo group and 16.4% (12.6% to 20.1%; n=28 studies) for the antidepressant group, but prevalence varied considerably across studies, most likely owing to differences in the length of follow-up. Dropout was higher in the placebo group (summary odds ratio 1.31, 1.06 to 1.63; n=27 studies).Conclusions Up to one year of follow-up, discontinuation of antidepressant treatment results in higher relapse rates among responders compared with treatment continuation. The lack of evidence after a one year period should not be interpreted as explicit advice to discontinue antidepressants after one year. Given the chronicity of anxiety disorders, treatment should be directed by long term considerations, including relapse prevalence, side effects, and patients' preferences.