At the 2015 REWARD/EQUATOR conference on research waste, the late Doug Altman revealed that his only regret about his 1994 BMJ paper ‘The scandal of poor medical research’ was that he used the word ‘poor’ rather than ‘bad’. But how much research is bad? And what would improve things? We used Cochrane systematic reviews to get an idea of how many trials are bad: we used the Cochrane assessment of trial risk of bias as high as being a proxy for bad. We were also interested in how many participants were in bad trials and how much these trials cost. We identified 96 reviews authored by 546 reviewers from 49 clinical Cochrane Review Groups that included 1659 trials done in 84 countries. In summary: most trials are bad. Only 133 (8%) of the 1640 presenting risk of bias information were low risk of bias. Most participants were in a bad trial and the money spent on them could, we estimate, fund the UK’s biggest public trial funding programme for at least 10 years. We have five recommendations: trials should be neither funded (1) nor given ethical approval (2) unless they have a statistician and methodologist; trialists should use a risk of bias tool at design (3); more statisticians and methodologists should be trained and supported (4); there should be more funding into applied methodology research and infrastructure (5). In short, most randomised trials are bad and most trial participants will be in one. The research community has tolerated this for decades. This has to stop: we need to put rigour and methodology where it belongs — at the centre of our science.
Ms Stefania Pirosca is a Neuroscience student at the University of Aberdeen and worked with Prof Treweek on two projects: one concerned with the improvements needed for trial design, and one concerned with the ‘scandal of bad research’. Stefania has thoroughly enjoyed her time as a health research intern and as her time at university is coming to an end, she wishes to pursue a PhD in Neuroscience. When she is not studying or working, she enjoys reading fiction, weightlifting and science podcasts.
Prof Shaun Treweek, University of Aberdeen, is a health services researcher interested in efficient trial design, particularly around recruitment and retention and the effective presentation of research evidence. He also helped to develop PRECIS-2, a tool to match trial design decisions to the information needs of those intended to use the results. He leads an initiative called Trial Forge (http://www.trialforge.org) that aims to be more systematic about how we identify, generate and use research evidence in making trial design, conduct, analysis and reporting decisions. In 2019 Trial Forge won the international Cochrane-REWARD Prize for outstanding work in reducing waste in research. Finally, Shaun is an Editor-in-Chief of the journal Trials.
