Series 1: Bad science
June 1, 2017
For a light start in the field of evidence based science Goldacre’s Bad science is a good and enjoyable read.
Goldacre is a British physician, academic and Guardian columnist, very enjoyable to read, though provocative.
An interesting follow up is Goldacre’s Bad Pharma, more closely oriented to the world of drug development. Unsurprisingly it has attracted some criticism by reknown statisticians working in this field, such as Stephen Senn’s:
Bad JAMA - slides
Bad JAMA - article
Bad JAMA/Pharma on the blog
Especially highlighting the bias which may be associated with (possibly unkown) filtering of the data before analysis, in the sense that the sample used for analysis is not a simple random sample, especially in relation to assess editorial bias against negative studies. Finding that the probability of acceptance of submitted studies is the same regardless of whether the study results are positive or negative is the same, does not imply no bias, since it does not account for the fact that negative studies submitted might be of higher quality on average, since authors are aware of the editorial pre-disposition against negative studies unless they are of outstanding quality.
Bad karma -article
Casting Stones, by Stephen Senn
A piece about the strategies to raise awareness about the problem of missing data from clinical trials - prompted by an initiative by BMJ to collect evidence for House of Commons Science and Technology Select Committee. It is argued once more that industry run trials are of higher quality (a point also admitted by Goldacre).
Clinical trial data: it’s publishing, Jim, but not as we know it
A piece again about the importance of publishing clinical trial data, the information generated by pharma companies and for which they are compensated by the sales. It argues however that it should be self-publication rather than in medical journals, which may for example delay the publication, reject negative studies and harm the quality of the study by requesting modifications to statistical analysis, in addition to the fact that regulators do a better job than journals. Closing remark from the article:
We are moving from an era of private data and public analyses to one of public data and private analyses. Just as we have learned to be cautious about data that are missing, we may have to be cautious about missing analyses also.