Saturday, September 15, 2007

Peer Review

I wrote a couple of pieces in 2005 and 2006 about the problems that scientific journals seemed to be having with regard to peer review of journal articles. Today I came across an article that goes beyond - really before - the question of peer review in attempting to judge the success or lack thereof of a scientific study. The article is well-known in scientific circles; it is the most downloaded paper ever published by PLos Medicine. The article, Why Most Published Research Findings Are False, is by John P.A. Ioannidis.

Ioannidis argues that the research findings are less likely to be true where
  • there is a small sample size
  • small effects - as opposed to large effects such as smoking on cancer - are studied
  • there are a considerable number of relationships being examined
  • there is a great degree of flexibility in designs, definitions, outcomes and analytical modes
  • there are financial and other interests present
  • the field is very hot and much in the news.
Interesting!

1 comment:

Anonymous said...

One has only to view the NBC Nightly news on a regular basis to confirm these findings. One week the research on a particular subject is positive, the next week it's negative. I exaggerate a little, but the context is relevant. I didn't read the complete article, but drug companies are now using us as human guinea pigs, and their results in one decade are being overturned in the next. Science is no longer the "exact science" it once was. The profit and loss columns have seen to that.