Peer-review of scientific papers does not guarantee good science

Some comments to my posts, especially those dealing with climate change, complain that some of the sources I cite are not from peer-reviewed scientific journals and therefore should not be considered as reliable sources. They belittle blog posts from scientists just because the research has not been published in a journal.

But, just because a paper appears in a peer-reviewed journal does not guarantee that it is correct. In fact, the number of retractions of papers published in peer-reviewed journals is rapidly increasing.

Canada’s Financial Post has an editorial by Dr. Bob Carter on the peer-review process titled: “Money corrupts peer-review process.” (See full article here.)

Carter explains how peer-review process works:

A potential scientific author conducts research, writes a paper on his or her results and submits the paper to a professional journal that represents the specialist field of science in question.

The editor of the journal then scan-reads the paper. Based upon his knowledge of the contents of the paper, and of the activities of other scientists in the same research field, the editor selects (usually) two persons, termed referees, to whom he sends the draft manuscript of the paper for review.

Referees, who are unpaid, differ in the amount of time and effort that they devote to their task of review. At one extreme a referee will criticize and correct the writing of a paper in detail, including making comments on the scientific content; at the other extreme, a referee may merely skim-read a paper, ignoring obvious mistakes in writing style or grammar, and make some general comments to the editor about the scientific accuracy, or otherwise, of the draft paper.

Neither type of referee, nor those who lie between, pretend to check either the original data or the detailed statistical calculations (or, today, complex computer modeling) that often form the kernel of a piece of modern scientific research.

Each referee makes a recommendation to the editor as to whether the paper should be published (usually with corrections) or rejected, the editor making the final decision regarding publication based on this advice.

In essence, then, peer-review is a technique of editorial quality control. That a scientific paper has been peer-reviewed is absolutely no guarantee that the science it portrays is correct.

Carter continues:

During the latter part of the 20th century, Western governments started channeling large amounts of research money into favored scientific fields, prime among them global-warming research. This money has a corrupting influence, not least on the peer-review process.

Many scientific journals, including prestigious internationally acclaimed ones, have now become captured by insider groups of leading researchers in particular fields. Researchers who act as editors of journals then select referees from within a closed circle of scientists who work in the same field and share similar views.

The Climategate emails showed this corrupting influence in action. “A worldwide network of leading climate researchers were revealed to be actively influencing editors and referees to approve for publication only research that supported the IPCC’s alarmist view of global warming, and to prevent the publication of alternative or opposing views.”

Good science is where you find it, whether in a peer-reviewed journal or not. In my opinion, insisting valid scientific points can come only from publication in a peer-reviewed journal is both scientific snobbery and an invalid appeal to authority.  By the way, Albert Einstein’s 1905 paper on relativity was not originally published in a peer-reviewed journal.

 Example of incorrect science published in a peer-reviewed journal: July 9, 2012, the Arizona Daily Star reported that two new studies showed that a study by NASA, published in the journal Science in 2010 was wrong. Back in 2010, I questioned the original study. See my post here.

There are two other considerations about scientific journals.  First, the process of getting a paper published is slow, whereas as discovery can immediately be made public through a blog.  Secondly, there is the consideration of expense and access.  Print journals cost money, so publishers charge subscription fees and often also charge the authors a fee.  That’s fine, but it limits access to the information.  There is some campaigning to require research funded by taxpayers to provide free access to the results.  Print journals probably could not survive under those conditions unless they, too, were funded by taxpayers.  Blogs by scientists cure those problems.  And, scientific blogs often have hundreds of “reviewers.”

See also:

The Case Against the IPCC and Proponents of Dangerous Anthropological Global Warming

The Assumed Authority

IPCC and Peer Review

Advertisements

2 comments

  1. So what’s the implication here? We should just forget peer review? Just forget review all together? Should we just support what we feel like seems right? Or maybe just follow and espouse whatever supports our personal opinions or political views? One would think that some review is better than none, but hey what do I know? I’m just some dumb blog reader looking to soak up fodder for my preconceived notions.

    1. The implication is stated in the second paragraph: just because a paper appears in a peer-reviewed journal does not guarantee that it is correct. Furthermore, blog posts by scientists should not be dismissed out of hand just because they are not published in a j ournal.

Comments are closed.