Tuesday, March 22, 2005

Blogs as a substitute for peer review 

Rick at Stones Cry Out has taken his hand to writing a critique of a study on the exit polls that showed a victory for John Kerry, which a group of researchers argue could be evidence that something odd happened in President Bush's victory. Mystery Pollster says it's a good skeptical essay, which I think says a great deal. You're encouraged to read it.

I think a more interesting point in this case is whether blogs can be used to review papers in progress of non-peer reviewed pieces (which was one point Rick made in his posts yesterday.) I think the answer is obviously yes, but does it break new ground? I could always publish a piece that critiqued another piece. If it was in a peer-reviewed journal my best recourse was always to go to the same journal -- indeed, most journals won't take a comment or critique of another journal's article that doesn't break new ground. For non-peer-reviewed articles, the comment has more outlets.

The blog can comment on either piece. The question is whether a blog comment is a good place for an academic to comment on a peer-reviewed article. I think the answer is mixed. A comment that passes through the same screen that the original article did and is published carries a great deal of weight. As a writer, you want that. But at the same time, journal editors also are humans with egos, and pointing out that they published a flawed piece isn't always well received.

Of course the blog's readership matters a great deal. I think that's one reason why it's a good advantage for those of us who do research at more teaching-oriented institutions. Blogs don't suffer from the same snobbery as the editor who, according to economic lore, scoffed at the idea of blind reviewing of articles. "Of course I want to know who wrote it," the story goes. "How else do I know if it's any good?"