Further thoughts on post-publication peer review (PPPR)
Sanjay Srivastava blogged some interesting thoughts about the process of post-publication peer review (PPPR), reflecting about his own comment on a PLOS ONE publication. I agree that open peer commentaries after publication are one important part of the future of scientific publishing. There were many cases where I wished to have the opportunity to publish such a commentary. In one case, I actually wrote a commentary on a paper published in Management Science – a strange story about managers, age, and testosterone, which received a lot of press coverage. I submitted it as a commentary to the journal, but it was rejected because of “lack of new results”. Now my commentary rests on SSRN and has been downloaded 5 times in 10 months – yippee-yeah! (probably 3 of these 5 are by myself …). But as SSRN does not allow peer commentaries I could not set a link from the original paper to my comment, and nobody finds it.
Other fields of science additionally established a pre-publication open peer review (also called the “pre-print culture”). Many researchers in mathematics or physics publish their preprints on arXiv and harvest open peer commentaries before submitting the manuscript to a peer-reviewed journal.
I believe devoutly that open PrePPR and PostPPR can significantly improve the quality of scientific output. But one crucial requirement indeed is etiquette, as Sanjay pointed out. I don’t want to see shitstorms coming over scientific articles, especially in the case of young scholars who worked hard to get their first paper published. Comments should be written in the spirit of a collaborative enhancement of research, and less in terms of “debunking”. We all are humans and mistakes can occur. Problems should be pointed out in order to strengthen scientific research, but in a friendly and constructive manner.
Researchers who conceive of science as a highly competitive business where claims have to be fortified and defended might have problems with open peer reviews (e.g., the escalation of the “Bargh rampage” [1][2][3]). But if we see science as a collaborative endeavour in search for knowledge, where no model is “right” but only “less wrong”, open peer reviews can be a very helpful tool.
————–
Some further readings:
- Neanderthal sex debate highlights benefits of pre-publication (news blog on nature.com)
- Interesting post by Jeremy Fox (and commenters), argueing for (standard) pre-publication review, from the perspective of marine biology and ecology
- The case for arXiv and a broader conception of peer-reviews (by Philippe Desjardins-Proulx)
The first CREDAM Award for creative data management goes to … the German government!
“If you torture the data long enough, it will confess.”
This aphorism, attributed to Ronald Coase, sometimes has been used in a disrespective manner, as if was wrong to do creative data analysis. This view obviously is misleading. In contrast, we at IRET have a much more positive and humanistic view of data management, and therefore we have made this aphorism to our leading guide in difficult times.
We at IRET have made it to our mission to proliferate and foster creative ways of data analysis. Therefore, we proudly introduce an award in recognition of outstanding data creativity: the CREDAM Award. CREDAM is both an acronym (CREative DAta Management), and a statement: credam (lat.) means “I will believe”, or “I will trust”.
This years CREDAM Award goes to …….. the German government!
A new report on poverty in Germany is going to be published soon. What does the data say?
| Year | Overall property in possession of rich households | Overall property in possession of complete lower half |
|---|---|---|
| 1998 | 45% | 3% |
| 2008 | 53% | 1% |
Seems like a pretty clear picture, and in a previous version of the report, the authors concluded (based on this and other data), that “income disparity increased” (see Süddeutsche Zeitung). But that is wrong!! But why is it wrong? Well, that interpretation “does not reflect the opinion of the German government”.
On the pressure of the leader of the minor coalition partner, Philipp Rösler (which currently would be elected by 4% of Germans), this conclusion was re-interpreted. Now, the report comes to the completely opposite conclusion: “income disparity decreases“!
As this is a great example of creative data analysis, which liberates us from restrictive and anally retentive “scientific” procedures, we are happy to award the first CREDAM trophy to the German government, especially Phillip Rösler. Congratulations!
(Maybe we should think about adopting this strategy for scientific reports as well. Given highly flexible approaches of data analysis, conclusions should rather be based on a majority vote of all (co-)authors and reviewers, not on empirical evidence.)