{"id":5985,"date":"2012-01-19T08:09:01","date_gmt":"2012-01-19T15:09:01","guid":{"rendered":"http:\/\/dabacon.org\/pontiff\/?p=5985"},"modified":"2012-01-19T08:09:01","modified_gmt":"2012-01-19T15:09:01","slug":"referee-hall-of-fameshame","status":"publish","type":"post","link":"https:\/\/dabacon.org\/pontiff\/2012\/01\/19\/referee-hall-of-fameshame\/","title":{"rendered":"referee Hall of Fame\/Shame"},"content":{"rendered":"

At the Pontiff, we are big fans of Science 2.0 in all its forms. But even within the traditional journal system, there are ways to improve the peer review system. One tragedy of peer review is how little credit the referees get for doing good work, and how there’s nothing to trouble them if they don’t but their own guilty consciences and a bunch of emails from an editor.
\nOne approach I recently came across is from an economics journal. They publish a list of their associate editors, together with average turnaround time for manuscripts<\/a>.<\/p>\n

\n\n\n\n\n\n\n\n\n
<\/th>\nEditorial Board Member<\/th>\nManuscripts<\/th>\n<\/tr>\n
<\/th>\nName <\/th>\n Affiliation<\/th>\n Reviewed<\/th>\n Avg Days<\/th>\n<\/tr>\n
1<\/td>\nBessembinder, Hank<\/p>\n University of Utah<\/p>\n 8 <\/p>\n14<\/p>\n
2<\/p>\n DeAngelo, Harry <\/p>\nUniversity of Southern California<\/p>\n 4<\/p>\n 18<\/p>\n
3<\/p>\n Dittmar, Amy<\/p>\n University of Michigan<\/p>\n 4<\/p>\n 25<\/p>\n
4<\/p>\n Duffie, Darrell<\/p>\n Stanford University <\/p>\n2<\/p>\n 12<\/p>\n
5<\/p>\n Fama, Eugene<\/p>\n University of Chicago<\/p>\n 6<\/p>\n 2
\n<\/table>\n

etc.\n<\/p><\/blockquote>\n

This is a nice start, but surely we could do more. When I teach, I get student evaluations. What if the authors of the papers I refereed also gave me 1-5 stars as a reviewer? Before you mention the obvious problem, the way to average the ratings would be by outcome: all the “reject outright” ratings are averaged together, all the “revise and resubmit” ratings are averaged together, etc., before these are all combined into a final score. That way, you couldn’t get high ratings just by always accepting everything.
\nClearly this proposal still needs more work. Any ideas?<\/p>\n","protected":false},"excerpt":{"rendered":"

At the Pontiff, we are big fans of Science 2.0 in all its forms. But even within the traditional journal system, there are ways to improve the peer review system. One tragedy of peer review is how little credit the referees get for doing good work, and how there’s nothing to trouble them if they … <\/p>\n

Continue reading “referee Hall of Fame\/Shame”<\/span><\/a><\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[32],"tags":[],"class_list":["post-5985","post","type-post","status-publish","format-standard","hentry","category-general"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/dabacon.org\/pontiff\/wp-json\/wp\/v2\/posts\/5985","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dabacon.org\/pontiff\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dabacon.org\/pontiff\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dabacon.org\/pontiff\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/dabacon.org\/pontiff\/wp-json\/wp\/v2\/comments?post=5985"}],"version-history":[{"count":0,"href":"https:\/\/dabacon.org\/pontiff\/wp-json\/wp\/v2\/posts\/5985\/revisions"}],"wp:attachment":[{"href":"https:\/\/dabacon.org\/pontiff\/wp-json\/wp\/v2\/media?parent=5985"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dabacon.org\/pontiff\/wp-json\/wp\/v2\/categories?post=5985"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dabacon.org\/pontiff\/wp-json\/wp\/v2\/tags?post=5985"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}