{"id":1810,"date":"2008-01-31T11:57:59","date_gmt":"2008-01-31T18:57:59","guid":{"rendered":"http:\/\/dabacon.org\/pontiff\/?p=1810"},"modified":"2008-01-31T11:57:59","modified_gmt":"2008-01-31T18:57:59","slug":"measuring-reviewers","status":"publish","type":"post","link":"https:\/\/dabacon.org\/pontiff\/2008\/01\/31\/measuring-reviewers\/","title":{"rendered":"Measuring Reviewers"},"content":{"rendered":"

I enjoy reviewing papers even knowing it sucks up too much of my time. I mean what better way is there to get out any inner angst than to take it out on the writers of a sub par paper? (That’s a joke people.) Reading Michael Nielsen’s post<\/a> taking on the h-index (Michael’s posting more these days!), reminded me of a problem I’ve always wondered about for reviewing.
\n
\nSuppose that in the far far future, there are services where you get to keep control of your academic identity (like which papers you authored, etc.) and this method is integrated with reviewing systems of scientific journals. (I call this Dream 2.0.) One benefit of such a setup is that it might be possible for you to get some sort of “credit” for your reviewing (altruists need read no further, since they will find any such system useless.) But the question is how should one measure “credit” for reviews. Certainly there is the number of reviews performed. But is there a better way to measure the “quality” of a reviewer than simply the number of reviews performed. Or the number of different journals for which the person reviews? Ideally I would think such a measure would punish a reviewer for letting papers through which never get cited. Or maybe punish a reviewer for not letting a paper through which gets accepted somewhere else and is successful. As a start, I think Cosma’s observation<\/a><\/p>\n

…passing peer review is better understood as saying a paper is not obviously wrong, not obviously redundant and not obviously boring, rather than as saying it’s correct, innovative and important.<\/p><\/blockquote>\n

is probably a nice place to start.
\nOh, and I certainly won’t claim that it is a problem of any real significance (this is to stop the Nattering Nabobs of Negativity<\/a> from yelling at me) It’s just a fun metric to try and define. Sort of like the quarterback rating.<\/p>\n","protected":false},"excerpt":{"rendered":"

I enjoy reviewing papers even knowing it sucks up too much of my time. I mean what better way is there to get out any inner angst than to take it out on the writers of a sub par paper? (That’s a joke people.) Reading Michael Nielsen’s post taking on the h-index (Michael’s posting more … <\/p>\n

Continue reading “Measuring Reviewers”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[70],"tags":[],"class_list":["post-1810","post","type-post","status-publish","format-standard","hentry","category-science"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/dabacon.org\/pontiff\/wp-json\/wp\/v2\/posts\/1810","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dabacon.org\/pontiff\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dabacon.org\/pontiff\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dabacon.org\/pontiff\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dabacon.org\/pontiff\/wp-json\/wp\/v2\/comments?post=1810"}],"version-history":[{"count":0,"href":"https:\/\/dabacon.org\/pontiff\/wp-json\/wp\/v2\/posts\/1810\/revisions"}],"wp:attachment":[{"href":"https:\/\/dabacon.org\/pontiff\/wp-json\/wp\/v2\/media?parent=1810"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dabacon.org\/pontiff\/wp-json\/wp\/v2\/categories?post=1810"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dabacon.org\/pontiff\/wp-json\/wp\/v2\/tags?post=1810"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}