Many of you have probably already seen this. Jorge Hirsch, a physicist from UCSD, has proposed an interesting way to measure research impact of an author. For details, see this Nature article or Hirsch’s original article physics/0508025. The basic idea of Hirsh’s h-index is very simple. The index is simply the number of papers which the author has written which have more citations than this number of papers. Thus, for instance, if an author had written five papers with the following number of citations, 10, 6,4,2, and 1, then the h-index would be three because the forth most cited paper has only two citations, which is less than four, but the third most cited paper has four citations which is greater than three. The highest h-index among physicists, Hirsch claims, is Ed Witten who has an h-index of 110. This means he has written 110 papers with greater than 110 citations. Wow! Another important quantity Hirsch defines is the average rate at which an h-index has been changing per year over a career. This is just a person current h-index divided by the time since they first started publishing. Witten has an astounding value of 3.9 increase in h-index on average per year over his career. What this all means is very much open to debate, but heck, it’s kind of fun!
One thing which is nice about the h-index is it is very simple to calculate it using the ISI Web of Science citation tools or, more dangerously, from citebase. My h-index from citebase (access to Web of Science is painful from my current computer location) is 12. The funny thing is that Hirsch says that this is about the h index (with large error bars) at which one should get tenure. Haha, very funny!
O brave new quantum world!
Two things: one, the nature link is to some UW page which requires a password. Two, why do you say that citebase is dangerous?
Citebase is dangerous because it’s incomplete, which could play havoc with a heavily thresholded measure like this.
Interestingly, citebase is probably less dangerous for our field (quantum information science) than for numerous other fields.
The major drawback of H-factor (which is not that easy to compute quickly, by the way, and doesn’t take into account first authorship) is that the number of publications and citations is extremely sub-field dependent.
It’s interesting that your h-index is impressively high (12) – I ran it for a couple of people I know:
one is an assistant professor (since 2003) at Princeton with current H-factor of 4, another is asst. prof. at U Wisconsin Madison (since 2002) with current H-factor 5. Yet another friend (also cond matter) just got a faculty position at Urbana with H=6.
All three are experimental cond. matter physisist – this is what you get for developing new and relatively unexplored fields.
You would think theorists would have higher H’s by a lot, but there are some exceptions here as well –
one friend of mine is starting assistant prof. in theory at Caltech with H=3. At the same time I know some people with H>~=10 who couldn’t get any decent offers for tenure-track positions.
I believe that taking into account quality of publications, how much the person contributed to those publications (first/second author etc.), the overall potential significance of the research, the presence of well-defined and novel direction is what the decisions for young physicists are based on – plus of course other parameters. H-index may be better for older, well-established scientists (and even then it has it’s own limitations), but tenure-track offers are made based primarily on “potential” rather on purely publication record, however intangible variable it may be.
So it all depends, interesting to see whether h-index becomes well-accepted measure, or will blow over like many other novelties.
So what would Einstein’s “H-factor” be just based on his annus mirabilis papers of 1905?
Citebase apparently doesn’t include biology papers so I wasn’t in there with my 1 paper. I am curious to see how many times it’s been cited, though…
Any gimmick like the H-index is at best an external measure of success. It is a way to roughly estimate the importance of an author’s papers if you are not qualified to read them. It’s like to ranking universities by their endowments. Of course it will be roughly correct.
All external measures of success and authority can only be roughly correct. All of them are also subject to manipulation. They cannot substitute for the thing that they are meant to approximate, the direct judgment of insiders. The dilemma is that that measure cannot be shared with outsiders, since accepting someone else’s judgment (say by letters of recommendation) is also an external measure.
I don’t think the h-index as some absolute major will be very important. For many of the reasons CME said: there are certainly many important quantifiable and intangible elements which go into assessments. But I do think it might catch on as one of a portfolio of interesting statistics.
Interestingly, physicists are often shocked that there are fields where the concept of “first authorship” does not exist…computer science for example doesn’t seem to have such a ranking for papers. But really if such mattered, shouldn’t we put percentages beside peoples names for a paper? Boy that would be dangerous…I wonder how many percentage points would cause good friends to become bitter enemies.
Mathematics might be almost alone in still having many solely authored papers. Even in math there is a trend toward more authors. A rule that I have heard is that if a math paper has two authors, both of them get 90% of the credit.
There are many parasitic practices in publication: excessive coauthorship, publishing LPUs (least publishable units), jumping on bandwagons, hyped results, overcitation (with expected reciprocation), etc. Each of these practices reflects one or another inaccuracy in external measures of success in science. Inaccuracy is essentially equivalent to manipulability.
If all external measures of success are inaccurate and therefore manipulable, what should we do? First, when you are an insider, just don’t resort to external measures of success. Instead rely on your own aesthetic judgment as to what work is important and why.
On the other hand, you may sometimes have to pass judgment on people beyond your expertise. For example if you are a dean. In that case you have to rely on these external yardsticks, but you should take them all skeptically. (All of them, including letters of recommendation and the department vote.) That’s why I don’t like some of the tone of Hirsch’s article, even though his actual idea is okay, even a bit clever.
The percentage distribution of contributions from various co-authors is a great idea IMPO, and could easily be done anonymously, by a web form, filled out by first author, who should have pretty solid judgement of which one of co-authors did what. As a first author, I am getting quite frustrated when 90% of work is done by 2, in some cases 3 people, yet there are 8 co-authors who need to be included for various political/historical reasons. In plenty of situations second, third co-authors (often grad students) get screwed from such practice the most, while some more senior collaborators get credit for doing absolutely (and literally) nothing. I guess as first author I shouldn’t care too much if a publication includes 3-4 people or 7-8 people, but it rubs me the wrong way, since everyone aside from first and maybe last author shares the same amount of “virtual” credit for a publication.
Phys Review journals (as an example) could easily have implemented such system, with contribution-weighed factors updated in increments of, say 3-4 publications – just so that you can’t reverse-engineer your most recent grade from increment of the average.
I think such “weight” system would
a) curb the parasitic practice of including people for political reasons
b) assign proper credit to 2nd, 3rd authors – in some cases 2nd author contributes almost equally to 1st author, while 3rd+ contribute much, much less, in other cases 2nd and 3rd contribute almost equally but considerably less than 1st, and so on – both should make collaborations more healthy and productive, with less “leeching” and bickering over who gets to have first authorship.
From citebase:
Cirac: 34
Shor: 22
DiVincenzo: 21
Preskill: 20
Bennett: 16
Grover: 8
Vazirani: 5
I think this reveals a major flaw with the h-index: it doesn’t assign proper weight to highly influential papers (quite a few of us have h-index > 10, but not many of us have been as influential as Vazirani and Grover. Bennett also seems short-changed).
A simple fix would be to include a citation weight for those papers contributing to a person’s index.
Heh. If you use Web of Science for Cirac, you get 51. Here are the highest people in quantum information I could find using Web of Science:
Peter Zoller 64
David Wineland 56
Lu Sham 52
Igancio Cirac 51
Eli Yablonovitch 46
Of course Lu Sham also has one of the highest cited papers of all time, with over 10000 citations! Interestingly, if one is interested in only quantum information science, then citebase probably has advantages over web of science…although it still misses some of the early papers altogether.
No, there is no simple fix. An insubstantial but well-positioned paper can get a lot of citations. Some people are much better at positioning than substance.
I’ll admit that this whole topic is sore for me because I usually feel like a shlemiel at positioning my work. There is nothing wrong with good positioning; I enjoy citations as much as the next person. But in practice, certainly in quantum information theory, I have to rely on other motivators.
Or rather, more like a schlimazel than a schlemiel.
You are absolutely right that the real value of a research paper is subjective. If you could compute the value of a paper with an algorithm, then science would be a competitive sport and not a creative process.
If a paper is highly cited, that does not exactly mean that it “generated” the activity of the people citing it. It is equally possible that the activity would have occurred anyway, just without that particular citation. And this associated activity may or may not be fruitful. I agree that if the paper really does generate the activity that cites it, and if that activity is important, then the paper is important.
For example, I see that (according to Google Scholar) Wolfram’s Book, “A New Kind of Science”, has 515 citations. I am skeptical that this book by itself is more valuable than my entire research career, even allowing that some of my papers might be “sleeping beauties”. I grant that the magnitude of the effect of this book is large, but I believe that if you also look at the direction, its net influence is negative.
On the other hand, Peter Shor’s factoring paper truly is revolutionary and substantial. I can only wish for even a fraction of that kind of impact from my work. In the tautological sense Shor’s paper is well-positioned, because it has a lot of citations. But you are right that this was not Shor’s doing, it was the response of the community. Even if the community had never responded, it would still be a great paper.
Actually, I don’t like the phrase “sleeping beauty” for papers that aren’t “yet” widely cited, although it’s better than directly equating importance with citation. I don’t think that the paper “Chtoucas de Drinfeld et correspondance de Langlands”, by Laurent Lafforgue is “asleep”, despite having only 43 citations in Google Scholar. After all, Lafforgue won the Fields Medal for it.
Greg, the label “insubstantial” is pretty subjective. I would say that if a paper generates a lot of interest and activity (e.g. in the form of a lot of citations) then it is substantial. Of course this is only a sufficient condition for “substantial”. There are many “sleeping beauties” out there, i.e., papers that were discovered years after their publication by the community. There was an interesting piece about this in Physics Today, June 2005 (“Citation Statistics from 110 Years of Physical Review”). And on the other hand some papers have a lot of impact without being “well positioned” (e.g. Shor’s factoring paper).
Dave, do you have a smart way to calculate the h-index using Web of Science? I.e. how do you get it to sort results by citation order?
“… do you have a smart way to calculate the h-index using Web of Science? I.e. how do you get it to sort results by citation order?”
Do a general search, then select “sort by times cited ” in the panel on the right.
I agree with everything “Greg the Schlimazel” said, but I’d go even further: the entire *purpose* of academia is to provide a refuge in which ideas aren’t judged solely by their popularity (or citation count, which is just popularity in a more restricted setting). If we adopted Daniel’s criterion, that “if a paper generates a lot of interest and activity then it is substantial,” then the writings of Deepak Chopra and Dr. Phil would be thousands of times more substantial than anything we’ll ever write.
Well, it seems to me that there are two different discussions going on here. One is about Science with a capital “S” and one is about science with a lower case “s.” The first of these is the science of big discoveries, things that change our view of the world, lead to entire new fields, or solve an important open problem. The second science, however, is the science of the masses: the science which doesn’t change the world, but which slowly but steadily pushes the ball forward. For Science, I think the h-index is a poor indicator. For science, I think the h-index, along with other quantitative measures, is more useful. How do you judge the masses of scientists who haven’t revolutionized their field. I think this is harder, and quantitative tools deserve a some standing here. However, it is still true that there is a lot missing from just someone’s citation statistics: potential to do Science chief among the missing statistics…how the person fits in with his or her community (research and institutional), etc.
I would prefer a silly “objective” measure that uses the same yardstick for all always over a subjective departmental evaluation committee where members have their own agenda. And, of course, who on such a committee has time and expertise to read all the papers of the people they evaluate.
Here’s some statistics on h-index for young (fresh PhD) scientists – reposted from another blog:
Analyzed are 72 PhD’s (for 2002, 2003 and 2004) from Harvard Physics and Applied Physics. Discarded are several high-energy PhDs. Analysis was done using Web of Knowledge which may not include conference proceedings and low-ranked journals, which shouldn’t affect analysis anyways.
Older PhDs (2002) obviously have an advantage over younger (2004) in terms of h-index, which tends to increase over time.
Average number of publications during PhD is 6.0 with average h-index 3.1.
3 out of 72 had h-index higher than 6 – Yaroslav Tserkovnyak (PhD 2003,l Halperin group) with h-index 10 and 25 publications, Jonathan Weinstein (PhD 2002, Doyle) with h-index of 9 and 9 publications and Oleg Shpyrko (PhD 2004, Pershan) with h-index 7 and 14 publications.
22 out of 72 had h-index from 4 to 6, and 47 had h-index 3 or less.
Condensed matter and AMO experimenters seem to do better than others – among 10 people with h-index 6 or higher, only 2 are string theorists )Volovich, Headrick), and one condensed matter theorist (Tserkovnyak) – others are CM experiment (Shpyrko, Topinka, Gordon, Prasad, Wu) or AMO experiment (Weinstein, Zabow).
13 out of 72 PhD’s had 10 or more publications – Yaroslav Tserkovnyak with 25, Meghan Valentine 17, Dan McKinsey 16, Oleg Shpyrko 14, Mark Topinka 14, Margaret Gardel 14, Carlo Mattoni 14, Sergei Dzhosyuk, Venrita Gordon 13, Brian LeRoy 12, Gary Zabow 12, Deiner 11, Fiete 10.
Once again, most (12 out 14) are CM or AMO experiment. They also on average take longer to get PhDs than high energy theorists, for example.
32 people had 3-9 publications and
27 had 3 or less.
All this is ridiculous. Everybody is talking about this h-index invented by a failed physicist, Jorge Hirsch. One quick look at his web page or any of his junk papers shows that the guy is unreasonable, and the fact that he called this thing “h-index†basically proves it (I bet “h†stands for “Hâ€irsch). He is a complete nobody in condensed matter theory and I guess this was the only way for him to get famous.
I do not know what is going on with theoretical physics. People are obsessed with things, which have absolutely no significance. The theoretical physics community reminds me of sheep, following a drunken shepherd. Even worse, because unlike sheep many physicists are unjustifiably complacent, often obsessed with their own greatness (especially so-called string theorists), which in my opinion is disgusting. There is no single field in theoretical physics, where people do reasonable things. Sometimes young people are forced to write nonsense papers, just because this is oftentimes the only way to get jobs or, say, to increase one’s h-index. May be the problem is that now there are no real leaders in physics, individuals of the caliber of Feynman or Landau, who would point out to the absurdity of the current state of affairs. Can you imagine Einstein, Feynman, or Bohr counting each other’s citations?
I myself feel very disappointed in physics. Although I recently got a faculty job, I am very seriously thinking of leaving Academia because I think that modern theoretical physics is a joke. To be successful one has to spend enormous amount of time doing stupid things, such as dealing with physics education people, who make you teach according to their idiotic methods or writing grant proposals about drawing so-called minorities into science. One does not have time anymore to do high-quality research; all time is consumed by different kinds of nonsense. And now one has to worry about this h-index. I am truly amazed that people take it seriously. Some departments already use it officially in evaluating their faculty and making decisions about promotion and tenure. This is just hilarious. I do not want to spend my life worrying about the h-index. I’d rather worry about the Dow Jones
Wow Sad Physicist, that is a lot of angst for one comment! Very sad.
Of course I’m sympathetic to a lot of what you say (I have no personal opinion about Hirsch, not being a condensed matter person, and don’t quite see how his personal physics has anything to do with the validity of the index he has produced to measure something which lazy tenure committees use. I also have a quite differing view of physics education people and the role of minorities in science.) But certainly if I take all of theory, the amount of noise is very very high. At least in quantum computing, there are gems among this noise, and, well the reason I keep on going is because the leaders in my field are very good at doing what you say: pointing the field in the direction away from the junk.
Personally, worrying about the Dow Jones also seems to me about as shallow as worrying about the h-index: both satisfy your ego, one your ego of the brain, and the other your ego to be richer than everyone else. Now worrying about the Dow Jones because how markets work and the whole theory of economics: THAT is a good reason to worry about the Dow Jones!
> I also have a quite differing view of physics
> education people and the role of minorities in
> science.
You different point of view is the correct one. If you want to find a good job or get funding that is exactly what you have to say: “I love education people and I will devote the rest of my life bringing minorities into science.†That is what I say. If you try to say anything different, the consequences will be quite serious. Look at what has happened to the president of Harvard, Lawrence Summers. He just said that “innate differences between men and women might be one reason fewer women succeed in science and math careers.†And now look at the outcome:
http://www.president.harvard.edu/speeches/2006/0221_summers.html
This is frightening… So, one has to be careful here.
> the leaders in my field are very good at doing
> what you say: pointing the field in the
> direction away from the junk.
Of course you are right here as well. I am sure they are doing great job pointing you in the right direction in the exciting field of two-level systems in single-particle quantum mechanics. Keep up the good work! And keep you h-index high!
Ouch.
Have a nice day.
FYI: There is a useful online tool to calculator an individual scientist’s H-index, using Google Scholar database.
H-index calculator of scientist impact
Publish or Perish is a software program that retrieves and analyzes academic citations and provides a wide range of citation metrics, including the h-index. It allows users to review both cited articles and citations in order to correct potential citation errors. It is free for non-profit use.
http://www.harzing.com/pop.htm
See also Reflections on the h-index andReflections on Google Scholar for a discussion of the validity, assumptions, and limitations of the underlying sources and methods used by Publish or Perish.
Dear sad physicist,
There is a lot of truth in some of the things you say, however I would not think that you will break out of your slump by putting down the work of the pontiff or others.
Ironically all three of the “real leaders” you mentioned earlier (Einstein, Bohr, Feynman) worried a great deal about “two-level systems in single-particle quantum mechanics” and Feynman is one of the founding fathers of quantum information. Einstein and Bohr spent weeks arguing with each other what it all meant.
If you got that all figured out, good for you, maybe you can let the rest of us know what the answers are. If not, quantum information seems to include a serious attempt to systematically approach the questions of the founding fathers that are (in my mind) still open. A quick look on current topics would also reveal, for example, that there is a close relation between quantum information processing and the interaction of spin systems. Pretty close to hardcore many-body theory from my lowly lab-rat perspective.
Dear h-index lovers!
I would like to bring to your attention a new paper by this crazy man, Jorge Hirsch (arXiv:0708.0646) “Does the h-index have predictive power?” (the answer is “yes,” it does have this power). It seems that he has invested a lot of his valuable time and applied his superior intellect to prove “…the superiority of the h-index.” Obviously, the main conclusion of this “work” is that if you have a high h-index now, you are much more likely to become a great scientist in the future (to be a great scientist, of course, is defined as having a high h-index). Thus, you will have the highest scientific potential just before you croak.
I view this paper as another sign that IDIOCRACY is becoming a reality in modern science. I decided that if this nonsense gets published in any serious scientific journal, I quit my faculty job (another job opening for you!)
Best,
Sad Physicist
Dude Sad Physicist, you’re STILL harping on this? Man, that is Sad.
To sad physicist. This reply might be 3 years late, but I can help it: Hirsch made great contributions in the 80 in the field of strongly correlated electrons (for instance, the Hirch-Fye algorithm to quantum Montecarlo). In the 90 he has also very important contributions (like a highly cited and influential paper on the spin hall effect) and, the most important, he thinks out of the beaten track. I wonder who’s the ignorant here. Are you still in academia or you move to Lehman Brothers?
Dear Professor Bacon:
What can I say, you are totally right and I have to admit that I was wrong. I totally agree with you now, Jorge Hirsch is a goddamn genius!
Also, thank you for your interest in my well-being and my career. Three years have passed and I am now a changed man. Unfortunately, I was fired from Lehman Brothers and I started a business: I now sell h-index extenders.
If you allow me, I would like to take the opportunity and post a testimonial about our services. V/R,
Sad (ex-)Physicist
TESTIMONIAL: ENLARGE YOUR H-INDEX!!!
Dear Fellow Physicists!
I would like to share with you a unique experience that has changed my professional life. What am I saying? It has changed my entire life! I hope that it may also help many other researchers, who might be suffering from the anxiety, associated with the small-h-index syndrome and large H-index envy, that I used to have, but have no longer!
Even though most Departments claim that the H-index size of their members is not the most important criterion for hiring, promotion and tenure, recent research (funded by the National Size Foundation) shows that size does matter. It is not always the deciding factor and in rare cases other criteria are used: There are still a few places left, where some old-fashioned senior physicists actually read candidate’s papers and look into the content (Ha!, can you imagine that!?) But fortunately people like that are a dying breed. Also, in cases involving women and underrepresented groups in science, measuring the H-index is inappropriate, and other measures have to be involved. But in most cases, the h-index size does become a crucial deciding factor and circumvents the need to read or understand research, thanks to the genius, Jorge Hirsch! But, now back to my story:
When I was hired by my Department, my H-index was really small. It did not matter much at first, during my honeymoon, when our feelings were fresh and I just took many things for granted. But as time passed, I noticed that eventough my colleagues were nice to me, they clearly were not completely satisfied with my performance. Of course, I performed all required faculty duties and went above and beyond, but nothing could completely satisfy the Department and the higher-ups in the University administration. The problem was my h-index size: It was very, very small. If I did not find a way to enlarge my H-index, I was facing the prospect of not getting tenure, and being replaced with someone else (with a bigger one).
This was when I developed a severe anxiety disorder, which now is widely-known as small H-index syndrome. I did not sleep well, checking my h-index every morning, hoping that it grew overnight. But, alas, it has never happened. I secretly checked h-indices of other members of the Department (thankfully, this information is now publically available), but it was mostly demoralizing: There are physicists here with really big ones and I knew that mine was not even close.
Then, I decided to put my suffering to an end and do everything possible to enlarge my H-index. So, I developed stretching techniques to extend my h-index and they worked! I believe that if you have determination and patience, you can make it as big as you wish. And you do not even have to work hard! You can continue doing crappy research and still enlarge your H-index very substantially. In fact, I developed techniques, where the more unreasonable your research is, the better it is for your h-index size. For example, you can move into an active field and start arguing with the leaders about well-accepted facts (an extreme example of this technique would be to write a comment on the Einstein’s work pointing out that the equation E=mc^2 is wrong and in fact it should be E=mc^3!). I can assure you that in most cases, your work will not go unnoticed!
Order NOW and you will learn more useful techniques that actually work (!) and will also receive 30 Minutes of FREE Live Tutoring on stretching your h-index!
30 day Free trial – for students and postdocs
Bronze package (10 citations) – Ideal for assistant professors
Silver package (20 citations) – Ideal for associate professors
Gold package (50 citations) – For full professors
Platinum H-index extender (100 citations) – For National Academy members!
http://scienceblogs.com/pontiff/2009/07/diary_of_a_sad_physicist.php
Sad (Ex-) Physicist: there is something extremely ironic about how wrong is the strategy you propose to increase your h-index:
“you can move into an active field and start arguing with the leaders about well-accepted facts”
If you look at the evolution of Hirsch career, you will notice a dramatic downturn in the number of citations of most of his papers after approximately 1990. And this was the consequence of doing exactly what you suggest to increase h: he moved into a very active field (High Tc superconductivity) and started arguing with the leaders about well-accepted facts (which does not make them correct, but that’s another story). In particular, he disputed the validity of BCS theory. Interestingly, he has only two highly cited papers after that, and none of them in superconductivity (the h-index one, and the one about Spin Hall effect).
Any of this makes h-index better. And you certainly have a point about people not reading the papers to judge their authors. But the point is being completely lost amongs all the bitterness and the anger in your rather obvious joke about h-enhancer.
I could not calculte mine. Is there any way that you could calculate my h index?
I am a high energy physicist.
You can find my 39 papers in high energy physics at stanford site
http://www.slac.stanford.edu/spires/find/hep/www?rawcmd=f+a+t.ibrahim
with total number of papers of 39 and of 1803 citations.
Sincerely yours
Tarek Ibrahim
Scientific contribution in academia is like the writing of a group of blind men with a dead elephant. Although, the activity of those blind man look absurd to onlookers and their writings will be laughing stock but happily the scientists are enclosed in institutes and there are no onlookers. University education was once about liberty from one’s parental job. With research becoming omnipresent in academia teaching shall become a family job with parents writing papers for their kids and securing citation for them.
Hey guys, i know this guy who’s got over 140 h- index, he also made this time machine, believe me!
regards
E
I need to calcualte my H-index value. Please, give me numerical example
h-index doesn’t make any sense. Although there are so many examples for that, few examples are given here. There is one scientist with h-index of 82 and 37,900 citations even at the beginning of her career. But she hasn’t written a single research paper in her life. Because she got the membership of many CERN groups in the world, she got her name among the thousand names given at the end of many research papers. She is not even a coauthor of any of those papers. Just because CERN members publish papers in so called US or European journals, google scholar search robot finds all those papers. Most of the CERN members got higher h-index and higher number of citations like that.
On the other hand, there are some researchers who have written 70 to 100 papers. But they have a lower h-index below 10 and less number of citations, just because google search robot can’t find even many good journals. Google search robot easily finds US journals, because it thinks that US journals are reputed journals. When I was doing my Ph. D at early nineties, I read several research papers. I found one good paper with original data of ferrite thin films published by some Japanese scientists on a Japanese journal. Few years after that, I found that some US researchers have deposited the same material on the same substrate using same techniques. But the data of US researchers are worse than the data published by Japanese researchers. But US researchers have published their worse data even after one year in US journal of applied Physics. So how can someone say that US journals are the best?