Jorge Hirsch: the man behind the metric
The h-index, conceived in 2005, is the number of papers by a particular author that receive h or more citations. The letter 'h' stands for 'highly cited'. It has already become one of the most widely used metrics for research evaluation, and has been adopted by bibliometricians and non-bibliometricians alike. Professor Jorge Hirsch, whose academic career in physics has taken him from Buenos Aires to Chicago to San Diego, talks to Research Trends about where it all started.
Research Trends (RT): What triggered your interest in bibliometrics?
Professor Jorge Hirsch (JH): There were two main reasons: I had trouble getting papers accepted in journals with the highest Impact Factors because of the controversial nature of my research. Fortunately, there were journals with lower Impact Factors that did accept my papers. Nonetheless, they were well cited, meaning other researchers found them useful.
A criterion often used in evaluating research achievement was to count papers published in high Impact-Factor journals; I wanted to provide an alternative criterion.
Secondly, I was on committees where I had to evaluate and compare research achievements of candidates for academic positions at my institution. I felt that too much weight was often placed on subjective criteria – such as letters of recommendation – rather than objective ones.
RT: How are bibliometrics perceived by physicists?
JH: Opinions are wide ranging: some hate them, some love them, and some have mixed feelings. There seems to be a strong correlation between how physicists perceive bibliometric indicators and how highly they rank with them as individuals. I imagine this is the case in other disciplines, too.
RT: How did you come up with the h-index?
JH: I have always paid a lot of attention to citations. If somebody writes a lot of papers that aren’t cited, it is very difficult to judge whether those papers have any value. In exceptional cases – for example, when research is very novel and not yet understood by the community – it does. But in most cases, un-cited papers are and remain irrelevant. So the number of papers an author writes is not a good indicator of the research achievement of that individual. The cumulative total number of citations for an individual is often not very useful either, because currently most research is collaborative and an author may receive a lot of citations for papers in which his/her role was not very important.
In response, I tried to look carefully at the entire citation record of the individual I was evaluating – that is, at the citation numbers for a large number of his/her papers. This is both time consuming and often inconsistent between candidates, so I wanted to devise an indicator that could be applied simply and consistently, and reflected achievement as much as possible.
Looking at the citation index of many physicists, I came up with the h-index in 2003 and started applying it to physicists I knew, immediately finding a strong correlation between my subjective opinion of them and the value of their h-index. I shared the idea with colleagues, several of whom gave me very positive feedback. Two years later, I decided to write a paper on it.
RT: Did you foresee the influence that the h-index would have on academia?
JH: I had not worked in bibliometrics before and was not totally familiar with the literature on the subject. I had recently read an article on bibliometrics by S. Redner in Physics Today (June 2005) that I found very interesting, and it made me realize how important people find these issues. But I had no idea how my paper would be received, nor whether it would be publishable in a scientific journal.
So I am certainly surprised and happy that my work has been well received. I am especially pleased that it’s attracted attention across all scientific disciplines, not just in physics or even natural sciences. I have some concern, however, that the h-index may sometimes be misused by over-relying on it, although I don't know of any specific instances.
RT: Do you intend to publish further work in bibliometrics?
JH: Yes. Although it is not the main focus of my research at present, I would like to understand the issues better and contribute to the subject.
RT: What do you think of the use of bibliometric indicators for evaluation purposes (e.g. grants, tenure, career advancement, funding, etc.)?
JH: I certainly think bibliometric indicators should play a role in evaluation, keeping in mind that there is a danger of over-reliance on them. Especially in life-changing decisions such as granting or denying tenure, the role of bibliometric indicators should be limited, and complemented by detailed analysis of the candidate and direct evaluation of the scientific content of their research. Such analysis should be especially thorough in cases where there is a large discrepancy between the direct evaluation and the collective evaluation of the scientific community as reflected in the bibliometric indicators.
I believe bibliometric indicators can be particularly useful in aiding decisions on distributing grant support; although it should be kept in mind that non-mainstream research can be undervalued by bibliometric indicators, and could still be highly deserving of support. Bibliometric indicators should always be used alongside other indicators and good judgment.