jump to navigation

The HIP-index: A Better Measure of Research Impact November 16, 2013

Posted by Andre Vellino in Bibliometrics, Citation Analysis, Statistical Semantics.
Tags:
trackback

hip-index

Eighteen months ago, Xiaodan Zhu, Peter Turney, Daniel Lemire and I embarked on an experiment to see if we could identify the features in an article that would enable us to identify the critical (vs. incidental) references.  We thought that being able to identify references that are crucial would help us devise a better researcher productivity index – one that was better than the h-index.

I am happy to report that we were successful!  In September I gave an overview presentation to the U. Ottawa School of Information Studies that describes the problem we were trying to solve, our methods and results. Since then our paper has been accepted for publication in JASIST, most likely in a 2014 issue.

To automatically identify the subset of references in a bibliography that have a central academic influence on the citing paper, we examined the effectiveness of a variety of candidate features – positional features, semantic features, context features and citation-frequency features – that might be predictors of the academic influence of a citation. We asked the authors of 100 papers to identify the key references in their own work and created a dataset in which citations were labeled according to their academic influence (note that this dataset is made available under the Open Data Commons Public Domain Dedication and License). We then used supervised machine learning to perform feature selection and found a model that predicts academic influence effectively using only four features.

The performance of these features inspired us to design an influence-primed h-index (the hip-index). Unlike the conventional h-index, the hip-index weights citations simply by how many times a reference is mentioned. We show that the hip-index has better precision than the conventional h-index at predicting ACL Fellows on a collection of 20,000 articles from the ACL Digital Archive of Research Papers.

P.S. (Nov. 18) Daniel Lemire in his related blog post gives the following credit, which I entirely share: Most of the credit for this work goes to my co-authors. Much of the heavy lifting was done by Xiaodan Zhu.

Comments»

1. drgunn - November 17, 2013

Nice work! I’ll be looking for the paper.

2. Andre Vellino - November 17, 2013

Thanks William.

3. Gilson Volpato - November 17, 2013

Nice idea. I predicted, about 13 years ago, that the relevance of the citation should be considered in the future of a scientometric index… now you showed us how to make this. Thanks a lot.

4. Not all citations are equal: identifying key citations automatically - November 18, 2013

[…] To this end, we proposed the hip-index and it appears to be better than the h-index. See Andre Vellino’s blog post on this […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: