Better metrics for high-impact research

Recommended Alternatives to the Journal Impact Factor

Though alternatives to the Journal Impact Factor have been developed (e.g., Eigenfactor Score, CiteScore, Scimago Journal Rank), none have been widely adopted. While each of these was developed to address specific concerns about the JIF, none address the underlying issues regarding the data used to calculate the metrics. The publication indexes on which these metrics are based are biased towards English-language publications and high-volume fields of research. Inclusion in these indexes does not reflect journal quality. Furthermore, the majority of citations continue to be accrued by a small fraction of published articles (i.e., Matthew Effect).

As publishing has transitioned to a networked environment, it is now possible to capture much more information about how scholars access and interact with research articles. To improve evaluation of scholars and their articles at IUPUI, we propose the use of article metrics to broaden the conversation about “high impact” research. Two types of article metrics, already used by some research areas and campus departments, could be readily adopted:

  • Normalized article citation metrics: These metrics, such as the Field Citation Ratio, Field Weighted Citation Impact, and Relative Citation Ratio, are metrics about an article. The normalization process corrects two main problems with the JIF. The normalized citation metric corrects for 1) the publication date; and, 2) disciplinary differences in the rate of publishing and citation.
  • Article usage metrics: Article downloads are another research metric, or indicator, that has shown potential relevance for gauging use and later citations. The benefit of using downloads is that they accrue in real-time, as users download the article for use. Particularly for recently published articles (less than 12 months since publication), article downloads may be the most salient indicator of interest and potential use.

When discontinuing use of the Journal Impact Factor is not immediately feasible, we recommend the following:

  • Contextualize the JIF: When using the JIF, present the JIF value along with the relevant research area, ranking in the assigned research area, and quartile ranking or percentile in the assigned research area. Some journals are assigned to multiple research areas; in that case, choose the research area most relevant to the evaluation task.
    • For example: The 2017 Journal Impact Factor for Virulence is 4.0, which puts the journal at a ranking of 9 of 93, or the 91st percentile, for journals in the research area “Infectious Diseases”.
  • Supplement the JIF: To provide a more complete picture of scholarly impact, use normalized article citation metrics and article usage metrics along with the contextualized JIF.

When discontinuing use of the JIF is possible, use article metrics.

  • In some fields, the JIF is simply not available or does not apply. Article metrics and qualitative reviews are better options for evaluating individual products and scholars. In addition to the field normalized citation scores and other article metrics described above, evaluators should consult with librarians to identify other approaches to gathering evidence of research impact. Qualitative approaches, testimonies from known experts in the field or comments from community stakeholders should not be overlooked.

Next Steps

Thank you for taking the time to review this information. We invite you to reach out to the IUPUI Research Metrics Services team with any questions, concerns, or to discuss further.

Download a PDF of this page to share with your colleagues.