Skip to main content

Scholarship Impact Metrics: Home

Research Services Librarian

Daniela Solomon's picture
Daniela Solomon
Contact:
Kelvin Smith Library Room 201-L
Email dxs594@case.edu


Research Services Librarian for Chemistry (interim) and Engineering: Biomedical; Chemical & Biomolecular (interim); Civil; Computer & Data Science; Electrical, Computer and Systems; Macromolecular Science (interim); Materials Science; Mechanical & Aerospace.

http://orcid.org/0000-0002-6200-5216
Phone (216)368-8790

Ask KSL

Reminder: Online Access

  • Library resources require going through CWRU Single Sign-On.
  • The best method is to follow links from the library website.
  • When logged in and a browser window is not closed, access should continue from resource to resource.
  • Remember to close your browser when done.
  • More information see this guide.

Measuring Impact

The main reason for the strong interest in adopting and using  scholarship metrics is the increased demand from funders and institutional administrations to demonstrate the impact of scholarly activities. 

There are two aspects that are taken in consideration when evaluating the impact of scholarly activities:

  • academic impact, defined as the contribution to science advancement
  • societal and economical impact, defined as the contribution to society and economy

This guide is a short introduction to the most common metrics used to measure academic impact and their flaws. As a direct result of the issues surrounding these metrics, the undiscriminating use of these metrics is inappropriate, and instead it is recommended to use a balanced and responsible approach as expressed in the Leiden Manifesto and the Metric Tide, see below.

Overall, it is agreed that use of metrics can support, but not supplant, the assessment of academic impact.

Academic Impact Metrics

The first metrics used for academic impact were number of publications and citations counts. In time, more and more citations based indices have been proposed. Additionally, the increased popularity of the social web has prompted the development of new forms of scholarship, which in turn, challenged the citations based metrics and created the need for new ways of measuring impact. One growing trend in this direction is altmetrics that measure the social media activity.  

Traditional Metrics

All scholarship metrics are calculated based on citations count, with citations being considered to represent a proxy for paper quality and its influence to another publication. 

Known citations issues include difficulty to interpret, long time to accumulate, source dependent and incomplete, discipline dependent (skewed toward STEM), publication venue dependent, language dependent, not available for new forms of scholarship, analysis is difficult to execute, measure only one aspect of the impact, and not useful in comparing scholars

​Recommended readings on the use of metrics in impact assessment

Altmetrics

As the scholarly communication is increasingly moving online, new scholars' expectation changed and the need for measuring web-based activity increased, leading to the development of altmetrics. 

The term "altmetrics" is a misnomer as it does not represent "alternative metrics," but instead, these represent an addition to traditional metrics by measuring social media reach of scholarly works. The "reach" is measured in article mentions, download counts, pageviews, social bookmarks, mentions on academic social sites, blogs, Facebook, Twitter, etc.

Recognized issue with altmetrics include measure reach and not quality, no wide availability, not sure what the numbers mean, and they are prone to gaming.

​Recommended readings on altmetrics