Scholarly Research Primer

Impact metrics are helpful and heavily used but should be viewed critically and applied with caution due to many issues that include:

• New research, new journals, early career scholars, and non-English content are under-represented by traditional impact metrics., e.g. Citation Tracking and Journal Impact Factor. 
• Typically focused on quantity (amount of output), not necessarily quality of the work.
• Publishers (eg. Elsevier-Scopus-Citescore) may rank their journals higher than the competition.
• Authors can manipulate their metrics through self-citing.
• No measurement of scholarly, non-academic work Eg. EBP study in a hospital setting.
• No standardization of metrics within or across academic fields.
• Publication frequency differs by format (paper, electronic) and time to publication (newspapers, blogs, journals, books).  This may impact similar content being less cited despite the quality of the research.
• Negative reviews of works cited as poor examples can increase item metrics.

Alternative metrics reveal how content outside the academic milieu is viewed and may be ranked for impact.  

Data Sources:

• Media, e.g. newspapers, tv news, journal websites
• Social media and web content, e.g. SSRN, LinkedIn, Twitter, Dataverse, Github, Facebook, citation management system bookmarks 
• Downloads of full-text articles, software etc. e.g. through a public policy think tank
• Comments/cites/mentions on blogs, online forums, podcasts etc.
• Faculty websites, e.g. university websites
• Public Policy documents, e.g. Center for Disease Control

Measurements:

• Mentions/ Shares:  # times research is mentioned in media or social media 
• Views/ Downloads: # times an article, dataset, presentation, etc., has been accessed from a given website or database
• Ratings/ Reviews: personal or professional evaluations of  research output reported through traditional or social media (e.g. book reviews in journals, product ratings on Amazon, etc.)