Scholarly Research Primer

Conducted AFTER the review (literature review, systematic review etc.) to assess if the studies used have methodological flaws that skew the study conclusions and, therefore, the judgment.

• NOT conducted at the abstract/ full-text consideration of scholarly/credible/peer-review content for a project. Even credible/scholarly/peer-review content may have risk-of-bias embedded in the study.
• Measures the RISK of bias, NOT the existence of bias.
• The measurement is done by asking methodological questions of the articles and assigning them a domain. Each domain is ranked with a number, stars, colors etc. Each piece is assigned an overall risk level after adding up the domain rankings.
• It is different than critical appraisal tools such as PICOT. Those tools examine the effectiveness of clinical interventions in evidence-based practice.
• Sometimes, the terms critical appraisal and risk-of-bias are treated as synonyms.  You can discern what is intended by attention to the outcome stated, such as evidence-based medicine (critical appraisal) or methodological assessment (risk of bias). 

Results are often represented in graph form, like the "traffic light" graph below. The color represents the reviewer's conclusion about the risk of each type of bias in each study. 

From: https://learning.edanz.com/what-is-risk-of-bias-in-a-systematic-review/