Table 1 Risk of bias for scientifically admissible reliability studies based on the modified QAREL criteria
| Author, Year | Study objective clearly described | Representative sample | Representative raters | Blinded to others’ findings | Blinded to own findings | Blinded to results of ref std | Blinded to clinical information | Blinded to additional clues | Order of exam varied | Appropriate time interval | Test applied correctly | Appropriate statistical measures |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Alyazedi et al. 2015 [19] | Y | Y | Y | Y | N/A | N/A | Y | Y | UC | Y | Y | Y |
| Arab et al., 2009 [20] | Y | Y | Y | Y | N | N/A | Y | Y | Y | Y | Y | Y |
| Downey et al., 2003 [21] | Y | Y | Y | Y | N/A | N/A | UC | UC | Y | Y | Y | Y |
| Hebert et al., 2015 [22] | Y | Y | Y | Y | N/A | Y | UC | UC | Y | Y | Y | Y |
| Hicks et al. 2003 [23] | Y | Y | Y | Y | N/A | N/A | UC | UC | UC | Y | Y | Y |
|
Jensen et al. 2013 [24] | Y | Y | Y | Y | N | N/A | UC | UC | Y | Y | Y | Y |
| Ravenna et al., 2011 [25] | Y | Y | Y | Y | N/A | N/A | N | UC | Y | Y | Y | Y |
| Schneider et al., 2008 [26] | Y | Y | Y | Y | N/A | N/A | Y | Y | Y | Y | Y | Y |
| Tong et al., 2006 [27] | Y | Y | Y | Y | N/A | N/A | N | UC | UC | UC | Y | Y |
| Walsh et al., 2009 [28] | Y | Y | Y | Y | N/A | Y | N | UC | N | Y | Y | Y |
| Weiner et al., 2006 [29] | Y | Y | Y | Y | N/A | UC | Y | UC | UC | Y | Y | Y |