| 
  • If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!

View
 

FrontPage

Page history last edited by Andrew Booth 9 years, 4 months ago

 

The QuaRT Wiki

 

Welcome to the Quality of Reporting Tool (QuaRT) Wiki

This wiki has been compiled as a single "one-stop shop" for information on the Quality of Reporting Tool (QuaRT) - a tool for quality assessment of research studies for potential inclusion in a qualitative evidence synthesis (qualitative systematic review). The QuaRT Tool was developed by Dr Christopher ("Chris") Carroll and Dr Andrew Booth at the School of Health & Related Research (ScHARR), University of Sheffield.  

 

Why QuaRT?

Although numerous tools for quality assessment exist (e.g. the CASP and QARI tools) the role of quality assessment in qualitative evidence synthesis remains contested. Some researchers resist the idea of quality assessment at all for qualitative research while many contest the particular approach to be used, the optimal tool or the individual criteria to be included. Even when the role of quality assessment has been accepted debate continues regarding whether the assessments should be used to exclude studies or simply to moderate the strength of recommendations according to study quality.

 

The co-developers of the QuART tool, Drs Carroll & Booth have attempted to advance the methodologies of quality assessment, and hence the methodological debate, by devising a pragmatic tool that is underpinned by the following assumptions:

  1. Referees, reviewers and commissioners of qualitative evidence syntheses will expect to see details of an approach to quality assessment
  2. Approaches to quality assessment may attempt to evaluate the design and conduct of a study but are limited to assessing only the report of the study, therefore a tool that explicitly restricts itself to assessing the quality of reporting is less vulnerable to questionable assumptions about the study design and conduct.
  3. The process of exploring a study report in a structured way is, in itself, valuable and fulfills criteria for being a "systematic approach" 
  4. Identifying which aspects of study reporting are addressed by which portions of a published article is a fairly accessible technical process, particularly when compared with judgements of appropriateness or epistemological "fit" required by other quality assessment tools. 
  5. Judgements about a study report should be supported by auditable text extracted from the Methods, or Study Limitations, sections of that study report. This enables experienced qualitative researchers to make subsequent judgements on the appropriateness of the methods, where required.
  6. Once quality assessments have been made, the resulting judgements should be used to inform the review and its recommendations. In particular qualitative sensitivity analysis could be utilised to explore the differential contributions of well-reported and poorly reported studies (Carroll, Booth, & Lloyd-Jones, 2012).  

 

Why the QuaRT wiki?

As co-developers of the QuaRT tool we are very well aware that much of the work to date has been performed either by ourselves or by reviews to which we have made an influential contribution. The pragmatic origins of the tool, fired in the crucible of a working Health Technology Assessment (HTA) report mean that we have not to date been able to undertake a systematic programme of methodological development, evaluation, publication and dissemination. For example it is only recently that we have generated the name "QuaRT" as a pragmatic response to identify and label the tool in the face of increasing research community interest. To partly compensate for this real world approach, and to cumulate collective experience with the QuaRT tool, both from ourselves and others we have assembled all available work to date on a single wiki for your convenience.

 

Methodological Note: Although the QuaRT tool can be used independently of any particular method of qualitative synthesis it has been most typically utilised alongside examples of "Best Fit" Framework Synthesis as both share similar pragmatic origins. In turn "Best Fit" Framework Synthesis may be performed in conjunction with any recognised tool for quality assessment, not simply the QuaRT tool. However not all quality assessment approaches will be so completely aligned with their subsequent synthesis method - a further advantage of a combined QuaRT/"Best Fit" Synthesis approach.  

 

For further information about QuaRT, or if you have used or plan to use QuaRT in your own qualitative systematic review, contact either Dr Christopher ("Chris") Carroll (c.carroll@sheffield.ac.uk) or Dr Andrew Booth (a.booth@sheffield.ac.uk)

 

A. The QuaRT tool

 

Carroll C, Booth A, Cooper K. A worked example of "best fit" framework synthesis: A systematic review of views concerning the taking of some potential chemopreventive agents, BMC Medical Research Methodology, 2011 11:29

http://www.biomedcentral.com/1471-2288/11/29 The tool itself is available at: http://www.biomedcentral.com/content/supplementary/1471-2288-11-29-s1.doc 

 

 

Table - An Overview of the four Dimensions in the QuaRT Tool

Criteria categorization and definition

Criteria categorization and definition

Question and study design:

Yes - if e.g., “a case study approach was used because . . .”, “interviews were used because . . .”

No - If paper does not specify question and study design

Unclear - if unsure

Method of data collection:

Yes - If details of the data collection method are given e.g., piloting; topic guides for interviews; number of items in a survey; use of open or closed items; validation, and so forth.

No - If only states “focus group”, “interviews were used” or “questionnaire was used”

Unclear - if unsure

Selection of participants:

Yes - If the selection of participants is described in full or explicitly as e.g., purposive, convenience, theoretical and so forth.

No - If only details of participants are given, e.g. age, gender, number

Unclear - if unsure

Methods of data analysis:

Yes - If full details of analysis method are given, e.g., transcription and form of analysis (with reference or full description of method), validation tests, and so forth.

No - If only states “content analysis” or that “data were analyzed”

Unclear - if unsure

B. The QuaRT worksheet

Worksheet pdf

Worksheet docx

 

C. First use of QuaRT in a published health technology assessment

Cooper K, Squires H, Carroll C et al. Chemoprevention of colorectal cancer: systematic review and economic model. Health Technology Assessment 2010; 14 (32). http://www.journalslibrary.nihr.ac.uk/hta/volume-14/issue-32 

 

D. Use of QuaRT in qualitative sensitivity analysis

Carroll, C., Booth, A., & Lloyd-Jones, M. (2012). Should we exclude inadequately reported studies from qualitative systematic reviews? An evaluation of sensitivity analyses in two case study reviews. Qualitative Health Research22(10), 1425-1434.

 

E. Other published examples of the use of QuaRT 

Carroll C, Lloyd-Jones M, Cooke J, Owen J. Reasons for the use and non-use of school sexual health services: a systematic review of young people's views. J Public Health (Oxf). 2012 Aug;34(3):403-10. doi: 10.1093/pubmed/fdr103. Epub 2011 Dec 18.

Carroll, C., Booth, A., Leaviss, J., & Rick, J. (2013). “Best fit” framework synthesis: refining the method. BMC Medical Research Methodology13(1), 37. http://www.biomedcentral.com/1471-2288/13/37 

Carroll, C., Rick, J., Leaviss, J., Fishwick, D., & Booth, A. (2013). A qualitative evidence synthesis of employees' views of workplace smoking reduction or cessation interventions. BMC Public Health13(1), 1095. http://www.biomedcentral.com/1471-2458/13/1095 

Fishwick, D., Carroll, C., McGregor, M., Drury, M., Webster, J., Bradshaw, L., ... & Leaviss, J. (2013). Smoking cessation in the workplace. Occupational medicine63(8), 526-536.

Field, B., Booth, A., Ilott, I., & Gerrish, K. (2014). Using the Knowledge to Action Framework in practice: a citation analysis and systematic review. Implementation Science9(1), 172. http://www.implementationscience.com/content/9/1/172/abstract 

 

F. The Co-Developers Approach to Quality Assessment

Carroll, C., & Booth, A. (2014).   Quality assessment of qualitative evidence for systematic review and synthesis: Is it meaningful, and if so, how should it be performed?. Research Synthesis Methodshttp://onlinelibrary.wiley.com/doi/10.1002/jrsm.1128/abstract  

Booth, A. (2007). Who will appraise the appraisers?—The paper, the instrument and the user. Health Information & Libraries Journal24 (1):72-6. http://scharrlatan.typepad.com/Booth2007.pdf  

Dixon-Woods, M., Sutton, A., Shaw, R., Miller, T., Smith, J., Young, B., Booth, A., & Jones, D. (2007). Appraising qualitative research for inclusion in systematic reviews: a quantitative and qualitative comparison of three methods. Journal of health services research & policy12(1), 42-47.

 

G. Work in progress using QuaRT 

Contextual Enablers and Barriers to the Implementation of home-based Palliative Care Interventions  Lisa Pfadenhauer, Louise Brereton, Jacob Burns, Ansgar Gerhardus, Kati Mozygemba, Kristin Bakke Lysdahl, Stephanie Polus, Eva Rehfuess, Gert Jan van der Wilt, Andrew Booth)

 

Comments (0)

You don't have permission to comment on this page.