Remember Me
Or use your Academic/Social account:


Or use your Academic/Social account:


You have just completed your registration at OpenAire.

Before you can login to the site, you will need to activate your account. An e-mail will be sent to you with the proper instructions.


Please note that this site is currently undergoing Beta testing.
Any new content you create is not guaranteed to be present to the final version of the site upon release.

Thank you for your patience,
OpenAire Dev Team.

Close This Message


Verify Password:
Verify E-mail:
*All Fields Are Required.
Please Verify You Are Human:
fbtwitterlinkedinvimeoflicker grey 14rssslideshare1
Elwyn, G.; O'Connor, A.; Stacey, D.; Volk, R.; Edwards, A.; Coulter, A.; Thomson, R.; Barratt, A.; Barry, M.; Bernstein, S.; Butow, P.; Clarke, A.; Entwistle, V.A.; Feldman-Stewart, D.; Holmes-Rovner, M.; Llewellyn-Thomas, H.; Moumjid, N.; Mulley, A.; Ruland, C.; Sepucha, K.; Sykes, A.; Whelan, T. (2006)
Publisher: BMJ Publishing Group Ltd.
Types: Research
Subjects: Research
OBJECTIVE: To develop a set of quality criteria for patient decision support technologies (decision aids). DESIGN AND SETTING: Two stage web based Delphi process using online rating process to enable international collaboration. PARTICIPANTS: Individuals from four stakeholder groups (researchers, practitioners, patients, policy makers) representing 14 countries reviewed evidence summaries and rated the importance of 80 criteria in 12 quality domains on a 1 to 9 scale. Second round participants received feedback from the first round and repeated their assessment of the 80 criteria plus three new ones. MAIN OUTCOME MEASURE: Aggregate ratings for each criterion calculated using medians weighted to compensate for different numbers in stakeholder groups; criteria rated between 7 and 9 were retained. RESULTS: 212 nominated people were invited to participate. Of those invited, 122 participated in the first round (77 researchers, 21 patients, 10 practitioners, 14 policy makers); 104/122 (85%) participated in the second round. 74 of 83 criteria were retained in the following domains: systematic development process (9/9 criteria); providing information about options (13/13); presenting probabilities (11/13); clarifying and expressing values (3/3); using patient stories (2/5); guiding/coaching (3/5); disclosing conflicts of interest (5/5); providing internet access (6/6); balanced presentation of options (3/3); using plain language (4/6); basing information on up to date evidence (7/7); and establishing effectiveness (8/8). CONCLUSIONS: Criteria were given the highest ratings where evidence existed, and these were retained. Gaps in research were highlighted. Developers, users, and purchasers of patient decision aids now have a checklist for appraising quality. An instrument for measuring quality of decision aids is being developed.
  • The results below are discovered through our pilot algorithms. Let us know how we are doing!

    • 1 O'Connor AM, Llewellyn-Thomas HA, Flood AB. Modifying unwarranted variations in health care: shared decision making using patient decision aids. Health Aff (Millwood) 2004;suppl web exclusive:VAR63-72.
    • 2 O'Connor AM, Fiset V, Degrasse C, Graham I, Evans W, Stacey D, et al. Decision aids for patients considering health care options: evidence of efficacy and policy implications. J Natl Cancer Inst 1999;25(monograph):67-80.
    • 3 Barratt A, Trevena L, Davey HM, McCaffery K. Use of decision aids to support informed choices about screening. BMJ 2004;329:507-10.
    • 4 Woolf SH, Chan EC, Harris R, Sheridan SL, Braddock CH, Kaplan RM, et al. Promoting informed choice: transforming health care to dispense knowledge for decision making. Ann Intern Med 2005;143:293-300.
    • 5 O'Connor AM, Stacey D, Entwistle V, Llewellyn-Thomas H, Rovner D, Holmes-Rovner M, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev 2003;(1):CD001431.
    • 6 Molenaar S, Sprangers MA, Postma-Schuit FC, Rutgers EJ, Noorlander J, Hendricks J, et al. Feasibility and effects of decision aids. Med Decis Making 2000;20:112-27.
    • 7 Estabrooks C, Goel V, Thiel E, Pinfold P, Sawka C, Williams I. Decision aids: are they worth it? A systematic review. J Health Serv Res Policy 2001;6:170-82.
    • 8 Entwistle V. The potential contribution of decision aids to screening programmes. Health Expect 2001;4:109-15.
    • 9 Whelan T, Levine M, Willan A, Gafni A, Sanders K, Mirsky D, et al. Effect of a decision aid on knowledge and treatment decision making for breast cancer surgery: a randomized trial. JAMA 2004;292:435-41.
    • 10 Kennedy AD, Sculpher MJ, Coulter A, Dwyer N, Rees M, Abrams KR, et al. Effects of decision aids for menorrhagia on treatment choices, health outcomes, and costs. JAMA 2002;288:2701-8.
    • 11 Evans R, Elwyn G, Edwards A. Making interactive decision support for patients a reality. Inform Prim Care 2004;12:109-13.
    • 12 Charles C, Gafni A, Whelan T, O'Brien MA. Treatment decision aids: conceptual issues and future directions. Health Expect 2005;8:114-25.
    • 13 Sepucha KR, Fowler FJ, Mulley AG. Policy support for patient-centered care: the need for measurable improvements in decision quality. Health Aff (Millwood) 2004;suppl web exclusive:VAR54-62.
    • 14 Altman DG. Better reporting of randomized controlled trials: the CONSORT statement. BMJ 1996;313:570-1.
    • 15 Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomized controlled trials: the QUOROM statement. Lancet 1999;354:1896-900.
    • 16 Cluzeau FA, Burgers JS, Brouwers M, Grol R, Mäkelä M, Littlejohns P, et al. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care 2003;12:18-23.
    • 17 Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 1999;53:105-11.
    • 18 Coulter A, Entwistle V, Gilbert D. Informing patients: an assessment of the quality of patient information materials. London: King's Fund, 1998.
    • 19 Gagliardi A, Jadad AR. Examination of instruments used to rate quality of health information on the internet: chronicle of a voyage with an unclear destination. BMJ 2002;324:569-72.
    • 20 Heaney DJ, Walker JJ, Howie JG, Maxwell M, Freeman GK, Berrey PN, et al. The development of a routine NHS data-based index of performance in general practice (NHSPPI). Fam Pract 2002;19:77-84.
    • 21 Campbell SM, Cantrill JA, Roberts D. Prescribing indicators for UK general practice: Delphi consultation study. BMJ 2000;321:425-8.
    • 22 Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CF, Askham J, et al. Consensus development methods, and their use in clinical guideline development. Health Technol Assess 1998;2(3):1-88.
    • 23 Jones J, Hunter D. Consensus methods for medical and health services research. BMJ 1995;311:376-80.
    • 24 Jones J, Hunter D. Using the Delphi and nominal group technique in health services research. In: Mays N, Pope C, eds. Qualitative research in health care. London: BMJ Books, 1999.
    • 25 Engels Y, Campbell S, Dautzenberg M, van den Hombergh P, Brinkmann H, Szécsényi J, et al. Developing a framework of, and quality indicators for, general practice management in Europe. Fam Pract 2005;22:215-22.
    • 26 Fitch K, Lázaro P, Aguilar MD, Kahan J, van het Loo M, Bernstein SJ. European criteria for the appropriateness and necessity of coronary revascularization procedures. Eur J Cardiothorac Surg 2000;18:380-7.
    • 27 Brook RH. Appropriateness: the next frontier. BMJ 1994;308:217-8.
    • 28 International Patient Decision Aid Standards (IPDAS) Collaboration. Background evidence reports from expert panels for 12 quality domains. http://ipdas.ohri.ca/ (accessed Jan 2006).
    • 29 Conover WJ. Practical nonparametric statistics. 2nd ed. New York: John Wiley & Sons, 1980.
    • 30 Ubel PA, Loewenstein G. The role of decision analysis in informed consent: choosing between intuition and systematicity. Soc Sci Med 1997;44:647-56.
    • 31 Kahnemann D. Experienced utility and objective happiness: a moment-based approach. In: Kahnemann D, Tversky A, eds. Choices, values and frames. New York: Cambridge University Press and Russell Sage Foundation, 2000:673-92.
    • 32 Schunemann HJ, Best D, Vist G, Oxman AD, Grade Working Group. Letters, numbers, symbols and words: how to communicate grades of evidence and recommendations. CMAJ 2003;169:677-80.
    • (Accepted 13 July 2006) Department of General Practice, Centre for Health Sciences Research, Cardiff University, Cardiff CF14 4YS Glyn Elwyn research professor, primary medical care Adrian Edwards research professor, primary medical care University of Ottawa and Ottawa Health Research Institute, Clinical Epidemiology Program, Ottawa, ON, Canada K1Y 4E9 Annette O'Connor professor School of Nursing, University of Ottawa, Ottawa, ON, Canada K1H 8M5 Dawn Stacey assistant professor Department of Family and Community Medicine, Baylor College of Medicine, Houston, TX 77098-3915, USA Robert Volk associate professor Picker Institute Europe, King's Mead House, Oxford OX1 1RX Angela Coulter chief executive Newcastle upon Tyne Medical School, School of Population and Health Sciences, Newcastle upon Tyne NE2 4HH
  • No related research data.
  • Discovered through pilot similarity algorithms. Send us your feedback.

Share - Bookmark

Cite this article

Collected from