Remember Me
Or use your Academic/Social account:


Or use your Academic/Social account:


You have just completed your registration at OpenAire.

Before you can login to the site, you will need to activate your account. An e-mail will be sent to you with the proper instructions.


Please note that this site is currently undergoing Beta testing.
Any new content you create is not guaranteed to be present to the final version of the site upon release.

Thank you for your patience,
OpenAire Dev Team.

Close This Message


Verify Password:
Verify E-mail:
*All Fields Are Required.
Please Verify You Are Human:
fbtwitterlinkedinvimeoflicker grey 14rssslideshare1
Read, Jonathon; Velldal, Erik; Cavazza, Marc; Georg, Gersende (2016)
Publisher: European Language Resources Association
Languages: English
Types: Unknown
Subjects: QA76.76.E95
In this paper we present the Corpus of REcommendation STrength (CREST), a collection of HTML-formatted clinical guidelines\ud annotated with the location of recommendations. Recommendations are labelled with an author-provided indicator of their strength of importance. As data was drawn from many disparate authors, we define a unified scheme of importance labels, and provide a mapping for each guideline.\ud We demonstrate the utility of the corpus and its annotations in some initial measurements investigating the type of language constructions associated with strong and weak recommendations, and experiments into promising features for recommendation classification, both with respect to strong and weak labels, and to all labels of the unified scheme. An error analysis indicates that, while there is a strong relationship between lexical choices and strength labels, there can be substantial variance in the choices made by different authors.
  • The results below are discovered through our pilot algorithms. Let us know how we are doing!

    • Blanco, E., Morante, R., and Sporleder, C. (2015). Proceedings of the Second Workshop on Extra-Propositional Aspects of Meaning in Computational Linguistics (ExProM 2015).
    • Bouma, G. (2009). Normalized (pointwise) mutual information in collocation extraction. In Proceedings of the Biennial GSCL Conference 2009, pages 31-40, Tu¨bingen, Germany.
    • Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20:37-46.
    • Crammer, K. and Yoram, S. (2001). On the algorithmic implementation of multiclass kernel-based vector machines. Journal of Machine Learning Research, 2:265- 292.
    • Field, M. J. and Lohr, K. N. (1990). Clinical Practice Guidelines: Directions for a New Program. National Academies Press.
    • Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5):378- 382.
    • Georg, G. and Jaulent, M.-C. (2007). A document engineering environment for clinical guidelines. In Proceedings of the 2007 ACM Symposium on Document Engineering, pages 69-78.
    • Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., FalckYtter, Y., Alonso-Coello, P., and Schu¨nermann, H. J. (2008). GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ, 336:924-926.
    • Hussain, T., Michel, G., and Shiffman, R. N. (2009). The Yale Guideline Recommendation Corpus: A representative sample of the knowledge content of guidelines. International Journal of Medical Informatics, 78:354-363.
    • Joachims, T. (1999). Making large-scale SVM learning practical. In B Scho¨lkopf, et al., editors, Advanced in Kernel Methods-Support Vector Learning. MIT Press.
    • Lomotan, E. A., Michel, G., Lin, Z., and Shiffman, R. N. (2009). How “should” we write guideline recommendations? Interpretation of deontic terminology in clinical practice guidelines: Survey of the health services community. Quality & Safety in Healthcare, 19:509-513.
    • Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S. J., and McClosky, D. (2014). The Stanford CoreNLP natural language processing toolkit. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 55-60.
    • Morante, R. and Sporleder, C. (2012). Proceedings of the Workshop on Extra-Propositional Aspects of Meaning in Computational Linguistics (ExProM 2012).
    • Read, J., Velldal, E., and Øvrelid, L. (2012). Topic classification for suicidology. Journal of Computing Science and Engineering, 6:143-150.
    • Shaneyfelt, T. M. and Centor, R. M. (2009). Reassessment of clinical practice guidelines: Go gently into that good night. JAMA, 301(8):868-869.
    • Shrier, I., Boivin, J.-F., Platt, R. W., Steele, R. J., Brophy, J. M., Carnevale, F., Eisenberg, M. J., Furlan, A., Kakuma, R., Macdonald, M. E., Pilote, L., and Rissignol, M. (2008). The interpretation of systematic reviews with meta-analyses: An objective or subjective process? BMC Medical Informatics and Decision Making, 8(1):19.
    • Turney, P. D. (2002). Thumbs up or thumbs down?: Semantic orientation applied to unsupervised classification of reviews. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 417-424.
    • U.S. Preventive Services Task Force. (2014). Grade definitions. http://www.uspreventive servicestaskforce.org/Page/Name/grade -definitions, October. Accessed 2016-03-01.
    • Velldal, E., Øvrelid, L., Read, J., and Oepen, S. (2012). Speculation and negation: Rules, rankers and the role of syntax. Computational Linguistics, 38(2):368-410.
    • Zhang, F. and Xu, K. (2015). Annotation and classification of an email importance corpus. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 651-656.
  • No related research data.
  • No similar publications.

Share - Bookmark

Download from

Funded by projects

  • EC | MUSE

Related to

  • fet-fp7FET Open: Challenging current thinking
  • fet-fp7FET Open: Machine Understanding for interactive StorytElling

Cite this article