Remember Me
Or use your Academic/Social account:


Or use your Academic/Social account:


You have just completed your registration at OpenAire.

Before you can login to the site, you will need to activate your account. An e-mail will be sent to you with the proper instructions.


Please note that this site is currently undergoing Beta testing.
Any new content you create is not guaranteed to be present to the final version of the site upon release.

Thank you for your patience,
OpenAire Dev Team.

Close This Message


Verify Password:
Verify E-mail:
*All Fields Are Required.
Please Verify You Are Human:
fbtwitterlinkedinvimeoflicker grey 14rssslideshare1
Cinelli, Marco; Coles, Stuart R.; Kirwan, Kerry (2014)
Publisher: Elsevier BV
Journal: Ecological Indicators
Languages: English
Types: Article
Subjects: QA, Ecology, Decision Sciences(all), Ecology, Evolution, Behavior and Systematics, T1
Sustainability assessments require the management of a wide variety of information types, parameters and uncertainties. Multi criteria decision analysis (MCDA) has been regarded as a suitable set of methods to perform sustainability evaluations as a result of its flexibility and the possibility of facilitating the dialogue between stakeholders, analysts and scientists. However, it has been reported that researchers do not usually properly define the reasons for choosing a certain MCDA method instead of another. Familiarity and affinity with a certain approach seem to be the drivers for the choice of a certain procedure. This review paper presents the performance of five MCDA methods (i.e. MAUT, AHP, PROMETHEE, ELECTRE and DRSA) in respect to ten crucial criteria that sustainability assessments tools should satisfy, among which are a life cycle perspective, thresholds and uncertainty management, software support and ease of use. The review shows that MAUT and AHP are fairly simple to understand and have good software support, but they are cognitively demanding for the decision makers, and can only embrace a weak sustainability perspective as trade-offs are the norm. Mixed information and uncertainty can be managed by all the methods, while robust results can only be obtained with MAUT. ELECTRE, PROMETHEE and DRSA are non-compensatory approaches which consent to use a strong sustainability concept, accept a variety of thresholds, but suffer from rank reversal. DRSA is less demanding in terms of preference elicitation, is very easy to understand and provides a straightforward set of decision rules expressed in the form of elementary “if … then …” conditions. Dedicated software is available for all the approaches with a medium to wide range of results capability representation. DRSA emerges as the easiest method, followed by AHP, PROMETHEE and MAUT, while ELECTRE is regarded as fairly difficult. Overall, the analysis has shown that most of the requirements are satisfied by the MCDA methods (although to different extents) with the exclusion of management of mixed data types and adoption of life cycle perspective which are covered by all the considered approaches.
  • The results below are discovered through our pilot algorithms. Let us know how we are doing!

    • Akadiri, P.O., Olomolaiye, P.O., 2012. Development of sustainable assessment criteria for building materials selection. Eng. Constr. Archit. Manag. 19, 666-687.
    • Antunes, P., R. Santos, N. Videira, F. Colaco, R. Szanto, E. R. Dobos, S. JKovacs, and A. Vari. 2012. Approaches to integration in sustainability assessment of technologies. PROSUITE Project. http://prosuite.org/c/document_library/get_- file?uuid=c378cd69-f785-40f2-b23e-ae676b939212&groupId=12772, (accessed on February 4, 2014).
    • Augusto, M., Figueira, J., Lisboa, J., Yasin, M., 2005. An application of a multi-criteria approach to assessing the performance of Portugal's economic sectors: methodology, analysis and implications. Eur. Bus. Rev. 17, 113-132.
    • Behzadian, M., Kazemzadeh, R.B., Albadvi, A., Aghdasi, M., 2010. PROMETHEE: a comprehensive literature review on methodologies and applications. Eur. J. Oper. Res. 200, 198-215.
    • Belton, V., Stewart, T.J., 2002. Multiple Criteria Decision Analysis: an Integrated Approach. Kluwer Academic Publisher.
    • Benoit, V., Rousseaux, P., 2003. Aid for aggregating the impacts in life cycle assessment. Int. J. Life Cycle Assess. 8, 74-82.
    • Błaszczynski, J., Greco, S., Matarazzo, B., Słowinski, R., Szelag, M., 2013. jMAF - Dominance-based rough set data analysis framework. In: Skowron, A., Skowron, A., Suraj, Z., Suraj, Z. (Eds.), Rough Sets and Intelligent Systems - Professor Zdzisław Pawlak in Memoriam. Springer, Berlin, Heidelberg, pp. 185-209.
    • Bockstaller, C., Guichard, L., Makowski, D., Aveline, A., Girardin, P., Plantureux, S., 2008. Agri-environmental indicators to assess cropping and farming systems. A review. Agron. Sustain. Dev. 28, 139-149.
    • Bockstaller, C., Guichard, L., Keichinger, O., Girardin, P., Galan, M.-B., Gaillard, G., 2009. Comparison of methods to assess the sustainability of agricultural systems. A review. Agron. Sustain. Dev. 29, 223-235.
    • Bond, A., Morrison-Saunders, A., Pope, J., 2012. Sustainability assessment: the state of the art. Impact Assess. Proj. Appraisal 30, 53-62.
    • Brans, J.P., Mareschal, B., 2005. PROMETHEE methods. In: Figueira, J., Figueira, J., Greco, S., Greco, S., Ehrgott, M., Ehrgott, M. (Eds.), Multi Criteria Decision Analysis: State of the Art Surveys. Springer, New York.
    • Brans, J.P., Vincke, P., Mareschal, B., 1986. How to select and how to rank projects. The PROMETHEE method. Eur. J. Oper. Res. 24, 228-238.
    • Buchholz, T., Rametsteiner, E., Volk, T.A., Luzadis, V.A., 2009. Multi criteria analysis for bioenergy systems assessments. Energy Policy 37, 484-495.
    • Cinelli, M., Coles, R.S., Kirwan, K., 2013a. Use of Multi Criteria Decision Analysis to Support Life Cycle Sustainability Assessment: An Analysis of the Appropriateness of the Available Methods. The 6th International Conference on Life Cycle Management, 25-28 August, 2013, Gothenburg, Sweden. , pp. 677-680.
    • Cinelli, M., Coles, S.R., Jørgensen, A., Zamagni, A., Fernando, C., Kirwan, K., 2013b. Workshop on life cycle sustainability assessment: the state of the art and research needs - November 26, Copenhagen, Denmark. Int. J. Life Cycle Assess. 18, 1421-1424.
    • Convertino, M., Baker, K.M., Vogel, J.T., Lu, C., Suedel, B., Linkov, I., 2013. Multicriteria decision analysis to select metrics for design and monitoring of sustainable ecosystem restorations. Ecol. Indic. 26, 76-86.
    • Danielson, M., Ekenberg, L., Idefeldt, J., Larsson, A., 2007. Using a software tool for public decision analysis: the case of Nacka municipality. Decis. Anal. 4, 76-90.
    • De Keyser, W., Peeters, P., 1996. A note on the use of PROMETHEE multicriteria methods. Eur. J. Oper. Res. 89, 457-461.
    • De Montis, A., De Toro, P., Droste-Franke, B., Omann, I., Stagl, S., 2000. Criteria for quality assessment of MCDA methods. 3rd Biennial Conference of the European Society for Ecological Economics, Vienna.
    • De Montis, A., De Toro, P., Droste-Franke, B., Omann, I., Stagl, S., 2005. Assessing the quality of different MCDA methods. In: Getzner, M., Getzner, M., Spash, C., Spash, C., Stagl, S., Stagl, S. (Eds.), Alternatives for Environmental Valuation. Routledge, Abingdon, pp. 99-133.
    • De Ridder, W., Turnpenny, J., Nilsson, M., Von Raggamby, A., 2007. A framework for tool selection and use in integrated assessment for sustainable development. J. Environ. Assess. Policy Manage. 9, 423-441.
    • De Smet, Y. 2014. D-SIGHT. http://www.d-sight.com/, (accessed on February 6, 2014).
    • Dembczynski, K., Greco, S., Słowinski, R., 2009. Rough set approach to multiple criteria classification with imprecise evaluations and assignments. Eur. J. Oper. Res. 198, 626-636.
    • Dyer, J.S., 1990. Remarks on the Analytic Hierarchy Process. Manage. Sci. 36, 249- 258.
    • Dyer, J.S., 2005. MAUT - Multiattribute Utility Theory. Springer New York. Multiple Criteria Decision Analysis: State of the Art Surveys. Springer, New York, pp. 265- 292.
    • EPA, U.S., 2006. Life Cycle Assessment: Principles and Practice. National Risk Management Research Laboratory, Cincinnati, OH, USA.
    • Fernandez, A., 1996. Software review: expert choice. OM/MS Today 23, 80-83.
    • Fiigueira, J., Greco, S., Ehrgott, M., 2005a. Multi Criteria Decision Analysis: State of the Art Surveys. Springer, New York.
    • Fiigueira, J., Mousseau, V., Roy, B., 2005b. ELECTRE methods. In: Fiigueira, J., Fiigueira, J., Greco, S., Greco, S., Ehrgott, M., Ehrgott, M. (Eds.), Multi Criteria Decision Analysis: State of the Art Surveys. Springer, New York, pp. 133-162.
    • Fiigueira, J., Roy, B., 2009. A note on the paper, “Ranking irregularities when evaluating alternatives by using some ELECTRE methods”, by Wang and Triantaphyllou, Omega. (2008). Omega 37, 731-733.
    • Fiigueira, J.R., Greco, S., Roy, B., Słowinski, R., 2013. An overview of ELECTRE methods and their recent extensions. J. Multi-Criteria Decis. Anal. 20, 61-85.
    • Gasparatos, A., El-Haram, M., Horner, M., 2008. A critical review of reductionist approaches for assessing the progress towards sustainability. Environ. Impact Assess. Rev 28, 286-311.
    • Gasparatos, A., Scolobig, A., 2012. Choosing the most appropriate sustainability assessment tool. Ecol. Econ. 80, 1-7.
    • Geldermann, J., Zhang, K., 2001. Software review: “Decision Lab 2000”. J. MultiCriteria Decis. Anal. 10, 317-323.
    • Gibson, R.B., 2006. Sustainability assessment: basic components of a practical approach. Impact Assess. Project Appraisal 24, 170-182.
    • Greco, S., Matarazzo, B., Slowinski, R., 1997. Rough approximation of a preferential information. Poznan University Technol. .
    • Greco, S., Matarazzo, B., Slowinski, R., 1998. A new rough set approach to evaluation of bankruptcy risk. In: Zopounidis, C., Zopounidis, C. (Eds.), Operational Tools in the Management of Financial Risks. Springer, US, pp. 121-136.
    • Greco, S., Matarazzo, B., Slowinski, R., 1999. The use of rough sets and fuzzy sets in MCDM. In: Gal Stewart, T.T., Gal Stewart, T.T., Hanne, T., Hanne, T. (Eds.), Multicriteria Decision Making. Springer, US, pp. 397-455.
    • Greco, S., Matarazzo, B., Slowinski, R., 2001a. Rough set approach to decisions under risk.. In: Ziarko, W., Ziarko, W., Yao, Y., Yao, Y. (Eds.), Rough Sets and Current Trends in Computing. Springer, Heidelberg, Berlin, pp. 160-169.
    • Greco, S., Matarazzo, B., Slowinski, R., 2001b. Rough sets theory for multicriteria decision analysis. Eur. J. Oper. Res. 129, 1-47.
    • Greco, S., Matarazzo, B., Slowinski, R., 2005. Decision rule approach. In: Fiigueira, J., Fiigueira, J., Greco, S., Greco, S., Ehrgott, M., Ehrgott, M. (Eds.), Multi Criteria Decision Analysis: State of the Art Surveys. Springer, New York, pp. 507-555.
    • Greco, S., Matarazzo, B., Słowinski, R., 2004. Axiomatic characterization of a general utility function and its particular cases in terms of conjoint measurement and rough-set decision rules. Eur. J. Oper. Res. 158, 271-292.
    • Haerer, W. 2000. Software review: Criterium Decision Plus 3.0. OR/MS Today 27.
    • Herva, M., Roca, E., 2013. Review of combined approaches and multi-criteria analysis for corporate environmental evaluation. J. Clean.Prod. 39, 355-371.
    • Hokkanen, J., Salminen, P., 1997. Locating a waste treatment facility bymulticriteria analysis. J. Multi-Criteria Decis. Anal. 6, 175-184.
    • Huang, I.B., Keisler, J., Linkov, I., 2011. Multi-criteria decision analysis in environmental sciences: ten years of applications and trends. Sci. Total Environ. 409, 3578-3594.
    • InfoHarvest. 2014. Criterium Decision Plus 3.0. http://www.infoharvest.com/ihroot/ infoharv/products.asp#CDP30, (accessed on February 6, 2014).
    • Keeney, L.R., 1974. Multiplicative utility functions. Oper. Res. 22, 22-34.
    • Keeney, L.R., Raiffa, H., 1976. Decisions with Multiple Objectives: Preferences and Value Tradeoffs. Wiley, New York.
    • Keeney, L.R., Raiffa, H., 1993. Decisions with Multiple Objectives: Preferences and Value Tradeoffs. Cambridge University Press.
    • Keeney, L.R., Wood, E.F., 1977. An illustrative example of the use of multiattribute utility theory for water resource planning. Water Resour. Res. 13, 705-712.
    • Khalili, N.R., Duecker, S., 2013. Application of multi-criteria decision analysis in design of sustainable environmental management system framework. J. Clean. Production 47, 188-198.
    • LAMSADE. 2014. ELECTRE software. CNRS - University Paris Dauphine. http://www. lamsade.dauphine.fr/spip.php?rubrique64, (accessed on February 6, 2014).
    • Le Teno, J.F., Mareschal, B., 1998. An interval version of PROMETHEE for the comparison of building products' design with ill-defined data on environmental quality. Eur. J. Oper. Res. 109, 522-529.
    • Linkov, I., Moberg, E., 2012. Multi-criteria decision analysis. Environmental Applications and Case Studies. CRC Press, United States.
    • Mareschal, B. 2014. PROMETHEE-GAIA.net. http://www.promethee-gaia.net/, (accessed on February 6, 2014).
    • Mareschal, B., De Smet, Y., Nemery, P., 2008. Rank reversal in the PROMETHEE II method. Some new results. International Conference on Industrial Engineering and Engineering Management 2008.
    • Mendoza, G.A., Martins, H., 2006. Multi-criteria decision analysis in natural resource management: a critical review of methods and new modelling paradigms. Forest Ecol. Manage. 230, 1-22.
    • Merad, M., Dechy, N., Serir, L., Grabisch, M., Marcel, F., 2013. Using a multi-criteria decision aid methodology to implement sustainable development principles within an organization. Eur. J. Oper. Res. 224, 603-613.
    • Munda, G. 2005. Multi criteria decision analysis and sustainable development Pages 953-986 in J. Fiigueira, S. Greco, and M. Ehrgott, editors. Multiple Criteria Decision Analysis: State of the Art Surveys. Springer, New York. http:// publications.jrc.ec.europa.eu/repository/handle/111111111/13077, (accessed on February 6, 2014).
    • Munda, G., 2008. The issue of consistency: basic discrete multi-criteria “Methods”. Social Multi-Criteria Evaluation for a Sustainable Economy. Springer, Heidelberg, Berlin, pp. 85-109.
    • Munda, G., Nardo, M., 2005. Constructing consistent composite indicators: the issue of weights. Institute for the Protection and Security of the Citizen.
    • Ness, B., Urbel-Piirsalu, E., Anderberg, S., Olsson, L., 2007. Categorising tools for sustainability assessment. Ecol. Econ. 60, 498-508.
    • O'Neill, J., Martinez-Alier, J., Munda, G., 1999. Commensurability and compensability in ecological economics. In: Cheltenham, M.O.C.C.S., Cheltenham, M.O.C.C.S. (Eds.), Valuation and the Environment: Theory, Method, and Practice. Elgar.
    • Omann, I., 2004. Multi-criteria Decision Aid as an Approach for Sustainable Development Analysis and Implementation. University of Graz.
    • Polatidis, H., Haralambopoulos, D.A., Munda, G., Vreeker, R., 2006. Selecting an appropriate multi-criteria decision analysis technique for renewable energy planning. Energy Sources Part B: Econ. Planning Policy 1, 181-193.
    • Pope, J., Annandale, D., Morrison-Saunders, A., 2004. Conceptualising sustainability assessment. Environ. Impact Assess. Rev. 24, 595-616.
    • Reuters, T., 2014. Web of Science. http://thomsonreuters.com/thomson-reutersweb-of-science/.
    • Roland, J., De Smet, Y., Verly, C., 2012. Rank reversal as a source of uncertainty and manipulation in the PROMETHEE II ranking. A first investigation (2012). 14th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems, Catania, Italy, July 9-13. , pp. 338- 346.
    • Rowley, H.V., Peters, G.M., Lundie, S., Moore, S.J., 2012. Aggregating sustainability indicators: beyond the weighted sum. J. Environ. Manage. 111, 24-33.
    • Roy, B., 1991. The outranking approach and the foundations of electre methods. Theory Decision 49-73.
    • Roy, B., 1996. Multicriteria Methodology for Decision Aiding. Kluwer Academic Publishers.
    • Roy, B., Słowinski, R., 2013. Questions guiding the choice of a multicriteria decision aiding method. EURO J. Decision Processes 1, 69-97.
    • Saaty, T.L., 1980. The Analytic Hierarchy Process. McGraw-Hill, New York.
    • Saaty, T.L., 1990. An exposition on the AHP in reply to the paper “Remarks on the analytic hierarchy process”. Manage. Sci. 36, 259-268.
    • Saaty, T.L., 2005. The analytic hierarchy and analytic network processes for the measurement of intangible criteria and for decision-making. In: Fiigueira, J., Fiigueira, J., Greco, S., Greco, S., Ehrgott, M., Ehrgott, M. (Eds.), in. Springer, New York.
    • Sadok, W., Angevin, F., -É. Bergez, J., Bockstaller, C., Colomb, B., Guichard, L., Reau, R., Doré, T., 2008. Ex ante assessment of the sustainability of alternative cropping systems: implications for using multi-criteria decision-aid methods. A review. Agron. Sustain. Dev. 28, 163-174.
    • Sala, S., Farioli, F., Zamagni, A., 2013a. Life cycle sustainability assessment in the context of sustainability science progress (part 2). Int. J. Life Cycle Assess. 18, 1686-1697.
    • Sala, S., Farioli, F., Zamagni, A., 2013b. Progress in sustainability science: lessons learnt from current methodologies for sustainability assessment: Part 1. Int. J. Life Cycle Assess. 18, 1653-1672.
    • Singh, R.K., Murty, H.R., Gupta, S.K., Dikshit, A.K., 2009. An overview of sustainability assessment methodologies. Ecol. Indic. 9, 189-212.
    • Slowinski, R. 2014. Laboratory of intelligent decision support systems. Poznan University of Technology. http://idss.cs.put.poznan.pl/site/idss-en.html, (accessed on May 3, 2014).
    • Slowinski, R. and Blaszczynaki, J. 2014. JMAF software. Poznan University of Technology. http://idss.cs.put.poznan.pl/site/139.html, (accessed on May 10, 2014).
    • Slowinski, R., Greco, S., Matarazzo, B., 2002. Axiomatization utility decision-rule of utility, outranking and decision preference models for multiple-criteria classification problems under partial inconsistent with the dominance principle. Control Cybernetics 31, 1005-1035.
    • Slowinski, R., Greco, S., Matarazzo, B., 2009. Rough sets in decision making. In: Meyers, R., Meyers, R. (Eds.), Encyclopedia of Complexity and Systems Science. Springer, New York, pp. 7753-7786.
    • Slowinski, R., Greco, S., Matarazzo, B., 2012. Rough set and rule-based multicriteria decision aiding. Pesquisa Operacional 32, 213-270.
    • Slowinski, R. and M. Szelag. 2014. JRank software. Poznan University of Technology. http://idss.cs.put.poznan.pl/site/146.html, (accessed on May 12, 2014).
    • Subramanian, V., Semenzin, E., Hristozov, D., Marcomini, A., Linkov, I., 2014. Sustainable nanotechnology: defining, measuring and teaching. Nano Today .
    • Szelag, M., Slowinski, R., Greco, S., Blaszczynaki, J. and Wilk, S. 2013. jRank - Ranking using Dominance-based Rough Set Approach. Poznan University of Technology. http://www.cs.put.poznan.pl/mszelag/Software/jRank/jrank.pdf, (accessed on May 6, 2014).
    • Tatham, E., Eisenberg, D., Linkov, I., 2014. Sustainable urban systems: a review of how sustainability indicators inform decisions. In: Linkov, I., Linkov, I. (Eds.), Sustainable Cities and Military Installations. Springer, Netherlands, pp. 3-20.
    • Teghem, J., Delhaye, C., Kunsch, P.L., 1989. An interactive decision support system (IDSS) for multicriteria decision aid. Math. Comput. Model. 12, 1311-1320.
    • UN, 2001. Indicators of Sustainable Development: Guidelines and Methodologies. United Nations Department of Economic and Social Affairs, Division for Sustainable Development, New York.
    • van der Werf, H.M.G., Petit, J., 2002. Evaluation of the environmental impact of agriculture at the farm level: a comparison and analysis of 12 indicator-based methods. Agric. Ecosyst. Environ. 93, 131-145.
    • Wang, J.-J., Jing, Y.-Y., Zhang, C.-F., Zhao, J.-H., 2009. Review on multi-criteria decision analysis aid in sustainable energy decision-making. Renew. Sustain. Energy Rev. 13, 2263-2278.
    • Wang, X., Triantaphyllou, E., 2008. Ranking irregularities when evaluating alternatives by using some ELECTRE methods. Omega 36, 45-63.
    • WCED, 1987. Our common future. Oxford University Press, Oxford.
    • Weistroffer, H.R., Smith, C.H., Narula, S.C., 2005. Multi Criteria Decision Support Software.. In: Fiigueira, J., Fiigueira, J., Greco, S., Greco, S., Ehrgott, M., Ehrgott, M. (Eds.), Multi Criteria Decision Analysis: State of the Art Surveys. Springer, New York.
    • Zamagni, A., P. Buttol, R. Buonamici, P. Masoni, J. B. Guine'e, G. Huppes, R. Heijungs, E. Van der Voet, T. Ekvall, and T. Rydberg. 2009. D20 Blue Paper on Life Cycle Sustainability Analysis. Institute of Environmental Sciences, Leiden University (CML). CALCAS Project, http://www.calcasproject.net/default.asp?site=calcas&- page_id=E2669B0F-9DB7-4D1E-95B0-407BC7949030.
  • No related research data.
  • Discovered through pilot similarity algorithms. Send us your feedback.

    Title Year Similarity

    Dynamic radiostereometric analysis for evaluation of hip joint pathomechanics


    Translational control of discoidin lectin expression in drsA suppressor mutants of Dictyostelium discoideum.


    Penerapan Dominance-based Rough Set Approach dalam Pemeringkatan Kualitas Desain Software Berorientasi Objek


    Road safety in Colombia on the decade of action for road safety: preliminary results 2010-2015


    Class-based Rough Approximation with Dominance Principle


    Rough set and rule-based multicriteria decision aiding


    Spare parts classification in industrial manufacturing using the dominance-based rough set approach


    Prediction Method based DRSA to Improve the Individual Knowledge Appropriation in a Collaborative Learning Environment: Case of MOOCs


    Классификация и фитоэкологическая оценка лесопарковой растительности г. Киева


    Application of dominance-based rough set approach (DRSA) for quality prediction in a casting process


    Greenweb and the landscape. A WebGIS –DRSA valuation pattern for the programs of slow viability in Syracuse Province


    Three Dimensional Multidirectional Geographical IP Traceback: Direction Ratio Sampling Algorithm


    A Novel Direction Ratio Sampling Algorithm (DRSA) Approach for Multi Directional Geographical Traceback


    Rough set and rule-based multicriteria decision aiding


    Multi Directional Geographical IP Traceback using Direction Ratio Sampling Algorithm (DRSA)




    Beyond Markowitz with multiple criteria decision aiding


    Decision tables aggregation in rough sets approximation


    Assessing rural sustainable development potentialities using a Dominance-based Rough Set Approach


Share - Bookmark

Cite this article