Benchmarking Aggregation-Based MCDM Methods: A Comparative Analysis of Ranking Consistency, Stability, and Robustness
Multi-Criteria Decision Making (MCDM) methods are widely applied in Decision Support Systems (DSS) to address selection problems involving multiple criteria. However, differences in aggregation mechanisms may affect ranking consistency, stability, and robustness under changing decision conditions. This study presents a benchmarking analysis of five aggregation-based MCDM methods, namely Simple Additive Weighting (SAW), Weighted Aggregated Sum Product Assessment (WASPAS), Additive Ratio Assessment (ARAS), Complex Proportional Assessment (COPRAS), and Multi-Attributive Ideal-Real Comparative Analysis (MAIRCA), for employee selection. Ranking consistency is evaluated using Spearman rank correlation, while stability and robustness are assessed using the Ranking Stability Index and Ranking Retention Rate. SAW, WASPAS, ARAS, and COPRAS achieve perfect consistency with the reference ranking, with a Spearman coefficient of 1.0000, while MAIRCA obtains 0.9879. ARAS and COPRAS achieve the highest stability, with an RSI of 97.22%, and robustness, with an RRR of 86.11%. SAW, WASPAS, and MAIRCA achieve an RSI of 95.56% and an RRR of 77.78%. Ranking changes are mainly limited to an exchange between Candidates 1 and 9, while Candidate 3 remains first across all scenarios. These findings suggest that ARAS and COPRAS exhibit comparatively higher ranking stability under the experimental conditions considered in this study.
Downloads
References
[1] T. Fujita, “The Hyperfuzzy VIKOR and Hyperfuzzy DEMATEL Methods for Multi-Criteria Decision-Making,” Spectr. Decis. Mak. Appl., vol. 3, no. 1 SE-Articles, pp. 292–315, Jan. 2026, doi: 10.31181/sdmap31202654.
[2] I. Badi, M. B. Bouraima, and C. Kiprotich Kiptum, “Evaluating the Barriers to Logistics Outsourcing through a Fuzzy Multi-Criteria Decision-Making Model,” Spectr. Mech. Eng. Oper. Res., vol. 3, no. 1, pp. 65–78, Apr. 2026, doi: 10.31181/smeor31202653.
[3] J. M. L. G. Leite, L. G. Marujo, and E. F. Arruda, “Novel Time Aggregation-Based Algorithms for Markov Decision Processes,” IEEE Trans. Automat. Contr., vol. 70, no. 12, pp. 8353–8360, 2025, doi: 10.1109/TAC.2025.3583262.
[4] M. Behera and A. C. Panda, “A q-rung orthopair fuzzy Gaussian aggregation-based decision-making framework for sustainable solid waste management,” Appl. Soft Comput., vol. 199, p. 115381, 2026, doi: 10.1016/j.asoc.2026.115381.
[5] R. S. Kanwal, S. M. Qurashi, N. Kausar, and F. T. Zahra, “A Hamacher Aggregation-Based Pythagorean Fuzzy Z-Number MCDM Framework for Sustainable Urban Green Space Planning,” J. Contemp. Decis. Sci., vol. 3, no. 1, pp. 1–15, 2026, doi: 10.67334/cds31202637.
[6] T. Widodo, D. Darwis, and S. Nassor, “Hybrid Modification Preference Selection Index with Optimized Pairwise Ratio Analysis Method for Multi-Criteria Decision Making: An Integrated Weighting and Ranking Approach,” J. Decis. Support Syst. Multi-Criteria Decis. Mak., vol. 1, no. 1, pp. 84–102, 2026, doi: 10.67449/jodesma.v1i1.5.
[7] G. Patel, S. Das, and R. Das, “Evaluation of optimal normalization techniques in multi-criteria decision-making to rank CMIP6 climate models,” Theor. Appl. Climatol., vol. 156, no. 7, p. 385, 2025, doi: 10.1007/s00704-025-05617-6.
[8] D. Štreimikienė, A. Bathaei, T. Baležentis, and J. Štreimikis, “Multi-criteria decision analysis of Circular Economy performance in the Baltic States: a comparative evaluation,” J. Bus. Econ. Manag., vol. 26, no. 5, pp. 1050–1070, 2025, doi: 10.3846/jbem.2025.24717.
[9] S. Sintaro, P. Palupiningsih, and J. Wang, “Comparative Analysis of Objective Weighting Approaches in Multi-Criteria Decision Making Using WASPAS,” J. Decis. Support Syst. Multi-Criteria Decis. Mak., vol. 1, no. 1, pp. 16–42, 2026, doi: 10.67449/jodesma.v1i1.2.
[10] D. A. Megawaty et al., “DAM Weighting: A New Approach to Determining Criteria Weighting in Multi-Criteria Decision Making,” Evergreen, vol. 13, no. 2, pp. 788–800, 2026, doi: 10.5109/7432655.
[11] M. Mesran et al., “Modification of Weighted Aggregated Sum Product Assessment Method to Improve Objective Weighting Accuracy in Multi-Criteria Decision Making,” Evergreen, vol. 12, no. 2, pp. 1213–1225, Jun. 2025, doi: 10.5109/7363505.
[12] P. K. Pramanik, S. Biswas, S. Pal, D. Marinković, and P. Choudhury, “A Comparative Analysis of Multi-Criteria Decision-Making Methods for Resource Selection in Mobile Crowd Computing,” Symmetry, vol. 13, no. 9. p. 1713, 2021. doi: 10.3390/sym13091713.
[13] E. Aytaç Adali, “A Novel MCDM Method: The Integrative Reference Point Approach,” Informatica, vol. 36, no. 3, pp. 625–655, 2025, doi: 10.15388/25-INFOR594.
[14] C.-N. Wang, N.-A.-T. Nguyen, and T.-T. Dang, “Sustainable Evaluation of Major Third-Party Logistics Providers: A Framework of an MCDM-Based Entropy Objective Weighting Method,” Mathematics, vol. 11, no. 19. p. 4203, 2023. doi: 10.3390/math11194203.
[15] P. Zandi, M. Ajalli, and N. S. Ekhtiyati, “An extended simple additive weighting decision support system with application in the food industry,” Decis. Anal. J., vol. 14, p. 100553, 2025, doi: 10.1016/j.dajour.2025.100553.
[16] H. Gaspars-Wieloch and D. Gawroński, “How can one improve SAW and max-min multi-criteria rankings based on uncertain decision rules?,” Oper. Res. Decis., vol. 34, no. 1, pp. 131–148, 2024, doi: 10.37190/ord240107.
[17] M. Bharadwaj. L, R. Sankargupta, R. Madurai Elavarasan, E. Devaraj, and A. Nandagopal, “Multi-Criteria Decision Analysis framework: Transitioning from coal to Large-Scale Photovoltaics in Australia for achieving SDG 7,” Appl. Energy, vol. 405, p. 127208, 2026, doi: 10.1016/j.apenergy.2025.127208.
[18] A. Biswas, K. H. Gazi, P. Bhaduri, and S. P. Mondal, “Site Selection for Girls Hostel in a University Campus by MCDM based Strategy,” Spectr. Decis. Mak. Appl., vol. 2, no. 1, pp. 68–93, Jan. 2025, doi: 10.31181/sdmap21202511.
[19] M. Hashemi-Tabatabaei, M. Amiri, and M. Keshavarz-Ghorabaee, “An Expected Value-Based Symmetric–Asymmetric Polygonal Fuzzy Z-MCDM Framework for Sustainable–Smart Supplier Evaluation,” Information, vol. 16, no. 3. 2025. doi: 10.3390/info16030187.
[20] S. L. Weng, H. L. Weng, F. L. Pei, S. H. Jaaman, and L. B. Swee, “Multi-Criteria Decision Making for the Selection of E-Commerce Platforms using AHP-TOPSIS Model,” J. Adv. Res. Appl. Sci. Eng. Technol., vol. 1, no. 2, pp. 120–136, 2024, doi: 10.37934/araset.60.1.120136.
[21] S. Dündar, “Prioritizing the Regional Preferences of Turkish Investors Regarding Foreign Direct Investment by ARLON Method,” Konya J. Eng. Sci., vol. 13, no. 3, pp. 927–946, 2025, doi: 10.36306/konjes.1648279.
[22] R. Raj, A. Singh, V. Kumar, T. De, and S. Singh, “Assessing the e-commerce last-mile logistics’ hidden risk hurdles,” Clean. Logist. Supply Chain, vol. 10, no. 1, p. 100131, 2024, doi: 10.1016/j.clscn.2023.100131.
[23] H. Gholami et al., “Mapping flood risk using a workflow including deep learning and MCDM– Application to southern Iran,” Urban Clim., vol. 59, p. 102272, 2025, doi: 10.1016/j.uclim.2024.102272.
[24] I. M. Hezam, A. R. Mishra, P. Rani, A. Saha, F. Smarandache, and D. Pamucar, “An integrated decision support framework using single-valued neutrosophic-MASWIP-COPRAS for sustainability assessment of bioenergy production technologies,” Expert Syst. Appl., vol. 211, p. 118674, Jan. 2023, doi: 10.1016/j.eswa.2022.118674.
[25] I. Z. Mukhametzyanov and D. Pamucar, “Equivalence of MCDM Methods and Synthesis of Solution Based on Ratings Obtained in Different Models,” Decis. Mak. Appl. Manag. Eng., vol. 8, no. 2, pp. 1–20, Aug. 2025, doi: 10.31181/dmame8220251473.
[26] N. Hendrastuty, S. Setiawansyah, M. G. An’ars, F. A. Rahmadianti, V. H. Saputra, and M. Rahman, “G2M weighting: a new approach based on multi-objective assessment data (case study of MOORA method in determining supplier performance evaluation),” Indones. J. Electr. Eng. Comput. Sci., vol. 38, no. 1, pp. 403–416, 2025, doi: 10.11591/ijeecs.v38.i1.pp403-416.
[27] S. Hadian, E. Shahiri Tabarestani, and Q. B. Pham, “Multi attributive ideal-real comparative analysis (MAIRCA) method for evaluating flood susceptibility in a temperate Mediterranean climate,” Hydrol. Sci. J., vol. 67, no. 3, pp. 401–418, Feb. 2022, doi: 10.1080/02626667.2022.2027949.
[28] Y. Rahmanto, J. Wang, S. Setiawansyah, A. Yudhistira, D. Darwis, and R. R. Suryono, “Optimizing Employee Admission Selection Using G2M Weighting and MOORA Method,” Paradig. - J. Komput. dan Inform., vol. 27, no. 1, pp. 1–10, Mar. 2025, doi: 10.31294/p.v27i1.8224.
[29] A. Yudhistira, S. Setiawansyah, T. Ardiansah, S. Maryana, Y. Yadin, and R. Oktaviani, “Development of Multi-Attribute Utility Theory Methods in Dynamic Decision Models Using Change-Data Driven,” Evergreen, vol. 11, no. 4, pp. 3279–3289, Dec. 2024, doi: 10.5109/7326962.
[30] M. Mohamed, S. Ayman, and J. Ye, “Assessment of Cybersecurity in Industry 4.0 using Delphi-Based Factor Relationships and Comprehensive Distance-Based Ranking Methods under Uncertainty,” Artif. Intell. Cybersecurity, vol. 1, pp. 21–36, Jun. 2024, doi: 10.61356/j.aics.2024.1296.
[31] D. A. Megaraty, H. Sulistiani, Setiawansyah, A. Qurania, Y. Yadin, and R. Oktaviani, “Integration Method Based on the Removal Effects of Criteria Weighting and MOORA Method: Wi-Fi Router Selection Case Study,” in 2024 International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS), 2024, pp. 241–246. doi: 10.1109/ICIMCIS63449.2024.10956545.
[32] I. Boukrouh, F. Tayalati, and A. Azmani, “A Comprehensive Framework for Supplier Selection: Using Subjective, Objective, and Hybrid Multi-Criteria Decision-Making Techniques With Sensitivity Analysis,” IEEE Access, vol. 12, pp. 145550–145569, 2024, doi: 10.1109/ACCESS.2024.3462348.
[33] M. R. Seikh and P. Chatterjee, “Determination of best renewable energy sources in India using SWARA-ARAS in confidence level based interval-valued Fermatean fuzzy environment,” Appl. Soft Comput., vol. 155, p. 111495, 2024, doi: 10.1016/j.asoc.2024.111495.
[34] S. Dündar, “Performance evaluation of IPARD-II rural development programs with integrated DIBR-RAWEC methods,” Pamukkale Üniversitesi Mühendislik Bilim. Derg., vol. 31, no. 3, pp. 339–350, 2025.
[35] T. Van Dua, D. Van Duc, N. C. Bao, and D. D. Trung, “Integration of objective weighting methods for criteria and MCDM methods: application in material selection,” EUREKA Phys. Eng., no. 2, pp. 131–148, Mar. 2024, doi: 10.21303/2461-4262.2024.003171.
[36] Gokhan Basar and Oguzhan Der, “Multi-objective optimization of process parameters for laser cutting polyethylene using fuzzy AHP-based MCDM methods,” Proc. Inst. Mech. Eng. Part E J. Process Mech. Eng., vol. 239, no. 4, pp. 2295–2309, Feb. 2025, doi: 10.1177/09544089251319202.
[37] T. Van Dua and D. D. Trung, “MEPSI (Mutriss Enhanced Preference Selection Index): a novel method for ranking alternatives,” EUREKA Phys. Eng., vol. 2024, no. 6, pp. 169–178, Nov. 2024, doi: 10.21303/2461-4262.2024.003408.
[38] J. Wang, S. Setiawansyah, and V. Saputra, “Hybrid Entropy and CRADIS Method Approach in Decision Support System for Selecting the Best Employees,” Build. Informatics, Technol. Sci., vol. 7, no. 4, pp. 2657–2668, Mar. 2026, doi: 10.47065/bits.v7i4.8985.
[39] T. Van Dua, “A Novel Approach for Criteria Weight Determination: A Case Study in Machine Ranking,” Eng. Technol. Appl. Sci. Res., vol. 16, no. 1, pp. 31333–31337, Feb. 2026, doi: 10.48084/etasr.15778.
[40] Hemn Othman Hama Ali and Karzan Mahdi Ghafour, “Enhancing Workforce Stability through Data-Driven Selection: An AHP-TOPSIS Approach for Healthcare HRM: Enhancing Workforce Stability through Data-Driven Selection,” Acad. J. Int. Univ. Erbil, vol. 3, no. 2, pp. 973–994, Apr. 2026, doi: 10.63841/iue32673.
[41] M. J. Naeini, M. Shakerian, S. Yazdanirad, and S. M. Mousavi, “Application of the SWARA–TOPSIS method for prioritizing turnover intention factors among nurses: a case study in an Isfahan hospital, Iran,” BMC Nurs., vol. 24, no. 1, p. 1415, 2025, doi: 10.1186/s12912-025-04066-w.
[42] M. I. Takaendengan, “Performance Comparison of Multi-Criteria Decision-Making Methods in Decision Support Systems,” J. Comput. Data Sci., vol. 1, no. 1, pp. 88–108, 2026, doi: 10.67449/jcoda.v1i1.5.
License
Copyright (c) 2027 Sumanto Sumanto, Mochamad Wahyudi, Lise Pujiastuti (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
