Combining experts’ ratings for partly overlapping concept lists: a feasibility test with cultural sustainability indicators

Abstract

Acquiring preference information from decision-makers and stakeholders may carry biasing effects due to question framing. In order to avoid unwanted distortions, respondent-driven querying methods are advisable to apply; however, concerning multiple stakeholders a challenge remains how to combine individually collected concepts and further on their individual valuations to an unified preference information. This paper introduces one solution: a semi-automatic stochastic simulation of joint preferences from partially overlapping individual concept lists and preference ratings. We used completed expert interview dataset of cultural sustainability indicators acquired for comparing bioenergy production chains. According to the results the approach seems generally applicable, but the feasibility may vary according to case characteristics. Combining concept list valuations with stochastic simulations may be more feasible the more similar the expected concept structures are. The presented method contributes particularly to planning processes in which democratic participation of a large number of stakeholders is needed in the goal setting phase. However, more tests with different decision problem types are needed to verify and refine the present findings.