Data from: Group elicitations yield more consistent, yet more uncertain experts in understanding risks to ecosystem services in New Zealand bays

Group elicitations with diverse experts can reduce expert overconfidence by allowing cross-examination and reassessment of prior judgements, but groups are also prone to uncritical “groupthink” errors. When the problem context is underspecified the probability that experts commit groupthink errors may increase. This study addresses how structured workshops affect expert variability among and certainty within responses in a New Zealand case study.

We find that experts’ risk estimates before and after a workshop differ, and that group elicitations provided greater consistency of estimates, yet also greater uncertainty among experts, when addressing prominent impacts to four different ecosystem services in coastal New Zealand. After group workshops, experts provided more consistent ranking of risks and more consistent best estimates of impact through increased clarity in terminology and dampening of extreme positions, yet probability distributions for impacts widened.

The results from this case study suggest that group elicitations have favorable consequences for the quality and uncertainty of risk judgments within and across experts, making group elicitation techniques invaluable tools in contexts of limited data.

Usage notes

Expert data for ranks, mode comparison, and interval comparison Multiple sheets in the workbook show data for expert responses ranking and quantifying impacts (with uncertainty) on ecosystem services. On ranking sheets, each row corresponds to ranking of different experts. On mode comparison, "absolute diff" indicates the difference of each expert's estimate of impact compared to the mean value.

On interval comparison, "range" indicates the interval range for each expert's uncertainty in the impact score. Expert_data_dryad.xlsx