Paper detail

Noise and fluctuations can undermine the efficiency of Majority Rule in Group Evaluation problems

Crowdsourcing is a mechanism by means of which groups of people are able to execute a task by sharing ideas, efforts and resources. Thanks to the online technologies, crowdsourcing has become in the last decade an even more utilized process in different and diverse fields. An instance of such process is the so-called "label aggregation problem": in practice, it is the evaluation of an item by groups of agents, where each agent gives its own judgment of it. Starting from the individual evaluations of their members, how can the groups give their global assessment? In this work, by means of a game-theoretical, evolutionary approach, we show that in most cases the majority rule (the group evaluation is the evaluation of the majority of its members) is still the best way to get a reliable group evaluation, even when the agents are not the best experts of the topic at stake; on the other hand, we also show that noise (i.e., fortuitous errors, misunderstanding, or every possible source of non-deterministic outcomes) can undermine the efficiency of the procedure in non-trivial situations. Therefore, in order to make the process as reliable as possible, the presence of noise and its effects should be carefully taken into account.