In 1981, a Swedish psychologist called Ola Svenson asked a group of 81 students from the University of Oregon to rate their driving skills. A little surprisingly, 93% of the students placed themselves ‘in the top 50% of drivers’.
In case Oregon students are exceptionally good drivers, it’s worth mentioning that this effect has since been replicated around the world – both with questions about driving skills and safety, and also with questions addressing respondents’ popularity, health habits and relationship success. Whatever sphere of competence being tested, the number of people rating themselves ‘above average’ is almost always paradoxically high.
The phenomenon of overrating one’s abilities is called ‘illusory superiority’, and reflects just one of the ways in which humans massively misjudge their own competence. Other well-documented ways to get it wrong are “ego-centric advice discounting” (preferring to stick with your own opinion rather than take the advice of an expert), the “hard-easy effect” (over-estimating your performance on hard tasks while simultaneously under-estimating your performance on easy tasks), and the “Dunning-Kruger effect” (thinking you’re about as competent as everyone else).
Although these effects have been fairly exhaustively studied at the individual level, it took a team of researchers from all over the world to ask how these competence biases might come into play for groups of people working together.
In a series of experiments, the researchers assembled pairs of people from Denmark, Iran and China (we’ll come to the choice of countries later). They then asked the pairs to play a game. (Note that we’re using the scientific term ‘game’, so it would be dangerous to assume the people involved had fun while they were playing it.)
Each person from every pair had to sit at a darkened desk – visually separated from their partner – wear headphones, and watch a computer screen. As each participant watched, a rather complicated image of six tiny greyscale patches appeared on the screen for a brief moment and then disappeared. Then after a short interval, another, very similar image of six little patches appeared on the screen before vanishing. In only one of the images, there would be a tiny difference in the light-dark contrast for one of the patches. The task for the participants was to identify which image, first or second, had the patch with tweaked contrast. Participants were also asked to record how confident they were in their guesses.
The task itself was clearly pretty difficult. But the key part of this experiment comes from the way in which participants were then obliged to work together.
After privately voting for the image they thought contained the high-contrast patch, and recording their confidence in their votes, one of two things could happen. If both participants had agreed on the first or second image, they would immediately receive feedback telling them whether they were right or wrong, and the computer would move on to the next set of images.
However, if they had voted for different images, one of the participants (switched throughout the experiment) would be asked to cast a final vote on behalf of the pair. This ‘arbitrator’ would be able to view both votes, as well as the confidence estimate of each participant. After the arbitrator had cast this final vote, both participants would again be presented with feedback about their performance and move on to the next set of images.
It was in this latter event that the researchers noticed some rather interesting behaviour. A lot of social theory – and let’s be honest, common sense – tells us that we ought to pay more attention to reliable sources. In other words, you might expect that people presented with information about their partner’s accuracy as well as their own might adjust their ‘final votes’ as they gain knowledge about their relative competence. If my partner is particularly good at identifying high-contrast patches flashed onto a computer screen – a useful skill to be sure – I might want to heed their advice in the event of a disagreement between our votes.
Indeed, technically speaking, the best way to attain the most correct answers in this game, when presented with the option of voting for the pair, is to multiply your own confidence by the probability that you’re right, and add that to your partner‘s confidence multiplied by the probability that they are right. This way, you weight everyone’s opinions with both their competence at the task, and how sure they are about their answer in this round.
This is not what people did. Not only that, people’s behaviour differed depending on how competent they were at the task.
People who found it very difficult to identify the correct image tended to over-represent themselves when given the chance to vote for the pair. (They also expressed more confidence in their incorrect answers than people who were good at the task.) Conversely, people who had high accuracy when it came to choosing the correct image tended to under-represent themselves. In other words, it seemed like the Dunning-Kruger effect, or ‘equality bias’ – where people assume they’re all about equally competent – was coming into play. And the pair’s score, as a result, was falling well behind the maximum.
The researchers wondered whether perhaps people behaved in this way because they found it difficult to remember their partner’s previous guesses. In a follow-up experiment, participants had to perform the same tasks but were given a running accuracy score for themselves and for their partners. But even when made completely aware of their average performance, participants in the study still behaved as though each person had equal competence and should therefore contribute equally to each ‘final vote’.
No doubt slightly exasperated, the researchers lastly tried giving people money rewards for correct answers. But still, when participants acted as arbitrators, they tended to rely equally on votes rather than favouring the guess of the more competent participant, and achieving a higher score as a result.
In other words, people seemed so set in their behaviour, the researchers couldn’t even shake them out of it with bribes…
Here is where the choice of countries comes in. One factor of human behaviour that has often been implicated in these games is trust. But trust is a partly cultural trait. In fact, a number of studies claiming ‘universal’ human behaviours in such games have been criticized as focusing too narrowly on ‘Western, Educated, Industrialized, Rich, Democratic’ countries (the inventor of that acronym must be pretty pleased with him/herself).
To avoid having to respond to this particular criticism, the researchers chose their countries carefully. The countries they picked had all been surveyed about which statement they preferred of “most people can be trusted” and “you can never be too careful when dealing with people”. They settled upon Denmark, China and Iran, which favoured the first statement approximately 71%, 52% and 11% of the time respectively. This sort of difference, the researchers argued, gave them the right to call their paper:
“Equality bias impairs collective decision-making across cultures.”
However, why we might prefer excessively equal representation in situations where some people are clearly more competent than others is a little unclear from the results in this paper. One possibility that the researchers suggest is that people are simply trying to avoid conflict by assuming equality – a sort of cognitive ‘meeting in the middle’. Another possibility is that it’s just a bit too challenging to calculate the optimal decision when both accuracy and confidence of more than one person are taking into account. Instead, we may have evolved to base our decisions on heuristics or ‘rules-of-thumb’ rather than precise equations.
But just as the paper discussed in a post last month questioned the notion that groups use social information optimally, so now this paper questions the notion that we properly weight that social information according to individual differences in competence.
Like various sub-disciplines in animal behaviour and social science before it, collective behaviour seems to be making the gradual (in my opinion healthy) move away from the study of living systems in terms of optimality. Instead, it is approaching an understanding which views living systems and the components that make them up them as subject to numerous, multi-scale trade-offs. In other words, it views them as evolved.