Ranker visitors come from a diverse array of backgrounds, perspectives and opinions. The diversity of the visitors, however, is often lost when we look at the overall rankings of the lists, due to the fact that the rankings reflect a raw average of all the votes on a given item–regardless of how voters behave on multiple other items. It would be useful then, to figure out more about how users are voting across a range of items, and to recreate some of the diversity inherent in how people vote on the lists.
Take for instance, one of our most popular lists: Ranking the Worst U.S. Presidents, which has been voted on by over 60,000 people, and is comprised of over a half a million votes.
In this partisan age, it is easy to imagine that such a list would create some discord. So when we look at the average voting behavior of all the voters, the list itself has some inconsistencies. For instance, the five worst-rated presidents alternate along party lines–which is unlikely to represent a historically accurate account of which presidents are actually the worst. The result is a list that represents our partisan opinions about our nation’s presidents:
The list itself provides an interesting glimpse of what happens when two parties collide in voting for the worst presidents, but we are missing interesting data that can inform us about how diverse our visitors are. So how can we reconstruct the diverse groups of voters on the list such that we can see how clusters of voters might be ranking the list?
To solve this, we turn to a common machine learning technique referred to as “k-means clustering.” K-means clustering takes the voting data for each user, summarizes it into a result, and then finds other users with similar voting patterns. The k-means algorithm is not given any information whatsoever from me as the data scientist, and has no real idea what the data mean at all. It is just looking at each Ranker visitor’s votes and looking for people who vote similarly, then clustering the patterns according to the data itself. K-means can be done to parse as many clusters of data as you like, and there are ways to determine how many clusters should be used. Once the clusters are drawn, I re-rank the presidents for each cluster using Ranker’s algorithm, and the we can see how different clusters ranked the presidents.
As it happens, there are some differences in how clusters of Ranker visitors voted on the list. In a two-cluster analysis, we find two groups of people with almost completely opposite voting behavior.
(*Note that since this is a list of voting on the worst president, the rankings are not asking voters to rank the presidents from best to worst, it is more a ranking of how much worse each president is compared to the others)
The k-means analysis found one cluster that appears to think Republican presidents are worst:
Here is the other cluster, with opposite voting behavior:
In this two-cluster analysis, the shape of the data is pretty clear, and fits our preconceived picture of how partisan politics might be voting on the list. But there is a bias toward recent presidents, and the lists do not mimic academic lists and polls ranking the worst presidents.
To explore the data further, I used a five cluster analysis–in other words, looking for five different types of voters in the data.
Here is what the five cluster analysis returned:
The results show a little more diversity in how the clusters ranked the presidents. Again, we see some clusters that are more or less voting along party lines based on recent presidents (Clusters 5 and 4). Cluster 1 and 3 also are interesting in that the algorithm also seems to be picking up clusters of visitors who are voting for people that have not been president (Hillary Clinton, Ben Carson), and thankfully were never president (Adolf Hitler). Cluster 2 and 3 are most interesting to me however, as they seem to show a greater resemblance to the academic lists of worst presidents, (for reference, see wikipedia’s rankings of presidents) but the clusters tend toward a more historical bent on how we think of these presidents–I think of this as a more informed partisan-ship.
By understanding the diverse sets of users that make up our crowdranked lists, we are able to improve our overall rankings, and also provide more nuanced understanding how different group opinions compare, beyond the demographic groups we currently expose on our Ultimate Lists. Such analyses help us determine outliers and agenda pushers in the voting patterns, as well as allowing us to rebalance our sample to make lists that more closely resemble a national average.
- Glenn Fox