They say you can’t compare apples and oranges. Yet the recommendation algorithms used by many review sites, are less than discerning — using seemingly irrelevant data about users’ preferences for gyms, for example, to suggest which cafes they might enjoy. Now an A*STAR researcher and his colleagues have devised a more sophisticated system that uses only the most relevant reviews written by each consumer to generate better recommendations for them.
Writing online reviews of products, venues and services can help others choose between the many options. But they also reveal information about the reviewer’s own tastes, which go beyond their purchase history. Sites mine the reviews that users write in an attempt to hone the recommendations they then make. This is a good idea, in principle, says Anh Tuan Luu, a computer scientist at A*STAR’s Institute for Infocomm Research. But, the systems making the recommendations tend to miss the mark.
The problem is the simplistic way that online platforms collect and compare data, says Luu. Most current systems combine all reviews that a user has ever written in one document, regardless of the product or service they are discussing. Similarly, they combine all reviews about an item written by other users in another single document. The system then compares these two documents, making recommendations based on any overlaps of interest it spots.
This strategy makes the mistake of weighting all reviews written by the user equally, even if they are about vastly different things, says Luu. “A user’s bad review about a coffee shop should be mostly irrelevant when deciding if a spa is a good match,” he says. “Not all reviews are created equal.”
Another problem is that documents listing every review written by a person can become unwieldy, as more irrelevant data is added, until eventually it hits an arbitrary cut-off. “The squashing of reviews into a single document is unnatural and ad-hoc,” says Luu.
Luu and his colleagues have devised a new algorithm which gives added weight to reviews that are directly related to the service or product in question. So when deciding if a coffee shop is a good match for a user, only the user’s previous reviews of eateries are considered, while their reviews of car mechanics, say, are ignored. The team’s system then compares this shorter subset of relevant reviews by the writer with reviews of a specific coffee shop by others.
The algorithm also looks for matches on a word-by-word level. If the reviewer mentions that they like cocoa, it would point to products which have received positive reviews containing the word chocolate. Similarly, if a user mentions enjoying a role-playing video game, the system searches out good reviews for other role-playing, rather than puzzle, games.
The team tested their algorithm on 24 benchmark datasets supplied by Amazon, including reviews for digital music, Android apps, video games and gourmet food, and on business reviews from Yelp. They evaluated their system’s performance by predicting user ratings for particular items and services and then compared them with the actual ratings given by those users. Their system outperformed two state-of-the-art systems, TransNet and DeepCoNN, by 19 and 71 per cent respectively.
The team was surprised to find that while the system needed to use many reviews to correctly match-up food and businesses preferences, it was able to correctly predict the writers’ tastes in apps and electronic games based on just one or two particularly insightful reviews.
“We hope that our model can be applied in current commercial sites,” says Luu.
The A*STAR-affiliated researchers contributing to this research are from the Institute for Infocomm Research.