These days, we live and buy by online reviews. Looking for a pair of headphones? Wondering what movie to stream or if you should splash out for the new PlayStation 5? Or perhaps you need a hotel to stay in and suggestions for the best baby back ribs in town? Well look no further than the Internet, for someone somewhere will surely have a recommendation to offer about the product, service or facility you are thinking of.
To say online reviews are incredibly popular would be an understatement. A 2016 study by the Pew Research Center found that 82% of American adults “sometimes or always” read online reviews before buying something new. Another study found that 91% of people regularly or occasionally read online reviews, with 84% trusting them as much as a personal recommendation. And in a different survey, nearly 7 in 10 participants indicated that they would be willing to pay up to 15% more for a product or service if assured they will have a better experience.
But who are the people behind such reviews and why do they write them? In most instances, users contribute comments altruistically, because they want to share their experiences with the product or service in question.
The twist comes when retailers offer incentives — free or heavily discounted products, or cash rewards — in exchange for a person’s review. “This alters the behavioural mode because now reviewers don’t do things out of their own free will or autonomy, but rather to fulfil the requirements set by other people,” says Dandan Qiao, an assistant professor at NUS Computing who studies online altruism and pro-social behaviour. “They feel some kind of outside control or pressure because they now receive financial incentives for their reviews.”
Platforms and retailers offer such rewards in hopes of generating content and sales, but existing research on the topic has shown that financial incentives can actually be detrimental to the quality of reviews. Aware of this, Qiao wanted to take things one step further.
“There’s another issue that the literature hasn’t addressed yet,” she says, “and that’s the question of what happens to the subsequent reviews a person writes when he or she previously received financial incentives?”
A lingering effect
Studying this spillover effect, as Qiao calls it, is important because it can impact a platform’s long-term development. To examine the phenomenon, Qiao and her collaborators used a large dataset comprising more than a million reviewers on Amazon. The reviews were made over a three-year period from 2011 to 2014. Using machine learning, the researchers identified reviewers who previously received incentives for their posts versus ones who did not. Their analysis led them to one main conclusion: financial incentives can generate a negative spillover effect on an incentive recipient’s subsequent, non-incentivised reviews. “If someone had previously received incentives, they would write fewer reviews for free in the future. And these subsequent unincentivised reviews tended to have an upward-biased sentiment, reduced review length, and less linguistic richness,” Qiao says.
The extent of this spillover effect varied depending on the type of product reviewed, the researchers found. It was less pronounced for search goods, items such as electronics that can be more objectively described and evaluated in terms of their attributes and functionality. However, experience goods — such as video games or beauty products, which tend to have more subjective features — were found to be more susceptible to the spillover effect of incentives.
The effect was also observed to be greater in those who had written more incentivised reviews. “The more financial incentives a person received, the more likely it is that his reviews declined in quality after receiving those incentives,” says Qiao.
“In contrast, the contributors who were more altruistically driven were less affected because they were more resilient against financial incentives,” she says.
Tread lightly
When asked what was the most surprising aspect of their findings, Qiao replies that it was the fact that they even observed the spillover effect at all. “We knew from earlier studies that when financial incentives are offered, review quality deteriorates,” she says. “But we didn’t know that these effects continue to influence reviewers’ subsequent unincentivised review contributions.”
“Therefore, we caution platforms against mindlessly providing incentives for short-term quantity and sentiment increase,” Qiao and her co-authors write in their recently published paper. Paid reviewers may offer more positive reviews, but this often comes at the cost of “lower contribution quality and subsequent lowered incentivised contributions.”
But if platforms and retailers decide to offer incentives, Qiao and her team have these bits of advice to offer: administer incentives to search goods (rather than experience goods), place a limit on the maximum number of incentives a contributor receives, and select more altruistic reviewers as incentive recipients.
Qiao and her collaborators have also conducted a study on how to mitigate the negative impacts of financial incentives on review writing. “When applying financial incentives, can we come up with some strategies to alleviate the adverse impacts while preserving the benefits of incentives?”
“The key point is to increase users’ intrinsic motivation,” she says. “For example, can we use stimulations like goal-setting or challenge-seeking to help?
To explore this notion, the team conducted a series of randomised controlled experiments on the crowdsourcing marketplace Amazon Mechanical Turk. Their paper shows that the combination of financial incentives and goal-setting or challenge-seeking can help solicit good-quality reviews, so that the adverse impacts induced by financial incentives can be mitigated. In the future, Qiao says they hope to explore what strategies can be used to mitigate the negative spillover effects on the unincentivised reviews written by people who were previously paid.
Paper:
Financial Incentives Dampen Altruism in Online Prosocial Contributions: A Study of Online Reviews
