When a reviewer loses their “elite” badge on the restaurant review site Yelp, the quality of their reviews declines, but the perceived value of those reviews remains surprisingly high because of how the site displays its status, according to a new study from the School of Management.

Available online before publication in Production and operations managementThe study investigated the impact of non-financial incentives such as badges and status on Yelp and found that when consumers see a badge indicating that a reviewer was once elite, they place more trust in these opinions, regardless of their depth or actual accuracy.

“Demoted reviewers feel they have been treated unfairly, leading them to reduce their efforts and produce lower-quality reviews,” says Wreetabrata Kar, study co-author and assistant professor of marketing. “But Yelp users can see a record of the years a reviewer held ‘elite’ status, which is a dominant indicator and leads readers to place more trust in the reviewer, even after he or she was demoted. This disconnect between actual and perceived quality can undermine the accuracy of reviews and potentially impact the long-term sustainability of the platform.

Using the Yelp Academic Dataset, researchers analyzed more than 6 million reviews of nearly 200,000 businesses written by more than a million reviewers between 2006 and 2018, focusing specifically on nearly 9,000 reviewers who were downgraded from elite status. They used propensity score matching and difference-in-differences techniques (methods that compare similar groups and measure changes over time) to establish a causal link between the loss of elite status and changes in review quality, while controlling for factors that could influence these results. results.

The researchers say these findings have broad implications for the design of online platforms that rely on user-generated content, highlighting the importance of balancing recognition and quality control to maintain user trust and confidence. integrity of the platform.

According to Kar, designing recognition systems is always a difficult task, because it requires striking a balance between exclusivity (the number of users benefiting from recognition) and the risk of alienation (how evaluators feel when their status is degraded).

“Platform managers should consider how recognition systems are displayed, as subtle design choices can have a significant impact on user behavior and perception, and may even undermine their goal of promoting a high-quality content,” he says.

Kar collaborated on the study with Vandith Pamuru, assistant professor of information systems at the Indian School of Business, and Warut Khern-am-nuai, associate professor of information systems at the Desautels Faculty of Management at the McGill University.