01 Jun 2015

Five-Star Research

Rating online reviews
Re: Jeremy Stoppelman (MBA 2005); Bill Heil (MBA 2009)
by April White


When Assistant Professor Michael Luca began studying online consumer reviews in 2008, then four-year-old Yelp.com, cofounded by Jeremy Stoppelman (MBA 2005), had 10 million reviews. Today, it has more than 71 million reviews, and the rapidly growing world of online consumer reviews has shown a measurable impact on a business’s bottom line. For Luca, who has authored several studies on the topic—and is himself an occasional Yelp critic—online reviews are “one of the most powerful information sources that has emerged in decades, if you interpret them correctly.”

Using Yelp reviews and financial data from Seattle restaurants, Luca found that a one-star increase in a Yelp rating increased an independent restaurant’s revenue by 5 to 9 percent. (Chain restaurants didn’t see the same effect.)

Yelp uses the arithmetic mean of reviewer ratings to arrive at a restaurant’s aggregate star rating. In his most recent paper, Luca and his fellow authors offered a new algorithm that takes into account reviewer accuracy, stringency, and other quirks as well as changes in a restaurant’s quality over time. If the algorithm were to be implemented, some rankings would change by as much as a quarter of a star.


Elite reviewers—a Yelp-awarded certification—have nearly double the impact of other reviewers. In another study looking at movie reviews, Luca found that consumer reviews could correctly predict expert reviews 85 percent of the time. (And yes, this is a real review, written by Bill Heil [MBA 2009]. Heil, who critiqued Spangler when he was an HBS student, now works at Yelp.)

Roughly 16 percent of restaurant reviews on Yelp are identified as potentially fraudulent by the site’s algorithm. Looking at the Greater Boston restaurant scene, Luca and his coauthors found that extreme reviews were more likely to be fraudulent, and restaurants that recently received bad reviews or faced increased competition were more likely to commit review fraud.


Next up: Luca is conducting experiments to see if posting health department inspection grades alongside Yelp reviews can change restaurant and consumer behavior.


Post a Comment