When Sheep Shop: Measuring Herding Effects in Product Ratings with Natural Experiments

by   Gael Lederrey, et al.

As online shopping becomes ever more prevalent, customers rely increasingly on product rating websites for making purchase decisions. The reliability of online ratings, however, is potentially compromised by the so-called herding effect: when rating a product, customers may be biased to follow other customers' previous ratings of the same product. This is problematic because it skews long-term customer perception through haphazard early ratings. The study of herding poses methodological challenges. Observational studies are impeded by the lack of counterfactuals: simply correlating early with subsequent ratings is insufficient because we cannot know what the subsequent ratings would have looked like had the first ratings been different. The methodology introduced here exploits a setting that comes close to an experiment, although it is purely observational-a natural experiment. Our key methodological device consists in studying the same product on two separate rating sites, focusing on products that received a high first rating on one site, and a low first rating on the other. This largely controls for confounds such as a product's inherent quality, advertising, and producer identity, and lets us isolate the effect of the first rating on subsequent ratings. In a case study, we focus on beers as products and jointly study two beer rating sites, but our method applies to any pair of sites across which products can be matched. We find clear evidence of herding in beer ratings. For instance, if a beer receives a very high first rating, its second rating is on average half a standard deviation higher, compared to a situation where the identical beer receives a very low first rating. Moreover, herding effects tend to last a long time and are noticeable even after 20 or more ratings. Our results have important implications for the design of better rating systems.


page 1

page 2

page 3

page 4


Feature-level Rating System using Customer Reviews and Review Votes

This work studies how we can obtain feature-level ratings of the mobile ...

Eliminating the effect of rating bias on reputation systems

The ongoing rapid development of the e-commercial and interest-base webs...

Parametric Empirical Bayes for Predicting Quality in Rating Systems

User-solicited ratings systems in online marketplaces suffer from a cold...

FairJudge: Trustworthy User Prediction in Rating Platforms

Rating platforms enable large-scale collection of user opinion about ite...

Content Rating Classification for Fan Fiction

Content ratings can enable audiences to determine the suitability of var...

Designing Informative Rating Systems for Online Platforms: Evidence from Two Experiments

Platforms critically rely on rating systems to learn the quality of mark...

Surveys without Questions: A Reinforcement Learning Approach

The 'old world' instrument, survey, remains a tool of choice for firms t...

Please sign up or login with your details

Forgot password? Click here to reset