An analysis of more than 700,000 TripAdvisor and Expedia reviews finds that soliciting reviews decreases organic review rates, and it worsens the quality of the reviews people do write.
Authored by WANG Jing
According to recent surveys, 65% of consumers trust online reviews more than ads, and nearly four out of five trust reviews from strangers online just as much as personal recommendations from their friends and family. Online reviews have become incredibly valuable to brands, and so, it’s no surprise that many sellers have begun taking steps to proactively solicit reviews from their customers.
In some cases, companies have attempted to pay or otherwise incentivize customers to leave reviews — but of course, this comes with obvious risks, from egregiously violating many ecommerce platforms’ terms of service to substantially reducing consumers’ trust in a brand. That’s where platform-endorsed review solicitation tools come in.
From Amazon’s Vine to TripAdvisor’s Review Express, many ecommerce platforms now offer tools specifically designed to help the businesses on their platforms collect reviews ethically and transparently. For many brands, using tools like these to solicit guest reviews seems like a no-brainer: They’re typically free or low-cost, and they’re an easy way to boost your review volume and potentially lift your ratings. But what actually happens when brands use these programs?
To explore this question, my colleagues and I analyzed more than 700,000 reviews from more than 500 hotels on TripAdvisor and Expedia. We compared TripAdvisor and Expedia ratings for the same hotels before and after they started using the TripAdvisor Review Express tool, which helps hotels send emails inviting guests to post a review after their stay (at the time, Expedia did not offer an equivalent tool, which enabled us to isolate the impact of using a solicitation tool on a brand’s ratings and reviews).
And what did we find? Asking for reviews did indeed increase the total number of reviews a hotel received, but with significant hidden costs: Organic reviews fell substantially, ratings were inflated with less informative feedback, and reviews became vaguer and less helpful.
Soliciting Reviews Boosts Volume and Ratings… But at What Cost?
Specifically, we found that participating in TripAdvisor’s solicitation program led to a striking 34.3% increase in the total number of reviews a hotel received per month. The average review rating also increased by 0.151 out of 5 stars — a significant increase in an industry where the difference between a 4.0 and a 4.5 can make or break a business. We also found that this ratings boost was highest among smaller, cheaper, less popular, and lower-rated hotels. But beyond these top-line numbers, we discovered two critical hidden costs associated with soliciting reviews:
Solicited Reviews Are Lower-Quality
First, we found that the solicited reviews that drove the volume increase were, on average, 16.9% shorter than organic reviews. This is critical, as prior research has shown that consumers perceive shorter reviews as less helpful, informative, and trustworthy.
We further verified this drop in quality by comparing the content in solicited and organic reviews with a sophisticated text analysis technique called a Structural Topic Model. Through this analysis, we found that organic reviews tended to be detailed, specific, and concrete, while solicited reviews were much more general and abstract. While organic reviews discussed details like room equipment, internet speed, breakfast options, parking, and the dining experience — valuable information that helps other customers make booking decisions — solicited reviews used vague terms like “recommendation and satisfaction,” “exceptional quality,” and “staff courtesy” to share largely positive but far less informative content.
Review Solicitation Reduces Organic Reviewers’ Motivation
Perhaps our most significant finding was the effect of solicitation on reviews that weren’t solicited: We discovered that after a hotel started asking for reviews, their volume of organic reviews fell by 15.5%. And while this could have been just the result of would-be organic reviewers using the convenient email link instead, we found that this was not the case.
Instead, our data demonstrated that the very experience of being solicited undermined people’s intrinsic motivation to leave a review. When customers who otherwise would have written a review saw that a hotel was proactively collecting reviews, their voluntary impulse to contribute to the community diminished. Because of this psychological effect, we found that after a hotel started soliciting reviews, the total number of reviews written by people whose first review had been organic decreased, and that people who had never written a solicited review reduced their contributions more than similar users who had previously written a solicited review.
This suggests that when hotels asked people to leave reviews, it actually devalued the act of review-writing for their most authentic advocates. What was once a voluntary, public-spirited contribution to a community became, in their eyes, a transactional response to a corporate request.
As a result, the practice of soliciting reviews silently corroded the value of the entire review ecosystem. After all, if a platform filled with solicited reviews becomes a place of pleasant but unsubstantiated praise, written by the customers whose input is least valuable while those whose reviews would be most helpful opt out, savvy consumers may quickly learn to discount it entirely.
Balancing the Pros and Cons of Soliciting Reviews
To be sure, we are not suggesting that companies should never solicit reviews. However, our research demonstrates that review solicitation comes with distinct tradeoffs. As such, there are several steps brands should take when determining whether or how to solicit reviews from their customers.
1. Know Your Baseline
As with any strategic decision, the pros and cons of soliciting reviews will depend on the specifics of your brand’s situation. One of the most substantial benefits of soliciting reviews is that it increases average ratings, but we found that this effect was not evenly distributed: Larger, more expensive, higher-class, more popular, and higher-rated hotels experienced less of a ratings lift than their smaller, independent, and lower-rated counterparts. So, if your business already has a strong reputation, the marginal gain from solicited reviews may be small, and thus perhaps not worth the loss of higher-quality, organic feedback. But for newer or struggling businesses looking to jump-start their online presence, the benefits of review solicitation may outweigh the risks.
2. Weigh Quantity Against Quality
Are 10 short, positive 5-star reviews better than three detailed, more critical 4-star reviews? It’s hard to say. The former might nudge your average rating up, which can absolutely bring benefits, but the latter are likely to be more trusted and more useful for converting discerning customers. In addition, critical reviews may also serve as a valuable source of customer feedback, without which businesses may struggle to improve their offerings. As such, it’s generally wise to prioritize the quality of reviews, not just their volume and positivity.
3. Protect Your Organic Advocates
Your most valuable reviewers are those who write voluntarily. These organic reviewers write the most detailed, helpful reviews — but their motivation is fragile, and it can be undermined when they are solicited for reviews. So, when possible, avoid bombarding your entire guest list with automated solicitations.
4. When Soliciting Reviews, Encourage Detailed Input
If you do decide to solicit reviews, craft your message carefully. Instead of a generic “Leave us a review,” try to encourage quality. Frame the request as an invitation to share detailed input to help future customers, and ask specific questions about your customers’ experiences with your business. This can nudge respondents toward more concrete and useful feedback, reducing the negative impact of solicitation on review length and quality.
5. Platforms: Inform and Empower Consumers
Finally, for review platforms like TripAdvisor, our research highlights the very real pitfalls that come with solicitation tools. While these services can boost review volume in the short term, they can also degrade the ecosystem’s long-term health and value proposition. To mitigate these risks, platforms can refine their disclosure language to emphasize to users that solicited reviews are still non-incentivized and legitimate. In addition, platforms can provide filters so that users can choose to see only organic or solicited content, and they can also invest in algorithms to correct for the positivity and brevity bias in solicited reviews, to maintain the fairness of their ranking systems.
Soliciting Reviews Comes at a Cost
In an economy of trust, the authentic voice of your organic reviewers is your most valuable asset. While blasting review solicitations to every customer can inflate your review metrics in the short term, it also erodes review quality, alienates your most genuine advocates, and fundamentally shifts conversations about your brand from valuable details to empty praise. So, before you hit “send” on that solicitation email, carefully consider the hidden costs: After all, you might just be trading quality for quantity, and long-term credibility for an artificial, short-term boost.
Wang Jing is an associate professor of information systems at HKUST, focusing on the gig economy, knowledge communities, online reviews, crowdfunding, crowdsourcing, social networks, and data mining.
This article draws on the research paper “The Pitfalls of Review Solicitation: Evidence from a Natural Experiment on TripAdvisor,” authored by GAO Baojun, WANG Jing, DING Xiaojie and GUO Yue.