Reviews have been one of Amazon’s strengths and one of its problems for a long time. The issues around fake reviews are rather well known and covered extensively in traditional media as well (here is a New York Times piece on those issues and Amazon’s attempts to solve them).

However, there is also another, less thoroughly covered story of Amazon review manipulation: the helpful or non-helpful review vote. Anyone can vote for or against a review with the simple click of a button, and this affects how reviews are displayed on the page. This feature is nowadays extensively used for manipulation of review rankings on product pages.

Amazon review system and the effect of non-helpful votes

The Amazon review system is based on an evaluation scale from 1 to 5 stars. Amazon considers reviews that award the product 1-3 stars as “critical” and those that award the product 4-5 stars as “favorable”.

The reviews that are displayed most prominently on a page are the ones that Amazon estimates to be the most helpful. How this is determined is not exactly clear, but one feature of the system is clear: the ratio of helpful/non-helpful votes is more important than the number of helpful votes. What this means in practice is that in most cases you can demote a review from the top of the page with a single non-helpful vote. Are you thinking of what I’m thinking? There are pretty good opportunities for manipulation here.

Manipulation of review ranking through non-helpful votes

The idea is not exactly new. Nick Daws wrote about this possibility in his blog in 2009, although he took the moral high ground and told that he does not give non-helpful votes for reviews of his own books.

Fast-forward to 2013 though and this practice has become commonplace. In one recent example, an author asked her fans on Facebook to downvote a critical review, and the fans responded by both downvoting the critical review and upvoting the favorable reviews.

I have also witnessed this behavior first-hand enough to believe that this is nowadays a very common practice. You see, I sometimes post book reviews on Amazon. I have posted a total of 16 reviews so far, 3 of which have been critical. Every critical review I have posted has received a non-helpful vote within 24 hours of being posted. None of my favorable reviews have received any non-helpful votes.

I will discuss these cases in a bit more detail, as they are a very nice sample of three different types of manipulation taking place.

Case 1: A book on ISO 9001 with some clearly false information in it. I bought a book on ISO 9001 that had no reviews so far (a risky move, I know). Turned out the book was not that good, as although it got some things right, it got quite a few things wrong as well. I wrote a critical review, and gave it two stars. The non-helpful vote arrived soon thereafter. A while after that, two 5-star reviews were posted by people who did not usually review books. Later on, I also got a helpful vote, but my review is still the least helpful of the three with 1 of 2 people finding it helpful whereas the other two have no votes in either direction. This is the case of downvoting critical reviews, combined with what I suspect are traditional fake 5-star reviews.

Case 2: A book on social business with an incorrect view of modern business practices, methods that are inferior to well-known ones published elsewhere, and the typical fad hype of “if you don’t do this, your business will go under and you will lose your job in the next 5 years.” I detailed its shortcomings and gave it two stars. The non-helpful vote arrived soon thereafter. In this case, a person who had been working with the author even came to my blog to post a comment on what a great guy he is (what does that have to do with his book, anyway?), and I received a couple of more non-helpful votes (which may have even been unsolicited). Again, I did receive a helpful vote as well. In this case, all the favorable reviews have one helpful vote, some have more (the ones that people actually agree with). This is the case of upvoting favorable reviews yourself (1 upvote each), downvoting critical reviews, and mobilizing your friends against downvotes.

Case 3: A book on developing your writing voice that was written with a fair bit of curse words (this author’s preferred voice) and was mostly about getting past the fear of writing with a little bit of advice on words to use and words to avoid. I wrote a somewhat critical review, three stars, and detailed how the book can be helpful for people who do not write out of fear, but also how it limits the range of acceptable voices to a certain area and the last few pages were pure advertisement of the author’s consultancy services with prices and all. The non-helpful vote arrived soon thereafter. In this case, all favorable reviews had at least 4 upvotes, and most of them exactly that number. This is the case of downvoting critical reviews and mobilizing your friends to upvote favorable reviews (exactly 4 upvotes each).

Advanced review ranking manipulation tactics

Based on how obvious the above behavior is, here is a devious manipulator’s guide to Amazon reviews:

• Do not downvote critical reviews immediately
• Mobilize your friends, but in varying numbers so that the number of downvotes and upvotes given to each review is not the same
• Cross-vote, so that no single person downvotes or upvotes all reviews
• Maintain a balance of five-star reviews and other reviews, too many five-star reviews compared to the rest seems unauthentic (Chris Eboch’s take on this)
• Post a critical review (1-3 stars) of your own that is not really critical at all and do not downvote it so that you can have it appear as the most helpful critical review instead of one that is actually critical

Then again, as there are no real consequences for the obvious manipulation going on today, I’m not sure whether there is any use to do it better to hide it. Of course, doing any of this is just wrong, so shame on you.

Can this be remedied through data analysis?

While regular users cannot see who has voted on the reviews, Amazon obviously has access to this data. Perhaps this data can be used to remedy the situation by monitoring who is voting on what, but this is still by no means easy, as the sample sizes are often low, and if the devious manipulator’s guidelines above are used, it becomes even more difficult to track this behavior.

One of the unfortunate side-effects of the current system is that  the community has become self-moderating and not submitting that many 1-3 star reviews. Here is a long discussion with many comments to this effect. This is one of the reasons why a good solution is needed to improve the reliability of the reviews.

For the time being, the only real remedy is judgment of the individual customer. Instead of looking at the star ratings, or even looking at the most helpful reviews, scan the reviews for the arguments presented in them, and make your call based on the arguments, not on quantified data. For this purpose, reviews can still be a powerful asset.

Photo: Used books by Benjamin Ragheb @ Flickr (CC)