In recent times, the Internet has been riddled with disinformation in the form of fake news articles and deep fakes. Social media platforms have been plagued by hoax articles, with false claims seemingly made by experts. Online markets such as Amazon have not been left unaffected by misinformation. Misinformation exists on such platforms through customer reviews, an important factor driving product sales. In recent times, a massive, complex underground market, spanning across multiple countries has developed which works hard at manipulating reviews on Amazon.
The goal of this market is to obtain a large number of verified, positive, five-star rated reviews for products. A review is a verified review if the person writing the review has purchased the product through Amazon. As a tactic to promote sales, Amazon always shows the ’top reviews’ for any product while viewing product information. Large number of positive verified reviews also help improve the product’s ranking. More sales and five-star reviews can also result in the product being marked as Amazon’s Choice or Best Seller in a particular category.
Reviews have become an increasingly important parameter for judging a product quality before making a purchase. of adults in the United States check product reviews before buying a product online [smith2016online]. Studies [guo2020positive] have shown that positive customer reviews strongly influence a buyer’s decision to purchase a new product.
Amazon sellers (not manufacturers) are at the heart of this underground market for fake reviews. These are third-party entities who can list products for sale. Such sellers recruit ’review agents’, who help them increase their sales numbers and gain positive reviews. The agents contact buyers, convince them to buy the product, leave a review, and then refund them via PayPal or Amazon gift cards. In this underground market, buyers are often referred to as ’jennies’. The seller gains a large number of organic, verified reviews, and the buyer gets to keep the product for free.
This can help sellers and their products to bootstrap themselves by gaining sales and reviews for new items. These activities, and any other activities which manipulate reviews are against Amazon’s terms of service. Offering discounts for reviews, or giving away free products for positive reviews is banned.
For products which need bootstrapping in order to promote sales and gain reviews, Amazon offers a program known as the Amazon Vine Program 111https://www.amazon.com/gp/vine/help. Highly rated reviewers are allowed to select products, get them for free and honestly review them. Sellers, however, prefer to go the underground route; they are guaranteed to get positive, five-star reviews irrespective of how the product is. The seller market is heavily saturated; according to a study [emon2019amazon], new products were uploaded on Amazon every second from Chinese sellers.
This network of sellers, agents, and reviewers spans across multiple countries. Our investigations revealed that nearly all of the sellers were from China. The agents worked out of India, Pakistan, China and Bangladesh. These agents reach out to Amazon buyers in the United States, Canada, Germany, UK and France. Buyers are contacted via targeted advertisements and dedicated groups on Facebook.
Contributions. In this work, we study this complex underground market and aim to understand it’s functioning. We reach out to agents and sellers to learn how the market works. Our investigation uncovers several evasion techniques used in order to avoid detection by Amazon.
We summarize our contributions as follows:
We study the intricacies, functioning and caveats of the underground market for incentivized Amazon reviews. To the best of our knowledge, this study is the first of its kind.
We interview agents and buyers involved in this market and study their motivations and incentives. In addition, we also identify key tactics used by reviewers to avoid detection by Amazon.
Through our research, we demonstrate how Facebook and Instagram, with their targeted advertisement program further fuel this market and attract more and more users.
We present recommendations of features that online markets can take into consideration for increased fidelity in detection of such incentivized reviews.
2 Related Work
The issue of illegal incentivized reviews has received wide media coverage [nguyen2018inside] [nguyen2019her] [crockett20195star]. Journalists have studied the market by speaking to sellers, buyers, agents, and even Amazon. These journalistic reports reveal that Amazon is aware of the incentivized reviews and says that they make for less than of the total reviews on the platform [nguyen2018inside]. They also reveal that the market operates through all social media communication channels: facebook, Slack, Discord and even Reddit [nguyen2019her]. One report [nguyen2018inside]
finds that incentivized reviews adversely hurt genuine sellers. According the report, a family run business for bedsheet fasteners suddenly suffered a 50% loss in revenue because of counterfeit products; these sellers manufactured duplicate products, and obtained incentivized reviews for them which ranked them higher in a short period. Another report reveals that Amazon is trying to crack down on these incentivized reviews using machine learning systems in combination with social media to detect them and stop them at source[crockett20195star]. A third report explores the seller perspectives in the market; sellers spend as much as in order to increase their product ratings. The report also reveals that the underground review market has given rise to several other businesses; those which claim to understand the parameters Amazon uses in order to detect whether a review is fake or not [leticia2019some].
We were unable to find substantial academic works studying the underground review market. A prior work [mayzlin2014promotional] studied fake reviews and their effects on travel websites TripAdvisor and Expedia. The closest and most relevant work to ours was He et al. [he2020market]. In this work, the authors study the underground market and how it functions, but focus more on the kind of the products that solicit reviews and characterizing product outcomes before and after reviews. Their study on products shows that these fake reviews lead to a significant increase in average rating and sales rank but the effect disappears after around a month. The authors also observe that as much as a third of the reviews are deleted by Amazon and most of them are five-star reviews. The dates of review deletion falls roughly a month or two after the campaign on Facebook has ended. The fact that Amazon aggressively deletes reviews is surprising; a recent study [ananthakrishnan2020tangled] shows that flagging reviews as fake but leaving them on the platform can increase consumer trust.
Our work is different from prior works. First, we study the underground review economy in a structured, qualitative manner by interviewing both agents and buyers/reviewers who participate. Second, we focus on how the reviewers attempt to evade detection by Amazon.
3 Market Functioning
Most sellers cover the PayPal fees incurred as part of the refund. Many of them also cover the sales taxes that buyers have to pay. A small number of products also offer additional commission on top of the PayPal fees and taxes.
3.2 Understanding the Process
Through our discussions with the buyers and agents, we were able to understand how the review-refund process works in detail.
The agent/seller engages with a potential buyer via Facebook or Instagram. This is generally through the buyer clicking on a targeted advertisement.
The buyer is asked for their Amazon profile. This is to look at whether the particular account will be able to leave a review or not. Buyers are also asked how old the account is, and if any reviews have been removed by Amazon.
The buyer is asked their PayPal email. This will be used to refund the buyer later on.
The agent shares the search keywords and product images. The buyer searches for the product, and sends a screenshot to confirm.
Once confirmed, the buyer places the order. They send a screenshot of the order consisting of the order number and total amount to the seller/agent.
After the order is received, the buyer leaves a review. Once the review is live, they send a screenshot to the seller/agent.
Upon checking the verified review, the seller/agent refund the buyer the full amount of the product via PayPal.
4 Research Methodology
In order to understand the underground market for reviews, we conducted a qualitative research study. We interviewed agents as well as buyers who are involved in these activities.
4.1 Participant Recruitment
We were able to find groups on Facebook which served as forums for sellers and agents to post about the products they had. From the posts and comments, we were able to infer whether a particular member was a buyer or a seller. We contacted agents and asked them to be part of our study. Out of these, responded positively and answered our questions.
Recruiting buyers for our study wasn’t as simple. We observed a lack of response from the first several buyers we contacted. We created a dummy account and posed as buyers, and we realized that buyer inboxes are flooded with messages from sellers and agents. Through Facebook groups, we contacted buyers out of which responded. We were able to identify other buyers by reading the reviews of the advertised products on Amazon; we tried to track them down on Facebook, and were able to recruit more. Thus, overall, we interviewed agents and buyers.
4.2 Interview Methodology
All of our interviews were conducted online in form of chat via Facebook messenger or email. Since engaging in activities related to incentivized reviews is against Amazon’s terms of service, participants did not agree to talk to us on the phone or via video calls.
|Agents||Communication and Outreach||How do the sellers get in contact with you? Can you share how you got involved in this market?|
|How do you look for potential buyers? How do you communicate with them?|
|What media do you use to communicate with buyers and sellers?|
|How do you protect your identity on social media?|
|Process Overview||How does the entire refund process work?|
|Are the product taxes and PayPal fees covered by the seller?|
|How do you share product information with buyers?|
|How do you make sure that reviews are not removed?|
|How do you keep track of orders and refunds?|
|Incentives||Are you aware that this is against Amazon’s TOS?|
|How much do you earn per review?|
|What do sellers gain through this activity, since they refund the full price?|
|How many products and sellers do you deal with?|
|Buyers||General Questions||How did you get introduced to these opportunities for reviews in exchange for products?|
|How do you keep track of items purchased and reviewed?|
|How do you search for the correct product?|
|What has been your experience in writing reviews and receiving the refund?|
|How do you make sure that your account is not flagged by Amazon for fraud?|
|Incentives||How many items do you review per week?|
|Are you aware that these activities violate Amazon’s TOS?|
|How much bonus do you earn on an average per week?|
4.3 Ethical Considerations
We did not collect any personally identifying information during the course of this study. Since all of our interactions with the participants were online, we are not even aware of the real identities of most people. Only out of the agents and out of the buyers used their real-world identities; the rest of them used a fake online alias. As a result, we do not have any information about sensitive or protected attributes.
All the participants were aware that the answers they provide will be reported as part of a research study. Participants were recruited only after they provided us consent for participation. All participants had the choice to decline to answer any of the questions, or withdraw their participation at any time.
Throughout the course of our study, we identified several sellers and brands which engage in the illegal activity of buying reviews on Amazon. However, we do not reveal their identities. If we were to do so, the sellers could trace the source back to our participants, which would jeopardize their anonymity.
5 Agent and Reviewer Mechanics
As part of our study, we seek to understand the motivation and incentives for both buyers and sellers in participating in such activities. Our interviews reveal that both buyers and agents are aware that writing incentivized reviews is against Amazon terms of service. In fact, of the reviewers we interviewed had a backup Amazon account with different payment methods. The incentive, however, is financial.
Agent Incentives. Agents are paid a commission on every review they secure. The commission ranges anywhere from USD to USD. This is still a significant amount in many developing countries; as an agent A3, who is based in Pakistan, tells us:
"I get at least $4 per review, which is roughly 660 PKR. I can buy at least five whole meals here for that amount. And I can get this in a matter of minutes, without much work."
The commissions add up quickly, leading to a significant amount at the end of the month. Most agents work on getting reviews as a second, night-shift job; this serves them well, as their target customers are mainly in western countries (A6). According to agent A28, based in Bangladesh:
"One month I earned almost $250; my regular job pays me around $100 a month. And it requires no additional expense; just an internet connection."
Buyer Incentives. A clear motivation for the buyers is the opportunity to get products for free [nguyen2019her]. A reviewer R20 says:
"I know it’s wrong….but when I moved into my new place, I was able to get a chair, desk, humidifier, kitchen utensils, storage boxes, vacuum cleaners and a lot more for free. I saved so much by just writing reviews."
Other buyers go a step ahead. They get their products for free by writing reviews, and then sell them online. Reviewer R1 tells us:
"The product is basically free. Once I get my refund, I sell it on Facebook marketplace for around of the price. I made over $300 last month."
5.2 Communication Tactics
Reviewers are contacted primarily via Facebook. There are two mediums by which agents recruit reviewers: Facebook groups and targeted advertisements.
Facebook Groups. We were able to find over groups on Facebook which served as a hub for reviewers and sellers to contact each other. The largest of these groups had close to two hundred thousand members. On these groups, agents post information about the products which they have. Many agents post wide calls for action (see Fig. 1. Reviewers can also ask for any products they want. Agents are aware that what they are doing is illegal, and hence, always operate with an alias. They also use minor perturbations on the keywords so that they may not be flagged by Facebook (see Fig. 3). Agent A25 says:
"I always share secondary contact information like email or WeChat, as FB often deletes our [fake] accounts. I also avoid posting sentences which convey that we are offering refund in return for review. I always use words like re3view, r*fuund, amz*n, etc."
Targeted Advertisements. This is primarily how people are exposed to this underground market for reviews. Agents have pages on Facebook, which can then deliver targeted advertisements to users. These advertisements commonly tout their business as ’looking for influencers’, or ’looking for testers’ for their product. A few sample sponsored posts can be seen in Fig. 2. Clicking on the advertisement leads to opening a chat with the page via Facebook Messenger. There is a chatbot which asks users for their Amazon account links, PayPal accounts, and explains the process. A particular page can deal with hundreds of products; see Fig. 4.
Facebook does seem to be on the lookout for such pages; a reviewer R18 shared that he received an email from one of the pages saying how their Facebook page has been blocked. However, agents simply create another page with a different account and start operating and delivering advertisements from that page. See Fig. 5 for a sample email that R18 received.
5.3 Malicious Actors
The underground market is not without it’s fair share of malicious actors. Such actors exists among both reviewers and agents. of the reviewers said that they had been scammed at least once. Reviewer R15 tells us:
"It happens often. Some agents submit your review to the seller and give their own PayPal for refund, which means that they get the money. But in that case, we can just return the product and get the refund from Amazon itself. So, not our loss."
Dishonest actors among reviewers seem to be fewer; only agents noted that they had been scammed at least once. Dishonest reviewers get the refund, and then return the product to Amazon within the 30-day return window; thus earning the full amount of the product without spending anything. This affects agents adversely; as agent A4 says:
"We are paid our commission at the end of the month. If an order is returned, the seller does not pay us the commission for that review."
There is also a system in place to identify such malicious actors; there are Facebook groups in which reviewers post agents who have scammed them, and agents post the reviewers who scammed them. of the reviewers we talked to said that they first check for the agent names on these groups before placing an order.
5.4 Role of Social Media
Our research reveals that Facebook plays an important role in the continued operation of the underground reviews market. Facebook facilitates the delivery of advertisements to a targeted audience; therefore the pages (as described in Section 5.2 are able to leverage search history, marketplace activity and other data available to Facebook to identify users who might be interested in writing such incentivized reviews. Further, a direct fallout of the targeted advertisements is the rabbit hole phenomenon; once a user clicks or interacts positively with an advertisement for free products, he goes down the rabbit hole and sees several more advertisements of similar pages. Reviewer R32 says:
"I just chanced upon this side-hustle one day; I clicked on the advertisement, and then saw a few more. Pretty soon, my news feed was full of such advertisements; every third or fourth post I saw was for free products in exchange for reviews."
Thus, targeted advertisements facilitate identifying users who might be complicit with writing incentivized reviews, and gradually sucks them deeper into the underground market. Further, since Instagram is also owned by Facebook, targeted advertisements can work across the platforms. The pages can leverage users Instagram activity data to target them. Once a user clicks on an advertisement on Facebook, he is shown similar advertisements on Instagram as well.
6 Evasion Tactics
In this section, we discuss the suggestions and tactics that reviewers are asked to follow. Agents and sellers provide these guidelines during the purchase and before writing a review. The goal of these is to minimize the probability of your purchase and the following review being considered a fraudulent activity, and subsequently being deleted by Amazon.
6.1 Finding the Right Product
In order for the process to work smoothly, buyers must purchase the exact product shown. The brand name and the seller must be the same. Sometimes, buyers may be forced to buy the item at a higher price in order to match the seller name. A reviewer(R1) notes:
"I once found the product with the correct brand after going to page 5 of Amazon search results, but the price shown was around $3 less than what the agent shared with me. I asked if it was okay and they declined. I had to go to the list of sellers available for that product and choose the specific one. That gave me the correct price."
Agents reveal that even if the same brand is purchased, which seller it was purchased from matters. Agents are hired by sellers. An agent(A7) tells us:
"We are paid our commission by sellers, not the brands. If the buyer chooses a different seller, our seller does not gain anything and his rank does not improve. Thus, he will not refund for such an order."
6.2 Organic Search for Product
Sellers always recommend buyers to search for the product with keywords, rather than provide a direct link or the brand name. Buyers are then asked to scroll through the products and find the correct item. According to agents, this is because Amazon can track that you landed on that product from a link, and can consider it to be suspicious. An agent (A3) notes:
"It is for security of your account. Sellers don’t allow us to share direct links with buyers, as Amazon will detect this, and may remove the review that you write."
Similarly, buyers are advised never to search for a product with the brand name in the search term. As far as possible, agents never directly disclose the brand name. A reviewer(R2) describes this:
"They never show us the brand. In the image they send us, the brand name is blurred out. I try to find the correct product and send them a screenshot or link to confirm."
6.3 Platform Engagement
Buyers are encouraged to engage with the Amazon platform while searching for the product. They are asked to look at similar products, browse through images, and read reviews. The goal of this is to simulate an organic product discovery experience. An agent(A8) says:
"We always ask buyers to spend at least 1 minute browsing similar products, and at least 15 seconds on checkout page since if search and purchase is done too quickly, it is suspicious to Amazon."
Buyers generally follow the guidelines, since their accounts are at risk. A reviewer(R1) also concurs, saying:
"They want us to convince amazon that we actually searched for, found and liked this product. I am also concerned for my account; I do not want to get banned from the site. So even when I have found the product, I browse as I would normally do. I read reviews, mark them as helpful and watch product videos."
6.4 Payment Restrictions
Buyers are advised not to pay using gift cards, and always make the full payment. In addition, Amazon at times displays coupons using which buyers can get a discount on the product; buyers are cautioned not to use these either. The reason behind this is that the seller has designated a fixed number of coupons, and these are to attract other, genuine customers to buy the product. An agent (A4) says:
"Most sellers tell us not to ask buyers to use coupons. If they do, they still refund them, as it is not a loss. But the coupons are mainly there for other customers (who we will not be refunding) to think that they are getting a discount."
At this time, we do not know why sellers do not want buyers to use gift card. None of the agents we spoke to were aware of the reasoning behind the instruction. However, one reviewer (R11) notes:
"….I generally try not to use the gift card balance. One time I accidentally used $5 from my gift card balance, and they refunded me $5 less. Now I have to manually uncheck any gift card balance available, since Amazon uses it by default."
6.5 Review Timing
Buyers are recommended to wait at least days before submitting a review on Amazon. Reviews which are submitted too soon arouse suspicious. According an agent (A32):
"…Please submit reviews 10-15 days after you receive the shipment. Use the product for a few days and then review. Else, your review will be removed by Amazon."
Buyers are, of course, eager to receive the refund as soon as possible. However, they understand the reasoning. Reviewer R4 says:
"It makes sense. I need to have experienced the product before writing a review, otherwise it is suspicious. I maintain a spreadsheet of items I purchased along with their dates, so I know when 10 days have passed to write the review."
Furthermore, agents prefer those accounts who have not written more than reviews in a week, and not more than per month. Agent A22 tells us:
"Sellers do not want reviews from accounts who write too many reviews, such reviews always get removed by Amazon. We decline if we see too many recent reviews, and all of them five-star."
6.6 Review Content
Agents provide detailed instructions to buyers on the content of the reviews. According to agents, factors like the length of the review, and presence of supplementary media (images and videos) affects whether Amazon will remove the review. All ratings have to be five-star. Agents always recommend writing a review of at least words. A reviewer (R19) tells us:
"Picture and video reviews are really important to them. Once, an agent offered me additional commission of $5 if I wrote a review of more than 200 words. And $5 more if I attached a picture or video to it."
In addition, they are cautioned never to strongly emphasize any particular seller. An agent (A30) notes:
"Never ever mention ’always buy from this seller’ in your review. We want the product from our seller to be highly ranked. If you write about a seller, Amazon moves the review to seller feedback, which sellers don’t want, because customers don’t view seller feedback often."
Agents seem to know a lot about the internal workings of amazons detection models. An agent A27 tells us:
"There are two computer programs that Amazon uses to detect reviews. One looks at content and the other looks at timing, search behavior, etc. We know that the review content program checks if you have shared your opinion and not simply information about the product."
6.7 Seller, Agent and Product Similarity
In general, buyers are not permitted to review multiple products from the same seller within a certain period of time. According to agent A21:
"Very suspicious if you buy too many products from same store. I never return to the same buyer before 2 months of the first review."
Reviewers have another thumb rule which they follow, and that is avoiding buying too many products offered by the same agent. Reviewer R15 shares his experience:
"I purchased a vacuum cleaner and sent the order screenshot to the agent. She confirmed my order, and showed me some more products to buy. When I went to amazon, I saw the SAME product as a suggestion, under the heading ’people with similar purchases to you also bought….’. It would have been very suspicious. Clearly, a pattern had been captured."
We observed that product similarity was not a concern amongst both buyers and agents. We expected that the same item purchased and reviewed from various brands from a single account was bound to be suspicious. However, that was not the case. Many buyers purchased the same (or similar item) from different brands and stores, and reviewed them. A reviewer R9 revealed that he had reviewed different pairs of headphones in the past months. Similarly, other reviewers reported having reviewed more than of the same product within the past months.
6.8 Account Information
Agents and sellers also take into consideration account metadata while analyzing whether a particular buyer’s review is likely to be removed or not. Of our participating agents, reported that they strongly prefer Amazon prime account holders. In addition, the account should be able to leave reviews on Amazon; there is a requirement to have spent a minimum amount in the past months in order to be eligible to write reviews. Agent A24 says:
"Amazon moderates all reviews before they go live on the product page. Reviews by prime accounts are approved in around hours, but it takes around hours (4 days) for reviews from non-prime accounts. "
According to agents, reviews from accounts who have Amazon Prime subscriptions are less likely to get removed. of our participating agents reported that when some products had some reviews removed, all of them were from non-Prime accounts. In addition, agents also check if a prospective buyer is an easy rater, that is, he has too many five-star reviews and no three or four-star reviews.
Agents also recommend that reviewers write reviews for items that they have purchased organically (that is, without rebate). They also recommend that not all reviews be five-star. An agent tells us:
"I always make sure that the profile has some three and four star reviews as well. Otherwise, Amazon might flag the person to be an easy rater. If he easily gives five stars to all purchases, they find it suspicious. "
Through our research, we identified key tactics used by buyers and agents to avoid detection. We also studied how the review market functions and what aspects are important for it. Based on this, we recommend some features that online markets like Amazon can use in order to detect such incentivized reviews. These features may help improve machine learning models and increase the fidelity of detections.
Emphasis on Sellers. Our investigations show that having the correct seller is important for getting a refund. Amazon generally shows users the product from the seller which is least expensive. Genuine buyers do not generally care which seller the product comes from, since Amazon always provides a guarantee for returns. Therefore, if a user deliberately changes the seller (as described in Section 6.1), it may indicate that the review he writes subsequently is incentivized and not a genuine one.
Unnecessary Media. As described in Section 6.6, agents believe that reviews with photos and videos attached to them are less likely to be removed. This leads to buyers overcompensating and attaching pictures even when not necessary. of our reviewers noted adding pictures and videos even when it did not make sense. For example, in products such as nutritional supplements, USB chargers, and memory cards, photos and videos do not really add anything to the review; everyone already knows what the product looks like in real life. Thus, adding multimedia when not required may indicate that the review is incentivized.
Ignoring Discounts. A reasonable consumer will try to get the best price for a particular product he is buying. However, as discussed in Section 6.4, buyers who are receiving a rebate are instructed to not use any coupons offered by Amazon. A reviewer R33 noted that he was to buy a robot vacuum cleaner, but the agent asked him not to select the $70 savings coupon that was offered. Therefore, when a buyer does not elect to apply the coupons displayed on the product page, it may indicate that the subsequent review is incentivized.
Avoiding Gift Card Balance. If there is any existing gift card balance in an account, Amazon automatically applies that to any purchase you make. In order to not use that balance, the user needs to select the payment methods menu, and manually uncheck the box corresponding to the gift card balance. If a buyer deliberately avoids using any existing points and gift card balance on his account, it may indicate that the purchase and review is incentivized.
Unreasonable Choice. Most of the buyers we interviewed reported that oftentimes the product they had to find was not displayed on the first page, and they had to scroll till they reached the fourth pr even fifth page. This, they noted, was not something they did for their genuine purchases; they almost always chose the products on the first page. In addition, buyers reveal that they often had to buy items which had little to no reviews, as opposed to competing products with ratings above and hundreds of reviews. Again, this was not something they did for their genuine purchases; they chose the well-rated products. Therefore, if a buyer carries out an excessive search into the fourth or fifth page, or chooses a product with few reviews even when competing products are available, it may indicate that the purchase is incentivized.
Review and Return. As noted in Section 5.3, buyers may attempt to return the product even after having a written a five-star review for it because they are being cheated, or are trying to scam the agents themselves. Such activity can be evidence of incentivized reviews. This can also manifest itself as a downgrading of the review after a few days.
In this work, we have studied the underground market of fake Amazon reviews. We interview several agents and buyers and discover that buyers are targeted via customized advertisements and Facebook groups. Although they are aware that these activities violate Amazon’s terms of service, incentives for both buyers and agents are monetary. Buyers are instructed to simulate organic search for a product, engage with content on Amazon, not use a gift card or discount codes, and write a positive, five-star review after a week in order to evade detection by Amazon. Our research shows the various evasion tactics used, and we present a set of recommendations that can help increase the fidelity of detections by Amazon.