A fake-review detector that shows its working
A system that flags AI-generated product reviews, shows a human moderator why, and is built to avoid falsely accusing real customers.
“love the fit. needed for work. this pair is my back up pair.”
Short, casual human reviews are where detectors go wrong. The project finds these cases, explains them word by word, and picks the model that punishes fewer real customers.
Accurate detectors exist. Detectors a moderator can trust are harder.
Cheap LLMs make it easy to flood a platform with fake reviews. Transformer classifiers can catch many of them, but they give a score with no reason attached. When one flags a real customer, the moderator has nothing to check it against. I set out to build a detector that behaves like a decision-support tool: accurate, but also open about why it made each call and where it's likely to be wrong.
Three questions a moderator needs answered
Is this review fake?
AI-written reviews are cheap to produce at scale. The detector has to catch them reliably, and the final model gets 97.9% of unseen reviews right.
Why was it flagged?
A score on its own can't be checked or argued with. Every flag comes with the words that drove it and a plain-English note.
What if it's wrong?
Accusing a real customer does real damage. The system is tuned to avoid that, and uncertain cases go to a person instead of being acted on automatically.
Decisions driven by the harm, not the leaderboard
I chose the smaller model because it harms fewer real users
RoBERTa catches the most fakes, but it wrongly flags almost twice as many genuine reviewers. I picked the model that makes fewer false accusations, because accusing a real customer is the costlier mistake in moderation.
Model output becomes a moderator's report
The report combines the prediction, the confidence and the most influential words into a short verdict with advice. Tools like GPTZero and DetectGPT stop at the score.
Uncertain cases go to a person
Reviews are routed to High risk Low risk or Ambiguous queues by confidence, so low-confidence calls are never actioned automatically.
Two models find each other's hard cases
I collected every test review where the models disagreed, then compared which words pushed each model to its opposite verdict.
Both transformers beat the baseline. DistilBERT makes the safer mistakes.
Genuine reviews wrongly flagged
Per 1,000 human-written reviews. Lower is better for users.
AI reviews missed
Per 1,000 AI-generated reviews. Lower is better for catching fakes.
RoBERTa misses about 5 fewer fakes per 1,000 but wrongly accuses about 26 more real customers per 1,000. For moderation, that trade isn't worth it. Overall accuracy: DistilBERT 97.9%, RoBERTa 96.9%, baseline 86.4%.
Who was right when the models disagreed?
244 of 8,105 test reviews (3%). These are the most ambiguous cases in the set.
Word-level explanations make the model's mistakes visible
| Triage queue (100 test reviews) | Count | Correct | Incorrect |
|---|---|---|---|
| High risk predicted AI | 51 | 48 | 3 |
| Low risk predicted human | 48 | 47 | 1 |
| Ambiguous manual review | 1 | 0 | 1 |
| Total | 100 | 95 | 5 |
The triage run is where I found the system's main weakness. Three false positives reached the high-risk queue with over 89% confidence, and a threshold can't catch that. I report this openly, and it's the main reason the tool supports moderators rather than replacing them.
What the project showed
- Style tells
AI reviews lean on generic praise and complaint words (“problem”, “materials”), while people hedge and tell stories (“actually”, “maybe”). That difference is what makes detection possible, and reading whole reviews in context adds more than 11 points of accuracy over simple word counting.
- Bigger ≠ better
RoBERTa is the larger and more aggressive detector. The lighter DistilBERT strikes the better balance for moderation, and it's cheaper to run.
- No length shortcut
Human reviews are about 30% longer in this dataset, so a model could cheat by judging on length alone. I tested for this, and length barely affects its predictions.
- Confident ≠ correct
Some false positives had over 99.9% confidence. Confidence thresholds help at the margins but don't make a detector safe on their own.
What this project shows I can do
Fine-tuning transformers
Adapted two pre-trained language models to this task, with a test set kept unseen until the very end.
Evaluation beyond accuracy
Judged models by the kind of mistakes they make and who those mistakes hurt, then tested for shortcuts.
Explainable AI
Word-level explanations for individual decisions, compared side by side across two models.
Building for real users
Output that non-technical moderators can act on, and a workflow that keeps a human in charge of uncertain decisions.
Research and writing
A dissertation that grounds the design in research on content moderation and explainable AI, and is candid about limitations.
What I'd do next
- The AI-written reviews were generated with GPT-2. The next step is testing on reviews from current LLMs and a wider range of prompt styles, since detectors often get worse on text from newer models.
- Try other explanation methods and check whether they point to the same words.
- Add human-written reviews from platforms other than Amazon.