Team project · Emory Computational Linguistics
AuthentiGuard
A Chrome extension that scores Twitter/X and Reddit posts for AI authorship right in the feed, backed by a FastAPI service that runs five detectors at once and shows you why it decided what it did.
What it is
Most AI-text detectors live in a separate tab. You copy a post, paste it into a website, wait, and read a number. AuthentiGuard puts that check where people actually read: inside the feed. It watches Twitter/X and Reddit with a MutationObserver, sends each new post to a local backend, and draws a small badge on the post within about five seconds. Green for human, red for likely AI, and a middle "uncertain" band when the signal is mixed.
Click the badge and it expands into a per-detector breakdown, so you can see the reasoning instead of trusting one confident-looking score. I built this with three classmates for Emory's Computational Linguistics course.
How it works
Every post runs through five detectors at the same time, fanned out asynchronously so the slowest one sets the pace. A fine-tuned RoBERTa classifier does most of the heavy lifting. Three separate LLM judges (Claude Sonnet 4.5, LLaMA 3.3 70B, and Mistral Large 2) each read the post and return a score plus a one-sentence reason. A handmade stylometric model adds another angle, measuring things like burstiness, type-token ratio, and how often common function words appear.
A weighted vote combines them: 0.40 RoBERTa, 0.40 the average of the three LLMs, and 0.20 stylometric. If a provider times out, its weight drops and the rest re-normalize, so one outage never blocks a verdict. Repeated posts hit a cache keyed on the hash of the text, which matters more than it sounds. Scrolling back up re-triggers the same posts constantly, and the cache keeps that from re-running five models every time.
Results, honestly
We tested it on a balanced 94-post slice of the RAID benchmark, stratified across 11 generator families and 8 writing domains. The full ensemble scored 0.87 AUROC, 0.76 F1, and a 0.14 false positive rate. The honest surprise was that RoBERTa on its own scored higher: 0.90 AUROC with zero false positives on that clean slice. Averaging in the weaker detectors pulled the leader down a little, which is exactly what a weighted vote does when one component already dominates the signal.
So on this benchmark, the ensemble did not beat RoBERTa on raw accuracy, and we wrote that up plainly rather than hiding it behind the headline number.
What I took away
The clean benchmark hides what the extra detectors are actually for. When a provider goes down, the badge still renders instead of failing. When the detectors disagree sharply (RoBERTa at 80%, Claude at 5%), that disagreement is information, and surfacing it beats hiding behind one number. We originally planned an XGBoost meta-classifier to fuse everything, but with only 94 examples it overfit, and a fixed weighted vote did better. That was the lesson that stuck: the fancier method is not automatically the right one, and at small sample sizes, simple and readable usually wins.
The other thing I did not expect was how much the interface mattered. People who would never paste a post into a detection tool will click a badge that is already sitting on the post. Where you put the feature changed whether anyone used it.