A Random Forest model that separates bots from real accounts by reading what they post and how they behave. 90% accuracy on about 10,000 held-out accounts. Published at ICAIML-AE 2026.
ICAIML-AE 2026, where the paper was published by Atlantis Press (Springer Nature)
When
2026
Team
With Akriti Biswas, guided by Dr. Vishal Shirsath
90%accuracy on held-out accounts
4,184bots caught
179real accounts wrongly flagged
01
The problem
Bot accounts are no longer easy to spot. Many of them post ordinary sentences and look like any other profile. What gives them away is the pattern: how often they mention other accounts, how their engagement compares with the number of followers they have, and the promotional phrases they repeat.
A detector that only reads text misses the behaviour, and one that only counts activity misses the language. The question behind this project was whether the two signals together separate bots from real people reliably enough to be useful as a backend layer for a platform.
02
The solution
I built the detector end to end in R. Behavioural features are engineered first: log-scaled follower counts, engagement and mention ratios, retweet and mention counts and the verified flag. Tweet text is cleaned, tokenised and turned into TF-IDF features with text2vec, keeping only terms that appear often enough to matter and not so often that they carry no signal.
Both feature sets are joined and fed to a Random Forest built with ranger (300 trees, mtry set to the square root of the feature count). The data is split 80/20 with caret so the evaluation is on accounts the model has never seen, and the same script produces the confusion matrix, ROC curve and the precision, recall and F1 charts that sit in the repository.
03
How it was built
About 60% of the effort went into preparing the data, around 35% into testing different approaches, and the remaining 5% into debugging, visualisation and documentation. The repository also contains the data-preparation step in full, including the part that adjusts counts and adds phrases for a 65% sample of records, so the way the dataset was shaped can be inspected.
Supervised models beat unsupervised ones, and combining behavioural and text features beat using either alone. The takeaway: understanding the data pays off more than trying another algorithm.
04
The result
On roughly 10,000 held-out accounts the model reaches 90.0% accuracy. It correctly flagged 4,184 bots and cleared 4,817 real accounts, with 179 real accounts wrongly flagged and 819 bots missed. The work was written up with co-author Akriti Biswas and faculty advisor Dr. Vishal Shirsath and published at ICAIML-AE 2026.
05
Tech stack
R
ranger
caret
text2vec
tidyverse
ggplot2
pROC
How it works
1DataA labelled set of social media accounts with their tweets and activity counts
2BehaviourLog follower count, retweets, mentions, engagement ratio, mention ratio and verified status
3TextTF-IDF features built with text2vec from the words accounts use
4ModelA Random Forest in ranger with 300 trees, trained on 80% of the data
5CheckConfusion matrix, ROC curve, precision, recall and F1 on the other 20%