City Research Online

Protecting Children from Online Exploitation: Can a Trained Model Detect Harmful Communication Strategies?

Cook, D. ORCID: 0000-0002-6810-0281, Zilka, M., DeSandre, H. , Giles, S. & Maskell, S. (2023). Protecting Children from Online Exploitation: Can a Trained Model Detect Harmful Communication Strategies? In: Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society. AIES '23: AAAI/ACM Conference on AI, Ethics, and Society, 8-10 Aug 2023, Montreal, QC, Canada. doi: 10.1145/3600211.3604696


The growing popularity of social media raises concerns about children's online safety. Of particular concern are interactions between minors and adults with predatory intentions. Unfortunately, previous research on online sexual grooming has relied on time-intensive manual annotation by domain experts, limiting both the scale and scope of possible interventions. This work explores the possibility of detecting predatory behaviours with accuracy comparable to expert annotators using machine learning (ML). Using a dataset of 6771 chat messages sent by child sex offenders, labelled by two of the authors who are forensic psychology experts, we study how well can deep learning algorithms identify eleven known predatory behaviours. We find that the best-performing ML models are consistent but not on par with expert annotation. We therefore consider a system where an expert annotator validates the ML algorithms outputs. The combination of human decision-making and computer efficiency yields precision - but not recall - comparable to manual annotation, while taking only a fraction of the time needed by a human annotator. Our findings underscore the promise of ML as a tool for assisting researchers in this area, but also highlight the current limitations in reliably detecting online sexual exploitation using ML.

Publication Type: Conference or Workshop Item (Paper)
Additional Information: This work is licensed under a Creative Commons Attribution International 4.0 License
Publisher Keywords: Child sexual exploitation, chat logs, machine learning, natural language processing, online grooming
Subjects: H Social Sciences > H Social Sciences (General)
Departments: School of Policy & Global Affairs
School of Policy & Global Affairs > Violence and Society Centre
SWORD Depositor:
[thumbnail of 3600211.3604696.pdf]
Text - Published Version
Available under License Creative Commons Attribution.

Download (1MB) | Preview


Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email


Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login