Researchers Explore the Use of LLMs for Content Moderation

AI in content moderation is far from a recent innovation: machine learning (ML) classifiers have been used for content moderation since around 2016. But since the technological advances of LLMs, platforms are accelerating the transition to AI for content moderation, laying off trust and safety workers and outsourced moderators in favor of automated systems. For instance, changes Meta announced to its content moderation practices in January included deploying “AI large language models (LLMs) to provide a second opinion on some content.” In August, TikTok announced it would lay off moderators in favor of AI systems as part of a reorganization plan intended to strengthen its “global operating model for trust and safety,” according to the Wall Street Journal.

The incentives for industry are straightforward, including cutting costs and reducing reliance on overburdened (and frequently exploited) contract workers, for whom the job of sifting through endless streams of unsavory user-generated content can have severe consequences. But alongside their promise, the use of LLMs for this purpose raises a host of questions, some of which are now the topic of study both in universities and in industry. The subject was a stand-out theme at the Trust and Safety Research Conference, held last month at Stanford.