AI is Removing Bottlenecks to Effective Content Moderation at Scale

In my nearly two decades working in trust & safety, I’ve helped build systems that processed billions of pieces of content. The hardest part of getting these systems to work is never the scale per-se — it is the policy interpretation. How do you translate a nuanced human judgment about what crosses a line into something a machine can evaluate consistently?

The traditional answer has been: you don’t, really. You accept that machine learning systems will imprecisely approximate human judgments (already themselves imperfect), and you build appeals processes and human review pipelines to catch the mistakes. That infrastructure is expensive, slow, and increasingly unsustainable as content volumes grow and AI-generated content floods every platform.

For the past three years, I’ve explored how artificial intelligence might be used to address the technical constraints shaping this result. That effort started at Stanford and is now the mission of Zentropi, a startup I co-founded with Samidh Chakrabarti. Our goal is to develop an AI content classification platform that changes how content moderation works. Our recent experiments suggest such systems have reached the point at which they could be deployed at scale.