The Denominator Problem in AI Governance

With the European Union’s AI incident reporting mandate taking effect in August and US states from New York to Colorado enacting their own AI reporting requirements this year, a basic measurement problem threatens to undermine all of these efforts before they begin. The OECD AI Incidents and Hazards Monitor catalogs thousands of documented harms across domains as varied as autonomous vehicles, deepfakes, algorithmic discrimination, and health chatbots.

The AI Incident Database, MIT FutureTech, and Arcadia Impact are collaborating on a project that attempts to move beyond raw incident counting toward something more analytically useful: classifying AI incident types into phases—from rare occurrence through rapid expansion to endemic patterns—by estimating factors such as the rate of harm and the scale of deployment. I became aware of what I call the denominator problem through my advisory work on this project, and I believe it names the single most important unsolved measurement challenge in AI governance today.