When Federal Agencies Pick AI Vendors, They Are Buying Different Policy Interpretations

If you are an OpenAI employee, you had an unusual view from your office this March: protesters in robot masks holding a “NO WAR STOP AI” sign and urging employees to quit. The protest captured the most dramatic fear about AI and the Pentagon, the use of AI with autonomous weapons, in battlefield decisions, and surveillance of American citizens.

But most government uses of these tools are likely to look more mundane. Agencies may use AI to summarize long documents, compare proposals, review comments, flag compliance issues, or help staff make sense of complex policy texts. Those uses sound administrative, but they still matter. When an agency changes AI vendors, it may also change what the system notices, downplays, or misses.

Take the Biden administration’s Framework for Artificial Intelligence Diffusion, the sweeping rule governing which countries can access the most powerful American AI systems, revoked by the Trump administration. If you ask ChatGPT, Claude, and Grok what this policy is about, on its face, the three chatbots agree: it is a national security document. The differences appear when you ask a more demanding question, the kind a trained policy analyst would ask. When we gave each model a detailed analytical checklist, ChatGPT and Claude identified cybersecurity safeguards, compliance requirements, and audit mechanisms embedded in the document that shape how it functions in practice. Grok, presented with the same checklist, still saw an export control document primarily. The safety and oversight provisions were invisible.