AI 'Trustwashing' Changes How Consumers Judge Credibility

Generative AI is changing how people decide what to believe. Large language models can produce fluent, confident answers across a wide range of topics at the drop of a hat, and many readers take that fluency as a proxy for truth. This is especially the case when those answers appear inside tools we already trust for everyday work, such as search engines, email clients, or document editors. Generative AI outputs gain credibility simply by appearing in familiar and trusted places where people search for information. Together, both the fluency of large language models and the prominent placement of their outputs create an environment where false or misleading information can appear reliable.

Users of these generative models are not always equipped to spot when AI output is wrong or misleading. In short, there is often a mismatch between how accurate users believe a model is and its actual accuracy. Studies show that models can produce incorrect answers even when they contain the right information, they simply fail to retrieve the information and yet still present their output with convincing confidence. On the human side, research in human-computer interaction shows that people tend to trust automated systems too readily. This so-called “automation bias” is especially strong when the system appears competent and users are trying to save time. These dynamics are amplified by “sycophancy,” where models optimized with human feedback echo a user’s stated views, thereby keeping its answers agreeable, and trusted, even when incorrect.