AI-ready data: Preparing government data for an AI-driven world
A look at why government data needs to change to keep pace with AI, and the standards that would get it there.
As large language models and AI chat tools become one of the fastest-growing ways people get information, the accuracy of what those tools say about government and public life depends on the data behind them. For decades, government statistical agencies have served as the backbone of authoritative information for the public and policymakers alike. But most of that data was built for a pre-AI world, and gaps in standardization, availability, and consistency limit how well AI systems can incorporate it and provide accurate facts.
We see the strongest path forward as a systems-based one: pairing large language models with information-retrieval methods — like retrieval-augmented generation, agents, or web search — and holding the results to a clear set of accuracy evaluations. Four criteria describe what it takes to get government data to that standard: accessible, understandable, accurate, and open.
Data summary
- Machine-readable access matters most. We recommend RESTful APIs with minimal rate limits, structured formats like JSON and CSV instead of PDFs, and metadata — including update timestamps — that can be retrieved programmatically through expanded, centralized catalogs like data.gov.
- Documentation needs to catch up to AI's needs. Data dictionaries, taxonomies, and "Data Cards" (modeled on the model cards used in AI transparency work) should spell out a dataset's origin, known biases, and intended uses, alongside semantic tagging that supports retrieval-augmented generation.
- Accuracy requires public benchmarks. We call for agencies to publish their own LLM evaluation datasets, regularly test AI models against those benchmarks, and maintain audit trails to catch errors or misuse.
- Openness depends on licensing and privacy design working together. Permissive licenses (CC0, ODC-BY), privacy safeguards like differential privacy, and free, anonymous API access all need to coexist with plain-language documentation of suppressed data, such as small-county figures.
- The Department of Commerce's continuously updated economic indicators are the example we point to for how this should work in practice — AI systems should fetch those figures live rather than rely on older, embedded statistics.
Keep exploring
- State of the Facts 2024: Americans’ views on data, election information, and AI - Americans are more likely to use “new media” as news sources, but less likely to trust them.
- How many Americans are using AI — and how? - 35% of American households are using AI to find information. 14% of people trust that information.
- How to use government data - The US government generates data on everything from unemployment rates to the status of endangered marine mammals. Here’s why that’s important.
- Why does USAFacts use US government data? - Here's why we believe it's critical to give Americans a site where the data is 100% sourced from the government itself.