IndQA sets a new bar for cultural understanding in AI

In the early stages of large language models, success was mostly measured by English fluency and performance on benchmarks such as SQuAD or MMLU. Current expectations extend well beyond benchmark accuracy. Users now expect models to handle local traditions, cultural idioms, and region-specific knowledge reliably. A Tamil speaker inquiring about Pongal customs, a Gujarati speaker seeking general information on family law norms, or a Kannada speaker asking about local festivals expects culturally grounded responses rather than literal translations or generic text.

This shift in expectations has revealed a persistent gap in modern AI systems. They can produce surface-level fluency in many languages, but still lack the cultural context necessary for accurate interpretation. This gap highlights the need for a different class of benchmark.

In November 2025, OpenAI introduced IndQA, a benchmark designed to assess the ability of AI systems to understand and reason about Indian languages and cultural contexts. It is not a translation dataset. It is a reasoning challenge built from the ground up by human experts across India. It tests whether models truly understand people, customs, stories, traditions, and daily life in South Asia.

This newsletter examines what IndQA is, its significance, the current results it reveals, and the opportunities that lie ahead for builders seeking to create culturally aware AI systems.

What is IndQA?#

IndQA (Indian Question-Answering) is a large benchmark designed to evaluate AI models on culturally grounded questions from India. According to OpenAI, the dataset comprises 2,278 questions spanning 12 languages and 10 cultural domains. These 12 languages include 11 Indian languages and English, which is included because of its widespread use in Indian education, media, and public life.