Nemotron-pretraining-Indic-v1: 2.8T tokens, open pipeline

NVIDIA and AI4Bharat have released Nemotron-pretraining-Indic-v1, an open pretraining corpus of approximately 2.8 trillion tokens spanning 22 Indian languages, alongside the GPU-accelerated data generation pipeline used to create it. The announcement, published on October 16, 2026, represents a substantial expansion of openly available training material for building AI models that serve Indian language speakers.

The 22 languages are those listed in India’s Constitution and serve as the first language for roughly 97 percent of Indians, representing nearly 1.2 billion people according to the 2011 Census. The corpus includes native-script data for all 22 languages and romanized text for twelve of them, spanning 34 language-script configurations. According to the announcement, the dataset contains 2,601.0 billion native-script tokens and 224.6 billion romanized tokens when counted with the Gemma-4 tokenizer.

What changed

NVIDIA partnered with IIT Madras, India’s premier engineering institute, and its AI4Bharat research lab to produce this release. AI4Bharat, which builds open-source datasets and models for Indian languages, had previously released Sangraha, an open pretraining corpus for 22 Indian languages. The new Nemotron corpus significantly expands that foundation and introduces an open pipeline that other teams can use to generate additional data from their own source materials.

The dataset draws from 15 subject-matter subsets, including web text, question-answer material, mathematics, and code. The pipeline translates English source documents into Indian languages while preserving technical content such as equations, tables, and code. Each record traces back to its original English source document, providing clear provenance for auditing and building on the work.

The workflow differs between language groups. For twelve languages, NVIDIA used a four-stage pipeline: full-document translation, quality assessment against the English source, targeted repair of low-scoring outputs, and conversion of accepted translations into Latin script. The team selected Gemma-4-31B-it for translation and quality evaluation, comparing models on complete documents to verify translation quality and preservation of technical content. Outputs that scored below 75 out of 100 on a rubric based on the open Multidimensional Quality Metrics framework were either repaired or excluded. This process recovered 438.4 billion tokens that might otherwise have been discarded, according to the announcement.

For the remaining ten languages, NVIDIA adapted the workflow using Indic Translate, a document-level translation model developed by Bodhan AI, a nonprofit center incubated at IIT Madras. This approach addressed long-document quality and script-stability issues identified during testing. For most of these languages, the pipeline followed the same four-stage pattern. For longer Santali documents, the team translated overlapping windows and reconstructed the full document. For Manipuri text, translation occurred into Bengali script before conversion to Meitei Mayek.

The romanized data creation for twelve languages employed a two-step approach. Native-script translations provided the starting point for Latin-script conversion, rather than direct translation from English. According to the announcement, this two-step method produced fewer hallucinated additions and less residual English than generating romanized text directly from English sources.

Why it matters

The release addresses a documented gap in AI training infrastructure. Mitesh Khapra, who heads AI4Bharat and was named to the 2025 TIME100 AI list, identified the core problem: “we do not have enough data for Indian languages.” Large language models built for English benefit from vast amounts of pretraining data, but comparable datasets for Indian languages have been scarce, limiting the capability and availability of language models that serve Indian speakers.

By releasing both the corpus and the generation pipeline, NVIDIA and AI4Bharat make large-scale Indic data generation more accessible to developers. Teams can use the corpus directly for pretraining, adapt the prompts and quality checks to their own needs, or bring their own source documents into the workflow. This approach gives model builders a path to create and scale language technology across Indian languages at a level previously unavailable in open form.

The data complements naturally authored Indic text, which remains important for local knowledge and cultural context. According to the announcement, the synthetic material brings “a broad mix of curated English material, including mathematics, code, and question-answer content” into Indian languages, giving teams more training material for building models that can explain technical concepts, work through problems, and answer questions in those languages.

NVIDIA also incorporated 10 billion Indic tokens from this corpus during continued pretraining of Nemotron 3.5 Super VL 120B, drawn from a twelve-language subset combined with natural Indic text.

What to test

Before adopting this corpus for production training, practitioners should verify several claims and design choices:

• Confirm that the 75-out-of-100 quality threshold for retained outputs matches your own accuracy and fluency requirements. The rubric covers meaning, fluency, terminology, style, and formatting, but your use case may weigh these factors differently.

• Test whether the balance of synthetic versus natural Indic text suits your language and domain. The corpus emphasizes technical content such as code and mathematics translated from English. Verify this distribution aligns with your target application.

• Validate the tokenization impact. The announcement states that corpus token counts depend on the tokenizer used and should be read separately from training-token counts. Experiment with your own tokenizer to understand how the corpus will scale during actual training.

• Examine script conversion quality, especially for romanized variants of the twelve dual-script languages. The announcement describes the two-step approach (native-script translation, then conversion to Latin script) but independent verification of accuracy is warranted.

• Review the traceability features. Each record includes a source identifier. Confirm this provenance metadata meets your audit and reproducibility requirements.

• Test the performance impact of different language mixtures. The corpus includes 34 language-script configurations with varying token counts. Measure how different combinations affect downstream model performance.

The conclusion

The Nemotron corpus and open pipeline represent a substantive shift in the availability of Indic language pretraining infrastructure. The decision to release the generation pipeline alongside the data itself is significant, as it enables the community to extend the work beyond the initial 2.8 trillion tokens and adapt the workflow to new domains and source materials.

Practitioners working on language models for Indian languages now have an open foundation for pretraining at scale. The next questions are adoption rate and iterative improvement. Watch for community contributions that extend the corpus, adapt the pipeline to new source domains, or experiment with different quality thresholds and language mixtures. Also observe whether this release influences other organizations to open similar resources for non-English languages.

Related reading

• Gemini 4 Argon: $2 per million input tokens, 95% cached discount
• NVIDIA Vera Rubin Architecture Advances AI Factory Standards