Sarvam AI plans trillion-parameter model, raises $234 million

Bengaluru-based startup Sarvam AI has announced plans to build a one-trillion parameter foundational model. The company has opened a San Francisco office and a Bay Area research lab, raising $234 million in the first close of its series-B funding round at a valuation of $1.5 billion, The Hindu reports. Foundational models are general-purpose systems that can perform new tasks without explicit programming, but their scope is predefined by training datasets.

Sarvam AI recently launched a developer platform, Epoch Builder Edition, offering 7 billion and 70 billion parameter models trained on 2-trillion tokens, supporting over 10 Indian languages. The company has also partnered with IBM to develop sovereign AI technologies in India. The Hindu notes that building a sovereign platform faces challenges around hardware acquisition, elite engineering talent, and training datasets, as pre-training can cost over $100 million per run and requires tens of thousands of GPUs.

However, Sarvam AI has an advantage in data efficiency with a custom tokenizer that reduces token fertility for Indic languages to 1.2-2 tokens per word, compared to 8 tokens per word with Western models. The development has clear geopolitical implications, The Hindu says, as the US and China compete with bleeding-edge models while other nations rush to build AI infrastructure.