TikTok parent company ByteDance is training a new AI model that could rival Anthropic’s Mythos, reaching up to 10 trillion parameters, as the TikTok owner steps deeper into the race to build advanced artificial intelligence systems capable of competing with leading US AI labs.
According to a Financial Times report, the Chinese company has begun pre-training the model, a process expected to take three to six months before it moves to fine-tuning and testing.
While the model size is yet to be determined, the scale reported places it closer to advanced AI systems developed by companies such as Anthropic and OpenAI.
Damilare Odedina
What does a 10 trillion parameter AI model mean?
Parameter count refers to the number of values an AI model learns during training. For context, more parameters generally allow a model to capture more complex patterns and knowledge.
At an estimated 10 trillion parameters, the TikTok owner’s model would dwarf Moonshot AI's Kimi K3, which, at the time of writing, is China's largest released model with 2.8 trillion parameters. It would also approach or exceed industry estimates for Anthropic's Mythos 5, whose most advanced models are estimated by industry analysts to contain trillions of parameters.
Still, researchers caution that bigger does not always mean better, as model performance also depends on training data, architecture, reinforcement learning, and optimization techniques, meaning a smaller, well-trained model can outperform a much larger one on many tasks.
Why ByteDance is betting on scale
Unlike several rivals that have relied on model distillation, the Chinese company is reported to be pursuing independent model development. The company believes building its own foundation model from the ground up offers the best chance of achieving world-leading performance over the long term.
Subscribe for free to continue reading this article
Subscribe SubscribeAlready Have an Account? Log In
