Reddit user runs massive AI model on Optane memory sticks
A Reddit enthusiast has demonstrated running a massive artificial intelligence model using only second-generation Optane memory sticks. The user executed Moonshot AI’s Kimi K3, containing 2.8 trillion parameters, at a speed of 0.85 tokens per second. This performance was achieved through a 1-bit quantization method on the hardware.
The experiment highlights the potential of using non-volatile memory for running large language models.
Read the full coverage
This brief was written with AI from the headlines and excerpts published by the outlets above, and may contain errors. Follow the links for the original reporting.