Reddit user runs massive AI model on Optane memory sticks

Published · Updated · Based on reporting by Tom's Hardware

A Reddit enthusiast has demonstrated running a massive artificial intelligence model using only second-generation Optane memory sticks. The user executed Moonshot AI’s Kimi K3, containing 2.8 trillion parameters, at a speed of 0.85 tokens per second. This performance was achieved through a 1-bit quantization method on the hardware.

Why it matters

The experiment highlights the potential of using non-volatile memory for running large language models.

Read the full coverage

More on: AI · #Reddit

This brief was written with AI from the headlines and excerpts published by the outlets above, and may contain errors. Follow the links for the original reporting.