DeepSeek Shows Necessity Is The Mother of Invention

While the rest of us were recovering from heavy Christmas dinners and too much time with relatives, Chinese quant trading firm High-Flyer Capital Management dropped the latest update on Thursday to DeepSeek, its open-weight large language model. Senior researchers at American firms including OpenAI and Meta Platforms almost immediately started investigating the Chinese rival’s technology and its claims about what little hardware it took to produce the model.

High-Flyer says it used just 2,048 Nvidia H800 chips and spent $5.5 million to train the model. The H800 is a dumbed-down version of the H100 chip that U.S. authorities allow Nvidia to sell in China. The result was a model with scores on popular benchmarks that in some cases beat models from OpenAI, Anthropic and Meta. By comparison, Meta said it trained its Llama 3 405B model, released in July, using 16,000 H100s, which are more expensive to run than the H800 chips.

High-Flyer said it used less precise computations than what you’d typically use during model training (“FP8 mixed precision training”) and designed the system’s algorithms, software and hardware to work together to make it easier to share data across chips. (By the way, “open-weight” refers to models that disclose their weights, or the settings that determine how a model responds to queries, but not information like training data.)

This isn’t the first time American AI researchers have been impressed with Chinese models. Despite restrictions on the sale of advanced Nvidia chips in China, researchers there have been able to make do with weaker and older-generation chips. Their success proves once again that necessity is the mother of invention—and that American researchers probably aren’t getting as much out of Nvidia’s hardware as they could be.