FP8 precision training on consumer-grade RTX 5090 GPUs enables dramatic cost reduction for language model pretraining

we train a collection of Puro-2B models from scratch on up to 1.4 trillion tokens with FP8 precision on consumer-grade RTX 5090 GPUs

http://arxiv.org/abs/2608.27370v1