AdvertisementAI leaders are rethinking data-heavy training for large language models. Traditional models scale linearly with data, but this approach may hit a dead end. Now, AI leaders are rethinking the conventional wisdom about how to train large language models. The focus on training data arises from research showing that transformers, the neural networks behind large language models, have a one-to-one relationship with the amount of data they're given. The money going into AI has largely hung on the idea that this scaling law "would hold," Scale AI CEO Alexandr Wang said at the Cerebral Valley conference this week, tech newsletter Command Line reported.

---

Persons: Alex Voica, Mohamed, Alexandr Wang, It's, Aidan Gomez, Gomez, Richard Socher, we're, Kevin Scott, Waleed Kadous, Uber Organizations: Meta, Google, University of Artificial Intelligence, Command, Microsoft, Sequoia Capital's, OpenAI's o1, o1 Locations: University, ChatGPT, gpt