When you purchase through links on our site, we may earn an affiliate commission.Heres how it works.
However, this still falls significantly short of even four-year-old models likeOpenAIs GPT-3 which featuredover 175 billion parameters.
This project started out with research on the open-source implementation and scaling of globally distributed AI model training.
Size isn’t everything though.
This reduces the risk of only a few large companies having access to this advanced technology.
For now, users can only contribute to the project through the companys own platform.
But in the future, you should be able to contribute to the models training with your own hardware.
The training is made possible through separate clusters of devices that process data to train the AI model.
The training framework can also handle nodes joining or leaving without leading to system crashes.
Delays with this catching-up process have been solved by having new nodes request checkpoints from their peers.
Its mainly training on a Hugging Face dataset called FineWeb-Edu which contains content from educational web pages.
More from Tom’s Guide
