😳Dataset published for training an improved alternative to LLaMa
A group of researchers from various organizations and universities (Together, ontocord.ai, ds3lab.inf.ethz.ch, crfm.stanford.edu, hazyresearch.stanford.edu, mila.quebec) are working on an open-source alternative to the LLaMa model and have already published dataset relevant to the one used to create the last one. The non-free but well-balanced LLaMa has been used as the basis for projects such as Alpaca, Vicuna and Koala.
Now RedPajama is published in the public domain - a dataset of texts containing more than 1.2 trillion tokens. The next step, according to the developers, will be the creation of the model itself, which will require serious computing power.