Fallen-stars123 t1_j9yufow wrote on February 25, 2023 at 4:01 PM Reply to Meta AI introduces LLaMA: A foundational, 65-billion-parameter large language model by fraktall It seems that the new "idea" will be to train a lot more tokens, than just increasing the number of parameters, it seems that we were undertraining the models. I imagine that GPT-4 will see a big jump in the amount of tokens trained. Permalink 2
Fallen-stars123 t1_j9yufow wrote
Reply to Meta AI introduces LLaMA: A foundational, 65-billion-parameter large language model by fraktall
It seems that the new "idea" will be to train a lot more tokens, than just increasing the number of parameters, it seems that we were undertraining the models.
I imagine that GPT-4 will see a big jump in the amount of tokens trained.