Raw LLM Responses

> How is anyone going to enforce it without obliterating privacy on the internet? Pandora’s box is already open. You need millions in hardware and millions in infrastructure and energy to run foundation training runs. ------------------------ LLaMA 2 65b, took 2048 A100s 21 days to train. For comparison if you had 4 A100s that'd take about 30 years. These models require fast interconnects to keep everything in sync. Assuming you were to do the above with 4090s to equal the amount of VRAM (163840GB, or 6826 rtx4090's) would still take longer because the 4090s are not equipped with the same card to card high bandwidth NVlink bus. So you need to have a lot of very expensive specialist hardware and the data centers to run it in. You can't just grab an old mining rigs and do the work. This needs infrastructure. And remember LLaMA 2 is not even a cutting edge model, it's no GPT4 it's no Claude 3 ------- It can be regulated because you need a lot of hardware and infrastructure all in one place to train these models, these places can be monitored. You cannot build foundation models on your own PC or even by doing some sort of P2P with others, you need a staggering amount of hardware to train them.