Meta Drops a 405-Billion-Parameter Model You Can Actually Run
Meta open-sourced Llama 3.1 405B under a permissive license. The weights match GPT-4 level performance on several benchmarks and can be downloaded for local inference or rented on cloud GPUs for roughly two dollars per million tokens.
This shows that frontier capability no longer requires a paid API subscription. Users gain the ability to fine-tune on private data and keep outputs inside their own infrastructure. The change encourages teams to evaluate cost and privacy trade-offs before defaulting to closed models.
Hugging Face hosts the model weights and reports more than 180,000 downloads in the first week. Independent developers have already published fine-tuned variants for legal document review that run on a single 8xA100 node with 15 percent lower latency than the base model.
Step 1: Go to huggingface.co/meta-llama/Meta-Llama-3.1-405B and accept the license. Step 2: Install the Hugging Face Transformers library and load the model with 4-bit quantization. Step 3: Run a short inference script; expect coherent 4K-token responses at roughly 12 tokens per second on an A100 GPU.