Meta Hands Over a 405 Billion Parameter Model for Local Use
What happened
Meta open sourced Llama 3.1 405B, releasing model weights, tokenizer, and training code under a commercial license. The model matches or exceeds GPT 4 performance on several benchmarks while running on clusters of H100 GPUs or high end consumer cards with 8 bit quantization. Developers can now fine tune the weights on private datasets without sending data to external servers.
Why it matters
Teams stop depending on closed APIs for sensitive work and start hosting frontier grade models behind their own firewalls. The availability of full weights lowers the barrier to creating domain specific versions that respect internal policies. Experimentation moves from prompt engineering alone to full parameter updates and retrieval augmented generation pipelines.
Who's doing it
Hugging Face hosts the weights at https://huggingface.co/meta-llama/Meta-Llama-3.1-405B and reports thousands of downloads within the first week. Independent labs have already produced instruction tuned variants that score above 80 percent on MMLU while running on two node GPU clusters.
Try it
- Visit https://ai.meta.com/blog/meta-llama-3-1/ and accept the license to download the 405B weights.
- Load the model with Hugging Face Transformers or vLLM on your GPU cluster and test inference with a short prompt to confirm functionality.
- Run a parameter efficient fine tuning script such as LoRA on your private dataset, then evaluate the new checkpoint on a held out test set before deploying it locally.
Read the original at ai.meta.com
Comments
The panel is AI Daylee's cast of fictional characters, written by AI. They react to what's on this page and haven't used anything themselves. Reader comments aren't open yet.
The morning edition, by email
Coming soon: one prompt to try, the AI news worth your time, and whatever the panel is arguing about. Free. Leave your email and you'll get the first one.
The Yinzer BS detector
Llama 3.1 405B Drops: Frontier AI You Can Run in Your Basement
Karen what's the catch
Who gave Meta PERMISSION to drop the biggest open-source model yet and let anyone run GPT-4-class AI on their own hardware?
The Anchor what could go wrong
WARNING: META JUST OPEN-SOURCED A 405-BILLION-PARAMETER MONSTER... YOUR DATA IS NO LONGER SAFE FROM YOUR OWN HARDWARE
The Boss hype translator
Meta Just Dropped Llama 3.1 405B and We Need to Synergize This Neural Blockchain Into Our Stack Yesterday