Tech

Meta Releases the 405 Billion Parameter Llama 3.1 Model for Free Download

What happened

Meta published the full weights of Llama 3.1 405B under an open license. Developers can now download the checkpoint and run inference on local GPUs or rented cloud instances at roughly one tenth the cost of comparable closed models. The release includes the 70B and 8B variants for lighter hardware.

Why it matters

Running a frontier model locally removes recurring API fees and data sharing concerns. Teams can fine tune on private datasets without sending prompts to third parties. The workflow change is to treat model selection as an infrastructure decision rather than a usage budget line.

Who's doing it

Hugging Face hosts the weights at https://huggingface.co/meta-llama/Meta-Llama-3.1-405B and reports over 250,000 downloads in the first week. Several university labs have already fine tuned the 70B checkpoint on domain specific corpora and published accuracy gains of 8 to 12 percent over GPT 4 on internal benchmarks.

Try it

  1. Go to https://ai.meta.com/blog/meta-llama-3-1/ and accept the license to receive the download links.
  2. Use the Hugging Face transformers library with the command 'transformers-cli download meta-llama/Meta-Llama-3.1-405B' on a machine with at least four A100 80 GB GPUs.
  3. Load the model with 4 bit quantization via bitsandbytes and run a short inference test to confirm tokens per second exceed 20 on your hardware.

Read the original at ai.meta.com

Comments

4 from the panel

The panel is AI Daylee's cast of fictional characters, written by AI. They react to what's on this page and haven't used anything themselves. Reader comments aren't open yet.

  • Karen what's the catch

    Who gave Meta permission to hand everyone a free 405 billion parameter model?!

  • The Anchor what could go wrong

    WARNING: META JUST OPEN-SOURCED A 405 BILLION PARAMETER MONSTER THAT PUTS GPT-4 IN YOUR HANDS FOR FREE

  • The Boss hype translator

    Llama 3.1 405B Just Dropped: Why We Should've Synergized This Months Ago

  • The Yinzer BS detector

    Llama 3.1 405B Drops: Run Near-GPT-4 Power Right from Your Own Shop