Meta's ambitious plans for Llama 4 aim to establish it as "the most advanced [model] in the industry next year," according to Mark Zuckerberg1. While specific details about Llama 4's capabilities remain undisclosed, Meta's focus on increasing compute power by tenfold suggests significant improvements in model performance and capabilities2. The company is preparing for a multi-year investment in AI infrastructure, with capital expenditures expected to rise substantially in 2025 to support future AI model training2. Although Meta does not anticipate immediate revenue from generative AI products, the long-term strategy involves developing advanced AI tools for various applications, including potential improvements in advertising personalization and content recommendations1.

Training Llama 4 is expected to require a staggering amount of computational power, potentially necessitating around 160,000 GPUs based on estimates from Llama 3's requirements1. This massive increase in compute demand reflects the growing complexity and scale of large language models. To support this expansion, Meta is significantly boosting its capital expenditures, with Q2 2024 seeing a 33% year-over-year increase to $8.5 billion2. The company's proactive approach to building AI training capacity demonstrates its commitment to maintaining a competitive edge in the rapidly evolving field of artificial intelligence.

The table above illustrates the substantial computational resources required for training top large language models (LLMs). Llama 4, Meta's upcoming model, is projected to require the highest number of GPUs at 160,000, which is a significant increase compared to its predecessor Llama 31. This massive jump in computational requirements underscores the rapid scaling of AI models and the intense competition in the field. GPT-4, developed by OpenAI, used 25,000 GPUs for training, while Grok 3, from xAI, is expected to use 100,000 GPUs2. These figures highlight the exponential growth in computational demands for advancing AI capabilities, with each new generation of models requiring substantially more resources than the last.

Meta is making substantial infrastructure investments to support its ambitious AI development plans, particularly for Llama 4. The company's capital expenditures rose by 33% to $8.5 billion in Q2 2024, primarily driven by investments in servers, data centers, and network infrastructure1. Meta is adopting a flexible approach to its AI infrastructure, allowing for the allocation of resources between generative AI training, inference, and core ranking and recommendation tasks as needed2. CFO Susan Li emphasized the company's strategy of staging datacenter sites at various development phases, enabling Meta to quickly scale up capacity while limiting long-term spending commitments3. This approach reflects Meta's forward-thinking strategy in AI development, prioritizing the ability to meet future computational demands over immediate revenue generation from generative AI products4.