In under two minutes, Meta’s CEO, Mark Zuckerberg, laid out the company’s AI plans, which included a plan to build an artificial intelligence system with the equivalent of 600,000 Nvidia GPUs.
“I’m bringing Meta’s AI research efforts closer together to support our long-term goals of building general intelligence, open sourcing it responsibly, and making it available and useful to everyone in all of our daily lives,” Zuckerberg said in a video posted on Twitter.
Zuckerberg’s announcement was an updated roadmap of Meta’s AI plans, which is built around the upcoming Llama3, which is currently being trained. It will succeed last year’s Llama2 model weights and tokenizers, which were major successes with just under 2 million combined downloads on Huggingface. Open-source developers have also released thousands of Llama2 forks.
Llama3 will compete with Google’s recently released Gemini model and OpenAI’s GPT-4 and upcoming GPT-5 models. OpenAI CEO Sam Altman has not talked about GPT-5 yet but has hinted that it would be much easier to handle text, speech, and images by supporting more data sources.
“We are building an absolutely massive amount of infrastructure to support this by the end of this year. We will have around 350,000 Nvidia H100 or around 600,000 H100 equivalents of compute if you include other GPUs,” Zuckerberg said.
That means Meta’s total number of GPUs will be more than the population of around 70 countries. Meta will also be using AMD’s MI300X GPUs. In fact, Meta deployed servers with the MI300X GPU in record time.
“[MI300X] leverages the OCP module, standard, and platform, which has helped us adopt it in record time. In fact, MI300X is one of the fastest deployment solutions in Meta’s history,” said Ajit Mathews, senior director of engineering at Meta, in an on-stage appearance at an AMD event last year.
Nvidia will start shipping the H200 GPUs next year to supplant its H100 GPUs. There are signs that Nvidia will scale up H100 GPU shipments, which have been in big demand, as it ramps up H200 shipments.
© Copyright RawNews1st
