Meta In-House Chip Designs Include Processing for AI, Video

Meta Platforms has shared additional details on its next generation of AI infrastructure. The company has designed two custom silicon chips, including one for training and running AI models and eventually powering metaverse functions like virtual reality and augmented reality. Another chip is tailored to optimize video processing. Meta publicly discussed its internal chip development last week ahead of a Thursday virtual event on AI infrastructure. The company also showcased an AI-optimized data center design and talked about phase two of deployment of its 16,000 GPU supercomputer for AI research.

“Investors have been closely watching Meta’s investments into AI and related data center hardware as the company embarks on a ‘year of efficiency’ that will result in the firing of 21,000 employees and major cost cutting,” CNBC reports, noting that “it’s expensive for a company to design and build its own computer chips.” Meta VP infrastructure Alexis Bjorlin told CNBC that Meta is confident the investment will be worth it in terms of improved performance.

The first-generation Meta Training and Inference Accelerator, or MTIA family of chips, are capable of training AI models and running them, including for “inference” tasks, “which is when an already trained AI model makes a prediction or takes an action,” CNBC writes.

Another chip, the Meta Scalable Video Processor, or MSVP, processes and transmits video using minimal energy. Bjorlin told CNBC the company developed the MSVP because “‘there was nothing commercially available’ that could handle the task of processing and delivering 4 billion videos a day as efficiently as Meta wanted.”

Meta VP engineering Aparna Ramani explained the new hardware was developed to work effectively with Meta’s home-grown PyTorch software, which CNBC calls “one of the most popular tools used by third-party developers to create AI apps.”

The MTIA chip has already been used for Meta’s social media recommendation engines, CNBC reports, adding that the chip is fabricated by TSMC using the 7nm process. But TechCrunch says the MTIA series won’t be broadly deployed until 2025.

“Custom AI chips are increasingly the name of the game among the Big Tech players,” TechCrunch writes, noting Google created the TPU (tensor processing unit) to train its PaLM-2 and Imagen models, while Amazon offers AWS customers its Trainium and Inferentia chips, “and Microsoft, reportedly, is working with AMD to develop an in-house AI chip called Athena.”

In a news announcement that runs down its latest AI developments, Meta said it’s also deploying CodeCompose, a generative AI-based coding assistant created in-house to make its developers more productive. The company also completed the final buildout of its Research SuperCluster (RSC) supercomputer, which features 16,000 Nvidia A100 GPUs. The SuperCluster trains Meta’s LLaMA language model, among other things.

The New York Times characterizes Meta as an outlier for deciding “to give away its crown jewels” by making LLaMA, which can power chatbots, open source.

Suggesting Meta is sitting out the AI arms race between Microsoft and Google, NYT reports there are those that feel Meta’s open sharing of powerful AI tools is arguably reckless. But, NYT writes, “driven by its founder and chief executive, Mark Zuckerberg, Meta believes that the smartest thing to do is share its underlying AI engines as a way to spread its influence and ultimately move faster toward the future.”

No Comments Yet

You can be the first to comment!

Leave a comment

You must be logged in to post a comment.