Meta is building more of its computing infrastructure in-house as it expands support for artificial intelligence, video and other workloads across its services. Its latest announcements include a custom chip for recommendation systems, a processor for video handling and a larger research supercomputer.
Custom chips for targeted workloads
The Meta Training and Inference Accelerator, or MTIA, is part of a family of chips intended to speed up AI training and inference. Inference means running a model after it has been trained. The MTIA is an ASIC, a chip that combines circuits designed to carry out tasks in parallel.
Meta says the first generation, MTIA v1, was created in 2020 using a 7-nanometer process. It has 128 MB of internal memory and can scale to up to 128 GB. In a benchmark designed by Meta, the company says the chip handled low- and medium-complexity AI models more efficiently than a GPU.
For now, its role is narrower than the name might suggest: Meta says MTIA is focused on inference for recommendation workloads across its apps, rather than training models. The company says it is refining the design and that the chip improves performance per watt for those tasks. It also says memory and networking remain areas of work, particularly as larger models require processing to be spread across multiple chips.
The effort follows a change in Meta’s hardware plans. Until 2022, it relied largely on CPUs and a custom chip for AI work. The company halted a planned large-scale rollout of that earlier chip and ordered billions of dollars’ worth of Nvidia GPUs, which required significant data center redesigns. Meta then planned a more ambitious in-house chip, due out in 2025, designed for both training and running AI models.
A supercomputer for research
While the MTIA is currently aimed at recommendation inference, Meta’s Research SuperCluster, or RSC, is built around Nvidia GPUs and supports research. First unveiled in January 2022, it has completed its second-phase buildout. Meta says the system now includes 2,000 Nvidia DGX A100 systems and 16,000 Nvidia A100 GPUs.
Meta says the RSC gives researchers a platform to train models using real-world examples from the company’s production systems. Its earlier AI infrastructure relied on open source and publicly available datasets. The company says researchers use the supercomputer for work in several areas, including generative AI.
At peak, Meta says, the RSC can reach nearly 5 exaflops of computing power. The company describes it as among the world’s fastest, while the article notes that some experts question the exaflops metric and that faster supercomputers exist. Meta says it used the system to train LLaMA, its large language model. The largest version was trained on 2,048 A100 GPUs over 21 days.
The RSC also reflects the push among major technology companies to build large-scale AI computing systems. Meta points to the research benefits of its system: researchers can work with examples drawn from the company’s own services and develop models that process different kinds of information.
Video processing moves to custom hardware
Meta also revealed the Meta Scalable Video Processor, or MSVP, its first in-house ASIC designed for video on demand and live streaming. Videos uploaded to Facebook or Instagram are converted into multiple bitstreams for different devices, formats, resolutions and quality levels. Meta says MSVP can be configured for high-quality on-demand video as well as the lower latency needed for live streams.
The company plans to move most stable and mature video processing workloads to MSVP. It expects to keep software video encoding for tasks that need special customization or significantly higher quality. Meta says work is continuing on techniques to improve video quality, including preprocessing and post-processing methods.
Meta has been exploring custom server-side video chips for years. Its latest processor is intended to support existing video needs and, in the future, additional use cases such as short-form video and delivery of generative AI and augmented or virtual reality content.
Infrastructure as part of Meta’s AI push
The hardware announcements fit into Meta’s larger effort to move faster on AI, particularly generative AI. The company has spent years hiring data scientists and building AI systems used in discovery, content moderation and advertising recommendations, but it has struggled to turn some research advances into products, especially in generative AI.
Meta says owning more of its hardware and software stack gives it control over systems from data center design through training frameworks. That approach can help tailor computing equipment to specific workloads, though the company’s current disclosures also show that different jobs call for different tools: recommendation inference on MTIA, research and training on GPU-based RSC, and video processing on MSVP.
The company’s plans reach beyond infrastructure. Meta has described exploring chat experiences in WhatsApp and Messenger, visual creation tools for Facebook and Instagram, and possible future video and multimodal experiences. The custom chips and supercomputer provide computing capacity for this broader work, while Meta continues developing its products and hardware.