Microsoft has developed its own custom AI chip and Arm-based CPU to drive its Azure data centers and equip the company and its enterprise customers for a future focused on AI. The Azure Maia AI chip and Azure Cobalt CPU are set to debut in 2024, aiming to alleviate the high demand for Nvidia's H100 GPUs and enhance the performance and power management of cloud services on Azure.
The Azure Cobalt CPU features a 128-core chip built on an Arm Neoverse CSS design and is tailored for powering general cloud services on Azure. Microsoft is currently testing the Cobalt CPU on workloads like Microsoft Teams and SQL server, with plans to offer virtual machines to customers in the near future. Initial testing shows performance improvements of up to 40% over the Arm-based servers currently used for Azure.
The Maia 100 AI accelerator is designed for running cloud AI workloads, such as large language model training and inference, and will be used to power some of Microsoft's largest AI workloads on Azure, including those from the partnership with OpenAI. Manufactured on a 5-nanometer TSMC process, Maia has 105 billion transistors and supports faster model training and inference times. Microsoft's involvement in standardizing the next generation of data formats for AI models and its collaboration with Open Compute Project (OCP) reflect its commitment to adapting entire systems to the needs of AI.
Microsoft has also implemented a unique liquid-cooled server processor with the goal of enabling higher server density at greater efficiencies, thereby impacting the capacity and deployment speed of AI servers in its data centers. The company is sharing its rack designs but keeping the Maia chip designs in-house. Maia 100 is currently undergoing testing on various AI workloads, and Microsoft is not yet sharing specific performance benchmarks.
While direct comparisons with competitors have not been disclosed, Microsoft emphasizes the importance of partnerships with Nvidia and AMD for the future of Azure's AI cloud. Diversification of supply chains is a key focus for Microsoft, and its custom chips aim to lower the cost of AI for its customers while also fulfilling its own Azure cloud workload requirements.
The naming convention of Maia 100 and Cobalt 100 hints at Microsoft's intention to produce second-generation versions, although the specific roadmap has not been revealed. The company's ability to rapidly deploy Maia and the impact of these chips on AI cloud service pricing remain to be seen as Microsoft continues to develop and refine its AI-powered offerings.
