Microsoft details its first AI SoC | Maia 100 promises affordable AI acceleration

Microsoft Unveils Maia 100: Its First Custom AI Accelerator The AI Chip Landscape

Maia 100 promises Currently, Nvidia’s AI chip segment is still the king of the hill with a 75-90% stake of the market. Nevertheless, the major hardware players and hyperscale’s are embarking on the development of their own set of hardware so as to contest this stand. Microsoft has now exposed information about its custom AI chip, representing a large leap in this rivalry objection.

Introducing Maia 100

This year’s Hot Chips conference gave me the opportunity to meet the Maia 100 and learn about its technology at Microsoft. This piece of machinery is a chip and it has a very special architecture which uses some sort of custom server boards, racks, and different software. It is designed to be the cost-effective solution for AI. Besides, it can easily run in Microsoft’s Azure data centers while implementing OpenAI models.

Technical Specifications

• • It is based on TSMC’s 5nm process node

• • 500W is a standard power draw, but the card can support up to 700W TDP

• • 64GB of HBM2E memory

• • High-speed tensor unit (16xRx16) for rapid processing

• • Support for various file types that include Microsoft’s MX format

Key Features

The Maia 100 is a great example of AI technology. Here are some brief points of the hardware:

• • A configurable superscalar engine with a Computer ISA

• • FP32 and BF16 data type support

• • DMA engine for different tensor sharding schemes

• • Hardware semaphores that allow the segmentation of processes

Developer Support

Microsoft’s new move, Maia SDK is being developed which I believe will be the most important thing for developers. This bundle of software includes:

• • Porting tools from PyTorch and Triton

• • Framework integration and developer tools

• • Two programming models and compilers

• • Optimized compute and communication kernels

• • Maia Host/Device Runtime for hardware abstraction
  • If you want to know more about those in depth, I recommend “Inside Maia 100” blog post written by Microsoft. It gives you comprehensive information about the SDK, backend network protocol, and optimization techniques.