Following Nvidia and AMD's footsteps, cloud service provider AWS has started to design its proprietary artificial intelligence (AI) chips. This move is deemed beneficial for the growth of the high bandwidth memory (HBM) market dominated by South Korean companies such as Samsung and SK Hynix.
According to industry sources quoted by Korean media Digital Times, the AI training chip Trainium 2 unveiled in AWS's re:Invent 2023 conference will let both Samsung and SK Hynix supply the HBM3 it needs. It is reported that Trainium 2, compared to the previous generation product, improves training performance by four times, featuring four 24GB HBM3 units, supporting a memory capacity of 96GB.
HBM was planned to be used in high-performance computing applications, but it has become a boon for AI servers following ChatGPT's rapid rise to fame over the past year. AWS used to purchase GPUs directly from Nvidia and AMD, but their supplies have been stripped by demand, and prices are too hefty. AWS now develops its own AI training chips.
It is worth noting the industry predicted that AWS would only use Samsung's HBM3. However, the current product also incorporates SK Hynix's HBM3, adopting a diversified supply chain strategy. Reportedly, SK Hynix, in addition to serving IC design companies such as AMD and Nvidia, will also supply HBM for Google TPU, and with AWS becoming a new customer, it has successfully ensured the supply in the data center domain.
Samsung is rumored to supply HBM required for data center applications to AWS, Google, and others. The competition in the HBM3 market is expected to intensify in the future.
Currently, Samsung and SK Hynix are preparing for the next generation of products. SK Hynix plans to start mass production of the fifth-generation product, HBM3E, in the first half of 2024. They are increasing production capacity by adding orders for Through Silicon Via (TSV) process equipment and Dual TC Bonder. To expand HBM production, Samsung is acquiring the building and equipment of its subsidiary, Samsung Display (SDC)'s Cheonan factory. The HBM3E is planned to go into mass production immediately after completing sample testing.
Article translated by Judy Lin
