High Bandwidth Memory

Interface

The HBM DRAM is tightly coupled to the host processor die with a distributed interface. The interface is divided into independent channels. The channels are completely independent of one another and are not necessarily synchronous to each other. HBM DRAM uses a wide-interface architecture to achieve high-speed, low-power operation. HBM1 DRAM used a 500 MHz differential clock CK_t / CK_c (where the suffix "_t" denotes the "true", or "positive", component of the differential pair, and "_c" stands for the "complementary" one). Commands are registered at the rising edges of CK_t and CK_c. Each channel interface maintained a 128‑bit data bus operating at this double data rate (DDR). HBM1 supported transfer rates of 1 GT/s per pin (transferring 1 bit), yielding an overall package bandwidth of 128 GB/s. ^[22]

HBM2

The second generation of High Bandwidth Memory, HBM2, also specified up to eight dies per stack and doubled pin transfer rates up to 2 GT/s. Retaining 1024‑bit wide access, HBM2 was able to reach 256 GB/s memory bandwidth per package. The HBM2 spec allowed up to 8 GB per package. HBM2 was predicted to be especially useful for performance-sensitive consumer applications such as virtual reality.^[23]

On January 19, 2016, Samsung announced early mass production of HBM2, at up to 8 GB per stack.^[24][25] SK Hynix also announced availability of 4 GB stacks in August 2016.^[26]

• 
  HBM2 DRAM die
• 
  HBM2 controller die
• 
  The HBM2 interposer of a Radeon RX Vega 64 GPU, with removed HBM dies; the GPU is still in place

HBM3

In late 2020, Micron unveiled that the HBM2E standard would be updated and alongside that they unveiled the then next standard known as HBMnext (later renamed to HBM3). This was to be a big generational leap from HBM2 and the replacement to HBM2E. This new VRAM would have come to the market in the Q4 of 2022. This would likely have introduce a new architecture as the naming suggests.

While the architecture might have been overhauled, leaks pointed to performance similar to the updated HBM2E standard. This RAM was likely to be used mostly in data center GPUs.^{[33][34][35][36]}

In mid 2021, SK Hynix unveiled some specifications of the HBM3 standard, with 5.2 Gbit/s I/O speeds and bandwidth of 665 GB/s per package, as well as up to 16-high 2.5D and 3D solutions.^[37][38]

On 20 October 2021, before the JEDEC standard for HBM3 was finalised, SK Hynix was the first memory vendor to announce that it had finished development of HBM3 memory devices. According to SK Hynix, the memory would have run as fast as 6.4 Gbit/s/pin, double the data rate of JEDEC-standard HBM2E, which formally topped out at 3.2 Gbit/s/pin, or 78% faster than SK Hynix's own 3.6 Gbit/s/pin HBM2E. The devices supported a data transfer rate of 6.4 Gbit/s and therefore a single HBM3 stack might have provided a bandwidth of up to 819 GB/s. The basic bus widths for HBM3 remained unchanged, with a single stack of memory being 1024-bits wide. SK Hynix would offer this memory in two capacities: 16 GB and 24 GB, aligning with 8-Hi and 12-Hi stacks respectively. The stacks consisted of 8 or 12 16 Gb DRAMs that are each 30 μm thick and interconnected using Through Silicon Vias (TSVs).^[39][40][41]

According to Ryan Smith of AnandTech, the SK Hynix first generation HBM3 memory had the same density as their latest-generation HBM2E memory, meaning that device vendors looking to increase their total memory capacities for their next-generation parts would need to use memory with 12 dies/layers, up from the 8 layer stacks they typically used until then.^[39] According to Anton Shilov of Tom's Hardware, high-performance compute GPUs or FPGAs typically used four or six HBM stacks, so with SK Hynix's HBM3 24 GB stacks they would accordingly get 3.2 TB/s or 4.9 TB/s of memory bandwidth. He also noted that SK Hynix's HBM3 chips were square, not rectangular like HBM2 and HBM2E chips.^[40] According to Chris Mellor of The Register, with JEDEC not yet having developed its HBM3 standard, might have meant that SK Hynix would have needed to retrofit its design to a future and faster one.^[41]

JEDEC officially announced the HBM3 standard on January 27, 2022.^[8] The number of memory channels was doubled from 8 channels of 128 bits with HBM2e to 16 channels of 64 bits with HBM3. Therefore, the total number of data pins of the interface was still 1024.^[42]

In June 2022, SK Hynix announced they started mass production of industry's first HBM3 memory to be used with Nvidia's H100 GPU expected to ship in Q3 2022. The memory would provide H100 with "up to 819 GB/s" of memory bandwidth.^[43]

In August 2022, Nvidia announced that its "Hopper" H100 GPU would ship with five active HBM3 sites (out of six on board) offering 80 GB of RAM and 3 TB/s of memory bandwidth (16 GB and 600 GB/s per site).^[44]

HBM-PIM

In February 2021, Samsung announced the development of HBM with processing-in-memory (PIM). This new memory brings AI computing capabilities inside the memory, to increase the large-scale processing of data. A DRAM-optimised AI engine is placed inside each memory bank to enable parallel processing and minimise data movement. Samsung claims this will deliver twice the system performance and reduce energy consumption by more than 70%, while not requiring any hardware or software changes to the rest of the system.^[55]

HBM4

In July 2024, JEDEC announced its preliminary specifications for HBM4.^[56] It lowered the data rate per pin to 6.4 Gbit/s/pin (the level of HBM3) but since it now uses a 2048-bit interface per stack (doubling that of the previous generations), it still achieves greater (1.6TB/s)^[57] data rate per stack than HBM3E. It will also allow 4GB layers (yielding 64GB in 16-layer configurations).

In April 2025, JEDEC released the official HBM4 specification.^[9] It supports transfer speeds of up to 8 Gb/s across a 2048-bit interface, with total bandwidth of up to 2 TB/s, and stack height of 4 to 16, with DRAM die densities of 24Gb or 32Gb, allowing for capacities up to 64GB. HBM4 is backwards compatible with HBM3 controllers. Samsung, Micron, and SK Hynix contributed to the standard.^[10]

Background

Die-stacked memory was initially commercialized in the flash memory industry. Toshiba introduced a NAND flash memory chip with eight stacked dies in April 2007,^[58] followed by Hynix Semiconductor introducing a NAND flash chip with 24 stacked dies in September 2007.^[59]

3D-stacked random-access memory (RAM) using through-silicon via (TSV) technology was commercialized by Elpida Memory, which developed the first 8 GB DRAM chip (stacked with four DDR3 SDRAM dies) in September 2009, and released it in June 2011. In 2011, SK Hynix introduced 16 GB DDR3 memory (40 nm class) using TSV technology,^[3] Samsung Electronics introduced 3D-stacked 32 GB DDR3 (30 nm class) based on TSV in September, and then Samsung and Micron Technology announced TSV-based Hybrid Memory Cube (HMC) technology in October.^[60]

JEDEC first released the JESD229 standard for Wide IO memory,^[61] the predecessor of HBM featuring four 128 bit channels with single data rate clocking, in December 2011 after several years of work. The first HBM standard JESD235 followed in October 2013.

Development

The development of High Bandwidth Memory began at AMD in 2008 to solve the problem of ever-increasing power usage and form factor of computer memory. Over the next several years, AMD developed procedures to solve die-stacking problems with a team led by Senior AMD Fellow Bryan Black.^[62] To help AMD realize their vision of HBM, they enlisted partners from the memory industry, particularly Korean company SK Hynix,^[62] which had prior experience with 3D-stacked memory,^[3][59] as well as partners from the interposer industry (Taiwanese company UMC) and packaging industry (Amkor Technology and ASE).^[62]

The development of HBM was completed in 2013, when SK Hynix built the first HBM memory chip.^[3] HBM was adopted as industry standard JESD235 by JEDEC in October 2013, following a proposal by AMD and SK Hynix in 2010.^[6] High volume manufacturing began at a Hynix facility in Icheon, South Korea, in 2015.

The first GPU utilizing HBM was the AMD Fiji which was released in June 2015 powering the AMD Radeon R9 Fury X.^[4][63][64]

In January 2016, Samsung Electronics began early mass production of HBM2.^[24][25] The same month, HBM2 was accepted by JEDEC as standard JESD235a.^[7] The first GPU chip utilizing HBM2 is the Nvidia Tesla P100 which was officially announced in April 2016.^[65][66]

In June 2016, Intel released a family of Xeon Phi processors with 8 stacks of HCDRAM, Micron's version of HBM. At Hot Chips in August 2016, both Samsung and Hynix announced a new generation HBM memory technologies.^[67][68] Both companies announced high performance products expected to have increased density, increased bandwidth, and lower power consumption. Samsung also announced a lower-cost version of HBM under development targeting mass markets. Removing the buffer die and decreasing the number of TSVs lowers cost, though at the expense of a decreased overall bandwidth (200 GB/s).

Nvidia's P100 and H100 devices were the first products to utilize HBM2 and HBM3 memory, respectively;^[69] AMD's Instinct MI400 series, launched in 2026, is among the first devices to utilize HBM4.^[70]