d-Matrix, a startup building AI inference chips, says Raptor, its second-generation accelerator, will stack 3D DRAM directly beneath the compute logic and arrive next year. Satyam Srivastava, a fellow at the company, laid out the plan to theCUBE at the SC26 supercomputing conference. d-Matrix first presented Raptor at Hot Chips in August as the first 3D DRAM accelerator for generative inference, aimed at the memory limits that hold back serving large models, and Tom's Hardware and ServeTheHome reported the technical detail of that talk.
In Raptor the logic die sits on top and the DRAM underneath, so the memory die doubles as the interposer. The logic is a TSMC 4nm die bonded face to face onto a custom DRAM die at a 36-micron pitch. Each card carries 32GB and 100 TB/s of bandwidth, at 0.37 picojoules per bit against roughly 2.4 for HBM4. The 32GB per card is a fraction of what HBM4 accelerators offer, so Raptor trades capacity for speed and efficiency. Srivastava summed it up as about ten times the energy efficiency and ten times the bandwidth of HBM, with capacity between SRAM and HBM. The memory is bonded to the compute rather than sitting beside it on a separate interposer, which is the design choice behind those claims.
“Our roadmap is still a work in progress, and we have a hard enough time trying to get one-high to work and work around all of the thermal issues of that.”
Sudeep Bhoja, co-founder and CTO of d-Matrix, at Hot Chips, at Hot Chips · tomshardware.com ↗
The performance figures are projections. A paper with the University of British Columbia estimates about 4.7 times the per-card throughput of HBM designs. Those figures come from early silicon, not shipping hardware: it points to 988 tokens per second per user on the 2.8-trillion-parameter Kimi K3 at a million tokens of context, and 72 cards would hold 2.3TB, enough for that model's weights at 4-bit for about 54 concurrent users. Tom's Hardware notes that no third party has measured Raptor, so every number here comes from d-Matrix and its early silicon rather than from independent testing.
There are trade-offs. Density is 11.4 MB/mm2, about half of HBM4's 21.9 to 26.3, and HBM4 accelerators carry 192GB to 288GB per card, so a Raptor system needs many more cards to hold a large model. The DRAM is built for a 105C junction temperature and refreshed eight times more often, which costs 1.37% of bandwidth, and it has 840 banks per chiplet with 72 spares. The vertical interface alone burns 296W of a 422W per-package budget, leaving 126W for everything else in the package. The DRAM maker is undisclosed, so volume rests on an unnamed supplier in what Tom's Hardware calls a very tight DRAM market, while Alchip handles design and packaging. The company has raised roughly $450 million so far, and TSMC makes the logic die.

d-Matrix image used by Tom's Hardware for its Hot Chips report on Raptor
d-Matrix · Image: Tom's Hardware
Raptor follows a line of test chips, each proving one piece of the design before the full product. Jayhawk proved die-to-die links, Jayhawk II digital in-memory compute, and Corsair combined both and entered full production this year. A Pavehawk test chip proves the 3D DRAM under the logic die. The comparison with Corsair shows the gap Raptor aims to fill: a pair of Corsair SRAM cards reaches about 300 TB/s but holds only about 4GB, while HBM4 packages top out near 20 TB/s with far more capacity. Raptor's DRAM sits between the two. A Corsair server paired with existing Hopper GPUs can lift utilization from about 20% to as much as 80%, and d-Matrix has adopted Nvidia's NVLink Fusion rackscale infrastructure to link its accelerators to Nvidia AI factories.
About d-Matrix
d-Matrix is an AI inference compute company that builds data-center inference platforms, including Corsair accelerators, JetStream network cards and Aviator software. Its CEO, Sid Sheth, said in November 2025 that it was started about six years earlier and focused on inference from day one.
In November 2025 it closed a $275 million Series C at a $2 billion valuation, bringing its total raised to $450 million, co-led by BullhoundCapital, Triatomic Capital and Temasek. Its Pavehawk 3D DRAM test silicon came online in September 2025, ahead of Raptor, and its Corsair accelerator, built on lessons from the Jayhawk and Jayhawk II test chips, entered full production in 2026.
d-Matrix argues that AI infrastructure is no longer a chip problem: memory bandwidth, energy, interconnect and software have to be solved together, and GPUs, CPUs and accelerators will be orchestrated as one heterogeneous cluster. For now Raptor has no firm date, volume or pricing, and CEO Sid Sheth told CNBC in June that it would launch in 2027. Raptor stacks a single DRAM layer, and a multi-layer stacking roadmap is still unfinished work. Samsung showed a zHBM concept at Hot Chips that stacks HBM on the processor, a rival take on bringing memory closer to compute.

A slide from d-Matrix's Hot Chips 2026 talk on its 3D-DRAM accelerator
Image: ServeTheHome
Sources · 3

Comments