Much has been said about a "memory wall" that emerges when the throughput needs of the system outstrip the performance of the memory. One possible solution is to leap right over that wall with high-bandwidth 3D-IC solutions. But there's both promise and peril with emerging 3D-IC memory standards, according to panelists at the MemCon conference August 6, 2013.
The panel was moderated by Jim Handy (right), memory expert and director of Objective Analysis. "We're running into a place where we've got all this bandwidth inside the DRAM chip and the processor," he said as he introduced the panel. "We've got what I would call an I/O wall where communications are too slow between the chips. There have been a lot of solutions - DDR1, DDR2, DDR3, and now DDR4 - but those kinds of things are running out of steam."
While there are a number of emerging memory standards, the panel focused mostly on three that are aimed at 2.5D packages, where dies are placed side-by-side on a silicon interposer, or 3D stacked die implementations that use through-silicon vias (TSVs). They are:
- Wide I/O, published Dec. 2011 by JEDEC, provides a 512-bit wide interface to memory and up to 17GBps. Wide I/O 2, now in development, ups the ante to 25.6GBps and 51.2GBps.
- Hybrid Memory Cube (HMC), developed by Micron and now managed by the HMC Consortium, stacks four or eight TSV-bonded DRAM dies on top of a logic layer, claiming potential performance up to 320 GBps.
- High Bandwidth Memory (HBM), an emerging JEDEC standard, leverages Wide I/O and TSV technology to reach speeds up to 256 GBps.
Here are the panelists, with their opening statements:
Becky Loop, principal engineer, Intel. "I work in the client PC working group, and our group focuses from tablets to high-end desktops. We want to go to a thinner form factor and still deliver the performance you're used to in a typical laptop. We are continuing to push the bandwidth wall but at the same time, we want lower power and smaller form factors."
Gopal Raghavan, fellow, Cadence. "We sell controllers you can use for a variety of applications. To a great extent we are protocol agnostic. Our goal is to solve your problem. We work with you to understand where the bottlenecks are."
Shafy Eltoukhy, vice president of operations, Open-Silicon. "We build SoCs with memory interfaces. HMC could be a very good solution, because you don't have to worry about building your stack; you just take the packaged part from HMC. On the other hand, if you want to differentiate yourself, Wide I/O and stacked DRAM would be a good solution."
Bill Gervasi, memory technology analyst, Discobolus Designs. "One of the things we're seeing is the fragmentation of the industry, and fragmentation doesn't work. We're seeing a lot of blue sky. A few minutes ago I saw a slide that showed five types of memory [standards] that are taking off. Are all going to succeed? I don't think so."
Some excerpts from the conversation follow.
Q: Wide I/O, HMC, HBM - What to Use When?
Gervasi: The idea of stacking memory devices onto a controller of some kind is a logical future, and we're going to see markets like graphics needing that High Bandwidth Memory alternative to push performance. On the other hand, Wide I/O is targeted more at mobile applications where performance doesn't have to be quite as aggressive and the power tradeoff is the most important thing.
Eltoukhy: With HMC they combine everything you want into one package, so you don't have to go through all the manufacturing issues. With Wide I/O you need to connect to the SoC and you need some type of interposer. You have to deal with manufacturing and yield and so on. But at the same time, it [Wide I/O] allows you to differentiate. You can optimize it in any way you want and add functionality to it.
Raghavan: Eventually it's a compromise. As a graphics person I may want HBM, but guess what, if HBM is very expensive, the people who know how to use DDR4 will come out ahead. DRAM is a very cost-sensitive market.
Gervasi: It's also possible that people will use multiple interfaces to solve different problems.
Loop: Which technology are you going to use for your highest bandwidth requirements, which one has capacity, which one has low power? A systems integrator needs to trade off one against the other and find a happy medium.
Q: Going from DDR4 to Wide I/O or HMC or HBM will really be a quantum leap. What's driving it?
Raghavan: One thing that drives it is network switching. Protocols are going up and typically you need between a 150GBps to 300GBps network interface for switching. Wide I/O is different - it's made for mobile applications. You cut back on bandwidth and save power.
Q: Interposers are quite pricey. How do we bring this technology to market?
Eltoukhy: The interposer is expensive now - we're still in a learning curve. We need to get to less than $500 per wafer to make it cost effective. However, the package and substrate guys are coming up with their own organic interposers. Performance won't be as good as silicon, but costs will be lower.
Q: What about yield?
Eltoukhy: Yield can also improve [with interposers]. We're hearing now that yield has gone from 60% to 90%.
Gervasi: 90% yield is not going to solve the yield problem! You'll come up with a compound yield issue. You stack four devices each yielding 90% and you're down to 64%. You can't make a market with that.
Eltoukhy: I'm talking about the overall interposer, not the stacking.
Gervasi: Yes, and the stacking is the weak link. I don't see a tester technology out there that is going to speed-bin devices before you stack them up. If you take four DDR4 2400 devices, put them together and package them, what yield are you going to get? You're not going to have a marketable product here.
A smartly designed HMC can work around yield problems. I actually think something like HMC could make it to market before the 3DS stack. The compound yield problem is going to kill them.
Q: Are thermal issues going to be a problem? DRAMs run hot.
Raghavan: Thermal management comes down to the controller. There are a lot of ways to manage it. I think the yield issues and the business model issues are more critical.
Gervasi: I don't know about that. If you think about HMC it is fully buffered DIMMs in a package. With fully buffered DIMMs, one of the weakest links was thermal.
Q: Multi-chip modules [MCMs] did not do well in the 1990s. The problem was, if an SRAM chip failed, a $1 SRAM could take down a $100 processor with it. Is this a concern with HBM and Wide I/O?
Raghavan: Yes, of course. The business model for Wide I/O is a big issue. You don't know what you need to buy, and how it is going to be priced, and what the yield loss is. The cost could be substantial.
Eltoukhy: If you just need large bandwidth and don't want to worry about all the manufacturing costs, just buy the HMC.
Q: MRAMs may be a nice addition to Flash controllers. Could we stack Flash plus MRAM?
Gervasi: I think the biggest hanging fruit in the industry now is heterogeneous systems. We could have subsystems of SRAM, DRAM, cache, MRAM, phase change...all these technologies. If you put a matrix together of the capabilities of these heterogeneous systems, and we structure our processes and our software to exploit them properly, I think we'll have phenomenal growth opportunities.
Conclusion - What's Your Favorite Technology?
Handy posed this question at the end of the panel, and the answers were surprising: LPDDR3 (Loop), DDR3 and DDR4 (Raghavan), DDRx and HMC (Eltoukhy), and DDR4 (Gervasi). With all the talk about new 3D memory technology, it looks like DDR isn't going away any time soon.
Richard Goering
Related Blog Posts
Semiconductor Memory Challenges Will Be Overcome, MemCon Keynoter Says
Wide I/O 2, Hybrid Memory Cube (HMC) - Memory Models Advance 3D-IC Standards
An Update on the JEDEC Wide I/O Standard for 3D-ICs
Wide I/O Memory and 3D ICs - A New Dimension for Mobile Devices
MemCon Samsung Keynote: New DRAM and Flash Memory Architectures are Needed