0:04 As chips speed up and get more capable,
0:06 they must also fetch more data and get
0:09 it faster. Most of the time that means
0:12 going off chip to some external memory
0:14 module. It slows things down and uses
0:18 energy. One alternative is to quote
0:20 embed some memory right alongside the
0:22 logic circuits on the chip. Embedded
0:25 memories. For years, two types of
0:28 embedded memories dominated. But things
0:30 are changing. In today's video, we take
0:32 a look at those, plus some of the next
0:34 generation memories coming down the pike.
0:37 pike.
0:39 40 years ago in the 1980s, there were
0:41 three big categories of discrete
0:44 standalone memory chips. The SRAMMs,
0:48 DRAM, and flash EROMs. But as time and
0:50 technology demands progressed, these
0:52 three changed like high school friends
0:55 after graduation. In the 1990s, the
0:57 process nodes used to make DAM and flash
1:00 memories greatly diverge from each other
1:02 as well as the nodes used for making
1:04 logic chips.
1:07 DRAMs transition from using flat planer
1:09 capacitors to vertical ones. And today,
1:11 the dominant DRAM nodes use these tall
1:14 and skinny capacitors stacked on top of
1:17 or below their axis transistors.
1:20 And as for the EROMs, they evolved into
1:23 the flash memories, NOR and NAND, with
1:25 planer NAND evolving yet again into the
1:28 lasagnaike 3D NAND. Such vertically
1:30 stacked NANDs are some of the most
1:32 scalable in the semiconductor world. I
1:35 did a video about it a while ago.
1:37 Now, as a standalone memory, SRAMM
1:39 hasn't had the same success as its two
1:42 friends. Unlike DRAMs and EROMs,
1:45 however, SRAMMs you can use only
1:47 transistors to store bits, which lets us
1:50 make it alongside the rest of the chip
1:52 on an embedded status without needing
1:56 any additional masks. In the late 1980s,
1:59 CPU makers started embedding SRAMs onto
2:01 their chips as cache to store important
2:04 data. It remains very significant and
2:05 the single largest embedded memory
2:09 market. However, in recent years, SRAMs
2:11 have found themselves on the ropes. For
2:13 one thing, it's a thick boy. The most
2:16 widely used SRAMM cell design uses six
2:18 transistors. That is a lot compared to
2:21 DRAM, which famously is just one
2:24 transistor and one capacitor. And that's
2:26 a problem because transistors aren't
2:29 getting any smaller nowadays. FABs have
2:31 optimized SRAMM to such an extent that
2:33 when they brag about their process
2:36 nodes, they use SRAMM density to do so.
2:39 One of the few hard numbers that TSMC
2:41 has publicly announced about their N2
2:43 process node is how it can stuff more
2:46 SRAMM onto the die.
2:48 With CPUs and other systems on chips
2:49 getting more advanced, you get
2:51 situations where a surprisingly
2:53 significant portion of certain chips is
2:56 just embedded SRAMM memory. Back in the
2:59 2000s, some high performance CPUs had as
3:02 much as 70% of their whole dyes being
3:05 just SRAMM.
3:07 So if SRAM is reaching its density
3:09 limits, why not embed something that can
3:11 be far denser?
3:14 That is why some have turned to embedded
3:17 DRAM or ED RAM. It is the same one
3:19 transistor, one capacitive structure
3:22 just embedded onto the die. With that
3:24 skinnier setup, we can stuff five to six
3:27 times more ED RAM than SRAMM onto the
3:30 same space. ED RAM also uses
3:32 significantly less power than SRAMM,
3:34 even if you still have to periodically
3:37 refresh them. Like with commodity DRAMs,
3:39 you use just a third of the power of
3:42 SRAMs, not to mention the power saved
3:45 from not going off chip. There are also
3:47 integration benefits since we are less
3:50 likely to get bad connections, bend pins
3:52 or other mechanical failure points, etc.
3:55 ED RAMs tend to be more reliable. Data
3:57 transfers to and from memory have bitter latency.
3:59 latency.
4:02 But what are the downsides? Memory and
4:04 logic process nodes are nowadays very
4:06 different. So producing ED RAM adds
4:08 maybe four to six masks to the
4:11 fabrication process which exposes your
4:15 chip to yield risk and higher costs. The
4:17 ED RAM market was once quite
4:19 considerable used for items like the
4:22 Xbox 360. However, its momentum has sort
4:24 of petered out in recent years with
4:26 fewer industry products being made with
4:30 it. However, there seems to be plenty of
4:31 compute and memory research done in
4:34 academia with it.
4:37 Also, like SRAMM, EDRAM is volatile.
4:39 Once the power goes out, everything is
4:42 forgotten. Ideally, we want something
4:44 nonvolatile, something that can hold its
4:48 data when the power goes off. So, over
4:50 time, vendors have embedded flash
4:52 memories onto the chip. Embedded flash
4:56 or e-lash. E-lash is a NOR type memory.
4:58 With NOR, we string together many
5:01 special memory cells, planer transistors
5:04 equipped with a floating gate. Electrons
5:06 are compelled into that floating gate
5:07 through an oxide, raising the
5:10 transistors threshold voltage. Nor
5:13 arranges these cells in such a way that
5:15 we can access them one at a time, random
5:18 access, at the cost of less density. Its
5:20 younger cousin NAND on the other hand
5:22 networks its cells together in strings
5:26 of 16 to 128 with each cell source
5:28 connected to its neighbor's drain. It
5:30 lets us pack cells very closely
5:32 together, but also means no random
5:35 access. We can only manipulate data in
5:38 blocks or pages. That is why we aren't
5:41 getting embedded NAND anytime soon.
5:44 Programming and erasing e-lash's nor
5:47 style networks require high voltages,
5:50 something like 9 to 18 volts to compel
5:52 the electrons to go in and out of the
5:54 floating gate. That's a problem because
5:56 standard logic transistors run at about
5:59 1 volt. To protect these neighboring
6:01 logic transistors from getting fried,
6:04 you need deep isolation trenches or some
6:07 kind of hardening. NAN requires even
6:09 higher voltages than NOR to program and
6:12 erase their long strings of cells which
6:16 is too high for the die anyway. So E
6:18 flash being NOR means it cannot achieve
6:21 the same density as NAN has. It also
6:24 does not write as fast as SRAMM or DRAM.
6:27 It cells also suffer the same endurance
6:29 issues as discrete flash memories
6:31 breaking down over repeated write
6:35 cycles. And same as ED RAM, there is a
6:37 fabrication cost. It requires an
6:39 additional six to eight mass steps,
6:42 which can be more than ED RAM. You take
6:45 on more yield risk. Today, e-lash is
6:47 most often used to store program code
6:50 and data for these small but vital chips
6:53 called microcontrollers or MCUs. These
6:55 are basically computers on a single
6:58 chip, though near as powerful as an
7:01 Intel or AMD CPU.
7:03 E Flash hit the bill, fit the bill
7:05 before these systems because it boots
7:08 fast, is power efficient, rewritable,
7:10 and can survive the rough conditions
7:13 that cars or other industrial devices
7:16 often experience. The e-lash automotive
7:19 MCU market is often cited as the second
7:21 largest overall embedded memory market
7:24 after SRAMM, though you can also find
7:26 them in edge AI and data center applications.
7:28 applications.
7:30 E-lash's most serious issue, however, is
7:33 scaling. Largely speaking, 28 nanometers
7:35 is the end of the road scaling wise for
7:39 El. Scaling down E-lash means making
7:41 smaller transistors and packing them
7:43 closer together. This becomes a serious
7:46 issue at 28 nmters, the last planer
7:48 transistor node.
7:51 There you have all the standard problems
7:53 of shrink, loss of control over the
7:56 gate, short channel effect, so on. par
7:59 for the course and why the logic fabs
8:02 switch to 3D finfats. But then there are
8:05 the flash memory related issues too. The
8:07 flash memory cells are now so physically
8:09 small that their floating gates contain
8:12 about 100 or so electrons for a
8:15 threshold of 1 volt. It takes fewer
8:18 electrons leaking to cause significant
8:21 degradation. And with the tunnel oxide
8:24 layer so thin now that is way more
8:26 likely. These scaling problems are why
8:30 the NAND makers switch to 3D NAND. You
8:32 loosen the floating gates as technical
8:34 requirements by resetting their sizes
8:38 from 28 nmters to 40 nmters, but then
8:40 stack them vertically to achieve massive
8:43 storage numbers. 3D NAND is made in a
8:46 parallel manner and very cool, some
8:48 cases literally, but not a valid
8:51 technical pathway for e-lash. Nor is it
8:53 economically feasible for the chip
8:55 designer to add what can be up to 10
8:57 additional mass to produce a wholly
9:01 different transistor type onto the chip.
9:03 Without a valid successor to e-lash,
9:06 OEMs of MCUs and such products might
9:08 move back to discrete memories, perhaps
9:10 using advanced packaging to put them
9:13 together. So fabs and startups have
9:16 suggested potential successors, the next
9:19 generation memories. And there are a lot
9:21 of next generation embedded memories out
9:24 there. Let me cut it down to a few such
9:26 with serious backing by major foundaries
9:30 like TSMC and Samsung. First up are the
9:33 magnetoresistive RAMs or M RAMs.
9:36 DRAMs and flash memories encode the bit
9:38 by storing a charge. The MRAM on the
9:40 other hand does it by manipulating
9:43 electrical resistance levels. An MRAM
9:46 cell is made up of two things. an access
9:48 transistor and the magnetic tunnel
9:51 junction or MTJ. The latter is where the
9:54 magic happens. The MTJ is a tiny
9:57 sandwich of ferroagnets and non-magnetic
10:00 layers. The simplest MTJ has two
10:03 ferroagnets and a very thin insulator
10:04 layer in between them. The top
10:08 ferroagnet is referred to as the free
10:11 layer. The ferroagnet on the bottom is
10:13 called the reference or fixed or pinned
10:16 layer. Both ferromagnets are usually
10:18 made from an iron alloy like cobalt iron
10:22 boron and depending on the variant you
10:25 may have several pinned layers. As for
10:28 the very thin maybe 1 to 2 nm thick
10:30 insulator layer that is most often made
10:33 of magnesium oxide.
10:36 The MTJ works by having an external
10:39 current orient to magnetization of the
10:42 MTJ's free layer as compared to its
10:45 reference or pinned layers. When the
10:48 magnetic moments of both ferromagnet
10:50 layers are parallel, then the whole MTJ
10:53 will have low electrical resistance. So
10:56 we can easily run a current through it.
10:58 And when the magnetic moments of the two
11:00 ferromagnets are not parallel to each
11:02 other or antiparallel then the
11:05 electrical resistance gets significantly
11:07 higher. So in a way it's like twisting a
11:10 faucet open or closed. We can map the
11:13 two high or low resistance states to a
11:16 bit. How is that done? With MRAMS, we
11:19 send small currents into the MTJ to try
11:21 and discern its resistance state,
11:23 comparing it against a middle point
11:26 reference level to determine the final value.
11:27 value.
11:29 The first MRAMs were introduced in the
11:32 1980s and are today known as
11:35 conventional or fieldswitched MRAMs.
11:38 These older memories use magnetic fields
11:41 to write to the MTJ, i.e. how we set the
11:43 free layer. This magnetic field was
11:45 created by running a current through a
11:48 wire making this flip an indirect
11:50 effect. Basically, that was how we wrote
11:54 data to the old fite core memories.
11:56 For this mechanism to work, the magnetic
11:59 field has to be strong enough to flip
12:02 the MTJ's magnetic state. The problem
12:05 was that as the MTJ got smaller, it gets
12:08 easier for that bit to accidentally flip
12:11 due to thermal noise.
12:13 Hard disk drives suffer this problem
12:15 too. They have something called the
12:17 super paramagnetic limit where the
12:19 grains in a bit get so small that
12:22 thermal energy can flip them. Anyway,
12:25 the response by engineers has been to
12:27 raise the quotequote flip limit. The
12:29 downside of doing that is that we need a
12:31 more powerful magnetic field to switch
12:34 it when we actually need to do so. When
12:37 the MTJ is small, controlling that field
12:40 is harder to do. In the 1990s, the field
12:42 switch MRAMM was replaced by a new
12:45 variant known as spin transfer torque
12:48 MRAM, STM RAM. Instead of using a
12:50 magnetic field, we send a special
12:53 current through the MTJ. Such electrons
12:55 flood into the free layer and flip it directly.
12:58 directly.
13:00 STM RAM is a very promising memory
13:03 technology. To start, it's nonvolatile,
13:04 so the data stays even after the power
13:06 goes off. It does not need to be
13:09 continually refreshed. The area savings
13:11 are also very significant. It is
13:13 basically just the MTJ and an access
13:17 transistor. A very DRMish setup. At the
13:20 5nmter node, we get like a 43% reduction
13:23 as compared to SRAMM.
13:25 Despite being a nonvolatile memory, you
13:28 can write to it in less than 10 nonds,
13:30 which is DAM like speeds and far faster
13:32 than flash memories 20 to 100
13:35 microsconds. And unlike flash memory,
13:39 the cells have very good endurance.
13:40 The biggest challenge involves things on
13:43 the manufacturing side. The technology
13:45 is technically CMOS compatible, but
13:48 fabbing the 15 to 20 various metal and
13:51 dialectric stacks that make up the MTJ
13:53 is challenging. In particular, the
13:56 insulating oxide barrier between the
13:58 free and pin layers needs to be about 1
14:02 to 2 nm wide. Common issues often happen
14:04 during the etch or post-tetch process
14:07 where oxygen impinges into the insulator
14:10 layer to create an effect called birds
14:13 beaking. There can also be issues with
14:15 the bottom electrode contact which
14:18 connects the MTJ to the metal lines.
14:21 Roughness in that contact can make the
14:24 MTJ's layers rough, too, which causes
14:26 the free layer to magnetically
14:28 quotequote sync with the reference
14:31 layer, thus making it harder to discern
14:35 the actual resistance level of the MTJ.
14:37 Beyond that, there are scaling issues at
14:39 the 3 to 5 nanometer class nodes. The
14:42 STM RAM gets so small that we need a
14:44 decently strong current to properly
14:46 write to it and avoid thermal induced
14:49 bit blips. But advanced node transistors
14:51 are so small and delicate that such a
14:54 current cannot be easily delivered.
14:57 The other major e-lash replacement is
15:00 resistive RAM or RAM or RAM. I'm going
15:03 to say RAM. RAM is another nonvolatile
15:06 memory that stores a bit using either a
15:09 high or low resistive state. Yes. So a
15:12 lot like MRAM. However, the way in which
15:14 they go about doing that is very
15:16 different and kind of fun. There are a
15:19 variety of RERAM cells, but the most
15:22 commonly used one is the filamentbased
15:25 RAM. It too is a sandwich of a metal
15:28 oxide insulator layer between two metal
15:30 electrodes. The oxide might be of
15:33 elements like hapneium, tantelum or
15:35 titanium, but research into more exotic
15:39 things like 2D materials is ongoing. The
15:42 electrodes can be titanium, platinum or
15:45 something else. The rea cell switches
15:48 between high and low resistive states by
15:52 creating set or destroying reset a small
15:55 conductive filament maybe as small as 10
15:58 by 10 nmters bridging the two
16:00 electrodes. That little filament is
16:03 essentially a wire through the naturally
16:06 insulating dialectric. We set or reset
16:08 the filament by applying a voltage or
16:11 current signal to the electrodes. Very
16:14 elegant, very simple concept.
16:16 The concept reminds me of another memory
16:19 called phase change RAM, which uses heat
16:22 to switch a ccogenide glass between an
16:25 amorphous or crystalline phase. In this
16:27 case, RAM does not require the phase
16:31 change. Producing embedded RAM requires
16:33 fewer mass depths than other embedded
16:36 non-volatile memories. It can scale down
16:38 to advanced nodes fairly well. It uses
16:40 less energy, is nonvolatile, and reads
16:43 writes very quickly. On the other hand,
16:45 there are some variability and endurance
16:48 issues. The set and reset processes
16:50 appears to be inherently random, leading
16:53 to inconsistent behaviors. Not what you
16:55 want in a semiconductor technology. And
16:57 I do wonder how many times can we make
16:59 and break the filaments before it starts
17:02 to exhibit weird behaviors. Broadly
17:04 speaking, the technology has commercial
17:06 potential, but it also feels like
17:08 something that pops his head up every so
17:10 often, attempting to ride the latest
17:13 significant trend. Between 2005 and
17:16 2015, RAM gained serious traction as a
17:19 potential successor to 2D NAND until the
17:22 rise of 3D NAND closed the door on that.
17:24 There were proposals to do stacked RAM,
17:26 vertical and horizontal, but those
17:28 failed to compete.
17:30 A few companies offer this technology
17:32 today. There's one called Weebit Bit
17:34 Nano from Israel that shows up a lot in
17:36 the literature. They've been around for
17:39 over 10 years licensing rea to customers
17:43 for enduser products. TSMC offers it as
17:45 an option for customers too. They have
17:47 published a few papers on the technology
17:50 though far fewer than what they have on
17:53 MRAM. It seems like TSMC is positioning
17:56 or TSMC and Samsung are positioning both
17:58 technologies as potential successors to
18:01 e-lash especially in the automotive MCU
18:03 space. There are others like the
18:05 affforementioned phase change RAM and
18:07 ferro electrics that might have a shot
18:10 but I think ST MRAM and RAM are the leaders.
18:12 leaders.
18:14 Despite both TSMC and Samsung
18:18 positioning ST MRAM or RAM e-lash
18:21 remains quite resilient. Why Elash is a
18:24 tried andrue solution in a space where
18:26 reliability matters more than raw
18:29 performance. Most MCUs microcontrollers
18:31 are still made using trailing edge nodes
18:34 like 65 nanmters, though this is
18:36 starting to change. And as we laid out
18:37 throughout this video, the next
18:40 generation contenders are not exactly
18:42 free of trade-offs. The semiconductor
18:43 industry is pretty conservative and
18:46 they're not apt to try new weird stuff
18:48 until they have to.
18:50 The current major hope for these
18:51 embedded nonvolatile memories
18:54 technologies is AI since embedded
18:56 memories are so close to the logic
18:58 literally there's some potential to
19:01 evade the vonoyoman bottleneck and that
19:04 seems to be the case with STM RAM.
19:06 Another option is for doing AI inference
19:08 on the device at low power and fast
19:11 latency maybe even using neuromorphic
19:13 principles to do so. This is more for
19:16 ream. In both cases, the technology
19:18 seemed to have gotten ahead of the use
19:20 case, but we shall see if the enduser
19:23 markets can get on board. All right,
19:24 everyone. That's it for tonight. Thanks
19:26 for watching. Subscribe to the channel.
19:27 Sign up for the Patreon, and I'll see
