The semiconductor industry is experiencing unprecedented demand driven by AI, leading to significant increases in capital expenditures, particularly for memory and storage components, which are now a major cost driver for hyperscalers.
Mind Map
Click to expand
Click to explore the full interactive mind map
And the reason for raising capex has
always just been we need more compute,
we need more flops, we need more
intelligence. This quarter there was
Hello everyone and welcome to another
semi-doped podcast. I'm Austin Lines
with Chip Strat and with me is Vic
Shaker from Vick's newsletter. Hey Vic.
Man, a lot has changed since the last
time we recorded a podcast. What What's
up with you?
Yeah, a lot has changed. So, after 15
years of working in the semiconductor
industry across like small, medium, and
large companies, I finally decided to
hang up that corporate hat and do the
Substack, this podcast
um full-time and see where it takes us,
you know, in the podcast front. And I
hope the substract continues to grow too
because it is so much fun that uh I
figured like okay let's give this a shot
because I learn so much from like
writing and you know reading about so
many different things and uh corporate
roles tend to be a little bit more like
hyperfocused and I've spent 15 years
doing kind of the same thing. So I was
like okay instead of trying to trying to
change roles everywhere I've kind of
managed to carve out this role for
myself. So I'm like, "Okay." I said,
"Let's try it for a while. What's the
worst that can happen? Let's try it."
>> Awesome. I love it. Yeah. So, okay. So,
you'll have more time presumably for
writing, podcasting, researching. Like,
what are you most excited about?
>> Yeah. I'm just going to do the same
thing like I'm not trying to uh add too
many things because I think we already
added one other thing to the semi-doped
umbrella which we haven't really spoken
about which is semi-doped also has a
substack presence now and it is a free
for all subscription it does not have
payw walls and what Austin and I do here
is uh just we just give our daily take
so this is like a once daily newsletter
uh that's short to read like within 3 to
5 minutes just with some key highlights
of what's happening in the day because
we monitor this stuff so much every day
now that we figured okay like there's so
much that goes like unmentioned because
we can't write everything in the
newsletter and we can't talk about
everything on a limited time podcast but
not everything requires a full deep dive
substack or a full hour podcast but if
we try to stick it in a podcast, it gets
fragmented. So, the best place for this
kind of information, but that's also
very relevant because the the reason you
sign up for the semi-doped Substack is
that, you know, you get you'll get one
email a day. You open it up, you know,
while you just have your coffee, you
just like glance through it. Anything
that picks your interest, you just read
that section or or not, you know, you
you'll get another email the next day
and it's an easy read. It's unlike our
deep dives on Substack which is highly
researched and thought through and all
that. These are just like quick hits. So
that is that is the second project
that's common to us apart from the Substack.
Substack.
>> Yes. Yes. Yeah. I'm excited about it.
I'm I'm looking forward to it. It's been
fun this week. Um you know I think for
listeners definitely go sign up. It's
just basically like Vic and I having a
quick water cooler conversation and that
you're part of you know hey here's this
news from yesterday. Vic will give a
take. I'll give a quick take. Move on to
the next thing. I think it'll be fun,
informative, and you'll just be more
intelligent about semiconductors and AI
for reading it.
>> Yeah, just semi-doped.com should get you
there. Or you can just search for it on
Substack Semi-Doped. You'll see the same
logo of this sub uh this podcast and
you'll find it there.
>> Exactly. And the last thing that is like
forming still in my life is the fact
that now I am available to do consulting
and my newsletter itself actually runs
under the umbrella of semi-exponent
which I came up as a name to say like
this is the most exponential technology
I like you know that ever existed. So I
was like okay semi-exponent is a nice
it's like you know what is the exponent
the exponent is zero means it's a flat
line but it's never zero. So anything
above zero is like an exponent. So I'm
like, okay, that's a nice name. So the
consulting arm is going to be under semi-exponent.
semi-exponent.
So now I'm just starting to find, you
know, clients who want to work with me
on various different aspects of
semiconductors, have chats, pick my
brain on various aspects of what I write
and talk about. Ah, we'll see how that
goes. Should be fun. I like talking to
people anyway.
>> Yeah. Yeah. Nice. Nice. So you'll be
researching, writing, talking and
consulting. That that does sound like plenty.
plenty.
>> Yeah, that's enough, right? That's why
that's why I'm not doing anything to the
substack. Things will continue as it is.
And uh I hope there'll be more research
involved because I have more time to
think through carefully a lot more stuff.
stuff.
>> Yeah, I've always tried to maintain a
high high level of, you know, quality on
the substack. So hopefully if I've been
doing my job well enough, you won't
notice any changes.
>> There you go. There you go. Love it. All
right, let's get into it. So this past
week, you were on vacation. There was
earnings week. They didn't wait for you.
Um and but I stayed on top of it and I
know you you've caught up a lot on it.
Um so of course listeners, I know you
know all about this. Microsoft, Google,
Meta, Amazon, Samsung, Sandis, Seagate,
Western Digital, Qualcomm, Cadence, NXP,
even Rivian, which I I love to follow
even though they're kind of downstream
as like a semiconductor consumer. Um,
all those companies reported impossible
to listen to all of it. Um, but we are
going to cover it. And I thought the
angle that we could take is actually
starting with memory and storage
companies. And here's why. You know, as
everyone's been tracking, the big four
hyperscalers have now committed to
nearly 700 billion of 2026 capex, which
is up from roughly 500 billion just in
2025. So they they every quarter the
story is always like, hey, did they keep
capex or did they raise it even more?
And the reason for raising capex has
always just been we need more compute,
we need more flops, we need more
intelligence. This quarter there was
something different which was yes we're
raising capex but it's because of
needing extra dollars allocated
specifically to cover rising component
costs. Memory is more expensive. Storage
is more expensive. Even things like
fiber and optics are are more expensive.
And so I thought that was an interesting
sort of shift of like, oh yeah, we are
increasing capex, but it's just to pay,
not to buy more flops, but just to pay
for the things that we already committed
to. Um, so I thought it'd be fun to
start with the beneficiaries of this
capex raise, which would be the memory
and storage companies that reported. So
we're going to start there. How does
that sound?
>> Yeah, let's do it. Memory and storage is
something I keep ranting on on the
podcast. Like, how much more expensive
is it going to get? Like, how much more
are they going to raise prices? They
expect a price hike every time,
>> but if the price hike isn't good enough
now, they're like, "Oh, what? You're
you're only doubling it. Why not? Why is
it not five times?"
>> Yes. Investors. Totally.
Man, aren't you happy that prices are
going up? Not going up enough. Yep. Your
stock goes down.
>> That is crazy.
>> So, all right. Let's start with Samsung
or Samsung Electronics. Um, I the number
that stuck out to me was that their
memory revenue was up 101%
year-over-year, which obviously is
pretty crazy for memory. Um, they said
as a company their Q1 revenue and
operating profit were at all-time highs.
HBM sales are expected to triple
year-over-year in 2026. and they expect
HBM 4 to be 50% plus of HBM sales by Q3.
So there's a mix shift going from HBM 3
3E to four. Um now for people who
haven't followed Samsung as closely in
the HBM market um the the story over the
past couple years was they used to be
there there's there's basically three
big players SKHX, Samsung and Micron.
And Samsung used to be, you know, up
there with SKH Highix, uh, having a lot
of market share on the order of even
just about this time, you know, last
year it was, uh, maybe like or actually
maybe like six quarters ago that it was
like 40% market share for Samsung. And
in 2025, it dropped drastically down to
like 13%, 15%, 20%. And so the story of
Samsung is fall they've fallen behind
and now they're trying to catch back up.
Now, of course, there's always a new
technology, HBM3, HBM3, HBM4, um, and
when a new standard comes out, there's
an opportunity to try to be first to it
and and regain some market share. And
so, um, Samsung on their call, they were
really trying to position that like,
sure, uh, the past is behind us, but
we're totally ready for HBM4. And so, I
thought I'd read a couple quotes quick
and then we can get into it. Um but they
wanted to make sure to say that they are
the first and they are the best around
HBM4. So the quotes from the call Jun
Kim EVP of memory sales said after we
became the world's first to commence
commercial shipment of HBM4 in February
and then also he called out so so right
there like saying in passing like don't
forget we were the world's first um
shipping HBM4 and then um he said the
differentiated performance of our HBM4
led to concentration of demand and our
production ready capacity is fully
booked and sold out. Our outstanding
performance has been translating into
actual premium on pricing. So the read
through there is
there's always been a question around
memory around storage which is is it is
it just a commodity a bits a bits a bit
it doesn't matter who it's from and here
uh Samsung was trying to say hey not
only we're first but our performance is
the best and that's why and and we've
got the capacity ready so that's why
we're sold out customers and that's why
we're able to have a premium on pricing
is because customers prefer us. Um
any reaction to that Vic? I remember
this there was this whole discussion
about uh how many gigabits per second uh
you can get out of HBM memory by
designing the base die for HBM4 to be uh
you know on a certain node or even if it
doesn't matter what node it is because
there was some discussion that Micron
was using a memory technology to make
the base die while the other competitors
like Samsung and SK were actually using
a two logic node
Um the point is that the speed becomes a
differentiating factor. So it's no more
memory is just memory.
How how it performs is become
has become a very important factor for
like which company Nvidia or AMD will
pick for their performance. And not only
that, it comes the other way too. The
Jedex spec for HBM4 is like I think 8
Gbit per second per lane. But then
because they want supremacy on inference
performance and tokens output and tokens
per watt per dollar and all that they
are pushing the speed per lane of HBM4
faster and faster and faster. So it's
become a competition as to who can get
to like 11 10 or 11 or 12 Gbps per pin
and that is not even is way beyond the
spec but it has become the thing that
drives sales and drives lock in because
once these hyperscalers choose and qualify
qualify
a HBM vendor it is a sticky decision
because qualification if you remember
like the HBM3
Samsung couldn't really get qualified at
Nvidia for a long time. They had so many
yield issues and things like that. So
qualification and performance makes this
very sticky. Uh as so it's not funible.
So you just can't take out Samsung and
drop in Micron tomorrow. Although there
are only three companies doing this.
>> Yes. And so zooming out and and hitting
on what you said again for listeners,
the the real interesting thing is there
is a spec this Jed spec JDC
um that defines
the performance level that and you know
the various other things that a spec
define about how it's supposed to work,
how it's supposed to communicate um that
these three companies are trying to hit
and and normally if everyone just hits
that spec exactly, it would be funible.
But but what Vic is saying is that, you
know, Nvidia said, "Hey, wait a minute.
I've got, you know, I've got to compete
against AMD and against XPUs, and if I
could if you could give me um memory,
HBM memory that's even faster, then I
can get even better, you know, tokens
per second, tokens per watt, that kind
of thing." And so actually there is a
pull from the the silicon vendors to the
memory manufacturers to say I know the
spec says 8 gigabits per second per pen
or whatever. Um can you go higher?
Right? And so then there's this
interesting like someone's trying to get
to 11 and now the others have to try to
get to 11 as well. So there's this
interesting pull to go faster and now
this kind of like unwritten spec that
this performance, you know, that they're
all trying to compete on and and so as
you move away from just everyone meets
the spec, we're all fungeible to like
how fast can you go, at what yield, at
what cost, it is more of a a a true
competition on the things that do allow
you to have premium pricing over
commodity pricing, which is which is
performance, you know, and yield and
cost and that kind of thing. So that's
that's a little bit of the story the
back the backstory here for HBM4 and how
Samsung is trying to regain market share
is to to truly compete on you know
performance. Okay. So here's an
interesting thing uh a as we move out of
that looking at the the competition
there between the three companies um and
go focusing back on Samsung another
quote that they had was our demand
fulfillment rate is now at a record low
customers who are concerned about supply
shortages are actually bringing forward
their demand for 2027 already so I you
know not surprising but again it's just
crazy to hear you know if customers are
asking for this much Maybe it used to be
like we can only give you 80% of that or
50%. Maybe now it's even lower than
that. I don't know. I'm making up the
numbers, but trying to illustrate the
point that customers are asking for a
lot of memory and the memory supplier is
saying I can give you less than ever. So
that would be uh the environment that
we're in.
>> Yeah, that's crazy to me. Uh how much
longer is this going to continue?
Because people, these hyperscalers are
increasing their capex just to pay these
memory companies. Literally, that's what
their capex is going into just to pay
these players for HBM and like nan
memory and things like this. I'm not
sure how much how much higher is going
to keep going. I I keep thinking that
that's it, but I'm always wrong.
>> Yeah, you know, that's it's an
interesting point. we can get into it
more uh both here and and later which is
I am very bullish on capex continuing to
get higher when it's buying more flops
when it's buying more compute because I
like everyone else believe that the more
compute we have the more intelligence we
have the more we can do you know I I too
have experienced I'm sure everyone has
you know um chat GBT or Gemini or cloud
code just spinning or saying like oh I'm
busy right now you know or like come
back. We've reached our limits. So,
that's frustrating and that just shows
that like dude, they need more GPUs.
They need more XPUs. But is this is a
different conversation when it's like
how high can you convince your CFO to
let capex go when you're not buying any
additional compute, but you're just
paying increased memory prices? Is that
going to be the straw that breaks the
camel's back?
>> Exactly. Because if you if like you say
if all of this money went into expanding
compute then a lot more users of AI
tools or whatever get actually something
out of it. They get better tools. They
use it to build better projects that
drives revenue that drives an economy.
It it completes the circle in some way.
What is happening now is if all the
capex is going to memory players it's
like you're siphoning all this money out
into somebody's pocket. it's not going
into the, you know, positive
reinforcement loop that we want to see.
So, this is a bit concerning for me.
>> No, you're totally right. I mean, put
succinctly, there's no ROI on that
additional capex
>> except for a few companies.
>> Yes. Yes. But for the hyperscalers, yes,
there's no ROI, no extra ROIC, return on
invested capital. It's just a tax
really. It's
>> it's a tax. It's a memory tag.
>> It's a memory tax. So, okay, let's move
on. Um, so we're going to talk SanDisk.
So, we've talked when we talked Samsung,
we were focused on HBM, focused on
memory. So, let's talk um o other memory
and storage. So, we're going to talk
SanDisk. But really quick, I thought I'd
give a quick history sidebar because we
haven't talked about SanDisk on the
podcast yet. So, um SanDisk, Western
Digital really quick, there are two
companies. Um back in 2016, Western
Digital actually acquired SanDisk for
around $20 billion. And the thesis I
think back then was having a full
storage portfolio. So can we own both
hard disk drives and nan flash and that
therefore we can sell hyperscalers a
complete stack. Makes a ton of sense
right now in the era we're in. But
prior to AI um that thesis didn't really
age well. I think they're they're very
different businesses. They have
different cycle dynamics, capital
intensity, customer mix. Um, so running
them under one roof wasn't as simple as
like, oh great, um, we share customers
and now we can more easily cross-ell
into them. It was actually sort of like,
ah, these are two different businesses
and there's some different customers at
play and we're not getting the quote
unquote synergies that we thought we
would. Um, so back in October of 2023,
Western Digital announced that they were
going to split SanDisk back out. And by
February 2025, that spin out happened.
SanDisk reemerged as a standalone
publicly traded company. And so when now
when you think about SanDisk, you can
think about NAND flash, SSDs, embedded
flash. Um, they have a joint venture on
some fabs in Japan with Kioia, Kioxia,
not sure how to pronounce that. Um and
then so SanDisk you can think of flash
SSDs etc. And then Western Digital is
eight hard disk drives only. Um and
>> they are working when I was doing a
little bit of research. Western Digital
is working on this interesting next
generation tech HMR hammer. It stands
for heat assisted magnetic recording.
And it I'd never heard of it, but uh the
goal is to achieve 100 terabyte plus
hard disk drive capacities for AI scale
data and they're planning volume
production in 2027. Have you heard of this?
this?
>> Yes. So, HMR has been there for a while
actually. It's not that new. >> Okay.
>> Okay.
>> HR has been cited for I've heard this at
least for like 5 years now. Uh I can't
pinpoint exactly how long it's been
around. It's always been the next uh you
know the next greatest thing in hard
disk drive technology. But then what has
happened recently is that if you look at
uh you know quad level cell SSDs QLC
SSDs the SanDisk ones for example the
highest capability the highest capacity
one is five uh 256 terabytes and you get
it in an SSD form factor already. So I'm
like what is this HMR and he's going to
give you what 100 terra it's not not
that great honestly today's day and age
of course the price point will be lower
it should be uh but you know HMR yeah
it's it's been around but it's I don't
know how relevant it is or how where we
are on that right now but I wanted to go
back in the history a little bit more I
think you'll find this fun so did do
does the name like Sanjay Meotra ring a
bell to you
Uh yes, Micron.
>> Yes. Micron. Micron. >> Yeah.
>> Yeah.
>> Do you know who founded SanDisk?
>> That person. >> Yes.
>> Yes.
>> Serious. What? >> Yes.
>> Yes. >> What?
>> What? >> Yeah.
>> Yeah.
>> Yeah. It's true. He was he was one of
the you know founded you know he was one
of the founding members of SanDisk. Uh
and you know this was like way before
his he became the CEO of sand of Micron
only in 2017 but he was the original >> wow
>> wow
>> uh founder of sand. So he's like real
real memory guy. And another fun bit of
trivia on the name, right? Uh one of the
other founders, um whose name I Eli
Harari, yeah, the one of the other
co-founders and so they've, you know,
they were trying to find a name for the
company or whatever. And then his
daughter comes in and it's like looks at
some of the discs lying there, you know,
you know, these these splatters or
something and like, oh yeah, that looks
like the sun. So they decided to call it Sundisk.
Sundisk.
>> No way.
>> You can look up early logos of Sundisk
and you'll see like a plate like thing
with like the sun's rays like coming out
of it. That's their logo. >> Serious.
>> Serious.
>> Yeah, we should put a picture if we find one.
one.
>> Yes, we should. Totally. Oh, great trivia.
trivia.
>> Yeah. Yeah. Yeah. And what happens is
that later they like sun micros
systemystems came after them for some
trademark stuff like oh why you can't
use sun in it or something so they
change sun to SanDisk
came about
>> fascinating yeah I was wondering I was
like sun how did it make it sand uh I
don't know
>> it changed only like seven years after
the company was even found like running
okay seven years after the company was
like fully functional they changed the
name to SanDisk thanks to Sun Micros
Systemystems Nice. Well, we should talk
to Sa San Sanjay sometime and ask him if
the sand came from his name, too.
>> Why not? Yeah.
>> Yeah. Well, yeah, we should seriously
see if he would talk to us about the
history. That's so fascinating that he
was a co-founder of SanDisk and now he's
a CEO of Micron. Super interesting.
>> Yes. Yes. I thought they mentioned that
as part of a history lesson here.
>> Oh man, good good history lesson. Okay,
so SanDisk, they reported earnings.
Their CEO is not Sanjay. it is someone
named David Gekckler I believe Sandis
CEO um so
>> this quarter they had revenue of almost
$6 billion they were up 97 their revenue
was up 97% sequentially so that's just
quarter over quarter um and of course
how do you do that you raise prices um
they're up 251% year-over-year
um here's a couple things that stood out
which are pretty crazy gross margin 78 8.4%.
8.4%.
That's like gross margins of a software company.
company. >> Yeah.
>> Yeah.
>> Yeah. Totally. Um
>> I have my notes here that it was 51.1%
the prior quarter. >> Okay.
>> Okay.
>> And you know the revenue estimate was
supposed to come in like I don't know
4.8 billion or something.
>> They come in a billion above that. I'm like
like
>> what like they are up 250% yearover-year
in like revenue
>> and you know the funny thing is that
they haven't shipped like that many
extra bits this is all pricing
>> yeah this is not as much you you think
like oh they sold maybe that much more
you know to account for like 97% quarter
over quarterart increase no not really
you can't bring on that much capacity
that quickly
>> I mean you're talking about making
wafers and stuff spend in a fab and that
stuff doesn't move in the time frame of
a quarter. So this is all price increases.
increases.
>> Totally. Wow. So you said their margins
were 51% last quarter and now they're up
to in the 70s.
>> 70s and I think they projected above 80
next quarter. >> Nuts.
>> Nuts.
>> The guidance is above 80.
>> They're going to make Yeah. Nvidia look
like they have work to do.
>> Nvidia is only like 75, right?
>> Yeah. Right. Totally. Totally. Of
course, Nvidia is a very very large
company and has had these margins for
quarter after quarter after quarter. So,
the question is um why why is Sandis
crushing it? Um and the what they the
story that I heard on the earnings call
was that they're they really believe
that the NAND market is transitioning
from commodity spot business to
something less cyclical. Um, and so the
the points that they made on the call,
and I'll read some quotes here, too. Um,
they have five multi-year supply
partnerships signed. Um, these three new
ones in quarter two, three. They'd
announced two previously and they they
had announced three more. Um, and those
three new ones account for $42 billion
of RPO remaining performance
obligations. You can kind of think of it
like a backlog. And this was the first
time they've actually disclosed that.
Um, so clearly there's long-term
agreements for many years and people are
like signing up and these customer
commitments um already cover onethird of
the fiscal year 27 bits. So it's not
just people signing up for 2026, but
they're already signing up and paying
for and committing to 2027 bits, which
actually have financial guarantees
backing them. So it's not, you know, it
people are putting their money where
their mouth is. Oh yeah, the CEO
Gekckler said he wanted to drive that
those long-term commitments to above 50%
of bits and already that one of the
longest contracts was for five years.
And so the quote that I thought was
pretty telling on how SanDisk is
perceiving this um from the CEO, he
said, "Last quarter, we were engaged in
discussions with customers on multi-year
supply partnerships, what we refer to as
new business models or NBMs. Um I am
pleased to share that we have
successfully advanced those
conversations with five multi-year
partnerships signed so far. They are
structured to lock in committed supply
for our customers and committed
financials for SanDisk. So kind of the
customers are prepaying almost. Our
customers commitments are backed by firm
financial guarantees. These partnerships
support durable structurally higher
earnings in a significantly more
predictable and less cyclical business
for SanDisk. We believe this marks a
fundamental evolution of our business
which is centered on deeper customer
alignment, enhanced visibility, and
long-term value creation. So, so far in
this quote, he said, "We're signing up
people for the long run and they're
paying for it." There hasn't yet, in my
opinion, been an argument for why it's
not cyclical. It could just be demand is
crazy and people are signing up. Um,
as we talked about earlier with Samsung,
when you're a commodity, when you're not
a commodity, you're actually competing
on um performance, for example. and he
hasn't talked about that yet. But then
later there was a quote that started to
get into this which I thought was
interesting. Um he said as AI models
scale from billions to trillions of
parameters and deployments advance from
simple inference to deep reasoning and
increasingly agentic systems NAND has
become a critical component of the
underlying infrastructure. inference
optimizations such as KV cache along
with workloads like rag require
substantial high performance low latency
flash to deliver real-time
responsiveness and quality of user
experience and then he goes on to say
basically NAND flash is emerging as the
only economically viable solution to
deliver that capacity performance and
efficiency and then he said you know
goes on to make the argument that
SanDisk is the best on these metrics and
therefore um that's why you know they're
capturing this value. So I want to throw
it over to you Vic. Do you think this is
SanDisk? Is this is this AI you know
truly needing lots of NAND and
performance matters to your economics to
your token cost? Um or and bits are
therefore no longer interchangeable.
Um or or is this just a demand thing
where it's like, hey, they're locking up
people for five years because they have
supply and it's sort of
preventing true competition on like
performance latency that that kind of thing.
thing.
>> Yeah. Yeah. This is this is good. Did
you look at uh the Deep Seek V4
announcement? Did you look into what is
happening there?
>> Oh, a little bit. But please tell us
about it. It's it's related
>> perfectly to LAN. You can see basically
how uh DeepSeek V4 has compressed KV
cache massively compared to the previous
version 3.2 2 I think and
you would think like why would you need
NAND now if your KV cache is compressed
the problem is that the KV cache still
when you're doing agentic AI and agentic
multi-turn AI none of it fits in HBM
you're running hundreds of agents they
all have to have long context long
running context multi-turn context all
of those key value cache pairs are just
too much to store in HBM or DAM. So if
you see the DeepSc V4 inventions
recently, it is optimized to be an
entirely SSDcentric
inference system where uh KV cache is
stored almost entirely on SSDs.
>> It is pretty amazing actually. And
earlier uh after GTC when uh Jensen
announced the inference context storage
system which now I think he calls CTX uh
context storage or whatever. So you
basically this is a bunch of uh SSDs in
a rack that is sitting there and
connected uh via like high bandwidth
fabrics to the GPUs so that they can
offload the KB cache matrices into this
this like SSD storage that is becoming
uh very important and at that time I
wrote a substract post pointing out how
this is going to change the inference
tokconomics forever because one of The
ways you can save on inference cost when
you're using, you know, the API. You can
look this up for any model is that
it really depends on a few things. First
of all, you've got the input tokens. You
got a certain number of tokens uh cost
per million tokens. Then you've got the
output number of u dollars per out
million output tokens, right? And it's
usually like a four or five is to one.
Output tokens are like five times more
expensive than input tokens because it
go under goes this like reasoning this
thinking process, right? The thinking
process costs a lot more money. So the
output tokens are more expensive than
input tokens. In the middle you have
another pricing point called the cash
hit or cash miss. So what that means is
that if you have KV cache and your uh
inference is able to reuse that KV cache
as much as you possibly can, right? You
have a cash hit and when you have a cash
hit, your cost of inference goes down
massively. So what people do now is to
basically have a bunch of system
instructions right up front and then
they'll like only append to the bottom
so that you can maintain your cache
hierarchy completely. So this is like
cash aware uh inferencing >> and
>> and
deepseek for example like there was some
metric saying that like in agentic use
you can make it use 95% of cash hit
rates. So 95% of the time it's hitting
cash. It in very few cases it even goes
to 99% it's hitting cash. So even now if
you go to the deepseek uh API pricing
page and see how much the cost has
dropped for the Deepseek V4 Pro. Okay,
it has gone down to a fraction of a cent
per million tokens. Okay, you can
compare that to Opus. I don't remember
the number right now but it's like
significantly high. So what this says is
basically SSD driven uh inference is
basically the only way forward. Deepseek
has kind of portrayed that clearly now
and you cannot store all of this
information on DM on or HBM. It's just
too much. It's too much and you almost
have an infinite storage of SSDs. You
know the cost per unit is like an order
of magnitude higher when it comes to DAM
versus SSD. So you've got this
essentially free storage and now you
have like capacity is a solved problem
if you go to SSDs. The only problem is
then uh you have this bandwidth
bottleneck because you can't access
stuff as fast uh from SSDs as you are right
right
>> you know so I've been meaning to read
this new paper that's come out from the
deepseek team called I have it on my
screen right now it's called dual path
breaking the storage bandwidth
bottleneck in LLM especially agentic
LLM. So I want to see exactly how all of
this ties in together. So I've basically
given a surface level overview of
justifying why SanDisk is saying that
nan storage is so important in the
future of agentic you know AI because
deepseek is already a data point that is
heading in this direction.
>> There's a long answer to your rather
simple question.
>> Oh this is this is so good. Okay so I'm
going to summarize it and then I have a
follow-up question for you. Okay, so
you're saying, hey, look, Deepseek is
showing us that uh SSDs are more
important than ever. NAN flash are more
important than ever. And if you think
about the tokconomics, it's way more
economical when you've got these cash
hits versus cash misses. So you got to
think really hard about can we have a a
very big cache or memory hierarchy and
can we store as much as possible and and
people are going to be very incentivized
even the end customers using APIs
running agents and stuff are going to be
incentivized to actually think about
memory and think about caching. Um
therefore SSD market is going to just
continue to grow. Um but my question for
you then is okay SSD market SSD TAM
going to explode. All good news. Invest
in these companies. But why SanDisk?
>> Not advice.
>> Oh yes. Not not this is not investment
advice. This is not investment advice.
We are just thinking very hypothetically here.
here.
>> Um do your own due diligence. You know
don't hold Vick and I accountable. Okay.
So uh the SSD TAM is going to explode
>> or is exploding. But is there is a bit
is an SSD bit a bit a bit or is is
SanDisk's bit better than someone else's?
else's?
I think a bit is a bit I mean is is NAND
is a rather established technology.
There isn't a controller
actually there is a controller
difference controller for the QLC NAND
is called Stargate and they're still
working on it and usually the more bits
you add so in going from a triple level
cell uh to a quadle cell you go from
having nine states a single cell has
nine states in a triple level cell in a
quad level cell it has 16 states a
single cell and now depending on how you
program the You should be able to
distinguish between 9 or 16 different
states otherwise you don't know what bit
it's holding. Is it holding like a
pattern 10 or pattern 15. So the
complexity of the controller for this
NAND gets significantly higher even when
you add a single bit like if you go to
pentle cells which exists in research
mode you will go to 32 levels. Now you
have to distinguish between 32 levels
you know that makes it very difficult.
Um I have a whole article on how all of
this works but yes there is a difference
in controller and how they how it works
and all that but it's not really a
differentiating factor. Okay the
controller needs to work well and as far
as I can understand it please correct me
if you know there are any storage
experts out there who actually work on
this stuff on a day-to-day basis they
always know better but as far as I know
a bit is a bit.
Yeah, we should follow up and bring on
someone Micron, SanDisk, Samsung,
whoever is listening out there. Of
course, to talk memory, but also to talk
storage. Let's I would love to talk
storage here and hear like a product
manager's argument for a bit is not a
bit because it probably comes down to
things like reliability or pricing per
bit or or other other factors than just
straight up read latency or right latency.
latency.
>> Yeah. Yeah. Yeah. Yeah. Maybe there's
some controller magic there because a
lot of the latency comes from adjusting
the voltages just right to read you know
this particular state of the cell. So
what it does is it it iteratively
programs the right voltage to reach that
state that takes time you know so if
there's like some fancy new algorithm
that can go quickly and you know that
you can reduce the latency that way that
may be a differentiating factor
>> actually from IEDM last year and if I
think we spoke about this on a past episode
episode
>> we spoke about a totally different kind
of cell where you can dramatically
increase uh the the the reliability
while still having like 36 states to a
cell, you know.
>> That's right. Yes. Circle one or something.
something.
>> The circle one. Yes. Yes.
>> Yes. Yes. All right. Listeners, go check
out our backlog. >> Yeah.
>> Yeah.
>> Okay. This has been a super good memory
deep dive. Let's let's carry on real
quick. Okay. So, let's talk hyperscaler
earnings quick. Um, so, hey, it was a
good quarter for Samsung. It was a good
quarter for SanDisk. Um, memory, storage
price through the roof, demand through
the roof. What does this mean for the
hyperscalers? We kind of already hinted
about it. Um, Microsoft
disclosed um $190 billion of capex for
2026. And the CFO, Amy Hood, said
roughly $25 billion of the calendar 2026
capex is specifically higher component
pricing. So, that's pretty crazy. We are
only a few years removed from someone
like Microsoft spending a total of $25
billion per quarter uh just on all of
their capex. And now we're talking about
in a year they're going to spend like
kind of a quarter's worth of capex just
on higher memory storage component prices.
prices. >> Um
>> Um
>> just give it to the memory guys.
>> Yeah, right. Pretty much. Uh hey, good
good time to be a memory guy. Meta Meta
raised their capex. Um and Zuck on the
call said most of the raise is higher
component costs particularly memory pricing.
pricing.
Google has a higher capex although they
didn't talk about that as much. Um
Sundar did say that their cloud their
cloud business was up 63%
um which was insane and Sundar said that
that would have actually been higher if
they were able to meet demand. So, still
talking about demand and capacity being
the the bottleneck there. And then
Amazon, they didn't uh talk specifically
about component prices and impacting
their capex, but but check this out.
This was an interesting qu like uh
commentary that came out. The CEO Andy
Jasse was asked if memory constraints um
are negatively impacting them. And he
answered by saying that for their cloud
business for AWS memory constraints are
actually driving cloud growth. And here
and that feels very counterintuitive.
And here's what he said. Who knows how
big this is in aggregate but he said one
of the interesting things that we see
right now with the change in price and
in supply on things like memory is it is
actually a further impetus pushing
companies who have been on premises
infrastructure into the cloud. And it's
because these suppliers, the memory
suppliers are prioritizing their very
largest customers, the hyperscalers,
cloud providers. And so therefore,
there's a number of enterprises who
can't get onrem infrastructure. It's
actually pushing them to speed up their
move to the cloud.
>> So I thought that was pretty interesting.
interesting.
>> And I think people are generally
comfortable in using like AWS cloud.
It's it's a very easy switch to go and
do your stuff on there because it has
been the the bread and butter for like
pretty much the software industry for
the last decade, right?
>> Yeah, totally. It it would be
interesting to dive in with them because
at this point you ask, well, who's still
running running on prem? And it's it's
got to be people who are still just
thinking about like, oh, what data do I
keep on prem because I'm in the
financial industry or health industry or
something. But even those folks are
still moving to the cloud and they're
getting pulled there faster because the
only place they can get compute and
memory is actually from the cloud
players which is very very fascinating. >> Yes.
>> Yes. >> Yeah.
>> Yeah.
>> If that's the only place you can get it,
that's where you go. I mean it doesn't
matter what you want, right?
>> It's like yo, sorry compliance team. I
know we're we're dotting all the eyes
and crossing all the tees, but literally
we have to we have to make this move
happen right now.
>> Yes. If you can buy memory from the
memory guys, then maybe.
>> Yeah. Yeah. Exactly. Exactly. Yeah. You
go, you go find it. You go find it.
Okay. Um, so, okay. AI accelerators were
talked a lot about across the
hyperscalers. Google said, you know,
publicly on their earnings call that the
TPU will be sold in a merchant capacity
for the first time and at multi-gawatt
scale. And and then actually if you look
in the 10 Q there was some risk language
now confirming that
>> the the risk to doing that is getting
cos and HBM allocation.
>> So I thought that that was interesting
>> and just go to EIB. Forget about co-ass.
>> Hey there you go. Yeah. Go to EIB.
>> That's right.
>> Intel. Intel for the win.
>> Yes. I hope to write something about
E-IB here shortly soon comparing it to
like co-was L which also has bridges but
to to unpack that for people um AWS
talked about trrenium um at a 20 billion
run rate growing triple digits and of
course people are not generally
asking to buy tranium and they're not
even necessarily uh renting it directly
but they are consuming Amazon Bedrock
and all these services that run on top
of tranium
Um, Tranium 2 largely sold out, Tranium
3 nearly fully subscribed and Jasse said
on the call that um, because they do
custom XPUs, they will save tens of
billions of dollars of capex savings
each year, which translates into several
hundred basis points of operating margin
advantage versus relying on merchant
chips. And on the one hand, normally I'd
been like, "Wow, that's so incredible.
They're going to save tens of billions
of dollars." But then right away I
thought, "Oh, so they can pay for their memory."
memory."
>> Yeah. Yeah.
>> We're making our own chips so we can pay
the memory guys. Yeah. Right now a funny
cycle. I don't know what what's what
what what is happening right now, but
yeah, we'll see.
>> Totally. And so um then Meta mentioned
of course they've had tons of uh press
releases building up to this quarter
about 1 gawatt worth of MTIA with
Broadcom and they had showed their road
map of you know four chips in two years
uh MTIA XPUs. We had a great
conversation with Meta recently on that
with Matt Steiner and hopefully we'll
have more getting into it further. Um
they now Meta did mention significant
deployments with AMD and then also
running on some new Nvidia. So they were
talking uh up all their multi- silicon
vendor uh partners. Um so with that
we've run long. I think we should call
it quits here. Thank you everyone for
listening. Um YouTube YouTube folks
thank you for your comments. We see that
you would like Vic to explain again from
our Google TPU one. Why is it seven hops
with I think it was with boardfly and
not 16 hops. So we will follow up with
you. We'll draw a picture sometime. So
keep writing your comments, keep writing
your questions. Thanks for everyone for
listening, watching us on YouTube,
watching us on X and reach out. Uh
thanks again. We'll talk to you guys later.
Click on any text or timestamp to jump to that moment in the video
Share:
Most transcripts ready in under 5 seconds
One-Click Copy125+ LanguagesSearch ContentJump to Timestamps
Paste YouTube URL
Enter any YouTube video link to get the full transcript
Transcript Extraction Form
Most transcripts ready in under 5 seconds
Get Our Chrome Extension
Get transcripts instantly without leaving YouTube. Install our Chrome extension for one-click access to any video's transcript directly on the watch page.
