Hang tight while we fetch the video data and transcripts. This only takes a moment.
Connecting to YouTube player…
Fetching transcript data…
We’ll display the transcript, summary, and all view options as soon as everything loads.
Next steps
Loading transcript tools…
How OpenAI Shapes Its Research And What's Next - EP 46 Mark Chen | Core Memory Podcast | YouTubeToText
YouTube Transcript: How OpenAI Shapes Its Research And What's Next - EP 46 Mark Chen
Skip watching entire videos - get the full transcript, search for keywords, and copy with one click.
Share:
Video Transcript
Video Summary
Summary
Core Theme
This content is an in-depth interview with Mark Chen, Chief Research Officer at OpenAI, discussing the company's research direction, talent acquisition strategies, competitive landscape, and the future of AI development, particularly focusing on the pursuit of Artificial General Intelligence (AGI) and its implications.
Mind Map
Click to expand
Click to explore the full interactive mind map • Zoom, pan, and navigate
on the recruitment wars. I mean, this
got a lot of attention clearly
>> and it looked like Meta was quite
aggressive. What exactly does this tit
for tat look like? What st what stage
are we at?
>> Yeah, I mean there is a pool of talent,
right? And everyone kind of knows who
they are. And um you know I think many
companies have realized that one of the
key ingredients not the only important
ingredient but one of the key
ingredients to building a great AI lab
is to get the best talent and um I think
not a surprise that you know Meta has
been aggressively employing this
strategy. Um
>> you know we haven't sat back idly and I
actually want to tell this story from
OpenAI's point of view. Um I think that
a lot has been made in the media of oh
you know there's this unidirectional
flow of talent over to Meta. Um but the
way that I've seen it is you know Meta
they've gone after a lot of people quite
unsuccessfully. So just to give you
context right within my staff within my
direct reports uh before they hired
anyone from open I think they went after
half of my direct reports and they all
declined. Um, and of course, you know,
if they have something like $10 billion
of capital per year to deploy towards
talent, um, they're going to get
someone. So, I actually feel like we've
been fairly good about protecting our
top talent. And, you know, it it's been
kind of interesting and fun to see it
escalate over time. Um you know uh some
interesting stories here are Zuck
actually went and yeah handd delivered
soup to people that he was trying to
recruit from us
>> like a a just a just to show how far he would
would
>> yeah I think he he handcooked the soup
and and you know it was it was shocking
to me at the time but you know over time
it I've kind of updated towards these
things can be effective in their own way
right and you know I've also delivered
soup to people that we've been
recruiting from from Meta.
>> You're doing a soup soup counting.
>> I' I've thought of, you know, if I had
an offsite, the next offsite for my
staff, I'm going to take them to a
cooking class. Okay.
>> And yeah, I mean, it's just been um
Yeah, but I I do think, you know, um
there's something I've learned about recruiting.
recruiting.
>> Did you cook your soup? >> Uh
>> Uh
it's better if you get like Michelin
star soup. [laughter] You know what I mean?
mean?
>> Yeah. No, no, no. I think Dejo is very,
very good and um probably better than
any soup I could cook. Um but yeah, I I
I do think there is something I've
learned about, you know, just um how to
go u aggressively after top talent. And
I think uh the the thing I've been
actually very inspired by is that um you
know at OpenAI um even among people who
who have for meta, I haven't heard
anyone say AGI is going to be developed
at Meta First. Um, everyone is very
confident in the research program at at
OpenAI and one thing that I've made very
clear to to my staff to to the whole
research ro is we don't counter uh
dollar for dollar with with Meta um and
the multiples that below what Meta is
offering that people are very happy to
stay at OpenAI gives me so much
conviction that you know people really
believe in the upside and believe that
we're going to do it
>> well and you and Alex Alex thing. He
used to be one of the math compet the the
the
>> Yeah. Yeah.
>> I'm sure you guys hung out.
>> Yeah. I mean, I I have hung out with
Alex a handful of times, but we don't do
much anymore. Yeah. I [laughter] mean, yeah.
yeah.
>> Why did soup become the thing? It was
just it just
>> I don't know. You know, it's there's
been soup, there's been flowers, there's
been anything you can think of under the
sun, but um I don't know. I think, you
know, life's an adventure. I I play into
the meme.
>> Yeah. Yeah.
>> Is there any any poker strategy to
employ like as you're you're thinking?
>> Well, again, I think it really goes back
to what I've said about the media
narrative. Um, the game is not to retain
every single person in the org. It's to
trust in this pipeline that we have for
developing talent and to understand who
the key people we [music] need to keep
are and to keep those. And I think we've
done a phenomenal job at that. [music]
>> We have a special treat today. I'm
excited. Mark Chen is here from OpenAI.
Um he's the chief research officer. He's
somebody I've gotten to know over the
last couple of years. Thank you so much
for coming. No, it's been great to know
you for for so long.
>> I feel like uh
you know there's a handful of people in
this world working on this very
important project and and I mean you're
right at the top of it. So it's it's so
cool to uh to have a chance to chat.
>> Yeah. Thanks for having me on.
>> It's a it's my pleasure and and I mean
there's a bunch of things that I want to
talk to you about because I've gotten to
know you like we said over those last
couple of years. I want to get I want I
want people to know a bit more about
your biography and but I also know
there's going to be AI enthusiasts who
who want us to go deep on a couple
things there. So we'll we'll try to do
everything. Um, I wanted to start just
by giving people a feel for your job,
which in my head
and you I mean just correct me if I get
any of this wrong, but you're you're you
know Sam has been he he's really into
research. He's the boss. He's kind of at
the top of the food chain. But then you
and Yakob are working
>> together to shape Open I Open AAI's
research direction.
and and then you're in this
additional part of this role is is is
actually deciding which compute goes
where onto these projects. So you kind
of have to chart where OpenAI is heading
and then the mechanics Yeah. of how
you're going to get there. Yeah. And
this always strikes me as a horrible job
because I picture people um doing
everything in their power to get GPUs
from YouTube.
>> It's true. people are very creative in
the ways that they try to make backroom
deals to to get the GPUs they need. But
yeah, I mean it is a big part of the
job, right? Uh figuring out the
priorities for the research or um and
also being accountable for execution. So
really to that first point um you know
there's this exercise that Jakob and I
do um every 1 to two months where we
take stock of all the projects at OpenAI
and uh it's this big spreadsheet about
300 projects and we go and try to deeply
understand each one as best as we can
and um really rank them and I think for
a company of 500 people it's important
for people to understand what the core
priorities are and for those to be
communicated clearly with explicitly
verbally and also through the way that
we allocate compute.
>> All right. What do we do at Core Memory?
We cover innovative, fastm moving,
forwardthinking companies, which is why
Core Memory is sponsored by Brex because
B is the intelligent finance platform
for many of these companies. 30,000
companies from startups to the world's
largest corporations
rely on Brexit technology for their
finances. They've got smart corporate
cards, high yield business banking, and
expense automation tools that are
fantastic. I hate doing my expenses. And
Brex's AIS software run right through
those expenses. Figure out where we're
spending money and take care of so much
stuff for you so you don't have to waste
your time on it yourself. Go to brex.com/cormemory
brex.com/cormemory
to learn more and just, you know, get
with the program. Let's get going. Let's
get out of this archaic finance software
and move toward the future core memory
and bre. So you've got when you're
talking about the 500, this these are
the 500. This is the heart of the
research team in an organization now
that's thousands of people. Yeah. Okay.
So, and then in that when you're talking
about this 300 projects, I I imagine I
mean obviously some of those are the
giant frontier models and then some are
probably experiments that people are
working on. And so like how do you possibly
possibly
keep track of all that and then come to
some sort of conclusion about what
merits GPUs and what doesn't.
>> Absolutely. So I think um it is very
important when doing this exercise to
keep the core road map um in focus and
one thing that differentiates I think
open AAI uh with other other big labs
out there is open AAI has always had
core exploratory research at its core.
Um we are not in the business of
replicating the results of other labs of
kind of catching up to other labs in
terms of benchmarks. That isn't really
our bread and butter. We're always
trying to figure out what that next
paradigm is and we're willing to invest
the resources to make sure that you know
we find that right and um I think most
people might be surprised at this but
more compute goes into that endeavor of
doing exploration than it is to training
the actual artifact. It must be it's
still got to be how do you stop yourself
from being persuaded by someone because
everybody's going to put I you know like
when I think about this sometimes I
picture when I was at the New York Times
you would have this page one
>> meeting where
>> everybody wants to be on page one. >> Yeah.
>> Yeah.
>> Everybody thinks their story is the most
important story. They're all doing their
very best job to tell you why this thing
is so important. Everybody in that room
has worked
>> weeks, months on on whatever they're
pitching. And so it feels like life and
death and that Yeah. I mean it just it's
it seems so difficult for me.
>> Yeah. No, it it is also a difficult
process and I think the hardest calls
you have to make are you know this is a
project that we just can't fund right
now. Um but I also think that's good
leadership. You need to clearly
communicate that hey these are the
priorities. This is what we're going to
talk about. These are the types of
results that we think move the research
program. And you know there can be other
things but those have to be clearly
number two.
>> And when you like you were talking about
um not being reactive to your
competitors. Yeah. When I was looking
through my notes, I I don't know if I
could go to the line
>> quick enough, but I mean, this this was
like a point of pride that I saw that
you feel like um
some of the other companies are well,
you know, you guys were in this position
where you were ahead um and and setting
the bar for others and so so they were
reactive, right, to to what you had
coming out. we happen to be doing this
interview a few days after Gemini 3 came
out and you know there is a degree to
which your rivals >> um
>> um
>> at times like yeah I mean there's this
back and forth going on and and I know
the benchmarks are sort of controversial
how valuable they are but you
[clears throat] know people go ahead on
these things so how do you also um as
time has gone on
maintain that luxury or that
intellectual position where you feel
like we're just going to do what we're
going to Yeah, I I think AI research
today, the landscape is just much more
competitive than it's ever been. Um, and
the important thing is to not get caught
up in that competitive dynamic because
you can always say, hey, you know, I'm
going to ship an incremental update that
puts me in front of my competitor for,
you know, a couple weeks or a couple
months. And I don't think that's the
long-term sustainable way to do research
because if you crack that next paradigm,
that's just going to matter so much
more, right? you're going to shape the
evolution of it. You're going to
understand kind of all the side research
directions um around that sphere of
ideas. And so when you think about kind
of um our RO program as an example of
this, right, we bet more than 2 years
ago that we're really going to crack RO
on language models. And this was a very
unpopular bet at the time. Uh you know,
right now it seems obvious, but back
then the environment was, hey, you know,
there's this pre-training machine that's
working great. There's this
post-training machine that's working
great. why invest in something else and
I think today everyone would totally
tell you you know thinking and language
models it's just a primitive you can't
have uh can't live without and um so
we're we're really there to make these
bold bets and to figure out how we can
scale and build the algorithms to really
scale to orders of magnitude more
compute than we have today. It's just
it, you know, I mean, and I get that
intellectually in my, you know,
it gets harder as you guys started as a
like basically a pure research company.
When you look at OpenAI today, I mean,
you have product line. There's parts of
OpenAI that look much more familiar to a
>> mature Microsoft or a Google where you
have product lines, you've got all these
different things that you have to serve.
Typically, I feel like you guys are
still young enough, so maybe you don't
have these exact pressures yet, but you
know, as those companies go on, it
always becomes, well, we're more focused
on the things that are serving the
bottom line than spending a ton of money
on research always seems to get like
dwindled down
>> over time.
>> Yeah. And I think that's really one of
the most special things about OpenAI. At
its core, we're a pure AI research
company. And I don't think you can say
that of many other companies out there.
And you know we were founded as a
nonprofit and I joined during that era
and I think the spirit is you know build
AGI advance AGI research at at all costs
um and do it in a safe way of course um
but yeah I actually do think that's the
best head fake to really creating value
right if you focus and you win at the
research the value is easy to create so
um I think there's a trap of getting too
lost into like oh you know um let's
drive up the bottom line. Uh when in
reality if you do the best research that
part of the picture is very easy.
>> And you you started in 2018 and so you
feel like that soul that that that um
>> yeah that core culture and that core
nucleus it's it's really persisted.
>> It's still there. What does Elon says uh
what does he he says we shouldn't call
any of you guys researchers. It's just
engineering, right?
>> Yeah. No, I think yeah, we No, it's it's
true because I feel like once you have
this hierarchy um and you elevate let's
say research science um as a thing
beyond engineering, you've completely
already lost the game cuz
you know when you're building a big
model uh so much is in the practice of
optimizing all of those you know little
percentages of you know how do you make
your kernels that much faster? um how do
you make sure the numeric all work um
and that's a deep engineering practice
and if you don't have that part of the
picture you can't scale to to the number
of GPUs we use today
>> so because I think there well okay but
there is like a mystique that surrounds
a researcher versus an engineer you know
what I mean so are you were do you feel
like um it is better to kind of stay
levelheaded on that is that is that kind
of what you're saying or or
>> on Well, I I just feel like researchers,
they come in so many different shapes,
you know? Uh some of our best
researchers, they're uh they're the type
that, you know, they come up with a
billion ideas, right? And many of them
are not good. But, you [laughter] know,
just when you're about to be like, ah,
is this person really worth it? They
come up with some, you know, phenomenal
idea. Um some of them are just, you
know, so good at kind of executing on on
the clear path ahead. And so there's
just so many different shapes of
researchers and I think it's hard to
just lump it into one stereotypical type
that works.
>> That makes sense. Um,
>> okay, I won't I won't belabor you with
too many competitive like rival
questions. It's just since Gemini 3 did
come out, I did wonder what happens with
you personally or the team when one of
your rivals puts it like does everybody
go and look and see what it can do? Is
there like a is there a prompt or a
question that you th you often throw at
these new models to see what they can do?
do?
>> Yeah. Yeah. So, um to speak to Gemini 3
specifically, you know, it's a pretty
good model. Um and I think one thing we
do is try to build consensus. You know,
um the benchmarks only tell you so much.
Um and just looking purely at the
benchmarks, you know, we actually felt
quite confident. um you know we have
models internally that uh perform at the
level of Gemini 3 and we're pretty
confident that we will release them soon
and we can release successor models that
are even better. Um but yeah again kind
of the benchmarks only tell you so much
and I you know I I think everyone probes
uh the models in their own way there
there is this math problem I like to
give the models uh
>> I I think so far none of them has quite
cracked it even the thinking models um
so yeah I'll wait for that
>> is this is this like a secret math problem
problem
>> oh no no um well if I announce it here
maybe it gets trained on but um yeah I I
do think uh it's one of the nice puzzles
of last Here it's this um the 42
problem. So you want to create this
random number generator mod 42 and you
have access to a bunch of primitives
which are random number generators
modulo primes less than 42. You want to
make as few calls on expectation to
these subg generators as possible. Um so
it's a very cute puzzle but um the
language models they get pretty close to
the optimal solution but I haven't seen
one quite crack it. Okay, this is we're
heading down a direction I want to ask
you about. But then just before we get
there, so I know you're I've seen you
you're very competitive. You've also
told me I think I found I love
competition. I hate to lose somewhere.
somewhere.
>> I really hate losing. I [laughter] hate losing.
losing.
>> Yeah. So I'm picturing I'm just curious
if this is at all right. I mean, if you
know if we know Gemini 3 or whatever is
coming out on a Thursday, I mean, are
you up at like midnight throwing that
problem at it or is it is it not quite
that drastic?
>> Um, no. I mean, I think it's in long
arcs, right? Um, and any endeavor, like
I, you know, I'm kind of a person who
has obsessions. I I think any endeavor
you have to play the long game. Um, and
you know, we've actually been focusing
on pre-training, specifically
supercharging our pre-training efforts
for the last half year. Um, and I think
it's a result of some of those efforts,
uh, together with Yakub, focusing and
building that muscle of pre-training at
at OpenAI. Uh, you know, crafting a
really superstar team around it, making
sure that all of the important areas and
aspects of pre-training are emphasized.
Um, that's what creates the artifacts
today that feels like we can go
head-to-head with Gemini 3 easily on on pre-training.
pre-training.
>> Okay. And I I want to ask about the
pre-training stuff because I' I've been
talking to all you guys about this a
lot, but but um Okay. But so but you're
saying that um you're less obsessed
about lobbing um problems at these new
models just when they appear and more at
this this long journey. >> Absolutely.
>> Absolutely.
>> Yeah. Okay. Um okay. Hey, the reason I
want to talk about sort the the puzzle
that you were at, I mean, I
>> you know, I first met Yakob before
OpenAI ever started when he was doing a
coding competition and I got I got like
super into coding competitions for a
while. There's this guy
>> Kennedy. I don't know if he's still
famous, but he was like the Michael
Jordan of of these coding competitions.
And so I went to watch one at uh
Facebook used to I don't know if they
still do, but they had an annual
>> Hacker Cup.
>> Yeah, Hacker Cup. And and that's where I
saw YaKob for the first time. And then I
know you I think did math competitions
in high school. Yeah.
>> Like probably grade school through high
school. And then you also did you also
do III?
>> So I got into coding really late in
life. Um it was a roommate in college
that convinced me to take my first
coding class and um I had all the hubris
of a mathematician at that time whereas
like you know math is the purest and
hardest science and that's where you
know you really prove your worth. I
mean, I think I was probably too into
the competition back then. Um, but yeah,
I mean, it became this super rewarding
endeavor and um and you know, it it
started out as purely a way to keep in
touch with my friends from college. Um,
>> you went to MIT.
>> Yeah, I went to MIT. Um, you know, I
graduated and every weekend we would
just log on and do these contests just
to to keep in touch with each other. And
um, you know, over time I found myself
having a talent for it. you know, I um
started competing fairly well and then
writing problems for for contests like
the USA Coding Olympiad. Eventually
started coaching that team and yeah,
it's been a great community where I've
met people like Scott that you know.
>> Yeah. Yeah. Okay. So, you So, I think
lots of people might be familiar with
like math competitions because they
probably see kids going through that.
The IOI and in these coding competitions
are a little bit different. I mean, it's
I mean, you'll know it so much better,
but when I saw them, I mean, it looks
like a it's almost like a word problem
that's a puzzle, and you're trying to
kind of find the most efficient and
correct way to solve that, and you're in
this race against everybody
>> and and everybody's like writing code on
their computer and then and then some
people try to get there really fast, but
then their thing kind of doesn't solve
the problem, right? And then, you know,
there's like this trade-off, right?
Absolutely. Right. And so so you you
actually were on the MIT team.
>> No, no, it's something I did after college.
college.
>> After college. Okay. But today you are
like the coach of the US national.
>> Yeah. One of the coaches.
>> One of the coaches. Okay. And and
>> was it last year or the year before?
Like the US like we hadn't won one in a
long long time, right? Yeah.
>> Yeah. Didn't we?
>> Yeah. Yeah. Yeah. So um Yeah. I mean the
the team I mean it's
you know you can never predict what the
makeup of top talent looks like every
year. Uh but we had a very spiky team I
think two years ago. Okay. And um yeah I
believe they won the Olympiad
>> because I feel like usually it's like
China or Russia or like uh
>> Barus and Poland. I mean right? Yeah.
And and so this compet the big
competition [clears throat]
>> takes place in a different country every year.
year.
>> What does it look like? How many people
show up?
>> Yeah. Yeah. So they they take the top
four students from every single country.
Um it is as much of a competition as it
is a social event, you know. Um this is
a tight-knit community. They all do go
on to do phenomenal things and um yeah,
it's this intense two-day contest where
each day you get just three problems. Um
5 hours to solve them and you can really
feel the adrenaline and uh all the
pressure in the room. Um but it's also
great fun. I think um people settle down
and they make you know lifetime friends
through it. What do you And like as
coach, I mean, you're so freaking busy,
man. How what do you uh how much time do
you spend on this? What does that look like?
like?
>> Honestly, um the kids are so
self-motivated. Sometimes it's really
about just managing their performance
and and strategy. Um I think, you know,
you're going to have good days, you're
going to have bad days, you're going to
have good hours within the contest, bad
hours, and you can't let that get into
your head. Um there's a lot of
similarities between managing
contestants and managing researchers. Uh
it's like on a much longer time scale,
but you know like researchers have good
months, bad months. You know, you can't
really let those strings of failures get
into your head because that's just the
nature of research, right? And um I
think a lot of its morale management um
at a certain point. Um
yeah, I think one other interesting
thing that contests have helped me
realize lately is when you put the
models and deploy them towards solving
these these contest problems which
they're quite good at these days. Yeah,
I was going to ask you about that.
>> Um they they work in a very different
way from humans. You know, we typically
think of these machines as you know
they're very good at pattern
recognition. You can take any problem if
it maps to a previous problem, it's
probably going to be able to solve it.
But what I've noticed is in some of the
previous IO, there's this problem like
message is very ad hoc. Um, I didn't
think the models would solve it at all,
but actually one of the easier problems
for the AI. So
>> yeah, I mean this has given me the sense
that AI plus humans in frontier
research, it's going to do something
amazing just because the AI has a
different intuition for what's easy and
what's not. So, okay. Is it vaguely, you
know, when when D mind did the whole
Alph Go thing,
>> you know, there was that moment where it
was doing things human, it was playing
in ways humans hadn't played before. So,
kind of like vaguely similar to that or
>> I I think so. I think so. Um I think
really with GPD 5 Pro, right? Um there's
been an inflection point
in frontier research. Um and one of the
best anecdotes I have for this is you
know I think 3 days after the launch um
I met up with a friend who was a
physicist and um you know he had been
playing around with the models uh felt
like you know they they were cute but
not super useful and I challenged him
with the pro model just try something
ambitious and you know he put in his
latest paper um it thought for 30
minutes and just got it and I would say
that that reaction in that moment Um
it was kind of like seeing Lisa doll
during that you know move 37 move 38. Um
and I just think that is just going to
keep happening more and more for
frontier mathematics for science for
biology material science. Um the models
have really gotten to that point.
I I was gonna ask you this question
which is not very original because I
think we've been doing this ever since
kind of big blue and and all the chess
stuff but yeah just as somebody who had
followed all these competitions if
>> I don't know there's a sadness when you
start seeing
>> these models solving these things that
were like the
>> this height of these achievement for
these very unique human minds.
>> Well um yes and no. I mean I was good at
competitive programming. I was never at
the absolute top and um maybe this is a
way to get revenge. No, [laughter] but I
I I do think um no there's certainly a
moment for myself, right? Um you we
tracked coding conscious performance
while we were developing reasoning
models for a while and you know at at
the start of the program you know they
were not super great you know uh at the
level of any average competitor going uh
going going into the contest. And yeah,
over time they just started creeping up
and up in in terms of capability and and
you still remember that moment when you
walk into the meeting and they have
where your performance is and then the
models exceeded that.
>> Um, man, that was also a shock to me.
It's just like, wow, we've automated to
this level of capability so fast.
>> And of course, you know, Yakob was there
still a bit smug, but within like one or
two months, it was also surpassing him. So,
So,
>> um, yeah, no, the models are at the
frontier today, right? Um it's so clear
by even through the results we've done
this summer um at coder right top
optimization competitive programmers in
the world um I think it achieves second
place there um and so really it's jumped
from you know hundth place last year to
top five this year and
>> like do you think we'll still be doing
these competitions in 10 years?
>> I think so. I mean they're just fun. Um,
I mean certainly a bunch of people who
use it to, you know, had their resume
are going to drop drop off from doing
it, but I think the people who've always
excelled at it the most are people who
just do it for the fun of it. And I
don't think that'll go away.
>> When I was doing this story, I mean,
they were telling me that like if you're from
from
>> Russia or I don't know which countries
that you basically get like an automatic
free ride to any university that you
want. I mean, I see the guys on the US
team go to like Harvard and MIT, so they
seem to be doing okay, but it was it
doesn't seem like the US has a [laughter]
[laughter]
>> Yeah. I mean, don't you think it's going
to Yeah. I mean, interviews, right?
They're going to be kind of broken going
forward. And I everyone's seeing this a
little bit. And, you know, even college
exams or college homework, it's it's all
broken at this point, right? And I I do
think we're going to need new ways of
assessing and gauging, you know, who's
performing well, who's learned the
>> where somebody's actually at.
>> Yeah. Yeah. So I mean I I've had this
idea here where um maybe for our
interviews we should just have
candidates talk to chat GPT and you know
it's a special kind of chat GPT where uh
the model is trying to gauge whether you
know the material or or whether you're
at the capability level to work at
OpenAI. Um, and you know, you have to
have this conversation with it that
convinces it deeply you belong at
OpenAI. And of course, you know, you
can't be allowed to jailbreak it, but
and we look at the transcript after, but
maybe like tests like this will more
accurately reflect in the future whether
you know.
>> So, you don't do that yet, but you're
thinking about
>> Yeah. Yeah. Just creative ways to revamp
the interviews.
>> Yeah. Yeah. Well, I mean, Silicon Valley
is famous for doing the like
>> brain teasers during the interviews and
everything. Yeah. Um, so you
I mean we tal you were very good at math
[laughter] growing up and and I think
you were you born on the east coast?
>> Uh yeah, born on the east coast
>> and then you lived on the west coast too.
too.
>> On the west coast and then you lived in
Taiwan for like for like um grade school
to high school.
>> Four years.
>> Okay. Your parents worked at Bell Labs. >> Yep.
>> Yep.
>> So you come from like engineering stock. [laughter]
[laughter]
Uh I mean it's a really interesting
background um just because you kind of
got like a flavor for all these
innovation hubs and and especially with
your parents being at at Bell Labs and
most of
>> I mean yeah I just grew up in a very
scientific environment you know dinner
table talk was puzzles and things like
that and um I also got kind of the more
traditional you know Bell Labs east
coast experience um on the west coast my
dad came to do a startup so a little bit
of that kind of new company got got
exposed to that when I was young as
well. And of course the big jump to
Taiwan, right? And I think it's a huge
culture shock. You you wear uniforms,
you're in a school, it has barbed wire
around the school, right? And um and
also getting exposed to kind of that
level of rigor. Um I think it was just a
number of really great experiences
growing. So, like the schools were harder
harder
>> um or
>> Well, I would say it was just much more
kind of uh you know, it's just there's a
little bit less flexibility and freedom
in the school system, but I think it
also teaches you something.
>> Yeah. Okay. Since day one, the Core
Memory podcast has been supported by the
fine people at E1 Ventures. They are a
young and ambitious VC firm in Silicon
Valley, investing in young and ambitious
companies and people. Thank you so much
to E1 Ventures for all your support. >> Yeah.
>> Yeah.
>> And you knew you wanted to come back to
the US for college.
>> Absolutely. Yeah. >> Okay.
>> Okay.
>> And then Okay. So, you're you're at MIT.
Um you're kind of like in this
interesting group. I guess MIT probably
always has a interesting
>> Oh, man. Yeah. 2012 was such a great group.
group.
>> Yeah. Like, who is there sort of like an
allstar list?
>> Oh, I mean it was a great year. Like, um
I don't know if you knew like Jacob
Steinhart, you know, he's doing Trans
Loose now. Um he uh and I used to do
projects together in in computer science
class. Um there was Paul Cristiano who's
um uh a bunch of really
>> phenomenal. He
>> worked at Open Yeah. Um a bunch of kind
of big names in in AI came from that year.
year.
>> And then and then we were talking about
the the competitive um coding like Scott
Woo who's at Cognition. I mean, he's
kind of like famous now as like a meme
on X for his math abilities, but and you
just got you got to know him through the
coding competition.
>> Oh, yeah. Through the coding community.
>> Okay. Okay. And then now I see the
competitive end
of you guys. The the output of this to
me look like poker these days. I think I
I was we were at an event which I think
I have to we have to keep secret or
something like the the specifics on this
event. But um I think I'm okay to talk
about this part which is like I
>> late at night
>> I'm walking by this table there's you
Scott I think Sham from Palunteer and
then and like a handful of other people
in a like a fairly int looked intense to
be maybe it's it's not for you guys but
like a fairly intense poker game. So you
guys, this is where you've applied your
your math and competitive skills now.
>> Yeah, I mean poker is a really fun game
and you know I've talked about my life
in terms of a series of obsessions.
Poker was definitely one of these
obsessions in in the past and um I think
the big revelation for me in poker was
um you know it's so much more a
mathematical game than a game of reading
people and bluffing. And I think the
more you learn about poker, the more you
update in that direction, right? Um, and
I think that's, you know, I used to be a
terrible bluffer. And when you know it's
mathematically correct to bluff, then
it's so easy, right? It's like you you
don't feel any nervousness around it.
Um, and yeah, it's just so interesting
that you have a game that I think is
perceived as so human. Um, but the
underlying mechanics and how to win are
so deeply mathematical. And yeah, I kind
of thought about this the other day
where you know there's something about
that in language modeling too, right?
You have um this deeply human process of
generating language, but there's this
mathematical machine that can really do
it as well as we can.
>> I think about that part all the time as
a writer and then I
>> did all this philosophy back in college
with Vickenstein and all these guys
thinking about these things. Yeah. Um,
well, how do you find like an edge? If
you and you and Scott both strike me as
like supernaturally good at math, I
don't understand how one of you is out
calculating the other person.
>> Um, well, no, I mean, it it is mostly um
a forum for us to just kind of um just
hang out and catch up with each other.
Um, and you know, today we don't take it
as ser I think there is an element to
taking something like poker very
seriously that takes the fun out of it.
and and so you know I my obsession with
poker I think has ended more than a
decade ago and now it's just fun.
>> You're just saying this cuz I saw Scott
win both [laughter] days. I think
>> you might be right about that.
>> He was taking it he was taking it quite
seriously. Um and and so like coming out
of college I mean you had in some sense
I was
>> Oh I beat him on the plane though.
>> You beat him on the plane right home.
Okay. All right. All right. So you did
you did was it just you verse him or it
was like a group thing
>> and maybe three or four people.
>> Okay. [laughter]
>> Um I feel like a lot of uh feel like
there there's three
I don't think I'm overgeneralizing too
far especially among like if you cut
back to the 2018 >> right
>> right
>> sort of time frame. Um, as far as like
people who were in AI at a high level, I
mean a lot had academic backgrounds. A
lot were math prodigies or had gone on
to sort of
>> take their math background and and get
into robotics or something physics like
that. And then and then there's this
other bucket which is people who had
gone to Wall Street and done
>> um high frequency trading and and quants
and and things like that. So that that
was the first path that you took was you
went straight from MIT to to Wall Street.
Street.
>> Yeah. I mean I don't wear that badge
with too much pride and [laughter] to be
honest like you know it it was a a a
path that was fairly common for you know
very quantitatively oriented kids at at
MIT. Um, and I mean it was certainly a
very meritocratic system, right? You
could apply intelligence and there's a
very concrete reward function, the the
amount of kind of profit that you would
make. Um, but I think culturally it was
hard for me. I it was a place where when
you discover something, your first
instinct is to just keep it away from as
many people as possible because your
knowledge is what gives you your worth.
and and so it felt like you would have
out of this an outgrowth of just even
internally at a company like these
competitive dynamics and people weren't
very trusting of each other. Um I think
it was also felt like such a closed
ecosystem, right? I think we today don't
feel too much like you know when when
someone in HFT finds a breakthrough that
makes their algorithm a little bit
faster, no one else feels it, right? And
I over time just kind of felt like you
know I woke up after four or five years
we're competing against the same exact
set of players. Everyone was a little
bit faster but had the world really
changed that much for it? Um, and it
felt like time to do something else,
right? It just a bunch of things lined
up back then. You know, there's the Alph
Go um, uh, match which I think was a
huge inspiration for a lot of people at
at OpenAI. And
>> did you play Go? Uh,
>> I did not, but
>> I think
the sense in which, you know, the the
model was able to do something creative,
I really wanted to understand what was
going on behind that. So you're watching
that happen and had you been had you
been at all reading AI research papers
and things like that or
>> to be honest no and then I saw that
event um it was really inspiring and
that's when I started doing my deep
dives into AI. So I um one of my goals
after seeing that was reproduce the DQN
results. um this is a network that was
able to uh play a lot of Atari games uh
effectively at a superhuman level and um
and going from there you know that's how
I got my start in AI
>> and you were doing that like on the side
as you so you just work all day and then
go back and try to
>> Yeah. Yeah. Yeah. >> Yeah.
>> Yeah.
>> Okay. I mean it is weird. I remember I
was interviewing George Hotz like
>> it might have been roughly 2018. Um
maybe a little before that, you know,
and he had just done this thing
>> building a self-driving car on his own
in his garage. And then, you know, I
mean, it's George, so he says
>> large statements sometimes that may may
or may not be like exact or spoton or or
apply to other people. But he's like
he's like AI is still so young. You can
you can basically learn the whole field
if you read I don't know what the number
was 10 20 30 research papers. I mean it
is fascinating to me that it's like
>> old in many ways stretching back decades
but this particular moment
>> it's very shallow. Um I I always give
this advice to people who are
intimidated by getting into AI. so
shallow like just spend three to six
months um picking some project like
maybe you know reproduce DQN and you can
get to the frontier very quickly. Um the
last couple years has added a little bit
of depth but it's not anything like you
know theoretical math or physics. Do you
think is this a field where
I asked Jaob this the other day I don't
know why I'm obsessed with this but you
know like in mathematics you see you see
people tend to do their best work in
their 20s or to have the big
breakthrough and then it's very hard as
they get older to like have that same
>> kind of moment um like what you're
saying you know are we are we dependent
on on young people reading these papers
and then having some insight or is this
is Is this something where it's more you
can you keep going throughout your whole career?
career?
>> I mean, I think you can keep going. I
mean, OpenAI itself does have a pretty
young culture, but I don't think you
need to be young to do good research. I
think there is something about being
young and having less priors about this
is the way it's done. Um, I think over
time you may develop your own vision.
Um, which is a good thing, right? Um but
it also locks you into a frame of mind
of like oh you know this is how research
is done this is how good results come
out and I think younger researchers tend
to have a little bit more plasticity
around that that concept. >> Yeah.
>> Yeah. >> Yeah.
>> Yeah.
>> Um so as your career is at open AI is
funny because it seems like you walked
in the door and had a very um important
large position from the get-go. But when
you first got there in 2018, it must
have been what, like uh 50 people.
>> Oh, no, it was much closer to 20.
>> Much closer to 20. Okay.
>> And it it really looked like two teams
back then. I came in as a resident. So
someone who, you know, clearly not a
specialist, not a PhD. Um I think I was
I was only a resident, you know,
throughout uh his tenure at OpenAI. So I
was very lucky in that regard, just kind
of learning the way that he thinks high
level about research. Um,
>> and a like a resident in this case is
you're just the right-hand person to
>> Oh, so it's um someone who comes in
usually from another field who openly I
wanted to invest in and train up in AI.
Yeah. And so I think the first part of a
residency is like a six-month compressed
PhD um and and then going from there to
to just you know getting into deeper and
deeper research projects.
>> So you're kind of like talking to IA
every day. is he kind of shaping that that
that
>> yeah he was responsible for for my
projects for my curriculum for my
learning and um and you know I would
just go to him for you know hey like
what's this about like why did people
pursue this
>> okay yeah
>> and you I mean I think if you go like on
LinkedIn I mean it would say you were
the head of Frontier Research as your
first job at open AI
>> oh no I I was in IC for threeish years
yeah so um I was doing independent
research projects Um, I worked in
generative modeling because that was
really kind of where I's focus was at
the time. And then,
>> um, only after a while did I start
managing teams.
>> And because most you're talking about
generative. I mean, most people point to
Dolly as maybe the kind of first big
project that the public would
>> mostly re is that is that fair?
>> Yeah. Yeah. So, um, I think that also
marked the transition between when I was
an IC and and a manager. So uh one of my
own kind of big projects and one I'm
still pretty proud of today is uh image
GBT uh this proof of concept that even
outside of language you could put things
like images into a transformer um and
the model would just internalize very
good representations and understand um
you know the content of images and um
it's kind of like a proof of concept
that you can do language modeling
outside of pure text and get really
really great representation. and scale
them to be state-of-the-art with other
methods. Um, that was I consider like a
precursor work to Dolly which I was on
the opposite side of managing. Um and um
I think between those uh another project
I'm really proud of uh
uh doing doing IC work on is uh codeex
where we
>> you know set up a lot of the framework
for evaluating um coding models and also
>> did a lot of in-depth study on how you
can take language models and make them
very good on code.
>> So and what made you pick open because I
I could see it two ways in my head. One is
is
>> is big fish in a small pond. There's
interesting people here. I remember the
open AI of 2018 with 20 people. In my
head, it was like, this is probably not
going to work. Um, you know, [snorts]
Google seems like they've got this
locked up and and this is like a pretty
small group of people trying to take on
something that appears to require many
billions of dollars of of capital. And I
mean, this was even before the scaling
stuff. It was just like Google had
invested so much [laughter] in in AI
already. Um kind of like in a different
form than we think, but you know, you're
already kind of doing translation on
your phone and things like that. So um
was that like a hard decision for you or
or you just stumbled into the OpenAI gig
so quickly that
>> Well, yeah, I mean I think there are two
things, right? You need ambition of
vision. That was certainly what OpenAI
had at the time, but you also need the
talent to back it up. And you know, I
did feel like OpenAI was one of the rare
places where the ambition was very
large, but the talent was also large
enough to fulfill that that gap. And you
know, I I was lucky I knew people like
Greg um before from from college. Yeah. And
And
>> Oh, so yeah, Greg was you over
overlapped at MIT.
>> Um I think we did some math contests
together. Okay. Yeah. Back in high
school. [laughter]
And um yeah, I I shot him a message
actually and I was like, "Oh, you know,
I I don't know if I have the right skill
set, but this sounds like a place that's
doing great work." So,
>> it still seems nuts till I just come at
this, you know, out of nowhere and and
now you're like leading [laughter] research.
research.
>> No, it's surreal to me, too. It's
surreal to me, too. Um, you know, even
that transition from IC to manager, I
was very hesitant about taking it. Um, I
didn't know if managing was a skill set
that I would be good at and I was really
enjoying IC work. I think uh um I was
having a lot of fun doing it, excelling
at it, building really great
collaborations. Um, but yeah, I mean
it's [snorts] really been a wild ride.
>> Yeah. Yeah. Well, okay, on that point, I mean,
mean,
>> you've always struck me as a very nice
like level-headed guy. I have to say,
you know, there's parts of OpenAI's
history that are are quite
>> dramatic, soap opera, like um a little
Game of Thronesy, you know, power
struggles and and like to me >> um
>> um
>> to be a manager in that I will say now
like I feel like things are a bit calmer
than they they were, but but when you
look backwards, it just seems like um I
don't know, you you're saying you had to
learn these skills feels. But
>> some of this feels like opposite. I
don't know you that well, but some of it
feels opposite to your personality. Um
to have to like deal with all of that.
>> Honestly, you know, I've been lucky at
opening eye. Um I genuinely say that and
um in the sense that I've had managers
that have really advocated for me. Um
you know, they they saw my talent and
and advocated for me. I think when I was
in IC, you know, W check, he was like,
"Oh man, you should bet on him for for
codeex." And um and then later on kind
of reporting to Bob, I I've never asked
for promotion or up level. And you know,
it's just organically happened. Um and
everyone kind of along the way has given
me great advice. Um I think part of
growing as in Manford is just getting
the reps. I don't think there's any
better place to get the reps than at at
OpenAI. You know, there's always
challenges to solve. Um, and yeah, I I
think kind of developing that
confidence. I actually think management
is something where it's really just
about the experience and um, you know,
there's I would say less so talent
involved in it.
>> Yeah. I don't want to like embarrass you
and I don't know if this will or won't
and I assume you probably don't want to
get too much into the the coup or the
blip or whatever. We want to talk about
anything. [laughter] Yeah. Yeah. Well, I
just I've I've interviewed so many
people about this now. And I'm also
gonna save my some of my gems for my
book. So,
>> I won't uh I won't up myself. But there
there's a couple moments in there where
you [clears throat]
>> um you know you help
>> get the researchers aligned around that
the petition to like bring Sam back and
then and then I think just that either a
day or two after that there's there's
kind of like this speech you know that
given I think at Greg's house maybe or
>> I think Chelsea's house
>> okay and and um you know both of those
struck me as pretty profound moments in
especially for um I guess like standing
up with for what you believe in and
rallying the troops. I mean, yeah, like uh
uh
you know, in a in a moment of crisis,
I don't know. So, did do those um Yeah.
I mean, that did feel like a very
pivotal moment for me. I think in the
days following the blip, right? There
was a lot of uncertainty. Um and you
know, myself, Nick, uh Barrett at the
time, we felt this responsibility of,
you know, the wolves are at the heels,
right? everyone's getting calls from
from all these competing labs being like
you should come work here instead and I
just set this goal of I will not lose a
single person um and we didn't um and it
was just you every day um opening up our
houses you know people could come here
they could you know have a place where
they let out their anxiety um and then
also just helping them keep in touch
with the leadership team um having a way
for them to feel like they could make a
difference and I think you know over
time people really felt this spirit of,
hey, we're all in this together. Um, how
do we make a difference? How do we
signal to the world that we're all
together? And, um, you know, we I've
been kind of driving back and forth uh
between a couple houses and, uh, we had
this idea of like, hey, you know, we
need to show the world that we're all
seriously aligned and we're going to
work for Sam. And, um, that's when the
petition came together. And I, you know,
the idea, I think, got solidified at
2:00 a.m. We got more than 90% of the
whole research or signed, I think, by
the morning. Um, and it was just
everyone like calling their friends
being like, "Hey, are you in or are you
are you not?" And yeah, I think in the
end, you know, it was very close to 100
people signing that petition. is well I
mean that must have put you in something
of a tough spot though just because
especially at the outset it was kind of
like Ilia and Sam were on opposite sides
and Ilia is your mentor and then I know
Ilia kind of
>> comes back um
>> um yeah I don't know was that awkward
>> um no no it was hard I mean it's it's a
low information environment um but
fundamentally it's just
uh and and yeah I mean I think at the
moment you know you could very
reasonably conclude like was like did
Sam do anything here, you know, is there
um but would Greg and Yakob like people
of super high integrity quit over that?
Um I just felt like, you know, there was
some part of the story that that was
being misrepresented here. >> Yeah.
>> Yeah. >> Yeah.
>> Yeah.
>> With the,
you know, Yakob's been there for a very
long time. Um like what should people
know about Yakob that they don't?
>> It's interesting cuz he's super funny guy.
guy.
>> He's hilarious. Oh my gosh. He has this
like sarcastic humor. Um, and yeah, it
cracks me up so much honestly. Like, um,
yeah, that's one of my favorite things
about OpenAI today. Just like the level
of alignment I have with Yakob. Um, I
feel like we go into a meeting, um, we
can just bounce off ideas and quickly
get to alignment and then, you know,
deliver the same message and kind of
like operate on different parts of a big
road map together. Um, and yeah, it's
just one of the big privileges I have
working at OpenAI. Yeah, I mean go going
to that actually that point about um you
know keeping people together like I
still feel that way about open eye
research. Um I think we're
>> still under attack. [laughter]
>> Yeah. No, I we are a family. Yeah. We're
always under attack. Look, when any and
this is how I know we're in the lead,
right? Any company starts, where do they
try to recruit from? It's OpenAI. Um and
you know, they want the expertise. They
want our our vision kind of our our
philosophy of the world. and we've made
so many star researchers, right? Um I
think opening I more than anywhere else
has been a place that makes names in in
AI today. Um and I still feel that same
level of protectiveness like you come
after I'm going to do anything in my
power to make sure you know they're
happy they're open and they understand
you know how their role fits into the
road map. I yeah I this is something I
battled with as I was doing the book or
even just watching events unfold in real
time is like when I go back through the
history I mean you've got
>> you've got Ilia in 2012 making sort of
like a big breakthrough and then you
know you've got no McGoo in 2017 doing
Transformers and then you've got Alec
Radford you know like sometimes the
story is these individuals
>> really pushing the field forward and it
feels like a field that's still so young
that you can have have this individual
and then and it seems like there's this
group of I don't know what the number
is, but let's call it like 8 to 10 who
seem to have
>> an ability to do that repeatedly and
they're really shaping where this is all
gone. And so when I started seeing like
John Schulman leave or Alec leave and
then you know there's kind of was like
wow okay well if you've lost a chunk of
this all-star team how do you it seems
like a kind of field where you you sort
of can't just replace that and yet you
know it was it was kind of like after
that that you guys
>> pushed forward on reasoning and and some
of these other spots. Yeah. So I don't
know I I've intellectually had trouble
>> I I do disagree with that as the the
overarching way to do good research
today. Um I think there's certainly a
lot of top down steer you know we bet on
directions but people it it open has
this beautiful culture of being bottom
up in a very deep way too where some of
the best ideas just organically emerge
from sometimes the most surprising of
places and um I think really the the
great thing has been just like watching
some of these bets unfold take shape get
scaled um and reasoning being a a core
example of that. Yeah. And okay, so in
this but like this idea that um like how
star dependent are we because you still
see Google spend an ungodly amount of
money to bring Gnome back, you know what
I mean? Yeah. And so this makes me
think, okay, this is how this works.
>> Yeah. I mean, I think it's a mix, right?
Like you have to you have to invest in
your pipeline because I'm very confident
in our ability to create stars. But
yeah, there's certainly very good people
out there and everyone knows that
they're good. Um I think if there's one
thing that you know on on the flip side
I've learned from from meta is you know
open can also go very aggressively after
star talent and um you know there's this
very aggressive recruiting approach that
you know I've taken a couple pages from
as well. Um [laughter] but yeah I I I
think it's we should always just be
trying to assemble the best team. >> Yeah.
>> Yeah.
>> Uh in service of the mission that we
want to accomplish. It's funny because
it is like a relatively small world and
like all you guys hang out even though
you you're like rivals and then
>> it must be weird cuz I know you're
friends with different people on some
level and then and then you're also
trying to steal all their
>> I mean yeah it's it's a brutally
competitive industry in all fronts,
right? Um but again that's what I love.
Um I'm a deeply competitive person. I
hate to lose and yeah on research on
recruiting all of these fronts um I'll
work very hard on them.
>> It reminds me because I'm like a
semiconductor well I'm a history nerd
but just
>> the early semiconductor days were not
that far off. I mean you had you had all
these semiconductor startups come at
once. They were all pushing the limits
of physics and somebody would discover
something at one.
>> They'd go to the bar and have a it's
like people they're engineers. they
can't like stop from like sort of
>> sharing knowledge with each other but
and then they're also getting
>> pulled like you know is hard each
company is is kind of quickly getting
this breakthrough in one way or another.
>> Yeah. I mean you raised an interesting
point of you know there is going to be
some base rate diffusion of ideas and um
I think there's two ways a company can
respond to that. You can create these
deep silos of like hey you know we're
going to protect information in all
these ways. I don't think OpenAI
operates that way and we don't think
that's the right way to operate. Um, we
just will outrun other people as fast as
we can and um I love the culture of
openness. People in research freely
share ideas and I think that's the way
to make the fastest progress.
>> And how like how do you Sam and Yakob
now work together? I I think people
sometimes um if you read the the
announcements and everything, you can
tell that Sam is researchoriented over
over like
>> day-to-day running of the company, you
know what I mean? You can tell research
is more of his his passion and and even
just like in the the titles and in the
way it's been organized, especially
recently. M
>> um you and Yakob are so deep on this
stuff and I know Sam is is technical but
>> you guys are are like in it all the time
and then you know Sam
>> is having conversations with everyone.
Yeah. I'm just curious about this
dynamic between the three of you and how
I mean are you guys I mean I guess
you're not always alignment on in
alignment on what is going to get the
resources but um yeah I was I was just
curious about you guys dynamic.
>> Yeah. Yeah. So, I mean, um, it's a very
tight cohort. You know, I talk to Sam
and Yakob every day. Um, and you know,
with Sam, he loves research. He loves
just learning about research. Um, he
loves talking to researchers. I think in
some ways he's very effective at getting
a pulse on the research or I rely on him
also to just, you know, are there any
hidden latent problems here? Um, go and
find them out, you know, surface them to
me. Jakob and I
>> personality or techno. It could be just
small things like oh you know um like
just even the way the office is laid out
like makes it harder for this team and
this team to collaborate um and the two
of them is like need to collaborate to
to help unlock this breakthrough that we
want. Um I mean all these things are you
know very very important and I think
Jakob and I we spend a lot of time
figuring out how to design the work for
success. You know I think um pairing
people with the right strengths
together. Um you know also how to like
incentivize people to work on directions
that we find are important. Um yeah that
that's a lot of the work that we do.
>> The and Sam what he um
like is he reading papers? Is he
chatting with you guys? Is
>> Yeah. Yeah. I mean, I think he he does
his fair share of reading papers. He
talks to researchers and just
understands how how they think about the
world, the type of research that they're
doing. Um, and of course, he's
responsible for a huge umbrella of
things outside of that.
>> All right, I'm going to ask some nerdy
questions now, but I'm going to try to
um I don't know if I can dwarf cash
level, but [laughter] I'm going to do my
best. And you know, I'll ask. I don't
know how top secret some of this stuff
is, but but um anyway, well, maybe
you'll slip up and we'll just we'll get
it out. Um you know, in the the meetings
I have been on, and I don't think I'm
revealing because we talked about a bit.
I think I'm safe here, but you know, pre
pre-training seems like this area where
it feel it seems that >> um
>> um
my sense is you you guys feel like
you've figured something out. you're
excited about. You think this is really
going to be like a major advance. It was
also, I think,
>> either a neglected spot or something of
a sore spot. You know, previously things
weren't maybe working [clears throat] exactly
exactly
>> how you guys had expected or hoped. Um,
like what can you tell us about what you figured out and and you know
what you figured out and and you know some sort of frame of reference on on
some sort of frame of reference on on we've seen these periodic big leaps
we've seen these periodic big leaps forward.
forward. >> Absolutely. So I think the way I would
>> Absolutely. So I think the way I would describe at a high level um the last two
describe at a high level um the last two years is you know we've put so much
years is you know we've put so much resourcing into into reasoning into
resourcing into into reasoning into understanding this primitive and making
understanding this primitive and making it work and it really has worked and I
it work and it really has worked and I do think one byproduct of that is you
do think one byproduct of that is you lose a little bit of muscle on uh your
lose a little bit of muscle on uh your other functions like pre-training and
other functions like pre-training and post- training. Um in the last six
post- training. Um in the last six months Jakob and I have done a lot of
months Jakob and I have done a lot of work to build that muscle back up. Um I
work to build that muscle back up. Um I think pre-training is really a muscle
think pre-training is really a muscle that you you exercise. You need to make
that you you exercise. You need to make sure you know all the info is fresh. You
sure you know all the info is fresh. You need to make sure uh people are working
need to make sure uh people are working on optimization at the frontier are
on optimization at the frontier are working on numericics at the frontier.
working on numericics at the frontier. And um I think you also have to make
And um I think you also have to make sure the mind share is there. Um that's
sure the mind share is there. Um that's kind of one one of the recent things
kind of one one of the recent things I've been focusing a lot on just kind of
I've been focusing a lot on just kind of directing and shaping what people talk
directing and shaping what people talk about at the company and very much today
about at the company and very much today that that is pre-training. Um, we think
that that is pre-training. Um, we think there's a lot of room in pre-training.
there's a lot of room in pre-training. You know, a lot of people say scaling is
You know, a lot of people say scaling is dead. Um, we don't think so at all. Um,
dead. Um, we don't think so at all. Um, in some sense, um, you know, all the
in some sense, um, you know, all the focus on RL, um, I I think, um, it's a
focus on RL, um, I I think, um, it's a little bit of an alpha for us because we
little bit of an alpha for us because we think there's so much room left in in in
think there's so much room left in in in pre-training and and I think as a result
pre-training and and I think as a result of these efforts, you know, we've been
of these efforts, you know, we've been training much stronger models and that
training much stronger models and that also gives us a lot of confidence
also gives us a lot of confidence carrying into, you know, Gemini 3 and
carrying into, you know, Gemini 3 and other releases coming this end of the
other releases coming this end of the year.
year. Like the way I picture it in my head
Like the way I picture it in my head sometimes is that
sometimes is that you guys have been on this you've just
you guys have been on this you've just been running so fast. The whole field
been running so fast. The whole field has been running so fast. And so we're
has been running so fast. And so we're at a moment where it's like, okay, we've
at a moment where it's like, okay, we've >> gathered up this vast volume of
>> gathered up this vast volume of information from the internet. We've
information from the internet. We've we've thrown it onto this supercomput
we've thrown it onto this supercomput and and that, you know, chat GPD pops
and and that, you know, chat GPD pops out and then we're just on this this
out and then we're just on this this like this incredible race that's going
like this incredible race that's going on. And so like when I hear you guys I'm
on. And so like when I hear you guys I'm just trying to think about this in like
just trying to think about this in like a
a to level set maybe for people who don't
to level set maybe for people who don't follow this as closely. Um
follow this as closely. Um so you you know in that initial moment
so you you know in that initial moment you just had so much data you're
you just had so much data you're throwing it at this machine you try to
throwing it at this machine you try to shape that data a bit initially and what
shape that data a bit initially and what now we're just learning like more
now we're just learning like more efficient ways to shape that just it's
efficient ways to shape that just it's not
not always clear on what the mistakes were.
always clear on what the mistakes were. Um, so I do think um, yeah, you touch on
Um, so I do think um, yeah, you touch on something I've been thinking about a
something I've been thinking about a lot, right? Um, when you think about
lot, right? Um, when you think about pre-training, right, you're taking human
pre-training, right, you're taking human written data and you're teaching the
written data and you're teaching the model how to essentially emulate it,
model how to essentially emulate it, right? Um, it understands human patterns
right? Um, it understands human patterns of writing. Um, and in some sense that
of writing. Um, and in some sense that also bottlenecks
also bottlenecks um, and and puts a ceiling on the
um, and and puts a ceiling on the capability that you're able to achieve,
capability that you're able to achieve, right? You you can't really surpass what
right? You you can't really surpass what humans have written when you're
humans have written when you're imitating what humans have written. And
imitating what humans have written. And so, you know, you you work on things
so, you know, you you work on things like RL um there, you know, you can
like RL um there, you know, you can really provide steer towards the hardest
really provide steer towards the hardest tasks that humans uh can come up with
tasks that humans uh can come up with and um have the model basically think
and um have the model basically think outside of the box, outside of um what
outside of the box, outside of um what it's learned from from imitating humans
it's learned from from imitating humans and achieve higher levels of capability.
and achieve higher levels of capability. But there is this kind of interesting
But there is this kind of interesting problem now of how do you go beyond um
problem now of how do you go beyond um what humans are able to do today? And I
what humans are able to do today? And I do find a serious measurement problem
do find a serious measurement problem there too. Um even in the sense of like
there too. Um even in the sense of like can humans judge superhuman performance
can humans judge superhuman performance in in the sciences, right? Um would how
in in the sciences, right? Um would how how would we know that like this
how would we know that like this superhuman mathematician is better than
superhuman mathematician is better than that superhuman mathematician? And uh we
that superhuman mathematician? And uh we really do need to kind of come up with
really do need to kind of come up with better evaluations for um what it means
better evaluations for um what it means to make progress in this world. Right?
to make progress in this world. Right? We've been lucky up to this point,
We've been lucky up to this point, right? There have been contests like the
right? There have been contests like the IMO IOI really just like gauging who's
IMO IOI really just like gauging who's the top one mathematician in the world
the top one mathematician in the world right um but when the model capabilities
right um but when the model capabilities go beyond humans there are no more tests
go beyond humans there are no more tests >> right okay you just made me think of a
>> right okay you just made me think of a question going back to the IOI stuff I
question going back to the IOI stuff I mean
mean >> and sorry we're going to come back I
>> and sorry we're going to come back I just you just totally popped in my head
just you just totally popped in my head I mean like often I would see the
I mean like often I would see the >> kids who were amazing at those
>> kids who were amazing at those competitions they would get hired
competitions they would get hired somewhere like a Google or Facebook or
somewhere like a Google or Facebook or something, but they weren't always like
something, but they weren't always like the,
the, you know, the top executive or the top
you know, the top executive or the top the the most famous engineer afterwards.
the the most famous engineer afterwards. And maybe it was like by choice, but I
And maybe it was like by choice, but I don't think Gennady was like the Michael
don't think Gennady was like the Michael Jordan ended up working at any of these
Jordan ended up working at any of these companies. And that totally could be by
companies. And that totally could be by choice. I'm not trying to disparrage
choice. I'm not trying to disparrage him, but but it's not clear to me um
him, but but it's not clear to me um >> like even Okay, so it's not clear to me
>> like even Okay, so it's not clear to me that the human who excels at that is is
that the human who excels at that is is necessarily like the greatest engineer
necessarily like the greatest engineer you're ever going to have. And so like
you're ever going to have. And so like yeah, may I mean if an AI is
yeah, may I mean if an AI is particularly good like what are we
particularly good like what are we learning?
learning? >> Yeah, that's a thing I quite like about
>> Yeah, that's a thing I quite like about working in AI. I think more so than uh
working in AI. I think more so than uh in in standard engineering culture, it
in in standard engineering culture, it is a meritocracy. Um in that, you know,
is a meritocracy. Um in that, you know, I've tried this many times before and
I've tried this many times before and learned this lesson many times before,
learned this lesson many times before, but it is hard to put in someone to lead
but it is hard to put in someone to lead a group who doesn't have the respect of
a group who doesn't have the respect of the researchers that they're leading. Um
the researchers that they're leading. Um and I think this is more so the case in
and I think this is more so the case in research than than anywhere else. You
research than than anywhere else. You have to make very strong technical calls
have to make very strong technical calls of like, you know, this is the right
of like, you know, this is the right right path when there's a disagreement.
right path when there's a disagreement. this is the right kind of project. Um,
this is the right kind of project. Um, and if you make those calls wrong, you
and if you make those calls wrong, you know, you lose the respect of of your
know, you lose the respect of of your researchers. So, um, yeah, one of the
researchers. So, um, yeah, one of the fun things in working with, uh, in AI
fun things in working with, uh, in AI and and creating a a strong AI or is,
and and creating a a strong AI or is, you know, all my bentures very deeply
you know, all my bentures very deeply technical and it's fun to talk to them
technical and it's fun to talk to them about the technical things.
about the technical things. >> Yeah. Yeah. Okay. And then Okay. On this
>> Yeah. Yeah. Okay. And then Okay. On this on I'm pre-training again for a second.
on I'm pre-training again for a second. Okay. You know, like to me in my head it
Okay. You know, like to me in my head it feels like
feels like Transformers
Transformers helped kick off this massive, massive
helped kick off this massive, massive leap. I mean, reasoning to me feels
leap. I mean, reasoning to me feels very comparable, if not even
very comparable, if not even >> sort of
>> sort of more amazing. I mean, are we
when I talked to you guys over the last few months, my, you know, and I can
few months, my, you know, and I can never tell if this is optimism,
never tell if this is optimism, >> if you guys are just putting the best
>> if you guys are just putting the best foot forward when I'm chatting to
foot forward when I'm chatting to everyone. My my sense when I talk to
everyone. My my sense when I talk to you, to Greg, to Yakob, you know, to Sam
you, to Greg, to Yakob, you know, to Sam is is that you guys kind of feel like
is is that you guys kind of feel like you've been putting in hard engineering
you've been putting in hard engineering work for like three, four, five years
work for like three, four, five years that hasn't fully
that hasn't fully >> manifested itself. Um, and so then I can
>> manifested itself. Um, and so then I can never tell how excited or not to be. I
never tell how excited or not to be. I mean like when you guys are hinting at
mean like when you guys are hinting at some of the stuff you're seeing do you
some of the stuff you're seeing do you feel like it is you can already tell
feel like it is you can already tell that it is a comparable leap forward
that it is a comparable leap forward >> in terms
>> in terms >> to these big epocle kind of things?
>> to these big epocle kind of things? >> I think so. You know um I I think when
>> I think so. You know um I I think when we launched GPD5 you know we
we launched GPD5 you know we >> we talked a lot about synthetic data as
>> we talked a lot about synthetic data as well. Um you know there are many other
well. Um you know there are many other threads of this form that we think are
threads of this form that we think are holding quite a bit of promise and that
holding quite a bit of promise and that we're scaling up pretty aggressively
we're scaling up pretty aggressively right now. And I think it's always about
right now. And I think it's always about maintaining that portfolio of bets. Uh
maintaining that portfolio of bets. Uh taking the ones that are providing more
taking the ones that are providing more empirical promise and and scaling and
empirical promise and and scaling and supporting them at at an even greater
supporting them at at an even greater degree.
degree. >> But it was like two weeks ago Andre
>> But it was like two weeks ago Andre Karpathy who used to work at open, you
Karpathy who used to work at open, you know, he went on Dark Casual's podcast
know, he went on Dark Casual's podcast and seemed to like deflate
and seemed to like deflate >> some giant portion of the AI industry by
>> some giant portion of the AI industry by saying, you know, I think he was saying
saying, you know, I think he was saying what that AGI was like 10 years 10 years
what that AGI was like 10 years 10 years off. And then when I hear and then I I
off. And then when I hear and then I I heard Daario talking about a week ago. I
heard Daario talking about a week ago. I mean he seemed to be holding on very
mean he seemed to be holding on very much to like massive scientific dis his
much to like massive scientific dis his his um what is he called the the nation
his um what is he called the the nation of geniuses. He seemed to be holding
of geniuses. He seemed to be holding still on kind of like maybe a little
still on kind of like maybe a little slower but like a two-year timeline on
slower but like a two-year timeline on that. Um you know
that. Um you know >> yeah when you heard what Andre said what
>> yeah when you heard what Andre said what did you
did you >> Yeah. I mean I think Twitter they love
>> Yeah. I mean I think Twitter they love this like cycle of you know it's so
this like cycle of you know it's so over. so back and you know whatever
over. so back and you know whatever plays into the narrative at the time I
plays into the narrative at the time I think you know just becomes amplified.
think you know just becomes amplified. Yeah, I'm trying to make a clip here,
Yeah, I'm trying to make a clip here, but you know, I the way I think about
but you know, I the way I think about it, yeah, I mean, it's like AGI, I mean,
it, yeah, I mean, it's like AGI, I mean, everyone defines their own point for
everyone defines their own point for AGI, I I think even at at OpenAI, um,
AGI, I I think even at at OpenAI, um, you can't get everyone in the same room
you can't get everyone in the same room and be like, hey, this is my clear
and be like, hey, this is my clear definition of AGI and it's it's it's
definition of AGI and it's it's it's consistent. And so, I kind of think
consistent. And so, I kind of think about it as something like, you know,
about it as something like, you know, you're in the industrial revolution,
you're in the industrial revolution, right? Do you consider
right? Do you consider the, you know, having machines make
the, you know, having machines make textiles, is that the industrial
textiles, is that the industrial revolution or is it the steam engine?
revolution or is it the steam engine? you know, everyone kind of has their
you know, everyone kind of has their different definition and um I think
different definition and um I think we're in the middle of this process of
we're in the middle of this process of producing AGI. For me, I think the thing
producing AGI. For me, I think the thing I index most on is are we producing
I index most on is are we producing novel scientific knowledge and are we
novel scientific knowledge and are we advancing the scientific frontier and I
advancing the scientific frontier and I feel since the summer there's been a
feel since the summer there's been a tremendous phase shift on that front.
tremendous phase shift on that front. >> Okay. like from stuff that you're seeing
>> Okay. like from stuff that you're seeing in ter the the first things that are
in ter the the first things that are jumping to my head are all these
jumping to my head are all these >> startups that are in the biotech space
>> startups that are in the biotech space that are showing you know oneshot
that are showing you know oneshot antibodies and and molecules but I have
antibodies and and molecules but I have no idea if that like what are you
no idea if that like what are you >> yeah yeah so I mean I was so inspired by
>> yeah yeah so I mean I was so inspired by that encounter with the physicists that
that encounter with the physicists that you know went back and thought hey well
you know went back and thought hey well we should just create open AI for
we should just create open AI for science and the goal being I think for
science and the goal being I think for the small set of scient scientists today
the small set of scient scientists today who realize the potential of these
who realize the potential of these models and feel like they want to lean
models and feel like they want to lean in and accelerate, we should do the best
in and accelerate, we should do the best that we can to accelerate them. And you
that we can to accelerate them. And you know I know there are similar efforts
know I know there are similar efforts that you know other companies um aim
that you know other companies um aim towards pushing the scientific frontier
towards pushing the scientific frontier but I think what we want to do and um I
but I think what we want to do and um I would say a little bit of a framing in
would say a little bit of a framing in terms of how we differ from let's say
terms of how we differ from let's say Google's uh efforts to to to work on
Google's uh efforts to to to work on science is we want to allow everyone
science is we want to allow everyone uh the ability to you know win the Nobel
uh the ability to you know win the Nobel Prize for themsel. Um, it's less so
Prize for themsel. Um, it's less so about us winning that at at OpenAI,
about us winning that at at OpenAI, which would be nice, but we want to
which would be nice, but we want to build the tooling and the framework so
build the tooling and the framework so that all scientists out there feel that
that all scientists out there feel that accelerative impact and we think we can
accelerative impact and we think we can push the field collectively.
push the field collectively. >> Well, and when the discoveries that
>> Well, and when the discoveries that you're saying you're excited about? I
you're saying you're excited about? I mean, are there any others like
mean, are there any others like specifically that that you've
specifically that that you've >> Yeah. Yeah. So, um, I think there's, you
>> Yeah. Yeah. So, um, I think there's, you know, if you want a a huge list of of
know, if you want a a huge list of of these, um, you can go on Seb's Twitter
these, um, you can go on Seb's Twitter account. Um so recently you know there's
account. Um so recently you know there's a GPD5 paper on an open convex
a GPD5 paper on an open convex optimization problem that you know uh is
optimization problem that you know uh is actually whose Twitter account?
actually whose Twitter account? >> Uh Sebastian. Okay. Yeah. Yeah. And you
>> Uh Sebastian. Okay. Yeah. Yeah. And you know it's like um very related to some
know it's like um very related to some of the core ML problems that we're
of the core ML problems that we're solving. Um I know there was um
solving. Um I know there was um >> I think people kind of dismiss these
>> I think people kind of dismiss these things as oh is it just fancy literature
things as oh is it just fancy literature search or something like that? Um it's
search or something like that? Um it's quite a bit more complicated than that.
quite a bit more complicated than that. And you know, there's some examples I
And you know, there's some examples I could go into, but
could go into, but >> I might I'm honestly overwhelmed at the
>> I might I'm honestly overwhelmed at the moment cuz, you know, I'm sort of a
moment cuz, you know, I'm sort of a generalist, but I cover biotech a lot.
generalist, but I cover biotech a lot. And it's like
And it's like >> every two days, man, I'm walking in and
>> every two days, man, I'm walking in and it's wow, we we're making an AI
it's wow, we we're making an AI scientist. We we one-shotted enhanced
scientist. We we one-shotted enhanced body and and then so like part of me
body and and then so like part of me gets excited and you know at least a
gets excited and you know at least a handful of these companies I know the
handful of these companies I know the people and they're real scientists and
people and they're real scientists and like but then there's so much of it that
like but then there's so much of it that I'm like either
I'm like either >> something amazing is happening or every
>> something amazing is happening or every it's it's like it's it's kind of too
it's it's like it's it's kind of too much for me to be able to discern where
much for me to be able to discern where reality is.
reality is. >> Yeah. I mean I wouldn't be surprised if
>> Yeah. I mean I wouldn't be surprised if it's happening in biology. Um personally
it's happening in biology. Um personally I have the most expertise in you know
I have the most expertise in you know computer science and mathematics and you
computer science and mathematics and you know we we do have the experts there
know we we do have the experts there that can confirm that these are
that can confirm that these are discoveries being made. So that's the
discoveries being made. So that's the thing that gives me the most confidence
thing that gives me the most confidence but I'm not surprised at all it's
but I'm not surprised at all it's happening in biology.
happening in biology. >> But like what you're saying is is kind
>> But like what you're saying is is kind of different than the I gr the narrative
of different than the I gr the narrative changes every like three weeks it seems
changes every like three weeks it seems like. But like what you're saying is is
like. But like what you're saying is is sort of different because the biggest
sort of different because the biggest knock even before Andre said that it
knock even before Andre said that it seemed to me from the you know what I
seemed to me from the you know what I was listening to um I was listening to
was listening to um I was listening to like a politics podcast um sagger I
like a politics podcast um sagger I think it's breaking points is their
think it's breaking points is their podcast you know he's he's
podcast you know he's he's >> pretty smart
>> pretty smart >> guy who's knowledgeable but I mean he's
>> guy who's knowledgeable but I mean he's just been on AI and the lack of
just been on AI and the lack of progress and this is all like make
progress and this is all like make believe and and all in and
believe and and all in and So, you know, if these discoveries
So, you know, if these discoveries aren't happening, I mean, I feel like
aren't happening, I mean, I feel like the public is aware of this.
the public is aware of this. >> Just to be clear, you know, um, while
>> Just to be clear, you know, um, while setting up Open AI for science, we've
setting up Open AI for science, we've talked to a lot of physicists, a lot of
talked to a lot of physicists, a lot of mathematicians, and actually most
mathematicians, and actually most of the people we've talked to aren't
of the people we've talked to aren't that bullish on AI. I think they still
that bullish on AI. I think they still believe, hey, you know, um, this thing
believe, hey, you know, um, this thing isn't something that can solve new
isn't something that can solve new theorems. There's no way it could do
theorems. There's no way it could do that. You know, there must be something
that. You know, there must be something else going on. And that's why I feel
else going on. And that's why I feel like empowering the set of people who
like empowering the set of people who really do believe and lean into it like
really do believe and lean into it like those people are going to just
those people are going to just >> you know outrun everyone else and we
>> you know outrun everyone else and we want to build the tools and convince
want to build the tools and convince people like this is the right way to do
people like this is the right way to do scientific research.
scientific research. >> Okay. And so I mean so like on that
>> Okay. And so I mean so like on that point I mean I grant you that
point I mean I grant you that everybody's definition of AGI is
everybody's definition of AGI is different but you're
different but you're >> like at least what I'm hearing is I mean
>> like at least what I'm hearing is I mean [clears throat] you whatever you want to
[clears throat] you whatever you want to call it um you feel like in the next
call it um you feel like in the next year or two is we're just seeing
year or two is we're just seeing dramatic things happen.
dramatic things happen. >> Yeah. I mean it is a bit of a meme,
>> Yeah. I mean it is a bit of a meme, right? It's like you ask someone when is
right? It's like you ask someone when is AGI? It's two years away, right? Um
AGI? It's two years away, right? Um >> and I don't think we're in that world
>> and I don't think we're in that world anymore. And it it's like these results
anymore. And it it's like these results in math and science that that are giving
in math and science that that are giving me this conviction. But at at OpenI
me this conviction. But at at OpenI within the research, we set two very
within the research, we set two very concrete goals, right? Within a year, we
concrete goals, right? Within a year, we want to change the nature of the way
want to change the nature of the way that we're doing research. And we want
that we're doing research. And we want to be productively
to be productively um relying on AI interns in in the
um relying on AI interns in in the research development process. And within
research development process. And within 2 and 1/2 years, we want AI to be doing
2 and 1/2 years, we want AI to be doing end-to-end research. And I think it's
end-to-end research. And I think it's very different right like today you know
very different right like today you know you come up with an idea you execute on
you come up with an idea you execute on it you implement it you debug it um it
it you implement it you debug it um it means within a year we're quite
means within a year we're quite confident we can get to a world where we
confident we can get to a world where we control the outer loop we come up with
control the outer loop we come up with the ideas but the model is in charge of
the ideas but the model is in charge of the implementation the debugging yeah
the implementation the debugging yeah >> okay are there I beyond pre-training
>> okay are there I beyond pre-training when I talk to you guys um
when I talk to you guys um sometimes I get the sense similar sort
sometimes I get the sense similar sort the thing it's like we all have in our
the thing it's like we all have in our heads at least people where I sit that
heads at least people where I sit that there's been this massive infrastructure
there's been this massive infrastructure build out that um the models seem to get
build out that um the models seem to get better every time you 10x them that um
better every time you 10x them that um you know there was a there was a story
you know there was a there was a story >> for a while that as you guys were going
>> for a while that as you guys were going from like four to five you weren't
from like four to five you weren't seeing the results you wanted even
seeing the results you wanted even though
though >> um you were getting more compute but
>> um you were getting more compute but then the more I talked to you guys
then the more I talked to you guys >> the more it sounds to me like you feel
>> the more it sounds to me like you feel we haven't actually that things were
we haven't actually that things were moving so fast back then that we haven't
moving so fast back then that we haven't actually seen um the moment where we
actually seen um the moment where we made the leap to the 10x computer. I
made the leap to the 10x computer. I don't know if I asked that question very
don't know if I asked that question very eloquently but
eloquently but >> yeah I mean I I do have a thought to
>> yeah I mean I I do have a thought to share here which is you know when people
share here which is you know when people ask me um like do you guys really need
ask me um like do you guys really need all this compute um it's such a shocking
all this compute um it's such a shocking question because you know dayto-day I'm
question because you know dayto-day I'm dealing with so many compute requests
dealing with so many compute requests and you know the really my frame of mind
and you know the really my frame of mind is you know if we had 3x the compute
is you know if we had 3x the compute today I could immediately
today I could immediately utilize that very effectively if we had
utilize that very effectively if we had 10x the compute today um probably within
10x the compute today um probably within a small number of weeks fully utilize
a small number of weeks fully utilize that productively
that productively And so I think the demand for compute is
And so I think the demand for compute is really there. I don't I don't see any
really there. I don't I don't see any slowdown. And yeah, I it almost baffles
slowdown. And yeah, I it almost baffles me when I hear people ask like, "Oh, do
me when I hear people ask like, "Oh, do you guys really need more compute?"
you guys really need more compute?" Yeah. Doesn't doesn't make sense to me.
Yeah. Doesn't doesn't make sense to me. >> And you think we
>> And you think we in the broad strokes of the question I
in the broad strokes of the question I asked badly. [laughter]
asked badly. [laughter] Um, do like along the lines of where you
Um, do like along the lines of where you guys seem very optimistic about what
guys seem very optimistic about what you've cracked on pre-training, are you
you've cracked on pre-training, are you equally not just like this demand that
equally not just like this demand that people want more GPUs, but are you are
people want more GPUs, but are you are >> do you see pretty clearly that that same
>> do you see pretty clearly that that same thing scaling is about to kick things
thing scaling is about to kick things higher?
higher? >> Yeah, we we absolutely want to keep
>> Yeah, we we absolutely want to keep scaling the models and I think we have
scaling the models and I think we have algorithmic breakthroughs that enable us
algorithmic breakthroughs that enable us to scale the models and um, you know, I
to scale the models and um, you know, I think there's a lot impressive about
think there's a lot impressive about Gemini 3. Um, one thing that kind of
Gemini 3. Um, one thing that kind of reading into the details that I've
reading into the details that I've noticed is, you know, when you look at
noticed is, you know, when you look at stuff like their SWE bench numbers,
stuff like their SWE bench numbers, there's still a big thing around data
there's still a big thing around data efficiency that they haven't cracked,
efficiency that they haven't cracked, right? They haven't made that much
right? They haven't made that much movement on it. And I think we have very
movement on it. And I think we have very strong algorithms there.
strong algorithms there. >> Yeah. Well, and there was this leaked
>> Yeah. Well, and there was this leaked memo from I mean, Sam was sounding quite
memo from I mean, Sam was sounding quite somber about Gemini 3, man. In this
somber about Gemini 3, man. In this memo, I'm trying to find the quote. Did
memo, I'm trying to find the quote. Did you Well, you obvious I'm sure you got
you Well, you obvious I'm sure you got the memo. Um
it like it it it seemed like a bit of a moment. Um yeah.
moment. Um yeah. >> Well, I do think part of Sam's job is to
>> Well, I do think part of Sam's job is to inject urgency and pace and and that's
inject urgency and pace and and that's also part of my job as well. Um I think
also part of my job as well. Um I think it is important for us to be laser
it is important for us to be laser focused on on scaling and I do think you
focused on on scaling and I do think you know Gemini 3 is
know Gemini 3 is exactly like the right kind of bet that
exactly like the right kind of bet that that Google should be pursuing. Um you
that Google should be pursuing. Um you know at the same time you know I would
know at the same time you know I would calibrate that by saying you know a
calibrate that by saying you know a large part of our jobs is to inject as
large part of our jobs is to inject as much urgency into the org as possible.
much urgency into the org as possible. Yeah.
Yeah. >> And um it is a good model. Um, I think
>> And um it is a good model. Um, I think we have a response. Um, and I think we
we have a response. Um, and I think we can execute even faster to the the
can execute even faster to the the follow-up.
follow-up. >> How much do you
>> How much do you um get involved with things like and I'm
um get involved with things like and I'm sure you're going to tell me exactly
sure you're going to tell me exactly what it looks like with Johnny Ives
what it looks like with Johnny Ives device.
device. >> Cool. Cool. Yeah.
>> Cool. Cool. Yeah. >> Yeah. Like is that is that an area that
>> Yeah. Like is that is that an area that um research plays in?
um research plays in? >> Yeah. Yeah, it is. And actually um I was
>> Yeah. Yeah, it is. And actually um I was just having dinner yesterday.
just having dinner yesterday. >> You can describe it to me if you want.
>> You can describe it to me if you want. >> Yeah, [laughter] absolutely.
>> Yeah, [laughter] absolutely. So it looks like this. Well, um
So it looks like this. Well, um yesterday I was uh [laughter]
yesterday I was uh [laughter] >> yeah just having uh dinner with Johnny
>> yeah just having uh dinner with Johnny uh with with some researchers as well um
uh with with some researchers as well um our head of pre-training and also post-
our head of pre-training and also post- trainining and um really the way I think
trainining and um really the way I think about chat GPT in the future right um
about chat GPT in the future right um today when you look at how you interact
today when you look at how you interact with chat GPT um it feels very dumb to
with chat GPT um it feels very dumb to me it doesn't feel very thinking native
me it doesn't feel very thinking native right and you go to it with a prompt
right and you go to it with a prompt right you get a response and then it's
right you get a response and then it's doing no productive work for you until
doing no productive work for you until you give it the next prompt. And if you
you give it the next prompt. And if you give it a similar prompt, you know, it's
give it a similar prompt, you know, it's going to think for the same amount of
going to think for the same amount of time, it hasn't gotten smarter because
time, it hasn't gotten smarter because you added asked the first prompt. And
you added asked the first prompt. And you know, I think the the future is
you know, I think the the future is going to be a world where, you know,
going to be a world where, you know, memory is going to be a a much kind of
memory is going to be a a much kind of improved feature. Every time you go go
improved feature. Every time you go go to chapd, it learns something deep about
to chapd, it learns something deep about you. It reflects about why you would ask
you. It reflects about why you would ask this question, related questions, you
this question, related questions, you know, anything. Um, and then the next
know, anything. Um, and then the next time you go to it, it's going to be that
time you go to it, it's going to be that much smarter. And I think it really begs
much smarter. And I think it really begs the question of how do you design a
the question of how do you design a device
device that has this as the dominating thesis.
that has this as the dominating thesis. Um, and yeah, I I thought that's been a
Um, and yeah, I I thought that's been a very productive experience.
very productive experience. >> Do you have one?
>> Do you have one? >> Do I have one? Um,
>> Do I have one? Um, >> I may or may not have one. [laughter]
>> I may or may not have one. [laughter] Um, what I think about when I think
Um, what I think about when I think about you guys talking to Johnny is that
about you guys talking to Johnny is that like at Apple,
like at Apple, you had this company that was centered
you had this company that was centered around hardware. It's something that
around hardware. It's something that Steve Jobs obsessed about all the time.
Steve Jobs obsessed about all the time. >> It's like,
>> It's like, >> you know, it's a craft. It's like an art
>> you know, it's a craft. It's like an art form. Um
form. Um whether it's you, Sam, Greg, Yakob,
whether it's you, Sam, Greg, Yakob, whomever, as far as I'm aware, none of
whomever, as far as I'm aware, none of you guys have really done a hardware
you guys have really done a hardware product before.
product before. >> Um
>> Um >> Sam seems to take design very seriously.
>> Sam seems to take design very seriously. I could tell from his the buildings in
I could tell from his the buildings in his house and things like that. But, you
his house and things like that. But, you know, there's no sort of track record to
know, there's no sort of track record to speak of of like I always thought of
speak of of like I always thought of Steve Jobs as having like taste,
Steve Jobs as having like taste, >> you know, and then and I've I've had a
>> you know, and then and I've I've had a couple
couple >> bosses over the years like Josh Taring
>> bosses over the years like Josh Taring used to run business week. He kind of he
used to run business week. He kind of he just always struck me as this guy. He
just always struck me as this guy. He just had taste, you know, whether it was
just had taste, you know, whether it was the way something looked, the way a
the way something looked, the way a story should be. There was like this
story should be. There was like this innate thing that was on this really
innate thing that was on this really high level. It strikes me that's kind of
high level. It strikes me that's kind of like what's required here. I guess
like what's required here. I guess that's why you have someone like Johnny
that's why you have someone like Johnny on some level, but but you have to have
on some level, but but you have to have this like
this like >> back and forth. How do we know that like
>> back and forth. How do we know that like any of you guys have taste and and are,
any of you guys have taste and and are, you know, can shape a hardware product?
you know, can shape a hardware product? >> Honestly, we don't need to have taste
>> Honestly, we don't need to have taste ourselves and and that that is Johnny's
ourselves and and that that is Johnny's job. He he's our discriminator on on
job. He he's our discriminator on on taste. And I think actually one thing
taste. And I think actually one thing that's been really nice is just
that's been really nice is just realizing that the way they work in
realizing that the way they work in design and the way we work in research
design and the way we work in research is there's some deep parallels there,
is there's some deep parallels there, right? there's like so much exploration
right? there's like so much exploration and ideation and you explore a bunch of
and ideation and you explore a bunch of hypotheses. You take your time. Um, and
hypotheses. You take your time. Um, and then you create kind of the thing that
then you create kind of the thing that you're happy, the artifact at the end
you're happy, the artifact at the end that you're happy about and
that you're happy about and >> [clears throat]
>> [clears throat] >> um, yeah, it's it's been really nice to
>> um, yeah, it's it's been really nice to kind of have them fold into the company
kind of have them fold into the company and there's just a lot more direct
and there's just a lot more direct communication about here's the
communication about here's the capabilities that we're going to ship
capabilities that we're going to ship and here's what the form factor looks
and here's what the form factor looks like and how to gel them.
like and how to gel them. >> Okay. And I this is like a crass way to
>> Okay. And I this is like a crass way to put this but um because I spend my life
put this but um because I spend my life adoring and talking to these people, but
adoring and talking to these people, but you know, sometimes I'm just like, man,
you know, sometimes I'm just like, man, I just don't know if a bunch of math
I just don't know if a bunch of math nerds are the ones that you want making
nerds are the ones that you want making like uh the AI computer, you know, but I
like uh the AI computer, you know, but I guess it is this blend that you're
guess it is this blend that you're talking about.
talking about. >> Uh yeah, I mean, honestly, um there
>> Uh yeah, I mean, honestly, um there Yeah, you're you're right in that the
Yeah, you're you're right in that the people who are the best at building AI
people who are the best at building AI capabilities are slightly different from
capabilities are slightly different from the people who have the best taste. And
the people who have the best taste. And we do have teams built of people who
we do have teams built of people who have really great taste for model
have really great taste for model behavior. And I think there's like a a
behavior. And I think there's like a a very different kind of philosophy and a
very different kind of philosophy and a very different
very different kind of set of questions you you need to
kind of set of questions you you need to keep asking yourself. Um one one example
keep asking yourself. Um one one example of like a a good taste question, right?
of like a a good taste question, right? Like um you can imagine this being like
Like um you can imagine this being like in the model behavior interview is like
in the model behavior interview is like what should Chapi's favorite number be?
what should Chapi's favorite number be? >> Um
>> Um >> what should it what number be?
>> what should it what number be? >> Chachi's favorite number be
>> Chachi's favorite number be >> Oh, okay. Okay. Okay. I'm curious what
>> Oh, okay. Okay. Okay. I'm curious what you would ask. You would answer
you would ask. You would answer >> what I think its favorite number should
>> what I think its favorite number should be.
be. >> Well, I have a stupid answer which is
>> Well, I have a stupid answer which is that I went to Pomona College and 47 is
that I went to Pomona College and 47 is this like number of lore there. So,
this like number of lore there. So, >> okay. Okay. Okay. Yeah. I mean, that's a
>> okay. Okay. Okay. Yeah. I mean, that's a good answer. Yeah. [laughter]
good answer. Yeah. [laughter] >> Um the Okay. Um I'm going to let you go
>> Um the Okay. Um I'm going to let you go in a second. You've been really
in a second. You've been really generous. I appreciate it. Um, is there
generous. I appreciate it. Um, is there Well, I'm going to ask you a question
Well, I'm going to ask you a question that chatbt told me to ask you, which is
that chatbt told me to ask you, which is is uh,
is uh, >> you know, it says like if you look back
>> you know, it says like if you look back in five years, um,
in five years, um, are there any kind of like small,
are there any kind of like small, fragile, nent ideas that you're seeing
fragile, nent ideas that you're seeing right now that you you your instinct is
right now that you you your instinct is telling you might be at the heart of a
telling you might be at the heart of a big breakthrough? Yeah, there's a couple
big breakthrough? Yeah, there's a couple I would say a handful of ideas. Um I
I would say a handful of ideas. Um I can't go into too much detail on them,
can't go into too much detail on them, but yeah, I'm really really excited to
but yeah, I'm really really excited to scale them up. Yeah. [laughter]
scale them up. Yeah. [laughter] >> Are there any hints any uh buckets of
>> Are there any hints any uh buckets of areas where these fall?
areas where these fall? >> Yeah, I mean I've been concentrating a
>> Yeah, I mean I've been concentrating a lot on pre-training. So, uh you know,
lot on pre-training. So, uh you know, some pre-training adjacent um a small
some pre-training adjacent um a small number of ideas in RL as well and a
number of ideas in RL as well and a small number of ideas of how to put it
small number of ideas of how to put it all together.
all together. >> Yeah. Okay. All right. I tried. I tried.
>> Yeah. Okay. All right. I tried. I tried. So, and you may or may not have a
So, and you may or may not have a device. And no, no hints.
device. And no, no hints. >> Yeah. Uh, no. No hints. [laughter]
>> Yeah. Uh, no. No hints. [laughter] Um, okay. Well, we covered tons of
Um, okay. Well, we covered tons of ground. I really appreciate it. Is there
ground. I really appreciate it. Is there I feel like I'm letting the nerds down a
I feel like I'm letting the nerds down a little bit. Um,
little bit. Um, as far as like the AI obsessives. Yeah.
as far as like the AI obsessives. Yeah. Are there any any um technical
Are there any any um technical anything you see people like kind of
anything you see people like kind of getting wrong about you guys at the
getting wrong about you guys at the moment that um you would set the record
moment that um you would set the record straight on?
straight on? >> Yeah, I I mean I think the most
>> Yeah, I I mean I think the most important thing is um just I think
important thing is um just I think anyone at OpenAI in research would tell
anyone at OpenAI in research would tell you that it is just a researchcentric
you that it is just a researchcentric company. It's a pure AI bet. um at the
company. It's a pure AI bet. um at the core of the company the ambition is to
core of the company the ambition is to build AGI it's to build it without
build AGI it's to build it without distractions and I think the you know
distractions and I think the you know anything when it comes to building
anything when it comes to building products it all flows very easily from
products it all flows very easily from that um yeah when it comes to what we
that um yeah when it comes to what we want to do in research it's
want to do in research it's you know we want to automate AI research
you know we want to automate AI research I think selfishly like we want to
I think selfishly like we want to accelerate our our own progress and then
accelerate our our own progress and then we want to automate scientific discovery
we want to automate scientific discovery and of course we want to automate the
and of course we want to automate the ability to do economically useful work
ability to do economically useful work and um I think all these pillars are
and um I think all these pillars are falling it
falling it and and you see that kind of the big
and and you see that kind of the big update in the last year has just been
update in the last year has just been like in that second pillar of automating
like in that second pillar of automating scientific research. It's happening.
scientific research. It's happening. >> Yeah.
>> Yeah. >> How old are you now?
>> How old are you now? >> Um 34 about to turn 35.
>> Um 34 about to turn 35. >> About to turn 35. Okay. Are you able to
>> About to turn 35. Okay. Are you able to have like a social life or are you are
have like a social life or are you are you
you >> No, honestly not. Um I yeah I think
>> No, honestly not. Um I yeah I think every day the last two weeks, you know,
every day the last two weeks, you know, it's been work calls till 1 2 a.m. Uh
it's been work calls till 1 2 a.m. Uh but I but I love doing it. It's just um
but I but I love doing it. It's just um you know, there's there's a lot of work
you know, there's there's a lot of work to get done. There's a lot of people I
to get done. There's a lot of people I want to recruit. There's um a lot of
want to recruit. There's um a lot of steering that needs to be done. And like
steering that needs to be done. And like why waste this golden moment? It's like
why waste this golden moment? It's like if we're in the middle of something like
if we're in the middle of something like an industrial revolution, um you got to
an industrial revolution, um you got to take as much advantage of it as
take as much advantage of it as possible.
possible. >> Yeah. I hear stories about you sleeping
>> Yeah. I hear stories about you sleeping at the office and
at the office and >> oh yeah that that was a fun one too. Um
>> oh yeah that that was a fun one too. Um no honestly it's just um yeah I think
no honestly it's just um yeah I think there are times in the company um I
there are times in the company um I think that was right after um Barra left
think that was right after um Barra left and and went to found their own company.
and and went to found their own company. Um, it just the job demands it and um,
Um, it just the job demands it and um, like I think when I peel it all back and
like I think when I peel it all back and examine that deep emotion, it's just
examine that deep emotion, it's just this protectiveness of of the research.
this protectiveness of of the research. >> That was after Mera mirror left.
>> That was after Mera mirror left. >> Yeah. Yeah. I spent spend spend a month
>> Yeah. Yeah. I spent spend spend a month kind of sleeping in in the office and
kind of sleeping in in the office and it's just like I I need to protect the
it's just like I I need to protect the research. They they feel like, you know,
research. They they feel like, you know, it feels like my baby. Yeah.
it feels like my baby. Yeah. >> So, you guys have gone through these
>> So, you guys have gone through these waves. There's the coup. Everybody's
waves. There's the coup. Everybody's trying to steal your people.
trying to steal your people. >> [snorts]
>> [snorts] >> I guess everybody's trying to steal your
>> I guess everybody's trying to steal your people all the time, but you have this
people all the time, but you have this inflection point. Mirror leaves, Meta
inflection point. Mirror leaves, Meta decides they're going to fire up this
decides they're going to fire up this massive lab. Do you think are we like
massive lab. Do you think are we like are we pass has everybody fired their
are we pass has everybody fired their their shot at this point?
their shot at this point? >> You know, um you know, I have a staff
>> You know, um you know, I have a staff meeting, right? I talked to talk to my
meeting, right? I talked to talk to my reports and I'm like, okay, well, here
reports and I'm like, okay, well, here here's the thing that I'm working on and
here's the thing that I'm working on and you know, once I get back to once I'm
you know, once I get back to once I'm done with this thread, you know, I'm I'm
done with this thread, you know, I'm I'm going to zoom out and you know, there's
going to zoom out and you know, there's no more fires. Um, no, no. I I've fully
no more fires. Um, no, no. I I've fully internalized at this point, you know,
internalized at this point, you know, the stakes are high enough for for
the stakes are high enough for for building AGI that there's always going
building AGI that there's always going to be something. And um, I think the
to be something. And um, I think the important thing is just being able to
important thing is just being able to understand what the important things are
understand what the important things are in the midst of all of all of these
in the midst of all of all of these these things going on.
these things going on. >> Do you like months have passed since
>> Do you like months have passed since there was sort of the deepseek moment or
there was sort of the deepseek moment or whatever? I guess it was like December
whatever? I guess it was like December 2024, I think.
2024, I think. >> Yeah. Yeah. earlier earlier this Yeah.
>> Yeah. Yeah. earlier earlier this Yeah. >> Yeah. Yeah. Or January. Yeah. I mean, is
>> Yeah. Yeah. Or January. Yeah. I mean, is there anything um now, you know, it felt
there anything um now, you know, it felt like people lost their minds for a
like people lost their minds for a second? Um just like reflecting on it
second? Um just like reflecting on it now um and seeing what they've done
now um and seeing what they've done since just just like thoughts, I guess,
since just just like thoughts, I guess, on open source models and Chinese open
on open source models and Chinese open source models.
source models. >> Yeah. I mean I think that was one of the
>> Yeah. I mean I think that was one of the first points in time when um I just
first points in time when um I just realized how important it is that we
realized how important it is that we just stay true to our research format. I
just stay true to our research format. I think when when that came out, you know,
think when when that came out, you know, it it went viral, right? Like everyone
it it went viral, right? Like everyone was like, "Oh man, like has OpenAI lost
was like, "Oh man, like has OpenAI lost its way? Are these models catching up?"
its way? Are these models catching up?" And um what's the response? What's the
And um what's the response? What's the response? What's the response? And I
response? What's the response? And I think rightfully the thing that we did
think rightfully the thing that we did was um we just stumbled down our own on
was um we just stumbled down our own on our own research program. And I don't
our own research program. And I don't think it was the right it was the wrong
think it was the right it was the wrong call at all. Like um I haven't seen the
call at all. Like um I haven't seen the Deep Seek follow-up model. Um, you know,
Deep Seek follow-up model. Um, you know, I I think they're they're a very strong
I I think they're they're a very strong lab, but fundamentally like let's just
lab, but fundamentally like let's just keep focusing on innovating. I think um
keep focusing on innovating. I think um you know, DeepC was a great kind of
you know, DeepC was a great kind of replication of the ideas in in our O
replication of the ideas in in our O series of models, but let's just focus
series of models, but let's just focus on innovating. Do you think 500 people
on innovating. Do you think 500 people is the does that number grow as the
is the does that number grow as the company grows or this is like the
company grows or this is like the optimum number for kind of like big
optimum number for kind of like big ideas you can chase at one time? Um no
ideas you can chase at one time? Um no honestly uh I feel like it can be done
honestly uh I feel like it can be done with even less and um again you know as
with even less and um again you know as we get AI researchers or AI interns uh
we get AI researchers or AI interns uh there's a real question of how do you
there's a real question of how do you design an around that but um I'm
design an around that but um I'm certainly a person who cares a lot about
certainly a person who cares a lot about heavy talent density um when like I I
heavy talent density um when like I I like to run a lot of experiments um of
like to run a lot of experiments um of this vein for for instance in quarter
this vein for for instance in quarter two of this year um I thought hey you
two of this year um I thought hey you know I'm just not going to open up any
know I'm just not going to open up any headcount for anyone research and you
headcount for anyone research and you know if you want to hire people uh you
know if you want to hire people uh you got to figure out who's not [snorts] on
got to figure out who's not [snorts] on the boat and um I think these kind of
the boat and um I think these kind of exercises are quite important you know
exercises are quite important you know you
you don't want a or to diffuse into
don't want a or to diffuse into something that's not manageable and you
something that's not manageable and you want to keep the talent bar very high
want to keep the talent bar very high >> I okay I promise this is the last
>> I okay I promise this is the last question so yeah sorry I have to set you
question so yeah sorry I have to set you free um uh the
free um uh the >> I remember being in a meeting and
>> I remember being in a meeting and >> I think you and Yakoba were kind of on
>> I think you and Yakoba were kind of on the same page here, but I remember you
the same page here, but I remember you for sure.
for sure. >> Um, sort of like this idea of of who
>> Um, sort of like this idea of of who gets attribution for a project and
gets attribution for a project and >> you seem to be of the stance that like
>> you seem to be of the stance that like people are obsessing over that a bit too
people are obsessing over that a bit too much and and clearly AI has its roots in
much and and clearly AI has its roots in academia where you are very proud when
academia where you are very proud when you have a paper and it's a big deal and
you have a paper and it's a big deal and and attribution is a huge thing. Um I
and attribution is a huge thing. Um I think I'm remembering that meeting right
think I'm remembering that meeting right and yours.
and yours. Yeah. And and so
Yeah. And and so >> what we've reached a new stage where
>> what we've reached a new stage where that is less of a big deal or it's just
that is less of a big deal or it's just you this is a company and who did what
you this is a company and who did what is less important.
is less important. >> I I actually really love this topic and
>> I I actually really love this topic and um I think overfixation on credit is a
um I think overfixation on credit is a very bad thing. Right. I I think you
very bad thing. Right. I I think you know but on the other hand I actually
know but on the other hand I actually feel like it's important as a company
feel like it's important as a company for us to recognize credit both
for us to recognize credit both internally and externally and a lot of
internally and externally and a lot of companies have actually shied away from
companies have actually shied away from this you know uh we've moved away from
this you know uh we've moved away from publishing papers credits lists I think
publishing papers credits lists I think broadly throughout the industry but
broadly throughout the industry but Jakob and I ended up making the call
Jakob and I ended up making the call that we're going to do it at OpenAI and
that we're going to do it at OpenAI and of course the counterargument is always
of course the counterargument is always like man you're like handing your top
like man you're like handing your top performers on a platter, you know,
performers on a platter, you know, everyone else is going to be recruiting
everyone else is going to be recruiting these guys aggressively. But I don't
these guys aggressively. But I don't think that's important, right? Like we
think that's important, right? Like we should just recognize the people who are
should just recognize the people who are doing great work. We should continue to
doing great work. We should continue to be this pipeline for creating AI
be this pipeline for creating AI superstars. And um yeah, honestly, it's
superstars. And um yeah, honestly, it's important for us to make names for the
important for us to make names for the people who are doing the best work at
people who are doing the best work at the company. So,
the company. So, >> but you seem to also be saying
>> but you seem to also be saying people, the individual researchers
people, the individual researchers should maybe obsess about this less.
should maybe obsess about this less. Where or am I am I totally misremeless?
Where or am I am I totally misremeless? >> No, I I think there was a sentiment in
>> No, I I think there was a sentiment in the room of of that form. Um, actually
the room of of that form. Um, actually Yakob and I held more of a dissenting
Yakob and I held more of a dissenting view on that.
view on that. >> Okay. Okay. Okay. It's been a while.
>> Okay. Okay. Okay. It's been a while. It's in my notes. Perfect.
It's in my notes. Perfect. >> Yeah. Yeah. Yeah. [laughter]
>> Yeah. Yeah. Yeah. [laughter] Yeah. But um I I think we got to give
Yeah. But um I I think we got to give credit where it's due even at the risk
credit where it's due even at the risk of everyone knowing who our top talent
of everyone knowing who our top talent is.
is. >> Okay. Okay. Um
>> Okay. Okay. Um >> I will make an even stronger statement
>> I will make an even stronger statement that I think OpenAI is the place where
that I think OpenAI is the place where we allow for the most external credit
we allow for the most external credit per capita.
per capita. >> Okay.
>> Okay. >> By a large margin.
>> By a large margin. >> Okay. All right. Well, I'm check my
>> Okay. All right. Well, I'm check my notes on. Well, now I've got more.
notes on. Well, now I've got more. >> Absolutely. Absolutely. [laughter]
>> Absolutely. Absolutely. [laughter] >> Yeah.
>> Yeah. >> I just I I just remember it being a
>> I just I I just remember it being a topic of discussion and and and I there
topic of discussion and and and I there were numerous opinions. So, um, that
were numerous opinions. So, um, that that's funny. Um, in that, okay, I lied.
that's funny. Um, in that, okay, I lied. Last question, I swear. So, you know,
Last question, I swear. So, you know, you got there in 2018. I mean, it was a
you got there in 2018. I mean, it was a research company. It was a nonprofit.
research company. It was a nonprofit. Um,
Um, the company started among the the
the company started among the the founders, you know, being this this
founders, you know, being this this counterwe to um to Google and and with
counterwe to um to Google and and with sort of, you know, making sure AGI
sort of, you know, making sure AGI arrived safely was kind of the goal. Um,
arrived safely was kind of the goal. Um, you came at this from highfrequency
you came at this from highfrequency trading and and and saw these
trading and and and saw these interesting things happening.
interesting things happening. >> You know, like how much in your
>> You know, like how much in your >> I'm sure you're going to say you want
>> I'm sure you're going to say you want this to happen safely. I get that. But
this to happen safely. I get that. But like if you look at your career path,
like if you look at your career path, you're a smart, curious human who saw
you're a smart, curious human who saw this interesting thing happening.
this interesting thing happening. It's not like a requirement that you
It's not like a requirement that you like really give a philosophically
like really give a philosophically about this or or want to see, you know,
about this or or want to see, you know, a super intelligence. Yeah. I mean, but
a super intelligence. Yeah. I mean, but anyway, like let's hear from you on on
anyway, like let's hear from you on on like why are you doing this in the first
like why are you doing this in the first place?
place? >> Yeah. So I think um really on the safety
>> Yeah. So I think um really on the safety and alignment piece um I managed the
and alignment piece um I managed the alignment team at OpenAI as well and um
alignment team at OpenAI as well and um I honestly feel like some of the grand
I honestly feel like some of the grand challenges over the next one or two
challenges over the next one or two years are alignment and I think for
years are alignment and I think for people paying attention to this slice of
people paying attention to this slice of research broadly in the field open I
research broadly in the field open I think has probably done the best work in
think has probably done the best work in the last year and why I say that is like
the last year and why I say that is like there's been so much work on things like
there's been so much work on things like scheming right the more RL compute that
scheming right the more RL compute that you pump into the model the more you can
you pump into the model the more you can measure
measure things like self-awareness,
things like self-awareness, self-preservation,
self-preservation, uh potentially even situations where the
uh potentially even situations where the model can scheme and and it's scary
model can scheme and and it's scary because the model can come to you with
because the model can come to you with the right answer at the end, the answer
the right answer at the end, the answer that you expect, but arrive at it from a
that you expect, but arrive at it from a very kind of twisted way, right? And um
very kind of twisted way, right? And um I think as the models do more complex
I think as the models do more complex tasks for us uh having a handle on what
tasks for us uh having a handle on what its thought process is um is going to be
its thought process is um is going to be super super important and okay chat told
super super important and okay chat told me to ask you a question along these
me to ask you a question along these very lines which is I mean you're
very lines which is I mean you're talking about a field mechanistic
talking about a field mechanistic interpretability where we're trying to
interpretability where we're trying to >> is a a term that captures trying to
>> is a a term that captures trying to understand this black box and how it
understand this black box and how it operates and and I guess the heart of
operates and and I guess the heart of the question was
the question was Do our skills at doing that keep up with
Do our skills at doing that keep up with the complexity of the AI systems or do
the complexity of the AI systems or do we just get to this runaway point where
we just get to this runaway point where it's like we're never going to learn how
it's like we're never going to learn how this thing works?
this thing works? >> Yeah. Um, so I think one of the
>> Yeah. Um, so I think one of the decisions that went all the way back to
decisions that went all the way back to 01's release, which I'm very proud of,
01's release, which I'm very proud of, is we decided that we weren't going to
is we decided that we weren't going to supervise the the model thinking
supervise the the model thinking process. Um, and I think when you put
process. Um, and I think when you put incentives into the model to uh, you
incentives into the model to uh, you know, give you a thinking process that
know, give you a thinking process that is appealing to a human, um, it won't
is appealing to a human, um, it won't necessarily be honest with you, right?
necessarily be honest with you, right? It won't say kind of tell you it's it's
It won't say kind of tell you it's it's its true intentions and and so we've
its true intentions and and so we've actually through that channel been able
actually through that channel been able to maintain observing the thinking
to maintain observing the thinking process of the model as a tool towards
process of the model as a tool towards understanding alignment. And um you know
understanding alignment. And um you know there was a paper that was published
there was a paper that was published just a couple months ago uh with deep
just a couple months ago uh with deep mind with anthropic um really exploring
mind with anthropic um really exploring kind of how this will evolve as a tool
kind of how this will evolve as a tool over time and and so you know I think
over time and and so you know I think we've made a lot of fairly good choices
we've made a lot of fairly good choices in in design here. Um yeah, I really do
in in design here. Um yeah, I really do worry about this world in the future
worry about this world in the future where the model will tell us something
where the model will tell us something super convincing but we can't be sure um
super convincing but we can't be sure um whether the model is aligned with us,
whether the model is aligned with us, right? align with our values and and so
right? align with our values and and so I think there are a lot of interesting
I think there are a lot of interesting directions here like um can you set up
directions here like um can you set up games right or can you set up frameworks
games right or can you set up frameworks or environments where you know like
or environments where you know like models supervise each other or they
models supervise each other or they co-evolve together in a certain way
co-evolve together in a certain way where like the only like stable
where like the only like stable equilibrium is one where you know the
equilibrium is one where you know the models are honest um and yeah I think
models are honest um and yeah I think there's a lot of very exciting work to
there's a lot of very exciting work to do there
do there >> okay all right okay I'll behave myself
>> okay all right okay I'll behave myself now thank you so much for joining us I I
now thank you so much for joining us I I am glad I'm old enough now that I don't
am glad I'm old enough now that I don't have to take a job interview from like a
have to take a job interview from like a super intelligent [laughter]
super intelligent [laughter] uh
uh chatbot that like I I feel like uh that
chatbot that like I I feel like uh that you can't sort of try to charm your way
you can't sort of try to charm your way past and and
past and and >> Great, Ashley. You would do great at
>> Great, Ashley. You would do great at >> I don't know, man. I don't know. I'm
>> I don't know, man. I don't know. I'm feeling okay, but I've I've I've old
feeling okay, but I've I've I've old enough not to have to probably do that.
enough not to have to probably do that. U Thank you, Mark, so much. You're I
U Thank you, Mark, so much. You're I know you're super busy, so thank you for
know you're super busy, so thank you for your time.
your time. >> Thank you so much for your time, too.
>> Thank you so much for your time, too. All right, man. It was fun. Really
All right, man. It was fun. Really pleasure.
pleasure. >> Okay.
>> Okay. [music] The Core Memory podcast is
[music] The Core Memory podcast is hosted by me, Ashley Vance. It is
hosted by me, Ashley Vance. It is produced by David Nicholson and me. Our
produced by David Nicholson and me. Our theme song is by James Mercer and John
theme song is by James Mercer and John Sortland. And the show is edited by the
Sortland. And the show is edited by the John Sortland. Thanks as always to Brex
John Sortland. Thanks as always to Brex and Elone Ventures for making this
and Elone Ventures for making this possible. Please visit our Substack,
possible. Please visit our Substack, YouTube, and podcast channels to get
YouTube, and podcast channels to get more of what Core Memory makes. Thanks
more of what Core Memory makes. Thanks y'all.
Click on any text or timestamp to jump to that moment in the video
Share:
Most transcripts ready in under 5 seconds
One-Click Copy125+ LanguagesSearch ContentJump to Timestamps
Paste YouTube URL
Enter any YouTube video link to get the full transcript
Transcript Extraction Form
Most transcripts ready in under 5 seconds
Get Our Chrome Extension
Get transcripts instantly without leaving YouTube. Install our Chrome extension for one-click access to any video's transcript directly on the watch page.
