good morning and welcome to Stanford for
our conference on promoting competition
in artificial intelligence my name is
Maria France I'm the director of the
business government and Society
initiative at the Stanford Graduate
School of Business artificial
intelligence or AI is the topic dour
from the classroom to the boardroom but
what makes today's conference unique is
its multifaceted look at the competitive
Dynamics across the AI landscape from
the chip to the app I'd like to thank my
colleagues at the Stanford Institute for
economic policy research and the
antitrust division of the US Department
of Justice for creating a compelling
program for many of us to learn and help
to shape future policies and strategies
so let's get started it's my privilege
to introduce our first Speaker Jonathan
caner Jonathan caner is the Assistant
Attorney General for the antitrust
division of the United States Department
of Justice he has served in that
position since November of 2021
throughout his career a ag caner has
been a leading advocate for strong and
meaningful antitrust enforcement and
competition policy having served as a
partner at two national leading law firms
firms
and as a founder of his own Boutique
antitrust Law Firm he began his career
as an attorney at the Federal Trade
commission's Bureau of competition
please join me in welcoming Jonathan [Applause]
caner well Maria thank you for that
incredibly warm introduction and for
having us here today um it's pretty
amazing um in academic
conference uh that is full first thing
in the morning and so um it really is I
think a statement to why this issue
matters so much and matters to so many
people there's a degree of urgency and
seriousness um and today's discussion is
really about bringing together people
who sometimes may not ordinarily come
together uh in the same room or on the
same uh panel and talk about issues that
are important and sometimes difficult to
talk about but
um that's what we're here to do it's
sometimes to confront the hard problems
uh and it's only when we do that that we
can actually start developing Solutions
um on beh half of the antitrust Division
I I want to thank Stanford Institute for
economic policy research uh and GSB uh
for hosting this conference it really is
amazing um but I would like to take a
moment uh if you will indulge me um to
acknowledge a particular specific person
here um our Chief Economist Susan ay um
who is not just the chief Economist of
the antitrust division but uh Stanford
Professor um she is one of a kind um
truly one of the smartest most
accomplished Minds in the world and has
been on the Forefront of so many of the
critical issues that um we're going to
talk about today uh and was doing that
before anyone else um I've had the
privilege and um and um honor of of
knowing Susan now for oh it must be 15
years and we've become good friends and
colleagues and thought partners and I'm
learning from Susan every day she is um
generous with her mind she is a leader
um a thought leader a leader of people
and somebody who um really is uh among
the smartest most talented and skilled
um thinkers of her generation in the
world and and we are honored every day
to work with her as I often say who
needs artificial intelligence when you
have the real thing and the real thing is
Susan um we meet today at the dawn of a
new technological Revolution AI has so
much promise but as with prior technological
technological
revolutions there will be threats to
confront um they're going uh ongoing
debates and we're hearing them all the
time and we'll hear them today uh that
impact Safety and Security um and talk
about how new technological developments
including AI can benefit Society but
also create risks for society uh the
antitrust laws play an important role uh
in this um development of new
technologies um there are um and it
exist broadly the antitrust laws to pre
preserve healthy and competitive markets
um today in this Workshop we're going to
focus on how to realize the competitive
promise of AI and reduce its threat to
competition in the American economy uh
and and reduce threats to competition in
the American economy um history teaches
us that effective antitrust enforcement
often coincides with major industrial
and technological change throughout
history we see examples of important
antitrust cases in fact some of the most
significant anti trust cases in history
that opened up markets and paved the way
for new companies and new Innovations to
arrive and to thrive um tr's Standard
Oil case is a famous early example known
to many splitting that oil trust into at
the time 34 independent companies today
we still um know and talk about and
purchase from The Offspring of Standard
Oil in companies including Exxon Mobile
and Chevron marathon and even BP which
merged with Amo and Arco um descends
from Standard Oil antitrust enforcement
actually resulted in the creation of new
competitors who thrived because of their
rivalry and we've seen lots of benefits
to this to society um in their
intervening Century we see a similar
story with uh the Bell System breakup
When AT&T was broken up into multiple
companies competition and telephone
markets thrived alongside those new
firms um today's a today we have AT&T
Verizon Lumen perhaps not as many as
some wish we would have uh but they are
direct descendants of successful
antitrust enforcement uh at the same
time that opened the technological space
to competition and created the room for
us Telecom and internet Industries to
develop and Thrive and more recently uh
the division's victory in the 20 1 case
against Microsoft for illegal
monopolization opened up the modern
digital economy and paved the way for
the success of today's firms and Beyond
many of whom inhabit um this very part of
of
California over and over again we see
that antitrust enforcement at moments of
industrial Evolution has the opportunity
to Spur innovation in its wake opening
the door to competition in new
competitors allows for the develop of
different business models and new
economies but we also see structures and
Trends in AI that should give us
pause AI relies on massive amounts of
data and computing
power which can give already dominant
firms a substantial Advantage powerful
Network and feedback effects may enable
dominant firms to control these new
markets an existing power in the digital
economy May create a power powerful
incentive to control emerging
innovations that will not only impact
our economy but the health and
well-being of our society and free expression
expression
itself these Technologies hold unbounded
promise for Innovations and change that
were once the exclusive domain of
Science Fiction at the same time AI
inputs and outputs have unique
characteristics that present new threats
to the markets for human ideas and innovation
innovation
generative AI for example leverages the
creations of humans knowledge paintings writing
writing
ideas absent competition to compensate
creators for their Works adequately
compensate creators for their Works AI
companies could exploit monopsony power
on levels that we have never seen before
with devastating
consequences everyone concerned about
human progress should be concerned about
that what incentive will tomorrow's
writers creators journalists thinkers
and artists have if AI has the ability
to extract their Ingenuity without appropriate
appropriate
compensation financially they will have
only those incentives that competition
competition between Foundation models
acting in concert with the IP system
creates in the absence of competition we
may see the problems Market power on the internet
internet
has caused in journalism spread to other
critical content creation
markets the people who create and
produce these inputs must be properly
compensated this of course is critical
to the Creator Community which is not
just about entertainment but
entertainment is very important but
about free
expression which is the most impactful
and beautiful form of human Innovation and
and
Ingenuity but it's also about so much
more it's about the Physicians and
patients whose Health Data is fed into
massive AI models it's about the
creators and artists whose words
thoughts and creativity are being
captured and used and of course the
journalists news outlets who are vital
to our
democracy there's more good news in the
history of industrial Evolution however
the ni trust laws adapt to changing
Market realities the principles of
competition enforcement apply whether an
innovation is powered by steam by
transistors or by reorganizing human
thought through machine learning at the
antitrust division we are actively
examining the AI
ecosystem both through our policy work
at events just like this
one and through our enforcement of the
Sherman and Clayton XS monopolizing
Upstream markets for Creative works is
monopolization whether or not a large
language model is involved D combining
to set prices with rivals in concerted
action whether or not an algorithm
assist that collision and risks fixing
prices if firms in the AI ecosystem
violate the antitrust laws the antitrust
division will have something to say about
about
that of course adapting our enforcement
program requires a deep understanding of
emerging AI ecosystem in different
sectors across the economy I'm
enormously excited to learn and to dis
discuss those issues today with an
incredibly impressive group of experts
today is really an important step
forward in our effort to learn and grow
it's our also an effort and this is
really I think a a distinctive feature
of today's
discussion um about bringing
stakeholders together from different
parts uh of the Continuum different
parts of the stack as we heard before
people who write code develop Hardware
write words and everything in
between so with that it's my pleasure to
kick us off and I'd like to introduce
vice president Jova she is the vice
president of the European commission for
values and
transparency from 2014 to
2019 vice president served as European
commissioner for justice consumers and
gender equality before that she served
as the minister for regional development
of the Czech Republic in her current
role vice president Jova leads the
commission's work on values and
transparency upholding the rule of law
and ensuring the Integrity of the
democratic system her deep well of
experience also
includes uh issues in consumer
protection data protection privacy
rights and importantly media Freedom we
are honored thrilled to have her with us
today thank you for joining us as part
of your us visit please join me in
you ladies and gentlemen this is exactly
what the AI will never replace for us
this human beings meetings and warm
words thank you very much for giving me
the opportunity to to speak uh at this
uh very important conference uh on a
very topical uh matter because uh the
artificial intelligence uh it's not the
matter of the future it's something we
live through and with already today and
uh over several years of my involvement
in the creation of the rules for AI and
digital world uh in the in the EU um I
had that feeling that somebody very
important is missing in that and it's
the sphere of research and science which
should have stronger voice I always was
uh participating in the interplay
between politicians and
Technologies and really the the voice of
research was missing and I think that
it's now time that uh the voice of
researchers and scientists is heard uh
more and uh where is Jonathan he
left uh I wanted also to thank thank him
and the Department of Justice for almost
10 years of fantastic
cooperation uh we worked uh together on
better understanding of uh the need uh
to protect privacy of people uh I had a
lot of discussions with the Department
of of Justice about how to cooperate
better in the field of Criminal Justice
and recently and this is my special
thanks uh we have very intense talks
about about how to uh bring Justice more
Justice into the I would call it
miserable War which is up and running in
Ukraine because we discuss about how to
uh establish the tribunal to punish the
uh war of aggression and and genocide
and we are also discussing how to use
the uh frozen assets from Russia to uh
uh reconstruct uh Ukraine so many things
in the times like this we only don't
need reliable Partners or Allies we need
friends and EU feels strongly especially
in these days that we have friends in
the United States so thank you very much
for that but coming to our today's topic
uh thank you for bringing uh the
spotlight into a very important
intersection of AI and
competition that is an overwhelming
agreement AI is a technology that can
bring many revolutionary
changes the paraphrase a quote from the
recent opam Heimer movie this isn't a
new technology it's a new
world it is one of the most powerful
Technologies ever invented and AI has a
potential of transforming everything not
only a competition
policy today I would like to talk to you
about two challenges of a I and the eu's
approach to it first AI challenges to
competition to stay faithful to theme
chosen by our hosts second I would like
to zoom out on the broader challenges
for AI of AI for democracy because this
is what is on my table in the European
commission this is at the heart of my
visit in California as part of what I
call a democracy tour in Europe we are
working hard to find answers to these
challenges brought to by digitalization
and shape our own approach to them we
start from a simple idea technology is
to serve the humans not the other way
around We Are Not Mere data fields we
are not just data donors for tech
companies to harvest from and then make
the decisions for us or make money on
our thoughts and
fears this is why we have chosen a
European model to regulate Technologies
in order to make sure they respect human
rights and I want to tackle one
skepticism that I have heard from many
concerns in the United States had on
namely that regulation stifles
Innovation I heard a lot of that when we
were introducing the general data protection
protection
regulation uh this uh concern this fear
did not materialize and I am sure it
will be also the case of AI I am sure
that Innovations and regulations can
work hand in hand we design laws to
address risks for the people or to open
markets that have been sealed by those
who have become too big to compete
against this brings Trust of of the
consumers and Innovations through
competition predictability and and
sufficient legal
certainty and where there is trust and
healthy competition there is investing
from both public and private
sources but there are indeed challenges
to competition and I'm sure you will
speak a lot about that
today the debate is very timely because
I still believe we have a narrow window
of opportunity to set the rules that
would allow competition to thrive in the
world of AI
in both the EU and the us we are facing
very similar problems to deal with the
challenge of the future it is good to
understand the past so we know where we
are coming from I am here thinking of
the digital disruption in the early
2000s social media digital marketplaces
online payments this was all so
exciting and this was the age full of
pioneer ing innovators some started in
the garage many of them at universities
or while dropping out of them the
innovators could not foresee or grasp
grasp
sorry the scale of impact their
Revolution would
bring today with a revolution some of
those things sound familiar but I also see
see
differences first the revolution is Led
in large part by incumbents powerful
companies with stakes in many many
markets second our awareness virtually
no one has doubts that AI will affect
every aspect of our work and life this
is what we didn't know at the beginning
of the
digitalization at the moment when some
boys started to invent things in the
garage I will speak with some of them here
here
also at my visit of
California uh the young inv inventors
who didn't know that one day they will
have impact on the whole world I used to
call them digital Gods when I came first
time to California in
2016 and now I think that we came to
agreement that we need meaningful rules
and for them the European rules are
quite demanding but what I hear from the
tech companies uh it's good that in
Europe which is such an important market
for us we have the necessary legal
certainty so but coming back from the
early uh Century the century and garage
uh coming back to
presents so our awareness is here now we
know that AI will affect every aspect of
our work and Life Third the technology
itself Ai and large language models have
he huge entry requirements they need
unprecedented amount of data and storage
among others the more the better in the
market where you need an entire
ecosystem to thrive it is hard to
imagine a teenager with a vision
challenging Microsoft and open eye AI or
metas Lama 2 or Google's de mind and as
many of you in the crowd today are
competition experts I am sure you can
immediately see barriers to entry
everywhere so how to answer these
challenges first I cannot help it we
have to deploy full human intelligence
and events like this we have to figure
it out and I'm sure we will second by
working together and this is why I
appreciate a Meeting of Minds with
Governor Newsome Europe and us must
continue to work together closely on
these issues to achieve a greater impact
and third by thinking out of silos
competition policy has to work together
with digital regulation that sets guard
rails in Europe This Means for instance
the digital markets
act let me now move to the challenges of
AI for
democracy democracy is an open debate
where we can argue and disagree seek for
compromise and then argue again we are
now few days ahead of European elections
I always say that the elections and
electoral campaigns have to remain fair
play of real human beings real Visions
people speaking to each other not the
competition of who will be better in
using manipulative techniques the this
is why are am I'm traveling now from
state to state in the EU and asking our
partners how they are ready to counter
these new
threats we have seen how digital
revolution has pushed us further into
bubbles or digital rabbit holes we are
still learning about the impact of
social media on our children and their
mental health and I do believe this
could be the main challenge for us in
the years to come
we are seeing in seeing in Europe how
Russia but also China Iran and other
actors use digital means to spread
disinformation and foreign
interference and ladies and gentlemen
this is a security risk to Europe and to
the United States as well Russia is
fighting in Ukraine with bombs and with
disinformation all over the world also
here in the United
States with AI they are just gaining a
new powerful tool to deploy old
tactics why they are doing this because
the essence of democracy is trust trust
to one another trust in the Democratic
institutions or the media without trust democracy
democracy
crumbles now with AI we can hardly trust
what we see or hear anymore deep fakes
synthetic images or videos not to
mention texts are fast becoming
indis IND
writer never again this [Laughter]
[Laughter]
word I was training the whole morning this
this [Laughter]
word but he wanted to
that uh but what I wanted to say yeah
the human eye cannot distinguish between
the false and the truth this is what is
say I already mentioned that I travel
from state to state on my democracy tour
I want to understand how well you can
countries and the authorities experts
and also the whole society are prepared
to digital threats to democracy and
security among other things to the fair or
or
unfair uh competition in electoral
campaign and this also is topic which
brings me to California because big Tech
has a tremendous role to play as well
what is being done here has a big impact
in Europe and in the rest of the world I
cannot cannot resist I have to share
with you the story when I first realized
that and I was here I think it was 2016
or 17 we had a big crisis in Spain the
separatists uh uh organized referendum
uh in
Catalonia and here in Silicon Valley I
received a lot of questions what what is
Catalonia Catalonia
and I didn't understand why they asked
me and they asked me because they had
hundreds of calls on closing this or
that website time is up I am in the
middle uh but this is a good story I
will finish
it uh that was the moment when I
realized that the boys in Silicon Valley
will not re decide on how our
referendums will
result and this was also the moment
which started first uh uh gentleman
agreements with the digital uh companies
and then the
regulation give me two more
minutes Madame
Professor thank
shter um yes uh we proposed an AI Act a
first comprehensive legislation on AI
including generative
AI we will enter it will enter into
force in July it is not only trust
fostering but also good for Innovation
first because of scale it means one set
of rules across all 27 European member
states second it gives the providers of
AI systems a stable and predictable
operating environment and sir said the
AI act encourages companies to develop
new products
easier I've been 10 years in the
commission over eight years we were
discussing how to regulate the digital
space eight years of discussions and
then regulation came the Digital
Services act for AI we feel we don't
have that time we had to act much
quicker so that's why we have the AI Act
and we also have uh relevant
International agreements like G7
agreement on secure uh
AI uh in the AI act uh we uh took the
risk-based approach we require the
companies to test the risky product and
the incidence with Tik Tock light or
recent launch of the Google search
product shows that this is only good for
consumers but good for companies as well
ladies and gentlemen the debate like you
are having today is not only needed but
it is urgent I don't have all the
answers I think nobody has this is why
we need Collective efforts Collective
wisdom to ensure competitive and
Innovative Market preserve democracy and
ultimately our values we still have a
chance to shape the AI Revolution so it
can become a Force for a positive change
rather than another risk that brings us
closer to a reality captured in a
dystopian TV show Black Mirror thank you
for your attention I wish you a very fruitful
fruitful [Applause]
debate okay I'll take the
so thank you so much Vice President
durova we are so honored to have you
here and I think this is maybe the the
first time in the history of timing that
uh someone actually listened to the
warning uh timing sign so uh so thank
you we we we really appreciate your
remarks so in this next part I'm I'm
Professor Susan ay from Stanford GSB and
I'm currently Chief Economist at the any
trust division at the doj and we're to
do the table setting for the rest of the
day we're really fortunate to have two
of the leading lights in AI from from
computer science and so we'll be
connecting the technology and the the
science to the competition issues we're
talking about today and to start off
with we are so fortunate to have Percy
leang here who's a professor of computer
science and all of the BIOS extended
bios for our speakers are on the website
so as a star AI professor at Stanford
and one of the leading lights of
national language processing you were
perfectly positioned to lead us as we
transitioned into this latest evolution
of AI um and indeed you were ahead of
the game um we were actually working
together at the Stanford um Institute
for human centered artificial
intelligence a few years ago and you had
the foresight to propose and start the
center for research on Foundation models
in 2021 before the rest of uh the world
was quite so aware of what this was all
going to look like um can you talk about
your vision for the center and why you
founded it and and how it's going yeah
yeah thanks for having me Susan so 2021
basically prehistory um in in AI terms
um for many uh chat gbt um 2022 December
was the sort of the turning point for me
it was May 2020 that's when opening
released their gpt3 model and that model
was a a huge model an order of magnitude
larger than before and it could it
demonstrated to me that this technology
and this mindset of training larger and
larger models was going to have a huge
impact on on society and at that time I
went around talked to many folks at the
uh at Hai and and say hey we got to pay
attention to this um you know
fortunately many people uh uh were also
uh compassionate um and agreed so we've
founded the center for research on
Foundation models um some people thought
I was crazy but that's um but um that's
fine turned out you were right maybe
still crazy both can be true um and and
but it was surprising it uh Chad gbt I
think really um demonstrated that this
was uh something to pay attention to and
it did come faster than I thought um but
so so what is foundation models about so
the idea of a foundation model is that
um someone can take a large amount of
data at scale usually scraped from the
internet and throw a huge amount of comp
computation power at it and this one
recipe can produce a single model that
is capable of demonstrating a large
range of tasks um not just high-end
generation uh but but also um you know
ability to solve many tasks so for the
first time we have more of these
generalist uh agents that um people had
aspired to for decades I've been a AI
researcher for you know a few decades
and this moment is very uh special um so
it's somehow the the the idea that you
can take you know capital and put it in
and sort of predictably get U more and
more capabilities that that was a sort
of a mindset uh change and I want to
also stress that Foundation models isn't
just about chachal language models it's
a much more general idea that any time
in any domain where you have a huge
amount of data whether it be um you know
Vision Videos time series retail data um
computation biology astronomy all these
areas we're seeing being transformed
because of this kind of remarkably
simple idea where you have a Transformer
architecture plus um you know gradient
Ascent plus um you know gpus um allows
you to in any gain give you sort of
remarkable uh
properties great thank you so much and
so what do the the center actually want
to do what are what are your goals in
this and hopes and and how you can
contribute to the conversation about
them yeah so why do we create this
Center so one thing that we were
reacting to was that at that time since
2020 and you see a kind of progress um
more in this direction is that the level
of transparency and openness that we see
in AI has diminished right in the
2010s when the Deep learning um
Revolution was um you know going strong
um there was a culture of researchers um
sharing data uh code models openly and
various companies would be able to uh
startups would be able to leverage this
and I think a lot of the Innovation came
from um this this openness but what has
happened over the last three years is
kind of the as capabilities go up uh
openness uh goes down so from the idea
that um initially the model weights um
which is something hopefully we'll get
to a little bit later are no longer
available to you know the stop of uh
Publications um and gating of access and
and this I think from a academic
researcher perspective was quite
concerning and not only from a
researcher perspective where maybe
Academia could no longer engage with
this important technology but also from
a societal aspect if we have this hugely
important technology which we don't
understand and as is is locked away
behind you know a guarded closely then
uh we might um be in a bad situation so
the center was created to with a mission
of making Foundation model is more
accessible more transparent um for uh
the world and so what are the kind what
can you have some examples of what how
that transparency can be beneficial what
are some of the sort of use cases from
the center that that that would be
helpful for from that transparency yeah
so one of the key things that we're
focused quite heavily on is uh
benchmarking and evaluation so we have
these models that are being developed
and released and people are using them
but we have shockingly little rigorous
evidence of what their actual
capabilities and risks are there are a
bunch of demos and things are being
deployed and one thing that we've
undertaken at that Center is developing
benchmarks so we developed something
called the holistic evaluation of
language models uh two years ago which
we've been extending o over time to look
at all the different capabilities and
risks um um of these models as best as
we can and to try to uh Benchmark them
and also make all these results uh
publicly available for people to to see
and one of the things that's difficult
about benchmarking is that because these
models are so General um how do you even
you know test this I don't like usually
making analogies with humans but it it's
kind of like humans you're measuring the
cap abilities of a of a human being
because they they are so versatile and
so this is something that's very very
tricky but um I think we've made a lot
of progress on this uh covering many
different um tasks from question
answering to summarization now we're
looking at multimodal models for visual
question answering and image generation
and looking at um starting to look at
other dialogue uh settings so and I
think one thing that's important is
having a third party sort of neutral
third party that's doing these
evaluations and committing to doing them
across all the the foundation models
that are um are you know Frontier and
and uh reporting the results in a
standardized uh way yeah so is it is it
fair to say that as of a couple of years
ago we didn't have a lot of science
about how you would evaluate these types of
of
models yeah so I think um and we still
don't have a lot lot of science behind
how to evaluate these mod models but we
have a a little bit more now um I think
the the field of evaluation is just very
much underdeveloped I think in the
history of machine learning it used to
be the case that you define the
benchmarks and these benchmarks would be
very difficult for example imag net when
imag net was created no vision system
could really do anything respectable on
it and it was a sort of a North star for
many researchers to keep on working on
image and then so that that was a
database with a bunch of images and
labels and so you could just test out
how your model worked against it and
there was a number right A number that
said okay you beat the last Benchmark
I'm the star of the AI conference this
year yeah yeah and now I think it's sort
of flipped where people are developing
models and now benchmarks are trying to
catch up to capture what these models
are doing and on top of that we're no
longer interested in just the accuracy
but we're interested in things like bias
and fairness and robustness and whether
the models uh know what they don't know
and not to mention different types of
risks uh you know we heard about
disinformation but we heard about you
know their cyber risks and all these
different risks and how do you measure
them in a way that can inform you know
uh businesses and and policy makers and
so do you think it's possible to do that
science within like one research group
in one place or um do you think you you
know need a larger Community working on
those problems I I think you definitely
need a larg lger community and this is
presum uh the way that Academia works is
that it's it's the community we do a lot
of work but there's also good work
happening at various places like you
know University of Washington and um
many other uh universities where these
data sets are created and there's often
um benchmarks that are created that
derive from these data sets and then
they're put together in you know in
different ways um and so so a lot of the
you know meth what we're missing is sort of
of
methodological um you know science for
how to conduct benchmarks I mean just as
a simple example one of the things that
you know chat gbd can do is it can
generate you know an essay or a summary
and this alone is is a very deep you
know problem how do you evaluate um
something that's so complex and of
course you run into cases where if it's
on a topic that is is very Niche then
hiring human experts is is going to be
very expensive so can you use uh
language models themselves to evaluate
but then you don't trust the language
models so it's kind of it's very
interesting I think technically to see
what you can do and I think you can
actually make some progress even though
it seems like you're kind of
bootstrapping yourself um great well so
so but in general so it sounds like your
Center is promoting the ability ility
for lots of researchers to be able to
contribute to this relatively nent but
important field um so do you could you
talk us through other than the open
weights what do you actually need to to
be able to do to do that kind of work so
if we want to have a large community of
people contributing to this nent science
which might take a long time to figure
out because as you mentioned the will
probably still be developing new
evaluation metrics in 10 years for new
risks that have been identified what
what do what do people need to be able
to contribute to that whether it's
researchers or whether it's startups
that are creating um tools that can help
other businesses deploy them safely like
what are the inputs to make that to
contribute to that yeah I think one
thing I'll start by saying is um in
Academia you know we're meant to do
fundamental research that moves the
needle and a lot of what we are missing
right now is just you know computer
um I think we can do a lot by taking
existing models benchmarking them um we
can even if we have open weights for in
the case of llama three or llama 2 we
can inspect them but I think there's no
substitute for actually being able to
train um you know models um for example
we're developing different types of
model architectures which are uh
different from the
Transformer um and to demonstrate the
the the value of those we need to be
able to train and the the resource
differential between Academia industry
is just multiple orders of magnitude and
I think it's unrealistic to expect that
them to be equal but I think we as a um
you know uh Community um let's say we
need to invest more in in computation so
that we can at least be in um playing
playing the game so just to make sure I
understand so there's a couple of
different kinds of research you can do
one is that you can take either a closed
or an open model if for the closed you
have to pay for it and then you can
evaluate it using evaluation metrics but
then there's other research that you can
do that kind of gets into the guts of it
that you could only do by training your
own models perhaps at smaller scale and
so could you talk about what's needed
for the different types and what other
inputs you might need for the second one
yeah so for the first typee which is
taking existing models and studying them
um you need access for and for the most
part we do have access um I think it
would help be helpful to have deeper
access for example with uh gp4 you can
only get access to Generations you can't
look inside what the model activations
uh um and there's also we don't have
access to training data so basic
questions in machine learning such as
did you is your test set in your
training set we have no ability to
really know for sure so there's some
asterisks on um the ability to do you
know proper research there but I would
say it's in a much better State and then
I would say this latter category of
models uh research which involves
intervening on the the architecture or
you know data um those require
substantially more resources and I think
um the the and I I think that you need
enough resources to be able to train a
large enough model to show that um it
can do something right so one of the
phenomenon that you see with um these
models is this emerging Behavior you
need to be at a certain scale before you
start seeing anything happening
otherwise it looks like nothing is
working nothing is is is
working and and I think the reason why
this that category is important but also
under resource is that you know we talk
about all these problems with these
models with AI biases uh you know
hallucinations people can you know
jailbreak and get these models to behave
in sort of unpredictable ways I think
these are really fundamental problems
that these models have and we're not
going to be able to solve them just by
you know patching things away um and
this requ requires really addressing the
the fundamental core of like how the data interacts with the model and
data interacts with the model and understanding that in a much um deeper
understanding that in a much um deeper way which unfortunately I think we don't
way which unfortunately I think we don't have the community doesn't have as much
have the community doesn't have as much patience or for this um but but I'm sort
patience or for this um but but I'm sort of pushing for more resourcing so we can
of pushing for more resourcing so we can actually get to the bottom of things and
actually get to the bottom of things and and this is a long-term agenda to get to
and this is a long-term agenda to get to the bottom things and hopefully develop
the bottom things and hopefully develop a a stronger
a a stronger Foundation great well so um one of the
Foundation great well so um one of the things you know when when you're doing
things you know when when you're doing the kind of uh fine-tuning or retrieval
the kind of uh fine-tuning or retrieval augment and generation or other types of
augment and generation or other types of uses of of open models you even
uses of of open models you even researchers or small startups need
researchers or small startups need resources they need compute it's it's
resources they need compute it's it's something that's been kind of amazing to
something that's been kind of amazing to me that um it's sort of like the first
me that um it's sort of like the first time in my lifetime where you can put
time in my lifetime where you can put out a problem and you know Stanford
out a problem and you know Stanford Master students can come out with
Master students can come out with something working in like 10 days which
something working in like 10 days which is it seems like kind of a miracle
is it seems like kind of a miracle because everything you think is going to
because everything you think is going to take two months and it takes two years
take two months and it takes two years but here people are are getting like
but here people are are getting like working prototypes up very very quickly
working prototypes up very very quickly all over campus and across the startup
all over campus and across the startup land and there's there's a number of of
land and there's there's a number of of pieces that go into making that possible
pieces that go into making that possible um so could you talk through um what
um so could you talk through um what some of those pieces are for
some of those pieces are for applications and and in particular I
applications and and in particular I think you co recently co-founded a
think you co recently co-founded a startup called together. that kind of
startup called together. that kind of contribut
contribut some of the pieces people might use but
some of the pieces people might use but I'd love to hear about about together
I'd love to hear about about together and what it does and also what are what
and what it does and also what are what are the pieces that go into this that
are the pieces that go into this that can make this spur Innovation you know
can make this spur Innovation you know across all of everyone yeah yeah I mean
across all of everyone yeah yeah I mean to start out with I mean I completely
to start out with I mean I completely agree with you it's sort of amazing what
agree with you it's sort of amazing what one can do even in the afternoon with
one can do even in the afternoon with these these models um because
these these models um because the because I think at some level if you
the because I think at some level if you can write English or or in maybe another
can write English or or in maybe another English you can write a prompt and you
English you can write a prompt and you can get you can sort of just State what
can get you can sort of just State what you want this model to do and if you're
you want this model to do and if you're lucky enough then it can actually do it
lucky enough then it can actually do it so as a result um this technology has
so as a result um this technology has really opened up the opportunity for
really opened up the opportunity for many people who are not uh don't have
many people who are not uh don't have machine learning phds to be able to
machine learning phds to be able to interact which I think is wonderful and
interact which I think is wonderful and we're seeing a lot of um you know uptake
we're seeing a lot of um you know uptake in other areas like law and and Medicine
in other areas like law and and Medicine where this normally wouldn't have um
where this normally wouldn't have um happened um I will uh put a little bit
happened um I will uh put a little bit asteris is on the the fact that there's
asteris is on the the fact that there's a gap between guine a demo up where you
a gap between guine a demo up where you look at it it's like wow that's pretty
look at it it's like wow that's pretty cool to an actual reliable product that
cool to an actual reliable product that you can ship to many many people so that
you can ship to many many people so that last mile problem is still um
last mile problem is still um challenging it might be a little bit
challenging it might be a little bit easier but um don't get too excited and
easier but um don't get too excited and I think let me just pause you there that
I think let me just pause you there that is a really excellent point and that
is a really excellent point and that I've certainly heard that from a lot of
I've certainly heard that from a lot of of people across the industry as well
of people across the industry as well because if you're going to if if you
because if you're going to if if you want to put something in front of your
want to put something in front of your end customers it's not enough that it
end customers it's not enough that it mostly is right it has to not say
mostly is right it has to not say offensive things or do something that
offensive things or do something that damages your reputation or creates risk
damages your reputation or creates risk and so that's partly where your
and so that's partly where your evaluation stuff comes in that those
evaluation stuff comes in that those tools are an important part but so let's
tools are an important part but so let's just talk though for the moment about
just talk though for the moment about getting to the demo step even like what
getting to the demo step even like what what um what are the pieces the inputs
what um what are the pieces the inputs to that so from an application developer
to that so from an application developer perspective right so you have your
perspective right so you have your problem um the thing that is probably
problem um the thing that is probably the easiest to start is you take gp4 or
the easiest to start is you take gp4 or Claud or gemini or if your favorite
Claud or gemini or if your favorite model and you prompt it just to see get
model and you prompt it just to see get a sense of what it can do right and this
a sense of what it can do right and this is I would really call this the
is I would really call this the prototyping phase right you're ideating
prototyping phase right you're ideating and you can brainstorm while you're
and you can brainstorm while you're getting feedback from this actual system
getting feedback from this actual system which is I think sort of incredible kind
which is I think sort of incredible kind of user interaction um uh new type of us
of user interaction um uh new type of us user interaction um and then I think
user interaction um and then I think there's a but of course that's not
there's a but of course that's not enough and it's also expensive to use
enough and it's also expensive to use keep on using gp4 so one thing that's I
keep on using gp4 so one thing that's I think also important is that if you're
think also important is that if you're you know you're a business and you have
you know you're a business and you have uh you know your own data and your own
uh you know your own data and your own particular needs and businesses you have
particular needs and businesses you have to have Generations that have a certain
to have Generations that have a certain tone or style or you don't want to have
tone or style or you don't want to have it say XYZ then this is where
it say XYZ then this is where fine-tuning um you know comes in you um
fine-tuning um you know comes in you um have to be able to you know spec fine
have to be able to you know spec fine tuning basically says I have I'm more
tuning basically says I have I'm more opinionated about what my system wants
opinionated about what my system wants to do so I'm going to take some work to
to do so I'm going to take some work to really flesh out here the cases where it
really flesh out here the cases where it should do X and here's the cases where
should do X and here's the cases where it should do y and then you you
it should do y and then you you fine-tune a model and on on top of that
fine-tune a model and on on top of that um you can connect these language models
um you can connect these language models with uh different types of of tools so
with uh different types of of tools so interacting with the the the world and
interacting with the the the world and this is you mentioned rag which stands
this is you mentioned rag which stands for retrieval augmented generation the
for retrieval augmented generation the idea that you can take your do internal
idea that you can take your do internal documents um and be able to look at them
documents um and be able to look at them and have that influence your generation
and have that influence your generation or you can if you have a database or
or you can if you have a database or different apis inside uh your your
different apis inside uh your your organization you can um query those as
organization you can um query those as well and have the language model
well and have the language model essentially as a um you know a sort of a
essentially as a um you know a sort of a driver that c dispatches to different
driver that c dispatches to different services and Aggregates and then
services and Aggregates and then generates the result May summarize it at
generates the result May summarize it at the end so if you want to build
the end so if you want to build something like that what do you what
something like that what do you what what do you need to build it and and how
what do you need to build it and and how does together. fit in yeah so together
does together. fit in yeah so together was founded two years ago on the premise
was founded two years ago on the premise that we wanted to build a platform that
that we wanted to build a platform that enabled um you know developers to be
enabled um you know developers to be able to use this technology in a easy
able to use this technology in a easy and cost effective and you know
and cost effective and you know performant way as as possible um so one
performant way as as possible um so one of the services that we provide is
of the services that we provide is simply uh know inference if you have
simply uh know inference if you have let's say a llama 3 an open model that
let's say a llama 3 an open model that you want to prompt and run you can use
you want to prompt and run you can use together just like the way you would use
together just like the way you would use a GPD 4 but at a you know fraction of
a GPD 4 but at a you know fraction of the price um you can also find it's kind
the price um you can also find it's kind of like a cloud computing solution for
of like a cloud computing solution for this that could host the model yeah so
this that could host the model yeah so so I I think one yeah that's a good
so I I think one yeah that's a good point so one thing that um people might
point so one thing that um people might maybe not quite realize is that there is
maybe not quite realize is that there is the open Ai and in Googles of the world
the open Ai and in Googles of the world that have their ape services and it's
that have their ape services and it's very convenient and then on the other
very convenient and then on the other hand you might hear these open weight
hand you might hear these open weight models like llama 3 or or mistol or
models like llama 3 or or mistol or mixol where you get the weights which is
mixol where you get the weights which is great for customization but you might
great for customization but you might hear that oh okay well I have to host it
hear that oh okay well I have to host it myself and I have to deal with it but
myself and I have to deal with it but that's not not really true because
that's not not really true because there's actually um multiple services
there's actually um multiple services including together that host these for
including together that host these for you so you really have the convenience
you so you really have the convenience of an an open AI like uh framework but
of an an open AI like uh framework but um you know cheaper and you get the
um you know cheaper and you get the benefits of transparency so you might
benefits of transparency so you might you might you load up your data so like
you might you load up your data so like when I see students doing here they
when I see students doing here they might load up their data they then could
might load up their data they then could fine-tune one of those open models using
fine-tune one of those open models using the the compute and and sort of pay as
the the compute and and sort of pay as you go but then there are also software
you go but then there are also software tools that people are using as well so
tools that people are using as well so the students don't necessarily have to
the students don't necessarily have to write all the code from scratch to do
write all the code from scratch to do the fine tuning can you just say briefly
the fine tuning can you just say briefly like how that step works so if you want
like how that step works so if you want to fine tuned there's no code actually
to fine tuned there's no code actually um you upload a file that has examples
um you upload a file that has examples of input output behaviors this is an
of input output behaviors this is an instruction um and this is the response
instruction um and this is the response you put them in a file you upload it you
you put them in a file you upload it you click you know finetune and then you get
click you know finetune and then you get your fine-tune model and then you can do
your fine-tune model and then you can do inference on this just like you would be
inference on this just like you would be able to do inference on any other um
able to do inference on any other um non-custom model yeah and so if we'
non-custom model yeah and so if we' wanted to do something like that like 15
wanted to do something like that like 15 years ago you know this would have been
years ago you know this would have been like you know all the time writing the
like you know all the time writing the code for the sastic gradient descent for
code for the sastic gradient descent for the the debugging and everything else
the the debugging and everything else but basically now literally you don't
but basically now literally you don't you you you can produce your own version
you you you can produce your own version with your own documents by pushing a
with your own documents by pushing a button yeah yeah and I think it's even
button yeah yeah and I think it's even better than that because 15 years ago
better than that because 15 years ago there were no Foundation models so you
there were no Foundation models so you basically have to start from scratch you
basically have to start from scratch you would gather your data set and then the
would gather your data set and then the problem is if you're starting from
problem is if you're starting from scratch you need a fair amount of data
scratch you need a fair amount of data you might need you know thousands or
you might need you know thousands or tens of thousands of examples to even
tens of thousands of examples to even get something that
get something that was interesting to play with and that
was interesting to play with and that meant you needed to First go you know
meant you needed to First go you know get a annotation team like hire
get a annotation team like hire annotators to create create this data
annotators to create create this data and you spend you know potentially
and you spend you know potentially months training animators getting the
months training animators getting the data and then you can train your model
data and then you can train your model and then you can see what it does and
and then you can see what it does and then if it doesn't work then you have to
then if it doesn't work then you have to go back but now because of foundation
go back but now because of foundation models you don't need that much data
models you don't need that much data because much of the common things about
because much of the common things about how you know English works or um you
how you know English works or um you know simple reasoning patterns the model
know simple reasoning patterns the model already knows about so all you need to
already knows about so all you need to do is supply information about your
do is supply information about your specific needs
specific needs and so you might be able to get away
and so you might be able to get away with you know hundreds of examples so so
with you know hundreds of examples so so the iteration Cycles are much shorter
the iteration Cycles are much shorter and then when you do prompting it's
and then when you do prompting it's often five examples or zero examples so
often five examples or zero examples so so I think that iteration cycle is much
so I think that iteration cycle is much much shorter great so actually we're
much shorter great so actually we're just running short on time but I'm going
just running short on time but I'm going to try to squeeze in my last favorite
to try to squeeze in my last favorite question um we have a lot of debates
question um we have a lot of debates about the risks and benefits of open and
about the risks and benefits of open and closed as an economist I'm particularly
closed as an economist I'm particularly interested in how you think about
interested in how you think about incremental risk of these open weight
incremental risk of these open weight models and could you just say briefly
models and could you just say briefly how you think about that yeah so one of
how you think about that yeah so one of the things that you might hear is that
the things that you might hear is that open weight and I use the word open
open weight and I use the word open weight instead of Open Source
weight instead of Open Source deliberately to be more specific is
deliberately to be more specific is about these model is that well people
about these model is that well people can use them for for harm you can f
can use them for for harm you can f tunea you can turn off the safety and
tunea you can turn off the safety and you can spread disinformation and and so
you can spread disinformation and and so on and I think it's really important to
on and I think it's really important to frame this conversation in terms of the
frame this conversation in terms of the whole picture which is first of all that
whole picture which is first of all that this model is a tool that is is uh used
this model is a tool that is is uh used um to potentially generate some uh some
um to potentially generate some uh some you know harmful content but then this
you know harmful content but then this harmful content also needs to be
harmful content also needs to be disseminated or in the case of let's say
disseminated or in the case of let's say biorisk you need to also you know
biorisk you need to also you know synthesize the you know the proteins and
synthesize the you know the proteins and so you need to think about the whole you
so you need to think about the whole you know picture and where you want to you
know picture and where you want to you know gate um another thing is that
know gate um another thing is that um the just because a model generates
um the just because a model generates let's say you know how do I make a bomb
let's say you know how do I make a bomb okay here's how you make a bomb um
okay here's how you make a bomb um doesn't mean that we should immediately
doesn't mean that we should immediately just shut it off because you can also
just shut it off because you can also you know find things like this on the
you know find things like this on the internet um you can find a lot of things
internet um you can find a lot of things about um you know um how to make
about um you know um how to make Chemical or bioweapons on you know on
Chemical or bioweapons on you know on Wikipedia and so there's a lot of other
Wikipedia and so there's a lot of other uh things so one thing that we've been
uh things so one thing that we've been trying to get people to uh think about
trying to get people to uh think about is the marginal risk of a particular
is the marginal risk of a particular technology with respect to what exists
technology with respect to what exists and and and I think only then can you
and and and I think only then can you make a meaningful calculation of you
make a meaningful calculation of you know whether introducing this um is
know whether introducing this um is going to what is the risk there um
going to what is the risk there um compared to the marginal benefits of
compared to the marginal benefits of having um these models and I I think the
having um these models and I I think the you know right now I don't see you know
you know right now I don't see you know substantial evidence that the marginal
substantial evidence that the marginal risk is is high and open AI actually
risk is is high and open AI actually also did you know certain studies the
also did you know certain studies the current models just aren't that um
current models just aren't that um impressive when it terms comes to like
impressive when it terms comes to like enabling much a broader uh surface area
enabling much a broader uh surface area of tax but this goes back to the point
of tax but this goes back to the point about evaluation we need to much more
about evaluation we need to much more rigorously evaluate the risks before we
rigorously evaluate the risks before we jump to conclusions and get our our
jump to conclusions and get our our hells and speculate about
hells and speculate about what you know could potentially happen
what you know could potentially happen well you've made every um intro
well you've made every um intro economics professor's heart sing with
economics professor's heart sing with marginal benefits and marginal cost so I
marginal benefits and marginal cost so I think that's a great place to close
think that's a great place to close thank you so much for being here thank
thank you so much for being here thank you great thank
you so next I'd like to welcome to the stage Andrew ing please come on up
stage Andrew ing please come on up Andrew well um and I just to remind
Andrew well um and I just to remind everyone if you're just tuning in all of
everyone if you're just tuning in all of the speaker bios are online um and so
the speaker bios are online um and so we're will'll be short with
we're will'll be short with introductions Andrew I think just to get
introductions Andrew I think just to get started you have a few slides to help
started you have a few slides to help frame some of your thoughts about open
frame some of your thoughts about open models open source and the role of
models open source and the role of competition and AI so I would love it if
competition and AI so I would love it if you could just kick us off with uh with
you could just kick us off with uh with with a few comments awesome thank thank
with a few comments awesome thank thank you Susan um it's good to hear pressy
you Susan um it's good to hear pressy good to see everyone so um I just Rec
good to see everyone so um I just Rec siiz and I'll be quick you know I I
siiz and I'll be quick you know I I think one of the hard things to
think one of the hard things to understand about AI is that is a general
understand about AI is that is a general purpose technology meaning is not useful
purpose technology meaning is not useful just for a single thing but it's useful
just for a single thing but it's useful for a very large multitude of tasks um I
for a very large multitude of tasks um I think there are some business
think there are some business representatives in the audience I think
representatives in the audience I think for businesses this is exciting because
for businesses this is exciting because it gives us the opportunity to identify
it gives us the opportunity to identify and build tons of applications in
and build tons of applications in healthcare Financial Services education
healthcare Financial Services education Logistics and on and on and on um and in
Logistics and on and on and on um and in terms of the registry landscape I think
terms of the registry landscape I think my my my key messages are one I think
my my my key messages are one I think it's important to distinguish between
it's important to distinguish between technology and applications um and
technology and applications um and second to promote competition and
second to promote competition and Innovation I feel like we should lean
Innovation I feel like we should lean into regulating applications rather than
into regulating applications rather than Technologies and and also um really
Technologies and and also um really promotes open source so um you know this
promotes open source so um you know this is what I think of as an AI stack
is what I think of as an AI stack different people have slightly different
different people have slightly different versions of this but I think the lowest
versions of this but I think the lowest level is a semiconductor level on top of
level is a semiconductor level on top of that are the clouds both of these are
that are the clouds both of these are relatively Capital intensive thus maybe
relatively Capital intensive thus maybe you know slightly concentrated and then
you know slightly concentrated and then the new layer has been the AI technology
the new layer has been the AI technology tooling layer the these provide
tooling layer the these provide Foundation models um um en large
Foundation models um um en large language models and so on and what I
language models and so on and what I seen with most waves of um Innovation is
seen with most waves of um Innovation is the media traditional media and social
the media traditional media and social media tends to focus on technology layer
media tends to focus on technology layer this is fun to talk about it's fun to
this is fun to talk about it's fun to talk about what's opening eye doing you
talk about what's opening eye doing you know and so on but I think that um
know and so on but I think that um there's an even bigger set of
there's an even bigger set of opportunities than this which is the
opportunities than this which is the application layer because for the
application layer because for the technology tooling layer and the layers
technology tooling layer and the layers below to be successful almost by
below to be successful almost by definition we need to applications on
definition we need to applications on top of them to generate even more
top of them to generate even more Revenue so they can afford to pay the
Revenue so they can afford to pay the technology layers underneath um and I
technology layers underneath um and I think in order to prevent choke points I
think in order to prevent choke points I think there's a lot of concern about
think there's a lot of concern about choke points AI technology layer we only
choke points AI technology layer we only a small number of companies have access
a small number of companies have access to Foundation models I think open source
to Foundation models I think open source is one of our best tools for promoting
is one of our best tools for promoting Innovation and preventing a new CH point
Innovation and preventing a new CH point from arising at AI technology layer and
from arising at AI technology layer and candidly I've been I would love if um
candidly I've been I would love if um governments all around the world can
governments all around the world can promote competition candidly I'd be
promote competition candidly I'd be quite happy if governments all around
quite happy if governments all around the world just don't stifle competition
the world just don't stifle competition but having having governments promote
but having having governments promote competition as opposed to merely avoid
competition as opposed to merely avoid stifling it seems like a a dream at this
stifling it seems like a a dream at this point in time but let me dig into that a
point in time but let me dig into that a little bit more um and and this is what
little bit more um and and this is what I mean by um technology versus
I mean by um technology versus applications and this is an important
applications and this is an important concept for all in government I hope I
concept for all in government I hope I hope you remember this part so I'm going
hope you remember this part so I'm going to talk about technology later
to talk about technology later as was an application layer so um an
as was an application layer so um an example of a technology might be an
example of a technology might be an electric motor and an electric motor
electric motor and an electric motor which can be used in lots of
which can be used in lots of applications could be put into a blender
applications could be put into a blender electrical vehicle a dial machine um or
electrical vehicle a dial machine um or also a guided bomb um and so I think of
also a guided bomb um and so I think of Technology as tools that can be applied
Technology as tools that can be applied in many different ways to solve many
in many different ways to solve many different problems like electric motor
different problems like electric motor um and then applications are specific
um and then applications are specific implementations designed to meet a
implementations designed to meet a particular customer need and this could
particular customer need and this could be a very beneficial need like a
be a very beneficial need like a dialysis machine or one that we don't
dialysis machine or one that we don't want adversaries to have access to like
want adversaries to have access to like a guided bomb um and I feel like if you
a guided bomb um and I feel like if you ask me to make electric motors safe I
ask me to make electric motors safe I don't know how to do that right the only
don't know how to do that right the only way to make an electric motor save is to
way to make an electric motor save is to make it so tiny that's basically useless
make it so tiny that's basically useless to everyone but and indeed blenders can
to everyone but and indeed blenders can be dangerous this is why we regulate
be dangerous this is why we regulate blenders we don't want blenders to you
blenders we don't want blenders to you know be dangerous in people's homes and
know be dangerous in people's homes and and damage and hurt children's Limbs and
and damage and hurt children's Limbs and so on um electric vehicles let's make
so on um electric vehicles let's make sure electric vehicles are safe and D
sure electric vehicles are safe and D machines boy that's a medical device
machines boy that's a medical device let's make sure to regulate that and
let's make sure to regulate that and guide the bombs yes we absolutely need
guide the bombs yes we absolutely need to regulate production of weapons and
to regulate production of weapons and Munitions um but if you try to say an
Munitions um but if you try to say an electric motor could be used by build a
electric motor could be used by build a bomb so let's make the electric motor
bomb so let's make the electric motor manufacturer guaranteed their electric
manufacturer guaranteed their electric motor will never be used by anyone to
motor will never be used by anyone to make a guide the bomb then the only way
make a guide the bomb then the only way is for the electric motor manufacturer
is for the electric motor manufacturer to shut down the business and that means
to shut down the business and that means you lose the blender EV and the DAT
you lose the blender EV and the DAT machine as well and this is what we're
machine as well and this is what we're seeing in in ai ai models lar language
seeing in in ai ai models lar language models Foundation models are general
models Foundation models are general purpose Technologies uh we can be use to
purpose Technologies uh we can be use to build a medical device to give medical
build a medical device to give medical advice um or to rank someone's Social
advice um or to rank someone's Social Media fee for good purposes or bad
Media fee for good purposes or bad purposes um it can be used to build a
purposes um it can be used to build a chat bot um it can be used to generate
chat bot um it can be used to generate political defects there are tons of
political defects there are tons of beneficial as well as a small number of
beneficial as well as a small number of harmful applications and I think risks
harmful applications and I think risks are a function of of the application not
are a function of of the application not to technology um and when you reg if
to technology um and when you reg if someone tries to regulate the general
someone tries to regulate the general purpose technology like electric motors
purpose technology like electric motors you know that tends to just stifle
you know that tends to just stifle Innovation because you're cutting off
Innovation because you're cutting off the Innovation at the root and shutting
the Innovation at the root and shutting down a huge range of beneficial
down a huge range of beneficial applications as well as maybe a small
applications as well as maybe a small number of harmful ones um so I feel like
number of harmful ones um so I feel like what we do need is um I do want medical
what we do need is um I do want medical devices to be safe I do want electrical
devices to be safe I do want electrical vehicles to be safe and what I'm seeing
vehicles to be safe and what I'm seeing unfortunately is many governments are
unfortunately is many governments are saying boy it's too hard to look at the
saying boy it's too hard to look at the applications that regulate them so let's
applications that regulate them so let's do the easy thing which is you know
do the easy thing which is you know let's just regulate the AI technology
let's just regulate the AI technology but that's a bad move I think we need to
but that's a bad move I think we need to do the harder but the right thing which
do the harder but the right thing which is look at the actual applications which
is look at the actual applications which where the risk are and pass appropriate
where the risk are and pass appropriate regulations so to to to protect um to
regulations so to to to protect um to protect citizens um and I've been really
protect citizens um and I've been really surprised at the intensity of the
surprised at the intensity of the lobbying um to shut down open source I
lobbying um to shut down open source I think maybe predictable there are some
think maybe predictable there are some you know companies that have invested
you know companies that have invested billions of dollars into training large
billions of dollars into training large Foundation models and when you've
Foundation models and when you've invested billions of dollars is really
invested billions of dollars is really inconvenient when some other company
inconvenient when some other company releases an open SCE or open weights
releases an open SCE or open weights model that severely degrades the value
model that severely degrades the value of that massive investment you made to
of that massive investment you made to train propriety model and I get it it's
train propriety model and I get it it's really annoying when someone open
really annoying when someone open sources something um but what has
sources something um but what has happened over the last year was really
happened over the last year was really intense lobbying under the guise of AI
intense lobbying under the guise of AI safety and you hear very smart people
safety and you hear very smart people say say things like of course I support
say say things like of course I support open source but don't you want open
open source but don't you want open source to be safe and and that's used as
source to be safe and and that's used as an excuse to put in place very
an excuse to put in place very burdensome licensing requirements that
burdensome licensing requirements that will stifle anyone's ability you know
will stifle anyone's ability you know that they'll severely stifle many team's
that they'll severely stifle many team's ability to release free open weights of
ability to release free open weights of Open Source software on the internet um
Open Source software on the internet um and you know in and and candidly I I
and you know in and and candidly I I think that um open source today open
think that um open source today open weight is open source is a key part of
weight is open source is a key part of the AI supply chain um I feel like the
the AI supply chain um I feel like the the one of the latest lines of attacks
the one of the latest lines of attacks on open source has been um to prevent
on open source has been um to prevent you know near peer adversaries from from
you know near peer adversaries from from getting access to it but candidly I I
getting access to it but candidly I I actually on on that ground is actually
actually on on that ground is actually worry I think um uh you know someday
worry I think um uh you know someday right some kid in some developing
right some kid in some developing economy will going to large out of
economy will going to large out of foundation model also large language
foundation model also large language model you know is democracy important
model you know is democracy important what's the role of Free Press is
what's the role of Free Press is independent Judiciary important and when
independent Judiciary important and when kids like that ask questions I would
kids like that ask questions I would quite like the lar language model to
quite like the lar language model to reflect the values of liberal democ
reflect the values of liberal democ racies I'm biased I like democracy and I
racies I'm biased I like democracy and I think that if democracies inflict this
think that if democracies inflict this self-inflicted wound of shutting down
self-inflicted wound of shutting down our open source then other countries
our open source then other countries will step into this um to the to the
will step into this um to the to the detriment of of democracy so I hope that
detriment of of democracy so I hope that one of the best ways to promote
one of the best ways to promote Innovation and competition we to protect
Innovation and competition we to protect open Souls I hope all of us will do that
open Souls I hope all of us will do that thank you great
thanks well that was that was great thank you so much and you've you've
thank you so much and you've you've really laid out a landscape of a lot of
really laid out a landscape of a lot of the things that people are worried about
the things that people are worried about and talking about um so when when you
and talking about um so when when you think about maybe I'll just pick up
think about maybe I'll just pick up where you left off um you mentioned that
where you left off um you mentioned that you thought other countries might do
you thought other countries might do something I think that relates a little
something I think that relates a little bit to the economist language of
bit to the economist language of incremental risk and incremental benefit
incremental risk and incremental benefit that we were just talking about with
that we were just talking about with Percy um can you talk through a little
Percy um can you talk through a little bit uh you know counterfactual world I
bit uh you know counterfactual world I guess you know what what happens without
guess you know what what happens without the some of these open weight models and
the some of these open weight models and and can you just flush out a little bit
and can you just flush out a little bit more what you might see Happening by Bad
more what you might see Happening by Bad actors or other countries or um and what
actors or other countries or um and what might happen in in our ecosystem yeah so
might happen in in our ecosystem yeah so let's see um I think that choke points
let's see um I think that choke points will arise um I I I see two
will arise um I I I see two possibilities both bad I think one would
possibilities both bad I think one would be if everyone is beholden to a very
be if everyone is beholden to a very very small number of foundation model
very small number of foundation model providers um uh that's creating a new CH
providers um uh that's creating a new CH Point um if we don't have access to open
Point um if we don't have access to open source I think Academia will suffer a
source I think Academia will suffer a lot of the work that pery talked about
lot of the work that pery talked about and that frankly many universities all
and that frankly many universities all around the world are doing I was
around the world are doing I was actually just in career couple weeks ago
actually just in career couple weeks ago at the government meeting prime minister
at the government meeting prime minister was there and I think that also you know
was there and I think that also you know I was hearing about Korean academics
I was hearing about Korean academics saying please protect open source
saying please protect open source because academics really need to to to
because academics really need to to to monitor uh uh to to to to do their work
monitor uh uh to to to to do their work so I think Academia would have a much
so I think Academia would have a much harder time and then a lot of innovation
harder time and then a lot of innovation just won't happen because I see so many
just won't happen because I see so many startups all around the world not just
startups all around the world not just in s all around the world um building on
in s all around the world um building on top of the open models and doing things
top of the open models and doing things that are simply not possible on top of
that are simply not possible on top of the Clos models maybe one example um
the Clos models maybe one example um llama llama 3 was released as open
llama llama 3 was released as open weights um and so you know llama 3 turns
weights um and so you know llama 3 turns out has a relatively short uh uh input
out has a relatively short uh uh input context length they can only accept a
context length they can only accept a relatively short input and so some team
relatively short input and so some team you know extended that to accept a much
you know extended that to accept a much longer input expanded context length and
longer input expanded context length and if lmer 3 had been released as an API
if lmer 3 had been released as an API only thing this work would have been
only thing this work would have been impossible um and so open source Builds
impossible um and so open source Builds on top of itself as a very rich
on top of itself as a very rich ecosystem that I think some companies
ecosystem that I think some companies through really intense lobing efforts
through really intense lobing efforts are really eager they try to shut off
are really eager they try to shut off and by the way and let me say so some
and by the way and let me say so some large companies have been saying stuff
large companies have been saying stuff like oh we totally support open source
like oh we totally support open source just not of the most advanced models we
just not of the most advanced models we got to keep that away from our
got to keep that away from our adversaries it's been over the last year
adversaries it's been over the last year it's been fascinating I've not spent a
it's been fascinating I've not spent a lot of time in DC I tend to stay stay
lot of time in DC I tend to stay stay here and just write software it's been
here and just write software it's been fascinating to see how the adversaries
fascinating to see how the adversaries of Open Source have shifted their
of Open Source have shifted their arguments multiple times um it was
arguments multiple times um it was initially AI could cuse all you know AI
initially AI could cuse all you know AI is like nuclear weapons analogy that
is like nuclear weapons analogy that makes no sense AI is intelligent nuclear
makes no sense AI is intelligent nuclear weapons P sub CID the strange analogy
weapons P sub CID the strange analogy when that argument lost credibility it
when that argument lost credibility it was then oh AI could create bioweapons
was then oh AI could create bioweapons but then um open and Rand published
but then um open and Rand published reports showing that the incremental
reports showing that the incremental risk um is very small uh so yes of
risk um is very small uh so yes of course AI could be used to create B
course AI could be used to create B weapons I'll just say that so can an
weapons I'll just say that so can an Excel spreadsheet um I mean Excel
Excel spreadsheet um I mean Excel spreadsheets I you can track experiments
spreadsheets I you can track experiments I mean Excel is really dangerous let's
I mean Excel is really dangerous let's regulate that so but AI so the evidence
regulate that so but AI so the evidence that AI helps more there is no evidence
that AI helps more there is no evidence as far as I'm aware that AI helps more
as far as I'm aware that AI helps more than you know an Excel spreadsheet or
than you know an Excel spreadsheet or web search for for creating bioweapons
web search for for creating bioweapons and then people have studied this so
and then people have studied this so when that credibility then it became oh
when that credibility then it became oh you know our adversaries China
you know our adversaries China specifically like got to got to keep
specifically like got to got to keep maybe Russia I don't know right so it's
maybe Russia I don't know right so it's been fascinating watching the shifting
been fascinating watching the shifting arguments of the of the open open source
arguments of the of the open open source and I've also been fascinated it's been
and I've also been fascinated it's been amazing to watch how effective some of
amazing to watch how effective some of these companies have been at convincing
these companies have been at convincing governments to regulate very clearly
governments to regulate very clearly against uh uh their own national
against uh uh their own national interests this this more mainly outside
interests this this more mainly outside the United States
the United States but thank you so much um well and so you
but thank you so much um well and so you know one
know one we're so fortunate to have you here from
we're so fortunate to have you here from so many different perspectives you have
so many different perspectives you have because of course you did foundational
because of course you did foundational research as a computer science professor
research as a computer science professor at Stanford um you cound co-founded
at Stanford um you cound co-founded corsera uh so anybody wants to learn
corsera uh so anybody wants to learn about this um you can watch his videos
about this um you can watch his videos um and you're now an also an investor
um and you're now an also an investor and an entrepreneur um can I turn this
and an entrepreneur um can I turn this around a little bit to think about the
around a little bit to think about the investor perspective if you are
investor perspective if you are investing in applications um how do you
investing in applications um how do you think about possible choke points and
think about possible choke points and risks that your Investments might um
risks that your Investments might um might experience yeah I think the the
might experience yeah I think the the the choke point that we all love to
the choke point that we all love to avoid a lot of it is at the um
avoid a lot of it is at the um Foundation model layer uh uh and I think
Foundation model layer uh uh and I think they'll be hard for um companies to
they'll be hard for um companies to establish a CH point there if we promote
establish a CH point there if we promote and invest in open source there's a
and invest in open source there's a government that does having
government that does having conversations with the where the
conversations with the where the administration was considering for their
administration was considering for their own procurement process to really have a
own procurement process to really have a bias fail colting open source based
bias fail colting open source based Solutions I don't know if they'll do it
Solutions I don't know if they'll do it but you know one government not the US
but you know one government not the US that that was s do was was considering
that that was s do was was considering this I think that type of move to
this I think that type of move to promote open source would very good um
promote open source would very good um there's one other layer that may to
there's one other layer that may to comment on that's kind of emerging which
comment on that's kind of emerging which is on top of the foundation model layer
is on top of the foundation model layer there's also another layer that's
there's also another layer that's emerging called an orchestration layer
emerging called an orchestration layer and what that means is um uh often um
and what that means is um uh often um when you're Building Systems you don't
when you're Building Systems you don't just call an L large language model once
just call an L large language model once you call it a bunch of times and you
you call it a bunch of times and you call it once as output goes into the
call it once as output goes into the second prom goes to the third prom you
second prom goes to the third prom you pull some data toloy a rag and so on so
pull some data toloy a rag and so on so that orchestration layer um right now is
that orchestration layer um right now is very Dynamic very competitive so there
very Dynamic very competitive so there is no CH point there at this moment but
is no CH point there at this moment but you know I definitely see uh more
you know I definitely see uh more startups at this point prior trying to
startups at this point prior trying to trying to become a dominant player but
trying to become a dominant player but that's very good right you you want you
that's very good right you you want you know um uh entrepreneurs with low
know um uh entrepreneurs with low resources aspiring to build the next
resources aspiring to build the next Google or the next you know mea or
Google or the next you know mea or something so I think it's very intense
something so I think it's very intense competition uh but I I could see a
competition uh but I I could see a future many many years from now if if
future many many years from now if if there's a new player that that that wins
there's a new player that that that wins so could you I'd love to hear you
so could you I'd love to hear you elaborate a little bit about that
elaborate a little bit about that because I know that you've talked about
because I know that you've talked about like AI agents as the next big wave in
like AI agents as the next big wave in applications and I think that kind of
applications and I think that kind of picks up on this orchestration layer and
picks up on this orchestration layer and I think you know most people in the
I think you know most people in the audience have seen the images they've
audience have seen the images they've seen the text they've played with the
seen the text they've played with the text but um it's these it's it's a
text but um it's these it's it's a little bit harder for the ordinary
little bit harder for the ordinary person to Intuit it like what's going on
person to Intuit it like what's going on behind the scenes and also frankly
behind the scenes and also frankly what's possible when you have multiple
what's possible when you have multiple agents kind of interacting together so
agents kind of interacting together so I'm wondering if you could talk us
I'm wondering if you could talk us through what makes you excited about
through what makes you excited about that or maybe a use case or an example
that or maybe a use case or an example if you have one sure yeah so I think one
if you have one sure yeah so I think one of the most exciting Trends in Tech
of the most exciting Trends in Tech right now is AI agents or agentic Ai
right now is AI agents or agentic Ai workflows and and lcy here's what I mean
workflows and and lcy here's what I mean um many of us are used to going into a
um many of us are used to going into a large langage model and prompting it uh
large langage model and prompting it uh to have it write a response right and
to have it write a response right and that's kind of like imagine you go to
that's kind of like imagine you go to someone and say I like you to write a
someone and say I like you to write a topic i' like you to write an essay on a
topic i' like you to write an essay on a certain topic but I need you to type
certain topic but I need you to type from start to finish without ever
from start to finish without ever hitting backspace or without ever going
hitting backspace or without ever going back to edit your work um or if you ask
back to edit your work um or if you ask AI to write code is like going to
AI to write code is like going to developer and saying I need you to just
developer and saying I need you to just type out a computer program and just
type out a computer program and just type it out from start to finish and
type it out from start to finish and then I want it to work and it turns out
then I want it to work and it turns out that you know despite the difficulty of
that you know despite the difficulty of this right like I can't write like that
this right like I can't write like that I can't cope like that AI actually does
I can't cope like that AI actually does surprisingly well at this task
surprisingly well at this task but it turns out that if you use an
but it turns out that if you use an agentic AI workflow in which you can ask
agentic AI workflow in which you can ask the AI write an outline and then think
the AI write an outline and then think about your outline do you need to do any
about your outline do you need to do any web research if so what do you want to
web research if so what do you want to type into the search bar a web search
type into the search bar a web search engine then go do a bunch of you know
engine then go do a bunch of you know Google or Bing or duck go searches or
Google or Bing or duck go searches or whatever and then get the results and
whatever and then get the results and then write the first draft and then read
then write the first draft and then read your own draft and think about where you
your own draft and think about where you want to improve it and iterate over your
want to improve it and iterate over your own work over and over so that's called
own work over and over so that's called an agent take workflow and it turns out
an agent take workflow and it turns out that this does way better than if you
that this does way better than if you force an AI system to write from start
force an AI system to write from start to finish um one the exciting things I
to finish um one the exciting things I found is that uh my team collected data
found is that uh my team collected data the kind of third party data uh sorry we
the kind of third party data uh sorry we we did a survey of uh uh AI uh large
we did a survey of uh uh AI uh large language models and agents applied to
language models and agents applied to writing code this a standard Benchmark
writing code this a standard Benchmark from open AI called human eval that
from open AI called human eval that measures how well AI systems right code
measures how well AI systems right code and it turns out that chat GPT or GPT
and it turns out that chat GPT or GPT 3.5 um had 48% right in term on on this
3.5 um had 48% right in term on on this Benchmark GPT 4 had I think I want to
Benchmark GPT 4 had I think I want to say 67% 67 or 68% um so gbd4 was a huge
say 67% 67 or 68% um so gbd4 was a huge improvement from 48 to 67 68% but it
improvement from 48 to 67 68% but it turns out the improvement from 3.5 gbd
turns out the improvement from 3.5 gbd 3.5 to gbd4 is dwarfed by the
3.5 to gbd4 is dwarfed by the improvement from if you wrap an agentic
improvement from if you wrap an agentic workflow around this and even using qbd
workflow around this and even using qbd 3.5 on this Benchmark can get kind of up
3.5 on this Benchmark can get kind of up to 95% accuracy um uh by having the AI
to 95% accuracy um uh by having the AI system write code and it run some tests
system write code and it run some tests fix his own code and just iterate over
fix his own code and just iterate over and over to keep on improving his code
and over to keep on improving his code until it works so I think for those of
until it works so I think for those of you interested about applications I
you interested about applications I think agentic AI um is going to be a
think agentic AI um is going to be a very large Trend that that I think we're
very large Trend that that I think we're actually seeing this in the field
actually seeing this in the field already my team AI fund we we we built
already my team AI fund we we we built startups we built one startup per month
startups we built one startup per month but for many applications um you know in
but for many applications um you know in legal work and Healthcare and tons of
legal work and Healthcare and tons of other applications we seeing these
other applications we seeing these agentic workflows um deliver really
agentic workflows um deliver really fantastic results that we couldn't with
fantastic results that we couldn't with other ways and so what else could it do
other ways and so what else could it do besides like search the web which again
besides like search the web which again I think we we we have some intuition for
I think we we we have some intuition for that but if you were building a custom
that but if you were building a custom one of those what are other kinds of
one of those what are other kinds of calls that that this type of agentic AI
calls that that this type of agentic AI could could make yeah so maybe some some
could could make yeah so maybe some some techniques I find exciting the gener ref
techniques I find exciting the gener ref one is reflection where we ask you to
one is reflection where we ask you to read over your own output and just see
read over your own output and just see if you have any ways to improve it that
if you have any ways to improve it that surprisingly helps quite a bit um to use
surprisingly helps quite a bit um to use where you can search the web or it can
where you can search the web or it can um search different information sources
um search different information sources or access your calendar with permission
or access your calendar with permission if that's what you needed to do or
if that's what you needed to do or sometimes it can also um make an API
sometimes it can also um make an API call to take an action so you know
call to take an action so you know people are building agents uh customer
people are building agents uh customer service agents for example they can make
service agents for example they can make an API call to retrieve information or
an API call to retrieve information or even make an API call to maybe issue a
even make an API call to maybe issue a refund you know um uh and then I think
refund you know um uh and then I think uh M uh and then agents can increasingly
uh M uh and then agents can increasingly plan out complex sequence of of steps
plan out complex sequence of of steps where we can say to do this time I need
where we can say to do this time I need to First do research then write an
to First do research then write an outline then fact check and then write
outline then fact check and then write the final jop or whatever it can decide
the final jop or whatever it can decide for itself what sequence of actions to
for itself what sequence of actions to take and then lost the multi-agent
take and then lost the multi-agent collaborations has been a fascinating
collaborations has been a fascinating Trend where it turns out that if you
Trend where it turns out that if you have two AI systems and you say you are
have two AI systems and you say you are the writer you're the researcher you're
the writer you're the researcher you're the fact Checker and then build three
the fact Checker and then build three agents uh and each agent is really built
agents uh and each agent is really built by prompting an OM you tell the dear
by prompting an OM you tell the dear your role is you are now a fact Checker
your role is you are now a fact Checker so go fact check these please well dear
so go fact check these please well dear you're now a writer so go write a clear
you're now a writer so go write a clear article uh then if you have multi-agents
article uh then if you have multi-agents interact that also uh really tends to
interact that also uh really tends to drive you know improved results so and
drive you know improved results so and it's a very nice and very exciting space
it's a very nice and very exciting space right frankly six years ago this stuff
right frankly six years ago this stuff it like kind of wasn't working six a
it like kind of wasn't working six a year ago it was demo whereare it was
year ago it was demo whereare it was great demos really couldn't get to work
great demos really couldn't get to work at all but now I feel like I'm seeing
at all but now I feel like I'm seeing month over month improvements on agentic
month over month improvements on agentic AI workflows um so this is actually a
AI workflows um so this is actually a very exciting space and and I think um
very exciting space and and I think um uh it does to hyper competitive right
uh it does to hyper competitive right now at the agentic layer and also the uh
now at the agentic layer and also the uh Innovation all the applications how do
Innovation all the applications how do you take this and apply this to you know
you take this and apply this to you know many different domains that's a very ex
many different domains that's a very ex that that layer of innovation feels like
that that layer of innovation feels like is exploding as well very exciting time
is exploding as well very exciting time to be in so one of the things that was
to be in so one of the things that was interesting about the the first large
interesting about the the first large language models is that they were mostly
language models is that they were mostly using open data that they scraped and
using open data that they scraped and later this afternoon we'll we're going
later this afternoon we'll we're going to come back and talk more about the
to come back and talk more about the scraping issue but they they were using
scraping issue but they they were using kind of common data and not as much like
kind of common data and not as much like proprietary data not as much like user
proprietary data not as much like user cck data or things like that but um you
cck data or things like that but um you know in some of these applications
know in some of these applications you're talking about and also I guess
you're talking about and also I guess previously the foundation models would
previously the foundation models would would kind of you know interact with
would kind of you know interact with lots of different things but they
lots of different things but they weren't they didn't they weren't like
weren't they didn't they weren't like directly connecting to other software
directly connecting to other software products so the agentic products you're
products so the agentic products you're talking about you mentioned they might
talking about you mentioned they might read your email um we later this
read your email um we later this afternoon we we'll we're going to talk
afternoon we we'll we're going to talk about um you know using like crms
about um you know using like crms talking to crms so some of these layers
talking to crms so some of these layers might now need to interoperate with
might now need to interoperate with other software products and they might
other software products and they might also possibly you know make more use of
also possibly you know make more use of user feedback data depending on what
user feedback data depending on what they're interoperating with so I'm sort
they're interoperating with so I'm sort of curious if you could do a little bit
of curious if you could do a little bit of like forward looking where you things
of like forward looking where you things are think things are going um what are
are think things are going um what are what are some of the inputs and
what are some of the inputs and bottlenecks that might come up in the
bottlenecks that might come up in the future implementations of these as they
future implementations of these as they start interoperating with more systems
start interoperating with more systems you know it's really interesting I I I
you know it's really interesting I I I really appreciate I think Susan has
really appreciate I think Susan has always been a very long-term forward
always been a very long-term forward thinker I have to say I find myself
thinker I have to say I find myself inspired by that view and honestly s not
inspired by that view and honestly s not I'll get to your question honestly what
I'll get to your question honestly what I'm really worried about is Les the
I'm really worried about is Les the longterm future is the awful regulations
longterm future is the awful regulations I'm seeing right now um that that just
I'm seeing right now um that that just as an example California s SP 1047 I
as an example California s SP 1047 I think it' be awful for American
think it' be awful for American innovation and for California inov I
innovation and for California inov I can't believe that California home of
can't believe that California home of such an Innovative place with so many
such an Innovative place with so many inventions is contemplating such
inventions is contemplating such ridiculous regulation is equivalent of
ridiculous regulation is equivalent of saying if your electric motor is used by
saying if your electric motor is used by anyone for an inferious purpose you
anyone for an inferious purpose you could be liable you know it's just awful
could be liable you know it's just awful and I don't see how the state of
and I don't see how the state of California could could seriously
California could could seriously contemplate something like that um sorry
contemplate something like that um sorry but to take a longer longer looking view
but to take a longer longer looking view you know I I I feel like um uh uh I
you know I I I feel like um uh uh I think that one interesting Dynamic would
think that one interesting Dynamic would be where will incumbents win and where
be where will incumbents win and where will new entrance have a shot so I think
will new entrance have a shot so I think um in the case of electronic health
um in the case of electronic health records you know I think we know that
records you know I think we know that epic as electronic health record system
epic as electronic health record system right has massive Market power immense
right has massive Market power immense concentration um so where will
concentration um so where will incumbents like that be able to lock
incumbents like that be able to lock down options to to to block innovators
down options to to to block innovators um I think I think those are those are
um I think I think those are those are issues worth thinking about
issues worth thinking about um yeah I don't know I I I think sorry
um yeah I don't know I I I think sorry yeah I I maybe part of me feels like for
yeah I I maybe part of me feels like for a lot of applications it feels like a
a lot of applications it feels like a relatively Dynamic landscape still um
relatively Dynamic landscape still um but I think access to data is going to
but I think access to data is going to be a thing um and I think I I I worry a
be a thing um and I think I I I worry a little bit about the um some of the
little bit about the um some of the emerging propriety deuse to access data
emerging propriety deuse to access data um one one of the debates is what should
um one one of the debates is what should fair use be in the era of of AI is it
fair use be in the era of of AI is it okay for a company to read something on
okay for a company to read something on the open internet and use it to train
the open internet and use it to train the AI model U my personal preference I
the AI model U my personal preference I believe the answer should be yes but
believe the answer should be yes but that's my kind of personal opinion um
that's my kind of personal opinion um and I realized that so I post on social
and I realized that so I post on social media so you uh New York Times suit open
media so you uh New York Times suit open AI uh and Microsoft I think and I found
AI uh and Microsoft I think and I found myself more on open AI side than on than
myself more on open AI side than on than on the New York Times side in that
on the New York Times side in that matter because um and and I think I
matter because um and and I think I learned an interesting lesson when I
learned an interesting lesson when I post on social media second said you
post on social media second said you know I as a human am allowed to read
know I as a human am allowed to read articles on the open internet and learn
articles on the open internet and learn from it I would like AI to be able to do
from it I would like AI to be able to do so too and I got a bit of you know hate
so too and I got a bit of you know hate SL criticism um including from some uh
SL criticism um including from some uh Sanford faculty um uh saying that I had
Sanford faculty um uh saying that I had no reason for making an equivalence
no reason for making an equivalence between humans and AI right because the
between humans and AI right because the point was yes humans can read stuff on
point was yes humans can read stuff on the internet but just because a human
the internet but just because a human can do it why is there the reason to let
can do it why is there the reason to let AI do it and I realized that
AI do it and I realized that I think there two views to AI um I tend
I think there two views to AI um I tend to view AI as an extension of myself I I
to view AI as an extension of myself I I think of as a tool like I use a web
think of as a tool like I use a web browser to do this and someone may use
browser to do this and someone may use accessibility software to make the font
accessibility software to make the font bigger or you know or just display a web
bigger or you know or just display a web page and high contract so to me that's a
page and high contract so to me that's a tool I'm going to use AI to do for me
tool I'm going to use AI to do for me something that I could do and so if I
something that I could do and so if I could do something why can't I use a
could do something why can't I use a tool to do it too I can read web pages I
tool to do it too I can read web pages I want to use a tool to do it more
want to use a tool to do it more efficiently and I realize there are
efficiently and I realize there are others I'm blessed with that view that
others I'm blessed with that view that view AI is almost a different species
view AI is almost a different species this this thing separate from themselves
this this thing separate from themselves and I don't think animals need to
and I don't think animals need to necessarily have the same rights as
necessarily have the same rights as humans you know I think human I mean I
humans you know I think human I mean I value human life and I mean you know
value human life and I mean you know sometimes if a bug is annoying me you
sometimes if a bug is annoying me you know I will not treat it very nicely so
know I will not treat it very nicely so I don't think I don't think all bugs
I don't think I don't think all bugs need to have the same I don't think
need to have the same I don't think other species you know need to have
other species you know need to have exact same rights as as humans and I
exact same rights as as humans and I think realiz there's some people that
think realiz there's some people that view AI as almost a separate species and
view AI as almost a separate species and so we don't need to give them the same
so we don't need to give them the same right to read the internet as as humans
right to read the internet as as humans do so I I tend to view AI as a tool and
do so I I tend to view AI as a tool and that's why I took a certain view but was
that's why I took a certain view but was interesting seeing you know these two
interesting seeing you know these two different uh so this is a very
different uh so this is a very interesting conversation and we I just
interesting conversation and we I just want to advertise uh that we will have a
want to advertise uh that we will have a panel at the uh later this afternoon and
panel at the uh later this afternoon and we will hear from the authors and
we will hear from the authors and Publishers um who probably have a a
Publishers um who probably have a a different view from you and uh and that
different view from you and uh and that will be that will be fleshed out um this
will be that will be fleshed out um this this conversation is one that is clearly
this conversation is one that is clearly extremely important to be had and you
extremely important to be had and you know the perspective of that is an input
know the perspective of that is an input to this open source which is you know
to this open source which is you know providing all of this Innovation seems
providing all of this Innovation seems like you know that's that's a a a
like you know that's that's a a a perspective um to be weighed yeah and
perspective um to be weighed yeah and just to say I I do acknowledge this is a
just to say I I do acknowledge this is a difficult issue I honestly don't know
difficult issue I honestly don't know what's the my gut was let's let AI read
what's the my gut was let's let AI read the open internet but I do think it's a
the open internet but I do think it's a difficult issue I don't think I I don't
difficult issue I don't think I I don't claim to know the right answer and I
claim to know the right answer and I think protecting the Free Press which is
think protecting the Free Press which is a key pillar of democracy I think there
a key pillar of democracy I think there are very difficult issues I I don't
are very difficult issues I I don't think I know the full answer to so very
think I know the full answer to so very good well one of the few things you may
good well one of the few things you may not know the full answer too uh we want
not know the full answer too uh we want we always wonder about the AI in your
we always wonder about the AI in your brain um well just in the last couple
brain um well just in the last couple minutes um our next panel after the
minutes um our next panel after the break is going to be about um Healthcare
break is going to be about um Healthcare and Ai and I know you also have a long
and Ai and I know you also have a long history with health applications you
history with health applications you mentioned um access to electronic
mentioned um access to electronic medical record data um which could be a
medical record data um which could be a a bottleneck here but I'm curious
a bottleneck here but I'm curious generally um you know what are other
generally um you know what are other ways potentially completely different
ways potentially completely different ways that you might see AI impacting the
ways that you might see AI impacting the health care industry and sort of what
health care industry and sort of what are the other inputs that are needed for
are the other inputs that are needed for this health Innovation you know it's
this health Innovation you know it's been so so um my my my group here at St
been so so um my my my group here at St has done a lot of work on Healthcare we
has done a lot of work on Healthcare we deployed stuff at St Hospital you kind
deployed stuff at St Hospital you kind of publishing a bunch of papers and so
of publishing a bunch of papers and so on it's been fascinating watching how
on it's been fascinating watching how difficult it is to approach Healthcare
difficult it is to approach Healthcare in the US um I feel like um and in my
in the US um I feel like um and in my team of AI fun we see opportunities left
team of AI fun we see opportunities left and right right there are so many ways
and right right there are so many ways we can drive efficiencies improve
we can drive efficiencies improve patient care make doctor's lives better
patient care make doctor's lives better and the problem we keep on running it
and the problem we keep on running it over and over is not that we don't know
over and over is not that we don't know how to use AI to make things better it's
how to use AI to make things better it's the Golden Market you know I think that
the Golden Market you know I think that with the US patient payer provider
with the US patient payer provider missign incentives lots of Choke points
missign incentives lots of Choke points how do you how do you get a how do you
how do you how do you get a how do you get a payer to you know how do you how
get a payer to you know how do you how do you get that to be a code so the
do you get that to be a code so the payer can reimburse and all that so
payer can reimburse and all that so those tend to be the problems not even
those tend to be the problems not even the technical problems or how the build
the technical problems or how the build things that can actually make lives
things that can actually make lives better um and then the the other Dynamic
better um and then the the other Dynamic I'm seeing is I'm seeing um uh because
I'm seeing is I'm seeing um uh because of this narly ball of stuff that's so
of this narly ball of stuff that's so hard to approach in the US I'm seeing a
hard to approach in the US I'm seeing a lot of um Innovation and and and
lot of um Innovation and and and startups Pro profit nonprofit academic
startups Pro profit nonprofit academic happening overseas rather than in the
happening overseas rather than in the United States uh was visiting Singapore
United States uh was visiting Singapore a few months ago I was seeing stuff
a few months ago I was seeing stuff deployed in hospitals in Singapore that
deployed in hospitals in Singapore that I've not seen deploy anywhere in the
I've not seen deploy anywhere in the United States so you know check on the
United States so you know check on the hospital in Singapore if you have pum or
hospital in Singapore if you have pum or read EHR and try to give the doctor
read EHR and try to give the doctor guidance on you know this do does it see
guidance on you know this do does it see something that maybe the doctor has
something that maybe the doctor has missed I see stuff deployed in Singapore
missed I see stuff deployed in Singapore which has a unified patient ID basically
which has a unified patient ID basically uh that we don't have in the US you know
uh that we don't have in the US you know low litigation
low litigation risks all sorts of things and then
risks all sorts of things and then recently aian recently working with a
recently aian recently working with a with a startup that decided to um pursue
with a startup that decided to um pursue India Golden Market because candidly
India Golden Market because candidly it's going to be much easier to take
it's going to be much easier to take certain things to Market in India than
certain things to Market in India than here in the United States and just as a
here in the United States and just as a as a you know as as a as a US citizen I
as a you know as as a as a US citizen I I just felt kind of sad that we can't
I just felt kind of sad that we can't seem to get the job done here in America
seem to get the job done here in America um well so thank you so much Andrew you
um well so thank you so much Andrew you your many different perspectives in here
your many different perspectives in here were so interesting and I also love that
were so interesting and I also love that you're not afraid to say what you think
you're not afraid to say what you think um so that it makes for a much more
um so that it makes for a much more powerful and interesting conversation
powerful and interesting conversation can I can I add one comment that might
can I can I add one comment that might gave into trouble okay all right so I I
gave into trouble okay all right so I I know that a lot government officials
know that a lot government officials here so this is I should say but
here so this is I should say but honestly I'm I'm I'm I'm Pro World prot
honestly I'm I'm I'm I'm Pro World prot Tech and I'm also pro-america one thing
Tech and I'm also pro-america one thing that saddened me was it felt like um uh
that saddened me was it felt like um uh it felt like pava governments have
it felt like pava governments have become anti-tech in a way that I've not
become anti-tech in a way that I've not seen before so in previous ad multiple
seen before so in previous ad multiple previous administrations not one
previous administrations not one multiple previous administrations if
multiple previous administrations if European Regulators were messing around
European Regulators were messing around with American companies our government
with American companies our government went in to argue the Europeans right and
went in to argue the Europeans right and I appreciated that now I kind of see
I appreciated that now I kind of see when you know foreign Regulators are
when you know foreign Regulators are messing with American companies feels
messing with American companies feels like sometimes our Administration
like sometimes our Administration doesn't lift a finger to help them um
doesn't lift a finger to help them um I'm sure I'm seeing only isolated cases
I'm sure I'm seeing only isolated cases I'm sure this is a broader you know
I'm sure this is a broader you know context I'm not seeing but I feel like
context I'm not seeing but I feel like I'm prot Tech I'm I'm Pro competition
I'm prot Tech I'm I'm Pro competition I'm Pro startup I'm also pro-american
I'm Pro startup I'm also pro-american big Tech um and and I wish that uh uh
big Tech um and and I wish that uh uh SLI Valley and America American Tech in
SLI Valley and America American Tech in general is very precious and then candid
general is very precious and then candid when I see politicians call up
when I see politicians call up individual
individual entrepreneurs Like Bernie Sanders call
entrepreneurs Like Bernie Sanders call up Elon Musk by name for the sin of
up Elon Musk by name for the sin of being successful and Wealthy you know
being successful and Wealthy you know and yes he's committed other sins but it
and yes he's committed other sins but it feels like it feels like a I wish I wish
feels like it feels like a I wish I wish I felt like America remained you know
I felt like America remained you know firmly Pro competition and Pro Tech
firmly Pro competition and Pro Tech including big Tech and small Tech and St
including big Tech and small Tech and St but let's let's just keep building this
but let's let's just keep building this together um yeah what so Andrew thank
together um yeah what so Andrew thank you so much for being here we're going
you so much for being here we're going to move to a coffee break and we'll come
to move to a coffee break and we'll come back in about 10 minutes for the next
back in about 10 minutes for the next session thank
session thank [Applause]
good morning everyone we're going to get started again uh so if I could ask
started again uh so if I could ask everyone to take their
seats I'm Jennifer Dixon I'm an assistant chief in our policy section at
assistant chief in our policy section at the antitrust Division and also special
the antitrust Division and also special counsel for IP and policy and I will be
counsel for IP and policy and I will be one of your MC's of the day so if you
one of your MC's of the day so if you have any questions please do uh reach
have any questions please do uh reach out to me um now that we've heard from
out to me um now that we've heard from uh Professor uh leang and Professor ing
uh Professor uh leang and Professor ing uh about Foundation models they set the
uh about Foundation models they set the stage about open and closed uh models
stage about open and closed uh models the benefits of of uh open- Source
the benefits of of uh open- Source models we've learned about applications
models we've learned about applications we're going to now apply that to our
we're going to now apply that to our discussion of healthc care applications
discussion of healthc care applications and I wanted to introduce our uh
and I wanted to introduce our uh moderator uh Katrina Rouse who is a a
moderator uh Katrina Rouse who is a a colleague of mine at the antitrust
colleague of mine at the antitrust division uh she is the director of the
division uh she is the director of the antitrust division's new task force on
antitrust division's new task force on healthc care uh she also is the deputy
healthc care uh she also is the deputy director of civil enforcement um and a
director of civil enforcement um and a special counsel for healthcare so thank
special counsel for healthcare so thank you Katrina and she will introduce the
you Katrina and she will introduce the rest of our panelists and all of their
rest of our panelists and all of their full bios are um in the program
full bios are um in the program materials so go ahead
materials so go ahead thanks very much Jennifer I am thrilled
thanks very much Jennifer I am thrilled to moderate this Healthcare Spotlight
to moderate this Healthcare Spotlight panel where we'll discuss the current
panel where we'll discuss the current state of competition in the healthare AI
state of competition in the healthare AI space the role that data access and
space the role that data access and Advantage plays in this competition what
Advantage plays in this competition what the dimensions of competition in this
the dimensions of competition in this space are and could be and the potential
space are and could be and the potential for algorithmic collusion among other
for algorithmic collusion among other topics I am really looking forward to a
topics I am really looking forward to a fascinating conversation today with
fascinating conversation today with these five exceptional panelists and
these five exceptional panelists and I'll now introduce them Elena Vach is a
I'll now introduce them Elena Vach is a partner at the VC fund General Catalyst
partner at the VC fund General Catalyst where she focuses on investing in life
where she focuses on investing in life sciences she serves on several boards
sciences she serves on several boards and prior to joining General Catalyst
and prior to joining General Catalyst she helped build the soft Bank Vision
she helped build the soft Bank Vision funds Healthcare
funds Healthcare portfolio David kizner is the general
portfolio David kizner is the general counsel and Chief privacy officer at viz
counsel and Chief privacy officer at viz AI a leading AI care coordination
AI a leading AI care coordination platform for disease detection and
platform for disease detection and workflow optimization prior to viai
workflow optimization prior to viai David served in legal roles for Stanford
David served in legal roles for Stanford University Stanford Healthcare Phillips
University Stanford Healthcare Phillips Healthcare and
Healthcare and Center next we have Z overmire he's the
Center next we have Z overmire he's the Blue Cross of California distinguished
Blue Cross of California distinguished associate professor of Health policy and
associate professor of Health policy and management at the UC Berkeley School of
management at the UC Berkeley School of Public Health where he does research at
Public Health where he does research at the intersection of machine learning
the intersection of machine learning medicine and health policy he is a
medicine and health policy he is a practicing emergency medicine physician
practicing emergency medicine physician and he's also a Founder he co-founded
and he's also a Founder he co-founded dandelion health an AI Innovation
dandelion health an AI Innovation platform that makes Healthcare data
platform that makes Healthcare data available to algorithm developers and he
available to algorithm developers and he founded a nonprofit Nightingale open
founded a nonprofit Nightingale open science which builds out data sets in
science which builds out data sets in partnership with health
partnership with health systems then we have Ben handle he's an
systems then we have Ben handle he's an associate professor of Economics at UC
associate professor of Economics at UC Berkeley and a faculty research fellow
Berkeley and a faculty research fellow at the National Bureau of economic
at the National Bureau of economic research his important research focuses
research his important research focuses on the microeconomics of consumer choice
on the microeconomics of consumer choice and Market structure in the healthcare
and Market structure in the healthcare sector with an emphasis on health
sector with an emphasis on health insurance
insurance markets and then finally We Have Allison
markets and then finally We Have Allison oliger she is the chief data officer and
oliger she is the chief data officer and director of The Office of Enterprise
director of The Office of Enterprise data and analytics at the centers for
data and analytics at the centers for Medicare and Medicaid services is in
Medicare and Medicaid services is in this role Allison oversees cms's data
this role Allison oversees cms's data and information product portfolio and
and information product portfolio and she manages the development advance of
she manages the development advance of advanced analytics using CMS
advanced analytics using CMS data so to begin I thought we could do a
data so to begin I thought we could do a quick overview of the major categories
quick overview of the major categories of healthcare AI applications I think we
of healthcare AI applications I think we all understand that where there is huge
all understand that where there is huge amounts of Health Data um we should be
amounts of Health Data um we should be able to use algorithms to make better
able to use algorithms to make better medical decisions but I'd really like to
medical decisions but I'd really like to drill down on the specifics here and I
drill down on the specifics here and I thought David maybe you could start by
thought David maybe you could start by giving us an overview and examples of
giving us an overview and examples of the clinical applications for healthcare
the clinical applications for healthcare AI including what viz AI does sure
AI including what viz AI does sure there's there's hundreds I will probably
there's there's hundreds I will probably focus on what viz does and unfortunately
focus on what viz does and unfortunately this isn't the exciting uh generative AI
this isn't the exciting uh generative AI uh this is kind of old school computer
uh this is kind of old school computer vision so um interrogating like a data
vision so um interrogating like a data set so uh CT uh studies or MRIs or
set so uh CT uh studies or MRIs or ultrasound and uh finding suspected
ultrasound and uh finding suspected conditions so the first one I'll talk
conditions so the first one I'll talk about which is um very relevant because
about which is um very relevant because we're in American stroke awareness right
we're in American stroke awareness right now is the first algorithm that viz
now is the first algorithm that viz developed called V viz lvo which stands
developed called V viz lvo which stands for large vessel occlusion so it's a
for large vessel occlusion so it's a type of stroke uh won't bore you with
type of stroke uh won't bore you with all the details but effectively there's
all the details but effectively there's two types of stroke there's a es schic
two types of stroke there's a es schic stroke which is a blockage and there was
stroke which is a blockage and there was a hemorrhagic stroke which is a bleed
a hemorrhagic stroke which is a bleed they're treated very very differently
they're treated very very differently and if you treat one like the other you
and if you treat one like the other you could kill a patient so uh there's a
could kill a patient so uh there's a saying in stroke time is brain the clock
saying in stroke time is brain the clock is ticking your brain is being starved
is ticking your brain is being starved of oxygen and for an es schic stroke a
of oxygen and for an es schic stroke a large vessal occlusion uh the doctors
large vessal occlusion uh the doctors joke they refer them to themselves as
joke they refer them to themselves as plumbers because you're literally
plumbers because you're literally clearing a block right and then you're
clearing a block right and then you're restoring blood flow and oxygen to the
restoring blood flow and oxygen to the brain the faster you do it the better
brain the faster you do it the better the outcomes you have so what does viz
the outcomes you have so what does viz lvo do uh when a patient presents
lvo do uh when a patient presents suspected of having a stroke in an
suspected of having a stroke in an emergency room they're going to get a CT
emergency room they're going to get a CT scan of their head
scan of their head you know typical workflow that would go
you know typical workflow that would go off to a radiologist to read uh they
off to a radiologist to read uh they they might you know pick it up quickly
they might you know pick it up quickly they' alert the emergency room um if
they' alert the emergency room um if that hospital is capable of doing a
that hospital is capable of doing a mechanical thrombectomy they could do it
mechanical thrombectomy they could do it if it's not they have to transfer the
if it's not they have to transfer the patient so viz um algorithm detects that
patient so viz um algorithm detects that suspected large vessal occlusion on a CT
suspected large vessal occlusion on a CT scan has a whole application for secure
scan has a whole application for secure communication and image viewing so that
communication and image viewing so that the care team can coordinate the care of
the care team can coordinate the care of that patient either get the patient
that patient either get the patient treated at that hospital or transferred
treated at that hospital or transferred as quickly as possible so that's that's
as quickly as possible so that's that's viz lvo and and having real world um
viz lvo and and having real world um Improvement in patient outcomes you know
Improvement in patient outcomes you know reducing costs for uh having better
reducing costs for uh having better Financial outcomes for Health Care
Financial outcomes for Health Care Systems and payers uh another algorithm
Systems and payers uh another algorithm I'll talk about is called viz HCM so
I'll talk about is called viz HCM so hypertrophic cardiomyopathy is a cardiac
hypertrophic cardiomyopathy is a cardiac condition um you may hear of this from
condition um you may hear of this from time to time about like an athlete in
time to time about like an athlete in the prime of their life who suddenly has
the prime of their life who suddenly has a heart attack and May die on the
a heart attack and May die on the practice field this is uh I believe it's
practice field this is uh I believe it's the number one cardiac killer of people
the number one cardiac killer of people under 30 and often times people are
under 30 and often times people are asymptomatic they don't even know that
asymptomatic they don't even know that they have this disease so viz um in
they have this disease so viz um in partnership with Bristol Meer squib
partnership with Bristol Meer squib developed an algorithm that detects
developed an algorithm that detects suspected HCM and just a simple 12 lead
suspected HCM and just a simple 12 lead ECG and again it's not acute like stroke
ECG and again it's not acute like stroke where the the time is really ticking but
where the the time is really ticking but it gets those um alerts to the right
it gets those um alerts to the right clinicians who can take action and help
clinicians who can take action and help the patient
the patient um and then I'll mention one that is uh
um and then I'll mention one that is uh generative Ai and probably the folks
generative Ai and probably the folks from Stanford who who work on on this
from Stanford who who work on on this know more about it but um when I was
know more about it but um when I was here at Stanford uh uh Stanford um is
here at Stanford uh uh Stanford um is beginning to use AI to help clinicians
beginning to use AI to help clinicians do a first draft of a response when
do a first draft of a response when patients interact with them over the My
patients interact with them over the My Health application which is part of Epic
Health application which is part of Epic um so the clinicians are intended to
um so the clinicians are intended to then you know review that but it helps
then you know review that but it helps them um respond to the overwhelming
them um respond to the overwhelming number of inquiries they get from
number of inquiries they get from patients like us so those are those are
patients like us so those are those are three examples in the healthcare space
three examples in the healthcare space can I hop in with a little bit about the
can I hop in with a little bit about the health system Beyond kind of the vi
health system Beyond kind of the vi point so you know just broadly General
point so you know just broadly General Catalyst invest in healthcare I'm a life
Catalyst invest in healthcare I'm a life sciences specialist but learned from my
sciences specialist but learned from my colleagues in healthc care and some of
colleagues in healthc care and some of the things that we're seeing in AI for
the things that we're seeing in AI for health care and health delivery are
health care and health delivery are things like viz AI which help enhance
things like viz AI which help enhance efficacy or quality so can I enable a
efficacy or quality so can I enable a doctor to treat a patient faster which
doctor to treat a patient faster which drives a better outcome or to lower an
drives a better outcome or to lower an error rate in things like Radiology or
error rate in things like Radiology or pathology so that's one overall category
pathology so that's one overall category or to find sepsis based on synthesizing
or to find sepsis based on synthesizing various data and run an alert everyone
various data and run an alert everyone hates epic sepsis alert it's been around
hates epic sepsis alert it's been around forever it's not AI drives people crazy
forever it's not AI drives people crazy but can you actually drive better
but can you actually drive better quality with these kinds of alerting
quality with these kinds of alerting that's one category is how can you use
that's one category is how can you use AI to drive better efficacy then there's
AI to drive better efficacy then there's better throughput or lower cost
better throughput or lower cost investors get seduced by this but it's
investors get seduced by this but it's not what health systems are solving for
not what health systems are solving for right so like better throughput is nice
right so like better throughput is nice um you know there are various ways you
um you know there are various ways you know in Radiology or pathology that you
know in Radiology or pathology that you can use AI to move faster but what
can use AI to move faster but what people really get excited about is
people really get excited about is driving patient impact differential
driving patient impact differential outcomes and then the last aspect is is
outcomes and then the last aspect is is taking cost and friction out of the
taking cost and friction out of the system so doctors don't like doing notes
system so doctors don't like doing notes imagine some of you guys are doctors in
imagine some of you guys are doctors in the audience um and also when you think
the audience um and also when you think about the overall costs in the Health
about the overall costs in the Health Care System you're paying a lot of
Care System you're paying a lot of people for things like healthcare bling
people for things like healthcare bling coding or nurse scheduling right there's
coding or nurse scheduling right there's a big Workforce shortage you're actually
a big Workforce shortage you're actually paying someone a lot of money to
paying someone a lot of money to schedule and reschedule nurses so there
schedule and reschedule nurses so there are places where AI can slot in to
are places where AI can slot in to relieve administrative burden which
relieve administrative burden which frees up money for research freeze up
frees up money for research freeze up money for patient care and so things
money for patient care and so things like that are as simple as nurse
like that are as simple as nurse scheduling cure as a company in our
scheduling cure as a company in our portfolio that has that among really
portfolio that has that among really broads set of offerings things like
broads set of offerings things like helping patients prepare for their
helping patients prepare for their appointments and giving nurses leverage
appointments and giving nurses leverage in that um and things like Healthcare
in that um and things like Healthcare billing and coding which are very manual
billing and coding which are very manual and labor intensive um you know in an
and labor intensive um you know in an Ideal World we could remove that as a
Ideal World we could remove that as a job category but given that it is a
job category but given that it is a category how can you make it high
category how can you make it high quality and faster using AA so that's
quality and faster using AA so that's kind of how I think about it like within
kind of how I think about it like within the broader Health Care delivery side is
the broader Health Care delivery side is most exciting appeals to my heart
most exciting appeals to my heart driving better patient
driving better patient outcomes important is better throughput
outcomes important is better throughput and then indirect effect but really good
and then indirect effect but really good for the health system are where can you
for the health system are where can you take things that are manual and labor
take things that are manual and labor intensive and transform them to be less
intensive and transform them to be less labor intensive and more comp compute
labor intensive and more comp compute enabled thank you and I thought we could
enabled thank you and I thought we could turn to Ben to discuss Healthcare
turn to Ben to discuss Healthcare applications on the insurer side yeah
applications on the insurer side yeah absolutely uh so I think one of the main
absolutely uh so I think one of the main points I'd like to make at an overview
points I'd like to make at an overview level is that in as many of you know in
level is that in as many of you know in the health insurance space insurers do a
the health insurance space insurers do a lot more than just providing insurance
lot more than just providing insurance so I think I think of insurance business
so I think I think of insurance business models a lot as relating to kind of
models a lot as relating to kind of developing a collection or a basket of
developing a collection or a basket of different rationing policies or
different rationing policies or different care provision policies that
different care provision policies that kind of
kind of get kind of care delivered to consumers
get kind of care delivered to consumers in a way that the insurer feels is kind
in a way that the insurer feels is kind of efficient uh and effective for them
of efficient uh and effective for them and of course insur private insurers do
and of course insur private insurers do this across a range of markets Medicare
this across a range of markets Medicare Advantage Medicare Part D the Affordable
Advantage Medicare Part D the Affordable Care Act exchanges and large employer
Care Act exchanges and large employer markets uh I think you know there's
markets uh I think you know there's insurers have always used data as a core
insurers have always used data as a core part of their businesses uh but I think
part of their businesses uh but I think that in recent years data has become
that in recent years data has become even more core part of what insures are
even more core part of what insures are building as kind of a business and
building as kind of a business and building as assets and together with
building as assets and together with that together with those data assets um
that together with those data assets um complimentary expertise in data
complimentary expertise in data analytics uh I think something that's
analytics uh I think something that's quite important here is that you know
quite important here is that you know for insurers to gain competitive
for insurers to gain competitive advantages in this space uh they're
advantages in this space uh they're really working hard to develop
really working hard to develop proprietary data sets and develop
proprietary data sets and develop proprietary data sets that um really are
proprietary data sets that um really are kind of span the whole vertical supp
kind of span the whole vertical supp supply chain of healthcare not just you
supply chain of healthcare not just you know we're looking to try and kind of
know we're looking to try and kind of rate risk of certain customers but all
rate risk of certain customers but all sorts of details on metadata relating to
sorts of details on metadata relating to the interactions between Physicians and
the interactions between Physicians and patients metadata related to claims
patients metadata related to claims denial and prior
denial and prior authorization uh things like that and so
authorization uh things like that and so I think you know just some of the
I think you know just some of the functions I think of insurers leveraging
functions I think of insurers leveraging these data and techniques to work on
these data and techniques to work on include you know prior
include you know prior authorization uh for drugs and for
authorization uh for drugs and for services
services uh how do we Implement claims denials in
uh how do we Implement claims denials in a way that's uh consistent with our
a way that's uh consistent with our policies that's kind of how insurers
policies that's kind of how insurers would describe it uh how do we Implement
would describe it uh how do we Implement value based care policies that also May
value based care policies that also May rely on data related to uh electronic
rely on data related to uh electronic health records uh that they're uh
health records uh that they're uh bringing in uh and then on the insurance
bringing in uh and then on the insurance side how do we find customers on the
side how do we find customers on the employer side and split their risk in a
employer side and split their risk in a way that kind of allows us to be
way that kind of allows us to be profitable more recently I think data uh
profitable more recently I think data uh one thing insurers have been using data
one thing insurers have been using data for us to highlight providers uh that
for us to highlight providers uh that they can look to purchase uh and I think
they can look to purchase uh and I think that more broadly uh in this space
that more broadly uh in this space vertical integration between insurers
vertical integration between insurers and other businesses um such as Pharmacy
and other businesses um such as Pharmacy benefit managers or
benefit managers or providers uh has been obviously a very
providers uh has been obviously a very strong Trend and I think that you know
strong Trend and I think that you know when we've thought about these things
when we've thought about these things we've thought about these things in
we've thought about these things in terms of the vertical economics of
terms of the vertical economics of pricing and models uh on kind of the
pricing and models uh on kind of the more core economic side
more core economic side but I think that one of the reasons
but I think that one of the reasons insurers are really you know looking to
insurers are really you know looking to you know work to to to to vertically
you know work to to to to vertically integrate with these kinds of fir with
integrate with these kinds of fir with these kinds of firms is they're looking
these kinds of firms is they're looking to assemble these large data sets across
to assemble these large data sets across the whole supply chain to use to gain a
the whole supply chain to use to gain a competitive advantage in their
competitive advantage in their businesses um it's all stopped
businesses um it's all stopped there thank you and Elena I thought you
there thank you and Elena I thought you could give us the overview and examples
could give us the overview and examples in the Pharma and Life Sciences space
in the Pharma and Life Sciences space just a quick PLL how many people in the
just a quick PLL how many people in the audience are interested in health
audience are interested in health systems and insurers as your primary
systems and insurers as your primary interest area how many people live in
interest area how many people live in payer provider
payer provider World High hands okay that's either
World High hands okay that's either you're very shy or not that many how
you're very shy or not that many how many people live in life sciences
many people live in life sciences World okay and then most of you won't
World okay and then most of you won't vote
vote um this is like worse than us voter
um this is like worse than us voter turnout okay um we're just trying to
turnout okay um we're just trying to gauge I'm just trying to gauge here how
gauge I'm just trying to gauge here how much interest there is in life science
much interest there is in life science but I'll take that as a little bit um so
but I'll take that as a little bit um so if you think about life sciences as
if you think about life sciences as Therapeutics Diagnostics research tools
Therapeutics Diagnostics research tools and then software and service that
and then software and service that enables that there are opportunities
enables that there are opportunities across the Spectrum on the drug
across the Spectrum on the drug Discovery or therapeutic side it can
Discovery or therapeutic side it can apply to everything from your target
apply to everything from your target Discovery so like what biology are you
Discovery so like what biology are you going after or what biology matters to
going after or what biology matters to how you make your chemical matter so
how you make your chemical matter so whether it's a small molecule or an
whether it's a small molecule or an antibody understanding where am I trying
antibody understanding where am I trying to bind to so understanding the target
to bind to so understanding the target protein or what chemical am I going to
protein or what chemical am I going to make whether it's chemical or protein
make whether it's chemical or protein what structure am I making you can use
what structure am I making you can use AI in all of
AI in all of that um so there's different ways to
that um so there's different ways to leverage AI depending on what your
leverage AI depending on what your problem is but it's highly relevant up
problem is but it's highly relevant up and down the value chain of drug
and down the value chain of drug Discovery the same thing in Diagnostics
Discovery the same thing in Diagnostics and it's a little bit more concrete so
and it's a little bit more concrete so back to sepsis are you mining the EMR or
back to sepsis are you mining the EMR or aggregating other sources of data to
aggregating other sources of data to provide an early warning system and an
provide an early warning system and an early intervention in cancer can you do
early intervention in cancer can you do this difficult signal to noise detection
this difficult signal to noise detection problem of finding rare circulating
problem of finding rare circulating tumor uh DNA or other forms of cancer
tumor uh DNA or other forms of cancer and distinguishing that from normal
and distinguishing that from normal healthy DNA in the blood um so in
healthy DNA in the blood um so in Diagnostics it's about ident identifying
Diagnostics it's about ident identifying something and surfacing that for
something and surfacing that for intervention and then in research it's
intervention and then in research it's actually really cool something that
actually really cool something that happened here at
happened here at Stanford um by a fellow named Renee
Stanford um by a fellow named Renee Casey is he started a company called
Casey is he started a company called metal Loop that does endtoend enablement
metal Loop that does endtoend enablement of research so everything from helping
of research so everything from helping you write your Grant to data collection
you write your Grant to data collection so pulling data out of the EMR playing
so pulling data out of the EMR playing data out of wearable so how do you get
data out of wearable so how do you get all of this multimodal data to data
all of this multimodal data to data analysis to writing your paper at the
analysis to writing your paper at the other end so basically for each of these
other end so basically for each of these areas you can take Ai and apply it to
areas you can take Ai and apply it to every step in the value chain you can't
every step in the value chain you can't have it do it for you but you can get a
have it do it for you but you can get a starting draft for your Grant you can
starting draft for your Grant you can gather multimodal data more easily you
gather multimodal data more easily you can analyze it more easily and you can
can analyze it more easily and you can put your results out more easily and
put your results out more easily and that's maybe like the most concrete
that's maybe like the most concrete example of the same concept whether it's
example of the same concept whether it's Therapeutics diagnostics enablement for
Therapeutics diagnostics enablement for research is take whatever value chain
research is take whatever value chain you had before before and think about
you had before before and think about where can AI solve a problem that
where can AI solve a problem that couldn't be solved before or help me do
couldn't be solved before or help me do this better faster cheaper one other
this better faster cheaper one other example on research I'll mention is um
example on research I'll mention is um screening for study subjects right like
screening for study subjects right like how can you find study subjects faster
how can you find study subjects faster to power a study could be hugely
to power a study could be hugely valuable saves time right and and gets a
valuable saves time right and and gets a study completed sooner so if you can use
study completed sooner so if you can use AI to you know find those patients
AI to you know find those patients sometimes needles in the hay stack
sometimes needles in the hay stack it's much better than waiting for the
it's much better than waiting for the principal investigator to have that
principal investigator to have that person show up in their clinic or their
person show up in their clinic or their Ed it's a huge deal because right now
Ed it's a huge deal because right now nurses have to go through electronic
nurses have to go through electronic medical records surfacing hey I think
medical records surfacing hey I think this patient could be a good fit for
this patient could be a good fit for your trial and then the PI says maybe
your trial and then the PI says maybe I'm going to enroll them um but if you
I'm going to enroll them um but if you have something like viai they can pull
have something like viai they can pull through scans and in real time say hey
through scans and in real time say hey this patient could be a relevant subject
this patient could be a relevant subject for this intervention especially for
for this intervention especially for things where you need to identify and
things where you need to identify and enroll patients quickly for sort of
enroll patients quickly for sort of urgent or critical studies it's a huge
urgent or critical studies it's a huge challenge for how do you find the
challenge for how do you find the patient to offer them that intervention
patient to offer them that intervention and usually involves having a nurse
and usually involves having a nurse sitting right there waiting for a
sitting right there waiting for a patient to happen um and then like
patient to happen um and then like operationally for a place like Stanford
operationally for a place like Stanford anywhere else the question is are you
anywhere else the question is are you going to fund or staff a clinical
going to fund or staff a clinical research nurse at all times where you're
research nurse at all times where you're taking risk on that person's salary
taking risk on that person's salary while hoping to have a Pharma partner
while hoping to have a Pharma partner sponsor it that's kind of apart um but
sponsor it that's kind of apart um but if you have aib based tools that can
if you have aib based tools that can enable patient identification then you
enable patient identification then you can bring in a nurse when needed without
can bring in a nurse when needed without having to take that risk on Staffing and
having to take that risk on Staffing and it means you can do a lot more trials
it means you can do a lot more trials without requiring an external sponsor or
without requiring an external sponsor or without requiring you to Bear the burden
without requiring you to Bear the burden of that cost that's just like a good
of that cost that's just like a good example of like one kind of like it
example of like one kind of like it sounds like a simple problem but really
sounds like a simple problem but really hard problem of how do you go from this
hard problem of how do you go from this patient could benefit from a trial to
patient could benefit from a trial to actually giving them access to that
actually giving them access to that trial
trial with that very helpful background I
with that very helpful background I wanted to have each panelist go through
wanted to have each panelist go through and give their take on the state of um
and give their take on the state of um Healthcare AI competition today and so
Healthcare AI competition today and so the question is are we seeing uh the
the question is are we seeing uh the amount of competition we would expect in
amount of competition we would expect in a competitive Marketplace and if not
a competitive Marketplace and if not what are the barriers to competition and
what are the barriers to competition and to make sure we're hearing from everyone
to make sure we're hearing from everyone I'll start with zad who we haven't heard
I'll start with zad who we haven't heard from yet um thanks so I thought um
from yet um thanks so I thought um instead of directly answering the
instead of directly answering the question I'd tell you a story that
question I'd tell you a story that speaks to the state of the market so a
speaks to the state of the market so a few years ago with some colleagues we
few years ago with some colleagues we started evaluating a family of
started evaluating a family of algorithms that um is sold to a number
algorithms that um is sold to a number of healthare systems around the country
of healthare systems around the country so just to give you a sense of um the
so just to give you a sense of um the the problem these algorithms are solving
the problem these algorithms are solving you've got a population of patients you
you've got a population of patients you know some of them are going to get sick
know some of them are going to get sick if you knew which one of them was going
if you knew which one of them was going to get sick you would do a lot of things
to get sick you would do a lot of things to help them prevent exacerbations of
to help them prevent exacerbations of chronic conditions flare ups things like
chronic conditions flare ups things like that today but you can't do that for
that today but you can't do that for everyone so algorithms are perfect for
everyone so algorithms are perfect for this job you find the people who are
this job you find the people who are going to get sick and you help them
going to get sick and you help them today so it's a great AI application and
today so it's a great AI application and there are a number of large companies
there are a number of large companies that are selling software to do this to
that are selling software to do this to healthcare system so this is at the
healthcare system so this is at the intersection of kind of clinical and
intersection of kind of clinical and insurance and um and and optimization
insurance and um and and optimization and what we found was that these
and what we found was that these algorithms which are screening um
algorithms which are screening um between 100 and 200 million people per
between 100 and 200 million people per year for this you know extra help with
year for this you know extra help with your chronic conditions um contained
your chronic conditions um contained this very large amount of racial bias um
this very large amount of racial bias um so they were putting healthier white
so they were putting healthier white patients ahead in line for extra help um
patients ahead in line for extra help um ahead of sicker black patients um and
ahead of sicker black patients um and and this was not known it was like this
and this was not known it was like this was not an efficient situation like
was not an efficient situation like after the study came out uh after a lot
after the study came out uh after a lot of senators sent letters to companies
of senators sent letters to companies and Hospital systems and things like
and Hospital systems and things like that people stopped using this we worked
that people stopped using this we worked with the team at the company to fix the
with the team at the company to fix the problem in the this is not like a design
problem in the this is not like a design thing this is just an error um but it
thing this is just an error um but it was not known and this was like hundreds
was not known and this was like hundreds of millions of patients per year being
of millions of patients per year being affected by this fairly large scale
affected by this fairly large scale racial bias now why why did this happen
racial bias now why why did this happen why were we the ones who um figured it
why were we the ones who um figured it out well there's a very simple answer
out well there's a very simple answer which is that we had access to the data
which is that we had access to the data that we needed to basically um do this
that we needed to basically um do this analysis and show that black patients
analysis and show that black patients were being discriminated against by this
were being discriminated against by this algorithm um and and I think that's a
algorithm um and and I think that's a microcosm of a much larger problem in
microcosm of a much larger problem in this area which is that the consumer is
this area which is that the consumer is not given the data that he or she needs
not given the data that he or she needs to make the decision about the algorithm
to make the decision about the algorithm that um that they're that they're
that um that they're that they're purchasing and I think that that is a
purchasing and I think that that is a huge problem and it and it limits the
huge problem and it and it limits the competitive this SM the competitiveness
competitive this SM the competitiveness of this Market it it prevents the market
of this Market it it prevents the market from being aimed in the right direction
from being aimed in the right direction and helping uh patients and helping the
and helping uh patients and helping the healthare
healthare system um Ben do you w to go next sure
system um Ben do you w to go next sure yeah um I'll be brief and I'll just talk
yeah um I'll be brief and I'll just talk a little bit about the role of data in
a little bit about the role of data in in in Insurance markets um you know I
in in Insurance markets um you know I think that data are just uh and this
think that data are just uh and this kind of a glomeration of data as an
kind of a glomeration of data as an asset it's kind of one ingredient that
asset it's kind of one ingredient that has kind of led to economies of scale
has kind of led to economies of scale for insurers both across kind of
for insurers both across kind of horizontal markets where insurers are
horizontal markets where insurers are merging with other insurers as well as
merging with other insurers as well as vertical markets where insurers are
vertical markets where insurers are merging with other parts of the um other
merging with other parts of the um other parts of the supply chain and I think
parts of the supply chain and I think that in Health policy we've seen this in
that in Health policy we've seen this in the past with things like accountable
the past with things like accountable care organizations but I think in in
care organizations but I think in in merger policy now both on the economic
merger policy now both on the economic side but also on on the legal side
side but also on on the legal side there's this tension where you know if
there's this tension where you know if if a merger leads to tangible
if a merger leads to tangible efficiencies or it's a vertical merger
efficiencies or it's a vertical merger and there's some presumed
and there's some presumed efficiencies uh we might kind of
efficiencies uh we might kind of historically de facto let this merger go
historically de facto let this merger go ahead when in fact when you have data
ahead when in fact when you have data and data analytics is kind of a central
and data analytics is kind of a central asset that's a part of this merger
asset that's a part of this merger it can lead to a number of asymmetries
it can lead to a number of asymmetries and a number of competitive advantages
and a number of competitive advantages where you know it's at some point it's
where you know it's at some point it's not obvious that these kind of
not obvious that these kind of increasing economies of scale are worth
increasing economies of scale are worth the trade-off with with Market power and
the trade-off with with Market power and I think that you know the new merger
I think that you know the new merger guidelines for example try to speak to
guidelines for example try to speak to these issues uh but I think that there
these issues uh but I think that there needs to be both kind of more
needs to be both kind of more development on the economic side but
development on the economic side but also uh on the on the law side on the
also uh on the on the law side on the legal side
legal side here Allison yeah just building on some
here Allison yeah just building on some of the points that the other panelists
of the points that the other panelists have made I think in the healthcare
have made I think in the healthcare space data is such an asset and as Ben
space data is such an asset and as Ben pointed out it has continued to grow as
pointed out it has continued to grow as an asset as more and more folks have
an asset as more and more folks have started to use artificial intelligence
started to use artificial intelligence but the other thing I think that's
but the other thing I think that's pretty unique about Healthcare that we
pretty unique about Healthcare that we have to think about with data is privacy
have to think about with data is privacy and that's something that uh also limits
and that's something that uh also limits the sharing of data the identification
the sharing of data the identification of healthcare data is really complicated
of healthcare data is really complicated if I know your doctor and your
if I know your doctor and your Healthcare conditions and your ZIP code
Healthcare conditions and your ZIP code I can probably figure out who you are
I can probably figure out who you are using social media or other publicly
using social media or other publicly available data and so I think there's
available data and so I think there's this kind of holding on to data because
this kind of holding on to data because we can get value from it but there's
we can get value from it but there's also holding on to data because we're
also holding on to data because we're worried about the risks to patients and
worried about the risks to patients and I think those are two things we we have
I think those are two things we we have to balance as we start to think about
to balance as we start to think about how we can make data more available to
how we can make data more available to promote competition in this space
Elena yeah um the panelists have covered it really well so far so just add maybe
it really well so far so just add maybe two small thoughts one is when we invest
two small thoughts one is when we invest in companies we take a look from the
in companies we take a look from the responsible Innovation perspective and
responsible Innovation perspective and so what we mean by that is we think
so what we mean by that is we think about intended and unintended
about intended and unintended consequences if this company scales what
consequences if this company scales what are the incentives that they will face
are the incentives that they will face and how will their behavior change as a
and how will their behavior change as a result of that so during the pandemic
result of that so during the pandemic you know a company called cerebral was
you know a company called cerebral was doing uh remote um psychiatric care
doing uh remote um psychiatric care which is really good on its face right
which is really good on its face right expand patient access to mental health
expand patient access to mental health the challenge is as a fee for service
the challenge is as a fee for service entity they were highly incentivized to
entity they were highly incentivized to churn patients and write lots of
churn patients and write lots of prescriptions that's just an example of
prescriptions that's just an example of how do you think about the intended and
how do you think about the intended and unintended consequences the intention
unintended consequences the intention was to expand patient access and address
was to expand patient access and address unmet medical need and the unintended
unmet medical need and the unintended consequence was an incentive to
consequence was an incentive to overprescribe or prescribe quickly
overprescribe or prescribe quickly without care um and so I think the same
without care um and so I think the same can be applied when you think about
can be applied when you think about businesses right what is the market
businesses right what is the market structure and how will this Market
structure and how will this Market structure incentivize or disincentivize
structure incentivize or disincentivize competition and then there are really
competition and then there are really smart people here uh whose job it is to
smart people here uh whose job it is to guide policy um or Sue uh to block uh
guide policy um or Sue uh to block uh unintended consequences and so we just
unintended consequences and so we just try to invest in companies where we
try to invest in companies where we think that their intention and their
think that their intention and their risk factors align with our mission um
risk factors align with our mission um and then count on the smart folks around
and then count on the smart folks around the table here to uh create the right
the table here to uh create the right Market structures for them to respond
Market structures for them to respond to and then I'll just say overall I
to and then I'll just say overall I think in the kind of the the viz space
think in the kind of the the viz space applying AI to images to detect things
applying AI to images to detect things and get patients are added to the care
and get patients are added to the care they need um Elena's firm uh has a
they need um Elena's firm uh has a portfolio comp that does very similar
portfolio comp that does very similar things so right on this panel you have
things so right on this panel you have two competitors and right here at
two competitors and right here at Stanford our third competitor uh was
Stanford our third competitor uh was created by a physician at Stanford so
created by a physician at Stanford so you know Ju Just Right Here in in this
you know Ju Just Right Here in in this this area we've got three Market
this area we've got three Market competitors so um but I do share what
competitors so um but I do share what what Andrew Ang mentioned at the end of
what Andrew Ang mentioned at the end of his speech the concern about data access
his speech the concern about data access so as um less so with medical images
so as um less so with medical images right I think we have Pathways to get
right I think we have Pathways to get access to that data but as you get into
access to that data but as you get into generative AI tools applied to health
generative AI tools applied to health record data you you do have powerful
record data you you do have powerful players that um hold that data and
players that um hold that data and there's there's other tools outside of I
there's there's other tools outside of I think anit trust you've got like cares
think anit trust you've got like cares act information blocking rules right so
act information blocking rules right so so long as they continue to play nicely
so long as they continue to play nicely and they allow Innovative companies to
and they allow Innovative companies to access that data then um then I'm not
access that data then um then I'm not concerned but if if that changes then I
concerned but if if that changes then I think it's something to keep an eye on
think it's something to keep an eye on yeah and I think our whole field owes a
yeah and I think our whole field owes a debt to the Regulators on making patient
debt to the Regulators on making patient data portable right like as a patient I
data portable right like as a patient I want to be able to take my data to
want to be able to take my data to wherever I seek care and then to
wherever I seek care and then to patients for then consenting their data
patients for then consenting their data into research um so I think that's a
into research um so I think that's a really good counterbalance regulation
really good counterbalance regulation that prevents the epics of the world
that prevents the epics of the world from keeping all of their data in a
from keeping all of their data in a Walled Garden or a health system saying
Walled Garden or a health system saying you can't take your data and switch to
you can't take your data and switch to another Health System there's actually
another Health System there's actually some really great regulation that lets
some really great regulation that lets you take your data with you and the next
you take your data with you and the next step could be can I take my algorithms
step could be can I take my algorithms with me so if someone develops an
with me so if someone develops an algorithm that's benefiting me as a
algorithm that's benefiting me as a patient can I Port that over just like
patient can I Port that over just like Regulators have said I'm allowed to Port
Regulators have said I'm allowed to Port my phone number can I take my health
my phone number can I take my health algorithms with me wherever I go I just
algorithms with me wherever I go I just want to point out one strange thing
want to point out one strange thing about this Market which and David you
about this Market which and David you brought up you know there are these
brought up you know there are these three companies doing similar things if
three companies doing similar things if you're hesitating among three cars you
you're hesitating among three cars you can go test drive the cars and you can
can go test drive the cars and you can make your decision one of the things
make your decision one of the things that's powered all of the advances in AI
that's powered all of the advances in AI over the past few decades is the ability
over the past few decades is the ability to see here are three algorithms let's
to see here are three algorithms let's just see how they're performing on this
just see how they're performing on this one gold standard data set and let's be
one gold standard data set and let's be able to compare their performance that's
able to compare their performance that's just what the algorithm is supposed to
just what the algorithm is supposed to be doing outputting a number on this
be doing outputting a number on this data set let's just see which one does
data set let's just see which one does it best and we can't do that with AI in
it best and we can't do that with AI in healthcare because there's so few ways
healthcare because there's so few ways for everyone to access these kinds of
for everyone to access these kinds of evaluations data set so it's taking the
evaluations data set so it's taking the The Supercharged fuel of AI out of
The Supercharged fuel of AI out of healthcare because we can't have access
healthcare because we can't have access to data where we can do these kinds of
to data where we can do these kinds of comparisons yeah and we kind of talked
comparisons yeah and we kind of talked about this a little bit in in the prep
about this a little bit in in the prep for this but the um I agree and like you
for this but the um I agree and like you can't necessarily like line up up like
can't necessarily like line up up like cars and say like this one has this much
cars and say like this one has this much horsepower and that one has that much
horsepower and that one has that much horsepower but um algorithm performance
horsepower but um algorithm performance I don't think is the fundamental issue
I don't think is the fundamental issue right it's like does it actually help
right it's like does it actually help patients um improve outcomes does it
patients um improve outcomes does it help reduce costs uh so at some point
help reduce costs uh so at some point you know the algorithm is is good enough
you know the algorithm is is good enough at doing what it's supposed to do but
at doing what it's supposed to do but the the the less exciting part of the AI
the the the less exciting part of the AI is the whole workflow tool on Healthcare
is the whole workflow tool on Healthcare the care coordination piece right
the care coordination piece right because if you have an amazing algorithm
because if you have an amazing algorithm but it goes nowhere and a clinician
but it goes nowhere and a clinician doesn't act on it a patient's outcomes
doesn't act on it a patient's outcomes aren't going to improve so um I I agree
aren't going to improve so um I I agree with you it would be wonderful to be
with you it would be wonderful to be able to kind of Benchmark and test and
able to kind of Benchmark and test and say Here's the actual positive
say Here's the actual positive predictive value of algorithm a b and c
predictive value of algorithm a b and c um but what I'm more interested in is
um but what I'm more interested in is like those real world studies done by
like those real world studies done by academic medical centers where they say
academic medical centers where they say hey before this algorithm was
hey before this algorithm was implemented what was our uh door to
implemented what was our uh door to needle time for stroke right after it my
needle time for stroke right after it my goodness we reduce it by 30 minutes and
goodness we reduce it by 30 minutes and that is like literally saving people's
that is like literally saving people's lives sending people home who are no
lives sending people home who are no longer disabled reducing costs for
longer disabled reducing costs for healthare right for for payers as well
healthare right for for payers as well because you're not supporting someone
because you're not supporting someone who may be disabled for the rest of
who may be disabled for the rest of their life so I I agree with you but I
their life so I I agree with you but I think it's more than just like straight
think it's more than just like straight up algorithm performance it would be
up algorithm performance it would be like it would be like buying a car based
like it would be like buying a car based upon horsepower and then realizing it's
upon horsepower and then realizing it's a two-seater and I can't get my kids to
a two-seater and I can't get my kids to soccer practice right and I think that
soccer practice right and I think that you know when you have a standard of
you know when you have a standard of evidence where you want to show impact
evidence where you want to show impact on patient outcomes if you want to run a
on patient outcomes if you want to run a randomized trial I think that would be
randomized trial I think that would be great like that but that seems like a
great like that but that seems like a whole other level of claims and evidence
whole other level of claims and evidence let's first figure out if the are
let's first figure out if the are working the way they're supposed to yeah
working the way they're supposed to yeah and some of that I will say like is is
and some of that I will say like is is um for The Med device algorithms right
um for The Med device algorithms right maybe I don't think the Epic sepsis ALG
maybe I don't think the Epic sepsis ALG algorithm for example was cleared as a
algorithm for example was cleared as a medical device but for um the algorithms
medical device but for um the algorithms that detect stroke for example they they
that detect stroke for example they they have to be cleared so there is some
have to be cleared so there is some floor level that FDA requires and also
floor level that FDA requires and also addresses some of the issues around bias
addresses some of the issues around bias right making sure that the data isn't
right making sure that the data isn't just from like one um you know one one
just from like one um you know one one hospital or or one type of patient and
hospital or or one type of patient and so um you have a diverse data set and it
so um you have a diverse data set and it performs at least at some minimum level
performs at least at some minimum level so you don't alert fatigue
so you don't alert fatigue Physicians I mean I think not to be
Physicians I mean I think not to be sitting in the Venture Corner over here
sitting in the Venture Corner over here but just to Echo what you're saying viz
but just to Echo what you're saying viz AI uh and AI doc the company we're
AI uh and AI doc the company we're invested in uh and any other AI player a
invested in uh and any other AI player a health system can say hey before I buy
health system can say hey before I buy it I want to run a pilot and I'm going
it I want to run a pilot and I'm going to run a pilot with all your competitors
to run a pilot with all your competitors let's see how it goes um and I think
let's see how it goes um and I think where viai and where AI doc have really
where viai and where AI doc have really differentiated is to your point in the
differentiated is to your point in the workflow like I met viai three four
workflow like I met viai three four years ago as an investor so my numbers
years ago as an investor so my numbers are totally out of date but I remember
are totally out of date but I remember then it was in like 400 Health Systems
then it was in like 400 Health Systems and had taken off like wildflyer not
and had taken off like wildflyer not because of the algorithm performance per
because of the algorithm performance per se but because of the alerting system to
se but because of the alerting system to Physicians it let them solve a problem
Physicians it let them solve a problem and the same thing with AI do it fits
and the same thing with AI do it fits into the physician workflow there's
into the physician workflow there's another company whose name I won't name
another company whose name I won't name that went bankrupt which was funded by
that went bankrupt which was funded by one of my competitors and they went to
one of my competitors and they went to Market saying we're going to replace
Market saying we're going to replace Radiologists well guess what
Radiologists well guess what Radiologists buy the AI um and they want
Radiologists buy the AI um and they want to know like does this make my life
to know like does this make my life easier does this drive patients outcomes
easier does this drive patients outcomes and then maybe I still want a job
and then maybe I still want a job tomorrow um so they didn't buy that one
tomorrow um so they didn't buy that one um you know and to your point um this is
um you know and to your point um this is like an audience participation uh
like an audience participation uh opportunity didn't the FDA say say to
opportunity didn't the FDA say say to Epic hey your sepsis thing should be
Epic hey your sepsis thing should be regulated anyone remember that anybody
regulated anyone remember that anybody anybody um not to speak out of school
anybody um not to speak out of school but I'm almost certain they said hey
but I'm almost certain they said hey this needs to come under software as a
this needs to come under software as a medical device regulation um so you do
medical device regulation um so you do have like a good balance there of saying
have like a good balance there of saying like hey there does have to be some bar
like hey there does have to be some bar for does this work um before you make it
for does this work um before you make it available
available right I think we had some really
right I think we had some really interesting discussion there on the
interesting discussion there on the dimensions of competition
dimensions of competition um I want to turn back to one of the
um I want to turn back to one of the earlier themes which is data access as a
earlier themes which is data access as a bottleneck and how Market participants
bottleneck and how Market participants who don't have ownership of a lot of
who don't have ownership of a lot of data can access it and I think um zad
data can access it and I think um zad and David both have interesting stories
and David both have interesting stories here so start with zad can you tell us
here so start with zad can you tell us about what prompted you to found night
about what prompted you to found night Andale and dandelion which both seem to
Andale and dandelion which both seem to be about um giving wider access to data
be about um giving wider access to data that people and companies can use to run
that people and companies can use to run a
a yeah I think a huge problem with the way
yeah I think a huge problem with the way this space is currently structured is
this space is currently structured is that the data live in one administrative
that the data live in one administrative enclave and the people who know how to
enclave and the people who know how to do machine learning live in a completely
do machine learning live in a completely different administrative enclave and it
different administrative enclave and it is really hard to either get the data
is really hard to either get the data out or to get those people in and so um
out or to get those people in and so um Nightingale open science is a nonprofit
Nightingale open science is a nonprofit um and this was a solution to the
um and this was a solution to the problem of lots of Berkeley PhD students
problem of lots of Berkeley PhD students coming to me and saying like oh I know
coming to me and saying like oh I know how to do this cool Machine Vision thing
how to do this cool Machine Vision thing it seems like it could be really useful
it seems like it could be really useful in healthcare I'd like to get involved
in healthcare I'd like to get involved in one and what I would say is like
in one and what I would say is like great we'll get your criminal background
great we'll get your criminal background check and your fingerprints done and
check and your fingerprints done and then we'll add you to the IRB and the
then we'll add you to the IRB and the data use agreement and then in two years
data use agreement and then in two years you'll and by that time of course
you'll and by that time of course they're already like working at Facebook
they're already like working at Facebook doing adclick optimization um and not
doing adclick optimization um and not doing Healthcare so uh Nightingale is um
doing Healthcare so uh Nightingale is um philanthropically funded we raised a few
philanthropically funded we raised a few million dollars from I'd say Tech Quant
million dollars from I'd say Tech Quant forward philanthropies Eric Schmidt Ken
forward philanthropies Eric Schmidt Ken Griffin Gordon and Betty Moore and we
Griffin Gordon and Betty Moore and we use that to create data sets around
use that to create data sets around interesting unsolved medical problems we
interesting unsolved medical problems we de identify the data inside of the
de identify the data inside of the health system we bring it out put it on
health system we bring it out put it on our Cloud you can access it within 24
our Cloud you can access it within 24 hours after you sign the data use
hours after you sign the data use agreement it's Nightingale science.org
agreement it's Nightingale science.org um so that is a kind of one data set at
um so that is a kind of one data set at a time solution uh it's not a market
a time solution uh it's not a market solution and so dandelion is a
solution and so dandelion is a for-profit company um that I uh helped
for-profit company um that I uh helped co-found and dandelion partners with um
co-found and dandelion partners with um a a set of large non-academic Health
a a set of large non-academic Health Systems where we get access to um all of
Systems where we get access to um all of their data and are able to De identify
their data and are able to De identify it within their infrastructure bring it
it within their infrastructure bring it out onto our cloud and then um allow a
out onto our cloud and then um allow a carefully curated set of clinical
carefully curated set of clinical algorithm developers to get access to
algorithm developers to get access to those data to build clinical products so
those data to build clinical products so this is a way to get the market um
this is a way to get the market um working and pointed in the right
working and pointed in the right direction for algorithm development so
direction for algorithm development so that people aren't just comp competing
that people aren't just comp competing on access to data but are competing on
on access to data but are competing on algorithm
algorithm quality and that would definitely be
quality and that would definitely be hugely valuable right because getting
hugely valuable right because getting getting the data so um uh every time viz
getting the data so um uh every time viz for example or ad do or any other
for example or ad do or any other company in the space moves into another
company in the space moves into another type of data you've got to gather that
type of data you've got to gather that data um it's not just that getting the
data um it's not just that getting the data too it's also annotating the data
data too it's also annotating the data you have to need to know whether it's
you have to need to know whether it's positive or false you know not not a
positive or false you know not not a positive it's a negative um scan
positive it's a negative um scan uh so uh how did viz do it initially I
uh so uh how did viz do it initially I think was by doing research agreements
think was by doing research agreements right but that's expensive it's time
right but that's expensive it's time consuming like you said you have to go
consuming like you said you have to go through irbs um uh it's costly because
through irbs um uh it's costly because academic medical centers have to have uh
academic medical centers have to have uh research coordinators people who pull
research coordinators people who pull data right so um it it it takes quite a
data right so um it it it takes quite a bit of time from there I will say once
bit of time from there I will say once um you have a commercial product
um you have a commercial product typically in the United States you'll be
typically in the United States you'll be able to De identify data um there's a
able to De identify data um there's a there's a little there's interesting
there's a little there's interesting interesting wrinkle with an FDA cleared
interesting wrinkle with an FDA cleared algorithm is that currently FDA doesn't
algorithm is that currently FDA doesn't allow that algorithm to improve in real
allow that algorithm to improve in real time right like if you use I don't know
time right like if you use I don't know like Gmail for example and it starts to
like Gmail for example and it starts to realize every time you sign your emails
realize every time you sign your emails with best regards it starts to suggest
with best regards it starts to suggest best regards right it doesn't have to be
best regards right it doesn't have to be trained with your data outside of that
trained with your data outside of that but for um cleared algorithms FDA
but for um cleared algorithms FDA cleared algorithms you have to train in
cleared algorithms you have to train in a non clinical setting so so when we
a non clinical setting so so when we engage our customers um healthc Care
engage our customers um healthc Care Systems like Stanford or or others uh we
Systems like Stanford or or others uh we get we negotiate the right to De
get we negotiate the right to De identify that data and train the
identify that data and train the algorithms and it's for the benefit of
algorithms and it's for the benefit of the Health Care system because their
the Health Care system because their data is being used to improve the
data is being used to improve the algorithm it's for the benefit of of viz
algorithm it's for the benefit of of viz and patience because the algorithm gets
and patience because the algorithm gets gets better and then you've got that
gets better and then you've got that potentially depending on the rights you
potentially depending on the rights you have you have that that data for other
have you have that that data for other purposes um but if you go to a whole new
purposes um but if you go to a whole new type of data like ultrasound right you'd
type of data like ultrasound right you'd have to somehow source that data and if
have to somehow source that data and if there was a way to do that to partner
there was a way to do that to partner with a company like yours that would be
with a company like yours that would be hugely beneficial because it's not easy
hugely beneficial because it's not easy to find the
to find the data Ben you talked a little bit about
data Ben you talked a little bit about how large data owners might have a
how large data owners might have a competitive Advantage here um can you
competitive Advantage here um can you share a little bit more about that
share a little bit more about that thinking do we see large data owners
thinking do we see large data owners taking advantage of their data access to
taking advantage of their data access to to do Health Care AI applications or to
to do Health Care AI applications or to disadvantage rivals in any way so any
disadvantage rivals in any way so any general thoughts there yeah I mean I I
general thoughts there yeah I mean I I think the answer is very likely yes
think the answer is very likely yes because they have the they have the
because they have the they have the incentive to do so uh and in some ways
incentive to do so uh and in some ways the fact that the data are proprietary
the fact that the data are proprietary make it harder to to assess whether
make it harder to to assess whether that's that's actually happening um I
that's that's actually happening um I think one good example of this is the um
think one good example of this is the um the United Healthcare merger with uh
the United Healthcare merger with uh change Healthcare where I was a uh
change Healthcare where I was a uh expert witness for the for the doj and
expert witness for the for the doj and there you know United was acquiring
there you know United was acquiring change and one of the aspects of this
change and one of the aspects of this merger is United is gaining access as a
merger is United is gaining access as a broad parent company to this great meta
broad parent company to this great meta data set that has lots of information on
data set that has lots of information on kind of the secret sauce rationing
kind of the secret sauce rationing Technologies of many of their Rivals
Technologies of many of their Rivals rival insurers so you can kind of name
rival insurers so you can kind of name all the other uh insurers you want and
all the other uh insurers you want and as you know by being able to acquire
as you know by being able to acquire that data set and own that data they
that data set and own that data they gain a serious competitive advantage in
gain a serious competitive advantage in terms of being able in real time to
terms of being able in real time to assess and learn about the strategies of
assess and learn about the strategies of their Rivals right and it for example if
their Rivals right and it for example if that merger didn't happen and change was
that merger didn't happen and change was an independent company it could be that
an independent company it could be that they're going to develop products that
they're going to develop products that are more publicly available or
are more publicly available or purchasable by all the
purchasable by all the insurers uh who can then use that to run
insurers uh who can then use that to run analytics for example on deidentified
analytics for example on deidentified data so I think it's kind of an
data so I think it's kind of an insurance side vers version of what zad
insurance side vers version of what zad was talking about and what we were
was talking about and what we were talking about in the health space but I
talking about in the health space but I think there's a real just kind of
think there's a real just kind of important discussion about you know what
important discussion about you know what data sets do we want to live in a
data sets do we want to live in a proprietary space under under companies
proprietary space under under companies and what data sets do we want to at
and what data sets do we want to at least live in a firm that's kind of
least live in a firm that's kind of independent of kind of the companies are
independent of kind of the companies are using it versus live under kind of a
using it versus live under kind of a more like regulator
more like regulator purview and to that point I wanted to
purview and to that point I wanted to ask what are the alternatives to
ask what are the alternatives to corporate Mass in of data in this
corporate Mass in of data in this space um is this should we be thinking
space um is this should we be thinking about as Po from a policy perspective
about as Po from a policy perspective making some of this data publicly
making some of this data publicly available through a regulator and I know
available through a regulator and I know CMS already does some of that so I
CMS already does some of that so I thought maybe Allison could talk a
thought maybe Allison could talk a little bit about that yeah so we do a
little bit about that yeah so we do a lot of work to make CMS data available
lot of work to make CMS data available for research um but it's the way the had
for research um but it's the way the had talked about you have to come through a
talked about you have to come through a data use agreement you have to tell us
data use agreement you have to tell us what research you're doing and also does
what research you're doing and also does a lot of work to make data available for
a lot of work to make data available for research but I think uh the role that
research but I think uh the role that Regulators can play is more what Elena
Regulators can play is more what Elena was talking about earlier the work that
was talking about earlier the work that CMS and onc have done to say Can giving
CMS and onc have done to say Can giving patients the ability to share their data
patients the ability to share their data with whoever they want to share it with
with whoever they want to share it with if they want their data to be used to
if they want their data to be used to support AI development great if they
support AI development great if they want to change their doctor they can
want to change their doctor they can take their data with them so allowing
take their data with them so allowing the flow of data encouraging the flow of
the flow of data encouraging the flow of data developing standards in this space
data developing standards in this space so that you know if you're working with
so that you know if you're working with one data set and you want to work with a
one data set and you want to work with a different data set you're not completely
different data set you're not completely rethinking your code development and all
rethinking your code development and all of that I think there's a lot more that
of that I think there's a lot more that that's kind of the space that we can
that's kind of the space that we can play in and allowing The Innovation to
play in and allowing The Innovation to happen that zad is doing for example
happen that zad is doing for example with night and Gale to say how do we
with night and Gale to say how do we open this data up how do we make it more
open this data up how do we make it more available how do we think about really
available how do we think about really Innovative techniques for de
Innovative techniques for de identification but also
identification but also you know to the point I was making early
you know to the point I was making early about earlier about privacy even when
about earlier about privacy even when the data is deidentified you're still
the data is deidentified you're still coming into an enclave you're still
coming into an enclave you're still signing an agreement saying you're not
signing an agreement saying you're not going to re-identify an individual so I
going to re-identify an individual so I think there's there's work that
think there's there's work that Regulators are doing to make data
Regulators are doing to make data available to encourage the seamless
available to encourage the seamless exchange of data but also allowing
exchange of data but also allowing Innovation to happen in the
Innovation to happen in the marketplace and I think we are running
marketplace and I think we are running out of time um do we do we have five
out of time um do we do we have five more minutes or we have one more minute
more minutes or we have one more minute okay do we need to take audience
okay do we need to take audience questions
questions now want to
now want to St okay well well let's see if the
St okay well well let's see if the audience has any questions we should
audience has any questions we should open it I won't hog these amazing
open it I won't hog these amazing panelists
panelists anymore but if you don't have questions
anymore but if you don't have questions I have
I have more okay well any anyone have a
more okay well any anyone have a question okay I wanted to ask um we
question okay I wanted to ask um we talked about data access as a a
talked about data access as a a bottleneck to competition I wanted to
bottleneck to competition I wanted to see are there other bottlenecks in the
see are there other bottlenecks in the space we should be thinking
space we should be thinking about I I I mean I think one that is
about I I I mean I think one that is that's pretty clear is data is kind of
that's pretty clear is data is kind of bottlenecks in in labor and
bottlenecks in in labor and sophisticated labor so um you know data
sophisticated labor so um you know data data Engineers data analysts uh you know
data Engineers data analysts uh you know a lot of the biggest companies get
a lot of the biggest companies get access to a lot of the best data
access to a lot of the best data engineers and data analysts and you know
engineers and data analysts and you know their expertise is crucial in developing
their expertise is crucial in developing these tools and so I think that's a
these tools and so I think that's a potential
potential bottleneck I think there's a particular
bottleneck I think there's a particular kind of expertise that in these domains
kind of expertise that in these domains like health or law or um hiring or
like health or law or um hiring or wherever AI is being used you actually
wherever AI is being used you actually need people who are kind of bilingual
need people who are kind of bilingual who understand that domain but also
who understand that domain but also understand how to do all of the data
understand how to do all of the data stuff and I think our educational system
stuff and I think our educational system is not very good at producing those
is not very good at producing those kinds of people unfortunately so ways to
kinds of people unfortunately so ways to uh get more of those um bying people I
uh get more of those um bying people I think would be really useful I'm really
think would be really useful I'm really optimistic because of work like what
optimistic because of work like what Allison is doing to standardize data and
Allison is doing to standardize data and make it more open um to address the
make it more open um to address the bottleneck in terms of you know data
bottleneck in terms of you know data analysis data engineers and then with
analysis data engineers and then with Next Generation models you don't need as
Next Generation models you don't need as much data like the early days of viai or
much data like the early days of viai or AI do was all about the painstaking
AI do was all about the painstaking aggregation and then manual in some
aggregation and then manual in some cases annotation of data and so with
cases annotation of data and so with Next Generation models with llms with
Next Generation models with llms with generative AI you can have zero shot or
generative AI you can have zero shot or small shot algorithm development and so
small shot algorithm development and so data becomes less of a bottleneck to the
data becomes less of a bottleneck to the application layer it's the bottleneck to
application layer it's the bottleneck to the base layer models but as Healthcare
the base layer models but as Healthcare people I think like it's an incredibly
people I think like it's an incredibly exciting time for application layer
and do patients have this vast knowledge of their data and the data the algorithm
of their data and the data the algorithm and whether they can transfer it or when
and whether they can transfer it or when they change Insurance especially you
they change Insurance especially you know the older populations I think uh
know the older populations I think uh there's a lot of role for just
there's a lot of role for just disseminating that information maybe uh
disseminating that information maybe uh to the patients I'm not sure if that's a
to the patients I'm not sure if that's a yeah I agree I think it is something
yeah I agree I think it is something that there's there's starting to be
that there's there's starting to be awareness that patients have this
awareness that patients have this ownership of their data and their right
ownership of their data and their right the right to take their data with them
the right to take their data with them but it is something that we need to
but it is something that we need to continue to do education on Medicare for
continue to do education on Medicare for example has a a service that we call
example has a a service that we call Blue Buton 2.0 which allows Medicare
Blue Buton 2.0 which allows Medicare beneficiaries to direct their data to
beneficiaries to direct their data to any application or service that they
any application or service that they want to use we haven't seen a huge
want to use we haven't seen a huge uptake especially among the you know the
uptake especially among the you know the Medicare age population who maybe aren't
Medicare age population who maybe aren't as as digitally savvy but we are
as as digitally savvy but we are starting to see growth in those types of
starting to see growth in those types of programs as we do more education and as
programs as we do more education and as more applications come on board and do
more applications come on board and do Outreach to beneficiaries offering their
services I want to thank the panelists so much for very eliminating insights
so much for very eliminating insights today I wish we had more time to talk we
today I wish we had more time to talk we could talk all afternoon but we have a
could talk all afternoon but we have a lot of other interesting panels so thank
lot of other interesting panels so thank you so much please join me in giving
you so much please join me in giving them a big round of applause
you all right we're going to move on to our next uh panel of the day after we
our next uh panel of the day after we studied our Healthcare applications
studied our Healthcare applications we're going to study uh and learn more
we're going to study uh and learn more about uh chips and Hardware that can
about uh chips and Hardware that can make some of these applications run and
make some of these applications run and we're very uh pleased to Welcome to the
we're very uh pleased to Welcome to the stage we have our uh two uh uh folks
stage we have our uh two uh uh folks that will be engaging in a fire side
that will be engaging in a fire side chat um first we have um Paula blizzard
chat um first we have um Paula blizzard who is the senior uh Assistant Attorney
who is the senior uh Assistant Attorney General in the antitrust section of the
General in the antitrust section of the California AG's office and she will
California AG's office and she will having be having this discussion with
having be having this discussion with the Victor Pang who is president at AMD
the Victor Pang who is president at AMD and has very deep knowledge of the chip
and has very deep knowledge of the chip industry so please let me welcome uh
industry so please let me welcome uh Victor and uh thank him for uh talking
Victor and uh thank him for uh talking to us about uh hardware and uh chips in
to us about uh hardware and uh chips in this space and uh then we will be joined
this space and uh then we will be joined by the rest of our panel uh discussion
by the rest of our panel uh discussion um in about 15 minutes so uh thank you
um in about 15 minutes so uh thank you Victor thank
Victor thank you thank you very much thank you to
you thank you very much thank you to Stanford and the US Department of
Stanford and the US Department of Justice for organizing this um as uh was
Justice for organizing this um as uh was mentioned I'm with the California
mentioned I'm with the California government um we are obviously very
government um we are obviously very focused and invested and excited about
focused and invested and excited about everything thing going on uh with AI
everything thing going on uh with AI here in
here in California um Victor Pang is the
California um Victor Pang is the president of AMD as was said he has um
president of AMD as was said he has um 40 years of experience in Innovative
40 years of experience in Innovative hardware and Chip development and he's
hardware and Chip development and he's responsible for the company's AI
responsible for the company's AI strategy um thank you for coming Victor
strategy um thank you for coming Victor thank you very much it's great to be
thank you very much it's great to be here um so hardware and chips have again
here um so hardware and chips have again a bit taken Center Stage they are
a bit taken Center Stage they are crucial to all the things uh that make
crucial to all the things uh that make these large language models run um and
these large language models run um and yet they are at the bottom of the stack
yet they are at the bottom of the stack those of you who think about Stacks
those of you who think about Stacks right the software at chat gpts at the
right the software at chat gpts at the top the chips at the bottom okay most of
top the chips at the bottom okay most of us will never touch one we'll never see
us will never touch one we'll never see one but they're super important so
one but they're super important so Victor help us out here give us a little
Victor help us out here give us a little bit of an overview what types of chips
bit of an overview what types of chips are important what do they do give just
are important what do they do give just a little primer on some of the acronyms
a little primer on some of the acronyms would be great yeah so for for some of
would be great yeah so for for some of you who um already can follow stochastic
you who um already can follow stochastic gradian descent the bear with me because
gradian descent the bear with me because it'll be a little bit more basic um you
it'll be a little bit more basic um you know look I think most people have heard
know look I think most people have heard about CPUs and certainly AMD you know
about CPUs and certainly AMD you know and and others have been uh in decades
and and others have been uh in decades in that business but you can think of
in that business but you can think of the CPU as a general jack of all trades
the CPU as a general jack of all trades right it could run all kinds of
right it could run all kinds of applications and software um everything
applications and software um everything from things that are more consumer like
from things that are more consumer like uh like on your your notebook or your
uh like on your your notebook or your laptop top you know web browsers your
laptop top you know web browsers your personal productivity video streaming or
personal productivity video streaming or in the server in the data centers
in the server in the data centers running transaction processing or other
running transaction processing or other kinds of more Enterprise kind of
kinds of more Enterprise kind of applications right so now you hear about
applications right so now you hear about gpus and gpus stands for graphics
gpus and gpus stands for graphics Processing Unit um so and you you sort
Processing Unit um so and you you sort of say well you know it's for AI so why
of say well you know it's for AI so why is it you know call Graphics Well turns
is it you know call Graphics Well turns out that you know graphics and AI um
out that you know graphics and AI um from a workload perspective has
from a workload perspective has similarities it's very um linear Al very
similarities it's very um linear Al very intensive um lots of Matrix
intensive um lots of Matrix multiplications it's also extremely
multiplications it's also extremely parallel um lots of data movement lots
parallel um lots of data movement lots of high bandwidth so it turns out
of high bandwidth so it turns out initially um before AI really exploded
initially um before AI really exploded on the scene researchers were doing a
on the scene researchers were doing a lot of uh development on Graphics gpus
lot of uh development on Graphics gpus right um and just confusing a little bit
right um and just confusing a little bit further like today we have uh two
further like today we have uh two different families of Graphics one
different families of Graphics one that's still for mainly for graphics
that's still for mainly for graphics processing gaming and and and
processing gaming and and and visualization and
visualization and but then we have another family that's
but then we have another family that's concentrated really on performing really
concentrated really on performing really large um deployment scale out extremely
large um deployment scale out extremely large and also in high performance
large and also in high performance Computing and this they're both qu gpus
Computing and this they're both qu gpus but in any event um then then you you've
but in any event um then then you you've probably heard of other kind of AI
probably heard of other kind of AI accelerat like tpus um you know lots of
accelerat like tpus um you know lots of different startups that have many
different startups that have many different architectures they're all
different architectures they're all broadly speaking a form of accelerator
broadly speaking a form of accelerator for AI applications right so um the the
for AI applications right so um the the other thing I would say say from
other thing I would say say from semiconductors to click down one level
semiconductors to click down one level two is that when you have these massive
two is that when you have these massive deployments of you know tens to hundreds
deployments of you know tens to hundreds of thousands and people are talking
of thousands and people are talking about you know millions of gpus um at
about you know millions of gpus um at that point too you also really need to
that point too you also really need to have a really good networking
have a really good networking architecture because you're taking these
architecture because you're taking these models or you're taking know large
models or you're taking know large workloads and you're actually dividing
workloads and you're actually dividing it across the entire uh data center you
it across the entire uh data center you know multiple Ikeas size buildings large
know multiple Ikeas size buildings large and you know your Bic um often times at
and you know your Bic um often times at that point is is more the network as
that point is is more the network as opposed to the compute on on giving node
opposed to the compute on on giving node um so there is a lot of complexity and
um so there is a lot of complexity and and maybe the the last thing I'll say
and maybe the the last thing I'll say even though there's a good deal of
even though there's a good deal of summarization going on in Andrew's talk
summarization going on in Andrew's talk which makes sense you know he just
which makes sense you know he just showed at the very bottom of the layer
showed at the very bottom of the layer as you say but even at that you know I
as you say but even at that you know I talked a lot about Hardware but there's
talked a lot about Hardware but there's a tremendous amount of software and
a tremendous amount of software and firmware you know embedded software so
firmware you know embedded software so to speak that makes that work otherwise
to speak that makes that work otherwise you really can't get any useful work out
you really can't get any useful work out of it so um these are systems that are
of it so um these are systems that are both hardware and software and it's all
both hardware and software and it's all really important to actually get the
really important to actually get the endtoend uh performance that you need
endtoend uh performance that you need yeah super interesting Okay so we've got
yeah super interesting Okay so we've got these Ikeas sized buildings full of
these Ikeas sized buildings full of billions upon billions of these little
billions upon billions of these little chips and they're all somehow being
chips and they're all somehow being coordinated with some software and some
coordinated with some software and some firmware um and then it goes on up to
firmware um and then it goes on up to ultimately sort of at some point get to
ultimately sort of at some point get to chat GPT and we're putting in little
chat GPT and we're putting in little little queries but let me ask you do do
little queries but let me ask you do do all of these things have to be
all of these things have to be integrated really well to work together
integrated really well to work together and what I'm thinking of is do we need
and what I'm thinking of is do we need one company that vertically integrates
one company that vertically integrates from the chip all the way up or are
from the chip all the way up or are there certain sort of demarcation
there certain sort of demarcation points yeah you know I I think um you
points yeah you know I I think um you know going back to the fact that you
know going back to the fact that you know AI is such a full stack you know
know AI is such a full stack you know and Andrew talked about the application
and Andrew talked about the application Level and and other middle layers and
Level and and other middle layers and you know the whole model was just one
you know the whole model was just one layer closest to the foundational
layer closest to the foundational Hardware layer um and as you said if you
Hardware layer um and as you said if you really you know poke down one more
really you know poke down one more there's actually more detail I guess
there's actually more detail I guess what I'd say is uh this is what makes
what I'd say is uh this is what makes both AI very interesting but also very
both AI very interesting but also very complex is you know we actually do have
complex is you know we actually do have to be aware certainly of the models but
to be aware certainly of the models but even higher up in in the stack and to
even higher up in in the stack and to make everything perform and we have to
make everything perform and we have to be aware to to Really devise a really
be aware to to Really devise a really good GPU and by the way CPUs are also
good GPU and by the way CPUs are also needed um it's it's GPU rich but you
needed um it's it's GPU rich but you also need CPUs and then you need the
also need CPUs and then you need the networking as I mentioned um
networking as I mentioned um so I guess what I say to have a
so I guess what I say to have a performant overall an system there has
performant overall an system there has to be at minimum kind of what I would
to be at minimum kind of what I would say co-optimization or or co-
say co-optimization or or co- understanding so it could be delivered
understanding so it could be delivered fully integrated but it should not only
fully integrated but it should not only be delivered that way um because you
be delivered that way um because you know basically you know the you could
know basically you know the you could have Innovation at every level and
have Innovation at every level and Andrew talked about that like it's
Andrew talked about that like it's tremendous activity in multiple levels
tremendous activity in multiple levels um and that's a really good thing
um and that's a really good thing because that drives you know better uh
because that drives you know better uh delivery uh lower cost higher
delivery uh lower cost higher performance enabling more powerful
performance enabling more powerful models enabling and performance just to
models enabling and performance just to make that concrete for everybody the
make that concrete for everybody the speed at which you get a response from
speed at which you get a response from your prompt that that's inference
your prompt that that's inference performance right and the amount of time
performance right and the amount of time and cost it takes you to train a new
and cost it takes you to train a new model maybe to do um you know bit more
model maybe to do um you know bit more accurate Imaging analysis of MRIs you
accurate Imaging analysis of MRIs you know that's a training kind of uh
know that's a training kind of uh performance level so it does matter
performance level so it does matter right um and so um the other thing I
right um and so um the other thing I would say is so you really should
would say is so you really should allow Innovation at every point and I
allow Innovation at every point and I think you know we've got tremendous
think you know we've got tremendous expertise in in AMD and and other
expertise in in AMD and and other companies but I I think it's hard to
companies but I I think it's hard to imagine that you know you would have the
imagine that you know you would have the best at every point in time at every
best at every point in time at every level of that stack so I do think it's
level of that stack so I do think it's important from an innovation and
important from an innovation and competitive and just forward progress to
competitive and just forward progress to sort of enable multiple players and uh
sort of enable multiple players and uh Andrew also mentioned open source you
Andrew also mentioned open source you know we we are well we're a big company
know we we are well we're a big company and we're for profit uh we embrace the
and we're for profit uh we embrace the open source from a perspective of of our
open source from a perspective of of our software stack where we open source a
software stack where we open source a tremendous amount of our software stack
tremendous amount of our software stack now there's still elements in areas
now there's still elements in areas where it's it's really super core to our
where it's it's really super core to our business that we don't necessarily do
business that we don't necessarily do that but I I just want to share that an
that but I I just want to share that an example of like not every big company
example of like not every big company that's that has a significant um you
that's that has a significant um you know stake in AI is taking the approach
know stake in AI is taking the approach sure everything is fully proprietary our
sure everything is fully proprietary our strategy in fact is General to be even
strategy in fact is General to be even if it's not open source the partner with
if it's not open source the partner with multiple other uh players like we're
multiple other uh players like we're doing that on the networking space right
doing that on the networking space right so we're enabling you know an ecosystem
so we're enabling you know an ecosystem of multiple players and we're enabling
of multiple players and we're enabling in that sense choice and I also think it
in that sense choice and I also think it drives you know Innovation more rapidly
drives you know Innovation more rapidly all right well I am all for more
all right well I am all for more Innovation but um it sounds like the
Innovation but um it sounds like the systems are massively huge physically as
systems are massively huge physically as well as um complicated and it needs to
well as um complicated and it needs to all work together really well so when
all work together really well so when you type in your prompt you get a fast
you type in your prompt you get a fast answer that is also accurate and has
answer that is also accurate and has been uh the system has been trained well
been uh the system has been trained well and yet to get the Innovation it has to
and yet to get the Innovation it has to be open at points so give us some more
be open at points so give us some more examples of how where in this stock it
examples of how where in this stock it can be more open um and uh I'll just
can be more open um and uh I'll just throw the last question to you a bit so
throw the last question to you a bit so should the government do that or this
should the government do that or this are the companies going to do it how's
are the companies going to do it how's AMD doing it and how are you looking at
AMD doing it and how are you looking at this problem well you know again we we
this problem well you know again we we are taking a strategy of um you know
are taking a strategy of um you know being enabling an ecosystem and working
being enabling an ecosystem and working with uh ecosystems and other partners
with uh ecosystems and other partners and you know it's not all you know in
and you know it's not all you know in the in the purest form open source but
the in the purest form open source but it is enabling multiple players and also
it is enabling multiple players and also interoperability I think you mentioned
interoperability I think you mentioned that before I didn't directly address
that before I didn't directly address that that's the other thing is even if
that that's the other thing is even if you do um feel like you have a turnkey
you do um feel like you have a turnkey solution and some customers want a
solution and some customers want a turnkey solution many customers have
turnkey solution many customers have their own special needs there really
their own special needs there really isn't a one-size fits-all and the other
isn't a one-size fits-all and the other thing is that this is a very very fast
thing is that this is a very very fast moving uh technology in Marketplace at
moving uh technology in Marketplace at every level of that full stack so I
every level of that full stack so I think you know I would say you know
think you know I would say you know enabling that um in the government's
enabling that um in the government's role what I would say is like you know
role what I would say is like you know you see how the Market's performing if
you see how the Market's performing if it's doing its right thing and
it's doing its right thing and competition is um you know it's a fair
competition is um you know it's a fair and Level Playing Field like you know we
and Level Playing Field like you know we we have leadership products you know we
we have leadership products you know we believe we're going to win that um and
believe we're going to win that um and we're taking a different approach a more
we're taking a different approach a more open approach and you know bringing
open approach and you know bringing other uh Co companies along with us um
other uh Co companies along with us um and we get really good feedback quite
and we get really good feedback quite frankly from a lot of our customers on
frankly from a lot of our customers on that um the key thing there though is
that um the key thing there though is that if it for whatever reason becomes
that if it for whatever reason becomes less of a Level Playing Field then I
less of a Level Playing Field then I think there is certainly a role for
think there is certainly a role for government and in one sense it's not
government and in one sense it's not different than what government has
different than what government has always done but I think the the
always done but I think the the additional challenge is it is very
additional challenge is it is very complex and it's changing very rapidly
complex and it's changing very rapidly and you know I completely agree with
and you know I completely agree with Andrew and it's interesting I've
Andrew and it's interesting I've interactions with Andrew but usually not
interactions with Andrew but usually not in this kind of
in this kind of context so so we never we didn't share
context so so we never we didn't share notes but you know his thing about uh
notes but you know his thing about uh the fact of you know look at what's
the fact of you know look at what's appropriate for the application and what
appropriate for the application and what the challenge the use case if you will
the challenge the use case if you will and then also uh Percy's comment about
and then also uh Percy's comment about marginal value versus marginal risk um
marginal value versus marginal risk um in fact internally we we've adopted
in fact internally we we've adopted because we have a responsible AI as well
because we have a responsible AI as well uh as you know within as a function
uh as you know within as a function within uh AMD and we look at both what
within uh AMD and we look at both what we're doing outbound technology we also
we're doing outbound technology we also look at how we use technology internally
look at how we use technology internally we we use the N for uh framework for
we we use the N for uh framework for risk identification and same thing it's
risk identification and same thing it's like you know there's certain risks if
like you know there's certain risks if you're talking about Medical Imaging in
you're talking about Medical Imaging in people's Health outcomes that that you
people's Health outcomes that that you know requires higher rigor perhaps
know requires higher rigor perhaps potentially higher guard rails but at
potentially higher guard rails but at the same time you want to enable data
the same time you want to enable data right so it doesn't mean everything is
right so it doesn't mean everything is you know it's a sort of um control and
you know it's a sort of um control and other areas where you know the outcome
other areas where you know the outcome of a recommending you know you didn't
of a recommending you know you didn't get my movie preference just right like
get my movie preference just right like you know okay you don't need to regulate
you know okay you don't need to regulate that too heavily so I think being uh
that too heavily so I think being uh commerate with the risk factor and also
commerate with the risk factor and also you know measuring uh marginal gain and
you know measuring uh marginal gain and marginal risk those are important things
marginal risk those are important things and just seeing if the marketplace is a
and just seeing if the marketplace is a level or not excellent thank you okay so
level or not excellent thank you okay so complicated for government as well
complicated for government as well because we're going to have to do it on
because we're going to have to do it on a risk basis one size is one regulation
a risk basis one size is one regulation will not fit all and whether you like
will not fit all and whether you like the Barbie or Oppenheimer or whether you
the Barbie or Oppenheimer or whether you need to get and MRI are probably very
need to get and MRI are probably very different questions um the time for our
different questions um the time for our little fireside chat is over so now I'm
little fireside chat is over so now I'm going to invite um or Jennifer is going
going to invite um or Jennifer is going to invite the rest of our panelists up
to invite the rest of our panelists up but we will continue the discussion um
but we will continue the discussion um about hardware and
about hardware and chips thank you Victor welcome we're
chips thank you Victor welcome we're going to do a little uh reshuffling here
going to do a little uh reshuffling here uh as as politics takes the seat of
uh as as politics takes the seat of moderator and we're going to welcome up
moderator and we're going to welcome up our panelists as I said all their full
our panelists as I said all their full full bios are uh available online um but
full bios are uh available online um but I will I will at least welcome them with
I will I will at least welcome them with their titles our our first panelist is
their titles our our first panelist is Chris wolf he's the global head of AI um
Chris wolf he's the global head of AI um and Advanced Services uh by by VMware at
and Advanced Services uh by by VMware at broadcom uh we have missar amiman
broadcom uh we have missar amiman founder and CEO of occi labs we have
founder and CEO of occi labs we have Alex Gainer is a colle at the FTC he's
Alex Gainer is a colle at the FTC he's the chief uh deputy chief technologist
the chief uh deputy chief technologist there and we do have a uh participant by
there and we do have a uh participant by by Zoom uh blanch uh saur uh board she's
by Zoom uh blanch uh saur uh board she's general counsel and Secretary of the
general counsel and Secretary of the board at mrol AI so she will be joining
board at mrol AI so she will be joining us by video um and the rest of our
us by video um and the rest of our panelists are up here on stage so I will
panelists are up here on stage so I will leave it to Paula thank
leave it to Paula thank you thank you very much welcome to my uh
you thank you very much welcome to my uh distinguished panel welcome to blunch as
distinguished panel welcome to blunch as well hello um
well hello um uh we were just chatting um with Victor
uh we were just chatting um with Victor Pang of AMD about hardware and chips and
Pang of AMD about hardware and chips and now we're going to get some other
now we're going to get some other perspectives we're going to broaden it
perspectives we're going to broaden it um to see what other players and
um to see what other players and stakeholders in the market um are
stakeholders in the market um are thinking about and what I'd like to do
thinking about and what I'd like to do is just go to um each panelist and give
is just go to um each panelist and give them a minute to tell you about the
them a minute to tell you about the position that they are coming from um
position that they are coming from um their initial thoughts a little bit
their initial thoughts a little bit about their company and what role um
about their company and what role um they're playing in the development vment
they're playing in the development vment of hardware and chips so Chris um Chris
of hardware and chips so Chris um Chris wolf Global head of AI and advanced
wolf Global head of AI and advanced systems VMware by broadcom give us some
systems VMware by broadcom give us some thoughts okay thanks for the thanks for
thoughts okay thanks for the thanks for the opportunity and I would say I think
the opportunity and I would say I think most of you are probably familiar with
most of you are probably familiar with with broadcom leader and semiconductors
with broadcom leader and semiconductors and also a lot of data center software
and also a lot of data center software from security to Mainframe to
from security to Mainframe to virtualization I'd say one thing that's
virtualization I'd say one thing that's been unique about my role is that we in
been unique about my role is that we in addition to Leading the product
addition to Leading the product development side for AI we also operate
development side for AI we also operate the internal AI services and that's
the internal AI services and that's driven our product strategy so some some
driven our product strategy so some some examples we run a chat service for
examples we run a chat service for product support and uh we're seeing just
product support and uh we're seeing just incredible efficiencies and accuracy uh
incredible efficiencies and accuracy uh from that particular service and this is
from that particular service and this is using an open source model but the meta
using an open source model but the meta point was when we started running our
point was when we started running our own internal AI Services we realized
own internal AI Services we realized that it would be difficult to take a uh
that it would be difficult to take a uh single beted on one one vendor or
single beted on one one vendor or technology provider we wanted to take a
technology provider we wanted to take a platform approach that would allow us to
platform approach that would allow us to very quickly on board uh new AI services
very quickly on board uh new AI services and new AI models at the speed of
and new AI models at the speed of software our Customer Support Service as
software our Customer Support Service as an example has had its foundation model
an example has had its foundation model changed three times in the last nine
changed three times in the last nine months uh because there's been better
months uh because there's been better technology that's come along and that's
technology that's come along and that's what's led us to what we call Private AI
what's led us to what we call Private AI which is about bringing the AI model to
which is about bringing the AI model to where your data is uh you can you can
where your data is uh you can you can see a lot of efficiencies that way you
see a lot of efficiencies that way you can maintain privacy and control of your
can maintain privacy and control of your data while still gaining the benefits of
data while still gaining the benefits of AI and we've seen this really resonate
AI and we've seen this really resonate with our customer base and that's where
with our customer base and that's where we uh are pushing some new solutions
we uh are pushing some new solutions that we have that
that we have that it's not just a A broadcom or VMware
it's not just a A broadcom or VMware perspective we see this as really
perspective we see this as really industry and there's lots of ways to
industry and there's lots of ways to gain these similar benefits but it
gain these similar benefits but it doesn't have to be a trade-off I'd say
doesn't have to be a trade-off I'd say is my first point and the second is you
is my first point and the second is you you can gain these benefits with
you can gain these benefits with probably less gpus than you think like
probably less gpus than you think like Percy mentioned retrieval augmented
Percy mentioned retrieval augmented generation this morning one of our
generation this morning one of our Flagship Services is running on 4 gpus
Flagship Services is running on 4 gpus and uh this is you know when you bring
and uh this is you know when you bring in Rag and inference you can see really
in Rag and inference you can see really incredible results with the smaller
incredible results with the smaller investment and often times more
investment and often times more predictable costs and a lower carbon
predictable costs and a lower carbon footprint so there's there's a lot out
footprint so there's there's a lot out there I mean there's lots of it's it's
there I mean there's lots of it's it's not about you know AI being good or bad
not about you know AI being good or bad is about how are you approach an AI and
is about how are you approach an AI and what's best for the organization but
what's best for the organization but there's a lot of choice that you can you
there's a lot of choice that you can you can take on to do things differently
can take on to do things differently great thank you okay forget the Ikeas
great thank you okay forget the Ikeas sized rooms we can do it with four gpus
sized rooms we can do it with four gpus which are probably in the size of your
which are probably in the size of your phone um uh let me jump over um to maer
phone um uh let me jump over um to maer Meo founder and CEO of oy Labs formerly
Meo founder and CEO of oy Labs formerly at bit fusion and VMware what are your
at bit fusion and VMware what are your opening thoughts uh yeah Maz M at a Labs
opening thoughts uh yeah Maz M at a Labs um so our mission is to quantify the
um so our mission is to quantify the world um so as you can imagine AI is
world um so as you can imagine AI is sort of everywhere that's sort of the
sort of everywhere that's sort of the buzzword and I think we're at a very
buzzword and I think we're at a very interesting time where we can actually
interesting time where we can actually drive efficiencies in sectors that don't
drive efficiencies in sectors that don't see a lot of efficiency so one of the
see a lot of efficiency so one of the first areas that we're looking at is in
first areas that we're looking at is in the charitable nonprofit
the charitable nonprofit space um so I don't know if you know
space um so I don't know if you know this is a huge space about2 trillion go
this is a huge space about2 trillion go through uh us-based nonprofits every
through uh us-based nonprofits every year $2 trillion um of the $2 trillion
year $2 trillion um of the $2 trillion $500 billion goes to charitable
$500 billion goes to charitable nonprofits so uh places that are not
nonprofits so uh places that are not hospitals or insurance companies um and
hospitals or insurance companies um and of the $500 billion $180 billion only go
of the $500 billion $180 billion only go to program expenses so that's 36% and
to program expenses so that's 36% and who knows how much of that is actually
who knows how much of that is actually effective maybe 10 or 20% and so this is
effective maybe 10 or 20% and so this is U one area where um you know we've
U one area where um you know we've engineered a vision system that allows
engineered a vision system that allows us to um basically capture human
us to um basically capture human development indicators food production
development indicators food production Water Production um literacy rate um how
Water Production um literacy rate um how much milk is being produced by a goat
much milk is being produced by a goat farm for example and we think that
farm for example and we think that creating this feedback loop will make
creating this feedback loop will make charal giving way more impactful um so
charal giving way more impactful um so that's kind of what we're looking on and
that's kind of what we're looking on and kind of answer your question about uh
kind of answer your question about uh competition um so we we have like a lot
competition um so we we have like a lot of uh thoughts on this I would kind of
of uh thoughts on this I would kind of crystallize it to three one is that AI
crystallize it to three one is that AI is not a differentiator and there's very
is not a differentiator and there's very little value capture from AI
little value capture from AI functionality intelligence or uh machine
functionality intelligence or uh machine learning or data science um and similar
learning or data science um and similar to that is that software is also not a
to that is that software is also not a differentiator it's sort of an expected
differentiator it's sort of an expected outcome the marginal cost of creating a
outcome the marginal cost of creating a new line of code is approaching zero and
new line of code is approaching zero and the marginal cost of creating software
the marginal cost of creating software that creates software is also
that creates software is also approaching zero so what do we take from
approaching zero so what do we take from that um we are not an AI company as a
that um we are not an AI company as a rule uh but we will use uh Ai and
rule uh but we will use uh Ai and software uh to accelerate our
software uh to accelerate our engineering so agente AI we accelerate
engineering so agente AI we accelerate our engineering our testing our
our engineering our testing our robustness in our in our services and of
robustness in our in our services and of course uh we uh use code generators uh
course uh we uh use code generators uh as fast as possible because as a startup
as fast as possible because as a startup we're trying to find the truth as as
we're trying to find the truth as as soon as possible right uh you know
soon as possible right uh you know satisfy a human need and see if you can
satisfy a human need and see if you can extract value from the market um so
extract value from the market um so that's a second rule third rule is deep
that's a second rule third rule is deep Tech is probably uh can be accelerated
Tech is probably uh can be accelerated that I think this is where a lot of
that I think this is where a lot of innovation and opportunity is so um do
innovation and opportunity is so um do we need Better Health Care yes do we
we need Better Health Care yes do we need to increase our lifespan yes do we
need to increase our lifespan yes do we need uh access to better food yes do we
need uh access to better food yes do we need to clean our environment yes these
need to clean our environment yes these are all deep Tech problems that could be
are all deep Tech problems that could be accelerated with AI and I think that's
accelerated with AI and I think that's where a lot of the Innovation and
where a lot of the Innovation and opportunities are excellent thank you
opportunities are excellent thank you very much um ai ai is not a
very much um ai ai is not a differentiator I thought we were here to
differentiator I thought we were here to talk about how it was um and um but we
talk about how it was um and um but we got in marginal costs so I'm certain
got in marginal costs so I'm certain that Susan is Happy um let me jump
that Susan is Happy um let me jump across the ocean all the way to France
across the ocean all the way to France um blanch thank you so much for joining
um blanch thank you so much for joining us and staying up late a bit um please
us and staying up late a bit um please give us a little bit of background about
give us a little bit of background about mrol and some opening thoughts and thank
mrol and some opening thoughts and thank you very much for having me um so yeah
you very much for having me um so yeah I'm I'm General councel and Secretary of
I'm I'm General councel and Secretary of the board as at mistal which is m a
the board as at mistal which is m a French company that has been founded a
French company that has been founded a year ago and that ALS also gives a sense
year ago and that ALS also gives a sense of how fast this environment is evolving
of how fast this environment is evolving uh because in a year we've been able to
uh because in a year we've been able to build and launch several uh Frontier
build and launch several uh Frontier models we've been able to uh do two
models we've been able to uh do two series uh to fund
series uh to fund fundraisings and uh and and a lot more
fundraisings and uh and and a lot more including launching a platform and and
including launching a platform and and building several Partnerships with some
building several Partnerships with some of which are public like Microsoft
of which are public like Microsoft iws or or snowflake so um that's going
iws or or snowflake so um that's going very fast we're only 55 right now so two
very fast we're only 55 right now so two months ago we were 35 and and that also
months ago we were 35 and and that also gives I think a sense of what it is to
gives I think a sense of what it is to be a startup in AI that has the ambition
be a startup in AI that has the ambition to compete against Giants because
to compete against Giants because because it is what it is uh so the
because it is what it is uh so the funders came from those giants because
funders came from those giants because they came from meta and and from Deep
they came from meta and and from Deep Mind and they wanted to to build
Mind and they wanted to to build something a bit different uh in Europe
something a bit different uh in Europe uh with the talents that we have also uh
uh with the talents that we have also uh in Europe uh and with a lot of funding
in Europe uh and with a lot of funding including a lot of American Funding
including a lot of American Funding which which is reality so I think it's
which which is reality so I think it's also important to to note that uh so um
also important to to note that uh so um what is interesting about M trial is uh
what is interesting about M trial is uh we have no connection to uh to any
we have no connection to uh to any pre-existing actors in the tech industry
pre-existing actors in the tech industry uh so we of course have certain tickets
uh so we of course have certain tickets in the fundraising of some of these
in the fundraising of some of these actors and we have some Partnerships
actors and we have some Partnerships with them but we have no connection in
with them but we have no connection in the sense that we are we are a pure
the sense that we are we are a pure player we are not integrated on on the
player we are not integrated on on the stack and we are in the middle of the
stack and we are in the middle of the value chain uh building uh building
value chain uh building uh building Frontier models and that makes our
Frontier models and that makes our position super interesting because of
position super interesting because of course we have a lot of work to create
course we have a lot of work to create the good relationships to get everything
the good relationships to get everything including the compute that is the topic
including the compute that is the topic of today to get everything to to make it
of today to get everything to to make it work uh but we are also in a very good
work uh but we are also in a very good position to be available everywhere
position to be available everywhere where the customers want us to be and to
where the customers want us to be and to be deployed the way they want us to be
be deployed the way they want us to be deployed so we Deploy on premise we
deployed so we Deploy on premise we Deploy on clouds we Deploy on our
Deploy on clouds we Deploy on our platforms and and many other deployments
platforms and and many other deployments could be could be considered so that's
could be could be considered so that's uh that's also a key differentiator that
uh that's also a key differentiator that makes our position interesting from a
makes our position interesting from a competition standpoint um and of course
competition standpoint um and of course and I heard a lot earlier about
and I heard a lot earlier about commitment Victor's commitment to open
commitment Victor's commitment to open source and I think we can can say the
source and I think we can can say the same some some of the models we we
same some some of the models we we launched have been on aachi too and we
launched have been on aachi too and we will keep doing that and uh we launched
will keep doing that and uh we launched yesterday cod cod estral a model
yesterday cod cod estral a model dedicated for code that was under our
dedicated for code that was under our first uh draft um mixed license meaning
first uh draft um mixed license meaning that the weights are open but uh it's
that the weights are open but uh it's for testing so so that's also an
for testing so so that's also an interesting experiment for for this
interesting experiment for for this model uh and we've had already a lot of
model uh and we've had already a lot of adoption so that's that's what we do so
adoption so that's that's what we do so um yeah I think this is what I wanted to
um yeah I think this is what I wanted to say about M maybe to finish on this
say about M maybe to finish on this introduction uh we've had already a lot
introduction uh we've had already a lot of interactions with Regulators in
of interactions with Regulators in Europe uh so I think uh Mo probably most
Europe uh so I think uh Mo probably most of you have but because those Regulators
of you have but because those Regulators are very active these days and but we
are very active these days and but we have been at the Forefront of that
have been at the Forefront of that experiment too and uh the challenges uh
experiment too and uh the challenges uh this environment is going very fast so
this environment is going very fast so the challenges are only going to grow
the challenges are only going to grow very fast but Regulators have to are
very fast but Regulators have to are considering acting fast or reacting fast
considering acting fast or reacting fast in a world where it's very difficult to
in a world where it's very difficult to anticipate the side effects of of those
anticipate the side effects of of those actions and reactions so uh I completely
actions and reactions so uh I completely sympathize with with the uh the effort
sympathize with with the uh the effort that is really super important and and
that is really super important and and I'm willing to contribute to how we can
I'm willing to contribute to how we can think of a an action that makes a lot of
think of a an action that makes a lot of sense for for keeping this industry very
sense for for keeping this industry very open thanks a lot great thank thank you
open thanks a lot great thank thank you love the perspective congratulations on
love the perspective congratulations on getting a product out yesterday um uh
getting a product out yesterday um uh okay let me turn briefly to Alex Alec
okay let me turn briefly to Alex Alec Gainer is a deputy chief technologist at
Gainer is a deputy chief technologist at the FTC the FTC and many other agencies
the FTC the FTC and many other agencies are bringing in more and more tech
are bringing in more and more tech people to help with these um these
people to help with these um these Market challenges but um maybe you can
Market challenges but um maybe you can give us a little introduction from how
give us a little introduction from how you see all this playing out sure good
you see all this playing out sure good morning so the role of uh the team I
morning so the role of uh the team I work on at the FDC is to provide
work on at the FDC is to provide technical expertise to the commission
technical expertise to the commission and to the attorneys we work with on uh
and to the attorneys we work with on uh how these Technologies show up in the
how these Technologies show up in the agency's enforcement work both in
agency's enforcement work both in consumer protection as well as in
consumer protection as well as in competition and so I should say that uh
competition and so I should say that uh you know these views are my own and
you know these views are my own and don't necessarily represent any of any
don't necessarily represent any of any individual commissioner the commission
individual commissioner the commission as a whole but uh a top level uh way I
as a whole but uh a top level uh way I Orient myself in this space is that
Orient myself in this space is that there is no single question of what is
there is no single question of what is competition in AI I've got at least
competition in AI I've got at least three that I think are you know top
three that I think are you know top level questions one is what are the
level questions one is what are the competitive Dynamics in companies
competitive Dynamics in companies building models themselves particularly
building models themselves particularly building foundational or Frontier models
building foundational or Frontier models so that that's one market uh or one
so that that's one market uh or one question of competitive Dynamics you
question of competitive Dynamics you might have uh a second question you
might have uh a second question you might have is how does AI
might have is how does AI impact uh the competitive Dynamics in
impact uh the competitive Dynamics in markets particularly in the supply chain
markets particularly in the supply chain for AI the chips cloud computing orchest
for AI the chips cloud computing orchest ation tools all the other software and
ation tools all the other software and Hardware you need to actually build and
Hardware you need to actually build and deploy these models that's a second
deploy these models that's a second question you might ask about the
question you might ask about the competitive Dynamics and a third
competitive Dynamics and a third question you might ask
question you might ask is uh how will AI impact competition in
is uh how will AI impact competition in markets it's deployed in whether that's
markets it's deployed in whether that's Healthcare or Finance uh or any other
Healthcare or Finance uh or any other Market that you might find uses uh for
Market that you might find uses uh for these models um and these questions
these models um and these questions don't uh this the state of competition
don't uh this the state of competition in any one of these questions doesn't
in any one of these questions doesn't necessarily have the same answer and
necessarily have the same answer and there are interventions you might
there are interventions you might consider that will improve competition
consider that will improve competition in one but might not impact another so I
in one but might not impact another so I think uh I will do my best to as I'm
think uh I will do my best to as I'm saying you know this has potential to be
saying you know this has potential to be good this has potential to be bad to be
good this has potential to be bad to be specific where where do I think it is
specific where where do I think it is the potential to be good or bad or you
the potential to be good or bad or you know show promise or any of those other
know show promise or any of those other things because I think I think it's very
things because I think I think it's very easy to talk past each other when
easy to talk past each other when something has potential advantages in
something has potential advantages in one area or disadvantages in another
one area or disadvantages in another there doesn't have to be a single answer
all right sounds like uh you've got your work cut out for you um Lots going on um
work cut out for you um Lots going on um now let me BR just bring it back to
now let me BR just bring it back to Victor and you can uh react a little bit
Victor and you can uh react a little bit to all the other panelists and or give
to all the other panelists and or give us a little more on amd's thoughts on
us a little more on amd's thoughts on yeah let me first do that because we
yeah let me first do that because we only kind of talked about CPS and GPS
only kind of talked about CPS and GPS but um you know AMD actually you know
but um you know AMD actually you know has of of course been in in CPUs and I
has of of course been in in CPUs and I kind of mentioned like there's that's
kind of mentioned like there's that's what's in your device you know your
what's in your device you know your notebook or your desktop versus what's
notebook or your desktop versus what's in the cloud so we do server and client
in the cloud so we do server and client CPUs we also have I mentioned um you
CPUs we also have I mentioned um you know the gpus that are very focused on
know the gpus that are very focused on really high performant uh Ai and that's
really high performant uh Ai and that's the Mi Instinct line and then we have
the Mi Instinct line and then we have the radio line which is still doing
the radio line which is still doing gaming which by the way of course in
gaming which by the way of course in gaming there is going to be some AI
gaming there is going to be some AI there but we also have um even more
there but we also have um even more special I won't go into the primer on
special I won't go into the primer on that but uh products from uh the zyink
that but uh products from uh the zyink acquisition uh which is a product called
acquisition uh which is a product called fpgas and adaptive so so's and and these
fpgas and adaptive so so's and and these are products that are in many many
are products that are in many many embedded products and the reason why I
embedded products and the reason why I want to first you know share the the
want to first you know share the the breath of the product portfolio is
breath of the product portfolio is because also it it's uh it's in many
because also it it's uh it's in many different products that actually you may
different products that actually you may not think you're touching it but you're
not think you're touching it but you're probably using an AMD product every day
probably using an AMD product every day um because we're in cars um we're in
um because we're in cars um we're in infrastructure we're in Communications
infrastructure we're in Communications infrastructure both wir and wireless and
infrastructure both wir and wireless and then of course we're in in like the more
then of course we're in in like the more traditional Computing things we are in
traditional Computing things we are in healthcare um so so it's quite Broad and
healthcare um so so it's quite Broad and the reason why I also mentioned that is
the reason why I also mentioned that is because you know AI is going to um I
because you know AI is going to um I think you know just you know
think you know just you know revolutionize and really en enhance um
revolutionize and really en enhance um everybody's everyday lives and of course
everybody's everyday lives and of course we have to be mindful about the negative
we have to be mindful about the negative things but um you know we do play across
things but um you know we do play across all of that and there's different
all of that and there's different software Dynamics around that um and I
software Dynamics around that um and I think that you know we think of this as
think that you know we think of this as um important on multiple levels because
um important on multiple levels because it really does you know hearing about
it really does you know hearing about the healthcare that's a perfect example
the healthcare that's a perfect example of you know you want to enable that
of you know you want to enable that Innovation while you're doing the proper
Innovation while you're doing the proper things on privacy and all the others and
things on privacy and all the others and I believe in this comment of the genius
I believe in this comment of the genius of the end it's like you can do the
of the end it's like you can do the right thing and still be a very
right thing and still be a very successful profitable business creating
successful profitable business creating value for society as a whole and
value for society as a whole and competing very rigorously right and I
competing very rigorously right and I think that's the goal of government
think that's the goal of government ensuring that sort of happening and when
ensuring that sort of happening and when it doesn't and Comm measure to sort of
it doesn't and Comm measure to sort of the risk factors maybe you know playing
the risk factors maybe you know playing a stronger role but what you really want
a stronger role but what you really want to see is that competition that you know
to see is that competition that you know you know and again the The Innovation
you know and again the The Innovation because that drives cost that drives
because that drives cost that drives greater value and so forth and um and
greater value and so forth and um and you know again we take an open approach
you know again we take an open approach that we try to enable an ecosystem as
that we try to enable an ecosystem as opposed to a more proprietary you know
opposed to a more proprietary you know locked in approach so excellent all
locked in approach so excellent all right so let's try and put this in a
right so let's try and put this in a little bit of perspective overall um
little bit of perspective overall um those of us who've been around for a
those of us who've been around for a while we have seen Tech come and go
while we have seen Tech come and go right we used to have mainframes and
right we used to have mainframes and then there were the rise of personal
then there were the rise of personal computers and now we have um we have
computers and now we have um we have more on our phones than um whole rooms
more on our phones than um whole rooms of mainframes used to be that we carry
of mainframes used to be that we carry around with us every day um so is
around with us every day um so is this um AI Hardware right now is this
this um AI Hardware right now is this cycle really any different um is it
cycle really any different um is it really any different than any of the
really any different than any of the other ones that have happened in the
other ones that have happened in the last 20 or 40 years and how and why
last 20 or 40 years and how and why might it be different so um to maer let
might it be different so um to maer let me start with you and let me say feel
me start with you and let me say feel free to all my panelists to to jump in
free to all my panelists to to jump in but muzzer kick us off
but muzzer kick us off um so I think the short answer is yes
um so I think the short answer is yes from my perspective from what I've uh
from my perspective from what I've uh observed um I like to think about or
observed um I like to think about or kind of describe this as um if if you
kind of describe this as um if if you guys have heard of the Canan effect
guys have heard of the Canan effect Canan effect is this idea that inflation
Canan effect is this idea that inflation is experienced by different parts of
is experienced by different parts of society differently so uh people closest
society differently so uh people closest to the money printer the central banks
to the money printer the central banks the primary dealers the insurance
the primary dealers the insurance companies Etc they get the value of the
companies Etc they get the value of the marginal dollar uh before anyone else
marginal dollar uh before anyone else and therefore they they're sort of this
and therefore they they're sort of this like wealth transfer from from society
like wealth transfer from from society to you know to to the financial sector
to you know to to the financial sector and I think that um and that's sort of
and I think that um and that's sort of an exponential function and and I think
an exponential function and and I think something like that is happening in the
something like that is happening in the AI space especially with uh companies
AI space especially with uh companies that have strong AI capabilities either
that have strong AI capabilities either the uh the the best model or the best
the uh the the best model or the best data lake or um or uh creating the best
data lake or um or uh creating the best AI Hardware is that they have this sort
AI Hardware is that they have this sort of feedback loop that they can they can
of feedback loop that they can they can sort of generate the next model faster
sort of generate the next model faster the next the next powerful model faster
the next the next powerful model faster better at accelerating rate uh versus
better at accelerating rate uh versus their competition so there is that
their competition so there is that feedback loop that I think is is
feedback loop that I think is is different and you can observe this by a
different and you can observe this by a lot of startups having to do really
lot of startups having to do really rapid pivots uh since the Advent of gpt3
rapid pivots uh since the Advent of gpt3 in November 2022 uh and then every
in November 2022 uh and then every subsequent release um people are
subsequent release um people are realizing that they sort of have to rip
realizing that they sort of have to rip out that intelligent portion that sort
out that intelligent portion that sort of was their core value and just replace
of was their core value and just replace it with a strong AA model and so I think
it with a strong AA model and so I think that is different where larger players
that is different where larger players not only have distribution but they have
not only have distribution but they have the speed that was usually the
the speed that was usually the characteristic of a small company got it
characteristic of a small company got it I could say I I also agree it's
I could say I I also agree it's different and I'll I'll point it three
different and I'll I'll point it three um three aspects you know one is you
um three aspects you know one is you know and showing my age right I started
know and showing my age right I started at a school in 82 so I've seen a lot of
at a school in 82 so I've seen a lot of stuff I've seen you know the the PC
stuff I've seen you know the the PC generation the mobile you know social
generation the mobile you know social media everything um you know I I think
media everything um you know I I think you know Percy and Andrew talked about
you know Percy and Andrew talked about this this is a very general purpose
this this is a very general purpose technology so the fact that it's so
technology so the fact that it's so Broad and it can appli for everything
Broad and it can appli for everything from healthare to forecasting to you
from healthare to forecasting to you know uh entertainment to
know uh entertainment to creativity infrastructure smart cities
creativity infrastructure smart cities um autonomous driving it's just so broad
um autonomous driving it's just so broad so so that's one factor that's different
so so that's one factor that's different the other factor is different we have
the other factor is different we have some of the other disruptions right so
some of the other disruptions right so if you had you know 56 years of MW and
if you had you know 56 years of MW and the power of semiconductors is
the power of semiconductors is incredible so algorithms and things that
incredible so algorithms and things that weren't useful before but now with the
weren't useful before but now with the computing power that you can have like
computing power that you can have like you said on an on on your phone or on a
you said on an on on your phone or on a device that you carry is is phenomenal
device that you carry is is phenomenal and we have the internet so the speed at
and we have the internet so the speed at which you know information travels the
which you know information travels the I've never seen a concept in you know
I've never seen a concept in you know Academia get commercialized so fast in
Academia get commercialized so fast in in the 42 years of my career right and
in the 42 years of my career right and the last thing is actually also just
the last thing is actually also just adoption because I remember it took a
adoption because I remember it took a long time for the internet to really I I
long time for the internet to really I I remember hearing about mosaic
remember hearing about mosaic you know but you have to be an Insider
you know but you have to be an Insider geek kind of person to know that and
geek kind of person to know that and then slowly there was AOL and all stuff
then slowly there was AOL and all stuff I mean but this is like and and it's
I mean but this is like and and it's true there was really something before
true there was really something before chap gbt but that is an amazing
chap gbt but that is an amazing phenomena right where my kids
phenomena right where my kids immediately within months were asking me
immediately within months were asking me questions about that they never ask me
questions about that they never ask me questions about my work that in itself
questions about my work that in itself is like you know unprecedented right um
is like you know unprecedented right um but you know not just in the US around
but you know not just in the US around the world right and there are people
the world right and there are people experimenting with it so so that human
experimenting with it so so that human um awareness and adoption and and that
um awareness and adoption and and that and the conversation that's amazing I
and the conversation that's amazing I have not seen that you know usually it
have not seen that you know usually it takes years before a new technology hits
takes years before a new technology hits people's um Consciousness like like it
people's um Consciousness like like it has so yep those are my three reasons
has so yep those are my three reasons why I think this time it's different yep
why I think this time it's different yep famous last words sometimes but Chris
famous last words sometimes but Chris did you have thought yeah I I think the
did you have thought yeah I I think the other part of this too I guess we're all
other part of this too I guess we're all in like Universal agreement here I wish
in like Universal agreement here I wish I had more success though in
I had more success though in communicating to my family what I do
communicating to my family what I do like if you ask my mom what I do she
like if you ask my mom what I do she like he works on
like he works on computers and then it would be like you
computers and then it would be like you had to go all the way to Brazil to fix
had to go all the way to Brazil to fix someone's computer and I'm like yeah Mom
someone's computer and I'm like yeah Mom I wish I just told them to reboot you
I wish I just told them to reboot you know but you know if you put that aside
know but you know if you put that aside there's a huge difference now with like
there's a huge difference now with like AI Hardware compared to something like a
AI Hardware compared to something like a mobile phone right where you'd have a
mobile phone right where you'd have a phone you'd have an OS and you'd have a
phone you'd have an OS and you'd have a fairly vertical stack with with there's
fairly vertical stack with with there's a ton of autonomy and flexibility with
a ton of autonomy and flexibility with AI Hardware today where you have
AI Hardware today where you have projects such as Ray to really help
projects such as Ray to really help orchestrate AI workloads you have uh AI
orchestrate AI workloads you have uh AI Frameworks like pytorch that can really
Frameworks like pytorch that can really help to create that neutrality you have
help to create that neutrality you have virtualization software like we provide
virtualization software like we provide that provides a lot of flexibility so
that provides a lot of flexibility so you there's tons of ways and lots of
you there's tons of ways and lots of different approaches you can take to
different approaches you can take to decouple applications from Hardware so
decouple applications from Hardware so it's a much different phenomenon than
it's a much different phenomenon than what you saw with a mobile phone in the
what you saw with a mobile phone in the past so I I'll just say I have no
past so I I'll just say I have no overarching answer to is AI different
overarching answer to is AI different than any past technology I'm happy to
than any past technology I'm happy to take a weit and see approach but I do
take a weit and see approach but I do think we are seeing something that is
think we are seeing something that is different from the last 20 years
different from the last 20 years particularly with re with respect to
particularly with re with respect to chips and Hardware which is that uh if
chips and Hardware which is that uh if you're not a practitioner this may sound
you're not a practitioner this may sound uh you know strange and if you are I
uh you know strange and if you are I guess it'll be boring but the last 20
guess it'll be boring but the last 20 years were really defined by a use of
years were really defined by a use of general purpose commodity Hardware in
general purpose commodity Hardware in the data center it was not about
the data center it was not about specialized chips for each workload if
specialized chips for each workload if you whether you were running a social
you whether you were running a social media application or an Enterprise
media application or an Enterprise application or as a software as a
application or as a software as a service business probably you were
service business probably you were running on roughly similar hardware and
running on roughly similar hardware and AI is really taking a different approach
AI is really taking a different approach in that it is fundamentally accelerated
in that it is fundamentally accelerated by gpus and even more specialized
by gpus and even more specialized hardware and it is really I think
hardware and it is really I think driving a Resurgence of uh what we might
driving a Resurgence of uh what we might call like an accelerator right dedicated
call like an accelerator right dedicated hardware for a specific task and I just
hardware for a specific task and I just think from the you know perspective of
think from the you know perspective of chips like that is something that's just
chips like that is something that's just different from how we you know the
different from how we you know the median business even the median Tech
median business even the median Tech business was operating five or 10 years
business was operating five or 10 years ago absolutely let me um let me go on to
ago absolutely let me um let me go on to my next question um what are the market
my next question um what are the market dynamics we're thinking about these
dynamics we're thinking about these chips that are now we're hearing more
chips that are now we're hearing more and more important that this is
and more important that this is different um you may need vast
different um you may need vast quantities of them um and we're here to
quantities of them um and we're here to talk about competition in markets so
talk about competition in markets so what are the market dynamics when we're
what are the market dynamics when we're thinking about these chips how do chips
thinking about these chips how do chips affect access to Capital how does access
affect access to Capital how does access to Capital affect access to chips um
to Capital affect access to chips um blanch can you start us off yes thank
blanch can you start us off yes thank you very much yes I will begin with
you very much yes I will begin with Basics so you've understand I'm not the
Basics so you've understand I'm not the the technical person in the panel but I
the technical person in the panel but I will give you my perspective on on how
will give you my perspective on on how we see the market of chips from really a
we see the market of chips from really a company that needs them uh so beginning
company that needs them uh so beginning with the basics uh the requirements for
with the basics uh the requirements for training are not the same as for
training are not the same as for inference it has been said in more
inference it has been said in more technical words but that's important to
technical words but that's important to recall the basics and uh servicing the
recall the basics and uh servicing the chips that are the ones that work for
chips that are the ones that work for training uh requires um some criteria
training uh requires um some criteria which which is um the most powerful
which which is um the most powerful chips so we speak about flops uh a chips
chips so we speak about flops uh a chips with lots of memory and also so those
with lots of memory and also so those chips today uh the h00 the H 200 that
chips today uh the h00 the H 200 that will come they are pretty rare that's
will come they are pretty rare that's the first point and also the third
the first point and also the third aspect is um for some reason that I
aspect is um for some reason that I cannot explain but this is what we
cannot explain but this is what we observe for some reason having those
observe for some reason having those chips is not enough you need a cloud
chips is not enough you need a cloud provider that has a very good
provider that has a very good infrastructure uh to assemble them by
infrastructure uh to assemble them by eight and to uh and to have uh to to
eight and to uh and to have uh to to assemble them in a way that uh does that
assemble them in a way that uh does that that creates a good connectivity between
that creates a good connectivity between the chips otherwise the training is
the chips otherwise the training is super long and it just fails or it costs
super long and it just fails or it costs too much and and that is something that
too much and and that is something that so the chips are rare and the providers
so the chips are rare and the providers that can uh the the the companies that
that can uh the the the companies that can assemble them and make them work in
can assemble them and make them work in a way that is efficient for training are
a way that is efficient for training are even more rare so um so if you have
even more rare so um so if you have providers that do not combine all those
providers that do not combine all those criteria you can do some inference with
criteria you can do some inference with them you can do some other things
them you can do some other things someone spoke about Rag and that's true
someone spoke about Rag and that's true you can do a lot of things with with
you can do a lot of things with with less power with less memory and and also
less power with less memory and and also maybe with some infrastructure that is
maybe with some infrastructure that is less efficient but you cannot do
less efficient but you cannot do training the other thing that is
training the other thing that is important for training is that you need
important for training is that you need a huge number of chips and you need them
a huge number of chips and you need them at the same place so that means that if
at the same place so that means that if you don't have them at the same place
you don't have them at the same place you will be able to train several models
you will be able to train several models of different sizes adapted maybe to the
of different sizes adapted maybe to the chips that you have available in the
chips that you have available in the same data center but you won't be able
same data center but you won't be able to build a huge model uh with your chips
to build a huge model uh with your chips scattered in different places so what
scattered in different places so what that means really practically is that
that means really practically is that you need a very big ticket of cash uh to
you need a very big ticket of cash uh to the same provider to be able to train
the same provider to be able to train big models and so uh if you don't have
big models and so uh if you don't have catch you don't have that but if you
catch you don't have that but if you don't have that you also have problems
don't have that you also have problems convincing your investors to get your
convincing your investors to get your next fundraising because you have
next fundraising because you have problems convincing them that you are a
problems convincing them that you are a realistic actor for for the future so
realistic actor for for the future so that's kind of what we're struggling
that's kind of what we're struggling with uh struggling not not so much
with uh struggling not not so much because we've managed so far uh and also
because we've managed so far uh and also because um maybe we took the opportunity
because um maybe we took the opportunity to not be being able to train all the
to not be being able to train all the time the biggest models uh to make sure
time the biggest models uh to make sure we trained the best smaller models which
we trained the best smaller models which also has some virtues because not all
also has some virtues because not all use cases
use cases requires the biggest models and it's
requires the biggest models and it's better for environment it's also been
better for environment it's also been said already but that's maybe how how
said already but that's maybe how how the lscape is um as of today so as of
the lscape is um as of today so as of today what we are experiencing and that
today what we are experiencing and that will be my conclusion at this point is a
will be my conclusion at this point is a massive bottleneck both on the provision
massive bottleneck both on the provision of chips and on the provision by the the
of chips and on the provision by the the the few companies that know how to
the few companies that know how to manage them in the right way to be able
manage them in the right way to be able to train models H and that means that
to train models H and that means that there is a cue and there is a cue for a
there is a cue and there is a cue for a company like ours on training but there
company like ours on training but there is also a cue for customers of ours that
is also a cue for customers of ours that would like to have their own
would like to have their own infrastructure for inference this time
infrastructure for inference this time so that reduces the possibility of
so that reduces the possibility of distribution on the side of of customers
distribution on the side of of customers uh to do on premises for example and
uh to do on premises for example and that also probably reduces the
that also probably reduces the possibility of new companies to to start
possibility of new companies to to start from scratch and convince all the all
from scratch and convince all the all these actors to provide them fast with a
these actors to provide them fast with a lot of
lot of computes all right well a a bottleneck
computes all right well a a bottleneck of chips you need a only one major
of chips you need a only one major provider sounds like an antitrust
provider sounds like an antitrust problem to me of course I'm an antitrust
problem to me of course I'm an antitrust lawyer so I see that I I feel compelled
lawyer so I see that I I feel compelled so say Victor we're we're coming to your
so say Victor we're we're coming to your rescue right we're going to we're going
rescue right we're going to we're going to like reduce that scarcity reduce the
to like reduce that scarcity reduce the TCO give you Choice um and you know look
TCO give you Choice um and you know look let's face it you know the Mi 300X which
let's face it you know the Mi 300X which is getting stood up in like it's public
is getting stood up in like it's public you know Azure and a lot of the public
you know Azure and a lot of the public you know infrastructure is going to get
you know infrastructure is going to get out there we're getting there um and we
out there we're getting there um and we really do you know have the leadership
really do you know have the leadership in inference exactly that for large
in inference exactly that for large language models you know we bench talk
language models you know we bench talk on things like llama and you mentioned
on things like llama and you mentioned also the distribution that's why we
also the distribution that's why we saying network equipment we also do have
saying network equipment we also do have networking equipment but we're also
networking equipment but we're also partnering with broadcom and with others
partnering with broadcom and with others actually to enable an open uh
actually to enable an open uh interconnect because you know it is true
interconnect because you know it is true you need powerful you know compute nodes
you need powerful you know compute nodes but you also for these large models you
but you also for these large models you need to have the connectivity because
need to have the connectivity because you could lose a lot of porns through
you could lose a lot of porns through that um you know now at the same time
that um you know now at the same time there are more Modz models and you know
there are more Modz models and you know one thing I didn't talk about earlier I
one thing I didn't talk about earlier I talked about how AMD spans end to end um
talked about how AMD spans end to end um we're in aipc so in your laptop we have
we're in aipc so in your laptop we have acceleration within that right so
acceleration within that right so there's no you don't have the same um
there's no you don't have the same um connectivity issues and that's going to
connectivity issues and that's going to go in in to to a large extent help
go in in to to a large extent help democratize maybe if you will like you
democratize maybe if you will like you know get get broader access but it
know get get broader access but it wouldn't be so for the really extremely
wouldn't be so for the really extremely large models that they they need the
large models that they they need the large infrastructure right but I I
large infrastructure right but I I really think this is goes to the heart
really think this is goes to the heart of you know if they're innovators like
of you know if they're innovators like mral is Bal neck what they need is they
mral is Bal neck what they need is they need a really active Lively Market at
need a really active Lively Market at that net foundational level and that's
that net foundational level and that's what we truly mean to provide and that
what we truly mean to provide and that is where if Beyond you know we're we're
is where if Beyond you know we're we're going to compete vigorously we have
going to compete vigorously we have leadership and inference right um but if
leadership and inference right um but if there are things that are not equal in
there are things that are not equal in competition or things like that's the
competition or things like that's the thing where you have to be careful of
thing where you have to be careful of and then I think you know outside of
and then I think you know outside of Regulation there's other things that I
Regulation there's other things that I think could be done by the government um
think could be done by the government um to helps for Innovation but anyway I
to helps for Innovation but anyway I just I just absolutely wanted to say
just I just absolutely wanted to say help is on the way I hope you look at
help is on the way I hope you look at AMD so all right well let me jump to um
AMD so all right well let me jump to um talking about the role of government
talking about the role of government here we potentially have a bottleneck we
here we potentially have a bottleneck we potentially have some super large
potentially have some super large players having super large roles um in
players having super large roles um in this industry um government can do sort
this industry um government can do sort of two things uh broadly one they can
of two things uh broadly one they can put money on the table they can put
put money on the table they can put subsidies on the table they can
subsidies on the table they can contribute in that way the other way is
contribute in that way the other way is through regulation um Alex um from the
through regulation um Alex um from the FTC perspective you said a minute ago
FTC perspective you said a minute ago you're going to have a wait and seee
you're going to have a wait and seee approach but I don't think that was a
approach but I don't think that was a global statement um correct tell us tell
global statement um correct tell us tell us a little bit about um what the ft is
us a little bit about um what the ft is thinking about the role of government in
thinking about the role of government in what is potentially a bottlenecked
what is potentially a bottlenecked environment controlled by a couple large
environment controlled by a couple large players yeah so the ftc's role is as an
players yeah so the ftc's role is as an enforcer of the nation's competition and
enforcer of the nation's competition and consumer protection laws and the first
consumer protection laws and the first order belief we have is that there is no
order belief we have is that there is no AI exemption to the rules that are on
AI exemption to the rules that are on the book existing competition laws apply
the book existing competition laws apply to this sector like they would any other
to this sector like they would any other I think the uh a caner said it well this
I think the uh a caner said it well this morning if you use an algorithm to
morning if you use an algorithm to collusively price fix that's is illegal
collusively price fix that's is illegal if you didn't use an algorithm right if
if you didn't use an algorithm right if you attempt to monopolize you know a
you attempt to monopolize you know a market whether it's for AI models or in
market whether it's for AI models or in the AI supply chain that is already a
the AI supply chain that is already a violation of our competition laws so I
violation of our competition laws so I think that that's the first order answer
think that that's the first order answer is we look to vigorously enforce the
is we look to vigorously enforce the laws that are on the books um in January
laws that are on the books um in January of this year we announced a 6B study uh
of this year we announced a 6B study uh which is an authority we have for a
which is an authority we have for a non-law enforcement investigation uh to
non-law enforcement investigation uh to better understand a market looking at
better understand a market looking at Investments and Partnerships between uh
Investments and Partnerships between uh large model developers and large comb
large model developers and large comb and Cloud providers and the focus of
and Cloud providers and the focus of that is to understand what is the nature
that is to understand what is the nature of these Investments and Partnerships
of these Investments and Partnerships and how do they impact the competitive
and how do they impact the competitive Dynamics in those marketplaces so that
Dynamics in those marketplaces so that that's something we're actively doing
that's something we're actively doing and I I don't have any results to share
and I I don't have any results to share but you know we're actively looking to
but you know we're actively looking to understand how the market is shaped by
understand how the market is shaped by uh those types of
uh those types of structures great thank you I'm going to
structures great thank you I'm going to flip back to blunch real briefly because
flip back to blunch real briefly because um Europe doesn't always do things
um Europe doesn't always do things exactly the way we do them in the US um
exactly the way we do them in the US um maybe you can just give a little
maybe you can just give a little International perspective in terms of
International perspective in terms of the role of government um in in France
the role of government um in in France or in
or in Europe I think it's it's the reputation
Europe I think it's it's the reputation indeed of Europe to be very good at
indeed of Europe to be very good at regulation I don't know if I I think
regulation I don't know if I I think it's great but it is what it is but for
it's great but it is what it is but for some purposes it is indeed great so I
some purposes it is indeed great so I think there are indeed two aspects to
think there are indeed two aspects to the topic one of them is really
the topic one of them is really encouraging this Innovation putting
encouraging this Innovation putting money on the table creating the
money on the table creating the universities and and the schools that
universities and and the schools that will accompany population to to ramp up
will accompany population to to ramp up to this Innovation make sure that there
to this Innovation make sure that there is no one left behind into this adoption
is no one left behind into this adoption and and also that the persons where
and and also that the persons where which have the jobs that are
which have the jobs that are revolutionized by this technical
revolutionized by this technical Revolution are accompanied to to to
Revolution are accompanied to to to change their way to work so I think that
change their way to work so I think that there's a lot to do for governments on
there's a lot to do for governments on the positive side uh and I think that
the positive side uh and I think that France is doing a lot on this and and
France is doing a lot on this and and this is maybe one of the very good
this is maybe one of the very good reasons why M TR is here today because a
reasons why M TR is here today because a decade ago the French government has
decade ago the French government has started to put billions on the table and
started to put billions on the table and and to make sure that there were some
and to make sure that there were some centers in France and Paris where
centers in France and Paris where students would be very good at Ai and
students would be very good at Ai and there would be some uh also some
there would be some uh also some welcoming some uh some of these
welcoming some uh some of these hyperscalers in Paris to have their Hub
hyperscalers in Paris to have their Hub their AI Hub in Paris to to keep a
their AI Hub in Paris to to keep a French ecosystem Lively also thanks to
French ecosystem Lively also thanks to us companies to to be frank uh so that
us companies to to be frank uh so that what happened and and a week ago our
what happened and and a week ago our French president maon announced some
French president maon announced some other hundreds of millions put on the
other hundreds of millions put on the table uh for um funding AI Innovation
table uh for um funding AI Innovation including chips uh and for training also
including chips uh and for training also um more people on AI so I think this is
um more people on AI so I think this is a very good thing to do and and it's
a very good thing to do and and it's very welcome in every country for sure
very welcome in every country for sure because really the purpose is to to
because really the purpose is to to leave no one behind uh and I'm sure
leave no one behind uh and I'm sure there is a lot of money in the US to be
there is a lot of money in the US to be put on that too on the other side of the
put on that too on the other side of the of the topic which is more the
of the topic which is more the regulation in the mere sense of the term
regulation in the mere sense of the term uh there is this AI act in Europe some
uh there is this AI act in Europe some might think it's going too soon because
might think it's going too soon because because we don't know yet what we
because we don't know yet what we regulating but it has the merits of uh
regulating but it has the merits of uh putting some uh some problematics on the
putting some uh some problematics on the table so that's always interesting but
table so that's always interesting but um so yeah there is a there is a thing
um so yeah there is a there is a thing which is it's going so fast that
which is it's going so fast that Regulators want to go fast which is
Regulators want to go fast which is really
really admirable but it's also a risk that if
admirable but it's also a risk that if they go too fast they regulate too much
they go too fast they regulate too much not on the right thing so maybe this is
not on the right thing so maybe this is a risk there and on more on the on the
a risk there and on more on the on the competition side all the European
competition side all the European authorities have been very active in
authorities have been very active in trying to ramp up on the topic and
trying to ramp up on the topic and understand and that's great because
understand and that's great because someday they will need to react and they
someday they will need to react and they will need to have all this science uh in
will need to have all this science uh in in in this area and honestly just by
in in this area and honestly just by answering them I've learned so much into
answering them I've learned so much into the that I think it's a very good
the that I think it's a very good initiative there are also been some
initiative there are also been some stronger initiatives into looking into
stronger initiatives into looking into Partnerships and here M believes it's a
Partnerships and here M believes it's a risk because pure players like us we
risk because pure players like us we need Partnerships to exist we need we
need Partnerships to exist we need we are not integrated on the value stack so
are not integrated on the value stack so we need to partner with hyperscalers for
we need to partner with hyperscalers for compute for distribution for almost
compute for distribution for almost everything and if we cannot do that uh
everything and if we cannot do that uh there is a risk that we just do not
there is a risk that we just do not exist so here we tend to think that
exist so here we tend to think that there is a risk risk that by reacting
there is a risk risk that by reacting too fast it could also create some side
too fast it could also create some side effects that could have contrary effects
effects that could have contrary effects to what competition law is here for so
to what competition law is here for so that's kind of the dynamic knowing the
that's kind of the dynamic knowing the topic being ready to react and to and to
topic being ready to react and to and to and regulate and and trying to
and regulate and and trying to anticipate the side effects which is
anticipate the side effects which is very difficult given how fast it's
very difficult given how fast it's moving got it thank you anyone else want
moving got it thank you anyone else want to jump in on the role of government
to jump in on the role of government Chris I would just say you know a couple
Chris I would just say you know a couple things that you know we're we're we're
things that you know we're we're we're very supportive of what's already
very supportive of what's already happened with the president's recent
happened with the president's recent executive order um as well as even the
executive order um as well as even the NY framework which is something our
NY framework which is something our internal AI policy also points to these
internal AI policy also points to these are very effective guidelines for
are very effective guidelines for steering the safe and secure adoption of
steering the safe and secure adoption of AI and it focuses on areas such as being
AI and it focuses on areas such as being able to uh AB test different AI models
able to uh AB test different AI models being able to have choice and this is
being able to have choice and this is what's Driven our both internal and
what's Driven our both internal and external approach to having a AI
external approach to having a AI platform the other thing I would just
platform the other thing I would just mention is like I've seen this a lot
mention is like I've seen this a lot like my career the last decade I've key
like my career the last decade I've key part of my role has been bringing new
part of my role has been bringing new technologies to Market and where I see a
technologies to Market and where I see a little bit of a gap in terms of support
little bit of a gap in terms of support from the government is the government's
from the government is the government's been very good in terms of trying to
been very good in terms of trying to promote Innovation but hasn't invested
promote Innovation but hasn't invested equally in promoting adoption of
equally in promoting adoption of innovation so sometimes getting that
innovation so sometimes getting that first mover agency to to to adopt
first mover agency to to to adopt technology is where that bottleneck
technology is where that bottleneck exists because if I'm a CIO of an agency
exists because if I'm a CIO of an agency I want to see which of my peers has
I want to see which of my peers has adopted first so incentivizing adoption
adopted first so incentivizing adoption is just as important as incentivizing
is just as important as incentivizing Innovation interesting interesting
Innovation interesting interesting thanks I wouldn't have guessed that like
thanks I wouldn't have guessed that like as Victor was saying everyone used chat
as Victor was saying everyone used chat GPT the week after it came out and um
GPT the week after it came out and um but maybe but there's a difference
but maybe but there's a difference between all of us typing something in
between all of us typing something in randomly um and really using it in a
randomly um and really using it in a business context where they're going to
business context where they're going to be looking at what are their competitors
be looking at what are their competitors doing a big part of my job last year was
doing a big part of my job last year was saying no to product teams wanting to
saying no to product teams wanting to embed chat GPT Integrations into
embed chat GPT Integrations into products got it okay um let's talk a
products got it okay um let's talk a little bit about vertical integration
little bit about vertical integration we've talked a little bit about the
we've talked a little bit about the stack chips on the bottom chat GPT on
stack chips on the bottom chat GPT on the top um and um and if the chip
the top um and um and if the chip companies um one of which is a very
companies um one of which is a very dominant position um in some of these
dominant position um in some of these areas um if it keeps integrating up the
areas um if it keeps integrating up the stack you can have a completely
stack you can have a completely integrated product which might be great
integrated product which might be great in some ways and might be horrible in
in some ways and might be horrible in some others um so um it might be good
some others um so um it might be good for uh it might be horrible or good at
for uh it might be horrible or good at different levels customers might have a
different levels customers might have a view um chip makers might have a view
view um chip makers might have a view people who are on the middle of the
people who are on the middle of the stack might have um a different view so
stack might have um a different view so um vertical integration what should
um vertical integration what should customers be thinking about what should
customers be thinking about what should government Regulators be thinking about
government Regulators be thinking about what should other companies be thinking
what should other companies be thinking about um Victor want to start sure and
about um Victor want to start sure and and again I'm going to pick up on um how
and again I'm going to pick up on um how it was already described several times
it was already described several times about you know it's you need to connect
about you know it's you need to connect up all these really powerful servers you
up all these really powerful servers you know GPU rich and also there CPUs in
know GPU rich and also there CPUs in there um you know now if you want to
there um you know now if you want to have choice and you want to have reduce
have choice and you want to have reduce the scarcity problem and therefore
the scarcity problem and therefore lowering costs and improving access to
lowering costs and improving access to innovators and so forth they need to
innovators and so forth they need to interoperate right so if the
interoperate right so if the connectivity in the network is
connectivity in the network is proprietary and closed and it's really
proprietary and closed and it's really hard for AMD operate with competition by
hard for AMD operate with competition by the way and AMD inter operates with you
the way and AMD inter operates with you know like traditionally with um you know
know like traditionally with um you know our comp competition and CPUs and and
our comp competition and CPUs and and others then that's challenge right
others then that's challenge right because that's that's blocking um a need
because that's that's blocking um a need from consumers and blocking ultimately
from consumers and blocking ultimately then you know progress right in
then you know progress right in Innovation and and Global um you know
Innovation and and Global um you know creating new value reducing the marginal
creating new value reducing the marginal cost of value and so forth right so I
cost of value and so forth right so I think there there's a you know because
think there there's a you know because there's a lot of discussion around
there's a lot of discussion around software and absolutely you have to look
software and absolutely you have to look at the software um interfaces and and
at the software um interfaces and and the layers in the stack but you also
the layers in the stack but you also have to think about the hardware and
have to think about the hardware and again I I I'm back to the there's
again I I I'm back to the there's nothing inherently wrong about vertical
nothing inherently wrong about vertical integration that happens a lot but if if
integration that happens a lot but if if it's closed and vertically integrated
it's closed and vertically integrated and then there's a extremely you know
and then there's a extremely you know persistent um position there then then
persistent um position there then then that that's a barrier right for for for
that that's a barrier right for for for customer choice for Innovation for
customer choice for Innovation for driving down cost for you know enabling
driving down cost for you know enabling you new value creation models so right
you new value creation models so right anyone else want to jump in leser I'll
anyone else want to jump in leser I'll uh so the way I think about this is that
uh so the way I think about this is that there's kind of a problem at the high
there's kind of a problem at the high end where a sort of dominant AI
end where a sort of dominant AI capability whether it's Hardware models
capability whether it's Hardware models or data could be used to reduce choice
or data could be used to reduce choice in somewhere else so one specific
in somewhere else so one specific example is um if you wanted to start a
example is um if you wanted to start a healthc care company that uses the
healthc care company that uses the highest end
highest end Ai and it has to be hit by compliant
Ai and it has to be hit by compliant there's only one cloud provider that can
there's only one cloud provider that can do that for you
do that for you uh if you want access to gpus well 40%
uh if you want access to gpus well 40% of nvidia's Revenue came from the
of nvidia's Revenue came from the hyperscalers you got to go to a large
hyperscalers you got to go to a large Club provider usually you won't have
Club provider usually you won't have access to those gpus if Google decides
access to those gpus if Google decides to say that hey if you want access to
to say that hey if you want access to the pedabytes and pedabytes of data that
the pedabytes and pedabytes of data that is generated every day by YouTube if you
is generated every day by YouTube if you want to if you want to train on that you
want to if you want to train on that you have to use
have to use gcp you have one choice so I think I
gcp you have one choice so I think I think um I think the vertical
think um I think the vertical integration I think one way to think
integration I think one way to think about is it's good in a way that gives
about is it's good in a way that gives you kind of a a solution that just works
you kind of a a solution that just works um but I think the the real problem is
um but I think the the real problem is where if you wanted to use a strong AI
where if you wanted to use a strong AI capability and that reduces choice in a
capability and that reduces choice in a different layer that's really where it's
different layer that's really where it's it's a little bit Pro problematic and
it's a little bit Pro problematic and that maybe that's a way that you know
that maybe that's a way that you know for example we can make proprietary
for example we can make proprietary models even widely distributed right or
models even widely distributed right or um maybe make uh compute or the
um maybe make uh compute or the distribution of compute somehow more
distribution of compute somehow more distributed so the role of government
distributed so the role of government then these are being these locked in
then these are being these locked in models what do we think I mean my
models what do we think I mean my perspective just as an engineer is
perspective just as an engineer is modularity tends to be good for
modularity tends to be good for Innovation because when you have a
Innovation because when you have a modular system you can have competition
modular system you can have competition for each component and more competitors
for each component and more competitors you know whether it's the same company
you know whether it's the same company competing at multiple layers or entirely
competing at multiple layers or entirely different companies competing at
different companies competing at different layers gets you more
different layers gets you more Innovation as they each strive to
Innovation as they each strive to perform you know the role of their you
perform you know the role of their you know component well and interoperate
know component well and interoperate with the other layers you know I think
with the other layers you know I think that's a tends to be a pro-innovation
that's a tends to be a pro-innovation Pro competition recipe when you have
Pro competition recipe when you have that
that modularity got it okay without saying
modularity got it okay without saying whether the government should do it it's
whether the government should do it it's still from an engineering standpoint yes
still from an engineering standpoint yes just yeah just as an engineer
just yeah just as an engineer that's all right I'm getting a sign that
that's all right I'm getting a sign that we have one minute left I'm going to go
we have one minute left I'm going to go to my last question we're going to call
to my last question we're going to call this a lightning round so everybody gets
this a lightning round so everybody gets you know two
you know two sentences 10 years from now crystal ball
sentences 10 years from now crystal ball what is the uh the AI Hardware uh market
what is the uh the AI Hardware uh market look like dominated by a one or two
look like dominated by a one or two players uh has it integrated all the way
players uh has it integrated all the way up the stack um has government
up the stack um has government intervened has the world fractured and
intervened has the world fractured and they've done something different in EU
they've done something different in EU as opposed to somewhere else um Chris
as opposed to somewhere else um Chris yeah I think it's going to be Lively and
yeah I think it's going to be Lively and robust there's going to be tons of
robust there's going to be tons of players you already see more accelerator
players you already see more accelerator vendors today than than we've
vendors today than than we've historically had with x86 CPU vendors
historically had with x86 CPU vendors and the thing that's going to really
and the thing that's going to really pick this up is going to be the
pick this up is going to be the prevalence of smaller domain specific AI
prevalence of smaller domain specific AI models that's what's going to really be
models that's what's going to really be the game changer here I I don't need a
the game changer here I I don't need a model today that tells me how to brush
model today that tells me how to brush my teeth if I'm at broadcom developing
my teeth if I'm at broadcom developing software right I need specialized models
software right I need specialized models and that lowers the comput ne uh needs
and that lowers the comput ne uh needs it lowers the energy consumption it
it lowers the energy consumption it lowers your carbon footprint AI is is
lowers your carbon footprint AI is is going to do a lot of really good and
going to do a lot of really good and it's you're going to see a broader
it's you're going to see a broader ecosystem as we go further outout okay
ecosystem as we go further outout okay Victor 10 years yeah I mean well first
Victor 10 years yeah I mean well first of all 10 years in AI is kind of like a
of all 10 years in AI is kind of like a hundred years of normal stuff so when
hundred years of normal stuff so when you have to realize that this is really
you have to realize that this is really futuristic projection but I I would just
futuristic projection but I I would just agree I mean I think if you're te just
agree I mean I think if you're te just generally you really are trying to have
generally you really are trying to have the positive outcomes where you're
the positive outcomes where you're mindful of some of the other things and
mindful of some of the other things and I do think it's going to change uh
I do think it's going to change uh everybody's lives on a daily basis um
everybody's lives on a daily basis um you know there's a I've been in
you know there's a I've been in Computing for a long time and there's a
Computing for a long time and there's a fundamental shift uh in Computing away
fundamental shift uh in Computing away from general purpose computes and much
from general purpose computes and much more towards acceler of some sort and I
more towards acceler of some sort and I also think it yes there will be very
also think it yes there will be very large foundational models but there will
large foundational models but there will also be much more mod models that you
also be much more mod models that you know in the bed space we're in like you
know in the bed space we're in like you know sensors intelligent sensors single
know sensors intelligent sensors single cameras you know let alone you know an
cameras you know let alone you know an edge Computing device um and there'll be
edge Computing device um and there'll be agents I I really liked also how uh
agents I I really liked also how uh Andrew talked about agents they'll be
Andrew talked about agents they'll be intelligent agents that are advocates
intelligent agents that are advocates for you right and I think they'll be um
for you right and I think they'll be um it's just really going to I hope that we
it's just really going to I hope that we will accelerate the outcomes in in
will accelerate the outcomes in in healthcare too because that's certainly
healthcare too because that's certainly whether you're in in it um I think all
whether you're in in it um I think all of us have you know have have in their
of us have you know have have in their personal life a touch point on
personal life a touch point on Healthcare so that's you know super um
Healthcare so that's you know super um important that we uh we get those
important that we uh we get those breakthroughs absolutely moer so I'm
breakthroughs absolutely moer so I'm going to say I don't know I don't think
going to say I don't know I don't think anyone knows uh because I think in the
anyone knows uh because I think in the last year it's been quite astonishing to
last year it's been quite astonishing to and to think about the next 10 years I
and to think about the next 10 years I think it's uh kind of unknowable but I I
think it's uh kind of unknowable but I I think there's two trends that we have to
think there's two trends that we have to look out for one is that um AI is a very
look out for one is that um AI is a very centralizing
centralizing Force you have more data you have more
Force you have more data you have more compute which gives you more data and
compute which gives you more data and more compute right you're able to
more compute right you're able to extract value You' be able to create
extract value You' be able to create your grow your team extract more
your grow your team extract more resources on the other hand one of the
resources on the other hand one of the really hopeful areas here is is around
really hopeful areas here is is around open source like without open source I'd
open source like without open source I'd be quite pessimistic uh so thank you for
be quite pessimistic uh so thank you for everyone you know contribute to open
everyone you know contribute to open source um so so maybe a lot of AI
source um so so maybe a lot of AI capabilities will be democratized so
capabilities will be democratized so that's kind of a hopeful hopeful uh
that's kind of a hopeful hopeful uh worldview all right blanch want to go
worldview all right blanch want to go next yes thank you so yeah I share
next yes thank you so yeah I share everything that has been said uh and I
everything that has been said uh and I truly believe that open source is going
truly believe that open source is going to create some more variety because
to create some more variety because everyone is able to to adapt open source
everyone is able to to adapt open source and to build on it so it's going to
and to build on it so it's going to Foster Innovation and and and open
Foster Innovation and and and open market for sure my vision is a world
market for sure my vision is a world where comput is is a is everywhere so
where comput is is a is everywhere so today it's difficult to get some
today it's difficult to get some European compute for example and and
European compute for example and and here I'm speaking about Europe I know
here I'm speaking about Europe I know I'm not even speaking about other areas
I'm not even speaking about other areas of the world so I think that in 10 years
of the world so I think that in 10 years I see a world where everyone has access
I see a world where everyone has access to AI everyone has access to computes
to AI everyone has access to computes and and we are Greener and better in the
and and we are Greener and better in the way we distribute compute and the way we
way we distribute compute and the way we use a variety of models that are smaller
use a variety of models that are smaller when the need is smaller and we I think
when the need is smaller and we I think we we we're going to be smarter in the
we we we're going to be smarter in the way we use the variety of AIS that will
way we use the variety of AIS that will are going to be available
are going to be available everywhere got it I hope so Alex final
everywhere got it I hope so Alex final words what's the yeah uh assuming AI
words what's the yeah uh assuming AI remains a transformational technology I
remains a transformational technology I it's an if I I don't take these things
it's an if I I don't take these things for granted
for granted uh it's my hope that events like this
uh it's my hope that events like this reflect uh you know an increased uh
reflect uh you know an increased uh Attunement to the importance of
Attunement to the importance of competition both for models themselves
competition both for models themselves but also in the supply chain for models
but also in the supply chain for models and I think we can hope that produces
and I think we can hope that produces positive results in competitive
positive results in competitive markets all right competitive markets
markets all right competitive markets five to 10 years we hope um thank you
five to 10 years we hope um thank you all thank you so much for the panelists
all thank you so much for the panelists thank you to all of you um I know I'm
thank you to all of you um I know I'm between you in lunch um so Jennifer's
between you in lunch um so Jennifer's probably going to orient you where to go
probably going to orient you where to go next yes yes thank you um we are going
next yes yes thank you um we are going to have lunch now out in the courtyard
to have lunch now out in the courtyard for about 45 minutes uh there's box
for about 45 minutes uh there's box lunch out there for you I encourage you
lunch out there for you I encourage you all to come back in 45 minutes because
all to come back in 45 minutes because we have a uh we're very grateful that we
we have a uh we're very grateful that we have recorded remarks from senator Amy
have recorded remarks from senator Amy kobitar who will be talking about her
kobitar who will be talking about her work uh to continue to promote
work uh to continue to promote competition in digital markets and AI in
competition in digital markets and AI in the Senate so please do come back for
the Senate so please do come back for those recorded remarks and about about
those recorded remarks and about about 15 in about 15 minutes in 45 minutes
15 in about 15 minutes in 45 minutes I'll give you more than 15 minutes uh
I'll give you more than 15 minutes uh and see you
seat all right um welcome back so for those of you who don't know me uh my
those of you who don't know me uh my name is jalo streer I'm the executive
name is jalo streer I'm the executive director of seer um we had a really
director of seer um we had a really great productive morning uh lots of
great productive morning uh lots of great discussions I really look forward
great discussions I really look forward to the second half of the conference uh
to the second half of the conference uh so thank you all for being here and I
so thank you all for being here and I also want to say thank you to our
also want to say thank you to our partners at the Department of Justice
partners at the Department of Justice and also at The Graduate School of
and also at The Graduate School of Business um so next we will hear from uh
Business um so next we will hear from uh some remarks by senator Amy kachar from
some remarks by senator Amy kachar from Minnesota I will not read her long list
Minnesota I will not read her long list of achievements uh but I just want to
of achievements uh but I just want to mention Senator clar is the first woman
mention Senator clar is the first woman elected to represent the state of
elected to represent the state of Minnesota in the US Senate and her 2021
Minnesota in the US Senate and her 2021 book antitrust was on the New York Times
book antitrust was on the New York Times best sellers list so let's hear from the
best sellers list so let's hear from the senator hello to everyone gathered for
senator hello to everyone gathered for today's important discussion on
today's important discussion on promoting competition in AI thank you to
promoting competition in AI thank you to everyone at Stanford and the justice
everyone at Stanford and the justice department who work to put this event
department who work to put this event together including my friend Assistant
together including my friend Assistant Attorney General Jonathan caner he's
Attorney General Jonathan caner he's just been a little bit busy lately like
just been a little bit busy lately like the work on Ticket Master I wish I could
the work on Ticket Master I wish I could be there with you in person as you know
be there with you in person as you know I actually wrote a book on
I actually wrote a book on competition and many of you may remember
competition and many of you may remember that during the presidential campaign I
that during the presidential campaign I had kind of a rivalry going with Pete
had kind of a rivalry going with Pete bootes well the first thing you should
bootes well the first thing you should know is we're actually good friends but
know is we're actually good friends but why did is this relevant to your topic
why did is this relevant to your topic today well when the campaign ended Pete
today well when the campaign ended Pete put out a New York Times bestselling
put out a New York Times bestselling book called trust not to be outdone
book called trust not to be outdone eight months later I put out my own New
eight months later I put out my own New York Times bestselling book for two
York Times bestselling book for two weeks called
weeks called antitrust it is no secret that our
antitrust it is no secret that our country has a competition problem that's
country has a competition problem that's why like you I'm working to restore
why like you I'm working to restore competition in our economy especially
competition in our economy especially when it comes to our digital markets
when it comes to our digital markets amid the rise of AI this work could not
amid the rise of AI this work could not be more important as we all know AI
be more important as we all know AI brings both opportunity and
brings both opportunity and uncertainty I think David Brooks put it
uncertainty I think David Brooks put it well when he wrote the people in I in in
well when he wrote the people in I in in AI seem to be experiencing radically
AI seem to be experiencing radically different brain States all at once I
different brain States all at once I found it incredibly hard to write about
found it incredibly hard to write about AI he said because it is literally
AI he said because it is literally unknowable whether this technology is
unknowable whether this technology is leading us to heaven or hell
leading us to heaven or hell his point is particularly resonant when
his point is particularly resonant when it comes to competition in AI the
it comes to competition in AI the emergence of any new technology can
emergence of any new technology can create enormous opportunities
create enormous opportunities opportunities when it comes to new
opportunities when it comes to new technologies I know Minnesota home of
technologies I know Minnesota home of Mayo Clinic we see enormous
Mayo Clinic we see enormous possibilities for curing diseases and
possibilities for curing diseases and treating diseases it also creates
treating diseases it also creates enormous opportunities for competition
enormous opportunities for competition new companies have seemingly spouted up
new companies have seemingly spouted up overnight firms no one had heard of a
overnight firms no one had heard of a few years ago like open Ai and anthropic
few years ago like open Ai and anthropic are becoming household names we've also
are becoming household names we've also seen a wave of smaller startups forming
seen a wave of smaller startups forming around AI Technologies with an estimated
around AI Technologies with an estimated more than 300 billion invested into over
more than 300 billion invested into over 25,000 Ai and machine learning startups
25,000 Ai and machine learning startups over the past three years but we may not
over the past three years but we may not necessarily be entering a golden age of
necessarily be entering a golden age of competition many of the most prominent
competition many of the most prominent AI startups are partner with the big
AI startups are partner with the big Tech incumbents rather than competing
Tech incumbents rather than competing with them of course business
with them of course business Partnerships Investments and even
Partnerships Investments and even Acquisitions are not necessarily
Acquisitions are not necessarily anti-competitive we all know that but we
anti-competitive we all know that but we can't make the same mistake with AI that
can't make the same mistake with AI that we did with prior Technologies by
we did with prior Technologies by allowing markets to consolidate and
allowing markets to consolidate and turning a blind eye to exclusionary
turning a blind eye to exclusionary conduct that's why we need to look out
conduct that's why we need to look out for anti-competitive behavior all the
for anti-competitive behavior all the way down the AI supply chain we must
way down the AI supply chain we must ensure the markets that are vital to the
ensure the markets that are vital to the safe and responsible development of AI
safe and responsible development of AI allow for the entry of new Innovative
allow for the entry of new Innovative startups that can compete the
startups that can compete the development of AI models requires
development of AI models requires enormous amounts of computing power that
enormous amounts of computing power that often only large firms can provide yet
often only large firms can provide yet cloud computing is largely controlled by
cloud computing is largely controlled by the very same firms developing the most
the very same firms developing the most advanced AI models as a result companies
advanced AI models as a result companies May preference proprietary AI models or
May preference proprietary AI models or products rather than supporting new or
products rather than supporting new or smaller firms terms even further down
smaller firms terms even further down the supply chain only one company
the supply chain only one company currently makes chips best suited for
currently makes chips best suited for the most advanced AI products of course
the most advanced AI products of course there's nothing wrong with making the
there's nothing wrong with making the best chips operating data centers or
best chips operating data centers or investing in startups but we must ensure
investing in startups but we must ensure that these markets are governed by
that these markets are governed by competitive market dynamics not Monopoly
competitive market dynamics not Monopoly power barriers to entry and
power barriers to entry and consolidation times of technological
consolidation times of technological transformation require critical thinking
transformation require critical thinking and Nuance to sure Innovation benefits
and Nuance to sure Innovation benefits consumers and businesses of all sizes
consumers and businesses of all sizes that's why discussions like this one are
that's why discussions like this one are so important it's also why I'm working
so important it's also why I'm working to ensure that our antitrust enforcers
to ensure that our antitrust enforcers have the resources and tools necessary
have the resources and tools necessary to carry out this vital task in many
to carry out this vital task in many cases these agencies even when you go
cases these agencies even when you go back to the time of Richard Nixon are
back to the time of Richard Nixon are shadows of their former C in terms of
shadows of their former C in terms of numbers of employees so last Congress we
numbers of employees so last Congress we passed my merger fee change legislation
passed my merger fee change legislation with Senator Chuck Grassley to give
with Senator Chuck Grassley to give enforcers at the FDC and at Justice the
enforcers at the FDC and at Justice the tools they need to keep a watchful eye
tools they need to keep a watchful eye over consolidation and anti-competitive
over consolidation and anti-competitive practices across the economy I also
practices across the economy I also continue to push Congress to pass my
continue to push Congress to pass my bipartisan Bill to prohibit online self
bipartisan Bill to prohibit online self preferencing by dominant Tech platforms
preferencing by dominant Tech platforms including AI platforms they spent the
including AI platforms they spent the tech companies about $300 million
tech companies about $300 million against a bill that's a minimum figure
against a bill that's a minimum figure uh in t tv ads I think at some point my
uh in t tv ads I think at some point my time will come I think at some point
time will come I think at some point when you look at what's going over the
when you look at what's going over the world we will come to an agreement on
world we will come to an agreement on some rules of the road when it comes to
some rules of the road when it comes to Tech
Tech competition to promote competition
competition to promote competition across all sectors of eon our economy I
across all sectors of eon our economy I also recently reinduced reintroduced my
also recently reinduced reintroduced my comprehensive legislation to empower
comprehensive legislation to empower enforcers to crack down on
enforcers to crack down on anti-competitive Acquisitions and
anti-competitive Acquisitions and exclusionary conduct that's a different
exclusionary conduct that's a different Bill yes we have a lot of work to do but
Bill yes we have a lot of work to do but as David Brooks wondered whether AI is
as David Brooks wondered whether AI is leading us to heaven or hell it's not
leading us to heaven or hell it's not out of our hands we don't just throw up
out of our hands we don't just throw up our hands like I'm afraid we did in the
our hands like I'm afraid we did in the past which has led to a lot of these
past which has led to a lot of these issues of what kids are exposed to in
issues of what kids are exposed to in the internet uh what's happening to our
the internet uh what's happening to our news organizations and democracy right
news organizations and democracy right now we have it in our power to put some
now we have it in our power to put some rules of the road in place we are not
rules of the road in place we are not just passive bystanders just few weeks
just passive bystanders just few weeks ago we passed three bipartisan bills out
ago we passed three bipartisan bills out of the rules committee that I chair to
of the rules committee that I chair to protect our elections from deceptive
protect our elections from deceptive uses of AI and that includes Banning
uses of AI and that includes Banning deep fakes a bill that I have with Josh
deep fakes a bill that I have with Josh Holly and Susan Collins that includes
Holly and Susan Collins that includes disclaimers on certain AI videos and
disclaimers on certain AI videos and Robo calls and the like otherwise people
Robo calls and the like otherwise people aren going to know if they're seeing the
aren going to know if they're seeing the candidate they love or the candidate
candidate they love or the candidate they don't like they won't know if
they don't like they won't know if they're real we cannot do that to our
they're real we cannot do that to our democracy and as a chair of of the
democracy and as a chair of of the competition policy and I trust in
competition policy and I trust in consumer rights subcommittee I'm
consumer rights subcommittee I'm fighting to ensure our markets remain
fighting to ensure our markets remain Fair open and contestable working
Fair open and contestable working together we can ensure that competition
together we can ensure that competition continues to drive innovation in
continues to drive innovation in America's economy for generations to
America's economy for generations to come I love Innovation I come from the
come I love Innovation I come from the state that gave the world everything
state that gave the world everything from the Post-it to the pacemaker and
from the Post-it to the pacemaker and certainly great great Innovations have
certainly great great Innovations have come out of Northern California but it
come out of Northern California but it is on all of us all of us to ensure that
is on all of us all of us to ensure that we see that next great product that next
we see that next great product that next gr Innovation some of the exciting
gr Innovation some of the exciting things we're seeing in AI but if we
things we're seeing in AI but if we don't put the guard rails in place that
don't put the guard rails in place that eventually is not going to work out for
eventually is not going to work out for people so I'm asking you to join us with
people so I'm asking you to join us with a business mind actually with a mind
a business mind actually with a mind that says we love competition we love
that says we love competition we love capitalism that's what's gotten us to
capitalism that's what's gotten us to where we are in America but we also know
where we are in America but we also know what Adam Smith warned our founding
what Adam Smith warned our founding father s watch out for the unbridled
father s watch out for the unbridled power of the army of monopolies if you
power of the army of monopolies if you don't put the guard rules in place you
don't put the guard rules in place you lose that Competitive Edge thank you
lose that Competitive Edge thank you enjoy today's discussion you got a lot
enjoy today's discussion you got a lot on your plate but I know you're up for
on your plate but I know you're up for it thanks
everyone okay so um now let's move on to the next panel uh our next panel will
the next panel uh our next panel will cover competition and innovation in
cover competition and innovation in Foundation models uh we will hear about
Foundation models uh we will hear about how companies are incorporating AI in
how companies are incorporating AI in their products and services and how
their products and services and how investors think about the value of AI
investors think about the value of AI for the world and the regulation in AI
for the world and the regulation in AI so this is a very complicated area very
so this is a very complicated area very important uh I'm really thrilled to
important uh I'm really thrilled to introduce our moderator uh professor
introduce our moderator uh professor mosam bayadi professor bayti is an
mosam bayadi professor bayti is an associate professor of operations
associate professor of operations information and Technology at Stanford
information and Technology at Stanford University Graduate School of Business
University Graduate School of Business uh so please join me in welcoming
uh so please join me in welcoming professor miad and uh the panel to the
professor miad and uh the panel to the [Applause]
welcome everyone and thank you for attending this panel uh where we explore
Chrome Extension