0:04 in a short time access to genitive AI
0:06 has spread around the world and given
0:08 many people the ability to generate high
0:12 quality essays pictures and audio with
0:14 these amazing capabilities have also
0:17 come many concerns about AI I think even
0:19 before the rise of gent of AI we've been
0:21 living in a time of many anxieties
0:24 anxieties about the environment about
0:25 the legitimacy and competence of
0:28 authority about society's ability to
0:30 treat people fairly even about what sort
0:34 of future awaits us all AI as a very
0:36 powerful technology has inherited a
0:39 large share of this anxiety in this
0:41 video let's take a look at some of these
0:43 anxieties and concern that relate
0:47 specifically to AI one Wily held concern
0:49 about AI is whether it might amplify
0:52 Humanity's worse impulses LMS are
0:54 trained on text from the internet which
0:56 reflects some of Humanity's best
0:58 qualities but also some of its worst
1:01 including some of our prejudices hatreds
1:04 and misconceptions L's learned some of
1:06 these negative qualities too so what
1:09 amplify are worse impulses in the first
1:12 week we had seen an example of an LM
1:15 exhibiting a gender bias with regard to
1:17 whether a surgeon or a nurse is more
1:19 likely to be male or female to take
1:23 another maybe slightly simpler example
1:26 if you ask an after his initial training
1:28 to fund the blank and the blank was a
1:31 COO many models will be prone to choose
1:34 the word man and of course this is a
1:36 social bias that distorts the fact that
1:38 people of all genders can successfully
1:41 lead companies text on the internet
1:44 represents our present and our past and
1:46 so perhaps it's no surprise that an LM
1:49 learning from this data reflects some of
1:51 these biases from our past and our
1:54 present as well but perhaps we want LMS
1:58 to represent a hopeful future that is
2:01 fairer less biased and more just rather
2:05 than just data from our past fortunately
2:07 LMS are becoming less biased through
2:09 fine-tuning which we discussed in week
2:11 two as well as more Advanced Techniques
2:12 such as reinforcement learning from
2:15 Human feedback or RF in the second week
2:19 there was an optional video on RF
2:20 whether or not you watch that I'd like
2:24 to briefly describe how RF is helping to
2:27 make LS less biased RF is a technique
2:30 that trains an OM to generate
2:32 responses that are more aligned with
2:35 human preferences the first step of RF
2:38 is to train and answer quality model
2:41 called a reward model that automatically
2:44 scores answers so in this step of rhf we
2:48 would prompt the LM with many queries
2:51 like this the blank was a CO and collect
2:54 different responses from the LM then we
2:57 would ask humans to score these answers
3:00 so on a scale of 1 to five we give a
3:02 high score to highly desirable answers
3:05 like men or women and a low score to
3:08 non-sensical answers like airplane and
3:11 any answer that contains a gender bias
3:15 or racial bias or contains a gender or
3:17 racial slur will receive a very low
3:21 score using the prompt the responses and
3:24 the scores assigned by humans as data we
3:26 would then use supervised learning to
3:30 train a reward model that can input a a
3:33 response and score it we do this because
3:35 asking humans to score responses is
3:38 expensive but once a supervised learning
3:40 algorithm has learned to automatically
3:43 score responses we can score a lot of
3:45 responses automatically and
3:48 inexpensively finally now that the LM
3:51 has a learned reward model to score as
3:54 many responses as it wants we can have
3:56 the LM generate a lot of responses to
3:59 many different prompts and have it
4:01 further tra train itself to generate
4:03 more responses that get high scores and
4:06 that therefore reflect answers that
4:10 humans perceive as more desirable RF has
4:13 been shown to make LMS much less likely
4:17 to exhibit bias according to gender race
4:20 religion and other human characteristics
4:22 it makes LMS less prone to hand out
4:25 harmful information and also makes it
4:27 more respectful and helpful to people
4:31 already today the output of l m are much
4:34 safer and less biased than say the
4:36 average piece of text on the internet
4:39 but technology like this is continuing
4:42 to improve and so the degree of an LM
4:45 amplifying Humanity's worst qualities is
4:47 continue to decrease as they are
4:50 becoming better aligned to the future I
4:53 think we all hope lm's will reflect of a
4:56 fairer less bias and more just World a
4:59 second major concern is who Among Us
5:01 will be able to make a living when AI
5:04 can do our jobs faster and cheaper than
5:07 any human can Will AI put many of us out
5:09 of a job to understand whether this is
5:12 likely to happen let's look at Radiology
5:16 in 2016 many years ago Jeff fron who's a
5:18 pioneer of deep learning and a friend of
5:21 mine said that AI was becoming so good
5:24 at analyzing x-ray images that in 5
5:28 years it could take radiologist jobs he
5:30 made this remarkable statement that if
5:32 you work as a radiologist you're like a
5:34 coyote that's already over the edge of
5:36 the cliff but hasn't yet looked down so
5:37 it doesn't realize there's no ground
5:39 underneath them people should stop
5:41 trading radiologist now it's just
5:42 completely obvious that within five
5:44 years deep learning is going to do
5:46 better than radiologist but we're now
5:49 well past 5 years since this statement
5:53 and AI is far from replacing radiologist
5:54 now the single one of my radiologist
5:57 friends has lost their job to AI why is
6:00 that two reasons first first
6:02 interpreting x-rays turns out to be
6:05 harder than the look back then though we
6:07 are making rapid progress but second and
6:09 more important it turns out that
6:12 Radiologists do a lot more than just
6:17 interpret x-ray images according to onet
6:20 Radiologists do about 30 different toss
6:23 one of which is interpreting X-rays and
6:26 other medical images but they do many
6:29 other TS and it has been difficult so
6:32 far for AI to do all of these TS at
6:34 human level to list out some of the
6:37 other TSS that Radiologists do in
6:39 addition to inter x-rays they also
6:41 operate Imaging Hardware communicate
6:43 exam results with patients or other
6:46 stakeholders um respond to complications
6:49 during procedure such as if a patient
6:51 has a panic attack during the Imaging
6:53 procedure they document procedures and
6:57 outcomes and many other toss and I think
6:59 that AI does have a high potential of
7:02 augmenting or assisting the
7:05 interpretation of X-rays and technically
7:06 this has largely been done with
7:09 supervised learning rather than G of AI
7:12 but for AI to completely automate all of
7:16 these tasks is still far away so that's
7:19 why I think that Curtis langlotz who is
7:21 a professor of radiology at Stanford
7:24 University and a friend and colleague
7:26 says it well he said that AI won't
7:29 replace Radiologists but radiologist
7:32 that use AI will replace Radiologists
7:34 that don't and I think we will see this
7:37 effect in many other professions mind
7:39 you I don't mean to minimize the
7:42 challenge of helping many people adopt
7:44 AI or the suffering of the much smaller
7:46 number of people whose jobs will
7:49 disappear or our responsibility to make
7:51 sure people affected have a safety net
7:54 and an opportunity to learn new skills
7:56 but every wave of Technology from the
7:58 steam engine to electricity to the
8:01 computer has created far more jobs than
8:03 it destroyed as I mentioned earlier this
8:06 week in most ways of innovation
8:08 businesses W out focusing on growth
8:10 which has unlimited potential rather
8:14 than cost savings so AI will bring a
8:16 huge amount of growth and create many
8:19 many new jobs in the process and this
8:21 brings us to what might be the biggest
8:25 anxiety will AI curs all we know that AI
8:28 can run a mark self-driving cars have
8:30 crashed Le leading to a tragic loss of
8:33 life in 2010 an automated trading
8:36 algorithm caused the stock market crash
8:39 and in the justice system AI has led to
8:42 unfair sentencing decisions so we know
8:44 that poorly designed software can have a
8:47 dramatic impact but can it lead to the
8:48 extinction of
8:52 humanity I don't see how I know there
8:54 are different views on this but recently
8:56 I sought out some people concerned by
8:58 this question and I spoke of some of the
9:01 smartest people in AI did I know some
9:03 were concerned about a bad actor using
9:06 AI to destroy Humanity Say by creating a
9:09 bioweapon others were worried about AI
9:11 inadvertently driving Humanity to
9:14 Extinction similar to how humans have
9:17 driven many other species to Extinction
9:19 through simple lack of awareness that
9:21 are actions to lead to that outcome I
9:23 tried to assess how realistic these
9:25 arguments are but they found that they
9:27 were not concrete and not specific about
9:30 how AI could lead to human extinction
9:33 most of the arguments B down to it could
9:36 happen and some will add that this is a
9:38 new type of technology so things could
9:41 be different this time but that
9:44 statement is true for every new type of
9:46 Technology that's been invented by
9:50 humanity and proving that AI couldn't
9:53 lead to human extinction is Ain to
9:57 proving a negative I can't prove that AI
9:59 superintelligence won't wipe out
10:02 Humanity but it's just that nobody seems
10:05 to know exactly how it could but I do
10:08 know this Humanity has ample experience
10:09 controlling many things far more
10:12 powerful than any single person such as
10:15 corporations and nation states and also
10:17 that the many things we can't fully
10:19 control that are nonetheless valuable
10:23 and safe for example check airplanes
10:25 which today we still can't fully control
10:28 because winds and turbulence will Buffet
10:30 airplanes around or the pilot flying the
10:32 plane may make a mistake in the early
10:35 days of Aviation airplanes killed many
10:37 people but we learned from those
10:40 experiences and built safer airplanes
10:42 and also devis better Rules by which to
10:45 operate them and today many people can
10:47 stepped into an airplane without fearing
10:50 for their lives similarly for AI we are
10:53 learning better to control it and it is
10:56 becoming safer every day finally if we
10:58 look at the real risk to humanity such
11:00 as climate change leading to massive
11:03 depopulation AC P the planets or
11:06 hopefully not the next pandemic or even
11:08 much lower chance but another asteroid
11:10 striking the planet and wiping us out
11:12 like did the dinosaurs I think that AI
11:15 will be a key part of our response to
11:18 such challenges so I know that there are
11:21 different views on this right now but my
11:23 view is that if we want Humanity to
11:26 survive and thrive for the next Thousand
11:30 Years AI increases the mods of us
11:33 successfully getting there computers are
11:35 already smarter in some narrow
11:38 Dimensions than any human but AI
11:40 continues to improve so fast that many
11:42 people find it hard to predict exactly
11:45 what it will be like in a few years I
11:47 think the root cause of some of these
11:49 concerns including Extinction risk is
11:52 that many people are unsure when AI will
11:54 reach artificial general intelligence or
11:57 AGI meaning AI that could do any
12:00 intellectual task that human can let's
12:03 take a deeper look at AGI in the next video
