YouTube Transcript:
Supervised Learning: Crash Course AI #2
Skip watching entire videos - get the full transcript, search for keywords, and copy with one click.
Share:
Video Transcript
Available languages:
View:
Hey, I’m Jabril, and this is CrashCourse AI!
Today, we’re going to try to teach John Green-bot something.
Hey John Green-bot!
John Green-bot: “Hello humanoid friend!”
Are you ready to learn?
John Green-bot: “Hello humanoid friend!”
As you can see, he has a lot of learning to do, which is the basic story of all artificial
intelligence.
But it’s also our story.
Humans aren’t born with many skills, and we need to learn how to sort mail, land airplanes,
and have friendly conversations.
So computer scientists have tried to help computers learn like we do, with a process
called supervised learning.
You ready, John Green-bot?
John Green-bot: “Hello humanoid friend!”
The process of learning is how anything can make decisions, like for example humans, animals,
or AI systems.
They can adapt their behavior based on their experiences.
In Crash Course AI, we’ll talk about three main types of learning: Reinforcement Learning,
Unsupervised Learning, and Supervised Learning.
Reinforcement Learning is the process of learning in an environment, through feedback from an
AI’s behavior, it’s how kids learn to walk!
No one tells them how, they just practice, stumble, and get better at balancing until
they can put one foot in front of the other.
Unsupervised Learning is the process of learning without training labels.
It could also be called clustering or grouping.
Sites like YouTube use unsupervised learning to find patterns in the frames of a video,
and compress those frames so that videos can be streamed to us quickly.
And Supervised Learning is the process of learning with training labels.
It’s the most widely used kind of learning when it comes to AI, and it’s what we’ll
focus on today and in the next few videos!
Supervised learning is when someone who knows the right answers, called a supervisor, points
out mistakes during the learning process.
You can think of this like when a teacher corrects a student’s math.
In one kind of supervised setting, we want an AI to consider some data, like an image
of an animal, and classify it with a label, like “reptile” or “mammal.”
AI needs computing power and data to learn.
And that’s especially true for supervised learning, which needs a lot of training examples
from a supervisor.
After training this hypothetical AI, it should be able to correctly classify images it hasn’t
seen before, like a picture of a kitten as a mammal.
That’s how we know it’s learning instead of just memorizing answers.
And supervised learning is a key part of lots of AI you interact with every day!
It’s how email accounts can correctly classify a message from your boss as important, and
ads as spam.
It’s how Facebook tells your face apart from your friend’s face so that it can make
tag suggestions when you upload a photo.
And it’s how your bank may decide whether your loan request is approved or not.
Now, to initially create this kind of AI, computer scientists were loosely inspired
by human brains.
They were mostly interested in cells called neurons, because our brains have billions
of them.
Each neuron has three basic parts: the cell body, the dendrites, and the axon.
The axon of one neuron is separated from the dendrites of another neuron by a small gap
called a synapse.
And neurons talk to each other by passing electric signals through synapses.
As one neuron receives signals from other neurons, the electric energy inside of its
cell body builds up until a threshold is crossed.
Then, an electric signal shoots down the axon, and is passed to another neuron -- where everything
repeats.
So the goal of early computer scientists wasn’t to mimic a whole brain.
Their goal was to create one artificial neuron that worked like a real one.
To see how, let’s go to the Thought Bubble.
In 1958, a psychologist named Frank Rosenblatt was inspired by the Dartmouth
Conference and was determined to create an artificial neuron.
His goal was to teach this AI to classify images as “triangles” or “not-triangles”
with his supervision.
That’s what makes it supervised learning!
The machine he built was about the size of a grand piano, and he called it the Perceptron.
Rosenblatt wired the Perceptron to a 400 pixel camera, which was hi-tech for the time, but
is about a billion times less powerful than the one on the back of your modern cellphone.
He would show the camera a picture of a triangle or a not-triangle, like a circle.
Depending on if the camera saw ink or paper in each spot, each pixel would send a different
electric signal to the Perceptron.
Then, the Perceptron would add up all the signals that match the triangle shape.
If the total charge was above its threshold, it would send an electric signal to turn on
a light.
That was artificial neuron speak for “yes, that’s a triangle!”
But if the electric charge was too weak to hit the threshold,
it wouldn’t do anything and the light wouldn’t turn on, that meant “not a triangle.”
At first, the Perceptron was basically making random guesses.
So to train it with supervision, Rosenblatt used “yes” and “no” buttons.
If the Perceptron was correct, he would push the “yes” button and nothing would change.
But if the Perceptron was wrong, he would push the “no” button, which set off a
chain of events that adjusted how much electricity crossed the synapses, and adjusted the machine’s
threshold levels.
So it’d be more likely to get the answer correct next time!
Thanks, Thought Bubble.
Nowadays, rather than building huge machines with switches and lights, we can use modern
computers to program AI to behave like neurons.
The basic concepts are pretty much the same:
First, the artificial neuron receives inputs multiplied by different weights, which correspond
to the strength of each signal.
In our brains, the electric signals between neurons are all the same size, but with computers,
they can vary.
The threshold is represented by a special weight called a bias, which can be adjusted
to raise or lower the neuron’s eagerness to fire.
So all the inputs are multiplied by their respective weights, added together, and a
mathematical function gets a result.
In the simplest AI systems, this function is called a step function, which only outputs
a 0 or a 1.
If the sum is less than the bias, then the neuron will output a 0, which could indicate
not-triangle or something else depending on the task.
But If the sum is greater than the bias, then the neuron will output a 1, which indicates
the opposite result!
An AI can be trained to make simple decisions about anything where you have enough data
and supervised labels: triangles, junk mail, languages, movie genres, or even similar looking
foods.
Like donuts and bagels.
Hey John Green-bot!
You want to learn how to sort some disgusting bagels from delicious donuts?”
John Green-bot: “Hello humanoid friend!”
John Green-bot still has the talk-like-a-human program!
Remember that we don’t have generalized AI yet… that program is pretty limited.
So I need to swap this out for a perceptron program.
Now that John Green-bot is ready to learn, we’ll measure the mass and diameter of some
bagels and donuts, and supervise him so he gets better at labeling them.
How about you hold on to these for me?
Right now, he doesn’t know anything about bagels or donuts or what their masses and
diameters might be.
So his program is initially using random weights for mass, diameter, and the bias to help make
a decision.
But as he learns, those weights will be updated!
Now, we can use different mathematical functions to account for how close or far an AI is from
the correct decision, but we’re going to keep it simple.
John Green-bot’s perceptron program is using a step function, so it’s an either-or choice.
0 or 1.
Bagel or donut.
Completely right or completely wrong.
Let’s do it. This here is a mixed batch of bagels and donuts.
This first item has a mass of 34 grams and a diameter of 7.8 centimeters.
The perceptron takes these inputs (mass and diameter), multiplies them by their respective
weights, then adds them together.
If the sum is greater than the bias -- which, remember, is the threshold for the neuron
firing -- John Green-bot will say “bagel.”
So if it helps to think of it this way, the bias is like a bagel threshold.
If the sum is less than the bias, it hasn’t crossed the bagel threshold, and John Green-bot
will say “donut.”
All this math can be tricky to picture.
So to visualize what’s going on, we can think of John Green-bot’s perceptron program
as a graph, with mass on one axis and diameter on the other.
The weights and bias are used to calculate a line called a decision boundary on the graph,
which separates bagels from donuts.
And if we represent this same item as a data point, we’d graph it at 34 grams and 7.8
centimeters.
This data point is above the decision boundary, in the bagel zone!
So all this means is that when I ask John Green-bot what this food is… he’ll say:
John Green-bot: “Bagel!”
And... he got it wrong, because this is a donut.
No big deal!
With a brand new program, he’s like a baby that made a random guess!
Because he’s using random weights right now.
But we can help him learn by updating-- his weights.
So we take an old weight and add a number calculated by an equation called the update
rule.
We’re going to keep this conceptual, but if you want more information about this equation,
we’ve linked to a resource in the description.
Now because our perceptron can only be completely right or completely wrong, the update rule
ends up being pretty simple.
If John Green-bot made the right choice, like labeling a donut as a donut, the update rule
works out to be 0.
So he adds 0 to the weight, and the weight stays the same.
But if John Green-bot made the wrong choice, like labeling a donut as a bagel, the update
rule will have a value -- a small positive or negative number.
He’ll add that value to the weight, and the weight will change.
Conceptually, this means John Green-bot learns from failure but not from success.
So he called this donut a bagel, and got the label wrong.
By pressing this “no” button, I’m supervising his learning and letting him know he made
the wrong choice.
So his weights update.
If we look back at the graph, we can see that when the weights update, the decision boundary
changes.
The data point we added is now below the line, in the donut zone.
Now, his perceptron will classify another item with this mass and diameter as a donut!
This next item [donut] has a mass of 26 grams and a diameter of 6.1 centimeters.
What do you think, John Green-bot?
John Green-bot: “Donut!”
He got it right!
When he took those inputs and did that same calculation, the sum was less than the bias.
That data point appeared below the decision boundary -- in the donut zone.
And so I’m going to push the “yes” button.
In this case, the update rule equation works out to 0, so the weights stay the same, and
so does the decision boundary.
Now we’ll do this 48 more times to train his perceptron.
After we’re done training John Green-bot’s perceptron, we have to test it on new data
to see how well he learned.
So I’ve got 100 new bagels and donuts for him to classify.
Woah. This is a big what? What is this?
John Green Bot: "Bagel."
Alright, alright. I'm just going to write down your answer.
Alright so overall, he classified 25 donuts and 75 bagels.
We can visualize the results on the graph with the decision boundary like this.
But we can also put the results in a table, called a confusion matrix, because it tells
us where John Green-bot was confused.
He got 8 donuts correct and 73 bagels correct.
But he said that a bagel was a donut twice, and that a donut was a bagel 17 times.
Using these numbers, we can calculate his overall accuracy by adding together what he
got right, which were 8 donuts and 73 bagels, and dividing by the total 100, to get 81%.
But to really understand what’s wrong, we need to look at his precision and his recall.
We can calculate these percentages for both foods, but we’ll focus on donuts right now.
Precision tells you how much you should trust your program when it says it's found something.
If John Green-bot tells me something’s a donut, I’m expecting to eat a donut.
I don’t want to bite into a bagel, because that would be a gross surprise.
Of the 10 items that he said were donuts, 8 were actually donuts.
So he was 80% precise, and I can be 80% sure he’s only handing me donuts when he says
he is.
Recall tells you how much your program can find of the thing you're looking for.
I’m really hungry, so I want as many donuts as possible.
But of the 25 items that were donuts, he correctly labeled 8 of them.
So his recall was just 32%, and he just handed me 32% of all the donuts.
The precision and recall depend on the criteria John Green-bot is using to make a decision:
diameter and mass.
And as we can see from this graph, he thinks that donuts generally have smaller diameters
and masses than bagels -- they’re small, fluffy treats.
So when it comes to classifying donuts, he has high precision.
Because if he says something’s a donut, we’re pretty sure it’s a donut, not a
disgusting bagel.
But John Green-bot has low recall, because this criteria didn’t account for the fact
that some donuts can be way bigger than the donuts we
used to train his perceptron.
They have a bigger diameter and mass, and they fall in the current “bagel zone,”
so he missed a lot of donuts when he was classifying.
Thanks John Green Bot.
Figuring out what criteria to use is the key to most AI challenges.
If we wanted better accuracy for this donut-bagel problem, maybe we should of used inputs besides
mass and diameter, like checking for seeds or sprinkles.
Generally, more inputs are better for accuracy, but the AI will need more processing power
and time to make decisions.
An ideal AI system would be small, powerful, and have perfect precision and perfect recall.
But in the real world, mistakes happen, so we have to prioritize based on our goals.
The AI filtering our inboxes needs to make sure we get all the important
emails, so it needs high recall.
But it’s okay if it isn’t very precise, because we can deal with some spam getting
through and don’t need only good emails.
Most AIs handle more complicated problems than sorting something into one of two categories,
though.
The world isn’t all donuts and bagels.
So to answer more complicated questions, we need more complicated AI.
Next time, we’ll combine artificial neurons to create an artificial neural network.
See you then!
Crash Course is produced in association with PBS Digital Studios.
If you want to help keep all Crash Course free for everybody, forever, you can join
our community on Patreon.
And if you want to learn more about how the human brain and nervous system works, check
out our Anatomy & Physiology videos about them.
Click on any text or timestamp to jump to that moment in the video
Share:
Most transcripts ready in under 5 seconds
One-Click Copy125+ LanguagesSearch ContentJump to Timestamps
Paste YouTube URL
Enter any YouTube video link to get the full transcript
Transcript Extraction Form
Most transcripts ready in under 5 seconds
Get Our Chrome Extension
Get transcripts instantly without leaving YouTube. Install our Chrome extension for one-click access to any video's transcript directly on the watch page.
Works with YouTube, Coursera, Udemy and more educational platforms
Get Instant Transcripts: Just Edit the Domain in Your Address Bar!
YouTube
←
→
↻
https://www.youtube.com/watch?v=UF8uR6Z6KLc
YoutubeToText
←
→
↻
https://youtubetotext.net/watch?v=UF8uR6Z6KLc