YouTube Transcript:
Statistics For Data Science: COMPLETE Course For Beginners (2025)| Statistics Tutorial | Intellipaat

Skip watching entire videos - get the full transcript, search for keywords, and copy with one click.

Video Transcript

Video Summary

Summary

Core Theme

This content is a comprehensive tutorial on statistics for data science, covering fundamental concepts from descriptive statistics to probability distributions and hypothesis testing, with practical Python implementations. It emphasizes building a strong statistical foundation essential for aspiring data scientists.

Mind Map

Click to expand

Click to explore the full interactive mind map • Zoom, pan, and navigate

[Music]

Welcome to Intellipad Statistics for

Data Science full course. Your complete

tutorial to master the most important

pillar of data science, statistics. If

you want to become a data scientist,

here's one truth you can't ignore.

Statistics is non-negotiable. That is

why we have created the statistics for

data science course designed for

absolute beginners who want to build

strong foundation that lead to real

world data science jobs. Let's face it,

machine learning, AI, even deep

learning, all of it stands on the

shoulder of statistical thinking. And in

this statistics for data science

tutorial, you will not only learn theory

but apply statistics principle

practically to different problem

statements. So not memorizing anything

here. You're solving data problems

hands-on using stat and little bit of

Python. In this statistics for data

science full course, we will start from

scratch. You will learn what is

statistics and data size, descriptive

versus inferial statistics, mean,

median, mode, variance, standard

deviation, probability theory and

distribution, binomial, normal, poison,

hypothesis testing, p values, confidence

interval, zed test, t test, ANOVA. We

have also included hands-on coding,

implementation of these stack concepts,

their real world example and jaw focused

explanations. So you will know exactly

why you are learning each concepts and

how it is used by data scientists in

companies like Amazon, Google and

Netflix. By the end of statistics course

for data science, you will be ready to

take a dig at machine learning and make

sense of numbers and figure out the

patterns between them. So let's get started.

started.

So tax is basically a branch of

mathematics, right? So how can I give

this answer is stat

large amount of data. So if we have a

smaller like suppose if I'm having a

company and I have only 10 employees

there. So I can easily manage the data

myself. I do not need any tools or

something to manage it. But if I'm

having large amount of data that time it

is difficult for me to manually analyze

the data. So that's why we need the

tools of stats.

stats.

Now the next question which is asked is

like okay we understood what is

statistics but why it is important for us.

Why is statistics important for us being

a data analyst or being a data

So guys, everybody knows what is the

difference between a raw data and information.

information.

Everybody knows what is the difference

Raw data means meaningless data, right?

which do not have any meaning and

information means meaningful data. Yes

So don't you think so your statistics is

actually helping you to convert your raw

So with the help of statistics we are

actually converting our raw data into

into information.

Also we can say that

also we can say that

from large data sets that we are

decisions etc. So this is how we can

answer this question.

from large data sets and which helps me

to make predictions, decisions and

all other things. So now we understood

that our data science or data analyst

field is full of analysis, analyzing the

data, doing predictions, finding

something and all that stuff. So at

stats also have the quality to do that.

So how much important this concept is

for us, right? I hope you got a basic

idea about this.

So now let us start with the another

topic which is population versus sample. Okay.

Okay.

So first let me explain you in a layman

language what exactly population and

sample is. Now suppose we live in a

planet of earth. Right?

Suppose this is your earth. If I want to

divide this earth into subp parts then I

think continents. So how many continents

we are having? That is seven. Right? So

I divided this earth into seven

different continents.

Right? That means the entire entity what

we are talking about is population.

the entire entity and the sub part of

Now how we can talk about when we have a

data frame that the entire data set what

we are having the entire CSV file what

we are having is the population and if I

take sub part of that data if I take the

sub part of that data that will be our

sample. So like how we do it in real

life like suppose you have some

employees data okay all employees data.

So all the managers all the HRs all the

interns all the senior employees

everyone have the data here and from

this your question is you need to find

what is the average salary of a managers

in the company. So what you will do is

you will filter out the data in which

you will have a sample only of managers

and from this data you will find what is the

the

average salary

of managers. Clear? Now the question

arises if this part is clear the

question arises is how do we choose samples?

samples?

Okay, we have some techniques guys

through which we decide samples and that

techniques are known as sampling

techniques. So there are many sampling

techniques but three most important

sampling techniques which we are going

to discuss. Okay. So the first sampling

technique is known as random sampling.

Okay. The second sampling technique

which is known as stratified sampling

and the third sampling technique is

known as systematic sampling. So I will

give you a hint and you will tell me

about these sampling techniques guys.

The thing is when you you know face

interviews it's not important that you

are you heard about all the concepts all

the terms there right? There might be

some new terms which will come in

interviews but according to the name you

need to answer something. It's very bad

if you are sitting blank in the

interview. So at least you need to say

something. So let's practice that thing.

Okay. So first we will talk about random

sampling. So from the name itself random

I think it's easy. What is random

sampling? So yes means taking data

randomly right? So nothing no pattern

nothing just randomly I will select like

suppose from the entire school you need

to randomly pick five students

irrespective of what class they are what

gender they are what age they are you

just randomly picking five student right

second is stratified sampling now from

the stratified word I think a strategy

English word strategy comes into my

So what could be this stratified sample?

So don't you think so the example which

we took above that from the entire

employees data I want the data I want

the sample only of managers. So because

I was having a strategy in my mind

statement was I need to find the average

salary of managers only. So when I was

creating sample I was thinking that only

managers should come in my data. So

don't you think so this example is of

stratified sampling where I am putting

the condition that designation of that

employee should be of managers only. So

I'm putting some conditions here. So

stratified sampling can also be known as

conditionalbased sampling where you are

putting some conditions.

Clear guys? So I'll give you one more example

example

like from the entire data set of all the

genders I want to create a sample of

women's only.

The third is systematic sampling.

So from the word systematic a systematic

order comes into my mind. Yes or no? A

systematic order it is ordered based

sampling. Order based what does it means

that like you are creating a sample.

Okay, you are creating this sample and

in this sample you are saying that I

want taking

every nth element. Every

Every

nth element. What does this means is

like suppose in your data points are

segregated like this 1 2 3 4 5 6 7 8 9

10 11 12 13 14.

You are creating a sample in which you

are saying I want every third element.

every third element.

So what points you will get in your

sample is I think 3 6 9

12. So you will be having a sample in

which you will be having the points 3 6

9 and 12.

So a order is maintained in this type of sampling.

sampling.

Okay. Let me make it more clear by

giving you the exact difference of three

of the sampling techniques. Okay. So

suppose you have a data in which the

points are stored like this. Male,

Male, female,

female,

male, male, female, female and male.

Suppose like this the data is arranged.

So if I want to perform random sampling

here. So in random sampling I can get

anything I can even get males I can even

get females anything I can get right if

I'm doing stratified sampling in

stratified sampling I am saying that I

want only females

only females so only females data I will get

get

number of females present I will get

only females data now if I want to do

systematic here in systematic I am

saying I want every second element. So

what elements I will get is first I will

get female then I will get male then I

will get again female. So like this I

will get the data getting the difference

between three of the sampling

techniques. Now my question to you is

you need to find in real time what is

the use of these three sampling techniques.

techniques.

So I'll give you one example of

systematic. Okay. So in systematics

suppose you're working in some factory

okay and you are there um product

management you're a manager of the

products there and you want to see the

production of the products now see like

if any new product is launched like we

talk about iPhones so recently which

iPhone is launched iPhone 15 was launched

launched

now when this iPhone 15 was launched do

you think they will produce lakhs of products.

products.

Do you think so? That they will produce

one lakh, 10 lakh or even kes of

products at one go which usually they

don't. Why? because they will first

create some samples like suppose uh this

is not the estimate number I'm just

giving you an idea like suppose they

will pin 10,000 phones in the market and

they will see what is the sales of this

10,000 if the sales is high they will

produce more phones if the sales is low

they will slow down the production

that's how any industry works now see

from that 10,000 phones what they have

buyed or what they have basically made.

Do you think so? They will manually

check each and every product that that

is working fine or not. No. That they

will convert this into batches. Batches

means batch one maybe of 500 phones.

Then 500 phones then 500 from this 500

phones. So suppose this is your

population now or 500 phones. And now

from this batch they will check every

second or every third product. Every

second or every third product. So if

there's like suppose 99.9%

of the checked products are working fine

that most probably this batch is fine.

This is how it works right means most

probably 500 phones will work fine.

Similarly they will check for this is

the example of systematic. So instead of

looking for each and every product

individually, they will look for every

second mobile or every third mobile and

see the percentage of like suppose they

checked 10 mobiles here. Now from this

10 mobiles all the 10 were working fine

or eight were working fine or five were

working fine. So 50% was fine, 99 was

fine or 100% was fine. So if the

percentage is good like they're saying

99% of the phones are working fine means

out of nine out of 10 nine phones were

working fine. So most probably the batch

of 500 will also work fine. Getting the

point? So here we can use systematic

sampling when we need to check the

condition or we need to check any

specific product in factories. So if it

is clear let us move to the next topic

which is central tendencies

central tendencies.

So suppose you have a data okay and you

are plotting the histogram of data. So

this is your x-axis and what is there in

y-axis of uh this histogram? What is the

y-axis of histogram? It's frequency or

count. Okay. So if you forget what

exactly histogram is, I'll give you

this is something what histogram looks

like. Yes or no? So for this data, if I

plotted this histogram and I smoothen

out this histogram like this, I smooen

out. Right? Now central tendencies will

help you to find where is the center of

the distribution. At what value the

center of the distribution lies.

So there are three types of central

tendencies mean median and mode. So the

distribution means the center is lying

at mean or the center is lying at median

or the center is lying at the mode

value. That's what your central

tendencies tells you. We have some

distribution of data like this, right?

So where is the center point of that distribution?

distribution?

Where is the center point of that

distribution? Does it center point is at

mean? That means the center of the data

is the mean of the data or the center of

the data is actually the median of the

data or the center of data is at the

mode of the data. That's what central

tendencies tells you.

So it is used to indicate

where does

middle or center of the distribution lies.

lies.

Okay. So central tendencies have three

points like three categories.

So I already told about that is your

mean, median and mode.

Mhm. Okay. So when we talk about mean,

so let us talk about these terms one by

one. What is the mean of the data? So it

is the average of the data. So what is

the formula of mean? Sum of observations

divided by total number of observations. Right?

Right?

Right. So suppose if I give you a data

of 7 10 15 27 38

42 96 tell me what is the mean of this

data. So basically summing up all the

values like this 7 + 10 + 96

divided by total number is 1 2 3 4 5 6

7. So that is approx 33.57.

So this was mean everyone is clear

right? Let's talk about median then what

is median? Median means the middle value

right? So suppose if I give you data

like this 20 10 40 5 4 2 3 50 tell me

what is the median of this data okay one

second I'll give you one more that is 49

one more in median middle value after

sorting the data so sorting can be even

in ascending or descending order so if I

sort this data so First I'll get 2 3 4

5. After five what we have? 10 20 30

49 50. And now you tell what is the

median? 10. Now 10 is the middle value

not 4. Okay. And suppose the same data I

am having without 49.

Without 49.

So if it is a odd number then it's fine.

We can easily find the median. If it is

even here it is even numbers. So that

time we need to take the two most center

elements and take the average of them. 5

+ 10 by 2 that is 15 by 2. So median

will be here 7.5. Here everyone this was

the basic concept which you should remember.

remember.

Now let us talk about mode. What is the

meaning of mode? most repeatable. So

suppose if I take the example we have 2

2 3 3 4 4 tell me what is the mode of

this data? Mode is three. Right? But if

my data is this one 2 3 3 4 4 2. Now

what is the mode? Now you can see we

have two elements. 1 is two which is

repeated three times and then three

which is also repeated three times mean

maximum repetition is three here. So

this time it is two modes we are having

that is two and three. So if we want to

use any one element either we can use

two here either we can use three here.

Clear? So don't get confused that if we

have multiple modes what to do in that

case? Do we need to subtract? Do we need

to take the average or something? No

average nothing these two elements will

be your mode. Clear? If no repetator

then no more. So basically if you got a

data like this. So here maximum time

repetition is basically one only. Now

because every element is repeated one

time only. So all will be considered as

the mode values. All the elements will

be considered as mode. So this was the

concept of mean, median and mode.

Okay. So let us jump to the next concept

which is of variation. What is the

meaning of variation

or we can call it as variance also. What

is the meaning of variance? So variance

actually tells you about the spread of

the data guys. Not only the spread

the spread of data from mean value.

Okay. So like suppose you have two types

of data in which the one points are like

this and in other the points are like

this. So can you tell me where is more

variance and where is less variance. So

we are basically calculating from the

mean. So suppose this is the mean of

this point and this is the mean of this

data. So with the help of variance we

are trying to calculate the distance of

each and every point from the value. Now

let's discuss about the formula of

variance standard variance formula. So

no need to memorize the formula. It is

even very very easy. So guys if you want

to calculate the distance between any

two points here like suppose it is x1

and x2. How you calculate the distance

between any two points? How you

calculate the distance between any two

points? Taking the difference x2 - x1

that will give me the distance right.

Similarly, variance we are calculating

the distance of the points from the

mean. Right? So that means that x minus

the mean of the data. Right? Then we

need to square it. Why? Because we don't

want any negative value. Right? And this

is for the one point like X but we have

multiple points in the data. So we will

do the submission and divided by the

capital N. This is how standard variance

formula looks like. So it was very easy,

right? Okay. Then when we talk about

standard deviation, it is nothing just

the under root value of standard variance.

Okay. So, next topic is to find the

range. So, tell me what is the range of

the data? It is the difference or

distance between minimum and maximum

value in the data. So, the formula will

be range is equals to max value of the

data minus minimum. So suppose you have

data points 7 8 2 9 10 11. Tell me what

is the range of this data? What is the

range of the data? So maximum value is

11 here. Minimum value is two here. So

we are getting the nine. So now let us

discuss about one more topic which is

percentiles. So everyone please focus on

this topic because it is something like

sometimes it feels difficult to

understand. So that topic name is

percentile. Okay. Let us discuss about

the topic of percentiles here. Now

suppose you have some data. So suppose

you have a data which stores the ages of

person. So in your data ages are stored

something like this. 1 age 2 4 5 7 10 12

6 5 and suppose 50. So what I can

of data is

less than or is of uh what we say small

children 99% data is of small children

whereas only one data point I'm having

here which is of old age person that's

what we can inference from here right

this is how your percentile will have

here your 99th percentile value

is equals to 50. What does this means?

That 99%age

of data is less than number 50. So this 99%

99%

of data will be less than number 50.

Getting the point? So percentiles helps

you to find a value below which certain

percentage of observations lie. So if

I'm talking about 99 percentile means

99% of the data. If I'm talking about

50% means 50%age of the data. If I'm

talking about 75% means 75% of the data.

Getting the point or all let me explain

you with one more example. So everyone

of you I hope you know what is the

marking system of competitive exams IIIT

J or IIT JAMS CAT GATE any exam guys

competitive exams do you know how the

marking system of that is they are not

percentile based that is percentile

based not percentage based percentile so

let me explain you so suppose you um any

topper in your class score 80 marks out

of 100. Okay, this is the marks of

topper. Now in this topper means means

rest of the complete class has marks

less than 18. That's why we are saying

the person is topper. Right? So that

means 100 percentile value here is

equals to 80 means 100%age

of other data points of other students

have marks which is less than 80.

Getting the point? So percentiles are

the scores that are used to describe a

value below which some percentage of

observations fall. If this is clear the

next topic is very very easy for you

guys which is quartiles. So quartiles I

think I can relate this word to a

quarter English word quarter and what is

quarter means 1 by4. So that means I

need to divide my data into four equal

parts. So the first part the first

division will be made at the value of 25 percentile.

percentile.

The second division will be made at the

value of 50th percentile and the third

division value will be made at 75%.

So for example if you have a data in

which your minimum value is 1 and the

maximum value is 100 and suppose here we

are having 15 45 and 80. Now what does

this means that at 25%

value is 15 means 25% of the data is

less than value 15. Similarly 50%age of

the data is less than value 45.

Similarly 75% of data is less than value 80.

80.

Getting the point? So this 25% is known

as Q1. 50% is known as Q2 and 75% is

known as Q3. That's just the notation

what we give. Right? Similarly, we have

one more term which we use which is IQR.

So, IQR full form is interquartile

range. What is the value? It's nothing

just the difference of two variables

that is Q3 minus Q1. Like we have read

similarly we have IQR. So Q3 means the

value of 25 percentile and Q1 means

value of 25 percentile. What does this

means that in this data what is your

IQR? So at 75 means Q3 we have 80 and at

Q1 we have 15. So 80 minus 15 which is

65. This is your IQR value. Right? Then

we will discuss about one topic which is

outliers. So in machine learning

outliers topic is very important. Let us

first understand the meaning of

outliers. So I call this as odd one

outliers. Like suppose you have a data

points like this 1 2 3 5 7 15 20 100.

Can you tell me any value here which is

odd one out value? 100. Right? because

the other values range is very less but

100 range is very high. So that odd one

out value is your outliers. Similarly,

if I take another set 0.02 01

1 2 3 5 7 15 20. Now can you tell me any

value which is outlier here or odd one

out? So in right hand side we have two

values 101 and in left hand side also we

have two values. So these values are

your outliers. So the extremist values

on the right or the left side is known

as the outright which is out of the

range. So if your data is this now like

don't consider 100. If your data is only

this then you don't have any outlier

because the range is quite okay here.

The range is quite okay. But if the

range of the data like here you can see

the range is 80. Whereas here it is just

five. You can see the difference. Then

only you will say it has outliers. It's

not necessary that every data set will

have outliers. Some have some don't

have. Clear? Now tell me which plot we

use to find the outliers. Which

visualization plot we use to find the

outliers? that do our data have outliers

or not which visualization plot we use

box plot. So how that box plot looks

like is so if you have these black

points like this then we say that the

data is outliers otherwise the data

don't have outliers. Similarly yes

boilent plot also boilent plot looks a

leaf shape. So if that leaf is also

having points that means outliers

otherwise no right

right

what is the meaning of correlation

correlation basically tells you the

relationship of columns relationship

of columns like if I'm increasing one

column what is the effect on the other

column okay so correlation is of three

types. Correlation is of three types.

What is that? Positive correlation,

negative correlation and zero

correlation. Now what does this means

that? So guys, do you remember the

concept of directly proportional and

inversely proportional?

What is directly? That if I'm increasing

one variable, the other variable will

also increase with it. like your uh

supply and demand. If demand of any

variable is increasing, the supply will

also increase. Right? Similarly,

inversely proportional, if one is

increasing, the other will decrease with

it. Like if you will increase the speed

of your vehicle, the time to get to your

destination will decrease. That's what

positive and negative correlation is.

Positive means directly proportional.

negative means inversely proportional

and zero means no effect that if I'm

increasing one variable or if I'm

decreasing one variable there is no

effect on the area now which

visualization tool we use

for correlation

which visualization tool we use heat map

so in ML you are going to use this heat

map a lot so you will automatically

remember this. Yeah. So this was all

about your descriptive statistics from

my end. Okay.

Regarding your probability. So first

tell me what is probability guys? What

idea you have about probability or we

can also talk like what are the things

scenarios here you can say that

probability will help me in data

science. Why probability is important

for data science. Probability basically

helps the data scientists. Like if we do

any kind of prediction or if we do any

kind of statistical analysis,

how probable that our analysis will be

right or how probable it will be wrong.

Like suppose I have forecasted that

tomorrow it will be real. Now if I know

the probability how much probability my

predictions will be correct I can

proceed like if it is raining my

probability comes out to be 90%. Then if

I'm working in some traffic agency

department and there I can help them to

reduce the traffic I can give them a

prior idea that tomorrow uh I need more

staff present I need more traffic police

as it is raining so it could be more

traffic. So it gives you the likelihood

or the probability that how much right

your predictions will be. So probability

helps data scientist and even data

analyst to access the likelihood of

events that in turns enhance efficiency

efficiency

of insights like what insights we are

basically making drawn from statistical models.

models.

Now tell me what intro you know about

probability. What exactly probability is?

is?

What is the meaning of probability

definition? Try to recall you see.

So probability is the way so way of

measuring how likely

something is to happen. Right? And

formula is like one of your student one

of your colleague has already given the

formula. So the formula is probability

is equals to favorable outcomes

favorable outcomes divided by

total number.

So let me also define about the

outcomes. So what do you understand by

favorable outcomes guys?

according to our needs. So the specific

outcomes we are interested in like

suppose if we toss a coin and I want

what is the probability of coming head.

So I want the probability of head right.

So head will be my favorable outcome

means the outcomes which you are

interested in. If you want to find what

is the probability that tomorrow it will

be raining. So yes will be your

favorable outcome. So the specific outcomes

you are interested in

right now. Then what are total outcomes?

What are total outcomes? We have the

formula as favorable outcomes divided by

total number of outcomes. So tell me

what are total outcomes? All the

possibilities. All the possible outcomes.

outcomes.

So let us discuss about some examples

simple examples. So first flipping a coin.

coin.

Flipping a single coin. Right? So what

is the sample space you will get? Either

you'll get head or you will get tail.

Right? So your favorable outcome suppose

we want to find

what is the probability of head right

then we need to find what is the total

outcomes here good two so now what will

be the probability that it's a head put

that into the formula yes 1 by 2 so here

it is basically 1 * head is coming so 1

by 2 which is.5 is the probability that

you will achieve leave a head right

so what is the sample space you're getting

getting

so either I can get both head either I

can get one head one tail first tail

then head or I can get both tails this

is the sample space right so favorable

outcome here suppose you want what is

the probability I will get head and head

to so how how many times it's repeating

what is the total outcome how many total

outcomes you're having four so the

probability will be 1 by 4 so these are

the very basic examples I hope you have

studied in your six seven standards as

well in your math session remember so

before moving ahead with further

advanced topics important terminologies

we use so I'll mention this topic as

important terminology ology.

So the first what we are going to

discuss is what do you understand by experiment?

experiment?

What is an experiment means? So an

experiment is an event where uncertain

outcomes are possible. This is how you

can define this terminology

of experiment. Then what is event guys?

What is the meaning of event? uh the

possible outcomes like if I'm having a

event I am doing this I am flipping a

coin so my task is to flip the coin

right that's what event is or we can

also say that I'll send you another so

the definition directly from the

Wikipedia is like the event means the

set of outcomes of an experiment for

example if If I'm flipping a coin, so my

sample space will be head or tail,

right? What is the event that I'm

getting a head or event of getting a

tail? Okay, so you can even consider it

as the subset of your sample space. So

an event is a set of outcomes of an experiment.

So let us discuss about sample space.

Now tell me what is sample space? All

the possible sample space means guys all

the possible outcomes. Like if I'm

tossing a coin, what are all the

possible outcomes? I can get a head, I

can get a tail, right? So when I'm

flipping only one coin, my sample space

is only two, which is I can get head or

I can get tail. Whereas if I'm flipping

two coins, this will be my sample space.

I can get both head. I can get first

head then tail otherwise first tail then

head and then both tail I can get

similarly if we have a deck of cards I

hope everybody knows about those deck of

cards game right so 52 cards exactly so

if I'm taking any card from that deck of

uh cards so what will be the sample

space so all the 52 cards which I'm

having in that deck will be in my sample

space So sample space basically means

sample space total outcome basically

means that we are counting that value

like total outcomes here we are having

two here we are having four if we are

counting that total number of outcomes

that is total outcomes if I'm just

writing that this is the sample space

here if I'm mentioning those outcomes

like what are all the possible outcome

that is your sample space. Okay, let me

talk about next more three they are also

important. What do you understand by

equally likely events guys? Equally likely

likely events.

events.

What is the meaning of equally likely

events? Equally likely guys from the

name itself try to understand like equal

what look likely means probability

events means the event what we having

don't you think so events which have the

equal probability from the name itself

we can define the definition right

events that have equal probability

for example your coin

The probability of head and the

probability of tail is equal that is 1

by 2 or when you have a dice. So the

probability of any number here is equals

to 1 by 6 means every outcome is having

equal number of probability that events

are known as equally likely events. So

Events that have the same chances or

probability of occurring. But here is

one note point guys. What is that? No

the events or the outcomes should be

independent of each other. They should

not depend upon each other. So if I am

toss uh tossing a coin my head and tails

are not dependent right like if first

time I'll get head second time it's

necessary I will get tail or something

no they are independent of each other so

that is very much necessary the outcome

of one event should be independent of

other. I hope it is clear.

The next one mutually exclusive events.

Mutually exclusive events. Try to think.

So mutually exclusive exclusive means

don't include anything. Right? Now

events that cannot happen

simultaneously. Right? That's are

mutually exclusive. Like again flipping

a coin. I cannot get head and tail both

together. Right? We cannot get any

either from dice also. I cannot get two

numbers same time. So events

that cannot

happen simultaneously.

Right? Another example of weather or

climate. Climate can be either hot or

cold. At one time it will be either hot

or it will be either cold. Coin example,

dice example, true false example or any

one zero example as well. Clear? Then

similarly we have one more which is

mutually inclusive.

Now tell me what would be this mutually inclusive

events can occur simultaneously

okay I'll give you one example then you

think of any other so if I'm taking a

card from the deck of card so I want to

find what is the probability

that the card is of um diamond. The card

is of diamond and a number four. So here

we are having two events together. One

is card is red and number is or we can

say card is red of number four. holiday

and Sunday like what is the probability

that it will be a Sunday and also a

national holiday. So, so these are the

few examples of mutually exclusive and

mutually inclusive events. Before moving

ahead, let's discuss about types of probability.

So, probability can be broadly classified

classified

into three types.

The first type is known as marginal.

The second type is known as joint and

the third type is known as conditional.

So probability is broadly classified

into these three types. So have you

heard about these types before? So let

us discuss with a very very beautiful

example. That example is actually I've

taken so that you can understand about

these three individually and also the

differences between these three. So

using one example only we will try to

distinguish between these three types.

So let me build the example first.

So a survey was conducted

with 500 strangers. Okay. There was a

survey regarding games that which game

is played by the person and the results

So the games like they asked that male,

female, total. So how many of them plays

football or rugby

or any other game? Okay, a survey was

conducted. So total we are having 500

persons. So from all the 500 persons

there were 270

ms and 230 females and the total who

plays rugby was 195 sorry football was

195 and for rugby was 125 and other was

180. Then from all the persons who plays

football and also who is male means how

many males were playing football that is

120. How many females were playing

football? That was 75. Then how many

males were playing rugby? That was 100.

And how many females were playing rugby?

That was 25. And how many males play

other sports? That was 50. And how many

females play? That is 130. Let me know

if this uh picture is clear like how it

came. So this was the number but if I

want to get the probability so I'll

mention here to get the proper probabilities

probabilities divide

divide

each by total number of persons that is

male

male total football

football

rugby others.

So kindly divide each and every value by

500 and tell me what will be the output.

So I'm giving you time of 2 minutes. You

make a note at your end where you will

divide each and every element by 500,

120 by 500, 75 by 500, 195 by 500. So

after 2 minutes I will start what

answers you get. Clear? Keep a note at

your end and divide each and everything

with 500. What is 120 by 500? 0.25 or 0.24.

0.24.

It's 0.24 24 not 25 0.24

0.24

75 divided by 500 good 0.15

Similarly 100 by 500.25

by 500 will be 0.05

125 by 500 will be 0.25 25 then 0.1 0.26

0.26 0.36

0.36

then 0.54

0.46 4 6 and right so we have find the

probabilities now let us discuss that

what is marginal joint and conditional

probability so first I will talk about

the joint probability joint probability

that was just an example through this

example we will understand about the

probabilities okay right now we were

just discussing about one example I have

built the plot now now I will use this

example to explain you about the

probabilities. Now joined guys from the

name itself it is clear that we are

joining two events that means don't you

think so it is kind of mutually

inclusive events means the probability

of the events occurring together. So in

our case don't you think the probab if

we want to find the probability that the

person is a male and play rugby

that is this probability is 0.2.

Similarly person is a female and plays

rugby that is 0.05. So these two two

probabilities are your joint

probability. What does this mean? That

it is used to calculate

the probability.

It is used to calculate the probability

of two events occurring together at the

same time. So it is denoted by P A and B

or probability of A comma D. It can be

more than two also. Mohammed depends

upon the example or depends upon the condition.

condition.

So in our example,

so in our example, joint probability

of someone being a male and liking

football, right? Or all the values which

are here 1 2 3 4. Don't you think so?

These probabilities are joint

probabilities. All the six highlighted

probabilities are the example of joint

probability. Joint probability means the

probability of two events occurring

together. So in this all probabilities

we have two events. What one is the

gender event other is the game event. So

here means that I am finding the

probability the person is a male and

playing football.

Right? Similarly here the example is

person is a female and playing football.

So two events we are having one is it's

a football and the gender is female.

Similarly here we can say that the

person is a male and play rugby whereas

here the person is a female and plays

rugby. Clear? But there is one note

point guys we have one note point. What

is that note is joint probability is symmetrical.

symmetrical.

Now what do you mean by symmetrical

here? What do you mean by symmetrical?

Means if I'm finding the probability of

A and B it should equals to probability

of B and A like if I'm finding what is

the probability the person is a male and

play rugby should be equals to what is

the probability the person first plays

rugby and then it is a male. So both of

the cases it should be equal. So that's

what joint probability should be symmetrical.

symmetrical.

What will be marginal? Try to think. Try

to think guys. What will be marginal? So

if joint means two events together,

don't you think? So marginal means a

single event we are talking about. So in

our example we are basically talking

about the person is either male or

female or either the person is playing a

particular game. So

marginal probability it is the

probability or it is the probability of

an event irrespective

of the outcome of another variable. So

it is denoted by probability

of B we are talking about. So like this

was an example. So here like if we see

that uh so from all the 500 people why

want to find the probability that the

person was male we don't care what game

he preferred we only prefer that the

person was a male to this probability similarly 0.4 for said that the person

similarly 0.4 for said that the person is a female or 500 people I want the

is a female or 500 people I want the people who plays football irrespective

people who plays football irrespective of what gender they are 0.39 0.25 25 and

of what gender they are 0.39 0.25 25 and 0.2. So don't you think so? These are

0.2. So don't you think so? These are the examples of marginal probabilities.

the examples of marginal probabilities. In these examples, we are talking about

In these examples, we are talking about the person is either male or female. We

the person is either male or female. We don't care what game they are playing.

don't care what game they are playing. We only care from 500 people. What is

We only care from 500 people. What is the probability the person was a male or

the probability the person was a male or the person was a female? I don't care

the person was a female? I don't care about the game. Similarly, these

about the game. Similarly, these probabilities were games irrespective of

probabilities were games irrespective of the gender. I want the probability of

the gender. I want the probability of people who plays football irrespective

people who plays football irrespective of they are males or females. So

of they are males or females. So individual probabilities is known as

individual probabilities is known as marginal probability. Individual means

marginal probability. Individual means that we are concerning only about one

that we are concerning only about one event. Here we have two events. Now one

event. Here we have two events. Now one is gender, other is game. So either we

is gender, other is game. So either we are focusing only upon the gender the

are focusing only upon the gender the person is male or female irrespective of

person is male or female irrespective of the game or we are only considered about

the game or we are only considered about the game irrespective of gender.

the game irrespective of gender. So either we will consider game or

So either we will consider game or either we will consider gender. Then

either we will consider gender. Then there's another which is conditional

there's another which is conditional probability.

Conditional probability. Now conditional probability means that it defines the

probability means that it defines the probability of one event occurring given

probability of one event occurring given that another event has occurred. That

that another event has occurred. That means one event already occurred. We

means one event already occurred. We want to find the probability of next

want to find the probability of next event which is dependent upon the

event which is dependent upon the previous event. So the formula of

previous event. So the formula of conditional probability is probability

conditional probability is probability of a comma b divided by probability of

of a comma b divided by probability of b. So here we are finding probability of

b. So here we are finding probability of a given b. Now if we talk about

a given b. Now if we talk about examples. Now suppose in our example

examples. Now suppose in our example only it is given that the person we

only it is given that the person we selected

selected is a female. So for example probability

is a female. So for example probability of preferring a sport

of preferring a sport given the observant

given the observant is female. Now one event already given

is female. Now one event already given that the person is female and we want

that the person is female and we want find the probability of preferring a

find the probability of preferring a sport suppose rugby right so probability

sport suppose rugby right so probability of we need to find the rugby given that

of we need to find the rugby given that it's a female so that will be

it's a female so that will be probability of like according to the

probability of like according to the formula a comma b means probability of

formula a comma b means probability of female plays rugby rugby divided by

female plays rugby rugby divided by probability that it is a female. So

probability that it is a female. So probability of female plays rugby. What

probability of female plays rugby. What is the probability? Tell me. It's a

is the probability? Tell me. It's a female and plays rugby. What is the

female and plays rugby. What is the probability guys? 0.05

probability guys? 0.05 divided by it's a probability that it's

divided by it's a probability that it's a female. What is the probability that

a female. What is the probability that it's a female? 0.46.

it's a female? 0.46. So this is how we calculate our

So this is how we calculate our conditional probability. Let's talk

conditional probability. Let's talk about one more that is base theorem.

Base theorem is actually nothing guys. It's the extension of conditional

It's the extension of conditional probability.

probability. Mathematical extension of conditional

Mathematical extension of conditional probability.

probability. So the formula of base theorem is

So the formula of base theorem is probability of b by a into probability

probability of b by a into probability of a divided by the same statement again

of a divided by the same statement again probability of b by a comma probability

probability of b by a comma probability of a plus probability of b when a did

of a plus probability of b when a did not occur multiply by probability of e.

not occur multiply by probability of e. It is nothing just the extension of your

It is nothing just the extension of your conditional probability the mathematical

conditional probability the mathematical extension. So derivation and all that

extension. So derivation and all that stuff will not be important guys for the

stuff will not be important guys for the base theorem. Okay. So this was all

base theorem. Okay. So this was all about your probability.

about your probability. So if no doubts let's discuss about the

So if no doubts let's discuss about the probability distribution.

probability distribution. Okay probability distribution guys it's

Okay probability distribution guys it's very very easy to understand. Suppose

very very easy to understand. Suppose you throw 100

you throw 100 dice. We throw a dice 100 times. Right

dice. We throw a dice 100 times. Right now I will record that how many times

now I will record that how many times I'll get one. How many times I'll get

I'll get one. How many times I'll get two? How many times I'll get other

two? How many times I'll get other outcomes. Okay, I'll record that 1 2 3 4

outcomes. Okay, I'll record that 1 2 3 4 5 6. Now suppose you got one five times

5 6. Now suppose you got one five times you got two

you got two suppose 10 times you got three suppose

suppose 10 times you got three suppose more number of times 25 times or you got

more number of times 25 times or you got four then you got five then you got six

four then you got five then you got six you have just count the frequency

you have just count the frequency right Now

right Now when I am smoothening out this graph

when I am smoothening out this graph like this, when I'm smoothing out this

like this, when I'm smoothing out this histogram like this, this smoothening

histogram like this, this smoothening part is your probability distribution.

part is your probability distribution. Clear? The smoothening part of this

Clear? The smoothening part of this is known as probability distribution.

So it is basically finding the probability

probability of different possible values of a

of different possible values of a variable. So here my possible values was

variable. So here my possible values was 1 2 3 4 5 6. So it is telling me what is

1 2 3 4 5 6. So it is telling me what is the probability. So here I can see that

the probability. So here I can see that the probability of coming three is the

the probability of coming three is the highest and probability of coming one

highest and probability of coming one and six is the lowest. So that's what

and six is the lowest. So that's what probability distribution helps you clear

probability distribution helps you clear the normal meaning of probability

the normal meaning of probability distribution. Yes or no? Now there are

distribution. Yes or no? Now there are two types of probability distribution

two types of probability distribution guys means two types of data we are

guys means two types of data we are having. So because we have two types of

having. So because we have two types of data we have two types of probability

data we have two types of probability distribution. So I hope everybody knows

distribution. So I hope everybody knows what is that two type of data discrete

what is that two type of data discrete and continuous. You know what is the

and continuous. You know what is the meaning of discrete and what is the

meaning of discrete and what is the meaning of continuous?

meaning of continuous? Yes. So ma means that for discrete data

Yes. So ma means that for discrete data we will be using discrete probability

we will be using discrete probability distribution

distribution and for continuous data we will be using

and for continuous data we will be using continuous probability distribution.

continuous probability distribution. Right? So I will write here used for

Right? So I will write here used for discrete value used for continuous

discrete value used for continuous values. Right? So can anybody give me

values. Right? So can anybody give me the examples of discrete data? Example

the examples of discrete data? Example of discrete data. So number of students

of discrete data. So number of students not marks number of students in a class

not marks number of students in a class because students can be either 30 or 31.

because students can be either 30 or 31. We cannot say we have 30.5 students.

We cannot say we have 30.5 students. Right? Number of students or we can say

Right? Number of students or we can say coin example head or tail or we can say

coin example head or tail or we can say dice example we can get 1 2 3 4 5 6 we

dice example we can get 1 2 3 4 5 6 we cannot get 1.1 2.1 like that right then

cannot get 1.1 2.1 like that right then what will be the example of continuous

what will be the example of continuous what will be the example of continuous

what will be the example of continuous values heartbeat good very nice weight

values heartbeat good very nice weight or height of a student loan amount

or height of a student loan amount temperature very nice

temperature very nice load these are the examples of

load these are the examples of continuous value now guys I'll tell you

continuous value now guys I'll tell you one important thing in reality now we

one important thing in reality now we will get this histogram you can see we

will get this histogram you can see we will get this histogram to achieve this

will get this histogram to achieve this curve we use a function of probability

curve we use a function of probability distribution

distribution we use a function okay so when We have a

we use a function okay so when We have a discrete data. The function name is PMF

discrete data. The function name is PMF which is probability

which is probability mass function. Using this function, we

mass function. Using this function, we will plot the discrete probability

will plot the discrete probability distributions.

distributions. Right? Similarly, when we have the

Right? Similarly, when we have the continuous data, the function we will

continuous data, the function we will use is PDF. The full form is probability

use is PDF. The full form is probability density

density function. So using these functions we

function. So using these functions we will plot the probability distribution

will plot the probability distribution curve. Now we will study about different

curve. Now we will study about different types of probability distribution guys

types of probability distribution guys we are having. So in discrete we have

we are having. So in discrete we have binomial probability distribution

binomial probability distribution but knowledge

but knowledge and poisons

and poisons whereas in continuous we have normal

whereas in continuous we have normal distribution.

distribution. So we will discuss about these

So we will discuss about these distributions one by one. So this was

distributions one by one. So this was the flow chart of probability

the flow chart of probability distributions.

distributions. First let us discuss about Bernnoli.

First let us discuss about Bernnoli. Bernnoli's distribution. It's very very

Bernnoli's distribution. It's very very easy. But knowledge distribution means

easy. But knowledge distribution means when distribution whenever our outcome

when distribution whenever our outcome is binary guys outcome is outcome of an

is binary guys outcome is outcome of an event is binary like real time where we

event is binary like real time where we can use it like suppose I want to find

can use it like suppose I want to find the probability that tomorrow it will be

the probability that tomorrow it will be raining or not one or zero head or tail

raining or not one or zero head or tail whenever we are having only two outcomes

whenever we are having only two outcomes we will be using the Bernoli is

we will be using the Bernoli is distribution what is the probability

distribution what is the probability that you will pass in tomorrow's exam

that you will pass in tomorrow's exam right but only one thing you need to

right but only one thing you need to note here what you need to note here is

note here what you need to note here is number of trials suppose if I am tossing

number of trials suppose if I am tossing a coin and I want to find the

a coin and I want to find the probability of head or tail so two

probability of head or tail so two events I'm having means two outcomes I'm

events I'm having means two outcomes I'm having that means I'll use boli's there

having that means I'll use boli's there but number of trials will be one means

but number of trials will be one means only one time I can toss was the coin

only one time I can toss was the coin only one time I can find it. So for this

only one time I can find it. So for this the function is PMF. Remember I told you

the function is PMF. Remember I told you about that we will use the function as

about that we will use the function as PMF. So let me give you the formula of

PMF. So let me give you the formula of PMF but you don't need to remember it.

PMF but you don't need to remember it. So don't worry it is just a mathematical

So don't worry it is just a mathematical formula. You don't need to remember this

formula. You don't need to remember this formulas. So it will be either the

formulas. So it will be either the probability will be yes

probability will be yes or the probability will be no. Here x is

or the probability will be no. Here x is equals to number of

equals to number of trials. So the formula exactly is don't

trials. So the formula exactly is don't go with the formulas

go with the formulas probability mass function.

probability mass function. Okay.

This is the PMF function. Here when we comes to real cases,

Here when we comes to real cases, do we need to find anything only one

do we need to find anything only one time? Means number of trials should be

time? Means number of trials should be one. Yes or no? No. Right? That's why we

one. Yes or no? No. Right? That's why we have the extension of this Bernoli.

have the extension of this Bernoli. We have the extension that is binomial

We have the extension that is binomial distribution.

Extension of Bernoli's

of Bernoli's P means probability. Here also we are

P means probability. Here also we are having two independent outcomes like

having two independent outcomes like where head and tail, true false, yes no

where head and tail, true false, yes no kind of situations but only the

kind of situations but only the difference is here number of trials will

difference is here number of trials will be infinite or multiple. So the formula

be infinite or multiple. So the formula here is again I'm saying no you do not

here is again I'm saying no you do not need to learn the formula only it is for

need to learn the formula only it is for the developers who actually creates this

the developers who actually creates this formulas but a basic understanding that

formulas but a basic understanding that is required n is equals to number of

is required n is equals to number of trial

trial x is equals to number of successes

x is equals to number of successes and q is equals to probability of 1

and q is equals to probability of 1 minus

minus So difference between bernoli and

So difference between bernoli and binomial is only one that is number of

binomial is only one that is number of trials. Clear?

trials. Clear? Bernoli means one trial. Binomial means

Bernoli means one trial. Binomial means multiple trials. That's it. Do you do

multiple trials. That's it. Do you do not read to uh remember the formulas.

not read to uh remember the formulas. Clear? Then let's discuss about the

Clear? Then let's discuss about the poisons distribution. Now think of a

poisons distribution. Now think of a situation guys that you are manager of

situation guys that you are manager of the server team in your company. Okay.

the server team in your company. Okay. Now it's a company of a call center

Now it's a company of a call center where you get several calls. Right? Now

where you get several calls. Right? Now everyone knows that if we will get calls

everyone knows that if we will get calls there will be a burden on the server if

there will be a burden on the server if the calls are very huge. Right? Now if

the calls are very huge. Right? Now if you are a manager of that you should be

you are a manager of that you should be aware about right what is the times when

aware about right what is the times when I get the highest falls in my customer

I get the highest falls in my customer care so that I can increase the load or

care so that I can increase the load or I can uh increase the capacity of the

I can uh increase the capacity of the servers right whereas what is the time

servers right whereas what is the time zone when I get very less calls so that

zone when I get very less calls so that I can leave the server as it is because

I can leave the server as it is because the thing is suppose the load of the

the thing is suppose the load of the server is less F and you got multiple

server is less F and you got multiple calls at that time your server will be

calls at that time your server will be overload and maybe it will crash also

overload and maybe it will crash also that time we use this poisons

that time we use this poisons distribution when we need to find the

distribution when we need to find the probability

probability of a certain duration

of a certain duration let's take another example suppose this

let's take another example suppose this Saturday Sunday you sold this many books

Saturday Sunday you sold this many books you want to find the probability that

you want to find the probability that what will be the sales in next

what will be the sales in next Saturday's and Sunday or you can say the

Saturday's and Sunday or you can say the arrival of emails. Getting it? So

arrival of emails. Getting it? So wherever you want to find the

wherever you want to find the probability between a specific time

probability between a specific time duration, you will use poison

duration, you will use poison distribution. There a discrete

distribution. There a discrete distribution

distribution estimates the likelihood of an event in

estimates the likelihood of an event in a given

a given time duration.

time duration. So here formula is lambda x / x

So here formula is lambda x / x factorial e minus lambda. So x means

factorial e minus lambda. So x means number of successes.

number of successes. Lambda means mean number of successes.

Lambda means mean number of successes. E means a constant value.

E means a constant value. So these were the three discrete

So these were the three discrete probability distribution.

So the last distribution is normal distribution which is very very

distribution which is very very important like very very important

important like very very important comparative to your other distributions

comparative to your other distributions also and also very very very very easy.

also and also very very very very easy. something when I will plot the

something when I will plot the distribution curve it could come to any

distribution curve it could come to any shape right the shape could be like this

shape right the shape could be like this or the shape could be like this or

or the shape could be like this or anything else right but when the shape

anything else right but when the shape of your distribution is like this

of your distribution is like this shape is known as bell curve so if the

shape is known as bell curve so if the shape of your distribution is like a

shape of your distribution is like a bell curve it's looking like a bell here

bell curve it's looking like a bell here right it's looking like a bell so the

right it's looking like a bell so the shape of the distribution is like a bell

shape of the distribution is like a bell we call it as normal distribution.

we call it as normal distribution. So when distribution

So when distribution follows

follows a bell curve

a bell curve we call it as normal distribution only

we call it as normal distribution only the shape guys

the shape guys know why this shape is so important

know why this shape is so important right why if I have the specific shape

right why if I have the specific shape we have a type of distribution there is

we have a type of distribution there is some special things about this what

some special things about this what the first property it has is this shape

the first property it has is this shape the mean, median and mode values will be

the mean, median and mode values will be equals to each other.

equals to each other. Means whenever we have this shape my 3 m

Means whenever we have this shape my 3 m of statistic values will be same. The

of statistic values will be same. The second property is it is symmetrical. It

second property is it is symmetrical. It is symmetrical. Symmetrical means mirror

is symmetrical. Symmetrical means mirror image. Left hand side will be equals to

image. Left hand side will be equals to right hand side. And the third property

right hand side. And the third property is a special case special case of this

is a special case special case of this normal distribution which is known as

normal distribution which is known as standard normal distribution.

standard normal distribution. Now what is the special case? So I know

Now what is the special case? So I know that in normal distribution

that in normal distribution my values of mean, median, mode will be

my values of mean, median, mode will be equal. But that could be equals to

equal. But that could be equals to anything. That could be equals to 6,

anything. That could be equals to 6, that could be equals to 10, that could

that could be equals to 10, that could be equals to 100, anything. But in

be equals to 100, anything. But in standard normal distribution that value

standard normal distribution that value will be always equals to zero

will be always equals to zero means at origin. What does this mean?

means at origin. What does this mean? That if I'm talking about the normal

That if I'm talking about the normal distribution that could be anywhere in

distribution that could be anywhere in the graph in but if I'm talking about

the graph in but if I'm talking about standard normal distribution that will

standard normal distribution that will only lie at the origin value and one

only lie at the origin value and one more standard deviation value will be

more standard deviation value will be equals to 1. That's what about your

equals to 1. That's what about your normal distribution is the function here

normal distribution is the function here is f_sub_x is equ= to 1 by sigma

is f_sub_x is equ= to 1 by sigma under<unk> 2 pi

under<unk> 2 pi exponential of - 1 by 2 x - mu by sigma

exponential of - 1 by 2 x - mu by sigma this is the function

this is the function where sigma is standard deviation

where sigma is standard deviation mu means mean.

mu means mean. Now there's one last last concept which

Now there's one last last concept which is similar to this you know normal

is similar to this you know normal distribution which is central limit

distribution which is central limit theorem. Anybody knows what is central

theorem. Anybody knows what is central limit theorem? It is related to this

limit theorem? It is related to this normal distribution only. Anybody knows

normal distribution only. Anybody knows what is it? It says that it's a like

what is it? It says that it's a like proven experimental technique guys. It's

proven experimental technique guys. It's not like we can see examples or we can

not like we can see examples or we can see something like that. No, it is just

see something like that. No, it is just an experimental value or experimental

an experimental value or experimental theorem which scientists discover. What

theorem which scientists discover. What they discovered is like whatever the

they discovered is like whatever the shape of the population is, the

shape of the population is, the population could be of any shape that

population could be of any shape that could be

could be this shape that could be this shape that

this shape that could be this shape that could be any shape of the distribution.

could be any shape of the distribution. Central limit says that if I am taking a

Central limit says that if I am taking a sample from this population

sample from this population and that size is greater than 30, it

and that size is greater than 30, it will that sample will most probably

will that sample will most probably follows a normal distribution.

follows a normal distribution. Means here sample population or sample

Means here sample population or sample distribution will not depend upon

distribution will not depend upon population.

population. Whatever the shape of the population is,

Whatever the shape of the population is, if I'm taking a sample greater than 30,

if I'm taking a sample greater than 30, it tries to or it tends to be a normal

it tries to or it tends to be a normal distribution. So let me define it

distribution. So let me define it properly so that you can even answer it.

properly so that you can even answer it. It's an important interview question as

It's an important interview question as well. So you can easily answer it. It

well. So you can easily answer it. It states that

states that if we take several

if we take several random samples

random samples from a population

from a population that has a finite variant, the mean of

that has a finite variant, the mean of the samples

the samples tends

tends to follow.

to follow. Clear? So here your sample size

Clear? So here your sample size should be greater than 30. That's what

should be greater than 30. That's what your central limit theorem is.

your central limit theorem is. So I'm importing the necessary libraries

So I'm importing the necessary libraries first. Now keep the name of the data as

first. Now keep the name of the data as uh data. Okay. So let it equore

CSV in which I am loading it country profile

profile variables dot CSV

variables dot CSV and TF dot head.

What is the shape of the guys? Uh complete shape of the file. So the

complete shape of the file. So the command which we can use is df dot shape

command which we can use is df dot shape here. So this will give me the shape of

here. So this will give me the shape of the data which is 229 and 50. So 229

the data which is 229 and 50. So 229 rows you're having and 50 columns you're

rows you're having and 50 columns you're having right?

having right? So you can see here like how many

So you can see here like how many columns 50 columns we are having right?

columns 50 columns we are having right? So df dot columns

So df dot columns it we are having 50 columns here. So

it we are having 50 columns here. So this data set I think everyone can

this data set I think everyone can understood that this is regarding the

understood that this is regarding the variables of the country. What is the

variables of the country. What is the name of the country? What is the region

name of the country? What is the region in which the country is surface area of

in which the country is surface area of the country? Population in thousand

the country? Population in thousand population density sex ratio and so on.

population density sex ratio and so on. We have these parameters. So if you from

We have these parameters. So if you from the geographic background you are well

the geographic background you are well aware about these parameters yes or no

aware about these parameters yes or no otherwise you can just understand it as

otherwise you can just understand it as the variables related to your country.

the variables related to your country. So from this data we need to just apply

So from this data we need to just apply the descriptive statistics concepts. So

the descriptive statistics concepts. So before that few commands which are very

before that few commands which are very very important for you. So the first

very important for you. So the first command is how to check the null values

command is how to check the null values in your data.

in your data. How to check the

How to check the null values in the data.

null values in the data. So write the name of the data frame dot

So write the name of the data frame dot is null command. Okay, this command will

is null command. Okay, this command will give you the null values. But the thing

give you the null values. But the thing is there is one problem with this

is there is one problem with this command. What is that? It will give me

command. What is that? It will give me the output in the boolean format. So you

the output in the boolean format. So you can see that wherever you will be having

can see that wherever you will be having a null value it will give you true

a null value it will give you true otherwise it will give you false right

otherwise it will give you false right but the thing is it is difficult for you

but the thing is it is difficult for you to manually see each and every value

to manually see each and every value wherever I'm having true right so what

wherever I'm having true right so what we use is dot sum here what this dot sum

we use is dot sum here what this dot sum will do is basically count the number of

will do is basically count the number of null values so right now you can see

null values so right now you can see that every in every column it is zero

that every in every column it is zero that means that you don't have null

that means that you don't have null values in your data. So again if I use

values in your data. So again if I use total sum it will give me the total

total sum it will give me the total number of null values in the data which

number of null values in the data which is zero in our case because we don't

is zero in our case because we don't have null case we have null values it

have null case we have null values it will give me the count of that null

will give me the count of that null value. Okay. Now tell me what if we have

value. Okay. Now tell me what if we have null values. Rajes first sum will give

null values. Rajes first sum will give me the null values in every column. You

me the null values in every column. You can see here and second value is

can see here and second value is actually counting these all numbers

actually counting these all numbers present here. The total count it is

present here. The total count it is given. Okay. Now the thing is guys what

given. Okay. Now the thing is guys what if we have right now we don't have but

if we have right now we don't have but what if we have null values? What to do

what if we have null values? What to do in that case? What if we have null

in that case? What if we have null values? So there are two methods we can

values? So there are two methods we can do it. One is drop the null values. So

do it. One is drop the null values. So if the values are very less, we can drop

if the values are very less, we can drop it. Otherwise we can fill the null

it. Otherwise we can fill the null values. Now we will fill the null values

values. Now we will fill the null values with what? We use the fill we use the

with what? We use the fill we use the fill now command to fill those values.

fill now command to fill those values. But with what we fill it with what we

But with what we fill it with what we fill the null values. Single sum is

fill the null values. Single sum is giving you the number of null values in

giving you the number of null values in each column separately and double sum is

each column separately and double sum is basically giving you the total count in

basically giving you the total count in the data frame. So it depends upon you.

the data frame. So it depends upon you. If you want to see in every column what

If you want to see in every column what are the null values, you can use single

are the null values, you can use single sum and if you want to use the total

sum and if you want to use the total null values in my data frame, I'll use

null values in my data frame, I'll use the doubles. So I'll give you a separate

the doubles. So I'll give you a separate command for it. Then if you're getting

command for it. Then if you're getting confused here so single sum is giving

confused here so single sum is giving you the null values in each and every

you the null values in each and every column and the double sum will give you

column and the double sum will give you the count. So I'll separated the command

the count. So I'll separated the command so that you can easily relate. So we

so that you can easily relate. So we will fill the null values guys most

will fill the null values guys most probably with mean or median or if it's

probably with mean or median or if it's a object data we will fill it with and

a object data we will fill it with and sometimes we fill it with zero also. Now

sometimes we fill it with zero also. Now how to check for duplicates?

how to check for duplicates? Do we have duplicate data or not? How we

Do we have duplicate data or not? How we check that? What is the command? It's df

check that? What is the command? It's df dot duplicated guys. It's not duplicate.

dot duplicated guys. It's not duplicate. It is duplicated. Not duplicates also.

It is duplicated. Not duplicates also. See if I'm writing here duplicates it

See if I'm writing here duplicates it will show me an error or it if I write

will show me an error or it if I write only duplicate again it will show me

only duplicate again it will show me error. So the command is duplicator. But

error. So the command is duplicator. But again the same problem we are having it

again the same problem we are having it will give me in boolean format. So we

will give me in boolean format. So we what we will use dot sum. So it will

what we will use dot sum. So it will count the duplicate values. So right now

count the duplicate values. So right now we have zero duplicate values. Now let

we have zero duplicate values. Now let us start our descriptive stats. So for

us start our descriptive stats. So for this randomly you can pick any column

this randomly you can pick any column guys. I am taking this population in

guys. I am taking this population in thousands column. Okay. So we will

thousands column. Okay. So we will initialize we will do our descriptive

initialize we will do our descriptive stats based on this column here. You can

stats based on this column here. You can pick any column as you want. Okay. But

pick any column as you want. Okay. But for this session I also first do with

for this session I also first do with what I am doing and for the practice you

what I am doing and for the practice you can take any other column. So for our

can take any other column. So for our analysis we have chosen this population

analysis we have chosen this population in thousands column. So first we will

in thousands column. So first we will find the mean. So first and what is the

find the mean. So first and what is the basic formula of mean guys? Like first

basic formula of mean guys? Like first let let us do it manually and then we

let let us do it manually and then we will use Python function. So what is the

will use Python function. So what is the formula of mean sum of observations

formula of mean sum of observations divided by total number of observations

divided by total number of observations right now let us do how we can if we

right now let us do how we can if we don't know what is the function and all

don't know what is the function and all how we can do it manually. So

how we can do it manually. So sum of observations means I need to take

sum of observations means I need to take the sum of this column population in

the sum of this column population in thousands. So I'll copy the name from

thousands. So I'll copy the name from here. So it's better to copy the name

here. So it's better to copy the name instead of writing it because in writing

instead of writing it because in writing we can miss some spaces and all which

we can miss some spaces and all which will cause me errors. So how to get the

will cause me errors. So how to get the sum guys? I think you should know the

sum guys? I think you should know the command which function will help me

command which function will help me first to get the sum. I think dot sum.

first to get the sum. I think dot sum. Yes or no? Dot sum. Now I can print sum

Yes or no? Dot sum. Now I can print sum of observations

of observations is sum. So sum of observations we got is

is sum. So sum of observations we got is this. Now we need to find the total. So

this. Now we need to find the total. So I'm choosing a variable again I'm

I'm choosing a variable again I'm choosing this column. How to find what

choosing this column. How to find what is the total number of observations

is the total number of observations here. So either you can use guys dot

here. So either you can use guys dot count function. Dot count function. It

count function. Dot count function. It will count the number of observations or

will count the number of observations or you can even use the length function.

you can even use the length function. Length will also give you the same

Length will also give you the same answer. Both you can do right. So we got

answer. Both you can do right. So we got sum, we got total. Now we can easily

sum, we got total. Now we can easily find mean is equals to sum divided by

find mean is equals to sum divided by total. Let's see what is the answer. So

total. Let's see what is the answer. So it's 32756.

So now similarly guys manually try to do median

median and mode also. How manually we can find

and mode also. How manually we can find median and how manually we can find

median and how manually we can find mode. So first we will find the length

mode. So first we will find the length of

of your column.

your column. Okay. So basically at first we need to

Okay. So basically at first we need to find is it a even length or an odd

find is it a even length or an odd length. So we can see here it is odd

length. So we can see here it is odd length. So basically if I divide here by

length. So basically if I divide here by two it will give me the index value of

two it will give me the index value of the what we say the middle element right

the what we say the middle element right now I can easily from you I can find the

now I can easily from you I can find the element at this index. So I'll take the

element at this index. So I'll take the column name but first we need to sort

column name but first we need to sort the values also. sort values

the values also. sort values because after sorting only we need to do

because after sorting only we need to do right and here I can find the index of

right and here I can find the index of 114

114 that is 223 but there is one more thing

that is 223 but there is one more thing guys there is one more thing you need to

guys there is one more thing you need to understand is what when we sorted the

understand is what when we sorted the values so if I sort here when we sorted

values so if I sort here when we sorted the values the index values are also

the values the index values are also changed

changed based on what sorting is done. So that

based on what sorting is done. So that means you also need to do reset index or

means you also need to do reset index or we can say ignore index. Ignore index is

we can say ignore index. Ignore index is equals to true. What this will do is it

equals to true. What this will do is it will basically give you the index values

will basically give you the index values from 012 instead of the shuffling one.

from 012 instead of the shuffling one. Clear everyone? And now I can find what

Clear everyone? And now I can find what is at 1 14 index. 548

is at 1 14 index. 548 is your median. Clear everyone? How we

is your median. Clear everyone? How we found the median?

found the median? Then how to find mode? Mode is I think

Then how to find mode? Mode is I think very very simple. Just value counts,

very very simple. Just value counts, right? Dot value comes

right? Dot value comes it will give me. So the highest time

it will give me. So the highest time repetitive is 2890

repetitive is 2890 51. So I can see here multiple modes I

51. So I can see here multiple modes I am having. So remember we discussed the

am having. So remember we discussed the example in theory also that we can have

example in theory also that we can have multiple modes. So here you can see here

multiple modes. So here you can see here that we do have multiple modes.

that we do have multiple modes. Now this is how we have done it manually

Now this is how we have done it manually right. But do we have functions for it?

right. But do we have functions for it? Yes we have. What is that functions?

Yes we have. What is that functions? So in real life we don't do manually we

So in real life we don't do manually we use functions. So we have for mean we

use functions. So we have for mean we have dot mean function.

have dot mean function. So I'll print this

So I'll print this mean is

similarly for median and for mode

and for mode for median and for mode we have these

for median and for mode we have these functions.

functions. Dear everyone so you can see here you

Dear everyone so you can see here you are getting multiple modes here. Why?

are getting multiple modes here. Why? because we have multiple values which is

because we have multiple values which is repeated equal number of times like the

repeated equal number of times like the example we discussed in theory. So

example we discussed in theory. So that's why you have multiple modes here.

that's why you have multiple modes here. Clear everyone. So in real we can use

Clear everyone. So in real we can use these functions to find mean, median and

these functions to find mean, median and mode.

mode. So see the difference in the output. You

So see the difference in the output. You have the index values jumbled up here,

have the index values jumbled up here, right? And in ignore index you don't

right? And in ignore index you don't have the jumbled up values.

have the jumbled up values. Okay. So we have found the mean, median,

Okay. So we have found the mean, median, mode. Then we will find the variance and

mode. Then we will find the variance and standard deviation.

standard deviation. For variance the function is dot var.

For variance the function is dot var. Okay. And for standard deviation the

Okay. And for standard deviation the function name is it is std.

function name is it is std. Run this and you can see variance of

Run this and you can see variance of your data is and standard deviation of

your data is and standard deviation of your data is this. Right? Now tell me

your data is this. Right? Now tell me relationship of this is very good

relationship of this is very good interview question also. What is the

interview question also. What is the relationship between variance and

relationship between variance and standard deviation? What is the

standard deviation? What is the relationship between variance and

relationship between variance and standard deviation? Guys, we have

standard deviation? Guys, we have studied this. Your standard deviation

studied this. Your standard deviation is equals to square root of

is equals to square root of standard variance. That was the

standard variance. That was the relationship. Now can we cross check it

relationship. Now can we cross check it that is it really true that means this

that is it really true that means this value should be the square root of this

value should be the square root of this value. So let us check how we can do it.

value. So let us check how we can do it. First import math in which std is equals

First import math in which std is equals to math dot square root

to math dot square root or the function name we can write sqrt

or the function name we can write sqrt also. Yes, just a minute. We can write

also. Yes, just a minute. We can write sqrt in which I will pass the variable

sqrt in which I will pass the variable name. So let me store it into a

name. So let me store it into a variable. So I'll take this value and

variable. So I'll take this value and store it into or I can we can directly

store it into or I can we can directly pass also instead of storing and st let

pass also instead of storing and st let it run. So don't you think so you are

it run. So don't you think so you are getting the exact value. So it is now

getting the exact value. So it is now proven that your standard deviation is

proven that your standard deviation is actually the square root of variance

actually the square root of variance only. So the thing which we have studied

only. So the thing which we have studied in theory we have proved it here.

in theory we have proved it here. Similarly how to find the range? We have

Similarly how to find the range? We have studied about range also. What is the

studied about range also. What is the formula to find the range?

formula to find the range? Maximum minus minimum. Right? So how we

Maximum minus minimum. Right? So how we can find the maximum?

can find the maximum? Which function we have for maximum dot

Which function we have for maximum dot max right? Similarly for minimum we have

max right? Similarly for minimum we have the function dot min right and we can

the function dot min right and we can print

print maximum

maximum value is max variable and print minimum

value is max variable and print minimum value is min variable. So this is your

value is min variable. So this is your maximum. This is your minimum. And we

maximum. This is your minimum. And we can easily find the range. Now range is

can easily find the range. Now range is equals to max minus min

equals to max minus min and print

and print range of the data is the range variable.

range of the data is the range variable. So this is how you can calculate the

So this is how you can calculate the range also. Clear?

range also. Clear? So you can see how these concepts are

So you can see how these concepts are very easy to implement right. So we

very easy to implement right. So we found mean, median, mode, we found

found mean, median, mode, we found range, we found variance, standard

range, we found variance, standard deviation. We are left with percentiles,

deviation. We are left with percentiles, quartiles

quartiles and correlation. These three topics is

and correlation. These three topics is what we left. So let's talk about

what we left. So let's talk about percentiles. So everyone remembers the

percentiles. So everyone remembers the concept of percentiles.

concept of percentiles. Let me relate to that example. Now

Let me relate to that example. Now suppose you have one student in the

suppose you have one student in the class who scored 75 marks out of 100.

class who scored 75 marks out of 100. Okay. And this 75 value is at 60th

Okay. And this 75 value is at 60th percentile. What does this means that

percentile. What does this means that 60%age

60%age 60%age

60%age of other students in the class have less

of other students in the class have less marks than 75.

marks than 75. So percentiles are the observations.

So percentiles are the observations. They will give you a value below which

They will give you a value below which some percentage of other values belong.

some percentage of other values belong. Clear?

Clear? So how we can find it using Python?

So how we can find it using Python? Let's see. So suppose if we want to find

Let's see. So suppose if we want to find 75 percentile, we will use the function

75 percentile, we will use the function np do. Okay, in which I will pass the

np do. Okay, in which I will pass the column name and comma which percentile I

column name and comma which percentile I want. So suppose I want to check what

want. So suppose I want to check what will be 75 percentile value. So print

will be 75 percentile value. So print this. Numpy is not defined. I think we

this. Numpy is not defined. I think we have run the code for numpy. Okay, we

have run the code for numpy. Okay, we have not run this as n.

have not run this as n. So you can see that at 75%

So you can see that at 75% value is 9193

value is 9193 which basically means that 75 percentile

which basically means that 75 percentile percentage of other values is less than

percentage of other values is less than this value. Clear? Similarly you can

this value. Clear? Similarly you can find 25 and you can find 15 also. So

find 25 and you can find 15 also. So give me what is the value of 25

give me what is the value of 25 percentile? what value you will get for

percentile? what value you will get for 25%.

25%. So I'll copy the same code here. I will

So I'll copy the same code here. I will change the variable name and I'll 25

change the variable name and I'll 25 and so yes 431.

and so yes 431. Similarly we can find the quartiles

Similarly we can find the quartiles also. So what was quartiles? Remember

also. So what was quartiles? Remember Q1, Q2, Q3, Q1 means 25 percentile, Q2

Q1, Q2, Q3, Q1 means 25 percentile, Q2 means 15 and Q3 means 75, right? So

means 15 and Q3 means 75, right? So either we can find the percentile also

either we can find the percentile also like here this 25 percentile is

like here this 25 percentile is basically my Q1. So like this also we

basically my Q1. So like this also we can find but we can use another function

can find but we can use another function also. What is that? I'll take the column

also. What is that? I'll take the column name dot quantile function in which I

name dot quantile function in which I will pass 0.25.

will pass 0.25. So that is 431. You can see exact value

So that is 431. You can see exact value you got. So you can write here Q1 is

you got. So you can write here Q1 is equals to print Q1. So either you can

equals to print Q1. So either you can find through percentiles also or you can

find through percentiles also or you can find it to quartiles also. Tell me what

find it to quartiles also. Tell me what value you will get for Q2 and Q3. Let's

value you will get for Q2 and Q3. Let's see for Q2 and Q3 what value you will

see for Q2 and Q3 what value you will get. Q2 means 50%.

get. Q2 means 50%. So this is giving you the value below

So this is giving you the value below which all other values all the 100% of

which all other values all the 100% of values belong. That is means we are

values belong. That is means we are finding the maximum value and you can

finding the maximum value and you can see here the maximum value is also same.

see here the maximum value is also same. So yes q2 and Q3 value also we got

So yes q2 and Q3 value also we got perfectly fine. See how easy it is to

perfectly fine. See how easy it is to implement in Python. Right? And the last

implement in Python. Right? And the last concept we are here with correlation.

concept we are here with correlation. So remember how many types of

So remember how many types of correlation we were having? We discussed

correlation we were having? We discussed positive like we discussed in the

positive like we discussed in the starting positive, negative and zero. So

starting positive, negative and zero. So how we find it? Function name is dot

how we find it? Function name is dot core.

core. So DF you can see you're getting the

So DF you can see you're getting the correlation values of each and every

correlation values of each and every column here right and also we can plot

column here right and also we can plot which visualization plot we use we have

which visualization plot we use we have discussed guys like that time you were

discussed guys like that time you were not aware about that plot so let me

not aware about that plot so let me discuss that plot with you which plot I

discuss that plot with you which plot I told you guys we use for correlation

told you guys we use for correlation heat map and how we can get that heat

heat map and how we can get that heat map SNS do heat map here we will find df

map SNS do heat map here we will find df core run this

core run this see you're getting the proper heat pipe

see you're getting the proper heat pipe now how to interpret the values you can

now how to interpret the values you can write here n not is equals to true so

write here n not is equals to true so you can see values are also visible but

you can see values are also visible but values are overlapped here right so I

values are overlapped here right so I will increase the figure size so plt dot

will increase the figure size so plt dot figure

figure inside that we will increase the figure

inside that we will increase the figure size is equals to I think 15 10 will

size is equals to I think 15 10 will work fine for my system. So yes you can

work fine for my system. So yes you can see the values here. Clear everyone? So

see the values here. Clear everyone? So you can see here some negative values

you can see here some negative values and you can see some positive values. So

and you can see some positive values. So negative means they are negatively

negative means they are negatively correlated and positive means they are

correlated and positive means they are positively correlated.

positively correlated. Clear everyone? So this is how we find

Clear everyone? So this is how we find the correlation.

the correlation. Just a quick info guys, Intellipath

Just a quick info guys, Intellipath offers a data science course in

offers a data science course in collaboration with iHub, IIT Riy which

collaboration with iHub, IIT Riy which will help you master concepts like

will help you master concepts like Python, SQL, machine learning, AI,

Python, SQL, machine learning, AI, PowerBI and more. With this course, we

PowerBI and more. With this course, we have already helped thousands of

have already helped thousands of professionals in successful career

professionals in successful career transition. You can check out their

transition. You can check out their testimonials on our achievers channel

testimonials on our achievers channel whose link is given in the description

whose link is given in the description below. Without a doubt, this course can

below. Without a doubt, this course can set your career to new heights. So visit

set your career to new heights. So visit the course page link given below in the

the course page link given below in the description and take the first step

description and take the first step towards career growth with the data

towards career growth with the data science course. So first thing first

science course. So first thing first what is the difference between statistic

what is the difference between statistic and statistics? So descriptive is done.

and statistics? So descriptive is done. So what is the difference between

So what is the difference between statistic and statistics? Statistic

statistic and statistics? Statistic versus statistics.

versus statistics. This is the first question I asked when

This is the first question I asked when I talk about statistics. If you observe

I talk about statistics. If you observe there's just one extra letter s in the

there's just one extra letter s in the end right that's the difference in the

end right that's the difference in the spelling not in the concept status uh

spelling not in the concept status uh statistic can be a single piece of data

statistic can be a single piece of data okay so you this is a question out of

okay so you this is a question out of syllabus right you might say you have

syllabus right you might say you have done this statistics and so I think this

done this statistics and so I think this is this is a more like a teacher's way

is this is a more like a teacher's way of testing where the you know unexpected

of testing where the you know unexpected question might come so see uh statist

question might come so see uh statist Statistic you are close statistic talks

Statistic you are close statistic talks about statistical technique

about statistical technique statistical technique like you know Z

statistical technique like you know Z test you know t test k square test right

test you know t test k square test right all these are your statistic more like

all these are your statistic more like it's an approach it's a method within

it's an approach it's a method within the universe of what statistics

the universe of what statistics statistic is a subject right so

statistic is a subject right so statistic is more like a method or

statistic is more like a method or approach a technique tech which you

approach a technique tech which you apply on a set of data which you have

apply on a set of data which you have collected over a piece of over a period

collected over a piece of over a period of time you have done some

of time you have done some transformation you are going to analyze

transformation you are going to analyze with the help of this test okay so

with the help of this test okay so statistics you already know it's a

statistics you already know it's a subject quite an quite an interesting

subject quite an quite an interesting subject how was your introduction to

subject how was your introduction to statistics like did you feel that

statistics like did you feel that statistics can be a really interesting

statistics can be a really interesting you know topic or interesting area where

you know topic or interesting area where you can learn how many of you guys felt

you can learn how many of you guys felt that that oh statistics something I

that that oh statistics something I should you know pick it up studied

should you know pick it up studied statistics till the descriptive was

statistics till the descriptive was taught to you right I know some of the

taught to you right I know some of the students are there okay reasonable quite

students are there okay reasonable quite interesting so let me couple of things

interesting so let me couple of things and let's see understand you few things

and let's see understand you few things uh that what is your understanding of

uh that what is your understanding of descriptive statistics a little bit more

descriptive statistics a little bit more so that I can use some of those things

so that I can use some of those things in inferential so I'm just making sure

in inferential so I'm just making sure so tell Tell me one thing if I talk

so tell Tell me one thing if I talk about salary right and I have a very

about salary right and I have a very good diagram for that. So you must have

good diagram for that. So you must have this is the time by the way guys. So

this is the time by the way guys. So those who have been to uh engine

those who have been to uh engine colleges or typically I would say I

colleges or typically I would say I December is day one uh December one is

December is day one uh December one is day zero for them. So this is the moment

day zero for them. So this is the moment I am sure my uh juniors will be getting

I am sure my uh juniors will be getting placed right. So this is a bell curve

placed right. So this is a bell curve which I put together and you might be

which I put together and you might be seeing something coming in the news

seeing something coming in the news right 1 K package let me tell you I have

right 1 K package let me tell you I have been asked this question a lot of time

been asked this question a lot of time one K package is a myth or more like

one K package is a myth or more like it's a misinformation

it's a misinformation given to uh news media print to get some

given to uh news media print to get some footage or mileage definitely some of

footage or mileage definitely some of the guys maybe one or two or three

the guys maybe one or two or three people in a batch they get one cr but

people in a batch they get one cr but that's all in dollars means it's not in

that's all in dollars means it's not in India, it's in US or likes of US. So

India, it's in US or likes of US. So converting dollar into rupees and then

converting dollar into rupees and then telling you are getting a package of 1

telling you are getting a package of 1 cr. The people forget uh that they have

cr. The people forget uh that they have to spend in dollar. They'll not take a

to spend in dollar. They'll not take a flight, come back to India and spend in

flight, come back to India and spend in rupees. Understand my point? So 1 K

rupees. Understand my point? So 1 K package is okay. U right? It's not like

package is okay. U right? It's not like in India hardly anybody gets as a

in India hardly anybody gets as a fresher 1 K package. If some companies

fresher 1 K package. If some companies giving their idiots the company owners

giving their idiots the company owners or the board is idiot okay doesn't make

or the board is idiot okay doesn't make sense so definitely you can see this is

sense so definitely you can see this is a skewed distribution and as you can

a skewed distribution and as you can also see I've already written it's a

also see I've already written it's a right skewed anybody why it is called

right skewed anybody why it is called right skew why is it called why is it

right skew why is it called why is it called so because it has a long tail

called so because it has a long tail has a long tail on the right side some

has a long tail on the right side some of the guys will be wondering what is

of the guys will be wondering what is that long tail let me highlight You'll

that long tail let me highlight You'll see it. Can you see the distribution is

see it. Can you see the distribution is going like this? Has a long tail. This

going like this? Has a long tail. This is what we say long tail and the peak is

is what we say long tail and the peak is on the left hand side. So it's a right

on the left hand side. So it's a right skewed. Okay. Now you must have been

skewed. Okay. Now you must have been taught uh when the data is dist when the

taught uh when the data is dist when the data is symmetric when the data is uh

data is symmetric when the data is uh having a shape like this the

having a shape like this the distribution. So then mean median mode

distribution. So then mean median mode all are going to be at the same place.

all are going to be at the same place. Remember now when the data is skewed

Remember now when the data is skewed then there is an issue doing mean here

then there is an issue doing mean here okay so far with me guys doing mean here

okay so far with me guys doing mean here and what is the reason for that because

and what is the reason for that because mean is sensitive to outliers I hope you

mean is sensitive to outliers I hope you have been must have been taught this

have been must have been taught this right mean is sensitive to outlier and

right mean is sensitive to outlier and this is a very very common concept are

this is a very very common concept are you guys aware mean is sensitive to

you guys aware mean is sensitive to outliers let me show you if you're not

outliers let me show you if you're not tell me one thing I've also written some

tell me one thing I've also written some package is okay let's say this is not 51

package is okay let's say this is not 51 by the way this is 5 LPA let me write

by the way this is 5 LPA let me write clearly else you start laughing at me 5

clearly else you start laughing at me 5 LPA let's say most of the guys are

LPA let's say most of the guys are getting 5 LPA that becomes if you think

getting 5 LPA that becomes if you think this is your you know this is your the

this is your you know this is your the peak of the salary you can say like a

peak of the salary you can say like a mode so this is the highest value which

mode so this is the highest value which you see in the normal distribution so

you see in the normal distribution so that's your highest frequency value

that's your highest frequency value which means it's the mode Right? Or you

which means it's the mode Right? Or you can say close to median. But it cannot

can say close to median. But it cannot be the average. You know why? Because of

be the average. You know why? Because of these guys who are sitting somewhere

these guys who are sitting somewhere here. Can you see this? Were sitting

here. Can you see this? Were sitting here. They are going to impact the

here. They are going to impact the average. Let me give a simple example.

average. Let me give a simple example. So let's say these are your salaries 35.

So let's say these are your salaries 35. I'm taking some numbers in LPA lakhs

I'm taking some numbers in LPA lakhs perm 50 and 80. Let's say. So let's say

perm 50 and 80. Let's say. So let's say these guys salary if you want to find

these guys salary if you want to find and this is all in LPA. So if you do the

and this is all in LPA. So if you do the math what is the mean salary? If someone

math what is the mean salary? If someone can do this for me you will get to how

can do this for me you will get to how much and I think I have 3 6 9 10. So

much and I think I have 3 6 9 10. So what is it coming guys? Did someone do

what is it coming guys? Did someone do 81.5?

81.5? No. How can be 81.5 average? Only two

No. How can be 81.5 average? Only two values are 5080 18.2 so many people are

values are 5080 18.2 so many people are telling. So let's say that's the

telling. So let's say that's the average. Okay? Right? But if you observe

average. Okay? Right? But if you observe closely, most of your data distribution

closely, most of your data distribution is not even higher than 10. Right? Guys,

is not even higher than 10. Right? Guys, can you see this? Just because of these

can you see this? Just because of these two guys who are like the IT that media

two guys who are like the IT that media guy who are getting this high package

guy who are getting this high package has impacted your overall salary. Why

has impacted your overall salary. Why why don't you do that? If you you know

why don't you do that? If you you know exclude these outliers and do this math

exclude these outliers and do this math again add this eight salary only and

again add this eight salary only and divide by 10 or sorry divide by 8 how

divide by 10 or sorry divide by 8 how much you are getting guys I think it

much you are getting guys I think it will be close to six or seven 6.5 okay

will be close to six or seven 6.5 okay thank you those who are helping me do

thank you those who are helping me do you think which one looks more real

you think which one looks more real closer to reality this one or this one

closer to reality this one or this one as an average salary

as an average salary definitely 6.5 right definitely 6.5

definitely 6.5 right definitely 6.5 Five. So you understand when I say mean

Five. So you understand when I say mean is sensitive to outliers and that's the

is sensitive to outliers and that's the reason people say whenever you are

reason people say whenever you are looking at a data with outliers you

looking at a data with outliers you should not go with mean you should go

should not go with mean you should go with median. Now what don't we find

with median. Now what don't we find median for this one. So just arrange 3 4

median for this one. So just arrange 3 4 3 4 5 6 8 10 11 50 8 4 6 8 I missed one

3 4 5 6 8 10 11 50 8 4 6 8 I missed one number. I think five is two times right

number. I think five is two times right guys. Okay. Now tell me guys what is the

guys. Okay. Now tell me guys what is the median? So I I'll come to URL 2 4 5

median? So I I'll come to URL 2 4 5 right. So what you will do my dear you

right. So what you will do my dear you will take first four and second four

will take first four and second four last four. So you will be left out with

last four. So you will be left out with what? What is the average? I'm d I'm

what? What is the average? I'm d I'm sure you must have been taught how to

sure you must have been taught how to calculate median. So what is the average

calculate median. So what is the average of these two? I'm not going to teach

of these two? I'm not going to teach that. I'm just making a Now tell me

that. I'm just making a Now tell me looking at this number and this number

looking at this number and this number which one is more realistic mean is more

which one is more realistic mean is more realistic or median is more realistic to

realistic or median is more realistic to the average which you might. So now you

the average which you might. So now you understand. So that's why if you see if

understand. So that's why if you see if some apps like glass door or some

some apps like glass door or some similar like pay scale if they record

similar like pay scale if they record average salary you should not never go

average salary you should not never go with that because let me also tell you

with that because let me also tell you there are students who are coming from

there are students who are coming from ISP and all and they're getting 30 lakhs

ISP and all and they're getting 30 lakhs as a fresher salary and then there will

as a fresher salary and then there will be student who are coming from normal

be student who are coming from normal college and they will be getting three

college and they will be getting three lakhs or five lakhs. So 30 lakhs guys if

lakhs or five lakhs. So 30 lakhs guys if you keep taking right blindly he will he

you keep taking right blindly he will he will be in a wrong mean he will in a

will be in a wrong mean he will in a bubble that thinking oh I am also going

bubble that thinking oh I am also going to get 30 lakhs so just to you know just

to get 30 lakhs so just to you know just to bust the this thought that you know

to bust the this thought that you know it doesn't work like this okay so does

it doesn't work like this okay so does it make sense guys now coming back to RN

it make sense guys now coming back to RN outlier identifying outlier treating

outlier identifying outlier treating outlier okay so remember I had taught

outlier okay so remember I had taught you guys box plot okay so box plot says

you guys box plot okay so box plot says that if something is in you know I'm

that if something is in you know I'm just more like you know revising

just more like you know revising revisiting few things so if you do a

revisiting few things so if you do a little math and try to identify an

little math and try to identify an outlier you will be able to do that so

outlier you will be able to do that so now my question is did you guys go

now my question is did you guys go through some of those examples where you

through some of those examples where you identify the outlier and got treated and

identify the outlier and got treated and treat the outlier already Right? So

treat the outlier already Right? So generally these there are two on the

generally these there are two on the upper side and two on the lower side we

upper side and two on the lower side we have outliers. Okay. So there is a

have outliers. Okay. So there is a statistical way to calculate these

statistical way to calculate these whiskers known as whisker. I have also

whiskers known as whisker. I have also talked about it in my visualization.

talked about it in my visualization. Let's say this is your upper threshold.

Let's say this is your upper threshold. So guys I'll not repeat it. I'll quickly

So guys I'll not repeat it. I'll quickly write the equation. You take the Q3 and

write the equation. You take the Q3 and in then you add the 1.5 times.

in then you add the 1.5 times. Statistically this number has been given

Statistically this number has been given to you. this whole formula you do the

to you. this whole formula you do the intercortile range which is Q3 minus Q1

intercortile range which is Q3 minus Q1 and multiply with 1.5

and multiply with 1.5 anything which is greater than this like

anything which is greater than this like these two guys are going to be outlier

these two guys are going to be outlier similarly you can also do lower

similarly you can also do lower threshold

threshold just refresh your memory so that will be

just refresh your memory so that will be Q1 minus 1.5 IQR IQR is enter range

Q1 minus 1.5 IQR IQR is enter range anything lower than this lower threshold

anything lower than this lower threshold like these guys are outlash and you can

like these guys are outlash and you can also make a little sense out of it. If

also make a little sense out of it. If someone is getting outclam something

someone is getting outclam something unexpected or weird, right? Everybody's

unexpected or weird, right? Everybody's getting five lakhs, six lakhs, 10 lakhs

getting five lakhs, six lakhs, 10 lakhs and somebody is getting 80 lakhs

and somebody is getting 80 lakhs definitely that is an outlier. That's a

definitely that is an outlier. That's a normal way of saying it. So outlier

normal way of saying it. So outlier means which is not normal which doesn't

means which is not normal which doesn't fit in normal scenario is an outlier

fit in normal scenario is an outlier literally I'm trying to explain

literally I'm trying to explain mathematically. This is how do you do

mathematically. This is how do you do that? Okay. Now guys, I'll jump one

that? Okay. Now guys, I'll jump one question and if you have been taught if

question and if you have been taught if you have been taught normal

you have been taught normal distribution, you should be able to take

distribution, you should be able to take it because then you must have been

it because then you must have been taught jet score as well. Guys, jet

taught jet score as well. Guys, jet score was taught to you guys. No. So

score was taught to you guys. No. So then what did you guys do in normal

then what did you guys do in normal distribution? There's no normal

distribution? There's no normal distribution without jet score. And

distribution without jet score. And without the normal distribution jet

without the normal distribution jet score, I cannot go into inferial. So

score, I cannot go into inferial. So it's a prerequisite. Min and max is

it's a prerequisite. Min and max is nothing. Maximum value in the data is

nothing. Maximum value in the data is maximum. Minimum is minimum. Okay.

maximum. Minimum is minimum. Okay. Simple. Why the outliers are not

Simple. Why the outliers are not considered as what they they are? See

considered as what they they are? See this is this minimum. Now this minimum

this is this minimum. Now this minimum is like for their they are actually

is like for their they are actually taking at lower threshold and upper

taking at lower threshold and upper threshold. Okay. So don't get confused.

threshold. Okay. So don't get confused. This is not our minimum maximum.

This is not our minimum maximum. Obviously minimum of the data will be

Obviously minimum of the data will be the outliers, right? If they are lower

the outliers, right? If they are lower the threshold values, right? Lower than

the threshold values, right? Lower than the low threshold. you you have a good

the low threshold. you you have a good point but they were talking about this

point but they were talking about this whisker okay so remember uh there's a

whisker okay so remember uh there's a bell curve okay standard standard normal

bell curve okay standard standard normal distribution

distribution let me do the let me do the thing okay

let me do the let me do the thing okay so one of the key distribution is normal

so one of the key distribution is normal distribution also known as bell's bell

distribution also known as bell's bell curve bell's curve it has gotten a name

curve bell's curve it has gotten a name bell curve anybody knows why it is

bell curve anybody knows why it is because if You go to a temple, right?

because if You go to a temple, right? And have you seen a bell, right? The

And have you seen a bell, right? The normal distribution can be, you know,

normal distribution can be, you know, shown as a bell curve. Okay? And here

shown as a bell curve. Okay? And here you go. Now, if you look at the normal

you go. Now, if you look at the normal distribution, okay, most of the thing uh

distribution, okay, most of the thing uh in the in the world can be model as a

in the in the world can be model as a bell curve. Trust me on that. So that's

bell curve. Trust me on that. So that's what I was telling right if the normal

what I was telling right if the normal distribution is symmetric your mean

distribution is symmetric your mean median mode all will be on the same

median mode all will be on the same point okay so ideally when we talk about

point okay so ideally when we talk about standard normal distribution you make

standard normal distribution you make sure that it is symmetric it's not

sure that it is symmetric it's not skewed okay and if you look at the the

skewed okay and if you look at the the top one not the bottom one the top one

top one not the bottom one the top one you must be seeing that I have given

you must be seeing that I have given three colors one is you know between mu

three colors one is you know between mu plus minus sigma. So I hope you know

plus minus sigma. So I hope you know what is mu, what is sigma. So this is

what is mu, what is sigma. So this is going to cover how much of your data?

going to cover how much of your data? 68.27%.

68.27%. Right? And then if you look at the from

Right? And then if you look at the from mu + - 2 sigma

mu + - 2 sigma 95% of the data will lie there. And if

95% of the data will lie there. And if you go and stretch it further mu plus -

you go and stretch it further mu plus - 3 sigma 99% of the data will be there.

3 sigma 99% of the data will be there. So you might be wondering why do we need

So you might be wondering why do we need normal distribution right? uh and why mu

normal distribution right? uh and why mu plus minus sigma is so important. See

plus minus sigma is so important. See this is what has been found or observed

this is what has been found or observed on a normal distribution because it's

on a normal distribution because it's all about probabilities of what data

all about probabilities of what data will lie where. So if you take any

will lie where. So if you take any normal distribution uh curve you can

normal distribution uh curve you can always find mu which is your more like

always find mu which is your more like expectation of x don't don't go into

expectation of x don't don't go into this it's a random variable I'm damn

this it's a random variable I'm damn sure you don't know. So expectation of X

sure you don't know. So expectation of X is generally written as mu which is your

is generally written as mu which is your you can say mean but it's not real mean

you can say mean but it's not real mean because mean of deterministic data is

because mean of deterministic data is your average but this is more like

your average but this is more like expected value of X. So I'm going to

expected value of X. So I'm going to write expected value and the moment

write expected value and the moment expectation come it's probability okay

expectation come it's probability okay then you also have sigma which is your

then you also have sigma which is your you know that right standard deviation

you know that right standard deviation which is square root of variance.

which is square root of variance. Now what we are saying that if you take

Now what we are saying that if you take a range of mu + minus sigma 68 I'll say

a range of mu + minus sigma 68 I'll say roughly 68% of the data it will have

roughly 68% of the data it will have plus - 2 sigma it will be 95% of the

plus - 2 sigma it will be 95% of the data and new plus - 3 sigma 99% of the

data and new plus - 3 sigma 99% of the data will lie there. This is the reason

data will lie there. This is the reason the professors or the college

the professors or the college universities professors they use this

universities professors they use this like a lot. So suppose they fit on a

like a lot. So suppose they fit on a normal distribution on your uh to

normal distribution on your uh to calculate grades right so what happens

calculate grades right so what happens guys let's say and definitely it will

guys let's say and definitely it will not be like symmetric this normal

not be like symmetric this normal distribution can be skewed so they know

distribution can be skewed so they know that so we have a mean somewhere here so

that so we have a mean somewhere here so this is going to be mu they know that in

this is going to be mu they know that in this there will be some sigma so this is

this there will be some sigma so this is mu plus sigma this will be mu + 2 sigma

mu plus sigma this will be mu + 2 sigma and this will be mu + 3 sigma and let's

and this will be mu + 3 sigma and let's say this is mu minus sigma and Then they

say this is mu minus sigma and Then they create bands. So this is going to be

create bands. So this is going to be their favorite student which is maybe

their favorite student which is maybe one or two who maybe one or two right

one or two who maybe one or two right and they will be getting let's say a

and they will be getting let's say a plus grade. Then they'll go to the next

plus grade. Then they'll go to the next band which is mu plus 2 sigma to mu plus

band which is mu plus 2 sigma to mu plus 3 sigma focus guys. So all those guys

3 sigma focus guys. So all those guys who are here they are going to get let's

who are here they are going to get let's say grade A, grade B, grade C, grade D

say grade A, grade B, grade C, grade D and everybody who is behind this are

and everybody who is behind this are going to get what grade? F grade which

going to get what grade? F grade which is a fail grade. Uh first of all this is

is a fail grade. Uh first of all this is just a highle overview of how grading

just a highle overview of how grading system work. It's not that

system work. It's not that straightforward and by the just to give

straightforward and by the just to give an example and this is the most of the

an example and this is the most of the guys generally fra means you have to

guys generally fra means you have to repeat the course and exam as well. Are

repeat the course and exam as well. Are you guys able to follow how normal

you guys able to follow how normal distribution can help you? As a matter

distribution can help you? As a matter of fact even the COVID cases were

of fact even the COVID cases were modeled as a normal distribution income

modeled as a normal distribution income can be modeled as a normal distribution.

can be modeled as a normal distribution. ID's uh performance or result can be

ID's uh performance or result can be modeled as a normal distribution. So

modeled as a normal distribution. So normal distribution is like bread and

normal distribution is like bread and butter. Okay. The beauty about normal

butter. Okay. The beauty about normal distribution is that it start from

distribution is that it start from negative. So it can fit all the real

negative. So it can fit all the real data from minus infinity to plus

data from minus infinity to plus infinity. No other distribution can do

infinity. No other distribution can do justice for a continuous one. Okay. Now

justice for a continuous one. Okay. Now coming back coming back how we are going

coming back coming back how we are going to use it. Okay. By the way, you

to use it. Okay. By the way, you understand what is the meaning of

understand what is the meaning of standard deviation? What what is

standard deviation? What what is interpretation of high standard

interpretation of high standard deviation versus low standard deviation?

deviation versus low standard deviation? Anybody? So, for example, I give you two

Anybody? So, for example, I give you two curves. Tell me which one is having the

curves. Tell me which one is having the high high uh higher you know standard

high high uh higher you know standard deviation. Higher standard deviation

deviation. Higher standard deviation curve is which color black or red? Where

curve is which color black or red? Where do you observe higher standard

do you observe higher standard deviation? First one, right? Black.

deviation? First one, right? Black. those who are saying red. So please pay

those who are saying red. So please pay attention standard deviation means

attention standard deviation means spreadness of the data right. So how

spreadness of the data right. So how spreadness means amount of variation.

spreadness means amount of variation. So standard deviation and you know why

So standard deviation and you know why do we need standard deviation? Because

do we need standard deviation? Because can you see that the mean of the data

can you see that the mean of the data the peak of the data is same right? So

the peak of the data is same right? So just saying mean, median, mode is not

just saying mean, median, mode is not enough a lot of time right it's all same

enough a lot of time right it's all same but these two distribution are quite

but these two distribution are quite different. So the reason is that why

different. So the reason is that why standard deviation is required because

standard deviation is required because standard deviation talks about the

standard deviation talks about the amount of variation or dispersion also

amount of variation or dispersion also known as dispersion or also known as

known as dispersion or also known as spread of data amount of variation or

spread of data amount of variation or dispersion to data or spread of data. So

dispersion to data or spread of data. So you guys can clearly see that this guy

you guys can clearly see that this guy the black one is like quite varying and

the black one is like quite varying and the red one is like less varying if you

the red one is like less varying if you look at the similar range that's why

look at the similar range that's why this okay now guys there is a

this okay now guys there is a probability density function for the

probability density function for the standard normal distribution so what

standard normal distribution so what happens from the mean from the mean so

happens from the mean from the mean so probability density function now you

probability density function now you might be wondering what is it don't

might be wondering what is it don't wonder much I'll just use this jargon

wonder much I'll just use this jargon for you although it should have been

for you although it should have been with you but I can't do much about it.

with you but I can't do much about it. Probability density function is actually

Probability density function is actually a mathematical function to represent the

a mathematical function to represent the any distribution. So for normal

any distribution. So for normal distribution you also have a probability

distribution you also have a probability distribution function. Density function

distribution function. Density function those who have done a little bit of

those who have done a little bit of random variable they know that. So it is

random variable they know that. So it is actually a mathematical equation

actually a mathematical equation 1 upon roo<unk> under 2 pi sigma is

1 upon roo<unk> under 2 pi sigma is outside e to the power -/ x - mu by

outside e to the power -/ x - mu by sigma square where x can be between this

sigma square where x can be between this doesn't look this looks better. Okay

doesn't look this looks better. Okay guys now this is your normal

guys now this is your normal distribution this is not standard. Now

distribution this is not standard. Now the problem this guy has

the problem this guy has this is normal distribution normal

this is normal distribution normal distribution. Now giving letting giving

distribution. Now giving letting giving a little idea suppose you talk about a

a little idea suppose you talk about a normal distribution curve and this is

normal distribution curve and this is how it is. I hope everybody knows this

how it is. I hope everybody knows this the area under this curve is the total

the area under this curve is the total probability. Come on right if it

probability. Come on right if it represents all the different different

represents all the different different probabilities the total data will have

probabilities the total data will have probability equal to one. I think this

probability equal to one. I think this is common sense. So then if it is

is common sense. So then if it is symmetric about let's say about the

symmetric about let's say about the center then can I say that this area

center then can I say that this area minus infinity to 0 let me take some

minus infinity to 0 let me take some time to shade it how much this will be

time to shade it how much this will be probability value what is the area under

probability value what is the area under this curve what is this area if total is

this curve what is this area if total is one what is this area similarly on the

one what is this area similarly on the other side you can say what is it you

other side you can say what is it you tell me another five so total is one

tell me another five so total is one right so far with me guys

right so far with me guys Total is 1.5

Total is 1.5 is one. Since it is symmetric, it is

is one. Since it is symmetric, it is easy to do some of these juggling. So

easy to do some of these juggling. So you can say minus infinity to 0 is one

you can say minus infinity to 0 is one part and 0 to infinity it is another

part and 0 to infinity it is another part. Right? Then it is symmetric. But

part. Right? Then it is symmetric. But look at this. Suppose this is a

look at this. Suppose this is a distribution right? This is also normal

distribution right? This is also normal distribution. It is like skewed

distribution. It is like skewed distribution. Ideally people don't say

distribution. Ideally people don't say this is a normal distribution. Let me

this is a normal distribution. Let me also tell you this. But let's say the

also tell you this. But let's say the distribution is not symmetric. Let's say

distribution is not symmetric. Let's say this is the zero here. See about the

this is the zero here. See about the peak it is symmetric only but it has

peak it is symmetric only but it has shifted. Can you see that guys? Hello.

shifted. Can you see that guys? Hello. Are you with me? Now this is not at

Are you with me? Now this is not at zero. It has shifted towards right. Now

zero. It has shifted towards right. Now you tell me if I ask what is this area

you tell me if I ask what is this area minus infinity to0 minus infinity to0.

minus infinity to0 minus infinity to0. What is this area? Will it be easy for

What is this area? Will it be easy for you to calculate? I think no. Because to

you to calculate? I think no. Because to find this area and if you have done

find this area and if you have done little bit of maths it means that you

little bit of maths it means that you are going to integrate this guy from

are going to integrate this guy from minus infinity to 0 dx and you will die

minus infinity to 0 dx and you will die doing the integration. So statisticians

doing the integration. So statisticians mathematicians they agreed that you know

mathematicians they agreed that you know boss we can't do it all the time. So

boss we can't do it all the time. So they said why don't we normalize this

they said why don't we normalize this make it a standard normal distribution

make it a standard normal distribution doesn't matter what data we are getting

doesn't matter what data we are getting end of the day we are going to make it

end of the day we are going to make it look like what you see on the first one

look like what you see on the first one and that's what where we got the normal

and that's what where we got the normal distribution to the standard normal

distribution to the standard normal distribution who will be doing this

distribution who will be doing this integration right it is doable but it's

integration right it is doable but it's not easily doable so they thought why

not easily doable so they thought why don't we introduce standard normal

don't we introduce standard normal distribution which is the in Python you

distribution which is the in Python you must have he got this like oh I

must have he got this like oh I standardize the data have you heard this

standardize the data have you heard this word normalize the data the data

word normalize the data the data scientists they keep telling this all

scientists they keep telling this all the time which is overused or abused but

the time which is overused or abused but what is the interpretation of it means

what is the interpretation of it means that you will take all the x value then

that you will take all the x value then what you will do you will subtract the

what you will do you will subtract the mean and divide by sigma and you will

mean and divide by sigma and you will call a new variable called zed and this

call a new variable called zed and this is where I introduce the zed

is where I introduce the zed Have you ever observed this in some at

Have you ever observed this in some at some point x - mu - x - mu by sigma. So

some point x - mu - x - mu by sigma. So basically what you're doing you're

basically what you're doing you're taking the whole data subtract the mean

taking the whole data subtract the mean and divide by sigma means normalizing by

and divide by sigma means normalizing by sigma. This is known as you know zed zed

sigma. This is known as you know zed zed value or zed score right. And what

value or zed score right. And what you're going to do with this the idea is

you're going to do with this the idea is that if you take this and find

that if you take this and find expectation of zed which is like

expectation of zed which is like expectation of x - mu sigma this is a

expectation of x - mu sigma this is a little bit coming from random variable

little bit coming from random variable so please excuse me for that but I want

so please excuse me for that but I want to show you so if you do the maths it

to show you so if you do the maths it comes to be expectation of x minus

comes to be expectation of x minus expectation of mu 1 by sigma expectation

expectation of mu 1 by sigma expectation of x is also average if you remember

of x is also average if you remember what I have given on the top see here

what I have given on the top see here wow here so using that you write this as

wow here so using that you write this as mu why I'm using curly braces sorry and

mu why I'm using curly braces sorry and then expectation of anything constant is

then expectation of anything constant is also mu cancel my so this is coming to

also mu cancel my so this is coming to zero so the beauty of zed score is that

zero so the beauty of zed score is that expectation of zed is what zero which is

expectation of zed is what zero which is mean which means that mean for standard

mean which means that mean for standard normal distribution is always zero hello

normal distribution is always zero hello mean for standard normal distribution is

mean for standard normal distribution is zero. It is a calculated mean always.

zero. It is a calculated mean always. It's not the natural mean coming from

It's not the natural mean coming from the data. You have standardized it. So

the data. You have standardized it. So don't miss this step in the whole scheme

don't miss this step in the whole scheme of things. I'll take a pause here. You

of things. I'll take a pause here. You have done this extra step on your data

have done this extra step on your data to normalize it. And that's the reason

to normalize it. And that's the reason that your expectation of zed is coming

that your expectation of zed is coming to be zero. And likewise sigma will come

to be zero. And likewise sigma will come out to be one which I'm not deriving

out to be one which I'm not deriving because it will take me to the extensive

because it will take me to the extensive intensive calculation on random variable

intensive calculation on random variable which is not there for you. Okay. Let's

which is not there for you. Okay. Let's not go too mathematical as so far with

not go too mathematical as so far with me guys. Now you might be wondering what

me guys. Now you might be wondering what is the benefit of this? So benefit is

is the benefit of this? So benefit is there.

there. The moment you do this you started with

The moment you do this you started with fxx mu sigma = 1 upon<unk> under 2 pi

fxx mu sigma = 1 upon<unk> under 2 pi sigma e to the power -/

sigma e to the power -/ x - mu by sigma square. Now what you

x - mu by sigma square. Now what you have done my dear you have taken this

have done my dear you have taken this guy and you have calling this as zed

guy and you have calling this as zed focus.

focus. So now suddenly what's happening the

So now suddenly what's happening the transition is happening and I'll write

transition is happening and I'll write it will become fz

it will become fz z mu becomes zero sigma becomes 1 and

z mu becomes zero sigma becomes 1 and then you will write 1 upon roo<unk>

then you will write 1 upon roo<unk> under 2 pi sigma is 1. So this will be a

under 2 pi sigma is 1. So this will be a shorter one e to the power minus j² by 2

shorter one e to the power minus j² by 2 guys does it look simpler than than the

guys does it look simpler than than the than the one which I just showed you. So

than the one which I just showed you. So right your your equation gets simpler

right your your equation gets simpler and you can generate a table out of it

and you can generate a table out of it and that's where you're going to use it.

and that's where you're going to use it. So when I say table then you'll be

So when I say table then you'll be looking at something. Yeah it is

looking at something. Yeah it is difficult to remember since I'm teaching

difficult to remember since I'm teaching I can but that's the reason they not

I can but that's the reason they not they're not hoping you to remember this.

they're not hoping you to remember this. They're telling you forgot forget about

They're telling you forgot forget about this. Why don't you learn this table?

this. Why don't you learn this table? Now this is the jet score table and we

Now this is the jet score table and we need to understand it uh in some time

need to understand it uh in some time but not not not now what this table does

but not not not now what this table does how it can help us and blah blah blah

how it can help us and blah blah blah right now tell me one thing this is

right now tell me one thing this is known as jet score table zed score table

known as jet score table zed score table okay I'm not in not paying too much

okay I'm not in not paying too much attention on uh so might be sounding

attention on uh so might be sounding like zed score but it is not it's not

like zed score but it is not it's not zed it's z score table or zcore table

zed it's z score table or zcore table okay

okay so Now if you see guys if you look at

so Now if you see guys if you look at you know Z =0 which is here this is zero

you know Z =0 which is here this is zero okay so all the values of you know zed

okay so all the values of you know zed is coming here okay on the left why it's

is coming here okay on the left why it's picking it up all the values are coming

picking it up all the values are coming on the left then can see zed value 0.1

on the left then can see zed value 0.1 2.3 and you have another set of values

2.3 and you have another set of values which are decimal values on the on the

which are decimal values on the on the horizontal line. So you have two line

horizontal line. So you have two line vertical line and horizontal line. Let's

vertical line and horizontal line. Let's say I'm trying to find a Z score for

say I'm trying to find a Z score for let's say some value like let's say what

let's say some value like let's say what is the Z score for one. So when the

is the Z score for one. So when the moment I say one it means that from 0 to

moment I say one it means that from 0 to 1 what is the value? So 0 to one how

1 what is the value? So 0 to one how will you find? You can clearly see that

will you find? You can clearly see that this is your value. What is that 3413?

this is your value. What is that 3413? It's your probability. Okay don't get me

It's your probability. Okay don't get me wrong it's your probability. Okay. So

wrong it's your probability. Okay. So let's say if I fit a normal distribution

let's say if I fit a normal distribution curve. I take an example to make you

curve. I take an example to make you understand this table. Let's say this is

understand this table. Let's say this is zero and this is one and this is minus

zero and this is one and this is minus one. I'm interested in finding the mu +

one. I'm interested in finding the mu + - sigma. Guys this is mu plus - sigma.

- sigma. Guys this is mu plus - sigma. Now mu is 0 and sigma is 1. Mu + -

Now mu is 0 and sigma is 1. Mu + - sigma. Basically you are trying to find

sigma. Basically you are trying to find 0 + - one. Now to get it you can also

0 + - one. Now to get it you can also understand that this is symmetric. If

understand that this is symmetric. If you find 0 to one you can also get the

you find 0 to one you can also get the whole thing and as this example say uh

whole thing and as this example say uh this always talks about 0 to zed right.

this always talks about 0 to zed right. So 0 to one if I say already I

So 0 to one if I say already I identified it. See zed 0 to one means in

identified it. See zed 0 to one means in the chart you have to find one. What is

the chart you have to find one. What is the value for one? Zed equal to 1. What

the value for one? Zed equal to 1. What is this value my dear? 3143

is this value my dear? 3143 and since it is symmetric it doesn't

and since it is symmetric it doesn't matter the Z value for minus1 to 0 will

matter the Z value for minus1 to 0 will be same area wise you can look at that

be same area wise you can look at that it's not in this table do you see any

it's not in this table do you see any negative value minus one or not right if

negative value minus one or not right if you do the total are you seeing a number

you do the total are you seeing a number which is very much familiar to you let

which is very much familiar to you let me show you in case you are not able to

me show you in case you are not able to recall what is this number guys mu plus

recall what is this number guys mu plus - 68.27 27 are you able to almost

- 68.27 27 are you able to almost getting the same hello in percentage

getting the same hello in percentage what is it 68.26% 26% of the data.

what is it 68.26% 26% of the data. Interesting. So this table works, right?

Interesting. So this table works, right? This table works. So what you can

This table works. So what you can actually calculate and find and then

actually calculate and find and then answer you are getting make sense guys.

answer you are getting make sense guys. Why it is 50%.

Why it is 50%. Tometric question and he was supposed to

Tometric question and he was supposed to find this angle. So so he supposed to

find this angle. So so he supposed to find this angle and then what he did he

find this angle and then what he did he drew this extension and then he was

drew this extension and then he was thinking that this will be 60. So this

thinking that this will be 60. So this angle will be 180 minus 60. So 120.

angle will be 180 minus 60. So 120. What is the basis for this? Are you

What is the basis for this? Are you observing it with the line? You're not

observing it with the line? You're not looking at any condition. What angle?

looking at any condition. What angle? What is the triangle shape? Right? So

What is the triangle shape? Right? So exactly you're doing the same that

exactly you're doing the same that sometime people like student do this

sometime people like student do this blunder. They they just observe thing

blunder. They they just observe thing and then answer. Here observation

and then answer. Here observation doesn't work all the time. You have to

doesn't work all the time. You have to go with the concept. The concept says

go with the concept. The concept says that if it is 0 to one, how will you

that if it is 0 to one, how will you find 0 to one in the Z score table? You

find 0 to one in the Z score table? You will go and find what is the value for

will go and find what is the value for one because this table gives you from 0

one because this table gives you from 0 to 0 to zed. So suppose you want to find

to 0 to zed. So suppose you want to find 0 to one. I can see that 0 to 1 is this.

0 to one. I can see that 0 to 1 is this. Then you might be wondering what is

Then you might be wondering what is this? You should ask that what is this?

this? You should ask that what is this? What is this? So suppose you want to

What is this? So suppose you want to find till 1.5,

find till 1.5, right? Or let's say you want to find

right? Or let's say you want to find 1.05. 05. How will you find 1.05? So

1.05. 05. How will you find 1.05? So you'll take one here my dear and 0.05

you'll take one here my dear and 0.05 you will look at here and then you will

you will look at here and then you will draw two lines and then where it is

draw two lines and then where it is intersecting that's the value. So what

intersecting that's the value. So what is the value guys? 3531. Are you getting

is the value guys? 3531. Are you getting how to use this? This is for decimal

how to use this? This is for decimal places. Okay don't feel that if you're

places. Okay don't feel that if you're not getting it uh what's going on? It's

not getting it uh what's going on? It's normal behavior with the normal

normal behavior with the normal distribution. Okay. Normal is not

distribution. Okay. Normal is not normal. seems doesn't seem like that

normal. seems doesn't seem like that right just trying to be you know making

right just trying to be you know making the moment like normal doesn't seem to

the moment like normal doesn't seem to be normal right why don't we do a

be normal right why don't we do a question right so let me take this top

question right so let me take this top 5%

5% of applicants

of applicants as measured by

as measured by GRE scores top 5% of applicants as

GRE scores top 5% of applicants as measured by GRE scores will receive

measured by GRE scores will receive scholarships

scholarships If GRE

If GRE is normally distributed with the mean

is normally distributed with the mean 500 and the variance right sigma square

500 and the variance right sigma square is variance so sigma is 100 being smart

is variance so sigma is 100 being smart how does how high does your GR score

how does how high does your GR score have to be to qualify for a scholarship

have to be to qualify for a scholarship real question we're trying to estimate

real question we're trying to estimate it have to be to qualify for a

it have to be to qualify for a scholarship

scholarship so this is for all those guys having

so this is for all those guys having American dream going to US going to US

American dream going to US going to US university get a job learn there study

university get a job learn there study there and settle there right so they

there and settle there right so they write GRE exam and definitely those who

write GRE exam and definitely those who have I know some of the guys might be in

have I know some of the guys might be in US already so they know it's not that

US already so they know it's not that easy first of all you need to have a lot

easy first of all you need to have a lot of money in the bank or you get a

of money in the bank or you get a scholarship there only two option yeah

scholarship there only two option yeah so did you understand the question guys

so did you understand the question guys you have a normal distribution mu is 500

you have a normal distribution mu is 500 sigma is 100 and they're asking that

sigma is 100 and they're asking that what should be the value sorry what

what should be the value sorry what should be the GR score to qualify for a

should be the GR score to qualify for a scholarship you will not be able to do

scholarship you will not be able to do this right now let me do it for you so

this right now let me do it for you so guys definitely when you look at this

guys definitely when you look at this it's a normal distribution it's a not a

it's a normal distribution it's a not a standard normal distribution it's a

standard normal distribution it's a normal distribution because why it is

normal distribution because why it is not a standard normal distribution still

not a standard normal distribution still you know awake and not sleepy

you know awake and not sleepy how So how do you realize that it is

how So how do you realize that it is normal distribution not a standard

normal distribution not a standard normal distribution? Come on Jantaa let

normal distribution? Come on Jantaa let me know what is in standard normal

me know what is in standard normal distribution what is the value of mu is

distribution what is the value of mu is it 500 it's normal distribution right

it 500 it's normal distribution right because mu is something non zero and

because mu is something non zero and sigma is also not one it is 100 so it's

sigma is also not one it is 100 so it's normal distribution cool now I needed to

normal distribution cool now I needed to convert into standard normal right so

convert into standard normal right so standard normal I have to get into how

standard normal I have to get into how will I get it I'll calculate zed which

will I get it I'll calculate zed which is x - mu by sigma which is basically x

is x - mu by sigma which is basically x - 500 by 100 right guys so far with me

- 500 by 100 right guys so far with me now here you'll need a little bit of

now here you'll need a little bit of help on the from the question itself

help on the from the question itself they'll give you in the question itself

they'll give you in the question itself that if you talk about that I need some

that if you talk about that I need some data about the threshold this threshold

data about the threshold this threshold will be given from the data itself and

will be given from the data itself and it's given right that 5% of the

it's given right that 5% of the applicants

applicants are going to you

are going to you received a scholarship like you know if

received a scholarship like you know if you are having 96 percentile 97

you are having 96 percentile 97 percentile you get the scholarship right

percentile you get the scholarship right remember those who have written CAT exam

remember those who have written CAT exam or any exam which are having percentile

or any exam which are having percentile percentile means suppose you are at 99

percentile means suppose you are at 99 what does it mean it means that 99% of

what does it mean it means that 99% of the students are behind you right hello

the students are behind you right hello so similarly for 95%

so similarly for 95% this is your 95% the whole of it 95 and

this is your 95% the whole of it 95 and this small part is your 5% let's say

this small part is your 5% let's say this is your 5% % and this all is your

this is your 5% % and this all is your 95%. So those who are going to be in the

95%. So those who are going to be in the pink area they are going to get the

pink area they are going to get the scholarship. Hello guys so far with me.

scholarship. Hello guys so far with me. So the major chunk is 95 obviously

So the major chunk is 95 obviously majority doesn't get scholarship right

majority doesn't get scholarship right so far with me 5%. Now this 5% cut off

so far with me 5%. Now this 5% cut off will be given from the will be given by

will be given from the will be given by other question itself. Generally when

other question itself. Generally when I'll teach you confidence interval

I'll teach you confidence interval hypothesis testing right. So I will give

hypothesis testing right. So I will give you this don't worry about it that 95%

you this don't worry about it that 95% means you have a probability there and

means you have a probability there and that probability is going to get

that probability is going to get converted into a jet score and that jet

converted into a jet score and that jet score will be 1.65 for the time being

score will be 1.65 for the time being just remember it. So alpha 0.05 gives

just remember it. So alpha 0.05 gives you the alpha cutoff right and for this

you the alpha cutoff right and for this corresponding jet score is for now you

corresponding jet score is for now you just take it as a value later I'll

just take it as a value later I'll explain okay it is 1.65 65 right why is

explain okay it is 1.65 65 right why is this coming because this is your

this coming because this is your confidence interval cutff and this is

confidence interval cutff and this is where your the magic happens okay so for

where your the magic happens okay so for the time being I'm just saying again and

the time being I'm just saying again and again forget how I have reached to this

again forget how I have reached to this how they have given you this number

how they have given you this number later you will see so now you have the

later you will see so now you have the jet score cut off anybody who is having

jet score cut off anybody who is having higher than this value will get the

higher than this value will get the scholarship so 1.65 65 x - 500 by 100.

scholarship so 1.65 65 x - 500 by 100. If you calculate it will come to 665.

If you calculate it will come to 665. Hello. So anyone who scoring more than

Hello. So anyone who scoring more than 665 they'll get a scholarship. Hello

665 they'll get a scholarship. Hello guys are able to follow this question.

guys are able to follow this question. So you might be wondering that how do

So you might be wondering that how do you how do you know that? Because see if

you how do you know that? Because see if this is your cutff which is 1.65 65 then

this is your cutff which is 1.65 65 then as you increasing as go towards the

as you increasing as go towards the right this value will increase now right

right this value will increase now right buddy the question is how high does your

buddy the question is how high does your GS go when I say how high means how

GS go when I say how high means how minimum you have to maintain to get the

minimum you have to maintain to get the scholarship higher cutff they're not

scholarship higher cutff they're not asking they're asking the cutff right

asking they're asking the cutff right suppose the qualifying marks is 665

suppose the qualifying marks is 665 you'll get the scholarship clear now

you'll get the scholarship clear now I'll explain that guys how did you get

I'll explain that guys how did you get that 1.65 65. Why don't you reverse

that 1.65 65. Why don't you reverse engineering it? Where is 1.6 here? 1.65

engineering it? Where is 1.6 here? 1.65 will be here. 0.05.

will be here. 0.05. Can we go there and see what is this

Can we go there and see what is this value my dear? Come on. What is this

value my dear? Come on. What is this value? This is what 4550.

value? This is what 4550. Can you put into

Can you put into into two this probability? How much is

into two this probability? How much is that?

that? 91% 90%.

91% 90%. We need 95.

We need 95. So it is close but not 100% close. Why

So it is close but not 100% close. Why I'm missing that? Because we are looking

I'm missing that? Because we are looking at a different chart all 1.65.

at a different chart all 1.65. No, this is not correct. Something is

No, this is not correct. Something is fishy with the chart. There's another

fishy with the chart. There's another distribution guys. Let me show you

distribution guys. Let me show you another table. Ideally, I use this table

another table. Ideally, I use this table for getting this more better. So this is

for getting this more better. So this is like you know the problem is that this

like you know the problem is that this chart is that this chart can only be

chart is that this chart can only be used and that's why we not use when

used and that's why we not use when something we are measuring from zero

something we are measuring from zero right guys remember they're saying 0 to

right guys remember they're saying 0 to zed but if you are trying to do from

zed but if you are trying to do from minus infinity till here right this

minus infinity till here right this value so then you should use this minus

value so then you should use this minus infinity to zed right this is the most

infinity to zed right this is the most appropriate chart now you tell me where

appropriate chart now you tell me where is 1.65 so this is 1.6 and 1.65 65. Now

is 1.65 so this is 1.6 and 1.65 65. Now let's see. So this is a value and the

let's see. So this is a value and the the closer values are this. Now you are

the closer values are this. Now you are making making sense guys. So it is

making making sense guys. So it is coming to what? Either you take this

coming to what? Either you take this higher cutff 95.05

higher cutff 95.05 right? So there are no concepts issue

right? So there are no concepts issue guys. Don't get excited. It's just that

guys. Don't get excited. It's just that we looking at a different table that

we looking at a different table that there are two formats to the table. One

there are two formats to the table. One is from minus infinity to zed and the

is from minus infinity to zed and the other is from zero to z. But we need

other is from zero to z. But we need this time we needed what? from here or

this time we needed what? from here or from starting. I think from starting.

from starting. I think from starting. Now guys, let me talk about important

Now guys, let me talk about important things which is I feel uh the first

things which is I feel uh the first question I generally ask moving away

question I generally ask moving away from descriptive to in in inferential is

from descriptive to in in inferential is that what is the difference between

that what is the difference between descriptive and inferential statistics?

descriptive and inferential statistics? Anyone anyone want to take a shot at

Anyone anyone want to take a shot at this answering it? Inferential is

this answering it? Inferential is conclusive. Descriptive is unbiased

conclusive. Descriptive is unbiased measurement. Okay. But someone has

measurement. Okay. But someone has answered in terms of what we will do

answered in terms of what we will do with what. Right? But what is the key

with what. Right? But what is the key difference? Descriptive, you know, if

difference? Descriptive, you know, if you go with the literal meaning of it.

you go with the literal meaning of it. Describing your data. Now that data will

Describing your data. Now that data will be population or sample, that is another

be population or sample, that is another discussion, right? But if you are

discussion, right? But if you are summarizing and presenting the data in a

summarizing and presenting the data in a meaningful way to describe it without

meaningful way to describe it without taking or making or generalizing any

taking or making or generalizing any inferences based on the presented data.

inferences based on the presented data. So inferential is conclusive. So if I

So inferential is conclusive. So if I can give you some pointers guys. Okay.

can give you some pointers guys. Okay. So descriptive if I may write

So descriptive if I may write descriptive statistics I'll just write

descriptive statistics I'll just write stats you know involves

stats you know involves involves the summarization

involves the summarization and presentation of data in a meaningful

and presentation of data in a meaningful way. It's not like it's good for nothing

way. It's not like it's good for nothing just it describes the data you know. So

just it describes the data you know. So what is the primary goal here guys?

what is the primary goal here guys? Primary or goal or objective is to

Primary or goal or objective is to describe the data obviously with the

describe the data obviously with the help of mean you know dispersion

help of mean you know dispersion measures of central tendency

measures of central tendency leveraging

leveraging you know measures of central tendency

you know measures of central tendency what is that guys what are the measures

what is that guys what are the measures of central tendency mean median mode

of central tendency mean median mode right must have been thought measures of

right must have been thought measures of dispersion right spreadness variance

dispersion right spreadness variance standard deviation right measure of

standard deviation right measure of shape what is measures of shape Okay, I

shape what is measures of shape Okay, I hope you must have been taught skewness

hope you must have been taught skewness ktosis right etc. is used when you want

ktosis right etc. is used when you want to make prediction or generalization

to make prediction or generalization about a population based on a sample.

about a population based on a sample. Yes, that's the application of inferial

Yes, that's the application of inferial statistics, right? Why do we need

statistics, right? Why do we need inferial? That's the answer to that

inferial? That's the answer to that question. We are still understanding

question. We are still understanding what's the difference between what is

what's the difference between what is what. So guys definitely you all of the

what. So guys definitely you all of the answers which you have given is correct

answers which you have given is correct that descriptive is all about population

that descriptive is all about population inferial is all about sample but one

inferial is all about sample but one thing which is very much you have to

thing which is very much you have to very much clear about that that

very much clear about that that descriptive stats do not make any

descriptive stats do not make any inferences okay that's why you have a

inferences okay that's why you have a separate branch all together do not you

separate branch all together do not you know draw conclusions

know draw conclusions or inferences but it doesn't mean that

or inferences but it doesn't mean that you will not find any logical conclusion

you will not find any logical conclusion right to RN point it does not draw any

right to RN point it does not draw any conclusion or inferences beyond what is

conclusion or inferences beyond what is presented in data so basically means

presented in data so basically means what generalizing it what you see in the

what generalizing it what you see in the data you tell it right it's an unbiased

data you tell it right it's an unbiased approach then I'll go into inferential

approach then I'll go into inferential so this is what descriptive is I'll get

so this is what descriptive is I'll get into inferential

into inferential if you remember in school days in the

if you remember in school days in the English MCB main course book those who

English MCB main course book those who have done CBSC they would know I don't

have done CBSC they would know I don't know it's still there or not but I'm

know it's still there or not but I'm talking about more than a decade ago. So

talking about more than a decade ago. So when I was in class 9 10 right so there

when I was in class 9 10 right so there we had a book called MCD main course

we had a book called MCD main course book there they used to give some you

book there they used to give some you know five or six lines right in the

know five or six lines right in the paper in the exam and then they used to

paper in the exam and then they used to ask what is the what are the what is the

ask what is the what are the what is the inference of this you know these four or

inference of this you know these four or five lines right remember somebody

five lines right remember somebody remembers that and then they used to

remembers that and then they used to give five option four option one would

give five option four option one would be correct right so obviously uh read

be correct right so obviously uh read the those four or five lines right and

the those four or five lines right and trying to make some conclusion on

trying to make some conclusion on inferences about it that's what you must

inferences about it that's what you must have done in English but it's a similar

have done in English but it's a similar kind of situation for statistics as home

kind of situation for statistics as home so inferial statistics as the name

so inferial statistics as the name suggest you draw some inferences right

suggest you draw some inferences right the literal meaning of inferences is

the literal meaning of inferences is conclusion some generalization

conclusion some generalization so I'll also keep talking and write so

so I'll also keep talking and write so that I can save some time involve making

that I can save some time involve making inferences

inferences And once you have done the inferences

And once you have done the inferences with you, you draw some conclusion

with you, you draw some conclusion about a population

about a population based on sample data and that will be

based on sample data and that will be random sample. There are there are

random sample. There are there are numerous way of doing the random sample.

numerous way of doing the random sample. So what is the goal? The primary goal is

So what is the goal? The primary goal is to

the goal is to generalize findings or insights

insights from a sample to a larger population,

from a sample to a larger population, right? And then you also have some

right? And then you also have some techniques there

techniques there leveraging. What are the techniques?

leveraging. What are the techniques? Generally we use guys like as simple as

Generally we use guys like as simple as doing hypothesis testing, right? So you

doing hypothesis testing, right? So you actually form a hypothesis and you test

actually form a hypothesis and you test it to come to a conclusion you know

it to come to a conclusion you know using confidence interval concept right

using confidence interval concept right CI concept and you will love it the way

CI concept and you will love it the way I'll teach you I can promise as simple

I'll teach you I can promise as simple as running some tests like t test right

as running some tests like t test right ANOVA analysis of variance etc okay so

ANOVA analysis of variance etc okay so this is my introduction to the

this is my introduction to the difference between these two guys now at

difference between these two guys now at least we have come we have come to a

least we have come we have come to a point where we know that you know

point where we know that you know descriptive is all about generalizing

descriptive is all about generalizing the data sorry uh describing the data

the data sorry uh describing the data inferential is all about making

inferential is all about making inferences or generalization about a

inferences or generalization about a larger data set uh from a sample of

larger data set uh from a sample of those right that is clear if that is

those right that is clear if that is clear guys so then I'll give you a movie

clear guys so then I'll give you a movie recommendation and this is just one

recommendation and this is just one summary which I have already written I

summary which I have already written I don't want to write see so if I have to

don't want to write see so if I have to summarize it in In a nutshell,

summarize it in In a nutshell, descriptive talks about describing the

descriptive talks about describing the visible characteristics of a data which

visible characteristics of a data which is can be a population or sample. So not

is can be a population or sample. So not always be like stable. I only do

always be like stable. I only do population with you can also find

population with you can also find descriptive statistics for sample. Who

descriptive statistics for sample. Who is stopping you? Meanwhile, inferential

is stopping you? Meanwhile, inferential uh talks about inferences or

uh talks about inferences or generalization. Look at the the green

generalization. Look at the the green highlighter about a larger data set

highlighter about a larger data set based on sample of those and that sample

based on sample of those and that sample will be random. There are lot of samples

will be random. There are lot of samples stratified strategic random simple

stratified strategic random simple random there are a lot of things are

random there are a lot of things are there let me talk about sample versus

there let me talk about sample versus population okay so whenever we talk

population okay so whenever we talk about sampling you know so sampling is

about sampling you know so sampling is very very important for any inferial

very very important for any inferial statistics uh right any sort of test it

statistics uh right any sort of test it cannot test everybody okay so let's say

cannot test everybody okay so let's say you are looking at a population which is

you are looking at a population which is let's the country population India has

let's the country population India has how many people how many we are 1.5 4 or

how many people how many we are 1.5 4 or we are now 1.5 how is it 1.4 1.5 what is

we are now 1.5 how is it 1.4 1.5 what is it 1.6 huh okay so 160 kores so let's

it 1.6 huh okay so 160 kores so let's say if I try to measure something 160 cr

say if I try to measure something 160 cr people this will be quite a lot of data

people this will be quite a lot of data isn't it hello and quite a lot of you

isn't it hello and quite a lot of you are if you are referring to what is

are if you are referring to what is India's population in 2023 I think it is

India's population in 2023 I think it is saying 1.43

saying 1.43 or 1.4 44 not 1.6 so yeah it is we are

or 1.4 44 not 1.6 so yeah it is we are close to 1.4243 4243 anyway so doesn't

close to 1.4243 4243 anyway so doesn't matter for my example suppose I ask you

matter for my example suppose I ask you to go and talk to everybody and ask

to go and talk to everybody and ask their mathematics marks right and give

their mathematics marks right and give me the mean of maths for Indians who

me the mean of maths for Indians who have studied maths mean except those

have studied maths mean except those guys who are in let's say and grade wise

guys who are in let's say and grade wise so grade one average marks grade two

so grade one average marks grade two grade four do you think it is even

grade four do you think it is even possible to do that I think No, I'm

possible to do that I think No, I'm asking you to go and talk to everybody,

asking you to go and talk to everybody, right? Okay, let me take a recent

right? Okay, let me take a recent example. Election just concluded, right?

example. Election just concluded, right? You must have heard about exit poll. One

You must have heard about exit poll. One of the famous things happens on media.

of the famous things happens on media. Exit poll. Anybody has any idea exit

Exit poll. Anybody has any idea exit poll. Exit poll is like they do give you

poll. Exit poll is like they do give you know agencies. They do the polling and

know agencies. They do the polling and they do the random polling, random

they do the random polling, random sampling of the voters. They ask in a

sampling of the voters. They ask in a constituency that okay tell us what do

constituency that okay tell us what do you think? They don't go and ask whom

you think? They don't go and ask whom did you vote for? Was it party A and

did you vote for? Was it party A and I'll not take any party name. So let's

I'll not take any party name. So let's say there are three parties you know

say there are three parties you know party ABC whom did you vote for? They

party ABC whom did you vote for? They don't ask. They have a very amazing set

don't ask. They have a very amazing set of questions with they call questionaire

of questions with they call questionaire and they go and they do this survey. But

and they go and they do this survey. But do you think is it possible to let's say

do you think is it possible to let's say even let's say just uh so I was in indor

even let's say just uh so I was in indor so MP indor let's say we go to vijanagar

so MP indor let's say we go to vijanagar anybody from indor say hi. So let's say

anybody from indor say hi. So let's say and MP election just concluded. Suppose

and MP election just concluded. Suppose this is a this is a nice place in in MP

this is a this is a nice place in in MP Indor. So even for a vizagar population

Indor. So even for a vizagar population let's say the population is you know

let's say the population is you know maybe let's say 20 lakhs. Okay

maybe let's say 20 lakhs. Okay hypothetically I'm just making some

hypothetically I'm just making some numbers. Is it easy to go and talk to

numbers. Is it easy to go and talk to everybody who are 18 plus who can vote?

everybody who are 18 plus who can vote? We can vote 18 plus or 21 plus. My GK

We can vote 18 plus or 21 plus. My GK has gone. I think 18 plus is the voter.

has gone. I think 18 plus is the voter. So 18 plus guys uh can vote. You're not

So 18 plus guys uh can vote. You're not going to ask all the voters that whom do

going to ask all the voters that whom do you vote for, right? It's not possible.

you vote for, right? It's not possible. So what this agency do guys? Agency do

So what this agency do guys? Agency do the random sampling of voters. So what

the random sampling of voters. So what they do? They are going to do random

they do? They are going to do random sampling of voters. And if I'm correct,

sampling of voters. And if I'm correct, they also pay the one who participate in

they also pay the one who participate in it but they never accept it by the way.

it but they never accept it by the way. So agencies do random sampling of

So agencies do random sampling of voters. they'll do in Vizanagar and ask

voters. they'll do in Vizanagar and ask a set of question to gauge the sentiment

a set of question to gauge the sentiment right you know ask a set of question to

right you know ask a set of question to gauge the sentiment and tie it back tie

gauge the sentiment and tie it back tie back to to which party these guys voted

back to to which party these guys voted for right and then what happens they

for right and then what happens they prepare a nice summary and if you have

prepare a nice summary and if you have seen so you must have seen something

seen so you must have seen something like this I cannot show you who is

like this I cannot show you who is getting what but I can definitely show

getting what but I can definitely show show you how does it look like right

show you how does it look like right have you seen this hello so can you see

have you seen this hello so can you see if I take the first number 120 to 140

if I take the first number 120 to 140 let's say this is party A party B party

let's say this is party A party B party C party D whatever it seems that party A

C party D whatever it seems that party A has a clear majority in most of the

has a clear majority in most of the agencies so coming back to this example

agencies so coming back to this example so the you know politicizing this who is

so the you know politicizing this who is what in this list let's focus on our

what in this list let's focus on our concept so what is this 22 to 140 what

concept so what is this 22 to 140 what what do you mean by that? So these

what do you mean by that? So these agency what it what they do right let's

agency what it what they do right let's understand the process 120 to 140 right

understand the process 120 to 140 right uh is the range they are given isn't a

uh is the range they are given isn't a range that minimum party A will get 122

range that minimum party A will get 122 seats and maximum they are going to get

seats and maximum they are going to get 140 hello so far with me guys this is

140 hello so far with me guys this is what we call it as confidence interval

what we call it as confidence interval interesting so we have been seeing these

interesting so we have been seeing these exit polls for long this is known as

exit polls for long this is known as confidence interval and they

confidence interval and they confidence score also that we feel 95 we

confidence score also that we feel 95 we feel 95% chances that party A will get

feel 95% chances that party A will get these many seats. This is known as conf

these many seats. This is known as conf confidence interval estimate. It's an

confidence interval estimate. It's an estimate right because you are not

estimate right because you are not talking to everybody. You are talking to

talking to everybody. You are talking to a sample sample of people. You are

a sample sample of people. You are asking those questions right? You are

asking those questions right? You are asking those questions and you are

asking those questions and you are coming to a conclusion that okay party A

coming to a conclusion that okay party A will get this these many seats because

will get this these many seats because don't forget that you did some sampling

don't forget that you did some sampling here right this is how sampling looks

here right this is how sampling looks like you go to a constituency you will

like you go to a constituency you will you know pick pick and ask questions

you know pick pick and ask questions this is your sample right and then you

this is your sample right and then you come back to generalize something about

come back to generalize something about that population point estimate versus

that population point estimate versus confidence interval estimate point

confidence interval estimate point estimate would be like Okay, I will say

estimate would be like Okay, I will say party A will get 130 seats. Period. You

party A will get 130 seats. Period. You cannot exactly predict it 130.

cannot exactly predict it 130. Astrologer do that. I think party A will

Astrologer do that. I think party A will get 130 seats. It is a point estimate

get 130 seats. It is a point estimate one value single value that is like you

one value single value that is like you know getting the mean. Generally people

know getting the mean. Generally people say this will be the mean of the sample

say this will be the mean of the sample and then I am saying that this will

and then I am saying that this will become population. So mean of the

become population. So mean of the population I'm predicting. So on an

population I'm predicting. So on an average I feel party will get 130. But

average I feel party will get 130. But tell me getting 130 correct is tough or

tell me getting 130 correct is tough or not. Tell me if the party gets 132

not. Tell me if the party gets 132 you're wrong. If the party get 130 28

you're wrong. If the party get 130 28 you are wrong. So point estimate

you are wrong. So point estimate you cannot you know put a confidence to

you cannot you know put a confidence to it. So that's why statistically

it. So that's why statistically we generally go with confidence interval

we generally go with confidence interval concept. Now if I'm getting 132 or 128

concept. Now if I'm getting 132 or 128 still both the options are going to fall

still both the options are going to fall in here. Hello and your analysis is

in here. Hello and your analysis is still correct. That happens with

still correct. That happens with clinical trials in you know vaccines as

clinical trials in you know vaccines as well in the covid hit us. So a lot of

well in the covid hit us. So a lot of vaccine trials would have happened and

vaccine trials would have happened and I'll take that example. I talked about

I'll take that example. I talked about that case study that vaccine trials

that case study that vaccine trials right vaccine trials are not going to be

right vaccine trials are not going to be 100% accurate and when the you know

100% accurate and when the you know pandemic happens right it's a mahammari

pandemic happens right it's a mahammari it's very tough because you have you are

it's very tough because you have you are running against the clock right you need

running against the clock right you need to get a vaccine in the market so you

to get a vaccine in the market so you cannot be like waiting for 99.99%

cannot be like waiting for 99.99% accuracy right you will go with some

accuracy right you will go with some conf interval generally there they use

conf interval generally there they use 95 because it's related to health you

95 because it's related to health you cannot administer something to someone

cannot administer something to someone who are having some co-orbidities they

who are having some co-orbidities they they might die right rather than they

they might die right rather than they die with the virus they die with because

die with the virus they die with because of your medicine. So you have to be very

of your medicine. So you have to be very careful but they do all those clinical

careful but they do all those clinical trials to come to the range of efficacy.

trials to come to the range of efficacy. So that's what is the efficacy of the

So that's what is the efficacy of the medicine how effective it is you know

medicine how effective it is you know are you it's a score. So that score if

are you it's a score. So that score if it comes in a threshold you go and

it comes in a threshold you go and release the vaccine in the market hello

release the vaccine in the market hello and then you put in the batch

and then you put in the batch production. So that's the difference

production. So that's the difference between the point estimate and the

between the point estimate and the confidence interval range. I I'll

confidence interval range. I I'll deliberate more on it. But point

deliberate more on it. But point estimate does not provide a room of

estimate does not provide a room of error. There's no margin of error. 130

error. There's no margin of error. 130 means 130. But confidence interval does

means 130. But confidence interval does provide margin of error. You can make

provide margin of error. You can make mistake and that you can also set it up.

mistake and that you can also set it up. Suppose you say 80% confident interval

Suppose you say 80% confident interval then your range will be like 100 to 140

then your range will be like 100 to 140 or 100 to 160. Are you feeling guys?

or 100 to 160. Are you feeling guys? Then you will say that what is this? You

Then you will say that what is this? You cannot have this much of range right 100

cannot have this much of range right 100 to 160 and then some party will uh you

to 160 and then some party will uh you know the majority is like only 60 let's

know the majority is like only 60 let's say majority to form a government is 70

say majority to form a government is 70 and you are putting a range of 100 to

and you are putting a range of 100 to 200 error cannot be the majority right

200 error cannot be the majority right are you getting my point what I'm saying

are you getting my point what I'm saying let's say to form the government the

let's say to form the government the majority is like 70 you are putting the

majority is like 70 you are putting the range of 100 that the gap is 100 so

range of 100 that the gap is 100 so that's not a

that's not a uh study so that's where range how much

uh study so that's where range how much we should Keep a percentage 95 99 90.

we should Keep a percentage 95 99 90. Generally these are the confidence

Generally these are the confidence interval levels 95 90 uh 99 99 but that

interval levels 95 90 uh 99 99 but that in any exit poll they follow a very good

in any exit poll they follow a very good uh set of approaches to run to come a

uh set of approaches to run to come a number which is your confident interval

number which is your confident interval that's what they show in the TV which

that's what they show in the TV which you are seeing right and they also claim

you are seeing right and they also claim that my analysis depends on blah blah

that my analysis depends on blah blah blah thing right that's why if you see

blah thing right that's why if you see some of the guys are like like rockstar

some of the guys are like like rockstar can you see these guys I am telling

can you see these guys I am telling you'll get exactly This is like ED they

you'll get exactly This is like ED they must be new. So definitely these are the

must be new. So definitely these are the top players right? C voter I have been

top players right? C voter I have been seeing since my childhood days. But this

seeing since my childhood days. But this is like new nation whatever. Can you see

is like new nation whatever. Can you see that they are giving the point estimate?

Just a quick info guys. Intellipath offers a data science course in

offers a data science course in collaboration with iHub IIT R key which

collaboration with iHub IIT R key which will help you master concepts like