0:03 okay so the bulk of this lesson has come
0:06 from our live thing when we actually ran
0:08 our experiments whether that was in
0:10 class or I'm recording this right now in
0:12 2020 so whether that ends up being
0:14 online for the coming school year but
0:16 we've already done some sort of
0:19 experiments based on our pulse rates and
0:21 I am recording this video before we've
0:22 actually done the experiments so I don't
0:24 know exactly how things played out right
0:26 here so I'm gonna talk in kind of a
0:28 generic sense about one type of
0:30 experiment I've done before but it
0:32 should still be helpful to hear how this
0:34 is organized even if we did something
0:36 slightly differently if you're like this
0:37 isn't helpful you're talking about a
0:39 different thing that we actually did in
0:41 class skip to the ends where I talk
0:43 about statistical statistical
0:44 significance because that's the most
0:47 important part of this lesson so
0:49 generally in class I don't do the
0:51 caffeine problem it's um kind of a
0:53 hassle to get it all set up and get the
0:56 caffeine poured and the grocery store to
0:57 get the caffeine and kids sometimes
0:59 aren't comfortable drinking soda in
1:01 class so it's just I found that it's
1:03 better to do other things to measure
1:06 pulse rate instead so one thing that
1:07 I've done which is what I'm gonna run
1:10 with in this example is I can look at
1:13 whether physical exercise increases
1:15 pulse rates would increase his pulse
1:17 rate which seems obvious that it would
1:18 but it's still kind of nice to do
1:20 something and to see results that
1:23 actually are significant so let's say
1:24 for my problem right here what I'm gonna
1:26 run with again we may have done
1:27 something differently in class I don't
1:30 know yet I can't see into the future but
1:33 what I'll generally do is something with
1:36 exercise so I'm gonna do three groups
1:37 here to be real fancy with it we're
1:41 gonna have no exercise at all we're
1:45 gonna have a group that does 60 seconds
1:49 of jumping jacks and we're gonna have a
1:53 group that does 60 seconds of push-ups
1:55 and then we're gonna see between those
1:57 groups if there is a difference in the
2:00 pulse rates before and afterwards so
2:02 let's talk about I'm not gonna have you
2:04 necessarily write these but think about
2:06 how we can make sure each of these four
2:08 principles of experimental design is
2:10 addressed within the context of this
2:13 study so think about that for a second
2:14 then I'll talk my way through it right
2:18 now comparison I've got three separate
2:20 groups and I'm going to compare my pulse
2:21 rates between the three groups that's
2:24 great I've done that and I need to do
2:26 that because if I didn't I wouldn't know
2:29 if the change I saw was because of the
2:32 exercise or because of something else
2:35 going on like my lesson etc etc so I've
2:38 talked about before on having different
2:40 groups that have all gone through the
2:42 same stuff is important I can't just
2:45 have everybody do the same exercise the
2:46 next thing I would need to do is have
2:48 random assignments so in my class
2:50 hypothetical class of 18 kids right here
2:54 I would randomly assign six kids to each
2:56 that's an arrow six kids to each of
2:59 those three groups and I would make sure
3:02 I did it randomly so like all the kids
3:04 who are in the same sports don't end up
3:06 in the same group and they're more fit
3:08 than everybody else and stuff like that
3:11 so by randomly assigning even amounts to
3:14 each of my groups I'm ensuring that my
3:16 groups aren't inherently unbalanced
3:20 which prevents confounding from Krank
3:23 control so with control I want to keep
3:24 everything else the same
3:27 so one way of doing that the other two
3:28 groups with jumping jacks and push-ups
3:30 are like getting up out of their chairs
3:32 and doing things I should probably have
3:35 my no exercise kids get up and stand up
3:36 anyway so it's not like they're still
3:38 sitting in a chair chillin they're still
3:41 doing something umm if I just had them
3:42 sitting there that I wouldn't know maybe
3:44 just by simply getting up there is a
3:47 difference being cause I'm recording
3:50 this video summer of 2020 so I still
3:52 don't know yet in 2020 if my kids are
3:54 going to be with me in class when we do
3:55 this lesson or if we're gonna do this
3:59 virtually so is experiments if we're
4:01 doing it from our own homes virtually
4:03 would be terrible in terms of control
4:05 because there are so many things that
4:06 are different in your house which is
4:09 your house versus your house so if we
4:10 are doing this virtually my experiments
4:12 had no control at all even if we're in
4:15 class I want to make sure like I said my
4:19 one group gets to like still stand up so
4:21 that's going on on I shouldn't have one
4:23 group in like hotter
4:24 part of the room or out the hallway
4:26 where the temperature might be different
4:29 etc etc etc make everything the same
4:29 that I can
4:32 and finally replication six people per
4:34 group is not great not making any
4:36 scientific journals with this amount of
4:38 data right here but I don't really have
4:41 control over that and my problem I can't
4:43 influence the number of people I have so
4:48 my replication is not ideal going in all
4:52 right so those are kind of how I am
4:54 addressing we're not addressing my
4:56 experimental design and I could make a
4:58 little diagram again but I've done that
4:59 before I don't really want to have you
5:02 guys write one of these whole deals the
5:04 idea is again I would take my 18 people
5:07 and I would randomly assign six of them
5:10 to each treatment so random assignment
5:14 you have six here six here 6 here I said
5:15 I didn't want to do it but it looks like
5:18 I'm kind of doing it anyway these people
5:20 can get treatment one which is no
5:23 exercise these people can get treated
5:27 two which is the jumping jacks and then
5:30 these people who get treatment 3 which
5:33 is push-ups it doesn't matter which I
5:35 call each of these because people were
5:36 randomly assigned so as long as I don't
5:38 like look at the people on the groups
5:40 and then decide what they get it's ok
5:43 these are in order but after I actually
5:46 do the exercise then what I'm gonna do
5:49 with each of the groups is I am going to
5:52 measure the pulses and then I'm going to
5:55 compare my treatment groups see if
5:57 there's a difference between the push-up
6:01 group and the jumping-jack group and the
6:04 no treatment group and that's basically
6:08 how my experiments gonna run so um
6:10 generally when I do these in class it
6:11 works a little better with exercise than
6:13 it used to with caffeine with caffeine I
6:15 would rarely see a difference between
6:16 the groups probably cuz I wasn't giving
6:18 enough caffeine and I wasn't doing a
6:19 good enough job controlling it's
6:22 actually very hard to execute a good
6:24 experiments where other things don't
6:26 come into play there's always things
6:28 that pop up that well that kind of mess
6:30 with my results so it takes a lot of
6:32 deliberate thoughts or and I put into
6:35 this to get good results from experiments
6:36 experiments
6:38 but let's do a hypothetical here so
6:40 again I don't know what happens when you
6:42 actually did the experiment with me but
6:46 just like before I'm gonna have a graph
6:51 or the change in pulse rates and I'm
6:54 gonna have 0 here at the middle so I
6:56 have my dot plots kind of stacked on top
6:57 of each other so you can see what's
6:59 going on we're gonna have our no
7:01 exercise we're gonna have our jumping
7:04 jacks and we're gonna have our push ups
7:08 let's say my no group hypothetically
7:12 looked like this so those are my six
7:13 people right there
7:15 and then let's say my jumping jacks
7:20 group hypothetically it looks like this
7:26 and then my push up group looks like
7:30 this just making stuff up right here in
7:33 this situation if I were to look at
7:35 these results what you want to look for
7:38 is little over that if you have lots of
7:40 overlap between your groups it's not
7:42 really convincing that there's a
7:45 difference between this and this if you
7:46 see lots of separation between the
7:48 groups that does suggest that maybe
7:51 there's a difference so if I were to
7:54 look at no exercise versus jumping jacks
7:56 fair amount of separation right there
7:58 not too much overlap same with no
8:02 exercise versus push-ups so in my
8:07 hypothetical right here I would say no
8:12 vs. either jumping jacks or push-ups
8:20 there is little overlap and little
8:25 overlap means there probably could be a
8:32 difference meanwhile if you look at
8:36 jumping jacks verse is the push-ups
8:39 jumping jacks versus push-ups have a lot
8:40 of overlap in them yes these people
8:42 right here are higher than anybody in
8:43 the push-up group but there's an awful
8:46 lot of overlap between these so this has
8:49 lots of overlap
8:53 and in general when there's lots of
8:57 overlap we're not convinced of a
9:01 difference so that is what you want to
9:02 look for in a nutshell when you're doing
9:04 these is there lots of overlap or is
9:06 there nots and this is so much more
9:08 complicated like they're not complicated
9:10 but complex then I'm making it up to me
9:13 right now we will talk a lot a lot a lot
9:16 this year about deciding whether results
9:18 are statistically significant this is
9:19 like our introductory chapter just kind
9:21 of dip our toes in the water aspect of
9:23 this look to see if there's separation
9:25 between your roots or not we will
9:27 revisit this many times over the course
9:32 of our time together in AP stats so
9:33 based on the results of the experiment
9:35 is there evidence that the one of the
9:37 groups has increased pulse rates and
9:39 then I'm gonna say again I'm just going
9:43 to write down a reminder to look for
9:47 overlap we want to see if there is
9:51 overlap or not and then assuming there
9:53 is a difference between the two groups
9:56 or the only three groups here what are
9:58 the possible explanations this is
10:00 getting at some of that more complicated
10:03 stuff that we will address a lot more as
10:06 class progresses here but anytime you
10:07 see a difference there are two
10:11 possibilities option one is there
10:20 actually is a difference caused by the
10:26 treatments so it's possible that I saw a
10:29 difference between jumping jacks and no
10:31 exercise because the jumping jacks
10:34 actually did cause a difference the
10:35 second option is sometimes a little
10:38 harder for kids to understand there is
10:42 no difference so there's no real
10:51 difference and the results occurred by
10:55 chance my face is in the way there sorry
11:02 so James um technically the chance is
11:03 due to rain
11:07 assignments you know by chance due to
11:13 random assignments so let me explain
11:15 what that means and I don't want to do a
11:17 super ton with this right now because we
11:18 will hit this again later in future
11:22 chapters a few true lessons but let's
11:24 say I had my no group that was like this
11:28 and I had my jumping jacks group it was
11:29 like this there's a lot of separation
11:32 there so this would lead me to believe
11:35 wow all these people did better than all
11:37 these people that happened probably
11:39 because the jumping jacks made a difference
11:40 difference
11:43 it's especially appearance if you have
11:45 only a little bit of data so let's say I
11:47 took the same problem but I had four
11:50 people in my study two per group so I
11:52 had these two kids right here
11:57 four actually let me close to zero so I
11:59 got my two people right here for this
12:01 group and I got my two people right here
12:05 for this group okay when you have too
12:08 little data the chances are like okay
12:10 maybe this person right here their pulse
12:12 rate was gonna go up either way no
12:14 matter what group they got putting in
12:15 they just happen to get put in this
12:18 group by random chance if they have got
12:19 put in the no root they still would have
12:21 increased the same amount because that
12:24 was just destined for happen anyway so
12:26 when you have too little data you can
12:27 make the arguments that the results
12:29 you're seeing are because of where
12:30 people were randomly assigned and not
12:33 because of the treatment itself that is
12:35 largely fixed by having enough data and
12:37 seeing enough separation and what's
12:38 going on but that's the basic idea going
12:43 on right there all right so the last
12:46 thing we need to talk about how can we
12:48 determine if the evidence is convincing
12:52 the evidence is considered convincing if
12:54 we can replicate it if we can do it
12:57 again and again again so we can do this
13:01 through replication or in other words
13:07 repeated trials so that is what it takes
13:09 to be able to decide that results are
13:11 convincing you do it again and see if
13:13 the same thing happens you can actually
13:15 conduct the experiments again or you can
13:16 use something called a simulation
13:18 you saw on the first day of school and
13:20 we will practice this a super time
13:22 between now and then
13:26 now results less than here results are
13:30 considered statistically significant if
13:36 they are unlikely to occur due to chance
13:39 so results are statistically significant
13:41 important vocab concept we will talk
13:43 about a lot in this class their
13:46 significance if they are unlikely to
13:49 occur by chance so if I were to deal out
13:52 my groups again and not reproduce my
13:55 results right here it may be that the
13:57 results I saw the first time were just
13:59 you the chance of how things were the
14:01 sign so you have to be able to repeat
14:03 the process and do it again to see if
14:06 it's due to chance or if it's actually
