Hello, and welcome to a module on bivariate regression in Python. First, let's start by loading in some
data. Let's try out the ADD.xlsx data this time, just to shake things up.
First, we'll be importing pandas as pd, drive from google.colab,
sm from statsmodels.api. We're importing this today to do our regression,
and we'll also be importing matplotlib.pyplot so we can do some visualizations of our regression as well.
Next, we'll mount our drive, set our file path, and remember we're using a different data set. So make sure your
file path is set to something different than what it usually is. And make sure to save the file in your Google Drive
so that it is available for you to use through your file path that I will read in your file path.
And let's print the head of our data.
So here we have some variables in our data set. The ones we're the most concerned with (we'll go over a couple of them). And we've
got gender here, which is pretty self-explanatory, repeat, which is if they have repeated a
grade, IQ scores, GPA, what I think stands for social problems
and drop out, which means if they have dropped out. And finally ADDSC,
which is ADD score. So this is a score that a student received on
an ADD or ADHD test. So what
is bivariate regression? Bivariate means two variables, where one is the independent
variable, the predictor. And one is the dependent variable, the outcome. We regress the two
variables when we want to do things like make prediction. For example, if we had a data frame with number
of hours studied and exam scores, we could make predictions about what a score would be given the number of hours study.
We can also use it to understand relationships. Bivariate regression in the above example, would help answer:
Does studying more actually lead to higher exam scores? The third way we can use it is to quantify impact.
The regression equation here tells us how much the dependent variable exam score changes
for each unit increase in the independent variable study hours. So let's take a look at
the variables in our data. What if we would like to predict GPA based on ADDSC? In other
words, our X variable independent will be ADDSC and our Y variable dependent
will be GPA. In this data set, the ADDSC scores
are ADHD test scores and GPA is for grades. So keep that in mind as go through
our test. So the first thing
we'll want to do in our code is set X and Y equal to a variable here. And this
just makes it easier for us to read the code later, rather than having all of our data frame
tags all over the place. So X is the independent variable. Y is
the dependent variable. Next we're going to add a constant for the intercept.
You have to do this manually in Python. The constant is a number that exists on the line of best fit. It is where
X is equal to zero. This line will add that content to our x values
and save it in a new variable called x_const, which stands for X constant.
Next we'll actually run our model. In this case we're using sm which is stats models that we imported
above. .OLS and in the parentheses our arguments are going to
be Y and the x constant that we created just a few moments
ago. Next we'll set our mod will use model.fit to actually fit the model to our data and save
it in a variable called results and then we'll print those results in the summary. Let's go ahead
and do it. Okay so this is a lot of numbers and we're not going to go
over every single number that's here but I will highlight a few of them for you so that you have an idea of what these numbers are trying to tell
you. So in our example summary output we can see that P, the P values less than 0.001. This is P right
here. It's 0.000 which gives us strong evidence that we can reject the null hypothesis.
This suggests that there is a relationship between ADDSC and GPA. If we want to know the strength of that
relationship we should look at the effect size or R-squared value. R-squared value is right
here. This top right number in this case R-squared
is 0.378 which means 37.8% of the variance of GPA is explained by ADDSC. So let's
talk about a few more of these numbers.
Next up we have the intercept. The intercept is this number right here.
It's the constant coefficient. That's where x is equal to 0. The intercept in
linear regression is the predicted value of y when x is 0. It's where the line crosses the x-axis. Kind of like
the starting point, the prediction. For example, if you're predicting someone's salary based on experience, the
intercept would be the predicted salary for someone with zero years of experience.
The standard error of the estimate tells us how much the actual data points differ from the predicted values on the regression
line. It's like saying, on average, our predictions are off by this much. Standard error is right here.
Unstandardized regression coefficient. And unstandardized regression coefficient tells you how much why changes
when x increases by one unit. It uses the original units of the variables, like dollars, hours,
or inches, so you can directly see the real world act.
So this is our unstandardized regression coefficient. And we can see that this indicates that there is a negative
relationship between ADD scores and GPA. We'll check out that a little bit more later.
P value. This is when you probably already know. It helps you measure the strength of the evidence against the null hypothesis. And we've
already pointed out where the P value is in our data here. R squared tells you how
much of variation y, the outcome is explained by x predictor. It's a number between 0 and 1. Ploser
to 1 means the model fits the data better. If r squared is 0.8, it means that 80% of the changes in
y can be explained by the model. And again, our r squared value is right here.
Beta shows how strongly each variable affects the outcome, but it uses standardized units. So you can
compare the importance of different predictors, even if they're measured differently. If one variable has a beta 0.7 and
another has a 0.2, the first one has a bigger impact on the outcome. Beta will not be produced automatically
by your code. So it will not appear in your results summary.
Okay, we've talked a little about what these variables mean, but let's take a look at what
these variables are, what sorry, not what the variables mean. We've talked about what these results mean, but let's take a
look at what these variables look like when they're actually produced on a scatterplot.
So we've done a scatterplot before previously. I won't be going over this code too much as a result, but you
can see that it's pretty much the same as what we've done before. Except now, we have a Y prediction using
the X constant, and then we're plotting X the Y predictor.
The blue and blue we're going to make it red, have a line at best fit.
Okay, and we can see that there is a negative relationship between these two variables,
meaning as ADDSC scores go up, GPA goes down.
This is probably not surprising. If you're familiar
with the literature on ADHD,
we can also check this by doing a correlation of ADDSC and GA. And we can indeed see
that GPA goes down as ADDSC goes up.
Okay, this should be all you need to get started on your
homework. Please let us know if there's anything that we do to help you understand better. If we have
tutors this semester, we've talked to tutors and if we don't have tutors, then we've talked to you on structure for help.
All right, have a great day and have fun learning python!