Hello and welcome to a module on calculating mean, standard deviation, and variance in Python. We'll also
be covering median and mode. These are commonly used to help us understand the shape of our data. First, let's
import some data to use. As usual, we'll be importing our libraries, our packages, up at the
top of our file here. We will be using statistics and numpy today to perform our calculations.
Next, we'll set our file path, read our file path in, save it to a data frame that
we'll call DF and print the head of DF so we can see what data we're working with.
As you can see, we've got the same class data that we've been using the whole time and today we'll be working with IV1.
So in this case, we're working with something called a sample. This means what we have is representative of a population,
but is not the entire population. There are two different packages we'll introduce today. The first is the
statistics package. So these are measures of central tendency,
mean, median and mode. Statistics.mean will find you the mean. Statistics.median
will find you the median. Statistics.mode will get you the mode.
In each of these cases, you just change out whatever variable it is you're trying to find for what
is inside these parentheses. And the next we have our
measures of dispersion, standard deviation and variance. First, the statistics package
that is statistics.stdev and then statistics.variance.
Now, if you had a variable that was a number and wasn't inside a data frame. So if you
had an array or a list of something like that in Python, or if you had
saved df, IV1 to another variable such as,
you know, var1 equals df,
IV1. Then you could actually just put
var1 in here and it would work just fine. I'm just directly referring
to the data frame because it is faster.
So these next lines will print out our answers. If we'd like to give something a title in an answer,
then we can do that with using f quotations, writing our title,
and then inside this quotations, we use curly brackets and we denote which variable we'd like to print out.
So in this case, let's go ahead and print out our mean, median, mode, standard deviation, and variance, and then
just for fun, I'll show you how to round these answers as well. You just add a dot 4f to get to
4 decimal places. If you wanted 3 decimal places, it would be dot 3f. If you wanted 2, it would be dot 2f,
and so on and so forth. So let's go ahead and run this. You
can see we've got our mean, median mode, which are the same in this data set. They won't always be the same.
Standard deviation of 18.53 variance of 343.41, and then this
is just our rounded to four decimal places of the answers. Now, never round your answers unless specifically
instructed to do so in the assignment. Generally, you just leave your answers exactly the way they come out.
So this statistics package just assumes we are working with a sample. So there's no extra work to do when using this package to find the standard
deviation and variance for sample data. The only two calculations that would be affected by this are standard deviation and variance.
Since for the sample calculation, we would divide by n-1 and in the population calculation, we simply
divide by n. But what if we are working with a population? There are times where this may come up.
If you end up working with big data, where data that is so large, it could reasonably be considered the entire population.
Or you may have data that has been collected, such as sales numbers that do represent the entire data set.
In that case, you could use NumPy. Now our mean and median aren't going to be affected by
this, but we'll go ahead and use NumPy to calculate them anyway. So we've got np.mean and np.median.
Find the mean and the median. NumPy lacks a mode function, unfortunately.
And now we'll calculate the standard deviation and variance for the population.
Remember NumPy assumes that you have a population. So it automatically calculate by dividing
by n instead of n minus 1. So for standard deviation, it's just STD for variance.
It's just VAR. And we're going to save those into two floats, called standard
deviation, underscore population, and var underscore population. We can
still calculate sample standard deviation and variance. But in order to do that, we have to add this argument
ddof equals 1 to let it know that we want to divide by 1 degree of freedom or n
minus 1 instead of n. Let's go ahead and print our results.
And as you can see, the standard deviation for the population is now slightly different at 18.47 whereas
before it was 18.53. Our variance is slightly different as well. For
population is 341.39 and for our sample is 343.41.
But you may recognize this the same sample deviation variance that we got before.
So as data sets get larger and larger, the differences between a sample calculation and a population calculation are smaller and
smaller until they are almost negligible. Our data set is small so the difference is more noticeable.
Remember in this class we are always working with samples. So no matter which method you choose to use to find
a measure of central tendency, make sure you are keeping that in mind. However, it is useful to know how to find population
standard deviation and variance. Here's a quick recap. The statistics package assumes you are working with
samples by default. NumPy assumes you are working with a population by default. NumPy
is still useful to learn as it is widely used in big data and other Python applications. You may choose either
method for your assignment but make sure you're doing sample calculations. Degrees of freedom equal to 1.