Hello, and welcome to a video on sub-setting in Python. What is sub-setting?
Sub-setting is the process of selecting a smaller part of a larger data set based on specific criteria.
Think of it like picking out only the red apples from the big basket of mixed fruit. In data analysis,
sub-setting allows us to focus on specific rows or columns that meet certain conditions, making it easier to
analyze relevant information. For example, if you have a data set of students test scores, you might subset
the data to look at only students who scored above 80. Let's do what we usually do and load in some data.
So we're going to import pandas as pd and from google.colab. We're going to import drive.
So we're going to mount our drive, set our file path, read in our file path into a data frame, and make some
variables. So let's go ahead and do that.
So now we have some variables to work with. Let's use that example of student test scores and
pretend IV2 is a column of test scores. What if we only wanted the data where IV2 was over 80
to do that? We're going to set a variable called scores over 80 and we'll put a data frame into
it. So this is letting Python know I want to make a new data frame. Based on this data
frame where DF2 IV2 is greater than 80. And we're going to
call that new data frame scores over 80.
Okay, so now we have a 124 rows.
And we can see that our original 170 rows now we only have
125. Because remember Python begins counting at zero. What if
we want to select only certain columns? Let's say we're only interested in IV2, RIV1 and
RIV2, where the score of IV2 is over 80. To do
that, we do the same thing as before. Except now we're going to place a comma after the 80.
I'm going to tell it, okay, but we only want these three columns. And
we'll select them using brackets. We'll call this data set
subset. Okay, that's much more manageable.
In pandas, dot-loc is a way to access or filter data based on labels, row and column names.
It is particularly useful when selecting rows and columns based on labels rather than numerical positions.
So you can see we've used dot-loc up here to select labels
and a position.
Okay, what if we have two conditions we'd like to meet? Let's say we want IV2 over 80
and RIV1 less than 9. But we still only want the columns IV2, RIV1 and RIV2.
We can combine the last two techniques into one like so. So we're going to use DF dot-loc
again to make a data frame where IV2 is greater than 85. And
that's what that ampersand symbol is here. And RIV1, less than
9. And in addition to that, we only want the variables IV2, RIV1,
and RIV2. Now we have a much smaller version
of our data set that will actually print the entirety of in our window
here. But not all data is
continuously measured and may not fit with this pattern. What if we have categorical data that we'd like to subset?
Let's take a look at some sample data. So here we'll make some quick sample data. We'll have
name, major, and score, and we'll have some names, a couple different majors for these people,
and whatever scores they've gotten on, maybe it's a test that we've recently had.
So we're going to subset only the psychology students. And to do this, we're going to say
major if major is equal to psychology, put it in data frame.
And we're going to call that data frame psychology students, and then we'll print it out.
So now we can see that only Alice and Charlie are part of the psychology program in this group. What if we wanted both
the psychology and biology students to do that? Well, use .isin to
give Python a list and say anything in this list. We want to know if
they're in that major. So we're selecting major from it.
But in addition, saying .isin this list of things that we're passing through.
And we'll call those science students. So now we have Alice, Bob,
Charlie, and Eve in our subsets. Alright, that's pretty much all the information
you need about subsets to do the homework. So have fun coding.