Pandas DataFrames

Pandas DataFrames

After this lesson you will be able to: create a DataFrame, inspect its structure, select rows and columns correctly with loc and iloc, and filter data by condition.

1. DataFrame and Series

A DataFrame is a table of data with rows and columns (like an Excel sheet); each column is a Series (a labeled sequence of values). Real data is usually loaded from a file: pd.read_csv("grades.csv").

import pandas as pd

df = pd.DataFrame({
    "name":  ["An", "Binh", "Chi", "Dung", "Em"],
    "major": ["IT", "IT", "Econ", "Econ", "IT"],
    "score": [8.5, 7.0, 9.0, 6.5, 7.5],
})
print(df.shape)     # (5, 3)  → 5 rows, 3 columns
print(df.head(2))   # first 2 rows
df.info()           # column names, data types, number of non-null values
print(df.describe())   # descriptive statistics for the numeric columns

Reading from a CSV file (the example below uses a string instead of a file so you can run it right away):

import io
csv = "name,score\nAn,8.5\nBinh,7\n"
small = pd.read_csv(io.StringIO(csv))
print(small.shape)   # (2, 2)

2. Selecting columns

df["score"]          # one column → a Series
df[["name", "score"]]  # several columns (two pairs of brackets) → a DataFrame
df[["score"]]        # one column, but wrapped in a list → still a DataFrame

The double brackets [[...]] mean "pass in a list of column names", so the result is always a DataFrame.

3. Selecting rows: loc and iloc

• loc selects by label (row/column name), and its slices include the end label.
• iloc selects by position (index number starting from 0), and its slices stop before the end position, like a list.

print(df.loc[0:2, "name"].tolist())    # ['An', 'Binh', 'Chi']   ← includes label 2
print(df.iloc[0:2]["name"].tolist())   # ['An', 'Binh']          ← stops before position 2

by_name = df.set_index("name")         # use the name column as the row labels
print(by_name.loc["Chi", "score"])     # 9.0   (by label "Chi")
print(by_name.iloc[0]["score"])        # 8.5   (by position 0, i.e. the row "An")

4. Filtering rows by condition

pass_list = df[df["score"] >= 7.5]
print(len(pass_list))                 # 3
print(pass_list["name"].tolist())     # ['An', 'Chi', 'Em']

# multiple conditions: use & (and), | (or), and put each condition in parentheses
print(df[(df["major"] == "IT") & (df["score"] > 7)]["name"].tolist())   # ['An', 'Em']

5. Adding columns, sorting, statistics

df["bonus"] = df["score"] + 0.5       # new column, computed row by row
print(df["score"].mean())             # 7.7
top = df.sort_values("score", ascending=False)
print(top.iloc[0]["name"])            # Chi  (highest score)

Common mistakes

• Confusing loc (label) with iloc (position), especially when the row labels are also numbers.
• Using and/or instead of &/| when filtering on multiple conditions.
• Forgetting that df["a"] is a Series while df[["a"]] is a DataFrame.

Key takeaways

• DataFrame = table; each column is a Series.
• Use df.shape, df.info() and df.describe() to get familiar with new data.
• loc is by label (end included), iloc is by position (end excluded).
• Filtering: df[condition]; for multiple conditions use &, | and parentheses.

Reference

McKinney, W. (2022). Python for Data Analysis: Data Wrangling with Pandas, NumPy, and Jupyter (3rd ed.). O'Reilly Media.