Indexing, Selection, and GroupBy
Explore precise data selection methods with pandas using loc for label-based and iloc for position-based indexing. Learn to apply boolean masks for filtering data like SQL WHERE clauses. Understand how groupby combined with aggregation functions summarizes data by groups, changing the data's granularity. This lesson helps you translate SQL queries into effective pandas operations for data analysis and reporting.
For each driver region, what is the average delivery time and how many late deliveries occurred? A SQL query would first keep the delivery rows that matter, then group those rows by region, and finally calculate summaries. In pandas, we make the same decisions with explicit row and column selection, a boolean filter, and groupby().agg().
Consider a delivery table whose index contains delivery_id values such as 1001, 1002, and 1003. The index is not merely a row counter here. It identifies a specific delivery, so we can use it to retrieve a precise range of records before building a report.
Two pandas indexers handle this job. Label-based indexing selects rows and columns by their names or index labels through loc. Position-based indexing selects them by their zero-based locations through iloc. Both can produce a smaller DataFrame, but they answer different questions. loc asks for delivery identifiers such as 1002 through 1004. iloc asks for the first three displayed rows.
Selecting rows and columns precisely
When delivery_id identifies each row, loc makes the identifier visible in the selection. For example, df.loc[1002:1004, ['region', 'minutes']] requests the deliveries labeled 1002, 1003, and 1004, with only the region and minutes columns.
iloc ignores delivery identifiers and counts positions instead. The expression df.iloc[0:3, 0:2] selects row positions ...