Engineering Python · NumPy & pandas Basics
Simple Transforms
In Engineering Python because feature columns are created and aggregated in pandas before they reach an ML estimator.
Assign new columns from expressions; use light `map` / elementwise ops; aggregate with one clear `groupby(...).mean()` pattern.
- Engineering Python
- Medium level
- 4 concepts
1Assigning a column
`df['z'] = df['x'] * 2` adds or replaces column `z`. Vectorised expressions beat row-wise Python loops.
Figure. df['z'] = df['x'] * 2 writes a new column from x = [1, 2, 3]. The z values are 2, 4, 6 — one product per row, no Python loop.
Assign
import pandas as pd
df = pd.DataFrame({'x': [1, 2, 3]})
df['z'] = df['x'] * 2
print(df)What does `df['z'] = df['x'] + 1` do?
- Creates/updates column z from column x
- Deletes x
- Trains a model
Column assignment is vectorised.
2Light map / elementwise
`.map` on a Series applies a function per element. Prefer arithmetic on columns when it expresses the idea directly.
Figure. .map applies one function to each Series element. Prefer a column expression when the idea is already arithmetic.
Takeaway
- Idea`.map` on a Series applies a function per element. Prefer arithmetic on columns when it expresses the idea directly.
3One groupby mean pattern
`df.groupby('site')['temp'].mean()` averages temp per site. Remember this one pattern; fancy aggregations wait for later courses.
Figure. site A has temps 20 and 22, so the group mean is 21. Site B has only 18, so its mean is 18. That is groupby('site')['temp'].mean().
Takeaway
- Idea`df.groupby('site')['temp'].mean()` averages temp per site. Remember this one pattern; fancy aggregations wait for later courses.
groupby mean
import pandas as pd
df = pd.DataFrame(
{'site': ['A', 'A', 'B'], 'temp': [20, 22, 18]}
)
print(df.groupby('site')['temp'].mean())What does df.groupby("city")["rent"].mean() compute in pandas?
- The total count of all flats across the entire dataset
- The median city population sorted in descending order
- A new DataFrame with city names replaced by integer IDs
- The average rent calculated separately for each distinct city group
groupby("city") groups rows by city value, and ["rent"].mean() aggregates the mean rent per group.
4Lab: column + groupby
Add a column and print a group mean.
Figure. double is temp × 2: 40, 44, 36. groupby('site')['temp'].mean() still averages the temps: site A is (20+22)/2 = 21.
Takeaway
- IdeaAdd a column and print a group mean.
Coding lab. Transform and groupby runs in the app, with checks on your output.
Notes
- In Engineering Python because feature columns are created and aggregated in pandas before they reach an ML estimator.
- `df['z'] = df['x'] * 2` adds or replaces column `z`. Vectorised expressions beat row-wise Python loops.
- `.map` on a Series applies a function per element. Prefer arithmetic on columns when it expresses the idea directly.
Exam traps & shortcuts
- Run one cell at a time and read stdout before changing more lines.
- Names are labels for values; rebinding a name does not rewrite old prints.
Recap
Assign columns vectorised; map lightly; remember groupby mean.
- Assigning a column
- `df['z'] = df['x'] * 2` adds or replaces column `z`. Vectorised expressions beat row-wise Python loops.
- Light map / elementwise
- `.map` on a Series applies a function per element. Prefer arithmetic on columns when it expresses the idea directly.
- One groupby mean pattern
- `df.groupby('site')['temp'].mean()` averages temp per site. Remember this one pattern; fancy aggregations wait for later courses.
- Lab: column + groupby
- Add a column and print a group mean.
Practise Simple Transforms
Reading is free and needs no account. Practice, mocks and progress live in the app.
- A 2-question practice set that ends the chapter
- 2 quick checks with worked explanations
- Timed mocks scored with the real marking scheme
- Readiness tracked per topic, kept on your device