11 NumPy essentials
Any changes you make to the code on this page, including your solutions to exercises, are temporary. If you leave or reload the page, all changes will be lost. If you would like to keep your work, copy and paste your code into a separate file or editor where it can be saved permanently.
Before we dive into Pandas, let us take a brief look at NumPy, the library for numerical computing that Pandas is built on. Understanding its basics – arrays, data types, and vectorized operations will make many Pandas concepts feel natural.
As usual, we first import the library (using its conventional alias np):
11.1 Arrays
The central data structure in NumPy is the array: an ordered collection of elements that, unlike a Python list, all have the same data type (the dtype). This restriction is what makes NumPy fast: the elements can be stored compactly in memory and processed by optimized machine code.
We can create an array from a list:
Other useful functions for creating arrays include np.arange (like range), np.linspace (evenly spaced values between two endpoints), and np.zeros/np.ones:
Indexing and slicing work as for lists:
11.2 Vectorized operations
The real power of NumPy is that arithmetic operations work on entire arrays at once – no loops needed. This is called vectorization:
NumPy also provides mathematical functions that are applied element-wise:
Vectorized operations are not only convenient to write, they are also much faster than Python loops – often by a factor of 10–100 for large arrays. Pandas inherits this behavior: when we later write df["weight"] / df["height"] ** 2, an expression involving entire columns, it is NumPy doing the work under the hood.
11.3 Boolean arrays
Comparison operators are vectorized too, producing arrays of True/False values:
Since True counts as 1 and False as 0, summing a boolean array counts the True values:
We can also use a boolean array to select elements. This is called boolean indexing (or masking), and it is the foundation of row filtering in Pandas:
11.4 Aggregations
Arrays have methods for computing summary statistics:
11.5 Missing values
NumPy provides a special floating-point value np.nan (“not a number”) to represent missing data. Note that any calculation involving nan results in nan:
Pandas uses np.nan to mark missing values, and we will see how to handle them properly in the chapter on data cleaning.
Exercise. The array below contains daily maximum temperatures in degrees Fahrenheit. Convert them to degrees Celsius using the formula \(C = (F - 32) \cdot 5/9\), and count how many days had a maximum temperature below freezing (0 °C).
Sample solution
temps_f = np.array([28.4, 35.6, 30.2, 41.0, 25.7, 33.8, 39.2, 27.5])
temps_c = (temps_f - 32) * 5 / 9
print(temps_c)
print("Days below freezing:", (temps_c < 0).sum())11.6 Summary
import numpy as np
np.array([1, 2, 3]) # array from a list (single dtype!)
np.arange(1, 10) # like range
np.linspace(0, 1, 5) # evenly spaced values
np.zeros(n), np.ones(n)
arr * 2, arr + arr # vectorized arithmetic (no loops)
np.sqrt(arr), np.log(arr) # element-wise functions
arr > 0 # boolean array
(arr > 0).sum() # count matching elements
arr[arr > 0] # boolean indexing (masking)
arr.min(), arr.max(), arr.mean(), arr.sum()
np.nan # missing value marker