Broadcasting With NumPy09/27/2000
This month's challenge will be to look at one of the most obvious but often confusing aspects of NumPy operations: broadcasting. When you perform operations between multiarrays of differing number of dimensions, one or both of the arrays may be broadcast so the dimensions will match. In its simplest form, this feature is very straightforward, but its use in complex expressions can be difficult to master. This month we’ll take a look at this fundamental capability of NumPy and how to put it to use.
NumPy does not inherently support the mathematics of linear algebra. The traditional functionality of linear algebra was left instead to library functions. NumPy's implementers focused on the complex capabilities of N-dimensional objects (the multiarray) --broadcasting provides that support. This combination approach led to a very powerful and useful package that satisfies a large development community. But it also led to some confusing clashes in terminology.
In the NumPy documentation, the word "rank" refers to the number of dimensions of the multiarray. It has nothing to do with the condition of the object itself (e.g., the number of linearly independent rows / columns, etc). A column vector is rank one, a matrix is always rank two, and an N-dimensioned multiarray is always rank N. In NumPy, rank refers to the shape of the multiarray. This will be confusing for any serious math types reading along. In traditional linear algebra, rank refers to related numbers within a matrix column or row.
Also in the NumPy documentation, the word "axis" refers to a dimension. Thus a rank 5 multiarray has five dimensions that can be referred to as the first through fifth axes. The length of an axis is the same as the length of a dimension.
These definitions can be confusing, especially when many of the terms are overloaded uses of traditional mathematical notions. Stick with it, however; soon you will be slinging the slang of NumPy with the best of them.
Broadcasting refers to smaller ranked multiarrays being either replicated or extended in a natural way to work with higher ranked multiarrays. By default, all binary operations, both mathematical (addition, subtraction, multiplication and division) as well as logical (and, or, and xor), work element by element between two multiarrays. The first thing a new NumPy user learns is that for binary operations to complete, the number and size of all dimensions must match. Well, that is not the whole truth. There are in fact several cases for which this is not true -- all are instances of broadcasting.
Before we dig into some examples, remember that you need to import the Numeric (NumPy) package to follow along. All examples can be run in the interactive Python command window once NumPy is installed. After starting up Python, execute the following command (this only needs to be done once per session).
>>> from Numeric import *
If you don't get any errors, you are all set to proceed. If an exception was raised, it probably means NumPy was not installed correctly; see the NumPy documentation for help.
Multiarray and scalar operations
For our first look at broadcasting, let's try to add a scalar to a multiarray. Given a 2-by-2 array
a and a scalar
b, the result is shown.
>>> a = array([[1,2],[3, 4]]) >>> a array([[1, 2], [3, 4]]) >>> b=1 >>> a+b array([[2, 3], [4, 5]])
The result is as expected. But wait! The rank of the two operands did not agree? The scalar was broadcast (replicated) in a natural way to allow the operation to complete. Essentially the scalar was turned into a 2-by-2 multiarray where each element had the value of
b; then the operation continued. The operator in this example was addition; as all operators exhibit the same behavior (element-by-element operation), one is as good as another for demonstration purposes. Go ahead and try the other operators to convince yourself. What would happen if you multiplied
b rather than adding them?