About NumPy Array Broadcasting

Arrays with different sizes cannot be added, subtracted, or generally be used in arithmetic. A way to overcome this is to duplicate the smaller array so that it has the dimensionality and size as the larger array. This is called array broadcasting and is available in NumPy when performing array arithmetic, which can greatly reduce and simplify your code. In this tutorial, you will discover the concept of array broadcasting and how to implement it in NumPy. After completing this tutorial, you will know:

•  The problem of arithmetic with arrays with different sizes.
• The solution of broadcasting and common examples in one and two dimensions.
• The rule of array broadcasting and when broadcasting fails.

1.1 Tutorial Overview

 This tutorial is divided into 4 parts; they are:

1. Limitation with Array Arithmetic

2. Array Broadcasting

3. Broadcasting in NumPy

4. Limitations of Broadcasting

1.2 Limitation with Array Arithmetic

You can perform arithmetic directly on NumPy arrays, such as addition and subtraction. For example, two arrays can be added together to create a new array where the values at each index are added together. For example, an array a can be defined as [1, 2, 3] and array b can be defined as [1, 2, 3] and adding together will result in a new array with the values [2, 4, 6].

a = [1, 2, 3]
b = [1, 2, 3]
c = a + b
print(c)
c = [1 + 1, 2 + 2, 3 + 3]
print(c)

Strictly, arithmetic may only be performed on arrays that have the same dimensions and dimensions with the same size. This means that a one-dimensional array with the length of 10 can only perform arithmetic with another one-dimensional array with the length 10. This limitation on array arithmetic is quite limiting indeed. Thankfully, NumPy provides a built-in workaround to allow arithmetic between arrays with differing sizes.

1.3 Array Broadcasting

Broadcasting is the name given to the method that NumPy uses to allow array arithmetic between arrays with a different shape or size. Although the technique was developed for NumPy, it has also been adopted more broadly in other numerical computational libraries, such as Theano, TensorFlow, and Octave. Broadcasting solves the problem of arithmetic between arrays of differing shapes by in effect replicating the smaller array along the last mismatched dimension.

1.4 Broadcasting in NumPy

We can make broadcasting concrete by looking at three examples in NumPy. The examples in this section are not exhaustive, but instead are common to the types of broadcasting you may see or implement.

1.4.1 Scalar and One-Dimensional Array

A single value or scalar can be used in arithmetic with a one-dimensional array. For example, we can imagine a one-dimensional array a with three values [a1, a2, a3] added to a scalar b.

a = [a1, a2, a3]
b

The scalar will need to be broadcast across the one-dimensional array by duplicating the value it 2 more times.

b = [b1, b2, b3]

The two one-dimensional arrays can then be added directly.

c = a + b
c = [a1 + b1, a2 + b2, a3 + b3]

The example below demonstrate this in NumPy.

# Example of broadcasting a scalar to a one-dimensional array in NumPy
# broadcast scalar to one-dimensional array
from numpy import array
# define array
a = array([1, 2, 3])
print(a)

# define scalar
b = 2
print(b)

# broadcast
c = a + b
print(c)

Running the example first prints the defined one-dimensional array, then the scalar, followed by the result where the scalar is added to each value in the array.

1.4.2 Scalar and Two-Dimensional Array

A scalar value can be used in arithmetic with a two-dimensional array. For example, we can imagine a two-dimensional array A with 2 rows and 3 columns added to the scalar b.

 

The scalar will need to be broadcast across each row of the two-dimensional array by duplicating it 5 more times.

 

The two two-dimensional arrays can then be added directly.

 

The example below demonstrates this in NumPy.

# Example of broadcasting a scalar to a two-dimensional array in NumPy
# broadcast scalar to two-dimensional array
from numpy import array
# define array
A = array([
    [1, 2, 3],
    [1, 2, 3]])
print(A)
# define scalar
b = 2
print(b)

# broadcast
C = A + b
print(C)

Running the example first prints the defined two-dimensional array, then the scalar, then the result of the addition with the value 2 added to each value in the array.

1.4.3 One-Dimensional and Two-Dimensional Arrays

A one-dimensional array can be used in arithmetic with a two-dimensional array. For example, we can imagine a two-dimensional array A with 2 rows and 3 columns added to a one-dimensional array b with 3 values.

 The one-dimensional array is broadcast across each row of the two-dimensional array by creating a second copy to result in a new two-dimensional array B.

 The two two-dimensional arrays can then be added directly.

 The example below demonstrates this in NumPy.

# broadcast one-dimensional array to two-dimensional array
from numpy import array
# define two-dimensional array
A = array([
    [1, 2, 3],
    [1, 2, 3]
])
print(A)
# define one-dimensional array
b = array([1, 2, 3])
print(b)
# broadcast
C = A + b
print(C)

Running the example first prints the defined two-dimensional array, then the defined onedimensional array, followed by the result C where in effect each value in the two-dimensional array is doubled.

 1.5 Limitations of Broadcasting

Broadcasting is a handy shortcut that proves very useful in practice when working with NumPy arrays. That being said, it does not work for all cases, and in fact imposes a strict rule that must be satisfied for broadcasting to be performed. Arithmetic, including broadcasting, can only be performed when the shape of each dimension in the arrays are equal or one has the dimension size of 1. The dimensions are considered in reverse order, starting with the trailing dimension; for example, looking at columns before rows in a two-dimensional case.

        This make more sense when we consider that NumPy will in effect pad missing dimensions with a size of 1 when comparing arrays. Therefore, the comparison between a two-dimensional array A with 2 rows and 3 columns and a vector b with 3 elements:

A.shape = (2 x 3)
b.shape = (3)

# In effect, this becomes a comparison between:
A.shape = (2 x 3)
b.shape = (1 x 3)

# This same notion applies to the comparison between a scalar that is treated as an array with the required number of dimensions:
A.shape = (2 x 3)
b.shape = (1)
# This becomes a comparison between:

A.shape = (2 x 3)
b.shape = (1 x 1)

# When the comparison fails, the broadcast cannot be performed, and an error is raised.

# The example below attempts to broadcast a two-element array to a 2 × 3 array. This comparison is in effect:

A.shape = (2 x 3)
b.shape = (1 x 2)

We can see that the last dimensions (columns) do not match and we would expect the broadcast to fail. The example below demonstrates this in NumPy.

# Example of broadcasting error in NumPy
# broadcasting error
from numpy import array

# define two-dimensional array
A = array([
    [1, 2, 3],
    [1, 2, 3]
])

print(A.shape)
# define one-dimensional array
b = array([1, 2])
print(b.shape)

# attempt broadcast
C = A + b
print(C)

Running the example first prints the shapes of the arrays then raises an error when attempting to broadcast, as we expected.

 1.6 Extensions

This section lists some ideas for extending the tutorial that you may wish to explore.

• Create three new and different examples of broadcasting with NumPy arrays.
• plement your own broadcasting function for manually broadcasting in one and twodimensional cases.
• Benchmark NumPy broadcasting and your own custom broadcasting functions with one and two dimensional cases with very large arrays.

1.7 Summary

In this tutorial, you discovered the concept of array broadcasting and how to implement in NumPy. Specifically, you learned:

• The problem of arithmetic with arrays with different sizes.
• The solution of broadcasting and common examples in one and two dimensions.
• The rule of array broadcasting and when broadcasting fails.