Vectorization and Broadcasting: NumPy

Vectorization:

In the realm of computer science, there exists a concept known as vectorization, a technique that transforms the conventional approach of using explicit loops into streamlined array operations capable of executing simultaneously. This innovative process enables the application of a series of operations across entire data structures, such as arrays or matrices, in a single stroke, rather than painstakingly handling each element one by one as shown in the Figure 1. The advantages of vectorization are particularly evident in terms of performance, as the operations tend to be carried out at a more fundamental level, utilizing machine-level instructions or compiled code, thereby circumventing the sluggishness associated with traditional loops.

Figure 1: Vectorization and machine level instruction

Vectorization in NumPy

NumPy leverages vectorization by applying operations across entire arrays without explicit loops in Python. These operations are handled by highly optimized, pre-compiled C functions, making them much faster. For instance, multiplying two arrays element-wise in NumPy is vectorized:

Here, the multiplication is vectorized and runs much faster than looping over each element.

Time complexity: loop vs vectorization

Here is an experiment demonstrating how vectorization significantly reduces execution time.

Figure 2: Time complexity loop vs vectorization

Broadcasting

Broadcasting is a feature that empowers NumPy to execute operations on arrays even when their shapes are not aligned. This elegant process involves the automatic expansion of one array, allowing it to conform to the dimensions of another. Rather than creating explicit duplicates of the data, NumPy opts for a more efficient approach, "broadcasting" the smaller arrays so that they seamlessly adapt to the larger arrays during various computations. It follows specific rules to stretch the dimensions of arrays so that element-wise operations are possible. For example:

Together, these concepts allow NumPy to perform fast, efficient numerical computations in a way that's both high-level (easy for the programmer) and high-performance.