Dot Product and Matrix Multiplication Explained

The dot product of a matrix is a basic linear algebra calculation used in machine learning to make computations more efficiently as data increases. Here’s how to calculate it.

Written by Soner Yıldırım
Published on Nov. 23, 2022
Dot Product and Matrix Multiplication Explained
Image: Shutterstock / Built In
Brand Studio Logo

Data is collected in many different formats from numbers to images to categories to sound waves. However, we need numerical data in order to analyze it on computers. Machine learning and deep learning models are data-hungry. The performance of them is highly dependent on the amount of data. Thus, we tend to collect as much data as possible in order to build a robust and accurate model. As the amount of data increases, the operations done with scalars become inefficient. We need vectorized or matrix operations to make computations efficiently. That’s where linear algebra comes in.

What Is the Dot Product of a Matrix?

A dot product of a matrix is a basic linear algebra computation used in deep learning models to complete operations with larger amounts of data more efficiently. It’s the result of multiplying two matrices that have matching rows and columns, such as a 3x2 matrix and a 2x3 matrix. It can also be calculated in NumPy using the np.dot operation. 

Linear algebra is one of the most important topics in the data science domain. In this post, we will cover two basic, yet very important, operations of linear algebra: Dot product and matrix multiplication. These basic operations are the building blocks of complex machine learning and deep learning models, so it’s important to understand them.

 

How to Find the Dot Product

The dot product of two vectors is the sum of the products of elements with regards to position. The first element of the first vector is multiplied by the first element of the second vector, and so on. The sum of these products is the dot product, which can be done with np.dot() function.

Let’s first create two simple vectors in the form of NumPy arrays and calculate the dot product.

Two numpy array.
Two numpy array. | Image: Soner Yildirim

The dot product of these two vectors is the sum of the products of elements at each position. In this case, the dot product is (1*2)+(2*4)+(3*6).

Dot product for the two numpy arrays.
Dot product for the two NumPy arrays. | Image: Soner Yildirim

Since we multiply elements at the same positions, the two vectors must have the same length in order to have a dot product.

A tutorial on the basics of a dot product. | Video 3Blue1Brown.

More on Data Science: A Step-by-Step Explanation of Principal Component Analysis

 

How to Calculate a Dot Product Matrix

In data science, we mostly deal with matrices. A matrix is a bunch of row and column vectors combined in a structured way. Thus, the multiplication of two matrices involves many dot product operations of vectors. It will be more clear when we go over some examples. Let’s first create two, 2x2 matrices with NumPy.

Two, 2x2 numpy arrays.

Two, 2x2 Num{y arrays. | Image: Soner Yildirim
Two 2x2 arrays.
Two 2x2 arrays. | Image: Soner Yildirim

A 2x2 matrix has two rows and two columns. The index of rows and columns start with zero. For instance, the first row of A (row with index zero) is the array of [4,2]. The first column of A is the array of [4,0]. The element of the first row and first column is four. We can access individual rows, columns or elements with the following NumPy syntax.

Numpy syntax to access the first column and row of the arrays.
NumPy syntax to access the first column and row of the arrays. | Image: Soner Yildirim

These are important concepts to comprehend in order to understand matrix multiplication.

The multiplication of two matrices involves dot products between the first matrix row and the columns of the second matrix. The first step is the dot product between the first row of A and the first column of B. The result of this dot product is the element of the resulting matrix at position [0,0] (i.e. first row, first column.)

Finding the dot product of two matrices.
Finding the dot product of two matrices. | Image: Soner Yildirim

So the resulting matrix, C, will have a (4*4) + (2*1) at the first row and first column. C[0,0] = 18.

The next step is the dot product of the first row of A and the second column of B.

Finding the dot product of the first row of A and second column of B.
Finding the dot product of the first row of A and second column of B. | Image: Soner Yildirim

C will have a (4*0) + (2*4) at the first row and second column. C[0,1] = 8.

First row A is complete, so we start on the second row of A and follow the same steps.

Continuing the dot product process multiplying the second row of A and the first column of B.
Continuing the dot product process multiplying the second row of A and the first column of B. | Image: Soner Yildirim

C will have a (0*4) + (3*1) at the second row and first column. C[1,0] = 3.

The final step is the dot product between the second row of A and the second column of B.

Final step in the dot product process multiplying the second row of A and the second row of B.
Final step in the dot product process multiplying the second row of A and the second row of B. | Image: Soner Yildirim

C will have a (0*0) + (3*4) at the second row and second column. C[1,1] = 12.

We’ve now seen how it’s done step-by-step. All of these operations can also be done using a np.dot operation:

Finding the dot product matrix using the np.dot operation.
Finding the dot product matrix using the np.dot operation. | Image: Soner Yildirim

As you may recall from vector dot products, two vectors must have the same length in order to have a dot product. Each dot product operation in a matrix multiplication must follow this rule. Dot products are done between the rows of the first matrix and the columns of the second matrix. Thus, the rows of the first matrix and columns of the second matrix must have the same length.

I want to emphasize an important point here: The length of a row is equal to the number of columns. Similarly, the length of a column is equal to the number of rows.

Consider the following matrix D:

An example of a 3x2 matrix.
An example of a 3x2 matrix. | Image: Soner Yildirim

D has three rows and two columns, so it’s a 3x2 matrix. The length of a row is two, which is the number of columns, and the length of a column is three, which is the number of rows.

Length of a row and column for matrix D.

Length of a row and column for matrix D. | Image: Soner Yildirim

That’s the long explanation, but the point is that to be able to perform a matrix multiplication, the number of columns in the first matrix must be equal to the number of rows in the second matrix.

More on Data Science: 4 Ways to Add a Column in Pandas

For instance, we can multiply a 3x2 matrix with a 2x3 matrix.

Multiplying a 3x2 and a 2x3 matrix to yield a 3x3 dot product matrix.
Multiplying a 3x2 and a 2x3 matrix to yield a 3x3 dot product matrix. | Image: Soner Yildirim

The shape of the resulting matrix will be 3x3 because we’re doing three dot product operations for each row of A, and A has three rows. An easy way to determine the shape of the resulting matrix is to take the number of rows from the first one and the number of columns from the second one:

  • 3x2 and 2x3 multiplication returns 3x3.
  • 3x2 and 2x2 multiplication returns 3x2.
  • 2x4 and 4x3 multiplication returns 2x3.

If the conditions we’ve been discussing are not met, matrix multiplication is impossible. Consider the following matrices C and D. They both are 3x2 matrices:

Attempting to multiply two 3x2 matrices.
Attempting to multiply two 3x2 matrices. | Image: Soner Yildirim

If we try to multiply them, we will get the following value error:

Value error for trying to multiply two 3x2 matrices.
Value error for trying to multiply two 3x2 matrices. | Image: Soner Yildirim.

We’ve now covered the basic, but very fundamental operations of linear algebra. These basic operations are the building blocks of complex machine learning and deep learning models. Lots of matrix multiplication operations are done during the optimization process of models. Thus, it’s very important to also understand the basics, as well.

Hiring Now
Magna International
Automotive • Hardware • Robotics • Software • Transportation • Manufacturing
SHARE