Matrix Multiplcation

What are Matrices?

Matrices are a rectangular arrangement of data. They can be numbers, variables, symbols or expressions. These are(https://vishalramesh.com/foundational-ml/math/notations-and-terminologies#maximas-and-minimas) represented in rows and columns. Here’s an example: \(numbers = \begin{bmatrix} 1 & 4 & 7 \\ 2 & 5 & 8 \\ 3 & 6 & 9 \end{bmatrix}\) or \(fruits = \begin{bmatrix} banana & apple & mango \\ jackfruit & tomato & papaya \\ \end{bmatrix}\)

The shape of a matrix is represented as the (https://vishalramesh.com/foundational-ml/math/notations-and-terminologies#maximas-and-minimas)number of rows X number of columns. The shape of the matrix numbers is 3X3 and the shape of the matrix fruits is 2X3.

Matrix Multiplication

Consider two matrices \(A = \begin{bmatrix} a_{00} & a_{01} & a_{02} \\ a_{10} & a_{11} & a_{12} \\ a_{20} & a_{21} & a_{22} \end{bmatrix} , B = \begin{bmatrix} b_{00} & b_{01} & b_{02} \\ b_{10} & b_{11} & b_{12} \\ b_{20} & b_{21} & b_{22} \end{bmatrix}\)

To multiply two matrices, you the the first row of the first matrix, and do a scalar multiplcation with the first column of the second matrix. This is the value of the first row’s first column. Repeat the process to build out the entire result matrix.

The product of these matrices \(A \cdot B\) or just \(AB\) is as follows. \(B = \begin{bmatrix} a_{00}b_{00} + a_{01}b_{10} + a_{02}b_{20} & a_{00}b_{01} + a_{01}b_{11} + a_{02}b_{21} & a_{00}b_{02} + a_{01}b_{12} + a_{02}b_{22} \\ a_{10}b_{00} + a_{11}b_{10} + a_{12}b_{20} & a_{10}b_{01} + a_{11}b_{11} + a_{12}b_{21} & a_{10}b_{02} + a_{11}b_{12} + a_{12}b_{22} \\ a_{20}b_{00} + a_{21}b_{10} + a_{22}b_{20} & a_{20}b_{01} + a_{21}b_{11} + a_{22}b_{21} & a_{20}b_{02} + a_{21}b_{12} + a_{22}b_{22} \end{bmatrix}\)

Check this out if you want a more visual explanation or just scroll to the end of this page for a visualizer.

• Matrix multiplication is not Commutative. So \(AB != BA\).
• To be able to multiply two matrices, the number of columns of the first matrix should be equal to the number of rows of the second matrix.

Why is this important in ML?

Machine Learning has a lot of linear algebra. We can represent these operations as matrices.

For example, consider the linear equation \(y = ax_{0} + bx_{1} + cx_{2}\). We can represent this using matrices as \(y = \begin{bmatrix} a & b & c \end{bmatrix} \times \begin{bmatrix} x_0\\ x_1\\ x_2 \end{bmatrix}\) and there are a lot of matrix “features” that make solving linear equations much easier and faster. You’ll see them as you go.

Another advantage of using Matrices in machine learning is that it lets us speed up our processing. GPUs are really poweful at executing instructions in parallel that you can parallelize a large number of operations if you can represent them as matrix operations and perform them on a GPU.

There are more uses of matrices in machine learning. You’ll learn about them as you understand the algorithms and their implementations.