Why Use Matrices
Matrix Notation
Matrix Basic Operations
Matrix Product
Matrix Operations Properties
Matrix Inverse
Calculating Inverses
Transformations
In our previous lecture, we used matrices as a shorthand for linear systems of equations
However, matrices can be used in multiple applications
Imagine, that a developer is considering three different methods to solve a problem, and each one requires a different number of operations A, B, and C
\[\begin{bmatrix} 3 & 2 & 4\\ 2 & 5 & 2\\ 8 & 1 & 1 \\ \end{bmatrix}\]
Now, imagine that each operation requires the following computational efforts:
A requires 3 clock-cycles
B 4 requires 3 clock-cycles
C 6 requires 3 clock-cycles
We can easily check which method is most efficient using...
\[\begin{bmatrix} 3 & 2 & 4\\ 2 & 5 & 2\\ 8 & 1 & 1 \\ \end{bmatrix}\cdot \begin{bmatrix} 3\\ 4\\ 6\\ \end{bmatrix} = \begin{bmatrix} 41\\ 38\\ 34\end{bmatrix}\]
Matrices are denoted by uppercase letters: A, B, C
The size of a matrix is denoted by its number of rows times number columns
For example, a 3 X 2 matrix has three rows and two columns
Only one row: row vector, u (boldface)
Only one column: column vector, v (boldface)
\[ A = \begin{bmatrix} a_{11} & a_{12} & \ldots & a_{1n}\\ a_{21} & a_{22} & \ldots & a_{2n}\\ \vdots & \vdots & \ldots & \vdots \\ a_{m1} & a_{m2} & \ldots & a_{mn}\\ \end{bmatrix}\]
The matrix element in its ith row and jth column is denoted by \[a_{ij}\ \text{or } (A)_{ij}\]
Matrices in which n = m are called square matrices
Elements in which i = j are called diagonal elements
Equality, addition, and subtraction are only defined if both matrices have the same size
For A = B, all corresponding elements need to be identical
For A + B, one needs to add each corresponding element
For A - B, one needs to subtract each corresponding element
\[A = \begin{bmatrix} 3 & 2 & 4\\ 2 & 1 & 3\\ 1 & 2 & 2\\ \end{bmatrix}, B = \begin{bmatrix} 1 & 1 & 1\\ 2 & 3 & 1\\ 2 & 1 & 2\\ \end{bmatrix}\]
\[C = \begin{bmatrix} 3 & 2\\ 2 & 1\\ \end{bmatrix}, D = \begin{bmatrix} 1 & 1 & 1\\ 2 & 3 & 1\\ \end{bmatrix}\]
The product of a matrix A by a scalar c is obtained by multiplying each element of A by c
I.e., \[cA\]
results in, \[ca_{ij}\ \forall\ i,j\]
The product C of two matrices A and B is obtained by:
→ Multiplying each element of the ith row of A by its corresponding elements in the jth column of B
→ Add the sum of these products
→ This results in the element Cij
A product is undefined if the number of columns in A is not the same as the number of rows in B
\[A = \begin{bmatrix} 3 & 2 & 4\\ 2 & 1 & 3\\ 1 & 2 & 2\\ \end{bmatrix}, B \begin{bmatrix} 3 & 2\\ 2 & 1\\ 1 & 2\\ \end{bmatrix}\]
\[C = \begin{bmatrix} 1 & 1\\ 2 & 3\\ \end{bmatrix}\]
The product of two matrices is defined as is because of its usefulnes
The product of two matrices do not follow some of the same properties, like the commutative law, of two scalars
A general system of equations such as ...
\[\begin{aligned} a_{11}x_{1} & + a_{12}x_{2} & + \cdots + & a_{1n}x_{n} & = b_{1} \\ a_{21}x_{1} & + a_{22}x_{2} & + \cdots + & a_{2n}x_{n} & = b_{2} \\ \vdots & + \vdots & + \cdots + & \vdots& = \vdots \\ a_{m1}x_{1} & + a_{m2}x_{2} & + \cdots + & a_{mn}x_{n} & = b_{m} \end{aligned} \]
... can be represented using matrix multiplication as
\[Ax = b \]
where A is a matrix, and x and b are column vectors
The transpose of a matrix A, AT, is obtained by flipping its rows with its columns
The trace of a matrix A is obtained by adding all entries of its main diagonal
A + B = B + A (commutative law for addition)
(A + B) + C = A + (B + C) (associative law for addition)
A(BC) = (AB)C (associative law for multiplication)
A(B + C)= AB + AC (distributive law)
a(B + C) = aB + aC
AB ≠ BA (commutative law for multiplication)
AB = AC does not imply that B = C (cancellation law)
AB = 0 does not imply that A and/or B are zero
\[A = \begin{bmatrix} 0 & 1\\ 0 & 2\\ \end{bmatrix}, B = \begin{bmatrix} 1 & 1\\ 3 & 4\\ \end{bmatrix}\]
\[C = \begin{bmatrix} 2 & 5\\ 3 & 4\\ \end{bmatrix}, D = \begin{bmatrix} 3 & 7\\ 0 & 0\\ \end{bmatrix}\]
A square matrix in which the main diagonal elements are one, and all other elements are zeroes is called an identity Matrix
It is normally denoted by I, or In
An mxn matrix A multiplied by an identity matrix results in itself:
\[AI_{n} = I_{m}A = A\]
The reduced row echelon form of a square matrix is either an identity matrix, or it has rows of zeroes
Proof: by definition
If there is a matrix B, such that AB = I, B is said to be the inverse of A
The inverse of a matrix is frequently denoted by: A-1
If A-1 exists, A is said to be invertible
If A-1 does not exist, A is said to be singular
\[A = \begin{bmatrix} 2 & 1\\ 1 & 1\\ \end{bmatrix}, A^{-1} = \begin{bmatrix} 1 & -1\\ -1 & 2\\ \end{bmatrix}\]
\[B = \begin{bmatrix} 1 & 3\\ 2 & 5\\ \end{bmatrix}, B^{-1} = \begin{bmatrix} -5 & 3\\ 2 & -1\\ \end{bmatrix}\]
If A-1 exists, it is unique
(AB)-1 = B-1A-1 (can be extended to three or more)
(A-1)-1 = A
(kA)-1 = k-1A-1
An Elementary Matrix E is obtained by performing a single row operation on I
Multiplying a matrix A by an elementary matrix E is equivalent of performing the same row operation on it
A combination of row operations can be performed by multiplying a matrix by consecutive elementary matrices
\[A = \begin{bmatrix} 2 & 0 & 1\\ 0 & 0 & 1\\ 0 & 1 & 0\\ \end{bmatrix}, E_{0} = \begin{bmatrix} 1 & 0 & 0\\ 0 & 0 & 1\\ 0 & 1 & 0\\ \end{bmatrix}\]
\[E_{1} = \begin{bmatrix} 1 & 0 & -1\\ 0 & 1 & 0\\ 0 & 0 & 1\\ \end{bmatrix}, E_{2} = \begin{bmatrix} 1/2 & 0 & 0\\ 0 & 0 & 1\\ 0 & 1 & 0\\ \end{bmatrix}\]
For this example, it is easy to show that \[ E_{0}E_{1}E_{2}A = I \]
It is, we can put matrices in reduced row echelon form by multiplying them by elementary matrices
If the matrix is invertible, this results in an identity matrix
The following statements are equivalent. They are all either true, or all false.
(a) A is invertible
(b) Ax = 0 has only the trivial solution
(c) The reduced form echelon of A is I
(d) A can be expressed as a product of Ek{k}
We can show that, if \[ E_{k} \cdots E_{2}E_{1}A = I \]
then, \[ A = E_{1}^{-1}E_{2}^{-1} \cdots E_{k}^{-1} \]
Therefore,
\[ A^{-1} = E_{k}\cdots E_{2}E_{1} \]
This means that we can invert matrices by first appending an identity matrix to its right
Then, we perform row operations to put the original matrix in reduced echelon form
If the reduced echelon form is an identity, the right part will be its inverse
Otherwise the matrix is singular
\[A = \begin{bmatrix} 2 & 0 & 1\\ 0 & 0 & 1\\ 0 & 1 & 0\\ \end{bmatrix}\]
\[\begin{bmatrix} 2 & 0 & 1 & \vdots & 1 & 0 & 0\\ 0 & 0 & 1 & \vdots & 0 & 1 & 0\\ 0 & 1 & 0 & \vdots & 0 & 0 & 1\\ \end{bmatrix} \]
The standard basis vectors for Rn are:
\[e_{1} = \begin{bmatrix} 1 \\ 0 \\ \vdots\\ 0 \\ \end{bmatrix},\ e_{2} = \begin{bmatrix} 0 \\ 1 \\ \vdots\\ 0 \\ \end{bmatrix},\ e_{n} = \begin{bmatrix} 0 \\ 0 \\ \vdots\\ 1 \\ \end{bmatrix}\]
Any vector in this space can be formed with a combination of standard basis vectors
Multiplying a vector w by a Matrix Amn effectively maps a domain Rn into a codomain Rm
A transformation maps each element in the domain to an element in the codomain
\[w = f(x),\ \text{or}\ w = A x\]
The matrix A that performs the transformation is called a standard matrix
\[\begin{aligned} w_{1} & = x_{1} \\ w_{2} & = x_{2} + x_{3} \end{aligned} \]
\[ \begin{bmatrix} w_{1}\\ w_{2}\\ \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 1\\ \end{bmatrix} \begin{bmatrix} x_{1}\\ x_{2}\\ x_{3}\\ \end{bmatrix} \]
Textbook: Sections 1.1, 1.2, 1.3, 1.4, 1.5, 1.8
Exercises
→ Section 1.3 1, 3, 7, 11, 13, 17*, 18*
→ Section 1.4 1, 23, 24
→ Section 1.5 1, 2, 3, 4, 7, 8, 11, 12, 13, 15, 16
→ Section 1.8 12, 13, 14, 15
* No need for row expansion (just do the product)