Matrix

A matrix over a field P is a mapping {1, 2, …, n} × {1, 2, …, m} → P. A matrix is usually denoted by a capital letter: A = (…).

Basic concepts

Put simply, a matrix is a table with n columns and m rows, although the way rows and columns are labelled is not always the same, so watch out for that. Each cell of the table contains a number or some other expression. A matrix does not have to contain only numbers, although at the start you will most likely not meet any other kind. So what might a matrix look like?

$A = (\begin{array}{ccc} 0 & 1 & 5 \\ 8 & 5 & 51 \end{array})$

This matrix has two rows and three columns. The entries of a matrix are written with subscripts and, instead of a capital letter, a lower-case letter is used: a₁₁ = 0 or a₂₃ = 51. The first subscript gives the row and the second gives the column.

Special types of matrix

Matrices can have various properties, and some special matrices even have names of their own.

A square matrix is a matrix with the same number of rows as columns. If a matrix is not square, it is rectangular. An example of a square matrix:

$A = (\begin{array}{ccc} 0 & 1 & 5 \\ 8 & 5 & 23 \\ 47 & 154 & 2 \end{array})$

A zero matrix is a matrix with zeros in every position: a_ij = 0.

$A = (\begin{array}{ccc} 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{array})$

An identity matrix is a square matrix with ones on the main diagonal and zeros everywhere else. The main diagonal is the “diagonal line” running from top left to bottom right. In other words, it is the entries where i = j.

$A = (\begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array})$

A matrix in row echelon form has any zero rows at the bottom (or has no zero rows), and each non-zero row starts with more zeros than the row before it. These are all matrices in row echelon form:

$A_{1} = (\begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & π \\ 0 & 0 & 1 \end{array}), A_{2} = (\begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{array}), A_{3} = (\begin{array}{ccccc} 1 & 1 & 1 & 1 & 8 \\ 0 & 0 & 0 & 5 & 1 \\ 0 & 0 & 0 & 0 & 5 \end{array})$

The transpose of a matrix A is the matrix A^T for which $a_{i j} = a_{j i}^{T}$ . That is, an entry that was in the ith row and jth column will be in the jth row and ith column of the transpose. In short, you swap the rows of the matrix for its columns.

$\begin{array}{rcl} {(\begin{array}{ccc} 0 & 1 & 5 \\ 8 & 5 & 23 \\ 47 & 154 & 2 \end{array})}^{T} & = & (\begin{array}{ccc} 0 & 8 & 47 \\ 1 & 5 & 154 \\ 5 & 23 & 2 \end{array}) \\ {(\begin{array}{ccc} 3 & 4 & 5 \\ 6 & 7 & 8 \end{array})}^{T} & = & (\begin{array}{cc} 3 & 6 \\ 4 & 7 \\ 5 & 8 \end{array}) \end{array}$

A symmetric matrix is a square matrix A that satisfies A = A^T. Entries that are mirror images of each other across the diagonal are equal. So we can write a_ij = a_ji.

$A = (\begin{array}{ccc} 9 & 3 & 4 \\ 3 & 7 & 0 \\ 4 & 0 & 2 \end{array})$

A skew-symmetric matrix is almost the same as a symmetric matrix, except that the entries on the other side of the diagonal have the opposite sign: A = −A^T. This means the entries on the main diagonal must be zero, because a = −a = 0.

$A = (\begin{array}{ccc} 0 & - 3 & - 4 \\ 3 & 0 & 5 \\ 4 & - 5 & 0 \end{array})$

A diagonal matrix is a matrix with zeros everywhere except on the main diagonal. To be more precise, every other entry must be zero, while the entries on the main diagonal are not restricted.

$A_{1} = (\begin{array}{ccc} 9 & 0 & 0 \\ 0 & 7 & 0 \\ 0 & 0 & 2 \end{array}), A_{2} = (\begin{array}{ccc} 3 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{array})$

Basic operations with matrices

We can add matrices, multiply them by a number, and even multiply matrices by each other.

Adding matrices is fairly intuitive. If the matrices are the same size (the same number of columns and rows), the resulting matrix has, in each position, the sum of the entries in the corresponding positions of the original matrices. In other words, if we add matrices A + B = C, then a_ij + b_ij = c_ij.

$(\begin{array}{ccc} 0 & 1 & 5 \\ 8 & 5 & 23 \\ 47 & 154 & 2 \end{array}) + (\begin{array}{ccc} 5 & 4 & 3 \\ 10 & 20 & 30 \\ 7 & - 54 & - 12 \end{array}) = (\begin{array}{ccc} 5 & 5 & 8 \\ 18 & 25 & 53 \\ 54 & 100 & - 10 \end{array})$

Addition of matrices is clearly commutative and associative: A + B = B + A and A + (B + C) = (A + B) + C.

Multiplying a matrix by a number is also intuitive. You take the number and multiply every entry of the matrix by it, nothing more: k · A = k · a_ij.

$5 \cdot (\begin{array}{ccc} 0 & 1 & 5 \\ 8 & 5 & 23 \\ 47 & 154 & 2 \end{array}) = (\begin{array}{ccc} 0 & 5 & 25 \\ 40 & 25 & 115 \\ 235 & 770 & 10 \end{array})$

Multiplying matrices

Multiplying matrices is a bit harder, because it is not as intuitive as you might expect. It is not enough to multiply the corresponding entries. First of all, the matrices must satisfy the condition that the number of columns of the first matrix equals the number of rows of the second matrix. Everything else can be arbitrary. Now we can define the product (n is the number of columns of the first matrix):

$(A \cdot B)_{i j} = \sum_{p = 1}^{n} a_{i p} \cdot b_{p j}$

That tells you a lot, doesn’t it? Let me try to make it clear for those who are a little slower on the uptake :-). Take the first row of the first matrix and the first column of the second matrix. Multiply the first entry by the first entry, add the product of the second entry and the second entry, and so on. That gives you the entry c₁₁ of the new matrix C. An example is by far the best way to see it. We will multiply these two matrices:

$A = (\begin{array}{cc} 1 & 2 \\ 3 & 4 \end{array}), B = (\begin{array}{cc} 5 & 6 \\ 7 & 8 \end{array})$

Now we pick the first row of the first matrix and the first column of the second matrix:

$A = (\begin{array}{cc} 1 & 2 \\ 3 & 4 \end{array}), B = (\begin{array}{cc} 5 & 6 \\ 7 & 8 \end{array})$

To keep the notation clear, we will call the new matrix C. To get the first entry of this matrix, we have to work out c₁₁ = a₁₁ · b₁₁ + a₁₂ · b₂₁. Note that the row comes first in the subscript, then the column. Substituting gives 1 · 5 + 2 · 7 = 19. The first entry has the value 19:

$C = (\begin{array}{cc} 19 & ? \\ ? & ? \end{array})$

We get the next entry, c₁₂, in the same way, except that we take the first row and the second column. This somewhat long-winded calculation always gives the entry that the row and column have in common. The first row and the first column share the position c₁₁, while the first row and the second column share c₁₂. The following diagram shows this nicely:

Product of matrices

Now let’s quickly work out the rest of the matrix:

$\begin{array}{rcl} c_{12} & = & a_{11} \cdot b_{12} + a_{12} \cdot b_{22} = 1 \cdot 6 + 2 \cdot 8 = 22. \\ c_{21} & = & a_{21} \cdot b_{11} + a_{22} \cdot b_{21} = 3 \cdot 5 + 4 \cdot 7 = 43. \\ c_{22} & = & a_{21} \cdot b_{12} + a_{22} \cdot b_{22} = 3 \cdot 6 + 4 \cdot 8 = 50. \end{array}$

We just write these numbers into a matrix:

$C = (\begin{array}{cc} 19 & 22 \\ 43 & 50 \end{array})$

You can check the result in Excel or in OpenOffice Calc, for example, which both have functions for working with matrices.

Now a few general facts about matrix multiplication. First of all, matrix multiplication is not commutative. In general A · B = B · A does not hold, although of course such a case can occur. But matrix multiplication is associative. It is even distributive over addition: A (B + C) = AB + AC. If we multiply two matrices a_ix · b_xn, the resulting matrix will be of size i × n (it will have as many rows as the first matrix has rows and as many columns as the second matrix has columns).

Examples

We are given these three matrices:

$A = (\begin{array}{ccc} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{array}), B = (\begin{array}{ccc} 4 & 2 & 8 \\ 10 & 12 & 4 \\ 4 & 5 & 9 \end{array}), C = (\begin{array}{cc} 8 & 9 \\ - 5 & 4 \\ 10 & - 1 \end{array}) .$

Work out the matrix product A · B.

$(\begin{array}{ccc} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{array}) \cdot (\begin{array}{ccc} 4 & 2 & 8 \\ 10 & 12 & 4 \\ 4 & 5 & 9 \end{array}) = (\begin{array}{ccc} 36 & 41 & 43 \\ 90 & 98 & 106 \\ 144 & 155 & 169 \end{array})$

Work out the matrix product B · A.

Watch out: matrix multiplication is not commutative, so we certainly cannot say that the result will be the same as in the previous case. We simply have to work it all out again:

$(\begin{array}{ccc} 4 & 2 & 8 \\ 10 & 12 & 4 \\ 4 & 5 & 9 \end{array}) \cdot (\begin{array}{ccc} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{array}) = (\begin{array}{ccc} 68 & 82 & 96 \\ 86 & 112 & 138 \\ 87 & 105 & 123 \end{array})$

Work out the matrix products A · C and C · B.

$(\begin{array}{ccc} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{array}) \cdot (\begin{array}{cc} 8 & 9 \\ - 5 & 4 \\ 10 & - 1 \end{array}) = (\begin{array}{cc} 28 & 14 \\ 67 & 50 \\ 106 & 86 \end{array})$

We cannot work out the second product, because the number of columns of the first matrix is different from the number of rows of the second matrix.

Elementary row operations

To work with matrices effectively, we need to define some elementary operations. First, we can multiply a row or column of a matrix by a non-zero number. The effect is the same as multiplying the whole matrix by a number k, but only in that one row or column.

The second operation is adding k times the jth row to the ith row, and the same for columns. It sounds a bit scary, but it is actually simple. Let’s demonstrate it for k = 1. We will start with this matrix:

$(\begin{array}{ccc} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{array})$

Now we add the second row to the first row. So we take the second row and add the numbers in matching positions to the numbers in the first row. Nothing happens to the numbers in the second row; only the first row changes:

$(\begin{array}{ccc} 5 & 7 & 9 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{array})$

Now we can carry on with the operations. This time we add twice the first row to the third row. We take the first row of this new matrix, multiply it by two, which gives the row (10, 14, 18), and add these numbers to the third row. Again, the first row does not change; only the third row does:

$(\begin{array}{ccc} 5 & 7 & 9 \\ 4 & 5 & 6 \\ 17 & 22 & 27 \end{array})$

Now we add the sum of the first and second rows to the third row. There is really nothing new here, because if we first add the second row to the third row and then add the first row, we must arrive at the same result. The sum of the first and second rows is (5, 7, 9) + (4, 5, 6) = (9, 12, 15). Adding this row to the third row gives (9, 12, 15) + (17, 22, 27) = (26, 34, 42).

$(\begin{array}{ccc} 5 & 7 & 9 \\ 4 & 5 & 6 \\ 26 & 34 & 42 \end{array})$

And one more column operation (these are not usually used as often, because they are not as easy to follow). We add the first column to the second column:

$(\begin{array}{ccc} 5 & 12 & 9 \\ 4 & 9 & 6 \\ 26 & 60 & 42 \end{array})$

And finally, we multiply the second row by two:

$(\begin{array}{ccc} 5 & 12 & 9 \\ 8 & 18 & 12 \\ 26 & 60 & 42 \end{array})$

Linear dependence

Now let’s explain what dependent rows and columns are. A row is linearly dependent if it can be expressed as a linear combination of the other rows of the matrix. In short, if you can add the rows together in various ways so that you end up with the row you are looking for, that row is linearly dependent. For example:

$(\begin{array}{ccc} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 6 & 9 & 12 \end{array})$

If you take twice the first row and the second row, you get the third row. If you subtract this combination from the third row, you get a zero row (a row containing only zeros). This row is dependent. So we carry out the operations as follows (multiply the first row by two, add it to the second row, and subtract it from the third row):

$(\begin{array}{ccc} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 6 & 9 & 12 \end{array}) \sim (\begin{matrix} 2 & 4 & 6 \\ 4 & 5 & 6 \\ 6 & 9 & 12 \end{matrix}) \sim (\begin{matrix} 2 & 4 & 6 \\ 6 & 9 & 12 \\ 6 & 9 & 12 \end{matrix}) \sim (\begin{matrix} 2 & 4 & 6 \\ 6 & 9 & 12 \\ 0 & 0 & 0 \end{matrix})$

Suppose we have a matrix A with n rows, where A_i denotes the ith row. Then we say that the matrix contains no linearly dependent row if:

$α_{1} A_{1} + α_{2} A_{2} + \dots + α_{n} A_{n} = 0$

if and only if α₁, α₂, …, α_n = 0. The zero on the right-hand side of the equation represents a zero row. In other words, the only solution of this equation is the trivial solution. If we find another solution, the matrix contains a linearly dependent row. The same applies to columns. For the previous matrix we would have:

$\begin{array}{rcl} 2 (\begin{array}{c} 1 & 2 & 3 \end{array}) + 1 (\begin{array}{c} 4 & 5 & 6 \end{array}) - 1 (\begin{array}{c} 6 & 9 & 12 \end{array}) = \\ = (\begin{array}{c} 6 & 9 & 12 \end{array}) - (\begin{array}{c} 6 & 9 & 12 \end{array}) = (\begin{array}{c} 0 & 0 & 0 \end{array}) \end{array}$

So the values of alpha are α₁ = 2, α₂ = 1, α₃ = −1.

The number of independent rows or columns is given by the rank of the matrix.

Non-singular and singular matrices

A matrix is called non-singular if it has maximum rank (that is, if it contains no linearly dependent row) and if it is a square matrix. A square matrix is called singular if it is not non-singular (that is, if the matrix contains at least one linearly dependent row). These two terms are quite important, and many definitions rest on them. A lot of things are defined only if the matrix is non-singular.

« Previous: Lattices

Next: Rank of a matrix »