Linear combination of vectors

A linear combination is a way of building a new vector from a given set of vectors using only addition and multiplication.

What a linear combination is

Let V be a vector space. We know that the space is closed under vector addition, so for two vectors x, y ∈ V we always have x + y ∈ V. If we add two vectors, we get another vector from the same space. The same is true for scalar multiplication. If we have some k ∈ ℝ, then k · x ∈ V too.

We can combine both operations – take k, l ∈ ℝ, and the expression k · x + l · y gives us a new vector, which we call v, and this vector v again lies in the space V. Why? We have k · x ∈ V, and certainly l · y ∈ V as well. We are adding two vectors of the space V, so by definition we must get a vector of the space V.

We can therefore say that the vector v is a linear combination of the vectors x and y. In general: if we have vectors x₁, x₂, …, x_n and real numbers k₁, k₂, …, k_n, which we call coefficients, then a linear combination of these vectors is the vector v that we get as

$$\mathbf{v}=k_1\cdot {\mathbf{x}}_1+k_2\cdot {\mathbf{x}}_2+\dots +k_n\cdot {\mathbf{x}}_n.$$

Example: take the space ℝ² and two vectors from it: (2, 3) and (0, 8). Now we choose the coefficients k₁ = 2, k₂ = 5. The linear combination of these vectors with coefficients k₁, k₂ gives the vector

$$2\cdot (2,3)+5\cdot (0,8)=(4,6)+(0,40)=(4,46)$$

We see that we have got the vector (4, 46), which belongs to the space ℝ². If we chose different coefficients, we would get a different vector. For k₁ = 0, k₂ = −1 we have:

$$0\cdot (2,3)-1\cdot (0,8)=(0,0)+(0,-8)=(0,-8)$$

Linearly dependent vectors

Linearly dependent and linearly independent vectors are closely related to linear combinations. We will show this with an example: take the vector space ℝ² and a set Q of vectors from this space:

$$Q=\{(1,1),(2,4),(3,5),(10,18)\}$$

We now want to know whether we can write any of these four vectors as a linear combination of the other three. Looking at the vectors, we can see that the vector (3, 5) is simply the sum of the first two.

$$(1,1)+(2,4)=(3,5)$$

So the vector (3, 5) is a linear combination of the vectors (1, 1) and (2, 4). Just for fun, let's remove this vector from the set Q. We get a new set Q₁, which looks like this:

$$Q_1=\{(1,1),(2,4),(10,18)\}$$

Can we find another vector in it that is a linear combination of the others? Yes, the vector (10, 18) is a linear combination of the first two vectors with k₁ = 2, k₂ = 4:

$$2\cdot (1,1)+4\cdot (2,4)=(2,2)+(8,16)=(10,18)$$

Again we remove this vector, and we get:

$$Q_2=\{(1,1),(2,4)\}$$

We ask again – can we get one of the vectors as a combination of the other? We cannot: no multiple of the vector (1, 1) will be equal to the vector (2, 4).

We have got a set of vectors in which none of the vectors can be written as a linear combination of the others. We call such a set a linearly independent set of vectors. If one vector can be written as a combination of the others, we say that it is a linearly dependent vector. If it cannot be written in this way, we say that it is a linearly independent vector.

The zero vector

The zero vector 0 plays an important role in linear dependence. Let's go back to the set of vectors Q₁ and the space ℝ². The set Q₁ looks like this:

$$Q_1=\{(1,1),(2,4),(10,18)\}$$

and we already know that the vector (10, 18) is dependent, because 2 · (1, 1) + 4 · (2, 4) = (10, 18). So what would happen if we subtracted the vector (10, 18) from the expression 2 · (1, 1) + 4 · (2, 4)? Or rather, if we added −1 times it? It would look like this:

$$2\cdot (1,1)+4\cdot (2,4)-(10,18)$$

This expression would certainly be equal to the zero vector:

$$2\cdot (1,1)+4\cdot (2,4)-(10,18)=(0,0)$$

From this we can deduce an interesting property. If the set of vectors x₁, x₂, …, x_n is linearly dependent, then there exist real coefficients a₁, a₂, …, a_n such that

$$a_1\cdot {\mathbf{x}}_1+a_2\cdot {\mathbf{x}}_2+\dots +a_n\cdot {\mathbf{x}}_n=\mathbf{0}$$

At least one coefficient a_i must be non-zero. If we put zero in for all the coefficients, we would always get a true equation, and so every set of vectors would turn out to be linearly dependent, which is clearly nonsense. The reason for the previous equation is simple. If, for example, the vector x₁ is linearly dependent, then

$$a_2\cdot {\mathbf{x}}_2+\dots +a_n\cdot {\mathbf{x}}_n={\mathbf{x}}_1$$

All we need to do is add −x₁ to this equation, and we get

$$-1\cdot {\mathbf{x}}_1+a_2\cdot {\mathbf{x}}_2+\dots +a_n\cdot {\mathbf{x}}_n=\mathbf{0}$$

How to tell whether vectors are dependent

If we have vectors x₁, x₂, …, x_n and want to find out whether they are linearly dependent or independent, it is enough to solve the equation above

$$a_1\cdot {\mathbf{x}}_1+a_2\cdot {\mathbf{x}}_2+\dots +a_n\cdot {\mathbf{x}}_n=\mathbf{0}$$

that is, to find the coefficients a_i. In the space ℝ³ we might have these vectors: (1, 2, 3), (2, 1, 7), (1, −4, 5). Since we want to find out whether they are dependent, we solve this equation:

$$a_1\cdot (1,2,3)+a_2\cdot (2,1,7)+a_3\cdot (1,-4,5)=(0,0,0)$$

We simply write this equation out using the definitions of vector addition and scalar multiplication. We have a₁ · (1, 2, 3) = (a₁, 2a₁, 3a₁) and so on. We write it out like this:

$$(a_1,2a_1,3a_1)+(2a_2,a_2,7a_2)+(a_3,-4a_3,5a_3)=(0,0,0)$$

Now we write out the addition. We always add in the same position, so we add the first component a₁ of the first vector, the first component 2a₂ of the second vector and the first component a₃ of the third vector, and all of this must equal the first component of the zero vector, that is, zero. The same goes for the second and third components. We get a system of linear equations:

$$\begin{array}{}{a}_1& +& 2a_2& +& a_3& =& 0\\ 2a_1& +& a_2& +& -4a_3& =& 0\\ 3a_1& +& 7a_2& +& 5a_3& =& 0\end{array}$$

We can write this system as an ordinary matrix:

$$\left(\begin{array}{cccc}1& 2& 1& 0\\ 2& 1& -4& 0\\ 3& 7& 5& 0\end{array}\right)$$

Notice that the columns of this matrix are in fact the original vectors. The first column of the matrix is the original vector (1, 2, 3), the third is the vector (1, −4, 5), and so on. When calculating, you do not always have to go through all these complicated steps: you build this matrix by putting the vectors you want to test for dependence into the columns, and then adding a zero column.

We can solve the system by Gaussian elimination. We reduce the matrix to row echelon form. First we add −2 times the first row to the second row, and −3 times the first row to the third row:

$$\left(\begin{array}{cccc}1& 2& 1& 0\\ 2& 1& -4& 0\\ 3& 7& 5& 0\end{array}\right)\sim \left(\begin{array}{cccc}1& 2& 1& 0\\ 0& -3& -6& 0\\ 0& 1& 2& 0\end{array}\right)$$

Now we can add 3 times the third row to the second row:

$$\left(\begin{array}{cccc}1& 2& 1& 0\\ 0& -3& -6& 0\\ 0& 1& 2& 0\end{array}\right)\sim \left(\begin{array}{cccc}1& 2& 1& 0\\ 0& 0& 0& 0\\ 0& 1& 2& 0\end{array}\right)$$

We do not even need to go any further: we managed to get a row of zeros, which tells us that the vectors are linearly dependent. But we can finish solving the system. We choose a parameter a₃ = t. Then from the equation

$$a_2+2a_3=0$$

which is the last row of the matrix, substituting a₃ = t gives the equation:

$$\begin{array}{rcl}{a}_2+2t& =& 0\\ a_2& =& -2t\end{array}$$

Now we substitute these values into the first equation, that is, the first row of the matrix:

$$\begin{array}{rcl}{a}_1+2a_2+a_3& =& 0\\ a_1-4t+t& =& 0\\ a_1& =& 3t\end{array}$$

So we have a₁ = 3t, a₂ = −2t, a₃ = t. If we substitute, say, t = 3, we get one particular solution a₁ = 9, a₂ = −6, a₃ = 3. We can substitute these coefficients into the original equation

$$a_1\cdot (1,2,3)+a_2\cdot (2,1,7)+a_3\cdot (1,-4,5)=(0,0,0)$$

and we get:

$$9\cdot (1,2,3)+(-6)\cdot (2,1,7)+3\cdot (1,-4,5)=(0,0,0)$$

We simplify the left-hand side:

$$\begin{array}{rcl}9\cdot (1,2,3)+(-6)\cdot (2,1,7)+3\cdot (1,-4,5)& =& (0,0,0)\\ (9,18,27)+(-12,-6,-42)+(3,-12,15)& =& (0,0,0)\\ (9-12+3,18-6-12,27-42+15)& =& (0,0,0)\\ (0,0,0)& =& (0,0,0)\end{array}$$

We see that if we choose the coefficients, for example, a₁ = 9, a₂ = −6, a₃ = 3, we get the zero vector from the vectors (1, 2, 3), (2, 1, 7), (1, −4, 5). If the system had only one solution, the zero solution, then the vectors would be linearly independent.