Column and Row Spaces

Theorem SLSLC showed us that there is a natural correspondence between solutions to linear systems and linear combinations of the columns of the coefficient matrix. This idea motivates the following important definition.

Definition CSM (Column Space of a Matrix) Suppose that $A$ is an $m\times n$ matrix with columns $\set{\vectorlist{A}{n}}$. Then the column space of $A$, written $\csp{A}$, is the subset of $\complex{m}$ containing all linear combinations of the columns of $A$, \begin{equation*} \csp{A}=\spn{\set{\vectorlist{A}{n}}}. \end{equation*}

Some authors refer to the column space of a matrix as the range, but we will reserve this term for use with linear transformations (Definition RLT).

## Column Spaces and Systems of Equations

Upon encountering any new set, the first question we ask is what objects are in the set, and which objects are not? Here's an example of one way to answer this question, and it will motivate a theorem that will then answer the question precisely.

Example CSMCS: Column space of a matrix and consistent systems.

So if we fix the coefficient matrix, and vary the vector of constants, we can sometimes find consistent systems, and sometimes inconsistent systems. The vectors of constants that lead to consistent systems are exactly the elements of the column space. This is the content of the next theorem, and since it is an equivalence, it provides an alternate view of the column space.

Theorem CSCS (Column Spaces and Consistent Systems) Suppose $A$ is an $m\times n$ matrix and $\vect{b}$ is a vector of size $m$. Then $\vect{b}\in\csp{A}$ if and only if $\linearsystem{A}{\vect{b}}$ is consistent.

Proof.

This theorem tells us that asking if the system $\linearsystem{A}{\vect{b}}$ is consistent is exactly the same question as asking if $\vect{b}$ is in the column space of $A$. Or equivalently, it tells us that the column space of the matrix $A$ is precisely those vectors of constants, $\vect{b}$, that can be paired with $A$ to create a system of linear equations $\linearsystem{A}{\vect{b}}$ that is consistent.

Employing Theorem SLEMM we can form the chain of equivalences

\begin{align*} \vect{b}\in\csp{A} \iff \linearsystem{A}{\vect{b}}\text{ is consistent} \iff A\vect{x}=\vect{b}\text{ for some }\vect{x} \end{align*}

Thus, an alternative (and popular) definition of the column space of an $m\times n$ matrix $A$ is

\begin{align*} \csp{A}&= \setparts{ \vect{y}\in\complex{m} }{ \vect{y}=A\vect{x}\text{ for some }\vect{x}\in\complex{n} } = \setparts{A\vect{x}}{\vect{x}\in\complex{n}} \subseteq\complex{m} \end{align*}

We recognize this as saying create all the matrix vector products possible with the matrix $A$ by letting $\vect{x}$ range over all of the possibilities. By Definition MVP we see that this means take all possible linear combinations of the columns of $A$ --- precisely the definition of the column space (Definition CSM) we have chosen.

Notice how this formulation of the column space looks very much like the definition of the null space of a matrix (Definition NSM), but for a rectangular matrix the column vectors of $\csp{A}$ and $\nsp{A}$ have different sizes, so the sets are very different.

Given a vector $\vect{b}$ and a matrix $A$ it is now very mechanical to test if $\vect{b}\in\csp{A}$. Form the linear system $\linearsystem{A}{\vect{b}}$, row-reduce the augmented matrix, $\augmented{A}{\vect{b}}$, and test for consistency with Theorem RCLS. Here's an example of this procedure.

Example MCSM: Membership in the column space of a matrix.

Theorem CSCS completes a collection of three theorems, and one definition, that deserve comment. Many questions about spans, linear independence, null space, column spaces and similar objects can be converted to questions about systems of equations (homogeneous or not), which we understand well from our previous results, especially those in Chapter SLE:Systems of Linear Equations. These previous results include theorems like Theorem RCLS which allows us to quickly decide consistency of a system, and Theorem BNS which allows us to describe solution sets for homogeneous systems compactly as the span of a linearly independent set of column vectors.

The table below lists these for definitions and theorems along with a brief reminder of the statement and an example of how the statement is used.

\begin{tabular}{|r|l|}

\multicolumn{1}{|l}{Definitionnbsp;NSM}
SynopsisNull space is solution set of homogeneous system
ExampleGeneral solution sets described by Theoremnbsp;PSPHS

\multicolumn{1}{|l}{Theoremnbsp;SLSLC}
SynopsisSolutions for linear combinations with unknown scalars
ExampleDeciding membership in spans

\multicolumn{1}{|l}{Theoremnbsp;SLEMM}
SynopsisSystem of equations represented by matrix-vector product
ExampleSolution to $\linearsystem{A}{\vect{b}}$ is $\inverse{A}\vect{b}$ when $A$ is nonsingular

\multicolumn{1}{|l}{Theoremnbsp;CSCS}
SynopsisColumn space vectors create consistent systems
ExampleDeciding membership in column spaces

## Column Space Spanned by Original Columns

So we have a foolproof, automated procedure for determining membership in $\csp{A}$. While this works just fine a vector at a time, we would like to have a more useful description of the set $\csp{A}$ as a whole. The next example will preview the first of two fundamental results about the column space of a matrix.

Example CSTW: Column space, two ways.

We will now formalize the previous example, which will make it trivial to determine a linearly independent set of vectors that will span the column space of a matrix, and is constituted of just columns of $A$.

Theorem BCS (Basis of the Column Space) Suppose that $A$ is an $m\times n$ matrix with columns $\vectorlist{A}{n}$, and $B$ is a row-equivalent matrix in reduced row-echelon form with $r$ nonzero rows. Let $D=\{d_1,\,d_2,\,d_3,\,\ldots,\,d_r\}$ be the set of column indices where $B$ has leading 1's. Let $T=\set{\vect{A}_{d_1},\,\vect{A}_{d_2},\,\vect{A}_{d_3},\,\ldots,\,\vect{A}_{d_r}}$. Then

1. $T$ is a linearly independent set.
2. $\csp{A}=\spn{T}$.

Proof.

This is a nice result since it gives us a handful of vectors that describe the entire column space (through the span), and we believe this set is as small as possible because we cannot create any more relations of linear dependence to trim it down further. Furthermore, we defined the column space (Definition CSM) as all linear combinations of the columns of the matrix, and the elements of the set $S$ are still columns of the matrix (we won't be so lucky in the next two constructions of the column space).

Procedurally this theorem is extremely easy to apply. Row-reduce the original matrix, identify $r$ columns with leading 1's in this reduced matrix, and grab the corresponding columns of the original matrix. But it is still important to study the proof of Theorem BS and its motivation in Example COV which lie at the root of this theorem. We'll trot through an example all the same.

Example CSOCD: Column space, original columns, Archetype D.

## Column Space of a Nonsingular Matrix

Let's specialize to square matrices and contrast the column spaces of the coefficient matrices in Archetype A and Archetype B.

Example CSAA: Column space of Archetype A.

Example CSAB: Column space of Archetype B.

Example CSAA and Example CSAB together motivate the following equivalence, which says that nonsingular matrices have column spaces that are as big as possible.

Theorem CSNM (Column Space of a Nonsingular Matrix) Suppose $A$ is a square matrix of size $n$. Then $A$ is nonsingular if and only if $\csp{A}=\complex{n}$.

Proof.

With this equivalence for nonsingular matrices we can update our list, Theorem NME3.

Theorem NME4 (Nonsingular Matrix Equivalences, Round 4) Suppose that $A$ is a square matrix of size $n$. The following are equivalent.

1. $A$ is nonsingular.
2. $A$ row-reduces to the identity matrix.
3. The null space of $A$ contains only the zero vector, $\nsp{A}=\set{\zerovector}$.
4. The linear system $\linearsystem{A}{\vect{b}}$ has a unique solution for every possible choice of $\vect{b}$.
5. The columns of $A$ are a linearly independent set.
6. $A$ is invertible.
7. The column space of $A$ is $\complex{n}$, $\csp{A}=\complex{n}$.

Proof.

## Row Space of a Matrix

The rows of a matrix can be viewed as vectors, since they are just lists of numbers, arranged horizontally. So we will transpose a matrix, turning rows into columns, so we can then manipulate rows as column vectors. As a result we will be able to make some new connections between row operations and solutions to systems of equations. OK, here is the second primary definition of this section.

Definition RSM (Row Space of a Matrix) Suppose $A$ is an $m\times n$ matrix. Then the row space of $A$, $\rsp{A}$, is the column space of $\transpose{A}$, i.e. $\rsp{A}=\csp{\transpose{A}}$.

Informally, the row space is the set of all linear combinations of the rows of $A$. However, we write the rows as column vectors, thus the necessity of using the transpose to make the rows into columns. Additionally, with the row space defined in terms of the column space, all of the previous results of this section can be applied to row spaces.

Notice that if $A$ is a rectangular $m\times n$ matrix, then $\csp{A}\subseteq\complex{m}$, while $\rsp{A}\subseteq\complex{n}$ and the two sets are not comparable since they do not even hold objects of the same type. However, when $A$ is square of size $n$, both $\csp{A}$ and $\rsp{A}$ are subsets of $\complex{n}$, though usually the sets will not be equal (but see exercise CRS.M20).

Example RSAI: Row space of Archetype I.

The row space would not be too interesting if it was simply the column space of the transpose. However, when we do row operations on a matrix we have no effect on the many linear combinations that can be formed with the rows of the matrix. This is stated more carefully in the following theorem.

Theorem REMRS (Row-Equivalent Matrices have equal Row Spaces) Suppose $A$ and $B$ are row-equivalent matrices. Then $\rsp{A}=\rsp{B}$.

Proof.

Example RSREM: Row spaces of two row-equivalent matrices.

Theorem REMRS is at its best when one of the row-equivalent matrices is in reduced row-echelon form. The vectors that correspond to the zero rows can be ignored. (Who needs the zero vector when building a span? See exercise LI.T10.) The echelon pattern insures that the nonzero rows yield vectors that are linearly independent. Here's the theorem.

Theorem BRS (Basis for the Row Space) Suppose that $A$ is a matrix and $B$ is a row-equivalent matrix in reduced row-echelon form. Let $S$ be the set of nonzero columns of $\transpose{B}$. Then

1. $\rsp{A}=\spn{S}$.
2. $S$ is a linearly independent set.

Proof.

Example IAS: Improving a span.

Theorem BRS and the techniques of Example IAS will provide yet another description of the column space of a matrix. First we state a triviality as a theorem, so we can reference it later.

Theorem CSRST (Column Space, Row Space, Transpose) Suppose $A$ is a matrix. Then $\csp{A}=\rsp{\transpose{A}}$.

Proof.

So to find another expression for the column space of a matrix, build its transpose, row-reduce it, toss out the zero rows, and convert the nonzero rows to column vectors to yield an improved set for the span construction. We'll do Archetype I, then you do Archetype J.

Example CSROI: Column space from row operations, Archetype I.
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 94, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 20, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.8716292381286621, "perplexity_flag": "head"}