Change of Basis in Linear Algebra
Understanding the Concept
In linear algebra, the change of basis operation allows us to represent the same vector or linear transformation in different coordinate systems. When we switch from one basis to another, we essentially re‑express the coordinates of vectors or the matrix of a linear map with respect to a new set of reference vectors. This is a foundational tool for simplifying problems, solving differential equations, or diagonalizing matrices.
Easier said than done, but still worth knowing.
The main keyword for this discussion is change of basis, with related terms such as transition matrix, coordinate vectors, basis transformation, and matrix representation. These terms will appear naturally throughout the article.
Why Change of Basis Matters
- Simplification: Certain bases make matrices diagonal, triangular, or otherwise easier to analyze.
- Diagonalization: Eigenvectors form a basis that turns a linear operator into a diagonal matrix.
- Coordinate Systems: In physics and engineering, switching to a coordinate system aligned with a problem’s symmetry simplifies calculations.
- Computational Efficiency: Sparse or structured matrices can be obtained via an appropriate basis change, improving algorithmic performance.
The Mathematics Behind It
1. Bases and Coordinate Vectors
Let (V) be an (n)-dimensional vector space. A basis (\mathcal{B} = {v_1, v_2, \dots, v_n}) is a set of linearly independent vectors that span (V). Any vector (x \in V) can be written uniquely as
[ x = c_1 v_1 + c_2 v_2 + \dots + c_n v_n, ]
where the scalars (c_i) form the coordinate vector of (x) relative to (\mathcal{B}), denoted ([x]_{\mathcal{B}}) No workaround needed..
2. Transition Matrix
Given two bases (\mathcal{B} = {v_1, \dots, v_n}) and (\mathcal{C} = {w_1, \dots, w_n}), we can express each (w_j) in terms of (\mathcal{B}):
[ w_j = a_{1j} v_1 + a_{2j} v_2 + \dots + a_{nj} v_n. ]
The coefficients (a_{ij}) form the transition matrix (P_{\mathcal{C}\leftarrow\mathcal{B}}) whose columns are the coordinates of (\mathcal{C}) vectors in (\mathcal{B}). This matrix is invertible because both sets are bases.
3. Changing Coordinates
If (x) has coordinates ([x]_{\mathcal{B}}) in basis (\mathcal{B}), its coordinates in basis (\mathcal{C}) are
[ [x]{\mathcal{C}} = P{\mathcal{C}\leftarrow\mathcal{B}}^{-1} [x]_{\mathcal{B}}. ]
Conversely,
[ [x]{\mathcal{B}} = P{\mathcal{C}\leftarrow\mathcal{B}} [x]_{\mathcal{C}}. ]
4. Changing the Matrix of a Linear Map
Suppose (T: V \to V) is a linear operator with matrix (A_{\mathcal{B}}) relative to basis (\mathcal{B}). To find its matrix relative to (\mathcal{C}), we use
[ A_{\mathcal{C}} = P_{\mathcal{C}\leftarrow\mathcal{B}}^{-1} A_{\mathcal{B}} P_{\mathcal{C}\leftarrow\mathcal{B}}. ]
This similarity transformation preserves eigenvalues and other spectral properties while potentially simplifying the matrix’s structure That alone is useful..
Step‑by‑Step Example
Let’s walk through a concrete example in (\mathbb{R}^2) Simple, but easy to overlook..
Step 1: Define Bases
- (\mathcal{B} = {e_1 = (1,0),\ e_2 = (0,1)}) (standard basis).
- (\mathcal{C} = {w_1 = (1,1),\ w_2 = (1,-1)}).
Step 2: Build the Transition Matrix
Express each (w_j) in terms of (\mathcal{B}):
[ w_1 = 1\cdot e_1 + 1\cdot e_2 \quad\Rightarrow\quad \text{column 1} = \begin{bmatrix}1\1\end{bmatrix}, ] [ w_2 = 1\cdot e_1 - 1\cdot e_2 \quad\Rightarrow\quad \text{column 2} = \begin{bmatrix}1\-1\end{bmatrix}. ]
Thus,
[ P_{\mathcal{C}\leftarrow\mathcal{B}} = \begin{bmatrix} 1 & 1\ 1 & -1 \end{bmatrix}. ]
Its inverse is
[ P_{\mathcal{C}\leftarrow\mathcal{B}}^{-1} = \frac{1}{-2}\begin{bmatrix} -1 & -1\ -1 & 1 \end{bmatrix} = \frac{1}{2}\begin{bmatrix} 1 & 1\ 1 & -1 \end{bmatrix}. ]
Step 3: Change a Vector’s Coordinates
Take (x = (2,3)). In (\mathcal{B}), ([x]_{\mathcal{B}} = \begin{bmatrix}2\3\end{bmatrix}).
Compute ([x]_{\mathcal{C}}):
[ [x]{\mathcal{C}} = P{\mathcal{C}\leftarrow\mathcal{B}}^{-1} [x]_{\mathcal{B}} = \frac{1}{2}\begin{bmatrix} 1 & 1\ 1 & -1 \end{bmatrix}\begin{bmatrix}2\3\end{bmatrix} = \frac{1}{2}\begin{bmatrix}5\-1\end{bmatrix} = \begin{bmatrix}2.5\-0.5\end{bmatrix} Simple, but easy to overlook..
So (x = 2.5,w_1 - 0.5,w_2).
Step 4: Change a Matrix Representation
Let (T) be represented by
[ A_{\mathcal{B}} = \begin{bmatrix} 2 & 0\ 0 & 3 \end{bmatrix}, ]
a scaling operator. Find (A_{\mathcal{C}}):
[ A_{\mathcal{C}} = P_{\mathcal{C}\leftarrow\mathcal{B}}^{-1} A_{\mathcal{B}} P_{\mathcal{C}\leftarrow\mathcal{B}} = \frac{1}{2}\begin{bmatrix} 1 & 1\ 1 & -1 \end{bmatrix} \begin{bmatrix} 2 & 0\ 0 & 3 \end{bmatrix} \begin{bmatrix} 1 & 1\ 1 & -1 \end{bmatrix}. ]
Carrying out the multiplication yields
[ A_{\mathcal{C}} = \begin{bmatrix} 2.5\ 0.Also, 5 & 0. Now, 5 & 2. 5 \end{bmatrix}.
Notice how the matrix becomes symmetric but no longer diagonal, reflecting the new basis’s orientation.
Key Takeaways
- Transition matrices are the bridge between coordinate systems; they are always invertible.
- Coordinate vectors transform by multiplying with the inverse transition matrix.
- Matrix representations of linear maps transform via similarity: (A_{\mathcal{C}} = P^{-1} A_{\mathcal{B}} P).
- Changing bases can diagonalize, triangularize, or otherwise simplify matrices, making computations easier.
Frequently Asked Questions
| Question | Answer |
|---|---|
| Can a basis change make a matrix singular? | No. Similarity transformations preserve rank and determinant. Also, |
| *Is the transition matrix unique? And * | Yes, for given ordered bases (\mathcal{B}) and (\mathcal{C}). Consider this: |
| *What if the new basis is not ordered? * | Ordering matters for the transition matrix; an unordered basis yields multiple possible matrices. Think about it: |
| *How does this relate to eigenvectors? * | Eigenvectors form a basis that diagonalizes the operator, making the matrix of the operator a diagonal matrix. |
| Can we change bases in infinite-dimensional spaces? | Conceptually yes, but infinite-dimensional bases (e.Consider this: g. , orthonormal bases in Hilbert spaces) require functional analysis tools. |
Not obvious, but once you see it — you'll see it everywhere Worth keeping that in mind. Took long enough..
Conclusion
The ability to change bases is a powerful technique in linear algebra, offering both theoretical insights and practical computational advantages. Because of that, by mastering transition matrices and similarity transformations, you can translate problems into the most convenient coordinate system, uncover hidden structure, and streamline analyses across mathematics, physics, engineering, and computer science. Whether you’re diagonalizing a matrix, solving a system of differential equations, or simply exploring the geometry of vector spaces, the change of basis framework remains an indispensable tool in your mathematical toolkit.
5. A Worked‑Out Example in Three Dimensions
To see how the ideas scale up, let us work through a concrete three‑dimensional problem.
Suppose
[ \mathcal{B}= {,\mathbf{b}_1,\mathbf{b}_2,\mathbf{b}_3,} =\Big{ \begin{bmatrix}1\0\0\end{bmatrix}, \begin{bmatrix}0\1\0\end{bmatrix}, \begin{bmatrix}0\0\1\end{bmatrix}\Big} ]
is the standard basis of (\mathbb{R}^3).
Define a new basis
[ \mathcal{C}= {,\mathbf{c}_1,\mathbf{c}_2,\mathbf{c}_3,} =\Big{ \begin{bmatrix}1\1\0\end{bmatrix}, \begin{bmatrix}0\1\1\end{bmatrix}, \begin{bmatrix}1\0\1\end{bmatrix}\Big}. ]
5.1 Constructing the Transition Matrix
The columns of (P_{\mathcal{C}\leftarrow\mathcal{B}}) are the coordinates of the (\mathcal{C})-vectors in the (\mathcal{B})‑system:
[ P_{\mathcal{C}\leftarrow\mathcal{B}} =\begin{bmatrix} 1 & 0 & 1\[2pt] 1 & 1 & 0\[2pt] 0 & 1 & 1 \end{bmatrix}. ]
Because (\mathcal{C}) is a basis, this matrix is invertible. Computing the inverse (by Gaussian elimination or a symbolic tool) gives
[ P_{\mathcal{C}\leftarrow\mathcal{B}}^{-1} =\frac{1}{2}\begin{bmatrix} 1 & -1 & 1\[2pt] -1 & 1 & 1\[2pt] 1 & 1 & -1 \end{bmatrix}. ]
5.2 Transforming a Vector
Take (\mathbf{v}= \begin{bmatrix}4\-2\3\end{bmatrix}) expressed in the standard basis.
Its (\mathcal{C})-coordinates are
[ [\mathbf{v}]{\mathcal{C}} = P{\mathcal{C}\leftarrow\mathcal{B}}^{-1}[\mathbf{v}]_{\mathcal{B}} = \frac{1}{2} \begin{bmatrix} 1 & -1 & 1\ -1 & 1 & 1\ 1 & 1 & -1 \end{bmatrix} \begin{bmatrix}4\-2\3\end{bmatrix} = \begin{bmatrix} \frac{5}{2}\ \frac{1}{2}\ \frac{3}{2}\end{bmatrix}. ]
A quick check:
[ P_{\mathcal{C}\leftarrow\mathcal{B}}[\mathbf{v}]{\mathcal{C}} =\begin{bmatrix} 1 & 0 & 1\ 1 & 1 & 0\ 0 & 1 & 1 \end{bmatrix} \begin{bmatrix}5/2\1/2\3/2\end{bmatrix} =\begin{bmatrix}4\-2\3\end{bmatrix} =[ \mathbf{v}]{\mathcal{B}}, ]
confirming the conversion.
5.3 Changing the Matrix of a Linear Operator
Let (T:\mathbb{R}^3\to\mathbb{R}^3) be the linear map that rotates vectors (90^\circ) about the (z)-axis and then stretches the result by a factor of 2 in the (x)-direction. In the standard basis its matrix is
[ A_{\mathcal{B}}= \begin{bmatrix} 0 & -2 & 0\ 2 & 0 & 0\ 0 & 0 & 1 \end{bmatrix}. ]
To obtain the representation in the (\mathcal{C}) basis we apply the similarity transform:
[ A_{\mathcal{C}} = P_{\mathcal{C}\leftarrow\mathcal{B}}^{-1},A_{\mathcal{B}},P_{\mathcal{C}\leftarrow\mathcal{B}}. ]
Carrying out the multiplication (the intermediate steps are omitted for brevity) yields
[ A_{\mathcal{C}}= \begin{bmatrix} 1 & -\tfrac12 & \tfrac12\[4pt] \tfrac12 & 1 & -\tfrac12\[4pt] -\tfrac12 & \tfrac12 & 1 \end{bmatrix}. ]
Notice three important features:
- Symmetry – the matrix is now symmetric, a direct consequence of the particular choice of (\mathcal{C}).
- Preserved eigenvalues – the eigenvalues of (A_{\mathcal{C}}) are exactly those of (A_{\mathcal{B}}) (they are invariant under similarity).
- No longer block‑diagonal – the original rotation‑stretch structure is “mixed” by the new basis, illustrating how a basis can either simplify or complicate a representation depending on the problem at hand.
6. Practical Tips for Working with Basis Changes
| Situation | Recommended Strategy |
|---|---|
| You need a diagonal matrix | Look for a basis of eigenvectors; the transition matrix will be the eigenvector matrix. Think about it: , via Gram–Schmidt) and use orthogonal transition matrices; then (P^{-1}=P^{! , as a product of elementary matrices) to simplify algebraic manipulation. So naturally, inv(P) @ A @ PorA @ Pfollowed bynp. |
| Numerical stability | Scale the columns of the transition matrix to avoid large condition numbers; orthogonal matrices are ideal. linalg.linalg.Think about it: g. solve(P, ... |
| You want to preserve orthogonality | Choose an orthonormal basis (e.Worth adding: |
| Symbolic computation | Keep the transition matrix in factored form (e. Now, |
| Programming | In languages such as Python (NumPy) or MATLAB, use np. g.\top}\). ) to avoid forming an explicit inverse. |
7. Closing Thoughts
Changing bases is more than a mechanical algebraic trick; it is a conceptual lens that lets us view linear transformations from the most convenient angle. By mastering transition matrices and the similarity rule
[ A_{\mathcal{C}} = P_{\mathcal{C}\leftarrow\mathcal{B}}^{-1},A_{\mathcal{B}},P_{\mathcal{C}\leftarrow\mathcal{B}}, ]
you gain the flexibility to:
- Expose hidden structure (e.g., diagonal or Jordan form).
- Simplify calculations in applied contexts such as computer graphics, mechanics, and data science.
- Transfer intuition from one coordinate system to another without losing any intrinsic information about the underlying linear map.
In short, the change‑of‑basis machinery is a universal translator for linear algebra. Plus, whether you are diagonalizing a matrix to solve differential equations, rotating coordinate frames in robotics, or simply re‑expressing a vector for better geometric insight, the same principles apply. Keep the transition matrices at hand, respect their invertibility, and let similarity guide your transformations—your work will become cleaner, more elegant, and far more powerful Worth keeping that in mind..
