Change of Variables in Quadratic Forms vs. Eigenvalue Decomposition

Question

Hey everyone! 👋 Struggling to wrap your head around change of variables in quadratic forms and eigenvalue decomposition? 🤔 Don't worry, you're not alone! I'm going to break down the key differences in a way that actually makes sense. Let's dive in!

william.johnston · Accepted Answer

📚 Change of Variables in Quadratic Forms vs. Eigenvalue Decomposition
Let's clarify what we mean by A and B in this context:

🧮 Definition of A: A real symmetric matrix representing the quadratic form.
 📈 Definition of B: In the context of change of variables, B represents the matrix used to transform the original variables into new variables ($x = By$).

📊 Comparison Table

Feature
 Change of Variables in Quadratic Forms
 Eigenvalue Decomposition

Goal
 Simplify a quadratic form by eliminating cross-product terms.
 Decompose a matrix into its eigenvalues and eigenvectors.

Matrix Requirements
 Real symmetric matrix.
 Square matrix (often symmetric, but not always required).

Transformation Matrix (B)
 Chosen to orthogonalize the quadratic form. Often involves completing the square or other algebraic manipulations to find a suitable $B$ such that $x = By$. The choice of $B$ is not unique.
 Formed by the eigenvectors of the matrix. This matrix (often denoted as $P$) diagonalizes the original matrix such that $A = PDP^{-1}$. Here, $D$ is a diagonal matrix with eigenvalues on the diagonal.

Result
 A simplified quadratic form with only squared terms.  For example, transforming $ax^2 + bxy + cy^2$ into $a'u^2 + c'v^2$ where $x, y$ are transformed into $u, v$.
 Expressing the original matrix as a product of matrices involving its eigenvalues and eigenvectors ($A = PDP^{-1}$).

Orthogonality
 The transformation matrix is chosen, but it doesn't necessarily *have* to be orthogonal, though using orthogonal transformations can simplify calculations and preserve geometric properties.
 The matrix formed by eigenvectors is orthogonal if the original matrix is symmetric.

Uniqueness
  The transformation matrix $B$ isn't unique.
  Eigenvalue decomposition is unique (up to the order and sign of eigenvectors).

Formulaic Representation
 $x = By$, where $x$ is the original variable vector, $y$ is the new variable vector, and $B$ is the transformation matrix.  The quadratic form $x^T A x$ becomes $y^T (B^T A B) y$.  The goal is to find a $B$ such that $B^T A B$ is a diagonal matrix.
 $A = PDP^{-1}$, where $A$ is the original matrix, $P$ is the matrix of eigenvectors, and $D$ is the diagonal matrix of eigenvalues. If A is symmetric, then $P^{-1} = P^T$.

🔑 Key Takeaways

🎯 Different Goals: Change of variables aims to simplify a quadratic form, while eigenvalue decomposition aims to decompose a matrix.
 🏗️ Matrix Structure: Change of variables works primarily on symmetric matrices representing quadratic forms. Eigenvalue decomposition applies to broader classes of matrices (though it's particularly well-behaved for symmetric matrices).
 🔄 Transformation Matrix: In change of variables, the transformation matrix is chosen to eliminate cross-product terms. In eigenvalue decomposition, the transformation matrix is formed by the eigenvectors of the matrix.
  📐 Orthogonality: While change of variable transformations can be orthogonal, they don't *have* to be. The matrix of eigenvectors *is* orthogonal for symmetric matrices in eigenvalue decomposition.

Change of Variables in Quadratic Forms vs. Eigenvalue Decomposition

🚀 Can't Find Your Exact Topic?

1 Answers

📚 Change of Variables in Quadratic Forms vs. Eigenvalue Decomposition

📊 Comparison Table

🔑 Key Takeaways

Join the discussion

Feature	Change of Variables in Quadratic Forms	Eigenvalue Decomposition
Goal	Simplify a quadratic form by eliminating cross-product terms.	Decompose a matrix into its eigenvalues and eigenvectors.
Matrix Requirements	Real symmetric matrix.	Square matrix (often symmetric, but not always required).
Transformation Matrix (B)	Chosen to orthogonalize the quadratic form. Often involves completing the square or other algebraic manipulations to find a suitable $B$ such that $x = By$. The choice of $B$ is not unique.	Formed by the eigenvectors of the matrix. This matrix (often denoted as $P$) diagonalizes the original matrix such that $A = PDP^{-1}$. Here, $D$ is a diagonal matrix with eigenvalues on the diagonal.
Result	A simplified quadratic form with only squared terms. For example, transforming $ax^2 + bxy + cy^2$ into $a'u^2 + c'v^2$ where $x, y$ are transformed into $u, v$.	Expressing the original matrix as a product of matrices involving its eigenvalues and eigenvectors ($A = PDP^{-1}$).
Orthogonality	The transformation matrix is chosen, but it doesn't necessarily have to be orthogonal, though using orthogonal transformations can simplify calculations and preserve geometric properties.	The matrix formed by eigenvectors is orthogonal if the original matrix is symmetric.
Uniqueness	The transformation matrix $B$ isn't unique.	Eigenvalue decomposition is unique (up to the order and sign of eigenvectors).
Formulaic Representation	$x = By$, where $x$ is the original variable vector, $y$ is the new variable vector, and $B$ is the transformation matrix. The quadratic form $x^T A x$ becomes $y^T (B^T A B) y$. The goal is to find a $B$ such that $B^T A B$ is a diagonal matrix.	$A = PDP^{-1}$, where $A$ is the original matrix, $P$ is the matrix of eigenvectors, and $D$ is the diagonal matrix of eigenvalues. If A is symmetric, then $P^{-1} = P^T$.