📚 Understanding Positive Semi-Definite vs. Positive Definite Quadratic Forms
Let's break down the key differences between positive semi-definite and positive definite quadratic forms. At first, it might seem a bit abstract, but once you grasp the definitions and how they relate to eigenvalues, it becomes much clearer!
Definition of a Positive Definite Quadratic Form
A symmetric matrix $A$ is positive definite if $x^TAx > 0$ for all non-zero vectors $x$. This means that no matter what vector $x$ you plug in (as long as it's not the zero vector), the result will always be a positive number. Importantly, all eigenvalues of a positive definite matrix are strictly positive.
Definition of a Positive Semi-Definite Quadratic Form
A symmetric matrix $A$ is positive semi-definite if $x^TAx \geq 0$ for all non-zero vectors $x$. The key difference here is that the result can be zero for some non-zero vectors $x$. This also means that all eigenvalues of a positive semi-definite matrix are non-negative (i.e., they can be zero or positive).
📊 Key Differences: A Side-by-Side Comparison
| Feature |
Positive Definite |
Positive Semi-Definite |
| Definition |
$x^TAx > 0$ for all $x \neq 0$ |
$x^TAx \geq 0$ for all $x \neq 0$ |
| Eigenvalues |
All eigenvalues are strictly positive ($> 0$) |
All eigenvalues are non-negative ($\geq 0$) |
| Zero Result |
$x^TAx = 0$ only when $x = 0$ |
$x^TAx = 0$ for some $x \neq 0$ |
| Geometric Interpretation |
Represents a strictly convex paraboloid. |
Represents a convex paraboloid (can be "flat" in some directions). |
| Invertibility |
Always invertible |
May not be invertible (if an eigenvalue is 0) |
🔑 Key Takeaways
- ➕ Positivity: Positive definite means strictly positive results for non-zero vectors, while positive semi-definite allows for zero results.
- 🔢 Eigenvalues: Positive definite matrices have all positive eigenvalues. Positive semi-definite matrices have all non-negative eigenvalues. This is super important for checking if a matrix is one or the other!
- 📉 Zero Vector: $x^TAx = 0$ only when $x$ is the zero vector for a positive definite matrix. For positive semi-definite matrices, you can find non-zero vectors where $x^TAx = 0$.
- 🔄 Invertibility: If a matrix is positive definite, it's always invertible. If it's positive semi-definite, it might not be invertible (specifically, if any eigenvalue is zero).
