POSITIVE-DEFINITE MATRIX

In linear algebra, a 'positive-definite matrix' is a Hermitian matrix which in many ways is analogous to a positive real number. The notion is closely related to a positive-definite symmetric bilinear form (or a sesquilinear form in the complex case).

Contents

Equivalent formulations

Quadratic forms

Negative-definite, semidefinite and indefinite matrices

Further properties

Non-Hermitian matrices

See also

References

Equivalent formulations

Let ''M'' be an ''n'' × ''n'' Hermitian matrix. Denote the transpose of a vector $a$ by $a^\{T\}$, and the conjugate transpose by $a^\{$
★ }.
The matrix ''M'' is ''positive definite'' if and only if it satisfies any of the following properties:
{| cellspacing="0" cellpadding="2"
|-
|valign="top"| '1.' || For all non-zero vectors ''z'' ∈ 'C'^''n'',
:$extbf\{z\}^\{$
★ } M extbf{z} > 0.
Note that the quantity $z^\{$
★ } M z is always real.
|-
|valign="top"| '2.' || All eigenvalues $lambda\_i$ of $M$ are positive. Recall that any Hermitian ''M'', by the spectral theorem, may be regarded as a ''real'' diagonal matrix ''D'' that has been re-expressed in some new coordinate system (i.e., $M\; =\; P^\{-1\}DP$ for some unitary matrix ''P'' whose rows are orthonormal eigenvectors of ''M'', forming a basis). So this characterization means that ''M'' is positive definite if and only if the diagonal elements of ''D'' (the eigenvalues) are all positive. In other words, in the basis consisting of the eigenvectors of ''M'', the action of ''M'' is component-wise multiplication with a (fixed) element in 'C'^''n'' with positive entries.
|-
|valign="top"| '3.' || The sesquilinear form
:$langle\; extbf\{x\},\; extbf\{y\}$
angle = extbf{x}^{
★ } M extbf{y}
defines an inner product on 'C'^''n''. (In fact, every inner product on 'C'^''n'' arises in this fashion from a Hermitian positive definite matrix.)
|-
|valign="top"| '4.' || ''M'' is the Gram matrix of some collection of linearly independent vectors.
:$extbf\{x\}\_1,ldots,\; extbf\{x\}\_n\; in\; mathbb\{C\}^k$
for some ''k''. More precisely, ''M'' arises by defining each entry
:$M\_\{ij\}\; =\; langle\; extbf\{x\}\_i,\; extbf\{x\}\_j$
angle = extbf{x}_i^{
★ } extbf{x}_j.
The vectors ''x_i'' may optionally be restricted to fall in 'C'^''n''. In other words, ''M'' is of the form ''A
★ A'' where ''A'' is not necessarily square but must be injective in general.
|-
|valign="top"| '5.' || All the following matrices have a positive determinant (the Sylvester criterion):

★ the upper left 1-by-1 corner of $M$

★ the upper left 2-by-2 corner of $M$

★ the upper left 3-by-3 corner of $M$

★ ...

★ $M$ itself
In other words, all of the leading principal minors are positive.
For positive semidefinite matrices, all principal minors have to be non-negative.
The leading principal minors alone do not imply positive semidefiniteness, as can be seen from the example
:
|}
These properties are equivalent: if a matrix satisfies one property, it satisfies them all.
For real symmetric matrices, these properties can be simplified by replacing $mathbb\{C\}^n$ with $mathbb\{R\}^n$, and "conjugate transpose" with "transpose."

Quadratic forms

Echoing condition 3 above, one can also formulate positive-definiteness in terms of quadratic forms. Let ''K'' be the field 'R' or 'C', and ''V'' be a vector space over ''K''. A Hermitian form
:$B\; :\; V\; imes\; V$
ightarrow K
is a bilinear map such that ''B''(''x'', ''y'') is always the complex conjugate of ''B''(''y'', ''x''). Such a function ''B'' is called ''positive definite'' if ''B''(''x'', ''x'') > 0 for every nonzero ''x'' in ''V''.

Negative-definite, semidefinite and indefinite matrices

The ''n'' × ''n'' Hermitian matrix $M$ is said to be 'negative-definite' if
:$x^\{$
★ } M x < 0,
for all non-zero $x\; in\; mathbb\{R\}^n$ (or, equivalently, all non-zero $x\; in\; mathbb\{C\}^n$). It is called 'positive-semidefinite' if
:$x^\{$
★ } M x geq 0
for all $x\; in\; mathbb\{R\}^n$ (or $mathbb\{C\}^n$) and 'negative-semidefinite' if
:$x^\{$
★ } M x leq 0
for all $x\; in\; mathbb\{R\}^n$ (or $mathbb\{C\}^n$).
Equivalently, a matrix is negative-definite if all its eigenvalues are negative, it is positive-semidefinite if they are all greater than or equal to zero, and it is negative-semidefinite if they are all less than or equal to zero.
A matrix ''M'' is positive-semidefinite if and only if it arises as the Gram matrix of some set of vectors. In contrast to the positive-definite case, these vectors need not be linearly independent.
For any matrix $A$, the matrix ''A''
^★ ''A'' is positive semidefinite, and rank($A$) = rank(''A''
^★ ''A''). Reversely, any positive semidefinite matrix $M$ can be written as ''M'' = ''A''
^★ ''A''; this is the Cholesky decomposition.
A Hermitian matrix which is neither positive- nor negative-semidefinite is called 'indefinite'.

Further properties

If $M$ is positive semi-definite, one sometimes writes $M\; geq\; 0$ and if $M$ is positive-definite one writes $M\; >\; 0$. This may be confusing, as sometimes nonnegative matrices are also denoted in this way. Our notion comes from functional analysis where positive definite matrices define positive operators.
For positive semi-definite matrices $M,N$ we write $Mgeq\; N$ if $M-N\; geq\; 0$, i.e. $M-N$ is positive semi-definite. Equivalently for $M>N$.
{| cellspacing="0" cellpadding="2"
|-
|valign="top"| '1.' ||
Every positive definite matrix is invertible and its inverse is also positive definite. If $M\; geq\; N\; >\; 0$ then $N^\{-1\}\; geq\; M^\{-1\}\; >\; 0$.
|-
|valign="top"| '2.' || If $M$ is positive definite and $r\; >\; 0$ is a real number, then $r\; M$ is positive definite.
If $M$ and $N$ are positive definite, then the sum $M\; +\; N$ and the products $MNM$ and $NMN$ are also positive definite. If $M\; N\; =\; N\; M$, then $M\; N$ is also positive definite.
|-
|valign="top"| '3.' || If $M=(m\_\{ij\})\; >\; 0$ then the diagonal entries $m\_\{ii\}$ are real and positive. As a consequence $ext\{tr\}(M)>0$. Furthermore
:.
|-
|valign="top"| '4.' || A matrix $M$ is positive definite, if and only if there is a positive definite matrix $B>0$ with $B^2\; =\; M$. One writes $B\; =\; M^\{1/2\}$. This matrix $B$ is unique (but only under the assumption $B>0$). If $M\; >\; N\; >\; 0$ then $M^\{1/2\}\; >\; N^\{1/2\}>0$.
|-
|valign="top"| '5.' || If $M,N\; >\; 0$ then $Motimes\; N\; >\; 0$. (Here $otimes$ denotes Kronecker product.)
|-
|valign="top"| '6.' || For matrices $M=(m\_\{ij\}),N=(n\_\{ij\})$ write $Mcirc\; N$ for the entry-wise product of $M$ and $N$, i.e. the matrix whose $i,j$ entry is $m\_\{ij\}\; n\_\{ij\}$. Then $M\; circ\; N$ is the Hadamard product of $M$ and $N$. If $M,N>0$ then $Mcirc\; N\; >\; 0$ and if $M,N$ are real matrices, the following inequality, due to Oppenheim, holds:
$det(Mcirc\; N)\; geq\; (det\; N)\; prod\_\{i\}\; m\_\{ii\}$
|-
|valign="top"| '7.' || Let $M\; >\; 0$ and $N$ hermitian. If $MN+NM\; geq\; 0$ ($MN+NM\; >\; 0$) then $Ngeq\; 0$ ( $N\; >\; 0$ ).
|-
|valign="top"| '8.' || If $M,Ngeq\; 0$ are real matrices then $ext\{tr\}(MN)geq\; 0$.
|-
|valign="top"| '9.' || If $M>0$ is real, then there is a $delta>0$ such that $Mgeq\; delta\; I$ where $I$ is the identity matrix.
|}

Non-Hermitian matrices

A real matrix ''M'' may have the property that ''x''^T''Mx'' > 0 for all nonzero real vectors ''x'' without being symmetric. The matrix
:
satisfies this property, because for all real vectors $x\; =\; (x\_1,\; x\_2)^T$ such that $x$
e 0,
:
In general, we have ''x''^T''Mx'' > 0 for all real nonzero vectors ''x'' if and only if the symmetric part, (''M'' + ''M''^T) / 2, is positive definite.
The situation for complex matrices may be different, depending on how one generalizes the inequality ''z''
^★ ''Mz'' > 0. If ''z''
^★ ''Mz'' is real for all complex vectors ''z'', then the matrix ''M'' is necessarily Hermitian. So, if we require that ''z''
^★ ''Mz'' be real and positive, then ''M'' is automatically Hermitian. On the other hand, we have that Re(''z''
^★ ''Mz'') > 0 for all complex nonzero vectors ''z'' if and only if the Hermitian part, (''M'' + ''M''
^★ ) / 2, is positive definite.
In summary, the distinguishing feature between the real and complex case is that, a bounded positive operator on a complex Hilbert space is necessarily Hermitian, or self adjoint. The general claim can be argued using the polarization identity. That is no longer true in the real case.
There is no agreement in the literature on the proper definition of ''positive-definite'' for non-Hermitian matrices.

See also

★ Square root of a matrix

★ Schur complement

★ Positive definite kernel

★ Positive-definite function

References

★ Roger A. Horn and Charles R. Johnson. ''Matrix Analysis,'' Chapter 7. Cambridge University Press, 1985. ISBN 0-521-30586-1 (hardback), ISBN 0-521-38632-2 (paperback).

★ Rajendra Bhatia. ''Positive definite matrices,''. Princeton Series in Applied Mathematics, 2007. ISBN 978-0691129181.