MATRIX NORM

In mathematics, a 'matrix norm' is a natural extension of the notion of a vector norm to matrices.

Contents

Properties of matrix norm

Induced norm

"Entrywise" norms

Frobenius norm

Trace norm

Max norm

Consistent norms

Equivalence of norms

Examples of norm equivalence

References

Properties of matrix norm

In what follows, $K$ will denote the field of real or complex numbers. Consider the space $K^\{m\; imes\; n\}$ of all matrices with $m$ rows and $n$ columns with entries in $K.$
A matrix norm on $K^\{m\; imes\; n\}$ satisfies all the properties of vector norms. That is, if $|A|$ is the norm of the matrix $A$, then

★ $|A|ge\; 0$ with equality if and only if $A=0$

★ for all in $K$ and all matrices $A$ in $K^\{m\; imes\; n\}$

★ $|A+B|\; le\; |A|+|B|$ for all matrices $A$ and $B$ in $K^\{m\; imes\; n\}.$
Additionally, some matrix norms defined on ''n''-by-''n'' matrices (but not all such norms) satisfy one or more of the following conditions which relate to the fact that matrices are more than just vectors:

★ $|AB|\; le\; |A||B|$

★ $|I|=1$ where $I$ is the identity matrix

★ $|A|=|A^$
★ | where $A^$
★ is the conjugate transpose of $A$ (or simply the transpose, for real matrices)
A matrix norm that satisfies the first additional property is called a 'sub-multiplicative norm.' The set of all ''n''-by-''n'' matrices, together with such a sub-multiplicative norm, is an example of a Banach algebra.
(In some books the terminology ''matrix norm'' is used only for those norms which are sub-multiplicative.)

Induced norm

If vector norms on ''K''^''m'' and ''K''^''n'' are given (''K'' is field of real or complex numbers), then one defines the corresponding ''induced norm'' or ''operator norm'' on the space of ''m''-by-''n'' matrices as the following maxima:
:$|A|=max\{|Ax|\; :\; xin\; K^n\; mbox\{\; with\; \}|x|le\; 1\}$
::$=\; max\{|Ax|\; :\; xin\; K^n\; mbox\{\; with\; \}|x|\; =\; 1\}$
::
e 0
ight}.
If ''m'' = ''n'' and one uses the same norm on the domain and the range, then the induced operator norm is a sub-multiplicative matrix norm.
For example, the operator norm corresponding to the ''p''-norm for vectors is:
:$left\; |\; A$
ight | _p = max limits _{x
e 0} rac{left | A x
ight | _p}{left | x
ight | _p}.
In the case of $p=1$ and $p=infty$, the norms can be computed as:
:$left\; |\; A$
ight | _1 = max limits _{1 leq j leq n} sum _{i=1} ^m | a_{ij} |
:$left\; |\; A$
ight | _infty = max limits _{1 leq i leq m} sum _{j=1} ^n | a_{ij} | .
These are different from the Schatten ''p''-norms for matrices, which are also usually denoted
by $left\; |\; A$
ight | _p .
In the special case of ''p'' = 2 (the Euclidean norm) and ''m'' = ''n'' (square matrices), the induced matrix norm is the ''spectral norm''. The spectral norm of a matrix ''A'' is the largest singular value of ''A'' or the square root of the largest eigenvalue of the positive-semidefinite matrix ''A''
^★ ''A'':
:$left\; |\; A$
ight | _2=sqrt{lambda_{mbox{max}}(A^
★ A)}
where ''A''
^★ denotes the conjugate transpose of ''A''.
Any induced norm satisfies the inequality
:$left\; |\; A$
ight | ge
ho(A),
where ρ(''A'') is the spectral radius of ''A''. Furthermore, we have
: $lim\_\{r$
arrinfty}|A^r|^{1/r}=
ho(A).

"Entrywise" norms

These vector norms treat a matrix as an $m\; imes\; n$ vector, and use one of the familiar vector norms.
For example, using the ''p''-norm for vectors, we get:
:$Vert\; A\; Vert\_\{p\}\; =\; Big(\; sum\_\{i=1\}^m\; sum\_\{j=1\}^n\; |a\_\{ij\}|^p\; Big)^\{1/p\}.\; ,$

Frobenius norm

For ''p'' = 2, this is called the 'Frobenius norm' or the 'Hilbert-Schmidt norm', though the latter term is often reserved for operators on Hilbert space. This norm can be defined in various ways:
:
where ''A''
^★ denotes the conjugate transpose of ''A'', ''σ_i'' are the singular values of ''A'', and the trace function is used. The Frobenius norm is very similar to the Euclidean norm on ''K''^''n'' and comes from an inner product on the space of all matrices.
The Frobenius norm is submultiplicative and is very useful for numerical linear algebra. This norm is often easier to compute than induced norms.

Trace norm

The 'trace norm' is defined as
:$|A|\_\{tr\}$
=operatorname{trace}(sqrt{A^
★ A})=sum_{i=1}^{min{m,,n}} sigma_{i}.
Clearly, for all $A\; in\; K^\{m\; imes\; n\}$ we have $|A|\_\{tr\}\; ge\; |A|\_\{F\}.$

Max norm

The 'max norm' is defined as $|A|\_\{max\}=max\{|a\_\{ij\}|\}.$

Consistent norms

A matrix norm $|\; cdot\; |\_\{ab\}$ on $K^\{m\; imes\; n\}$ is called consistent with a vector norm $|\; cdot\; |\_\{a\}$ on $K^n$ and a vector norm $|\; cdot\; |\_\{b\}$ on $K^m$ if:
:$|Ax|\_b\; leq\; |A|\_\{ab\}\; |x|\_a$
for all $A\; in\; K^\{m\; imes\; n\},\; x\; in\; K^n$. All induced norms are consistent by definition.

Equivalence of norms

For any two vector norms ||·||_α and ||·||_β, we have
:$rleft|A$
ight|_lphaleqleft|A
ight|_etaleq sleft|A
ight|_lpha
for some positive numbers ''r'' and ''s'', for all matrices ''A'' in $K^\{m\; imes\; n\}$. In other words, they are ''equivalent norms''; they induce the same topology on $K^\{m\; imes\; n\}$.
Moreover, when $Ain\; mathbb\{R\}^\{n\; imes\; n\}$, then for any vector norm
||·||, there exists a unique positive number ''k'' such that ''k||A||'' is a (submultiplicative) matrix norm.
A matrix norm ''||·||_p'' is said to be ''minimal'' if there exists no other matrix norm ''||·||_q'' satisfying ''||·||_q≤||·||_p'' for all ''||·||_q''.

Examples of norm equivalence

For matrix $Ainmathbb\{R\}^\{m\; imes\; n\}$ the following inequalities hold:

★ $|A|\_2le|A|\_Flesqrt\{n\}|A|\_2$^[1]

★ $|A|\_\{max\}\; le|A|\_2lesqrt\{mn\}|A|\_\{max\}$

★

★

★ $|A|\_2lesqrt\{|A|\_1|A|\_infty\}$

References

★ Roger Horn and Charles Johnson. ''Matrix Analysis,'' Chapter 5, Cambridge University Press, 1985. ISBN 0-521-38632-2.

★ L. Thomas, Norms and Condition Numbers of a Matrix [1]

★ James W. Demmel, Applied Numerical Linear Algebra, section 1.7, published by SIAM, 1997.