Norms of Vectors

A vector norm is a single number which represents the size or `length' of a vector. For example, if the exact solution to a problem is a vector $\mbox{\boldmath$\space x $ } \in \hbox{{\sf I}\kern-.4em\hbox{\sf C}}^n$, but when we evaluate it we obtain the vector $\mbox{\boldmath$\space x $ } +\mbox{\boldmath$\space \delta $ } \mbox{\boldmath$\space x $ }$, then the error vector is $\mbox{\boldmath$\space \delta $ } \mbox{\boldmath$\space x $ }$. We want a single number to represent the size of this vector; such a number is $\left\Vert \kern.05em \mbox{\boldmath$\space \delta $ } \mbox{\boldmath$\space x $ } \kern.05em \right\Vert$. There are many different norms. In $\hbox{{\sf I}\kern-.15em\hbox{\sf R}}^3$, for example, we can use the Euclidean norm $\left\Vert \kern.05em \mbox{\boldmath$\space \delta $ } \mbox{\boldmath$\space x $ } \kern.05em \right\Vert _2=(\delta x_1^2+\delta x_2^2+\delta x_3^2)^{1/2}$, where $\delta x_1, \delta x_2, \delta x_3$ are the components of $\mbox{\boldmath$\space \delta $ } \mbox{\boldmath$\space x $ }$. This of course corresponds to the geometric length of the vector.

Although only three vector norms are used, it is worth giving the axiomatic definition, since this is a starting point for matrix norms.

Definition 2.1.1   A vector norm $\left\Vert \kern.05em . \kern.05em \right\Vert$ on $\hbox{{\sf I}\kern-.4em\hbox{\sf C}}^n$ is a mapping of $\hbox{{\sf I}\kern-.4em\hbox{\sf C}}^n$ into $\hbox{{\sf I}\kern-.15em\hbox{\sf R}}$ with the properties:

(i)

$\left\Vert \kern.05em \mbox{\boldmath$\space x $ } \kern.05em \right\Vert\geq 0... ...e x $ } \kern.05em \right\Vert=0\iff \mbox{\boldmath$\space x $ } = \mbox{\bf0}$,

(ii)

$\left\Vert \kern.05em \alpha \mbox{\boldmath$\space x $ } \kern.05em \right\Ver... ...f C}},~ \mbox{\boldmath$\space x $ } \in \hbox{{\sf I}\kern-.4em\hbox{\sf C}}^n$,

(iii)

$\left\Vert \kern.05em \mbox{\boldmath$\space x $ } +\mbox{\boldmath$\space y $ ... ... x $ } ,\mbox{\boldmath$\space y $ } \in \hbox{{\sf I}\kern-.4em\hbox{\sf C}}^n$.

These properties ensure that the norm is a length; (i) says that length is non-negative, (ii) gives scaling of length, while (iii) is the triangle inequality.

In Linear Algebra, we defined a vector norm from an inner product $\left\Vert \kern.05em \mbox{\boldmath$\space x $ } \kern.05em \right\Vert=\sqrt... ...\mbox{\boldmath$\space x $ } ,\mbox{\boldmath$ x $ } \kern.05em \right\rangle }$. It is easy to check that the norm so defined satisfies the axioms above. Here we shall use special cases $p=1,2,\infty$ of the Hölder norm

$$\left\Vert \kern.05em \mbox{\boldmath$ x $ } \kern.05em \right\Vert _p=\left\{ \sum _{i=1}^n\vert x_i\vert^p\right\} ^{1/p}.$$

$\left\Vert \kern.05em . \kern.05em \right\Vert _1$ and $\left\Vert \kern.05em . \kern.05em \right\Vert _2$, the Euclidean norm, are self-evident. Writing $\left\Vert \kern.05em \mbox{\boldmath$\space x $ } \kern.05em \right\Vert _p=k\left\{ \sum _{i=1}^n\left\vert \frac{x_i}{k}\right\vert^p\right\} ^{1/p}$, where $k=\stackrel{\max }{_i}\vert x_i\vert$, letting $p\to \infty$ and noting that the quantity we are taking the pth root of is bounded by n, we find that the norm is just k. Thus $\left\Vert \kern.05em \mbox{\boldmath$\space x $ } \kern.05em \right\Vert _{\infty }=\stackrel{\max }{_i}\vert x_i\vert$.

It is straightforward to show that each of these is indeed a vector norm (indeed, it can be shown for general p). Only the third property presents any difficulty. For example,

$$\begin{eqnarray*}\left\Vert \kern.05em \mbox{\boldmath$ x $ } +\mbox{\boldmath$ ... ...n.05em \mbox{\boldmath$ y $ } \kern.05em \right\Vert _{\infty }. \end{eqnarray*}$$

The result for $\left\Vert \kern.05em . \kern.05em \right\Vert _2$ was proved in Linear Algebra last year.

There are a few analytic results we shall need:

1.

$\left\Vert \kern.05em \mbox{\boldmath$\space x $ } \kern.05em \right\Vert$ is a continuous function of the components of $\mbox{\boldmath$\space x $ }$.

2.

A continuous function on a compact set attains its maximum and minimum values on the set.

3.

For each pair of vector norms $\left\Vert \kern.05em . \kern.05em \right\Vert,\left\Vert \kern.05em . \kern.05em \right\Vert'$ there exist positive numbers m,M such that for all $\mbox{\boldmath$\space x $ } \in \hbox{{\sf I}\kern-.4em\hbox{\sf C}}^n$

$$m\left\Vert \kern.05em \mbox{\boldmath$ x $ } \kern.05em \rig... ...ert \kern.05em \mbox{\boldmath$ x $ } \kern.05em \right\Vert'.$$

The last property is referred to as the equivalence of norms.