The Riesz Representation Theorem – Part 2

So I should explain: this is one of those weird weeks where I have actually finished all my homework for the week two days early, so I actually have some time to write on the blog! So, let’s talk about some more functional analysis. Last time we talked about $L(X,Y)$, the set of bounded linear maps $f:X\to Y$. In particular we defined $X^*$ to be the set of all bounded linear functionals on a normed space $X$. We showed that this is a normed space (the reader should verify that an equivalent definition is $||T||=sup\{||Tx||:||x||=1\}$) and I claimed that $X^*$ is always complete with respect to this norm. Let’s prove this:

Proposition: Let $X$ be a normed space. Then $X^*$ is always complete with respect to the norm given in the previous post.

Proof: Suppose $\{f_n\}\subset X^*$ is a Cauchy sequence. If $x\in X$, then the sequence $\{T_n(x)\}\subset\mathbb{C}$ satisfies $|T_n(x)-T_m(x)|=|(T_m-T_n)(x)|\leq||T_m-T_n||\cdot||x||\to0$ by assumption, and hence this sequence converges. Define $T(x)=lim_{n\to\infty} T_n(x)$. It is clear that $T$ is linear. Note that for all $x\in X$,

$|T(x)|=|lim_{n\to\infty} T_n(x)|=lim_{n\to\infty}|T_n(x)|$

But for each $n$ we have $|T_n(x)|\leq||T_n||\cdot||x||$. Now, by the reverse triangle inequality, $|\hspace{1mm}||T_n||-||T_m||\hspace{1mm}|\leq||T_n-T_m||$, so the numerical sequence $\{||T_n||\}$ is Cauchy, hence is (uniformly) bounded, say by $C$. Then for all $n$, we have $|T_n(x)|\leq C||x||$, and taking limits gives $|T(x)|\leq C||x||$, hence $T$ is bounded. Now given $\epsilon>0$, choose $N$ so that $m,n\geq N$ implies $||T_n-T_m||<\epsilon/2$. Then for all $x\in X$ with $||x||=1$, we have

$|T_n(x)-T_m(x)|\leq ||T_n-T_m||\cdot||x||=||T_n-T_m||<\epsilon/2$

But then taking $n$ to infinity, this shows that for all $x$ with $||x||=1$ we have $|T(x)-T_m(x)|\leq \epsilon/2$, and taking the supremum over such $x$ preserves the inequality, thus for $m\geq N$, we have $||T-T_n||<\epsilon$.

I presented this proof because I realize in the last post I didn’t prove much and I wanted to give you a feel for how to work with these operator norms. Note above that this proof carries through mutatis mutandis if $T\in L(X,Y)$ for any Banach space $Y$. Okay, let’s shift gears and talk about inner products. Recall that an inner product on a vector space $V$ is a map $\langle\cdot,\cdot\rangle:V\times V\to\mathbb{C}$ so that 1) $\langle x,x\rangle\geq 0$ for all $x\in V$ with equality if and only if $x=0$; 2) If $x,y,z\in V$, $\lambda\in\mathbb{C}$, then $\langle x+\lambda y,z\rangle=\langle x,z\rangle+\lambda\langle y,z\rangle$, and 3) $\langle y,x\rangle=\overline{\langle x,y\rangle}$. There are several familiar facts from finite dimensional linear algebra whose proofs carry over almost verbatim to the arbitrary inner product space setting, so I will list them here without proof. Let $(H,\langle\cdot,\cdot\rangle)$ be an inner product space, then

Facts

1. The Cauchy-Bunyakovsky-Schwartz Inequality: For all $x,y\in H$, $|\langle x,y\rangle|\leq ||x||\cdot||y||$ with equality if and only if $x=\lambda y$ for some $\lambda\in\mathbb{C}$, where $||x||=\langle x,x\rangle^{1/2}$.

2. The map $x\mapsto||x||$ is a norm on $H$. Unless stated otherwise, we always assume $H$ to be equipped with this norm.

3. The Parallelogram Law (and its converse): For all $x,y\in H$, we have

$||x+y||^2+||x-y||^2=2(||x||^2+||y||^2)$.

Moreover, if $(X,||\cdot||_p)$ is any normed space such that $||\cdot||_p$ satisfies this expression, then there is an inner product $\langle\cdot,\cdot\rangle_p$ so that $\langle x,x\rangle_p^{1/2}=||x||_p$ for all $x\in X$. For a detailed explanation of the second result, check out this Wikipedia article.

4. The Pythagorean Theorem: Let $\{x_i\}_1^n\subseteq H$ be an orthogonal set of vectors. That is, we have $\langle x_i,x_j\rangle=0$ if $i\neq j$. Then $||\sum_1^n x_j||^2=\sum_1^n||x_i||^2$.

5. Let $E\subseteq H$ be a subset. We say vectors $x,y\in H$ are orthogonal and write $x\perp y$ if $\langle x,y\rangle=0$. We define $E^\perp=\{x\in H:x\perp y\text{ for all }y\in E\}$. Then $E^\perp$ is a closed subspace of $H$. If $E$ is a closed subspace, then $E^\perp$ is called the orthogonal complement of $E$

This last definition is very important. In some sense this is what will allow us to prove the Riesz Representation Theorem. In fact, we have the following

Lemma/Proposition/Theorem: Let $M$ be a closed subspace of a Hilbert space $H$. Then $H=E\oplus E^\perp$. Moreover, if we write $x=y+z$ with $y\in M$, $z\in M^\perp$, then $y,z$ are the unique elements of $M,M^\perp$ whose distance to $x$ is minimal.

Proof (sketch): Given $x\in H$, let $\delta=inf\{||x-y||:y\in M\}$ and let $y_n$ be a sequence in $M$ so that $||x-y_n||\to\delta$. Apply the parallelogram law to show that $\{y_n\}$ is a Cauchy sequence (you may need to use that $(y_n+y_m)/2\in M$) and let $y=lim_{n\to\infty}y_n$ and let $z=x-y$. Since $M$ is closed, we have $y\in M$ and $||x-y||=\delta$. To show that $z\in M^\perp$, let $u\in M$. By multiplying by an appropriate (complex) scalar, we may assume $\langle z,u\rangle\in\mathbb{R}$. Thus the function $f(t)=||z+tu||^2$ is real for all $t\in\mathbb{R}$. Elementary calculus shows that $f$ has a minimum at $t=0$, and so $f'(0)=2\langle z,u\rangle=0$, which shows $z\in M^\perp$. I’ll leave it to the reader to verify the uniqueness statements as well as the statement about distance to $M^\perp$.

Some remarks are in order. First, if $X$ is a Banach space and $M$ is a closed subspace of $X$, we say $M$ is complemented if there exists a closed subspace $N$ such that $X=M\oplus N$. The above theorem tells us that in a Hilbert space, every closed subspace is complemented. It is a non-trivial theorem (due to Lindenstrauss and Tzafriri) that if $X$ is a Banach such that every closed subspace of $X$ is complemented, then $X$ is isomorphic to a Hilbert space. The question of which subspaces of a Banach space are complemented is open.

Given a closed subspace $M\subseteq H$, we can define a map $P:X\to M$ as follows: given $x\in H$, write $x=y+z$ uniquely with $y\in M$ and $z\in M^\perp$. Then define $P(x)=y$. Note that $(I-P)(x)=z$, so $I-P:X\to M^\perp$. Moreover, we have $P^2=P$ and $(I-P)^2=I-P$. A map $T$ so that $T^2=T$ is called a projection. For the special case of $P$ above, $P$ is called the orthogonal projection onto $M$ and $I-P$ is the orthogonal projection onto $M^\perp$. It can be shown that in a Banach space, a subspace $M$ is complemented if and only there exists a continuous projection onto $M$. For more information on projections, check out this link.

Alright, enough asides! The real reason I presented the above theorem was to prove the following

Corollary: Riesz Representation Theorem: Let $H$ be a Hilbert space and $f\in H^*$. Then there is a unique $y_f\in H$ so that $f(x)=\langle x,y_f\rangle$ for all $x\in H$. Moreover, $||f||=||y_f||$.

Proof: For uniqueness, suppose $z$ is another vector satisfying the theorem. Then for all $x\in H$ we have

$\langle x,y_f-z\rangle=\langle x,y_f\rangle-\langle x,z\rangle=f(x)-f(x)=0$

and so $y_f-z\in H^\perp=\{0\}$, i.e. $y_f=z$. For existence, if $f=0$, then clearly $y=0$ works. Otherwise $\text{ker}(f)$ is a proper subspace of $H$, so its orthogonal complement contains a non-zero vector $z$ of norm one. Let $u=f(x)z-f(z)x$ and note that $f(u)=0$, so $u\in\text{ker}(f)$. Thus

$0=\langle u,z\rangle=f(x)||z||^2-f(z)\langle x,z\rangle=f(x)-\langle x,\overline{f(z)}z\rangle$

so letting $y_f=\overline{f(z)}z$ gives the existence. Finally, note that $|f(y_f)|=\langle y_f,y_f\rangle=||y_f||\cdot||y_f||$, so $||f||\geq||y_f||$, and by Cauchy-Schwartz, for all $x\in H$ we have $|f(x)|=\langle x,y_f\rangle|\leq||y||\cdot||x||$, so $||f||\leq||y_f||$, which completes the proof.

So what does this tell us? Well, note that if $y\in H$, then the map $f_y(x):=\langle x,y\rangle$ is a linear functional on $H$. It’s easy to see that $y\mapsto f_y$ is linear, and if $\langle x,y\rangle=0$ for all $x$, then $y=0$ by the same argument as above, so this embeds $H$ into $H^*$. What the above theorem tells us is that this correspondence is surjective, and so we have a conjugate-linear, isometric isomorphism between $H$ and $H^*$. Repeating the construction again, we see that $H\simeq H^{**}$. It’s natural to ask if there are other classes of spaces where we can write down such a relation. I’ll write about these in later posts, but I think the next few posts I write will give some examples of applications of the Riesz representation theorem and perhaps a few more facts about Hilbert spaces. For example, how many different Hilbert spaces are there? Well, if you haven’t seen the result before, the answer may surprise you! Unfortunately I’ll probably have to start working on homework again soon, so I don’t know when I’ll get to write again, but until then, enjoy!