The Riesz Representation Theorem – Part 2

So I should explain: this is one of those weird weeks where I have actually finished all my homework for the week two days early, so I actually have some time to write on the blog! So, let’s talk about some more functional analysis. Last time we talked about L(X,Y), the set of bounded linear maps f:X\to Y. In particular we defined X^* to be the set of all bounded linear functionals on a normed space X. We showed that this is a normed space (the reader should verify that an equivalent definition is ||T||=sup\{||Tx||:||x||=1\}) and I claimed that X^* is always complete with respect to this norm. Let’s prove this:

Proposition: Let X be a normed space. Then X^* is always complete with respect to the norm given in the previous post.

Proof: Suppose \{f_n\}\subset X^* is a Cauchy sequence. If x\in X, then the sequence \{T_n(x)\}\subset\mathbb{C} satisfies |T_n(x)-T_m(x)|=|(T_m-T_n)(x)|\leq||T_m-T_n||\cdot||x||\to0 by assumption, and hence this sequence converges. Define T(x)=lim_{n\to\infty} T_n(x). It is clear that T is linear. Note that for all x\in X,

|T(x)|=|lim_{n\to\infty} T_n(x)|=lim_{n\to\infty}|T_n(x)|

But for each n we have |T_n(x)|\leq||T_n||\cdot||x||. Now, by the reverse triangle inequality, |\hspace{1mm}||T_n||-||T_m||\hspace{1mm}|\leq||T_n-T_m||, so the numerical sequence \{||T_n||\} is Cauchy, hence is (uniformly) bounded, say by C. Then for all n, we have |T_n(x)|\leq C||x||, and taking limits gives |T(x)|\leq C||x||, hence T is bounded. Now given \epsilon>0, choose N so that m,n\geq N implies ||T_n-T_m||<\epsilon/2. Then for all x\in X with ||x||=1, we have

|T_n(x)-T_m(x)|\leq ||T_n-T_m||\cdot||x||=||T_n-T_m||<\epsilon/2

But then taking n to infinity, this shows that for all x with ||x||=1 we have |T(x)-T_m(x)|\leq \epsilon/2, and taking the supremum over such x preserves the inequality, thus for m\geq N, we have ||T-T_n||<\epsilon.

I presented this proof because I realize in the last post I didn’t prove much and I wanted to give you a feel for how to work with these operator norms. Note above that this proof carries through mutatis mutandis if T\in L(X,Y) for any Banach space Y. Okay, let’s shift gears and talk about inner products. Recall that an inner product on a vector space V is a map \langle\cdot,\cdot\rangle:V\times V\to\mathbb{C} so that 1) \langle x,x\rangle\geq 0 for all x\in V with equality if and only if x=0; 2) If x,y,z\in V, \lambda\in\mathbb{C}, then \langle x+\lambda y,z\rangle=\langle x,z\rangle+\lambda\langle y,z\rangle, and 3) \langle y,x\rangle=\overline{\langle x,y\rangle}. There are several familiar facts from finite dimensional linear algebra whose proofs carry over almost verbatim to the arbitrary inner product space setting, so I will list them here without proof. Let (H,\langle\cdot,\cdot\rangle) be an inner product space, then


  1. The Cauchy-Bunyakovsky-Schwartz Inequality: For all x,y\in H, |\langle x,y\rangle|\leq ||x||\cdot||y|| with equality if and only if x=\lambda y for some \lambda\in\mathbb{C}, where ||x||=\langle x,x\rangle^{1/2}.

  2. The map x\mapsto||x|| is a norm on H. Unless stated otherwise, we always assume H to be equipped with this norm.

  3. The Parallelogram Law (and its converse): For all x,y\in H, we have


    Moreover, if (X,||\cdot||_p) is any normed space such that ||\cdot||_p satisfies this expression, then there is an inner product \langle\cdot,\cdot\rangle_p so that \langle x,x\rangle_p^{1/2}=||x||_p for all x\in X. For a detailed explanation of the second result, check out this Wikipedia article.

  4. The Pythagorean Theorem: Let \{x_i\}_1^n\subseteq H be an orthogonal set of vectors. That is, we have \langle x_i,x_j\rangle=0 if i\neq j. Then ||\sum_1^n x_j||^2=\sum_1^n||x_i||^2.

  5. Let E\subseteq H be a subset. We say vectors x,y\in H are orthogonal and write x\perp y if \langle x,y\rangle=0. We define E^\perp=\{x\in H:x\perp y\text{ for all }y\in E\}. Then E^\perp is a closed subspace of H. If E is a closed subspace, then E^\perp is called the orthogonal complement of E

This last definition is very important. In some sense this is what will allow us to prove the Riesz Representation Theorem. In fact, we have the following

Lemma/Proposition/Theorem: Let M be a closed subspace of a Hilbert space H. Then H=E\oplus E^\perp. Moreover, if we write x=y+z with y\in M, z\in M^\perp, then y,z are the unique elements of M,M^\perp whose distance to x is minimal.

Proof (sketch): Given x\in H, let \delta=inf\{||x-y||:y\in M\} and let y_n be a sequence in M so that ||x-y_n||\to\delta. Apply the parallelogram law to show that \{y_n\} is a Cauchy sequence (you may need to use that (y_n+y_m)/2\in M) and let y=lim_{n\to\infty}y_n and let z=x-y. Since M is closed, we have y\in M and ||x-y||=\delta. To show that z\in M^\perp, let u\in M. By multiplying by an appropriate (complex) scalar, we may assume \langle z,u\rangle\in\mathbb{R}. Thus the function f(t)=||z+tu||^2 is real for all t\in\mathbb{R}. Elementary calculus shows that f has a minimum at t=0, and so f'(0)=2\langle z,u\rangle=0, which shows z\in M^\perp. I’ll leave it to the reader to verify the uniqueness statements as well as the statement about distance to M^\perp.

Some remarks are in order. First, if X is a Banach space and M is a closed subspace of X, we say M is complemented if there exists a closed subspace N such that X=M\oplus N. The above theorem tells us that in a Hilbert space, every closed subspace is complemented. It is a non-trivial theorem (due to Lindenstrauss and Tzafriri) that if X is a Banach such that every closed subspace of X is complemented, then X is isomorphic to a Hilbert space. The question of which subspaces of a Banach space are complemented is open.

Given a closed subspace M\subseteq H, we can define a map P:X\to M as follows: given x\in H, write x=y+z uniquely with y\in M and z\in M^\perp. Then define P(x)=y. Note that (I-P)(x)=z, so I-P:X\to M^\perp. Moreover, we have P^2=P and (I-P)^2=I-P. A map T so that T^2=T is called a projection. For the special case of P above, P is called the orthogonal projection onto M and I-P is the orthogonal projection onto M^\perp. It can be shown that in a Banach space, a subspace M is complemented if and only there exists a continuous projection onto M. For more information on projections, check out this link.

Alright, enough asides! The real reason I presented the above theorem was to prove the following

Corollary: Riesz Representation Theorem: Let H be a Hilbert space and f\in H^*. Then there is a unique y_f\in H so that f(x)=\langle x,y_f\rangle for all x\in H. Moreover, ||f||=||y_f||.

Proof: For uniqueness, suppose z is another vector satisfying the theorem. Then for all x\in H we have

\langle x,y_f-z\rangle=\langle x,y_f\rangle-\langle x,z\rangle=f(x)-f(x)=0

and so y_f-z\in H^\perp=\{0\}, i.e. y_f=z. For existence, if f=0, then clearly y=0 works. Otherwise \text{ker}(f) is a proper subspace of H, so its orthogonal complement contains a non-zero vector z of norm one. Let u=f(x)z-f(z)x and note that f(u)=0, so u\in\text{ker}(f). Thus

0=\langle u,z\rangle=f(x)||z||^2-f(z)\langle x,z\rangle=f(x)-\langle x,\overline{f(z)}z\rangle

so letting y_f=\overline{f(z)}z gives the existence. Finally, note that |f(y_f)|=\langle y_f,y_f\rangle=||y_f||\cdot||y_f||, so ||f||\geq||y_f||, and by Cauchy-Schwartz, for all x\in H we have |f(x)|=\langle x,y_f\rangle|\leq||y||\cdot||x||, so ||f||\leq||y_f||, which completes the proof.

So what does this tell us? Well, note that if y\in H, then the map f_y(x):=\langle x,y\rangle is a linear functional on H. It’s easy to see that y\mapsto f_y is linear, and if \langle x,y\rangle=0 for all x, then y=0 by the same argument as above, so this embeds H into H^*. What the above theorem tells us is that this correspondence is surjective, and so we have a conjugate-linear, isometric isomorphism between H and H^*. Repeating the construction again, we see that H\simeq H^{**}. It’s natural to ask if there are other classes of spaces where we can write down such a relation. I’ll write about these in later posts, but I think the next few posts I write will give some examples of applications of the Riesz representation theorem and perhaps a few more facts about Hilbert spaces. For example, how many different Hilbert spaces are there? Well, if you haven’t seen the result before, the answer may surprise you! Unfortunately I’ll probably have to start working on homework again soon, so I don’t know when I’ll get to write again, but until then, enjoy!


About Ryan

I'm a software developer at Hudl where I work on awesome software. Before that, I was a grad student in mathematics, interested in probability theory as well as analysis, more on the side of functional analysis and less on the side of PDEs. Apart from that I'm pretty lame. Though I do enjoy watching football, playing golf, and playing the trumpet.
This entry was posted in Analysis, Linear Algebra. Bookmark the permalink.

3 Responses to The Riesz Representation Theorem – Part 2

  1. Z Norwood says:

    Put your posts on the site map, please! But thanks for posting.

  2. Ryan says:

    How do you do that?

  3. Pingback: Moving Around a Sphere – Hilbert Spaces Part 3 | whateversuitsyourboat

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s