Digression: Hilbert Spaces – Part 1

Hello all, I hope your semesters are going well! I am on spring break this week, so I’ve got a bit of time, and I figured instead of getting a head start on my homework for next week, I’d write a post on the blog! So last time, I talked about the Riesz representation theorem for Hilbert spaces, where we showed that $H\simeq H^{**}$ for any Hilbert space $H$, and that in fact $H\simeq H^{*}$ (conjugate linear), which is in some sense the strongest form of reflexive we could ask for (this is what happens in finite dimensional spaces, for example). A natural question to ask, then, is just how nice do things actually get? I’ve remarked before that in finite dimensions, all norms are equivalent, all linear maps are continuous, and so up to an inner-product preserving map, there is only one normed space of dimension $n$ for each $n\in\mathbb{N}$. We will, in fact, prove a much stronger generalization of this fact now.

Recall that, in linear algebra, a Hamel basis for a vector space $V$ is a maximal linearly independent set in $V$. That is, a set $\{b_i\}_{i\in J}$ is a Hamel basis given any $v\in V$, there exist unique scalars $\{c_{k_1},\dots,c_{k_n}\}$ s.t. $v=\sum_{j=1}^n c_{k_i}b_i$.

Examples: Let $X=l^2$, i.e. the set of all sequences such that $\sum_1^\infty |a_j|^2<\infty$ and let $Y\subset X$ be the set of all sequences $\{a_n\}_1^\infty$ such that $a_k=0$ for all but finitely many $k$. Let $\delta_j$ defined by $\delta_j(n)=0$ if $n\neq j$ and $1$ if $n=j$ (so, for example, $\delta_2=(0,1,0,0,\dots)$), and let $B=\{\delta_j:j\in\mathbb{N}\}$. Then $B\subset Y\subset X$. $B$ is a Hamel basis for $Y$, since given any $\{a_j\}\in Y$, there is $N\in\mathbb{N}$ so that $n\geq N$ implies $a_n=0$. It is clear $\{a_n\}=\sum_{j=1}^N a_j\delta_j$ and it is not hard to show this representation is unique. However, $B$ is not a Hamel basis for $X$, since there is no finite subset of $B$ so that $\{a_n\}=\sum_{j} c_j\delta_j$, where $a_k=1/2^k$.

Of course, as Zach writes, every vector space possesses a Hamel basis, so there is some Hamel basis for the space $X$ above. A key part of the definition of a Hamel basis is that every element of $V$ is a finite linear combination of basis elements. Unfortunately, this makes Hamel bases not very useful for analysis, where we have the machinery to deal with infinite linear combinations. We would really like to be able to say that the set $B$ above is a basis for $X$, morally it feels right. We want these sequence spaces to have “countable dimension”, but it’s not hard to show (as a corollary to the Baire Category Theorem) that in fact any complete normed space cannot contain a countable Hamel basis. To resolve this problem, we’ll pull a very typical mathematical trick, we’ll simply redefine what we mean by basis! I’ll make the following definition for a Hilbert space, and at the end of the post I’ll make some comments on generalizing the ideas to other Banach spaces.

Definition: Let $H$ be a Hilbert space. A set $\{e_\alpha\}_{\alpha\in A}$ is called an orthonormal sequence if $\langle e_\alpha,e_\beta\rangle=\delta_{\alpha,\beta}$ for all $\alpha,\beta\in A$. We say the set is an orthonormal basis if it is an orthonormal sequence and for all $x\in H$, we have

$x=\sum_{\alpha\in A} \langle x,e_\alpha\rangle e_\alpha$

where the sum on the right has only countably many non-zero terms and converges in the norm topology no matter how the terms are ordered.

Example: Let $X,B$ be as in the above example. If for $f,g\in H$ we define $\langle f,g\rangle=\sum_{n\in\mathbb{N}} f(n)g(n)$, then it is easy to see that $\langle\cdot,\cdot\rangle$ defines an inner product on $X$, and $X$ is complete with respect to this inner product. The reader may easily verify that $B$ is an orthonormal basis for $X$.

A quick comment: can we really define sums over uncountable indexing sets? The reader should try to prove that if we have any uncountable sum of real numbers $\sum_{\alpha\in A}a_\alpha$, then either $a_i\neq 0$ for at most countably many $i\in A$, or the sum is infinite. To avoid issues of defining convergence of arbitrary sums in arbitrary normed spaces, we essentially require that we deal with only sums containing only countably many non-zero terms, and of course we know how to talk about norm convergence of such sums. In fact, the following proposition will show that these are really the only situations we need to concern ourselves with.

Proposition Let $\{e_\alpha\}_{\alpha\in A}$ be an orthonormal sequence in a Hilbert space $H$. Then TFAE:
1) $\{e_\alpha\}_{\alpha\in A}$ are an orthonormal basis for $H$.
2) If $\langle x,e_\alpha\rangle=0$ for all $\alpha\in A$, then $x=0$.
3) Given any $x\in H$, we have $||x||^2=\sum_{\alpha\in A} |\langle x,e_\alpha\rangle|^2$.

The property 2) is called completeness and the equation in 3) is called Parseval’s identity. I’ll leave the proof to the reader; you may find it useful to use Bessel’s inequality, that state if $\{e_\alpha\}_{\alpha\in A}$ is an ONS in $H$, then for all $x\in H$, we have $\sum_{\alpha\in A} |\langle x,e_\alpha\rangle|^2\leq||x||^2$, and so in particular the set $\{\alpha:\langle x,e_\alpha\rangle\neq0\}$ is countable. Okay, so now what do we do? Well, in linear algebra, one often defines a basis, then proceeds to show that every vector space has a basis and that any two bases of the space have the same cardinality, often called the dimension of the vector space. Then often one shows that two finite dimensional vector spaces are isomorphic if and only if they have the same dimension. Well, turns out that if we use our new definition of basis, we can get the same results for Hilbert spaces! Let’s first note that

Proposition Every Hilbert space has an orthonormal basis
Proof: Let $H$ be a Hilbert space and let $\mathcal{O}$ denote the collection of all orthonormal sets in $H$. If $H=\{0\}$, then the result in trivial, otherwise $H$ contains a vector $x$ of unit norm. The set $\{x\}$ is trivially an ONS, so $\mathcal{O}$ is non-empty. Partially order $\mathcal{O}$ by inclusion. If $\mathcal{C}$ is a linearly ordered subset of $\mathcal{O}$, then let $B=\bigcup_{C\in\mathcal{C}} C$. It is clear that if $x\in B$, then $\langle x,x\rangle=1$, and if $x,y\in B$, then there is $C$ so that $x,y\in C$ (since $\mathcal{C}$ is linearly ordered), so $\langle x,y\rangle=0$, and so indeed $B$ is an ONS. By Zorn’s lemma, there exists a maximal element $E=\{e_\alpha\}_{\alpha\in A}$ in $\mathcal{O}$. Suppose $\langle x,e_\alpha\rangle=0$ for all $\alpha\in A$. If $x\neq0$, let $v=x/||x||$. Then $\langle v,e_\alpha\rangle=0$ for all $\alpha\in A$, and so the set $E\cup\{v\}$ is an ONS in $H$ containing $E$, which contradicts the maximality of $E$, so indeed we must have $x=0$, which shows $E$ is complete, and hence is an ONB for $H$.

This proof isn’t very tough, it’s very similar to the proof that every vector space has a basis, and is a pretty standard Zorn’s lemma argument. In fact, a lot of these proofs fall out nicely for Hilbert space cases. As I’ve hinted at, I’ll write a little bit (at some point) about how one can define bases for Banach spaces that admit countable linear combinations of elements. Unfortunately proofs don’t work so nicely in these cases, we really do need the nice geometry that is given by the complete inner product in a Hilbert space. I was going to write more for this post, but I just noticed it’s already pretty long, and we’re at a pretty nice place for a break. Next time, I’ll try to convince you that any two orthonormal bases for $H$ must have the same cardinality (so we can (well) define the dimension of $H$), and then show you that two Hilbert spaces $H,H'$ are (unitarily) isomorphic if and only if they have the same dimension and then close by giving you my promised comments on bases in Banach spaces. Until then, good luck with any exams and homeworks you may have coming up!