Hahn–Banach theorem

The Hahn–Banach theorem is a central tool in functional analysis. It allows the extension of bounded linear functionals defined on a vector subspace of some vector space to the whole space, and it also shows that there are "enough" continuous linear functionals defined on every normed vector space to make the study of the dual space "interesting". Another version of the Hahn–Banach theorem is known as the Hahn–Banach separation theorem or the hyperplane separation theorem, and has numerous uses in convex geometry.

The theorem is named for the mathematicians Hans Hahn and Stefan Banach, who proved it independently in the late 1920s. The special case of the theorem for the space ${\displaystyle C[a,b]}$ of continuous functions on an interval was proved earlier (in 1912) by Eduard Helly,^[1] and a more general extension theorem, the M. Riesz extension theorem, from which the Hahn–Banach theorem can be derived, was proved in 1923 by Marcel Riesz.^[2]

The first Hahn–Banach theorem was proved by Eduard Helly in 1912 who showed that certain linear functionals defined on a subspace of a certain type of normed space (${\displaystyle \mathbb {C} ^{\mathbb {N} }}$) had an extension of the same norm. Helly did this through the technique of first proving that a one-dimensional extension exists (where the linear functional has its domain extended by one dimension) and then using induction. In 1927, Hahn defined general Banach spaces and used Helly's technique to prove a norm-preserving version of Hahn–Banach theorem for Banach spaces (where a bounded linear functional on a subspace has a bounded linear extension of the same norm to the whole space). In 1929, Banach, who was unaware of Hahn's result, generalized it by replacing the norm-preserving version with the dominated extension version that uses sublinear functions. Whereas Helly's proof used mathematical induction, Hahn and Banach both used transfinite induction.^[3]

The Hahn–Banach theorem arose from attempts to solve infinite systems of linear equations. This is needed to solve problems such as the moment problem, whereby given all the potential moments of a function one must determine if a function having these moments exists, and, if so, find it in terms of those moments. Another such problem is the Fourier cosine series problem, whereby given all the potential Fourier cosine coefficients one must determine if a function having those coefficients exists, and, again, find it if so.

Riesz and Helly solved the problem for certain classes of spaces (such as ${\displaystyle L^{p}([0,1])}$ and ${\displaystyle C([a,b])}$) where they discovered that the existence of a solution was equivalent to the existence and continuity of certain linear functionals. In effect, they needed to solve the following problem:^[3]

(The vector problem) Given a collection ${\displaystyle \left(f_{i}\right)_{i\in I}}$ of bounded linear functionals on a normed space ${\displaystyle X}$ and a collection of scalars ${\displaystyle \left(c_{i}\right)_{i\in I},}$ determine if there is an ${\displaystyle x\in X}$ such that ${\displaystyle f_{i}(x)=c_{i}}$ for all ${\displaystyle i\in I.}$

If ${\displaystyle X}$ happens to be a reflexive space then to solve the vector problem, it suffices to solve the following dual problem:^[3]

(The functional problem) Given a collection ${\displaystyle \left(x_{i}\right)_{i\in I}}$ of vectors in a normed space ${\displaystyle X}$ and a collection of scalars ${\displaystyle \left(c_{i}\right)_{i\in I},}$ determine if there is a bounded linear functional ${\displaystyle f}$ on ${\displaystyle X}$ such that ${\displaystyle f\left(x_{i}\right)=c_{i}}$ for all ${\displaystyle i\in I.}$

Riesz went on to define ${\displaystyle L^{p}([0,1])}$ space (${\displaystyle 1<p<\infty }$) in 1910 and the ${\displaystyle \ell ^{p}}$ spaces in 1913. While investigating these spaces he proved a special case of the Hahn–Banach theorem. Helly also proved a special case of the Hahn–Banach theorem in 1912. In 1910, Riesz solved the functional problem for some specific spaces and in 1912, Helly solved it for a more general class of spaces. It wasn't until 1932 that Banach, in one of the first important applications of the Hahn–Banach theorem, solved the general functional problem. The following theorem states the general functional problem and characterizes its solution.^[3]

The Hahn–Banach theorem can be deduced from the above theorem.^[3] If ${\displaystyle X}$ is reflexive then this theorem solves the vector problem.

A real-valued function ${\displaystyle f:M\to \mathbb {R} }$ defined on a subset ${\displaystyle M}$ of ${\displaystyle X}$ is said to be dominated (above) by a function ${\displaystyle p:X\to \mathbb {R} }$ if ${\displaystyle f(m)\leq p(m)}$ for every ${\displaystyle m\in M.}$ Hence the reason why the following version of the Hahn–Banach theorem is called the dominated extension theorem.

The theorem remains true if the requirements on ${\displaystyle p}$ are relaxed to require only that ${\displaystyle p}$ be a convex function:^[7][8] $${\displaystyle p(tx+(1-t)y)\leq tp(x)+(1-t)p(y)\qquad {\text{ for all }}0<t<1{\text{ and }}x,y\in X.}$$ A function ${\displaystyle p:X\to \mathbb {R} }$ is convex and satisfies ${\displaystyle p(0)\leq 0}$ if and only if ${\displaystyle p(ax+by)\leq ap(x)+bp(y)}$ for all vectors ${\displaystyle x,y\in X}$ and all non-negative real ${\displaystyle a,b\geq 0}$ such that ${\displaystyle a+b\leq 1.}$ Every sublinear function is a convex function. On the other hand, if ${\displaystyle p:X\to \mathbb {R} }$ is convex with ${\displaystyle p(0)\geq 0,}$ then the function defined by ${\displaystyle p_{0}(x)\;{\stackrel {\scriptscriptstyle {\text{def}}}{=}}\;\inf _{t>0}{\frac {p(tx)}{t}}}$ is positively homogeneous (because for all ${\displaystyle x}$ and ${\displaystyle r>0}$ one has ${\displaystyle p_{0}(rx)=\inf _{t>0}{\frac {p(trx)}{t}})=r\inf _{t>0}{\frac {p(trx)}{tr}}=r\inf _{\tau >0}{\frac {p(\tau x)}{\tau }}=rp_{0}(x)}$), hence, being convex, it is sublinear. It is also bounded above by ${\displaystyle p_{0}\leq p,}$ and satisfies ${\displaystyle F\leq p_{0}}$ for every linear functional ${\displaystyle F\leq p.}$ So the extension of the Hahn–Banach theorem to convex functionals does not have a much larger content than the classical one stated for sublinear functionals.

If ${\displaystyle F:X\to \mathbb {R} }$ is linear then ${\displaystyle F\leq p}$ if and only if^[4] $${\displaystyle -p(-x)\leq F(x)\leq p(x)\quad {\text{ for all }}x\in X,}$$ which is the (equivalent) conclusion that some authors^[4] write instead of ${\displaystyle F\leq p.}$ It follows that if ${\displaystyle p:X\to \mathbb {R} }$ is also symmetric, meaning that ${\displaystyle p(-x)=p(x)}$ holds for all ${\displaystyle x\in X,}$ then ${\displaystyle F\leq p}$ if and only ${\displaystyle |F|\leq p.}$ Every norm is a seminorm and both are symmetric balanced sublinear functions. A sublinear function is a seminorm if and only if it is a balanced function. On a real vector space (although not on a complex vector space), a sublinear function is a seminorm if and only if it is symmetric. The identity function ${\displaystyle \mathbb {R} \to \mathbb {R} }$ on ${\displaystyle X:=\mathbb {R} }$ is an example of a sublinear function that is not a seminorm.

The dominated extension theorem for real linear functionals implies the following alternative statement of the Hahn–Banach theorem that can be applied to linear functionals on real or complex vector spaces.

The theorem remains true if the requirements on ${\displaystyle p}$ are relaxed to require only that for all ${\displaystyle x,y\in X}$ and all scalars ${\displaystyle a}$ and ${\displaystyle b}$ satisfying ${\displaystyle |a|+|b|\leq 1,}$^[8] $${\displaystyle p(ax+by)\leq |a|p(x)+|b|p(y).}$$ This condition holds if and only if ${\displaystyle p}$ is a convex and balanced function satisfying ${\displaystyle p(0)\leq 0,}$ or equivalently, if and only if it is convex, satisfies ${\displaystyle p(0)\leq 0,}$ and ${\displaystyle p(ux)\leq p(x)}$ for all ${\displaystyle x\in X}$ and all unit length scalars ${\displaystyle u.}$

A complex-valued functional ${\displaystyle F}$ is said to be dominated by ${\displaystyle p}$ if ${\displaystyle |F(x)|\leq p(x)}$ for all ${\displaystyle x}$ in the domain of ${\displaystyle F.}$ With this terminology, the above statements of the Hahn–Banach theorem can be restated more succinctly:

Hahn–Banach dominated extension theorem: If ${\displaystyle p:X\to \mathbb {R} }$ is a seminorm defined on a real or complex vector space ${\displaystyle X,}$ then every dominated linear functional defined on a vector subspace of ${\displaystyle X}$ has a dominated linear extension to all of ${\displaystyle X.}$ In the case where ${\displaystyle X}$ is a real vector space and ${\displaystyle p:X\to \mathbb {R} }$ is merely a convex or sublinear function, this conclusion will remain true if both instances of "dominated" (meaning ${\displaystyle |F|\leq p}$) are weakened to instead mean "dominated above" (meaning ${\displaystyle F\leq p}$).^[7][8]

Proof

The following observations allow the Hahn–Banach theorem for real vector spaces to be applied to (complex-valued) linear functionals on complex vector spaces.

Every linear functional ${\displaystyle F:X\to \mathbb {C} }$ on a complex vector space is completely determined by its real part ${\displaystyle \;\operatorname {Re} F:X\to \mathbb {R} \;}$ through the formula^{[6][proof 1]} $${\displaystyle F(x)\;=\;\operatorname {Re} F(x)-i\operatorname {Re} F(ix)\qquad {\text{ for all }}x\in X}$$ and moreover, if ${\displaystyle \|\cdot \|}$ is a norm on ${\displaystyle X}$ then their dual norms are equal: ${\displaystyle \|F\|=\|\operatorname {Re} F\|.}$^[10] In particular, a linear functional on ${\displaystyle X}$ extends another one defined on ${\displaystyle M\subseteq X}$ if and only if their real parts are equal on ${\displaystyle M}$ (in other words, a linear functional ${\displaystyle F}$ extends ${\displaystyle f}$ if and only if ${\displaystyle \operatorname {Re} F}$ extends ${\displaystyle \operatorname {Re} f}$). The real part of a linear functional on ${\displaystyle X}$ is always a real-linear functional (meaning that it is linear when ${\displaystyle X}$ is considered as a real vector space) and if ${\displaystyle R:X\to \mathbb {R} }$ is a real-linear functional on a complex vector space then ${\displaystyle x\mapsto R(x)-iR(ix)}$ defines the unique linear functional on ${\displaystyle X}$ whose real part is ${\displaystyle R.}$

If ${\displaystyle F}$ is a linear functional on a (complex or real) vector space ${\displaystyle X}$ and if ${\displaystyle p:X\to \mathbb {R} }$ is a seminorm then^{[6][proof 2]} $${\displaystyle |F|\,\leq \,p\quad {\text{ if and only if }}\quad \operatorname {Re} F\,\leq \,p.}$$ Stated in simpler language, a linear functional is dominated by a seminorm ${\displaystyle p}$ if and only if its real part is dominated above by ${\displaystyle p.}$

The proof above shows that when ${\displaystyle p}$ is a seminorm then there is a one-to-one correspondence between dominated linear extensions of ${\displaystyle f:M\to \mathbb {C} }$ and dominated real-linear extensions of ${\displaystyle \operatorname {Re} f:M\to \mathbb {R} ;}$ the proof even gives a formula for explicitly constructing a linear extension of ${\displaystyle f}$ from any given real-linear extension of its real part.

Continuity

A linear functional ${\displaystyle F}$ on a topological vector space is continuous if and only if this is true of its real part ${\displaystyle \operatorname {Re} F;}$ if the domain is a normed space then ${\displaystyle \|F\|=\|\operatorname {Re} F\|}$ (where one side is infinite if and only if the other side is infinite).^[10] Assume ${\displaystyle X}$ is a topological vector space and ${\displaystyle p:X\to \mathbb {R} }$ is sublinear function. If ${\displaystyle p}$ is a continuous sublinear function that dominates a linear functional ${\displaystyle F}$ then ${\displaystyle F}$ is necessarily continuous.^[6] Moreover, a linear functional ${\displaystyle F}$ is continuous if and only if its absolute value ${\displaystyle |F|}$ (which is a seminorm that dominates ${\displaystyle F}$) is continuous.^[6] In particular, a linear functional is continuous if and only if it is dominated by some continuous sublinear function.

The Hahn–Banach theorem for real vector spaces ultimately follows from Helly's initial result for the special case where the linear functional is extended from ${\displaystyle M}$ to a larger vector space in which ${\displaystyle M}$ has codimension ${\displaystyle 1.}$^[3]

This lemma remains true if ${\displaystyle p:X\to \mathbb {R} }$ is merely a convex function instead of a sublinear function.^[7][8]

The lemma above is the key step in deducing the dominated extension theorem from Zorn's lemma.

When ${\displaystyle M}$ has countable codimension, then using induction and the lemma completes the proof of the Hahn–Banach theorem. The standard proof of the general case uses Zorn's lemma although the strictly weaker ultrafilter lemma^[11] (which is equivalent to the compactness theorem and to the Boolean prime ideal theorem) may be used instead. Hahn–Banach can also be proved using Tychonoff's theorem for compact Hausdorff spaces^[12] (which is also equivalent to the ultrafilter lemma)

The Mizar project has completely formalized and automatically checked the proof of the Hahn–Banach theorem in the HAHNBAN file.^[13]

The Hahn–Banach theorem can be used to guarantee the existence of continuous linear extensions of continuous linear functionals.

In category-theoretic terms, the underlying field of the vector space is an injective object in the category of locally convex vector spaces.

On a normed (or seminormed) space, a linear extension ${\displaystyle F}$ of a bounded linear functional ${\displaystyle f}$ is said to be norm-preserving if it has the same dual norm as the original functional: ${\displaystyle \|F\|=\|f\|.}$ Because of this terminology, the second part of the above theorem is sometimes referred to as the "norm-preserving" version of the Hahn–Banach theorem.^[15] Explicitly:

The following observations allow the continuous extension theorem to be deduced from the Hahn–Banach theorem.^[16]

The absolute value of a linear functional is always a seminorm. A linear functional ${\displaystyle F}$ on a topological vector space ${\displaystyle X}$ is continuous if and only if its absolute value ${\displaystyle |F|}$ is continuous, which happens if and only if there exists a continuous seminorm ${\displaystyle p}$ on ${\displaystyle X}$ such that ${\displaystyle |F|\leq p}$ on the domain of ${\displaystyle F.}$^[17] If ${\displaystyle X}$ is a locally convex space then this statement remains true when the linear functional ${\displaystyle F}$ is defined on a proper vector subspace of ${\displaystyle X.}$

Proof for normed spaces

A linear functional ${\displaystyle f}$ on a normed space is continuous if and only if it is bounded, which means that its dual norm $${\displaystyle \|f\|=\sup\{|f(m)|:\|m\|\leq 1,m\in \operatorname {domain} f\}}$$ is finite, in which case ${\displaystyle |f(m)|\leq \|f\|\|m\|}$ holds for every point ${\displaystyle m}$ in its domain. Moreover, if ${\displaystyle c\geq 0}$ is such that ${\displaystyle |f(m)|\leq c\|m\|}$ for all ${\displaystyle m}$ in the functional's domain, then necessarily ${\displaystyle \|f\|\leq c.}$ If ${\displaystyle F}$ is a linear extension of a linear functional ${\displaystyle f}$ then their dual norms always satisfy ${\displaystyle \|f\|\leq \|F\|}$^{[proof 3]} so that equality ${\displaystyle \|f\|=\|F\|}$ is equivalent to ${\displaystyle \|F\|\leq \|f\|,}$ which holds if and only if ${\displaystyle |F(x)|\leq \|f\|\|x\|}$ for every point ${\displaystyle x}$ in the extension's domain. This can be restated in terms of the function ${\displaystyle \|f\|\,\|\cdot \|:X\to \mathbb {R} }$ defined by ${\displaystyle x\mapsto \|f\|\,\|x\|,}$ which is always a seminorm:^{[note 4]}

A linear extension of a bounded linear functional ${\displaystyle f}$ is norm-preserving if and only if the extension is dominated by the seminorm ${\displaystyle \|f\|\,\|\cdot \|.}$

Applying the Hahn–Banach theorem to ${\displaystyle f}$ with this seminorm ${\displaystyle \|f\|\,\|\cdot \|}$ thus produces a dominated linear extension whose norm is (necessarily) equal to that of ${\displaystyle f,}$ which proves the theorem:

The continuous extension theorem might fail if the topological vector space (TVS) ${\displaystyle X}$ is not locally convex. For example, for ${\displaystyle 0<p<1,}$ the Lebesgue space ${\displaystyle L^{p}([0,1])}$ is a complete metrizable TVS (an F-space) that is not locally convex (in fact, its only convex open subsets are itself ${\displaystyle L^{p}([0,1])}$ and the empty set) and the only continuous linear functional on ${\displaystyle L^{p}([0,1])}$ is the constant ${\displaystyle 0}$ function (Rudin 1991, §1.47). Since ${\displaystyle L^{p}([0,1])}$ is Hausdorff, every finite-dimensional vector subspace ${\displaystyle M\subseteq L^{p}([0,1])}$ is linearly homeomorphic to Euclidean space ${\displaystyle \mathbb {R} ^{\dim M}}$ or ${\displaystyle \mathbb {C} ^{\dim M}}$ (by F. Riesz's theorem) and so every non-zero linear functional ${\displaystyle f}$ on ${\displaystyle M}$ is continuous but none has a continuous linear extension to all of ${\displaystyle L^{p}([0,1]).}$ However, it is possible for a TVS ${\displaystyle X}$ to not be locally convex but nevertheless have enough continuous linear functionals that its continuous dual space ${\displaystyle X^{*}}$ separates points; for such a TVS, a continuous linear functional defined on a vector subspace might have a continuous linear extension to the whole space.

If the TVS ${\displaystyle X}$ is not locally convex then there might not exist any continuous seminorm ${\displaystyle p:X\to \mathbb {R} }$ defined on ${\displaystyle X}$ (not just on ${\displaystyle M}$) that dominates ${\displaystyle f,}$ in which case the Hahn–Banach theorem can not be applied as it was in the above proof of the continuous extension theorem. However, the proof's argument can be generalized to give a characterization of when a continuous linear functional has a continuous linear extension: If ${\displaystyle X}$ is any TVS (not necessarily locally convex), then a continuous linear functional ${\displaystyle f}$ defined on a vector subspace ${\displaystyle M}$ has a continuous linear extension ${\displaystyle F}$ to all of ${\displaystyle X}$ if and only if there exists some continuous seminorm ${\displaystyle p}$ on ${\displaystyle X}$ that dominates ${\displaystyle f.}$ Specifically, if given a continuous linear extension ${\displaystyle F}$ then ${\displaystyle p:=|F|}$ is a continuous seminorm on ${\displaystyle X}$ that dominates ${\displaystyle f;}$ and conversely, if given a continuous seminorm ${\displaystyle p:X\to \mathbb {R} }$ on ${\displaystyle X}$ that dominates ${\displaystyle f}$ then any dominated linear extension of ${\displaystyle f}$ to ${\displaystyle X}$ (the existence of which is guaranteed by the Hahn–Banach theorem) will be a continuous linear extension.

The key element of the Hahn–Banach theorem is fundamentally a result about the separation of two convex sets: ${\displaystyle \{-p(-x-n)-f(n):n\in M\},}$ and ${\displaystyle \{p(m+x)-f(m):m\in M\}.}$ This sort of argument appears widely in convex geometry,^[18] optimization theory, and economics. Lemmas to this end derived from the original Hahn–Banach theorem are known as the Hahn–Banach separation theorems.^[19][20] They are generalizations of the hyperplane separation theorem, which states that two disjoint nonempty convex subsets of a finite-dimensional space ${\displaystyle \mathbb {R} ^{n}}$ can be separated by some affine hyperplane, which is a fiber (level set) of the form ${\displaystyle f^{-1}(s)=\{x:f(x)=s\}}$ where ${\displaystyle f\neq 0}$ is a non-zero linear functional and ${\displaystyle s}$ is a scalar.

When the convex sets have additional properties, such as being open or compact for example, then the conclusion can be substantially strengthened:

Then following important corollary is known as the Geometric Hahn–Banach theorem or Mazur's theorem (also known as Ascoli–Mazur theorem^[22]). It follows from the first bullet above and the convexity of ${\displaystyle M.}$

Mazur's theorem clarifies that vector subspaces (even those that are not closed) can be characterized by linear functionals.

Since points are trivially convex, geometric Hahn–Banach implies that functionals can detect the boundary of a set. In particular, let ${\displaystyle X}$ be a real topological vector space and ${\displaystyle A\subseteq X}$ be convex with ${\displaystyle \operatorname {Int} A\neq \varnothing .}$ If ${\displaystyle a_{0}\in A\setminus \operatorname {Int} A}$ then there is a functional that is vanishing at ${\displaystyle a_{0},}$ but supported on the interior of ${\displaystyle A.}$^[19]

Call a normed space ${\displaystyle X}$ smooth if at each point ${\displaystyle x}$ in its unit ball there exists a unique closed hyperplane to the unit ball at ${\displaystyle x.}$ Köthe showed in 1983 that a normed space is smooth at a point ${\displaystyle x}$ if and only if the norm is Gateaux differentiable at that point.^[3]

Let ${\displaystyle U}$ be a convex balanced neighborhood of the origin in a locally convex topological vector space ${\displaystyle X}$ and suppose ${\displaystyle x\in X}$ is not an element of ${\displaystyle U.}$ Then there exists a continuous linear functional ${\displaystyle f}$ on ${\displaystyle X}$ such that^[3] ${\displaystyle \sup |f(U)|\leq |f(x)|.}$

The Hahn–Banach theorem is the first sign of an important philosophy in functional analysis: to understand a space, one should understand its continuous functionals.

For example, linear subspaces are characterized by functionals: if X is a normed vector space with linear subspace M (not necessarily closed) and if ${\displaystyle z}$ is an element of X not in the closure of M, then there exists a continuous linear map ${\displaystyle f:X\to \mathbf {K} }$ with ${\displaystyle f(m)=0}$ for all ${\displaystyle m\in M,}$ ${\displaystyle f(z)=1,}$ and ${\displaystyle \|f\|=\operatorname {dist} (z,M)^{-1}.}$ (To see this, note that ${\displaystyle \operatorname {dist} (\cdot ,M)}$ is a sublinear function.) Moreover, if ${\displaystyle z}$ is an element of X, then there exists a continuous linear map ${\displaystyle f:X\to \mathbf {K} }$ such that ${\displaystyle f(z)=\|z\|}$ and ${\displaystyle \|f\|\leq 1.}$ This implies that the natural injection ${\displaystyle J}$ from a normed space X into its double dual ${\displaystyle V^{**}}$ is isometric.

That last result also suggests that the Hahn–Banach theorem can often be used to locate a "nicer" topology in which to work. For example, many results in functional analysis assume that a space is Hausdorff or locally convex. However, suppose X is a topological vector space, not necessarily Hausdorff or locally convex, but with a nonempty, proper, convex, open set M. Then geometric Hahn–Banach implies that there is a hyperplane separating M from any other point. In particular, there must exist a nonzero functional on X — that is, the continuous dual space ${\displaystyle X^{*}}$ is non-trivial.^[3][25] Considering X with the weak topology induced by ${\displaystyle X^{*},}$ then X becomes locally convex; by the second bullet of geometric Hahn–Banach, the weak topology on this new space separates points. Thus X with this weak topology becomes Hausdorff. This sometimes allows some results from locally convex topological vector spaces to be applied to non-Hausdorff and non-locally convex spaces.

The Hahn–Banach theorem is often useful when one wishes to apply the method of a priori estimates. Suppose that we wish to solve the linear differential equation ${\displaystyle Pu=f}$ for ${\displaystyle u,}$ with ${\displaystyle f}$ given in some Banach space X. If we have control on the size of ${\displaystyle u}$ in terms of ${\displaystyle \|f\|_{X}}$ and we can think of ${\displaystyle u}$ as a bounded linear functional on some suitable space of test functions ${\displaystyle g,}$ then we can view ${\displaystyle f}$ as a linear functional by adjunction: ${\displaystyle (f,g)=(u,P^{*}g).}$ At first, this functional is only defined on the image of ${\displaystyle P,}$ but using the Hahn–Banach theorem, we can try to extend it to the entire codomain X. The resulting functional is often defined to be a weak solution to the equation.

To illustrate an actual application of the Hahn–Banach theorem, we will now prove a result that follows almost entirely from the Hahn–Banach theorem.