Operators with invariant subspaces but no hyperinvariant ones

← All digests

Family 293 · Operator algebras
Importance 5/5 · Provisional editorial rating
Sources consulted 2026-10-25

1. The question

A matrix has invariant subspaces: subspaces that it maps into themselves. For a bounded operator on an infinite-dimensional Hilbert space, the general invariant-subspace question is much subtler. One can also ask for a subspace preserved by every operator commuting with the given operator. Such a subspace is called hyperinvariant.

Family 293 claims operators with no nontrivial closed hyperinvariant subspace. One manuscript obtains them from weighted irrational rotations with no invariant projection in a specified containing factor. The other gives a direct construction whose ordinary invariant subspaces are plainly present, while additional commuting operators prevent any of them from being hyperinvariant.

The distinction is essential. The family claims a negative answer to the hyperinvariant-subspace problem and a related transitive-algebra question. It does not claim a negative answer to the ordinary Hilbert-space invariant-subspace problem.

2. Background and essential definitions

A closed subspace $L\subseteq H$ is invariant for $T\in B(H)$ if $T(L)\subseteq L$. If $p$ is its orthogonal projection, this is equivalent to

$$(1-p)Tp=0.$$

It does not require $pT=Tp$; that stronger condition would make both $L$ and its orthogonal complement invariant.

The commutant of $T$ is $\{T\}'=\{A:AT=TA\}$. Hyperinvariance requires invariance under every element of this commutant. An operator algebra is transitive when its only common closed invariant subspaces are zero and the whole Hilbert space. Thus a transitive commutant is precisely the absence of nontrivial hyperinvariant subspaces.

A bounded operator is norm-quasinilpotent if

$$\lim_{n\to\infty}\|T^n\|^{1/n}=0.$$

Its spectrum is then $\{0\}$. Unlike a finite-dimensional nilpotent matrix, it need not have a power that is exactly zero.

A type $\mathrm{II}_1$ factor $M$ is a von Neumann algebra with scalar center and a normalized trace $\tau$, but no minimal nonzero projections. An invariant subspace is affiliated with $M$ when its orthogonal projection lies in $M$. This restricts which invariant subspaces are being considered.

If $T\in M$, every hyperinvariant subspace has its projection in $M$. Indeed, the unitaries in the commutant $M'$, and their inverses, commute with $T$, so hyperinvariance makes the subspace reducing for them. Its projection therefore belongs to $M''=M$. Excluding affiliated invariant projections consequently excludes hyperinvariant subspaces, though it need not exclude all invariant subspaces.

3. Development history and previous partial results

Invariant-subspace theory has many positive results under additional assumptions. In finite factors, Brown measure supplies a spectral distribution for nonnormal operators. Haagerup–Schultz associate invariant projections to regions of the complex plane, with projection trace equal to the region’s Brown mass. A Brown measure that is not concentrated at one point therefore gives a nontrivial projection.

The point-mass case is not settled by that construction. It does not automatically have no invariant subspaces: Dykema–Haagerup’s quasinilpotent DT-operator has a rich family of affiliated hyperinvariant subspaces, and Tucci constructed other quasinilpotent examples with invariant projections.

For weighted irrational rotations, earlier work identifies Brown measure as uniform measure on a circle with radius given by the weight’s Fuglede–Kadison determinant. Zhu, Fang and Shi studied the zero-determinant case and asked whether nontrivial affiliated invariant subspaces must still exist. They had already exhibited continuous one-zero weights with zero determinant. M1 adds the claimed exclusion of every invariant projection for suitable such weights.

A second history connects hyperinvariance with transitive algebras. Arveson’s density theorem says that a transitive algebra containing a maximal abelian self-adjoint subalgebra of $B(H)$ is strongly dense in $B(H)$. M2’s scalar multipliers have infinite multiplicity and are not maximal abelian in $B(H)$, so that hypothesis does not apply.

Classical work of Domar and Grabiner analyzes shifts with tightly ordered invariant-subspace lattices. M2 uses those ideas while coupling different fibers through an additional commuting operator. The source also notes that a Kérchy–Pearcy preprint claiming an equivalence between ordinary invariance and hyperinvariance was withdrawn. Such an equivalence is not used here.

4. What the new manuscripts claim

M1: a counterexample at every irrational angle

Fix any irrational $\theta\in(0,1)$. In the irrational-rotation factor $R_\theta$, let the generating unitaries satisfy

$$VU=e^{2\pi i\theta}UV.$$

M1 constructs a continuous weight $f:\mathbb T\to[0,1]$, on the unit circle, such that

$$f^{-1}(0)=\{1\},\qquad \int_{\mathbb T}\log f\,dm=-\infty,$$

where $m$ is normalized circle measure. Nevertheless, for $T=Uf(V)$, every projection $p\in R_\theta$ satisfying $(1-p)Tp=0$ is zero or one.

The angle is prescribed before the weight is chosen; no Diophantine restriction is imposed. The operator is nonzero and norm-quasinilpotent. The logarithmic integral means that the determinant $\exp(\int\log f\,dm)$ is zero, although $f$ is positive almost everywhere.

By the projection observation above, M1 obtains a nonzero norm-quasinilpotent operator with no nontrivial hyperinvariant subspace on every separable infinite-dimensional complex Hilbert space.

M2: an explicit mechanism with ordinary invariant tails

M2 works on

$$H_0=L^2(\mathbb Z_2;\ell^2(\mathbb Z)),$$

where $\mathbb Z_2$ is the compact probability group of 2-adic integers, not the two-element group. Its translation $x\mapsto x+1$ is an ergodic probability-preserving transformation.

The operator acts fiberwise as a weighted bilateral shift,

$$S_xe_j=\beta_j(x)e_{j+1},\qquad \beta_j(x)>0.$$

M2 states that every fiber’s closed invariant subspaces are exactly the coordinate tails $K_{\geq h}$, together with zero and the full fiber. It also gives

$$\|S\|=e^{-10},\qquad \|S^n\|\leq\exp\left(-10\left\lfloor\frac{(n+1)^2}{4}\right\rfloor\right).$$

A bounded contraction $V$ commutes with $S$, transports the base point from $x$ to $x+1$, and moves fiber coordinates backward. Together with all scalar multiplication operators and $S$, it generates a transitive algebra.

Hence $\{S\}'$ is a proper strongly closed unital transitive algebra. This also contradicts the assertion that every unital transitive operator algebra must be strongly dense in $B(H)$. Ordinary invariant subspaces remain: every fixed tail space $L^2(\mathbb Z_2;K_{\geq h})$ is invariant under $S$.

5. Why the advance matters

Importance: 5/5. If correct, these constructions resolve the hyperinvariant-subspace problem negatively and establish a sharp separation between invariance under one operator and invariance under all its commuting symmetries.

The rotation result also answers a specific question inside the hyperfinite factor, one of the most tractable infinite-dimensional operator algebras. It shows that zero-determinant weighted rotations can behave more radically than the existence of earlier quasinilpotent examples suggested.

M2 makes the mechanism understandable: ordinary invariant tails exist in abundance, but a commuting operator pushes them in a direction incompatible with a globally invariant measurable choice. Its explicit power bound additionally controls the whole operator, not merely its individual fibers.

6. The main idea

Force a projection to contradict weighted orthogonality

M1 first constructs a measurable weight. If an invariant projection existed, the shift representation would express it as a measurable field of projections. Finite trace and ergodicity force its central diagonal coordinate strictly between zero and one almost everywhere. Both the projection and its complement therefore contain vectors with nonzero central coordinates.

Invariance imposes weighted orthogonality identities. The constructed weight has rare but very deep drops: a selected finite window remains controlled, while sufficiently distant terms are strongly suppressed. A finite convolution coefficient forced by the central coordinates cannot be canceled by the suppressed tail, contradicting the identity.

The source then smooths and rearranges the logarithmic drops to obtain a continuous weight with exactly one zero. The change is expressed through a measurable coboundary and an intertwining multiplier, possibly unbounded. A separate domain-and-trace argument transfers any hypothetical invariant projection back to the measurable model. This transfer is needed; smoothing alone would not preserve the obstruction automatically.

Couple forward shifts to a backward operator

M2 chooses weights whose logarithms have controlled quadratic behavior at both ends of the bilateral sequence. This gives the stated classification of fiber invariant subspaces and the uniform decay of powers.

To construct the backward intertwiner, write its fiber action as $V_xe_j=b_j(x)e_{j-1}$. Requiring $VS=SV$ determines a recurrence for the coefficients. Opposite logarithmic changes at the two ends make the solution decay in both directions. A measurable normalization produces one bounded contraction over the entire probability space.

A tail cannot move downward forever in distribution

A subspace invariant under all scalar multipliers has a measurable projection field. Invariance under $S$ then makes each fiber a tail, described by a measurable label $h(x)$ taking integer values or the symbols for full and zero fibers.

The backward intertwiner forces

$$h(x+1)\leq h(x)-1$$

where the label is finite. Probability preservation rules out this perpetual strict descent for an integer-valued measurable label. Ergodicity rules out mixing full and zero fibers nontrivially. Only the whole space and zero remain.

All the operators used in this transitivity argument commute with $S$. Thus their lack of a common invariant subspace excludes a hyperinvariant subspace for $S$, even though $S$ alone preserves the fixed coordinate tails.

7. Scope, limitations, and open questions

M1 excludes invariant projections inside the specified rotation factor. It does not exclude arbitrary invariant projections in $B(H)$. M2 explicitly retains ordinary invariant tails, so the ordinary invariant-subspace problem remains outside the conclusion.

The nonzero quasinilpotent operators are nonscalar, ensuring their commutants are proper. Strong closedness means closure under pointwise norm convergence on vectors; the resulting transitive algebras are not claimed to be self-adjoint. Arveson’s theorem is avoided by failure of its maximal-abelian hypothesis, not by contradicting that theorem.

The two constructions are different. M2 imports only the elementary commutant and measurable-field facts identified in its introduction from M1; M1 uses no result from M2. The quantitative power bound belongs to the 2-adic construction, not automatically to every rotation example.

Both sources used here are dated September 27. Their claims are reported with these precise distinctions between factor affiliation, ordinary invariance and hyperinvariance.

8. Sources and further reading

• Haagerup and Schultz, Invariant Subspaces for Operators in a General II1-factor, for the Brown-measure projection theorem.
• Kérchy and Pearcy, Hyperinvariant subspaces of block-triangular operators on Hilbert space, whose withdrawal notice is relevant to the distinction discussed above.
• M1 cites the original Zhu–Fang–Shi question and earlier quasinilpotent examples; M2 specifies its weighted-translation criterion and its relation to Arveson’s density theorem.

Source manuscripts

• M1. OpenAI, Invariant-projection counterexamples for every irrational rotation (September 27, 2026).
• M2. OpenAI, Backward intertwiners and a transitive commutant (September 27, 2026).

Sources consulted on 25 October 2026 at repository revision 1581d4c. Download the manuscript citations.