CC BY
23
Boundary values of holomorphic functions and spectra of some unitary repesentations
Yurii A. Neretin*
Moscow State Institute of Electronics and mathematics baksan@dionis.iasnet.ru
These notes are based on my lectures on discrete spectra given in Tambov in August 1996 (school ’’Analysis on homogeneous spaces”)
Now spectra in various problems of noncommutative harmonic analysis are completly or partial}' evaluated. It is well-known that sometimes such spectra contain discrete increments. Quite often such discrete increments are singular (’’exotic”) unitary representations and it is very difficult to construct these unitary representations by other way, see [Puk], [Nai], [Boy], [Ism], [Moll-3], [Str], [Far], [F-J], [Sch], [Kobl], [Kob2], [RSW], [Tsu], [How], [Ada], [Li], [Pat], [BO]
It was observed in [Nerl], [01s2], [01s3], [NO], [Ner2] that very often discrete increments to spectra in various problems of noncommutative harmonic analysis (decomposition of tensor products, decomposition of restrictions, decomposition of induced representation ) are related to some functional-theoretical phenomena, namely to so-called ’’trace theorems ” (i.e theorems about existence of restrictions of discontinuous functions to submanifolds, for this type theorems see [RS],IX.3,IX.9 and references to this sections, [Bar],chapter 5, [NR] , [Rud], 11.2).
The simplest case of this phenomenon is the tensor product of two representations of complementary series of SL-2(M. ).Recall the definition of representation Ts of the complementary series. The space 'H; of the repesentation Ts (where 0 < s < 1) is the space of functions on the circle z\ = eprovided with the scalar product
< fi, /2 >=
г2тг /-2ТГ
Jo Jo
|sin(<pi — <^2)/2|(1+s)/2 -
The representation Ts is defined by the formula
7 (s 0/w = /0fc + 5r‘
The representation TSl ® TS2 acts in the space ® Hje of functions on two-dimensional torus zi — e'4’1, Z2 = e,<h equipped with the scalar product
f f _ / [ [~* r1' , 1p2)d<pld<j)2dTp1dlp2
Jo Jo Jo Jo [sin^i -^1)/2|(1+-0/2|sin(^2 - ^2)/2|(l+-»)/2
The group 5^2(1. ) acts in this space by the formula
(TSl®TS2)( % M/(z1|22) = /(p^,p^)|k1+ar-1|6z2+ar-1 \ 0 a / bzi + a 0Z2 + a
If .si + S2 > 1 then there exists well-defined operator R of restriction of a function / £ 'Hjoa to the
diagonal A : </>j = <^2- We emphasis that functions / £ ® W/ are discontinuous and hence / haw
no values in a individual point of the torus. Neverless the operator R of the restriction of a function / to the diagonal is well defined . Observe that R is interwinning operator from TSi ®TS2 to TSl+S2_j. Hence TJl+i2_i is a subrepresentation in TSl ® Ts.2.
Existense of the embedding T5l4.S2_i to TSl <S)TSJ was obtained in [Puk] (see also [Nai]). The construction with restriction to the diagonal was observed in [Nerl] (see also [NO] ).
Various consructions of the same type are contained in [N0],[Nerl]-[Ner2],[01s3]. In [NO] we used this approach for constructions of singular unitary representations of the groups U(p, q), 0(p, q), Sp(p, q).
^supported by RFRF ,grant 95-01-00814) and Russian program of support of scientific schools
These notes is'some kind of addendum to the papers [NO],[Ner2]. The aim of these notes is to forllate o^n probl-o. on discrete increments to spec,,, and trace theorems and to discuss a
relationship between spectral ^ ^ th„ls y.F.Molchanov, H Schl.chtkrull,
G Zuckerman^ B.Orsted, M.Flensted-Jensen, G.OIafsson, A.Dvorsky, A.G.Sergeev, R.S.Ismagdov, R.Howe, V.M.Gichev, V.V.Lebedev for discussions, comments and references.
1 Boundary values of holomorphic functions.
1.1. Let fl C C " be a open domain, let 5a be its boundary, let Q be the closure of Q. We say that O is
a regular circle domain if
a) for all z G fi and A G C such that |A| £ 1 we have Az G fi
Let A'(1! ^be a “pt^g tond for LlncT^NG]) in fl satisfying the condition
K(ei<l>z, e’^u) = I<{z,u)
w b- b. - 7» r:rP::n“ »»-
Theorem 1A (see I JJ exi3i dmQst ^ onMxM mih respeci to the measure fix fi
I K-e !‘(M xtkcrc ,„><* « f»nc„on S(, „) €
LHl, X x ,) »C* «*•< №.«)!■Z\Z i M i. ^
Then the operator of restriction of a function f G H on the set m j r
H — L\M, n)
The following natural problem arises. reproducing kernel and let H be
^ i.- т „( о с Г n he a a onen domain. Let De а кртииишц,
the!” -Id h“e,t L M b!a submanifold in - *"^0, “ ^
existence of restriction operator from H ”me is ls followin6 case . Let П - C/K
Remark. A case which is mteres ing or ^ ^ ^ ^ ^ ghijov
be a homogeneous Cartan domain. Let Q e a su ^group opcrator from я to some hilbert
boundary of fi. Then operator R о res nc _ fNer2] and below sections 2 and 3
space of funtions on M.See discussion of such restriction [ j, I ^ triction operator exists.
Following subsection show that Theorem 1.1 doesn t cover all cases then a restnc Р
1.2. Denote bv Bq the unit ball
|Zl|2 + ■ ■ •+ \Zq\ < 1
in С q. Let 7(f) be a C^curve in the dBq = S2*"1.
Theorem 1.2 (see [NR],[Rud]) Let 7(t) satisfies the condition
Vi : Im < т(0л'(0 >Ф 0 ^
Then for each f G tf°°(B?) the nontangent limit f(z) as z — 7(t) exists almost sure on 7(i).
Denote by the space of all holomorphic functions of polynomial groutb m B<
f£D’^3N-.Snp\f(z)\(l-\z\2)N<oo
„ is well known that each function / i= O' has limit on the boundary in the sence of distributions (see
[RS],1X 3 for discussion of such type ^e°re™ “d ав, шф,пд lhe condition (I). Псп
Theorem 1Л.(ш [KerS])Un{l) nC cantor from D' to
the operator R of restriction of holomorphic function to Д j
the space of distributions on 7(i). |z I < 1 Let Tn be the torus 2 = e<*',. .., z = e<*» . Let
Denote by Pn the polydisk Iz^ < 1, - - •, l2nl < Lel
7(T) = (^i(<). • ■ •. ^«(0) be a C°°-curve in Tn such that
Vi : ^(0 > >0
. Denote by D' the space of holomorhic functions of polynomial grouth in P".
’ 387
Theorem 1.4.(see [NerS]) Operator R of restriction of holomorphic function f on Pn to the curve 7(t) extends to the bounded operator from the space D' to the space of distributions on the curve 7(t)
Let C C N be a open domain. Let M be a submanifold in the Shilov boundary of f2. Denote by Trn the tangent space to M in the point N G M. We identify Tm with a linear submanifold in C n. Denote by Sm the linear submanifold which consists of vectors
i ■ (v — m) + to
where v G Tm. Assume that for each point m G M there exists a open cone Cm C Tm with the vertex rn and 6-neighbourhood Ot(m) of to such that
Cm nft D Of (m) fl Cm
Conjecture.Each holomorphic function of polynimial grouth in f2 has restriction to M in the sense of distributions.
Theorems 1.3-1.4 are partial cases of this conjecture. It is also similiar to the standard facts on limits of functions of polynomial grouth on the whole Shilov boundary mentioned above (see [RS],IX.3 ). Neverless I couldn’t find this fact in the literature.
2 Positive defined kernels on riemann noncompact symmetric spaces
2.1. Matrix balls BP:t. Let p < q. Denote by BPit the space of all complex p x g-matrices 2 such that ||z|| < 1. The group U(p, q) consists of (p + q) x (p + q)-matrices g = ^ satisfying the condition
•■(J = -“)
The group U(p,q) acts on BPiq by the transformations
z i—► := (a + zc)~1(b + zd) (2)
The stabilizer of the point z = 0 consists of matrices having the form ^ ^ ^ where a G U(p), d G i'{q)
.Hence BPiq is the symmetric space
BP]q = U(p,q)/(U(p) x (7(g))
Consider the function
* \ ] — 25
Ls(z,u) = | det(l — zu where z, u G Bp g.
Theorem 2.1. Let s = 0, 1, 2,. . . ,p— 1 or s > p- 1. Then the function Ls(z, u) is a positive defined kernel on Bp q.
(this theorem is a consequence of the theorem 2.2 below)
Consider the hilbert space defined by the positive defined kernel Ls(z,u).. This space contains the total system of vectors G Bp q such that
< ^2,'I'u >= Ls(z,u)
We associate to each vector h G Hs the function fh on Bp q by the rule
fh(2) =< h, ipz >
It is easy to prove that fh is a real analytic function 011 BPiq. We will identify the space Hs with its image in the space of real analytic functions on BP:q.
The group U(p,q) acts in H, by the unitary operators
As(g)f(z) = f(z[g])| det(a + zc)T2s
Problem Decompose the repersentaUon As
We will name representations Ts by spherical kern-representations of the group U(p,q).
Some partial cases of this question were discussed in the end of 70-ies (See [Ber2],[Rep],[Gut], in fact there was discussed only the case when s is large. In this case the most interesting phenomena don’t appear). Then such problems were more or less forgotten. In last several years some this problem attracted interest again(see [NO], [00], [OZ],[Dij]).
I would like to try to explain why this problem is interesting and also to discuss some approaches to this problem.
2.2. Another formulation of the problem.
Theorem 2.2 (see [Berl]) Let Let s = 0,1, 2,.. ., p - 1 or s > p - 1. Then the kernel
Ks(z, u) = det~s(l - zu*)
is positive defined on Bpq.
Denote by V, the hilbert space of holomorphic functions on BPtq defined by the kernel Ks(z,u) (see for instance [Berl], [NO]). The group U(ptq) acts in Vs by the unitary operators
T.(; bd^=f(^)ders(a + zc)
where 2^1 is given by the formula (2).
Remark. If 5 is integer then Ts is a representation of the group U(p, q) itself. If s is not integer then Ts is a representation of the universal covering group of U(p, q).
Denote by T* the representation contragradient to T3. Consider the tensor product Ts ® T* . This
representation acts in the space of holomorphic functions on Bp q x Bp q by the operators
(Ts ® Ts*) ( “ bd ) = f(Zuz,) = f(z^,z^)(a + z1c)-(a + z2c)-
The U(p,q)-invariant scalar product in the space of holomorphic functions on Bp%q x ? is defined by
the reproducing kernel
M(zi,z2:,ui,u2) - det_s(l - ziw*)det“5(l - z2u*2)
Consider the operator
I ■ Vs O V, Hs
defined by the formula
If(z)=f(z,z)
Obviously I is a unitary operator interwining
ts ® t; ~ as
Hence we can formulate our problem in the form:
Problem.Decompose the tensor product Ts ®T*.
2.3.Orbits of the group U(p, q) on the Shilov boundary.
Denote by MPi? the Shilov boundary of Bp>q. Elements of MPtq are matrices z satisfying the condition
■ z ■ z" — 1
In the other words 2 is a matrix of a isometric embedding Cp —> Cq. Hence Mp<q is complex Stiefcl manifold.
^ The Shilov boundary of BPi,xBPi, is M„xMM . The group U{p,q) has (p+1) orbits on MPiqxMp,q. Ihe unique invariant of a orbit is the number
a = rk(z — u) (3)
We denote by Ea the orbit corresponding to a given invariant a (i.e. is the set of all pairs z,u G MPA x MPt? such that (3) is satisfied)
Orbit H0 is compact, orbit Hp is open. For all a the closure of is Ua<aE(J.
2.4. Restriction to U{p, q)-orbits. Fix a orbit EQ of U(p, q) in the Shilov boundary of Bpq x Bp<q, It can happens (and it realy happens) that for small 5 function f E Hs - VS®VS has well defined restriction to the orbit Ea. In this case the restriction operator is a interwmmng operator from V, x
to some hilbert space of functions on Ea. r t +•
It can also happened (and it realy happens) that for small a all first partial derivatives of function
f <z Hs = Vs® Vs have well defined restriction to the orbit Sa etc.(see discussion of this phemmenon m
[NO] section ) ^ ^ ^ ^ Consder the maximal number ra such that all partial derivatives of
functions feVs®Vs have well-defined restrictions to S„. (this numbers aren’t known, but Theorem 1.1 give possibility to estimate them. I don’t know are such estimates strict or not).If restrictions of functions
, V. r -----
/ g Hs = Vs ® vs to Ea don’t exist we suppose ra - -1.
Remark For large s restriction operators don’t exist, i.e. we have ra - -1 for all a.
For each a consider some i = 1, • ■ ■, ra. Denote by Q[a,ra] the space of functions f £ Vs ® Vs such
that all partial derivatives of / of orders < i equals zero on ■—*0, ■
We obtain a filtration
0cQ\p- l,Tp-i] c Q[p- i.^-i - i] c-cfltP-Mlc
C Q[p — 2, TP_2] C • • ■ C Q\p - 2,1] C Q{p - 2,0] C ■ • ■ (4)
... c Q[ 0, To] c • • • Q[ o, 1] c Q[0,0]cV,® V,
Remark . For large 5 this filtration is trivial, neverless for small s it is quite long.
Consider representations of U(P,q) in subquotients of this filtration. Obviously A, - Ts ® ls i-
eauivalent to the direct sum of the subquotients. .
Remark. The representations of U(p, q) in the subquotients have simple interpretations. For instance
V®Vs/Q[ 0 0] is a subspace in the space of functions on the orbit Eo . The space Q[0,1]/Q[0, 0] asubspace
of sections of norma! bundle to the orbit . The space Q[0 1}Q 0,2] ,s a subspace „.space of
sections of symmetric square of normal bundle etc., see discussion in [NO],o I. ^
It is natural to hope that spectrum in each subquotient is more or less ’ uniform , i.e orbit strucure give separation of quite complicated spectrum of As to the different types (compaire with [GG]-proje ct ,
2 5 Large , If s is large enough then the restriction operators don’t exist. In this case he representation is equivalent to standart representation of the group U{j>, q) m L* on nemann symmetric space U(p,q)/(U(p) X U(q)) see [Ber2],[Rep],[Gut],[00]. Sufficient not nessessary) condition for this is
s>D_Lff_l(ie Ts is a element of Harish-Chandra discrete seri es).
2 6 Limit as s - oc. Concider the system of vectors VzeHs. Let X be a distribution in Bp,q with a compact support. Consider the vector 0(X) £ Hs defined by the equality
0(x) = [ det-5(l - zz*)x{z)^zdzdz
JBp,q
Consider a scalar product {, ■}, in the spase of distributions on Bp,q with compact support given by the formula
, ^ rv w- f I det(l — zz*)(det(l — ui£l sxi{z)x2{u)dzdjdudu
{X1>X2}3 :-< ©(Xi).0(X2) >- JB"'JB"I det2(l - zu*)
We can identify the space H, with the completion of the space of distributions with respect to scalar produt {■,•},. The group U(p, q) acts in this space of distributions by the formula
Bs(g)f(z) = f{z[9])
(the formula doesn’t depend of s, neverless the scalar product and spectra of representation depend of s
essentially)
Denote by w(s) the integral
w(s) = j det5 (1 z z* )
J Bp,q
Then for all continuous functions on Bp<q with compact support
lim I Mz)Mz)dzdJ
5^+00 lj(s) J Bp,q
It is natural to think that the limit of kern-reprcsentations as s —► +00 is the canonical representation of U(p,q) in the space L2 on riemann symmetric space U(p,q)/(U(p) x U(q)).
2.7. Restriction to the compact orbit. The part of spectrum which corresponds to the compact orbit Eo is purely discrete and it consists of quite exotic representations of U(p, q) and this is relatively simple way for constructing singular unitary representations of U(p,q), see [NO].
2.8. Spherical kern-representations of other classical groups. All classical riemann noncompact symmetric spaces G/K (up to centrum of G) can be realized as matrix balls (see [Ner3]). Namely the space G/K is the space of matrices z over the field K = 1R. , C , 1HI (see below) satisfying the additional condition (see below) such that ||z|| < 1.
Now we are enumerate symmetric spaces G/K, fields and additional conditions.
1*. U(p,q)/(U(p) x U(q)) is the space of p x q matrices over C .
2* . Sp(2n,lR )/U(n) is the space of symmetric n x n-matrices over C .
3*. SO*(2n)/U(n) is the space of skew symmetric n x n- matrices over C .
4*. 0(p, q)/(O(p) x 0(q)) is the space of p x g-matrices over ffi .
5*. GL(n,ffi )/0(n) is the space of symmetric n x n-matrices over ffi .
6’. 0(n, C )/0{n) is the space of skew-symmetric n x n-matrices over ® .
7*, GL(n,C )/U(n) is the space of hermitian n x n-matrices over C .
8*. Sp(p, q)/(Sp(p) x Sp(q)) is the space of p x g-matrices over H .
9*. GL(n, H )/Sp(p, q) is the space of hermitian n x n-matrices over H .
10*. Sp(2n, C )/Sp(n) is the space of skew hermitian (i.e. z = — z*) n x n-matrices over HI .
In all cases enumerated above the group G acts on the matrix ball G/K by fractional-linear transformations
z 1—> z^ := (a + zc)_1(6 + zd) (5)
Now let us consider positive definite kernels on G/K having a form
Ls(z,u) = | det(l — zu*)|"2s
(conditions for positive-definiteness are different for different spaces) Consider the hilbert space defined by the positive defined kernel Ls(z, u). We identify this space with some space of real-analytic functions on G/K by the same way as in 2.1 . The group G acts in Hs by the unitary operators
As{g)f(z) = f(zlg])\det(a + zc)\~2s .
We say As is a spherical kern-rcpresentation of G.
2.9. Nonspherical kern-representations. Fix a matrix ball G/K and a finite dimensional euclidian space y. Denote by GL(y) the group of invertible linear operators in y. We say that a function
L : G/K x G/K - GL(y)
is a matrix-valued positive definite kernel if the function
L({z,0-,iu,r])) :=< K(z, u)£, 77 > ; (z, £), (u, r/) 6 G/K x y
is a positive defined kernel on G/K x y.
Let we have a matrix-valued positive defined kernel on G/K. Then there exist a hilbert space H and
a map : G/K x y such that
a) The map ^ is linear on each fibre zxycG/ICxy.
b) < $(z,0, n) >H=< K(z,u)Z,r] >y
c) The image of map is dence in H.
For each h E H we define a function /;, : G/K —» 3^ by the rule < //,,£ >y=< /1,^(2,^) >//.
Consider a symmetric spaces having the form 2’, 3', 5“, 6“, 7”, 9', 10*. Consider a finite dimensional
irreducible representation p of the group GL{ji,'K ) or of its universal covering. Assume that the function
L(z, u) = p( 1 — zu*)
is a matrix valued positive definite kernel. Then we consider the associated hilbert space Hp of real-analytic functions G/K —> y and the unitary kern-representation of G in Hp given by the formula
TP(g)f{z) = p(a + zc)f{z[3]) (6)
Consider the cases r,4*,9*. Consider a finite-dimensional irreducible representation p = pi <g> p2 of the
group GL(p,K ) x GL(q,K ) or of its universal covering. Assume that the function
Lp(z, u) pi(l - zu*) ® p2{ 1 - u*z)
is a positive defined matrix-valued kernel on G/K. Then the group G acts in the associated space of
real-analytic functions on G/K by the formula
Tp(g)f(z) = (pi(a +zc)® p2(d-cz[g]))f(zl3])
Remark. Our arguments from subsections 2.5 are valid for general kern-representations.
2.10. Another description of kern-representations (see [00]). For G = Sp(2n,E.), U(p,q),
SO* (2n) a kern representation is a tensor product of irreducible highest weight representation of G and
irreducible lowest weight representation of G.
In other cases a kern representation of G is a restriction of a highest weight representation of the group G* to the symmetric subgroup G:
4*. G = 0(p,q) G* — U(p, q) 5*. G = GL{n, 1) G* = Sp(2n,l)
6*. G = 0(n, C ) G* = SO*{‘In) V G = GL(n, C ) G* = U(n,n)
8* G = Sp(p,q) G* = U{2p,2q) 9* G = GL(n,M) G* = SO*(2n)
10* G — Sp(2n, C ) G* = Sp(4rc,E )
Remark. The cases 1* - 3* can be described by the same way . We have G* = G x G and the embedding G -+ G* is given by the formula g (g, ge) where 9 is the outer automorphism of G.
Remark. There are some additional possibilities related to highest weight representations of 0(p, 2) and two exeptional groups (see [00]).
2.11. Action of Olshanskii semigroup, (see [Olsl]) For each matrix ball G/K consider the set
of matrices g = ^ ° such that the mapping (6) maps the matrix ball to itself. Obviously T is a
semigroup and the group G is the group of invertible elements of F. The formula (6) defines representation of semigroup T. This representation is irreducible and all irreducible representations of T can be obtaned by this way(see[01sl]). See [Olsl], [Ner3],[Ner4],appendix A, for the explicit desvription of semigroups T. Moreover kern-representations extends to representations of some categories(see[Ner3],[Ner4],appendix
A)- . -
I don’t know any applications of these phenomena to harmonic analysis of kern-representations of groups.
2.12. Bibliographical comments.
a) Let T be a highest weight representation of G = Sp(2n,R ),SO*(2n), U(p,q), . .. and 5 be a lowest weight representation of G. Assume that T, S be elements of Hansh-Chandra discrete series. Then T®S is equivalent to a representation of G induced from irreducible representation of K (see [Ber2],[Rep],[Gut]).
b) Restriction of a spherical highest weight representation of G* to the symmetric subgroup G (i.e spherical kern-representation, see notations of 2.10 ) is equivalent to canonical representation of G in
L2(G/K),see [00]. . . . T
c) Discrete spectra associated to the compact orbit in Shilov boundary were investigated in [NO] for the case G = 0(p,q). Analogical results are valid for G - (7(p, g), 5p(p, g)([01s2],[NO],7.12.). One method of separation of discrete spectrum is discussed in [Nerl], [NO],7.1-7.8. I think that the restriction operator to the compact orbit doesn’t exist for G ^ U(p, q), Sp(p, q), 0(p,q).
d) The spectrum for spherical kern-representation of U(2,2) was obtained in [0Z2] . For 1 < s < 3/2 the spectrum consist of two different pieces. One of pieces coincides with the spectrum of L2(U(2, 2)/{U{2) x U(2)). Another piece is a integral of notrivial representations. It is natural to think (it is not proved) that this piece of spectrum is associated to noncompact U(2, 2)-orbit in the Shilov
boundary of B2 2 x B2,2- .
If it is so it is the unique known case when spectrum associated to noncompact orbit is observed. It is natural to think that such spectra exist in various spectral problems(not only for kern-representations).
e)Spectrum of spherical kern-representations of U(p, 1) is obtained in [Dij].
f) Plancherel formula for tensor product of highest weight and lowest weight representations of SL(2,M ), see [Mol4]. I don’t know analogical results for other kern-representations.
g) Nonspherical kern-representations have discrete spectrum which is not associated to compact orbit. Some possibilities to observe it are contained in the following two sections. A way to observe Harish-Chandra discrete series increments using trace theorems is proposed in [Ner2],
h) Let p be the same as in 2.10. Let p,(7) = det-5(7)^(7). Then limit of TPi as s —► 00 is the representation of G induced from finite dimensional representation of K (i.e the representation in sections of vector bundle on G/K). The theory of such representations is more or less equivalent to Harish-Chandra theory of L2(G).
3 Dual pairs '
3.1. Spectrum of dual pairs. Consider the harmonic representation W2n (= Weil representation = Segal-Shale-Weil representation = Friedrichs-Segal-Berezin-Shale-Weil representation = oscilator representation ) of the group Sp(2n,№. ) (see [KV],[Ner4]) for discussion of this representation).
Consider the following subgroups in the simplectic group it (noncompact Howe dual pairs):
Sp(2k(p + q), ffi ) D Sp(2k, R ) x 0(p, q)
Sp(2(k + l)(p + q),R) 3 U(k,l) X U(p,q)
Sp(4k(p + q), 1) D SO*(2k) x 5p(p, q)
Let us restrict Win to these subgroups and then let us restrict to
0(p, q), Sp(2k, 1 ), U(k, /), U(p, q), SO*(2k), SP(P, q)
It was proved in [Ada],[Li] that the spectra of these restrictions have discrete increments. This construction is one of standard way to obtain singular unitary representations of groups U(p, q), 0(p, q), Sp(p, q).
Proposition 3.1. (see [NO]) Each representation ofG — 0(p, q),Sp(2k, R ), U(k, I), U(p, q), SO*(2k), Sp(p, q) which occurs in spectra of dual pair discretly (resp. weakly) occurs in spectra of some kern-representation discretly (resp.weakly).
This proposition is more or less obvious. Concider for instance the case Sp(2k,W. ) x 0(p,q). The restriction of W2k(p+q) to the subgroup
Sp(2k, R ) C Sp(2k, R ) x 0(p, q) C Sp(2k(p + q), M )
is equivalent to the representation
KP ® TC)®’ (7)
First tensor factor is a direct sum of highest weight representations and second tensor factor is a direct sum of lowest weight representations . Hence (6) is a direct sum of kern-representations.
Consider the following subgroups in Sp(2k{p + q), E ):
Sp(2k(p + q), R ) D 0(p,q) x Sp(2k,Hk) D 0(p,q)
|| nun
Sp(2k(p + q),M) D U(p,q) x U(k) D U(p,q)
The restriction of W2k(P+q) to U(p,q) is a direct sum of highest weight representations and hence the
restriction of W2k(p+q) to 0(p,q) is a direct sum of kern-representations.
3.2. Restriction to orbits. We realize the group Sp(2N, E ) as the group of (Ar + N) x (AT + N)-
. . ( $ \ . . matrices with complex coefficients having the form g = I — 1 and satisfying the condition
1 0 \ t /10
9 ' 1 0 1 ) 9 \ 0 1
Denote by Cn the space of complex symmetric N x TV-matrices z . The group Sp(2n}M. ) acts on Cjv by the fractional linear transformations and we have Cn = Sp(2N, E )/U(N). Consider a reproducing kernel
K(z, u) = det-1^2(l — zu*)
on Cat. Denote by H the associated hilbert space.. The group Sp(2N, R ) acts in H by the unitary operators
^2y5) = /(z^)det-1/2($ + z*)
The representation W^n is one of two irreducible components of the representation W2n-
Again we have question about restrictions of holomorphic functions to 0{p,q) x Sp(2k, 1 )-orbits in the Shilov boundary of CN. It seems that a orbit structure of Shilov boundary in this case is very complicated. In any case there exists a orbit
T - Sp(2k, E )/U(n) x 0{p,q)/Q
where Q is stabilizer of maximal isotropic subspace in pseudoeuclidean space E p+q . I can show, that operator of restriction to orbit T exists and this observation give a way to observe a part of discrete spectra for dual pair. It is interesting to calculate this part of spectrum.
Another question which seems interesting to me: is it possible to obtain by such way some handble realizations of some Hansh-Chandra discrete series representations.
4 Space L2 on Stiefel manifolds
4.1. Stiefel manifolds. We name by Stiefel manifolds the following 10 types of homogeneous spaces
Q IQ.
1°. 0(p,q)/0(p-t,q- s) 2°. U{p,q)/U(p-t,q- s)
3°. Sp(p,q)/Sp(p — t,q — s) 4°. Sp(2n, E )/Sp(2(n — t), E )
5°. Sp(2n,C)/SP(2(n-t),C) 6°. 0{n,C)/0(n-t, C)
7°. SO*(2n)/SO*(2(n -1))
8° - 10°. The spaces of all linear embeddings
I»-'-,!'1 cn"t-Cn HP-1—Hn
In the last 3 cases the group G is GL(n, E ), GL(n, C ), GL(n, HI) respectively and Q is the group of matrices having the form
( 0 * )
Remark The Stiefel manifold Sp(2n, E )/Sp(2(n - t), E ) is the space of isometric embeddings of the space K 21 equipped with a nondegenerate skew symmetric bilinear form to the space E equipped with nondegenerated skew symmetric bilinear form. Other Stiefel manifolds 1° - 7° have the analogical descriDtion
4.2. Additional symmetries. Consider the case G/Q = Sp(2n, E )/Sp{2(n — t), E ). Then the eroup Sp(2t,R ) acts by the obvious way on the space of symplectic-isometnc embeddings E
E 2"(sinse it acts on the space E 2‘) . Hence the manifold 5p(2n,E )/Sp(2(n - t),R ) is a Sp(2t,R ) x 5p(2n,E )-homogeneous space:
5p(2n,E )/Sp(2(n -i),® ) = {Sp{2t,R ) x Sp(2n,E ))/(Sp(2*,R ) x Sp(2(n - t), E ))
Analogical additional group of symmetries exists in all cases 1° - 10°. These additional symmetries are useful since the spaces L2(G/Q) have G-spectrum of infinite multiplicity.
4.3. Spectra of L2 on Stiefel manifolds. A few is known about spectral decomposition of L2(G/Q). Neverless it is known that this problem is interesting. See [Sch] and [Kob] for Flensted-Jensen type constructions of discrete spectra in L2 on
0{p,q)/0(p-r,q) U{p,q)/U(p-r,q) Sp(p, q)/Sp(p - r, q)
For the case r = 1 the Plancerel formula is obtained in [OZ1]. Some constructions for discrete increments to spectra of
L2{U{p,q)/(U(p-t,q-t) X U(p) X U{q))) C L2{U(p, q)/{U(p) X U(q)))
are contained in [RSW].
The cases G/Q where G = GL(n,E ),GL(n,C ),GL(n,M ) are very simple.
Proposition 4.1. Each representation of G which is contained in spectra L2(G/Q) discretly(resp. weakly) is contained in specrtrum of some kern-representation of G discretly (resp. weakly).
Proof. We use arguments from [How],[NO], Concider for instance the case G = Sp(2n,R). The representation ’
of Sp(2n, 1 ) is equivalent to representation of Sp(2n, 1) in I2(l 2n). Hence the representation
(w2n ® w2*n f2k = wf2k ®{w;n)®2k
is equivalent to representation of Sp(2n,R) in L2 on the space Mat2k]2n of all 2k x 2n-matrices. A generic orbit of Sp(2n, 1) in Mat2k<2n is a Stiefel manifold Sp(2n, 1 )/Sp(2(n - k), 1).
Remark. For other groups G Proposition 6.1 can be proved by the same arguments. The basic observation is
W-
2 n
(see real model of harmonic representation in [KV]).
4.3. Some pseudoriemann symmetric spaces. By the obvious way we have
L2(0(p,q)/(0(p) X 0(p-r,q))) C L2(0{p,q)/0(p-r,q))
L2(U(P>l)/(U(r)xU(p-r,q))) C L2(u(p,q)/U(p~r,q))
L2(Sp(p, q)/(Sp(r) X Sp(p - r, q))) C L2 (Sp(p, q)/Sp(p - r, q))
and spectra of these spaces are contained in spectra of Stifel manifolds. I don’t know such embeddings of spectra for other pseudoriemannian symmetric spaces.
References
[Ada] Adams J. Discrete spectrum of reductive dual pair 0(p, q), Sp(2m). Inv.math., 74(1983).449-475.
[Bar] Bari N.K. Treatise on trigonometric series. Fizmatgiz (1961) (Russian) ; English translation Perg-amon Press (1964)
[Berl] Berezin F.A. Quantization in compex symmetric spaces Izv. Akad.Nauk SSSR, Ser.Mat., 39(1975)(russian); English translation Math.in USSR Izv. 9(1975) ’ ’
[Ber2] Berezin F.A. A connection between the co- and contravariant symbols of operators on classical symmetric spaces Dokl. Akad. Nauk SSSR, 241 (1978), Nl(Russian); English translation in Sov. Math. Dokl., 19(1978), 786-789.
[Boy] Boyer Ch.P. On complementary series of SO(p, 1) J.Math.Phys., 14(1973), 609-617
[BO] Bunke U., Oldrich Spectrum of Keinian manifolds, to appear
[Dij] van Dijk G.Canonical repesentations, to appear.
[EHW] Enright T.J., Howe R., Wallach N. Classification of unitary highest weight modules. Representation theory of reductive groups , Birkhauser,97-143(1983).
[Far] Faraut J. Distributions spheriques sur les spaces hyperbohque. J.Math.Pure Appl. 58(1979),369-444
[F-J] Flensted-Jensen M. Analysis on Non-Riemanian symmetric spaces Regional Conference series in Math.61, AMS, Providence, 1986 '
[GG] Gelfand I.M., Gindikin S.G.Complex manifolds whose skeletons are semisimple lie groups, and analytic discrete series of representations. Funkt.Anal i Prilozh. 11(1977),N4(Russian) ; English translation Funct.Anal.Appl 11,258-265
[Gut] Gutkin
E. Coeffitients of Clebsh-Gordon for the holomorphic discrete series. Lett.Math.Phys.3(1979),185-
[How] Howe R. On some results of Stnshartz and of Rallis and Schiffmann J.Funct.Anal., 32(1979) 297303
[Ism] Ismagilov R.S. On the representations of Lorents group which are unitary in indefinite metric.. Trudy MIEM,2(1996), 492-504. (Russian)
Kashwaxa M., Vergne M. On Segal-Shale-Weil representation and the harmonic polynomials. Inv.Math.44 (1978) , 1-47
[Kobl] Kobayashi T.Singular unitary representations and discrete series for indefinite Stiefel manifolds. Mem. AMS,95 (1992)
[Kob2] Kobayashi T. Discrete decomposability of the restrictions of Aq(\) with respect to reductive subgroups and its applications. Inv.Math.117(1994), 181-205.
[Li] Li J.S. Singular unitary representations of classical groups. Inv.Math., 97(1989), 235-257.
[Moll] Molchanov V.F. Analogue of the Plancherel formula for hyperboloids. Dokl.Akad.Nauk SSSR 171(1966) (Russian) ; English translation Sov.Math.Dokl 7( 1968), 1382-1385
[Mol2] Molchanov V.F. Restriction of complementary series representations to pseudoortihonal group of lower dimension. Dokl.Akad.Nauk SSSR, 237(1977),N4(Russian) ; English translation Sov.Math.Dokl. 18(1977), 1493-1497
[Mol3] Molchanov V.F. The Plancerel formula for hyperboloids. Trudy MIAN, 147 (1980) 65-85 (Russian)
; English translation Proc.Steklov Inst.,Math. 147, (1981),63-83
[Mol4] Molchanov V.F. Tensor products of unitary representations of 3-dimensional Lorents group Izv.Akad.Nauk SSSR,ser.Mat. 43(1979), N4(Russian) ; English translation Math USSR Izv., 15
(1980), 113-143
[Nail Naimark M.A. Decomposition of tensor product of irreducible representations of Lorents group into irreducible components.Ill Trudy Moscovskogo Matem. Obshchestva 10(1961), 181-216.(Russian)
[NR] Nagel A., Rudin W. Local boundary behavoir of bounded holomorphic functions. Can. J. Math. ,30(1978), 583-592
fNerl] Neretin Yu A. Representations of complementary series entering discretely in tensor products of unitary representations. Funkt.Anal i Pnlozh. 20(1986)N4,79-80(Russian) ; English translation Funct.Anal.Appl 20, 68-70
[Ner2] Neretin Yu. A. Restriction of function which is holomorphic in a domain to a curve m boundary, and discrete SL(2,M)-spectra. (to appear)
[Ner3] Neretin Yu.A. Extensions of representations of classical groups to representations of categories. Algebra i Analiz 3(1992), Nl(Russian) ; English translation St-Petersburg Math. J. 3(1992), 147-169
[Ner4] Neretin Yu.A. Categories of symmetries and infinite dimensional groups. Oxford Univ. Press (1996)
TNOl Neretin Yu A., Olshanskii G.I. Boundary values of holomorphic functions, singular unitary representations of groups 0(p,q), and their limits as q - oo. Zap.Nauchn.Semin.POMIRAN, 223(1995) 9-91 (Russian) ; English translation J.Math.Sci(1996).
[OO] Olafsson G., 0rsted B. Bargmann transform for symmetric spaces (to appear)
[Ols 1] Olshanskii G.I. Invariant cones in Lie algebra, Lie semigroups, and the holomorphic discrete series. Funkt.Anal i Prilozh. 14(1980),N3 (Russian) ; English translation Funct.Anal.Appl 15(1981)
[01s2] Olshanski G.I. Singular unitary representations of the groups 0(p, q),U(p, q), Sp(p, g)(nonpublished notes) '
r01s3l Olshanskii G I Unitary irreducible representations of the groups U(p, q) which admit the pass to the limit asq-> oo Zap.Nauchn.Semin.LOMI 172(1989), 114-220.(Russian) ; English translation
J.Sov. Math. (199 )
[OZ1] Orsted B., Zhang G. L2-version of Howe correspondence II. J.Math.Pure.Appl., 74(1995), 165-183.
[0Z2] 0rsted B., Zhang G. Tensor products of analytic continuations of discrete series. Can J Math fto appear) 'v
[pat] Patterson S.3 The Laplacian operator on a nemann surface. I,II,III Compos.Math. 31(1975) 83107, Compos.Math. 32(1976), 71-112, Compos.Math. 33(1977) , 227-259. '
[Puk] Pukanszky L On the Kronecker products of irreducible representations of 2x2 real ummodular group. Trans. Amer. Math. Soc.,100 (1961), 116-152
[RSW] RawnsleyJ Schmid W., Wolf J. Singular unitary representations and indefinite harmonoc theory J.Funct.Anal.,01(1983),1-114. y
[RS] Reed M, Simon B. Methods of modern mathematical physics. II. Fourier analysis. Self-adjointness Acad.Press (1975). '
[Rep] Repka J. Tensor products of holomorphic discrete series. Can.J.Math. 31(1979), 836-844
[Rud] Rudin W.The function theory in the unit ball in Cn. Springer(1980).
[Sch] Schhchtkrull H.A series of unitary irreducible representations induced from symmetric subgroup of a semisimple Lie group. Inv.Math.,68(1982), 497-516.
[Str] Stnshartz R.S. Harmonic analysis oh hyperboloids J.Funct.Analysis, 12(1973), 341-383.
[Tsu] Tsuchikawa The Plancherel transform on SL2{k) and its applications to tensor products of irreducible representations J.Kyoto Univ., 22(1982), 369-473.
