Semicontinuity of the Automorphism Groups of Domains with Rough Boundary

Abstract

Based on some ideas of Greene and Krantz, we study the semicontinuity of automorphism groups of domains in one and several complex variables. We show that semicontinuity fails for domains in ℂ^𝑛, 𝑛>1, with Lipschitz boundary, but it holds for domains in ℂ¹ with Lipschitz boundary. Using the same ideas, we develop some other concepts related to mappings of Lipschitz domains. These include Bergman curvature, stability properties for the Bergman kernel, and also some ideas about equivariant embeddings.

1. Introduction

A domain in ℂ^𝑛 is a connected open set. If Ω is a domain, then we let Aut(Ω) denote the group (under the binary operation of composition of mappings) of biholomorphic self-maps of Ω. When Ω is a bounded domain, Aut(Ω) is a real (never a complex) Lie group.

A notable theorem of Greene and Krantz [1] says the following.

Theorem 1.1. Let Ω₀ be a smoothly bounded, strongly pseudoconvex domain with defining function 𝜌₀ (see [2] for the concept of defining function). There is an 𝜖>0 so that, if 𝜌 is a defining function for a smoothly bounded, strongly pseudoconvex domain Ω with ‖𝜌₀−𝜌‖_𝐶^𝑘<𝜖 (some large 𝑘) then the automorphism group of Ω is a subgroup of the automorphism group of Ω₀. Furthermore, there is a diffeomorphism Φ∶Ω→Ω₀ such that the mapping Aut(Ω)∋𝜑↦Φ∘𝜑∘Φ⁻¹ (1.1) is an injective group homomorphism of Aut(Ω) into Aut(Ω₀).

In what follows we shall refer to this result as the “semicontinuity theorem.”

It should be noted that, although this theorem was originally proved for strongly pseudoconvex domains in ℂ^𝑛, the very same proof shows that the result is true in ℂ¹ for any smoothly bounded domain Ω₀. In fact the proof, while parallel to the original proof in [1], is considerably simpler in the one-dimensional context.

The original proof of this result, which was rather complicated, used stability results for the Bergman kernel and metric established in [3] and also the idea of Bergman representative coordinates. An alternative approach, using normal families, was developed in [4]. The paper [5] produced a method for deriving a semicontinuity theorem when the domain boundaries are only 𝐶². The more recent work [6] gives a new and more powerful approach to this matter of reduced boundary smoothness. The paper [7] gives yet another approach to the matter and proves a result for finite type domains.

It is geometrically natural to wonder whether there is a semicontinuity theorem when the boundary has smoothness of degree less than 2. On the one hand, experience in geometric analysis suggests that 𝐶² is a natural cutoff for many positive results (see [8]). On the other hand, Lipschitz boundary is very natural from the point of view of dilation and other geometric operations.

The purpose of this paper is to show that the semicontinuity theorem fails for domains in ℂ^𝑛, 𝑛>1, with Lipschitz boundary. But it holds for domains in ℂ¹ with Lipschitz boundary. The reason for this difference is connected, at least implicitly, with the failure of the Riemann mapping theorem in several complex variables. We shall explain this point in more detail as the presentation develops.

2. The Several-Complex-Variable Situation

The main result of this section is the following.

Theorem 2.1. Let 𝑛>1 and consider domains in ℂ^𝑛. There is a sequence Ω_𝑗 of strongly pseudoconvex domains with Lipschitz boundary and another domain Ω with Lipschitz boundary so that Ω_𝑗→Ω in the Lipschitz topology on defining functions and so that(a)for each 𝑗,Aut(Ω_𝑗)=ℤ;(b)Aut(Ω)={𝑖𝑑}.

See [9] for a consideration of strongly pseudoconvex domains with less than 𝐶² boundary. This result shows that the semicontinuity theorem fails for domains with Lipschitz boundary.

It should be understood that all the domains considered in this paper have finite connectivity. In particular, the complement of the domain only has finitely many components. And each component of the complement has Lipschitz boundary. We do not allow boundary components that are a single point. Each boundary component is the closure of an open set.

We shall use some ideas in [10] in constructing the example enunciated in the theorem. We shall make our construction in ℂ². But it is easy to produce analogous examples in any ℂ^𝑛.

Proof of the Theorem. Let 𝜓∈𝐶^∞_𝑐(ℂ^𝑛) be such that (i)supp𝜓⊆𝐵(0,1); (ii)𝜓≥0; (iii)𝜓(0)=1.
We will build our domains by modifying the unit ball 𝐵 in ℂ². We will make particular use of these automorphisms of the unit ball, for 𝑎 a complex number of modulus less than 1: Ψ_𝑎𝑧₁,𝑧₂=𝑧₁−𝑎1−𝑎𝑧₁,1−|𝑎|²𝑧₂1−𝑎𝑧₁. (2.1) See [11].
We define 𝜂₁𝑧₁,𝑧₂=−1+𝑧₁²+𝑧₂²−1𝜓𝑧1010₁,𝑧₂−34,12. (2.2) Set 𝑈₁=𝑧₁,𝑧₂∈ℂ²∶𝜂₁𝑧₁,𝑧₂<0. (2.3) Clearly 𝑈₁ is a domain with smooth boundary. It is a ball with a “bump” attached at the point (3/4,1/2).
Now define Ω₁=^∞_𝑗=−∞Ψ^2𝑗_1/10𝑈₁. (2.4) We see that Ω₁ has infinitely many bumps which accumulate at the points (1,0) and (−1,0). It is because of those accumulation points that the boundary of Ω₁ is only Lipschitz.
In general we let, for 𝑘≥2, 𝜂_𝑘𝑧₁,𝑧₂=−1+𝑧₁²+𝑧₂²−110^𝑘×𝜓10^𝑘𝑧₁,𝑧₂−12^𝑘−1−12^2𝑘1,1−2^𝑘. (2.5) Set 𝑈_𝑘=𝑧₁,𝑧₂∈ℂ²∶𝜂_𝑘𝑧₁,𝑧₂<0. (2.6) Clearly 𝑈_𝑘 is a domain with smooth boundary. It is a ball with a “bump” attached at the point ((1/2)^𝑘−1−(1/2)^2𝑘,1−(1/2)^𝑘).
Now define, for 𝑘≥2, Ω_𝑘=Ω_𝑘−1∪^∞_𝑗=−∞Ψ^{2𝑗+𝑘−1}_1/10𝑈_𝑘. (2.7) We see that Ω_𝑘 has infinitely many bumps which accumulate at the points (1,0) and (−1,0). It is because of those accumulation points that the boundary of Ω₁ is only Lipschitz.
Finally we let Ω=^∞_𝑘=1Ω_𝑘. (2.8)
Now it is clear that Ω_𝑘→Ω in the Lipschitz topology on defining functions. Furthermore, the ideas in [10] show that the automorphism group of Ω_𝑘 consists precisely of the mappings Ψ^{2𝑗+𝑘−1}_1/10, 𝑗∈ℤ. So the automorphism group of Ω_𝑘 is canonically isomorphic to ℤ. But it is also clear that the automorphism group of Ω consists of the identity alone.
That completes the construction described in the theorem.

3. The One-Variable Situation

The one-variable result is the following.

Theorem 3.1. Consider domains in ℂ¹. Let Ω₀⊆ℂ¹ be a bounded domain with Lipschitz boundary and defining function 𝜌₀. If 𝜖>0 is sufficiently small then, whenever Ω is a bounded domain with Lipschitz boundary and defining function 𝜌 satisfying ‖𝜌₀−𝜌‖_Lip<𝜖 then the automorphism group of Ω is a subgroup of the automorphism group of Ω₀. Moreover, there is a diffeomorphism Φ∶Ω→Ω₀ so that the mapping Aut(Ω)∋𝜑↦Φ∘𝜑∘Φ⁻¹∈AutΩ₀ (3.1) is an injective group homomorphism.

We see here that the situation is in marked contrast to that for several complex variables. Our proof of this result will rely on uniformization for planar domains, a result which has no analogue in several complex variables.

Proof of the Theorem. Fix the domain Ω₀ and let Ω be of distance 𝜖 from Ω₀ in the Lipschitz topology.
It is a standard result of classical function theory that a finitely connected domain in the plane, with no component of the complement equal to a point, is conformally equivalent to the plane with finitely many nontrivial closed discs excised—see [12]. Call this conformal mapping the “normalization” of the domain. What is particularly nice about this result is that the proof is constructive and it is straightforward to see that the normalization of Ω is close to the normalization of Ω₀ just because Ω is close to Ω₀. Indeed the normalization of Ω will be close to that of Ω₀ in the 𝐶² topology. Just because once it is close in the Lipschitz topology then it is automatically close in a smoother topology (because the boundary consists of finitely many nontrivial circles).
Thus we may apply the one-dimensional version of the semicontintuity theorem for 𝐶² boundary to see that the automorphism group of the normalization of Ω is a subgroup of the automorphism group of Ω₀. And the diffeomorphism Φ exists as usual. Now we may use the normalizing conformal mapping to transfer this result back to the original domains Ω₀ and Ω.
That completes the proof.

We note that another approach to construct the normalization map is by way of Green’s functions. This method is also quite explicit and constructive. Stability results for elliptic boundary value problems are well known. So this again leads to a proof of the semicontinuity theorem by transference to the normalized domain.

4. Related Results in One Complex Dimension

Key to the work of Greene and Krantz in [3] and [1] is a stability result for the Bergman kernel. In that theorem, the authors consider a base domain Ω₀ and a “nearby” domain Ω. As usual, we define “nearby” in terms of closeness of the defining functions in a suitable topology. But it is useful to note that, in this circumstance, there is a diffeomorphism Π∶Ω→Ω₀, which is close to the identity in a suitable 𝐶^𝑘 topology. With this thought in mind, Greene and Krantz proved the following.

Theorem 4.1. Let Ω₀ be a fixed, smoothly bounded, strongly pseudoconvex domain. Let Ω be a domain which is “𝜖-close” to Ω₀ in a 𝐶^𝑘 topology. Let Π be the mapping described in the preceding paragraph. If 𝜖 is small enough, then the Bergman kernel 𝐾_Ω for Ω is close to 𝐾_Ω0∘Π in the 𝐶^𝑚 topology for some 0<𝑚<𝑘.

This result also holds in one complex dimension, and the proof in that context is actually much easier.

Our remark now is that this theorem is actually true in the Lipschitz topology. We use the argument of the last section. Namely, if Ω is close to Ω₀ in the Lipschitz topology, then the normalization of Ω is close to the normalization of Ω₀ in a smooth topology. This one-dimensional version of Theorem 4.1 applies to the normalized domains. The result follows.

5. Equivariant Embeddings

A lovely result of Maskit [13] is the following.

Theorem 5.1. Let Ω⊆ℂ be any planar domain. Then there is a univalent, holomorphic embedding Φ∶Ω→ℂ so that the automorphism group of the image domain Φ(Ω) consists only of linear fractional transformations.

An elegant corollary of Maskit’s result is that if 𝜑 is any automorphism of a planar domain that fixes three points then 𝜑 is the identity mapping. This follows because it is clear that any linear fractional transformation that fixes three points is the identity.

We would like to remark here that the ideas in this paper give a “poor man’s version” of this theorem. For let Ω be any domain with Lipschitz boundary as we have been discussing. So each component of the complement is the closure of a region having Lipschitz boundary. Now the normalizing map sends this domain Ω to a planar domain bounded by finitely many disjoint circles. It is easy to see, using Schwarz reflection and Schwarz’s lemma, that any conformal self-map of such a domain must be linear fractional. So any such map that fixes three points must be the identity.

6. The Bun Wong/Rosay Theorem

A classical result in several complex variables is the following (see [14, 15]).

Theorem 6.1. Let Ω⊆ℂ^𝑛 be a bounded domain. Let 𝑃∈𝜕Ω and assume that 𝜕Ω is strongly pseudoconvex in a neighborhood of 𝑃. Suppose that there are a point 𝑋∈Ω and automorphisms 𝜑_𝑗 of Ω such that 𝜑_𝑗(𝑋)→𝑃 as 𝑗→∞. Then Ω is biholomorphic to the unit ball.

In a similar spirit, Krantz [16] proved the following result.

Theorem 6.2. Let Ω⊆ℂ be a bounded domain and let 𝑃∈𝜕Ω have the property that 𝜕Ω near 𝑃 is a 𝐶¹ curve. Suppose that there are a point 𝑋∈Ω and automorphisms 𝜑_𝑗 of Ω such that 𝜑_𝑗(𝑋)→𝑃 as 𝑗→∞. Then Ω is conformally equivalent to the unit disc.

In this section we will reexamine Theorem 6.2 in the context of this paper, that is, in relation to finitely connected domains with Lipschitz boundary. As noted, such a domain is conformally equivalent to a domain Ω whose boundary consists of finitely many circles. Now we have the following possibilities.(a)If 𝜕Ω consists of just one circle, then Ω is the disc, and there is nothing to prove. (b)If 𝜕Ω consists of two circles, one inside the other, then Ω is (conformally equivalent to) an annulus. Then the automorphism group of such a domain is two copies of the unit circle. In particular, it is compact. So the hypotheses of Theorem 6.2 do not obtain.(c)If 𝜕Ω consists of two circles, neither of which is inside the other, then the domain is unbounded. The automorphism group of such a domain is compact, and the hypotheses of Theorem 6.2 do not apply.(d)If 𝜕Ω consists of at least three circles, with all the circles but one lying inside the other one, then it is well known (see [17] or [18]) that the automorphism group of Ω is finite. Then the hypotheses of Theorem 6.2 do not obtain.

Thus, we see by inspection that Theorem 6.2 is true in the context of the domains that we have been discussing in this paper.

7. Curvature of the Bergman Metric

It is a matter of considerable interest to know the curvature properties of the Bergman metric on a planar domain. In particular, negativity of the curvature near the boundary is a useful analytic tool (see [3]). If Ω is a planar domain with Lipschitz boundary, then its normalized domain is bounded by finitely many circles. The asymptotic boundary behavior of the Bergman kernel on such a domain is very well understood—see [19]. In particular, the kernel near a boundary point 𝑃 is asymptotically very much like the kernel for the disc. Thus, a straightforward calculation confirms that the curvature of the Bergman metric near the boundary is negative. Of course this statement pulls back to the original domain in a natural way.

8. Closing Remarks

It is natural to want to consider the results presented here in either the 𝐶¹ topology or even the 𝐶^2−𝜖 topology. At this time the techniques are not available to attack those questions.

In several complex variables, one would also like to prove semicontinuity theorems for broad classes of domains. This will be the subject for future papers.