The cosmological constant problem is principally concerned with trying to understand how the zero-point energy of quantum fields contributes to gravity. Here we take the approach that by addressing a fundamental unresolved issue in quantum theory, we can gain a better understanding of the problem. Our starting point is the observation that the notion of classical time is external to quantum mechanics. Hence there must exist an equivalent reformulation of quantum mechanics which does not refer to an external classical time. Such a reformulation is a limiting case of a more general quantum theory which becomes nonlinear on the Planck mass/energy scale. The nonlinearity gives rise to a quantum-classical duality which maps a “strongly quantum, weakly gravitational” dynamics to a “weakly quantum, strongly gravitational” dynamics. This duality predicts the existence of a tiny nonzero cosmological constant of the order of the square of the Hubble constant, which could be a possible source for the observed cosmic acceleration. Such a nonlinearity could also be responsible for the collapse of the wave function during a quantum measurement.

1. Introduction

The observed notion of time, with which we are so familiar, is external to quantum mechanics. It is part of a classical spacetime geometry, which comprises of a spacetime manifold and the metric. The metric is determined by classical matter fields via the field equations of general relativity. In principle, the Universe could be in a state in which there are no classical matter fields, but only quantum fields. In such a situation, the metric of the Universe will in general no longer be classical, but will undergo quantum fluctuations. It is known from the Einstein hole argument that in order for the spacetime manifold to have a physically meaningful point structure, a well-determined classical metric (which is a solution of the Einstein equations) must reside on the manifold. When the metric is undergoing quantum fluctuations, the point structure of the spacetime manifold is destroyed, and one no longer has a classical notion of time [1].

Nonetheless, one should be able to describe the dynamics of a quantum system, even if an external classical time is not available. Such a description should become equivalent to standard quantum mechanics as and when a dominant part of the Universe becomes classical, so that a classical time now exists. In arguing for the existence of such a reformulation, one is led to conclude that standard linear quantum theory is a limiting case of a more general quantum theory which is nonlinear on the Planck mass/energy scale [2]. This conclusion is independent of any specific mathematical structure which one would like to use to develop the reformulation.

What is our most reliable guideline towards the construction of such a reformulation of quantum mechanics? A natural mathematical structure which forgoes the point structure of spacetime is a noncommutative spacetime. We construct the reformulation by pursuing the following proposal: in the reformulation, relativistic quantum mechanics is the same theory as noncommutative special relativity, with a specific set of commutation relations imposed on noncommuting coordinates and momenta. The physical principle is that the basic laws are invariant under “inertial” coordinate transformations of noncommuting coordinates. One is naturally led to attach an antisymmetric part to the Minkowski metric. The theory is supposed to describe dynamics when gravity can be neglected (like in special relativity). In the present context, this amounts to the requirement that the total mass/energy in the system be much smaller than Planck mass/Planck energy. As and when an external time becomes available, this reformulation should become equivalent to standard quantum mechanics. These aspects will be described in Section 2.

The nonlinear generalization of this reformulation, a hitherto unnoticed feature which arises naturally, describes the dynamics of the system when its energy becomes comparable to Planck energy. The Schrödinger equation becomes nonlinear and the gravitational dynamics is now a noncommutative general relativity. The physical principle now is that basic laws are invariant under general coordinate transformations of noncommuting coordinates. This is supposed to generalize general covariance to the noncommutative case. When the mass/energy becomes much larger than the Planck scale, the dynamics is assumed to reduce to classical general relativity, and classical mechanics. This is discussed in Section 3.

The presence of the nonlinearity has two important consequences. Firstly, the antisymmetric part of the gravitational field associated with this nonlinearity suggests the existence of a quantum-classical duality, as a consequence of which one can match a dominantly quantum sector of the theory to a dominantly classical sector. This is the subject matter of Section 4. In turn this helps us understand why the cosmological constant should be nonzero and yet have the very small value it does. This is the main part of the paper, and it will be presented in Section 5.

The second important consequence of the nonlinearity has to do with the nonlinearity in the Schrödinger equation, which becomes relevant in the vicinity of the Planck mass scale. This can lead to a breakdown of quantum superposition, and it could lead to the collapse of the wave-function during a quantum measurement. What is important for us here is that the parameters influencing the collapse of the wave-function are in principle measurable in the laboratory. These are the same parameters which are responsible for the existence of the quantum-classical duality, and for the nonzero value of the cosmological constant. Thus our explanation for the origin of the dark energy is in principle testable experimentally, via the quantum measurement process. This aspect is investigated in Section 6.

The arguments of this paper suggest that a dynamically evolving “cosmological constant-like” term is present throughout the history of the Universe. At any given epoch, such a term is supposedly of the order of the square of the Hubble constant at that epoch. The cosmological viability of such a scenario will be discussed in Section 7.

In this paper we have attempted to keep the discussion compact, so as to provide an essential overview of the arguments. More detailed discussions can be found in [13].

2. Quantum Mechanics as a Noncommutative Special Relativity

The quantum dynamics of a relativistic particle of mass is described here as a noncommutative special relativity. Gravity is neglected in this small mass limit since this approximation is equivalent to setting . We outline here a proposal for the desired reformulation, using the illustrative case of a two-dimensional noncommutative spacetime described by coordinates . It should be said at the outset that our treatment is heuristic, and a rigorous mathematical description remains to be developed. We assume that associated with the 2d noncommutative spacetime, there is a line element:

which has an antisymmetric component. We call such a spacetime a quantum Minkowski spacetime, and the noncommuting coordinates are assumed to obey the commutation relations

We will comment on the function shortly.

We assume that a suitable differential calculus can be defined on this spacetime. Then, in analogy with special relativity, we introduce a velocity and a momentum . It is evident from the form of the line element (1) that the following Casimir relation holds

The specific structure of the commutation relations above is such that the momenta, as well as the coordinates, do not commute with each other. Moreover, while appears in one of the relations, it is which appears in the other relation. This is motivated by the expectation that one should be able to derive the uncertainty relations of quantum theory, and the quantum commutation relation from these underlying relations [2].

The function in (2) has to be chosen so that the momenta commute with the Casimir relation. It is easy to show that in fact there is no nontrivial solution in two dimensions; the only solution is , which is clearly not of interest. However, in dimensions three or higher, there appears to be no constraint that , although the exact form of remains to be found. Our subsequent discussion here does not depend on the form of , and it suffices to use the 2d example to illustrate our ideas.

Dynamics is defined by assuming that the momenta are gradients of a complex action . This converts the Casimir relation into a noncommutative Hamilton-Jacobi equation, which is the equation of motion. This is the theory we call a noncommutative special relativity.

As and when an external classical spacetime becomes available, the Klein-Gordon equation of standard linear quantum mechanics can be recovered from this reformulation by the correspondence rule

The justification for this rule has been discussed in [2]. On the right hand side of this equation, the momenta are again defined as the gradients of a complex action , and the wave-function defined as . Substituting for the wave function on the right-hand side of (4) and equating this expression to lead to the Klein-Gordon equation. In this sense one can recover standard quantum mechanics from an underlying formulation as a noncommutative special relativity.

3. A Noncommutative General Relativity

When the mass of the particle becomes comparable to Planck mass, its self-gravity can no longer be neglected.The noncommutative line element (1) is modified to the curved noncommutative line element

Correspondingly, the Casimir relation (3) is generalized to

and the correspondence rule is generalized (4) to

It is important now to note that if one rewrites this Hamilton-Jacobi equation in terms of the wave-function, one no longer gets the linear Klein-Gordon equation. This is because the metric appears in the equation. In the simplest case, where is a function of , and the diagonal components of the metric are approximated to unity, we get the equation of motion

which is equivalent to a nonlinear Klein-Gordon equation [2].

The noncommutative metric is assumed to obey a noncommutative generalization of Einstein equations, with the property that goes to one for , and to zero for . Also, as one recovers classical mechanics, and in the limit standard linear quantum mechanics is recovered.

In the mesoscopic domain, where is away from these limits and the mass is comparable to Planck mass, both quantum and gravitational features can be defined simultaneously, and new physics arises. The antisymmetric component of the gravitational field plays a crucial role in what follows.

4. A Proposed Quantum-Classical Duality

4.1. Motivation for the Duality

In general relativity, the Schwarzschild radius of a particle of mass can be written in Planck units as , where is Planck length and gm is the Planck mass. If the same particle was to be treated, not according to general relativity, but according to relativistic quantum mechanics, then one half of the Compton wavelength of the particle can be written in Planck units as . The fact that the product is a universal constant cannot be a coincidence; however, it cannot be explained in the existing theoretical framework of general relativity (because herein ) and quantum mechanics (because herein ).

One could attempt to trivialize this observation by saying that in general relativity, the only length scale that can be constructed is proportional to mass, and in relativistic quantum theory, the only length scale that can be constructed is inversely proportional to mass. However, what is non-trivial is that these both length scales have a fundamental physical meaning attached to them. Hence their inverse relation to each other does call for an explanation and is a signal that both general relativity and relativistic quantum theory must be limiting cases of a deeper underlying theory. In fact we have argued for the existence of such a theory in the previous section for entirely different reasons.

The only plausible way to explain this inverse relation is to propose a duality between a pair of solutions of the theory—a duality which maps the Schwarzschild radius for the first solution to the Compton wavelength for the second solution. Hence we propose and justify the following quantum-classical duality: the weakly quantum, strongly gravitational dynamics of a particle of mass is dual to the strongly quantum, weakly gravitational dynamics of a particle of mass .

It follows that the dimensionless Schwarzschild radius of is four times the dimensionless Compton-wavelength of .

The origin of this duality lies in the requirement that there be a reformulation of quantum mechanics which does not refer to an external classical spacetime manifold. The implied nonlinearity leads to a quantum gravity theory of which general relativity and quantum theory are natural approximations, and the duality is inevitable. Its existence does not depend on the use of noncommutative geometry for the mathematical formulation of the theory. The use of noncommutativity serves to illustrate and justify the duality.

The Planck mass demarcates the dominantly quantum domain from the dominantly classical domain and is responsible for the quantum-classical duality. As is evident from (8), the effective Planck constant is , going to zero for large masses, and to for small masses, as expected. Similarly, the effective Newton gravitational constant is expected to be , going to zero for small masses, and to for large masses.

Thus the parameter space is strongly quantum and weakly gravitational, whereas is weakly quantum and strongly gravitational. The Compton wavelength for a particle of mass gets modified to , and the Schwarzschild radius for a mass gets modified to . We propose that the dynamics of a mass is dual to the dynamics of a mass if . This holds if and

If (9) holds, the solution for the dynamics for a particle of mass can be obtained by first finding the solutions of (8) for mass , then replacing by , and finally writing instead of , wherever appears.

We can deduce the functional form of by noting that the contribution of the symmetric part of the metric, , to the curvature, grows as , whereas the contribution of the antisymmetric part must fall with growing . This suggests that grows linearly with ; thus

and implies ; and we set since this simply defines as the scaling mass. Hence we get , which satisfies (9) and thus establishes the duality. The mapping interchanges the two fundamental length scales in the two solutions: Compton wavelength and Schwarzschild radius.

The duality we observe is holographic, by virtue of the abovementioned relation . Thus, the number of degrees of freedom that a quantum field associated with the particle possesses (bulk property) should be of the order of the area of the horizon of the dual black hole in Planck units (boundary property), that is, . This value of could be interpreted as follows: the infinite number of degrees of freedom associated with a quantum field in the flat spacetime continuum limit (when no artificial high-energy cutoff has been imposed) has been replaced by this finite value. More correctly however, the effective number of degrees of freedom is actually of the order , because we have and so the highest energy associated with a mode of the quantum field cannot be more than Planck mass.

In summary, we see here a new picture for the dynamics of a particle. A particle need not be either quantum or classical, but there is a third possible kind of dynamics, mesoscopic dynamics, which interpolates between quantum and classical. This dynamics is described by a nonlinear Schrödinger equation (see (13) below). The nonlinear term depends on the newly introduced parameter , and its nature is such that the nonlinearity vanishes in the small mass limit, , . On the other hand, the nonlinear Schrödinger equation reduces to Newton's classical laws of motion in the limit , . This interpolating behaviour, where one makes a transition from quantum to classical mechanics via an intermediate nonlinear quantum mechanics, is not ruled out by experiment. Its verification or otherwise in the laboratory will constitute a crucial test of these ideas.

5. The Cosmological Constant Problem

The quantum-classical duality helps understand why there should be a cosmological constant of the order of the observed matter density, a possible explanation for the observed cosmic acceleration. If there is a nonzero cosmological constant term in the Einstein equations, of the standard form , it follows from symmetry arguments that in the noncommutative generalization of gravity, a corresponding term of the form should also be present. This latter term vanishes in the macroscopic limit but is present in the microscopic limit .

However, when , the effective gravitational constant goes to zero, so cannot be sourced by ordinary matter. Its only possible source is the zero-point energy associated with the quantum particle . Since this zero-point energy is necessarily nonzero, it follows that is necessarily nonzero. This same manifests itself on cosmological scales, where is nonvanishing, because is non-vanishing, even though goes to zero on cosmological scales, because goes to zero. Essentially we are saying that we have to examine the two limits of : the microscopic limit and the macroscopic limit; the value of arising at one of the limits will clearly be the same as its value at the other limit.

This solves the vexing problem of the cancellation of (i) a bare coming from general relativity and (ii) a coming from the zero-point energy of quantum fields. This problem arises in the first place because we have allowed ourselves to treat general relativity and quantum theory as completely disconnected theories. The nonlinearity of the theory suggested here, the consequent duality, and the introduction of the antisymmetric component of the metric compel us to treat the two theories as limiting cases of an underlying theory, and to conclude that the so-called bare and the “quantum ” are one and the same thing. The question of their mutual cancellation does not arise any longer.

The value of can be estimated by appealing to the deduced quantum-classical duality. The total mass in the observable Universe is , where is the present value of the Hubble constant. The mass dual to this is , and is roughly the magnitude of the zero-point energy in the ground state. In a higher mode, the energy is a multiple of the ground state energy, and we write it as , recalling that the effective Planck constant runs with energy. To obtain a rough estimate, we take to be one for energies up to and zero for energies beyond . We then see that the total contribution to the zero-point energy is

It is remarkable that Planck's constant drops out of the sum! The vacuum energy density, and hence the value of the cosmological constant, is which is of the order of the observed value of .

We note that the ground state energy is being mapped to a total energy , which is an instance of a UV-IR mixing, or equivalently, a quantum-classical duality. As goes to zero, the IR limit goes to zero, whereas the UV limit diverges.

Clearly, nothing in this argument singles out today's epoch; hence, it follows that there is an ever-present , of the order , at any epoch, with being the Hubble constant at that epoch. This solves the cosmic coincidence and fine-tuning problems. However, issues related to an ever-present will have to be addressed—we will return to this aspect in the last section.

5.1. Understanding

The standard quantum field theoretic cosmological constant problem does not arise here because we have brought in a new scale, the Hubble constant. In effect, we are proposing that since is the age of the Universe, there is a fundamental minimum frequency, that is, . All allowed frequencies are discrete multiples of , with the maximum being at Planck frequency. As a result, the net zero-point energy comes out to be . By itself, this is higher than , but we must recall that duality demands this much to be the classical contribution to the cosmological constant. Hence the energy density is found by dividing the total energy by the volume of the observed Universe, giving a value for that matches with observations. Thus although the argument for obtaining the magnitude of given here draws input from quantum theory, our argument is completely different in concept of what is suggested by quantum field theory. For us, duality is playing a crucial role.

6. Testing for Dark Energy Through Quantum Measurement

We would now like to suggest that the above proposal for the origin of the cosmological constant can in principle be tested in the laboratory by examining the quantum mechanics of mesoscopic systems, because the latter is also affected by the nonlinearity of the underlying theory.

Firstly, as discussed above, the effective Planck constant is , and using the form of that we have, we can write

Thus a measurement of the Planck constant for a “mesoscopic particle” with mass approaching of the Planck mass will show a deviation from the standard value. By particle, we mean a composite object in the required mass range whose internal degrees of freedom can be neglected.

Secondly, the nonlinearity can result in a breakdown of quantum superposition during a quantum measurement, leading to collapse of the wave-function and a finite lifetime for superpositions. A great deal has been written about the physics of quantum measurement over the last century or so. It is fair to say that there are essentially only two possibilities: either the wave function collapses during a quantum measurement, or it does not. If it does not, then the many worlds interpretation holds, and the different worlds do not interfere because of decoherence. If the wave function does collapse, then a modification of the Schrödinger equation in the mesoscopic domain is indicated. We have argued that the nonlinearity resulting from removal of external time favors the collapse picture [1].

As we discussed above, on the Planck mass/energy scale, the Klein-Gordon equation becomes nonlinear. In the nonrelativistic limit, it results in the following nonlinear Schrödinger equation:

This equation can be rewritten as

where and . Norm is preserved during evolution, provided the probability density is defined as .

Since nonlinearity is negligible for the quantum system, prior to the onset of a quantum measurement, evolution is described by

thus preserving superposition. The onset of measurement corresponds to mapping the state to the state of the final system as

where is the state the measuring apparatus would be in, had the initial system been in the state .

Evolution is now described by the equation

where and . is the total mass of the final system, which includes the quantum system as well as the measuring apparatus. The states cannot evolve as a superposition because the evolution is now non-linear. However, the initial state at the onset of measurement is a superposition of the . This superposition must thus break down during further evolution, according to the law

Note that the 's have been set to be different for different states. This is to be expected because will be determined by the quantum state, and setting it as a function only of to begin with was a leading order approximation, applied for simplicity. We thus get

and only the state with the largest survives [1]. In this manner, the inclusion of a nonlinear term breaks superposition.

In order to recover the Born probability rule, it is essential that the 's be random variables, with a suitable probability distribution. Only further development in theory can determine wherether the 's are indeed random, and if so, what their probability distribution is. A highly plausible candidate for a random variable is the phase of the quantum state at the onset of measurement. Although the phase evolves in a deterministic manner, it is effectively random, because the time at which the measurement begins is arbitrary.

From (18) we can define the lifetime of a superposition

Since is strictly equal to one in standard linear quantum mechanics, quantum superposition has an infinite lifetime in the linear theory. However, the situation begins to change in an interesting manner as the value of the mass approaches and exceeds . Since we know that in this limit approaches zero, we can neglect , and the superposition lifetime will then essentially be given by

We can get a numerical estimate by noting that we are close to the classical limit, where the phase coincides with the classical action in the Hamilton-Jacobi equation. To leading order, the magnitude of the classical action is given by , where is the time over which we observe the classical trajectory; approximately, this could be taken to be the value of the phase , and is then roughly given by

For a measuring apparatus, if we take the linear dimension to be, say, cm, and the time of observation to be, say, seconds, we get the superposition lifetime to be seconds. We can get a very rough estimate of for a mesoscopic system using (22), and taking  cm,  gm, and with . This gives seconds. Thus an experimental detection of dependence of superposition lifetime on the mass (equivalently number of degrees of freedom) of the system could be indicative of the nonlinearity.

The third possible way in which a nonlinearity of this nature can be detected is through rapid successive measurements of a quantum observable. Suppose a certain outcome for an observable results from the random variable being in a certain range . Suppose now that a second measurement is made sufficiently quickly with the eigenbasis slightly rotated. Because the random variable will not have changed to a value sufficiently different from the original one, the result of the second measurement will show a correlation with the result of the first measurement, contrary to what standard quantum mechanics predicts.

A more detailed discussion of the physics of measurement described here will be presented elsewhere [4].

7. Can There Be an Ever Present ?

A positive cosmological “constant” which is of the order of at every epoch is obviously not a constant and does not satisfy the standard equation of state . Furthermore, by increasing the rate of expansion in the early Universe, it spoils the consistency between theory and observation with regard to the abundance of light elements. It also makes galaxy formation more difficult later during the evolution of the Universe. It is thus evident that although the cosmic coincidence problem can be solved by an ever-present , one has to ascertain that the resulting cosmological model must be consistent with observation. A way out, as has been suggested by Sorkin, is to have a cosmological constant whose mean value is zero, but which has fluctuations with a typical magnitude of the order of [5]. As a starting point, this seems like a reasonable possibility for us also, considering that in our model, the origin of lies in the zero-point energy contribution coming from quantum theory. However, the development of a cosmological model in the context of our scenario is an issue we have not yet addressed, and we leave this for future investigation.

A phenomenological model for an ever-present has been partially developed in the context of the causal set approach to quantum gravity [6]. In this approach, a fluctuating of the order is predicted because is conjugate to the spacetime four volume, and this volume itself is subject to quantum fluctuations. The phenomenological model is specified by choosing a suitable equation of state for and expressing as a stochastic function of the four volume. A numerical study by the authors shows tracking behavior in , as well as fluctuations. For a suitable choice of a free parameter, a consistent with the present observed value is reproduced. It has however been pointed out by Barrow [7] that the model is very strongly constrained by the magnitude of the CMB anisotropy on the last scattering surface. It remains to be seen whether a way can be found out to overcome this constraint, by constructing an inhomogeneous version of the phenomenological model, or otherwise. An alternative investigation on the origin of based on quantum gravitational fluctuations has been carried out by Padmanabhan [8, 9].

On a more general note, we observe that the theoretical prediction of an ever-present nonzero cosmological “constant” of the order of is independent of the details of the cosmological model. Essentially, all we have assumed is a homogeneous and isotropic cosmology, but we have placed no a priori restrictions on the evolutionary history of the scale factor. Thus although we originally set out to seek an explanation for the observed cosmic acceleration in the framework of the standard Big Bang cosmology, we could turn things around and ask the following question: given an ever-present , does it admit a nonstandard cosmology consistent with observations? To us, the answer to this question is not obvious, and in our view the question merits further careful examination.

8. Concluding Remarks

Our use of noncommutative spacetime has a conceptually different origin as compared to applications based on the seminal work of Doplicher, Fredenhagen, and Roberts [10]. In the latter, spacetime noncommutation relations are deduced as a consequence of the joint application of quantum uncertainty relations and the rules of general relativity on the Planck length scale. One then envisages that quantum field theories exhibit effects induced by these spacetime commutation relations, on the Planck length scale. Also, on these scales general relativity could be assumed to be replaced by a noncommutative gravity theory which should eventually be quantized.

For us, the starting point has been that there should be a reformulation of quantum mechanics which does not refer to a classical time. This leads to the conclusion that linear quantum theory is a limiting case of an underlying theory which becomes nonlinear on the Planck energy scale. This is the principle difference from the theories referred to in the previous paragraph—the latter assume a strict validity of linear quantum theory at all scales. For us, this nonlinearity is responsible for the explanation of the tiny observed cosmological constant, and possibly also the collapse of the wave-function during a quantum measurement. In order to arrive at the proposed reformulation of quantum mechanics, we are led to suggest noncommutativity not only in spacetime, but also in momentum space. While the detailed theory remains to be developed, some consequences of the heuristic discussions given here can be tested in the laboratory.


The author would like to thank Aruna Kesavan, Kinjalk Lochan, and Aseem Paranjape for useful discussions.