Can the quantum state be interpreted statistically?

A new preprint entitled The Quantum State Cannot be Interpreted Statistically by Pusey, Barrett and Rudolph (henceforth known as PBR) has been generating a significant amount of buzz in the last couple of days. Nature posted an article about it on their website, Scott Aaronson and Lubos Motl blogged about it, and I have been seeing a lot of commentary about it on Twitter and Google+. In this post, I am going to explain the background to this theorem and outline exactly what it entails for the interpretation of the quantum state. I am not going to explain the technicalities in great detail, since these are explained very clearly in the paper itself. The main aim is to clear up misconceptions.

First up, I would like to say that I find the use of the word “Statistically” in the title to be a rather unfortunate choice. It is liable to make people think that the authors are arguing against the Born rule (Lubos Motl has fallen into this trap in particular), whereas in fact the opposite is true.  The result is all about reproducing the Born rule within a realist theory.  The question is whether a scientific realist can interpret the quantum state as an epistemic state (state of knowledge) or whether it must be an ontic state (state of reality). It seems to show that only the ontic interpretation is viable, but, in my view, this is a bit too quick. On careful analysis, it does not really rule out any of the positions that are advocated by contemporary researchers in quantum foundations. However, it does answer an important question that was previously open, and confirms an intuition that many of us already held. Before going into more detail, I also want to say that I regard this as the most important result in quantum foundations in the past couple of years, well deserving of a good amount of hype if anything is. I am not sure I would go as far as Antony Valentini, who is quoted in the Nature article saying that it is the most important result since Bell’s theorem, or David Wallace, who says that it is the most significant result he has seen in his career. Of course, these two are likely to be very happy about the result, since they already subscribe to interpretations of quantum theory in which the quantum state is ontic (de Broglie-Bohm theory and many-worlds respectively) and perhaps they believe that it poses more of a dilemma for epistemicists like myself then it actually does.

Classical Ontic States

Before explaining the result itself, it is important to be clear on what all this epistemic/ontic state business is all about and why it matters. It is easiest to introduce the distinction via a classical example, for which the interpretation of states is clear. Therefore, consider the Newtonian dynamics of a single point particle in one dimension. The trajectory of the particle can be determined by specifying initial conditions, which in this case consists of a position \(x(t_0)\) and momentum \(p(t_0)\) at some initial time \(t_0\). These specify a point in the particle’s phase space, which consists of all possible pairs \((x,p)\) of positions and momenta.

Classical Ontic State

The ontic state space for a single classical particle, with the initial ontic state marked.

Then, assuming we know all the relevant forces, we can compute the position and momentum \((x(t),p(t))\) at some other time \(t\) using Newton’s laws or, equivalently, Hamilton’s equations. At any time \(t\), the phase space point \((x(t),p(t))\) can be thought of as the instantaneous state of the particle. It is clearly an ontic state (state of reality), since the particle either does or does not possess that particular position and momentum, independently of whether we know that it possesses those values[1]. The same goes for more complicated systems, such as multiparticle systems and fields. In all cases, I can derive a phase space consisting of configurations and generalized momenta. This is the space of ontic states for any classical system.

Classical Epistemic States

Although the description of classical mechanics in terms of ontic phase space trajectories is clear and unambiguous, we are often, indeed usually, more interested in tracking what we know about a system. For example, in statistical mechanics, we may only know some macroscopic properties of a large collection of systems, such as pressure or temperature. We are interested in how these quantities change over time, and there are many different possible microscopic trajectories that are compatible with this. Generally speaking, our knowledge about a classical system is determined by assigning a probability distribution over phase space, which represents our uncertainty about the actual point occupied by the system.

A classical epistemic state

An epistemic state of a single classical particles. The ellipses represent contour lines of constant probability.

We can track how this probability distribution changes using Liouville’s equation, which is derived by applying Hamilton’s equations weighted with the probability assigned to each phase space point. The probability distribution is pretty clearly an epistemic state. The actual system only occupies one phase space point and does not care what probability we have assigned to it. Crucially, the ontic state occupied by the system would be regarded as possible by us in more than one probability distribution, in fact it is compatible with infinitely many.

Overlapping epistemic states

Epistemic states can overlap, so each ontic state is possible in more than one epistemic state. In this diagram, the two phase space axes have been schematically compressed into one, so that we can sketch the probability density graphs of epistemic states. The ontic state marked with a cross is possible in both epistemic states sketched on the graph.

Quantum States

We have seen that there are two clear notions of state in classical mechanics: ontic states (phase space points) and epistemic states (probability distributions over the ontic states). In quantum theory, we have a different notion of state — the wavefunction — and the question is: should we think of it as an ontic state (more like a phase space point), an epistemic state (more like a probability distribution), or something else entirely?

Here are three possible answers to this question:

  1. Wavefunctions are epistemic and there is some underlying ontic state. Quantum mechanics is the statistical theory of these ontic states in analogy with Liouville mechanics.
  2. Wavefunctions are epistemic, but there is no deeper underlying reality.
  3. Wavefunctions are ontic (there may also be additional ontic degrees of freedom, which is an important distinction but not relevant to the present discussion).

I will call options 1 and 2 psi-epistemic and option 3 psi-ontic. Advocates of option 3 are called psi-ontologists, in an intentional pun coined by Chris Granade. Options 1 and 3 share a conviction of scientific realism, which is the idea that there must be some description of what is going on in reality that is independent of our knowledge of it. Option 2 is broadly anti-realist, although there can be some subtleties here[2].

The theorem in the paper attempts to rule out option 1, which would mean that scientific realists should become psi-ontologists. I am pretty sure that no theorem on Earth could rule out option 2, so that is always a refuge for psi-epistemicists, at least if their psi-epistemic conviction is stronger than their realist one.

I would classify the Copenhagen interpretation, as represented by Niels Bohr[3], under option 2. One of his famous quotes is:

There is no quantum world. There is only an abstract physical description. It is wrong to think that the task of physics is to find out how nature is. Physics concerns what we can say about nature…[4]

and “what we can say” certainly seems to imply that we are talking about our knowledge of reality rather than reality itself. Various contemporary neo-Copenhagen approaches also fall under this option, e.g. the Quantum Bayesianism of Carlton Caves, Chris Fuchs and Ruediger Schack; Anton Zeilinger’s idea that quantum physics is only about information; and the view presently advocated by the philosopher Jeff Bub. These views are safe from refutation by the PBR theorem, although one may debate whether they are desirable on other grounds, e.g. the accusation of instrumentalism.

Pretty much all of the well-developed interpretations that take a realist stance fall under option 3, so they are in the psi-ontic camp. This includes the Everett/many-worlds interpretation, de Broglie-Bohm theory, and spontaneous collapse models. Advocates of these approaches are likely to rejoice at the PBR result, as it apparently rules out their only realist competition, and they are unlikely to regard anti-realist approaches as viable.

Perhaps the best known contemporary advocate of option 1 is Rob Spekkens, but I also include myself and Terry Rudolph (one of the authors of the paper) in this camp. Rob gives a fairly convincing argument that option 1 characterizes Einstein’s views in this paper, which also gives a lot of technical background on the distinction between options 1 and 2.

Why be a psi-epistemicist?

Why should the epistemic view of the quantum state should be taken seriously in the first place, at least seriously enough to prove a theorem about it? The most naive argument is that, generically, quantum states only predict probabilities for observables rather than definite values. In this sense, they are unlike classical phase space points, which determine the values of all observables uniquely. However, this argument is not compelling because determinism is not the real issue here. We can allow there to be some genuine stochasticity in nature whilst still maintaining realism.

An argument that I personally find motivating is that quantum theory can be viewed as a noncommutative generalization of classical probability theory, as was first pointed out by von Neumann. My own exposition of this idea is contained in this paper. Even if we don’t always realize it, we are always using this idea whenever we generalize a result from classical to quantum information theory. The idea is so useful, i.e. it has such great explanatory power, that it would be very puzzling if it were a mere accident, but it does appear to be just an accident in most psi-ontic interpretations of quantum theory.  For example, try to think about why quantum theory should be formally a generalization of probability theory from a many-worlds point of view.  Nevertheless, this argument may not be compelling to everyone, since it mainly entails that mixed states have to be epistemic. Classically, the pure states are the extremal probability distributions, i.e. they are just delta functions on a single ontic state. Thus, they are in one-to-one correspondence with the ontic states. The same could be true of pure quantum states without ruining the analogy[5].

A more convincing argument concerns the instantaneous change that occurs after a measurement — the collapse of the wavefunction. When we acquire new information about a classical epistemic state (probability distribution) say by measuring the position of a particle, it also undergoes an instantaneous change. All the weight we assigned to phase space points that have positions that differ from the measured value is rescaled to zero and the rest of the probability distribution is renormalized. This is just Bayesian conditioning. It represents a change in our knowledge about the system, but no change to the system itself. It is still occupying the same phase space point as it was before, so there is no change to the ontic state of the system. If the quantum state is epistemic, then instantaneous changes upon measurement are unproblematic, having a similar status to Bayesian conditioning. Therefore, the measurement problem is completely dissolved within this approach.

Finally, if we allow a more sophisticated analogy between quantum states and probabilities, in particular by allowing constraints on how much may be known and allowing measurements to locally disturb the ontic state, then we can qualitatively explain a large number of phenomena that are puzzing for a psi-ontologist very simply within a psi-epistemic approach. These include: teleportation, superdense coding, and much of the rest of quantum information theory. Crucially, it also includes interference, which is often held as a convincing reason for psi-ontology. This was demonstrated in a very convincing way by Rob Spekkens via a toy theory, which is recommended reading for all those interested in quantum foundations. In fact, since this paper contains the most compelling reasons for being a psi-epistemicist, you should definitely make sure you read it so that you can be more shocked by the PBR result.

Ontic models

If we accept that the psi-epistemic position is reasonable, then it would be superficially resonable to pick option 1 and try to maintain scientific realism. This leads us into the realm of ontic models for quantum theory, otherwise known as hidden variable theories[6]. A pretty standard framework for discussing such models has existed since John Bell’s work in the 1960′s, and almost everyone adopts the same definitions that were laid down then. The basic idea is that systems have properties. There is some space \(\Lambda\) of ontic states, analogous to the phase space of a classical theory, and the system has a value \(\lambda \in \Lambda\) that specifies all its properties, analogous to the phase space points. When we prepare a system in some quantum state \(\Ket{\psi}\) in the lab, what is really happening is that an ontic state \(\lambda\) is sampled from a probability distribution over \(\mu(\lambda)\) that depends on \(\Ket{\psi}\).

Representation of a quantum state in an ontic model

In an ontic model, a quantum state (indicated heuristically on the left as a vector in the Bloch sphere) is represented by a probability distribution over ontic states, as indicated on the right.

We also need to know how to represent measurements in the model[7].  For each possible measurement that we could make on the system, the model must specify the outcome probabilities for each possible ontic state.  Note that we are not assuming determinism here.  The measurement is allowed to be stochastic even given a full specification of the ontic state.  Thus, for each measurement \(M\), we need a set of functions \(\xi^M_k(\lambda)\) , where \(k\) labels the outcome.  \(\xi^M_k(\lambda)\) is the probability of obtaining outcome \(k\) in a measurement of \(M\) when the ontic state is \(\lambda\).  In order for these probabilities to be well defined the functions \(\xi^M_k\) must be positive and they must satisfy \(\sum_k \xi^M_k(\lambda) = 1\) for all \(\lambda \in \Lambda\). This normalization condition is very important in the proof of the PBR theorem, so please memorize it now.

Overall, the probability of obtaining outcome \(k\) in a measurement of \(M\) when the system is prepared in state \(\Ket{\psi}\) is given by

\[\mbox{Prob}(k|M,\Ket{\psi}) = \int_{\Lambda} \xi^M_k(\lambda) \mu(\lambda) d\lambda, \]
which is just the average of the outcome probabilities over the ontic state space.

If the model is going to reproduce the predictions of quantum theory, then these probabilities must match the Born rule.  Suppose that the \(k\)th outcome of \(M\) corresponds to the projector \(P_k\).  Then, this condition boils down to

\[\Bra{\psi} P_k \Ket{\psi} = \int_{\Lambda} \xi^M_k(\lambda) \mu(\lambda) d\lambda,\]

and this must hold for all quantum states, and all outcomes of all possible measurements.

Constraints on Ontic Models

Even disregarding the PBR paper, we already know that ontic models expressible in this framework have to have a number of undesirable properties. Bell’s theorem implies that they have to be nonlocal, which is not great if we want to maintain Lorentz invariance, and the Kochen-Specker theorem implies that they have to be contextual. Further, Lucien Hardy’s ontological excess baggage theorem shows that the ontic state space for even a qubit would have to have infinite cardinality. Following this, Montina proved a series of results, which culminated in the claim that there would have to be an object satisfying the Schrödinger equation present within the ontic state (see this paper). This latter result is close to the implication of the PBR theorem itself.

Given these constraints, it is perhaps not surprising that most psi-epistemicists have already opted for option 2, denouncing scientific realism entirely. Those of us who cling to realism have mostly decided that the ontic state must be a different type of object than it is in the framework described above.  We could discard the idea that individual systems have well-defined properties, or the idea that the probabilities that we assign to those properties should depend only on the quantum state. Spekkens advocates the first possibility, arguing that only relational properties are ontic. On the other hand, I, following Huw Price, am partial to the idea of epistemic hidden variable theories with retrocausal influences, in which case the probability distributions over ontic states would depend on measurement choices as well as which quantum state is prepared. Neither of these possibilities are ruled out by the previous results, and they are not ruled out by PBR either. This is why I say that their result does not rule out any position that is seriously held by any researchers in quantum foundations. Nevertheless, until the PBR paper, there remained the question of whether a conventional psi-epistemic model was possible even in principle. Such a theory could at least have been a competitor to Bohmian mechanics. This possibility has now been ruled out fairly convincingly, and so we now turn to the basic idea of their result.

The Result

Recall from our classical example that each ontic state (phase space point) occurs in the support of more than one epistemic state (Liouville distribution), in fact infinitely many. This is just because probability distributions can have overlapping support. Now, consider what would happen if we restricted the theory to only allow epistemic states with disjoint support. For example, we could partition phase space into a number of disjoint cells and only consider probability distributions that are uniform over one cell and zero everywhere else.

Restricted classical theory

A restricted classical theory in which only the distributions indicated are allowed as epistemic states. In this case, each ontic state is only possible in one epistemic state, so it is more accurate to say that the epistemic states represent a property of the ontic state.

Given this restriction, the ontic state determines the epistemic state uniquely. If someone tells you the ontic state, then you know which cell it is in, so you know what the epistemic state must be. Therefore, in this restricted theory, the epistemic state is not really epistemic. Its image is contained in the ontic state, and it would be better to say that we were talking about a property of the ontic state, rather than something that represents knowledge. According to the PBR result, this is exactly what must happen in any ontic model of quantum theory within the Bell framework.

Here is the analog of this in ontic models of quantum theory.  Suppose that two nonorthogonal quantum states \(\Ket{\psi_1}\) and \(\Ket{\psi_2}\) are represented as follows in an ontic model:

Psi-epistemic model

Representation of nonorthogonal states in a psi-epistemic model

Because the distributions overlap, there are ontic states that are compatible with more than one quantum states, so this is a psi-epistemic model.

In contrast, if, for every pair of quantum states \(\Ket{\psi_1},\Ket{\psi_2}\), the probability distributions do not overlap, i.e. the representation of each pair looks like this

Psi-ontic model

Representation of a pair of quantum states in a psi-ontic model

then the quantum state is uniquely determined by the ontic state, and it is therefore better regarded as a property of \(\lambda\) rather than a representation of knowledge.  Such a model is psi-ontic.  The PBR theorem states that all ontic models that reproduce the Born rule must be psi-ontic.

Sketch of the proof

In order to establish the result, PBR make use of the following idea. In an ontic model, the ontic state \(\lambda\) determines the probabilities for the outcomes of any possible measurement via the functions \(\xi^M_k\). The Born rule probabilities must be obtained by averaging these conditional probabilities with respect to the probability distribution \(\mu(\lambda)\) representing the quantum state. Suppose there is some measurement \(M\) that has an outcome \(k\) to which the quantum state \(\Ket{\psi}\) assigns probability zero according to the Born rule. Then, it must be the case that \(\xi^M_k(\lambda) = 0\) for every \(\lambda\) in the support of \(\mu(\lambda)\). Now consider two quantum states \(\Ket{\psi_1}\) and \(\Ket{\psi_2}\) and suppose that we can find a two outcome measurement such that that the first state gives zero Born rule probability to the first outcome and the second state gives zero Born rule probability to the second outcome. Suppose also that there is some \(\lambda\) that is in the support of both the distributions, \(\mu_1\) and \(\mu_2\), that represent \(\Ket{\psi_1}\) and \(\Ket{\psi_2}\) in the ontic model. Then, we must have \(\xi^M_1(\lambda) = \xi^M_2(\lambda) = 0\), which contradicts the normalization assumption \(\xi^M_1(\lambda) + \xi^M_2(\lambda) = 1\).

Now, it is fairly easy to see that there is no such measurement for a pair of nonorthogonal states, because this would mean that they could be distinguished with certainty, so we do not have a result quite yet. The trick to get around this is to consider multiple copies. Consider then, the four states \(\Ket{\psi_1}\otimes\Ket{\psi_1}, \Ket{\psi_1}\otimes\Ket{\psi_2}, \Ket{\psi_2}\otimes\Ket{\psi_1}\) and \(\Ket{\psi_2}\otimes\Ket{\psi_2}\) and suppose that there is a four outcome measurement such that \(\Ket{\psi_1}\otimes\Ket{\psi_1}\) gives zero probability to the first outcome, \(\Ket{\psi_1}\otimes\Ket{\psi_2}\) gives zero probability to the second outcome, and so on. In addition to this, we make an independence assumption that the probability distributions representing these four states must satisfy. Let \(\lambda\) be the ontic state of the first system and let \(\lambda’\) be the ontic state of the second. The independence assumption states that the probability densities representing the four quantum states in the ontic model are \(\mu_1(\lambda)\mu_1(\lambda’), \mu_1(\lambda)\mu_2(\lambda’), \mu_2(\lambda)\mu_1(\lambda’)\) and \(\mu_2(\lambda)\mu_2(\lambda’)\). This is a reasonable assumption because there is no entanglement between the two systems and we could do completely independent experiments on each of them. Assuming there is an ontic state \(\lambda\) in the support of both \(\mu_1\) and \(\mu_2\), there will be some nonzero probability that both systems occupy this ontic state whenever any of the four states are prepared. But, in this case, all four functions \(\xi^M_1,\xi^M_2,\xi^M_3\) and \(\xi^M_4\) must have value zero when both systems are in this state, which contradicts the normalization \(\sum_k \xi^M_k = 1\).

This argument works for the pair of states \(\Ket{\psi_1} = \Ket{0}\) and \(\Ket{\psi_2} = \Ket{+} = \frac{1}{\sqrt{2}} \left ( \Ket{0} + \Ket{1}\right )\). In this case, the four outcome measurement is a measurement in the basis:

\[\Ket{\phi_1} = \frac{1}{\sqrt{2}} \left ( \Ket{0}\otimes\Ket{1} + \Ket{1} \otimes \Ket{0} \right )\]
\[\Ket{\phi_2} = \frac{1}{\sqrt{2}} \left ( \Ket{0}\otimes\Ket{-} + \Ket{1} \otimes \Ket{+} \right )\]
\[\Ket{\phi_3} = \frac{1}{\sqrt{2}} \left ( \Ket{+}\otimes\Ket{1} + \Ket{-} \otimes \Ket{0} \right )\]
\[\Ket{\phi_4} = \frac{1}{\sqrt{2}} \left ( \Ket{+}\otimes\Ket{-} + \Ket{-} \otimes \Ket{+} \right ),\]

where \(\Ket{-} = \frac{1}{\sqrt{2}} \left ( \Ket{0} – \Ket{1}\right )\). It is easy to check that \(\Ket{\phi_1}\) is orthogonal to \(\Ket{0}\otimes\Ket{0}\), \(\Ket{\phi_2}\) is orthogonal to \(\Ket{0}\otimes\Ket{+}\), \(\Ket{\phi_3}\) is orthogonal to \(\Ket{+}\otimes\Ket{0}\), and \(\Ket{\phi_4}\) is orthogonal to \(\Ket{+}\otimes\Ket{+}\). Therefore, the argument applies and there can be no overlap in the probability distributions representing \(\Ket{0}\) and \(\Ket{+}\) in the model.

To establish psi-ontology, we need a similar argument for every pair of states \(\Ket{\psi_1}\) and \(\Ket{\psi_2}\). PBR establish that such an argument can always be made, but the general case is more complicated and requires more than two copies of the system. I refer you to the paper for details where it is explained very clearly.

Conclusions

The PBR theorem rules out psi-epistemic models within the standard Bell framework for ontological models. The remaining options are to adopt psi-ontology, remain psi-epistemic and abandon realism, or remain psi-epistemic and abandon the Bell framework. One of the things that a good interpretation of a physical theory should have is explanatory power. For me, the epistemic view of quantum states is so explanatory that it is worth trying to preserve it. Realism too is something that we should not abandon too hastily. Therefore, it seems to me that we should be questioning the assumptions of the Bell framework by allowing more general ontologies, perhaps involving relational or retrocausal degrees of freedom. At the very least, this option is the path less travelled, so we might learn something by exploring it more thoroughly.

  1. There are actually subtleties about whether we should think of phase space points as instantaneous ontic states. For one thing, the momentum depends on the first derivative of position, so maybe we should really think of the state being defined on an infinitesimal time interval. Secondly, the fact that momentum appears is because Newtonian mechanics is defined by second order differential equations. If it were higher order then we would have to include variables depending on higher derivatives in our definition of phase space. This is bad if you believe in a clean separation between basic ontology and physical laws. To avoid this, one could define the ontic state to be the position only, i.e. a point in configuration space, and have the boundary conditions specified by the position of the particle at two different times. Alternatively, one might regard the entire spacetime trajectory of the particle as the ontic state, and regard the Newtonian laws themselves as a mere pattern in the space of possible trajectories. Of course, all these descriptions are mathematically equivalent, but they are conceptually quite different and they lead to different intuitions as to how we should understand the concept of state in quantum theory. For present purposes, I will ignore these subtleties and follow the usual practice of regarding phase space points as the unambiguous ontic states of classical mechanics. []
  2. The subtlety is basically a person called Chris Fuchs. He is clearly in the option 2 camp, but claims to be a scientific realist. Whether he is successful at maintaining realism is a matter of debate. []
  3. Note, this is distinct from the orthodox interpretation as represented by the textbooks of Dirac and von-Neumann, which is also sometimes called the Copenhagen interpretation. Orthodoxy accepts the eigenvalue-eigenstate link.  Observables can sometimes have definite values, in which case they are objective properties of the system. A system has such a property when it is in an eigenstate of the corresponding observable. Since every wavefunction is an eigenstate of some observable, it follows that this is a psi-ontic view, albeit one in which there are no additional ontic degrees of freedom beyond the quantum state. []
  4. Sourced from Wikiquote. []
  5. but note that the resulting theory would essentially be the orthodox interpretation, which has a measurement problem. []
  6. The terminology “ontic model” is preferred to “hidden variable theory” for two reasons. Firstly, we do not want to exclude the case where the wavefunction is ontic, but there are no extra degrees of freedom (as in the orthodox interpretation). Secondly, it is often the case that the “hidden” variables are the ones that we actually observe rather than the wavefunction, e.g. in Bohmian mechanics the particle positions are not “hidden”. []
  7. Generally, we would need to represent dynamics as well, but the PBR theorem does not depend on this. []

Creative Commons License
Can the quantum state be interpreted statistically? by Matthew Leifer, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Noncommercial 3.0 Unported License.

131 Responses to Can the quantum state be interpreted statistically?

  1. Mateus Araújo

    Matt,
    You’re right, my simplification weakens the result. Although I find hard to believe that anyone would take seriously a theory that is epistemic for pairs of states but not for triples. My goal was to get rid of the multipartite argument, which I still think is unessential.
    Anyway, I’m looking forward to your anti-realist paper (not very interested in the philosophical argument).

  2. Hi Hrvoje, I think you’ve understood the claim correctly. Psi being “ontic” doesn’t mean that psi is among the beables postulated by the theory; it just means that it is a function of the posited beables. (Elsewhere I gave this example: in this usage of the terminology, the electromagnetic energy density \rho is “ontic” for classical electromagnetism, even though usually we would say that, for that theory, only the E and B fields “really exist”. The energy is just something it’s sometimes convenient for theorists to calculate. The point is, if you know the exact configuration of E and B, you can calculate \rho, and that’s all that’s meant by saying it’s “ontic”.)

    I haven’t read the Smolin paper you linked to, but it’s certainly possible it’s compatible with the theorem in just the way you described. Another such example (which I know you are at least somewhat familiar with) is my “TELB” paper (http://arxiv.org/abs/0909.4553). This is a nice example because, at least on its own terms, the theory obviously doesn’t posit the (big universal) wave function as a beable; but (just as obviously) the big wave function is “ontic” in the sense that, given the exact configuration of all the beables, you can calculate the wf.

    (On the other hand, the thing that makes it nice as an example here is also what makes it hard to take seriously as a theory: the obviousness of being able to reconstruct the wf from the posited beables leaves the theory very much open to the charge of being merely the usual pilot wave theory with a thinly-veiled different mathematical representation of the same old quantum state. But hey, you gotta start somewhere.)

  3. Matt Leifer,

    Your concluding comments
    “For me, the epistemic view of quantum states is so explanatory that it is worth trying to preserve it. Realism too is something that we should not abandon too hastily. Therefore, it seems to me that we should be questioning the assumptions of the Bell framework by allowing more general ontologies, perhaps involving relational or retrocausal degrees of freedom. At the very least, this option is the path less travelled, so we might learn something by exploring it more thoroughly.”

    suggest the existence of an ontological quantum mechanics complying with epistemic requirements (e.g., statistical verifications of theoretical predictions) as well as philosophically realistic assumptions, leaving out any underlying hidden determinist framework. Put it differently, an ontological status would be ascribed to the wavefunction. Yet this ontic content underlying wavefunction should not match the following Bohr’s conception:

    ‎”There is no quantum world. There is only an abstract physical description. It is wrong to think that the task of physics is to find out how nature is. Physics concerns what we can say about nature”

    Summing up, building up an ontolological theory of quantum mechanics implies that we are going beyond both Einstein and Bohr, as long as the ontological character of the concept of probability turns out to be pointed out. Accordingly, the assumption of universality of the wavefunction as well as the concept of isolated system should be revised. This view goes beyond the Bohmian mechanics as well. In fact, I have set out to look at these novelties in my monograph

    A. O. Bolivar, Quantum Classical Correspondence: Dynamical quantization and the Classical Limit (Springer –Verlag, Berlin, 2004).

    Bolivar.

  4. Hrvoje Nikolic

    Hi Travis,

    Thank you for your notes, with which I absolutely agree.

  5. Hrvoje Nikolic

    One additional note on Bohmian trajectories.

    Let as accept this definition of “being ontic” according to which the wave function is ontic due to the PBR theorem. Then, by the same definition, the set of all Bohmian trajectories associated with a given wave function is also ontic. But that leads to a funny question: As the set of all Bohmian trajectories is ontic, does it mean that each particular trajectory is ontic too?

  6. Hrvoje, That’s a great example to show that this definition of “being ontic” maybe isn’t capturing exactly what we would normally think of this term as meaning. Note also that the full set of Bohmian trajectories will be ontic also in the many worlds interpretation!

  7. I am still just a student, so excuse my lack of knowledge.

    Does this theorem greatly affect the debate in the foundations of QM?

    To me it seems like this theorem rules out indeterminism more than anything, couldn’t there still be some other “deeper” theory of reality that does not have the wavefunction itself as a ontological feature (like it does in Bohm and MWI), but rather that there is a deeper foundamental theory beneath the quantum realm?
    Or is this now completely ruled out ?

  8. Quantumental,

    The theorem assumes that the predictions of quantum theory are correct, and that the goal of any deeper underlying theory is to reproduce them. We could, of course, find out that the predictions of quantum theory are not correct in some regime, e.g. at higher energies than we have done experiments at so far, in which case we would need a completely new theory, which would still have to contain quantum theory as an approximate limit. If that happens then all bets are off regarding ontology. However, as long as quantum theory continues to hold, the theorem tells us that any deeper realist theory must include the wavefunction as part of its ontology (or at least, whatever the ontology is, it uniquely fixes the wavefunction). This is so unless we are prepared to consider ontologies that are more exotic than those usually considered, e.g retrocausal influences or relational degrees of freedom.

  9. Mateus Araújo

    I contacted the authors, and it turns out my would-be simplification is just an old result by Caves, Fuchs, and Schack http://arxiv.org/abs/quant-ph/0206110 . Well, I guess it was too easy to be unknown.

  10. Matt Leifer,

    Thanks for your answer.
    If I understood correctly this means that one could still find nonlinear dynamics beneath the wavefunction?
    Or something like what Gerard ‘t Hooft believes, where determinism and realism is restored on the planck scale.
    So these sort of realist determinist interpretations are still not ruled out?

    What about Many Worlds, does it affect it in any way?

  11. Quantumental,

    I don’t think it rules out ‘t Hooft’s proposal, which would be called “superdeterministic” in the context of Bell’s theorem. I don’t think that the ontic status of the wavefunction is a big issue for him anyway, since he is more concerned with restoring realism, locality and determinism. I think he would be happy with those three, regardless of the status of the wavefunction in the final theory. That said, I am not sufficiently familiar with his work to say for sure.

    The theorem has no consequences for many-worlds because, firstly, the wavefunction is already ontic in many worlds and, secondly, the theorem is based on a framework in which measurements only have single outcomes, so it does not apply to many-worlds.

  12. Thanks a lot Matt Leifer,

    I guess the interpretation debate will go on for quite a few decades more or perhaps centuries or forever.

    MWI has seriously problems with probability that cause most people to consider it ruled out, but obviously you never know….

  13. Thanks, Matt for putting the PBR theorem in context. Wonderful survey of the foundational issues.

    PBR proof depends critically on the clever detector which is designed to click on 4 sums of basis states. This is a very odd sort of detector because one cannot infer the input from the output whenever the input is a basis state. In this (degenerate) case all the basis states map to the (0. 0. 0. 0) output and hence cannot be distinguished.

    If one breaks the degeneracy by adding deltas and epsilons to the basis states
    so that they no longer register as (0, 0, 0, 0), then to conserve probability the
    detector states must be massively renormalized.

    Since the PBR proof depends on normalization conventions, the fact that the normalization of the detector states is unstable to small perturbations suggests that these states are idealized (non-physical). I suspect that when
    a more realistic detector is used –in which the degenerate (0,0,0 0) outcome is excluded–the PBR proof will not go through.

  14. Nick,

    I don’t quite understand what you are trying to say here. First of all, what do you mean by “basis states”? There are three bases involved here: the |0>,|1> basis, the |+>,|-> basis, and the entangled basis that the measurement is performed in. Which one are you referring to? Secondly, what do you mean by the (0,0,0,0) output?

    I also should say that there is an argument in the paper that the result is robust to noise. I did not mention it in the post because I wanted to focus on the theoretical result rather than the possibility of experimental test, and I haven’t checked it in detail. In any case, PBR claim that if you can prepare the states and perform the measurement with good accuracy then you can put a bound on the overlap between the probability distributions representing distinct states. Obviously, you won’t be able to say that there is no overlap at all.

  15. The gist of my inquiry is this: we know from the CHSH inequality that Bell’s theorem is ROBUST–that it does not depend on perfect correlations at zero relative angle nor exact detector settings. Is the PBR theorem similarly ROBUST or is it ULTRA-SENSITIVE? In particular does PBR depend for its validity on forcing the basis states EXACTLY into the pencil-standing-on-its-point (0, 0,0,0) detector state or is some wiggle room permitted?

  16. The “basis states” are the four raw possibilities <1<+, <1<-, <0<+, <0<-. Each of these when inputted into the detector states (I would not call them "entangled" but implemented with AND gates since the states <1 and <+ are independent and phase has no meaning here.) gives the result (0, 0, 0, 0)–that is none of the AND gates is triggered since these are pure states and there is nothing for them to AND with.

    I envision the magic detector as being implemented by two beam splitters sensitive 1. to <0 and <1 and 2. sensitive to <+ and <-. the outputs of these detectors are pairwise combined with AND gates to implement the 4 orthogonal detector states. An inputted basis state, such as <1<+ has nothing to AND with so none of the 4 orthogonal detector states is triggered. That's what I mean by (0,0,0,0).

    Perhaps I am wildly off base here. How do you envision the 4 detector states being realized?

  17. Nick,

    In one sense the result is not robust. If you define psi-epistemic to mean exactly zero overlap in the distributions representing different pure states then obviously any noise in the state preparations or measurements destroys the argument (in this sense it is more like Kochen-Specker than Bell). However, PBR claim in the paper that they can bound the extent to which the distributions overlap when there is noise, and if there is a small amount of noise then there is a small amount of overlap. If there is only a small amount of overlap then it is tempting to say that the theory is “almost” psi-ontic. There is probably more work to be done in order to quantify this precisely.

    Regarding the measurement, I am not sure that I get how your implementation is supposed to work. One thing I am certain about is that the four states in the measurement basis are entangled, and in fact maximally entangled. Therefore, you can’t make the measurement by measuring each system separately and then post-processing the result classically using AND gates. Somehow, you have to implement a unitary operation that maps each of the four states in the basis to the states |00>, |01>, |10> and |11> and then you can measure each system separately in the |0> vs |1> basis. This unitary operation requires a coherent interaction between the two systems. I don’t think it can be done with just linear optics (beam splitters and the like). How you would implement it depends on the kind of system you use for the experiment, but it is not more difficult in principle than the kind of measurement you need to do for a teleportation experiment. Therefore, any system that we have successfully used for teleportation, e.g. photons or ions in an ion trap, could be used.

    Note however that this only works for the two states used in the example. For the full proof, you want to do something similar for any pair of states and for some of them you would need a lot of systems and a lot of interaction. A quantum circuit is given in the paper for implementing the measurement. If we had a quantum computer it certainly could be done with any pair of states. For pairs of states that only require a few qubits (approx < 10) it could be done with the kind of setups that we currently use for ion trap quantum computing.

    Finally, let me say that this is early days on the theoretical side as well, so we may be able to find a version of the result that is easier to test experimentally. For example, it may be possible to prove a PBR-like result using only a single system, by considering a variety of different states and measurements. Obviously, the proof strategy would have to be different from the one used by PBR, since that would require perfect distinguishability of non-orthogonal pure states. Maybe some extension of the idea that Mateus was discussing in earlier comments could be used.

  18. Well, it’s clear I’m really confused about PBR. But so are smarter folks like Stapp and Motl. In olden days physicists computed preposterous properties that the ether had to possess; today we calculate preposterous properties of hidden variables.

    K-S showed that hidden variables must be “contextual”–that is, depend on what compatible measurements are concurrently carried out, Bell (in a special case of K-S) showed that HVs must be non-local.

    Now PBR claim to establish yet another restriction on HVs. But how best to express the PBS restriction unambiguously in ordinary language?

    From your description PBR seems to have to do with the “overlap” of quantum states. Orthogonal states have no overlap but there still exist measurements where a state and its orthogonal state will both give detector clicks.

    However there also do exist special measurement situations where a state and its orthogonal state can be unambiguously distinguished.

    Is one of the accomplishments of PBR to demonstrate a clever measurement situation in which non-orthogonal states (possessing probabilistic overlap)
    can be unambiguously distinguished?

    As in the case of Maxwell’s luminiferous ether which was progressively burdened with preposterous properties until its banishment by Einstein, what, in plain language, is the additional burden that PBR now are placing on the poor beleaguered hidden variables?

  19. Nick,

    When I was talking about overlap, I was referring to the overlap of the probability distributions over the hidden variables that represent the quantum states in the hidden variable theory and not the overlaps (inner products) of the quantum states themselves. PBR do not show that nonorthogonal states can be distinguished. [Note: You should be careful about using the phrase "unambiguously distinguished" because there is a technical sense in which this can always be done. For any pair of states, you can find a 3 outcome measurement such that if you get outcome 1 then you know it was the first state, if you get outcome 2 then you know it was the second state, and if you get outcome 3 then you have failed, i.e. you don't know which state was prepared. Of course, the probability of outcome 3 increases with the inner product of the states and becomes 100% when the states are identical. This is usually called "unambiguous discrimination", but I take it that you were not referring to this, but rather to the idea of distinguishing states in a single shot without any failure probability.]

    The extra burden on hidden variable theories entailed by the PBR theorem is just that, if you know the full hidden variable state, then you must also know the wavefunction. The wavefunction must just be a function of the ontological state, and in this sense it must be ontological itself. Now, as it happens, all of the viable hidden variable theories that have been constructed so far, e.g. Bohmian mechanics, have the wavefunction as part of the ontological state. Now we know why.

  20. Thanks for the clarification, Matt. What’s needed here is a single new word to characterize PBR’s discovery such as “contextual” for K-S and “non-local” for Bell. “Ontological” is much too broad and suggests that hidden variables “really exist”. To answer the question : “Is the wavefunction statistical?” in the negative easily implies some sort of determinism. That is a very bad title.

    Here’s your chance to coin a new word. And if that word truly captures the gist of PBR, word X will remind us of something non-obvious about quantum theory. I still don’t grasp the essence of this proof so am not a candidate for adding an essential new descriptor to the physics canon.

    As an indication of how fundamental PBR’s discovery of the necessary X-ness of quantum reality seems to be, I note that to prove his theorem, Bell needed 4 Hilbert-space dimensions, K&S required 3 dimensions, but PBR can prove property X using only 2.

  21. I think “psi-ontic” is the word, but I can’t take credit for it, since Rob Spekkens coined the term.

  22. “Psi-ontic” is a perfect word for specialists and the in-crowd
    but it will never make the front page of the New York Times.

    We need to find a more ordinary-language term for “property X” discovered by PBR.

    PBR’s work concerns a new property of the Qubit–the simplest possible quantum entity, which dwells in two-D Hilbert space and hence is immune to K-S and Bell’s theorem which hold for 3- and 4-D spaces and larger.

    If my understanding is correct, PBR have proved that an elemental qubit Q(0), if undergirded, supported, defined by a HAREM OF HIDDEN VARIABLES. refuses to share a single member of its harem with any other qubit Q(n) that dwells in the same Hilbert space–barring entanglement between Q(0) and Q(n))–no matter how strong the inner-product overlap may be between Q(0) and Q(n).

    Ordinary language words that this newly-discovered quantum behavior suggests are “patriarchal, territorial, exclusive, jealous” but these terms seem too anthropomorphic for describing an objective property of nature. “Psi-ontic” is perfect for the specialist but we sorely need a term that has significance for Joe and Jenny Six-pack, the celebrated person in the street.

  23. Matt–
    If PBR have indeed proved that each independent qubit must possess exclusive hidden-variable support, I’d like (with apologies to John Donne) to propose the term “islandic” as an ordinary-language expression for “property X” of quantum reality.

    Each (solitary, unentangled) qubit is an island,
    Entire of itself, alone in David Hilbert’s sea.
    Solo, unmated, hermit qubit
    Ask not for whom the Bell tolls
    The Bell tolls not for thee.

  24. Hrvoje Nikolic

    A new paper on the PBR theorem:
    http://xxx.lanl.gov/abs/1111.6304

  25. Another new paper on a related result (apparently; haven’t had time to go over it in detail) that cites this very blogpost as ‘an additional informative discussion’ (which it very much is, by the way!):
    http://arxiv.org/abs/1111.6597

  26. Pingback: Statistical Interpretation of Quantum State « The Quantum Channel

  27. Pingback: La explicación del teorema que sacudió los cimientos cuánticos « Francis (th)E mule Science's News

  28. Pingback: On Determinism | Cosmic Variance | Theoretical Physics

  29. Someone cited you over at a forum as saying that if retrocausallity is not true you would have to become an Everettian? Is this a recent statement in the light of these results?

  30. Quantumental,

    As I see it, Everett is a viable realist fallback option for me if nothing else can be made to work out. I would strongly prefer an interpretation that is both realist and psi-epistemic as I think that psi-epistemicism has greater explanatory power than psi-ontology. In light of the PBR theorem, the ontology of any viable realist and psi-epistemic interpretation is going to have to be different from the kinds of ontology included within the Bell framework. Retrocausaility and relational degrees of freedom are two open possibilities, and there may be more that we haven’t thought of yet. Of those two possibilities, retrocausality is the one that makes the most sense to me and I have some idea about how to go about investigating it. I am concerned that relational theories will end up being so close to Everett that it won’t be worth making the distinction, but maybe advocates of that approach can convince me otherwise.

    If I find out that retrocausal theories are not viable in some compelling sense then, unless some other possibility has been uncovered by then, I will be compelled to give up either realism or psi-epistemicism. If I were to give up psi-epistemicism then I would probably become an Everettian. I used to be strongly against that interpretation, but the Saunders-Wallace programme has made it much more intelligible and acceptable to me. If I were to give up realism, then I would have to move closer to people like Chris Fuchs and other neo-Copenhagenists, although I do think there are problems with those interpretations as they are currently formulated. I am not completely sure whether realism or psi-epistemicism is more important to me, and my opinion on the matter fluctuates from day to day. Therefore, it is quite likely that I might have said that I would become an Everettian if retrocausality fails on a day in which my realist conviction was the stronger of the two. As things stand, I am quite glad that I don’t have to make the choice just yet.

  31. Does the KS theorem imply a contextual ontology?

  32. In the context of what I am calling the “Bell framework” for hidden variable theories, yes. If you introduce different kinds of ontology, e.g. retrocausality, then you would have to define what is meant by noncontextuality in that framework before you could say anything definite.

  33. Before an electron is observed, what is it that exists?

  34. Thanks Matt, that clears that up!

    When you say “the work of Wallace-Saunders” are you referring to their work on probability?
    If so, I’d say that’s a debate that seems to have no end in sight.
    I am sure you are familiar with the works of Kent, Price, Lewis, Albert etc. on why the decision-theoretic approach fails?

    I agree realism is very important, but I am not sure if MWI could give us realism.
    Have you read these 2 papers by Valia Allori and Bradley Monton on why we should keep 3Space and a “primitive ontology” as fundamental?

    http://spot.colorado.edu/~monton/BradleyMonton/Articles_files/Monton%20Against%203N%20D%20Space.doc

    http://www.niu.edu/~vallori/AlloriWfoPaper-Jul19.pdf

    I think these might be of interest to all realists :)

  35. Quantumental,

    I am referring both to their work on probability and to their particular structuralist reading of the decoherence program. With regards to the criticisms of the decision theoretic approach, I would say that none of the critics have responded adequately to Wallace’s challenge to identify which of the axioms of his argument is at fault. Instead, they appeal to intuitions about what it seems natural that one ought to do in a many worlds scenario. These are strong intuitions, but by no means conclusive.

    If there is a problem with the decision theoretic approach, I think it is that there is a conflation of the roles of probability and utility that are usually distinct in classical decision theoretic arguments. Indeed, it is telling that Wallace’s response to more than one of his critics includes the claim that the behaviour they argue for can be arranged by picking a utility function appropriately. The problem is particularly apparent in the “fusion program” due to Greaves (I believe), which argues that the mod squared amplitudes of the branches should be viewed as a “caring measure”, i.e. it measures how much value you should attach to your successors who live in a particular branch. However, the amount to which you value an outcome would normally be described by the utility function, so it seems to me that there are two utility functions in the fusion approach and, hence, it is not surprising that one can negate the effect of one of them by choosing the other appropriately. If you agree with the intuitions of the critics then we ought to be doing this routinely. The Born rule would then be of no consequence, since we would always be overruling it with an extremal utility assignment.

    Because of this, I would prefer an approach where the probabilities are viewed as representing subjective uncertainty, as they are in the classical case, i.e. I am going to become one of my successors but I currently don’t know which one. This is problematic because it breaks the symmetry of the many-worlds approach, in which all branches are supposed to exist on an equal footing. Nevertheless, there were some articles in the Many Worlds at 50 volume arguing that this type of view might be possible. I do not, at present, understand any of them, but in any case I don’t want to get derailed by long arguments defending views of quantum theory that I don’t currently hold. These are things that I would have to think about more seriously if the psi-epistemic program were to fail, and I don’t think that has happened yet.

    Thanks for the links to the two papers. I agree with some of what they say, but there is also a lot that I disagree with. A blog comment is not an appropriate venue to do them justice though.

  36. I think Jacques Mallah points out which axioms are unjust in this paper: http://arxiv.org/abs/0808.2415

    The approach you seem to want of a many worlds view seems impossible in a scheme where the world “branches”.
    It would take something the lines of Many Bohmian Worlds where the worlds evolve completely indepdently of eachother I think.

    Do you remember the name of the approaches or which authors were suggesting them?

    I don’t think the psi-epistemic programme has failed yet either, but I still maintain that determinism and realism has to come first.

  37. The author may have been Saunders, but I am not sure. In any case, there is definitely a paper about it in the proceedings of the 50th anniversary of Everett conferences (http://philpapers.org/rec/SAUMWE). I am not sure if the approach has a specific name, but the words “subjective uncertainty” definitely appear in the title and/or abstract. Unfortunately, I don’t have the book to hand at the moment.

    Regarding the Mallah paper, there are several points I could make in response. Firstly, the “amount of conciousness” interpretation of probability is not the one that Wallace subscribes to. According to him, all successors are on an equal footing and it is just that, for some reason, we care about some of them more than others. Now, if Mallah had an argument that the “amount of conciousness” interpretation was the only one admissible in a many worlds scenario then that would be a mark against Wallace. However, as far as I can see, there is no such argument other than a bald statement that it must be so.

    Secondly, if you are a Bayesian, then a decision theoretic account of probability is simply the definition of what a probability is. There is no need to establish that probabilities can be derived from a decision theoretic argument because that is simply a tautology within that approach. Now, of course, one might not be a Bayesian. However, I would say that, whatever interpretation of probability that you subscribe to, it must always be the case that it is rational to use those probabilities in the standard way when making decisions, i.e. by maximizing expected utility. If not, there will be a wide range of common applications of probability that your conception of the term cannot account for. Therefore, one needs to be able to get a decision theoretic account off the ground as a bare minimum, even if you think that more than that is needed, e.g. it is more of a fundamental requirement then showing that typical observers will observe relative frequencies close to the probabilities, which some people may also want in addition.

    Thirdly, once you have accepted that a decision theoretic account is needed then one must be aware of the fact that even the classical arguments are not unproblematic. Their assumptions need not always hold and, if you are a good subjective Bayesian, then you should adopt a more general theory as and when they fail to hold for some particular situation, e.g. one could use a partial order as advocated by Keynes. I must admit that this is not universally accepted amongst Bayesians, many of whom think that it is always irrational to deviate from probability theory. However, I do not think that it is irrational and so, for me, it is less about whether Wallace’s axioms always hold than it is about whether they hold in situations that are analagous to those in which the classical arguments hold. Many of Mallah’s criticisms are of the “I can imagine situations in which this axiom does not hold” variety, and these are not compelling for me unless it is also shown that the classical axioms would hold in analagous situations.

    That said, I was actually being a bit oversimplistic in saying that I endorse the Deutsch-Wallace derivation. What I actually like, is a combination of the Greaves-Myrvold derivation (also to be found in the Everett at 50 volume) with some of the ideas of Deutsch-Wallace. Greaves-Myrvold start out by generalizing the standard Savage axioms in such a way that they do not assume a single universe. Actually, the assumption that they drop is the idea that the current state of the universe determines the outcomes of future measurements uniquely, which is of course false in many worlds but also in other types of theory, e.g. those with genuine stochasticity. In any case, at this level, their argument is very general and doesn’t assume much about physics. It is hard to find criticisms with this part of the derivation that do not also apply to the classical argument because the two are so closely related. However, at this point we only have a very weak theorem that says we have a different classical probability space for each choice of measurement (in other words we have the state space of a semi-classical test space if you are familiar with that terminology). Greaves and Myrvold then go on to use the de Finetti theorem to argue that the Born rule can be obtained from this empirically by looking at the outcomes of experimental data. I am not so keen on this part of the argument, as I feel that there should be a justification of the Born rule based on the mathematical structure of quantum theory rather than just invoking statistics, i.e. I think that if you just decide to believe in many-worlds quantum theory and have never seen any experimental data then there should be something about the theory that still compels you to assign Born-rule probabilities. Therefore, at this point, I would bring in a few Deutsch-Wallace type principles in order to complete the proof. Firstly, I would assert that there is some global unitarily evolving wavefunction that describes the full ontology of the theory, even though I might not actually know what it is. Then, I would invoke a variant of measurement neutraility to derive the noncontenxtuality assumption of Gleason’s theorem, which would allow me to infer the Born rule, conveniently skipping over all the principal principle nonsense in Wallace’s approach.

    I realize that this is just a sketch of an argument, but I believe that Wallace already has an argument like this in his Ph.D. thesis, which should be published as a book fairly soon. At least, there is a footnote that hints at this in the Everett at 50 volume. In any case, I think this version of the argument is much more robust against criticism than Wallace’s original argument, since it apes the classical argument so closely and only drops in a couple of implications of many-worlds at the very end. It does not, however, address my concern that some people might always want to assign extremal untilities in branching scenarios. I don’t think any decision theoretic argument can really do that, as it is really more of a question of ethics than anything else. Restoring a subjective uncertainty view of probability would help, since then we would have the same moral intuitions as in the classical case, but I don’t really understand the attempts to do this at the moment.

  38. Dear Matt,
    Although this discussion seems to have subsided, I still would like to:
    1) join most others in congratulating and thanking you for your clear exposition of PBR and surrounding issues;
    2) make a point which isn’t just about terminology.

    You write:
    ‘ First up, I would like to say that I find the use of the word “Statistically” in the title to be a rather unfortunate choice.’
    I believe they made exactly the right choice, though, because their terminology exposes the fact that they make use of standard statistical (or one should say probabilistic) reasoning in passing from the hidden states to the wave function. As such, they rule out option 1) precisely as you state it, that is,
    ‘Wavefunctions are epistemic and there is some underlying ontic state. Quantum mechanics is the statistical theory of these ontic states in analogy with Liouville mechanics.’
    BUT: they do not rule out the first sentence by itself. The point is that, in what you quite rightly call the ‘standard Bell framework for ontological models.’ one implicitly assumes the classical probability calculus. The use of this calculus should, in my opinion, always be explicitly added to the assumptions leading to Bell’s Theorem and related results, and hence also to PBR. Your conclusion that
    ‘The PBR theorem rules out psi-epistemic models within the standard Bell framework for ontological models’ is correct as stated (and I applaud your precision at this point), but one cannot say (as some surely would be inclined to do) that ‘The PBR theorem rules out psi-epistemic models’ full stop.

    To restate my point, your replacement of ‘statistical’ by ‘ontic’ hides an important ingredient in practically all discussions of hidden variable models underlying QM, namely the assumption of classical probability theory. Why do I find this an important point? Surely, most readers of this blog will be aware of the shaky philosophical and conceptual status of classical probability: the frequency interpretation (which would justify the axioms and hence the calculus of classical probability if it were correct) has been discredited by almost every commentator, Popper’s propensity interpretation is now widely (and rightly) seen as empty, and finally the Dutch Book arguments of the De Finetti and his followers are so human-oriented that even Bayesians and other subjectivists should raise their eyebrows in falling back on it. In any case, I would be baffled if anyone would dare to justify the statistical averaging over invisible hidden variables underlying quantum theory by a theory of betting!

    To close this comment: although a detailed comparison between the strength of Bell-type theorems and Conway-Kochen-Specker type theorems has not, to my knowledge, been made in the literature, I would maintain that the latter are a priori stronger since they rely on weaker assumptions. For, as you know, the K-S and Free Will Theorems do not rely on classical or any other kind of probability at all.

    Best wishes, Klaas

  39. There is a lot to respond to here, so I don’t know if I can do all of it justice, but here are my immediate thoughts:

    - The use of the word “statistical” is, at the end of the day, just a matter of terminology and we should be able to get past it. Once we are clear on how the authors are using it, it doesn’t really make a great deal of difference whether or not you like their usage. However, I still maintain that it was a mistake to use this terminology because it has turned a lot of people off the result on the basis of the title alone. The problem is that the “statistical interpretation of the wavefunction” is interpreted by many people as nothing more than the correctness of the Born rule. This is not an unreasonable reading of the terminology, since Max Born’s Nobel prize citation reads, “for his fundamental research in quantum mechanics, especially for his statistical interpretation of the wavefunction”. As a result, since I wrote this blog post, my email inbox has been full of people complaining that PBR do not actually disprove the Born rule and so the result must be uninteresting. I agree with the first part, but I think that failure to appreciate the significance of what it is that they actually do prove is a mistake, and the title is helping a lot of people to make that mistake.

    - Regarding the use of the classical probability calculus, I am with you, I think, in believing that quantum theory is best understood as a generalization of classical probability theory. I am also with you in thinking that the foundations of probability are a mess, or rather that your average foundationally inclined physicist has a rather poor understanding of them. However, I don’t agree with your assessment of the subjective Bayesian approach. For me, it is the best possible response to the fact that we can’t solve the problem of induction by deductive means, or rather we can’t solve the statistical variant of the problem where the aim is to deduce probabilities from relative frequencies. The subjective Bayesian responds to this by falling back on a description of what an agent expects to happen on the basis of data, rather than attempting to describe what will happen. In doing so, he makes it clear that the usual description of statistical inference depends on many assumptions, e.g. exchangeability, and that no one is forced to agree that these assumptions must hold in any given situation. However, if we do have a common set of assumptions, e.g. we all adopt exchangeability, then the theory explains how observing data will end up causing us to reach agreement. Admittedly, it may be preferable to have a theory of why we reach agreement that also implies that the agreement that we reach is objectively true, as the frequentist and propensity accounts purport to maintain. However, I think that the problem of induction implies that such a theory of truth from statistics is logically impossible. Therefore, I am inclined to accept the subjective Bayesian account as the best that one can hope for. At the very least, it forces us to state the assumptions on which probabilistic assessments depend explicitly, rather than sweeping them under the carpet and pretending that they don’t exist as is usually done in objectivist accounts.

    Given that I have adopted the subjective Bayesian approach, it is then necessary to determine what it means to have a generalization of probability theory, since that is how I want to understand quantum theory. It is not enough to just point to the Kolmogorov axioms and generalize them formally, as is usually done, since those axioms are not what gives probability its real-world meaning. For a subjective Bayesian, generalizing probability theory can only mean that one or more of the assumptions usually made in the decision theoretic arguments for classical probability do not hold in general. This is not a crazy thing to think, since those assumptions have been questioned already for reasons that have nothing to do with quantum theory, and a variety of generalizations have been proposed (e.g. upper and lower probabilities, or partially ordered comparative probabilities in general). Admittedly, none of these generalizations have seen widespread application, but I view it as a virtue of the subjective Bayesian approach that it leads to them. (Generally, if you are doing work on the foundations of some theory and are adopting the assumption that the theory in question must always hold at the outset then you are doing it wrong.) In the case of quantum theory, it is very easy to see which assumptions do not hold. For the Dutch book argument, it is the idea that all bets that you might make about the properties of some physical system can simultaneously be resolved. For the Savage argument, it is the idea that there is a pre-existing “state of the world” that determines the consequences of all possible actions uniquely. Depending on how one chooses to interpret quantum theory, these assumptions may not hold, e.g. they would not hold in Copenhagenish views with the notion of complimenatarity, in the many-worlds view, or in spontaneous collapse views (they do hold in Bohmian mechanics though). If we drop these assumptions then we end up with a more general theory (technically a theory of probability measures on semi-classical state spaces — this is the Greaves-Myrvold theory that I mentioned in my last comment). Both classical and quantum probability can be embedded in this structure, although we are still a long way from pinning down a C*-algebraic structure uniquely.

    Given this account of how the foundations of probability need to be generalized, we can then ask whether the adoption of classical probability theory is an extra assumption that we need to make explicit in no-go theorems like Bell, PBR, etc. or whether it is really just an implication of realism. Certainly, if we are dealing with a (single universe) deterministic theory then it is not an extra assumption because this implies the “all bets can be resolved” and “state of the world” assumptions of Dutch Book and Savage. If the theory has some genuine stochasticity then things are less clear. Savage’s axiom implies determinism, and dropping it leads to Greaves-Myrvold, so it seems like we are in good shape here. However, the idea that there is some ontic degree of freedom that causes outcomes probabilistically may be strong enough to get us back to classical probability theory. I do not know for sure, so it is worth thinking about. On the other hand, Dutch book makes no assumption of determinism, since simultaneous resolvability of all combinations of bets does not imply it. One could argue it either way, but I would be inclined to say that the idea that this can be done in principle, if not in practice, is part of the definition of what we mean by “realism”, i.e. the idea that those bet outcomes are caused by something. This would imply that the use of classical probability is not an extra assumption. However, I admit that you might not find this reasoning compelling, especially if you are not a subjectivist.

    - Finally, regarding the relative merits of Bell, KS, free-will theorem etc., that is certainly a topic for debate. Describing all my views on this would require a blog post of its own, so I’ll just make a couple of comments. Firstly, the KS theorem assumes outcome determinism, so one is always free to drop that assumption rather than noncontextuality as a response. In this sense, Bell, PBR, etc. are stronger because they do not assume determinism. In my opinion, Rob Spekkens has the best definition of what noncontextuality means in the absence of determinism, but his definition does involve probabilities so the perceived advantage of KS disappears. However, this doesn’t apply to the free-will theorem because restricting attention to deterministic theories can be justified by locality and convexity, as is done in Bell’s theorem. Nevertheless, the convexity argument shows that probabilities are hiding in the background of this theorem as well. As a side note, “experimental tests” of KS (with scare quotes to indicate that I am skeptical of their relevance) generally do make probabilistic assumptions, as they are based on inequalities derived in a similar way to Bell’s theorem.

    Generally speaking, I think that PBR will turn out to be the strongest of the no-go results, which is why I am so keen on promoting it. I think it may imply all of the others in some suitable sense. For example, given PBR, the EPR argument is enough to establish nonlocality, without having to bother with Bell inequalities. It also pretty simply implies Rob Spekkens notion of “preparation contextuality”, but I have to admit that I haven’t been able to figure out how it is connected to the traditional KS version of contextuality as of yet.

  40. Thanks a lot for your indepth answer.
    I happen to be in contact with the author of the paper so I will tell him to read your post and see what he thinks…

    Meanwhile, you might find this interesting, I think this is a candidate for psi-epistemic realist and deterministic interpretation that the PRB theorem does not affect:
    http://de.arxiv.org/abs/1112.1811

  41. Quantumental,
    Thanks for your comment. I did look at that ‘t Hooft paper when it came out. He basically describes a way of reformulating classical theories such that they look more like quantum theory (it reminded me a bit of the Koopman-von Neumann construction). In his reformulation, you get a wavefunction and this can clearly be interpreted in a psi-epistemic manner, so it is indeed an interesting way of constructing psi-epistemic theories. However, these theories are not quantum theory and they do not reproduce the predictions of quantum theory. ‘t Hooft says as much in the paper as he expects that the fundamental degrees of freedom of his theory will not be the ones that we conventionally use in quantum theory and that quantum theory will break down at some scale in order to allow for wavefunction collapse to be described by this sort of theory rather than quantum theory. Therefore, I would not really call this an “interpretation” of quantum theory, but rather an alternative to quantum theory.

    Personally, although I am confident that quantum theory is not the final theory, I am skeptical that nature will choose to violate it a way that immediately solves the measurement problem. Instead, I imagine that the next theory will look even weirder than quantum theory. Therefore, attempts at modifying quantum theory that are motivated just by solving the measurement problem and nothing else are not that appealing to me. I could be wrong of course, and these approaches do deserve to be investigated and tested by the physics community, but my prior probability for them being correct is sufficiently low that I won’t be working on them myself. The exception is nonequilibruim Bohmian mechanics, as this has structure that I find interesting and is at least a very concrete proposal for post-quantum physics. However, I view this more as a conceptual investigation into the theoryspace around quantum theory rather than as a plausible candidate for future physics.

  42. Matt, given the KS, PBR, etc. theorems, why are not more people investigating contextual ontologies??

  43. Because most people think of these theorems as either reasons to give up on realism or vindications of their preferred ontology that has already been constructed.

  44. Matt–
    Am enjoying your explications of the PBR theorem. But I have (at least)
    one nagging question. If the polarization states H, V, R and L are all psi-ontic,
    do you believe there is an ontic difference between PUP–a random mixture of H and V states, and CUP–a random mixture of R and L states? Each of these two mixtures is represented by the same density matrix. Hence they are experimentally identical. But are these two quantum states ontically distinct? (The acronyms PUP and CUP stand for “Plane-UnPolarized” and “Circular-UnPolarized” light.)

  45. Yes. That is a consequence of the PBR theorem.

    Well, let me back up a bit. It is a consequence of PBR under one additional assumption, which is that hidden variable theories are convex under mixing, i.e. if quantum states \(\Ket{\psi_j}\) are represented in a HVT by the distributions \(\mu_j(\lambda)\), and if I prepare the state \(\rho = \sum_j p_j \Ket{\psi_j}\Bra{\psi_j}\) by generating a classical random variable (e.g. by flipping coins, rolling dice, etc.) with outcome probabilities \(p_j\) and then preparing \(\Ket{\psi_j}\) when the outcome is \(j\), then this results in the hidden variable distribution \(\mu(\lambda) = \sum_j p_j \mu_j(\lambda)\). Formally, one could imagine theories in which this wasn’t true, e.g. the coin flipping mechanism is somehow correlated with the ontic state of the quantum system, but it would be pretty loopy to do so.

    With this convexity assumption, the fact that two ensembles represented by the same density matrix must be represented by different distributions in a hidden variable theory was already known prior to the PBR theorem, and was called “preparation contextuality” by Rob Spekkens (see http://arxiv.org/abs/quant-ph/0406166). The PBR theorem implies an even stronger result, which is that the probability distributions representing any two ensembles with no pure states in common (as in the PUP vs. CUP example) can have no overlap.

  46. A very interesting paper that came out today. Two of the authors are the same as per PBR :

    “Many quantum physicists have suggested that a quantum state does not represent reality directly, but rather the information available to some agent or experimenter. This view is attractive because if a quantum state represents only information, then the collapse of the quantum state on measurement is possibly no more mysterious than the Bayesian procedure of updating a probability distribution on the acquisition of new data. In order to explore the idea in a rigorous setting, we consider models for quantum systems with probabilities for measurement outcomes determined by some underlying physical state of the system, where the underlying state is not necessarily described by quantum theory. A quantum state corresponds to a probability distribution over the underlying physical states, in such a way that the Born rule is recovered. We show that models can be constructed such that more than one quantum state is consistent with a single underlying physical state-in other words the probability distributions corresponding to distinct quantum states overlap. A recent no-go theorem states that such models are impossible. The results of this paper do not contradict that theorem, since the models violate one of its assumptions: they do not have the property that product quantum states are associated with independent underlying physical states.”

    The quantum state can be interpreted statistically
    http://lanl.arxiv.org/pdf/1201.6554.pdf

  47. Hi Leifer,
    I must thank you for this post, it has helped a lot in getting through the PBR result.

    However I yet have some difficulties with the PBR result, have posted them here: http://goo.gl/EjCgi. Pleas take a look and help me in resolving.

  48. There is something that intrigues me about the PBR result. To
    illustrate it assume that the preparation step starts like a typical
    EPR scenario: two 1/2 spin particles, say an electron and a positron,
    in a spin singlet are separated spatially, and the positron is subject
    to spin measurements in the z or x axis. Due to entanglement, the
    measurement performed on the positron will put the electron in one of
    the following states:

    |0> = |spin up in the z axis>

    |1> = |spin down in the z axis>

    |+> = |spin pointing towards the positive x axis> = 1/sqrt(2) (|0> + |1>)

    |-> = |spin pointing towards the negative x axis> = 1/sqrt(2) (|0> – |1>)

    We can do the same with a second electron-positron pair, and this
    produces a second electron in one of the four states mentioned above.

    Next, ignore all cases in which the outcome is different from |0>|0>,
    |0>|+>, |+>|0>, |+>|+>, i.e., we pay attention only to what happens
    when each the two electrons come out with either spin up in the z
    axis, or spin pointing towards the positive x axis (which can be
    determined by looking at the outcome of the measurement performed on
    the corresponding positron).

    If we measure the two electrons with the device described in the PBR
    paper, the statistical results will be very different e.g. for states
    |0>|0> and |+>|+>, in particular if the electrons come out as |0>|0>,
    then the measuring device will never yield \xi_1, however if they come
    out as |+>|+>, the device will have to yield \xi_1 a percentage of
    times (I believe 50% of the times if my computations are correct).

    Everything looks fine up to here, but let’s now assume that the
    preparation step of each electron is performed by a sequence of two
    measurements on the corresponding positron, the first one (call it
    event ‘E1′) of spin in the z direction, and the second one (event
    ‘E2′) in the x direction. Let E3 be the event consisting of the
    electrons arriving to the measuring device. Things may be arranged so
    that, given two observers in relative motion, both of them see events
    E2 and E3 happening after event E1, but for one of them E2 happens
    BEFORE event E3, and for the other one even E2 happens AFTER event E3.

    Now, if we pay attention only to the cases in which the electrons end
    up in state |0>|0> after the first z-spin measurement on the
    positrons, we conclude that the first observer will never see the
    measuring device yielding \xi_1. However, in about 25% of the cases
    the subsequent x-spin measurement on the positrons will put the
    electrons in state |+>|+>, and the second observer will see those
    electrons arriving in that state |+>|+> to the final measuring device.
    According to the above reasoning the second observer should see the
    the final measuring device yielding \xi_1 in about 50% of those cases,
    contradicting what the first observer is supposed to see.

    Am I missing something?

  49. Yes. After E1, the electron spins will be oriented in the z direction and the entanglement between the electrons and positrons will no longer exist due to the collapse caused by E1. E2 will not therefore cause the electrons to change their state. It only affects the positrons and these are no longer entangled with the electrons.

  50. Ok, let’s suppress the first measurement (event ‘E1′), and consider only electron-positron pairs that happen to be in the desired states – both electrons in |0>|0> (z-spin up). We may not know which ones they are, but if the quantum state is real there must be a proportion of cases in which the particles happen to be produced in those states, and each pair electron-positron is still entangled. The question is what happens in those cases – the observers still seem to reach different conclusions on the statistical outcome of the final measurement on that particular set of particles.

Leave a Reply