On the Network-Wide Gain of Memory-Assisted Source Coding

Mohsen Sardari, Ahmad Beirami, Faramarz Fekri
Email:{mohsen.sardari, beirami,
This material is based upon work supported by the National Science Foundation under Grant No. CNS-1017234.

Several studies have identified a significant amount of redundancy in the network traffic. For example, it is demonstrated that there is a great amount of redundancy within the content of a server over time. This redundancy can be leveraged to reduce the network flow by the deployment of memory units in the network. The question that arises is whether or not the deployment of memory can result in a fundamental improvement in the performance of the network. In this paper, we answer this question affirmatively by first establishing the fundamental gains of memory-assisted source compression and then applying the technique to a network. Specifically, we investigate the gain of memory-assisted compression in random network graphs consisted of a single source and several randomly selected memory units. We find a threshold value for the number of memories deployed in a random graph and show that if the number of memories exceeds the threshold we observe network-wide reduction in the traffic.

I Introduction

Several studies have demonstrated the existence of considerable amount of redundancy in the Internet data traffic, where a few major dimensions have been identified as the main sources of redundancy in the network traffic. For example, the contents of a web server contains more than redundant data on the average within a ten-day period [1]. Further, there are some popular files in each server that may be requested by several clients in the network. This redundancy in the data may be leveraged to reduce the communication within the network. The existing redundancy elimination techniques are mostly based on end-to-end caching mechanisms, where the redundant content is only cached in the server and the client for future reference [2]. However, these end-to-end approaches do not efficiently leverage the redundancy in the network because there is no memorization in today’s Internet except end-to-end caching mechanisms [1].

Recently, a few studies considered the deployment of redundancy elimination techniques within the network [3, 4], where the intermediate nodes in the network have been assumed to be capable of caching of the previous communication and processing of the data. These works studied the network flow reduction via ad-hoc solutions such as deduplication of the repeated segments of the traffic without any connection to the information theory. The objective of this paper is to study this problem from an information theoretic point of view. We assume that some intermediate nodes (referred to as memory nodes) are capable of the memorization (to be defined later) of the previous communications which have passed through them. We further assume that the memorized content will be used in a memory-assisted source coding in the network, which we refer to as network flow compression with memory. However, several questions remain open regarding memory-assisted compression of the network flow. Does the deployment of memory in the network provide any fundamental benefit over end-to-end solutions? How much saving could be achieved using network compression? How can the savings be achieved?

This paper attempts to answer the above fundamental questions. To the best of our knowledge, this is the first work which addresses the memory-assisted redundancy elimination of the flow from the information theoretic point of view. The data that is transmitted inside the network have different spatial and temporal probability distributions. Thus, prior knowledge of the probability distributions underlying the contents may not be assumed. Hence, one important characteristic of any compression solution is that it must be universal in the sense that it must be able to remove redundancy without knowing the statistics and nature of the data [5, 6].

In this paper, we focus our study on the fairly broad class of parametric information sources, which include the class of Markov sources of any finite order [7]. We formulate the problem as memory-assisted compression of the network flow, where the redundancy in the traffic data is to be removed. In this context, the redundancy elimination could be viewed as universal source coding, where the goal is to represent the data with a minimum length codeword [7]. However, as we will discuss in Sec. II, this problem is different from the distributed source coding problem because of memory units. We investigate the fundamental gain of the memory-assisted network flow compression over end-to-end universal compression techniques, where the content is compressed at the server and routed via routers without any memorization inside the network. Throughout this paper, we focus on the problem involving a single source (content server) that is fixed in the network and extensions to multiple sources is left as future work.

In what follows, we first describe the memory-assisted source coding in Sec. II. The memory deployment problem and the gain of memorization in networks is discussed in Sec. III. In Sec. IV we study memory-assisted network flow compression on Erdős-Rényi (ER) random graphs and find a threshold value for the number of memory units. Finally, simulation results are provided in Sec. V.

Ii Memory-Assisted Source Coding

In what follows, we introduce the memory-assisted source coding via the sample network depicted in Fig 1, which is consisted of a server , a memory unit (which also acts as a router), and the clients and . We assume that the communication is as follows. First, client , who has not previously communicated with the server, acquires the sequence from the server through the intermediate node . Next, client , who also has not previously communicated with , acquires the sequence from through . In order to show the benefits of the deployment of memory in the router, we compare three schemes:

  • NcompNmem (No compression with no memory), which does not apply any compression and does not utilize the memory unit.

  • UcompNmem (Universal compression with no memory), which only applies end-to-end universal source coding at the source without using the memory unit.

  • UcompWmem (Universal compression with memory), which assumes that the router has memory and utilizes the memory unit when compressing the data at the source.

In all of the above scenarios, we assume that the client has no previous communication with the server since it is usually the case in networks. On the other hand, the memory/router is capable of memorizing the communication of between and in order to better compress (on the link from to ) that is then being delivered to . We will later demonstrate that even if sequences and are independent given that the source model is known, the memorization of in can result in the reduction of the communication for the transfer of from to . This seemingly counter intuitive phenomenon is due to the fact that the source model is not known a priori (at ). The underlying source coding must be universal, which imposes a compression overhead when the length of the sequence is finite (small) [7]. On the other hand, sequence does indeed contain some information about the unknown source parameters to the extent that an infinite length sequence can be used to identify all of the unknown parameters of the source. This side information can be memorized at the memory unit and the source for the compression of . Then the memory unit can decode using the side information and send to . It is important to note that the saving of memory-assisted compression in terms of flow reduction is observed in the link. For example, if and are unit size and the memorization helps to compress by a factor of 2, the total flow is reduced from , to bithop, where there is a gain in the link between and .

Fig. 1: The basic memory-assisted network flow compression scenario.

While relevant, the memory-assisted source coding problem is different from those addressed by distributed source compression techniques (i.e., the Slepian Wolf problem) that target multiple correlated sources sending information to the same destination [8, 9]. As described in the above example, the memory-assisted source coding gain is due to the fact that the source parameter is unknown. Therefore, when the length of the sequence increases, the memory-assisted source coding gain, with respect to UcomNmem, vanishes since the source parameter can be well estimated using the sequence, and hence, can be compressed to fundamental limit (i.e., entropy rate). On the other hand, in the Slepian-Wolf, the gains are achievable in the asymptotic regime. Further, the memorization of a sequence that is independent of the sequence can result in a gain in memory-assisted compression of whereas, in the Slepian-Wolf problem, the gain is due to the bit by bit correlation between the two sequences. The memory-assisted compression problem is also distinct from the lossless source coding techniques such as LZ and CTW that simply remove redundancy in a single piece of content without regard to any memorization at the intermediate node  [5, 6].

Next, we characterize the benefits of network memory in the context of a universal source coding problem for the class of smooth parametric sources [7, 10]. Let denote the length function that describes the codeword associated with the sequence . The expected codeword length quantifies the compression performance of UcompNmem, where denotes the expectation operator. For an asymptotically optimal code in the sense that it achieves the entropy rate, we have as , where denotes the entropy of a sequence of length . In the case that the router is capable of memorization (UcompWmem), assume that is another sequence of length from the same source that generates . By context memorization, we mean that both the source and the memory unit have already visited the sequence . Let be a regular length function where and have access to a context memory of length from the previous communications. Then, the expected codeword length with memory characterizes the compression performance of UcompWmem for in the link from to , in Fig 1.

Let be defined as the ratio of the expected codeword length of UcompNmem to that of UcompWmem as

Further, let be a real number such that . We denote as the fundamental gain of the context memorization on the family of parametric sources on a sequence of length using a context sequence of length for a fraction of the sources, which is defined as follows:

In other words, the fundamental gain of memorization is at least for a fraction of the sources in the family.

In [10], Beirami and Fekri studied the fundamental limits of compression without memory. In [7], they extended their study to the behavior of the memorization gain with respect to the sequence length and the memory size for different source models. In the rest of this paper, we investigate the effect of the context memorization gain on a network given that the memory-assisted compression gain (hereafter, referred as ) is known.

Iii Memory Deployment Problem in Networks

In this section, we will investigate the memorization gain on a network. A network is represented by an undirected graph where is the set of nodes (vertices) and is the set of edges connecting nodes and . We consider a single source which is the content server, and a set of memories chosen out of nodes. The content server is assumed to be a parametric information source [7]. We assume that each client requests a small to moderate length size sequence from the content server. As discussed in Sec. II, there is a fundamental limit beyond the entropy on the universal compression of a small to moderate length sequence. Therefore, UcompNmem will only be able to compress the content to a value which may be significantly larger than the entropy of the sequence. On the other hand, a memory nodes is capable of memorizing the communication between the server and some client node .

To investigate the gain of memorization in the compression of the network flow, we must consider two phases. The first is the memorization phase in which we may assume all memory units have visited some sufficiently long sequence (or equivalently, a sufficient collection of small to moderate length sequences) from the source. This phase is realized in actual communication networks by observing the fact that a sufficient number of clients may have previously retrieved small to moderate length sequences from the server such that, via their routing, each of the memory units has been able to memorize the source. In the second phase, which is the subject of this section, we assume each node in the entire network may request (a small to moderate length) content from the server, uniformly. Our goal is to characterize the memorization gain in the compression of the sequences that are retrieved in the second phase. The above view simplifies our study as we are not concerned with the transition phase during which the memorization is taking place in the memory units. Hence, we can assume each memory unit will provide the same memory-assisted compression gain of of the link from source to itself that depends on the length of the sequence that is being transmitted as well as the length of the sequence that is memorized in the memory unit, as described in Sec. II.

The goal of the memory deployment is to minimize the total cost of communication between the source and destinations in the network, measured by bithop, by deploying a set of memories . We wish to study the behavior of the total savings in terms of bithop, as a function of the number of the memories, place of memories, size of memories, length of the sequences, and the information source model.

In a network with source and a set of destinations , let be the flow destined to . The distance between any two nodes and is shown by . The distance is measured as the number of hops in the shortest path between two nodes. As we will see later, introducing memories to the network will change the lowest cost paths from the source to destinations, as there is a gain associated with the portion of the path. Therefore, we have to modify paths accounting for the gain of memories. Accordingly, for each destination , we define effective walk, denoted by , which is the ordered set of nodes in the modified (lowest cost) walk between the source and . Then, we partition the set of destinations as , where is the set of destinations observing a memory in their effective walk, and . The total flow is then defined as

Using (3), we define , called the effective distance from source to , as

In short, the effective distance is the distance when memory-assisted source compression is performed and hence the gain applies. By definition, . For simplicity, we assume for all destinations. Hence,

In a general network where every node can be a client, we define a generalized network-level gain of memory deployment as a function of memorization gain , as follows:

where is the total flow in the network under UcompNmem, i.e., .

In order to show the challenges of the memory deployment problem, we show as to how a single memory changes the effective paths in a network with a single source.

Fig. 2: The shortest walk between the source and destination is not necessarily a path when we have memory in network.

Consider the network with the source node placed as shown in Fig. 2. The destinations are nodes , and . The effective walks from the source to destinations are obviously the shortest paths when there is no memorization (UcompNmem). As shown in the figure, the placement of memory changes the effective path to while the shortest paths from the source to , , and are the same as the effective paths. Without memory, the shortest path to is two hops long (), while the memory totally changes the effective walk distance to to as depicted in the figure.

In order to extend our study to analyze the achievable gains in general networks and study the behavior of , in the next section, we consider the memory deployment gain in the network graphs that resemble the ER random graph [11]. The ER random graph is the building block of the recent models for complex graphs and hence the results would be useful in much broader contexts. We specifically direct our attention to connected random graphs since they better describe real networks.

Iv Gain of Memory Deployment on the Network Flow Compression

Definition 1

An ER random graph is an undirected, unweighted graph on vertices where every two vertices are connected with probability .

Definition 2

Let be any two vertices. The diameter of a connected graph is defined as . Similarly, the average distance of a connected graph is defined as .

The following properties hold for ER random graphs:

  1. If , then almost surely (a.s.) has isolated vertices and thus disconnected.

  2. If for some constant , then is a.s. connected and every vertex asymptotically has degree  [12].

  3. The diameter of is almost surely .

  4. The average distance in , denoted by , is

    provided that goes to infinity as (this condition is satisfied in the connected regime) [13].

Again, the main question is how scales with . In order to characterize the gain of memory placement, we consider connected , with a single source node and all other nodes as destinations. Since the expected degree of all nodes in ER graph is the same and every vertex is a destination with equal probability, we select memories uniformly and random. Theorem 1, below, provides the scaling of with respect to :

Theorem 1

Suppose is the number of deployed memories in an ER random graph. Let be a positive real number.

  • If , then . 111In this paper, we have used the following asymptotic notation: iff .

  • If , then all the destinations benefit from memory and

We first find an upper bound on the number of destinations benefit form each memory. This upper bound is sufficient to derive part (a) of the theorem. For the second part, we find a lower bound on the number of benefiting destinations.

To characterize , we first need to find . The average distance from the source to a node is . Thus, . For large , (6) results in

Next, we need to find . For every memory we consider a neighbourhood as shown in Fig. 3. This neighborhood consist of all vertices within distance from . We choose such that, almost surely, all nodes in would benefit from the memory . Clearly, if , the benefit of the memory for node vanishes and only nodes at distances less than benefit from the memory . Given , we denote this set of nodes benefiting from by .

Since memories are uniformly placed, the average value of in is equal to . Similarly, the average of is also . Hence, solving for in (7) and then using the result on the average distance in (6), we conclude

The following lemma, by Chung and Lu [13], gives an upper bound on the total number of vertices in the neighborhood , where is the set size operator.

Lemma 2 ([13])

Assume a connected random graph. Then, for any , with probability at least , we have , for .

Using Lemma 2 and (8), we deduce that

Therefore, the total number of nodes gaining from the memories is upper-bounded by As we will see, from (9) it is clear that the gain of memory vanishes if is chosen small. The value is the threshold value for the network-wide gain. More accurately, if , there is no gain from memories.

For all the nodes in , we have a flow gain of . Let , then we have

where inequality in (10) follows from the double counting of the destination nodes that may reside in more than one neighborhood. Also, (IV) follows from replacing (9) in (10).

Fig. 3: Memory Neighborhood

Since we need more than memory units to have a network-wide gain, the next question is as to how scales when the number of memory units exceeds . To answer this question, we need to establish a lower-bound on the neighborhood size and the number of nodes benefiting from memory. Further, we have to account for the possible double counting of the intersection between the memory neighborhoods. We use the following concentration inequality from [13] to establish the desired bound.

Proposition 3 ([13])

If are non-negative independent random variables, then the sum holds the bound

This inequality will be helpful to show that the quantities of interest concentrate around their expected values.

The following lemma provides a lower-bound on the neighborhood size and the lower-bound on , as we show, is immediate.

Lemma 4

Consider a set of vertices of such that . For , with probability at least , we have

The vertex boundary of , denoted by , consists of all vertices in adjacent to some vertex in .

Let be the indicator random variable that a vertex is in , i.e., . Then,

where the inequality in (13) follows from

and the second part holds because . Since, ’s are non-negative independent random variables, by applying Proposition 3 with , with probability at least we have

By picking a single vertex and applying (14) inductively times, and then adding up the number of adjacent nodes, we obtain (12).

Now that we have a lower-bound on the number of nodes benefiting from each memory, we show that by increasing the number of memories beyond , memories cover all the nodes in the graph effectively and hence all the nodes would gain from the memory placement.

In order to limit the intersection between the neighborhoods, we reduce to as below:

With this choice of , by lemmas 2 and 4, we deduce that the probability that a random node belongs to the neighborhood of the memory is . Hence, the expected number of the covered nodes is

where (16) holds by choosing .

To show that the number of covered nodes is concentrated around its mean, we use Prop. 3 again with . Then, with probability at least we have

Hence, the memories cover, almost surely, all of the nodes.

Network-level gain
Fig. 4: Network-level gain versus for different network sizes and .

Since all nodes are covered with high probability, we can associate each node with a neighborhood , for which nodes’ distances in the neighborhood from memory are .

By (16), we can bound the network-wide gain of the memory from below. We have

where (17) holds because the distance of the nodes from memory is , asymptotically almost surely.

As the number of memories becomes close to , i.e., , the gain , as expected. In the next section, we verify our result in memory-assisted source coding and the network-wide gain of memory via numerical simulations.

V Simulation Results

In this section, we first demonstrate our theoretical results through an example. We consider the source to be a first-order Markov source with alphabet size equal to . In [7], Beirami and Fekri derived a lower bound on the memorization gain as a function of the sequence length and the memory size. For example, they showed that kBMB is about , i.e., with a memory of MB, a gain of is obtained on the memory-assisted compression of kB long sequences [7]. Fig. 4 presents the simulation results for network-wide gain for different network sizes versus when . The rightmost solid curve is our theoretical result in (17). For small values of , the network-wide gain would be 1 for , while for large , tends to . Also, as increases, simulation results approach the theoretical limit for both small and large values of .

For everything else, email us at [email protected].


Spin dynamics in a two dimensional quantum gas

Poul L. Pedersen QUANTOP, Institut for Fysik og Astronomi, Aarhus Universitet, Ny Munkegade 120, 8000 Aarhus C, Denmark.    Miroslav Gajdacz QUANTOP, Institut for Fysik og Astronomi, Aarhus Universitet, Ny Munkegade 120, 8000 Aarhus C, Denmark.    Frank Deuretzbacher Institut für Theoretische Physik, Leibniz Universität Hannover, Appelstraße 2, 30167 Hannover, Germany.    Luis Santos Institut für Theoretische Physik, Leibniz Universität Hannover, Appelstraße 2, 30167 Hannover, Germany.    Carsten Klempt Institut für Quantenoptik, Leibniz Universität Hannover, Welfengarten 1, 30167 Hannover, Germany.    Jacob F. Sherson QUANTOP, Institut for Fysik og Astronomi, Aarhus Universitet, Ny Munkegade 120, 8000 Aarhus C, Denmark.    Andrew J. Hilliard QUANTOP, Institut for Fysik og Astronomi, Aarhus Universitet, Ny Munkegade 120, 8000 Aarhus C, Denmark.    Jan J. Arlt QUANTOP, Institut for Fysik og Astronomi, Aarhus Universitet, Ny Munkegade 120, 8000 Aarhus C, Denmark.
March 24, 2022

We have investigated spin dynamics in a 2D quantum gas. Through spin-changing collisions, two clouds with opposite spin orientations are spontaneously created in a Bose-Einstein condensate. After ballistic expansion, both clouds acquire ring-shaped density distributions with superimposed angular density modulations. The density distributions depend on the applied magnetic field and are well explained by a simple Bogoliubov model. We show that the two clouds are anti-correlated in momentum space. The observed momentum correlations pave the way towards the creation of an atom source with non-local Einstein-Podolsky-Rosen entanglement.

Since the optical trapping of Bose-Einstein condensates (BECs) enabled the investigation of quantum gases with multiple spin components, spinor condensates have become a particularly rich research field Stamper-Kurn and Ueda (2013); Kawaguchi and Ueda (2012). While initial work focused on an understanding of the ground state and dynamical properties of spinor condensates Stenger et al. (1998); Chang et al. (2004); Widera et al. (2005), recent experiments have started to exploit their properties for applications in other fields. In particular, the production of entangled states through spin dynamics Gross et al. (2011); Hamley et al. (2012) has spawned interest in spin dynamics for their applications in precision metrology Lücke et al. (2011); Gross et al. (2010).

Spin dynamics in a trapped quantum gas are strongly influenced by the geometry of the confining potential. In particular, highly asymmetric optical traps provide a way to reduce the dimensionality of a trapped quantum gas, both with respect to the motional and spin degrees of freedom Sadler et al. (2006). Thus, tailored confining potentials offer new avenues for exploiting spin dynamics, e.g., the generation of correlated pairs of atoms in well-defined motional states Scherer et al. (2013), similar to work on four-wave-mixing of ultra-cold atoms in an optical lattice Hilligsøe and Mølmer (2005); Campbell et al. (2006); Bonneau et al. (2013).

In this Rapid Communication, we investigate spin dynamics in a quantum gas confined to two dimensions (2D) by an optical lattice. We show how the spin excitation modes in the 2D potential lead to ring-shaped density distributions with a superimposed angular density modulation in time-of-flight images. The angular structure is traced to the matter-wave interference between multiple spin excitation modes with angular momentum. The observed density distributions may also be interpreted as several wave packets propagating in 2D with well-defined momentum.

We investigate spin dynamics in a Rb BEC prepared in (). By making several standard approximations to treat atomic collisions at ultra-low temperatures, one finds that only collisions that preserve the total magnetization can occur Stamper-Kurn and Ueda (2013). Thus, the spin dynamics lead to scattering into () and (), but for short evolution times, scattering between and predominates; i.e., . By treating the condensate as a classical field and as small fluctuations , the dynamics may be described by

where , , is the radial coordinate, is the radial trapping frequency of the confining potential, and is the mass. The 2D interaction energies and respectively describe spin independent and dependent contact collisions Deuretzbacher et al. (2010). The 2D chemical potential is given by and is the quadratic Zeeman energy difference between and for a magnetic field of magnitude . The first two lines of Eq. (1) describe the internal dynamics of and , respectively, while the final line describes scattering between and . Given our focus on spin-changing collisions during short evolution times, we neglect the significantly weaker dipolar interaction: in of Rb, the magnetic dipole interaction strength is on the order of 5% of the spin-dependent contact interaction Stamper-Kurn and Ueda (2013); Vengalattore et al. (2008).

(Color online) (a) Absorption image after Stern-Gerlach separation and time-of-flight for
Figure 1: (Color online) (a) Absorption image after Stern-Gerlach separation and time-of-flight for and 8 ms evolution time. (b) Time evolution of relative population in . The data have been fitted with an exponential function for the first 8 ms; the statistical uncertainty of each point is on the order of the symbol size. (c) Mean relative population in after 8 ms evolution time; the solid line is a guide to the eye.

We realize a 2D spinor gas by preparing a BEC in an optical lattice. The BEC is produced in in a Ioffe-Pritchard magnetic trap and contains on average atoms with no discernible thermal fraction  Bertelsen et al. (2007). The BEC is loaded into a red-detuned vertical optical lattice at wavelength  nm and depth , where is the recoil energy. The lattice is formed by retro-reflecting a beam with a waist of m. At the chosen lattice depth, the radial and axial trapping frequencies are  Hz and  kHz respectively, and tunneling between lattice sites is negligible. The lattice loading is performed by simultaneously reducing the current in the magnetic trap coils and increasing the lattice intensity over 110 ms. This results in lattice sites occupied by independent BECs. The atoms are prepared in by two sequential microwave pulses via at a constant magnetic bias field of 285 mG in a horizontal direction, which is maintained throughout the experiment. Following preparation in the lattice, the cloud has a finite thermal component: on average, we obtain atoms in the BEC and in a thermal fraction at temperature  nK. The thermal component arises from a technical limitation in switching off our magnetic trap. Spin dynamics are initiated by applying an additional magnetic field along the vertical direction. This field is turned on with a linear ramp over 1 ms and adds vectorially to the magnetic bias field to set ; the resulting magnetic field is held constant for a variable evolution time. The spin dynamics are brought to an end by switching off the optical lattice.

We probe the result of the spin dynamics by Stern-Gerlach separation and absorption imaging along the vertical direction after 20 ms time-of-flight. To avoid saturating the optical depth of our imaging system for the cloud, a third microwave pulse is applied just after the lattice is switched off to transfer part of the population to , which is transparent to the imaging light Ramanathan et al. (2012). To obtain accurate atom numbers, the imaging system was calibrated following Reinaudi et al. (2007); Gajdacz et al. (2013). Finally, the point-spread function of the imaging system is well-described by a Gaussian function with waist 5.72 m. A typical absorption image is shown in Fig. 1(a). For this image the peak optical depth of atoms in state was reduced to 1.5. The clouds have an interesting ring structure with a number of peaks on the circumference and will be discussed in detail below.

Figure 1(b) shows the time evolution of the population in for Hz in terms of the relative population . The data have been fitted by an exponential function from 0-8 ms. This dependence is expected because Eq. (1) describes an initial exponential amplification analogous to parametric down conversion in non-linear optics Klempt et al. (2010). The fitted value of the time constant was  ms. After 8 ms, the population deviates from exponential growth, indicating the breakdown of the ‘linear regime’ in which depletion of can be neglected. Additionally, we begin to observe a small population in at this time: at 8 ms, the relative population in is on average 0.25%, i.e., on the order of our resolution limit.

(a) Absorption images for several values of
Figure 2: (a) Absorption images for several values of ; the gray-scale was adjusted for different settings of to enhance visibility. The schematic for  Hz illustrates momentum conservation in a spin-changing collision. (b) Simulated time-of-flight density distributions for the same values of .

The spin dynamics that transfer atoms from to exhibit a clear dependence on , as shown in Fig. 1(c). The data show a broad peak centered at  Hz and for higher , the population in drops gradually to zero. The -dependence is also evident in the density distributions after time-of-flight. Figure 2(a) shows absorption images along the vertical axis for several values of . The spatial structure of the clouds changes from a singly-peaked distribution at low to a ring-shape with density modulations around the circumference at higher .

Typically, ring-shaped density distributions in such time-of-flight images indicate the presence of orbital angular momentum Ryu et al. (2007), and the appearance of density modulations around the circumference could be interpreted as the matter-wave interference pattern of two or more angular momentum eigenstates Scherer et al. (2013); Moulder et al. (2012). For large , one may obtain a good understanding of the ring formed in a simplified, ‘free-space’, picture. In this picture, one can regard the spin-changing collision as scattering two atoms from the stationary BEC into in a ring in momentum space with radius . Momentum conservation requires that the two scattered atoms propagate in opposite directions, as indicated in Fig. 2(a) for  Hz. Although, in principle, the system exhibits polar symmetry, bosonic stimulation breaks the symmetry leading to azimuthal modulations, which result in the formation of counter-propagating wave packets. Interestingly, these wave packets should comprise an Einstein-Podolsky-Rosen (EPR) entangled pair in momentum and spin state  Einstein et al. (1935); Bohm and Aharonov (1957).

(Color online) (a) (Solid line) Effective potential experienced by
Figure 3: (Color online) (a) (Solid line) Effective potential experienced by and (red dashed line) the cylindrical box approximation. (b) Instability rates for . The peak instability rate is that of the mode; the peak rate decreases monotonically with increasing . (c) Instability rates in the range to  Hz. (d) A close-up of (c) showing the instability rates at  Hz.

To elaborate on the qualitative picture presented so far, we investigate the excitation spectrum of the states that arises from the Hamiltonian in Eq. (1). In the Thomas-Fermi approximation, in each lattice site has the shape of the inverted confining potential. Accordingly, atoms in experience a flat potential bottom plus a small parabolic repulsion  Scherer et al. (2010). Outside the BEC, the potential rises steeply and the effective potential may be approximated by a cylindrical box Scherer et al. (2013). Figure 3(a) shows the effective potential experienced by and the cylindrical box approximation. This motivates the simplified Hamiltonian for excitations of the system

The eigenstates of the cylindrical box potential are  Scherer et al. (2010),

where is the Thomas-Fermi radius of the cloud, are Bessel functions of the first kind, and is the ’th zero of . The eigenstates have energies . The substitution of into Eq. (2) leads to the matrix elements . Cylindrical symmetry of and orthogonality of for different yields . Additionally, a numerical evaluation of the matrix elements shows that is a good approximation. With this simplification, Eq. (2) may be put into a symmetric form and diagonalized through a Bogoliubov transformation. The excitation energies are

The eigenvalues are either real or imaginary depending on the interplay of , and : a real determines the standard phase evolution of an eigenstate; an imaginary value describes unstable evolution, i.e., growth or decay in the amplitude of an eigenstate. In the following, we focus on the unstable evolution and refer to as instability rates.

For our experimental parameters, the instability rates form a dense ‘forest’ of overlapping resonances. The density of atoms sets the width and peak value of each resonance, and it is the extreme compression along the lattice symmetry axis that causes many modes to overlap. For , the central site has atoms, and the peak value of the mean field repulsion is 55 Hz (see Fig. 3(a)). To illustrate how the modes change in shape as a function of , Fig. 3(b) shows for in the central lattice site. The peak instability rate is reached by the mode due to its large overlap with the BEC in . Figure 3(c) shows the instability rates for all relevant modes in the range to  Hz: it is clear that a single value supports many unstable modes.

We may quantify the multi-mode character of the spin dynamics by investigating the instability rates at a given . We focus on the high ‘free-space’ regime. Figure 3(d) shows the instability rates in a narrow interval around  Hz, for which 19 modes are unstable. We quantify multi-mode amplification by the ratio , where is the mean value of the instability rate of the unstable modes, and is the mean difference of instability rate between modes. For , all modes grow in population almost equally on a timescale , as though degenerate in instability rate. In the case of the most unstable modes in Fig. 3(d),  Hz and  Hz, for which the time scale for population growth is 2.7 ms, consistent with the experimentally measured value of 3.0 ms (see Fig. 1(b) and associated discussion). Only on a timescale  ms will the evolution of different unstable modes become resolvable, but this lies in the non-linear regime, well outside the 8 ms evolution time we employ (see Fig. 1(b)).

The expansion of excitations in in terms of states with angular momentum arises naturally from the cylindrical symmetry of the optical lattice, but this choice of basis is arbitrary. In the ‘free-space’ picture, two atoms from the stationary BEC undergo a momentum-conserving spin-changing collision: in the cylindrical basis, one spin state gains positive angular momentum while the other gains , and the total angular momentum remains zero.

We simulate the experimental results by forming a superposition state in each lattice site comprised of the set of modes that have a finite instability rate at a given . In light of earlier work Scherer et al. (2010, 2013), we form a general superposition state for one spin state (e.g., ) in each lattice site, consisting of positive and negative angular momentum components. The expansion coefficients are sampled from appropriate probability distributions for each realization of the simulation Scherer et al. (2013). The state is given by , where the complex amplitude is , and and are sampled separately. The first factor, , is the square root of the probability to obtain atoms in mode in ; i.e., a two-mode Fock state Scherer et al. (2013). The second factor, , establishes a random phase between and modes, and between different lattice sites. For each lattice site the BEC population is calculated and is evolved for 20 ms in free space 111We model the transfer to the optical lattice by assuming that the atoms are ‘frozen’ by the lattice at . The BEC in the relaxed magnetic trap is taken to have a Thomas-Fermi profile along the lattice symmetry axis and we numerically integrate the profile over successive intervals of to obtain the atom number in each site.. The superposition across all lattice sites is then formed and the squared modulus is taken to obtain the density distribution. Finally, the simulated density distributions are convolved with the measured point spread function.

(Color online) (a) Cloud radius after 20 ms time-of-flight. (Black points) experimental data, (red squares) simulations, (blue line) ballistic expansion model (offset to begin at the average value of the mean field repulsion across the populated lattice sites). (b) Density correlation function at
Figure 4: (Color online) (a) Cloud radius after 20 ms time-of-flight. (Black points) experimental data, (red squares) simulations, (blue line) ballistic expansion model (offset to begin at the average value of the mean field repulsion across the populated lattice sites). (b) Density correlation function at  Hz (mean of 54 realizations for both simulations and experimental data): (black line) cross-correlation for clouds, (blue dash-dot line) mean auto-correlation function for clouds, (red dotted line) auto-correlation for simulations.

Examples of the simulated density distributions obtained following this procedure are shown in Fig. 2(b). The simulated distributions reproduce the transition from a density distribution with a central peak at low to the striking ring structures observed in the experiment at high . This transition can be understood from the mode structure in Figs. 3(b) and (c): the dominant mode extends from to  Hz, meaning that in this interval will have non-zero density in the center of the cloud, even after time-of-flight. For larger , modes without population in the center and a range of angular momenta predominate in the experimental and simulated density distributions. While the simulations capture the overall ring structure well, it is clear that they show more azimuthal structure than the observed density distributions.

To make a quantitative comparison of the simulated and experimental density distributions, we first investigate the size of the clouds. Figure 4(a) shows the cloud radius of the experimental and simulated density distributions. The box model simulations show good agreement with the experimental cloud size, with deviations arising for low and high where the cylindrical box approximation breaks down. At low , this arises from the  Hz repulsive bump (see Fig. 2(a)), and at high , the box potential underestimates the energy imparted to the scattered atoms due the finite slope of the walls. One can obtain a simple estimate of the cloud radius in the high regime using the ‘free space’ picture: if each wave packet gains , the position of an atom assuming ballistic expansion is given by , where is the in-trap radius and is the time-of-flight.

To study the structure around the circumference of the experimental and simulated density distributions, we employ an angular density correlation function. Momentum conservation in the collision process requires that a density peak at angle in a cloud leads to an anti-correlated density peak located at in the component (see Fig. 2(a) for  Hz). We define the correlation function , where are the two angular density distributions and the angled brackets denote the mean over . The correlation function for  Hz is shown in Fig. 4(b) and exhibits the expected anti-correlation between the two clouds. In order to analyze the -mode composition of the states generated by the spin dynamics, we also show the auto-correlation function for the experimental and simulated density distributions. A comparison of these indicates that only a subset of the energetically allowed modes contribute to the observed experimental states. Nonetheless, the auto-correlation function confirms that several states contribute to the observed distributions because is approximately flat away from the peak at due to the destructive interference of several frequency components; in the ‘free space’ picture, this amounts to random spontaneous symmetry breaking of the polar symmetry.

In conclusion, we have investigated spin dynamics in a 2D quantum gas and have observed qualitatively new features such as ring-shaped density distributions in time-of-flight. A theoretical analysis of the spin excitation modes in the system provided a qualitative understanding of the ring-shaped clouds and showed their angular structure is due to the interference of several modes. In future work, we will gain a full understanding of the unstable spin modes by studying the dynamics in a single site of the optical lattice. Finally, tailoring the trapping potential further will enable the preparation of anti-correlated wave-packets of quantum degenerate atoms with the aim of producing an atom source with non-local EPR entanglement.

We acknowledge support from the Danish National Research Foundation, the Danish Council for Independent Research, the Lundbeck Foundation, the DFG (SA1031/6), the Cluster of Excellence QUEST and the German-Israeli Foundation.

For everything else, email us at [email protected].


Bayesian Quantile Regression Using Random B-spline Series Prior

Priyam Das1 and Subhashis Ghoshal 1
11Deaprtment of Statistics, North Carolina State University, NC, USA
August 9, 2021

We consider a Bayesian method for simultaneous quantile regression on a real variable. By monotone transformation, we can make both the response variable and the predictor variable take values in the unit interval. A representation of quantile function is given by a convex combination of two monotone increasing functions and not depending on the prediction variables. In a Bayesian approach, a prior is put on quantile functions by putting prior distributions on and . The monotonicity constraint on the curves and are obtained through a spline basis expansion with coefficients increasing and lying in the unit interval. We put a Dirichlet prior distribution on the spacings of the coefficient vector. A finite random series based on splines obeys the shape restrictions. We compare our approach with a Bayesian method using Gaussian process prior through an extensive simulation study and some other Bayesian approaches proposed in the literature. An application to a data on hurricane activities in the Atlantic region is given. We also apply our method on region-wise population data of USA for the period 1985–2010.

1 Introduction

In a regression model, if the distribution of the response variable is highly skewed, traditional mean regression model may fail to describe interesting aspect of the relationship between the prediction and response variables. For instance, if we deal with the income distribution data of a certain state or country, the traditional mean regression will get affected by the outliers, i.e., the income of top 1–2 % people of that region, and hence is of limited use for prediction of income of general people. This situation arises often in business, economics, environmental and many other fields. As an alternative to traditional linear regression, quantile regression is one of the most popular and useful regression technique.

Extensive research have been done on quantile regression. Most of them were approached  from  the  frequentist  perspective ([Koenkar (2005)]). [Koenkar and Bassett (1978)] estimated the th regression quantile for the dataset , by minimizing the loss function with respect to and . Due to computational convenience and other theoretical properties, this method remained popular for long time. Later on, many other methods of quantile regression were proposed both in frequentist and Bayesian framework. [Yu and Moyeed (2001)] introduced Bayesian quantile regression for independent data. A few generalization and extension of quantile regression were proposed in [Kottas and Gelfand (2001)],  [Gelfand and Kottas (2003)],  [Kottas and Krnjajic (2009)]. These   papers mainly focused on quantile regression for a single quantile level for censored independent data. [Geraci and Bottai (2007)] proposed the single level quantile regression model with subject specific random intercept term which accounts for within group correlation. The drawback of separate quantile regression for different quantiles is that the natural ordering among different quantiles is not ensured. In other words, the lower quantile estimation curves might cross the upper quantile curves which violates the ordering of quantiles of a fixed distribution.

There exist a few methods addressing the crossing issue in estimating multiple quantile regression. [He (1997)] proposed a method of estimating non-crossing quantile curves assuming a heteroskedastic model for response variable. Under this assumption, predictors might affect the response variable via a location and scale change. [Neocleous and Portnoy (2008)] proposed a method of estimating the quantile curve by linear interpolation on the estimated quantile curves on a grid of quantiles. [Takeuchi (2004)] and [Takeuchi et al.(2006)] used support vector machines (SVM, [Vapnik (1995)]) for non-crossing quantile regression. But, as mentioned in [Shim and Lee (2010)], a disadvantage of using SVM for non-crossing quantile regression is that when multiple quantiles are needed, every adjacent pair of conditional quantile functions should be computed. [Shim et al.(2009)] proposed non-crossing quantile regression using doubly penalized kernal machine (DPKM). For this method also, we need to compute quantile curves separately for each quantile. The method proposed in [Wu and Liu (2009)] sequentially updates the quantile curves under the contraint that the upper quantile curves stay above the lower ones. A drawback of this method is that the predicted quantile curves are dependent on the grid of quantiles for which we want to calculate the quantile curves.

Quantile regression for fixed number of levels of quantiles with monotonicity constraint was proposed in [Dunson and Taylor (2005)] and [Liu and Wu (2011)]. The method proposed in [Bondell et al.(2010)] is sensitive to the number of prediction quantile levels. For spatial quantile regression [Reich et al.(2011)] used Bernstein polynomial to maintain the monotonicity constraint of the quantile function. [Reich (2012)] proposed non-crossing multiple quantile regression using piece-wise Gaussian basis functions to interpolate the quantile function at different quantile levels. [Reich and Smith (2013)] suggested the use of different symmetric and asymmetric quantile functions for modeling based on the data. Models based on a fixed number of quantiles do not give the estimates of all quantiles and may also be sensitive to the number and the location of the grid points of the quantile levels, which is not desirable.

Instead of fitting quantile curves for a fixed set of quantiles, a more informative picture emerges by estimating the entire quantile curve. Suppose denote the -th conditional quantile of a response at , being the predictor. A linear simultaneous quantile regression model for at a given is given by

where is the intercept and is the slope smoothly varying as function of . Thus estimating the quantile function involves non-parametric estimation of the function and . The main challenge in fitting this kind of model remains in complying with the monotonicity restriction of the predicted quantile lines as a function of all values of the predictor .

[Tokdar and Kadane (2012)] obtained a very useful characterization of the monotonicity constraint (see Equation (1) below). They used the characterization to propose a suitable prior on the quantile function in a Bayesian approach, and computed the posterior distribution of the quantile curves. More specifically, they used two (possibly) dependent Gaussian processes and to induce prior on and in such a way so that remains a monotonically increasing function of .

Gaussian processes do not have any shape restriction. To induce monotonicity property on and , [Tokdar and Kadane (2012)] took the running integral of the exponential transformation of two (possibly correlated) Gaussian processes. To evaluate the likelihood, it requires solving the equation (see Equation (6) below). Without having a convenient expression, the solution needs to be done completely numerically which involves computing integrals of transformed Gaussian process over a very fine grid. Substantial improvement in computing approach is possible using a finite random series prior ([Shen and Ghoshal (2015)]). Especially using a B-spline basis, monotonicity can be ensured by choosing a prior only on coefficients in increasing order. Piece-wise polynomial representation of splines allows obtaining solving analytically. Thus we can reduce computation time substantially using a random series prior based on the B-spline basis.

2 Model Assumptions

We observe independent random samples of an explanatory variable and a response variable , both of which are assumed to be univariate. By monotonic transformations, we transform both of them to the unit interval. From Theorem 1 of [Tokdar and Kadane (2012)], it follows that a linear specification , , is monotonically increasing in for every if and only if

where and are positive constants and are monotonically increasing in . Because also has domain , we have the boundary conditions and . Now, putting in Equation (1), and using , we obtain

which implies and hence (1) reduces to

Now putting and using , we obtain

which implies . Therefore in the present context, the quantile regression function has representation

Equation (3) can be re-framed as

where and denotes the slope and the intercept of the quantile regression, which are smooth functions of . Provided that is strictly increasing in for all , the conditional density for at given is given by

where solves the equation

Then the joint conditional density of given is given by .

3 Regression with Spline

Function estimation on a bounded interval through B-spline basis expansion is one of the most convenient approaches. To construct a prior on the quantile function , or equivalently on and , we need to ensure their monotonically increasing properties. If the coefficients in a B-spline basis expansion are in increasing order, then the corresponding quantile function will be an increasing function of ([Boor (2001)]). In fact, if the target function is strictly increasing, the quality of the approximation of spline expansions is maintained when the coefficients are restricted by increasing order ([Shen and Ghoshal (2015)]). This motivates us to use B-splines with increasing coefficients as the basis of the quantile function. The degree of B-spline determines the degree of smoothness, in that quadratic splines are continuously differentiable, cubic splines are continuously twice differentiable and so on, and are able to optimally approximate functions of smoothness index only up to the degree of splines. Thus optimal estimation of smoother functions requires higher degree splines. On the other hand, computational complexity increases with the degree of the B-spline basis. Smoothness of order higher than two is typically not visually distinguishable from smoothness of order , and therefore splines of degree up to 3 suffices in most applications. In case of quadratic B-spline basis, is a linear combination of piecewise second degree polynomials in terms of , and hence restricted to an interval between two knots, is an increasing quadratic function. Thus we can solve for each data points analytically, unlike the case of Gaussian process prior, where one has to use numerically integrate and apply iterative Newton-Raphson method to solve them for each realization of the quantile process from its posterior distribution. Due to the increasing complexity and higher computation time, results for with B-splines of order higher than 3 have not been shown in this paper. For a cubic B-spline, is linear combination of piecewise third degree polynomials. Hence, for any given interval in the domain, is a monotonically increasing third degree polynomial. Hence in this case also, can be solved analytically. However the analytic algorithm for solving cubic equation requires computing all three roots even though only one root falls in the admissible interval, which makes the cost of computing roots in a cubic equation substantial. Due to the monotonicity of , the bisection method of finding roots is an attractive alternative. In our numerical experiments, we found that for cubic splines the bisection method is slightly faster than the analytical approach.

Let be the equidistant knots on the interval where , . For B-spline of th degree, the number of basis functions is . Let be the basis functions of th degree B-splines on on the above mentioned equidistant knots. We consider the following basis expansion of the quantile functions through the relations

Taking ensures and ensures . The increasing values of the coefficients takes care on monotonicity ([Boor (2001)]). Thus, and are monotonically increasing functions of from onto . Let, and be defined by

and the restrictions in (7) equivalently be written as

Therefore the spacings of the B-spline coefficients take value in the unit simplex. A natural prior is thus given by the uniform distribution on the unit simplex, or the Dirichlet prior with parameter . The Bayesian procedures based on quadratic splines (i.e. ) will be called the Quadratic Spline Simultaneous Quantile regression (QSSQR) and that based on cubic splines (i.e. ) will be called the Cubic Spline Simultaneous Quantile regression (CSSQR).

3.1 Model Fitting

To compute the log-likelihood function, we proceed like [Tokdar and Kadane (2012)]. By (5), the log-likelihood is given by

where is obtained from (6).

Due to the monotonicity of and , any convex combination of them will have one and only one solution on the interval . To evaluate the log-likelihood we compute the derivatives of the basis splines ([Boor (2001)]),

The log-likelihood thus reduces to

3.2 MCMC and Transition Step

In Equation (10), we note that and are on the unit simplex. We shall use a Metropolis-Hastings algorithm to obtain Markov Chain Monte Carlo (MCMC) samples from the posterior distribution. In MCMC, to move on the simplex, we generate independent sequences and , , from for some .

Define and for . Consider the proposal moves and given by

As shown in the appendix, the conditional distribution of given is

Similarly, we can find the conditional distribution of . Now from the given set of values for and the updated values and can be found using the following relation.

While evaluating the likelihood, using the relations mentioned in Equation (8) and (9), we can find the likelihoods at the initial and destination points respectively in terms of and , and and . The acceptance probability in Metropolis-Hastings algorithm is then given by

where denotes the likelihood and is the uniform Dirichlet prior density on the corresponding parameters.

3.3 Choosing value of and Model Averaging

In the previous sections, we discussed fitting spline basis functions for fixed number of partitions of equal length in , the transformed domain of . In other words, we developed the methodology for fixed number of basis functions. The number of basis functions controls the smoothness and hence the quality of the estimate. For smaller value of , the bias is high but variability is less, so there is a bias-variance trade-off. Moreover MCMC chain runs faster with better mixing if the value of is smaller. It is desirable to determine the value of based on the data to apply the right amount of smoothness. In the Bayesian setting, it is natural to put a prior on the smoothing parameter and make inference based on the posterior distribution. For computational efficiency, the range of should be a finite collection of consecutive integers. Once we fix the range of all possible values of , we can either consider empirical Bayes selection based on the marginal likelihood of , or a Bayesian model averaging. A common approach to posterior computation for a parameter space of varying dimension is through the reversible jump MCMC method, but its implementation can be challenging. An alternative approach is provided by the method described in [Chib and Jeliazkov (2001)]. In this approach marginal probabilities corresponding to different values of are obtained from separate MCMC output for each value of . Since all these chains are run independently of each other, parallel computing can be implemented, which can greatly offset the cost of running separate chains for all feasible values of . Both the empirical Bayes approach by identifying the value of with highest posterior probability, and the hierarchical Bayes approach based on Bayesian model averaging, can be implemented using this method, and they have identical computing cost.

Consider the th degree B-spline with many partitions of of equal length. The equidistant knot sequence is given by such that for . Let be the parameter of interest. We denote the whole data by , and with a slight abuse of notation, denote the joint density of also by . The posterior density is then given by

over , a subset of , specified by the Equation (10). For any , the logarithm of the marginal likelihood is given by

It is recommended to take to be a point where it has high density under the posterior. For any given value of , we can easily calculate the first two terms of the Equation (15). Our goal is to estimate the posterior ordinate given the posterior sample , where denotes the number of draws from the posterior sample after burn-in. Let us denote the proposal density by for the transition from to . Suppose the probability of move is given by

[Chib and Jeliazkov (2001)] proposed a simulation consistent estimate of the posterior ordinate given by

where are the sampled draws from the posterior distribution and are the draws from , given the fixed value . Then the estimated logarithm of the marginal is given by

As mentioned in [Chib and Jeliazkov (2001)], the computation time of the marginal likelihood of each of the probable model is small and is almost readily available after the MCMC chain finishes. After finding the marginal likelihood for each of those models, the value of corresponding to the model with the highest marginal likelihood can be treated as the best possible value of . Suppose, we fix the domain of to be where is a collection of discrete natural numbers. Let, denotes the marginal likelihood of . Now select the best possible value of by

Afterwards, in this paper, we named the method of estimation with B-spline basis functions corresponding to the to be the empirical Bayes (EB) method.

Instead of choosing the value of by the empirical Bayes method, another possibility is to perform model averaging with respect to the posterior distribution of over . To find the weights of the corresponding models, we calculate the marginal likelihood by the Metropolis-Hastings algorithm for each following the method described in [Chib and Jeliazkov (2001)]. After obtaining the marginal likelihoods for each value of , we obtain the weighted average of the estimated quantile curves where weights are proportional to their marginal likelihoods. Let, and denote the weights and the marginal likelihoods for . Let and be the posterior means of and respectively for given . Then we have

Monotonicity of for each ensures that is also monotonic. Similarly, is also monotonic. We note that the estimated quantile functions for each is totally determined by the estimated coefficients of the corresponding basis functions. Since their relation is also linear, to calculate the overall posterior mean quantile function, it suffices to obtain the posterior mean of the coefficients of the basis functions for each . With the knowledge of weights and posterior mean of basis coefficients for each , we can derive the estimated slope, intercept and quantile functions easily without saving the quantile function values over grid points. For the rest of this paper, we called this to be the hierarchical Bayes (HB) method.

4 Large-sample Properties

In this section, we give arguments that indicate the posterior distribution based on a random series of B-splines concentrates near the truth. We assume that the true quantile regression function (which necessarily has the linear structure of the form is absolutely continuous with respect to the Lebesgue measure and is the corresponding “quantile density function”. We assume that satisfies the following regularity condition:

(A) For any , there exists such that for any strictly increasing, continuous, bijection such that , we have for all .

We also assume that the true conditional density function is positive and continuous (and hence is bounded above and below) and that the distribution of is . Let stand for the true conditional distribution function.

Clearly the random quantile regression function (i.e., in the model following the B-spline basis expansion prior) is differentiable with quantile density function . Let the corresponding conditional density function be denoted by and the cumulative distribution function . We assume that the prior on is positive for all values.

It follows from a well-known theorem of [Schwartz1965] that the posterior distribution of the joint distribution of is consistent with respect to the weak topology if for every , the prior probability of

is positive. Below we show that the condition holds at the true for the B-spline random series prior.

We follow the line of argument given in [Hjort and Walker (2009)] although in our case the presence of the conditioning variable makes the situation considerably more complicated. Define . Note that if is , then has conditional density . Hence

where we have used the relation between quantile density and probability density and , and equivalently and .

Note that the assumption implies that and and are continuous functions, and hence for a sufficiently large number of knots , both functions can be approximated by a linear combinations of B-splines up to any desired degree of accuracy. Fixing at a sufficiently large value (which has positive prior probability), we find the approximating coefficients, and then consider all vector of coefficients in a small neighborhood around the vector of approximating coefficients. By the choice of continuous and positive prior density on the coefficient vector in the random spline series, it follows that the event has positive prior probability. Thus neighborhoods of the true quantile function get positive prior probabilities. In view of the portmanteau theorem characterizing weak convergence in terms of convergence of quantile functions (see [van der Vaart (1998)], Lemma 21.2), it follows that weak neighborhood of the true distribution get positive prior probabilities, and so do uniform neighborhoods of the true distribution in view of Polya’s theorem. This means, in view of the Condition (A) on , that the second term in (19) can be arbitrarily small with positive probability.

For the first term in Equation (19), by the same arguments we find that is uniformly close to with positive probability, and hence the integrand is small with positive probability since is assumed to be bounded below.

Thus the prior puts positive mass in neighborhoods defined by Equation (18), and hence by Schwartz’s theorem, the posterior probability of all conditional distributions satisfying

tends to one for any continuous function on and .

The above conclusion on posterior consistency is about the conditional distribution function in terms of the weak topology, which does not immediately show concentration of posterior for the quantile function in neighborhoods of the true quantile function. To this end, we further assume that the collection of true conditional distribution function is equicontinuous, i.e., given , there exists such that whenever , we have that for all . Then by an easy modification of the proof of Polya’s theorem (see Lemma 2.11 of [vanderVaart]), it follows that for all implies that . Thus the posterior probability of tends to one for any . Now for any , as ,

As is bounded below by a positive number, this implies that the posterior probability of tends to one for any .

5 Simulation Study

In Section 2, we note that after some monotonic transformation, every quantile regression function can be represented in the form mentioned in equation (3). For simulation purposes, we consider two different true quantile functions structure. For each cases, we compare the true and the estimated values of the slope, intercept and the quantile regression function for and , and for sample size by our proposed methods, Gaussian SQR (GSQR) proposed in [Tokdar and Kadane (2012)], Bernstein polynomial SQR (BPSQR) proposed in [Reich et al.(2011)] and qreg and qreg_spline functions under the BSquare package in R by [Smith and Reich (2013)] (see [Reich (2012)],[Reich and Smith (2013)] for details).

For our proposed methods, suppose be the equidistant knots on the interval such that for all . Since MCMC does not mix very well for large values of , we considered only first 8 possible values of for either cases ( for QSSQR and for CSSQR). Separately, for quadratic and cubic B-spline methods, for each of the values of , we run 20000 iterations with 5000 burn-in. For QSSQR, while evaluating likelihood, we can solve (6) analytically very fast. Although CSSQR can also be solved analytically, we note that the procedure is considerably slower than bisection method. We apply the latter method with precision of , after finding the interval where is located for , by linear search.

To calculate the marginal likelihood of each model, we need to calculate the expression given by Equation (16). To calculate the numerator of that term, we considered the last 5000 iterations and we take for where denotes the -th update of the parameter in MCMC. Thus the numerator term is evaluated by the end of MCMC chain. To calculate the denominator term, we consider 5000 draws of {} from . Then we computed the marginal likelihoods of each model. We also computed the estimates corresponding to weighted average quadratic and cubic spline models taking the weights proportional to the marginal likelihoods over the domain of values of . We used parallel computing for different values of to obtain posterior probabilities.

To implement the GSQR (Gaussian process SQR) method, proposed in [Tokdar and Kadane (2012)], we took equidistant knots over the interval , the starting and the ending knots being at 0 and 1 respectively. At each iteration step, we save the values of the Gaussian process at those knots and use those values for likelihood evaluations. We ran 20000 iterations with 5000 burn-in. We evaluate the likelihood function by solving (6) for each data-points using Newton-Raphson method, as mentioned in [Tokdar and Kadane (2012)]. We took the precision for convergence criteria to be .

While estimating the quantile regression (QRF) functions using the functions qreg and qreg_spline functions under the BSquare package in R by [BSquare], for each case we performed 50000 iterations with 10000 burn-in. For BPSQR (Bernstein polynomial SQR) method we performed 10000 iterations with 1000 burn-in. The function (implemented in R by [Reich et al.(2011)]) for estimating the quantile functions used for this method is available at the link reich/code/SpaceQRapprox.R.

For our proposed method, we calculate the uniform posterior credible region for the quantile regression function (QRF) for three distinct co-variate values for for three different sizes of sample . To find the posterior credible region for the QRF for a given co-variate value using Empirical Bayes method, we calculate the posterior mean of the estimated QRF for all for each value of at that given covariate value . Since we run iterations with burn-in , only last are used for calculating the posterior mean. For a given and covariate value , define such that

where denotes the QRF for in the -th iteration after burn-in for B-spline method with partitions and denotes the posterior mean of the QRF for for B-spline method with partitions. After that we calculate the percentile of for each . Thus we find the size of the posterior credible region for each value of . To find the coverage, we repeat the simulation study times under different random number seeds and count the number of times the distance of the true QRF from the estimated EB estimate for all is not more than the size of the posterior credible region for EB approach of corresponding .

In HB method, we again have iterations with burn-in and only last are used for calculating the posterior mean. To find the posterior credible region for the QRF for a given co-variate value in this case, we calculate such that

where denotes the QRF for in the -th iteration after burn-in for B-spline method and denotes the posterior mean of the QRF for for B-spline method. The weights corresponding to each to find and are the same and they are derived as mentioned in section 3.3. Then we find the size of the posterior credible region by calculating the percentile of . To find the coverage, we repeat the simulation study times under different random number seeds and count the number of times the distance of the true QRF from the estimated HB estimate for all is not more than the size of the posterior credible region for HB approach.

It is well-known that posterior credible band of smooth functions have an under coverage property (see [Cox (1993)], [Knapik et al.(2011)],[Szabo et al.(2015)], [Yoo and Ghoshal (2016)]). To alleviate the problem, undersmoothing or modifying the credible region is needed. We inflate the obtained credible region by blowing the radius of the region by a slowly increasing factor. We choose the inflation factor in our examples. This works for all the sample sizes and under different simulation settings and different true quantile regression functions. We report two simulation studies in this paper described in Sub-section 5.1 and 5.2. We have also provided the posterior coverage with and without inflation of uniform posterior credible interval for three different sample sizes under two different true quantile regression functions.

5.1 First Study

Consider where

and . We note that and are strictly increasing function from to satisfying and . Note then the conditional quantile function is given by where and . Observe that since the quantile function is the inverse of the cumulative distribution function, for the random variable has conditional quantile function . We generate values of the predictor variable independently from . Then we simulate variable from the following equation

where , and ’s are i.i.d. , . We simulated observations from the distribution and compared the estimated results with the true ones for both Gaussian and B-spline methods.

The comparative study of the performances in estimation of our method with other methods under this simulation study has been provided in Figure 1. We found that there is not much difference between the estimates given by quadratic and cubic B-spline approaches. Hence, for convenience, we only compared the output of our Hierarchical Bayes QSSQR method with other proposed methods in these figures. The root mean integrated squared error (RMISE) is given by the square root of the average of the square of the differences of the estimated and the true values of the curves at those grid points. In Table 1, we compared the RMISE for above mentioned estimated curves for all methods. It can be noted that in our simulation study, the estimated slope, intercept, quantile regression function for and are curves on the domain . To calculate the RMISE, we divide the interval using partition such that such that for all . We note that our proposed methods have lower RMISE of estimation curves than that of GSQR. Among other methods, qreg function from BSquare worked quite good, though our method yields lower RMISE values for most of the cases. Among our set of proposed methods, we do not see any noticeable improvement by using CSSQR instead of QSSQR. We note that the HB approach has lower RMISE values than the corresponding EB alternative.

(a) Intercept
(b) Slope
(c) Estimated QRF at
(d) Estimated QRF at
(e) Estimated QRF at
(f) Estimated QRF at
(g) Estimated QRF at
(h) Estimated QRF at
Figure 1: (First Simulation study) Comparison of true and estimated intercept, slope and estimated quantile regression functions (QRF) at , and for different methods QSSQR(HB), GSQR, qreg, qreg_spline and BPSQR.
Methods Intercept Slope
QSSQR(HB) 0.039 0.059 0.029 0.037 0.030 0.008 0.035 0.038
QSSQR(EB) 0.097 0.088 0.071 0.081 0.037 0.025 0.076 0.066
CSSQR(HB) 0.026 0.048 0.034 0.042 0.05 0.021 0.03 0.051
CSSQR (EB) 0.033 0.069 0.034 0.041 0.051 0.051 0.051 0.014
GSQR 0.075 0.052 0.091 0.101 0.112 0.121 0.14 0.089
qreg 0.100 0.019 0.089 0.082 0.074 0.106 0.142 0.026
qreg_spline 0.101 0.003 0.093 0.084 0.077 0.103 0.118 0.019
BPSQR 0.267 0.353 0.177 0.118 0.063 0.181 0.168 0.145
Table 1: (First Simulation study) Comparison of RMISE of estimation of slope, intercept and quantile regression function at , and under different methods QSSQR(HB); QSSQR(EB); CSSQR(HB); CSSQR(EB); GSQR; qreg; qreg_spline and BPSQR.

In Table 2, we present estimation accuracy and coverage with and without inflation of the proposed methods for three different sample sizes . We note that the posterior coverage of the inflated credible band for the Hierarchical Bayes are typically better than that of corresponding Empirical Bayes. No noticeable improvement is noticed by using CSSQR instead of QSSQR.

Size Coverage Size Coverage Size Coverage
Table 2: (First Simulation study) Size and posterior coverage of inflated (in bold) and regular uniform posterior credible interval of estimated quantile regression function for for for three different sizes of sample for QSSQR(HB), QSSQR(EB), CSSQR(HB) and CSSQR(EB).
(a) QSSQR :
(b) CSSQR :
(c) GSQR :
(d) QSSQR :
(e) CSSQR :
(f) GSQR :
(g) QSSQR :
(h) CSSQR :
(i) GSQR :