<blockquote>
<p>Generating random galaxy catalogs for correlation functions</p>
<p>... the random catalog can be just be a uniform distribution of points in the same volume as the real data.</p>
<p>However, real-world galaxy catalogs are not homogenous boxes, and have very irregular shapes and observationally induced redshift and angular distributions. Even some mock-catalogs are endowed with these properties to better represent real data. In these cases, how is a random catalog best produced? ...</p>
<p>If the answer is that some nonuniform distribution should be used, is there a usual method for doing so? The literature I have found (e.g. for the SDSS published data, and for galaxy mocks mimicking SDSS's constraints) does not discuss this in any way. <strong>Note, if this is a highly involved process beyond a brief summary and a source or two, I do not expect a full answer here</strong>. Rather, I am trying to determine if there is an existing set of standard methods which are documented but have eluded me thus far.</p>
</blockquote>
<p>For SDSS see:</p>
<ul>
<li><p>&quot;<a href="https://www.aanda.org/articles/aa/full/2006/08/aa3902-05/aa3902-05.right.html" rel="nofollow noreferrer">Large scale correlations in galaxy clustering from the two degree field galaxy redshift survey</a>&quot; (2006), by N. L. Vasilyev, Yu. V. Baryshev and F. Sylos Labini.</p>
</li>
<li><p>&quot;<a href="https://arxiv.org/abs/1303.2432" rel="nofollow noreferrer">The SDSS Galaxy Angular Two-Point Correlation Function</a>&quot; (12 Mar 2013), by Y. Wang, R. J. Brunner, J. C. Dolence. The include their SQL query to the SDSS catalog archive server:</p>
</li>
</ul>
<blockquote>
<p><strong>1 INTRODUCTION</strong></p>
<p>One of the most powerful and simplest probes of the galaxy distribution is the two-point angular correlation function, which quantifies the excess probability above a random distribution of finding one galaxy within a specified angle of another galaxy. For the case of a Gaussian random field, the two-point angular correlation function and its Legendre transform pair provide a complete statistical characterization of the galaxy clustering (see, e.g., Peebles 1980). Even for the case of non-Gaussianity, the two-point angular correlation function provides a simple and important statistical test of galaxy formation models (Tegmark et al. 2004). The two-point angular correlation function has been studied at bright magnitudes from the data releases from the Sloan Digital Sky Survey (SDSS) such as the Early Data Release (EDR; Connolly et al. 2002).</p>
<p>...</p>
<p><strong>To accurately calculate the galaxy two-point angular correlation function, we must first minimize potential systematic effects in the galaxy catalog used to measure the correlation function</strong>. The systematics of the SDSS EDR were thoroughly studied by Scranton et al. (2002). To minimize the systematic effects of seeing and Galactic extinction, they determined that the SDSS EDR galaxy sample had to be masked to exclude regions with seeing greater than 1.&quot;75 and reddening &gt; 0.2 magnitudes. Given the importance of minimizing the impact of systematic effects on the galaxy two-point angular correlation function and the significant changes that were made in the SDSS data processing pipeline between the SDSS EDR and the SDSS DR7, we have repeated many of the tests presented in Scranton et al. (2002) by using the SDSS DR7 data. <strong>In this paper we present the methods used to contain these systematic effects, the results of these systematic tests, the actual galaxy two-point angular correlation function for the SDSS DR7, and our massively parallel implementation that rapidly calculates correlation functions for large data sets</strong>.</p>
</blockquote>
<p>Also: &quot;The galaxy correlation function and power spectrum&quot; (<a href="http://www.uio.no/studier/emner/matnat/astro/AST4320/h12/undervisningsmateriale/galaxyclustering.pdf" rel="nofollow noreferrer">.PDF</a>).</p>
<p>Wikipedia's &quot;<a href="https://en.wikipedia.org/wiki/Correlation_function_(astronomy)" rel="nofollow noreferrer">Correlation function (astronomy)</a>&quot;.</p>
<p>CalTech: &quot;<a href="https://ned.ipac.caltech.edu/level5/March04/Jones/Jones5.html" rel="nofollow noreferrer">Measurements of Clustering</a>&quot; (Use [Next] or [Contents] buttons to view all the pages).</p>
<p>There are fixups applied to the raw data to <a href="https://astronomy.stackexchange.com/a/26667/18054">correct for redshift and angular diameter distance</a>.</p>
<hr />
<p>For example catalogs and a description of the procedures see Shaun Cole's website: &quot;<a href="http://star-www.dur.ac.uk/%7Ecole/mocks/main.html" rel="nofollow noreferrer">Mock Galaxy Redshift Catalogues</a>&quot; where software and full details of the N-body simulations and the contruction and properties of the mock catalogues is provided for both 2dF Galaxy Redshift Survey (2dFGRS)  and SDSS galaxy redshift surveys.</p>
<p>Halo bias relation to dark matter and N-body simulations are discussed in &quot;<a href="https://arxiv.org/abs/1806.05870" rel="nofollow noreferrer">BAM: Bias Assignment Method to generate mock catalogs</a>&quot; (15 June 2018), by A. Balaguera-Antolínez, Francisco-Shu Kitaura, Marcos Pellerejo-Ibañez, Cheng Zhao, and Tom Abel.</p>
<p>Also check out the (under construction, and too busy working and writing papers) website: &quot;<a href="https://cgs.obs.carnegiescience.edu/CGS/Publications.html" rel="nofollow noreferrer">The Carnegie-Irvine Galaxy Survey (CGS)</a>&quot; - with missing papers available on arXix:</p>
<p>&quot;<a href="https://arxiv.org/abs/1111.4605" rel="nofollow noreferrer">The Carnegie-Irvine Galaxy Survey. I. Overview and Atlas of Optical Images</a>&quot;</p>
<p>&quot;<a href="https://arxiv.org/abs/1111.4606" rel="nofollow noreferrer">The Carnegie-Irvine Galaxy Survey. II. Isophotal Analysis</a>&quot;</p>
<p>&quot;<a href="https://arxiv.org/abs/1212.2639" rel="nofollow noreferrer">The Carnegie-Irvine Galaxy Survey. III. The Three-Component Structure of Nearby Elliptical Galaxies</a>&quot;</p>
<p>&quot;<a href="https://arxiv.org/abs/1603.00876" rel="nofollow noreferrer">The Carnegie-Irvine Galaxy Survey. IV. A Method to Determine the Average Mass Ratio of Mergers That Built Massive Elliptical Galaxies</a>&quot;</p>
<p>&quot;<a href="https://arxiv.org/abs/1707.03570" rel="nofollow noreferrer">The Carnegie-Irvine Galaxy Survey. V. Statistical study of bars and buckled bars</a>&quot;</p>
<p>&quot;<a href="https://arxiv.org/abs/1806.06591" rel="nofollow noreferrer">The Carnegie-Irvine Galaxy Survey. VI. Quantifying Spiral Structure</a>&quot;</p>
<p>&quot;<a href="https://arxiv.org/abs/1806.06350" rel="nofollow noreferrer">The Carnegie-Irvine Galaxy Survey. VII. Constraints on the Origin of S0 Galaxies from Their Photometric Structure</a>&quot;</p>

CC BY-SA 4.0
2018-06-18T22:04:48.713Z
I will look over these sources but it looks like you've hit the nail on the head. If I don't see any other answers soon I will accept yours.
Davis
CC BY-SA 4.0
2018-06-19T22:15:08.103Z
@Davis - I ran into a bit more somewhat related information while researching another question. Apparently [SDVision](http://irfu.cea.fr/Projets/COAST/SDvision.html) is the premier software for cosmology data visualization. It is available by request (as a *black box*) or for licensed user it enables you to "introduce personal modifications", see the link. They will likely inquire about your status as a researcher.
Rob
CC BY-SA 4.0
2018-06-18T20:36:24.750Z
2018-06-19T05:04:32.330Z
2020-06-17T09:47:40.653Z
Rob
<p>The galaxy correlation function, when computed directly from a galaxy catalog, relies on computing a randomly generated counterpart catalog.</p>

<p>When dealing with a simulated galaxy mock-catalog, where the sample of galaxies is roughly uniform over a large box (e.g. the galaxy mocks created from Millennium/Millennium II), the random catalog can be just be a uniform distribution of points in the same volume as the real data.</p>

<p>However, real-world galaxy catalogs are not homogenous boxes, and have very irregular shapes and observationally induced redshift and angular distributions. Even some mock-catalogs are endowed with these properties to better represent real data. In these cases, how is a random catalog best produced? With some sort of probability distribution in coordinates which attempts to mimic the "non-physical" (i.e. not due to observational constraints) part of the distribution? Or a simple binary mask which filters out empty regions, but otherwise leaves the random catalog uniform.</p>

<p>If the answer is that some nonuniform distribution should be used, is there a usual method for doing so? The literature I have found (e.g. for the SDSS published data, and for galaxy mocks mimicking SDSS's constraints) does not discuss this in any way. Note, if this is a highly involved process beyond a brief summary and a source or two, I do not expect a full answer here. Rather, I am trying to determine if there is an existing set of standard methods which are documented but have eluded me thus far.</p>

<galaxy><cosmology><data-analysis><galaxy-cluster><python>
Generating random galaxy catalogs for correlation functions
Davis
# Generating random galaxy catalogs for correlation functions

(Asked by: username_0 on 2018-06-14)

The galaxy correlation function, when computed directly from a galaxy catalog, relies on computing a randomly generated counterpart catalog.

When dealing with a simulated galaxy mock-catalog, where the sample of galaxies is roughly uniform over a large box (e.g. the galaxy mocks created from Millennium/Millennium II), the random catalog can be just be a uniform distribution of points in the same volume as the real data.

However, real-world galaxy catalogs are not homogenous boxes, and have very irregular shapes and observationally induced redshift and angular distributions. Even some mock-catalogs are endowed with these properties to better represent real data. In these cases, how is a random catalog best produced? With some sort of probability distribution in coordinates which attempts to mimic the "non-physical" (i.e. not due to observational constraints) part of the distribution? Or a simple binary mask which filters out empty regions, but otherwise leaves the random catalog uniform.

If the answer is that some nonuniform distribution should be used, is there a usual method for doing so? The literature I have found (e.g. for the SDSS published data, and for galaxy mocks mimicking SDSS's constraints) does not discuss this in any way. Note, if this is a highly involved process beyond a brief summary and a source or two, I do not expect a full answer here. Rather, I am trying to determine if there is an existing set of standard methods which are documented but have eluded me thus far.

**Tags:** galaxy, cosmology, data-analysis, galaxy-cluster, python


---

## 1 answers

### Answer by username_1 on 2018-06-18. Score: 2 | Accepted answer

> Generating random galaxy catalogs for correlation functions
>
> ... the random catalog can be just be a uniform distribution of points in the same volume as the real data.
>
> However, real-world galaxy catalogs are not homogenous boxes, and have very irregular shapes and observationally induced redshift and angular distributions. Even some mock-catalogs are endowed with these properties to better represent real data. In these cases, how is a random catalog best produced? ...
>
> If the answer is that some nonuniform distribution should be used, is there a usual method for doing so? The literature I have found (e.g. for the SDSS published data, and for galaxy mocks mimicking SDSS's constraints) does not discuss this in any way. **Note, if this is a highly involved process beyond a brief summary and a source or two, I do not expect a full answer here**. Rather, I am trying to determine if there is an existing set of standard methods which are documented but have eluded me thus far.

For SDSS see:

* "[Large scale correlations in galaxy clustering from the two degree field galaxy redshift survey](https://www.aanda.org/articles/aa/full/2006/08/aa3902-05/aa3902-05.right.html)" (2006), by N. L. Vasilyev, Yu. V. Baryshev and F. Sylos Labini.
* "[The SDSS Galaxy Angular Two-Point Correlation Function](https://arxiv.org/abs/1303.2432)" (12 Mar 2013), by Y. Wang, R. J. Brunner, J. C. Dolence. The include their SQL query to the SDSS catalog archive server:

> **1 INTRODUCTION**
>
> One of the most powerful and simplest probes of the galaxy distribution is the two-point angular correlation function, which quantifies the excess probability above a random distribution of finding one galaxy within a specified angle of another galaxy. For the case of a Gaussian random field, the two-point angular correlation function and its Legendre transform pair provide a complete statistical characterization of the galaxy clustering (see, e.g., Peebles 1980). Even for the case of non-Gaussianity, the two-point angular correlation function provides a simple and important statistical test of galaxy formation models (Tegmark et al. 2004). The two-point angular correlation function has been studied at bright magnitudes from the data releases from the Sloan Digital Sky Survey (SDSS) such as the Early Data Release (EDR; Connolly et al. 2002).
>
> ...
>
> **To accurately calculate the galaxy two-point angular correlation function, we must first minimize potential systematic effects in the galaxy catalog used to measure the correlation function**. The systematics of the SDSS EDR were thoroughly studied by Scranton et al. (2002). To minimize the systematic effects of seeing and Galactic extinction, they determined that the SDSS EDR galaxy sample had to be masked to exclude regions with seeing greater than 1."75 and reddening > 0.2 magnitudes. Given the importance of minimizing the impact of systematic effects on the galaxy two-point angular correlation function and the significant changes that were made in the SDSS data processing pipeline between the SDSS EDR and the SDSS DR7, we have repeated many of the tests presented in Scranton et al. (2002) by using the SDSS DR7 data. **In this paper we present the methods used to contain these systematic effects, the results of these systematic tests, the actual galaxy two-point angular correlation function for the SDSS DR7, and our massively parallel implementation that rapidly calculates correlation functions for large data sets**.

Also: "The galaxy correlation function and power spectrum" ([.PDF](http://www.uio.no/studier/emner/matnat/astro/AST4320/h12/undervisningsmateriale/galaxyclustering.pdf)).

Wikipedia's "[Correlation function (astronomy)](https://en.wikipedia.org/wiki/Correlation_function_(astronomy))".

CalTech: "[Measurements of Clustering](https://ned.ipac.caltech.edu/level5/March04/Jones/Jones5.html)" (Use [Next] or [Contents] buttons to view all the pages).

There are fixups applied to the raw data to [correct for redshift and angular diameter distance](https://astronomy.stackexchange.com/a/26667/18054).

---

For example catalogs and a description of the procedures see Shaun Cole's website: "[Mock Galaxy Redshift Catalogues](http://star-www.dur.ac.uk/%7Ecole/mocks/main.html)" where software and full details of the N-body simulations and the contruction and properties of the mock catalogues is provided for both 2dF Galaxy Redshift Survey (2dFGRS) and SDSS galaxy redshift surveys.

Halo bias relation to dark matter and N-body simulations are discussed in "[BAM: Bias Assignment Method to generate mock catalogs](https://arxiv.org/abs/1806.05870)" (15 June 2018), by A. Balaguera-Antolínez, Francisco-Shu Kitaura, Marcos Pellerejo-Ibañez, Cheng Zhao, and Tom Abel.

Also check out the (under construction, and too busy working and writing papers) website: "[The Carnegie-Irvine Galaxy Survey (CGS)](https://cgs.obs.carnegiescience.edu/CGS/Publications.html)" - with missing papers available on arXix:

"[The Carnegie-Irvine Galaxy Survey. I. Overview and Atlas of Optical Images](https://arxiv.org/abs/1111.4605)"

"[The Carnegie-Irvine Galaxy Survey. II. Isophotal Analysis](https://arxiv.org/abs/1111.4606)"

"[The Carnegie-Irvine Galaxy Survey. III. The Three-Component Structure of Nearby Elliptical Galaxies](https://arxiv.org/abs/1212.2639)"

"[The Carnegie-Irvine Galaxy Survey. IV. A Method to Determine the Average Mass Ratio of Mergers That Built Massive Elliptical Galaxies](https://arxiv.org/abs/1603.00876)"

"[The Carnegie-Irvine Galaxy Survey. V. Statistical study of bars and buckled bars](https://arxiv.org/abs/1707.03570)"

"[The Carnegie-Irvine Galaxy Survey. VI. Quantifying Spiral Structure](https://arxiv.org/abs/1806.06591)"

"[The Carnegie-Irvine Galaxy Survey. VII. Constraints on the Origin of S0 Galaxies from Their Photometric Structure](https://arxiv.org/abs/1806.06350)"

> **Comments:**
> **username_0**: I will look over these sources but it looks like you've hit the nail on the head. If I don't see any other answers soon I will accept yours.  
> **username_1**: @username_0 - I ran into a bit more somewhat related information while researching another question. Apparently [SDVision](http://irfu.cea.fr/Projets/COAST/SDvision.html) is the premier software for cosmology data visualization. It is available by request (as a *black box*) or for licensed user it enables you to "introduce personal modifications", see the link. They will likely inquire about your status as a researcher.  

---

