<h2>AstroPy</h2>

<p>Jake Vanderplas (Director of Open Software at the University of Washington eScience Institute) has written a Pythonic Preambulations post <a href="https://jakevdp.github.io/blog/2015/06/13/lomb-scargle-in-python/" rel="nofollow noreferrer">Fast Lomb-Scargle Periodograms in Python</a> which <em>by definition appears to be</em> a good place to start to learn how to do Lomb-Scargle Periodograms... Fast... in Python. </p>

<p>I have not read it yet in full, but I've found other posts of his to be extremely useful. </p>

<p>He starts out by explaining the implementation in <a href="https://www.astropy.org/" rel="nofollow noreferrer">AstroPy</a> which if you are going to be doing Astronomy, is probably a good thing to become familliar with.</p>

<h2>SciPy</h2>

<p>Alternatively, there is <a href="https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.lombscargle.html" rel="nofollow noreferrer"><code>scipy.signal.lombscargle</code></a> which you can try assuming you have a current version of SciPy installed but want to try something different than AstroPy or want to compare performance or results. </p>

<h2>StackOverflow</h2>

<p>I found <a href="https://stackoverflow.com/search?q=Lomb-Scargle">over 40 posts containing 'Lomb-Scargle' in Stack Overflow</a> which is a very good sign!</p>

<p>I notice that you've been active there and know how to ask well-received questions. Since your asking about speed and errors, StackOverflow is your go-to source! However, don't forget to be as clear as possible and quote all error messages and symptoms, and see if you can somehow add a short Minimum Complete and Verifiable Example or <a href="https://meta.stackoverflow.com/q/366988">MCVE</a>. That's hard when your problem results from a very large dataset I know.</p>

CC BY-SA 4.0
2019-04-16T13:52:54.097Z
I upload a test data set. See that data link please. The data is a series of photon arrival times. We need to bin the data  and  then do lomb-scargle or fft. The problem is our low computing efficiency now. We need a faster method.
questionhang
CC BY-SA 4.0
2019-04-16T13:56:56.673Z
@questionhang thanks, that's great! I recommend two things, 1) wait for someone with specific experience to add another answer here, and 2) consider asking a separate question in StackOverflow as I discuss above. It's really great that you are so responsive to comments and questions. As you know, in SO they can come very quickly! :-)
uhoh
CC BY-SA 4.0
2019-04-16T13:23:14.327Z
2019-04-16T13:29:24.877Z
2019-04-16T13:29:24.877Z
uhoh
<p>Our <a href="https://clbin.com/C7Jcz" rel="nofollow noreferrer">data set</a> has <span class="math-container">$10^4$</span> data points, but has a long baseline and many gaps.</p>

<p>If we bin the data, there would be <span class="math-container">$10^8$</span> data points (<span class="math-container">$[t,\rm {value}]$</span>), but only about 1% are non-zero values.</p>

<p>How to improve the detection efficiency? </p>

<p>Is a multi-threading way possible (especially for Lomb-Scargle)?</p>

<p>For example, if I use LombScargle in <em>astropy.stats</em>,</p>

<pre><code>freq, power = LombScargle.autopower(minimum_frequency=0.5, maximum_frequency=1.5, normalization='standard')
</code></pre>

<p>there are two problems:</p>

<ol>
<li>It is slow and easy to get <code>memoryError</code>.</li>
<li>It can't utilize the computing power as much as possible.</li>
</ol>

<p>So my question is about the efficiency. For the data set above, what is the best method?</p>

CC BY-SA 4.0
2019-04-16T08:45:04.970Z
Question might be a better fit for [Computational Science SE](https://scicomp.stackexchange.com/).
StephenG - Help Ukraine
CC BY-SA 4.0
2019-04-16T11:07:31.053Z
[Understanding the Lomb-Scargle Periodogram](https://arxiv.org/abs/1703.09824) explains that technique nicely. Would you simply like to know if it can be (or has already been) written in a function particularly conducive to multi-threading? First glance of my (non-expert eyes) suggests that would be really straightforward, but rather than multithreading, I think the calculation is easy vectorized. I think you could ask a more specific, practical question about the technique here (how to set it up with your data) and a more computational question in CompSci as mentioned above.
uhoh
CC BY-SA 4.0
2019-04-16T13:02:13.457Z
@uhoh could you suggest some method which can improve the computing efficiency?
questionhang
CC BY-SA 4.0
2019-04-16T13:08:04.490Z
okay that would definitely be a good, and separate question, asked separately, in [Computational Science SE](https://scicomp.stackexchange.com/). However, before you ask it, please find a way to explain *clearly* how you are doing it now. You can't ask how to improve something if you don't show your current method and state its speed. In the mean time, I will take a quick look at what you've added here :-)
uhoh
CC BY-SA 4.0
2019-09-28T22:49:12.093Z
@StephenG yep that makes more sense.
Inertial Ignorance
<binary-star><pulsar><algorithm>
How to improve efficiency in periodicity detection?
questionhang
# How to improve efficiency in periodicity detection?

(Asked by: username_0 on 2019-04-16)

Our [data set](https://clbin.com/C7Jcz) has $10^4$ data points, but has a long baseline and many gaps.

If we bin the data, there would be $10^8$ data points ($[t,\rm {value}]$), but only about 1% are non-zero values.

How to improve the detection efficiency?

Is a multi-threading way possible (especially for Lomb-Scargle)?

For example, if I use LombScargle in *astropy.stats*,

```
freq, power = LombScargle.autopower(minimum_frequency=0.5, maximum_frequency=1.5, normalization='standard')

```

there are two problems:

1. It is slow and easy to get `memoryError`.
2. It can't utilize the computing power as much as possible.

So my question is about the efficiency. For the data set above, what is the best method?

**Tags:** binary-star, pulsar, algorithm

> **Comments:**
> **username_1**: Question might be a better fit for [Computational Science SE](https://scicomp.stackexchange.com/).  
> **username_2**: [Understanding the Lomb-Scargle Periodogram](https://arxiv.org/abs/1703.09824) explains that technique nicely. Would you simply like to know if it can be (or has already been) written in a function particularly conducive to multi-threading? First glance of my (non-expert eyes) suggests that would be really straightforward, but rather than multithreading, I think the calculation is easy vectorized. I think you could ask a more specific, practical question about the technique here (how to set it up with your data) and a more computational question in CompSci as mentioned above.  
> **username_0**: @username_2 could you suggest some method which can improve the computing efficiency?  
> **username_2**: okay that would definitely be a good, and separate question, asked separately, in [Computational Science SE](https://scicomp.stackexchange.com/). However, before you ask it, please find a way to explain *clearly* how you are doing it now. You can't ask how to improve something if you don't show your current method and state its speed. In the mean time, I will take a quick look at what you've added here :-)  
> **username_3**: @username_1 yep that makes more sense.  

---

## 1 answers

### Answer by username_2 on 2019-04-16. Score: 1

## AstroPy

Jake Vanderplas (Director of Open Software at the University of Washington eScience Institute) has written a Pythonic Preambulations post [Fast Lomb-Scargle Periodograms in Python](https://jakevdp.github.io/blog/2015/06/13/lomb-scargle-in-python/) which *by definition appears to be* a good place to start to learn how to do Lomb-Scargle Periodograms... Fast... in Python.

I have not read it yet in full, but I've found other posts of his to be extremely useful.

He starts out by explaining the implementation in [AstroPy](https://www.astropy.org/) which if you are going to be doing Astronomy, is probably a good thing to become familliar with.

## SciPy

Alternatively, there is [`scipy.signal.lombscargle`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.lombscargle.html) which you can try assuming you have a current version of SciPy installed but want to try something different than AstroPy or want to compare performance or results.

## StackOverflow

I found [over 40 posts containing 'Lomb-Scargle' in Stack Overflow](https://stackoverflow.com/search?q=Lomb-Scargle) which is a very good sign!

I notice that you've been active there and know how to ask well-received questions. Since your asking about speed and errors, StackOverflow is your go-to source! However, don't forget to be as clear as possible and quote all error messages and symptoms, and see if you can somehow add a short Minimum Complete and Verifiable Example or [MCVE](https://meta.stackoverflow.com/q/366988). That's hard when your problem results from a very large dataset I know.

> **Comments:**
> **username_0**: I upload a test data set. See that data link please. The data is a series of photon arrival times. We need to bin the data  and  then do lomb-scargle or fft. The problem is our low computing efficiency now. We need a faster method.  
> **username_2**: @username_0 thanks, that's great! I recommend two things, 1) wait for someone with specific experience to add another answer here, and 2) consider asking a separate question in StackOverflow as I discuss above. It's really great that you are so responsive to comments and questions. As you know, in SO they can come very quickly! :-)  

---

