<p>You could try to use approaches that are similar to measure the strength (and delay) of a P3 response in ERP data. There are many ways described to measure the P3 responses (or any other response you are interested in) in the book of <a href="https://mitpress.mit.edu/books/introduction-event-related-potential-technique" rel="nofollow">Luck: An introduction to the event related potential technique</a>. These include:</p>

<ul>
<li>Maximum value (within a time frame; sort of what you are doing)</li>
<li>Mean peak value (averaged over a time frame)</li>
<li>Area of the response ($amplitude * time$)</li>
</ul>

<p>I am probably missing a few, so you might want to check out that book. These may provide more reliable results than solely looking at max-min differences.</p>

CC BY-SA 3.0
2016-08-12T10:20:56.977Z
Thanks for the advice Robin! Your previous answer was also extremely useful (and informative to someone like myself), however for the sake of this question I'll be choosing this as the best answer. Cheers
GroomedGorilla
CC BY-SA 3.0
2016-07-27T12:06:11.680Z
2016-07-27T12:06:11.680Z
Robin Kramer-ten Have
<p>In this <a href="http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6346834" rel="nofollow">conference paper</a> they propose an algorithm that roughly goes as follows (if you can't access it, just add "sci-hub.bz" after ".org" in the url and it redirects you directly to the pdf) :</p>

<ul>
<li>Divide channels into four groups: 1) frontal, 2) central, 3) temporal and 4) parietal+occipital.</li>
<li>For each group take three signals: 1) the raw signal, 2) the alpha band(8-12Hz, using a 3rd order butterworth band-pass filter) and 3) the beta band(13-35Hz).</li>
<li>For each group and each signal calculate the four metrics: 1) max amplitude, 2) standard deviation, 3) kurtosis and 4) skewness. For the frequency band signals they also take the mean power and the standard deviation of the power.</li>
</ul>

<p>They then go on and find optimal thresholds for each metric, group, and signal by using a differential evolution algorithm on training data, so unfortunately it's not immediately applicable for you. As I see it, you have at least two options here:</p>

<ol>
<li>Create 5 seconds long epochs of your data and label a lot of them. Use that data to learn optimal thresholds (unfortunately they don't give any details on how to do that, so you would have to figure that out yourself). This would obviously take a lot of effort but at the end you can be quite confident about your classification.</li>
<li>Just try some value ranges for the different metrics, you can orient yourself on the values they found in the paper (you have to read them from the plots though).  Check the resulting classifications. Pick the threshold values that give the "best" results by inspections. This is not as systematic as the first option but might fit your needs already. For example if you just want to avoid to have segments that were falsely classified as "clean" you might just go for very conservative/safe threshold values. There is a <strong>big danger</strong> here though, that you will get lost in an endless loop of "maybe if I adjust this threshold a bit more" and will never stop exploring. To help with that, you might also reduce the number of metrics, groups or signals.</li>
</ol>

CC BY-SA 3.0
2016-08-12T08:40:15.850Z
Thanks for the tips! The paper you linked is also extremely handy to have, Cheers!
GroomedGorilla
CC BY-SA 3.0
2016-07-27T17:10:52.607Z
2016-07-27T17:10:52.607Z
awakenting
<h2>Short answer</h2>

<p>This answer is not on how to find clean data. Instead, I would like to argue that you can clean your data. </p>

<h2>Long answer</h2>

<blockquote>
  <p>We collected our data in a museum environment (very noisy in many ways), with participants sitting as still as possible for 2-minute sessions at a go. For this reason, there is a high likelihood of art[i]facts being found in the data due to movement, the environment and possibly even inaccuracies from the device itself. </p>
</blockquote>

<p>This is indeed true. The Emotiv Epoc has shown to not be the best EEG headset and introduces a lot of noise (e.g. <a href="https://www.researchgate.net/profile/W_David_Hairston/publication/263286124_A_Comparison_of_Electroencephalography_Signals_Acquired_from_Conventional_and_Mobile_Systems/links/544f01db0cf2bca5ce90c2fa.pdf" rel="nofollow noreferrer">Ries et al., 2014</a>). In this study, where they used an odd-ball paradigm (a highly controlled lab experiment), the authors found that significantly more trials had to be removed, compared to medical grade EEG (BioSemi) and another wearable EEG headset (B-Alert X10). </p>

<blockquote>
  <p>From what I've discussed with the team, I've been told that removing artefacts wouldn't solve the issue </p>
</blockquote>

<p>It is true that removing muscle artifacts (they were sitting still right?) and system noise will not be very helpful. The data will be contaminated throughout the task, and removing that would likely also remove some actual data. It may be interesting to investigate at what frequency the system adds noise. You could do a measurement with the system while it's not equipped to the head, and then calculate the frequency spectrum with an FFT. This will give you an indication of how your actual data was affected. </p>

<p>You can remove ocular artifacts though. As you showed in the picture, there is only one clear peak in that five second period. That particular patterns is a clear eye blink. Another EOG artifact is an eye movement, which can be seen in the picture below as the jumps in data in the top two channels (between 3 and 3.5 seconds).Both artifacts can easily and rather safely be removed due to their distinct patterns. For this I always use an independent component analysis (ICA; <a href="http://sccn.ucsd.edu/~scott/pdf/NIPS95b.pdf" rel="nofollow noreferrer">Makeig et  al., 1996</a>). This is also explained by <a href="https://sccn.ucsd.edu/eeglab/download/eeglab_jnm03.pdf" rel="nofollow noreferrer">Delorme and Makeig (2004)</a> for EEGLab specifically (and <a href="http://www.fieldtriptoolbox.org/example/use_independent_component_analysis_ica_to_remove_eog_artifacts" rel="nofollow noreferrer">here</a> is a tutorial that explains it with FieldTrip).<br>
<a href="https://i.sstatic.net/fV2WO.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/fV2WO.png" alt="enter image description here"></a></p>

<p><strong>What would that mean for your data?</strong><br>
Whenever you got rid of the EOG artifacts, your data should be pretty clean. Then, you do not need to search for clean periods but you can select the same (or a random) period of time for each participant. However, if you really do not want to clean your data, this answer doesn't help you. </p>

CC BY-SA 3.0
2016-07-27T11:55:28.397Z
2016-07-27T11:55:28.397Z
Robin Kramer-ten Have
<p>For automation, you need to write <a href="https://sccn.ucsd.edu/wiki/Chapter_02:_Writing_EEGLAB_Scripts" rel="nofollow noreferrer">a MATLAB script that controls EEGLAB</a>. One option is to use the Automatic Artifact Removal toolbox, which you can find a link to <a href="https://sccn.ucsd.edu/wiki/EEGLAB_Plugins" rel="nofollow noreferrer">here</a>.</p>
<p>Generally, min, max, abs(max-min) criteria suffice to mark the artifactual events, followed by artifact rejection. However, line noise and environmental noise must be dealt with separately. The influence of eye movements can also be removed - either using ICA, which would be a timely process for 300 data sets (assuming you do not have have access to a MATLAB computer cluster), or computing virtual EOG channels and excluding data based on some criterion for those peaks. Compute the channel Af4-Af3 for horizontal eye movements and F7-F3 for vertical. Tweak the parameters for a single noisy data set first. If that works, batch process the files with the same criteria. I would start with Steven Luck's criteria.</p>
<p>Artefacts will go away once you start averaging across trials because of common mode rejection, but they can be influential if not dealt with.</p>

CC BY-SA 3.0
2016-07-28T22:05:26.297Z
2016-07-28T22:05:26.297Z
2020-06-17T09:55:31.633Z
noumenal
<p>I'm working on some EEG data we collected using the Emotiv Epoc headset using 14 channel sensors. </p>

<p>I need to isolate chunks of data where all 14 channels are stable (non-noisy) for a given length of time (e.g. 5 seconds). I've written a MATLAB script for this, such that I calculate the standard deviation of each channel and pass an analysis window over the data, comparing the difference between maximum and minimum values in the window to this standard deviation (i.e. where maximum - minimum is less than the standard deviation [or 3-5 standard deviations], I treat that window as stable).</p>

<p>Based on this criteria, I am having some success in finding 5 second long segments but am worried this data may not be reliable (or not as stable/artefact free as desired).</p>

<p>Are there any better ways of performing this analysis? Even better, are there any functions built into EEGLAB for this very purpose?</p>

<p><strong>Further info:</strong></p>

<p>I've selected only the relevant channels and can now look through my EEG data files.</p>

<p>I am no neuroscience expert, actually I'm rather new to this sort of analysis, so apologies if this question is overly basic or off-point.</p>

<p><strong>EDIT</strong></p>

<p>We collected our data in a museum environment (very noisy in many ways), with participants sitting as still as possible for 2-minute sessions at a go. For this reason, there is a high likelihood of artefacts being found in the data due to movement, the environment and possibly even inaccuracies from the device itself (since we went through a rather large number of participants, cleaning and charging the headsets in between trials). </p>

<p>From what I've discussed with the team, I've been told that removing artefacts wouldn't solve the issue (though admittedly I'm no expert as to why this would be). </p>

<p>For clarity, the data we've collected looks similar to the plot below. </p>

<p><a href="https://i.sstatic.net/J0Nbj.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/J0Nbj.png" alt="Noisy Data"></a></p>

<p>And we're looking for a minimum of 5 seconds of stable data, closer to the plot below.</p>

<p><a href="https://i.sstatic.net/F2BHZ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/F2BHZ.png" alt="Stable Data"></a></p>

<p>Also, I know this data selection can be done manually using pop_eegplot, however since we have close to 300 data files I'm looking to automate this (and hopefully get more reliable selections).</p>

CC BY-SA 3.0
2016-07-21T20:25:19.677Z
Hi, welcome at CogSci. Is there a particular reason that you only want to select artifact free data periods? Or is it also okay to correct for artifacts, that is to clean your data? I bet there is a function that performs an independent component analysis (ICA) which is capable of identifying artifactual components such as blinks, muscle noise and heartbeats, and subsequently removes them. Perhaps you can tell a little more about your experiment and what you want to calculate.
Robin Kramer-ten Have
CC BY-SA 3.0
2016-07-22T11:40:38.767Z
Hi @RobinKramer, thanks for the welcome! I've added a bit more detail to my question in case this helps. 
From what I'm told, since our data collection was particularly noisy for a number of reasons, it seems that we're specifically looking to pick out stable data rather than correct it.
GroomedGorilla
CC BY-SA 3.0
2016-07-26T11:32:07.447Z
Another question. What are you hoping to find in these 5 seconds of data. Did you present them with particular stimuli to which you want to see the response or something, or do you want to calculate the power of the different brain waves/frequency bands?
Robin Kramer-ten Have
CC BY-SA 3.0
2016-07-26T13:05:08.647Z
Spot on again. I'm looking for the frequency content as responses to specific (rather lengthy) stimuli. Ultimately, I will be calculating the power of the different bands per sensor to determine which areas were activated more than others per stimulus.
GroomedGorilla
<eeg>
Isolating stable EEG data across all channels in EEGLAB
GroomedGorilla
# Isolating stable EEG data across all channels in EEGLAB

(Asked by: username_0 on 2016-07-21)

I'm working on some EEG data we collected using the Emotiv Epoc headset using 14 channel sensors.

I need to isolate chunks of data where all 14 channels are stable (non-noisy) for a given length of time (e.g. 5 seconds). I've written a MATLAB script for this, such that I calculate the standard deviation of each channel and pass an analysis window over the data, comparing the difference between maximum and minimum values in the window to this standard deviation (i.e. where maximum - minimum is less than the standard deviation [or 3-5 standard deviations], I treat that window as stable).

Based on this criteria, I am having some success in finding 5 second long segments but am worried this data may not be reliable (or not as stable/artefact free as desired).

Are there any better ways of performing this analysis? Even better, are there any functions built into EEGLAB for this very purpose?

**Further info:**

I've selected only the relevant channels and can now look through my EEG data files.

I am no neuroscience expert, actually I'm rather new to this sort of analysis, so apologies if this question is overly basic or off-point.

**EDIT**

We collected our data in a museum environment (very noisy in many ways), with participants sitting as still as possible for 2-minute sessions at a go. For this reason, there is a high likelihood of artefacts being found in the data due to movement, the environment and possibly even inaccuracies from the device itself (since we went through a rather large number of participants, cleaning and charging the headsets in between trials).

From what I've discussed with the team, I've been told that removing artefacts wouldn't solve the issue (though admittedly I'm no expert as to why this would be).

For clarity, the data we've collected looks similar to the plot below.

[![Noisy Data](https://i.sstatic.net/J0Nbj.png)](https://i.sstatic.net/J0Nbj.png)

And we're looking for a minimum of 5 seconds of stable data, closer to the plot below.

[![Stable Data](https://i.sstatic.net/F2BHZ.png)](https://i.sstatic.net/F2BHZ.png)

Also, I know this data selection can be done manually using pop\_eegplot, however since we have close to 300 data files I'm looking to automate this (and hopefully get more reliable selections).

**Tags:** eeg

> **Comments:**
> **username_1**: Hi, welcome at CogSci. Is there a particular reason that you only want to select artifact free data periods? Or is it also okay to correct for artifacts, that is to clean your data? I bet there is a function that performs an independent component analysis (ICA) which is capable of identifying artifactual components such as blinks, muscle noise and heartbeats, and subsequently removes them. Perhaps you can tell a little more about your experiment and what you want to calculate.  
> **username_0**: Hi @RobinKramer, thanks for the welcome! I've added a bit more detail to my question in case this helps.  From what I'm told, since our data collection was particularly noisy for a number of reasons, it seems that we're specifically looking to pick out stable data rather than correct it.  
> **username_1**: Another question. What are you hoping to find in these 5 seconds of data. Did you present them with particular stimuli to which you want to see the response or something, or do you want to calculate the power of the different brain waves/frequency bands?  
> **username_0**: Spot on again. I'm looking for the frequency content as responses to specific (rather lengthy) stimuli. Ultimately, I will be calculating the power of the different bands per sensor to determine which areas were activated more than others per stimulus.  

---

## 4 answers

### Answer by username_1 on 2016-07-27. Score: 0 | Accepted answer

You could try to use approaches that are similar to measure the strength (and delay) of a P3 response in ERP data. There are many ways described to measure the P3 responses (or any other response you are interested in) in the book of [Luck: An introduction to the event related potential technique](https://mitpress.mit.edu/books/introduction-event-related-potential-technique). These include:

* Maximum value (within a time frame; sort of what you are doing)
* Mean peak value (averaged over a time frame)
* Area of the response ($amplitude \* time$)

I am probably missing a few, so you might want to check out that book. These may provide more reliable results than solely looking at max-min differences.

> **Comments:**
> **username_0**: Thanks for the advice Robin! Your previous answer was also extremely useful (and informative to someone like myself), however for the sake of this question I'll be choosing this as the best answer. Cheers  

---

### Answer by username_2 on 2016-07-27. Score: 1

In this [conference paper](http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6346834) they propose an algorithm that roughly goes as follows (if you can't access it, just add "sci-hub.bz" after ".org" in the url and it redirects you directly to the pdf) :

* Divide channels into four groups: 1) frontal, 2) central, 3) temporal and 4) parietal+occipital.
* For each group take three signals: 1) the raw signal, 2) the alpha band(8-12Hz, using a 3rd order butterworth band-pass filter) and 3) the beta band(13-35Hz).
* For each group and each signal calculate the four metrics: 1) max amplitude, 2) standard deviation, 3) kurtosis and 4) skewness. For the frequency band signals they also take the mean power and the standard deviation of the power.

They then go on and find optimal thresholds for each metric, group, and signal by using a differential evolution algorithm on training data, so unfortunately it's not immediately applicable for you. As I see it, you have at least two options here:

1. Create 5 seconds long epochs of your data and label a lot of them. Use that data to learn optimal thresholds (unfortunately they don't give any details on how to do that, so you would have to figure that out yourself). This would obviously take a lot of effort but at the end you can be quite confident about your classification.
2. Just try some value ranges for the different metrics, you can orient yourself on the values they found in the paper (you have to read them from the plots though). Check the resulting classifications. Pick the threshold values that give the "best" results by inspections. This is not as systematic as the first option but might fit your needs already. For example if you just want to avoid to have segments that were falsely classified as "clean" you might just go for very conservative/safe threshold values. There is a **big danger** here though, that you will get lost in an endless loop of "maybe if I adjust this threshold a bit more" and will never stop exploring. To help with that, you might also reduce the number of metrics, groups or signals.

> **Comments:**
> **username_0**: Thanks for the tips! The paper you linked is also extremely handy to have, Cheers!  

---

### Answer by username_1 on 2016-07-27. Score: 0

## Short answer

This answer is not on how to find clean data. Instead, I would like to argue that you can clean your data.

## Long answer

> We collected our data in a museum environment (very noisy in many ways), with participants sitting as still as possible for 2-minute sessions at a go. For this reason, there is a high likelihood of art[i]facts being found in the data due to movement, the environment and possibly even inaccuracies from the device itself.

This is indeed true. The Emotiv Epoc has shown to not be the best EEG headset and introduces a lot of noise (e.g. [Ries et al., 2014](https://www.researchgate.net/profile/W_David_Hairston/publication/263286124_A_Comparison_of_Electroencephalography_Signals_Acquired_from_Conventional_and_Mobile_Systems/links/544f01db0cf2bca5ce90c2fa.pdf)). In this study, where they used an odd-ball paradigm (a highly controlled lab experiment), the authors found that significantly more trials had to be removed, compared to medical grade EEG (BioSemi) and another wearable EEG headset (B-Alert X10).

> From what I've discussed with the team, I've been told that removing artefacts wouldn't solve the issue

It is true that removing muscle artifacts (they were sitting still right?) and system noise will not be very helpful. The data will be contaminated throughout the task, and removing that would likely also remove some actual data. It may be interesting to investigate at what frequency the system adds noise. You could do a measurement with the system while it's not equipped to the head, and then calculate the frequency spectrum with an FFT. This will give you an indication of how your actual data was affected.

You can remove ocular artifacts though. As you showed in the picture, there is only one clear peak in that five second period. That particular patterns is a clear eye blink. Another EOG artifact is an eye movement, which can be seen in the picture below as the jumps in data in the top two channels (between 3 and 3.5 seconds).Both artifacts can easily and rather safely be removed due to their distinct patterns. For this I always use an independent component analysis (ICA; [Makeig et al., 1996](http://sccn.ucsd.edu/~scott/pdf/NIPS95b.pdf)). This is also explained by [Delorme and Makeig (2004)](https://sccn.ucsd.edu/eeglab/download/eeglab_jnm03.pdf) for EEGLab specifically (and [here](http://www.fieldtriptoolbox.org/example/use_independent_component_analysis_ica_to_remove_eog_artifacts) is a tutorial that explains it with FieldTrip).  
[![enter image description here](https://i.sstatic.net/fV2WO.png)](https://i.sstatic.net/fV2WO.png)

**What would that mean for your data?**  
Whenever you got rid of the EOG artifacts, your data should be pretty clean. Then, you do not need to search for clean periods but you can select the same (or a random) period of time for each participant. However, if you really do not want to clean your data, this answer doesn't help you.


---

### Answer by username_3 on 2016-07-28. Score: 0

For automation, you need to write [a MATLAB script that controls EEGLAB](https://sccn.ucsd.edu/wiki/Chapter_02:_Writing_EEGLAB_Scripts). One option is to use the Automatic Artifact Removal toolbox, which you can find a link to [here](https://sccn.ucsd.edu/wiki/EEGLAB_Plugins).

Generally, min, max, abs(max-min) criteria suffice to mark the artifactual events, followed by artifact rejection. However, line noise and environmental noise must be dealt with separately. The influence of eye movements can also be removed - either using ICA, which would be a timely process for 300 data sets (assuming you do not have have access to a MATLAB computer cluster), or computing virtual EOG channels and excluding data based on some criterion for those peaks. Compute the channel Af4-Af3 for horizontal eye movements and F7-F3 for vertical. Tweak the parameters for a single noisy data set first. If that works, batch process the files with the same criteria. I would start with Steven Luck's criteria.

Artefacts will go away once you start averaging across trials because of common mode rejection, but they can be influential if not dealt with.


---

