<p>I do have a large dataset containing nearly a decade of particulate matter measurements. At the lower precision limit of the measuring instrument, there are values given like <code>&lt; 8</code>. For statistical analysis however, a defined value is needed.</p>

<p>What is the best possible way to deal with this? </p>

<ul>
<li>Replacing it as missing values?</li>
<li>Replacing it by <code>8</code> or even <code>7</code> (<code>&lt; 8</code>)?</li>
<li>Assuming <code>0</code>?</li>
<li>Assuming the mean value between something like <code>0</code> and <code>8</code>; e. g. <code>4</code>?</li>
</ul>

<p><em>Or is there a better solution, maybe considering the natural background value?</em></p>

<p>EDIT:
I'd like to calculate the intra-day- and intra-year distribution, as well as searching for patterns by considering wind directiond and speed.</p>

CC BY-SA 3.0
2016-10-03T10:05:16.557Z
**It depends on what you want to measure/calculate.** So we need more info.
Jan Doggen
CC BY-SA 3.0
2016-10-03T10:39:55.970Z
@JanDoggen: Added some more information, thanks!
Arne
CC BY-SA 3.0
2016-10-03T13:03:03.007Z
Perhaps one standard deviation around the limit of determination would suffice, but it depends upon the range of the dataset you are using, and the accuracy / precision of whatever you are trying to analyse. If the limit of determination, say 7 or 8, has a high frequency relative to other values, then you may have a statistical problem from too uniform a lower limit, in which case, counter-inuitively,  it may be beneficial to introduce synthetic randomness into your lowest value. I sometimes do this when I have to synthesize missing rainfall data.
Gordon Stanger
CC BY-SA 3.0
2016-10-04T07:45:51.783Z
In addition to Gordon Stanger's comment: replacing `<8` values by missing values might alter the distribution but replacing `< 8` values by noise (or replacing it by other non-missing values) might yield artificial correlations. Therefore, it might be reasonable to do the noise and the missing-value approach and get a feeling on which one is actually working 'better'. If there are less than approximately 5% (not a fix threshold) I would suggest to use missing values.
daniel.heydebreck
CC BY-SA 3.0
2016-10-04T07:48:45.287Z
Maybe consider changing the title of your question. Reading the title, I assumed that the question would be focused on programming-related.
daniel.heydebreck
CC BY-SA 3.0
2016-10-04T22:28:31.763Z
If your data is dense enough, replacing by missing-value may not be a bad idea. But if the data is not sufficiently dense, I would suggest interpolating to the nearest values. Either way the outliers are bad and should be thrown away imo.
Gemechu Fanta Garuma
<measurements><air-pollution><statistics>
Determining '< x' values as numeric
Arne
# Determining '< x' values as numeric

(Asked by: username_0 on 2016-10-03)

I do have a large dataset containing nearly a decade of particulate matter measurements. At the lower precision limit of the measuring instrument, there are values given like `< 8`. For statistical analysis however, a defined value is needed.

What is the best possible way to deal with this?

* Replacing it as missing values?
* Replacing it by `8` or even `7` (`< 8`)?
* Assuming `0`?
* Assuming the mean value between something like `0` and `8`; e. g. `4`?

*Or is there a better solution, maybe considering the natural background value?*

EDIT:
I'd like to calculate the intra-day- and intra-year distribution, as well as searching for patterns by considering wind directiond and speed.

**Tags:** measurements, air-pollution, statistics

> **Comments:**
> **username_1**: **It depends on what you want to measure/calculate.** So we need more info.  
> **username_0**: @username_1: Added some more information, thanks!  
> **username_2**: Perhaps one standard deviation around the limit of determination would suffice, but it depends upon the range of the dataset you are using, and the accuracy / precision of whatever you are trying to analyse. If the limit of determination, say 7 or 8, has a high frequency relative to other values, then you may have a statistical problem from too uniform a lower limit, in which case, counter-inuitively,  it may be beneficial to introduce synthetic randomness into your lowest value. I sometimes do this when I have to synthesize missing rainfall data.  
> **username_3**: In addition to username_2 username_2's comment: replacing `<8` values by missing values might alter the distribution but replacing `< 8` values by noise (or replacing it by other non-missing values) might yield artificial correlations. Therefore, it might be reasonable to do the noise and the missing-value approach and get a feeling on which one is actually working 'better'. If there are less than approximately 5% (not a fix threshold) I would suggest to use missing values.  
> **username_3**: Maybe consider changing the title of your question. Reading the title, I assumed that the question would be focused on programming-related.  
> **username_4**: If your data is dense enough, replacing by missing-value may not be a bad idea. But if the data is not sufficiently dense, I would suggest interpolating to the nearest values. Either way the outliers are bad and should be thrown away imo.  

---

## 0 answers

