<p>40% missing data is huge. Missing data analysis is complicated on the underlying distribution. If the data set is periodic then missing data periodicity is needed. Non-periodic data can be solved using regression, I think <code>SciPy</code> has an automated method for this, but the volume of missing data makes this approach complex. If you can bring the missing data down to 20% by compartmenatlising the data set that would be the starting point.</p>

CC BY-SA 4.0
2020-11-14T09:10:12.963Z
2020-11-14T09:10:12.963Z
M__
<p>This looks like a TMT proteomics experiment. What tools/pipeline did you use to arrive at your quantitative values?</p>
<p>There are a variety of tools that are useful for analyzing this type of data and supply appropriate imputation methods. If you are comfortable programming in R, one I would suggest is the MSstatsTMT R package: <a href="http://msstats.org/msstatstmt/" rel="nofollow noreferrer">http://msstats.org/msstatstmt/</a></p>
<p>That being said, any ways you can improve the upstream analysis to decrease the amount of missing values will be extremely beneficial.</p>

CC BY-SA 4.0
2020-12-15T00:47:48.457Z
2020-12-15T00:47:48.457Z
Will Fondrie
<p>I am currently working on a dataset that contains 50 samples (10 samples * 5 blocks). The features of the date set are:</p>
<ol>
<li><p>The data is perfectly balanced between blocks, with equal treatment representation in each block. Each block contains 2 control samples (CTL) that are pools of all other treatment samples. These are technical replicates that are identical across multiplexes.</p>
</li>
<li><p>There is a lot of missing data (~40%),</p>
</li>
<li><p>most of the missing data is missing by block, meaning that the protein is present in some blocks, but not others.</p>
</li>
<li><p>When looking at the expression of these missing proteins in the other blocks, they tend to be around the median value for the experiment, which suggests to me that these values are missing completely at random.</p>
</li>
</ol>
<p>I could proceed using complete cases, but then I'm missing out on a large portion of data. I am considering using an imputation method, but I'm concerned that (due to the nature of the missing data) that I will just be introducing a block effect into the data.</p>
<p>Any input on potential methods for imputation would be greatly appreciated.</p>
<p>Experimental design:</p>
<pre><code>SampleNumber    Multiplex   Treatment
1   A   CTL
2   A   CTL
3   A   TMT1
4   A   TMT1
5   A   TMT2
6   A   TMT2
7   A   TMT3
8   A   TMT3
9   A   TMT4
10  A   TMT4
11  B   CTL
12  B   CTL
13  B   TMT1
14  B   TMT1
15  B   TMT2
16  B   TMT2
17  B   TMT3
18  B   TMT3
19  B   TMT4
20  B   TMT4
21  C   CTL
22  C   CTL
23  C   TMT1
24  C   TMT1
25  C   TMT2
26  C   TMT2
27  C   TMT3
28  C   TMT3
29  C   TMT4
30  C   TMT4
31  D   CTL
32  D   CTL
33  D   TMT1
34  D   TMT1
35  D   TMT2
36  D   TMT2
37  D   TMT3
38  D   TMT3
39  D   TMT4
40  D   TMT4
41  E   CTL
42  E   CTL
43  E   TMT1
44  E   TMT1
45  E   TMT2
46  E   TMT2
47  E   TMT3
48  E   TMT3
49  E   TMT4
50  E   TMT4
</code></pre>
<p>The data is in a matrix format, columns = samples and rows = individual proteins, similar to gene expression.</p>
<p>The controls are technical replicates, but they are pools of all other 40 samples. This is for TMT proteomics, where the pools are used by the MS to identify peptides across the multiplexes, not necessarily for multiplex correction. I've included multiplex (as well as treatment) in the linear model to identify changes associated with the treatment, while removing the effect of the multiplex. I have confirmed (with PCA and PVCA) that the multiplex effect is negligible after including it in the lm. My problem at the moment is the missing data.</p>
<p>I calculated the missing percentage from the data matrix by taking the number of missing values divided by the total number of potential values. he data is in a matrix format, columns = samples and rows = individual proteins, similar to gene expression. To calculate the missing values, I divided the total number of NA's by the total number of values in the matrix (number of samples * number of proteins). Missing data is common in proteomics, but normally this can be attributed to something, most often being close to the limit of detection. But that is not the case here, if a value is missing, it is missing in the whole multiplex, including the pool, where we know it should be found.</p>
<p>The missing values in this case are random, and this is definitely due to the MS. It will select the first ~65k peptides that it identifies to trigger an additional MS event, so if it doesn't see that first peptide, it will be missing from the entire multiplex. There are methods for imputation of data that is missing at random, but I'm concerned that it will just be imputing by multiplex.</p>

<r><imputation><proteomics>
Block wise protein imputation
# Block wise protein imputation

(Asked by: username_0 on 2018-09-28)

I am currently working on a dataset that contains 50 samples (10 samples * 5 blocks). The features of the date set are:

The data is perfectly balanced between blocks, with equal treatment representation in each block. Each block contains 2 control samples (CTL) that are pools of all other treatment samples. These are technical replicates that are identical across multiplexes.

There is a lot of missing data (~40%),

most of the missing data is missing by block, meaning that the protein is present in some blocks, but not others.

When looking at the expression of these missing proteins in the other blocks, they tend to be around the median value for the experiment, which suggests to me that these values are missing completely at random.


I could proceed using complete cases, but then I'm missing out on a large portion of data. I am considering using an imputation method, but I'm concerned that (due to the nature of the missing data) that I will just be introducing a block effect into the data.
Any input on potential methods for imputation would be greatly appreciated.
Experimental design:
SampleNumber    Multiplex   Treatment
1   A   CTL
2   A   CTL
3   A   TMT1
4   A   TMT1
5   A   TMT2
6   A   TMT2
7   A   TMT3
8   A   TMT3
9   A   TMT4
10  A   TMT4
11  B   CTL
12  B   CTL
13  B   TMT1
14  B   TMT1
15  B   TMT2
16  B   TMT2
17  B   TMT3
18  B   TMT3
19  B   TMT4
20  B   TMT4
21  C   CTL
22  C   CTL
23  C   TMT1
24  C   TMT1
25  C   TMT2
26  C   TMT2
27  C   TMT3
28  C   TMT3
29  C   TMT4
30  C   TMT4
31  D   CTL
32  D   CTL
33  D   TMT1
34  D   TMT1
35  D   TMT2
36  D   TMT2
37  D   TMT3
38  D   TMT3
39  D   TMT4
40  D   TMT4
41  E   CTL
42  E   CTL
43  E   TMT1
44  E   TMT1
45  E   TMT2
46  E   TMT2
47  E   TMT3
48  E   TMT3
49  E   TMT4
50  E   TMT4

The data is in a matrix format, columns = samples and rows = individual proteins, similar to gene expression.
The controls are technical replicates, but they are pools of all other 40 samples. This is for TMT proteomics, where the pools are used by the MS to identify peptides across the multiplexes, not necessarily for multiplex correction. I've included multiplex (as well as treatment) in the linear model to identify changes associated with the treatment, while removing the effect of the multiplex. I have confirmed (with PCA and PVCA) that the multiplex effect is negligible after including it in the lm. My problem at the moment is the missing data.
I calculated the missing percentage from the data matrix by taking the number of missing values divided by the total number of potential values. he data is in a matrix format, columns = samples and rows = individual proteins, similar to gene expression. To calculate the missing values, I divided the total number of NA's by the total number of values in the matrix (number of samples * number of proteins). Missing data is common in proteomics, but normally this can be attributed to something, most often being close to the limit of detection. But that is not the case here, if a value is missing, it is missing in the whole multiplex, including the pool, where we know it should be found.
The missing values in this case are random, and this is definitely due to the MS. It will select the first ~65k peptides that it identifies to trigger an additional MS event, so if it doesn't see that first peptide, it will be missing from the entire multiplex. There are methods for imputation of data that is missing at random, but I'm concerned that it will just be imputing by multiplex.

**Tags:** r, imputation, proteomics

---

## 2 answers

### Answer by username_1 on 2020-11-14. Score: 1

40% missing data is huge. Missing data analysis is complicated on the underlying distribution. If the data set is periodic then missing data periodicity is needed. Non-periodic data can be solved using regression, I think `SciPy` has an automated method for this, but the volume of missing data makes this approach complex. If you can bring the missing data down to 20% by compartmenatlising the data set that would be the starting point.


---

### Answer by username_2 on 2020-12-15. Score: 1

This looks like a TMT proteomics experiment. What tools/pipeline did you use to arrive at your quantitative values?

There are a variety of tools that are useful for analyzing this type of data and supply appropriate imputation methods. If you are comfortable programming in R, one I would suggest is the MSstatsTMT R package: <http://msstats.org/msstatstmt/>

That being said, any ways you can improve the upstream analysis to decrease the amount of missing values will be extremely beneficial.


---

