<p>Adding to Devon's correct answer: Depending on the amount of images you got, there is also the possibility to avoid the feature engineering by building a two level architecture:</p>

<ul>
<li>The first level identifies the regions of interest in your image and scales these regions to the same size.</li>
<li>In the second level, the recognized areas are fed to a (multilayer) neural network/deep neural network to be classified.</li>
</ul>

<p>By using this approach, the number of needed datapoints is lower compared to a plain CNN method. </p>

<p>The drawback is that identifying the regions of interest itself might be a difficult task as you have either to label these regions manually to apply a machine learning approach at this first level, or you have to "engineer" criteria which make the extraction possible.</p>

<p>I have no idea how these images look like, but maybe you are able to extract the regions using <a href="https://opencv.org/" rel="nofollow noreferrer">OpenCV</a>.</p>

CC BY-SA 4.0
2018-07-20T12:13:25.490Z
Very interesting.  I will check this out also.  I have been working with some clustering to isolate some examples of objects of interest.   I haven't perfected this process yet as the fluorescence data is somewhat noisy.  

The same question for you as Devon, is it possible to send the image in as a 3d stack array?  Or will the image necessarily be sent in on layer of the stack at a time?  My interpretation thus far is the latter
thietpas
CC BY-SA 4.0
2018-07-20T12:21:20.527Z
At least to OpenCV and a [CNN](https://en.wikipedia.org/wiki/Convolutional_neural_network) you can present the images one at a time in a 3D format, i.e.: _width x height x color_. That means you total dataset has to have the format _number_of_images x width x height x color_
phngs
CC BY-SA 4.0
2018-07-20T13:45:39.273Z
sounds good thank you the assistance and advice!
thietpas
CC BY-SA 4.0
2018-07-20T09:42:54.883Z
2018-07-20T09:42:54.883Z
phngs
<p>Whether to quantify things at all with depend on the technique you want to use. An end-to-end CNN (convolutional neural network) for example would just be fed the 3D image for training and predictions. However the major downside to CNNs is that you need a LOT of data to train them, though perhaps you can get lucky and find an existing CNN that you can use and just retrain the last couple layers.</p>

<p>Other than that, you're going to need to come up with as many (hopefully useful) metrics as you can. There are then a lot of techniques one could try, ranging from simple things like regression to random forests to SVMs, among many others. In general I think simple regression and random forests are the easiest to get started with.</p>

CC BY-SA 4.0
2018-07-20T12:10:00.390Z
Excellent.  Thank you!  Now If i understand correctly, one feeds the image in 1 dimension essentially?  That is to say the total number of elements reshaped into one dimension?   From what I've seen the layers are all 1 dimensional unless the images have color layers
thietpas
CC BY-SA 4.0
2018-07-20T13:13:43.857Z
Typically yes, though in reality it's a contiguous stretch of memory already, so you're just ignoring the dimensional aspects when running the CNN.
Devon Ryan
CC BY-SA 4.0
2018-07-19T21:27:20.460Z
2018-07-19T21:27:20.460Z
Devon Ryan
<blockquote>
  <p>What supervised machine learning techniques will evaluate the raw
  image data itself - either for training or testing?</p>
</blockquote>

<ol>
<li><p>You can use convolutional neural networks (CNN). They will directly use the raw image data itself.</p></li>
<li><p>After applying CNNs, you can also use recurrent neural layers (RNN), to achieve better results (e.g. gated recurrent layers).</p></li>
</ol>

<p>Hope this helps. As a reference you have <a href="http://ncbi.nlm.nih.gov/pubmed/?term=3D%20CNN%20fluorescence" rel="nofollow noreferrer">this article</a>: </p>

<blockquote>
  <p>Gudla PR1,2, Nakayama K2,3, Pegoraro G1,2, Misteli T2. Article:
  SpotLearn: Convolutional Neural Network for Detection of Fluorescence
  In Situ Hybridization (FISH) Signals in High-Throughput Imaging
  Approaches. Send to Cold Spring Harb Symp Quant Biol. 2017;82:57-70.
  doi: 10.1101/sqb.2017.82.033761. Epub 2017 Nov 28.</p>
</blockquote>

CC BY-SA 4.0
2019-01-17T13:59:40.827Z
Please include some links to resources on CNN and RNN
Bioathlete
CC BY-SA 4.0
2019-01-17T14:39:31.327Z
Gudla PR1,2, Nakayama K2,3, Pegoraro G1,2, Misteli T2.

SpotLearn: Convolutional Neural Network for Detection of Fluorescence In Situ Hybridization (FISH) Signals in High-Throughput Imaging Approaches.

Send to
Cold Spring Harb Symp Quant Biol. 2017;82:57-70. doi: 10.1101/sqb.2017.82.033761. Epub 2017 Nov 28.

Abstract and full text link can be found here:

https://www.ncbi.nlm.nih.gov/pubmed/?term=3D+CNN+fluorescence
Wolfgang123
CC BY-SA 4.0
2019-01-17T15:02:16.080Z
Please edit your answer to include those links
Bioathlete
CC BY-SA 4.0
2019-01-17T15:23:53.750Z
Authors: Gudla PR1,2, Nakayama K2,3, Pegoraro G1,2, Misteli T2. 



Article: SpotLearn: Convolutional Neural Network for Detection of Fluorescence In Situ Hybridization (FISH) Signals in High-Throughput Imaging Approaches. Send to Cold Spring Harb Symp Quant Biol. 2017;82:57-70. 

doi: 10.1101/sqb.2017.82.033761. Epub 2017 Nov 28. 


you can find this article in https://www.ncbi.nlm.nih.gov/pubmed/

A detailled link is here:

Abstract and full text link can be found here: ncbi.nlm.nih.gov/pubmed/?term=3D+CNN+fluorescence
Wolfgang123
CC BY-SA 4.0
2019-01-17T11:48:16.793Z
2019-01-20T15:33:03.583Z
2019-01-20T15:33:03.583Z
Wolfgang123
<p>Some general questions...</p>

<p>So it appears with most classification learners one must come up with a series of quantifiable variables to associate with each observation.  This might include mean and std deviation intensity values, texture, and zernike polynomials.  Does anyone have any other suggestions as to how one might quantify 3-dimensional fluorescence pixel data features?  </p>

<p>Once appropriate quantification's have been acquired for a given class one can then train a classifier.  My next question is - does one then have to segment test images and create a matrix of the same feature variables from that segmented image to use the classifier? Or can one use the raw image itself?</p>

<p>What supervised machine learning techniques will evaluate the raw image data itself - either for training or testing? Can manually annotated data sets or clustered data sets provide a direct model for training data?  That is to say, are there supervised machine learning algorithms which accept pure pixel data as models for training?</p>

<p>Are there reliable pre-trained classifiers that are already implemented for these type of fluorescence data sets?  </p>

<p>Any resources or advice is greatly appreciated.</p>

<p>Thanks in advance for any input!</p>

<3d-structure>
biological fluorescence supervised machine learing
# biological fluorescence supervised machine learing

(Asked by: username_0 on 2018-07-19)

Some general questions...

So it appears with most classification learners one must come up with a series of quantifiable variables to associate with each observation. This might include mean and std deviation intensity values, texture, and zernike polynomials. Does anyone have any other suggestions as to how one might quantify 3-dimensional fluorescence pixel data features?

Once appropriate quantification's have been acquired for a given class one can then train a classifier. My next question is - does one then have to segment test images and create a matrix of the same feature variables from that segmented image to use the classifier? Or can one use the raw image itself?

What supervised machine learning techniques will evaluate the raw image data itself - either for training or testing? Can manually annotated data sets or clustered data sets provide a direct model for training data? That is to say, are there supervised machine learning algorithms which accept pure pixel data as models for training?

Are there reliable pre-trained classifiers that are already implemented for these type of fluorescence data sets?

Any resources or advice is greatly appreciated.

Thanks in advance for any input!

**Tags:** 3d-structure

---

## 3 answers

### Answer by username_1 on 2018-07-20. Score: 2

Adding to Devon's correct answer: Depending on the amount of images you got, there is also the possibility to avoid the feature engineering by building a two level architecture:

* The first level identifies the regions of interest in your image and scales these regions to the same size.
* In the second level, the recognized areas are fed to a (multilayer) neural network/deep neural network to be classified.

By using this approach, the number of needed datapoints is lower compared to a plain CNN method.

The drawback is that identifying the regions of interest itself might be a difficult task as you have either to label these regions manually to apply a machine learning approach at this first level, or you have to "engineer" criteria which make the extraction possible.

I have no idea how these images look like, but maybe you are able to extract the regions using [OpenCV](https://opencv.org/).

> **Comments:**
> **username_0**: Very interesting.  I will check this out also.  I have been working with some clustering to isolate some examples of objects of interest.   I haven't perfected this process yet as the fluorescence data is somewhat noisy.    The same question for you as Devon, is it possible to send the image in as a 3d stack array?  Or will the image necessarily be sent in on layer of the stack at a time?  My interpretation thus far is the latter  
> **username_1**: At least to OpenCV and a [CNN](https://en.wikipedia.org/wiki/Convolutional_neural_network) you can present the images one at a time in a 3D format, i.e.: _width x height x color_. That means you total dataset has to have the format _number_of_images x width x height x color_  
> **username_0**: sounds good thank you the assistance and advice!  

---

### Answer by username_2 on 2018-07-19. Score: 2

Whether to quantify things at all with depend on the technique you want to use. An end-to-end CNN (convolutional neural network) for example would just be fed the 3D image for training and predictions. However the major downside to CNNs is that you need a LOT of data to train them, though perhaps you can get lucky and find an existing CNN that you can use and just retrain the last couple layers.

Other than that, you're going to need to come up with as many (hopefully useful) metrics as you can. There are then a lot of techniques one could try, ranging from simple things like regression to random forests to SVMs, among many others. In general I think simple regression and random forests are the easiest to get started with.

> **Comments:**
> **username_0**: Excellent.  Thank you!  Now If i understand correctly, one feeds the image in 1 dimension essentially?  That is to say the total number of elements reshaped into one dimension?   From what I've seen the layers are all 1 dimensional unless the images have color layers  
> **username_2**: Typically yes, though in reality it's a contiguous stretch of memory already, so you're just ignoring the dimensional aspects when running the CNN.  

---

### Answer by username_3 on 2019-01-17. Score: 0

> What supervised machine learning techniques will evaluate the raw
> image data itself - either for training or testing?

1. You can use convolutional neural networks (CNN). They will directly use the raw image data itself.
2. After applying CNNs, you can also use recurrent neural layers (RNN), to achieve better results (e.g. gated recurrent layers).

Hope this helps. As a reference you have [this article](http://ncbi.nlm.nih.gov/pubmed/?term=3D%20CNN%20fluorescence):

> Gudla PR1,2, Nakayama K2,3, Pegoraro G1,2, Misteli T2. Article:
> SpotLearn: Convolutional Neural Network for Detection of Fluorescence
> In Situ Hybridization (FISH) Signals in High-Throughput Imaging
> Approaches. Send to Cold Spring Harb Symp Quant Biol. 2017;82:57-70.
> doi: 10.1101/sqb.2017.82.033761. Epub 2017 Nov 28.

> **Comments:**
> **username_4**: Please include some links to resources on CNN and RNN  
> **username_3**: Gudla PR1,2, Nakayama K2,3, Pegoraro G1,2, Misteli T2.  SpotLearn: Convolutional Neural Network for Detection of Fluorescence In Situ Hybridization (FISH) Signals in High-Throughput Imaging Approaches.  Send to Cold Spring Harb Symp Quant Biol. 2017;82:57-70. doi: 10.1101/sqb.2017.82.033761. Epub 2017 Nov 28.  Abstract and full text link can be found here:  https://www.ncbi.nlm.nih.gov/pubmed/?term=3D+CNN+fluorescence  
> **username_4**: Please edit your answer to include those links  
> **username_3**: Authors: Gudla PR1,2, Nakayama K2,3, Pegoraro G1,2, Misteli T2.     Article: SpotLearn: Convolutional Neural Network for Detection of Fluorescence In Situ Hybridization (FISH) Signals in High-Throughput Imaging Approaches. Send to Cold Spring Harb Symp Quant Biol. 2017;82:57-70.   doi: 10.1101/sqb.2017.82.033761. Epub 2017 Nov 28.    you can find this article in https://www.ncbi.nlm.nih.gov/pubmed/  A detailled link is here:  Abstract and full text link can be found here: ncbi.nlm.nih.gov/pubmed/?term=3D+CNN+fluorescence  

---

