<p><a href="http://www.utstat.toronto.edu/~rsalakhu/papers/pami_submit.pdf" rel="nofollow">This</a> 2013 paper integrates deep learning models with structured hierarchical Bayesian models, for the net to learn novel concepts from very few training examples. It shows encouraging results.</p>

<p>Another approach is to import a general-purpose top-performing neural net, and replace the top fully connected layers with initialised ones (including an output layer that corresponds to the number of classes in your specific task). <a href="https://github.com/sermanet/OverFeat" rel="nofollow">OverFeat</a> is an example, for image classification - it was trained on ImageNet.</p>

CC BY-SA 3.0
2014-04-25T14:00:49.610Z
2014-04-25T14:00:49.610Z
Alex Dalyac
<p>Has any research been done about / are people interested in incorporating prior knowledge of good high-level features into a deep neural net?</p>

<p>I know this somewhat defeats the purpose of feature learning but consider the following applied scenario: image classification task, 20 classes,  'small' training set (10k), expert human knowledge of high-level features. </p>

<p>Ideal would be to compensate for lack of training cases by incorporating knowledge of these features into the architecture (or via other means). How?﻿</p>

<ne.neural-evol>
Neural Networks: Incorporate feature-specific knowledge
Alex Dalyac
# Neural Networks: Incorporate feature-specific knowledge

(Asked by: username_0 on 2014-04-25)

Has any research been done about / are people interested in incorporating prior knowledge of good high-level features into a deep neural net?

I know this somewhat defeats the purpose of feature learning but consider the following applied scenario: image classification task, 20 classes, 'small' training set (10k), expert human knowledge of high-level features.

Ideal would be to compensate for lack of training cases by incorporating knowledge of these features into the architecture (or via other means). How?﻿

**Tags:** ne.neural-evol


---

## 1 answers

### Answer by username_0 on 2014-04-25. Score: 2

[This](http://www.utstat.toronto.edu/~rsalakhu/papers/pami_submit.pdf) 2013 paper integrates deep learning models with structured hierarchical Bayesian models, for the net to learn novel concepts from very few training examples. It shows encouraging results.

Another approach is to import a general-purpose top-performing neural net, and replace the top fully connected layers with initialised ones (including an output layer that corresponds to the number of classes in your specific task). [OverFeat](https://github.com/sermanet/OverFeat) is an example, for image classification - it was trained on ImageNet.


---

