<p>In the classic PAC learning (i.e., classification) model, rare instances are not a problem. This is because the learner's test points are assumed to come from the same distribution as the training data. Thus, if a region of space is so sparse as to be poorly represented in the training sample, its probability of appearing during the test phase is low.</p>

<p>You'll need a different learning model, which explicitly looks at type-I and type-II errors, or perhaps some combined precision-recall score. Here again, I don't think there are any results indicating that a specific class of algorithms is particularly poorly suited for this task, but I could be wrong.</p>

<p>The closest I can think of is sensitivity to outliers --- AdaBoost is known to have this property, for example.</p>

CC BY-SA 4.0
2018-08-07T17:59:51.900Z
2018-08-07T17:59:51.900Z
Aryeh
<p>Is there any result showing that models (say SVM, Neural-Net, kNN, etc) will have difficulty in learning "rare" instances/tail phenomena? </p>

CC BY-SA 4.0
2018-08-08T08:35:13.173Z
I think you accepted an answer too quickly -- it's a good answer, but there could be more possibilities out there as well.
usul
CC BY-SA 4.0
2018-08-08T14:13:54.173Z
@usul thanks for the comment. Do you have any further suggestion? Would be happy to hear additional thoughts.
Daniel
<machine-learning><lg.learning><pac-learning>
Difficulty of "learning" rare instances
Daniel
# Difficulty of "learning" rare instances

(Asked by: username_0 on 2018-08-07)

Is there any result showing that models (say SVM, Neural-Net, kNN, etc) will have difficulty in learning "rare" instances/tail phenomena?

**Tags:** machine-learning, lg.learning, pac-learning

> **Comments:**
> **username_1**: I think you accepted an answer too quickly -- it's a good answer, but there could be more possibilities out there as well.  
> **username_0**: @username_1 thanks for the comment. Do you have any further suggestion? Would be happy to hear additional thoughts.  

---

## 1 answers

### Answer by username_2 on 2018-08-07. Score: 10 | Accepted answer

In the classic PAC learning (i.e., classification) model, rare instances are not a problem. This is because the learner's test points are assumed to come from the same distribution as the training data. Thus, if a region of space is so sparse as to be poorly represented in the training sample, its probability of appearing during the test phase is low.

You'll need a different learning model, which explicitly looks at type-I and type-II errors, or perhaps some combined precision-recall score. Here again, I don't think there are any results indicating that a specific class of algorithms is particularly poorly suited for this task, but I could be wrong.

The closest I can think of is sensitivity to outliers --- AdaBoost is known to have this property, for example.


---

