<p>Answering your edited question: any learning algorithm (more specifically: supervised classification) algorithm <em>always</em> (either implicitly or explicitly) works with a well-defined class of <em>hypotheses</em> or mappings from examples to labels. In the case of the perceptron in my previous answer, it works with linear separators (and hence will not consider hypotheses involving square roots). A standard no-free-lunch theorem shows that an algorithm whose hypothesis space is too rich will no be able to learn (it will overfit).</p>

CC BY-SA 3.0
2017-01-18T12:11:45.563Z
2017-01-18T12:11:45.563Z
Aryeh
<p>A perceptron
<a href="https://en.wikipedia.org/wiki/Perceptron" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Perceptron</a>
or SVM
<a href="https://en.wikipedia.org/wiki/Support_vector_machine" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Support_vector_machine</a>
learning algorithm would find a separating hyperplane -- which in your case, would be something like $w=(0,1,0,0,1)$ with bias $b=-1$.</p>

CC BY-SA 3.0
2017-01-18T08:08:42.223Z
2017-01-18T08:08:42.223Z
Aryeh
<p>I am truly fascinated by algorithms learning on their own with a little help from humans and as a newbie in this field (with programming experience mainly in C/C++), seek your help to obtain the big picture before delving into specifics. I am not a researcher, but broadly want to know what is happening under the hood. A nudge in the right direction is all I need.</p>

<p>What I have learnt so far can be summarized as:</p>

<ul>
<li>There is both supervised and unsupervised learning.</li>
<li>Supervised learning is typically used to solve regression and classification </li>
<li>Unsupervised learning is used to solve clustering related problems</li>
<li>In supervised learning, we provide training data that are labelled</li>
<li>Each training data example can comprise multiple features.</li>
<li>An ML algorithm reads the labelled data, possibly finds patterns and "deduces" a general rule that maps input to output (e.g. mapping between a feature vector and a label)</li>
<li>If now an unlabeled feature vector was given, the algorithm uses the "deduced" rule and classifies the input into a label class.</li>
<li>I also understand that there are quite a few classification algorithms like SVM, Nearest neighbor, Decision Trees etc.</li>
<li>Each of these algorithms have different criteria that leads them to different "deduced" rules on basis of which, they perform classification.</li>
</ul>

<p>So the concepts am so desirous to understand at a broad level are:</p>

<ul>
<li>How does an algorithm determine inter-relation ship among features?</li>
<li>How does it know what features are to be considered and what could possibly be disregarded?</li>
<li>What does a deduced rule physically look like?</li>
<li>Is there a standard suite of rules to choose from?</li>
<li>How does the algorithm arrive at such a rule?  </li>
</ul>

<p>For example, if I had a dataset as the following:</p>

<pre><code>0 1 0 0 1 1   +
1 1 1 0 1 1   +
1 1 0 1 1 0   +
1 0 1 0 1 1   -
1 0 1 1 1 1   -
0 1 1 0 0 0   -
</code></pre>

<p>In the above example, the label is + if the second and fifth element were 1. How will an algorithm deduce this? </p>

<p>I shall be truly grateful for any clarification you provide.</p>

<p>EDIT</p>

<p>Based on Jan's recommendation, my question boils down to the following:</p>

<p>Assume x1, x2 ... xn are attributes/features. Assume h(y) is the mapping function and it exists. How does the algorithm decide that h(y) should be (x1 + x2 + x9) /5 and not h(y) = Sq Root(x1^2 + X9^3) + k</p>

<p>Given the size of the dataset, the possibilities are endless, isn't it?</p>

<p>h(y) could either be a direct classification or some value that could lead to a class subsequently.  </p>

CC BY-SA 3.0
2017-01-18T07:24:54.103Z
I think this question may be too broad to attract a satisfactory answer.
kodlu
CC BY-SA 3.0
2017-01-18T07:31:47.717Z
You are entitled to your opinion. But the main reason why people come to these forums is because they need the spark/insight/intuition which classic literature does not provide. Satisfactory answer or not is for the OP to decide and not you!
Raj
CC BY-SA 3.0
2017-01-18T08:29:26.140Z
I agree this question is too broad. Also there are several questions here, please ask only one definitely answerable question per post.
Jan Johannsen
<machine-learning>
Machine Learning: How ML algorithms build classification rules
Raj
# Machine Learning: How ML algorithms build classification rules

(Asked by: username_0 on 2017-01-18)

I am truly fascinated by algorithms learning on their own with a little help from humans and as a newbie in this field (with programming experience mainly in C/C++), seek your help to obtain the big picture before delving into specifics. I am not a researcher, but broadly want to know what is happening under the hood. A nudge in the right direction is all I need.
What I have learnt so far can be summarized as:

There is both supervised and unsupervised learning.
Supervised learning is typically used to solve regression and classification 
Unsupervised learning is used to solve clustering related problems
In supervised learning, we provide training data that are labelled
Each training data example can comprise multiple features.
An ML algorithm reads the labelled data, possibly finds patterns and "deduces" a general rule that maps input to output (e.g. mapping between a feature vector and a label)
If now an unlabeled feature vector was given, the algorithm uses the "deduced" rule and classifies the input into a label class.
I also understand that there are quite a few classification algorithms like SVM, Nearest neighbor, Decision Trees etc.
Each of these algorithms have different criteria that leads them to different "deduced" rules on basis of which, they perform classification.

So the concepts am so desirous to understand at a broad level are:

How does an algorithm determine inter-relation ship among features?
How does it know what features are to be considered and what could possibly be disregarded?
What does a deduced rule physically look like?
Is there a standard suite of rules to choose from?
How does the algorithm arrive at such a rule?  

For example, if I had a dataset as the following:
0 1 0 0 1 1   +
1 1 1 0 1 1   +
1 1 0 1 1 0   +
1 0 1 0 1 1   -
1 0 1 1 1 1   -
0 1 1 0 0 0   -

In the above example, the label is + if the second and fifth element were 1. How will an algorithm deduce this? 
I shall be truly grateful for any clarification you provide.
EDIT
Based on Jan's recommendation, my question boils down to the following:
Assume x1, x2 ... xn are attributes/features. Assume h(y) is the mapping function and it exists. How does the algorithm decide that h(y) should be (x1 + x2 + x9) /5 and not h(y) = Sq Root(x1^2 + X9^3) + k
Given the size of the dataset, the possibilities are endless, isn't it?
h(y) could either be a direct classification or some value that could lead to a class subsequently.

**Tags:** machine-learning

> **Comments:**
> **username_1**: I think this question may be too broad to attract a satisfactory answer.  
> **username_0**: You are entitled to your opinion. But the main reason why people come to these forums is because they need the spark/insight/intuition which classic literature does not provide. Satisfactory answer or not is for the OP to decide and not you!  
> **username_2**: I agree this question is too broad. Also there are several questions here, please ask only one definitely answerable question per post.  

---

## 2 answers

### Answer by username_3 on 2017-01-18. Score: 0 | Accepted answer

Answering your edited question: any learning algorithm (more specifically: supervised classification) algorithm *always* (either implicitly or explicitly) works with a well-defined class of *hypotheses* or mappings from examples to labels. In the case of the perceptron in my previous answer, it works with linear separators (and hence will not consider hypotheses involving square roots). A standard no-free-lunch theorem shows that an algorithm whose hypothesis space is too rich will no be able to learn (it will overfit).


---

### Answer by username_3 on 2017-01-18. Score: 0

A perceptron
<https://en.wikipedia.org/wiki/Perceptron>
or SVM
<https://en.wikipedia.org/wiki/Support_vector_machine>
learning algorithm would find a separating hyperplane -- which in your case, would be something like $w=(0,1,0,0,1)$ with bias $b=-1$.


---

