<p>This is supervised learning with a vector-valued output, i.e. learning a vector to vector mapping.  There's a large literature on this in the neural network community. One application is generating control signals for a physical device (e.g. pointing a camera based on visual input).  Here's a typical reference:    </p>

<p>Suh, I.H. and Kim, T.W. Fuzzy membership function based neural networks with applications to the visual servoing of robot manipulators.  IEEE Transactions on Fuzzy Systems, 2(3):203-220, 1994. </p>

<p>From another angle, this can be viewed as <a href="http://en.wikipedia.org/wiki/Multi-task_learning" rel="nofollow">multitask learning</a>, a currently popular area in machine learning.   If one knows more about constraints among the output values, then it could usefully be viewed as a <a href="http://en.wikipedia.org/wiki/Structured_prediction" rel="nofollow">structured prediction</a> problem, also recently popular. </p>

CC BY-SA 3.0
2011-12-27T20:31:44.540Z
2011-12-27T20:31:44.540Z
DavidDLewis
<p>Without going into detail about your example, this seems to be a <a href="http://en.wikipedia.org/wiki/Multiclass_classification" rel="nofollow">multiclass</a> prediction problem.  As the outputs are on a range of possibilities, instead of counting mistakes (<a href="http://en.wikipedia.org/wiki/0-1_loss_function" rel="nofollow">0-1 loss</a>), you can pick a desirable <a href="http://en.wikipedia.org/wiki/Loss_function" rel="nofollow">loss function</a> for your application (especially if you're in the <a href="http://www.cc.gatech.edu/~ninamf/ML11/lect0913.pdf" rel="nofollow">non-realizable case</a>) and then a suitable learning algorithm.  Because you have two outputs, you can make each prediction separately to make things simpler.</p>

CC BY-SA 3.0
2011-12-27T16:12:38.793Z
2011-12-27T16:12:38.793Z
Lev Reyzin
<p>The objective is to build a classifier that produces M correct outputs when given N inputs. </p>

<p>Let a "sample" be M outputs and N inputs. Each output is some function some of the N inputs but the functions are not given. You only know that some of the N inputs can possibly combine to produce each of the M outputs.</p>

<p>Several samples are available for the machine learning algorithm to learn from, and in each sample, the functions for each output produced from what input sets are the exactly same.</p>

<p>It sounds like supervised learning but it doesn't seem to match exactly.</p>

<p>I'll provide an example:</p>

<p>__Sample 1:</p>

<p>Inputs: people = 6; salary = 100</p>

<p>outputs: width = 600; height = 100</p>

<p>__Sample 2:</p>

<p>Inputs: people = 3; salary = 60</p>

<p>outputs: width = 180; height = 60</p>

<p>__Sample 3:</p>

<p>Inputs: people = 5; salary = 101</p>

<p>outputs: width = 505; height = 101</p>

<p>The correct functions are then: width = people * salary; height = salary</p>

CC BY-SA 3.0
2011-12-23T09:21:55.513Z
Such functions can be learned using genetic programming. An application very close to this is described in the book: Programming collective intelligence: building smart web 2.0 applications by  Toby Segaran.
Dave Clarke
CC BY-SA 3.0
2011-12-23T10:46:07.430Z
Wow genetic programming is awesome! I downloaded pyevolve and it can classify my example almost perfectly! The only imperfection being some junk in the functions; e.g. making functions like a * b + b - b instead of a * b.
Eric
CC BY-SA 3.0
2011-12-23T11:02:05.707Z
You can probably post-process the output to get rid of obvious junk.
Dave Clarke
CC BY-SA 3.0
2011-12-23T12:04:32.340Z
I found a way that seems to be working: Get the RMSE of each instance between inputs and outputs, add to it (the number of nodes in the function divided by 100).

Two correct functions will have scores e.g. 0.14 and 0.03. One of them contains lots of junk and the other less. Both are preferable to incorrect functions.
Eric
CC BY-SA 3.0
2011-12-23T12:05:27.697Z
What if you simply tried to use linear regression?
Jukka Suomela
CC BY-SA 3.0
2011-12-23T12:07:35.457Z
Jukka: I tried linear regression before but have experienced trouble with multi-collinearity. Regression could also only solve simple cases like the above; I had no idea how to add stuff like "max" and "min". (e.g. for above example, what if width and height was bounded between 0 and 1000 and additional samples were added to reflect this? Genetic programming seems to be able to handle max & min well, but my attempts with regression couldn't overcome that obstacle.)
Eric
CC BY-SA 3.0
2011-12-30T05:14:13.460Z
The title of the question is not very informative (it is not possible to understand what is "this" without reading the post). Please use a more informative title. By the way, you may want to check the Q&A sites dedicated to machine learning linked in the [FAQ].
Kaveh
CC BY-SA 3.0
2012-01-01T16:08:39.330Z
Kaveh: I didn't know what to call "this"...
Eric
<machine-learning>
What kind of machine learning is this?
Eric
# What kind of machine learning is this?

(Asked by: username_0 on 2011-12-23)

The objective is to build a classifier that produces M correct outputs when given N inputs.

Let a "sample" be M outputs and N inputs. Each output is some function some of the N inputs but the functions are not given. You only know that some of the N inputs can possibly combine to produce each of the M outputs.

Several samples are available for the machine learning algorithm to learn from, and in each sample, the functions for each output produced from what input sets are the exactly same.

It sounds like supervised learning but it doesn't seem to match exactly.

I'll provide an example:

\_\_Sample 1:

Inputs: people = 6; salary = 100

outputs: width = 600; height = 100

\_\_Sample 2:

Inputs: people = 3; salary = 60

outputs: width = 180; height = 60

\_\_Sample 3:

Inputs: people = 5; salary = 101

outputs: width = 505; height = 101

The correct functions are then: width = people \* salary; height = salary

**Tags:** machine-learning
> **Comments:**
> **username_1**: Such functions can be learned using genetic programming. An application very close to this is described in the book: Programming collective intelligence: building smart web 2.0 applications by  Toby Segaran.  
> **username_0**: Wow genetic programming is awesome! I downloaded pyevolve and it can classify my example almost perfectly! The only imperfection being some junk in the functions; e.g. making functions like a * b + b - b instead of a * b.  
> **username_1**: You can probably post-process the output to get rid of obvious junk.  
> **username_0**: I found a way that seems to be working: Get the RMSE of each instance between inputs and outputs, add to it (the number of nodes in the function divided by 100).  Two correct functions will have scores e.g. 0.14 and 0.03. One of them contains lots of junk and the other less. Both are preferable to incorrect functions.  
> **username_2**: What if you simply tried to use linear regression?  
> **username_0**: Jukka: I tried linear regression before but have experienced trouble with multi-collinearity. Regression could also only solve simple cases like the above; I had no idea how to add stuff like "max" and "min". (e.g. for above example, what if width and height was bounded between 0 and 1000 and additional samples were added to reflect this? Genetic programming seems to be able to handle max & min well, but my attempts with regression couldn't overcome that obstacle.)  
> **username_3**: The title of the question is not very informative (it is not possible to understand what is "this" without reading the post). Please use a more informative title. By the way, you may want to check the Q&A sites dedicated to machine learning linked in the [FAQ].  
> **username_0**: Kaveh: I didn't know what to call "this"...  

---

## 2 answers

### Answer by username_4 on 2011-12-27. Score: 2 | Accepted answer

This is supervised learning with a vector-valued output, i.e. learning a vector to vector mapping. There's a large literature on this in the neural network community. One application is generating control signals for a physical device (e.g. pointing a camera based on visual input). Here's a typical reference:

Suh, I.H. and Kim, T.W. Fuzzy membership function based neural networks with applications to the visual servoing of robot manipulators. IEEE Transactions on Fuzzy Systems, 2(3):203-220, 1994.

From another angle, this can be viewed as [multitask learning](http://en.wikipedia.org/wiki/Multi-task_learning), a currently popular area in machine learning. If one knows more about constraints among the output values, then it could usefully be viewed as a [structured prediction](http://en.wikipedia.org/wiki/Structured_prediction) problem, also recently popular.


---

### Answer by username_5 on 2011-12-27. Score: 1

Without going into detail about your example, this seems to be a [multiclass](http://en.wikipedia.org/wiki/Multiclass_classification) prediction problem. As the outputs are on a range of possibilities, instead of counting mistakes ([0-1 loss](http://en.wikipedia.org/wiki/0-1_loss_function)), you can pick a desirable [loss function](http://en.wikipedia.org/wiki/Loss_function) for your application (especially if you're in the [non-realizable case](http://www.cc.gatech.edu/~ninamf/ML11/lect0913.pdf)) and then a suitable learning algorithm. Because you have two outputs, you can make each prediction separately to make things simpler.


---

