<p>I want to solve this classification problem. Basically what I have is a sequence of feature vectors $\mathbf{x}_1,\mathbf{x}_2,\dots,\mathbf{x}_N$, and each feature vector is sequential in time. I want to predict the class label $y$ based on the observation of these $N$ feature vectors. The outcome label $y$ can be from a set of two possible classes $y \in \{A,B\}$. So my question is: What machine learning algorithm is best suited for this kind of problem? I know sequential learning like Hidden Markov Model, but they are more suitable for the case where each observation $\mathbf{x}_i$ corresponds to a hidden state $y_i$, versus here in my case only the final class label is needed. There is no hidden states along the time in my problem. So the goal is to predict the class label $y$ based on a sequence of feature vectors $\{\mathbf{x}_i\}_{i=1}^{N}$. Any suggestions?  </p>

CC BY-SA 3.0
2013-01-29T20:04:08.910Z
Search for "classification algorithms for longitudinal data" or "classification algorithms for time-series data" (depending on the type of your data). You can also search for "longitudinal data analysis". Also, this question seems to fit better in http://stats.stackexchange.com/ , since this site is about Theoretical Computer Science.
George
CC BY-SA 3.0
2013-01-29T20:14:59.683Z
Is time a relevant feature or are you saying that the algorithm is "online"?
Lev Reyzin
CC BY-SA 3.0
2013-01-29T20:21:02.730Z
What is your training data like? Do you ever get to observe the true label y for any of the examples? The HMM you mentioned is an *unsupervised* learning algorithm (as in, you never get to see any labels), while the setting you seem to be describing suggests that labels of some sort ought to be available.
Aryeh
CC BY-SA 3.0
2013-01-29T22:17:11.793Z
@LevReyzin All the feature vectors $x_1, x_2, ..., x_N$ are in time sequence. I want to predict the label $y$ given a sequence of x.
tonga
CC BY-SA 3.0
2013-01-29T22:21:56.900Z
@Aryeh: I don't have any label information for each observed $\mathbf{x}_i$ at time instance $i$. My goal is to classify the whole sequence to either $y \in A$ or $y \in B$. So I don't know any labeling information of underlying states. Is my problem the same as Hidden Markov Model?
tonga
CC BY-SA 3.0
2013-01-30T11:53:50.520Z
How do you expect *any* algorithm to produce labels if has not seen any? Suppose I show you pictures of unknown animals and ask you to predict whether they are zorks or kroz's -- without ever giving you a labeled example of either. How would you proceed?
Aryeh
CC BY-SA 3.0
2013-01-31T19:56:31.380Z
@Aryeh: Maybe I should restate my problem a bit clearer. Given a sequence of feature vectors $x_1, x_2, ..., x_N$ with each $x_i$ a $M \times 1$ vector, I also know that the sequence can come from either class A or class B. In my training, I know the label $y$ given a sequence. In prediction, given a new sequence, I want to tell if $y \in A$ or $y \in B$. Is this the same as HMM? Thanks.
tonga
<machine-learning><ds.algorithms>
What machine learning algorithm solves this problem?
tonga
# What machine learning algorithm solves this problem?

(Asked by: username_0 on 2013-01-29)

I want to solve this classification problem. Basically what I have is a sequence of feature vectors $\mathbf{x}\_1,\mathbf{x}\_2,\dots,\mathbf{x}\_N$, and each feature vector is sequential in time. I want to predict the class label $y$ based on the observation of these $N$ feature vectors. The outcome label $y$ can be from a set of two possible classes $y \in \{A,B\}$. So my question is: What machine learning algorithm is best suited for this kind of problem? I know sequential learning like Hidden Markov Model, but they are more suitable for the case where each observation $\mathbf{x}\_i$ corresponds to a hidden state $y\_i$, versus here in my case only the final class label is needed. There is no hidden states along the time in my problem. So the goal is to predict the class label $y$ based on a sequence of feature vectors $\{\mathbf{x}\_i\}\_{i=1}^{N}$. Any suggestions?

**Tags:** machine-learning, ds.algorithms
> **Comments:**
> **username_1**: Search for "classification algorithms for longitudinal data" or "classification algorithms for time-series data" (depending on the type of your data). You can also search for "longitudinal data analysis". Also, this question seems to fit better in http://stats.stackexchange.com/ , since this site is about Theoretical Computer Science.  
> **username_2**: Is time a relevant feature or are you saying that the algorithm is "online"?  
> **username_3**: What is your training data like? Do you ever get to observe the true label y for any of the examples? The HMM you mentioned is an *unsupervised* learning algorithm (as in, you never get to see any labels), while the setting you seem to be describing suggests that labels of some sort ought to be available.  
> **username_0**: @LevReyzin All the feature vectors $x_1, x_2, ..., x_N$ are in time sequence. I want to predict the label $y$ given a sequence of x.  
> **username_0**: @Aryeh: I don't have any label information for each observed $\mathbf{x}_i$ at time instance $i$. My goal is to classify the whole sequence to either $y \in A$ or $y \in B$. So I don't know any labeling information of underlying states. Is my problem the same as Hidden Markov Model?  
> **username_3**: How do you expect *any* algorithm to produce labels if has not seen any? Suppose I show you pictures of unknown animals and ask you to predict whether they are zorks or kroz's -- without ever giving you a labeled example of either. How would you proceed?  
> **username_0**: @Aryeh: Maybe I should restate my problem a bit clearer. Given a sequence of feature vectors $x_1, x_2, ..., x_N$ with each $x_i$ a $M \times 1$ vector, I also know that the sequence can come from either class A or class B. In my training, I know the label $y$ given a sequence. In prediction, given a new sequence, I want to tell if $y \in A$ or $y \in B$. Is this the same as HMM? Thanks.  

---

## 0 answers

