GMM in speech recoginition using HMM-GMM

The previous answer was wrong so I removed it.

Here goes my second attempt after reading Speech and Language processing by daniel Jurafsky and James H Martin(good book to read).

The 39 features associated with an observation/acoustic is considered to have come from mixtures of multivariate gaussian.

Why Mixture of MV gaussian ? Assuming a single MV gaussian for each state(phones) is a strong assumption which might not be true.

How does HMM comes into picture with GMM in ASR: Consider an uni-variate case where a single cepstral feature(usually it is 39) is represented by a single gaussian and HMM state has a mean value and variance which generate the particular observation. To get which observation was produced by which state is a part of decoding problem.

Let me know if this is right ?


You can use grid_obj.predict(X) or grid_obj.best_estimator_.predict(X) to use the tuned estimator. However, I suggest you to get this _best_estimator and train it again with the full set of data, because in GridSearchCV, you train with K-1 folds and you lost 1 fold to test. More data, better estimates, right?


When using Absolute Error in Gradient Descent, how to calculate the derivative?

The gradient of MAE is not continuous in $y_{pred} = y_{true}$ and therefore there is no defined (bounded, direction independent) derivative at that point.
Elsewhere you have -1, where  $y_{pred} > y_{true}$ and +1 where  $y_{pred} < y_{true}$
Usually frameworks like TensorFlow, Keras, etc... use an approximate derivative for that point.


Does nearest neighbour make any sense with t-SNE?

No, it is not necessary that this is the case, however, this is, in a convoluted way, the goal of T-SNE.

Before getting into the meat of the answer, let's take a look at some basic definitions, both mathematically and intuitively.

Nearest Neighbors: Consider a metric space $\mathbb{R}^d$ and a set of vectors $X_1, ..., X_n \in \mathbb{R}^d$, given a new vector $x \in \mathbb{R}^d$, we want to find the points such that $|| X_1 - x || \le ... \le ||X_n - x ||$. Intuitively, it's just the minimum of the distances using a suitable definition of norm in $\mathbb{R}^d$.

Now coming to whether the nearest neighbors actually matter while applying dimensionality reduction. Usually in my answers, I intend to rationalize something with mathematics, code and intuition. Let us first consider the intuitive aspect of things. If you have a point that is a distance $d$ away from another point, from our understanding of the t-sne algorithm we know that this distance is preserved as we transition into higher dimensions. Let us further assume that a point $y$ is the nearest neighbor of $x$ in some dimension $d$. By definition, there is a relationship between the distance in $d$ and $d + k$. So, we have our intuition which is that the distance is maintained across different dimensions, or at least, that is what we aim for. Let's try to justify it with some mathematics.

In this answer I talk about the math involved in t-sne, albeit not in detail (t-SNE: Why equal data values are visually not close?). What the math here is, is basically maximizing the probability that two points remain close in a projected space as they are in the original space assuming that the distribution of the points is exponential. So, looking at this equation $p_{j | i} = \frac{exp(\frac{-||x_j - x_i||^2}{2\sigma^2})}{\sum_{k \neq i}{exp(\frac{-||x_j - x_i||^2}{2\sigma^2})}}$. Notice that the probability is dependent on the distance between the two points, so the further apart they are, the further apart they get as they get projected to lower dimensions. Notice that if they are far apart in $\mathbb{R}^k$, there is a good chance they will not be close in the projected dimension. So now, we have a mathematical justification as to why the points "should" remain close. But again, since this is an exponential distribution, if these points are significantly far apart, there is no guarantee that the Nearest Neighbors property is maintained, although, this is the aim.

Now finally a neat coding example which demonstrates this concept too.

from sklearn.manifold import TSNE
from sklearn.neighbors import KNeighborsClassifier
X = [[0],[1],[2],[3],[4],[5],[6],[7],[8],[9]]
y = [0,1,2,3,4,5,6,7,8,9]
neighs = KNeighborsClassifier(n_neighbors=3)
neighs.fit(X, y)
X_embedded = TSNE(n_components=1).fit_transform(X)
neighs_tsne = KNeighborsClassifier(n_neighbors=3)
neighs_tsne.fit(X_embedded, y)
print(neighs.predict([[1.1]]))
>>>[0]
print(neighs_tsne.predict([[1.1]]))
>>>[0]

Although this is a very naive example and doesn't reflect the complexity, it does work by experiment for some simple examples.

EDIT: Also, adding some points with respect to the question itself, so it is not necessary that this is the case, it might be, however, rationalizing it through mathematics would prove that you have no concrete outcome (no definitive yes or no).

I hope this cleared up some of your concerns with TSNE.


Multiple linear regression for multi-dimensional input and output?

You are asking about multioutput regression. The class you talked about sklearn.linear_model.LinearRegression supports this out of the box.
import numpy as np
from sklearn.linear_model import LinearRegression

# features
A = 10
# number of values to predict
B = 15
# number of rows in dataset
m = 100

x = np.ones((m, A))
y = np.ones((m, B))

model = LinearRegression()
model.fit(x, y)

sklearn.linear_model.LinearRegression actually just creates B models. However it optimises calculations using vectorisation.
It actually is exactly the same as a fully connected layer in a neural network which has no activation function.
You can read more about it here:
https://machinelearningmastery.com/multi-output-regression-models-with-python/


Confidence interval interpretation in linear regression when errors are not normally distributed

In case you might want to try another way to find confidence intervals, and in addition to the nice and rigourous answer by Peter, I would also consider a resampling method like bootstrapping as a robust way to find confidence intervals.
One key advantage is that it does not assume any kind of distribution, being a distribution-free method to find your coefficients estimates.

In the case of finding a confidence interval for a linear regression, the steps would be:

Draw n random samples (with replacement) from your dataset, where n is the bootstrap sample size
Fit a linear regression on the bootstrap sample from step 1
Repeat steps 1 & 2 n_iters times, where n_iters will be the number of bootstrap samples and linear regressions made on them
Now that we have n_iters values for the linear regression coefficients, we can find the interval limits via the min, median and max percentiles (e.g. for a 95% CI: percentile 2.5, 50 and 97.5) to find the coefficient estimate together with the CI limits

Please mind the variability of the confidence intervals along the x-axis values, taking into account the sampling error of the coefficients estimates (good source of read: https://greenteapress.com/wp/think-stats-2e/)

The associated code of my example including the plot can be found here


Should I use epochs > 1 when training data is unlimited?

If the data is unlimited, how would you have an epoch to begin with? For example, if you are analyzing tweets, you could never finish an epoch will all the tweets, since there will be an endless supply of new tweets. A much better approach will be to do some online or streaming learning.

Would it make sense to create a subset by ignoring new incoming tweets or data in general?

That really depends on your problem. For example if you have a datastream of tweets, or any other where new data is being produced in real time, you would miss out any trends or patterns that emerged after you sampled from the stream. Are those missed patterns relevant to your problem? They may or may not be.


Better way of writing following Pandas Python code

I was provided with this answer by someone:

# Set the index of the dataframes to 'Country' so that you can get the countrywise
# medal count
gold.set_index('Country', inplace = True)
silver.set_index('Country', inplace = True)
bronze.set_index('Country', inplace = True)

# Add the three dataframes and set the fill_value argument to zero to avoid getting
# NaN values
total = gold.add(silver, fill_value = 0).add(bronze, fill_value = 0)

# Sort the resultant dataframe in a descending order
total = total.sort_values(by = 'Medals', ascending = False)

# Print the sorted dataframe
print(total)


Is there a closed formula/function for decision trees?

It can be expressed as a linear combination of indicators, where the weights are the training averages on each partition of the space, and the indicator is the indicator for that partition.
Having said that, I haven't come across any software that automatically does it for you. I also think the analytic expressions might become very large when dealing with moderately wide trees.


For answering questions from graph, you should not visualize it. Visualizing graphs is for sake of having an overview on how it looks like in general. There are Graph Visualization techniques that show graph for sake of getting some initial insight (e.g. if there are visually obvious communities or not).

Your question has an analytic answer. After constructing your graph object, you can get the adjacency matrix as a Python array or matrix. This matrix will be asymmetric as outgoing and incoming degrees are different of course. In Networkx you can get in-degree and out-degree information as seen below. Then the main story starts!

The most influential of a social network can be seen as the most central node. Centrality measures capture them for you. The simplest one is degree centrality. It simply says the node with highest degree has most influence in graph. In your case be sure how you model "the guy who is retweeted"as he has the most influence. If being retweeted is in-degree in your modeling then this code get them for you:

import networkx as nx
import operator

g=nx.digraph.DiGraph()
g.add_edges_from([(0,1),(1,2),(2,3),(4,2),(4,3)])
dict_deg = {ii:jj for (ii,jj) in g.in_degree}
print('In-Degree Dictionary\n',dict_deg)

m = max(list(dict_deg.values()))
print('The node(s) number',[i for i, j in enumerate(list(dict_deg.values())) if j == m],'have the most influence!')

Output:

In-Degree Dictionary
 {0: 0, 1: 1, 2: 2, 3: 2, 4: 0}
The node(s) number [2, 3] have the most influence!

And the graph itself:

pos = nx.spring_layout(g)
_=nx.draw(g,label=True,pos=pos)
_=nx.draw_networkx_labels(g,pos=pos)

I hope it answered your question. If not please drop a comment so I can update.


state-action-reward-new state: confusion of terms

The TD Target (for learning update) for using $\hat{q}(s,a)$ neural network in Q-learning is:

$$r + \text{max}_{a'} \hat{q}(s',a')$$

In order to calculate this, you need a starting state $s$, the action taken form that state $a$, and the resulting reward $r$ and state $s'$.

You need the $s, a$ to generate the input to the neural network (what you might call train_X for supervised learning). You need $r, s'$ to generate the TD Target shown above as an output to learn for regression (in supervised learning, that would be train_y). And you need to work through all possible $a'$ based on $s'$ in order to find the maximum value for the TD Target equation used in Q learning - other RL algorithms may use variations of this for calculating the TD Target.

This means your suggestions are all close but not quite right.

  Should one rely on the B2 state to iterate over possible actions from this state (next state) to get an approximation of highest reward (max Q)?

Sort of. The B2 state is $s'$ from the equation, so is responsible for calculating the TD target for state-action value $q(s,a)$.

  Then, why do we store the A1 and move-to-B2 information at all in the replay buffer?

You still need to know $s$ is A1, because the representation of A1 (and whichever action is taken) will be the input to your network.

  Or I am wrong and we just use the A1 and iterate over possible actions (including that to B2) to get the max Q?

Iterate over actions from B2 using your neural network to pick the highest estimate.

  I think I have found an answer ). We need to store previous state (A1) and action (move to B2) in order to create the state-action distribution, which will be met with the expected long-term reward distribution, that we get after the next state routine. Right?

I'm not sure I fully understand this, but it does not seem quite right. One thing that might be confusing you is having your actions as "move to state". Whilst this is quite normal in many deterministic environments, especially board games, most RL formula and tutorials are written using separate state/action pairs, so whilst getting this straight in your mind, best to find another way to represent actions - e.g. move piece from X to Y, or place new piece at (I,J) . . . you can return to "change state from A1 to B2" as the representation later. This is actually more efficient and is called the after-state representation, but most of the literature will show state-action value functions.


F - measure derivation (harmonic mean of precision and recall)

why the equality of both partial derivatives correspond to these
  hypothesis. I would rather understand when one partial derivative
  equals the other partial derivative multiplied by minus one.

Your intuition of "trading off" by "subtracting" a value is correct when you speak in terms of $\Delta R$ and $\Delta P$ (as you yourself noticed in the edit), but is not valid if you speak in terms of derivatives.

Think of it this way: a partial derivative with respect to a parameter tells you by how much your metric changes if you improve that variable. By the very design, $F$ will always increase if you improve either $R$ or $P$ - its partial derivatives are positive everywhere.

They are not equal everywhere, however. At some configurations $(R,P)$ you would gain more by improving one parameter rather than the other. Say, if your $R$ is much lower than $P$, your $F$-score would increase more if you improved the $R$ a bit rather than $P$.

In the case of an $F_1$ metric, $R$ and $P$ enter the formula symmetrically and it is kind-of easy to see that whenever $R=P$, either of the components contributes equally to the increase of $F$. In fact, the set of points where $R$ and $P$ are "equal in importance" is exactly the line $R=P$ (which can also be written as $R/P=1$, if you prefer).

Now consider the equation for $F_\beta$:

$$\frac{\text{constant}}{\beta^2\frac{1}{R} + \frac{1}{P}}.$$

Let us see how much it changes in response to increasing $R$ or $P$. As both parameters are in the denominator, the change in value of $F_\beta$ is proportional to the change in the value of its denominator, hence we may simplify a bit and only consider the change in

$$\beta^2\frac{1}{R} + \frac{1}{P}$$

You can now compute that a $dx$ increase in $R$ would decrease the value of the expression above by $\beta^2\frac{dx}{R^2}$. A similar increase in $P$ would decrease the value of the above expression by $\frac{dx}{P^2}$. There is a set of points $(R,P)$ where these effects are exactly equal. These are the points where
$$\beta^2\frac{dx}{R^2} = \frac{dx}{P^2} \quad\Rightarrow\quad \frac{R}{P}=\beta$$

When $\beta > 1$ you are effectively putting more marginal importance on recall, in the sense that until your recall is at least $\beta$ times the precision, it is always beneficial to work on it, rather than improving precision.

  What am I missing?

You are missing the fact that the condition $\frac{\partial F_\alpha}{\partial P} = \frac{\partial F_\alpha}{\partial R}$ does not hold everywhere, but just on the set of points $R=\beta P = \sqrt{\frac{1-\alpha}{\alpha}}P$. This line intersects the level curve at just a single point.

The line with slope -1 would be the tangent to the level line at that point - the green dashed line on the illustration below.


How many features do you generally use for your ML Model?

The number of features should take into account the number of examples you have in the training data.

If your model training takes too long, have you considered principal components analysis or feature selection?  Both techniques can cut down the number of features into a more manageable size without throwing away information which describes the target variable.


Grid search grows geometrically with the number of hyperparameters and values for each hyperparameter. It is best practice to minimize the search space to fewer options.
Increased performance on training dataset is no guarantee of increased performance on test set.


What's wrong with my gradient descent? (simple, one variable)

You set your model parameters using np.array([[0], [0]]), meaning that your parameter array is of shape (2, 1). Changing this to np.array([0, 0]) (with shape (2,)) decreases the loss over time as expected using a learning rate $\alpha$ of 0.01. Looking more closely at the difference of using the two, I see that your way returns way higher losses (exactly 100x). Using a smaller $\alpha$, e.g. 0.0001, with your implementation also works:

Therefore the main reason of your implementation is your choice of $\alpha$, which is too high, preventing the model to reach the optimum.
EDIT: The issues is twofold, the first is indeed the shape or your theta array. The second has to do with the fact that you are using the theta array in your for loop which always contains the original theta parameters, as you are not storing and using the updated theta1 and theta2. The following code seems to give a good result of $\theta_1 \approx -3.2$ and $\theta_2 \approx 1.12$ using 1000 iterations with $\alpha=0.01$
import numpy as np

# load and transform data
data = np.genfromtxt("data.txt", delimiter=",")
X = np.stack((np.ones(data.shape[0]), data[:,0]), axis=-1)
y = data[:,1]

m = X.shape[0]
iter = 1000

# calculating the cost function
theta = np.array([[0.3], [0.5]])
def h(X, theta):
    return np.matmul(X, theta)

def cost(X, y, theta, m):
    return 1/(2* m) * np.sum((h(X, theta)-y)**2)

print(cost(X, y, theta, m))
# 1-variable gradient decent
cost_vals = []
def grad_decent(X, y, theta, m, alpha):
    theta1 = theta[0, 0]
    theta2 = theta[1, 0]
    for i in range(0, iter):
        theta = np.array([theta1, theta2])
        cost_vals.append(cost(X, y, np.array([theta1, theta2]), m))
        theta1 = theta1 - alpha/m * np.sum((h(X, theta)-y) * X[:, 0])
        theta2 = theta2 - alpha/m * np.sum((h(X, theta)-y) * X[:, 1])
        # print(f"Iteration {i}: ", theta1, ", ", theta2)

    return np.array([[theta1], [theta2]])

grad_decent(X, y, theta, m, 0.01)
# 1740.4098044406999
# array([[-3.20070025],
#        [ 1.12320526]])


How to predict the dealer whether pick up the goods next month?

Sure.

Question you should answer is it multiclass or mutlilabel.

In any case I would really advise you to take a look at this list of similiar problems/competitions. Since your question is open-ended I could write 10 pages of approaches. Yours best bet is to invest some time, find similiar problem and look what other people did and dig it up.


How to Improve Low Accuracy Keras Model Design?

Looking at the training epochs, it seems to me you set a patience parameter that is too short. Please consider removing early stopping at all, for a model trained on 1500 observations only. Early stopping comes useful for particularly heavy models, but in this you shouldn't need it.

I think the size of each mini-batch is very small. That would make gradient descend very noisy, please consider increasing its size, or using full batch training as well.

Additionally, I think you have implemented a Network that is too big. Your input is very small, therefore you don't need to expand its signal on layers of size 64. There are too many nodes that are trying to "learn" not many things, IMHO. A good architecture could be:

model = Sequential()
model.add(Dense(6, input_dim=6, activation='relu'))
model.add(Dense(6, activation='relu'))
model.add(Dense(1, activation=None))

That would make your model faster to train, and ensure that each node is learning relevant features of your data.

I would also change the output layer. Since you want to predict an outcome, you need an output node with no activation (i.e. linear activation). That is mandatory for regression tasks with unbounded output.

Additional things you can try are:

change dropout levels (but for such a small network it might not be needed at all),
try regularization techniques, such as Batchnorm, L1 - L2 regularization, different weights initialization... you name it,
try alternative activation functions.

Also, for medium-to-small size datasets, it is possible for other ML algorithms such as Random Forests or SVMs to beat Neural Networks in performance.


Using a neural network for prediction on natural language data can be a tricky task, but there are tried and true methods for making it possible.

In the Natural Language Processing (NLP) field, text is often represented using the bag of words model.  In other words, you have a vector of length n, where n is the number of words in your vocabulary, and each word corresponds to an element in the vector.  In order to convert text to numeric data, you simply count the number of occurrences of each word and place that value at the index of the vector that corresponds to the word. Wikipedia does an excellent job of describing this conversion process.  Because the length of the vector is fixed, its difficult to deal with new words that don't map to an index, but there are ways to help mitigate this problem (lookup feature hashing).

This method of representation has many disadvantages -- it does not preserve the relationship between adjacent words, and results in very sparse vectors.  Looking at n-grams helps to fix the problem of preserving word relationships, but for now let's focus on the second problem, sparsity.

It's difficult to deal directly with these sparse vectors (many linear algebra libraries do a poor job of handling sparse inputs), so often the next step is dimensionality reduction. For that we can refer to the field of topic modeling:  Techniques like Latent Dirichlet Allocation (LDA) and Latent Semantic Analysis (LSA) allow the compression of these sparse vectors into dense vectors by representing a document as a combination of topics.  You can fix the number of topics used, and in doing so fix the size of the output vector producted by LDA or LSA. This dimensionality reduction process drastically reduces the size of the input vector while attempting to lose a minimal amount of information.

Finally, after all of these conversions, you can feed the outputs of the topic modeling process into the inputs of your neural network.


Applications and differences for Jaccard similarity and Cosine Similarity

The answer from saq7 is wrong, as well as not answering the question.

∥A∥ means the $L2$ norm of $A$, i.e. the length of the vector in Euclidean
space, not the dimensionality of the vector $A$.  In other words, you
don't count the 0 bits, you add up only the 1 bits and take the square root.

Sorry I don't have a real answer as to when you should use which metric,
but I can't let the incorrect answer go unchallenged.


Interpretation of an SVD for recommender systems

First of all, note that the dot product between two movies/users is by definition the correlation between them. So your intuition for treating it as a similarity measure is not wrong.

Now, applying SVD to $A$ is simply a re-representation of it in some other basis. Let's call it feature space. If we'll take it one step further and keep only $k$ leading vectors of $U$ and $V$, we'll get two matrices: $U^{(k)} \in \mathbb{R}^{f,k}$ and $V^{(k)} \in \mathbb{R}^{k,u}$ (where $f$ and $u$ are the dimensions you supplied in your question). where $U^{(k)}\cdot V^{(k)} \approx A^{(k)}$ is a $k$-rank approximation for $A$ ** (note it has the same dimensions).

So you can of $U^{(k)}$ as a matrix which rows movies, and each movie has some $k$ latent features. Similarly, $V^{(k)}$ will be a "list" (columns) of users and each user is described by some $k$ values (latent features).

Note that you can "split" the scaling matrix $\Sigma^{(k)}$ between $U^{(k)}$ and $V^{(k)}$: $U'^{(k)} = U^{(k)}\Sigma^{(k)0.5}$ and $V'^{(k)} = V^{(k)}\Sigma^{(k)0.5}$, to make up for the lack of scale of $U$ and $V$.

This factorization can be elaborated into more interpretable methods such as LDA.

** The variational approach to singular values tell us that this approximation minimizes the difference between $A$ and any other $k$ rank matrix in terms of Frobenius-norm and operator-norm (and possibly other norms I'm not aware of).


Theoretically, the formula with two matrices is more clear and self-evident, I think that's the reason why it's used more often.
In practice, both approaches are actually used in production and hence are equivalent. It's just a matter of preference.

Tensorflow

For example, Tensorflow is often optimized for performance. Here's how basic RNN cell is implemented there (tensorflow/python/ops/rnn_cell_impl.py):

def call(self, inputs, state):
    """Most basic RNN: output = new_state = act(W * input + U * state + B)."""

    gate_inputs = math_ops.matmul(
        array_ops.concat([inputs, state], 1), self._kernel)
    gate_inputs = nn_ops.bias_add(gate_inputs, self._bias)
    output = self._activation(gate_inputs)
    return output, output

A single matrix multiplication is more efficient, so it's applied, even though the comment describes the expanded formula with two matrices.

Keras

On the other hand, Keras often chooses simplicity and clarity over performance. Here's its implementation (keras/layers/recurrent.py):

def call(self, inputs, states, training=None):
    prev_output = states[0]

    ...

    if dp_mask is not None:
        h = K.dot(inputs * dp_mask, self.kernel)
    else:
        h = K.dot(inputs, self.kernel)
    if self.bias is not None:
        h = K.bias_add(h, self.bias)

    ...
    output = h + K.dot(prev_output, self.recurrent_kernel)
    if self.activation is not None:
        output = self.activation(output)

The class thus makes its it easy to access two matrices separately (self.kernel and self.recurrent_kernel).

Pytorch

Pytorch approach is closer to keras. Moreover, not only they use separate kernel matrices, they also have two bias vectors, and these four arrays are accessible for the client code.

Despite these differences all three libraries are functionally equivalent.


Looking at your graphic there seems to be a change in the series behaviour right about when your models start predicting 0. Since you have had the same issue with several models I would say that this is a data issue.

Have you checked looking at these values?
Try plotting the 1-period return?
Try skipping the parts in the validation set that work and make prediction directly from when the model is starting to fail - or even a bit after i.e X_test[1200:] or X_test[2000:]


Sort dataframe by date column stored as string

Your month column is really a partial date (i.e. the year is missing), but you could still just convert that column to a proper date format and then sort.
You will have to specify a format, so the parser knows roughly what to do.

df["date"] = pd.to_datetime(["20-Apr"], format="%d-%b")
df.sort_values("date")   # ascending by default

Without any year information, it will add the year as 1900.

If you want to add e.g. this year, you could do the following, adding 12*20 months to the new date column:

df["fixed_date"] = df.date.apply(lambda x: pd.tseries.offsets.shift_month(x, 12*20))


Each time series can be decomposed in at least three elements:

Trend
Seasonal component
Noise

An additive model can be explained as:

y = Trend + Seasonal + Noise

while a multiplicative component as:

y = Trend * Seasonal * Noise

The main difference is that in the case of additive models all three components impact your dependent variable independently. In case of multiplicative models instead, the impact of each component is dependent from the others.

The choice of the "right" model depends on the problem at hand. If you assume that the Seasonal effect is independent from the level of the Trend, then use an additive model. If you think the value of the Trend is going to affect the magnitude of the Seasonal effect, then go for the multiplicative one.

Hope this helps, otherwise let me know.


Error in Convolutional Neural network for input shape

The convolution operation used in CNNs are batch operations. The customary inputs for these networks, at least for Keras and Tensorflow, are channels last and channels first. The first dimension specifies the number of data in your batch, the second and third can be height and width if you choose channels-last and the last one will be the number of channels of your input, for instance the number of channels for grey-scaled images is one whilst for coloured images is three. If you use channels first, the first dimension would be the number of data in your batch, the second would be the number of channels and finally, the third and fourth dimensions would specify the number of height and width respectively.

For your case, it seems you are using MNIST data-set. What you need to do is just reshaping your input images to one of the standard mentioned manners. You can resize your input to (1000,28,28, 1).


Here's my two cents:

Yes, if you want to perform enseble each model should be trained on the same set of data. The technique you described is stacking, because you "stack" each prediction and simply use the majority.
Ensemble Learning, to me, si more "complex", infact you can combine the different predictions from k models via linear combination, where you estimate the coefficients of the combination itself.

More precisely, you have k prediction for each of your test observations, from k models (ie: 3 or more), and you need to find the best combination (for each obs.) of these, to have the best "accuracy" (any kind of metric).

Keep in mind that it's best to combine models that have low or even opposite correlation, so that you "diversify" your predictions, that would mean better results.

Often peole train lots and lots of different model, than use a simple Neural Network using the predictions as inputs, and the true labels from the test set as the truth, just as a normal class model would do.

See the papers in the References here as a starting point.

Bagging is basically a special case of stacking via majority vote, see:

Bagging.

You propose to the same type of model (ie: decision trees) the train dataset multiple times, making new different extraction, with replacement. This is like submitting new entries, but basically you are using the same data all over. Then you average all the predictions again, this helps with overfitting but it's not always the case, because you are using the same model, and so each si highly correlated with the other.

Yes, in boosting, just like bagging you use the same model (usually a weak learner). At each stage, the learner learns from the errors made at the previus stage.

Here an excellent visual example of what's going on.

To be clear, stacking and ensemble learning can be used with lots of different models on the same train set (it's even better if the model in the "ensemble" are quite different). You can combine each prediciton with different techniques, from simple average, to the more complex use of Neural Networks.

Bagging and Boosting are based on the use of the same model, one using the same obs. multiple times, the latter uses the same weak learner on the same data, but each makes predictions using the errors made by the previous one, in a loop.

Hope this helps you a little.


for i in range(len(Rows))):
    row = [Rows[i][0] + '_' + Rows[i][1] + '_' + Rows[i][2]]
    row.extend(Rows[i][3:])
    Rows[i] = row


Difference between text-based image retrieval and natural language object retrieval

Disclaimer: I can only answer for the NLP part since I'm no expert for image processing.
I assume that text-based image retrieval is the task of finding the image (or the part of an image) which corresponds to a short text which exclusively describes the object. Practically it means that any content word (i.e. excluding grammatical words like determiners) in the text refers directly to the object: "a bike", "a black cat", "the red car", etc. For a ML process it means that there's nothing to analyze in the text, every word can directly be associated with a characteristic of the image.
By contrast Natural Language object retrieval involves analyzing the text. For instance "the cat on the left of the picture" is different than "the picture on the left of the cat", even though the words are the same. Additionally there can be different ways to refer to the same object: "the book at the left of the shelf" may be the same as "the leftmost book" or "the book next to the green book". There are usually many ways to express the same meaning with language, and that makes the task much more complex. Additionally I would assume that mapping positional descriptions to the image characteristics can be tricky: "the man behind the tree" or "the second bridge" in a 2D image requires the model to "understand" depth. In a picture with two dogs, "the small dog" requires the model to "understand" size relation between objects. Humans intuitively know how to interpret these sentences, but for a machine Natural Language Understanding hasn't been solved yet (it might never be).


Estimating destination according to previous data

This is a spatio-temporal clustering problem that is likely best solved with a Markov model. You could reasonable group this into machine learning or data mining. Develop your model using machine learning and then (the data mining part) leverage those pattern recognition techniques (that have been developed in machine learning). I think there are at least one or two threads on this over at Cross-Validated that go into more detail.

Here are a couple of papers to look at if you are just getting started.

Using GPS to learn significant locations and predict movement across multiple users

Predicting Future Locations with Hidden Markov Models


You should look at a classifier based on Dynamic Time Warping (DTW) distance. DTW is a method that calculates an optimal match between two given sequences (e.g. time series). It has been used in a supervised learning setup, in particular it has been reported to achieved state of the art results, when used in a nearest neighbour classifier.


Assigning values to missing target vector values in scikit-learn

The model likely won't have much predictive power if the input is a single day. No weather patterns longer than one day can be captured that way.

Instead you should aggregate the days together. You can come up with different features that describe your larger, aggregated unit of time (months, year). For example mean precipitation is a very simple one. Binning the data and using counts within those bins would also work.

More advanced options would roll the time all the way up to a full year and learn a feature set at that level.


AlexNet CNN how can it be applied to my case?

Potentially what you could do is use a GMM with 3 modes to cluster the image into 3 "partitions". The first would contain all the pixels with blue, the second all the pixels with brown, and the final should contain the background pixels. You would cluster based on the raw RGB values (unless you could calculate some more semantic deep learning based feature for each pixel to cluster). You would then do some form of colour sampling on the pixels assigned to each of the 3 modes to determine which are the brown blobs and which are the blue blobs. You can count easily by computing the binary image from the GMM blobs.


To learn machine learning which one is good?

I can think of the following pros and cons for each. As for learning to code your own machine learning algorithms such as logistic regression:

You will definitely learn more about specific algorithms, their details, role of different parameters and etc. That is also a good practice of coding itself. Then you can validate your implementation by benchmarking it against other packages and implementations.
You will have more freedom in controlling different aspects of your method. You can add functions and modules as you wish and do not necessary have to deal with predefined variables, methods and etc.

On the other hand, implementing algorithms when it is not necessary and you can just use existing packages is like reinventing the wheel. It normally takes a lot of time and you have to verify your results for each one of them. Packages like sklearn are popular because of the following:

A group of people are working on those, constantly making them up to date, testing the  methods in different situations for different needs. That makes packages like sklearn very dependable and usually scalable.
If you have a question about them, there are tons of resources out there; documentation, forums, source code, communities like StackOverflow where thousands of people are eager to help you literally for any error you face while running your code.
Another important feature is automated hyperparameters tuning. Most of machine learning algorithms have a series of hyperparameters that need to be optimized in order to achieve the best performance. Packages like sklearn efficiently search for the optimal tuning parameters (or "hyperparameters") for your machine learning model in order to maximize its performance.
Still if you are interested in implementing machine learning algorithms and like the coding, you can always contribute to the existing packages. They usually have Github repositories where you can raise an issue, ask for a new feature or provide help improving them.

All in all, if you have enough time and you are keen to learn low level details about those models, go ahead and give implementation them a shot. That is certainly fun. However, if you need to get to the results as soon as possible and looking for a reliable package where a huge group of people both in industry and academia are already using, sklearn, pandas and others are you options.

Hope this is helpful and good luck.


Well ... the simplest approach is using Fuzzy String Matching and it will work. Just go through the examples in python implementation of it (fuzzywuzzy) and you will understand how it works. You need to find a threshold by practice to determine if two strings are similar enough to be considered as same concept.

If it didn't work please drop a line in the comments so I can propose more sophisticated algorithms.

Good luck!


Database missing extensions:tym_system_time,quantile orange widget error

This is not really a proper place to ask questions about software issues, but rather about data mining concepts.

However, I'll answer your question.
Your database is missing both extensions as specified in the error message. Here are some brief instructions.


You can't put v_length in the regression - that'd be a form of data leakage. However, you are right to be thinking about "how to tell the model that I've already observed some length". This can be accomplished with some survival analysis math. For censored observations, what you want is

$S(l \;|\; l > \text{observed v_length})= P(L > l \;|\; L > \text{observed v_length})$.

Let's analyze this:

$P(L > l \;|\; L > \text{observed v_length}) = \frac{P(L > l \;\text{and}\; L > \text{observed v_length})} {P(l > \text{observed v_length})}$

$\;\;\;\;\;\;=\frac{P(L > l)} {P(L > \text{observed v_length})} = \frac{S(l)}{S(\text{observed v_length})} $

This gives you a new survival curve that you can use to do imputation (take the median or mean of the new survival curve).

In python's lifelines package, this calculation is done behind the scenes when the conditional_after argument is used in the prediction methods, see docs here. I'm not sure if R packages have something like this.

It also doesn't make sense to plot the observed vs predicted, because some observed values are truncated (censored) - hence a difference in observed and predicted could be due to censoring rather than bad prediction. For example, using your values above, I may plot the point (1.70, 25.0). The 1.70 is censored, and the 25.0 is the predicted value. There is a big difference between these values, but we don't know if that difference is just because of the censoring or if our prediction is just really off.

My advice would be to focus on finding a good model that maximizes the out-of-sample likelihood or AIC (Why likelihood? It's a good measure of survival fit).


What is local-NMF? How is it better than original NMF?

The "parts-based" is explicitly stated in the text to be a synonym to "non-subtractive". It is a natural feature of NMF. Its underlying assumption is that all elements of a source matrix are non-negative, as well as that the source matrix is a product of two other matrices, all elements of which are non-negative. Thus the source matrix is a sum of independent "parts". If the parts were allowed to be negative, the problem would lose capacity for unique solutions, since then, for example one could generate out of thin air fake components that are equal and opposite.

The "local" refers to spatial locality. Especially in image processing, frequently each component is expected to be concentrated within a small region, such that all related pixels are not too far from each other. This is not achieved by naive implementation of NMF, which results in components "smeared" across the entire domain, being obviously non-local.


Can I specify the root node splitting feature in XGBoost?

I do not have the whole answer, but I believe that xgboost allows warm-starting, which implies starting from a provided point.

xgb.train(…,xgb_model=model_to_start_from)


How to deploy machine learning models as a chrome extension?

Since what you want to do is to apply a pre-trained model and use the predictions on the client side, you have two options:

Client-side application of the model: the extension would have to download the model file and store it, then it would need to compute the predictions from this model. This would probably require the SVM library used for the training to be available on the client side, this might be an issue.
Server-side application of the model: the extension would have to upload the data to the server, then the server computes the predictions using the model, and sends the predictions back to the client.

Usually option 1 is preferable since it doesn't require transfering the input data to the server and then sending back the predictions. It's also safer with respect to the user privacy, their data is not uploaded anywhere. However it requires the client to be able to compute the predictions, so probably requiring the SVM library to be installed and the extension must be able to communicate with it, so it might not be feasible.


Trying to change pandas column dtype from str to float

In pandas the object type is used when there is not a clear distinction between the types stored in the column.

So, I guess that in your column, some objects are float type and some objects are str type. Or maybe, you are also dealing with NaN objects, NaN objects are float objects.

a) Convert the column to string: Are you getting your DataFrame from a CSV or XLS format file? Then at the moment of reading the file, you can specify that that column is an str type or just make the type conversion of the column you are dealing with.

b) After that, you can apply the string changes and/or deal with the NaN objects.

c) Finally, you transform your column into float type`.


What is the best way to classify data not belonging to set of classes?

Just a proposal of a method to try out.

Stage $1$: Use one-class SVM to assign those images that do not belong to the set of predefined classes as the $9$-th class.

Stage $2$: For those images that passes through your filter, let the multi-class SVM assign them to one of the $8$ classes.


Unsupervised Anomaly Detection in Images

There seems to be a clear line between samples with anomalies and those without. Label them and run a classifier? e.g SVM. One shouldn't use more complex models than needed.

Also, be a bit careful with trying to detect all kind of anomalies. It's not always realistic and might lead you away from more straightforward, and perhaps better solutions.

You could also try something as simple as a histogram from one part of the image. Looks like it should be able to find the anomalies.


Why the results from Matlab curve fitting function differ when using custom equation?

I think your value of $y$ is too small. So, when the sum of squares is calculated it is already below the threshold, and the coefficients are accepted as good ones.

And the power function is fitted differently. $log(y) = log(a) + x*log(b)$ is a linear regression that has a direct formula. While when you input it manually it is something close to gradient decent to minimize sum is squares.

I do not know Matlab well, but you can multiply $y$ by $10^6$ and fit the curve. Then the real $a$ coefficient is $a/10^6$. Or try clicking fit options button and search the threshold value there.


Feature Scaling both training and test data

Generally speaking, best practice is to use only the training set to figure out how to scale / normalize, then blindly apply the same transform to the test set.

For example, say you're going to normalize the data by removing the mean and dividing out the variance. If you use the whole dataset to figure out the feature mean and variance, you're using knowledge about the distribution of the test set to set the scale of the training set - 'leaking' information.

The right way to do this is to use only the training set to calculate the mean and variance, normalize the training set, and then at test time, use that same (training) mean and variance to normalize the test set.

As for the point in your question, imagine using the training mean and variance to scale the training set and test mean and variance to scale the test set. Then, for example, a single test example with a value of 1.0 in a particular feature would have a different original value than a training example with a value of 1.0 (because they were scaled differently), but would be treated identically by the model.  This is where the bias would come from.

Hope this helps!


One-Dimensional Convolutional Neural Network

is it fixed 1-D filter within a specific time interval?

Yes. The same as filters in 2D. Adjacent filters may even have no overlap with each other.

1D CNN is almost the same as 2D CNN both mathematically and visually by setting the second dimension (either the horizontal or vertical one in visualizations) to 1. This way, 1D filters are placed (possibly with some overlap) in one dimension instead of 2D filters being spread in two dimensions.

The below image shows a filter set with shared parameter $W$ covering the overlapping regions of the input.

By shared parameter we mean $f_i=\mbox{ReLU}(\mbox{sum}(W \odot  \mbox{region}_i))$, where $\odot$ is a point-wise product between a region of input and $W$.


Which phrase should be returned in case of multiple matches when comparing text?

This is usually done by carefully choosing two things:

The sentence representation. Word count is the most simple option but there can be many others: TFIDF weights, with/without removing stop words, with/without lemmatization, etc. In a DL approach the sentence would be represented as a sentence embedding.
The similarity measure between two sentences. Again there are many options, in BoW approaches the standard ones would include counting words in common (e.g. Jaccard) and cosine TFIDF.

So the answer is: it depends on the similarity score. A complex similarity score like cosine TFIDF rarely produces ties so the highest score can be selected. More simple methods give ties, and then the logical answer is to return all the tied sentences.


Changing colors on Orange Scatterplot widget?

I just realised I'd missed the 'colour' widget. By connecting the data table to the 'colour' widget, I could set what colour I wanted different data sets to be. Then I could attach the colour widget to my scatter plot, and BAM, the colours I wanted. I needed to make sure the scatter plot widget was connected to both the colour widget and the data table. There may be a more streamlined way to do it, but that worked.

(Posting my own answer rather than deleting, in case anyone else has similar issues).


Train LSTM model with multiple time series

As you mention, all rooms have similar properties, thus it makes sense developing a single LSTM model for all of them.

You have to pre-process your data so that you have input vectors $X_i$ of 7 time steps and 1 time step as label vector $Y_i$:

$
X_i = [x_i[t-6 ],x_i[t-5 ],x_i[t-4 ],x_i[t-3 ],x_i[t-2],x_i[t-1 ],x_i[t]]\\
Y_i = [x_i[t+1]]
$

where the subscript $i$ denotes the $i_{th}$ vector.

Let's imagine you can generate 100 vectors for each room, you'll end up with a dataset of $10^4$ input vectors. The number of vector per room will depend on the overlap you use, for instance, if you have data for 100 days per each room and you use an overlap of 99 days (your window slides one day in the future), you'll have 99 vectors for each room.

To train the LSTM you use the typical Mini-batch training. Make sure you don't propagate the state for batch sample $i$ to sample $i+1$ in order to treat them individually (in Keras you set the stateful flag to False)


Why does prediction by a consensus of classifier work better than prediction by a single classifiers?

For a very simple example, imagine you have three independent classifiers that each have 60% accuracy. If you use any one of them to classify a random sample, you have a 60% chance of getting it right.

Now use an ensemble classifier that takes the majority vote. Below are the probabilities that exactly N of the individual classifiers are correct.

0 correct - 6.4%
1 correct - 28.8%
2 correct - 43.2%
3 correct - 21.6%

Your ensemble classifier has the correct output when 2 or 3 of the component classifiers are correct, which happens 64.8% of the time. Your ensemble classifier outperforms each of the individual component classifiers! You'll get this advantage by combining any classifiers that perform better than random.


Convert filters pre-trained with ImageNet to grayscale?

I doubt either of your proposed approaches will work, these convolutional layers learn all kind of relationships between the colors which will be non-existent in grayscale, upsampling it to RGB makes your input space very different from the input space that ImageNet was trained on and averaging these filters of the trained ImageNet model makes a whole lot of assumptions that are unlikely to hold. The second approach does seem more promising than the first, so if you want to test an approach I would go for the second one. It's possible that it is at least a better initialization than randomly. I think you will end up needing a lot of data and computation anyway, so it might be better to start off with your own network shape and start from scratch.


Why are there currently no content-based evaluation metrics for information retrieval?

Using a word-based metric would explicitly favor word-level retrieval methods.  The theory is that (just as you suggest with dwell time), the URL-level metric measures more directly the desired result.

More concretely, consider a search of "alcohol from potatoes." Assume we have two pages:

1) A page that is simply a grocery list (containing "alcohol" and "potatoes")
2) A page that describes vodka or moonshine which only uses the words "ethanol"
   and "potatoes" but not "alcohol."

A word-level metric would prefer the first result, since it contains both alcohol and potatoes..  A URL-level metric would likely label the second result as more relevant, which would allow a model to form associations beyond simple word-matching (i.e. it could learn that "alcohol" and "vodka" are associated).


What is the difference between handcrafted and learned features

In short

The classification is the following:

handcrafted are features that are manually engineered by the data scientist.
learned features are ones that are automatically obtained from a machine learning algorithm

Let me give you an example:

Suppose you are doing an image classification task, where you wanted to classify cats from dogs. You want to build a classifier but you are faced with the dilemma, how do I input my data to the classifier?. You have two options:

Use the raw pixel data. The issue with this is that you have a vast feature space, which makes it hard for models to generalize.
Attempt to extract features from the image so that you can reduce your feature space.

Now if you choose the second you have two more options:

Manually define a set of features and extract them. Some examples include edge detection, corner detection, histrograms, etc. The problem with this approach is that nothing guarantees that the number of corners is a good descriptor for classifying cat and dog images.
The alternative is to train a ML model to identify and extract useful features for this specific classification task. This is exactly what a Convolutional Neural Network does. It searches for what features are best to classify the images?.

Traditionally the first approach was used extensively in Machine Learning. That changed however with the arrival of Deep Learning.

An example of what features CNNs identify when classifying different types of images:

Note that these features aren't general, low-level features like edges or corners; instead they are tailored for each class. That is the power of training a model to extract features.

Comparison

Because learned features are extracted automatically to solve a specific task, they are extremely effective at it. In fact deep learning models that perform feature extraction and classification outperform models that classify manually extracted features by a large margin. This is one of the reasons why deep learning is so popular.

On the other hand, we have no control on what features the model will extract from the data. In many cases these features are only good for classifying the data and have no real-world interpretation. They are only good for the task that they were trained for.


The best will be to have domain knowledge and understand how this feature affects the target and what will be the best way to encode it:
First way: treat them all as a categorical feature with high cardinality.
Second way: split them into two columns, one categorical and one numerical. And then treat them separately.
Without domain knowledge in this case I cant think of on something better. Answering this question would help: Is the number really a number? Why can I have numbers and strings in the same column? How will this information might help my model?


What if there is huge difference between train data and test data?

To me it seems like you are training on the wrong data. The results of your algorithms will only be as good as the data you train it on. First question you should ask yourself is, if it is possible to transform or update your dataset to make it more realistic. Without deeper insight into the nature of the problem, it is, unfortunately, very hard to help in this regard. Still, in my honest opinion, this will do for the cleanest solution.

If you can not update your training data to represent the real-world distribution, your problem is quite similar to trying to transfer simulation data to real-world applications, like often seen in robotics. Usually, the authors try to learn the transformation between the simulation and real-world data explicitly like shown in this nice paper about optimization. Briefly summarized, the authors model the data by two Gaussian Processes (GPs), one resembling simulation data, whereas the other is the transformation I mentioned above. Thus, your real world data would result as the sum of the two kernels from the GPs. I am sure a similar approach is applicable for your problem, even without using GPs as you (most likely) are not interested in Bayesian Optimization.

Still, if you want go for this method, you will also need to gather some more data from your real distribution, just not a whole new dataset.


Adding extra (meaningful) features does not improve model performance

I tried with different features: in one dataset with less features engineering, i.e., using only features from Text, I got a maximum value of F1-score equal to 68%. With more features, that I thought to be significant for improving the model, I am getting max 64%, that is weird considering the problem (email classification for spam detection).

Typically this would happen if the model is overfit: not enough data and/or too many features make the model pick patterns which happen by chance in the training data.
Usually with text one has to remove the least frequent words in order to avoid overfitting. You might also want to check the additional features, remove anything which happens too rarely.

Also, the confusion matrix, has given me weird outputs

       0      1
0    [[2036  161]
1    [   1 2196]]

Observations:

True class 0 has 2036+161 = 2197 instances, true class 1 has 1+2196=2197 instances. These results are obtained with the resampled data.
Assuming class 1 is positive: 2196 True Positives (TP), 2036 TN, 161 FP (true positive predicted as negative) and 1 FN (true negative predicted as positive).

recall = 0.999, precision = 0.932. That's an f1-score somewhere higher than 0.95 (probably due to the resampled data).

The second confusion matrix is also clearly obtained with the resampled data, and it shows perfect performance (F1-score is 1).

These matrices show the performance obtained on the resampled data, so it's similar to the performance on the training data. Since the performance on a real test set is much lower, this confirms strong overfitting.


Multi target regression when some targets have correct NA labels

One option is to modify your network to output a binary label for each body part, signalling whether the part is visible or not.

You would need the following changes:

Modify your network architecture to return a real number in the range $[0, 1]$ for each body part. A sigmoid would be an appropriate activation for the last layer. Its semantics would be the probability that the $i^{th}$ body part is visible.
Extend your training and test data with appropriate values for the new labels.
Change your loss to only take into account the difference between body part positions (expected vs predicted) if the part was expected to be found (e.g. by multiplying). I'm not very sure whether you should use the real labels or the predicted ones here (or even an average of both).
Add a new term (added to the original loss) to your loss with the cross entropy of the body part presence flag. You should probably add a weighting factor $\alpha$ to this new loss term to have a proper scale with respect to the position distance loss term.


Understanding Weighted learning in Ensemble Classifiers

Adaboost is an Ensemble Classifier, which means that it consists of several weak classifiers that are able to consider weights in the training set. These weights can be used to inform the training of each weak classifier. For example, decision trees can be grown in a way that favor splitting sets of samples with higher weights.


Detecting Different Distributions in data

It's not really feasible to label points as coming from distribution A or B if there is be overlap amongst the distributions. You can't know whether the point fell near the mean of A or the tail of B. In the simplest case, consider two normally distributed datasets with overlap - how can you know which generated it?

You could estimate the probability of each point falling under A or B, but that doesn't really solve the problem.

A different approach, but one I'm not sure will help, would be to fit a new distribution defined as the convolution of A and B. The ex-gaussian distribution is one such example. But you'd have to do the maths to figure out the PDF equation, and that still won't necessarily label your points.


I don't know if it's the best way, but I'd write a function and apply it to the df['size']:
def rename_size(size):
    if size not in ['M','S','L']:
        return np.nan
    else:
        return size
df['size']= df['size'].apply(rename_size)


What algorithmic approach for selecting similar, relevancy based documents

Check out term frequency-inverse document frequency (TF-IDF), if you're parsing large chunks of text for mentioned topics. The metric measures importance of topics bounded by how often they appear across a corpus of documents, which helps to weed out topics that may be very common, but are very common to all discussions, and therefore not very useful.

It's a simple calculation, but there is a ruby gem that calculates it directly: tf-idf-similarity.

Hope this helps.


SPARK, ML: Naive Bayes classifier often assigns 1 as probability prediction

The reason NB is called "Naive" is that is makes the assumption that the predictive variables are all independent.
This assumption usually skews the model scores (which, under the above naive assumption are unbiased probability estimates) towards 0 or 1.

In your case, e.g., the presence of words flower and petal indicate gardening category, but, because the presence of these words is not independent (if one is present, the other is likely to be present too), the model will over-value their appearance. Taking the extreme case, if words A and B appear only together, then

P(Category=X | A & B) = P(Category=X | B) = P(Category=X | A)

and thus one should multiply by the odds ratio of A&B once, not twice, as the Naive Bayes algorithm requires.

Your remedy is to use calibration (a separate model which maps model scores to probabilities).


Peformance evaluation of ranking algorithms

Kaggle's famous competition Chess ratings - Elo versus the Rest of the World, that aimed "to discover whether other approaches can predict the outcome of chess games more accurately than the workhorse Elo rating system", used this structure

  Competitors train their rating systems using a training dataset of over 65,000 recent results for 8,631 top players. Participants then use their method to predict the outcome of a further 7,809 games

A similar structure - starting from a complete dataset, using first part
 for training and last part to check the outcome - could be useful to measure the performance of the ranking algorithms.


Can you combine two xgboost models into one?

No: the trees' results are added together to produce the final score, so combining two models would produce outputs roughly twice as large as desired.  (Gradient boosted trees change the target labels being fitted by each tree, so the 101st tree has "reset" the targets when training.)


Why does bidirectional LSTM have half the parameter count compared to LSTM in keras?

For Unidirectional LSTM the number of parameters are
4*[(numHiddenUnit+inputSize)*numHuddenUnits+numHuddenUnits]
where 4 is for 4 LSTM gate equations.
For your case numHuddenUnits = 256, inputSize is 28 gives the result 291840
For birectional LSTM the number of parameters are
2 * 4 * [(numHiddenUnit+inputSize)*numHuddenUnits+numHuddenUnits]
where 2 is due to bi-directional weights and 4 is for four gate equations
For your case numHuddenUnits = 128, inputSize = 28 gives the result 160768


How to read the output of Binary cross entropy?

Binary cross-entropy is a simplification of the cross-entropy loss function applied to cases where there are only two output classes. Essentially it can be boiled down to the negative log of the probability associated with your true class label.

Cross entropy

Cross entropy is defined as

$L = -\sum ylog(p)$

where $y$ is the binary class label, 1 if the correct class 0 otherwise. And $p$ is the probability of each class.

Let's look at an example, if for an instance $X$ the output label is 0 and your model output was $[0.7, 0.3]$. Then we can see that the loss function using binary cross entropy is

$L = -(1*log(0.7) + 0*log(0.3)) = 0.155$

Notice how the loss is only affected by the true label's prediction probability.

Binary cross-entrop

Binary cross entropy is for an example with 2 output classes and can be simplified as

$L = (ylog(p) + (1-y)log(1-p)) = (1*log(0.7) + (1-1)log(1-0.7)) = 0.155$

Your example

For your example, $[1,0,0,0,0]$ and predictions $[0.1 0.5 0.1 0.1 0.2]$ the loss is

$L = -(1*log(0.1) + 0*log(0.5) + 0*log(0.1) + 0*log(0.1) + 0*log(0.2)) = 1$


As u said that one has to train a model in order to calculate Hellinger distance. I am not sure that which model u are talking about but for now I assume that u might have thinking about the latent model so according to me there is not mandatory to train a latent model to produce a document vector. one can create a document vector from the directly word-doc matrix and can calculate Hellinger distance. One simple trick is that I used is first calculate a bag of word representation after getting doc-word matrix by gensim utilities. kindly review it and let me know your opinion.


Handle 50,000 classes in OneVsRestClassifier

As you already know, the main culprit here is the large number of classes (50k). For every sample data, you have a label of size 50k. Even if a sample only belongs to a single class, the label will still be of size 50k.

For example, sample1 has a label of A and B while sample2 has a label of A. Sample3 on the other hand has C for its label and sample4 has A. We can visualize it like this:

sample1 - [1, 1, 0, 0, 0, 0, ..., 0]
sample2 - [1, 0, 0, 0, 0, 0, ..., 0]
sample3 - [0, 0, 1, 0, 0, 0, ..., 0]
sample4 - [1, 0, 0, 0, 0, 0, ..., 0]

One solution here is to use Siamese neural network. This takes in two input at a time. Instead of training directly to learn classes as the output, we are training to learn the similarity between two samples. We first select random pairs. We use the label 1 if two samples belong to the same class; else 0.

sample1 & sample2 - 1
sample1 & sample3 - 0
sample2 & sample4 - 1
sample3 & sample4 - 0

This way, even if there is a lot of classes, you are only training with 2 labels, 1 or 0, thus, avoiding memory error.

The problem now is that you are doing a multi-classification task. For instance, sample1 and sample 2 are the same class A but sample1 also belongs to class B. My suggestion, but I'm not totally sure here, is to add multiple instance of that pair.

sample1 & sample2 - 1
sample1 & sample2 - 0
sample1 & sample3 - 0
sample2 & sample4 - 1
sample3 & sample4 - 0

As for the large number of training samples (1 million), this can be handled by using small batches during training.


PyTorch is too heavy for deployment, is there a light weight pytorch for running neural networks?

One thing to try is to create a lean version of Pytorch that only has what you need. I did something similar for an AWS Lambda layer, here's what I was able to delete (this was for Pytorch 1.1, things may have changed since):

find . -type d -name "tests" -exec rm -rf {} +
find . -type d -name "__pycache__" -exec rm -rf {} +
rm -rf ./{caffe2,wheel,pkg_resources,boto*,aws*,pip,pipenv,setuptools}
rm ./torch/lib/libtorch.so
rm -rf ./torch/test

If that's not enough, you need to look into using ONNX or TorchScript/torch.jit. ONNX exports your model to a binary protobuf file. TorchScript exports it to a serialized form that can be run in a C++ environment.


You need to use appropriate activations. If you were using softmax for those two components, they'd be constrained to sum to one. LeakyReLU not only doesn't impose this constraint: it can output any real value, so it isn't even constrained to [0,1] for those components individually.


Different learning rate for each of the layers?

In trivial update rules like gradient descent, the learning rate is important and it somehow specifies the speed you go downhill. In popular papers like Adam optimisation technique, and in non-paperised(!) popular solution namely RMSProp the authors cared that the slope of different features may vary differently and in a direction you may need to go faster due to its slope. Consequently, They decided to set the learning rate and update each parameter based on its own slope and this learning rate is somehow affected by the slope of each direction independently to the other dimensions. The motivation is this. As far as I know, you just need to set the learning rate for your optimisation and it will be adapted by itself.


At first glance, your conclusion appears correct, but there are some important caveats to keep in mind.

First, what are the sizes of your training and validation sets?  If your validation set is too small, then the observed difference may not be statistically significant.

Second, you should verify that your validation set is a representative sample.  (i.e. it should come from the same distribution as the training set).  If it's not representative, then it may give poor estimates of performance.

Third, when tuning hyperparameters, it's a good idea to split your dataset into three shards - training, validation, and testing.  You can use the training and validation sets to find optimal hyperparameters (as you have done), then use the testing set to generate a performance estimate for the tuned model.  If you trust the validation accuracy obtained during hyperparameter tuning, you are liable to a subtle form of overfitting where hyperparameters are specialized for the validation set.

Finally, if you have the computational resources, then it's always a good idea to evaluate accuracy with cross-validation rather than a train-test split.  This will give you a more robust estimate of accuracy.

If you've checked all these boxes, then you have good reason to believe that 500 estimators is better than 100 estimators!

  [S]hould I change the validation data few times and see if similar kinds of increment happens by shifting from 100 to 500 estimators?

Yes, it's always a good idea to try many different configurations of hyperparameters.  You can use scikit-learn's GridSearchCV or RandomizedSearchCV to easily run a search over the hyperparameter space.


How can I find contextually related words and classify into custom tags/labels?

You are working with word classification so you really don't have any contextual information that you can leverage. So the best thing would be to use freely available contextual information called as word2vec. I would suggest using the pretrained GLOVE Word2Vec as they are much cleaner than the Google ones. I am going to suggest two approaches for doing this:

Naive Approach

Load the pretrained word vectors.
For every new word calculate cosine similarity with every item in your training list.
Predict the label for the word which is closest to your new word.
Alternatively you can calculate cosine similarity with the labels and just predict the label which comes the closest.

Better Approach

Load the pretrained word vectors.
Extract the vectors for every item in the training set.
One hot encode the labels.
Train a neural net or any multiclass classification algorithm using the vectors as features and save the model.
For every new word extract the vector and run the model on this vector.
The predicted label is your output.

You can use the first approach to run quick tests and check the efficacy of the approach and then move on to the second one if you found it fit to your use.


Does it make sense to label a dataset manually?

The chicken egg dilemma

What came first: the labeled data or the machine learning model?

If you have labeled data, then you can train a machine learning model.
If you have a trained machine learning model, then you can label data.

Precision vs. Recall

Suppose you are in case 2 (you have a model). Then how is the precision and recall of your model? Unless both are $100\%$, you might still want to label data.

                       recall
                 100%     |     < 100%
          --------------------------------------
          Your model is   | You can confidently
p   100%  perfect. You    | trust the labels you
r         don't need more | get. But your model
e         data.           | might miss some.
c         --------------------------------------
i         You get         | You get some
s  < 100% labels, but you | labels, but you
i         still need to   | still need to
o         confirm that    | confirm that
n         they are correct| they are correct

Most people will be at the lower-right quadrant (precision and recall will be $\lt100\%$). The goal will be to get as close as possible to $100\%$. Therefore, the model will output labels which can be the basis for manual labeling.

Your case

If you can label all rows with some rules and you are creating these rules, then, in fact, you are already manually labeling data. Further, you could use a decision tree later on, to model all your rules in order to deal with new data points that come your way.


What advantage does Guassian kernel have than any other kernels, such as linear kernel, polynomial kernel and so on?

The advantange is that it is related to euclidean distance, and if your problem benefits from this type of feature, e.g. you have a highly non-linear problem where values nearby some vector point tend to be of a different class, it will be able to find relationships that other kernels would not, and create highly complex decision boundaries.


Stanford NER is not properly extracting percentages

Looking at your code, I dont believe there is any mistake in it. The Stanford NER tagger is modelled by a Conditional Random Field Classifier and is hence bound to make mistakes. If you are able to check for a large number of samples and calculate an accuracy metric on the basis of that you might get a good idea if the pre-trained model is useful for you. Else, I suggest that you train your own model using. The stanford NLP group provide an implementation for training, you just have to give it the data in the required format. Look at this blog to get a good idea on how to do it.


It seems there is a small bit of ambiguity in the question. But my best guess is that you are looking for stratifiedkfold (Emre has mentioned that in a comment) or second best guess is labelkfold.
See: http://scikit-learn.org/0.17/modules/generated/sklearn.cross_validation.StratifiedKFold.html#sklearn.cross_validation.StratifiedKFold
and
http://scikit-learn.org/0.17/modules/generated/sklearn.cross_validation.LabelKFold.html#sklearn.cross_validation.LabelKFold

It would help if you specify which inbuilt cross_validation in sklearn you tried, since there are several to choose from:
http://scikit-learn.org/0.17/modules/classes.html#module-sklearn.cross_validation

Additionally, please also let us know about the number of data points available, since that will contribute to deciding on the numeric value of k. The point being that you don't want to bias your training due to an counter-productively high value of k for the available n data points. Herein lies the reason for the labelkfold approach since it will ensure that the same data point does not repeatedly occur in multiple folds, which would end up biasing your classifier, and we want to avoid this.

Finally, if your training time is of the order of a few minutes, you could try several cross_validation schemes. It will be a good learning experience and might provide some useful insights into the data :-)

--
Since, you are exploring the Iris dataset, I've gone ahead and created a small demo of the effect of different cross validation methods on classifier evaluation. I've put in a small range for relevant parameters for each of the methods considered.

I've included my code for you to experiment with, you can try more cross-validation methods. Iris data set has small set of instances and there isn't too much 'craziness' in the distribution of the classes. If you tried it on some real-world data with a lot of bias and noise, you might see a more pronounced effect of cross-validation (use error-bars).

--

    """
Evaluate cross-validation schemes for Iris dataset
"""

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn import svm
from sklearn import datasets
from sklearn.model_selection import train_test_split
from time import time
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import StratifiedKFold
from sklearn.model_selection import RepeatedStratifiedKFold

iris = datasets.load_iris()
X, y = iris.data, iris.target

""" visually inspect iris dataset """
iris_sns = sns.load_dataset('iris')
g = sns.pairplot(iris_sns, hue='species', markers=["o", "s", "D"])
plt.savefig(filename='iris_pairplot.png')
""" Dimensions of data matrix and target labels """
print(X.shape, y.shape)

"""
cross-validation: the apparent silver bullet to over-fitting
The available data and label matrices are split into set of training and validation matrices
We first begin with mere evaluation of classifier trained on different random sets
of training and validation data sampled from the entire available data.
By averaging the computed classification performance on each training subset,
we will acquire a statistically reliable estimate of how this classifier might perform on unseen test data.
Subsequently, we will train classifier on multiple subsets of training data to actually improve the
learnt classifier itself.
"""

""" The effect of train-test split sizes"""
test_sizes = [0.1 * i for i in range(1, 10)]
scores = []
for test_size in test_sizes:
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=test_size, random_state=round(time()))
    print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)
    clf = svm.SVC(kernel='linear', C=0.1).fit(X_train, y_train)
    scores.append(clf.score(X_test, y_test))

fig1 = plt.figure(1)
plt.plot(test_sizes, scores, 'k-d')
plt.title('Iris dataset, effect of proportion of train-test splits')
plt.xlabel('proportion of test-images')
plt.ylabel('classification score')
plt.savefig('train-test-splits.png')
""" The effect of number of cross-validations """
cvs = [i for i in range(2, 11)]
cv_scores_means = []
cv_scores_std = []
for cv in cvs:
    clf = svm.SVC(kernel='linear', C=0.1, random_state=round(time()))
    scores = cross_val_score(clf, X, y, cv=cv)
    cv_scores_means.append(scores.mean())
    cv_scores_std.append(scores.std())

fig2 = plt.figure(2)
plt.errorbar(cvs, cv_scores_means, yerr=cv_scores_std, fmt='k-o')
plt.title('Effect of number of cross-validations')
plt.xlabel('number of cross-validation')
plt.ylabel('mean classification score')
plt.savefig('number-of-cv.png')
""" The effect of k-stratified splits"""
num_splits = [i for i in range(2,11)]
scores_splits = []
for n_splits in num_splits:
    skf = StratifiedKFold(n_splits=n_splits)
    clf = svm.SVC(kernel='linear', C=0.1)
    scores = []
    for train, test in skf.split(X, y):
        clf.fit(X[train], y[train])
        scores.append(clf.score(X[test], y[test]))
    scores_splits.append(np.average(scores))

fig3 = plt.figure(3)
plt.plot(num_splits, scores_splits, 'k-s')
plt.title('Effect of number of stratified splits')
plt.xlabel('number of splits')
plt.ylabel('mean classification score')
plt.savefig('num-stratified-splits.png')
""" the effect of k-label splits"""
num_folds = [i for i in range(2, 11)]
num_repeats = [i for i in range(2, 10, 2)]
line_colors = ['r', 'g', 'b', 'k', 'y']
scores_repeats = []
for n_repeat in num_repeats:
    scores_folds = []
    for n_fold in num_folds:
        rskf = RepeatedStratifiedKFold(n_splits=n_fold, n_repeats=n_repeat, random_state=round(time()))
        clf = svm.SVC(kernel='linear', C=0.1)
        scores = []
        for train, test in rskf.split(X, y):
            clf.fit(X[train], y[train])
            scores.append(clf.score(X[test], y[test]))
        scores_folds.append(np.average(scores))
    scores_repeats.append(scores_folds)
fig4 = plt.figure(4)
handles = []
for i in range(len(num_repeats)):
    h, = plt.plot(num_folds, scores_repeats[i], line_colors[i]+'-x', label="#rep: "+str(num_repeats[i]))
    handles.append(h)
plt.title('Effect of number of splits and repeats for repeated stratified k fold')
plt.xlabel('number of splits')
plt.ylabel('mean classification score')
plt.legend(handles=[h for h in handles])
plt.savefig('num-split-repeat-stratified.png')
plt.show()


Optimising for Brier objective function directly gives worse Brier score than optimising with custom objective - what does it tell me?

Brier score has known shortcomings for very rare or very frequent events.

Binary logistic regression objective function is relatively more robust than the Brier score to the rate of rare or frequent events.

It is possible that the difference in performance between the two objective functions is due to the event frequency in the data set.


Is it better to use a MinMax or a Log Return normalization to predict stock price movements?

Log returns are symmetric compared to percentage change. Log(a/b)=log(b/a) and this (less skewness), in theory, leads to better results for most models (linear regression, neural networks).
Neural networks like lstm work better if the values are close to zero, but the difference in normalizations is usually not that big.
Any returns (log or percentage) are better than raw values because prices change according to previous prices. Their absolute (raw) values have almost negligible influence compared to previous price.

I would recommend first to convert to log returns and then normalize. If it is daily prices then I would divide the log returns by something like 0.05. Price changes have very heavy tails in a distribution so I would not suggest using minmax because then you divide by something like 0.5 (which probably was in great depression) and get all values too close to zero. Dividing by standard deviation should also be good.

But reality is different than theory, so it is better to benchmark. Maybe percentage changes are better because this is the number people see and react to. And markets are a lot about psychology.

And be prepared to see very high errors and bad models. Financial markets are badly predictable both in practice and theory. According to economic theory if they were predictable and people are rational and have unlimited credit lines then any possibility of earning additional money compared to the whole market will be closed in milliseconds. Only if you find some way to analyze data that noone is currently using only then will you be able to earn money. Neural networks were discussed in 1990s to predict financial markets. So LSTM is not really new in 2018.


Discovering important topics in corpus of text using metadata and text content

There is no way to keep a supervised model update as the data distribution changes over time without labeling new instances.


What exactly is the "hyperbolic" tanh function used in the context of activation functions?

The hyperbolic trig functions follow the equation for a Rectangular hyperbola, which is something you should be familiar from analytical geometry.  The recentgular hyperbola is defined by $x^2 - y^2 = 1$ and if you let $x = cosh(t)$ and $y = sinh(t)$ and plug it into the rectangular hyperbola equation you can verify this fact quite easily.

From this idea you can derive the rest of the hyperbolic trig functions.  (Also they reason they are given trig names is because they resemble the properties trig functions have).


How to interpret this Plot of Model Loss from a BiLSTM model?

It looks like your train/val loss curves have a very large generalisation gap, which suggests that your model is overfitting. THis simply means it does a great job making predictions for the training set but a terrible one for your validation set. This appears to be the case even in early epochs, since valid loss appears to never improve.
I see you have shuffle set to False. Is that related to shuffling datapoints in the batches? The unfortunate behaviour in training may as well trace back to the train and validation sets being very different. I suggest

stratified train/val split
QA your train and val sets (e.g. class ratio in each set)
shuffle datapoints in your batches


LightGBM vs Sklearn LightGBM- Mistake in Implementation- Exact same parameters giving different results

A kind soul from Github was able to answer this.
The answer is this:
for the parameters consistency problem, you can try to use a new lgb_train before lgb.train, like
lgb_train = lgb.Dataset(x_train, y_train)
best_gbm = lgb.train(params=best, train_set=lgb_train, num_boost_round=num_boost_round)

lgb_train is lazy-inited, and only inited one-time, so it will be constructed in the cv part. And some parameters (like min_child_samples) in that part may change the lgb_train. Therefore, lgb_train may is inited by different parameters. (So it is better to use a new lgb.train in cv part as well.)


Is there any tool for data visualization and manipulation?

What language are you using? A quick Google search of {lang} visualization can help you with this.

For example, Python has Bokeh, Pandas, and Seaborn. Bokeh is for interactive visualizations which can help with digging deeper into the visualizations. See:

Are there any python based data visualization toolkit?


Reducing noisy data from non normal distribution of data with std deviation?

First, good practice to raise validity concerns here when removing outliers and/or filtering data. This may have strong affects of validity of results. An intro is here: When to remove outlier in preparing features for machine learning algorithm .

Second, is it possible to address the small dataset problem -- Can you collect more data? Redefine the population to produce more data? Use another data set that is similar in developing the model?

Lastly, this seems to be a filter problem, to get started on a solution, check MATLAB documentation for filters.

If the results are going to be used anywhere, probably a good idea to document all of your decisions and the first two concerns. Absent much more detail, experience says there is a pretty high risk to any conclusions based on this model.


How to convert nested list into a single list in python?

You are getting the error because of string value contains in the nested list.
You need to handle string value as well.

You can fix the error by following codes:

from collections import Iterable

def single_list(list,ignore_types=(str)):
for item in list:
    if isinstance(item, Iterable) and not isinstance(item, ignore_types):
        yield from single_list(item,ignore_types=(str))
    else:
        yield item

Item_list = [10,20,[30,40],[50,'Null',70],100]
items_single=single_list(Item_list)
for item in items_single:
    print(item)


My NN has less than 1% accuracy rate, what's wrong with my model/dataset?

You note that y_train.values[0] returns 106.0 - is that correct? Sigmoid activation binds the output to between 0 and 1, so it's not possible for it to predict a 106.

Try changing the activation for your final layer to relu too.

You should also switch to a more regression-appropriate metric like mean_squared_error or R2. Accuracy is more of a classification thing.


What do I do when my column has 50% data missing?

It depends on your knowledge of the problem. First, you should classify why is it missing??

Structurally missing data
Structurally missing data is data that is missing for a logical reason. In other words, it is data that is missing because it should not exist.check this

Structurally missing data is data that is missing for a logical reason. In other words, it is data that is missing because it should not exist.
Missing completely at random (MCAR) No pattern in missing data
Missing at random (MAR)An alternative assumption, known somewhat confusingly as missing at random (MAR), instead assumes that we can predict the value that is missing based on the other data.

Then choose your imputing technique based on your knowledge of the problem if you have other cases you can try (from this source):

Encode NAs as -1 or -9999. This works reasonably well for numerical features that are predominantly positive in value, and for tree-based models in general. This used to be a more common method in the past when the out-of-the box machine learning libraries and algorithms were not very adept at working with missing data.
Casewise deletion of missing data. Here you simply drop all cases/rows from the dataset that contain missing values. In the case of a very large dataset with very few missing values, this approach could potentially work really well. However, if the missing values are in cases that are also otherwise statistically distinct, this method may seriously skew the predictive model for which this data is used. Another major problem with this approach is that it will be unable to process any future data that contains missing values. If your predictive model is designed for production, this could create serious issues in deployment.
Replace missing values with the mean/median value of the feature in which they occur. This works for numerical features. The choice of median/mean is often related to the form of distribution that the data has. For imbalanced data, the median may be more appropriate, while for symmetrical and more normally distributed data, the mean could be a better choice.
Label encode NAs as another level of a categorical variable. This works with tree-based models and other models if the feature can be numerically transformed (one-hot encoding, frequency encoding, etc.). This technique does not work well with logistic regression.
Run predictive models that impute the missing data. This should be done in conjunction with some kind of cross-validation scheme in order to avoid leakage. This can be very effective and can help with the final model.
Use the number of missing values in a given row to create a new engineered feature. As mentioned above, missing data can often have lots of useful signal in its own right, and this is a good way to encode that information.


How to make custom callback in keras to generate sample image in VAE training?

I think the two following links could help you 1, 2.

The first one is a tutorial, which introduces you how to display images in TensorBoard.
If you look at the part on the confusion matrix, you should find a way to make your desired callback, i.e. a callback which involves displaying an image.

The second link is also a tutorial that shows you how to write your very own custom callback in TensorFlow with Keras.

I think by combining both tutorial you should have a pretty good idea how to do what you want.


Handling actions with delayed effect (Reinforcement learning)

I thought of Experience and Replay as a mechanism to handle this delayed effect.

Experience Replay will not help directly here, it is a mechanism to make learning more stable and effective from limited data, but it will not address delayed returns.

Any approach which learns Q should cope fine with delayed rewards. So Monte Carlo Control, SARSA, Q-learning, DQN and all their variants are in theory capable of learning the delayed reward. The total reward is the ultimate effect of any action.

However, I'm guessing just because it is not mentioned and you are asking this question, that you may still have a problem . . .

You claim that the action has a "delayed effect on the environment". You need to change that, because it implies that your state representation is incorrect for your problem. Clearly making an order for something changes the environment. An environment with an order in progress is different, in a way that is critically important to your problem, than one without an order in progress.

Probably what you are missing is a state representation that captures what your action has actually done. Without that, there are hidden variables (orders currently being handled) that a Q function cannot learn about because it is not in the state, $s$ for $Q(s,a)$. For RL to be reliable, the state value $s$ has to capture all relevant information about how future state changes and rewards will progress. For instance, if you were writing an agent to control a swinging pendulum (a standard toy problem), then you don't just need the position of the pendulum, you also want its velocity in order to predict where the pendulum might end up before any action is taken. For your issue, the current stock is like the pendulum's position, and you need to track orders "in progress" as they are a bit like velocity in that they will cause further changes to state regardless of action.

To address this, you will need to add a representation of recent stock orders to your state. I would guess something that describes the contents of the order (similar to your current stock representation) and a countdown for how many days left until it is delivered (or if they are not that reliable, something similar that correlates with likely arrival time). That state should be changed immediately in response to the action that caused the order, otherwise the agent will not learn the association between action and its effects, and will treat orders arriving as some kind of random effect from the environment.

Once you have a representation like this, then I think that is enough, and the next steps are finding the right hyper-parameter values to learn effectively. This will be easier initially if you have a simulated environment, although you could also learn off-policy from historical data using something like DQN, provided you have a lot of historical data to work with.

  I thought of just adding the quantity reordered to the shelf at a later time and let the agent learn it's effects. Will this suffice?

I think this is similar, but probably not enough. The most important thing to do is associate change to state with the action that caused it. Adding the new stock as it arrives you should do anyway, but the agent needs to learn that the earlier action is what caused this, and that is only possible if that action actually changes the state in some way.


How backpropagation through gradient descent represents the error after each forward pass

To get total error before back propagating - it is common to take an average of all the forward-pass errors. This is what's done in RNN such as LSTM.

In the case of linear regression and logistic regression, The traditional Mean Squared Error Function can produce such a value.

In essence, this value is represented by an average of errors: $Q(w) = 1/n{\sum_{i=1}^n Q_i(w)}$

Also, as a reminder, speaking of an actual backpropagation - from wikipedia:

When used to minimize the above function, a standard (or "batch") gradient descent method would perform the following iterations:

$$w:=w - {\eta}\nabla Q(w) = w - \eta{\sum_{i=1}^n}\nabla Q_i(w)/n$$

notice the $/n$ When used with the ${\sum_{i=1}^n}$ it results in the average of all gradients

:= means 'becomes qual to'

$\eta$ is the learning rate


Is it better to have one model with more categories or less with two for multi-label classification?

Generally there are two main options for multi-label classifications:

A binary model for every class, in this case the labels are supposed to be independent of each other.
A joint model which learns all the classes together, so at training stage the class represents the three values for the labels, for instance:

class "NNN" means no label at all
class "NNY" means no for question, no for complain and yes for complements
class "NYN" means no for question, yes for complain and no for complements
...
class "YYY" means yes for question, yes for complain and yes for complements

In theory there are $2^3$ possible classes for 3 labels, but in practice there might be less combinations in the data (as suggested in the question). If the labels depend on each other (for instance if it's unlikely to have a document which is both complain and complements), a joint model is more appropriate. However a joint model might need more instances to be trained properly, since it has more work to do than a binary model.


Multi-class classification with extremely small dataset

Yes, you can employ different methods. You can employ deep learning models, but you should not train them from scratch. You should employ transfer learning. Due to the fact that your dataset is small, you should utilize a deep learning model that is already trained. Next, you should replace the last layer with another layer that has the same number of neurons as your classes. The connections of this newly added layer should have random weight at first. Finally, you will freeze all the weights except for the newly added layer. In this case, your model will have a nice capacity to learn your data, and it will not overfit it. You may want to see the following links:

Pr. Ng's video
Another helpful video

You can also utilize SVM with soft-margin to have good generalization.
About the number of samples, it cannot be said in advance. Moreover, for different tasks, it may be different. By the way, someone who looks at your data can easily figure out you have small dataset.