what is the first input to the decoder in a transformer model?

At each decoding time step, the decoder receives 2 inputs:

the encoder output: this is computed once and is fed to all layers of the decoder at each decoding time step as key ($K_{endec}$) and value ($V_{endec}$) for the encoder-decoder attention blocks.
the target tokens decoded up to the current decoding step: for the first step, the matrix contains in its first position a special token, normally </s>. After each decoding step $k$, the result of the decoder at position $k$ is written to the target tokens matrix at position $k+1$, and then the next decoding step takes place.

For instance, in the fairseq implementation of the decoding, you can see how they create the target tokens matrix and fill it with padding here and then how they place an EOS token (</s>) at the first position here.

As you have tagged your question with the bert tag, you should know that what I described before only applies to the sequence-to-sequence transduction task way of using the Transformer (i.e. when used for machine translation), and this is not how BERT works. BERT is trained on a masked language model loss which makes its use at inference time much different than the NMT Transformer.


Gradient Descent Step for word2vec negative sampling

Looks good to me. This derivative is also presented in the paper (equations 56-58).

The paper you're linking to is the most advanced attempt - at least to best of my knowledge - to explain how word2vec works, but there is also a lot of other resources on the topic (just search for word2vec on arxiv.org). If you're interested in word2vec, you may find GloVe interesting too (see: Linking GloVe with word2vec).


Delete/Drop only the rows which has all values as NaN in pandas

The complete command is this:

df.dropna(axis = 0, how = 'all', inplace = True)

you must add inplace = True argument, if you want the dataframe to be actually updated. Alternatively, you would have to type:

df = df.dropna(axis = 0, how = 'all')

but that's less pythonic IMHO.


Exponential Linear Units (ELU) vs $log(1+e^x)$ as the activation functions of deep learning

ReLU and all its variants ( except ReLU-6 ) are linear i.e $ y = x $ for values greater than or equal to 0.

This gives ReLU and specifically ELU an advantage like:

Linearity means that the slope does not plateau or saturate when $x$ becomes larger. Hence, the vanishing gradient problem is solved.

Now, the graph $ y = log( 1 + e^x ) $ isn't linear for values > 0.

For larger negative values, the graph produces values which are very close to zero. This is also found in sigmoid where larger values produce a fully saturated activation. Hence, $ y = log( 1 + e^x ) $ can raise problems which sigmoid and tanh suffer.

About ELU:

ELU has a log curve for all negative values which is $ y = \alpha( e^x - 1 )$. It does not produce a saturated firing for some extent but saturates for larger negative values.

See here for more information.

  Hence, $ y = log( 1 + e^x ) $ is not used because of early saturation
  for negative values and also non linearity for values > 0. This may
  produce problems and even bring down some features which ReLU and
  variants exhibit.


Understanding a dataset (prior to applying ML models) with no metadata given

The aim is to predict pm2.5 ( target variable ).

Step 1: Data Cleaning. Remove unwanted features and fill the missing values.

Step 2: To learn about the features, perform data visualization. You can plot a linear plot with TEMP and pm2.5 and see how it varies with change in temp.

Step 3: The next step is to find the relationship between features. Some features are not needed for prediction. Remove those features.

Step 4: Apply a suitable machine learning technique and predict.


What affects the magnitude of lasso penalty of a feature?

When you increase the scale of $X_1$ by 10, the scale of the corresponding weight in you linear regression sees its scale divided by 10 (it is exactly 10 for standard linear regression and the same order of magnitude holds for lasso loss). As a consequence, the $L_1$ penalty for this coefficient is also divided by a factor 10 and it is less likely to be set to 0.


Extracting NER from a Spanish language text file

I suggest you take a look at the Python library Spacy it has a model for spanish language that includes NER.


How to automatically test for the best parameters for transformed independent variable in linear model

One approach would be to use an algorithm designed for non-convex problems like Bayesian optimization. However, if you have already evaluated a fine grid of parameters this is unlikely to offer significant improvement. Here is an example of how you could implement Bayesian optimization for this problem.

First, we need some data. Just for fun let’s extract the data from the images you posted (in brief since this is off topic).

In Mathematica:

img = Import[NotebookDirectory[]<>"LHrXQ.png"]
img2 = ImageResize[ImageTake[img, {40, 450}, {50, 1000}], 200]
pixels = {(#[[1]]-10)*3.8,#[[2]]*9+100}&/@PixelValuePositions[img2,Black, 0.4];

ListPlot[pixels, Frame->True, ImageSize->500]

Now in order to use Bayesian optimization we need to define the objective as a function of parameters A and B. Here we will maximize the R^2 value for the fitted model.

In python:

import pandas as pd
import numpy as np
import random
from sklearn.linear_model import LinearRegression

data = pd.read_csv('mathematica_data.csv')

def objective(params):
    """Whatever you want to do for your regression."""

    # So it works with GpyOpt
    A = params[0][0]
    B = params[0][1]

    temp = data.copy()

    # Transform variable
    xt = [np.arctan((x - A)/B) for x in data['x1'].tolist()]
    temp['x1'] = xt

    # Fit a linear model
    reg = LinearRegression().fit(temp.drop('y', axis=1), temp['y'])

    # Compute scores of interest
    r2 = reg.score(temp.drop('y', axis=1), temp['y'])

    # GPyOpt will minimize so we want -f
    return - r2

Now use GPyOpt to optimize the objective.

import GPyOpt

domain = [{'name': 'A', 'type': 'continuous', 'domain': (-300.0,300.0)},
          {'name': 'B', 'type': 'continuous', 'domain': (0.1,300.0)}]

bo = GPyOpt.methods.BayesianOptimization(f=objective,
                                         domain=domain,
                                         model_type='GP',
                                         acquisition_type='EI',
                                         initial_design_numdata=10,
                                         initial_design_type='random',
                                         acquisition_jitter=0.01,
                                         num_cores=-1,
                                         de_duplication=True,
                                         exact_feval=True)

# Run optimization
bo.run_optimization(max_iter=100)

We can plot the optimizers convergence:

bo.plot_convergence()

And information about how it sampled the parameters:

bo.plot_acquisition()

Of course using only the independent variable extracted from your plot even the best R^2 values suggests there is little to no relationship between arctan((x-A)/B).


How to decide power of independent variables in case of non-linear polynomial regression?

If all you care about is the quality of predictions (as opposed to explanatory power), skip linear models altogether and use gradient boosted trees instead. Gradient boosting can generally learn polynomial splines with ease, and you don't have to manually make a bunch of polynomial predictors yourself.

By the way, gradient boosting is implemented in Python's scikit-learn library, R's caret library, and Java/Scala's Weka library.


You do not need to use Word Embedding to solve this problem.

You are performing set membership check. In this problem, each book title and author combination is an element. You can create a hash for each element and perform constant time lookup. This assumes you have consistent normalization for each data source.


What you are looking for is feature contributions to the final score of an observation having a positive or negative price. Feature importances from a decision tree or random forest is not going to help you because a feature's importance is fixed across all observations (wines). They tell a story about the overall model, but nothing about the individual observations. So if you want to know what caused a wine having a negative or positive price, you should look at feature contributions.

Let's elaborate a little bit.

Let's forget about decision trees, and go for ensemble of them, say random forest, for all the good reasons. (I highly recommend you to read why ensemble models uniformly beat the base learner, and how they help with the high bias - high variance problem base learners might have.)

The first step would be to label your dependent variable, price, as $1$ and $0$, depending on the price being positive and negative ($y = 1$ iff $price > 0$). After the typical exploratory analysis (where you can look at boxplots and stuff as a beginning study) and feature engineering, you can fit your random forest to predict a wine having a positive or negative price (i.e. $y$ being $1$ or $0$).

Once you do that, you will have an estimated probability for each and every wine, representing the classification model's estimation of that wine's price being positive or negative. Something like this:

Wine Prob
A    0.4
B    0.7
C    0.9
D    0.3

These probabilities actually consist of the summation of a bias term and the individual contributions of all features to that observation only:

$$P(y_i = 1) = bias + \sum_{i=1}^{m}(contributionOfFeature_i),$$

where $m$ is the total number of features. This is how you can analyze each and every feature's contribution on the final probability of a particulat wine's price being positive or negative.

Regarding how to code this, if you are using scikit, you can use this very easy and convenient tool:

https://github.com/andosa/treeinterpreter


Feature importance with high-cardinality categorical features for regression (numerical depdendent variable)

It depends on how you're one-hot encoding them. Many automated solutions for that will name all the converted booleans with a pattern so that a categorical variable called "letter" with values A-Z would end up like:

letter_A, letter_B, letter_C, letter_D,....

If after you've figured out feature importance you've got an array of feature and the associated weight/importance, I would analyze the array and perhaps sum up the feature importance weights for anything starting with "letter%".


Neural net layer that preserves spatial information

Rather than trying to predict each value explicitly, you could pick a parametric function that you think might be expressive enough to describe your target function (for example, a polynomial of order 6), and get your network to learn the parameters of that polynomial that give you a minimum loss. This is basically polynomial regression.  Since your function is periodic you might use a more cycle-based basis set, like linear combinations of sine waves.

For example:

1) Input data to the network
2) output a set of values that you treat as parameters to your function
3) for each target point, compare it to the predicted value from your function
4) calculate a loss (like mean squared error) based on the difference between the function predictions and the targets.

Using a function will (1) restrict the space of possible outputs, and (2) make nearby outputs be self-similar.


Predicting topics for customer reviews based on topics mapped to n-grams?

You might want to look at the Facebook starspace library - their examples are similar but with suggesting Facebook pages and hashtags.

https://github.com/facebookresearch/StarSpace


GPU Accelerated Data Processing for R in Windows

From my experience setting up GPU processing for R is hard, setting it up on a Windows machine is even harder. Additionally, GPU processing can only be used for very specific types of calculations.

If you just want to setup GPU processing for the sake of it, then my answer is quite useless.

If you however care about general performance optimization of your system and your code, I advise to check the following steps:

Use Microsoft R Open instead of Base R because it automatically enables multicore processing on your machine.
Vectorize your code
Use libraries such as data.table instead of dataframes
Avoid growing objects

In general, R's performance depends strongly on your code quality. A very good summary on what you can and should do is provided in R Inferno by Patrick Burns.


Unsure of how to implement an equation in PyTorch

Check this GitHub repo for implementing SummaRuNNer with Pytorch.

The PyTorch Implementation Of SummaRuNNer


I assume that you are referring to policy gradient estimates. Adding any kind of function to your policy estimation,  which is dependent on the state of the environment, first of all, does not bias your gradient estimator (Proof Here).

The basic idea of subtracting a baseline from your action value function (and thus forming the advantage function) is that an unbiased estimator of your policy gradient is still unbiased if a constant is subtracted from that estimator. Then, that constant can be chosen appropriately in order to reduce the variance of the new estimator by optimization. If you have access, you can find a very good explanation in the book Statistical Reinforcement Learning: Modern Machine Learning Approaches in section 7.2.2. Also [2] and section 3 in [3].

As you mention it can be viewed as a control covariate addition [4] which is used to reduce variance in Monte Carlo estimations. A good choice for that function is to use the usual value function ($V(s)$) which reduces the variance of your estimate.

Hope it helps!


Upload 'tsv' file to Google Colab / Jupyter Notebook

You first need to upload your file. The io.BytesIO only reads from the uploaded. So first run:

from google.colab import files
uploaded = files.upload()

and select the file you would like to upload.

Also, when you load it into your pandas, you need the sep='\t':

tsk = pd.read_csv(io.BytesIO(uploaded['train.tsv']), sep='\t')


It seems that Watson doesn't allow some accounts to access some regions. Just choose another region, like "US South", when creating the resource, and the problem will be gone.


You have your model saved as the variable lm. You can use the lm.predict(X_test) for any other test scenario. Note that your X_test should be similar to your X_train meaning if you have transformations made on your X_train, you need to do similar transformation on X_test too. You can use pickle in the following way

import pickle
filename = 'model.pckl'
pickle.dump(model, open(filename, 'wb'))

#To load the model from disk, use this
model = pickle.load(open(filename, 'rb'))
prediction = model.predict(X_test)


Is it necessary to sort time series data on datetime stamps?

Sorting data won't affect the training of your model, it is similar to changing the random seed.

It can affect the validation that you are doing. In case you do time series you can do sliding window or roll-out-window, that they need the data to be sorted before the splitting.

It seems that you want to do time series regression with supervised learning so what you probably want to do is change your problem to a supervised learning problem. You can see a bit more here. But this is me guessing what you wnat from a couple lines.


What kinds of learning problems are suitable for Support Vector Machines?

SVM can be used for classification (distinguishing between several groups or classes) and regression (obtaining a mathematical model to predict something). They can be applied to both linear and non linear problems.

Until 2006 they were the best general purpose algorithm for machine learning.  I was trying to find a paper that compared many implementations of the most known algorithms: svm, neural nets, trees, etc. I couldn't find it sorry (you will have to believe me, bad thing). In the paper the algorithm that got the best performance was svm, with the library libsvm.

In 2006 Hinton came up with deep learning and neural nets. He improved the current state of the art by at least 30%, which is a huge advancement. However deep learning only get good performance for huge training sets. If you have a small training set I would suggest to use svm.

Furthermore you can find here a useful infographic about when to use different machine learning algorithms by scikit-learn. However, to the best of my knowledge there is no agreement among the scientific community about if a problem has X,Y and Z features then it's better to use svm. I would suggest to try different methods. Also, please don't forget that svm or neural nets is just a method to compute a model. It is very important as well the features you use.


Using LSTM to clear up corrupted text files

You should use a Seq2Seq (which uses LSTM/GRU/RNNs) architecture for the task of cleaning text. The encoder network would take in the "noisy" sequence and the decoder would generate the "cleaned" sequence.

Seq2Seq models is often used for Neural Machine Translation (NMT).

The next thing is you will need a very large data set. Fortunately this shouldn't be too hard to generate noise injections (of real words, symbols, fake words, letter omissions etc) into sequences of text.

A good guide for a similar task is in the following post. The implementation uses Keras and has all of the code. https://towardsdatascience.com/how-to-create-data-products-that-are-magical-using-sequence-to-sequence-models-703f86a231f8


Handling missing values to optimize polynomial features

There is no globally - one could say even locally - ideal way to deal with missing data. This aspect points to incompleteness in the data you're feeding your algorithms, and imputing is simply a technique meant to fill gaps.

Data imputation's motivation is to make feature distribution in your sets the closest possible to whatever real-world distribution it attempts to portray.

The intuition behind why the median is what worked better for your scenario is not something I could be precise about without having access to the data you worked upon, but the positive result is deeply related to your feature distribution, which for the missing points in your data is better represented by the feature's median than all other metrics you've calculated. I'd recommend reading material such as this article that both explains and shows implementation of different imputation techniques - KNN, as you mentioned, being one of them. One of the advantages is seeing how different methods work for a given distribution:

As you have already pointed in the end of your question, methods such as KNN are some straightforward means of imputation that could benefit your case better than mean/median imputation. The biggest difference between those is that KNN better preserves the variance in your data, whereas mean imputation (as you can see from the above image) shifts missing data towards a single value.

Since no method is 100% globally optimal, I'd advise you to try them - KNN, Multiple Interpolation and the likes - and compare. Invest the appropriate amount of time on the techniques that make the most sense for your data.


It depends on the problem you are trying to solve. If you know the signal in the dataset already, the words which decide your decision then go with Bag of Words. This is useful when you are doing something like text classification.
On the other hand, TF-IDF is useful when you don't know the signal in the dataset. If you want to do text similarity, then, this is a good option.


How to compare and find common values from different columns in same dataframe?

let's consider we have a data frame named df, then one approach might be saving these two columns in different dataframes and then trying to compare them and find out their similarities, hence we would have:

column1 = df.iloc[:,1].values
column2 = df.iloc[:,2].values

Then, saving all the indices in column1 where the set exists in column2

equal_indices = []
for i in range(len(column1)):
    for j in range(len(column2[or column1, since they are equal])):
        if column1[i] == column2[j] and column2[i]==column1[j]:
            equal_indices.append(i)
            print(i,j)
            print(column1[i], column2[i])

Now equal_indices contains all the indices you want. Then you can delete the rows with similar columns from column1 or column2. or just return the found indices of column1 from the dataframe df:

df.iloc[[equal_indices]]


Determine Optimal CNN Complexity: Achieve ~100% Training Accuracy, Then Add Regularization?

Yes, that is in my opinion the best methodology for approaching Deep Learning. This isn't supported by just me but by many researchers in the field. (I can't cite it but I'm pretty sure even Andrew Ng mentioned it in a lesson in the Deep Learning Specialization in coursera.)

The first step is to make sure you have a model with a sufficient capacity to be able to learn all the relationships (useful and not) in the training set.

Then you start applying regularization techniques (dropout, augmentation, early stopping, etc.), to prevent overfitting. During this step your training accuracy is expected to take a drop (or at least take longer to reach 100%), but the accuracy on the validation set should increase.

This methodology, however, is only applicable in the field of deep learning, because of the strong regularization techniques available.


How to reduce pipeline footprint in scikit-learn?

I don't know if it is still an open question, anyway.

What is painful when using TFidf, it is the size of the ouput sparse matrix. II think it is your problem here. In order to pass this, you should limit the amount of word you want to treat. I didn't try with your case but you should try CountVectorizer before TfidfTransformer or directly TfidfVectorizer.
In CountVectorizer and TfidfVectorizer from sklearn, you can specify the minimum and maximum number of document any term can be found in (min_df and max_df).
This will reduce the size of the output A.

Hope it helps a bit


How to use the same minmaxscaler used on the training data with new data?

You are refitting scaler_x on your test set, which you don't want.  Change this line:

xaa = scaler_x.fit_transform(xaa)

to

xaa = scaler_x.transform(xaa)

You are getting [-1, -1, ..., -1] because with one sample, each feature is equal to the minimum.


Need for Dense layer in Text Classifcation

In neural networks meant for classification, you need a linear layer before the softmax to project the internal representation, which has some dimensionality $d_i$, to the output space, which has dimensionality $d_o$ equal the number of choices (5 in your case).
So you either place a Dense(5) layer after the BiLSTM or you take the output of the BiLSTM "manually" and implement the projection.
The code above has some strange things:

Uses numpy.dot to multiply the output of the BiLSTM. Is this a typo and you actually meant tf.dot or tf.matmul?
The model ends with a tf.keras.layers.Dense(1), maybe because it was originally meant for binary classification.
Has both a Dense layer and then a dot product (i.e. matrix multiplication). These two operations are equivalent to a single Dense layer, so it is pointless to have both.

So yo answer your question: assuming that the np.dot actually means a tf matrix multiplication, then the Dense layer in the model is pointless.


Will I be qualified for data science roles with a PhD in Engineering and Machine Learning?

Write blogs, papers, and so on. Take part in the open source community. This PhD does neither qualify you nor not. But when you are able to show your passion you have got a ticket.


Keras does not include by itself any means to export a TensorFlow graph as a protocol buffers file, but you can do it using regular TensorFlow utilities. Here is a blog post explaining how to do it using the utility script freeze_graph.py included in TensorFlow, which is the "typical" way it is done.
However, I personally find a nuisance having to make a checkpoint and then run an external script to obtain a model, and instead prefer to do it from my own Python code, so I use a function like this:
def freeze_session(session, keep_var_names=None, output_names=None, clear_devices=True):
    """
    Freezes the state of a session into a pruned computation graph.

    Creates a new computation graph where variable nodes are replaced by
    constants taking their current value in the session. The new graph will be
    pruned so subgraphs that are not necessary to compute the requested
    outputs are removed.
    @param session The TensorFlow session to be frozen.
    @param keep_var_names A list of variable names that should not be frozen,
                          or None to freeze all the variables in the graph.
    @param output_names Names of the relevant graph outputs.
    @param clear_devices Remove the device directives from the graph for better portability.
    @return The frozen graph definition.
    """
    graph = session.graph
    with graph.as_default():
        freeze_var_names = list(set(v.op.name for v in tf.global_variables()).difference(keep_var_names or []))
        output_names = output_names or []
        output_names += [v.op.name for v in tf.global_variables()]
        input_graph_def = graph.as_graph_def()
        if clear_devices:
            for node in input_graph_def.node:
                node.device = ""
        frozen_graph = tf.graph_util.convert_variables_to_constants(
            session, input_graph_def, output_names, freeze_var_names)
        return frozen_graph

Which is inspired in the implementation of freeze_graph.py. The parameters are similar to the script too. session is the TensorFlow session object. keep_var_names is only needed if you want to keep some variable not frozen (e.g. for stateful models), so generally not. output_names is a list with the names of the operations that produce the outputs that you want. clear_devices just removes any device directives to make the graph more portable. So, for a typical Keras model with one output, you would do something like:
from keras import backend as K

# Create, compile and train model...

frozen_graph = freeze_session(K.get_session(),
                              output_names=[out.op.name for out in model.outputs])

Then you can write the graph to a file as usual with tf.train.write_graph:
tf.train.write_graph(frozen_graph, "some_directory", "my_model.pb", as_text=False)

Credits: jdehesa
You can find more at:

How to export Keras .h5 to tensorflow .pb?

Keras as a simplified interface to TensorFlow: tutorial


Preventing fitting Regression CNN to the mean when dataset has only few outliers

I'm not sure what you are trying to do. CNNs are good for image-related tasks, as they attempt to extract spatially local features from the input data. They can be used for regression problems but only as long as the input resembles an image.

Random Forests on the other hand are bad at image-related tasks, unless some sort of feature extraction has been performed beforehand.

Does your dataset consist of images? If not don't use CNNs!


Which GUI library to use with Deep Learning

It really depends on how far you want to go.
If you are very serious about GUI apps, PyQt is the only way to go.
Qt5 is the gold standard for cross-platform GUI right now.
But, for basic applications, you are good to go with Tkinter.
I have never used Kivy, and I don't know many people who use it.


How to force weights to add to $1$ in Linear regression

Suppose the problem formulation is

$$\min_{a,b} \sum_{i=1}^n(y_i -a_0-\sum_{j=1}^pb_jx_{ij} )^2$$

subject to $$\sum_{j=1}^p b_j=1$$
$$b \ge 0$$

then this is an instance of a quadratic programming problem.

There are solvers such as Gurobi (free for academia) and CVXOPT (freeware) that can manage a quadratic programming problem. An example of CVXOPT code to solve a QP can be found here.

Note that the objective function can be written as

\begin{align}&\|Y-a_0e-Xb\|^2\\&=\left\|Y-\begin{bmatrix}e & X \end{bmatrix}\begin{bmatrix}a_0 \\ b\end{bmatrix}\right\|^2\\
&=\begin{bmatrix}a_0 & b \end{bmatrix}\begin{bmatrix} e^T\\X^T\end{bmatrix}\begin{bmatrix} e &X\end{bmatrix}\begin{bmatrix}a_0 \\ b \end{bmatrix}-2Y^T\begin{bmatrix} e &X\end{bmatrix}\begin{bmatrix}a_0 \\ b \end{bmatrix}+\|Y\|^2\end{align}

Also, you might like to check with people in your field what is the common package that they use.


Why did Logistic regression perform better than svm?

There is a theorem in machine learning literature which is called “No free lunch theorem”. The essence of NFL is that there is no universal model which performs best for every problem and every dataset. So, according to NFL, you can not expect SVM to outperferm logistic regression in all situations and contexts. If your classes were linearly separable SVM would be perfect with 100% accuracy but otherwise you shouldn’t expect it to necessarily outperform ligistic regression. So, whether SVM or ligistic regression are better choice highly depends on the problem and on the available dataset.


The reason for weights in machine learning is actually a lot easier than it seems. It's the way by which our model learns some underlining function and performs the classification or regression. We tune these weights in order to model some underlining function which can map our input to a desired output. Either a class in classification, or a range of values in a regression.

Let's look at an easier machine learning model so that we can understand why weights are needed.

Linear Regression

This is just a straight line which splits data into two sections. Let us apply this model in 2D, where we have two features $x = [x_1, x_2]$, for example weight and height. And labels $y$ which will split our data into $y \in \{men, women\}$.

A random line in this space is defined as

$0 = -2x_2 + x_1 + 1$.

Assume this is our boundary line. Let's just assume that women fall above this line and men fall below it. Like this

Now if we get a new data point $x_{new} = [2, 8]$ we will label this as being a woman. So the entire decision is based on the numbers $1, -2, 1$ from our linear equation. We need to tune these values using the training data. We usually call these trainable parameters the weights $w$ associated with the features $x$ and we also add a bias $b$. In general the linear separator in 2D is

$0 = w_1x_1 + w_2x_2 + b$.

Our predicted label is $\hat{y} = w_1x_1 + w_2x_2 + b$. If $\hat{y} > 0$ then woman, else man.

In $n$ dimensions this can be written as a matrix multiplication as

$\hat{y} = w^Tx + b$

Obviously, a linear separator is not sufficient for most classification tasks. Things are not always linearly separable. So we use more complex models.

Neural networks

In neural networks each node is associated with a function much like the linear separator. However, we will use the sigmoid function

$\sigma(w^Tx) = \frac{1}{1 + exp^{-(w^Tx + b)}}$.

The weights here have the same effect. They will modulate the input values $x$ such that we are able to learn some classification or regression.

How do the weights affect the decision boundary. We want the two different classes, circles and x's, to be on opposite sides of this boundary in order for us to be able to correctly classify them.

The weights are trained iteratively using gradient descent. We can see that the decision boundary starts off terribly and then gets progressively better.

$0 = -1.0 - 1.0x_1 - 1.0x_2$

$0 = -16.0 - -39.0x_1 - 71.0x_2$

$0 = -36.0 - -94.0x_1 - 61.0x_2$

$0 = -83.5 - -134.0x_1 - 76.0x_2$

$0 = -88.5 - -114.0x_1 - 98.5x_2$

As you can see we changed the weights until we were able to find this ideal boundary between the two classes. If you want to find our more about how the gradient descent algorithm works to tune these weights you can look here.


Transition from clustering to classification?

In order to build a model to make predictions you need a labeled training set, that is, a training set in which each training example is assigned a class label. Training sets are usually labeled by human experts that use their domain knowledge to manually classify the examples in the training set. You have already done that, as described in your first paragraph.

However, sometimes this process is expensive. In order to decrease cost sometimes semi-supervised learning is applied. In semi-supervised learning, and under some assumptions, a small amount of labeled, together with a large amount of unlabeled data, are used to build a predictive model. The unlabeled data are assigned labels during the training process.

This seems to match your idea in your last paragraph, and it is already implemented in Python. I think that if you found clear clusters in your data it may work very well. Worth a try at least.


Structuring experiment/training data with months in mind

Sounds like your problem is predicting a value for a specific month given historical data. That could be modeled as a time series problem.
Also, you also say the data strategy changed over time. That means you can not make a stationary process assumption.


What is "GOAL" in terms of Reinforcement Learning specified in these papers?

After some discussions with Kiuhnm in RL Group Discord channel, I understand it now.
In the Horde paper it is easy, i.e. the goal might be state or a time. This sub-agent (called demon in the paper) might have a goal of running for 10 time steps, and you might have a reward function specific for it.

In the UVFA paper, the example was to define goals as a state and that's just for simplicity. Let's say you are in a room and your state is 2-dimensional position. Your goal maybe going to next room, and that means another 2-dimensional position. UVFA proposes that, the network will generalize over goals, and make your goal achieving easier in the following tasks.

That pretty much sums it up. Thanks to everyone who did give a thought.


What does the Orange widget "Image Embedding" do exactly?

The source reveals it's likely Google's Inception V3 model.


Sementic segmentation data and model compile in Keras

Two things that stand out is that you are using sigmoid as activation which is used for binary classification (it just squash values between 0 and 1). The other is that your learning rate is 5x lower than the default value.

Also because you are classifying images, I think the title is misleading; semantics is associated with "meaning" i.e. with language.


Opening a 20GB file for analysis with pandas

If it's a csv file and you do not need to access all of the data at once when training your algorithm, you can read it in chunks. The pandas.read_csv method allows you to read a file in chunks like this:

import pandas as pd
for chunk in pd.read_csv(<filepath>, chunksize=<your_chunksize_here>)
    do_processing()
    train_algorithm()

Here is the method's documentation


It's hard to tell exactly what you're asking, and exactly what your data looks like. Nonetheless, I think you're asking how to convert a list to a numpy.array.

Try this:

np.asarray(data["names"])


i have to classify an sms into categories like educational, bank related,etc. Is this a problem of topic modelling or text classification?

Topic modeling is usually used in the context of unsupervised learning, while classification is used for supervised learning. So it more depends on your dataset, does it have labels that you can learn from (classification) or does it need to figure out topics itself (topic modeling)?


What is the advantage of using RNN with fixed timestep length over Neural Network?

It's true that they both use 3 samples to predict the next sample. But the difference is obviously how they learn it. Deep Neural Nets will treat time-distributed samples such as they were single sample at once. However, RNN will learn it as a sequence by its sequential nature, part by part.

Each activation layer of (uni-directional) RNN will take a part of a sequence, make prediction and feed the prediction on that part forward to the next activation layer which will take the next part of the sequence as input. Important part is that the prediction is feeded forward so that the next neuron learning the next part of the sequence will also affected from the prediction of the previous neuron as well as the input sequence, unlike classical neural networks. It can get easy to understand when you look at the visual example below:

Hope I could help.


Should the times series be made stationary before doing a clustering analysis?

Not sure, but I don't know any task related to time series algorithms and analysis for which it wouldn't be useful to make the time series stationary first. You simply get rid of a lot of unexplainable anomalies this way.
Maybe only if you use several different time serieses of the same period, then one could try to find some usefulness in comparing the unstationary serieses.
The effects of the unstationary series, are not explainable by the same series alone.


Both train and test error are decreasing in XGBoost iterations

This is a common situation and can be caused by a number of things, such as:

Too much variance in the model (e.g. did the hyperparameter tuning identify low values for eta, alpha, lambda, gamma, subsampling/column sampling/row sampling, minimum child weights and/or high values for depth and number of leaves?)
Having a small dataset with high variance.  XGBoost is prone to overfitting and this is exacerbated when there isn't much data for it.
Not shuffling the data so that the validation set has lower variance than (or differs in distribution from) the training and test sets.  train_test_split shuffles by default, so this shouldn't be a problem if you've used it as indicated in the question for generating the validation set also.

Stopping training early is a perfectly valid and common method of increasing bias in a model and doing so based on a validation set is acceptable.  It is preferable for the training to stop based on the test set.  To achieve this you could address the potential issues above.  Without knowing more about the data and how the validation set was generated, the most reliable advice I can give is to alter the hyperparameters to increase model bias.


I suggest to give transfer learning a try for two reasons:

Your dataset is relatively small
Skin, faces and similar human body features have been successfully classified or segmented by existing neural networks

Specifically, you might want to have a look at Skin Lesion Analysis Toward Melanoma Detection. While the tasks were slightly different it did include a segmentation task of Melanomas. Here is an example of what they did:

And this is a snapshot of how some of the models performed:

Source: both images are from the paper linked above

It might be close enough to your task to re-use their networks or some part of it. Once you have found a transfer network, you can vary the degree to which you replace or retrain it`s layers to see what works. Given your relatively small dataset I would not retrain too much but start with the final layers only.

However, depending on how exactly your dataset looks like, there might be other tasks (e.g. from security applications where one needs to detect, classify or mask an object (e.g. a person or a face)) which resemble yours even better. Therefore, before digging into a specific model and apply transfer learning I'd do some more research.


Can a Conditional GAN be used for a regression task, where the condition can take any continuous value?

I think it exists as it is mentionned in the article Intra-class Variation Isolation in Conditional GANs from Richard T. Marriott, Sami Romdhani et Liming Chen in 5.2

Sadly I found nothing about those CC-GANs, eventhough it is said "classical"!


How to go about creating embeddings (especially, token to Id mapping) for categorical columns in tensorflow 2.0+?

Embedding() layers take sequences as inputs. If there is no time dependency in your data (such as for time series datasets or language corpora) then those layers cannot be used. What kind of data do you have?

Alternatively, you can use dimensionality reduction, one-hot encoding or, in some cases, label encoding. I personally suggest dimensionality reduction whenever possible, even if it's the most computationally intensive choice.


How to measure confidence in prediction?

If you've got enough information on the underlying distributions of your data, classical statistical theory on regression models is an obvious alternative. These models always come with nice derivations of error estimates. See Frank Harrell's "Regression Modelling Strategies" for a comprehensive overview.


Training on data with inherently non-applicable data cells

Welcome to the site!

The usual approach to missing values is to handle them manually. There are a few algorithms which can do this automatically, such as LightGBM and XGBoost, but in most cases it's better for model performance to decide on how you should indicate that a value is missing in your data.

For example in with a Pandas dataframe in Python, I may decide to replace all NA/NaN values in a particular column which should contain positive integers with -99:

dataframe[column] = dataframe[column].fillna(-99)


RNNs are not designed to do language modeling exclusively, they are designed to process time series data, and language happened to be representable as time series.
There is plenty of papers demonstrating how to use RNNs to do classification and regression on time series (awesome list of papers).

One-hot encoding is often used in cases where the input is discrete and not a number that can be directly fed into a model. However, one-hot encoding is actually not always the norm for language-modeling, some research actually map each character (or each word depending on how one wants to model the problem) to a unique numerical identifier (e.g. with $id\in \left\{ 0\ldots n \right\}$; with $n$ the size of the vocabulary) and the model map that identifier to a vector representation. This is particularly useful when the size of the vocabulary is huge and you want to avoid having to deal with big one-encoded vectors. Take a look at word2vec and word2vec Tensorflow for more details about this.

In your case, you want to process sensor data. There is no need for one-hot encoded because your data are continuous and numerical. In other words, you can input the recorded EEG data directly; although it's usually better to clean them up and normalize them beforehand. There is actually plenty of papers about EEG data processing with RNNs.

Regardless of the type of data, the idea to fed them into a RNN remains the same: provide a data sample $x$ for each timestep $t$.

For EEG data recorded for 5 timesteps with 1 electrode $a$, you would have a 1-dimensional vector with one sample per timestep:

$a = [0.12, 0.44, 0.134, 0.39, 0.23]$

$inputvector = [0.12, 0.44, 0.134, 0.39, 0.23]$

For EEG data recorded for 5 timesteps with 3 electrodes $a, b, c$, you would have a 3-dimensional vector with one sample per timestep:

$a = [0.12, 0.44, 0.134, 0.39, 0.23]$

$b = [0.43, 0.92, 0.3, 0.37, 0.4]$

$c = [0.13, 0.1, 0.4, 0.21, 0.14]$

$inputvector = [[0.12, 0.43, 0.13], [0.44, 0.92, 0.1], ...]$

I highly recommend you to read some of the literature I linked above.


Implementing sklearn's FeatureHasher on Unseen Data

Feature hashing just applies a fixed hash function to its input strings; it doesn't need to have seen any data.  Note the docstring for the fit method:

No-op.
This method doesn’t do anything. It exists purely for compatibility with the scikit-learn transformer API.

There may be collisions, and you won't know it with unseen data; your model will be forced to consider these random collisions as the same, but this may not be an issue.  Feature hashing is really only meant for situations where you need to keep memory usage down.


Volcano-plot and MA-plot are graphical methods. MA-plot of what? Everything in your screenshot is bioinformatics analysis. The plots are part of the analysis but not the name of the analysis itself.


How do you classify groups when the inputs are paired?

That is called two-alternative forced choice (2AFC) task, a common technique in Psychology research. The Diffusion Decision Model (DDM) is used to analyze the data.


Kernel regression results in diverse outputs

You have a very small number of observations (32?) and a non-trivial number of predictors.

It is known that the cross-validation function possesses multiple local minima/maxima. If you increase the number of multistarts to, say, nmulti=100 (add this option to your call to npregbw()), you ought to see that the same results occur on each invocation of the optimization process.

Note you can do this all in the call to npreg() and skip the bandwidth call for convenience (npreg() will call npregbw() automatically but accept the arguments intended for npregbw()).

Also, you will get the same results if you restart R each time you run the routine (seeds are set automatically to ensure this).

model <- npreg(power ~ temperature
                     + prevday1
                     + prevday2
                     + prev_instant1
                     + prev_instant2
                     + prev_2_hour,
                regtype="ll",
                bwmethod="cv.aic",
                nmulti=100,
                data=new_tr_dat)

To see whether things are stable with respect to the number of multistarts, look at the value of the cross-validation function and summary provided. Also, you can look at partial regression plots along with resampled variability bounds via plot().

summary(model$bws)
plot(model,common.scale=FALSE,plot.errors.method="bootstrap")

Also, I note that your predictor `temperature' is discrete, so you might consider using

+ ordered(temperature)

(i.e. use a discrete support kernel). Doing so reveals that there is little signal in this model, but the same holds for a simple parametric model (adjusted r-squared is negative).

model.lm <- lm(model$bws$formula)
summary(model.lm)

Hope this helps!


Why does the transformer positional encoding use both sine and cosine?

If you read the mentioned answer, I guess you already have the notion of the need for a encoding way to represent the position of the word in the input.
In order not to use a sequence of integers (1, 2, 3, ... n) because of the lack of boundary in the value and the magnitude, a float friendly is preferred. But, just using a limited (0 to 1) option means you need to know the length of the sequence beforehand.
This is why the authors propose a cyclic solution. Sin and cos functions can benefit from this idea and thus are the choice here. But why using both instead of one?
It is not clearly answered in the original paper, but if you go though this article(which I urge you to) the reason is that you can use a linear transformation to go from the two functions, to the same functions with an offset:

And if you wonder why having this property available is useful, if you dig into the comments below the article you will find the explanation you seek:

I suppose that this encoding framework enables the model to attend to relative positions by simply generating a transformation matrix, which only depends on the k. To be clear, I hypothesize that given any input PE(pos), the model can create the attention query matrix Q that targets PE(pos+k) by multiplying the PE(pos) with a weight matrix T (the transformation matrix). The weight matrix T, which could be parameters of a single feed-forward layer, can be learned during the training process.

Basically the model can learn a matrix not dependant on t as part o the trining to generalize all the range of encoding values.
Can we just use one of the other?
You could encode the values of the position, yes. But you would not have the property of having this linear transformation available, which seems to be a very important part for the learning process to succeed.


How I understand your data is a single feature of temperature. LSTM works with RNN time series like (batch_size, timesteps, feature). In your case it is (none, 10000, 1). Forget about the first one, it's technical.


How to select the split point for Continuous Attribute Age

For 35 split, there is an error in the denominator. It should be 5 as the total number of items for D<35 is 5

D<35 => [2+, 3-], Entropy (D<35) = -[2/5 x $log_2$(2/5) + 3/5 x $log_2$(3/5)]


How to use Keras predict_generator() for segmentation output?

Okay, I checked that m.predict_generator returns a numpy array so I just plotted it using plt.imshow(). Obviously I'd to do some slicing and squeeze on it.

Sorry if it was too trivial


On the choice of LSTM input/output dimension for a spatio-temporal problem

On time-series models

All models that you have mentioned are correct and practical depending on the problem (the index $n$ is not required). The second one however produces redundant results which is a waste of computation. Even
$$
(X_{t} ; X_{t+1} ; X_{t+2}) \rightarrow (Y_{t-1})
$$
is correct, if you are fitting on an archive and want to predict a year given the covariates from the next three years.

But only the last model
$$
(X_{t-2} ; X_{t-1} ; X_{t}) \rightarrow (Y_{t+1})
$$
is a forecasting model. So in general, if you want to interpolate into the next $k$-th year from now $t$, you should use:
$$
(X_{t-2} ; X_{t-1} ; X_{t}) \rightarrow (Y_{t+k})
$$

or

$$
(X_{t-2} ; X_{t-1} ; X_{t}) \rightarrow (Y_{t+1},...,Y_{t+k})
$$
Even a better model that takes advantage of known $Y$'s in the past would be:

$$
(X_{t-2}|Y_{t-2} ; X_{t-1}|Y_{t-1} ; X_{t}|Y_{t}) \rightarrow (Y_{t+k})
$$

where $|$ denotes vector concatenation to produce a 100 + 1 dimensional vector for each known year.

As a personal opinion, for the time-series prediction task, 24 data points per year is very small compared to the dimension of $X$, which is 100. 1200 samples for $X \rightarrow Y$ regression (ignoring the time) is more practical; if selecting 10 from 100 covariates is possible even better.

Because of the small data set, I would suggest:

$(Y_{t-m} ;...; Y_{t-1} ; Y_{t}) \rightarrow (Y_{t+k})$ for time series prediction, and
$X \rightarrow Y$ regression for estimating the relation between X and Y.

Relation to LSTM and RNN

If we use LSTM/RNN to model time-series, they would be stateful. That is, when input $X_{t-2}$ is fed to an LSTM, it keeps an internal state (hidden state) to be combined with the next input $X_{t-1}$ and so on. Regarding the input/output dimension, here is an RNN animation from a post on medium by Raimi Karim that shows an arbitrary step among 3 steps of feeding $(X_{t-2} ; X_{t-1} ; X_{t})$ to the network:

As you see, dimension and number of inputs are independent of output. We can feed 5 inputs $X_{t-4}$ to $X_{t}$, each 100 dimension (100d) and receive a 1d output by setting the dimension of hidden states to 1d, or setting it to 10d and use an extra dense layer at the end to convert 10d to 1d, or receive a 50d output, or a 150d (three 50d) output, etc.

Word "stateful" in Keras (source)

LSTM and RNN are stateful by definition, this [badly named] variable in Keras means

  If stateful=True, the last state for each sample at index i in a batch will be
  used as initial state for the sample of index i in the following
  batch. Fabien Chollet

For example, if each batch has 24 samples indexed from 0 to 23 (each sample could have the form $(X_{t-2}, X_{t-1}, X_{t}, Y_{t+1})$), then the last hidden state $h$ from 8th sample will be used as the initial hidden state for 8th sample in the next batch. Except for special cases that there is a temporal order between batches and their samples, this must be set to False.


Data-preprocessing for Machine Learning model

You should definitely not use one hot encoding with values which represent numbers, as this removes the natural order between your intervals.

So these values should be represented as numbers:

Either with the average of the limits indeed
Or a simple integer encoding of the intervals, e.g. tumor sizes 0-4, 5-9, 10-14,... would be represented as 0,1,2,...


Build a model to classify given string/text input

From the sample you have here it is not obvious that you need to apply some fancy ML technics. A simple rule based approach might also give you great results.
For instance:

if the string contains only numbers, then return B
If the string starts with 5 letters then return A
If the string starts with 2 letters and and then numbers return D
Etc.

That being said, to transform your strings into numbers a simple approach -looking at your data- could consist in assigning to each character a value in the set {0,1,...,9,10,..35,36}, 0 being assigned to the value 0, 9 to the value 9, A to the value 10, Z to the value 35, and NULL to the value 36 (as your strings don't have the same size, it might come in handy to introduce some placeholder for the blank values so that all your final vectors get to be of the same size)

Hope this helps!


There's a bit of confusion about your task: first, apparently by "emails" you mean email addresses, not the full content of the emails, right?
What you're doing looks like record linkage. The complexity of comparing every pair of records (here email addresses) can be decreased using the technique called blocking, where roughly similar pairs are grouped together in order to minimize the number of real comparisons (see Wikipedia page).
Now you're talking about using embeddings, and in this case what you'd need is character embeddings. However it's very unlikely that embeddings will help with the efficiency issue, quite the opposite actually.


FP16, FP32 - what is it all about? or is it just Bitsize for Float-Values (Python)

FP32 and FP16 mean 32-bit floating point and 16-bit floating point. GPUs originally focused on FP32 because these are the calculations needed for 3D games. Nowadays a lot of GPUs have native support of FP16 to speed up the calculation of neural networks. If you look at some benchmarks (https://blog.slavv.com/titan-rtx-quality-time-with-the-top-turing-gpu-fe110232a28e) you will see that GPUs that support FP16 are almost twice as fast calculating FP16 than FP32. Taking into account that newer cards that support FP16 (like NVidia 2080 series) are also about 20% faster for FP32 compared to their predecessor (1080) you get an increase of 140% to train FP16 neural networks compared to FP32 on previous cards.

But there is a caveat. Your neural network needs to be written using FP16 and it should also have the same accuracy. FP16 have lower accuracy by design, because they have much less bits to represent the same number. Some choose to use mixed precision models in order to be fast and accurate (https://hackernoon.com/rtx-2080ti-vs-gtx-1080ti-fastai-mixed-precision-training-comparisons-on-cifar-100-761d8f615d7f). But you see in the last link that the speed is faster for mixed precision, but is not 2 times faster as when you use pure FP16. You could also theoretically use FP32 weights and convert some of them to FP16 weights, but the accuracy could fall.

So in the end you need to understand whether you could rewrite your neural network to use FP16 fully or partially. If you cannot then you do not get any additional benefits from FP16 compatible cards.

The maximum value for FP16 is 65504 and the minimum is 5.96 × 10−8. The best precision for 5989.12345 will most likely be 5988.0 (played with bits on https://www.h-schmidt.net/FloatConverter/IEEE754.html) If this precision and magnitude is not enough for you then you could scale your data before training to fit FP16 and then train with double the speed. Or use mixed precision models where you have FP32 as input and then reduce precision in later layers.


torch.argmax has an extra argument dim which you can specify such that the maximum value is taken over a specific dimension. If you specify the dimension which represents the number of images it will return an array of indices where each value is for one image. For example:
import torch

# 3 images with 5 classes
t = torch.randn(3, 5)

# tensor([[-1.2917,  1.3740,  0.6967, -0.0575,  0.3702],
#        [ 0.5428,  1.0863,  0.3951,  1.8535,  1.0926],
#        [ 0.5865,  0.8522, -0.6858,  0.5297, -0.1320]])

# get the argmax over the first dimension, which specifies the number of images
torch.argmax(t, dim=1)

# tensor([1, 3, 1])


Understanding Embeddings input and output sizes

Both are correct.

Too clarify: when training in mini batches, it is more common to do padding after selecting eachmini batch. Say in your mini batch of size 16, if the longest sentence is of length 35, you want to pad this mini batch to 16x35x100.


Logistic Regression outperforms MLPClassifier

In ML, all algorithms are useful depending on the dataset. Its naive to generalize an algorithm to be always better than the other.

In your case since you have only 90 examples, mlpclassifier couldn’t have trained proper as compared to logreg.

A general suggestion: if you post like rows of your dataset along with accuracy of the two algorithms, it would have been a bit easier to discuss why there were differences.


Create clusters based on specific keywords

Methods for generating fuzzy (or soft) rather than crisp clusters may be applicable to your problem. One implementation for soft clustering in Python is a variant of DBSCAN called HDBSCAN. This link explains soft clustering using this algorithm. In soft clustering each point's similarity to each cluster can be measured. If a point is sufficiently similar to multiple clusters, then the clusters may overlap in the feature space.
There are also algorithms for explicitly generating overlapping cluster assignments from the literature such as:

In Overlapping Correlation Clustering (Bonchi 2013) each point may be assigned to multiple clusters based on pairwise similarities between points in the input set.
In Model-based Overlapping Clustering (Banerjee 2005) a generative approach is used to modeling the clusters.

For a survey of overlapping clustering approaches, see Overlapping Clustering: A review (Baadel 2016).


NLP: Getting the top 5 or top 10 predictions

There's predict_proba method in xgboost. You get probabilites for each class, sort them and take top 5.


Best neural network architecture for object detection without location

It mostly depends on the amount of data you have available for training.

However, unless you have an extremely limited dataset and you are unable to generate synthetic data, CNNs are currently the best approach for solving your task.  With sufficient data, they are very robust.  To improve their performance, you can augment your dataset by randomly rotating, flipping, brightening, contrasting, scaling, cropping, blurring, and adding noise to the images throughout training.

It also depends on your speed requirements and computational resources.  Traditional methods have the potential to be quite fast, but if you only care about accuracy, a deep network would probably perform better.  Note that some CNNs are also quite fast, such as MobileNets.

There are a few reasons you might go "old school" and use template matching and more traditional approaches as suggested by @TheDjentleman:

You do not have enough data to make a CNN robust AND you cannot generate synthetic data
CNNs do not meet your speed requirements AND the traditional methods do
You do not have sufficient resources to train a deep network, such as GPU(s).  FYI Google developed a cloud computing tool for researchers and developers at a pretty reasonable cost.

If none of those criteria fit your situation, go with a CNN.

Lastly, you do not necessarily need a generic binary classification CNN.  You can use an object detection network or even a semantic segmentation network and just ignore the location data if you do not need it.


Tensorflow simple neural network has very bad performance in regression (curve fitting) problem

I am not sure which network training function is your MATLAB feedforwardnet is using. Also, most likely the activation function in your MATLAB net is 'tansig'. Try using 'tanh' activation function in your python/keras code.


This is a typical application of the ARIMA model. By choosing the proper order (p,q and d in the wikipedia page), this model can be trained effectively with relatively small amount of data.


Extract 2 pieces of information from a string - what to use?

Maybe you can use Python with dictionaries.

You can set a group of word in the dictionary that could be filled each time with new words you find.

  To find new word (if the pattern remains the same as your example) you
  can grab the "brand" (Samsung) and the next word will be the model
  (UE65KS7500).

This is a good resource


Doc2vec(gensim) - How to calculate the most similar sentence and get its label?

As far as I understood you are using type of TV as tag of particular sentence , and you are using doc2vec model for future classification . So As above answer is suggesting that model will learn semantic meaning of type of TV(tag).
let's suppose s is your future sentence for prediction. then you use infer vector.

model = Load_model('model.doc2vec')
infer_vector = model.infer_vector(s)
similar_documents = model.docvecs.most_similar([infer_vector], topn = 1)

here similar document is list of tuples. where first element is label.
let me know if this help you.


Why is this binary state format so much more efficient than a non-binary one?

You are increasing the state space by introducing non-binary options. This increases the complexity the model has to learn to reach the same level of performance.
In particular, you are creating a nested structure which makes it more difficult to assign reward credit to a specific option. The hierarchical options are "Should the snake move?" then "If yes, which direction?" make it slower to assign credit to the directional movement. A flat structure "Should the the snake move up?" allows credit to be more directly assigned to that decision.


PyTorch CNN network outputs homogeneous results

So, I think I figured out what the problem was. It had nothing to do with the algorithm in itself but the data. The data which was being fed to the network was a sparse tensor; the thing is that the network wasn't being able to learn from its mistakes because every time it got it wrong it received a zero correction to the weight.

Because of that, I simply made a transformation $X\mapsto 2X-1$ on the tensor, switching the zeros by -1's and the networks started giving heterogeneous results.


Propensity Modeling, still use Test/Train Split?

You are right; you shouldn't predict on the same data you have used for training your model. If your goal is to only output probabilities and you don't mind achieving this by having not a single classifier but a number of them, you can potentially use nested cross-validation to achieve that.

In the outer cross-validation steps, you break your data in $N_{out}$ folds, where one fold is kept for testing the model and the remaining $N_{out} - 1$ folds are used to train. You then feed this training dataset (which is similar, in nature, to that 70% you have mentioned in OP) to the inner cross-validation step with $N_{in}$ folds. Here, you will compare different models and choose the best performing one. This chosen model is then passed to the outer fold (see above) and can be used to obtain probabilities for the 1 fold that was left out. You then repeat the same procedure for other folds in the outer fold. At the end of this nested cross-validation process, you will have probabilities for all your rows (but they will have come from $N_{out}$ different classifiers, each corresponding to one of the outer folds).

Note that the ultimate purpose of nested cross-validation is not to do this but it will give you what you want as a by-product.


What are the differences and advantages of TensorFlow and Octave for machine learning?

Octave is a great language for prototyping and experimenting with ML algorithms, as it has built-in support for numerical linear algebra such as matrix and vector calculations. Octave is optimized for rapid calculations, which is very useful in Machine Learning. It is also quite easy to do matrix multiplications in Octave as Matrices are first-class objects in Octave.

Tensorflow is indeed a versatile platform for machine learning with an ever-expanding list of packages and frameworks getting built.

Octave is a good tool for learning the essentials and internals of mathematics of machine learning and Tensorflow is a good platform for building industry solutions for machine learning projects. Hence both are good for their own purposes.


I don't know of such a dataset, but I would suggest you to borrow ideas from mobile phone accelerometer projects. e.g.  the links below can be a good inspiration.

To proceed, I would record their daily movement and generate the normal movement thresholds (probably you could just use an off the shelf android phone utilizing its motion sensor).

Based on the collected dataset, you will define what are the normal movements (e.g. you can start with fixed rules). Later you dynamically adjust based on the feedback from your parents.

Concerning machine learning algorithms, After you collect high dimensional data for long enough time you can proceed with the algorithms you mentioned above.

(e.g. setting it as a classification algorithm)

(speed, time, pt1, pt2, ...) -> fall
(speed, time, pt1, pt2, ...) -> no_fall

good luck!

[1] step detection https://github.com/danielmurray/adaptiv

[2] sensors overview https://developer.android.com/guide/topics/sensors/sensors_overview


Can I do bagging method as improvement technique to decision tree in research?

Let's clarify a few things first:

The bagging technique is an ensemble method which is not specific to decision trees, it can be applied to any classification method.
It's worth noting that there is another ensemble method specifically for decision trees, it's called Random Forest. While it's not the same method, it is known to generally improve performance compared to a regular Decision Tree algorithm like C4.5.

These techniques exist because they have been proved to improve performance in general, but whether they would improve performance on a specific problem (and by how much) has to be tested.
Also just to be clear: these techniques are already established, so using them wouldn't be an original research contribution in the field of Machine Learning. Their application to a new problem in a specific research domain might be a contribution to this domain, but that depends on the specific context.


Experience replay in Reinforcement learning - Batch Size

Definitely using experience replay can slow down the agent processing each time step, because typically on each time step, a result is stored (possibly requiring another result to be deleted), a mini-batch number of results are chosen randomly and fetched, then the function approximator (usually a NN) has to be run forward for the mini-batch to discover max Q values (potentially multiple times per item if the NN uses action as input instead of multiple action values as output). Then it has to run a forward pass to calculate error values and back propagation and weight updates over the mini-batch. All that can take considerable amount of CPU and memory bandwidth compared to the same system running without experience replay.

However, there are a lot of factors to consider whether this is important. Here are a few that I can think of:

Typically the environments using DQN agents are not training in real time. Often they run multiple times the speed of real time in order to collect data quickly and solve the environment faster.
Obviously something without the overhead of experience replay could collect more data in the same real time if the environment is simulated, but actually the stabilising effect of experience replay is more important and the effective "wall clock" speed of learning is faster and more reliable, using less game play experience.
Due to the way mini-batch processing is parallelised for neural networks on GPUs, there is not such a high overhead for increasing mini-batch size as you may think. At least up to the limits of GPU. There's still a cost for memory transfers etc per item, so it's not completely free.

  when the program starts the fit function on the replay I guess it is not sending the action to the agent,

That is correct, the agent is updating its internal estimates of values in order to learn the optimal control actions for later actions. The mini-batch actions are not taken, just used to refine the estimates.

  but the environment is still running.

That is only correct in real-time systems. The Gym environments are not real-time as far as I know. They all run at the speed that the agent wants them to - they are effectively a turn-based game for the agent to play, although many of them are based on systems that run at a specific real-time rate. However, real-time reinforcement learning and control systems are a thing in other domains. If learning must be done online in such a system, then typically you have some choices on how to design the system:

Have fixed time steps, and allow for maximum computation time between each one (with perhaps a margin of error).
Asynchronous interactions with the environment - with fixed time steps for sampling observations and taking actions - and learning continuously sampling from recent history and updating value functions and/or policy.
Allow for variable length time steps. I don't know much about this approach, but it is an option.

Which to use depends on the problem, and a bit of guesswork.


How do you use KS-test in a data science report?

In your case, the null hypothesis $H_0$ is that your sample follows your the distribution that your model has learned. The alternative hypothesis $H_1$ is that it follows some other distribution. Assuming you have fixed your significance level $\alpha$ to be $0.05$ (the most common choice for $\alpha$, but up to you if you want to go lower), getting a p-value lower than that means you should reject the null hypothesis.
The p-value can be interpreted as the probability of a type I error, in other words a false positive: the probability that you reject the null hypothesis when it is in fact true. In your case, rejecting the hypothesis means stating that there is statistically significant evidence that the distribution your model has learned is not the underlying distribution of the sample. So yes, you would like as large a p-value as possible.
You are using a Kolmogorov-Smirnov test to compare your sample to a reference distribution, in this case, so it's a one-sample KS test. The way I would put it is that getting a high p-value means that: "it is highly unlikely that your model has learned a wrong distribution". In other words, it is highly likely it has learned a pretty good approximation of the underlying distribution. However, nothing is certain when doing statistical hypothesis testing!
I'm not sure what you're showing on your plots though, since there doesn't seem to be an empirical cumulative distribution function on them (lines look smooth).


The cross entropy formula takes in two distributions, $p(x)$, the true distribution, and $q(x)$, the estimated distribution, defined over the discrete variable $x$ and is given by

$$H(p,q) = -\sum_{\forall x} p(x) \log(q(x))$$

For a neural network, the calculation is independent of the following:

What kind of layer was used.
What kind of activation was used - although many activations will not be compatible with the calculation because their outputs are not interpretable as probabilities (i.e., their outputs are negative, greater than 1, or do not sum to 1). Softmax is often used for multiclass classification because it guarantees a well-behaved probability distribution function.

For a neural network, you will usually see the equation written in a form where $\mathbf{y}$ is the ground truth vector and $\mathbf{\hat{y}}$ (or some other value taken direct from the last layer output) is the estimate. For a single example, it would look like this:

$$L = - \mathbf{y} \cdot \log(\mathbf{\hat{y}})$$

where $\cdot$ is the inner product.

Your example ground truth $\mathbf{y}$ gives all probability to the first value, and the other values are zero, so we can ignore them, and just use the matching term from your estimates $\mathbf{\hat{y}}$

$L = -(1\times log(0.1) + 0 \times \log(0.5) + ...)$

$L = - log(0.1) \approx 2.303$

An important point from comments

  That means, the loss would be same no matter if the predictions are $[0.1, 0.5, 0.1, 0.1, 0.2]$ or $[0.1, 0.6, 0.1, 0.1, 0.1]$?

Yes, this is a key feature of multiclass logloss, it rewards/penalises probabilities of correct classes only. The value is independent of how the remaining probability is split between incorrect classes.

You will often see this equation averaged over all examples as a cost function. It is not always strictly adhered to in descriptions, but usually a loss function is lower level and describes how a single instance or component determines an error value, whilst a cost function is higher level and describes how a complete system is evaluated for optimisation. A cost function based on multiclass log loss for data set of size $N$ might look like this:

$$J = - \frac{1}{N}\left(\sum_{i=1}^{N} \mathbf{y_i} \cdot \log(\mathbf{\hat{y}_i})\right)$$

Many implementations will require your ground truth values to be one-hot encoded (with a single true class), because that allows for some extra optimisation. However, in principle the cross entropy loss can be calculated - and optimised - when this is not the case.


Machine learning methods on 1 feature dataset

Can I use any machine learning methods having only one feature?

Yes!

In fact, many NLP classifications tasks are in this format. Given 1 piece of text, classify something. For example:

Given 1 review, classify the sentiment
Given 1 news article, classify the topic
Given 1 chat message, classify the intent

And now you have:

Given 1 name, classify the Fullname

Can a better method be used?

Like you mentioned, you could just find the most common Fullname for a given name and every time you get a name you have a lookup table for the Fullname. However, what will happen when a name you have never seen before appears, how do you classify it? Are you also assuming that you already have the full list of Fullnames?

Assumption: you know all Names and Fullnames

In this case, do as you suggested. Create a dictionary mapping Name-Fullname by finding the most common Fullname for every Name.

Assumption: you know all Fullnames but not all Names

Let say you have the mappings:

Peter -> Johnson
John  -> Smith

Then, there is a name you have never seen before, Pete for example, which does not appear in your mapping table.

You could try two approaches:

The simple way - find which name in the mapping is closest to Pete using some word distance measure, like Levenshtein.
The more robust way - forget the notion of mapping table and use a machine learning model. You will need the following things:

A text vectorizer to transform your text into a numerical vector. I would suggest a character level n-gram TF-IDF.
A classifier. If you use the vectorizer I suggested, then you will need a linear classifier, like an SVM.

If you go to with approach two, when you encounter the name Pete, it will be spit into n-grams (e.g. [pe, et, te, pet, ete]) and vectorized.

Assumption: you don't know all Fullnames and you don't know all Names

This gets more interesting because you could be working with Fullname generation.

It could be used when you move to names from other countries as well.

For example, you already have the mapping:

Peter -> Johnson
John  -> Smith

Then you start dealing with Dutch names and encounter Pieter and Jan.
Then you might want to get the following results where even the Fullnames are different:

Pieter -> Janssen
Jan    -> Smeets

For this, you could use a seq-to-seq Recurrent Neural Network. The architecture can be similar to ones used for neural language translation.

However, all embeddings you create have to be character level. Instead of learning an embedding for every word, you learn for every character. You also feed your network one character at a time. This way, you will be less likely to find "out of vocabulary" tokens (except for when you find character from another alphabet).


Trying to implement a "smart compose" feature

From the same blog,

  In this hybrid approach, we encode the subject and previous email by averaging the word embeddings in each field. We then join those averaged embeddings, and feed them to the target sequence RNN-LM at every decoding step.

The BoW part of their hybrid approach is to get the general context of the email conversation by averaging the word embeddings in the subject and the previous email. Passing this vectorized representation of the general context at each step of the RNN-LM, which takes into account word order, helps get predictions that are more tailored to the subject of the conversation.

Note that at each step, the RNN-LM is getting 3 types of input -

Embedding for the previous word - For immediate context.
State vector - For localized context
Subject and previous email averaged embeddings - For a wider, conversation-level context

Hopefully, it is clearer now, that the model incorporates order with the use of RNN-LM and the BoW part is to condition the output to the subject of the conversation.


Yep this is a common problem. What I would do is use SKLearns label encoder. With a similar API to SKLearn models, it can be fit to your category - meaning that any new data passed through the encoder object is encoded in the same fashion. For example

# Import the encoder
from sklearn.preprocessing import LabelEncoder

# Fit it to your training set + transform
encoder = LabelEncoder()
train[categorical_feature] = encoder.fit_transform(train[categorical_feature])

# Use it on the test set to ensure the same transformation
test[categorical_feature] = encoder.transform(test[categorical_feature])

Huge caveat # 1

This assumes every possible category is present in your training set. If you think this isn't the case, fit the encoder to a dataset that does have them all.

Huge caveat # 2

Depending on the model you're using, categoricals still aren't 'processed' when they've been label-encoded , you probably still need to one hot encode (SKLearn has a package for that too).


Yes, in this case train_x and train_y both have just 1 column.  RNN learns patterns in just that variable and tries to predict next instance based on the patterns.

Usually, this is just a theoretical exercise. In real world, you will have multiple features.

Some examples :

https://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-python-keras/

https://machinelearningmastery.com/time-series-forecasting-long-short-term-memory-network-python/


Early-Stopping for logistic regression. Theano

Patience is the number of training batches to do before you stop. iter is the number of training batches you've seen. Each iteration, you decide whether or not your validation is lower than your previous best. The improvement is only stored if the score is lower than improvement_threshold * validation_score.

It appears that patience_increase is a multiplier.  Every time you have a new lowest score, you up the total number or training batches to iter*patience_increase, but not below the current value of patience.

validation_frequency is just the number of batches between times you check the validation score.


Entity Linking is a type of supervised machine learning, thus many of the common performance metrics could be used. In particular, creating a confusion matrix would identify where one label was predicted but the ground-truth was different. Confusion matrices can be calculated with counts or normalized, a normalized data would be an estimate of "ambiguity degree" relative to the other labels in the current dataset.

Other classification measures such has F-score, precision, and recall could also be used. In particular, low precision for a label would suggest the model has trouble disambiguating entities from nearby text. "Cheap and easy entity evaluation" goes into more technical details.

Inter-rater reliability could also be used, the raters could be different humans or different models. If the joint-probability of agreement between different raters is low then entities could be regarded as difficult to disambiguate.

The performance also depends on the relative value of an exact match vs partial match.


Generating a set of different scenarios based on some initial observations

You could do block bootstrapping. For example GDP was 100, 150, 300, 100, 130. In percentage it is +50%, +100%, -67%, +30%. You choose blocks of two. So you have two blocks (+50%, +100%) and (-67%, +30%). So you take random blocks for your scenario. Possible variants:

+50%, +100%, -67%, +30%
+50%, +100%, +50%, +100%
-67%, +30%, -67%, +30%
-67%, +30%, +50%, +100%

The blocks allow you to hold autocorrelation in the data and if one block includes all variables then all correlations will also remain the same.

An explanation on how to do block bootstrapping on Cross Validated StackExchange site:
https://stats.stackexchange.com/questions/25706/how-do-you-do-bootstrapping-with-time-series-data


Why would one crossvalidate the random state number?

I personally think that the general idea of optimising your model with different random seeds is not a good idea. There are many other, more important, aspects of the modelling process that you can worry about, tweak and compare before spending time on the effects of random initialisation.

That being said, if you just want to test the effect of random initialisation of model weights on a final validation metric, this could be an approach to do so. Kind of the reverse argument to my point above. If you can show for different random seeds (ceteris paribus: with all other parameters equal) that the final model performs differently, it shows maybe that their is either inconsistency in the model, or a bug in the code even. I would not expect a well-validated model to give hugely differing results if being run with a different random seed, so if it does, it tells me something weird is going on!


How to process multivariant questions in Orange

Since you say the order of picks doesn't matter, I think your data will be more easily manageable (in any tool, including Orange) if you first transform the database into a wide table of one-hot encoded features, with every option its own column:

id    weekend_home    weekend_friends    weekend_family    weekend_work
1     1               1                  1                 0
2     1               1                  0                 0
3     0               1                  0                 0
4     1               1                  0                 0
5     1               1                  1                 0


What the function is doing for each variable?

Record the Out-Of-Bag (OOB) accuracy for each tree.
"shuffle" or permute the values of that variable.  This means you take all the values of that variable in the data, and assign those values randomly back out to the observations, which is a way of introducing noise and getting rid of the signal that that variable provided.
Now it finds the OOB accuracy again, but this time the values for that variable are incorrect since we permuted it.  By introducing noise where your model expects signal, you should see a decrease in performance.
Compare the original accuracy in (1) to the accuracy in (3) for each variable.  If the model performance decreases a lot for a variable in step (3) compared to (1), then it is deemed to have greater importance.

Why does removing the most important variable not have a negative affect on accuracy? (my guess)

Probably because that important variable is correlated with some other variable(s) you have.  Your model can capture the information contained in that missing important variable by using a few other variables to make up for it.  When you drop the important variable, which other variables see a notable gain?


Every one knows data-driven modeling, but what is model-driven (or non data-driven) modeling?

If the model is not derived from the data then it must be built manually, so non data driven means rule based.
This was the big trend in AI in the 80s before Machine Learning, these pre-ML automatic prediction systems were called expert systems and they were quite successful at the time in industry (here are some examples of applications).
The way one would build a system for a regression task is essentially this: do the whole regression analysis manually, find the parameters and hard-code them in the prediction system.
As far as I know Machine Learning has pretty much made this kind of rule-based systems obsolete, due to their total lack of flexibility and the very high cost in manual labor to build one.


how classification scores are interpreted?

If the scores have values that are higher than 1, I wouldn't call them probabilities. Probabilities should always be between 0 and 1.
And indeed, the higher the score, the more likely an example is to be positive, this is the most natural interpretation of a score.


Where is the cost parameter C in the RBF kernel in SVM?

The C parameter in SVMs doesn't have to do anything with the kernel function. C is the penalty associated to the instances which are either misclassified or violates the maximal margin. As you may know already, SVM returns the maximum margin for the linearly separable datasets (in the kernel space).

It might be the case that the dataset is not linearly separable. In this case the corresponding SVM quadratic program is unsolvable. To make it solvable, we redefine the quadratic program to return the maximum margin with respect to some error cost C for the points violating the maximum margin (misclassified samples also violate the maximum margin). Below you can see a simple example I found on the Internet. This uses linear kernel but the story is the same for other non-linear kernels as well.

The effect C has on the classification result is shown in the picture below:

As you can see, if you set the error penalty to be very high, you will end up with less misclassifications and fewer support vectors (the highlighted points are SVs). However, if you set it too high when you are using non-linear kernels (specially Gaussian) you might overfit.

If you are interested in the theory behind it, here is what we are trying to optimize:

If we write its dual, we'll get:

See how convenient it is! No C in the kernel function!

In other words, you are mapping the samples to a higher dimensional space using the kernel function, then solving it using some penalty term C.


How do I access data on my EBS Volume from R-Studio Server on Ubuntu EC2 Instance

I see this question got up-voted so someone else is looking for the answer. Here it is:

I was logged in as the default user on Ubuntu instance when i did the

$s3cmd get s3://data-analysis/input-data/filename.csv

The data got saved from the s3 bucket to the home directory of the user named ubuntu. this can be verified with :

$ df -h /home/ubuntu/

Since I had followed randy zwitch's tutorial on installing R on aws - i had created a user named rstudio and that was the user name I was using to log-in to rstudio server. Hence I had to move the file from the default 'ubuntu' user's home directory to the user 'rstudio'. this can be done with:

$ sudo mv /home/ubuntu/filename.csv /home/rstudio

Happy to answer follow up questions.


Training a convolutional neural network for image denoising in Matlab

So, I can point a few things that may cause a sub-optimal result:

Your model may be too simple. Although using CNNs, which are pretty much the state of the art in Neural Networks based image processing, you need to pay attention to the archietcture you are using. For instance, you can take a look into that paper, where the authors have used CNNs + residual learning to denoise images
Your dataset isn't complex enough/isn't big enough. As far as you said, you have pre-trained the network on a much larger dataset, however, the notion of "large dataset" isn't precise enough. Maybe you don't have enough data to train the model. In that sense, it may be useful use a well documented existing architecture where you can compare your results with those reported, and see if you indeed have the data needed to train it.
Another concern is about the noise on train/test data. If your model is to complex, and your data too poor, it may be memorizing the mechanics of denoising gaussian noise (which seems to be the noise component in your training data), and hence does not generalize good on Poisson noise. To solve this, you may consider change the noise component on the train data (if it was generated artificially), or getting a different dataset.
Change losses, take a look on other metrics. Loss choice have a great impact on the final model, since they determine how the gradient is calculated. For instance, following Goodfellow, Bengio and Aaron reasoning on the book "Deep Learning",

  For example, mean squared error applied to the pixels of an image implicitly
  specifies that an underlying cause is only salient if it significantly changes the
  brightness of a large number of pixels. This can be problematic if the task we
  wish to solve involves interacting with small objects.
  Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016.

Hence, despite looking into RMSE, I may recommend Peak to Signal Noise Ratio (PSNR) or Structural Similarity (SSIM).


Q1: I do not really understand how the situation in your Q1 is possible - I would expect an error to be thrown about as a mismatch in shape. For example, when I change the number of classes in the final dense layer, I do indeed get an error.

model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),
                 activation='relu',
                 input_shape=input_shape))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
# adjust the number of layers from 10 to 20 - which throws an error
model.add(Dense(20, activation='softmax'))
model.compile(loss= 'categorical_crossentropy',
          optimizer = 'rmsprop',
          metrics=['accuracy'])

Output:

  ValueError: A target array with shape (60000, 10) was passed for an
  output of shape (None, 20) while using as loss
  categorical_crossentropy. This loss expects targets to have the same
  shape as the output.

Perhaps you can post your code when you build in the layers.

Q2: The list of probabilities that are output by the model will match the order of the classes on which you trained it. For example, if the order of the classes for the digits 0-9 is [0, 1, 2, 3, ... 9] and the model outputs [0.91, 0.01... 0.01] for a certain image, then the model thinks there is a 91% chance that the image is a '0'


A simple representation of the problem in terms of a genetic algorithm could be something like this:

A gene represents a chemical and the values it takes represent the proportion of the chemical in the drug
An individual is a combination of N genes/chemicals, each with their particular proportion
A cross-over is the combination of the genes of two individuals A and B, either by picking randomly the value of either A or B for each gene or by some other kind of aggregation. A mutation is the random modification of a gene to a different proportion.

The standard genetic algorithm works like this:

Randomly pick a set of say 100 individuals (first generation)
Calculate the "performance" of every individual, i.e. evaluate how good this particular combination of chemicals proportions is.
Select say the top 10 individuals according to their performance, then produce the next generation of 100 individuals by cross-over among these top 10. Optionally add some random mutations to the new individuals' genes.
Iterate again from step 2. Keep iterating unless some stop condition is satisfied, for example the average performance over the last 5 generations doesn't increase anymore.

As far as I understand the task, the main problem is the evaluation in step 2: if there is no automatic way to evaluate the performance of a combination of chemicals, it's impossible to use the genetic algorithm. In theory the evaluation doesn't need to be automatic, one could imagine doing a manual experiment for every individual, but that means performing thousands of experiments, a lot of them probably useless.


How to approach a Data Science case study question?

This question (something I've asked variants of several times in interviews) has absolutely nothing to do with statistical or other quantitative procedures. What is being asked here is for an understanding of the overall data mining process itself.  The first thing to determine is what the definition of success. So you have to ask. The stakeholder usually will not volunteer this unless asked anyway. Then, depending on the answer describe the overall process for data mining based on this end goal.


Supervised learning for a turn-based game?

Maybe the correct way of addressing this is by making sub optimizations of every step, even though it could be done by regression, I would suggest decision trees.

You have and advantage: A game is made of discrete steps, so in every moment you can "stop" and decide the best move based on your (possibly comprehensive) history of moves.

Supervised learning vs reinforcement learning for a simple self driving rc car


Random Forest where objective is not to replicate past classifications

Your target variable should always reflect what you're trying to optimize (maximize or reduce), so if you want to maximize revenue, you should make revenue your target variable. I would use the coupons as independent variables. When you want to see whether to offer a coupon or not for a particular customer, feed a new record into your model for every permutation of coupon / no coupon and see which has the highest prediction. There might be other/better ways of doing this, but I think this should get what you're looking for.


PPO, A2C for continuous action spaces, math and code

Packt Publishing's "Deep Reinforcement Learning Hands-On" has an entire chapter on continuous action spaces.  Here is the math in the book:

and the code accompanying the book:  code repo for book

The code for the math above is in:

  The complete source code is in Chapter14/02_train_a2c.py ,
  Chapter14/lib/model.py and Chapter14/lib/common.py . Most of the code
  will already be familiar to you, so the following includes only the
  parts that differ. Let's start with the model class defined in
  Chapter14/lib/model.py .


Fitting an arimax model on out of sample dataset

the arima function from the stats library does not take an argument called model, which is why you are receiving an error. Here is the function signature:

arima(x, order = c(0L, 0L, 0L),
      seasonal = list(order = c(0L, 0L, 0L), period = NA),
      xreg = NULL, include.mean = TRUE,
      transform.pars = TRUE,
      fixed = NULL, init = NULL,
      method = c("CSS-ML", "ML", "CSS"), n.cond,
      SSinit = c("Gardner1980", "Rossignol2011"),
      optim.method = "BFGS",
      optim.control = list(), kappa = 1e6)

The returned object, does contain the model. Read the related documentation for more details.

I think your workflow is perhaps a little confused. You fit a model first on 20 data points, which is fine (more data would be nice!). You make some forecasts and plot it, which is also good - you can see if the model learned much and if there is perhaps some systematic error, e.g. simply predicting the previous time-step and not a more intelligent trend.

The final step, however, should be to once again make predictions on your hold-out data; the last 4 data points. You should not fit another model to the hold-out data! Just predict what you model would say for that data, which it has never before seen.

The reason we work like this, is so we can assess the model's performance independently from the data that was used to train it. We want to know what will happen in the future, when you get your 25th datapoint.

Have a look at this nice tutorial, which explains the mains concepts of ARIMA and has a working example. Here is a very similar tutorial, but it is a video.


Classification model accuracy with ensemble methods

This comes from the Binomial distribution, where you have $n=1000$ independent trials (models), $p=0.51$ of each model being right and since you care about the majority vote you want to have at least $k=500$ successful trials. That leads to:
$$\text{Pr}(k\geq500 \text{ models are right}) = \sum^{1000}_{k=500}\binom{1000}{k}0.51^{k}(1-0.51)^{1000-k}=0.74675\approx0.75$$
Here is how I calculated it:
    import numpy as np
    from scipy.stats import binom
    np.sum([binom.pmf(k,1000,0.51) for k in range(500,1001)])


How do neural networks account for outliers?

Here's a math answer for you.

Neural network is an approximation function $f(\theta)$ of the joint distribution $p(X, Y)$ of input data $X$ and labels $Y$. The learning process is the process of tweaking parameters $\theta$ to make $f$ as close as possible to $p$

$$f(\theta) \approx p(X, Y)$$

Side note: usually $f$ is considered to approximate the conditional $p(Y|X)$, but it can be viewed more generally.

So, in this terms, outliers are the values $(x, y)$ clearly outside of the distribution $p(X, Y)$. Hence, a neural network can not account for them, this has to be done separately. One possibility is to gather more data to make these outliers look ordinary, i.e., from $p(X, Y)$.

As for overfitting, it's completely different term, which relates to inability to generalize. This happens because in practice the true distribution $p(X, Y)$ is never fully known. Instead, the researches have the sample of it -- $(\hat{X}, \hat{Y})$, a.k.a. the training data. If the dimensionality of $\theta$ is large, $f$ can approximate $p(\hat{X}, \hat{Y})$ so well that it actually learns the noise and fails to capture $p(X, Y)$. There are ways to deal with overfitting, most importantly regularization, which is equivalent to adding noise to $(\hat{X}, \hat{Y})$. So the answer to this question is: it can be extended to account for it, by using special techniques. But if the researcher does nothing, the neural network won't learn to deal with overfitting by itself.


Partitioning data into features/labels and train/test after reading from csv file

I don't know exactly how your data is but y = data.temp may be a Series containing the string values which should be cast to float values. Try to change it to the following alternative.

  y = data.temp.astype(float)