Understanding the mathematics of AdaGrad and AdaDelta

With regard to resources:

In my opinion, ADADELTA: An Adaptive Learning Rate Method (the original ADADELTA paper) explains (in sections 1-3) both ADAGRAD and ADADELTA in a quite accessible manner.
I found Adaptive Subgradient Methods for Online Learning and Stochastic Optimization to be less accessible, but it is the original ADAGRAD paper, so it is probably worth a shot.
An overview of gradient descent optimization algorithms (a blog post by Sebastian Ruder) also helped me understand both ADAGRAD and ADADELTA.

Here are some central quotes from ADADELTA: An Adaptive Learning Rate Method, along with some examples and short explanations:

ADAGRAD

  The update rule for ADAGRAD is
  as follows: $$\begin{matrix}\Delta x_{t}=-\frac{\eta}{\sqrt{\sum_{\tau=1}^{t}g_{\tau}^{2}}}g_{t} &  &  & (5)\end{matrix}$$
  Here the denominator computes the $l2$ norm of all previous
  gradients on a per-dimension basis and η is a global learning
  rate shared by all dimensions.While there is the hand tuned global learning rate, each dimension has its own dynamic rate.

I.e. if the gradients in the first three steps are $g_{1}=\left(\begin{gathered}a_{1}\\
b_{1}\\
c_{1}
\end{gathered}
\right)\,,\,g_{2}=\left(\begin{gathered}a_{2}\\
b_{2}\\
c_{2}
\end{gathered}
\right)\,,\,g_{3}=\left(\begin{gathered}a_{3}\\
b_{3}\\
c_{3}
\end{gathered}
\right)$, then: $$\begin{gathered}\Delta x_{3}=-\frac{\eta}{\sqrt{\sum_{\tau=1}^{3}g_{\tau}^{2}}}g_{3}=-\frac{\eta}{\sqrt{\left(\begin{gathered}a_{1}^{2}+a_{2}^{2}+a_{3}^{2}\\
b_{1}^{2}+b_{2}^{2}+b_{3}^{2}\\
c_{1}^{2}+c_{2}^{2}+c_{3}^{2}
\end{gathered}
\right)}}\left(\begin{gathered}a_{3}\\
b_{3}\\
c_{3}
\end{gathered}
\right)\\
\downarrow\\
\Delta x_{3}=-\left(\begin{gathered}\frac{\eta}{\sqrt{a_{1}^{2}+a_{2}^{2}+a_{3}^{2}}}a_{3}\\
\frac{\eta}{\sqrt{b_{1}^{2}+b_{2}^{2}+b_{3}^{2}}}b_{3}\\
\frac{\eta}{\sqrt{c_{1}^{2}+c_{2}^{2}+c_{3}^{2}}}c_{3}
\end{gathered}
\right)
\end{gathered}
$$
Here it is easier to see that each dimension has its own dynamic learning rate, as promised.

Problems of ADAGRAD that ADADELTA tries to counter

  The idea presented in this paper was derived from ADAGRAD in order to improve upon the two main drawbacks
  of the method: 1) the continual decay of learning rates
  throughout training, and 2) the need for a manually selected
  global learning rate.

The second drawback is quite self-explanatory.

Here is an example for when the first drawback is an issue:
Consider a case in which the absolute value of each component of $g_2$ is much larger than the absolute value of the respective component of the gradient in any other step.
For any $t>2$, it holds that every component of $\sqrt{\sum_{\tau=1}^{t}g_{\tau}^{2}}$ is bigger than the absolute value of the respective component of $g_2$. But the absolute value of every component of $g_2$ is much bigger than the absolute value of the respective component of $g_t$, and so $\Delta x_t$ is very small.
Moreover, as the algorithm progresses, it gets closer to a minimum, so the gradient gets smaller, and so $\Delta x_t$ becomes smaller and smaller.
Thus, it might be that the algorithm virtually comes to a standstill before reaching a minimum.

ADADELTA

Instead of considering all of the gradients that were calculated, ADADELTA considers only the last $w$ gradients.

  Since storing $w$ previous squared gradients is inefficient,
  our methods implements this accumulation as an exponentially
  decaying average of the squared gradients. Assume at
  time $t$ this running average is $E\left[g^{2}\right]_{t}$ then we compute:
  $$\begin{matrix}E\left[g^{2}\right]_{t}=\rho E\left[g^{2}\right]_{t-1}+\left(1-\rho\right)g_{t}^{2} &  &  & (8)\end{matrix}$$
  where $\rho$ is a decay constant [...].  Since we require the square root of this quantity
  in the parameter updates, this effectively becomes the $\text{RMS}$
  of previous squared gradients up to time $t$:
  $$\begin{matrix}\text{RMS}\left[g\right]_{t}=\sqrt{E\left[g^{2}\right]_{t}+\epsilon} &  &  & (9)\end{matrix}$$
  where a constant $\epsilon$ is added to better condition the denominator

($\text{RMS}$ stands for Root Mean Square.)

Similarly:
$$E\left[\Delta x^{2}\right]_{t-1}=\rho E\left[\Delta x^{2}\right]_{t-2}+\left(1-\rho\right)\Delta x_{t-1}^{2}$$
$$\text{RMS}\left[\Delta x\right]_{t-1}=\sqrt{E\left[\Delta x^{2}\right]_{t-1}+\epsilon}$$
And finally:

  [...] approximate $\Delta x_{t}$ by compute the exponentially
  decaying $\text{RMS}$ over a window of size $w$ of previous $\Delta x$ to
  give the ADADELTA method:
  $$\begin{matrix}\Delta x_{t}=-\frac{\text{RMS}\left[\Delta x\right]_{t-1}}{\text{RMS}\left[g\right]_{t}}g_{t} &  &  & (14)\end{matrix}$$
  where the same constant $\epsilon$ is added to the numerator $\text{RMS}$ as
  well. This constant serves the purpose both to start off the first
  iteration where $\Delta x_{0}=0$ and to ensure progress continues to
  be made even if previous updates become small.[...]The numerator acts as
  an acceleration term, accumulating previous gradients over a
  window of time [...]

I.e. if the gradient in step $r$ is $g_{r}=\left(\begin{gathered}a_{r}\\
b_{r}\\
c_{r}
\end{gathered}
\right)$ and $\Delta x_{r}=\left(\begin{gathered}i_{r}\\
j_{r}\\
k_{r}
\end{gathered}
\right)$, then:
$$\begin{gathered}\Delta x_{t}=-\frac{\text{RMS}\left[\Delta x\right]_{t-1}}{\text{RMS}\left[g\right]_{t}}g_{t}=-\frac{\sqrt{E\left[\Delta x^{2}\right]_{t-1}+\epsilon}}{\sqrt{E\left[g^{2}\right]_{t}+\epsilon}}g_{t}=\\
\\
-\frac{\sqrt{\rho E\left[\Delta x^{2}\right]_{t-2}+\left(1-\rho\right)\Delta x_{t-1}^{2}+\epsilon}}{\sqrt{\rho E\left[g^{2}\right]_{t-1}+\left(1-\rho\right)g_{t}^{2}+\epsilon}}g_{t}=\\
\\
-\frac{\sqrt{\rho\left(\rho E\left[\Delta x^{2}\right]_{t-3}+\left(1-\rho\right)\Delta x_{t-2}^{2}\right)+\left(1-\rho\right)\Delta x_{t-1}^{2}+\epsilon}}{\sqrt{\rho\left(\rho E\left[g^{2}\right]_{t-2}+\left(1-\rho\right)g_{t-1}^{2}\right)+\left(1-\rho\right)g_{t}^{2}+\epsilon}}g_{t}=\\
\\
-\frac{\sqrt{\rho^{2}E\left[\Delta x^{2}\right]_{t-3}+p^{1}\left(1-\rho\right)\Delta x_{t-2}^{2}+p^{0}\left(1-\rho\right)\Delta x_{t-1}^{2}+\epsilon}}{\sqrt{\rho^{2}E\left[g^{2}\right]_{t-2}+p^{1}\left(1-\rho\right)g_{t-1}^{2}+p^{0}\left(1-\rho\right)g_{t}^{2}+\epsilon}}g_{t}=\\
\\
-\frac{\sqrt{\rho^{t-1}E\left[\Delta x^{2}\right]_{0}+\overset{t-1}{\underset{r=1}{\sum}}\rho^{t-1-r}\left(1-\rho\right)\Delta x_{r}^{2}+\epsilon}}{\sqrt{\rho^{t-1}E\left[g^{2}\right]_{1}+\overset{t}{\underset{r=2}{\sum}}\rho^{t-r}\left(1-\rho\right)g_{r}^{2}+\epsilon}}g_{t}
\end{gathered}
$$

$\rho$ is a decay constant, so we choose it such that $\rho\in\left(0,1\right)$ (typically $\rho\ge0.9$).
Therefore, multiplying by a high power of $\rho$ results in a very small number.
Let $w$ be the lowest exponent such that we deem the product of multiplying sane values by $\rho^w$ negligible.
Now, we can approximate $\Delta x_{t}$ by dropping negligible terms:
$$\begin{gathered}\Delta x_{t}\approx-\frac{\sqrt{\overset{t-1}{\underset{r=t-w}{\sum}}\rho^{t-1-r}\left(1-\rho\right)\Delta x_{r}^{2}+\epsilon}}{\sqrt{\overset{t}{\underset{r=t+1-w}{\sum}}\rho^{t-r}\left(1-\rho\right)g_{r}^{2}+\epsilon}}g_{t}=\\
\\
-\frac{\sqrt{\overset{t-1}{\underset{r=t-w}{\sum}}\rho^{t-1-r}\left(1-\rho\right)\left(\begin{gathered}i_{r}^{2}\\
j_{r}^{2}\\
k_{r}^{2}
\end{gathered}
\right)+\epsilon}}{\sqrt{\overset{t}{\underset{r=t+1-w}{\sum}}\rho^{t-r}\left(1-\rho\right)\left(\begin{gathered}a_{r}^{2}\\
b_{r}^{2}\\
c_{r}^{2}
\end{gathered}
\right)+\epsilon}}\left(\begin{gathered}a_{t}\\
b_{t}\\
c_{t}
\end{gathered}
\right)\\
\downarrow\\
\Delta x_{t}\approx-\left(\begin{gathered}\frac{\sqrt{\overset{t-1}{\underset{r=t-w}{\sum}}\rho^{t-1-r}\left(1-\rho\right)i_{r}^{2}+\epsilon}}{\sqrt{\overset{t}{\underset{r=t+1-w}{\sum}}\rho^{t-r}\left(1-\rho\right)a_{r}^{2}+\epsilon}}a_{t}\\
\\
\frac{\sqrt{\overset{t-1}{\underset{r=t-w}{\sum}}\rho^{t-1-r}\left(1-\rho\right)j_{r}^{2}+\epsilon}}{\sqrt{\overset{t}{\underset{r=t+1-w}{\sum}}\rho^{t-r}\left(1-\rho\right)b_{r}^{2}+\epsilon}}b_{t}\\
\\
\frac{\sqrt{\overset{t-1}{\underset{r=t-w}{\sum}}\rho^{t-1-r}\left(1-\rho\right)k_{r}^{2}+\epsilon}}{\sqrt{\overset{t}{\underset{r=t+1-w}{\sum}}\rho^{t-r}\left(1-\rho\right)c_{r}^{2}+\epsilon}}c_{t}
\end{gathered}
\right)
\end{gathered}
$$


How the vector of weights assigned to a neural network?

It looks like when you say "way of assigning the weights", that you mean "what order are weights counted in, so that I know which weights connect between which neurons".

There is no formal "correct" way of doing this for all neural networks. However, in practice for feed-forward networks like your diagram, you would choose to use a matrix, not a vector, to represent the weights connecting layers. That is how pretty much all standard libraries will represent weights.

A matrix uses two indices (call them $i,j$ in this case) to identify a single scalar value. If we call the weight matrix $W$, then an individual weight is $W_{ij}$.

To determine how the weights connect between neurons, then you index the input layer neuron with $i$ and the output layer neuron with $j$.

In math notation, looking at your diagram, call each input value $x_i$ and each hidden value $h_j$, then the formula for calculating a single $h_j$ would be:

$h_j = f(b_j + \sum_{i=1}^{N} W_{ij}x_{i})$

Where $f()$ is a transfer function (such as sigmoid), $N$ is the number of input features, and $b_j$ is the bias for hidden layer neuron $j$.

You will also often see this written using matrix notation:

$\mathbf{h} = f(W\mathbf{x} + \mathbf{b})$

. . . this is not only clear and simple notation, but using matrix maths like this to describe a neural network is what allows us to use high performance libraries on GPUs such as TensorFlow.


How to get sentence from embedding vector with Universal Sentence Encoder?

You cannot run the universal sentence encoder in reverse. There is no practical way to take an arbitrary embedding vector and get a sentence.

My suggestion would instead be to find the sentence in your data with the embedding closest to your center. Euclidean distance works well, specially if you used K-means or another euclidean method to create your clusters.


How to force weights to be non-negative in Linear regression

What you are looking for, is the Non-negative least square regression.
It is a simple optimization problem in quadratic programming where your constraint is that all the coefficients(a.k.a weights) should be positive.

Having said that, there is no standard implementation of Non-negative least squares in Scikit-Learn. The pull request is still open.

But, looks like Scipy has implemented the same.

PS: I haven't tried the scipy version. I found it solely by googling around.


It totally depends on you and your problem. In fact, if the distribution of the data must be unchanged, oversampling is not a suitable solution.
If you can change the Loss function of the algorithm, It will be very helpful. There are many useful metrics which were introduced for evaluating the performance of classification methods for imbalanced data-sets. Some of them are Kappa, CEN, MCEN, MCC, and DP.

Disclaimer:

If you use python, PyCM module can help you to find out these metrics.

Here is a simple code to get the recommended parameters from this module:

>>> from pycm import *

>>> cm = ConfusionMatrix(matrix={"Class1": {"Class1": 1, "Class2":2}, "Class2": {"Class1": 0, "Class2": 5}})

>>> print(cm.recommended_list)
["Kappa", "SOA1(Landis & Koch)", "SOA2(Fleiss)", "SOA3(Altman)", "SOA4(Cicchetti)", "CEN", "MCEN", "MCC", "J", "Overall J", "Overall MCC", "Overall CEN", "Overall MCEN", "AUC", "AUCI", "G", "DP", "DPI", "GI"]

After that, each of these parameters you want to use as the loss function can be used as follows:

>>> y_pred = model.predict      #the prediction of the implemented model

>>> y_actu = data.target        #data labels

>>> cm = ConfusionMatrix(y_actu, y_pred)

>>> loss = cm.Kappa             #or any other parameter (Example: cm.SOA1)


How to present longitudinal data to LSTM for multiclass prediction

Use a LSTM, where the input is a sequence, specifically the sequence of records for that patient.  You can train the LSTM to predict anything you want -- e.g., the next step.  In LSTMs, the sequence normally is fixed-length (the same length for all people; you pad shorter sequences); each element of the sequence is a feature vector of same length (the same length for each element in the sequence.)  Each person in your dataset gives you at least one training instance for the LSTM.
It's beyond the scope of this question how to encode inputs, but you can read about standard methods; it's not specific to a LSTM.  For instance, you can encode categorical variables with a one-hot vector, and continuous variables directly.  Similarly for outputs, but you'll need to choose an appropriate loss function (e.g., cross-entropy loss for categorical variables, MSE for continuous values, etc.).  None of that is specific to LSTMs, so read up on how that is done with neural networks and the same methods will apply.
You'll probably need a lot of data to get the LSTM to work well.


is SST=SSE+SSR only in the context of linear regression?

if suppose regression wasn't linear, would SST=SSE+SSR still hold? If yes, why? If no, why?

Just to be clear that with linear regression it is perfectly OK to model nonlinear associations such as $y = 2x + 3x^2 + 17log(x)$ simply by including the relevant nonlinear terms, because it would still be linear in the parameters. I guess you are aware of this, but just wanted to make sure. In those cases, SST=SSE+SSR will hold.
Now, the crux of the matter is that SST=SSE+SSR is actually a special case that only holds when the model is linear in the parameters.  When we are dealing with a nonlinear model such as logistic regression, or any Generalised Linear Model, the situation is quite different because we model the linear predictor's association with the response variable via a link function so that a simple sum of squared deviations does not meaningfully reflect the variability in the response because the variance of an individual response depends on its mean.
There is a very good further explanation of this on Cross Validated, by Ben, and so I will just post it here, for completeness:

The sums-of-squares in linear regression are special cases of the more general deviance values in the generalised linear model.  In the more general model there is a response distribution with mean linked to a linear function of the explanatory variables (with an intercept term).  The three deviance statistics in a GLM are defined as:
$$\begin{matrix}
\text{Null Deviance} \quad \quad \text{ } \text{ } & & \text{ } D_{TOT} = 2(\hat{\ell}_{S} - \hat{\ell}_0), \\[6pt]
\text{Explained Deviance} & & D_{REG} = 2(\hat{\ell}_{p} - \hat{\ell}_0), \\[6pt]
\text{Residual Deviance}^\dagger \text{ } & & \text{ } D_{RES} = 2(\hat{\ell}_{S} - \hat{\ell}_{p}). \\[6pt]
\end{matrix}$$
In these expressions the value $\hat{\ell}_S$ is the maximised log-likelihood under a saturated model (one parameter per data point), $\hat{\ell}_0$ is the maximised log-likelihood under a null model (intercept only), and $\hat{\ell}_{p}$ is the maximised log-likelihood under the model (intercept term and $p$ coefficients).
These deviance statistics play a role analogous to scaled versions of the sums-of-squares in linear regression.  It is easy to see that they satisfy the decomposition $D_{TOT} = D_{REG} + D_{RES}$, which is analogous to the decomposition of the sums-of-squares in linear regression.  In fact, in the case where you have a normal response distribution with a linear link function you get a linear regression model, and the deviance statistics reduce to the following:
$$\begin{equation} \begin{aligned}
D_{TOT} = \frac{1}{\sigma^2} \sum_{i=1}^n (y_i - \bar{y})^2 = \frac{1}{\sigma^2} \cdot SS_{TOT}, \\[6pt]
D_{REG} = \frac{1}{\sigma^2} \sum_{i=1}^n (\hat{y}_i - \bar{y})^2 = \frac{1}{\sigma^2} \cdot SS_{REG}, \\[6pt]
D_{RES} = \frac{1}{\sigma^2} \sum_{i=1}^n (y_i - \hat{y}_i)^2 = \frac{1}{\sigma^2} \cdot SS_{RES}. \\[6pt]
\end{aligned} \end{equation}$$
Now, the coefficient of variation in a linear regression model is a goodness-of-fit statistic that measures the proportion of the total variation in the response that is attributable to the explanatory variables.  A natural extension in the case of a GLM is to form the statistic:
$$R_{GLM}^2 = 1-\frac{D_{RES}}{D_{TOT}} = \frac{D_{REG}}{D_{TOT}}.$$
It is easily seen that this statistic reduces to the coefficient of variation in the special case of linear regression, since the scaling values cancel out.  In the broader context of a GLM the statistic has a natural interpretation that is analogous to its interpretation in linear regression: it gives the proportion of the null deviance that is explained by the explanatory variables in the model.
Now that we have seen how the sums-of-squares in linear regression extend to the deviances in a GLM, we can see that the regular coefficient of variation is inappropriate in the non-linear model, since it is specific to the case of a linear model with a normally distributed error term.  Nevertheless, we can see that although the standard coefficient of variation is inappropriate, it is possible to form an appropriate analogy using the deviance values, with an analogous interpretation.

$^\dagger$ The residual deviance is sometimes just called the deviance.

Now as to the other part of the question:

If the answer to the above question is a no, then a follow-on question I would like to pose is: why is R2 the go-to measure for assessing the regression performance of a model or a technique?

My  answer to this is that $R^2$ is NOT the go-to measure for assessing regression performance. On the contrary, $R^2$ is very poor. I would recommend reading this thread on CV
Is $R^2$ Useful or Dangerous
This has currently 10 answers, many of which are good, but please pay particular attention to the answer by whuber.


KeyError: Selecting text from a dataframe based on values of another dataframe

You can consider this simple example, and take this forward to solve your problem. I have data set of quotes about animals and fruits. and I need to find out the top occurring word in each category. Count Vectorizer will be useful here
Consider data:

Code Snippet:
from sklearn.feature_extraction.text import CountVectorizer

def return_word_count_segment_wise(data, type):

    tfidf_vec = CountVectorizer(max_features=5)
    model = tfidf_vec.fit(data[data['Type'] == type].description)
    model_transform = tfidf_vec.transform(data[data['Type'] == type].description)

    feature_list = model.get_feature_names();
    count_list = model_transform.toarray().sum(axis=0)
    return dict(zip(feature_list,count_list))

return_word_count_segment_wise(data, 'Animal')

Outputs: {'cats': 3, 'is': 2, 'love': 4, 'my': 4, 'than': 3}
return_word_count_segment_wise(data, 'Fruits')

Outputs: {'fruit': 8, 'of': 5, 'that': 2, 'the': 3, 'we': 3}

Answering question asked in comment:
Try to merge both the dataframes, and then call the function while filtering out the customer segment using class (1/2/3)
merged_df = pd.merge(badges, comments, on = 'UserId')

return_word_count_segment_wise(merge_df, 1) # Get top 10 words for Gold class
return_word_count_segment_wise(merge_df, 2) # Get top 10 words for Silver class
return_word_count_segment_wise(merge_df, 3) # Get top 10 words for Bronze class

And just in case you can't merge, you can filter out the another dataframe using this dataframe
to_check = comments[comments['userId'].isin(Badges[Badges['class'] == 1].userId)]
return_word_count_segment_wise(to_check, 3)


XGBoost: Can the features in test data be a subset of the features used to train the model?

All the variables used to train the model must be present in the test set.

This is because you used all the variables to create the rules. Hence we would need those to score them.

If you are using python to do one hot encoding using fit or fit_transform functions in sklearn you will use the same object to transform the test set using transform function.

This will ensure the variables to be consistent in test and train.


Kmeans clustering with multiple columns containing strings

After some more research we found this library: https://github.com/nicodv/kmodes.

The library k-modes is used for clustering categorical variables. It defines clusters based on the number of matching categories between data points. (This is in contrast to the more well-known k-means algorithm, which clusters numerical data based on Euclidean distance.) The k-prototypes algorithm combines k-modes and k-means and is able to cluster mixed numerical / categorical data.

Because the dataframe contains categorical data we can't visualize it in a scatterplot. So I added the number representing the cluster the row was assigned to, for every row to get some form of visualization.

Normally you can only cluster ordinal data, because clustering happens based on distance. So I don't know to what extent this is reliable.


Genetic Optimization, Heuristic regarding choosing the number of generations and population size

Is there a heuristic formula or a study of some sort that can guide me regarding the best

Maybe, but I don't know any. My approach is to run a few experiments and observe the speed of convergence with different values for the size of the population, typically in the range 50 to 500. My experience is that usually this parameter doesn't have a big impact (especially compared to rate of mutation/crossover) so I tend to stay on the low side for efficiency reasons.

It's not technically required to specify the number of generations provided there is a criterion to check convergence, either manually or programmatically. I think it's more important to have such a criterion, because there's a always a risk that the predefined number of generations won't be sufficient.


Why does adding data augmentation decrease training accuracy a tiny bit?

The obvious reasons why data augmentation might reduce the train accuracy is -
As you know, deep learning models are data hungry. If the model don't get enough data to recognize the patterns then it will try to memorize the dataset. Bigger models tend to memorize the data instead of finding patterns, because they are big enough to do so. When model memorizes the training data it will definitely perform very good on training set and poorly on validation set.
And as you said data augmentation is a regularization technique. In regularization your model weights are penalized more to make sure they don't over fit. As a result, your model cannot perform well on training set (depending on how much regularzation is used), but as an advantage model will try to find generalized patterns in the dataset and this will also help at the time of validation.
I could find one research paper which has exhaustive experiments about data augmentation and regularization.


Before trying out deep learning I suggest you to try out Random Forests as the nature of your data seems non-linear to me and RFs are good at fitting non-linear data. If it doesn't work out well try doing a simple feed forward neural network with non-linear activation function as it also fits non linear functions really well. Also, machine learning is more about trying and testing as every dataset is different and you can never know in advance which model will work out well.


Grouping with non-sequential index (datetime) [Pandas] [Python]

Try this:
If needed, reset the index (this should not be a date time)
datos_frecuencia.groupby(datos_frecuencia.index // 3).agg({"tiempo":lambda x: x.head(3).min(),"state":sum})

Output:


Clarifying my understanding of on-policy RL (online SARSA)

I want to understand if I must use exactly the same Q function (and policy) to get A and A'. If I update Q function in each iteration, it follows that the next action in a subsequent iteration will be derived using the latest Q updated, while the previous action was obtained using the former Q.\ On the other hand, I really can make A and A' with exactly the same Q, and only after that update the Q. So I will always consider A and A' derived using the same function.
Which is more orthodox / correct?

The algorithm pseudocode given is more orthodox, since in order to revise the value of $A$ you would have to "roll back" the environment and see where the newly-sampled $A$ would take you from state $S$. To make this clearer, you can see that:
select action a' using a policy based on Q

could be re-phrased:
select action a' by sampling epsilon-greedy function over Q(s',*)

. . . you cannot do that unless you have the value of $S'$, and you may only have that value if you have already taken action $A$ when in state $S$. Changing $A$ at that stage therefore means going back in time . . .
In practice it doesn't matter much, even if you have capability to roll back (in a simulator, or in a planning algorithm). If your policy is based on e.g. $\epsilon$-greedy over the current Q values, then you are performing SARSA for optimal control (as opposed to prediction). In that case, changing Q means changing the policy. "On-policy" in SARSA for control must allow for the non-stationarity of the policy. Occasionally that means that the $A'$ value you just chose would have been chosen with a lower probability in a more optimal policy. But you chose it anyway this time, and the agent should choose it less often in future. The learning-rate based updates will remove estimation bias due to earlier poor/too-frequently-sampled choices over time.
Revising a single step "mistake" is possible, but not common practice in a purely online algorithm. I have not seen it in planning look-ahead or offline algorithms either that I have studied. I don't know for certain, but I suspect the occasional boost to learning you might get from revising the immediate part of the trajectory is too small to be worth the loss of generality of the algorithm. You may find it does help sometimes though, and worth an experiment to review whether it is helpful, provided you are working with a simulator/planner where rolling back state is relatively easy.

Note that the way you are thinking does turn up again when using function approximators (e.g. linear functions or neural networks) in semi-gradient versus "true gradient" methods, where instead of this being an issue with which Q values to use, it is an issue with calculating gradients due to the TD error, when your TD target is based on the same parameters that you are are taking the gradient for. In semi-gradient methods, this issue is ignored, and the methods still work OK. However, the "true gradient" methods are more theoretically correct.


what does this doc2vec based ML predict?

The code is trying to predict sentiment of documents (Amazon reviews or yelp reviews).

Kneighbors classifier works with vectors for input. A pre-trained doc2vec model is used to convert each review to a vector.
The classifier is trained to predict sentiment label from vector representing each review.

Accuracy is mean accuracy on the dataset: percentage of reviews which were correctly classified (as positive or negative).


Best practices for selecting categorical features

For removing categorical variable, generally used techniques is chi-square distribution based selection. The technique falls under general umbrella term univariate feature selection.


What is the relation between input into LSTM and number of cells?

Consider RNN networks as a simple MLP which for each time span t, takes the inputs of that time and the outputs of the previous step. Actually each time, you unroll the network. Consequently, the number of cells does not have any relation to the input size or the length of the time series. Take a look at here for a better understanding.


Are Hadoop and Python SciPy used for the same?

I think you're quite confused.

Hadoop is a collection of software that contains a a distributed file system called HDFS. Essentially HDFS is a way to store data cross a cluster.  You can access file stores as you would in a local file store (with some modification) and modify things via Java API.  Furthermore, ON TOP OF the file system there exist a MapReduce engine that allows for distributive workflow.

Python on the other hand is a generic programming language that can be made to do a myriad of task such as build a web applciation, to generating reports and even peforming analytics.

SciPy is a package that can be used in conjunction with Python (and often numpy) as a way to perform common scientific task.

Truthfully, they focus on different paradigms.  If you have LARGE DATA (ie terabytes worth of it), it might be worth wild to setup a hadoop cluster (ie multiple servers and racks) and use Java MapReduce, Hive, Pig or Spark (of which there is a python version) to do analytics.

If your data is small or you only have one computer, then it probably makes sense to just use python instead of adding the overhead of setting up hadoop.

Edit:  Made correction via comment.


How does a convolutional ply differ from an ordinary convolutional network?

It seems that a convolutional ply is exactly the same as an ordinary convolutional layer. From their paper, they argue that the term "CNN layer" usually refers to a convolutional layer followed by a pooling layer. In an attempt to reduce confusion, they name the convolutional part a "convolution ply" and a the pooling part a "pooling ply":

  In CNN terminology, a pair of convolution and pooling layers in Fig. 2 in succession is usually referred to as one CNN “layer.” A deep CNN thus consists of two or more of these pairs in succession. To avoid confusion, we will refer to convolution and pooling layers as convolution and pooling plies, respectively.

Ironically, this has increased the confusion, leading to this post. At the time I suppose it was not common to have several convolutional layers in a row before a pooling layer, but we see this often in modern architectures.

To answer your other question about the network structure; they state the structure of the network that they use in the Experiments section (Section V-B). To hopefully reduce confusion, I've replaced the word "ply" with "layer":

  In these experiments we used one convolution [layer], one pooling [layer] and two fully connected hidden layers on the top. The fully connected layers had 1000 units in each. The convolution and pooling parameters were: pooling size of 6, shift size of 2, filter size of 8, 150 feature maps for FWS, and 80 feature maps per frequency band for LWS.


What is the allowable limit of oversampling?

This depends on the nature of your data. If you can effectively simulate 435 samples using any given oversampling methods such as SMOTE or ADASYN for instance, then I would say oversampling would be better. Because it would provide data for various scenarios. But if exact replication is not possible and if replicated there might be an inherent problem with the model then you should choose to undersample.

A good example for undersampling would be that for example if you're conducting a scientific experiment and the data you have are limited to a few scenarios and the other data is yet to be fully observed, you would choose to undersample.

Oversampling would be the reverse case, where you can effectively simulate data for various classes based on some parameter and the generated data can mimic actual data and scenarios, then you should oversample.


Why we prefer VIF if we can find multicollinearity from correlation matrix as well?

The correlation matrix is not a reliable measurement for multicollinearity  because it only considers the pairwise effects. Unfortunately, multicollinearity is defined as:

  Phenomenon in which two or more predictor variables in a multiple regression model are highly correlated,

Do you see the point? You'll need to consider the correlation with all other variables in your data set, not just 1-to-1 pairwise comparison.

VIF addresses the issue.


How do I remove outliers from my data? Should I use RobustScaler? I am aware I can use DecisionTree but I want to use XGBoost

First of all, you don't need to remove outlier because "Decision family algorithm" like XGBoost can handle it.
Secondly, you can use Tukey method (Tukey JW., 1977):

def detect_outliers(df,n,features):
    outlier_indices = []
    # iterate over features(columns)
    for col in features:
        # 1st quartile (25%)
        Q1 = np.percentile(df[col], 25)
        # 3rd quartile (75%)
        Q3 = np.percentile(df[col],75)
        # Interquartile range (IQR)
        IQR = Q3 - Q1
        # outlier step
        outlier_step = 1.5 * IQR
        # Determine a list of indices of outliers for feature col
        outlier_list_col = df[(df[col] < Q1 - outlier_step) | (df[col] > Q3 + outlier_step )].index
        # append the found outlier indices for col to the list of outlier indices
        outlier_indices.extend(outlier_list_col)
        # select observations containing more than 2 outliers
        outlier_indices = Counter(outlier_indices)
        multiple_outliers = list( k for k, v in outlier_indices.items() if v > n )
        return multiple_outliers
Outliers_to_drop = detect_outliers(data,2,["col1","col2"])
data.loc[Outliers_to_drop] # Show the outliers rows
# Drop outliers
data= data.drop(Outliers_to_drop, axis = 0).reset_index(drop=True)

https://www.kaggle.com/yassineghouzam/titanic-top-4-with-ensemble-modeling

And Thirdly, I suggest you try discrete (binning) continuous variable instead of remove outlier for xgboost.


I think that resources about speech processing would be a good introduction to audio files manipulation for machine learning in general.

For a broader introduction to audio,
Machine Learning for Audio, Image and Video Analysis: Theory and Applications, Reviews
Book by Alessandro Vinciarelli and Francesco Camastra could be a good read.

Also, check out some blogs like this one: https://www.analyticsvidhya.com/blog/2018/01/10-audio-processing-projects-applications/ (as you mention deep learning in the tags) that can give a good introduction with some hands-on experience.


What is the number of neurons for the input layer of the BERT?

It is not very clear what you are referring to with "number of input neurons".
The input layer in BERT is an embedding layer, which is a table of vectors. Each of those vectors has dimensionality 768, and each vector is associated to one of the tokens in the vocabulary (so the number of vectors in the embedding table is the vocabulary size).


How to create a general plot for this problem

I think your data has three variables: customer_ID, number_of_products_sold, and feature_x. Time is not, as far as I can tell from your description, relevant.
To get all three relevant variables plotted in one figure, start with the data for a single customer and show number_of_products_sold versus feature_x as a scatter plot (removing the time aspect). This scatter plot with, say, circles for each data point, could be extended to multiple customer_IDs by overlaying scatter plots with different marker types.


How to check if the dataset contains sufficient information to predict a continuous variable

You are describing something that you can create a series of scatterplots for. Hold your dependent variable constant and then just run scatterplots against all possible features and see what comes out. You can visually determine what makes sense (or at least what makes sense as a starting point) and then go from there.

Also, don't forget that this is where you will also need to apply some of your own knowledge around transformations. Just because a scatterplot isn't obvious doesn't mean that it can't be used


R, keras: How to get output of a hidden layer?

You can get the answer here.
Here it is:
model <- ...  # create the original model
layer_name <- 'my_layer'
intermediate_layer_model <- keras_model(inputs = model$input,
                                    outputs = get_layer(model, layer_name)$output)
intermediate_output <- predict(intermediate_layer_model, data)


How to detect Anomalies in raw level data?

Should I create a heuristic that checks every column or is there a machine learning algorithm I can apply?

I would go with unsupervised or semi-supervised learning. Use 'normal' data as training data and then predict what is normal and what is abnormal. Depending on your needs you can fine-tune the cut-off.

One-class SVM should work. Or a combination of one-class SVM and SV Data-Description for possibly better results.


Statistical comparison of model performance when training and validation data is always the same

If your model is deterministic (no randomness), then repeating the training/testing on the exact same set of data is pointless - you will get the exact same answer every time. The benefit of cross-validation is that it provides an unbiased estimate of your model performance, and does so by using different perturbations of the train/test data. You can still do something similar, selecting 80% of your training data and testing on some subset of the test data, and repeatedly doing a resampling. There's a slight difference from traditional CV, where your train/test set are mutually exclusive and essentially define one another, whereas in this case, your training and test datasets can be defined totally independently (but that shouldn't be a problem).

Incidentally, what model are you using that you expect can accurately predict data that's completely unlike what it's been trained on? Usually the point of the training data is to provide the model with examples of the data you expect to see, along with the correct output. It's not clear to me why you'd want to train a model to classify tweets in English, but then test how it performs at classifying Dutch tweets - the model has never seen Dutch, so I don't expect it would perform well. It seems this evaluation would test how similar English and Dutch are, rather than testing how good your classification model is.


Which is the best method for Neural Network Layers in Keras

The sequential API is the simplest.  It allows you to declare a sequence of layers with an single input and a single output.  Naturally, this API is a good choice for sequential networks - those where data flows through each layer in sequence.
Pros: straightforward to read and write.
Cons: Difficult to accommodate multiple inputs/outputs.  Cannot define arbitrary acyclic networks.  No layer reuse.
The functional API allows you more freedom in how you connect inputs and outputs of different layers.  You can route the output of any layer to any other layer, which allows you to define arbitrary acyclic graphs.  Use this API when you need a network architecture which is not sequential.
Pros: More flexibility.  Can define arbitrary networks with multiple inputs/outputs, residual layers, etc.
Cons: More effort to read/write.  Higher potential for bugs (since you have to connect layers manually)
Subclassing is actually a different mechanism than the sequential or functional API.  The APIs above are used to build custom networks by linking together pre-existing layers.
By contrast, subclassing keras.layers.Layer is used to build your own custom layers.  This is useful if you are doing cutting-edge research or if you are trying to implement some technique from the literature that is not part of the standard Keras library.  After you have created a custom layer, you can use it with either the sequential or functional APIs to include your layer in a network.
Pros: Ultimate flexibility - you can make your layer do whatever you like.
Cons: Massive effort compared to using pre-built layers.


One way to look at this is through the idea of under-/overfitting

First off, here is a sketch of the generally observed relationship between bias and variance, in the context of model size/comlpexity:

Say you have a model which is learning quite well, but your test accuracy seems to be pretty low: 80%. The model is essentially not doing a great job of mapping input features to outputs. We have a high bias. But for a wide variety of input (assuming a good test set), we consistently obtain this 20% error; we have a low variance. We are underfitting

Now we decide to use a bigger model (e.g. a deep neural network), which is able to capture more details of the feature space and so maps inputs to outputs more accurately. We now have an improved test accuracy: 95%. At the same time, we notice that several runs of the model produce different results; sometime we have 4% error, and sometimes 6%. We have introduced a higher amount of variance. We are perhaps somewhere around the optimum model complexity shown on the graph above.

You say ok... let's create a monolithic neural network. It totally nails training and ends with a perfect accuracy: 100%. However, the test accuracy now drops to 90%! So we have zero bias, but a large variance.
We are overfitting. The model is almost as good as a look-up table for training data, but doesn't generalise at all when it sees new samples. Intuitively, that 10% error corresponds to a difference in distribution between the training and test sets used $\rightarrow$ the model knows the training distribution in extreme detail, some of which do not apply to the test set (i.e. the reality).

In summary: The bias tends to decrease faster than the variance increases, because you can likely still make a more competitive model for your dataset; the model is underfitting. It is like the low-hanging fruit that you can easily get - so an incremental improvement on the red curve above gives a big decrease in bias (increase in performance). Obviously that pattern cannot go on indefinitely, with each increment in model complexity, you get a lower increase in performance; i.e. you have diminishing returns. Furthermore, as you begin to overfit, the model becomes less able to generalise and so exhibits larger errors on unseen data; variance is creeping in.

For some more intuition between bias/variance in machine learning, I'd recommend this talk by Andrew Ng. There is also a text summary of the talk, for a quicker overview.

For a brief but more mathematical explaination, head over to this post of Cross-Validated. The second answer there is very recent and is perhaps better than the (old) accepted answer.


Can distribution values of a target variable be used as features in cross-validation?

There is nothing necessarily wrong with this.  If you have no better information, then using past performance (i.e., prior probabilities) can work pretty well, particularly when your classes are very unevenly distributed.

Example methods using class priors are Gaussian Maximum Likelihood classification and Naïve Bayes.

[UPDATE]

Since you've added additional details to the question...

Suppose you are doing 10-fold cross-validation (holding out 10% of the data for validating each of the 10 subsets). If you use the entire data set to establish the priors (including the 10% of validation data), then yes, it is "cheating" since each of the 10 subset models uses information from the corresponding validation set (i.e., it is not truly a blind test). However, if the priors are recomputed for each fold using only the 90% of data used for that fold, then it is a "fair" validation.

An example of the effect of this "cheating" is if you have a single, extreme outlier in your data. Normally, with k-fold cross-validation, there would be one fold where the outlier is in the validation data and not the training data. When applying the corresponding classifier to the outlier during validation, it would likely perform poorly. However, if the training data for that fold included global statistics (from the entire data set), then the outlier would influence the statistics (priors) for that fold, potentially resulting in artificially favorable performance.


Connection between Regularization and Gradient Descent

The fitting procedure is the one that actually finds the coefficients of the model. The regularization term is used to indirectly find the coefficients by penalizing big coefficients during the fitting procedure. A simple (albeit somewhat biased/naive) example might help illustrate this difference between regularization and gradient descent:

X, y <- read input data
for different values of lambda L
    for each fold of cross-validation using X,y,L
        theta <- minimize (RSS + regularization using L) via MLE/GD
        score <- calculate performance of model using theta on the validation set
    if average score across folds for L is better than the current best average score
        L_best <- L

As you can see, the fitting procedure (MLE or GD in our case) finds the best coefficients given the specific value of lambda.

As a side note, I would look at this answer here about tuning the regularization parameter, because it tends a little bit murky in terms of bias.


Cosine similarity of averaged random word vectors

Nothing too surprising here. As you sample more and more words, the sample mean is a better and better estimator of the population mean. This is called the law of large numbers.


extracting information in presence of noise

Usually removing noise is a process depends on the type of noise.
Noises can be high or low frequency , you are using low frequency filters ( average and median) .
1. You did not describe the source of noise, we can not tell if it is a high frequency or low frequency .
2. Try to use a higher order low pass filter


Should I resize my test images when feeding to a CNN model?

Firstly let's start with the last question. It is normally not possible to increase the size of an image, that is, increasing its resolution without using AI solutions. But one famous solution to overcome this problem is padding the image to increase its size. But depending on the filter size, it may poorly perform if the original image is too small compared to the original input size, and filter dimensions are big. Also, in any case, if a model designed to accept the high resolution, would not perform similarly with small images because it will not get the same size of input or information (padding will just contain dummy data).
However, you can build a model that can accept different sizes, although the one that you have built will not. Your model is configured to receive 32x32x3, that is, 3072 input nodes. Thus, you have to resize all of your high-resolution images to 32x32x3 and pad the smaller images.


What is/are the default filters used by Keras Convolution2d()?

By default, the filters $W$ are initialised randomly using the glorot_uniform method, which draws values from a uniform distribution with positive and negative bounds described as so:
$$W \sim \mathcal{U}\left(\frac{6}{n_{in} + n_{out}}, \frac{-6}{n_{in} + n_{out}}\right),$$

where $n_{in}$ is the number of units that feed into this unit, and $n_{out}$ is the number of units this result is fed to.

When you are using the network to make a prediction, these filters are applied at each layer of the network. That is, a discrete convolution is performed for each filter on each input image, and the results of these convolutions are fed to the next layer of convolutions (or fully connected layer, or whatever else you might have).

During training, the values in the filters are optimised with backpropogation with respect to a loss function. For classification tasks such as recognising digits, usually the cross entropy loss is used.
Here's a visualisation of some filters learned in the first layer (top) and the filters learned in the second layer (bottom) of a convolutional network:

As you can see, the first layer filters basically all act as simple edge detectors, while the second layer filters are more complex. As you go deeper into a network, the filters are able to detect more complex shapes. It gets a little tricky to visualise though, as these filters act on images that have been convolved many times already, and probably don't look much like the original natural image.


How the Support Vector Machine will perform if the bias b = 0 in the equation of hyperplane?

I think that it will simply behave as a normal SVM, with the difference that being the hyperplane anchored to the origin, you are blocking one degree of freedom and as such, the optimization algorithm used to find the parameters w might not find the best solution.
At this point, I am wondering why you should set b=0.


Which database to use for storing machine learning data?

I share here the results of my quick research:
I think I will create a mongodb database to store everything, because it is easy to use and accross the NoSQL databases it is the one my company knows best.
If you want to use a database, you may also consider things like parallel access, distributiveness of the database, resilience, etc.
I think more advanced solutions like shared filesystems, spark, hadoop, sciDB (in particular, sciDB seems specialized for analytics on arrays only) etc are good for data lakes and when we want to process ALL the data or a big part of it at the same time, per batch or through a pipeline for example. Therefore, I don't need it for now, and the simpler the best.


Why Is Overfitting Bad in Machine Learning?

Overfitting is empirically bad.  Suppose you have a data set which you split in two, test and training.  An overfitted model is one that performs much worse on the test dataset than on training dataset.  It is often observed that models like that also in general perform worse on additional (new) test datasets than models which are not overfitted.

One way to understand that intuitively is that a model may use some relevant parts of the data (signal) and some irrelevant parts (noise).  An overfitted model uses more of the noise, which increases its performance in the case of known noise (training data) and decreases its performance in the case of novel noise (test data).  The difference in performance between training and test data indicates how much noise the model picks up; and picking up noise directly translates into worse performance on test data (including future data).

Summary: overfitting is bad by definition, this has not much to do with either complexity or ability to generalize, but rather has to do with mistaking noise for signal.

P.S. On the "ability to generalize" part of the question, it is very possible to have a model which has inherently limited ability to generalize due to the structure of the model (for example linear SVM, ...) but is still prone to overfitting.  In a sense overfitting is just one way that generalization may fail.


Linear regression with a fixed intercept and everything is in log

Chi squared test does not serve any purpose. The nonlinear equations/functions can be handled by transforming them  in linear functions. The linear model can be used once we transform the nonlinear data/relations into linear format. chi squared test checks for variability. You seem to be interested in sum total of surface (area) i.e. linear model and not a linear regression.


I'd say there are pros and cons of using FeatureHasher for this purpose.
If you really striving to use it, then just instantiate it like this:
In [1]:
from sklearn.feature_extraction import FeatureHasher
h = FeatureHasher(n_features=5, input_type='string')
f = h.transform(mail_id)
f.toarray()

Out[1]:
array([[ 1.,  0.,  0.,  0.,  0.],
      [ 0., -1.,  0.,  0.,  0.],
      [ 1.,  0.,  0.,  0.,  0.],
      [ 0.,  0., -1.,  0.,  0.],
      [ 0., -1.,  0.,  0.,  0.]])

So, after you have instantiated it, just .transform each your upcoming mail_id and use results in upstream applications ( like online learning, for instance ).
Obviously n_features is some knob to tune. But this has its flip side: the cardinality of mail ids is apriori high, so unless you have very limited amount of users you will need enormous n_features to minimize collisions.
The better would be to take logs, where your ids coappear, and learn item2vec style model. This will deliver much denser (and meaningful) representation of mail_ids than FeatureHasher would do.
Also, take a look at this.


Error: float() argument must be a string or a number, not 'StandardScaler'

.fit method returns the standard scalar object. You are using that to train the model. please use fit_transfor or transform after the fit. like below

sc_x.fit(x)
x = sc_x.transform(x)

or

x = sc_x.fit_transform(x)


How to bin Colour values (categorical- high cardinality) to primary colours?

Since there are small, finite number of color options, it would be straightforward to manually create a dictionary where the key is the primary color and values are a set of other color values.
In Python, it would be something like:
colors = {"blue": {"blue", "teal"},
          "green": {"green", "olive"},
         }


What is the difference between ImageNet and ImageNet1k? How to download it?

The ImageNet dataset consists of more than 14M images, divided into approximately 22k different labels/classes. However the ImageNet challenge is conducted on just 1k high-level categories (probably because 22k is just too much).

ImageNet Stats

When people mention results on the ImageNet, they almost always mean the 1k labels (if some paper uses the original 22k labels, they would surly mention it). So basically ImageNet=ImageNet-1k.

Regarding downloading the dataset. Since you are downloading ImageNet for your personal usage (GAN training) and not to participate in one of the challenges, it doesn't really matter, so just download the latest dataset labeled "Download links to ILSVRC2017 image data".


Calculate the number of params of a neural network

There is not enough information to compute forward the number of parameters of each layer, as we don't know the number of input channels to conv2d_7 or its kernel width. Nevertheless, we can try to guess the missing pieces of information with reasonable assumptions from the number of parameters in each layer, which maybe helps you understand the number of parameter computations:

conv2d_7 has an output shape with 64 channels. This means that the filter tensor is $(c \times w \times w + 1) \times 64$ (the "+ 1" is for the bias), where $c$ is the number of input channels and $w$ the kernel size. The number of parameters in this layer is 640, which means that $w \times w \times c + 1= 10$. I would guess that $c=1$ and $w = 3$.
max_pooling2d_7, flatten_7 and dropout_2 don't have trainable parameters (= 0)
dense_14 has 1384576 parameters and, from the output shape, we know that the projection matrix is $? \times 128$. The previous layer has 10816 as output dimensionality. From this, we know that dense_14 has a projection matrix of dimensionality $10816 \times 128$ and a bias vector of 128 dimensions (totaling 10816 * 128 + 128 = 1384576)
dense_15 has 1290 parameters and, from the output shape, we know that the projection matrix is $? \times 10$. The previous layer has 128 as output dimensionality. From this, we know that dense_15 has a projection matrix of dimensionality $128 \times 10$ and a bias vector of 10 dimensions (totaling 128 * 10 + 10 = 1290).

Adding up the three computed number of parameters gives the total: $1384576 + 640 + 1290=1386506$.


Validation Curve Interpretations for Decision Tree

A few comments on the top of my head:

both parameters min_samples_leaf and max_depth are not very important for decision trees, so it's not surprising not to see much variation (or not all) across different values:

the fact that min_sample_leaf doesn't influence the performance simply means that the algorithm finds enough good predictors in the features to create leaves with a high number of instances, apparently always more than 30.
I'm less sure about max_depth, but I assume that it's a parameter which is used to prune the tree in order to avoid overfitting. This is exactly what happens here as soon as it's increased above 5: the algorithm creates a deeper (more complex) tree by using very specific conditions on the features which turn out to be specific to the training set, hence the divergence between training score and CV score (that's a clear sign of overfitting).

Value 5 for max_depth is indeed optimal, as seen on the second graph (due to overfitting with higher values). The value 19 returned by grid search for min_samples_leaf is as optimal as any other value between 1 and 30, as seen on the first graph: grid search just happened to pick this one but it doesn't have any impact anyway.
The 82% accuracy is completely normal: that's the Y axis on both graphs ;) It indeed looks like a decent performance, but we can't say for sure since there's no comparison: maybe the dataset is super easy and 82% is just the majority class, or maybe it's super hard and 82% is a great achievement.

Actually these graphs don't show much about the learning, they just show that the parameters studied are not really relevant. To observe something more interesting try:

Ablation study: pick a random subset of the training set instances and train only with this subset. Do this for say 10%, 20%, ..., 100% of the data, then plot the performance as a function of the size of the training set. This will show how many instances are needed for the model to reach its max performance (educated guess: not that many).
Feature selection: use only the N most informative features (as measured by information gain for instance) and plot the performance for different values of N. If you're lucky you might see an increase until a particular optimal value of N followed by a small decrease when uninformative features are added. If it happens, that last part would be due to overfitting.


Random Forests with complementary features

Notice that random forests (and decision trees in general) do not assume that the given features are independent. On the contrary, a typical classification path from the root to a leaf in one particular tree/classifier in the random forest would be e.g. to apply a different rule on the successes feature, based on the value of the attempts feature. So as one comment suggests, the algorithm will be able to identify certain dependencies.

However, you need to keep in mind that decision trees (and in certain sense random forests as a consequence) define only linear separations between classes. Thus, to enhance the domain space, you might want to try to "hint" to the algorithm some additional meta-features, and possible semantics among the features. For example, have you considered also introducing success ratio (successes divided by attempts) as an additional feature?

Notice that additional features are not guaranteed to help, even if the easiest way to know is to try them. The reason for this is that the algorithm might already be able to learn the additional semantics you give it. Having said that, to me it is not obvious that a random forest would be able to "learn" a feature like success ratio.


Two-class classification model with multi-type input data

With the help from a colleague, we came up with this:

set.seed(666)

#Dummy data
dat <- data.frame(samples=c(rep(paste0("A_",1:9800)),rep(paste0("B_",9801:10000))))
dat$label <- c(rep("A",9800),rep("B",200))
dat$continuous1 <- c(rnorm(9800, 100, 45),rnorm(200, 0, 25))
dat$continuous1[dat$continuous1<0] <- 0
dat$categorical1 <- c(rep(1:100,98),rep(1:100,2))
dat$categorical2 <- c(rep(1:98,100),rep(99:100,100))

pool_A <- factor(1:15,levels=1:20)
pool_B <- factor(16:20,levels=1:20)
dat_categorical3 <- vector("list",10000)
names(dat_categorical3) <- c(as.character(dat[dat$label=="A",]$samples),as.character(dat[dat$label=="B",]$samples))
for(i in 1:9800){
  dat_categorical3[[i]] <- (as.numeric(table(sample(pool_A, 20, replace=T))) > 0) + 0L
}
for(i in 9801:10000){
  dat_categorical3[[i]] <- (as.numeric(table(sample(pool_B, 20, replace=T))) > 0) + 0L
}
dat_categorical3_tensor <- do.call(rbind,dat_categorical3)

## Split data for training-validating-testing

# As we have much fewer Bs than As, each partition has to have a more or less equal number of Bs(?)
training_Bs <- sample(x=as.character(dat[dat$label=="B",]$samples), size=67, replace=F)
validating_Bs <- sample(x=setdiff(as.character(dat[dat$label=="B",]$samples),training_Bs), size=67, replace=F)
testing_Bs <- setdiff(as.character(dat[dat$label=="B",]$samples),c(training_Bs,validating_Bs))

# We also parition the data containing As equally, though this could probably be done in e.g., 80-10-10
training_As <- sample(x=as.character(dat[dat$label=="A",]$samples), size=3267, replace=F)
validating_As <- sample(x=setdiff(as.character(dat[dat$label=="A",]$samples),training_As), size=3267, replace=F)
testing_As <- setdiff(as.character(dat[dat$label=="A",]$samples),c(training_As,validating_As))

# Put together
training_dat <- dat[dat$samples %in% c(training_As,training_Bs),]
training_categorical3_tensor <- dat_categorical3_tensor[rownames(dat_categorical3_tensor) %in% c(training_As,training_Bs),]
training_labels <- ifelse(training_dat$label=="A",0,1)
training_dat2 <- as.matrix(training_dat[,c(3:5)]) #use this

validating_dat <- dat[dat$samples %in% c(validating_As,validating_Bs),]
validating_categorical3_tensor <- dat_categorical3_tensor[rownames(dat_categorical3_tensor) %in% c(validating_As,validating_Bs),]
validating_labels <- ifelse(validating_dat$label=="A",0,1)
validating_dat2 <- as.matrix(validating_dat[,c(3:5)]) #use this

testing_dat <- dat[dat$samples %in% c(testing_As,testing_Bs),]
testing_categorical3_tensor <- dat_categorical3_tensor[rownames(dat_categorical3_tensor) %in% c(testing_As,testing_Bs),]
testing_labels <- ifelse(testing_dat$label=="A",0,1)
testing_dat2 <- as.matrix(testing_dat[,c(3:5)]) #use this

## Keras model

# Input layers
all_dat_input <- layer_input(shape = 3, dtype = 'float32', name = 'all_dat_input')
categorical3_indices_input <- layer_input(shape = 20, dtype = 'float32', name = 'categorical3_input')
input_tensor <- c(all_dat_input, categorical3_indices_input)

# Output layers
all_dat_out <- all_dat_input %>%
  layer_dense(units = 128, activation = 'relu') %>%
  layer_dropout(rate = 0.5) %>%
  layer_dense(units = 128, activation = 'relu') %>%
  layer_dropout(rate = 0.5)

categorical3_indices_out <- categorical3_indices_input %>%
  layer_dense(units = 128, activation = 'relu') %>%
  layer_dropout(rate = 0.5) %>%
  layer_dense(units = 128, activation = 'relu') %>%
  layer_dropout(rate = 0.5)

output_tensor <- layer_concatenate(c(all_dat_out, categorical3_indices_out)) %>%
  layer_dense(units = 128, activation = 'relu') %>%
  layer_dropout(rate = 0.5) %>%
  layer_dense(units = 128, activation = 'relu') %>%
  layer_dropout(rate = 0.5) %>%
  layer_dense(units=1, activation="sigmoid")

model <- keras_model(inputs=input_tensor, outputs=output_tensor)

# Compile
model %>% compile(
  optimizer = "rmsprop",
  loss = "binary_crossentropy",
  metrics = "accuracy"
)

# Fit
history <- model %>% fit(
  x=list(training_dat2,training_categorical3_tensor),
  y=training_labels,
  batch_size=256,
  epochs=20,
  validation_data=list(list(validating_dat2,validating_categorical3_tensor),validating_labels)
)

# Compare with put aside testing data
results <- model %>% evaluate(list(testing_dat2,testing_categorical3_tensor),testing_labels)

results
$loss
[1] 0.0000001584833
$acc
[1] 1

# Using the trained network to generate predictions on new data
predictions <- model %>% predict(list(testing_dat2,testing_categorical3_tensor))

head(predictions[3267:3332])
[1] 0.9999921 0.9999992 0.9999999 0.9999996 0.9999999 0.9999542
# the network correctly identifies Bs as Bs with a confidence between >99%

(Remember that this is dummy data!)

I'm currently looking into feature importance to see which data inputs contributed more than others to the prediction. lime seems to be made for this!


How would you use a series of events as a factor for a NN?

Your input matrix should have dimensions (# of instances, length of history of events, 1). For example, I have text data that is sequential and I will cut it up into sequences which are sequentially dependent. Each row in the matrix is the sequence, the history of events. My label for each of these is the next sequential letter, or event in your case.

The input $X$ has the following shape

print(X.shape)

  (163717, 100, 1)

The LSTM model in Keras.

# define the LSTM model
model = Sequential()
model.add(LSTM(256, input_shape=(X.shape[1], X.shape[2])))
model.add(Dropout(0.2))
model.add(Dense(y.shape[1], activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam')
model.fit(X, y, epochs=20, batch_size=128, callbacks=callbacks_list)

$y$ is my targets. Can be any kind of classification.

If you have 1000 instances of these sequences of events which have 50 historical values then your matrix should have shape (1000, 50, 1)

If you want to add structured data you can always do this after the LSTM layers into the densely connected layer. You can have two models whos outputs are concatenated and then added together. This is a lot easier to do with the functional API of Keras as it is much more flexible and allows you to interact with the tensors more directly. For example

from keras.models import Sequential, Model
from keras.layers import Concatenate, Dense, LSTM, Input, concatenate
from keras.optimizers import Adagrad

seq_input = Input(shape=(X.shape[1], X.shape[2]))
seq_model = LSTM(256)(seq_input)
seq_model = Dropout(0.2)(seq_model)
seq_model = Dense(100, activation = 'softmax')(seq_model)

metadata_input = Input(shape=(8, ))
metamodel = Dense(10, )(metadata_input)

merged = concatenate([seq_model, metamodel])
merged = Dense(64, activation = 'softmax')(merged)
merged = Dense(y.shape[1], activation = 'softmax')(merged)

model = Model(inputs=[seq_input, metadata_input], outputs=merged)
model.compile(loss='categorical_crossentropy', optimizer='adam')

This example is as above. However, I add a metadata input of size 8 to the model after the LSTM layers. This will contribute in the output as it will embed the information from those features in the last few layers of the model.


Any non Deep Learning python packages for sequence classification.?

Have you thought of using Conditional Random Fields? I had used it for my thesis some years ago and papers from Prof. Sunita Sarawagi were a great resource for me. You can use the CRFsuite library which also offers a python wrapper.


'PCA' object has no attribute 'explained_variance_'

PCA is an estimator and by that you need to call the fit() method in order to calculate the principal components and all the statistics related to them, such as the variances of the projections en hence the explained_variance_ratio.
pca.fit(preprocessed_essay_tfidf) or pca.fit_transform(preprocessed_essay_tfidf)


Generating rules from data for classification

If the data is unlabeled you either

manually label the data (if you have not too many samples)
manually label part of the data, train a classification model on it (and validate your model!) and use the model to label the unlabeled part, to speed up the labeling process
use pretrained classification/segmentation models (if your data is images)
use unsupervised learning algorithms to find patterns in the data, then manually inspect the groups obtained to find recognizable patterns and label the groups accordingly. The algorithm you should use depends on the type of features you have. With numerical features you can use distance-based clustering techniques like k-means, or dimensionality reduction techniques like Self-Organizing Maps or t-SNE

Basically, it depends on what kind of data you have and how many samples. I believe that more information on your data is required for providing more specific answers.


What are the consequences of not freezing layers in transfer learning?

I think that the main consequences are the following:

Computation time: If you freeze all the layers but the last 5 ones, you only need to backpropagate the gradient and update the weights of the last 5 layers. In contrast to backpropagating and updating the weights all the layers of the network, this means a huge decrease in computation time. For this reason, if you unfreeze all the network, this will allow you to see the data fewer epochs than if you were to update only the last layers weights'.
Accuracy: Of course, by not updating the weights of most of the network your are only optimizing in a subset of the feature space. If your dataset is similar to any subset of the imagenet dataset, this should not matter a lot, but, if it is very different from imagenet, then freezing will mean a decrease in accuracy. If you have enough computation time, unfreezing everything will allow you to optimize in the whole feature space, allowing you to find better optima.

To wrap up, I think that the main point is to check if your images are comparable to the ones in imagenet. In this case, I would not unfreeze many layers. Otherwise, unfreeze everything but get ready to wait for a long training time.


how to create word2vec for phrases and then calculate cosine similarity

You cannot apply word2vec on multiple words. You should use something like doc2vec, which gives a vector for each phrase:

phrase = model.infer_vector(['microsoft', 'visual', 'studio'])

You can also average or sum the vectors of words (from word2vec) in each phrase, e.g.

phrase = w2v('microsoft') + w2v('visual') + w2v('studio')

This way, a phrase vector would be the same length as a word vector for comparison. But still, methods like doc2vec are better than a simple average or sum. Finally, you could proceed to compare each word in the first list to every phrase in the second list, and find the closest phrase.

Note that a phrase like "cloud computing" has a completely different meaning than the word "cloud". Therefore, these phrases, specially if frequent, better to be treated as a single word, e.g.

phrase = w2v('cloud_computing')

Extra directions:

Here is an answer by Astariul on stackoverflow that uses a function from word2vec package to calculate similarity between two sets of words.
Take a look at fastText that works better when there is a lot of misspelled, or out-of-vocabulary words.


Best tool for text pre-processing, involving tokenization, lemmatization, stop-word removal, feature vector extraction?

Comparing two libraries or tools in terms of these things is somewhat that is opinion dependent. Some people prefer NLTK for doing almost all the tasks. spacy has also gained quite reputation. But what is better for you depends on what you want do. In my personal experience, I have found that NLTK along with gensim libraries is all that I need to do all the Natural Language Processing tasks.


Choosing an optimizer to perfectly fit a neural networks to training data

There's no free lunch, only bunch of tips. Your search space always depends on the problem you're tackling, and there's no one that fits them all. You need to run experiments to check this. Additionally, your net's architecture is one of the parameters, so it's hard to tell if your net will even be of sufficient capacity, without knowing the exact functions, you want to approximate. At first glance, optimizer is last thing you should worry about.


CTry to change

model.add(LSTM(50, activation='relu'))

to

model.add(LSTM(50, activation='relu', return_sequences=True))

as explained in the documentation ( https://keras.io/layers/recurrent/#lstm ) return_sequences allow the LSTM to return the complete sequence of vectors of the LSTM rather than a single vector.


Try:
# Create an example dataframe
df = pd.DataFrame({"genres":["Fantasy|Sci-Fi",
                "Action|Adventure|Fantasy",
                "Thriller",
                "Action|Adventure|Thriller|bbv","Action","Action|Adventure|thriller"]})

# Get a dataframe with as many columns as there are genres
df = df.genres.str.get_dummies(sep = "|")

# Get the genres as values
df = df.multiply(df.columns)

# Rename the columns to have the genre id
df.columns = ["genre_" + str(x) for x in range(len(df.columns))]

Input:

Output:

EDIT:
you can simply use pandas assign method:
df.assign(genre1= df.genres.str.split("|", expand = True).iloc[:,:1],
          genre2 = df.genres.str.split("|", expand = True).iloc[:,1:2])

*Output:


Difference between using BERT as a 'feature extractor' and fine tuning BERT with its layers fixed

No, approaches 1 and 2 are not the same:

In approach 1 (feature extraction), you not only take BERT's output, but normally take the internal representation of all or some of BERT's layers.

In approach 2, you train not only the classification layers but all BERT's layers also. Normally, you choose a very low learning rate and a triangular learning rate schedule to avoid catastrophic forgetting.

There are many scientific articles studying how to best use BERT in transfer learning scenarios. This one may be a good starting point: https://www.aclweb.org/anthology/W19-4302/


Reversing the input and output of an ML algorithm to Optimize

The problem is a bit vague but here a couple ideas:

Just use the Y variables as features and predict the X variables. The most basic option is to consider every Xi as independent of the others, and train an independent model for each of them. A more advanced approach is to train a joint model which predicts all the Xi together (i.e. the class would be for instance "1,1,0"), but this probably requires a lot more data.
In the idea of optimization techniques, there might be some way to use genetic learning in order to obtain the optimal X values given the Y values. However I don't see how to design the population with many different Y combinations, but maybe it's possible.


What are the good parameter ranges for BERT hyperparameters while finetuning it on a very small dataset?

How many classes do you have? Bert can handle a high-quality 12k dataset for binary classification. I recommend duplicating your positive test case 4x and sampling a 5k test cases from your negative class. This will give you a balanced dataset. Then implement BERT in google colab using the original GitHub repository from the google Bert team. You can find a functional collab here: https://colab.research.google.com/github/tensorflow/tpu/blob/master/tools/colab/bert_finetuning_with_cloud_tpus.ipynb
On collab, TPUs takes about 10 to 15 minutes to run. All you have to do is use write your data processor.
The following parameters are a good place to start.

TRAIN_BATCH_SIZE = 32
EVAL_BATCH_SIZE = 8
PREDICT_BATCH_SIZE = 8
LEARNING_RATE = 2e-5
NUM_TRAIN_EPOCHS = 3.0
MAX_SEQ_LENGTH = 128
# Warmup is a period of time where hte learning rate
# is small and gradually increases--usually helps training.
WARMUP_PROPORTION = 0.1
# Model configs
SAVE_CHECKPOINTS_STEPS = 1000
SAVE_SUMMARY_STEPS = 500


Time series binary classification probability smoothing

You can use a total variation regularizer (https://en.wikipedia.org/wiki/Total_variation_denoising), it's a penalty for abrupt changes of neighbor values. It's usually used for images, that's why its TF version (https://www.tensorflow.org/api_docs/python/tf/image/total_variation) operates with 4D tensors, but if you're writing your model in pytorch for instance, it's easy to implement that regularizer yourself. Also possibly you don't need it if you've got enough data and target values there are already smooth. Your ML algorithm would just learn that smoothness from data, the only 2 cases you'd need it is when your dataset is small or when your training targets aren't smooth, but testing targets should be smooth.


Given the context of your question I assume your are referring to the k-NN classifier. The idea of the classification method is you have a set of feature vectors $f_i$, in general, $f_i \in \mathbb{R}^d$  where $d$ is the dimension of the vectors. Additionally you have a class $c_i$ for each $f_i$. To classify an unseen feature vector $f_j$ you select the $k$ nearest neighbors (under some distance, usually the euclidean distance) and the most common class among the neighbors is your prediction.

So, summarizing, the input of the algorithm are the feature vectors $f_i$ and the classes $c_i$. For more on the algorithm see this wikipedia link.

The image (from wikipedia) below illustrates a typical example, in this example the classes are: red triangle,blue square and the point to predict is the green circle. Each feature vector is a two dimensional point, i.e. $f_i, f_j \in \mathbb{R}^2$. In this situation the predicted class will be red triangle.

For an explanation using both Python and R see this.


Most machine learning around ICD codes deals with auto-encoding documents or NLP to extract ICD codes automatically from documents. Once you have them, most applications I have seen use them as is.

One simple alternative would be to assign the codes to relevant categories to reduce the number of levels and make them more interpretable. For example

injury types (e.g. F00–F99 for mental disorders)
body parts (e.g. S10-S19 & .. for neck injuries)
severity of injury etc.

depending on your application. Of course, if you don't want to define the categories beforehand, this can be done using some clustering or embedding method etc.


Algorithm for campaign optimization (Digital Advertising)

sounds like a great place for a weighted average and a little algebra.

budget_n = (revenue_n / total_revenue) * total_budget

This is a simple solution you may have already tried, but in principle, the revenue fraction can be as complex as you need it to be as all the fractions across all campaigns add up to 1.


A Proper Outlier Detection For the Attached Figure

One approach would be to fit a linear regression and look at Cooks Distance to detect outliers.
https://en.m.wikipedia.org/wiki/Cook%27s_distance

Some R background: http://r-statistics.co/Outlier-Treatment-With-R.html

Some Python discussion: https://stackoverflow.com/a/52322232/9524424


How to convert 200 bag of word features to several continuous features

Latent dirichlet allocation (LDA) would be a good one to try, as it is tailored for bag-of-words representations. LDA is a topic model, which basically means that it assigns $n$ topics, and figures out the words that associate with each topic. The output is a vector of $n$ continuous values where each value indicates membership to the corresponding topic. You should be able to find an implementation of LDA for whatever language you're using easily (but don't confuse it with linear discriminant analysis, also commonly referred to as LDA).

You could also try principal components analysis or singular value decompositions, which are other standard dimensionality reduction techniques.


Data wrangling for a big set of docx files advice!

The big problem with docx files is that they have a ton of content related to formatting that most people find irreverent when scraping docx files.  Hence, one approach is to convert the docx files to something more friendly like json that excludes all the irrelevant bits, and then use existing frameworks like JSON schema for pattern matching and jmespath for data extraction.  There is a python package called simplify-docx for converting Docx files to JSON and a toy website where you can try it out.  (Disclaimer: I'm the author of both)


Regression with LSTM network: use multiple time series as input

I think what you're looking for is to change your dataset into a supervised learning problem. Keras has its own TimeSeriesGenerator but that wont be of use as it only has a single step output. Someone created an alternate version of the TimeSeriesGenerator. Using this, you can specify the inputs and outputs per sample to change your data into a supervised learning problem which will then be able to fit seamlessly in your LSTM network.
Hope that helps.


Transfer learning within Tensorflow's inception model

To freeze the lower layers during training, the simplest solution is to give the optimizer the list of variables to train, excluding the variables from the lower layers:

train_vars = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES,
                               scope="hidden[34]|outputs")
training_op = optimizer.minimize(loss, var_list=train_vars)

The first line gets the list of all trainable variables in hidden layers 3 and 4 and in the output layer. This leaves out the variables in the hidden layers 1 and 2.


How to divide 7 non-continuous result values into 5 legend categories?

You do not have to use shades of the same colour.  You can keep 7 colours if they have sufficient contrast between them.

The "Gradient" and "Indexed" colour schemes of ColorData may be be used to select the 7 colours.

Some examples include.

In Wolfram Language

BarLegend[{#, {0, 7}}, Range@7,
    LegendLabel -> #,
    LegendLayout -> "Row"] & /@
  {"BrightBands", "DarkBands", "Rainbow", "LightTemperatureMap", "TemperatureMap", "ThermometerColors"} //
 Multicolumn[#, 3] &

BarLegend[{#, {0, 7}}, Range@7,
    LegendLabel -> Last@ColorData[#, "AlternateNames"] <> " (" <> ToString@# <> ")",
    LegendLayout -> "Row"] & /@
  {96, 109, 110, 111} //
 Multicolumn[#, 2] &

The ColorFunctionScaling and ColorFunction options will need to be set to use the scheme.

BarChart[Range@7,
 ColorFunctionScaling -> False,
 ColorFunction -> ColorData[{"TemperatureMap", {0, 7}}]
 ]

For a large number of discrete values a "Gradient" colour scheme can be used without restricting the number of contours.

For example

Plot[x, {x, 0, 41},
    ColorFunctionScaling -> False,
    ColorFunction -> ColorData[{#, {0, 41}}],
    PlotLabel -> #,
    Filling -> Axis,
    PlotLegends -> Automatic,
    ImageSize -> Medium
    ] & /@ {"TemperatureMap", "ThermometerColors"} //
 Column

Hope this helps.


Using class weights with validation data in Keras

Class weights make sense only in the context of a loss function. When you validate your model you are making predictions and comparing to ground truth using a metric - but in that phase you aren't propagating back any changes, so weights are useless.


After several attemps I have the following piece of code. I created two new networks and transferred the weights from the initial network to them.

# first part of initial model
part1_model = tf.keras.models.Sequential([
  tf.keras.layers.Conv2D(32, (3,3), activation='relu', input_shape=(28, 28, 1)),
  tf.keras.layers.MaxPooling2D(4, 4),
  tf.keras.layers.Conv2D(32, (3,3), activation='relu'),
  tf.keras.layers.MaxPooling2D(2,2),
  tf.keras.layers.Flatten()
])
part1_model.layers[0].set_weights(model.layers[0].get_weights())
part1_model.layers[1].set_weights(model.layers[1].get_weights())
part1_model.layers[2].set_weights(model.layers[2].get_weights())
part1_model.layers[3].set_weights(model.layers[3].get_weights())
part1_model.layers[4].set_weights(model.layers[4].get_weights())
part1_model.summary()

# second part of initial model
part2_model = tf.keras.models.Sequential([
    tf.keras.layers.Dense(128, activation='relu',input_shape=(1, 128)),
  tf.keras.layers.Dense(10, activation='softmax')
])
part2_model.summary()
part2_model.layers[0].set_weights(model.layers[5].get_weights())
part2_model.layers[1].set_weights(model.layers[6].get_weights())
# predictions holds prediction for test set from initial model
predictions = model.predict(test_images)

# predictions1 holds output of first part when test set is used as input
predictions1 = part1_model.predict(test_images)

import numpy as np
# tmp is used to transform prediction1 in a format recognizable from part2
tmp=np.zeros((10000,1,128))
for i in range(0,10000):
    tmp[i,:,:]=predictions1[i,:]

# predictions2 holds the output of second part when the result of first part (predictions1) is used as input
predictions2 = part2_model.predict(tmp)

# check that the result of initial model is the same with the result of the two parts
ok=0
for i in range(0,10000):
    if np.argmax(predictions[i])!=np.argmax(predictions2[i]):
        print(i," False")
    else:
        ok=ok+1
print(ok)
# also sample some cases and check probabilities
print("last entry")
print(predictions2[9999])
print(predictions[9999])
print("2nd entry")
print(predictions2[1])
print(predictions[1])


Is Keras a fancy tool or is it actually helpful in research or both?

In general any tool that many researchers use is by definition useful/helpful.

For the particular case of Keras and other neural network frameworks (like PyTorch, TensorFlow, etc), a lot of people use them. You can see this by reading papers in the topic, as it is usually mentioned which framework the implementation is made in. You can also check "popularity" by counting references, for example for keras and tensorflow, both have thousands of citations.

In general you should prefer well known tools that are used in the field. These are validated and you can at least have a degree of trust that the results they produce are correct (minus user mistakes, of course). Implementing a neural network framework from scratch is not easy and any reviewer can point out that there could be programming errors and/or mistakes. Also as you mention there is the advantage of reproducibility, as it is much more likely that people can run your code.

Also note that many frameworks were actually made by researchers, so they are specifically targeted for other researchers, and their design is made for easy experimentation. In keras for example, it is easy to implement some custom functionality as a loss or layer, so you can integrate it with another network design. A framework can have another target userbase such as deployment for low power platforms or computers, which Keras doesn't.

Finally, you have to consider that since a lot of people use Keras, there is a large community around it from where you can get support, such as Stack Overflow or the keras-users google group.  Using custom code will make this more difficult or impossible.

If you want not to depend on a specific framework, you can always export networks in ONNX format so you can use it with another framework or your own code.


Time series prediction using ARIMA vs LSTM

Statement 1 is correct, statement 2 is correct, but requires elaboration, and statement 3 is incorrect for seasonal ARIMA:

The following might point you in the right direction but hopefully you'll get a few more answers with more depth in the arena of LSTM.

You mention that you have tried both algorithms and that you are simply trying to figure out which one is better, which leads me to think you may be having more trouble with the data science process and cross validation than with the specifics of the models.

Time series in general:

Time series, in general, are difficult to forecast. If they were easy to forecast then all data scientists would be wealthy, having accurately forecast the value of all of the stocks. The reality is that hedge funds, on average, do not outperform the market and that time series forecasting is typically very poor and applies only to very short durations.  The main problems are that there is a lot of noise, there are many hidden influences, models are overly simplistic, influencers do not behave as we think they should, the interplay between linearity and nonlinearity is subtle and confusing, ... ad infinitum.

ARIMA

You are incorrect in your assessment that ARIMA requires stationary time series to forecast on.  Non-seasonal ARIMA has three input values to help control for smoothing, stationarity, and forecasting ARIMA(p,d,q), where:

p is the number of autoregressive terms,
d is the number of nonseasonal differences needed for stationarity, and
q is the number of lagged forecast errors in the prediction equation.

By contrast seasonal ARIMA has six input values ARIMA(p,d,q,P,D,Q), where:

P is the number of seasonal autoregressive terms,
D is the number of seasonal differences, and
Q is the number of seasonal moving-average
terms.

Subject to the qualifying statements above, I suggest playing with seasonal ARIMA to get a feel for the intricacies involved in smoothing, de-seasoning, de-trending, de-noiseing, and forecasting.

LSTM

I don't know enough about LSTM to add much here. I will add that red flags tend to be raised when someone begins at data science exercise with deep learning. I suggest learning as much as you can using ARIMA and then applying some of your ARIMA expertise to help you learn LSTM. Neural networks can be a very powerful tool, but they:

can take a long time to run,
often require more data to train than other models, and
have lots of input parameters to tune.

Cross validation and comparing models:

Time series are fun in that all training data can usually be turned into supervised learning training sets.  Once can simply take a time series and roll back time. That is... pick a point in time and pretend that you don't have any additional data, then produce a forecast and see how well you did.  You can march through the time series doing this $n$ times in order to get an assessment of the performance of your model and to compare models while taking the necessary precautions to prevent overfitting.

Hope this helps and good luck!


Integrate remote sensing & GIS data in a CNN

Keras functional API can be used to create models with multiple inputs (E.g. : Image, Categorical and Numerical).

From Geotiff, you can extract :

TIFF
CSV with other features

TIFF will be an input to stack of CNN layers and other features will feed dense layers. Output from both of these is combined in a dense layer.

Example : https://www.pyimagesearch.com/2019/02/04/keras-multiple-inputs-and-mixed-data/


There's a lot of ways to do this, one approach is to use token-based matching. You can use this to easily find any "tokens" in the text, like names, places, or just plain words.

Methodology

I'd recommend using Rule-based Entity Recognition in spaCy. You'll define the "rules" of what the entity looks like, here's the example from the docs where we define the following patterns to find:

An entity type of Organization and the word Apple
An entity type of Location and the words san and francisco

Here's that in code (live example):

from spacy.lang.en import English
from spacy.pipeline import EntityRuler

nlp = English()
ruler = EntityRuler(nlp)

# These are the rules you define, look at the docs to see what your options are.
# You don't have to use the "label", you can just look for a "pattern" if you want.
patterns = [{"label": "ORG", "pattern": "Apple"},
            {"label": "GPE", "pattern": [{"LOWER": "san"}, {"LOWER": "francisco"}]}]

ruler.add_patterns(patterns)
nlp.add_pipe(ruler)

# The text you're searching through to find your patterns
doc = nlp("Apple is opening its first big office in San Francisco.")

# This prints out the matches
print([(ent.text, ent.label_) for ent in doc.ents])

The output of this code is: [('Apple', 'ORG'), ('San Francisco', 'GPE')]

Usage

Thankfully spaCy has some fantastic online tools for helping you write your patterns, I highly recommend you check these links out.

Install spaCy
Evaluate if you should use rules or a model (I suggest rules but I could be wrong), and if you should use token matcher or phrase matcher
Read the documentation to determine how to write your patterns
Test your patterns using the Live Rule-based Matcher Explorer
Use the code snippet above or the code samples in the docs to see how to use your new patterns


Isn't (steps_per_epoch = total training data/batch size)?

As clearly mentioned in the documentation:
Steps_per_epoch is total number of steps (batches of samples) before declaring one epoch finished and starting the next epoch. The default None is equal to the number of samples in your dataset divided by the batch size, or 1 if that cannot be determined. It is a optional parameter and is useful when passing an infinitely repeating dataset.


Very good question!
An approach towards your goal is Center loss. The loss rewards if points of a class gather around a center. You can adapt this approach for your Autoencoder.


Comparing youtube view counts between historical release dates

A simple option would be to use the average number of views per day, this way the value is normalized and independent of how recent the video is. Of course averaging is simplistic, since the views don't follow a linear trend.
A more advanced option would be to model the evolution of views across time (for instance assuming a normal distribution), then estimate the parameters for every particular video and compare the predicted total number of views. This would be more complex due to the difficulty of estimating the parameters from an incomplete distribution.


Adaptive learning of user's IoT setting preferences

This sounds like a fun problem but it very open ended!  I will provide some links from the Scikit-Learn User Guide.  There are many more reading options, but the SKL Userguide has lots of examples and usually links to academic publications for more in-depth reading.  Another great resource is: Introduction to Statistical Learning, and if you are good at math: Elements of Statistical Learning

Model Selection

The first big question is whether the settings you are trying to predict are ordinal values or continuous values?  Continuous value will allow you to use regression methods, where as ordinal values will give you a choice between regression methods and classification schemes.  I would suggest employing a couple of different models to start and then selecting the best one moving forward.  Some possible candidates are:

Linear regression (with nonlinear features as appropriate) because linear models are usually insightful,
Support Vector Classification/Regression (SVC/SVR) as these are often very accurate classifiers. and either
Naive Bayes or Random Forests as these often give good results where other models fail.

Initial Training Data

You will need to prime your solution with some deterministic data or data from another dataset that you have adapted to fit your model.  I suggest priming the system with common sense values and then employing some sort of algorithm to forget those values as the system becomes well-trained.

Moving Forward

You will have to decide whether each user's model will use other user's data or not.  This is sometimes solved via clustering methods like k-means or DBSCAN), where by users are clustered into groups each cluster has a different model associated with it.  I would suggest retraining (fitting) the model on regular time frames initially, and then investigating the possibility of moving to an online learning system.

I absolutely suggest using a well known library for your problem. Scikit-Learn is great for the reasons mentioned above and for prototyping, but doesn't scale particularly well. H20 and Mahout are high quality scalable libraries that I would recommend for a big data production system.  Mahout in Action is a great book to learn from, also.

Hope this helps!


Probability vs recall for time series classification task

[edited]

The Area Under the Curve (AUC) is a good fit for this kind of problem, but it's not exactly a standard case. The ROC curve is built by ranking your instances by their predicted probability of fault, for example:

p(faulty)   gold
0.03        0
0.14        1
0.19        0
0.23        0
0.31        1
0.32        0
0.65        1
0.78        1
0.83        0
0.90        1

Each level of probability corresponds to a possible threshold for the binary classification problem. The number of True/False Positive/Negative can be calculated for every possible level of threshold:

p(faulty)   gold TP TN FP FN
                 5  0  5  0
0.03        0    5  1  4  0
0.14        1    4  1  4  1
0.19        0    4  2  3  1
0.23        0    4  3  2  1
0.31        1    3  3  2  2
0.32        0    3  4  1  2
0.65        1    2  4  1  3
0.78        1    1  4  1  4
0.83        0    1  5  0  4
0.90        1    0  5  0  5

At this stage one obtains exactly the data required for a standard ROC curve: from these values precision and recall scores can be computed for every level and plotted on the ROC curve. The AUC is used in the usual way.

This method takes into account the continuity of the probabilities while matching predictions against binary classes. An example of problem where this evaluation method is used: https://pan.webis.de/clef15/pan15-web/author-identification.html


How to draw a hyperplane using the weights calculated

I think a better to way to think about a neural net as you have it written is that it is a function.

$$f_{nn} : \mathbb{R}^n \rightarrow \{0, 1\}$$

which is generally a non-linear decision boundary. You did not indicate your network structure and activation function. If your data is linearly separable then a single layer perceptron will define weights for you in the input layer, plus a bias input node for the intercept.

If you specifically want a separating hyperplane you could consider support vector machines. See here.

https://stackoverflow.com/a/8021219/431895


How to handle a feature vector that could be variable length?

There are many options.
The most common method to handle variable length vectors is to pad the vectors. Add zeros as placeholders to the shorter length vectors until all the vectors are the same length. Then it can be modeled similar to any other feature vector.
Another option is to take the norm of the vector. That would yield a single scalar that could be used in the machine learning model.
Ultimately, the way to handle a feature vector that could be variable lengths depends on what the feature represents.


You might be able to fit your training into a Google Colab session. You can search for tutorials using different libraries, here is one for PyTorch. Google Colab is essentially a free online Jupyter notebook, and makes K80 GPU available for acceleration.

Sessions on Colab are limited to ~12 hours maximum, so you would need to start with some simple variants and build up to max allowed training time (this is good practice anyway). There is no guarantee that this will be enough for your game, but you should learn a lot even if the resulting agent is not perfect.

It is not clear whether you have already figured this out, but you most likely will not be able to host the Android App natively to run as the environment, and will need to code some kind of simulation or re-implement the game. I would also assume it is a card, board or turn-based strategy game with perfect information, if you are hoping to use an AlphaZero-based learning agent with it (AlphaZero could probably be adapted to video games, but I would not expect it be as efficient as simpler algorithms without a planning phase, such as A3C in that case).


I would recommend to have a look at intent classification. Once intent of the conversation is known, it could be routed to specific departments.

Basically in chat bot development, we try to do intent classification and entity extraction.

Intent classification is simple text classification and you would need training data.


The complexity of one generation will be O(N/2). In order to make the whole population the same, you will need N generations. My answer will be O(N**2).

You can also evaluate the time complexity by actually measuring the run time and averaging.

If you don't do crossover, mutation or replacement you are just selecting the best of your individuals.