Creating a posterior distribution for classic coin flipping in python using grid search

In short

The problem assumes a uniform prior distribution function. All possible $P(H)$ are equally likely. Because they are standardizing the probability distribution function at the end it does not matter what value is placed in

prior=np.repeat(1,grid_points)

Likelihood function

The likelihood function answers the question, how probable is the prior which we assumes.

For example, when flipping a fair coin we can assume that the probability of getting a heads is $P(H) = 0.5$. Now, if we do an experiment and in 3 flips we observe 3 heads then $P(HHH|P(H)=0.5) = 0.5^3$. This tells us that their is only a 0.125 chance that we selected the correct prior.

$Posterior \propto Likelihood \times Prior$

The code

As you observed it is strange that they set their entire list with the value $5$. However this will have no effect on the results because the before last line standardizes the results. Try changing the $5$ to a different value and you will see that your results remain unchanged.

Let's go through the code

grid = np.linspace(0, 1, grid_points)

This creates an array between $0$ and $1$. This is the different probabilities we will assume to calculate our likelihood. For example if we use 5 grid_points we have $[0, 0.25, 0.5, 0.75, 1]$. So we will assume that $P(H)$ is equal to these values in turn and get the resulting likelihood for each value.

likelihood=stats.binom.pmf(heads, tosses, grid)

We can now set out prior however we wish. We can assume that the coin is fair, or we can have no clue and just assume that it's a uniform distribution. This is essentially what they are doing here. The value they chose makes no difference in the results. However, I think for clarity making it equal to 1 is more obvious.

prior=np.repeat(1,grid_points)

To calculate the posterior we multiply the prior by the likelihood

unstd_posterior = likelihood * prior

And then to get a probability distribution function we standardize the function.

posterior = unstd_posterior / unstd_posterior.sum()


Machine learning algorithms for forming Homophones from input dataset word

I have very limited knowledge about homophones generator.
I feel to make a homophone detector, one should focus more on the phonetics of the word rather than the spellings.

try to make a word-phonetics list dataset and then train a model.
focus on Levenstein/fuzzy/edit distance between the phonetics of words.

eg - two, too and to all have the same phonetics - T UW .
try this website to find phonetics - http://www.speech.cs.cmu.edu/cgi-bin/cmudict?in=to
they have already mapped the words and phonetics.
I think finding homophones in non-english languages would be an uphill task.


Best file format for transfer of EHR data

If your are using python, I suggest using Numpy's npy format. An NPY file is a NumPy array file created by the Python software package with the NumPy library installed. It contains an array saved in the NumPy (NPY) file format. NPY files store all the information required to reconstruct an array on any computer, which includes dtype and shape information.

You can also consider using hdf or hdf5 formats both of which are supported in Python. Hierarchical Data Format (HDF) is an open source file format for storing huge amounts of numerical data. It’s typically used in research applications (meteorology, astronomy, genomics etc.) to distribute and access very large datasets without using a database. One can use HDF5 data format for pretty fast serialization to large datasets.


when to use certain metrics for splitting decision trees?

This is not really a question about decision trees, but about properties to difference metrics. I would be surprised if you find significant differences in the final performance of your decision tree, for different metrics, but if you do find big differences, please comment!

So I don't think it matters very much what you choose here, but for example, the chi-squared numerical value has a specific meaning that is only valid if the data is normally distributed. If it is not, then the chi-squared can be artificially big, or small. But the best split will probably be the one with the lowest chi-squared, so that's why I don't think there is a big impact, even if your data is not normally distributed.

About the Gini coefficient: A given split results in two samples that you're comparing with the truth. The Gini coefficient has a downward bias for small samples, so if one of your samples after splitting is small, then your metric will be biased. That's not great, so probably, don't use the Gini coefficient in a decision tree.


Why does this paper claim to have found a minimal width of $d_{in}+1$?

From the abstract of the paper:

Specifically, we answer the following question: for a fixed $d_{in} \ge 1$,
what is the minimal width $w$ so that neural nets with ReLU
activations, input dimension $d_{in}$, hidden layer widths at most $w$,
and arbitrary depth can approximate any continuous, real-valued
function of $d_{in}$ in variables arbitrarily well?

"Real-valued function" means that the output dimension $d_{out}=1$. Their Theorem 1 is more general and says that for any $d_{in}, d_{out} \ge 1$ you will need $d_{in}+1 \le w \le d_{in}+d_{out}$.


Object detection is not improving although loss is decreasing

So the question asks why you are seeing a decrease in the loss function (for both training and validation?), but you are also observing decreasing generalisation performance over iterations.
One first thought could be due to the loss function that you have chosen might not be appropriate for your task.


R Studio - grepl compare a column in a dataframe to a list of pattern

TL;DR: grepl expects its first argument to be a string (length 1), not a vector. You can solve this with combinations of sapply and lapply (see below), but you are better served using a single regular expression that captures what you want to match in df1.MATCH and not use df2.PATTERN at all. This second option is much faster (if less intelligle) for a large data set. For this type of work, it is worth learning how to use regular expressions to their full potential.

df1 %>% filter(grepl(pattern = "^((ABC)( )*)+$", x = df1.MATCH, ignore.case = TRUE))

Explanation

The documentation for grepl shows the following usage:

grepl(pattern, x, ignore.case = FALSE, perl = FALSE,
  fixed = FALSE, useBytes = FALSE)

The pattern argument is first, and this argument should be a string (one element). You are providing df1.MATCH to this argument, which is a vector.

We could use sapply to apply grepl to each element of df1.MATCH.

sapply(df1.MATCH, grepl, x = df2.PATTERN)
       ABC   abc   BCD
[1,]  TRUE FALSE FALSE
[2,] FALSE  TRUE FALSE
[3,]  TRUE  TRUE FALSE

However, look at the output! You probably did not want a matrix. What happens when we run your grepl one just the first element of df1.MATCH?

grepl("ABC",df2.PATTERN)
[1]  TRUE FALSE  TRUE

We get a vector because grepl is checking ABC against each element of df2.PATTERN. To get a useful logical vector for filtering, you need to return a logical vector of the same length as df1.MATCH. I see two ways to do it.

  Method 1: Use any

Since you want to know which elements in df1.MATCH match any elements in df2.PATTERN, you can use any, which returns TRUE if any element in its arguments is TRUE. We need a little bit different syntax to make this work. We need to wrap grepl in lapply to make a list of three vectors (one for each element in df1.MATCH1) that feeds into sapply wrapped any. If we just use sapply, any will only return one value since we have a matrix input.

any(grepl("ABC", df2.PATTERN))
[1]  TRUE

sapply(
  lapply(df1.MATCH, grepl, x = df2.MATCH),
  any)
[1]  TRUE  TRUE FALSE

  Method 2: Write a better regular expression.

You want to match the contents of df1.MATCH against possible values that look like abc, ABC, ABC ABC, or ABC abc, etc. You can encompass all of this in a single regex string. The string you want is

"^((ABC)( )*)+$"
^                   # Nothing else before this
(ABC)               # Must contain ABC together as a group
( )*                # followed by any number of spaces (including 0)
((ABC)( )*)+        # Look for the ABC (space) pattern repeated one or more times
$                   # And nothing else after it

Then use grepl with ignore.case = TRUE:

grepl("^((ABC)( )*)+$", df1.MATCH, ignore.case = TRUE)
[1]  TRUE  TRUE  FALSE

  Benchmarking

In a large dataset, one of these will perform faster. Let's find out. Your benchmark results will vary by your machine's resources.

df1.MATCH <- sample(c("ABC", "abc" ,"BCD"), size = 100000, replace = TRUE)
df1 <- data.frame(df1.MATCH)
df2.PATTERN <- c("ABC", "abc", "ABC abc")

library(rbenchmark)

benchmark("any lapply" = {
              df1 %>%
                filter(sapply(lapply(df1.MATCH, grepl, x=df2.PATTERN), any) )
          },
         "better regex" = {
              df1 %>%
                filter(grepl("^((ABC)( )*)+$", df1.MATCH, ignore.case = TRUE))
          }
          )

          test replications elapsed relative user.self sys.self user.child sys.child
1   any lapply          100  149.13   70.678    147.67     0.39         NA        NA
2 better regex          100    2.11    1.000      2.10     0.02         NA        NA

It looks like the improved regex method is significantly faster. That's because it is performing only one operation per row (grepl) before filtering. The other method is performing four operations per row: lapply is performing grepl three times (one for each element of df2.PATTERN, and sapply then performs any for each list element (each row).


One reason for sure is, that you can easialy open it in spreadsheet viewer.


Credit scoring using scorecardpy with XGBoost

It happened to me as well, although I used a logistic regression model not XGBoost.

The problem is not about which model to choose, but rather there is something wrong with woebin_ply function. I didn't read the source code but the woe value I'm getting doesn't match the value for the corresponding bin/input value (You can double check your results as well).

After manually matching input value with bin with corresponding woe value, my scorecard model performs at similar level with my benchmarking models.

Hope this help!


Do modern R and/or Python libraries make SQL obsolete?

R and SQL are two completely different beasts. SQL is a language that you can use to query data that is stored in databases as you already experienced. The benefits of SQL versus R lays mostly in the fact of the database server (MS SQL, Oracle, PostgreSQL, MySQL, etc.).

Most, if not all, modern database servers permit multiple users to query data from the same data source and insert, update and delete data in the same tables all while ensuring that the data remains consistent. This is essential for say recording a bank transaction. Can you imagine running a bank on R? That's where database servers come in. They ensure ACID properties of procedures run on the database. ACID stands for Atomicity, concurrency, isolation and durability (see ACID description on wikipedia). R is a single user platform where everything happens in memory. So, if your computer stops working halfway in a big operation, your data will not be stored. You are also the only person who can access the data. To be clear, R is not considered an alternative for database servers and/or SQL.

Another main advantage of database servers is that a good database design will ensure that you can query your database fast by performing query optimization. To achieve this database servers keep track of the design of a table. See for a full discussion of this topic the wiki page. R cannot perform query optimization. Poor database design,  can lead to slow execution of your queries. Database servers can also perform optimization over queries that query multiple tables if foreign keys are properly used in the database design.

The SQL language has a very different syntax and I share your experience that it is shorter to write data munging steps using the data table or dplyr syntax. However, sometimes your data is too big for R or you need to store the results in the database as part of a periodic batch job, which will require to code your logic in SQL.

In my experience there are particular use cases for SQL and R/Python. SQL is great for storing business critical data and for allowing multiple people to access, modify, insert and delete data in a centralized environment. For any one-off data munging R and Python are great. If your data munging needs to be periodically executed, you will need to port your R/Python script to SQL.


Why can't I use 2D-arrays as features for CCA (Canonical Correlation Analysis) classifier?

No, coding-wise, it cannot use multi-dimensional arrays as single features. You can flatten the array though. Essentially, you want to reshape your input to be (189, 250).


Machine Learning algorhtm that can be trained to learn Pig-Latin

Look at Neural Machine Translation models, you could either 'cheat' and tokenize both the English and the Pig-Latin words or you could make it character based which I think is more interesting and it allows to generalize for words out of dictionary. These models are called sequence to sequence models and there are plenty of implementations that can take pairs of sequences to translate. I think this task can be solved with accuracy close to 100%.

Here is an implementation in Keras: https://github.com/farizrahman4u/seq2seq

Here is an official TensorFlow tutorial: https://github.com/tensorflow/nmt


Meaningful predictive analytics for small (n=114) dataset with just 1 explanatory variable and 1 response variable?

Create additional features from the given features. From the Monday "date" feature, the features ("month","year") can be found. Additionally, months of the year contain either 4 or 5 Mondays, so a "week" feature can be created with a value 1...5 that can be used to represent the week of the month that the Monday occurred on. Additional features can be created by calculating trends (Average Shipments for the previous N shipments,...etc.). This gives the algorithm more information to work with and should provide more refined results.


Clustering customer dataset to find customer patterns

Suggestion: use an indicator variable for your "order"+"orderAmount" data. The table would look like this, which seems more suitable for distance metrics:

name      | tv | hifi | cd | timestamp
customer1 | 1  | 0    | 0  | 8:30
customer2 | 0  | 1    | 0  | 12:00
customer3 | 0  | 3    | 0  | 12:30
customer4 | 2  | 0    | 0  | 9:30
...

If you do so, then I'd also suggest to normalize those indicator variables (make sure they are in the interval [0, 1], otherwise you could have a order of 1000 "cd" completely dominating the maximum of 10 "tv" for example).

Another thing to watch out is the possibility of binning. If the time of the day doesn't matter much, you could group all the orders from the same day and customer in a single row.

Also, if you can manually label some examples, you could use a semi-supervised algorithm, and maybe it would have better performance than completely unsupervised clustering. Some possible algorithms are HMRF-KMeans, co-training variants, and Spy EM.


What features used by CNN model should a feature store actually store?

Disclaimer: i am one of the authors of the referenced link.

If you are doing pure image classification/segmentation, I would say no.
I would not store the output of convolution layers, noramlly. In principle, the output of models can be stored in the feature store, however.

Feature stores store data in tabular file formats (like parquet) and are used to store both immutable data (like images) and features that change over time (number of clicks on the website in the last 1 hour). These latter historical/contextual features need to be constantly updated by feature pipelines. Typically, you cache them in an online feature store, so that your model can retrieve them to build feature vectors.
You could store image data in the feature store, but I would normally just store the path to an image (on a distributed file system or object store), as well as all the metadata about the image that i need to make predictions. By storing image paths, it is easy to retrieve them to build training datasets in file formats designed for efficient training of DNNs on image data, like Petastorm.


ML algorithms for defining NORMAL user behavior

I think it depends mostly on how difficult/expensive/time-consuming it is to gather data for abnormal behavior.It also depends on whether you'd expect this abnormal behavior to belong to one (or a fixed number of) classes or it could be anything that deviates from normal.

If you can afford to collect labelled data and expect the abnormal behavior to be limited to a fixed number of classes, then the supervised approach is very likely going to give you better results.

Instead, if any of the above hypothesis do not hold, then I think the semi-supervised approach (as you said, One Class SVM or similar techniques) is in my opinion a good way to go.


How to search collection of podcasts (.mp3 files)?

Transcribe then topic model the recordings. This will let you know which podcast talks about the subject of interest, then you can search the transcript. If you really need it to pinpoint the moment when the subject was discussed, you'll need a temporal topic model like so but that would add a good bit of complexity.

Google has a good service for the first part. You can use an open source library like gensim for the second part.


How can I train a many-to-one RNN with an array of 2D matrices?

you can use CNN just before inputting your raw matrix into RNN. An other option, which is just made for that purpose is ConvLSTM. Several research papers show that ConvLSTM can be used to process in sequential frames of video which is similar to your input type.


Which Algorithm did OpenAI used to create a hide and seek playing Agent?

This is specified in the original paper that led to that video:

Policies are optimized using Proximal Policy Optimization (PPO) (Schulman et al., 2017) and Generalized Advantage Estimation (GAE) (Schulman et al., 2015)

They, nevertheless, used concepts often used in evolutionary algorithms, specifically competitive co-evolution, as the goal of the seeking agents is contrary to the goal of the hiding agents, and this drives the reinforcement signal.


Classifying hand written digits dataset using Probabilistic neural network (PNN)?

Reducing the sample size is pretty much up to you.  PNN have the drawback that the size of the network after training is equal to the size of the training dataset.  Therefore it's a bit impractical to use for large datasets.  I also believe that there is evidence that suggest the performance may also decrease when the layers are big.

I don't have access to the paper but I imagine a big reason why they reduced the sample was to avoid the memory error that you're encountering right now.  I also suspect they carefully selected a good representation for the sample.


It is kind of unfortunate that no further explanations were given in the paper about this. In my opinion Hard negative mining is actively used by architectures like SSD, actually by boosting its performance with the use of selective training.
I could speculate about what the authors meant, namely that a one-stage detector does not have per construction any element capable of performing the hard negative mining, whereas Faster-RCNN (sparse predictor) does as it relies on two different stages (classification/bounding box regression + RoI generation) to discriminate the results actively.

Dense prediction means that RoIs are a dense sample of possible boxes around points of interest (image or feature maps). Object Detectors whose output is the subset of the top relevant RoIs (and output features) out of these are dense predictors. They are planned as a self-contained framework that runs in a one-stage manner. Dense prediction is also part of two-stage detectors as you can see in the image, but specific boxes are used for further processing (e.g. a box refinement method).


I don't think you need to worry, instead I would ask myself if the accuracy I'm getting is good enough for the task that the NN is supposed to do.
Having higher training loss than validation loss can mean different things:

Your validation data is easier to assess than training data. If the train/validation split is done randomly and there is enough data in both subsets, this shouldn't be the case.
You're using dropout in training but not in validation. This is the default of some deep learning libraries, and it makes sense. If this is the case and you want to see less of a gap, try to reduce the amount of dropout rate and you'll see less of a gap.

To sum up, I don't think it's an issue but you might be able to improve your validation performance by reducing the amount of regularization or increasing the complexity of the NN. However, this is just a hypothesis and the only way to know is to re-train and check the new performance.

Edit
By default, keras doesn't do dropout in prediction so this is likely your case since you have high dropout rates.


looking for approaches to detecting outliers in individuals unequal sequential time series

Outlier detection doesn't sound like the most promising approach to me, as you have a model for the data.  Some ideas you could try: use a hypothesis test to check the hypothesis that the stress values fit iid Gaussian with pre-defined standard deviation and unknown mean; use linear regression to fit a line that predicts stress as a function of time, and see if the slope is greater than zero by a statistically significant amount; etc.


Support vector machine margin term, norm or norm squared?

It is indeed for computational tractability. You would not lose the all-important sparsity, since that is provided by the second term. The original formulation of SVM, which you can find on Wikipedia actually uses $\lVert w \rVert$:

Minimize $\|{\vec {w}}\|$ subject to ${\displaystyle y_{i}({\vec {w}}\cdot {\vec {x}}_{i}-b)\geq 1}-\xi_i$

Obviously, minimizing $\lVert w \rVert$ is the same as $\lVert w \rVert^2$. A quantative difference arises when introducing the constraint using the Lagrangian function, but qualitatively it is the same.


what can be done with the following data?

I think you're looking at it the wrong way. First, you need to have a question and then you may find out whether the data could potentially hold the answer to that question. Right now, you're asking us to what questions this data may hold the answer. And that is a very difficult question!

But to provide you with at least some ideas:

Find out what people deem important (the correlation between overall rating and the other ratings in (linear) regression analyses)
Find out the relation between certain characteristics of people (age/gender/nationality, if available) and their overall rating (or whatever rating available)

Basically, the relation between certain variables and their outcomes. But make sure you have a clear question before you start looking for answers.


Predicting car failures with machine learning

I think you should first clearly specify what the covariates are, what is the target variable is and what is your goal.

Therefore, if you have attributes about the car as covariates and target variable is failure time (car failed in 1y, 2y...), then the best approach is indeed Survival analysis, because you try to model time to failure.

On the other side, if you target variable is just a failure - yes or no, then its a classification problem. For that, simple models such as Decision trees or Logistic regression are very well suited.

Dont use algorithm just because its fancy or dont dislike other approaches just because they are not "Machine Learning".

You said you want to start with Machine learning, so go ahead. Dont blindly fit whatever blackbox models, start with simple ones and look inside, how they work.

That being said, pick something more simple. Because Survival analysis requires knowledge or regression and a bit more of statistics.

Best of luck.


Best way to narrow down a list and rank based on attributes?

So it a little hard to know how to answer the question. Why do you need to cut the number of records down? If you want to rank them, rank them to what purpose? Ranking implies there are positives and negatives, or at least less positives.

One way you could approach this is to perform PCA. That said, I think you should consider disqualifying some fields, like the address fields.


Is it possible to deploy a python trained machine learning model (e.g. a .pkl file) in C language?

Library http://www.picklingtools.com/

#include "chooseser.h"

// assume that pickle file is "loaded_pkl";
int main()
{

  Val loaded_pkl;
  LoadValFromFile(loaded_pkl, "my_pkl.pkl", SERIALIZE_P0);
  cout << loaded_pkl << endl;

}


Why is this series of house prices being thrown into one bin with plt.hist?

Can't reproduce:

>>> housing = pd.DataFrame([410000, 610000, 579950, 559950, 404950],
                           columns=['price'], dtype=int)
>>> housing.price
0    410000
1    610000
2    579950
3    559950
4    404950
Name: price, dtype: int64

>>> plt.hist(housing.price)

Resolution

You have an outlier at 1e8 while most observations fall into 1e7 range.


How does Wav2Vec 2.0 feed output from Convolutional Feature Encoder as input to the Transformer Context Network

They key is that the 768 dimensional vector of the transformer is the size of a single input... let me explain.

You start with a variable length audio input
This is passed through a temporal CNN network, which will give you outputs, called $z_1$ to $z_T$ by the paper - where T is variable across the batch and is the number of timesteps in a particular audio input (different for different audio inputs).
Each of these $z$ are passed as 768 dimensional vectors to the
transformer, i.e. there are $T$ 768-dim vectors being passed to your
transformer

As you can imagine, practically you would need to pass a mask to the transformer too, so it knows what the variable $T$ is for the different inputs in your batch.


I took your exact code, replaced
model.add(Activation('sigmoid'))

by
model.add(Activation('relu'))

and indeed I experienced the same problem than you: only 55% accuracy, which is bad...
Solution: I rescaled the input image values from [0, 255] to [0,1] and it worked: 93% accuracy with ReLU! (inspired from here):
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers.core import Dense, Activation
from keras.utils import np_utils

(X_train, Y_train), (X_test, Y_test) = mnist.load_data()

X_train = X_train.reshape(60000, 784)
X_test = X_test.reshape(10000, 784)
X_train = X_train.astype('float32') / 255
X_test = X_test.astype('float32') / 255

Y_train = np_utils.to_categorical(Y_train, 10)
Y_test = np_utils.to_categorical(Y_test, 10)

batch_size = 100
epochs = 15

model = Sequential()
model.add(Dense(100, input_dim=784))
model.add(Activation('relu'))
model.add(Dense(10))
model.add(Activation('softmax'))

model.compile(loss='categorical_crossentropy', metrics=['accuracy'], optimizer='sgd')

model.fit(X_train, Y_train, batch_size=batch_size, epochs=epochs, verbose=1)

score = model.evaluate(X_test, Y_test, verbose=1)
print('Test accuracy:', score[1])

Output:

Test accuracy: 0.934

Potential explanation: when using an input in [0, 255], then when doing the weighted sum for the layer $L$: $z = a^{(L-1)} w^{(L)} + b^{(L)}$, the value $z$ will often be big too. If $z$ is often big (or even if it's often > 0), let's say around 100, than $ReLU(z) = z$, and we totally lose the "non-linear" aspect of this activation function! Said in another way: if the input is in [0, 255], then $z$ is often far from 0, and we totally avoid the place where "interesting non-linear things" are going on (around 0 the ReLU function is non linear and looks like __/)... Now when the input is in [0,1], then the weighted sum $z$ can often be close to 0: maybe it sometimes goes below 0 (since the weights are randomly-initialized on [-1, 1], it's possible!), sometimes higher than 0, etc. Then more neuron activation/deactivation is happening... This could be a potential explanation of why it works better with input in [0, 1].


Since I was also looking for it just yesterday being a newbie,sharing what I found , As per Orange FAQ it says

and it directs to the link @K3---rnc mentioned.

But i found similar request as well not sure how well it works-

Orange Data Mining load saved models

https://stackoverflow.com/questions/45598627/orange-save-model-in-python-script


Pixel labelled image is must for semantic segmentation using deep learning in matlab?

If you have the bounding boxes then you can use MaskRCNN and it will be able to give you those segmentations.
If you don't, look at unsupervised segmentation. Also, you need something as an output if you want to try out supervised learning.


How to impute Missing values not the usual way?

To decide which strategy is appropriate, it is important to investigate the mechanism that led to the missing values to find out whether the missing data is missing completely at random (MCAR), missing at random (MAR), or missing not at random (MNAR).

MCAR means that there is no relationship between the missingness of the data and any of the values.
MAR means that that there is a systematic relationship between the propensity of missing values and the observed data, but not the missing data.
MNAR means that there is a systematic relationship between the propensity of a value to be missing and its values.

Given what you have told its likely that its MCAR. (assumption is that you already tried to find this propensity yourself (domain knowledge) or build a model between the missing columns and other features and failed in doing so)

Some other techniques to impute the data, I would suggest looking at KNN imputation (from experience always solid results) but you should try different methods

fancy impute supports such kind of imputation, using the following API:

from fancyimpute import KNN

# Use 10 nearest rows which have a feature to fill in each row's missing features
X_fill_knn = KNN(k=10).fit_transform(X)

Here are different methods also supported by this package:

  •SimpleFill: Replaces missing entries with the mean or median of each
  column.

  •KNN: Nearest neighbor imputations which weights samples using the
  mean squared difference on features for which two rows both have
  observed data.

  •SoftImpute: Matrix completion by iterative soft thresholding of SVD
  decompositions. Inspired by the softImpute package for R, which is
  based on Spectral Regularization Algorithms for Learning Large
  Incomplete Matrices by Mazumder et. al.

  •IterativeSVD: Matrix completion by iterative low-rank SVD
  decomposition. Should be similar to SVDimpute from Missing value
  estimation methods for DNA microarrays by Troyanskaya et. al.

  •MICE: Reimplementation of Multiple Imputation by Chained Equations.

  •MatrixFactorization: Direct factorization of the incomplete matrix
  into low-rank U and V, with an L1 sparsity penalty on the elements of
  U and an L2 penalty on the elements of V. Solved by gradient descent.

  •NuclearNormMinimization: Simple implementation of Exact Matrix
  Completion via Convex Optimization by Emmanuel Candes and Benjamin
  Recht using cvxpy. Too slow for large matrices.

  •BiScaler: Iterative estimation of row/column means and standard
  deviations to get doubly normalized matrix. Not guaranteed to converge
  but works well in practice. Taken from Matrix Completion and Low-Rank
  SVD via Fast Alternating Least Squares.

EDIT: MICE was deprecated and they moved it to sklearn under iterative imputer


Why aren't Genetic Algorithms used for optimizing neural networks?

Training Neural Networks (NNs) with Genetic Algorithms (GAs) is not only feasible, there are some niche areas where the performance is good enough to be used frequently. A good example of this is Neuroevolution of augmenting topologies or NEAT, which a successful approach to generating controllers in simple environments, such as games.

In the more general case though, the approach does not scale well to large, deep networks with many parameters to tune.

Genetic algorithms and other global searches for optimal parameters are robust in ways that gradient-based algorithms are not. For instance, you could train a NN with step function activations, or any other non-differentiable activation functions. They have weaknesses elsewhere. One thing relevant in the case of GAs used for NNs, is that weight parameters are interchangeable in some combinations but heavily co-dependent in other combinations. Merging two equally good neural networks with different parameters - which you would do in cross-over in a GA - will usually result in a third network with poor performance. NEAT's success is partially in finding a way to address that issue by "growing" the NN's connections and matching them up between similar neural networks.

Gradient-based approaches are much more efficient. In general, and not just in domain of NNs, if you can calculate gradient of a function with respect to parameters, then you can find optimal parameters faster than most other optimising techniques. An accurate gradient guarantees at least a small improvement from a single evaluation, and most other optimisers fall into a generate-and-retry paradigm which cannot make that kind of guarantee. The weakness of tending to find local optima has turned out not be a major hindrance for the loss functions in NNs, and has been tackled with some degree of success using extensions to basic gradient descent such as momentum, RPROP, Adam etc.

In practice on a large multi-layer network, gradient methods are likely orders of magnitude faster than GA searches such as NEAT for finding network parameters. You won't find any GA-trained CNNs that solve ImageNet, or even MNIST, where the GA has found the network weights unaided. However, GAs, or at least some variants of them, are not 100% ruled out. For instance this 2017 blog reviews recent papers including Large-Scale Evolution of Image Classifiers which explores using GAs to discover NN hyperparameters which is an important task in machine learning, and not very tractable using gradient-based methods.


Beautiful Soup is specifically designed for web crawling and scraping, but is written for python and not R


How many trees does a Random Forest need?

By other posts and this one seems what you don't have a clear intuition of the n_estimators of the random forest.

I am going to assume that you are referring to the n_estimators (from this other question). n_estimators is the number of trees that your 'forest' has. Not the depth of your tree. That is another parameter.

If you are referring to max_depth = 100, 100 splits it can be a lot. Feel free to plot one tree and see what it is doing. see this link

The number of rows and columns is not necessarily important, what is important is the complexity of your problem.


How to know when to stop trainning a deep network?

Thats the whole point behind early stopping, don’t train the network to too small training error. It is always a bit tricky with early stopping but following framework might help.

When training, also output validation error

• Every time validation error improved, store a copy of the weights

• When validation error not improved for some time, stop

• Return the copy of the weights stored

After early stopping of the first run, train a second run and reuse
validation data
How to reuse validation data

Start fresh, train with both training data and validation data up to
the previous number of epochs
Start from the weights in the first run, train with both training data
and validation data until the validation loss < the training loss at the
early stopping point


Two-tower net does not learn when made deep

I now have a working network. It turned out that the gradients were all zeros after only about 3000 update step. I tried two approaches to fix this - using Batch Normalization after each activation function in the feed-forward net and changing the activation function from ReLU to Leaky ReLU. Both worked, and I ended up using the Leaky ReLU without normalization.
For the full thread in the PyTorch discussion forum here


in which case can I say that the data are bad and I ll achieve nothing using machine learning on it

NN is not a magic bullet

Not every model works well on every dataset, is the ethos.

Even tough NN is very powefull for certain tasks you could find tasks where linear regression would be a better choice.

Quality of data could mean 2 things:

a) data is unstructured and not processed. Think in terms of some super ugly text logs

b) data is uninformative. There are no patterns that can be extracted no matter what you try.

To conclude, experiment with different approaches and always have a holdout dataset to do sanity checks


What statistical method can be applied in my case?

There are a number of methods for this. Here's a list:

You can build a Regression model and observe the p-values of the coefficients of each variable.
Pearson Correlation
Spearman Correlation
Kendall Correlation
Mutual Information
RReliefF algorithm
Decision trees
Principal Component Analysis (which you have tried)

etc.

You can search for other methods with these keywords:

feature selection, variable importance, variable ranking, parameter selection

They are all almost the same, just different terminology in different fields.

However, the issue that you MUST pay attention here is "10" observations are almost nothing. It will be extremely difficult to trust any outcome, regardless of the method you use.


Cheat Sheet of UNIX commands for Data Science

I would recommend this one - http://www.cheat-sheets.org/saved-copy/ubunturef.pdf

Its comprehensive, but not data science specific.

There are a number of great cheat sheets here that may be what you are looking for:https://dzone.com/refcardz

If you are wanting to create your own, this is a very nice tool: http://www.cheatography.com/


Validation loss is lower than the training loss

It is certainly correct in the sense that it is a legitimate neural network. The dropout layer introduces noise that is not injected during the test period. The goal is to combat overfitting so that the error in your test set will be lower due to better generalization. Applying the dropout layer on top of the input layer however throws away a lot of information, meaning it will be difficult to learn a good strategy. Usually dropout layers are applied after the first transformations, which means the neural network has to rely on multiple different transformations that will mean something similar, reducing complexity. However, if you put it over your input layer you are throwing away information that is not accessible in any other way,


Need help in understanding a small example

Regarding your first question, by dividing by the length, you get the percentage of each category, and you can see that one category has a very low percentage.

If you use the regular train_test_split, the proportion of the categories will be different in each set, as the split is made randomly, and this can introduce a bias.

Imagine categories with a very low number of observations, you could even have categories missing entirely in either of the sets, which will cause trouble for your model.

The use of stratified sampling allows you to have the same proportion for the categories. That is why you see the same results: it is the goal of this sampling.


Handling a Pandas Data Frame containing multiple non-ordinal categorical features

It looks like a case where target encoding will shine.

Target encoding replaces a category for the mean target in that category. You have to be careful not to overfit, but it is a very effective method to use categorical features with many levels.

There's a python package that implements target encoding using the syntax of Scikit Learn.


How to use MNIST dataset to make predictions on similar images (colorblindness charts)?

I think the $RGB$ color components of the pixels form vectors that if you see them in spherical coordinates, angles are colors, and radius is intensity.

I tried it and ended up with:

Build a 3-class classifier for the angles calculated for each pixel (background and two colors).

If you end up with only 2 classes (the third one has very few members), you have an intensity problem, and you should reclassify for radius (or just take a threshold value).

If you have 3 classes, you have a color problem. Drop the class of the (black) background color (probably just the class of the top left pixel. Also drop the class with the largest mean distance to the center of the image, the second background.

The remaining pixels will be white, the others black.

You still have a picture with circles forming the digit, but a pooling filter will even it out.

3-way classification directly on the image was unsuccessful, because the distance between light gray and dark gray is similar to light gray/light purple. The angles are much more separable.


The perceptron algorithm was invented in 1957 at the Cornell Aeronautical Laboratory by Frank Rosenblatt (Wikipedia EN)

It was the first neural network like architecture. It was thought that (a combination of) perceptrons could learn anything. There was a lot of controversy about this and after a while Marvin Minsky showed that a single perceptron could not solve the XOR problem. This was the end of the first neural network hype.

After a couple of decades the multilayer perceptron was invented, and with it the backpropagation algorithm. The backpropagation algorithm can learn weights that are not directly connected to the output unit. However, it needs a differentiable activation function. Hence, the ramp activation function was not sufficient anymore and others like the sigmoid and tanh were used for this.

Besides, the perceptron was first implemented using electronic boards and not with the software available to us today. Binary thresholds made this complicated process a lot easier.

So, in short Rosenblatt could have chosen for any version of the heaviside step function but he chose for the one where $w \cdot x + b = 0$ leads to a zero output. If you think binary values this makes sense, a zero input (i.e. False) leads to a zero output.


How to define confusion matrix for classification?

You are just predicting if Play = Yes or Play = No.

The confusion matrix would look like this:

             Predicted
          +------+------+
          |  Yes |  No  |
    +-------------------+
A   |     |      |      |
c   | Yes |  TP  |  FP  |
t   |     |      |      |
u   +-------------------+
a   |     |      |      |
l   | No  |  FN  |  TN  |
    |     |      |      |
    +-----+------+------+

TP: True positives
FP: False positives
FN: False negatives
TN: True negatives

The accuracy can then be calculated as (TP + TN)/(TP + FP + TN + FN).


Non-convex feasible region in a linear programming problem

Sorry, this cannot be done with a continuous LP solver. As you observed, this construct introduces a non-convexity. However, it can be handled by a Mixed Integer Programming (MIP) solver.

$x=0$ or $\ell\le x\le u$ is what is known as a semi-continuous variable. Many advanced MIP solvers support this variable type directly.

If a MIP solver does not support this, we can introduce a binary variable $\delta \in \{0,1\}$ and use the constraint:

$$ \ell \cdot \delta \le x \le u \cdot \delta  $$

to simulate a semi-continuous variable.

COIN-OR has a capable MIP solver called CBC (I believe it supports semi-continuous variables directly).


Shaping columns (and column headers) into multi-index rows using pivot/pivot_table

For this I would perform two steps, first transforming the data from a wide to long format using pandas.melt (i.e. transform the TP columns over the rows) and then use pandas.pivot to get the desired format.
result = (
    df
    .melt(id_vars=["POC_Code", "Trading_date"])
    .pivot(index=["Trading_date", "variable"], columns="POC_Code")
    .reset_index()
)

result

# Trading_date variable  value
#                         ARA    BPE
#   1/04/2018      TP1  10120   3545
#   1/04/2018      TP2  10170   4329
#   1/04/2018      TP3  10200   3465
#   2/04/2018      TP1  11760  12749
#   2/04/2018      TP2  11790  13793
#   2/04/2018      TP3  11750  13943
#   3/04/2018      TP1  25930  11358
#   3/04/2018      TP2  25890  15448
#   3/04/2018      TP3  25860  16132

If you want to change the column names you can simply assign the new column names to df.columns:
result.columns = ["Trading_date", "TP", "ARA", "BPE"]


How to measure F1 score and NMI for clustering task?

Use the original papers, and books, on F1 and NMI.

There is little reason to only use a 2016 arxiv (non-reviewed) paper that uses them, because they have been introduced long before.

I don't see F1 mentioned anywhere in the paper,and it would be an uncommon choice for clustering (see literature for details).


Neural Network for regression with one dependent and one independent variable

Your predictions are not actually that bad. At the very last line of your code, print the expected value too at each line (that is x_test[i]= y_test[i]^2+0.2).


Dummies Variables and Scaling in Regression Problems

Would this approach create an inappropriate bias considering the different scale from dummies and the scaler numerical? Or, at least, is it correct?

"It is fine" because One-Hot-Encoded features don't have a very large scale. Issue of scale comes when features are at a very different scale e.g. 1 vs 105 etc.
Standardizing them will disturb the sparsity(i.e. log of zeroes). Normalizing them will not have any impact.
What is incorrect here is -
 You are standardizing the dataset prior to the split. You must do the other way round.
What is optional here is -
 You don't need scaling and OHE for a Tree-based model. Just label encode, split and move.


Do Clustering algorithms need feature scaling in the pre-processing stage?

Clustering algorithms are certainly effected by the feature scaling.

Example:

Let's say that you have two features:

weight (in Lbs)
height (in Feet)

... and we are using these to predict whether a person needs a 'S' or 'L' size shirt.

We are using weight+height for that, and in our trained set let's say we have two people already in clusters:

Adam (175Lbs+5.9ft) in 'L'
Lucy (115Lbs+5.2ft) in 'S'.

We have a new person - Alan (140Lbs+6.1ft.), and your clustering algo will put it in the cluster which is nearest.  So, if we don't scale the features here, the height is not having much effect and Alan will be allotted in 'S' cluster.

So, we need to scale it. Scikit Learn provides many functions for scaling. One you can use is sklearn.preprocessing.MinMaxScaler.


If your goal is to transform your matrix into a number (your similarity measure), you may want to use a matrix norm.

For instance, using the Frobenius norm on your example would return 1.488086.


K-Means clustering for mixed numeric and categorical data implementation in C#

The solution is described here: K-Means clustering for mixed numeric and categorical data

C# implementation can be found in ALGLIB library, which I strongly recommend: http://www.alglib.net/translator/man/manual.csharp.html#gs_packages


Why do pre-trained CNNs use low image resolution?

Would it potentially have a big impact to use higher resolution images?

Yes, if you increase the input size to your convolutional neural network, the size of each activation map for each layer increases, so you will have more computation. Also if you use same architecture, the number of neurons and consequently the number of parameters, in dense layers increases.

  Does it make sense and is it possible to use a pre-trained network on low resolution and re-training the last layer/classifier with higher resolutions?

The answer is no. When you train a network with a special size of input, you reserve variables to hold the weights and middle variables. If you increase the size of input, dense layers will have different size, so their number of wights should vary too.

To wrap up, for networks with classification tasks, it is appropriate to pass the network small size of images. For other tasks like edge detection where the information of edges can be destroyed by resizing, you have to be careful. In those cases you have to find an appropriate size of the image in order to keep the important information. The small size of the inputs is for reducing number of operations and number of parameters.


First off, it's always helpful to think geometrically about what complex numbers are, and what arithmetic operations achieve.

In your function, you are using the mean and standard deviation of the absolute value of these complex numbers. That means that if you perform your operation to the absolute value of your data:

(tmp - tmp.mean()) / tmp.std()

you will end up with normalized data of mean 0 and standard deviation 1.

Going back to thinking geometrically, when you perform your original operation:

(x_source - tmp.mean()) / tmp.std()

you are essentially moving your data's mean tmp.mean() units to the left, then scaling horizontally by 1/tmp.std().

Notice none of this is vertical shift or scaling, so something smells funny.

What I would do: I would normalize each coordinate independently.

Finding the mean is fine -- the mean of complex data points is the same as means of components:
$$
\bar z = (\bar x , \bar y)
$$
So you can subtract the mean of the $x$ value's from each input value's $x$-coordinate. Ditto for $y$.

Then you divide the real component by the standard deviation of the real component, and ditto for the imaginary component.

It could also be appropriate to divide by the standard deviation of the (new) norms. This would ensure good properties involving your data lying within a circle of a certain radius.

Code:

real_data = real(x_source)
imag_data = imaginary(x_source)

real_data = ( real_data - real_data.mean() ) / real_data.std()
imag_data = ( imag_data - imag_data.mean() ) / imag_data.std()

x_source_norm = real_data + i * imag_data


Is it correct to join training and validation set before inferring on test-set?

Yes, once you optimized your model and parameters with the validation set, it is advised to train your final model on the combination of the training and validation sets before applying it to the test set. Indeed, you can see the validation set as a subset of the training set. It is used to tweak your models and your parameters, but once it is done, it would just be a waste to not use the validation set for training during the testing phase.

Remember that once you tested your model on the test set, you should not tweak it anymore.


What is the best references / books to get familiar with machine learning algorithms?

If you want to get more into Machine Learning and the mathematical/theoritical aspects that it includes, the book mentioned below is an option:

Machine Learning by Tom Mitchell, McGraw Hill, 1997.


including Zeros counts categories with pandas value_counts()

I got the answer use of Series.reindex with a list of all categories and if not match replace to 0
  categories = ['a', 'b', 'c', 'd', 'e']
  pd.concat([df.col1.value_counts().reindex(categories[::-1], fill_value=0),        df.col1.value_counts(normalize=True).reindex(categories[::-1], fill_value=0).mul(100).round(1).astype(str) + '%'],axis=1, keys=('Counts','Percentage'))

This worked for me


What is the proper way to use early stopping with cross-validation?

I suspect this is a "no free lunch" situation, and the best thing to do is experiment with (subsets) of your data (or ideally, similar data disjoint from your training data) to see how the final model's ideal number of estimators compares to those of the cv iterations.

For example, if your validation performance rises sharply with additional estimators, then levels out, and finally decreases very slowly, then going too far isn't such a problem but cutting off early is.  If instead your validation performance grows slowly to a peak but then plummets with overfitting, then you'll want to set a smaller number of estimators for the final model.  And then there are all the other considerations for your model aside from straight validation score; maybe you're particularly averse to overfitting and want to set a smaller number of estimators, say the minimum among the cv iterations.

Another wrench: with more data, your model may want more estimators than any of the cv-estimates.  If you have the resources to experiment, also look into this.

Finally, you may consider leaving an early-stopping validation set aside even for the final model.  That trades away some extra training data for the convenience of not needing to estimate the optimal number of estimators as above.


Quantify the amount of differences of one data set to another

Just average (or pool, moree generally aggregate) some of the standard measures:

if you look closly python implementation is already general for some of them, meaning that you can apply them directly on dataset and not individually on the vectors

Examples: Cosine similarity, Eucledian distance, Manhattan/city-block distance, Chebychev distance, Minkowsky dustance. For computer vision, similarity measures are: Hausdorff Distance (the two data vectors will be considered close if every point of each set is close to some point of the other set), Bhattacharyya Distance (measure of similarity of two probabilty distributions), Bhattacharyya coefficient (measure of relative closeness of the two vectors) and Mahalanobis Distance (a specific case of Bhattacharyya Distance).


How to interpret Correlation along with Coefficients of multiple linear regression ?£

The  table of correlation coefficients shows the pairwise correlation between the variables in your data set: on a range from 0 (no correlation) to 1 (full correlation), to what extent does variation in one variable explain variation in the other variable?

The coefficients from the regression table, on the other hand, describe the relation between y and the different x's all else being equal. For example, the coefficient estimate of x1 tells you that, provided x2 to x4 are held constant, y is expected to change by .959 units when x1 changes by 1 unit. To understand the relation between your y and the different x's, these coefficients are usually more informative than pairwise correlations.

In addition to the strength of the effect (the site is the coefficients) which can be used to assess the substantive significance of a predictor variable, the regression table gives p-values that are used to judge the statistical significance of a predictor variable. A low p-value (conventionally below .05) suggests statistical significance (meaning a low probability of observing your data under the null hypothesis).

So looking at your regression output, you can see that all four predictor variables have a statistically significant relation with y. X1 to x3 are more significant than x4, both in terms of statistical and substantive significance. The contrast between the strong pairwise correlation between x4 and y and the small regression coefficient is due to multicollinearity between the x's: x4 covaries with the other predictors, and it is this covariance which accounts for the correlation with y.


What annotators are used in Cohen Kappa for classification problems?

What 2 annotators or observers does Cohen Kappa use in classification problems when you compute the score in scikit learn?

The two annotators are exactly the two parameters you fed the method with: cohen_kappa_score(y_test, predictions), i.e. the truth/labels and your predictions. And these are then measured in terms of their "inter-rater" agreement.


What is the benefit of splitting tfrecord file into shards?

Splitting TFRecord files into shards helps you shuffle large datasets that won't fit into memory.

Imagine you have millions of training examples saved on disk and you want to repeatedly run them through a training process. Furthermore, suppose that for each repetition of the training data (i.e. each epoch) you want to load the data in a completely random order.

One approach is to have one file per training example and generate a list of all filenames. Then at the beginning of each epoch you shuffle the list of filenames and load the individual files. The problem with this approach is that you are loading millions of files from random locations on your disk. This can be slow especially on a hard disk drive. Even a RAID 0 array will not help with speed if you are loading millions of small files from random locations. The problem gets even worse if you are accessing the files over a network connection.

Another approach is to read the training examples in sequence from one large TFRecord file and shuffle the examples in memory using a shuffle buffer. However, the shuffle buffer typically cannot be larger than the DDR memory available to your CPU. And if the shuffle buffer is significantly smaller than your dataset, then it may not adequately shuffle the data. The data may be "locally" shuffled but not "globally" shuffled. That is, examples from the beginning of the dataset may not be shuffled with examples from the end of the dataset.

A good solution is to use a balanced combination of the above two approaches by splitting your dataset into multiple TFRecord files (called shards). During each epoch you can shuffle the shard filenames to obtain global shuffling and use a shuffle buffer to obtain local shuffling. A good balance will make the shards large enough to prevent disk speed issues but will keep the shards small enough to allow for adequately shuffling by a shuffle buffer.

Here are the exact steps:

Randomly place all training examples into multiple TFRecord files (shards).
At the beginning of each epoch, shuffle the list of shard filenames.
Read training examples from the shards and pass the examples through a shuffle buffer. Typically, the shuffle buffer should be larger than the shard size to ensure good shuffling across shards.
Pass the shuffled examples into your training process.


What is the proper way to use time-series data for classification?

Welcome to the community!

Well ... the answer needs a bit of explanation. It is not about MLPclassfier or another python function. It is about background in Signal Processing/Analysis specially for bio-signal.

For different bio-signals like EEG there has been feature extraction studies for long while. For example in your problem, BCI, you probably know P300, etc which are common features (I am pretty outdated. You may have a look at recent review papers.). So usually you extract features from these signals and use them as the input of your classifier (those X, Y values that you mentioned). I explain a bit:

In General

There are general feature extraction methods from signals. They either extracted from signal in time-domain or frequency domain or time-frequency domain.

Have a look at those links and get an overview.

EEG

But for specific bio-signals you don't need to re-invent the wheel. There is a long history of methods for EEg signal analysis. For BCI in particular, there are again reviews to help you find good feature extraction and classification methods.

Finally my point is that, MLPclassifier is just a python function. According to my experience as a ML scientist coming originally from Biomedical Data Analysis, the question is more about "what to do?" before "how to do?". I hope it helped. If you had more questions please comment so I can update my answer.

Good Luck!


Applying activation on part of the layer in Keras

YOLO architecture uses the softmax activation function determining the classes of objects in bounding boxes in the output layer. But from the code you have shared, sigmoid is used in the last layer for prediction. It seems that you're trying to implement YOLOv3.
Please look at this paper original paper. This will help to give a better understanding.
Authors of YOLOv3 have refrained from softmaxing the classes since the method rests on the assumption that classes are mutually exclusive. For example, if there are classes like “cat” and “animal” in the dataset and one of the objects in the bounding boxes is a cat, this assumption fails, because a cat is also an animal. Instead, independent logistic classifiers predict each class score and a threshold is used to perform multilabel classification for objects detected in images. An element belonging to a certain class will not be influenced by the decision of that element belonging to another class (binary cross-entropy loss).


Correct output labelling for RNNs/ANNs with sequential data and single label

Recurrent Neural Networks can learn very complex mappings between sequences (unlike Feed Forward Nets which are capable of transforming only fixed-sized vectors). You really only need a RNN which takes the sequence of the form $(a_t, b_t)$ as the input (assuming $t \in \{1...T\}$ you have seq. $a$ and $b$ both with $T$ samples and you have to classify these signals as one of $k$ classes). Then you can run the following procedure (pseudocode) for the sampling (given already trained model):

for t in (1..T)           //loop over an input sequence
    rnn.input[0] = a[t]   //assign t-th point of the blue seq.
    rnn.input[1] = b[t]   //assign t-th point of the green seq.
    rnn.forward()         //make forward pass
rnn.output.normalize()    //renormalize output probabilities
class_prob = rnn.output.max()
class      = rnn.output.argmax()

For example we have the following sequence with length 4:
signal A: {1, 3, 4, 6}
signal B: {4, 3, 0, 1}
Then in the 1st timestep we assign vector $(1, 4)$ to the input layer, then $(3, 3)$ and so on. At each timestep we propagate the input forward the network updating its hidden state which is responsible for remembering past datapoints. After every forward pass our neural network generates a prediction at the output layer which is temporarily uninmportant. This fluctuating prediction reflects the rnn's belief about the part of our dataset- it have seen only $t$ first input vectors. After we propagated entire signals we can do the classification.
Your RNN's output layer should be $k$-dimensional and its $i$-th component is the pseudo-probability of the signal beeing in class $i$. For example you can do argmax (take the index with greatest prediction) or sample with a softmax distribution. For example if you have class $a$ and class $b$ and your output vector is $(5, 15)$ then your signal is classified as $b$. Or if you do softmax then $softmax(5, 15) = (0.000045, 0.999955)$ so you assign the class $b$ with nearly 100% probability.
Training RNN is much more triky. I would suggest you taking some existing toolkit which have RNN such as LSTM already implemented. To train Recurrent NN we can apply the above procedure but at the end (after iterating over the sequence) we calculate Gradient of the loss function and do a backward pass. However this technique is called Backpropagation through time (it is really similar to ordinary FFN backprop, but we must "unfold" out RNN in time to adjust recurrent connections). Also notice that if you choose to have softmax output units your loss function should be a Cross-Entropy loss function.
But what is the expected output vector to compute gradient loss from? If your dataset contains $N$ (green-blue) sequences each of them is in $k_n$ class then their expected output vector is simply one-hot vector $(0...,1,...0)$ where $1$ is at $k_n$-th component only. For example if you have a sequence classified as $b$, then its output vector target is $(0,1)$. @EDIT: Suppose that as in a previous example your sequence have class $b$ so you expect to see the vector $(0,1)$. You take now the prediction of the net (say $(0.34, 0.57)$) and calculate the error function as normal f.e. squared error or cross-entropy.

More information you can learn from karpathy and hinton


How can we detect the existence of outliers using mean and median?

No. Unless you have some other idea of the scale and spread and distribution of the feature values. You can construct data sets that have any given mean and median with no outliers or with one massive outlier.

For example, f2 looks like a well-balanced set of numbers with a very close mean and median, maybe all those values are from a N(0.184,3) distribution. Scale that up by a linear transformation to an N(X,Y) and you'll get a mean and median much like f6. Exercise: find X and Y.


This depends on the data in your minority classes.

The data in each class can be considered as a sample of observations from some population. This sample may or may not represent well the whole population of instances from that class.

If the sample represents the population well, then oversampling will introduce only a small error. However, if the sample does not represent the population well, then oversampling will produce data that have statistical properties different from those of the population.

All estimates (like confidence intervals, prediction intervals) are calculated from the statistical properties of the sample (mean, variance, etc), the exact calculations being different for different distributions and learning algorithms. If statistical properties of your oversampled data are different from the statistical properties of their populations, you will get wrong estimates of the confidence and prediction intervals for your model.

I will illustrate this with an example.

Let's assume that you have 2-dimensional data (two features in each observaton) that belong to 2 classes. Data in each class are normally distributed with the standard deviation for each feature = 1. The population mean of the class 1 is (-2, 0). The population mean of the class 2 is (1, 0).

I illustrate a large population, taking 500 points for each class. The classes can be separated by logistic regression line as follows:

The regression line is almost exactly between the population means and is almost vertical because both ordinates of the population means are zero. If I take a couple of thousand points then it will be vertical.

The code for this picture:

import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LogisticRegression

n_points1 = 500
n_points2 = 500
mu1 = np.array([-2,0])
mu2 = np.array([1,0])

sig = 1

cl1 = sig * np.random.randn(n_points1,2) + mu1
cl2 = sig * np.random.randn(n_points2,2) + mu2

print('Red points:')
print('mu_x = %.3f, mu_y = %.3f, std_x = %.3f, std_y = %.3f'
    % (np.mean(cl2[:,0]), np.mean(cl2[:,1]), np.std(cl2[:,0]), np.std(cl2[:,1])) )

y1 = np.zeros((cl1.shape[0],1))
y2 = np.ones((cl2.shape[0],1))

X = np.vstack((cl1,cl2))
y = np.vstack((y1,y2))

logreg = LogisticRegression()

logreg.fit(X, y.ravel())
w = logreg.coef_[0]
a = -w[0] / w[1]

xx = np.linspace(-4,4)
yy = a * xx - (logreg.intercept_[0]) / w[1]

plt.scatter(cl1[:,0], cl1[:,1], s = 50, marker='.', color='green')
plt.scatter(cl2[:,0], cl2[:,1], s = 50, marker='.', color='red')
plt.plot(xx, yy, 'b-')
plt.xlim((-5, 4))
plt.ylim((-5, 5))
plt.grid()
plt.show()

Now, let's assume that the red class is under-represented (a minority class). I take 500 points for the green class, and 10 points for the red class. Then I oversample the red data by two methods. One is duplicating them 50 times, like bootstrap resampling (I color them red), and another is something like SMOTE (they are magenta). This is not exactly SMOTE. I simply added data that are in the middle between red data points. I was too lazy to calculate nearest neighbors for each observation in this simple example but it illustrates SMOTE nevertheless because in SMOTE, synthetic examples are generated inside the convex hull of existing minority examples which reduces the variance of the data.

This is what I get:

The blue line is for bootstrap, and the black line is for SMOTE. The sample means and standard deviations for the minority class are as follows:

Red points:
mu_x = 1.600, mu_y = -0.182, std_x = 0.719, std_y = 0.753

Magenta points:
mu_x = 1.625, mu_y = -0.174, std_x = 0.513, std_y = 0.550

The x-component of the sample mean is overestimated (it should be = 1) and standard deviations are underestimated (they should be = 1). As a result, the line separating the classes is different from the true line on the previous picture, so a lot of new data will be classified incorrectly.

Let's take a couple of more samples randomly from the same population. Again, the size of the minority class is 10, and I resample them using the same two methods.

Red points:
mu_x = 0.944, mu_y = 0.289, std_x = 0.943, std_y = 0.867

Magenta points:
mu_x = 0.974, mu_y = 0.298, std_x = 0.700, std_y = 0.617

This time x-component of the sample mean is OK, its y-component is a little overestimated, and the standard deviation is underestimated. The separating line is again incorrect.

Red points:
mu_x = 0.922, mu_y = -0.236, std_x = 1.066, std_y = 1.097

Magenta points:
mu_x = 0.924, mu_y = -0.244, std_x = 0.757, std_y = 0.749

In the last picture we are lucky to get a sample that has almost the same mean and standard deviation as the population. Therefore, the separating line is very close to the line in the first picture where the data were balanced. Notice that the standard deviation for SMOTE is always smaller because new data are added between the existing data, not outside of them.

You might consider undersampling instead of oversampling. Check this link.

The code for the last 3 pictures:

import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LogisticRegression

n_points1 = 500
n_points2 = 10
mu1 = np.array([-2,0])
mu2 = np.array([1,0])

sig = 1

cl1 = sig * np.random.randn(n_points1,2) + mu1
cl2 = sig * np.random.randn(n_points2,2) + mu2

cl2 = np.tile(cl2,(n_points1//n_points2,1)) # oversampling

# A kind of SMOTE but not quite
cl3 = np.zeros_like(cl1)
for k in range(len(cl3)):
    i = np.random.randint(low=0, high=n_points2, size=2)
    cl3[k,:] = (cl2[i[0]]+cl2[i[1]])/2

print('Red points:')
print('mu_x = %.3f, mu_y = %.3f, std_x = %.3f, std_y = %.3f'
    % (np.mean(cl2[:,0]), np.mean(cl2[:,1]), np.std(cl2[:,0]), np.std(cl2[:,1])) )

print('Magenta points:')
print('mu_x = %.3f, mu_y = %.3f, std_x = %.3f, std_y = %.3f'
    % (np.mean(cl3[:,0]), np.mean(cl3[:,1]), np.std(cl3[:,0]), np.std(cl3[:,1])) )

y1 = np.zeros((cl1.shape[0],1))
y2 = np.ones((cl2.shape[0],1))

X = np.vstack((cl1,cl2))
X3 = np.vstack((cl1,cl3))

y = np.vstack((y1,y2))

logreg = LogisticRegression()

logreg.fit(X, y.ravel())
w = logreg.coef_[0]
a = -w[0] / w[1]

xx = np.linspace(-4,4)
yy = a * xx - (logreg.intercept_[0]) / w[1]

plt.scatter(cl1[:,0], cl1[:,1], s = 50, marker='.', color='green')
plt.scatter(cl2[:,0], cl2[:,1], s = 50, marker='.', color='red')
plt.plot(xx, yy, 'b-')

logreg.fit(X3, y.ravel())
w = logreg.coef_[0]
a = -w[0] / w[1]
yy = a * xx - (logreg.intercept_[0]) / w[1]

plt.scatter(cl3[:,0], cl3[:,1], s = 10, marker='.', color='magenta')
plt.plot(xx, yy, 'k-')
plt.xlim((-5, 4))
plt.ylim((-5, 5))
plt.grid()
plt.show()


Scan based operations are basically all the operations that require evaluating the predicate on an RDD.

In other terms, each time your create an RDD or a DataFrame in which you need to compute a predicate like performing a filter, map on a case class, per example, or even explain method will be considered as a scan based operation.

To be more clear, let's review the definition of a predicate.

A predicate or a functional predicate is a logical symbol that may be applied to an object term to produce another object term.

Functional predicates are also sometimes called mappings, but that term can have other meanings as well.

Example :

// scan based transformation
rdd.filter(!_.contains("#")) // here the predicate is !_.contains("#")

// another scan based transformation
rdd.filter(myfunc) // myfunc is a boolean function

// a third also trivial scan based transformation followed by a non scan based one.
rdd.map(myfunc2)
   .reduce(myfunc3)

If you want to understand how spark internals work, I suggest that you watch the presentation made by Databricks about the topics


I figured out the issue. The "predict" function expects a batch of input arrays, so it expects x to have shape (n, 36) where n is the number of examples. After adding :

x = x.reshape( (1,36) )

the code works fine


How do i perform feature selection for clustering?

The data described in tables above is for purchases. You, however, want to segment users. Therefore, the first thing is to derive a dataset of user data.
As a first approach, I suggest that you create a table with one row per customer ID, where you have information about how many products the customer has purchased of each of the possible hierarchy values (e.g. customer ID: 312425, tableware products purchased: 5, cookware products purchased: 10, bath scale-manual products purchased: 0), and then do clustering on that.


Linear Regression - finding thetha using Normal equation

Consider the 1-D equivalent of the table you have provided. In this case, you have one input x and one output y (see figure). If you try to fit the data points with y=mx equation, you can not fit the data successfully. You need an equation like y=mx+c to have a good fit. y=mx will be a good fit only if the line goes through the origin.
Now you can look at

y=mx+c

as

y = m * x + c * 1
y = m * X1 + c * X0
y = W1 * X1 + W0 * X0

I hope the figures below will clarify why we need the intercept term X0 and its weight W0=c.


Detecting Offensive Text Content in English and German

Toxic comment classification challenge might be a good place to start.  It contains a set of comments and 6 binary classifications indicting if it's a toxic comment and of which type.

I imagine this would be a sufficient start.


While using reindex method on any dataframe why do original values go missing?

Regarding your code the tols must be cols?

cols = pd.MultiIndex.from_product([['A','B'],['Y','X']
['P','Q']])
tols.set_axis(cols, axis = 1, inplace = False)

This must be:

cols = pd.MultiIndex.from_product([['A','B'],['Y','X']
['P','Q']])
cols.set_axis(cols, axis = 1, inplace = False)

Personally I would have removed nan with a default value before performing the multi-indexing.

df_noNAN = from_product.fillna(value=-1)

In your case it looks like the inplace is producing a bug and better reported to the pandas team. On Stack exchange they are extremely responsive.


Why don't tree ensembles require one-hot-encoding?

The encoding leads to a question of representation and the way that the algorithms cope with the representation.

Let's consider 3 methods of representing n categorial values of a feature:

A single feature with n numeric values.
one hot encoding (n Boolean features, exactly one of them must be on)
Log n Boolean features,representing the n values.

Note that we can represent the same values in the same methods. The one hot encoding is less efficient, requiring n bits instead of log n bits.
More than that, if we are not aware that the n features in the on hot encoding are exclusive, our vc dimension and our hypothesis set are larger.

So, one might wonder why use one hot encoding in the first place?

The problem is that in the single feature representation and the log representation we might use wrong deductions.

In a single feature representation the algorithm might assume order. Usually the encoding is arbitrary and the value 3 is as far for 3 as from 8. However, the algorithm might treat the feature as a numeric feature and come up with rules like "f < 4". Here you might claim that if the algorithm found such a rule, it might be beneficial, even if not intended. While that might be true, small data set, noise and other reason to have a data set that mis represent the underlying distribution might lead to false rules.

Same can happen in logarithmic representation (e.g., having rules like "third bit is on). Here we are likely to get more complex rules, all unintended and sometimes misleading.

So, we should had identical representations, leading to identical results in ideal world. However, in some cases the less efficient representation can lead to worse results while on other cases the badly deduce rules can lead to worse results.

In general, if the values are indeed very distinct in behaviour, the algorithm will probably won't deduce such rule and you will benefit from the more efficient representation. Many times it is hard to analyze it beforehand so what you did, trying both representations, is a good way to choose the proper one.


Is the magnitude of the gradient a weakness of Gradient Descent?

The magnitude doesn't say anything about the path to the optimum, in fact nothing in gradient decent knows anything about the optimum and the algorithm relays on local information only. However if you assume that the space of available solutions is relatively smooth and doesn't contain sharp drops, you can treat the magnitude of the gradient as a score of your confidence in your current surrounding.

So if the gradient magnitude is high, you are fairly certain that you are far from a good solution and you can make big steps and if the magnitude is small, than you are closer to a good solution and should make smaller changes.

The learning rate is just a hyper-parameter that controls how much we are adjusting the weights of our network with respect that said loss gradient. It can also be viewed in terms of your confidence in the surrounding solutions space. It also helps the model to converge deeper into a specific minimum point (as we reduce the learning rate during training).


Python package for machine-learning aided data labelling

It sounds like you are looking for active learning.  In active learning, the classifier learns which samples would be most useful to have labelled by a human.

There are many techniques for active learning, and many ways to adapt an existing (standard) learning algorithm to the active learning setting.  The particular approach you mentioned is called "uncertainty sampling", and can be applied to any standard classifier that outputs confidence/certainty scores.  There are other selection methods as well, which may perform better in some settings.

You can also apply unsupervised methods to cluster the samples, then label one or a few samples from each cluster.


The standard way to evaluate a recommendation engine is by using the RMSE (root mean square error) of the predicted values and the ground truth.

It is almost a SOP that, after finishing developing a recommendation engine, we will evaluate this engine by comparing its RMSE with other famous, common recommendation algorithms like SVD, tranditional CF, even RBM, etc.

Some terms mentioned above do not seem to be related with recommendation, but you can easily find on the internet how these techniques can be used in this topic.


Restrictions on my skewed validation data

I suggest using your whole validation set but providing class-specific metrics. E.g., AUC only considering validation rows with targets in a single class. That will help you see if you're underperforming for a specific class. You can then say "This model earned 0.xxx AUC on the whole validation set and no worse than 0.yyy for any single class."


Python Text Classification - Data that does not fit into any category

The main assumption in supervised learning is that the training set is a representative sample of the space of all possible instances for the problem.
This is why in the regular classification setting there is simply no way to consider a "not in any class" option: what the model learns is to distinguish between the classes seen in the training set, nothing else. In the standard one-vs-rest multi-class setting It's important to understand that the model doesn't predict whether an instance is likely to belong to "class A" in general, it only predicts whether the instance is more likely to belong to class A compared to any other known class. This is a common misunderstanding, but one-vs-rest is not an answer to the problem of "not in any class".
There are options to deal with this problem, but one must design the system so that the model can deal with such cases. In general the problem is actually open-set classification: in regular classification the set of possible classes is closed, as mentioned above. The open-set classification setting is much less standard and usually harder, since the model has to make a prediction about something it does not know from its training data. Here are a few options:

Treat the problem as an outlier detection problem: before running the regular classifier, eliminate abnormal instances which don't look like the regular training set cases.
Multi-label classification: in multi-label classification an instance may belong to zero, one or several classes. A binary classifier is trained for every class, and an instance may belong to none of the classes. This is more flexible than multi-class classification, but it's still closed-set classification: for example the model is supposed to have seen examples in the training set which don't belong to any class.
One-class classification is a type of classification where the model learns to identify a single class "in general", not by contrast to any other class. This is proper open-set classification: if a model is trained for every known class and every model predicts the instance as negative, then the instance doesn't belong to any known class.


What are advantages of oversampling over changing threshold for unbalanced classes?

I would suggest to not use oversampling because of the disadvantages you listed above. Important one is that it increase noise, may contain biased data & may have outliers that demands another pre-processing step. I would suggest go for undersampling the majority class. But for your answer here is the link which will help you further if you are opting for oversampling


Without putting in the time to look through Azure's documentation, my guess is that their PCA method is really just a way to do a feature reduction, then use some algorithm they have to classify. Best thing to do is try both methods and then CV and compare performances. https://gallery.azure.ai/Experiment/1219e87f8fb84e88a2e1b54256808bb3


I would say that option 1 will not work out too well: in my experience, the model will either only be good for the first or last model you train, depending on how much freedom you give the algorithm to change weights as time goes on (e.g. with the learning rate).

You really need to decide what you are going to be predicting. Is it the pollution level for a single city: Which features do you have for each city?

It could can make sense to train all cities at the same time if the features you have are also general ones that really can explain the target variable. So if you have temperature, humidity, some transport statistics for that city etc. then training everything together could make sense.

I would think about each sample leading to one target pollution level, and if that sample has enough information (based on the features) to distinguish itself from samples of the other cities, the model should pick up on and leverage those subtleties in the data.


All the tokens are inferred at once, independently from one another.


Correlation based Feature Selection vs Feature Engineering

What helps the model more, keeping all features or removing correlated ones?

There is some theory about it but in the end Machine Learning is try and error. You should give it a try with all features and then doing a feature selection to see if you are able to improve your model. What works for some models doesn´t necessarily have to work for the rest of the models.

If you want to select which features help your model you could do otherwise, instead of having all and removing, starting with one and adding features and only keeping them in case that it boosts the performance of the model. There are cases when you add a feature and the performance of the model drops.

There are a lot of ways in which we can think of feature selection, but most feature selection methods can be divided into three major buckets. From this source

Filter based: We specify some metric and based on that filter features. An
example of such a metric could be correlation/chi-square.
Wrapper-based: Wrapper methods consider the selection of a set of features as a search problem. Example: Recursive Feature Elimination
Embedded: Embedded methods use algorithms that have built-in feature selection methods. For instance, Lasso and RF have their own feature selection methods.


Can I completely cancel the effects of using a smaller batch size by reducing the learning rate?

Are you aware of this article/tutorial?

https://machinelearningmastery.com/how-to-control-the-speed-and-stability-of-training-neural-networks-with-gradient-descent-batch-size/

I did not study for this answer, but I would guess that when you assure that the batches are randomized, a smaller learning would give the same result.

My guess would be that you divide the learning rate by the square root of the batch size reduction factor.


Tuning C hyper parameter in Soft Margin SVM in Matlab

The easiest way to tune a single hyperparameter is to use what is called the elbow method. Do the following:

Define a range of C you want to try, i.e C = [1.0, 1.5, 2.0, ...]
Loop over all values of C in your range

Train a new model with the current value of C
Evaluate each model on the validation set and store the results

Plot your metric over over your range of C's

If you didn't choose a too narrow range you should visually see your optimal value of C by finding the value that either minimize or maximize your metric.

If you need to tune C alongside other hyperparameters you can use strategies such as grid search, randomized search or bayesian search.


Where can I find dataset for word analogy task?

If i understand correctly you want a long text file of some sort that can easily be analysed. I would suggest that you use project Gutenberg which publishes thousands of free to use books in plain UTF-8 text format. For example here is Pride and Prejudice by Jane Austin.


Combining spatial input with a label as input for CNN using Keras

Here the viewpoint ($v_i$) and corresponding frame ($f_i$) are not concatenated. $v_i$ is simply an index of $f_i$. As mentioned in their section 3.1, $v_i$ is the timestamp and $f_i$ is the actual frame (image). The convolution network $M_\theta$ is applied on $f_i$, not $v_i$.

After the convolution network $M_\theta$, the output of different frames $r_i$ are added together (notice the "+" in the figure, as well as the equation $r=r_1+...+r_m$ in section 3.2.


Is there a fundamental difference from creating a model for each value in a category?

Yes there is.
If a model is trained for each specific value of a variable (a category), then only the subset of data for this category can be used to train and test the model. As a consequence each model has a smaller number of instances to be trained from. Consequences:

In the case of a small category, there might not be enough instances to obtain a reliable model.
Every model is independent. This can be good or bad depending on whether this independence is also true in the data or not, or to what extent:

If the features behave in a completely different way depending on the category, then it's better to create individual models since each can really exploit the specific patterns for this category.
If the features have a very similar behavior across the categories, then independent models by category would potentially lose a lot of information.

In conclusion the choice often depends on:

How much data is available for each category.
How independent are the other features with respect to the category.


What you described is the more-or-less standard approach, with a couple of caveats.

Leakers

Remember that we live in a stochastic world, so a perfect model is immediately suspect of using leakers, i.e., variables which have a causal connection with the target. In your case, e.g., it would be the presence of a service contract for the machine (which would indicated that the company actually does own one). This, building an interpretable model and examining the most powerful variable for such causal connections (and eliminating them) would be a good idea.

Better Approach

However, one could improve on this approach in a very powerful way by replacing the current state of the company with its historical state at the time of the machine purchase. This way one eliminates the leakers automatically and also gets a better model because the predictor values which are more relevant: after all, you are trying to predict which company is ready to buy the machine now rather that which company bought it in the past when its conditions were very different.

In fact, your approach works precisely because it is an approximation of this approach (approximating the company state at the time it bought the machine with its current state) and the absurdity/uselessness of the perfect model that you mention is the artifact of your approach being an approximation of The Right Thing.


I've read all but the last one. If you are serious about computer vision you won't get much out of this reading list. However, if you are set on these books I would start with Goodfellow's Deep Learning which is fairly self-contained and the only book that covers computer vision enough to make it rewarding for that purpose. You can branch out from here and read additional material referenced in the book where you lack background.

If you are less concerned with computer vision you can start with the other books, they are often read in the following order:

An Introduction to Statistical Learning with Applications in R
The Elements of Statistical Learning
Bishop's Pattern Recognition and Machine Learning

If you finished with these you could tackle McKay which is quite dry and boring. Shai Shalev-Shwartz et al. is a nice book that covers a lot of the material in the other books but from the perspective of Empirical Risk Minimization and covers Vapnik–Chervonenkis theory. It is more theoretical and is more structured in the classical theorem-proof-corollary scheme. The perspective taken is less practical but adds to the other books.

I don't know the book on Convex Optimization but you can probably read it in any order. If you want to read all of the books and don't care how long it takes, you can probably read that first as it sounds more foundational.


Is there a thumb-rule for designing neural-networks?

This question has been answered in detail on CrossValidated: How to choose the number of hidden layers and nodes in a feedforward neural network?

However, let me add my own two cents:

There is no magic rule for choosing the best neural network architecture, but if you can find an architecture someone has used to solve a similar problem this is often an excellent starting point.

The best places to look are official or unofficial examples using popular neural network libraries  such as Keras, PyTorch, or Tensorflow, and architectures described in academic literature. keras/examples on github is a great resource.

These architectures were likely chosen after lots of trial and error, so most of the work will have been done for you.


Should your learning rate get smaller as your neural net gets larger?

The learning rate affects how much your model weights are changed with each update. So the question is what is a good learning rate? This depends on your problem space, dataset and network parameters (ie. Activation functions, complexity, regularisation, optimizers, etc).
Too high a learning rate, your model may not reach convergence due to weights updated to “overshoot” the optimum.
Too low a learning rate, your model may also not reach convergence due to taking too many epochs, as the updates to weights are tiny.
It is best to experiment with the defaults ~0.01, and adjust accordingly based on visualizing the validation loss and training loss change over epochs. Also try out different optimizers (Adam, RMSprop, etc) as these adapt your learning rate over time.
Another approach is to implement a learning rate scheduler, that decays your learning rate over time - the idea being as a model nears the global minima, you want to fine tune its descent with smaller weight updates.


Usage of Precision Recall on an unbalanced dataset

When evaluating your algorithms, especially when your dataset is unbalanced, you should use more metrics than just accuracy. The accuracy is how many examples you have correctly identified in total. As you have seen if you have an unbalanced dataset where 0.5% of your instances are 1's then this will result in 99.5% accuracy if you blindly set all your outputs as zeroes. This is obviously wrong albeit the high accuracy. The accuracy is calculated as

$Accuracy = \frac{\sum{TP} + \sum{TN}}{\sum{TP} + \sum{TN} + \sum{FP} + \sum{FN}}$

where TP is true positive, TN is true negatives, FP is false positives and FN is false negatives.

If you want to capture the performance of your unbalanced dataset you should look into the percentage of FP and FN you are calculating. You can do this using the sensitivity and the specificity. Calculate the sensitivity as

$Sensitivity = \frac{\sum{TP} }{\sum{TP}  + \sum{FN} }$

and the specificity as

$Specificity = \frac{\sum{TN} }{\sum{TN}  + \sum{FP} }$.

An ideal classifier should have the accuracy, specificity and sensitivity all be 1. This would mean every sample is correctly classified. In your case where you are getting very high false negatives, you will see that your sensitivity will be very low. This is a measure with which you can state that your algorithm is performing poorly. It is good form to always include these metrics in any statistical study you are doing. Accuracy alone is not sufficient to prove that you are obtaining good results.

Moreover, there is the receiver-operator curve (ROC). This will tell you your false positive rate for any true positive rate. You can then calculate the area under this curve (AUC) to get a comparable metric of performance.

All of these should be used together when exclaiming the performance of your algorithm. The ROC and AUC can be omitted however leaving out the sensitivity and specificity of your algorithm is unwise.


Why Adaboost SAMME needs f to be estimable?

Think for a while, if $f$ is not estimable, it can have any constant added to it with no difference on the result of the process.
This means that if no other constraints are imposed, $f$ is not well/uniquely defined and in fact represents a whole class of functionals. This obviously needs to be fixed and some natural constraint needs to be added (to uniquely fix $f$).
On the other hand, the additional constraint can also be seen as a free parameter of the process which nevertheless needs to be fixed somehow in a concrete instance of the process in order to take place.
In an analogy with physics, the algorithm is "gauge-invariant", but for any concrete physical problem to be solved some "gauge" needs to be chosen and fixed.
They choose to impose the symmetric constraint which reduces to the usual AdaBoost in the 2-class case.
One can impose another constraint on $f$ (than the symmetric one), choosing to satisfy other criteria if so desirable (as long as it can uniquely fix $f$).
For example, the general non-symmetric constraint is also valid:
$$f_1 + \dots + f_K = c$$
for arbitrary constant $c$. This also fixes $f$ uniquely and it can introduce bias favoring certain classes over others (eg for an imbalanced problem). Additionaly, unless $c=0$, does not reduce to the (symmetric) AdaBoost in 2-classes case (which may or may not be desirable).


Clamping Q function to it's theoretical maximum, yes or no?

As MountainCar is often solved with $\gamma = 1$ and a negative reward per timestep, you would immediately hit a problem with your ability to calculate a maximum action value in that case. However, I don't think that is your problem here, the discounted return with positive reward at the end should still encourage desired behaviour for the problem.

It is likely that you are experiencing known problems with RL, and the "deadly triad":

Function approximation (neural network)
on a bootstrap method (Q-Learning or any TD-learning approach)
off policy (learning optimal policy from non-optimal behaviour*, which is a feature of Q-learning)

This combination is often unstable and difficult to train. Your Q value clamping is one way to help stabilise values. Some features of DQN approach are also designed to deal with this issue:

Experience replay. Agent does not learn online, but instead puts each sample (S, A, R, S') into a memory table and trains the neural network on mini-batches sampled from this memory. Commonly this minibatch update is run on every step (after enough experience collected) and might be e.g. size 32. So learning updates happen faster than experience is collected.
Frozen bootstrap target network. The network used to calculate $\operatorname{max}_{a'} Q(s', a')$ when setting the target values to learn (in td target $R + \gamma \operatorname{max}_{a'} Q(s', a')$) is kept stable, and refreshed to use a copy of the current weights after a certain number of steps (e.g. 100 or 1000, or once every 10 episodes).

* Technically off-policy learning is learning the cost function for any policy $\pi$ from a different behaviour policy $b$, but the common case is for control systems attempting to find an optimal policy from an exploratory policy.


The number of folds is nothing you can really calculate. It is more a parameter you choose by good judgement. Typically used values are between 5 and 10, but you can even go up to the so-called leave-one-out cross validation, in which each fold consists of all but one observation. The arguments that guide your decision are typically the following:
- computational cost: the more folds you use, the more computations you need.
- variance: the more folds you use, the higher the variance of the cross validation result
- bias: the more folds you use, the smaller the bias of the cross validation result.

In your case, depending on the exact context in which you want to apply cross validation, you will probably want to choose between 5 and 10 folds.

For more details, you might want to consult Kohavi's 1995 paper on cross validation.