Creating a custom dataset for object detection

I have built many data sets. The latest was a data set of species of birds. I has 100 species of birds so I had 100 classes. For each species (class) I had 100 training images, 5 test images and 5 validation images. Thats as total of 11,000 images.. The classifier I built had a final accuracy of 98% on 500 test images (5 test images per specie)
Here are some things to consider in building your data set.
1- crop the data or not- depending on the images you generated via your cell phone what is on average the ratio of the pixels of the region of interest (example a milk bottle in the refrigerator) to the total pixels in the image. Your classifier will always work better when this ratio is high. You want your classifier to learn to recognize a milk bottle as a class.Any part of the image that is not a milk bottle is essentially noise.  The rule of thumb I use to build a high quality data sets is 50%. That is that on average for your data set the region of interest should occupy 50% of the pixels in the image.
2- how many training- validation-test images should you have? depends on the problem but if you follow the 50% rule above I think you can build a solid data set with about 100 images per class, 5 validation images per class and 5 test images per class so you would need about 110 total images per class.
3- image augmentation - even with the small number of images you can create a good classifier using image augmentation. This is a way  of artificially expanding the size of your data set. If you use Keras, the ImageDataGenerator provides several transforms that can be used to augment your data set. Documentation is at https://keras.io/preprocessing/image/.
4-image size - CNN's operate with a fixed size for all input images. Again in general the larger the image the better the result(assuming the 50% rule is used). However you pay a price for large images in terms of computation time and memory usage. I have consistently used 224 X 224 X 3 (color images). Depending on the importance of color(ie how important is color to discriminating between classes) you may be able to just use gray scale images  224 X 224 X 1.

5- avoiding BIAS in your data set - make it diverse as possible.
   It is easy to accidentally build an unintentioned BIAS in your data set. Here is an example. A guy created a data set with 2 classes. One set had images was of various kinds of dogs. The other set had only images of wolves. He wanted to build a classifier that if given an image would classify it as either a dog or a wolf. He built his data set and trained the classifier with great results 99% training accuracy 98% validation accuracy. That is a surprising result since a lot of dogs very closely resemble a wolf. But when random images of a dog or a wolf were presented the accuracy was essentially 50%. WHy? Well he built a big bias in his data set. 90% of the training and validation images for wolves had a white background of snow. Since he did not crop the images most of the image was white. For the dogs his images were mostly of dogs NOT in snow. So what the CNN learned was "if background is white must be a wolf, if background is not white must be a dog. So think about avoiding building an unintentional bias in your data set. In your case say you have a class ketchup bottle. Now if all you take are images of HEINZ ketchup bottles (probably with the text HEINZ on the label) your model will not know anything about ketchup bottles that aren't Heinz ketchup bottles. Good luck I hope your task works out well.
One more thing, why are you considering doing all the work of taking cell phone images? Why not just do a Google search for the items of interest and download the images. For example do a search for "ketchup bottle images" You will find hundreds of images (with diversity which is important) that you can just download.


Not sure if this is time series data, but it looks like it might be.

For any given narrow window of time there is a distribution that looks like it's centered somewhere a bit higher than y=200 and has a reasonably stable spread. So whatever that distribution is, that's your model.

Then monitor the recent window, and treat that as a sample. Perform a goodness of fit test against the model. If it fails then the points in the window are anomalous.

UPDATE: Based on what you say about the data representing roughly a minute of time, it looks like you could use a window that's about 1 second wide. Run a hypothesis test to see if the data in the window comes from the typical distribution, and if not, just mark all the points in that window as being anomalous. If you need finer resolution than 1s, simply shrink the window.


Backtracking the updater source code, it looks like "extra nodes" are calculated this way:

At each boosting stage, looking at the gradient boost tree,

Extra Nodes = (the total number of nodes) - (the number of start roots) - (the number of deleted nodes)

At each boosting stage, there might be different starting roots (sub trees) and different deleted (so far) nodes. The extra nodes can provide some intuition into how much your processing tree is utilized.

updater_prune.cc

tree_model.h

xgboost "train" api


Logistic regression with unbalanced data, scoring based only on rare class

Firstly, when you have an imbalanced dataset accuracy is not a good metric to be using (see https://en.wikipedia.org/wiki/Precision_and_recall#Imbalanced_data). You should consider what the ultimate use-case of this model is and what metric is properly capturing the performance of the model considering that use case. For example, when classifying the presence of cancer, false negatives are much more undesirable than false positives so you would want to ensure you are using a metric that captures that appropriately.
In sklearn there is a class_weight parameter of the LogisticRegression model which allows you to essentially weigh misclassifications of different classes differently. Setting this to 'balanced' will automatically adjust this weight to be inversely proportional to the amount of samples of that class in your data which might be beneficial. You may want to adjust this in a custom manner also.
Changing the metric you are evaluating on doesn't change the actual training of the model, so I am guessing that your custom implementation of logistic regression should not function significantly differently to the sklearn version in terms of performance (if it does their may be other issues), it seems you are just using a different metric. There are also a number of other metrics besides accuracy that you can use in sklearn (https://scikit-learn.org/stable/modules/model_evaluation.html#model-evaluation) perhaps consider balanced-accuracy to begin with. It is also not to hard to apply your own custom metric to the results from the sklearn logistic regression model.
Tools such as the classification_report (https://scikit-learn.org/stable/modules/generated/sklearn.metrics.classification_report.html) and/or confusion matrix (https://scikit-learn.org/stable/modules/generated/sklearn.metrics.confusion_matrix.html) can also be enlightening when dealing with imbalanced data.


Is my understanding of On-Policy and Off-Policy TD algorithms correct?

1) With an on-policy algorithm we use the current policy (a regression model with weights W, and ε-greedy selection) to generate the next state's Q.

Yes. To avoid confusion, it may be better to use the terms "behaviour policy" for the the policy that controls current actions and "target policy" for the policy being evaluated and/or learned.

  2) With an off-policy algorithm we use a greedy version of the current policy to generate the next state's Q.

Sort of. The only requirement for an algorithm to be off-policy is that the target policy is different to the behaviour policy. The usual target policy in Q-learning is not necessarily a greedy version of the behaviour policy, but is the maximising policy over Q. However, if the behaviour policy is ε-greedy over Q, and adapting to updates in Q, then yes your statement holds.

  3) If an exploration constant ε is set to 0, then the off-policy method becomes on-policy, since Q is derived using the same greedy policy.

This is true when comparing SARSA with Q learning, but may not hold when looking at other algorithms. This greedy-only action selection would not be a very efficient learner in all environments.

  4) However, on-policy method uses one sample to update the policy, and this sample comes from on-line world exploration since we need to know exactly which actions the policy generates in current and next states. While off-policy method may use experience replay of past trajectories (generated by different policies) to use a distribution of inputs and outputs to the policy model.

Experience replay is not directly related to on-policy vs off-policy learning. Technically though, yes when the experience is stored and used later, that makes it off-policy for SARSA if Q values have changed enough between the sample and current parameters of the learning agent.

However, you will see experience replay used more often with off-policy methods, since off-policy learners that boot-strap (i.e. use Q value of next state/action to help estimate current Q value) are less stable when used with function approximators. Experience replay helps to address that problem.


Should reinforcement learning always assume (PO)MDP?

How does reinforcement learning algorithms work without the assumption of (PO)MDPs?

It doesn't. The theory of reinforcement learning is tied very strongly to an underlying MDP framework. The RNN-based solutions that you are referring to are fully compatible with such an MDP model, and don't even require a POMPDP to be useful.

Without the core guarantees of a (PO)MDP model, or something closely equivalent, it is not clear that any learning could occur, with any kind of agent. The MDP model of an environment is about describing consistent behaviour, that is predictable to some degree, and random/stochastic otherwise, and the predictable parts make it amenable to at least some optimisation. The split into states, actions, time steps and rewards help organise the thinking around this. They are not necessary for other kinds of policy-search approaches, such as genetic algorithms. However, if you try to break away from something that would fit to a (PO)MDP, it would break any other kind of meaningful policy too:

If actions had no consequence, then you could learn the value of being in a particular state, but you could not optimise an agent. This could be modelled as a Markov Reward Process, provided state transitions were not completely random, otherwise just learning associations of state to reward using a supervised learning approach would be the best you could do.
If rewards were not consistently based on any data available to the agent, not even history, but not random, then there is no way to learn how to predict or optimise rewards.
Similarly for state transitions, if they bear no relation to any information known about the environment, current state or history, but are not random, then there is no way to learn about the non-randomness, and no kind of agent could generate a meaningful policy to take advantage of knowledge about the system, because the knowledge available is not relevant. However, if the current state still influenced what rewards were available to which actions, then a contextual bandit approach might work (plus a supervised learning approach could predict currently available rewards).

When the information about rewards or state transitions is not directly available, but can be inferred or guessed at least partially from history or context, then you can model this as a POMDP.

One common scenario you can face is that you have available some observations about the environment, but are not sure how to construct a state description that has the Markov property. The velocity of an object might be such a detail, when your observations only give you positions. Technically a POMDP and this observation/state mismatch are the same basic issue, if you decided arbitrarily that your observation was the state.

When faced with this mismatch between easily-available observations and a state description based on history that would be more useful, you can either try to engineer useful features, or you can turn to learning models that will infer them. That is where using RNNs can come in useful as part of RL, and they can help with both observation to state mapping and also inferring more complex hidden state variables in POMDPs. Use of hidden markov models to model a "belief state" that augments the observed state is similar to the latter use of RNNs.


How to pass linear regression weights to Xgboost regressor?

Answer is NO
Why? weights are hyperparameters of the linear regression model and they are not the same as the ones for xgboost or catboost.
What you can do is combine models (if you really want to use xgboost or catboost additionally) SEE this
NOTE just because model is more powerfull it does not mean that it will beat linear regression on every dataset ;)


Can PCA be applied to reduce dimensionality of only a subset of features?

Yes, absolutely. Simply split your data into two sets feature-wise, apply PCA to one of them, and then stick them back together again. How to actually perform this will vary depending on your programming language/frameworks, but it is trivially easy in python + pandas, for example.


How to get sensitivity and specificity for multi-class classification for each fold of cross validation?

I think this error comes from confusion_matrix(), here we have three "types_of_target": multiclass, multilabel-indicator, continuous-multioutput.

For example, np.array([1, 0, 2]) is multiclass, it's one-hot-encoding np.array([[0,1,0],[1,0,0],[0,0,1]]) is multilabel-indicator, what we predict np.array([0.3,0.4,0.3],[0.7,0.2,0.1],[0.1,0.1,0.8]) is continuous-multioutput.

The input of confusion_matrix must be of type "multiclass".

I think you can try

confusion_matrix(y_true.argmax(axis=1),np.rint(y_pred).argmax(axis=1))

by converting y_true from multilabel-indicator to multiclass, and y_pred from probs(continuous-multioutput) to one-hot(multilabel-indicator) then multiclass.


How do the policy gradient's cost function and gradients work?

In policy gradients we are interested in maximizing the expected reward. For this we assume that the expected reward is parametrized by parameters $\theta$ (e.g. a Neural Network. This means that in order to maximize the expected reward we need to find these parameters. In math notation:
$$
\theta^{\star}=\arg \max _{\theta} J(\theta) =\arg \max _{\theta} E_{\tau \sim p_{\theta}(\tau)}\left[\sum_{t} r\left(\mathbf{s}_{t}, \mathbf{a}_{t}\right)\right]
$$

where $\tau$ is a trajectory sampled from policy $p_\theta$. To solve this we need gradient ascent so our parameters are updated: $\theta = \theta + \alpha\nabla J(\theta)$. Thus, if we compute the gradient of the expected reward we will get the proper update of our parameters towards greater expected rewards. You can take a look at Likelihood Ration and REINFORCE which explains analytically the whole optimization process.

To solve this you need the Policy Gradient theorem which will lead you to the equation you have. In other words, by using the gradient form you mentioned we are updating our parameters towards higher expected rewards. Also, PGs are more close to classification (cost-sensitive) rather than regression.

In terms of Neural Networks (and assuming stochastic policies), typically, your inputs are going to be states and your outputs action distribution and/or expected reward, thus mapping states to action probabilities (and/or reward predictions). The additional reward prediction it has been shown that leads to better results than using only the policy loss, as it drives the NN's representations to also predict expected rewards (instead of only action distributions).


I would doubt there's a single correct answer for best available architecture, but the current best results come from this paper by NVIDIA and this technical report by Adobe. The latter paper is the second iteration of the repo you linked. Both papers focus on non-rectangular inpainting, which seems to be your targeted interest. Neither has publicly available code, however, so if you're interested in seeing what those models can accomplish you'll have to re-implement them yourself!

Edit: NVIDIA released a PyTorch implementation since the initial answer, here's the repo.


Why would a validation set wear out slower than a test set?

It is difficult to say without access to the original author. However, I expect this refers to the ability of using each set to realise its purpose.

A validation set's purpose is to select hyperparameters that perform the best according to some metric. The best measurement on the validation set should always have the highest expectation of being the best in reality. If you make very many measurements, then the absolute probability of the best measurement being the real best could be low, but the chances of a generally poorly performing set of hyperparameters winning overall do not increase as fast. You can be reasonably certain that you have picked "one of the best" plus "the one with highest probability of being the best" even though that might be e.g. just a 10% chance if you have run 100s of validations.

A test set's purpose is to measure a metric without bias. If you use this for model comparison or selection, then this can be affected by maximisation bias - because there is uncertainty in the measurement, focusing on the relative values and picking a "best" almost certainly over-estimates the true value. This effect happens very quickly. If you measure metrics for two sets of hyperparameters and pick the best one, you should already expect that the value you got for the metric is an over-estimate. Note you still expect on average that you have picked the better option, but you cannot trust the measurement as much.


Discarding rare words when comparing texts - per text, per comparison, or per codex?

Tf-idf (text frequency - Inverness document frequency) will highlight your interests.

https://en.m.wikipedia.org/wiki/Tf–idf


How do data types influence hardware (CPU / GPU / TPU) performance?

The variation in performance gains you are seeing from reduction precision might be do to different frameworks using different types. Even after downcasting data types, some operations will automatically upcast types. You mention using Pandas and TensorFlow / Keras. Mixing these frameworks leads to unwanted recasting of data types. It is better to use a single framework to avoid recasting.
There is no software (i.e., language, library, or model) agnostic answer. Hardware utilization is software specific. In order to get the reduced precision gains, the same software stack should be throughout the entire modeling process. That software should be designed for the specific chip (i.e., CPU, GPU, or TPU).


Forecasting using Boosting methods on Non-stationary Time Series data

As @10xAI said, a tree-based gradient boosted approach may miss the mark for time series because it cannot forecast a growing trend.  However, we can apply gradient boosting methodology to any algorithm.  You can mess around with some code I wrote that is based on gradient boosting and decomposition: LazyProphet.  The code is badly written and I think the example data pulls break now but the method itself tends to produce some decent results.
Essentially if we do boosting with some piecewise approach we can get new changepoints at each boosting round and update our seasonality + exogenous measures.  I use binary segmentation so it ends up being very similar to a wild binary segmentation approach for change points.  Round 0 the 'trend' is just the mean/median, then you measure seasonality and set your 'y' to the original time series - (trend + seasonality).  Round 1 then finds the optimal point which splits the data into 2 and fits a trend estimator (could be the mean kind of like a tree output) then measures seasonality and adds these measures to what was found in round 0 to find the new residuals to fit for the next round and so on.  Hopefully this image makes it clearer:
I do have a much better written and more generalized approach coming soon!


To see if SVM can capture any signal at all, try to balance your data: create training and test sets that consist of exactly 50% positive and 50% negative samples (i.e., by subsampling randomly from whichever one is bigger). Also standardize the data (subtract the mean and divide by standard deviation).

(For balancing, you might try changing the class_weight parameter in sklearn, but we found the manual method (subsampling) to work better.)


Why not give them a name describing their purpose?

df_csv = read_csv('customer_data_export.csv')
df_date_filtered = df.query("date > '2017-01-10'")
df_grouped_by_trans_id = df_date_filtered.groupby('transaction_id').sum()

#cleanup
rm(df_csv, df_date_filtered)

plot_data = pivot_table(df_grouped_by_trans_id, columns='weekday', rows='n_items')


What morphological operations can be used to vectorise road surface binary rasters?

Perhaps another solution to your question: There are datasets that contain road surfaces, such as the (Dutch) BGT-database.


The purpose of smoothing is to prevent a language model from assigning zero probability to unseen events.

That is needed because in some cases, words can appear in the same context, but they didn't in your train set. Smoothing is a quite rough trick to make your model more generalizable and realistic. You can also see it as a tool to prevent overfitting.


You could simply compute the distance to each cluster center provided in the cluster_centers_ attribute (once the KMeans instance is fitted). The predict method actually does that for the closest cluster center.


When in doubt simply build a model and test how good it is (accuracy, MSE, whatever).

If those KPIs are not up to par, think about improving the parameters, feature engineering, etc.

I find this approach to be more valuable than to spend a lot of time analyzing correlations and other classical statistical analysis.

If simple models (Regression, GLM, RandomForest, XGB) do not produce results with any sensible accuracy and you see no path to successful feature engineering then you have your answer as well.

The only downside to this approach might be a perfectly good model on the first try ;)


Have a look at fast.ai (https://www.fast.ai), only heard the best about it. Supposedly,  it's completely free and easy to get into.

Other often mentioned resources are the course by Andrew Ng (https://www.coursera.org/specializations/deep-learning and/or https://www.coursera.org/learn/machine-learning) or the (also free) book by Ian Goodfellow (https://www.deeplearningbook.org).


As the model is not trained to recognize an image from this new specific class, the only thing it will do, is to give a probability-or similarity measure  for each of the classes for which the model has been trained on. Hence in a Classification problem, the class with the highest probability for this testing image will be the classification output.


How can precision be less than one in Leave-One-Subject-Out binary classification if each subject contains only one class

Your reasoning is correct that the gold standard class is the same for all the instances in a single leave-one-out test set (under the assumption that the test patient cannot become sick at some point in time, thus having both healthy and sick status).
What you're missing is the aggregation across multiple test sets: a full leave-one-out experiment repeats this process for every single patient, i.e. if there are N patients then there are N unique pairs of (training set, test set). Here is a pseudo code to show this clearly:
correct,incorrect=0
for every patient p:
    training_set = all the patients except p
    test set = patient p
    model = train(training_set)
    // to simplify I count the patient as one instance, it's easy to count instances instead
    if model.predict(test_set) == gold_standard(p):
       correct += 1
    else:
       incorrect += 1
accuracy = correct / (correct + incorrect)

The calculation across all the patients can lead to some predictions being correct and some others being incorrect, this is why the accuracy, precision or recall can be any value between 0 and 1.


How to evaluate RMSE with standard deviation

The residuals of your model seem to vary by a large extent (as denoted by RMSE, which is in the same units as the target). Without any context on the problem you are solving, it would be hard to decide whether the model is good or not. But, it does appear to be on the poorer side.


How can I use a multivariable list of strings as a single X variable / feature in a Neural Network Model

For your problem you can easily use PCA to remove the redundant features and only keep the viable information. Since you are using neural networks, you anyways wont have any interpretation layer, so PCA will not be a problem.


How to extract and classify data from a column in excel?

You need to perform a few preprocessing steps.

Convert your excel file into some sort of text file (csv is possibly the easiest)
Manipulate the file with python, either directly reading the file or with libraries such as the csv python module or pandas.

Last word of advice: Regular expressions are wonderful, but I think you might be using them for the wrong task, I strongly recommend you to take a programmatic approach.


Does this line in Python indicate that KNN is weighted?

Yes, the line indicates that KNN is weighted and that the weight is the inverse of the distance. All of this can easily be found in scikit-learn's documentation

Also, pro-tip, you can find an object's documentation using the help function. In this case:

from sklearn.neighbors import KNeighborsClassifier

print(help(KNeighborsClassifier))

As to how weighted KNN works, I actually made a learning unit on my website about this.

In short, weighting the instances means that instead of each neighbor having an equal vote, their vote is weighted. This means that if for instance, in my 3 nearest-neighbors, 2 of them are of class A but have low weights, and 1 of them is of class B and has a high weight, KNN might still pick class B even though more neighbors are of class A.


Are there any tools/ python packages for Fuzzy Grouping?

Probably there are more direct solutions, but you can try using some sort of fuzzy matching to get the most similar words to each word. That returns you an unweighted graph where the nodes are words and the edge indicates whether the world is similar. then, you can try clustering the resulting graph, for example with the Louvain's algorithm. Than would return non overlapping groups of related words.


In R, it's an option of the cross validation function : xgb.cv
See the documentation here : https://www.rdocumentation.org/packages/xgboost/versions/0.4-4/topics/xgb.cv


How to calculate the distance between two locations using Haversine Formula?

I am not an expert in geometrics but I found that this function has already been supported by sklearn. So I decided to help you define a generic function.
from sklearn.metrics.pairwise import haversine_distances
from math import radians
import pandas as pd

def distance(location1, lat, lon):
  location1_radian = [radians(_) for _ in location1]
  location2 = [lat, lon]
  location2_radian = [radians(_) for _ in location2]
  result = haversine_distances([location1_radian, location2_radian])
  result = result * 6371000/1000  # multiply by Earth radius to get kilometers

  return result[0][1]

df = pd.DataFrame({
    'Latitude': [-0.123684, -0.129212, -0.123234],
    'Longitude': [51.485020, 51.507426, 52.476264]
})

df.apply(lambda x: distance([0.1245, 51.685], x.Latitude, x.Longitude),axis=1)
# 0    35.440884
# 1    34.434944
# 2    92.195943
# dtype: float64


Ranked tag recommendation for university courses

This will most likely turn out to be multiple binary classification problems.

Instead of building just one classifier, you would have to build as many classifiers as there are tags. Each classifier's task would be to predict whether a certain tag exists or not. A tag specific probability threshold can then be learnt by tuning over the precision-curve where course wise precision recall can be calculated.

The percentile of the predicted probability for each tag can be used to obtain a ordered (sorted) list of tag suggestions.

This way of converting a multi-class problem to multiple binary classification problems is known as binary relevance. While it is very efficient, it assumes that the labels themselves are independent. However, picking the probability threshold for each tag in order to optimize the course wise precision recall curve can lead to modeling dependence in the tags.


Because y_true and y_pred have shapes (N,). Where N is the number of examples. By setting the axis to -1 you are calculating the sum over the last dimension, which happens to be over N (which is the batch)


Reason for generally using RMSE instead of MSE in Linear Regression

However RMSE seems similar to MSE and is the root of it, gradient of RMSE with respect to $i^{th}$ prediction differs from that of MSE.
$$\frac{\sigma{RMSE}}{\sigma{y_i}} =\frac{1}{2}\frac{1}{\sqrt{MSE}}\frac{\sigma MSE}{\sigma y_i}$$
Gradient of RMSE is equal to the gradient of MSE multiplied by this $\frac{1}{2}\frac{1}{\sqrt{MSE}}$ value which is constant and is called learning rate. And it shows that RMSE and MSE cannot be interchangeably used when using gradient based methods like when it comes to use linear regression with gradient descent optimization algorithm.
Further, in cases when it is better to give more weight (higher error) to large errors, RMSE can be a better error measure. I believe it depends on your data distribution which one of them to pick to use. But in linear regression, maybe not caring about the points which are considerably off the actual prediction line leads to being biased to considering these points and having a bad model. So, it might be better to give a higher error value to these points.
This and this might also help.


Predicting next action to take to reach a final state

The problem you've described can be formalized as a Markov decision process, the foundational problem of reinforcement learning. In broad strokes, reinforcement learning is concerned with how agents (robot) in a given environment (room) out to take actions (movements from one state to another) to maximize some notion of reward.

Formalizing your problem requires defining a few parts of an MDP model:

Set of states $S$
Set of actions $A$
A reward function $R(s)$, defining the reward of arriving in a given state. In your case, a simple scheme of $R(s) = \mathbb{1}(s = s_{goal})$ is one option.
A transition function $T(s,a,s')$ giving the probability of winding up in state $s'$ having taken action $a$ from state $s$.(Note that you can model deterministic transitions by returning a value of $1$ for a single $s'$.)

If the problem goes on infinitely, you'll also need a discount factor $\gamma$.

In reinforcement learning, the term optimal policy describes a function that returns the best action to take from a given state. I.e., this gives the recommendation you're looking for.

If you know all of the model components described above—or can at least derive ones that match your problem—you can use a variety of planning algorithms to find the optimal policy, e.g. value iteration or policy iteration.

If you don't know the rewards and transitions—say, for example, that the robot is seeking a sensor attached to a charging station and you don't know where it is or the size of the enclosing room—you'll likely need to explore algorithms that observe action-outcome pairs and try to learn optimal policy from these learning episodes.

A full description of these is beyond the scope of your question, but a good place to start is Sutton & Barto's Reinforcement Learning: An Introduction. An html version is freely available. Another resource is RL udacity course produced by Georgia Tech.

In your example, you may also want to research potential-based reward functions. Very loosely, one potential might be the robot's distance to goal state, and the reward would be based on changes to this potential value. (This is described in a paper by Ng, Harada and Russell, and in unit 6 of the GA Tech course mentioned above.)


One option is to call them synonyms, a word or phrase that means exactly or nearly the same as another word or phrase.
Entity linking is mapping words of interest to corresponding unique entities in a knowledge base. It is useful to think of the unique entities in knowledge base as hashes and all surface forms as synonyms.


classification performance metric for high risk medical decisions

Check ROC curve, increase the threshold and measure PPV.
Also you can not use only one in isolation, you have to check sensitivity, specificity and PPV in order to understand the complete scenario.


Importance of feature selection for boosting methods

There is a difference between boosting and features selection.
It is very important to understand that the original boosting algorithm or bagging algorithm have been modified and augmented with many features selection and/or data sampling ( over/ down/ synthetic) to improve the accuracy.
Let us talk about the difference between bagging and boosting :
Booth of them are random subspace based algorithms, the difference is in bagging we used uniform distribution and all the samples have the same weight , in boosting we use non- uniform distribution, during the training the distribution will be modified and difficult samples will have higher probability. The second difference is the voting. In bagging it is average voting , in boosting it is an weighted voting.

Features selection algorithms try to find the best set of features that can separate the classes. But there is no explicit consideration for difficult or easy samples and what is the used training algorithm.
In boosting, the algorithm selects the feature that minimize the error , the error is the sum of prob "weights" of samples that miss classified, since the difficult samples have higher weights , the selected feature will be the one that better distinguish between the difficult samples.

FE ( Features, data) --> feature set
Boosting ( features, data, base learners type, initial distribution, difficult samples) --> feature set


Where to find height dataset, or datasets in General

The Galton Height dataset seems to be exactly what you're looking for.

There are a ton of repositories of open datasets online.  For beginners, I often see the UCI Machine Learning Repository.  It has quite a few datasets which are easy to work with.

Some other resources for datasets:

Kaggle - hosts data science competitions, but you can download all the datasets without participating in a competition.
Data.gov - You can freely download data from US govt agencies
AWS Open Data - You need a AWS account, but these datasets are free-to-download


How to deal with missing data for only some categories

There are three types of missing data: Missing Completely At Random (MCAR), Missing At Random (MAR) and Missing Not At Random (MNAR).

Your case is the second, where according to wikipedia it:

  occurs when the missingness is not random, but where missingness can be fully accounted for by variables where there is complete information

This means that the presence or not of entries in zone can be derived from the column Produit.

Because the values aren't missing completely at random, normal imputation techniques (e.g. fill with the most common value) shouldn't be applied. Instead I'd recommend treating the missing values as their own category. Just create a category (let's say not available) and fill the missing with this value. From a data science view this makes more sense.


Keras Sequential model returns loss 'nan'

To sum up the different solutions from both stackOverflow and github, which would depend of course on your particular situation:

Add regularization to add l1 or l2 penalties to the weights. Otherwise, try a smaller l2 reg. i.e l2(0.001), or remove it if already exists.
Try a smaller Dropout rate.
Clip the gradients to prevent their explosion. For instance in Keras you could use clipnorm=1. or clipvalue=1. as parameters for your optimizer.
Check validity of inputs (no NaNs or sometimes 0s). i.e df.isnull().any()
Replace optimizer with Adam which is easier to handle. Sometimes also replacing sgd with rmsprop would help.
Use RMSProp with heavy regularization to prevent gradient explosion.
Try normalizing your data, or inspect your normalization process for any bad values introduced.
Verify that you are using the right activation function (e.g. using a softmax instead of sigmoid for multiple class classification).
Try to increase the batch size (e.g. 32 to 64 or 128) to increase the stability of your optimization.
Check the size of your last batch which may be different from the batch size.


How to find a changing relation between continuous variables?

Besides doing initial EDA and plotting dependent vs independent variables, it would seem prudent to try to first identify the "maintenance" points. According to your description of the problem, we can expect to see a decline in output o punctuated by single spike-step indicating improved performance. In other words you should in general see a negative slope interspersed with very high positive slope. We see a similar type of spike behavior in EEG signal, where the "blips" or "spikes" correspond to unwanted artifacts (e.g., eye blinks). These can be identified relatively easily by running a sliding window over the data and looking for big changes in slope. The time segments marked in green below are identified as artifacts by monitoring for changes in slope. You can look into EEGlab and ERPlab -- which are Matlab based -- to find out more about this approach.

For feature engineering, because of the input-to-response lag, you will want to initially include variable lags. For example, for variable a_left and a corresponding change in output o, you would want to add as features a_left_1, a_left_2, ... a_left_n, where each feature refers to the input value at some lagged time prior to the observed change in behavior.


Agent Collapse / Overfitting during Training

There are lots of things that might lead to this performance. With the very limited information provided I can only guess and suggest the reasons.
I have personally used lots of Pacman variations (including this one) with RL models successfully. My first initial guess is that you are stopping exploration too early. Pacman environments are very hard to solve (up to the last level). You can get a reasonable performance though within a couple of days training (depending on your system). I highly doubt that the agent underfits/overfits as it will take time for the agent to reach  a good score. Also underfitting/overfitting would be better observable when you test your agent (you do not update weights). I would let exploration to become minimum once the agent reaches the last episode. Then I would test the current state of the agent with very little or none amount of exploration (full exploitation).


Automatically Detect Valid/Interesting pivot/drill downs of a dataset

The automated generation of pivot tables is a well-studied topic. In fact, Google has a patent on the technology.

The general approach is:

Identify low-cardinality, categorical data features that have many repetitions. Those features become the row and column candidates for the pivot table.
Identify numerical data that are the candidates for the data (or fact) cells in the pivot table.
Select an aggregation function (e.g., count, sum, average, …).
Select an evaluation method. The most common evaluation methods are human rating or look-a-like modeling with historical data.

Generating useful pivot tables typically requires domain knowledge.


It is perfectly fine. When you encode you want to extract the most information possible. You can apply one encoding to each feature, or even two.

Also you can encode numerical variables or bin them and then encoding.

For different encoding techniques, I recommend Category Encoders.

Be aware that some encodings can make your model decrease its performance, so you should select features somehow, not by adding more encodings your model will perform better.


Isolation forest sklearn contamination param

I think the answer is "it depends". The contamination parameter simply controls the threshold for the decision function when a scored data point should be considered an outlier. It has no impact on the model itself.

Depending on your application it could make sense to use some statistical analysis to get a rough estimate of the contamination. (If you could narrow it down more specifically then you wouldn't need the isolation forest...)

Usually, I would say, this is informed by some business expectation. Something like "we expect to see a similar rate to our competitors who have reported a rate of x".

Or you expect a certain number of outliers in your dataset. Then you can use the raw scores to find a threshold that gives you that number and set the contamination parameter retrospectively when applying the model to new data.


Regression model performance with noisy dependent variable

Adding white noise is simple enough and should work, alternatively you could permute the values of your variable, which is another commonly used method.


What is the general approach I can use to predict whether a domain is related to a brand using a supervised learning algorithm?

It will be almost impossible to do this your way as you arw trying to derive context based solely on the domain names, and domain names more than often don't carry a specific context. Since the word ups occurs in each domain and the words coming before or after hardly carry much specific information. If you could extract some information about the website by hitting the url, you will not need any machine learning at all. I am not saying this can't be done through ML, all I am saying it won't be that effective. If you use scikit learn you will need to find a way to tokenize the domain, in your case a character level tokenization will be useful, throw in a BOW (Bag of words, characters in your case) create one-hot vectors out of it and use them as features to predict the label. If you have a lot and I mean lots you can train a character level convolutional neural network and it might perform better. Cheers.


How to visualize image segmentation results

If you are saying your model produces a vector of length N for each pixel (holding the probability of each class for that pixel), then you are doing the right thing to select the maximum value, of course.

For the plotting, if you leave the image in 128x128, your output will simply be gray-scale and if you are plotting the raw maximum values you took from each vector, I imagine the image will be very noisy.

You need to map the class of each pixel to a predefined colour mapping. For example, if you have the classes: tree, car, building, sky, you should decide on a colour for each of those. For example, specify a dictionary mapping each class to a colour:

colour_mappings = {
    'tree': (255, 255, 255)   # white
    'car': (255, 0, 0)        # red
    'building': (0, 255, 0)   # green
    'sky': (0, 0, 255)        # blue
    }

No when you have the output matrix of size 128x128, you can use filtering to replace each of your classified pixels to the colour you put in the mapping above.

Example

I use the colour mapping above along with a simulated matrix of pixel classification results (for simplicity just 5x5, not 128x128), which is randomly filled with values from [0, 1, 2, 3]:

classes = array([[0, 3, 3, 1, 0],
                 [2, 2, 0, 3, 2],
                 [3, 1, 0, 0, 1],
                 [1, 3, 2, 2, 2],
                 [1, 3, 1, 2, 1]])

Now create an array with three channels, which we will fill based on our results. I just initialise it with zeros, which would be black in colour, which is ok because non of my classes in the mapping above are black (just to avoid confusion):

# initialise an empty (black) matrix to hold the image
segmented_image = np.ones((5, 5, 3))          # we require three channel dims for RGB colour

Let's say the classes above (in the same order) have the numbers 0, 1, 2, 3 - you could then fill the empty image matrix with the colours as follows.

# fill the segmented image based on the class, using your mappings
segmented_image[classes == 0] = colour_mappings['tree']

This will produce the following image, where we see all pixels from the class 'tree' now show the mapped colour, white:

Now we can do the same for the remaining classes:

segmented_image[classes == 1] = colour_mappings['car']
segmented_image[classes == 2] = colour_mappings['building']
segmented_image[classes == 3] = colour_mappings['sky']

This produces the following final image:

Which can be checked against the classes matrix shown above.


How to find what values are assigned to labels that where encoded using LabelEncoder?

Use the classes_ attribute of your LabelEncoder. For example:

le = preprocessing.LabelEncoder()
le.fit(places)
print(le.classes_)

The index of the label in le.classes_ is the encoded value of the label. See another example here.


In generative adversarial models (GANs), why should we solve min-max problem and not max-min?

You can use either a min-max or a max-min game, it doesn't matter. Look, this is the "usual" loss function (i. e. if you use the original loss proposed by Goodfellow et al. in their paper on GANs, though I use a slightly different notation):
$$\min_{G}\max_{D}\mathcal L_{\text{GAN}}\left(D, G \right) = \mathbb E_{x \sim p_{\text{data}\left( x\right) }}\left[ \log \left\{ D\left( x \right) \right\} \right] + \mathbb E_{z\sim p_{z}\left( z\right)}\left[ \log\left\{ 1 - D\left( G\left( z\right) \right) \right\}\right].$$
Now, you could also write this as:
$$\max_{G}\min_{D}\mathcal L_{\text{GAN}}\left(D, G \right) = \mathbb E_{x \sim p_{\text{data}\left( x\right) }}\left[ \log \left\{ 1 - D\left( x \right) \right\} \right] + \mathbb E_{z\sim p_{z}\left( z\right)}\left[ \log\left\{ D\left( G\left( z\right) \right) \right\}\right].$$
I hope that clarifies your doubt.


predicting number of customers based on long/lat coordinates

The answer relies on the question: are sales related to location?

There might also be some linear relation between location and sales in the sense that "the further you go away from city centre, the less books people read", which is definitely worth checking by simply running a correlation analysis between longitude,latitude and sales and see it in a matter of seconds using either Matlab or Python (Pandas).

Another thing that you can do, since you have two inputs, you can plot the sales for each location (longitute,latitude) on a 2D colormap (heatmap), where color intensity shows number of sales. Let us assume that light and dark colors correspond to low and high sales respectively. If you see that similar color intensity is gathered in same regions of the map, it means that location actually affects the sales. On the other hand, if you see randomly colored points in a small area, this means that sales are irrelevant to location.


Relation between an underlying function and the underlying probability distribition function of data

The function you are thinking about (both statement 1 and statement 2) is the same function.  Allow me to use your framework to describe it.

The domain of the neural network is the range of $X_1$ and there exists some distribution function $F_{X1}(x)$ that describes how the values of $X_1$ are generated.  The same is true for $X_2$ and $F_{X2}(x)$.  Then there exists $f$ which maps:

$$
f: \mathbf{range}(X_1) \rightarrow \mathbf{range}(X_2)
$$

Note that this is not $\dot{f}: X_1 \rightarrow X_2$, (not sure if we can this easily define a function whose domain and range are spaces with different densities, an interesting idea to explore but off-topic for us here).

Now.  Here I'll diverge from your framework.  We will assume that we in fact constructed and trained a neural network which estimates (with good confidence) $f: \mathbf{range}(X_1) \rightarrow \mathbf{range}(X_2)$.  What we can say about this neural network and about $f$, is:

$f$ can be used to predict, given a value from the range of $X_1$ a corresponding value in the range of $X_2$.  This is by our understanding that our model is a good predictor.
The model models $F_{X2}(x)$.  In other words $f$ is $F_{X2}(x)$; and it models the distribution of $P(X_2)$.

Why is that?

In order for $f$ to be a model of $F_{X2}(x)$ then we must have:

$$
f: \mathbf{domain}(X_2) \rightarrow \mathbf{range}(X_2) \Longleftrightarrow
f: \mathbf{range}(X_1) \rightarrow \mathbf{range}(X_2)
$$

Which is true by the way how we designed our neural network.  We said that the network receives the range of $X_1$ as its input, which is the domain of the neural network.  We also said that the output of the network is the range of $X_2$, therefore the network itself if the model of $P(X_2)$.

Common Sense Reasoning

The neural network receives values taken from $F_{X1}(x)$ as the input but compares its output against values in $F_{X2}(x)$, and it can only change (train) itself against values it can compare something to (its output).  In other words, the network does not attempt to understand the distribution of its input.  If we have the input as just a single sample from $F_{X1}(x)$ we should still be able to evaluate $f$ correctly.

Reality may sometimes seem different though, and the distribution of inputs actually may affect the training of a neural network in practice.  For example, classes/regions with very little support would impact the score of the network (and consequently its estimate of $P(X_2)$.  Yet, the idea of little support is not relevant for us since we are thinking of $X_1$, which is a random variable and its expected value would be from a big enough number of samples.

Extra Note

The constrain that $X_2$ must be from the same sample space ($\Omega$) as $X_1$ is not needed, although possible.  This constrain would affect how the network is trained (it is a constrain after all), yet it will be the same constrain on how $P(X_2)$ looks.


What are the most concrete and easiest to understand applications of deep learning in the industry?

This seems like a matter of opinion to me, but here are some applications of machine learning that you seen very often in Silicon Valley:

Advertising: what to show whom and when? The data comes fast and furious so the engineering part is quite sophisticated.
Recommendation: similar to the advertising problem, except there are no advertisers, and the volume is slower, so you can afford to use more sophisticated machine algorithms.
Security: did somebody outside penetrate your system? did somebody inside abuse your system?
Text/audio/video processing: information retrieval on media. Requires finding good representations of the items using machine learning; e.g., fetch all pictures similar to the input, where the similarity can be construed in various ways, such as similarity of texture, style, color, content, etc.

These are not necessarily simple, but they're representative. If you want to give a demo, I suggest the last one. You could show them how to find similar images.


How to plot logistic regression decision boundary?

Your decision boundary is a surface in 3D as your points are in 2D.

With Wolfram Language

Create the data sets.

mqtrue = 5;
cqtrue = 30;
With[{x = Subdivide[0, 3, 50]},
  dat1 = Transpose@{x, mqtrue x + 5 RandomReal[1, Length@x]};
  ];
With[{x = Subdivide[7, 10, 50]},
  dat2 = Transpose@{x, mqtrue x + cqtrue + 5 RandomReal[1, Length@x]};
  ];

View in 2D (ListPlot) and the 3D (ListPointPlot3D) regression space.

ListPlot[{dat1, dat2}, PlotMarkers -> "OpenMarkers", PlotTheme -> "Detailed"]

I Append the response variable to the data.

datPlot =
 ListPointPlot3D[
  MapThread[Append, {#, Boole@Thread[#[[All, 2]] > 40]}] & /@ {dat1, dat2}
  ]

Perform a Logistic regression (LogitModelFit). You could use GeneralizedLinearModelFit with ExponentialFamily set to "Binomial" as well.

With[{dat = Join[dat1, dat2]},
 model =
  LogitModelFit[
   MapThread[Append, {dat, Boole@Thread[dat[[All, 2]] > 40]}],
   {x, y}, {x, y}]
 ]

From the FittedModel "Properties" we need "Function".

model["Properties"]

  {AdjustedLikelihoodRatioIndex,    DevianceTableDeviances, ParameterConfidenceIntervalTableEntries,
  AIC,    DevianceTableEntries,   ParameterConfidenceRegion,
  AnscombeResiduals,  DevianceTableResidualDegreesOfFreedom,  ParameterErrors,
  BasisFunctions, DevianceTableResidualDeviances, ParameterPValues,
  BestFit,    EfronPseudoRSquared,    ParameterTable,
  BestFitParameters,  EstimatedDispersion,    ParameterTableEntries,
  BIC,    FitResiduals,   ParameterZStatistics,
  CookDistances,  Function,   PearsonChiSquare,
  CorrelationMatrix,  HatDiagonal,    PearsonResiduals,
  CovarianceMatrix,   LikelihoodRatioIndex,   PredictedResponse,
  CoxSnellPseudoRSquared, LikelihoodRatioStatistic,   Properties,
  CraggUhlerPseudoRSquared,   LikelihoodResiduals,    ResidualDeviance,
  Data,   LinearPredictor,    ResidualDegreesOfFreedom,
  DesignMatrix,   LogLikelihood,  Response,
  DevianceResiduals,  NullDeviance,   StandardizedDevianceResiduals,
  Deviances,  NullDegreesOfFreedom,   StandardizedPearsonResiduals,
  DevianceTable,  ParameterConfidenceIntervals,   WorkingResiduals,
  DevianceTableDegreesOfFreedom,  ParameterConfidenceIntervalTable}

model["Function"]

Use this for prediction

model["Function"][8, 54]

0.0196842

and plot the decision boundary surface in 3D along with the data (datPlot) using Show and Plot3D

modelPlot =
 Show[
  datPlot,
  Plot3D[
   model["Function"][x, y],
   Evaluate[
    Sequence @@
     MapThread[Prepend, {MinMax /@ Transpose@Join[dat1, dat2], {x, y}}]],
   Mesh -> None,
   PlotStyle -> Opacity[.25, Green],
   PlotPoints -> 30
   ]
  ]

With ParametricPlot3D and Manipulate you can examine decision boundary curves for values of the variables. For example, keeping x fixed and letting y vary or vice versa.

Manipulate[
 Show[
  modelPlot,
  ParametricPlot3D[
   {x, u, model["Function"][x, u]}, {u, 0, 80}, PlotStyle -> Orange],
  ParametricPlot3D[
   {u, y, model["Function"][u, y]}, {u, 0, 10}, PlotStyle -> Purple],
  PlotLabel ->
   StringTemplate["model[`1`, `2`] = `3`"] @@ {x, y, model["Function"][x, y]}
  ],
 {{x, 6, Style["x", Orange, Bold]}, 0, 10, Appearance -> "Labeled"},
 {{y, 40, Style["y", Purple, Bold]}, 0, 80, Appearance -> "Labeled"}
 ]

Update

You can also plot contours of the probability in 2D.

plot = ListPlot[{dat1, dat2}, PlotMarkers -> "OpenMarkers", PlotTheme -> "Detailed"];

Manipulate[
 db = y /. First@Quiet@Solve[model["Function"][x, y] == p, y];
 Show[
  plot,
  Plot[db, {x, 0, 10}, PlotStyle -> Red]
  ],
 {{p, .5}, 0, 1, Appearance -> "Labeled"}
 ]

Hope this helps.


Compressing profiles with a large number of dimensions

This seems similar to the collaborative filtering problem of assigning ratings to movies based on similar users and past reviews: https://en.wikipedia.org/wiki/Collaborative_filtering. The essence of this approach is to find a low-rank factorization of the matrix of ratings, where each user has one rating for each movie/product, and some of the ratings are unseen.


Classification using xgboost - predictions

The gradient boost algorithm create a set of decision tree.

The prediction process used here use these steps:

for each tree, create a temporary "predicted variable", applying the tree to the new data set.
use a formula to aggregate all these tree. Depending on the model:

bernoulli: 1/(1 + exp(-(intercept + SUM(temporary pred))))
poisson, gamma: exp(intercept + SUM(temporary pred))
adaboost: 1 /(1 + exp(-2*(intercept + SUM(temporary pred))))

The temporary "predicted variable" is a probability, having no sense by its own.

The more tree you have, the more smooth is your prediction.( as for each tree, only a finite set of value is spread across your observations)

The R process is probably optimised, but it is enough to understand the concept.

In the h2o implementation of the gradient boost, the output is a flag 0/1.
I think the F1 score is used by default to convert probability into flag. I'll do some search/test to confirm that.

In that same implementation, one of the default output for a binary outcome is a confusion matrix, which is a great way to assess your model ( and open a whole new bunch of interrogations).

The intercept is "the initial predicted value to which trees make adjustments". Basically,just an initial adjustment.

In addition: h2o.gbm documentation


Is it possible to predict using fewer number of features than the number of features that was used in training the model?

Machine learning model is nothing but an equation (hypothesis), derived from the features (data set) used for model training. Using training dataset we optimize this equation by changing the coefficient(also known as weight) values with each iteration(or with each batch).

For example:

 where x1, x2, x3 are features and theta1, theta2, theta3 are weights...that we have to optimize with each iteration.

  In order to get the results from this function, we need some value for
  x1, x2,and x3.  This function wont work in case you feed just either
  of these values.

  Best we can do in your case is to feed some average values for rest of
  the missing features. But then wont expect much from the model.


Machine learning without explicit training

Nearest neighbour algorithms (kNN and variants) do not have a training phase. They work by storing all the labelled examples, and using them directly for inference on new data.

There are some caveats:

"Training" is fast as it just stores each example once, but inference is slow, as typically it involves searching for relevant examples. This can be improved by indexing routines (which are not part of the algorithm, but might be part of a specific implementation).
Although there are no parameters to train, there may still by hyper-parameters to select, which will involve running a cross-validation exercise with hold-out data and varying the choices, e.g. of number of nearest neighbours to consider, or best distance metric to use to determine closest neighbours.
Data preparation (normalisation) and feature engineering are often required to get the best results out of kNN, and also require multiple attempts and tests.

In general, approaches that store and query labelled data without a training phase are instance based learning


TensorFlow: trainable_variables() is empty

There are functions in the Estimator class that handle this, namely: get_variable_names and get_variable_value.

I guess they want to avoid polluting the global scope. Note that you can pass a name parameter to your layers on creation to disambiguate them.


Sampling trying to keep as much multivariate variance as possible

It depends on what you mean by sampling. Is it sampling between or within features?

For between features, scikit-learn has a built-in option for VarianceThreshold which removes features whose variance does not meet some threshold.


Why it is taking more training time on one machine comparatively to similar another?

In machine learning, while your choice of inputs and hyper-parameters do matter, most of the time convergence ultimately comes down to how lucky you are. If you're lucky you might reach optimum in a very short period of time, whereas if you're unlucky you might get stuck in a local optimum with a huge error and get stuck in a very long loop.

That's one reason; another is possibly that if you're using TensorFlow and you installed the version for non-GPU computers, and your friends installed the version for computers with GPUs.


You can simply generate them yourself. They are simply representations of all integers up to $2^{32}$

Which format do you want the address? There are many ways to represent them according to your requirements, due to the fact that they are simply representation of a set of normal numbers:

standard dot-separated e.g. 192.168.0.1
unsigned int32 : e.g 3548551264
hex, dotted hex, octal bytes

You could generate them in Python like this:

def generate_ips():
    """Generate all possible IPv4 addresses, on by one"""
    for i in range(255):
        for j in range(255):
            for k in range(255):
                for l in range(255):
                    yield "{0}.{1}.{2}.{3}".format(i, j, k, l)

The you must loop on the generator function like this to actuall use the values:

for ip in generate_ips():
    print(ip)

# output
0.0.0.0
0.0.0.1
0.0.0.2
0.0.0.3
0.0.0.4
0.0.0.5
0.0.0.6
0.0.0.7

I used a generator (with the yield keyword instead of saving all values in a list) because it will just give you one at a time and not consume all those Gb of memory that you mentioned!

If you want to store all values in a list for some reason, you can alter the function to append each IP address to a list, then return that:

def generate_ips():
    """Return a list of all possible IPv4 addresses"""
    all_ips = []
    for i in range(255):
        for j in range(255):
            for k in range(255):
                for l in range(255):
                    ip = "{0}.{1}.{2}.{3}".format(i, j, k, l)
                    all_ips.append(ip)
    return all_ips


There are a number of implementations of Word2Vec, but most assume the basic unit to be 'sentences' - though they don't care what those sentences look like. If you are using something like gensim you will need each sentence in its own list, and each sentence will be a list of tokens. If you are using another package you may be reading from the disk in which case you likely need all documents concatenated with each newline representing a new sentence.

The fundamental consideration in deciding on the size of a 'sentence' (whether it's the entire note, all a patient's notes, or a single sentence) is what surrounding words should be used to train the model. The W2V model will consider all words within a certain distance of the target word when tuning its vector representation (regardless of if it's cbow or skipgram), but will not look beyond a 'sentence' boundary. So if your fundamental unit is a document not a sentence, associations will bleed across sentences (this may or may not be what you want).

As an aside, consider the nature of the documents first. Patient notes can be messy and full of automatically generated text, so it can be important to strip out or replace certain strings. Similarly, they often use whitespace and newlines rather than punctuation. You might want to consider treating existing whitespace as the end of a sentence and you might want to use a parser to replace dates, names, etc with default tokens to increase generalizability.


Model selection for a "set of linear models"?

A generalized linear model (glm) appeared to be work reasonable.

glm(y ~ x * c, data=df) and extensions hereof.


What is Sentiment Bias? How will it affect a Lexicon Based Sentiment Analysis?

What the authors call sentiment bias is the tendency for such systems to give strong results, either positive or negative, as opposed to neutral. Very often sentences or documents are more or less sentiment-neutral, but accumulating the positive or negative weights associated to individual words makes it more likely to result in a non-zero "sentiment value".

Very simple example:

I am comparing deep learning and lexicon/rule-based models for sentiment analysis.
               +1   +1                   -1         +1         +1

Individually words such as "deep", "learning, "model", "sentiment" can be considered positive; "rule" can be considered negative. As a result your sentence would receive a strong positive score of +3, even though it's actually neutral.


Using GridSearchCV for custom kernel SVM in scikit-learn

Look at the source code and modify it to meet your conditions.

https://github.com/scikit-learn/scikit-learn/blob/ab93d65/sklearn/model_selection/_search.py#L685


I have a solution that might not be very standardized but will suffice your need. Copy all of your data into any advanced text editor like, Notepad++ or Sublime Text. Next, Use CTRL+H to toggle the find and replace feature. Find : '\n' and Replace with ''.


Need help understanding LSTMs' backpropagation and carousel of error

Part 1: Derivation

It's important to note that $\delta z \neq z^{-1}$.

So on page 11 when he says he plans to derive $\delta I^t, \delta h^{t-1}$, and $\delta w^t$ he's invoking the chain rule rather than inverse matrix multiplication.

Let's derive $\delta I^t$.

$\delta I^t = \frac{\partial E}{\partial I^t}$ by definition
$\delta I^t = \frac{\partial z^t}{\partial I^t} * \frac{\partial E}{\partial z^t} $ by the chain rule

2.1.1 Recall that $z^t = W * I^t$

2.1.2 Therefore $\frac{\partial z^t}{\partial I^t} = W^T$

2.2.1 $\frac{\partial E}{\partial z^t} = \delta z^t$ by definition
Therefore $\delta I^t = W^T * \delta z^t$

The author continues by noting that since $I^t = \begin{bmatrix}x^t\\h^{t-1}\end{bmatrix}$, that $\delta h^{t-1}$ can be retrieved from $\delta I^t$ where  $\delta h^{t-1}$ would be the dx1 terms after the first nx1 terms.

Along the same vein to derive $\delta W^t$,

$\delta W^t = \frac{\partial E}{\partial W^t}$ by definition
$\delta W^t = \frac{\partial E}{\partial z^t} * \frac{\partial z^t}{\partial W^t} $ by the chain rule

2.1.1 Recall that $z^t = W * I^t$

2.1.2 Therefore $\frac{\partial z^t}{\partial W^t} = (I^t)^T$

2.2.1 $\frac{\partial E}{\partial z^t} = \delta z^t$ by definition
Therefore $\delta W^t = \delta z^t * (I^t)^T$

Part 2: Constant Error Carousels

The following explanation draws heavily from the 14th slide of this lecture, and from the LSTM section of this blog, which I would highly recommend referencing. Consider this portion of my answer the TL;DR version of these two links.

The Constant Error Carousel (CEC) as you might well know is the magic of the LSTM in that it prevents vanishing gradients. It's denoted as follows:

$c_{t+1} = c_t *$ forget gate + new input $*$ input gate

In the case of regular RNNs during backpropagation, the derivative of an activation function, such as a sigmoid, will be less than one. Therefore over time, the repeated multiplication of that value against the weights $f'(x) * W$ will lead to a vanishing gradient.

In the case of an LSTM, we only multiply the cell state by a forget gate, which acts as both the weights and the activation function for the cell state. As long as that forget gate equals one, the information from the previous cell state passes through unchanged.

So in our case, the parameters we derived $h_t$ and $h_{t-1}$ are the hidden states which as part of the input vector $I^t$, will be filtered through the forget gate, and affect the cell state accordingly. The $\delta W$ we solved for is the parameter we use to update all our weights/gates.


Binary Classification without machine learning tools

How about implementing one of the most basic binary classifiers by hand without any libraries, Logistic Regression?
Here is the explanation of the Logistic Regression in subtle terms:

https://www.youtube.com/watch?v=6tByJTacCOc&index=6&list=PLBAGcD3siRDguyYYzhVwZ3tLvOyyG5k6K

Here is the description of writing a simplified version of a binary classifier in Python, which will take a few buckets of lines in the vectorized form:

https://www.youtube.com/watch?v=ERIFzbTdIm0&index=14&list=PLBAGcD3siRDguyYYzhVwZ3tLvOyyG5k6K

If you feel uncomfortable with the implementation or confused about something, I'm sure that if you watch the videos in between the two that I provide, you will be able to easily write the algorithm that the video describes. I hope I could help; if you are not happy with the offered solution, I will be around for alternatives. Have fun.


Machine learning algorithms and Computer Vision technologies for detecting 52 playing cards deck

Yes, you can recognize these cards. For easy implementation, you can check here and here. Also, you can build your custom neural network model with tensorflow, keras, pytorch etc. Recognizing visually "52 card deck" is already solved problem. Because, cards have good features & landmarks. You either can use neural networks or "old school" computer vision algorithms.


Why don't you import the data into R and use some built-in function for encoding? createDummyFeatures from mlr package is a function, which can do one-hot encoding of categorical variables.

https://www.rdocumentation.org/packages/mlr/versions/2.13/topics/createDummyFeatures


Algorithm Suggestions for a Self Project

I would start with EDA, as usual. Plot stuff. For categorical data, what counts do you have? For quantitative data, perform the usual summaries and correlations. Since you don't have a question in mind already, I would then move on to unsupervised learning algorithms to suggest patterns in the data. Clustering is definitely a good option here (probably preceded by feature scaling). The unsupervised learning algorithms can lead into supervised algorithms, with all the associated statistical tests.

Without knowing your data, you simply cannot choose algorithms.


Discrimination vs Calibration - Machine Learning Models

Discrimination is the separation of the classes while calibration gives us scores based on risk of the population.

For example, there are 100 people that we’d like to predict a disease for and we know that only 3 out of 100 people have this disease. We get their probabilities from our model. Due to good predictability power, our model predicts probabilities between 0-0.05 for 70 people and 0.95-1 for 30 people. This is a good discrimination between classes. We now know that 30 people are at high risk considering only discrimination. But we also know that only 3 out of 100 people get the condition which is 3% prevalence. We use the 3% prevalence to calibrate our scores which will give the actual risk based on population of 100. That means, 0.95 x 0.03 = 0.0285 is their actual risk to the disease.

This is a very crude approach, there are advanced techniques like Kernels, Platt Scaling etc.,


You can use the pandas rolling method for that.

back = 3
cols = ['high','low']

# calculate rolling mean on the dataframe's columns of interest
df_back_average = df[cols].rolling(back).mean()

# rename columns before concatenating the result back
df_back_average.columns = str(back) + '_back_average_' + df_back_average.columns

pd.concat([df[cols],df_back_average], axis=1)

high        low         3_back_average_high     3_back_average_low
0.000442    0.000445    NaN                     NaN
0.004580    0.004650    NaN                     NaN
0.004820    0.004890    0.003281                0.003328
0.004890    0.004920    0.004763                0.004820
0.004750    0.004920    0.004820                0.004910
0.004610    0.004860    0.004750                0.004900
0.004620    0.004800    0.004660                0.004860


Wikipedia says: "Assign each observation to the cluster whose mean yields the least within-cluster sum of squares (WCSS)"

I think in your case, this is translatable to: $c_i$ is assigned to the closest centroid by euclidean distance.

For your second question, the centroid should $\mu$ should have the same number of dimensions as each training point $x_i$. They are both points in the co-ordinate system.

You can use a high number of features with K-means, for example, text analytics might reduce a corpora of news articles to 10,000+ dimensions. Depending on the package you use these might be represented as a sparse matrix.


predict_generator gives "Found 0 images belonging to 0 classes."

Make a folder Structure.

For instance:

Training_Data
       |
        ---Cat
       |
       -----Dog

Because keras expect your folder in catag


What is the difference between LS-SVM and P-SVM?

Least squares support vector machines were first introduced by Suykens and Vandewalle in 1999. They are slightly prior to proximal SVM proposed by Fung and Mangasarian in 2001. It appears that the latter did not know about LS-SVM and re-discovered the approach in a very similar manner.

Both methods are very close in the aspect that, instead of solving a quadratic programming (quadratic problem optimization with linear constraints), they solve a linear system. The trick to do this is to transform the optimization constraint from an inequation to an equation. Both methods use the same trick.

There appear to be minor differences though. In the conclusion of their paper on P-SVM, Fung and Mangasarian note:

  Least squares are also used in [33] to construct an
  SVM, but with the explicit requirement of Mercer’s positive
  definiteness condition [35], which is not needed here. Furthermore,
  the objective function of the quadratic program
  of [33] is not strongly convex like ours. This important feature
  of PSVM influences its speed as evidenced by the many
  numerical comparisons given here but not in [33].

Where [33] is the original LS-SVM paper by Suykens and Vandewalle.

As of today, a quick internet search shows that LS-SVM are much more common. Whether it is because of anteriority / initial popularity, because they perform better, and/or because their implementation is easier, however, I don't know.


Naive Bayes Classifier - Discriminant Function

In Naive Bayes, for the case of two classes, a discriminant function could be $$D(\boldsymbol{x}) = \frac{P(\boldsymbol{x}, c=1)}{P(\boldsymbol{x}, c=0)}$$ which can be anywhere in $[0, +\infty)$, and decides $c=1$ if $D(\boldsymbol{x})>1$, $c=0$ otherwise, or it could be the logarithm of that value

$$d(\boldsymbol{x}) = \text{log}\frac{P(\boldsymbol{x}, c=1)}{P(\boldsymbol{x}, c=0)}=\text{log}P(\boldsymbol{x}, c=1)-\text{log}P(\boldsymbol{x}, c=0)$$
which can be anywhere in $(-\infty, +\infty)$ (handling zero probability as a special case), and decides $c=1$ if $d(\boldsymbol{x})>0$, $c=0$ otherwise.

As a side note, $P(\boldsymbol{x}, c=k)$ in Naive Bayes is calculated as
$$P(\boldsymbol{x}, c=k)=P(c=k)\prod_{i=1}^{d}P(x_i|c=k)$$
or equivalently for log probabilities as
$$\text{log}P(\boldsymbol{x}, c=k)=\text{log}P(c=k) + \sum_{i=1}^{d}\text{log}P(x_i|c=k)$$


how to improve searching index in dataframe

This probably doesn't take into account that dates are sorted and thus performs as O(n). Try using binary-search on dates, that would perform as O(log(n)) (avoid implementing your own, look for a standard module, maybe it exists in numpy).


Comparison between cost functions to determine the "best" model?

tl;dr since you are referring to accuracy, I'm guessing you have a classification task.
There is no way to evaluate a model's actual performance, by its loss.
The goal of the loss function is to train the model and not to show how well this model classifies.
What you should do to see the best model is to evaluate it by a classification metric (e.g. accuracy). You should note that it is best to measure these on the validation set to avoid overfitting.

Some things to consider...

Different loss functions, don't measure the same thing. For example take into consideration MAE and MSE. MSE penalizes larger errors disproportionately, compared to MAE. Which is the best? Depends on the problem and what you want!

Classification depends on your model's predictions, which are by their nature continuous. However, when we are referring to the "best model" we want the one that classifies better. This can cause misleading effects, because a model can reduce its loss by becoming more "confident" in the samples it classifies as correct (usually due to overfitting).

You can look at the following as an example of this:
label | model1 | model2
  0   | 0.995  |  0.5
  0   | 0.400  |  0.5
  1   | 0.600  |  0.5
  1   | 0.600  |  0.5

model1 is a better classifier ($3/4$ correct predictions), but model2 ($2/4$) would have the smallest loss*.
*technically this depends on the loss function, but this stands on MAE, MSE and cross entropy losses.


plot show relationship between independent variable and dependent variable(Binary)

There is package named Boruta in R/Python. This is also called as predictor importance test. Used to pick the important variable.

Uses of this package:

Boruta is a feature selection algorithm. Precisely, it works as a wrapper algorithm around Random Forest. This package derive its name from a demon in Slavic mythology who dwelled in pine forests.

We know that feature selection is a crucial step in predictive modeling. This technique achieves supreme importance when a data set comprised of several variables is given for model building.

Boruta can be your algorithm of choice to deal with such data sets. Particularly when one is interested in understanding the mechanisms related to the variable of interest, rather than just building a black box predictive model with good prediction accuracy.

The outcome would be a box whisker plot with their importance with the target variable. You need not worry about categorical and continuous variables. you just need to cast them correctly before passing the data to the algorithm.

You can go through this Link for better understanding on the package and implementation of the same.

This is the sample plot:

Let me know if you need any additional information.


Interpolation in nlp - definition of O term

Terminology point: those symbols aren't O, but thetas Θ.

Confusingly, these values labelled theta are normally referred to as lambdas, as in the page you quote. They are weights used in interpolation (as opposed to backoff) that sum to 1, and can be calculated from the corpus itself via a variety of methods:

  How are these λ values set? Both the simple interpolation and conditional interpolation λs are learned from a held-out corpus. A held-out corpus is an additional
  training corpus that we use to set hyperparameters like these λ values, by choosing
  the λ values that maximize the likelihood of the held-out corpus. That is, we fix
  the N-gram probabilities and then search for the λ values that when plugged into
  Eq. 4.24 give us the highest probability of the held-out set. There are various ways
  to find this optimal set of λs. One way is to use the EM algorithm defined in Chapter 7, which is an iterative learning algorithm that converges on locally optimal λs
  (Jelinek and Mercer, 1980).

  Speech and Language Processing. Chapter 4: N-Grams (4.4.3 Backoff and Interpolation) (p.15)


how to implement Excel vlookup on Python

You will want to load your data in two pandas data frames (if you haven't already)
Next you will need to use dataframe.join():

Note: You might want to think about the mode (inner, outer, left or right)


On-line random forests by adding more single Decisions Trees

There's a recent paper on this subject (On-line Random Forests), coming from computer vision. Here's an implementation, and a presentation: Online random forests in 10 minutes


What would be the target input for Transformer Decoder during test phase?

At training time, the input to the decoder is the target sentence tokens, which are indeed unknown at the test time. What you call the second input are the desired outputs, which are not usually referred to as an input to the decoder, 1. for clarity, 2. they are technically input to the loss function.
At test time, we do not need the loss function, but we still need to pass some input to the decoder. The decoding proceeds autoregressively, i.e., at each decoding step, we execute the decoder layers and get a probability distribution over the target tokens. We select one token (typically the best-scoring one, but it gets trickier with beam search) and append it to the input of the decoder. It means that the input to the decoder is generated one token at a time, gradually as the sentence is decoded.


The information for this plots is provided by a meteorological service in a Geospatial Format.  A common format for weather data is ESRI shape file format (SHP). I gave a presentation on an introduction to modelling hurricane losses a couple of years ago (YouTube). There is a link to the code here.  An example example storm track with intensity bands that was produced is:

Hope this helps.


Too low accuracy on MNIST dataset using a neural network

What am I missing?

Incorrect architecture for the classification task. You have a single binary output, trained using binary_crossentropy, so the NN can only classify something as in a class (label 1) or not (label 0). Instead, you most likely want 10 outputs using softmax activation instead of sigmoid, and categorical_crossentropy as the loss, so that you can classify which digit is most likely given an input.
Incomplete processing of MNIST raw data. The input pixel values in X_train range from 0 to 255, and this will cause numeric problems for a NN. The target labels in y_train are the digit value (0,1,2,3,4,5,6,7,8,9), whilst for classification you will need to turn that into binary classes - usually a one-hot coding e.g. the label 3 becomes a vector [0,0,0,1,0,0,0,0,0,0].

Scale the inputs - a quick fix might be X_train = X_train/ 255 and X_test = X_test/ 255
One-hot code the labels. A quick fix might be y_train = keras.utils.to_categorical(y_train)

I made those changes to your code and got this after 10 epochs:

val_loss: 0.1194 - val_acc: 0.9678


Could we use image hasing techniques for images classification tasks?

You could, but it won't be very effective.

Image hashing is aimed at detecting two instances of almost the same image.  So, if your training set contains an image of a dog, and the test set contains an almost-identical image, then using image hashing you could use that to learn the label of the test set image.  But in practice that doesn't provide much generalization power.  In practice, we want to take hundreds of different images of dogs, and use that to learn to recognize new images of docs, so that if in the test set we are given a totally new image of a dog, we can still classify it as a dog.  Image hashing won't help with that.

In other words, image hashing isn't designed for this sort of thing and will work poorly.


Optimizer for Convolutional neural network

Yes, you can use the same optimizers you are familiar with for CNNs.

I don't think that there is a best optimizer for CNNs. The most popular in my opinion is Adam. However some people like to use a plain SGD optimizer with custom parameters.

An excellent article explaining the differences between most popular gradient descent based optimizers can be found here.


With regards to VC-dimension, why can you shatter 3 points with circles but not 4 points?

Given $4$ points $A,B,C,D$. If they do not lie on the boundary of a convex hull, then it is impossible to shatter the inner point from the boundary.

So assume they lie on the boundary of the hull. So they form a convex quadrilateral. Meaning $\angle A+\angle B +\angle C +\angle D =360^\circ$
Then we can assume w.l.o.g. $\angle A +\angle C \leq 180^\circ$, where $A$ and $C$ are opposite points.

Now the claim is that you cannot have a circle containing $A,C$, but not $B,D$.

Assume that you have such a circle, that contains $A,C$ but not $B,D$. Then we can make the circle smaller if necessary such that $A,C$ lie on the boundary, but $B,D$ is still not contained in the circle.

But now since the the points lie outside the circle $\angle B +\angle D < 180^\circ$. But this is a contradiction thus such a circle does not exist.

This last part has to do with the fact that for a circular quadrilateral opposite angels sum up to $180^\circ$ together with the fact that an angle of a point outside a circle is smaller that on the circle.

https://en.wikipedia.org/wiki/Cyclic_quadrilateral


convert predict_proba results using class_weight in training

The correction that you are talking about is called probability calibration -- you want the "real" probability of an observation being in each class, right?

The two most common approaches to probability calibration are Platt Scaling and Isotonic regression. Since you are training on a balanced training set (which is the right thing to do in your case because the original dataset is unbalanced), you can apply these techniques afterwards on your test set.

Sorry I can't explain the techniques fully here, but hopefully knowing the names of these terms gives you a starting point.


MinMaxScaler when LSTM predictions fall outside of training range?

Inverse-transforming with MinMaxScaler should be capable of producing something outside of the training data's range.  It seems that, in your use case, using a final activation that lands in $[0,1]$ might not be appropriate.  Even if you transform the training data to land in, say, $[0,0.7]$, applying a sigmoid or some-such on the final layer seems to lack motivation.

As to the question of whether it can be done: yes, just with a little roundabout thinking.  You can't specify what you want the scaler to think your data's max and min are, but you can specify the output range you want (parameter feature_range), which amounts to the same thing.


Understanding Training and Test Loss Plots

Assuming that the train and validation sets in the curves under comparison are the same, the best curve is probably the one with the lowest validation loss value.

Numbering your figures from left to right and from top to bottom, I would say the best one is #5 (second row, second column).

Now, let's break down what is going on in each plot:

Very high values, seemingly random, no decrease whatsoever in either train or validation losses: the model is not learning; probably there's something wrong with either the model or the optimization process, or maybe some hyperparameter value is terribly wrong.
Descending values for both training and validation losses, with validation loss having a gap with the training one, and both stabilized (i.e. neither of them will probably go any lower -if in doubt about this, leave them more training time-): training seems Ok, but there is room for improvement if you regularize your model so that you get your training curve upper and the validation one lower.
Initially, both curves descend, then validation starts going up around step 800: overfitting. You should try regularizing the model and, if that's not effective, use early stopping to use the model that performs best on the validation data. You may also try with some hyperparameter tunning, or having a learning rate schedule that makes it smaller with time.
Both curves are descending and it seems they will keep doing that for a while: training is not finished, leave it training more time.
Both curves descend, despite the initial plateau, and reach a low point, with no gap between training and validation curves: you can probably improve the model weight initialization. Anyway, this plot seems the best, as the validation curve reaches the lowest value and there is no overfitting.
Both curves go up: there is something wrong, probably in how you define your loss function optimization process.

I don't see any clear case of underfitting among your plots. In an underfitting scenario, we would see that the model learns something but both the training and validation losses stabilize at too high values. This would suggest a lack of capacity of the model, preventing it to properly capture the data distribution with respect to the labels.


You can do something like that for matrix

mat = new float*[rows];

for (ushort i = 0; i < rows; i++) {
    mat[i] = new float[cols](); //init 0

}


Do categorical features always need to be encoded?

You have partly answered this question yourself ("because converting to integers implies that there is an ordering between features").

I will just clarify the terminology a bit more.

Categorical data: information has categories, but no natural ordering defined between them (gender, name of user's cat)
Ordinal data: information has categories with natural ordering defined between them (annual income scale defined in terms of categories \$40000,   \$40000 - $80000).

If the variable is of type ordinal, you can replace it with integers and proceed with the algorithm. If it is categorical, it should be converted as well as encoded.

Hope this helps.


Neural Net for regression task on images only learning mean of training data

I took a deeper look at this problem with some example data (that the OP provided) and some simplifications. I just predicted brick height param as opposed to the 17 example params in the question. I used the Keras library in Python to explore a few different architectures.

Initially I replicated the problem, simple CNNs were predicting identical values for all heights, and the best loss (around 0.0053 on the sample data) was when this predicted the mean very precisely for all inputs. For some reason the network could not find the visual structure that is obviously in the data.

After some experimentation, I had a network that functioned as intended. It was surprisingly deep for such a simple problem. My best guess at the reason why is that the combined overlap of all the convolutions needs to be of the same size order as the features being detected.

Some important details:

Check that the order of the dimensions of the images when read with the image library you are using match the expected structure that the CNN is working on. If the dimensions are in a different order, this will not cause any error, but it scrambles the interpretation used by the CNN and prevents it learning anything but the mean.
Avoid using pooling layers, as they obscure pixel locations of features from further layers.
Check that your parameters are not ambiguous. The two "brick width" parameters seem interchangeable for instance. That makes it much harder for the network to predict them, and it is likely to output a rough mean for the example even if you avoid the problems with architecture.
The problem is harder to find a working architecture for than I first thought. I am not entirely sure why that is the case, but am guessing it is to do with the size of the features that need to be isolated in order for the network to "measure" them. Eventual working architectures were quite deep e.g. 10 convolutional layers, despite the pixel-level structures being quite obvious and clear in the training data.

Here is a gist of my solution in Python/Keras. Summary of architecture in the gist: 4 times Conv2d layers with 8 feature maps and 5x5 kernel. Dropout 25%. 4 more times Conv2d layers with 16 feature maps and 5x5 kernel. Dropout 50%. Fully connected layer with 256 outputs. Dropout 50%. Fully connected later with 128 outputs. Dropout 50%. Single output neuron. All layers use ReLU activation, except output which is linear.

The example re-sizes images to 50x50 for speed. However, using 100x100 images works fine. I added a couple of convolutional layers to it for that (one each for the 8 features and 16 features section), and get a reasonable loss of ~0.00025 on a validation set. It looks possible to get a lower loss still with more training examples, more epochs and perhaps more feature maps. Many of the predictions were very good, close to the actual value.

Interestingly, the worst errors were for short bricks predicting close to a 2:1 or 3:2 ratio with the ground truth, implying that the network had found correct structure but was being confused by the periodic nature of the bricks. I think this is something that could be resolved with more training examples (and perhaps more features in higher layers) and it should be possible to get much lower loss metric on brick height.

I think that the corresponding tiny-cnn structure looks like this:

net << convolutional_layer<relu>(100, 100, 5, 3, 8)
    << convolutional_layer<relu>(96, 96, 5, 8, 8)
    << convolutional_layer<relu>(92, 92, 5, 8, 8)
    << convolutional_layer<relu>(88, 88, 5, 8, 8)
    << convolutional_layer<relu>(84, 84, 5, 8, 8)
    << dropout(80 * 80 * 8, 0.25)
    << convolutional_layer<relu>(80, 80, 5, 8, 16)
    << convolutional_layer<relu>(76, 76, 5, 16, 16)
    << convolutional_layer<relu>(72, 72, 5, 16, 16)
    << convolutional_layer<relu>(68, 68, 5, 16, 16)
    << convolutional_layer<relu>(64, 64, 5, 16, 16)
    << dropout(60 * 60 * 16, 0.5)
    << fully_connected_layer<relu>(57600, 256)
    << dropout(256, 0.5)
    << fully_connected_layer<relu>(256, 128)
    << dropout(128, 0.5)
    << fully_connected_layer<identity>(128, 1)


RL: Collecting States (training data) in real-life. Must use fixed timestep?

Your "fixed timestep" idea is actually very similar to a common technique called frame skipping. Instead of waiting a fixed amount of time, agents wait a fixed number of frames $k$ before choosing a new action. In the meantime, they repeat their most recently chosen action.

Frame skipping was included as part of the Atari 2600 Arcade Learning Environment. It was also used in the foundational DQN paper. Common values of $k$ are 3, 4, and 5. The value chosen depended on the game being played, since different games had important events happen at different time resolutions. In these papers, frame skipping enabled training to happen roughly $k$ times faster. So this is definitely a valid technique to try.

I actually think this would generally be less of a concern in the robotics application. Forward propagation usually happens much more quickly than real-world event timescales. As an example, Stanford famously applies RL to fly small helicopters, which requires considerable precision.

Finally, if your forward propagation really is taking too long, you should consider a faster architecture. One approach would be just to make your neural net smaller. You might consider policy distillation for compressing a large, trained network into a smaller one. Also, make sure you're not using some ridiculously slow activation function like sigmoid or tanh. ReLU is the common choice if you don't need a bounded output for a given neuron. If you do, I recommend softsign.

If your time bottleneck is actually in action selection, due to a large action space and using a value network, you should seriously consider switching to a policy-based method (e.g. actor critic). This would help because sampling from a distribution over actions would potentially be much faster than the $\max$ operation involved in value-based methods. You can read more about this in Section 13.7 of Sutton and Barto's RL book.


I suspect you're working with the Ames house price dataset - one of Kaggle's introductory competitions.

Replacing the missing values with the dataset mean / median is very general.

I believe you have a variable which describes which neighbourhood the house is in. A more focussed approach would be to replace that missing value in GarageYrBlt with the mean / median for that neighbourhood (subject to additional sensibility checks like the garage not being built before the house).


Statistical significance test in deep learning for regression problems

You could use either the k-fold or the test-only.

The k-fold is a possibility which gives you more robustness because you tried on every possible combination of train/test, but you are still avoiding overfitting if you use the train/test partition.

Just make sure you use the same dataset in all the comparisons you make.


How to input a 3d model into ML algorithm?

Convolutional Neural Networks can work with 3 dimensional data, and they don't require much feature engineering. Provided you have enough training data, that seems like a way to go. As long as your data is in the form of a grid it is actually easy-peasy. Just imagine an image-like pipeline, but of the form [x,y,z,n]. Where n is the number of examples. As opposed to [x,y,n] (black and white) or [x,y,c,n] (color-channels).

It might be the case however that your data is in a vector format (as tends to happen with building models). I do believe there are experiments with straight up graphs and deep learning, but my feeling is that you might be better rasterising your model in that case.

You will probably need to add some layers to compress the data concepts (conv and pools etc) and end with a fully connected layer of 1 node (since you are looking for a scalar).


What methods can be used to detect duplicacy in image dataset?

I think the dhash technique might help. It essentially creates a signature for each image, then you could isolate the duplicated images. 50M could take a while, so perhaps you can try that with a smaller subset and see how well it works.


Can a Box plot be used for finding the useful features from the dataset?

This is actually very simple: the more separation you see between the boxes, the stronger the 'predictive power' of the covariate.
Boxplots can be actually viewed as histograms looked at from the top. The box itself encompasses 50% of the data (from 25th to 75th percentile), and the line inside the box is the median. Whiskers show you the bounds of the data, up to 1.5 IQR (if I remember correctly). Anything over that range is an outlier.
Consider 2 plots: displacement and acceleration. In the displacement plot the boxes are completely separated, meaning that at least 50% of the data 'mass' is in a completely different place for mpg01='0' than for mpg01='1'. On the other hand, in acceleration the top of one box is around the median in another box - indicating poor separation.


When classifying documents with naive Bayes, which probabilities do you multiply in case of repeated terms?

You could use either of the two approaches:

Just count whether your documents in your corpus contain or not a word $w$ and then estimate the likelihoods of $P(w_1|C)...P(w_n|C)$, as you describe
You could count how many instances of any word has each of the documents of your corpus, but then the estimation of the likelihoods becomes a bit more complicated than what you describe; in essence you need to use a multinomial distribution: $P(w_1|C=k) = \sum_{i=1}^{N}x_{it}z_{ik}/\sum_{s=1}^{|V|}\sum_{i=1}^{N}x_{is}z_{ik}$, where $N$ is the number of documents, $x_{it}$ in the numerator is the counts of word $w_1$ per document, $s$ in the denominator iterates the number of terms $|V|$ in your corpus and $x_{is}$ is the number times each term appears in each document, and finally $z_{ik}$ is 1 if the document belongs to class k, 0 otherwise, in both numerator in the denominator

Hopefully the short explanation makes sense, anyway you have a very complete and useful reference here (the same notation in the formula above).