One Challenging Neural Network Question and MATLAB Simulation

I believe the solutions should be a total of 3 neurons for each question. This is because two neurons are sufficient to calculate the lines that separate all the points. Then, an extra neuron can be used to decide to which class does a point belong to. Let's see why:
1st Question: Output of the neurons $=1$ or $=0$
First, we should note that both classes are separable if we use two lines:

Let's call these lines $l_1$ and $l_2$ (the order doesn't matter). Because of the fact that a line depends only on 2 parameters (let's call them $k$ and $c$), we know that the points $(x_1,x_2)$ that belong to these red and blue lines will  satisfy:
$$ l_1 = x_1 + k_1\,x_2 + c_1 = 0 \\
l_2 = x_1 + k_2\,x_2 + c_2 = 0$$
If a point $(x_1,x_2)$ doesn't belong to these red or blue lines (like all the points in the picture), then the above equalities would not hold.
In this situation, we have that the points lying at one side of the line will lead to a quantity $l_i > 0$, and the points to the other side will lead to a quantity $l_i < 0$ (with $l_i$ being $l_1$ or $l_2$).
Given this, we know that the first thing a neuron computes is the weighted sum of its inputs using its weights ($w_i$) and bias ($b_i$):
$$ w_ix_1+w'_ix_2+b_i$$
This means that we can calculate the previous lines using two neurons. This can be achieved by setting $w_i = 1$, $w'_i=k_i$ and $b_i = c_i$. Just like in the next NN:

So, for example, the first neuron will be calculating the value of $l_1$ and the second neuron will be calculating the value of $l_2$.
As said earlier, all the points that need to be classified will have values of $l_i$ $>0$ or $<0$, so we can use an activation function that returns a neuron output of $1$ if $l_i>0$ and $0$ if $l_i<0$.
This way, we will end up with four possible combinations of outputs:
$$ \begin{align}
(0,0)\,\, \text{and} \,\, (1,1)\quad &\text{For one class}\\
   (1,0)\,\, \text{and} \,\,(0,1)\quad &\text{For the other class}
\end{align}$$
Thereby we can use this to finally know to which class does a point belong to. Note that the first class has these values repeated $\Rightarrow$ If we substract them we will end up with a value of $0$. However, if we substract the values of the second class, we would end up with $1$ or $-1$.
Hence we can use an activation function at the output neuron that outputs a value of $1$ when the substraction of its inputs is $0$ and that outputs a value of $0$ otherwise, achieving the correct classification. Thereby, our final NN would be like:

2nd Question: Output of the neurons $=1$ or $=-1$
The only thing that changes w.r.t. the previous reasoning is that the neurons will output $-1$ when $l_i<0$ (instead of outputing $0$). This way, we would have the next four cases with the hidden layer output:
$$ \begin{align}
(-1,-1)\,\, \text{and} \,\, (1,1)\quad &\text{For one class}\\
(1,-1)\,\, \text{and} \,\,(-1,1)\quad &\text{For the other class}
\end{align}$$
Where again, the first class has these values repeated and the second class doesn't $\Rightarrow$ we can still use the previous NN to correctly classify the data.
Edit regarding the comments
The above explanation respects the constraints given by the question (neuron outputs $\in\{0,1\}$ or $\in\{-1,1\}$), but it also assumes that we are able to choose freely how is the activation function w.r.t. its inputs.
Below in the comments we have talked about using simpler activation functions, concretely perceptrons, which have the next behaviour:
$$ \text{output} = \begin{cases}
1 \text{ if } \mathbf{w\cdot x}+ b > 0\\
0 \text{ otherwise}
\end{cases}$$
Perceptrons have already been considered above in the answer for the first two neurons, which are able to transform our initial data to XOR data.
Given this, it's important to note that 3 perceptrons are enough to model any logical function (as argued in posts like this one from Medium). This means that with the above reasoning we would need 3 more neurons, making a total of 5 neurons needed.


Crowdsource your results.

Your problem is very similar to writing Amazon Alexa skills. Creating intents, utterances and slots for Alexa skills. One of the recommended methods for developing this section of Alexa skills is to simply ask your friends and peers how they would go about asking for a given outcome. Crowdsourcing your inputs will give you some robust results because other people may think quite differently than you. I would recommend hopping into whatever Slack/Discord/chat channels you belong to and asking there. There may be more formal sources for such information out there but I am not directly aware of them.


The simplest method is to calculate the euclidean distance between the baricenters of the two distributions; This do not take in account of the variance between the distributions, however.

If you want something more accurate, you can add to that distance the spread of the distributions; between the others, you can calculate the spread as the standard deviation of the distances between the baricenter (already calculated) and each of the points.

You can also think of using a join metric of the first two (baricenter distance and spread distance).

An example in Python. Take the three distributions in the image:

As you can see, distribution A and B are centered around the same point, but they have different spreads. Distribution A and C have different centers, but their spreads are more similar.

Here the code to calculate the distances I described. The more the distances are small, the more the distributions are similar.

#create three dummy distributions
dist_a=[]
dist_b=[]
dist_c=[]
for i in range (100):
    dist_a.append(np.random.randn(2)+10)
    dist_c.append(np.random.randn(2)+25)
    dist_b.append(np.random.randn(2)*5.5+10)

plt.scatter([a for a, _ in dist_a], [b for _, b in dist_a], label='distribution a')
plt.scatter([a for a, _ in dist_b], [b for _, b in dist_b], label='distribution b')
plt.scatter([a for a, _ in dist_c], [b for _, b in dist_c], label='distribution c')

plt.legend()

#calculate baricenters
bc_a=np.mean(dist_a, axis=0)
bc_b=np.mean(dist_b, axis=0)
bc_c=np.mean(dist_c, axis=0)

#calculate the distance between baricenters
dist_a_b=np.linalg.norm(bc_a-bc_b)
dist_a_c=np.linalg.norm(bc_a-bc_c)
dist_b_c=np.linalg.norm(bc_b-bc_c)

print("baricenter distante between distribution A and distribution B=", dist_a_b)
print("baricenter distante between distribution A and distribution C=", dist_a_c )
print("baricenter distante between distribution B and distribution C=", dist_b_c )
print ("\n")

#calculate the spread of the distributions, e.g. their standard deviation
spread_a=np.std(dist_a)
spread_b=np.std(dist_b)
spread_c=np.std(dist_c)

dist_spread_a_b=np.abs(spread_a-spread_b)
dist_spread_a_c=np.abs(spread_a-spread_c)
dist_spread_b_c=np.abs(spread_b-spread_c)

print("spread distance between distribution A and distribution B=", dist_spread_a_b)
print("spread distance between distribution A and distribution C=", dist_spread_a_c)
print("spread distance between of distribution B and distribution C=", dist_spread_b_c)
print ("\n")

#put in a single metric. NB, the paramenter of this join is subjective, and depend on the usecase
#alpha=0 : don't care about the euclidean distance between the baricenters
#alpha=1 : don't care about the spread distance between the baricenters

alpha=0.3
joint_metric_a_b=alpha*dist_a_b + (1-alpha)*dist_spread_a_b
joint_metric_a_c=alpha*dist_a_c + (1-alpha)*dist_spread_a_c
joint_metric_b_c=alpha*dist_b_c + (1-alpha)*dist_spread_b_c

print("joined metric distance between distribution A and distribution B=", joint_metric_a_b)
print("joined metric distance between distribution A and distribution C=", joint_metric_a_c)
print("joined metric distance between distribution B and distribution C=", joint_metric_b_c)

Outputs:

baricenter distante between distribution A and distribution B= 0.22454217332627005
baricenter distante between distribution A and distribution C= 21.028862497007008
baricenter distante between distribution B and distribution C= 20.98580645790957

spread distance between distribution A and distribution B= 4.153324630270008
spread distance between distribution A and distribution C= 0.004700454831506384
spread distance between of distribution B and distribution C= 4.158025085101515

joined metric distance between distribution A and distribution B= 2.974689893186887
joined metric distance between distribution A and distribution C= 6.311949067484157
joined metric distance between distribution B and distribution C= 9.206359496943932

Following the first metric (euclidean distance between baricenters), distribution A and B are more similar.

Following the second metric (spread distance), distribution A and C are more similar.

The third metric is tunable, by the parameter alpha, which accept values between 0 and 1. Depending on your usecase, you can be more interested that the distributions lie around the same point, and so you care more on the distance between their baricenters, or that the distributions have the same spread, even if their baricenters are slightly displaced. So, you have to adapt the paramenter alpha to your case.


Classifcation of non-linear regressions based on their shapes

As a quick brainstorming idea, you could create new attributes based on your original ones.
Let's say you have a 2D feature set x and y; so, you could test if adding x2, y2 , x•y, x3... as new attributes, and applying regularization, gives you as important features the ones you expect for each type of regression (test it with hand made examples for exponential regression, logarithmic, etc)


It can approximate, but as the input grows the error of that approximated function will also grow exponentially, right?

You are right, the neural network uses ReLu functions to approximate the output $f(x) = x^2$ and there is some error incurred, but you are forgetting that

Non-linear functions also do approximation since we use only some subset of activation functions and incur some error which can be as huge as with ReLus
There is also a number of neurons to consider. The more neurons/layers you have - the better the approximation can be (since more small-interval ReLus are fit to better approach the shape of $x^2$). If your input size grows, you should also consider increasing the number of neurons/layers, in this way the approximation will always be close enough.

Now x² is a simple curve. How can ReLU perform better for real data which will be way more complicated than x²?

The main reason for this is that even though other activation functions can approximate $x^2$ or something else better, they are not optimised as fast. Generally speaking, non-linear activation function decision surface is more complex than the loss created by ReLus and can contain better global minimum but this minimum is harder to find, and, thus, with the less sophisticated loss created by ReLus, we can find a better optima using some gradient descent procedure, and it will be much faster.


Product classification according to description

So the question is what should the training and test sets consist of for classifying products by their product description.
To clarify, for any machine learning task we require data. In the case of supervised machine learning, the dataset will be labelled (in your case with USPSC codes).
The dataset is then split into a training, (validation if possible) and test set.
So the answer to your question would be that both trading and test sets will have the same format in that they contain the input text which will go into the model. The only real difference is that the labels in the training set are observed so that the model can find an underlying function which translates the text into a class. In the test set, the labels are used to evaluate the model.


Keras next(); what does (2, 256, 128, 128, 3) mean

Keras generator returns a Tuple for data and label. So is that 2.
First you should unpack it with a tuple then use image and labels.

yielding tuples of (x, y) where x is a numpy array containing a batch of images with shape (batch_size, *target_size, channels) and y is a numpy array of corresponding labels

img, labels = traindata.next()
img.shape, labels.shape

Output - ((16, 224, 224, 3), (16, 2))
We are also supposed to return a Tuple of batches of images/labels in case we build a custom generator for Keras.


K.gradients gives type error where both arguments are tensors

I am very sorry but this was a careless error.  In the critic network I had different pathways for state and action, joining them right before the Q-value output layer but...

I actually had state as the input for both pathways rather than action for the action pathway.

So of course, if the action is not present in a network, there is no way to use the chain rule to get the dQ(s,a)/da gradient of output w.r.t. actions.

Here is the code for clarification:

def build_model(self):
    #lrelu = LeakyReLU(alpha=0.1)
    #Define input layers
    states = layers.Input(shape=(self.state_size,), name="states")
    actions = layers.Input(shape=(self.action_size,), name="actions")

    #Add hidden layers for state pathway
    net_states = layers.Dense(units=32, use_bias=False)(states)
    net_states = layers.BatchNormalization()(net_states)
    net_states = layers.LeakyReLU(alpha=0.1)(net_states)

    net_states = layers.Dense(units=64)(net_states)
    net_states = layers.BatchNormalization()(net_states)
    net_states = layers.LeakyReLU(alpha=0.1)(net_states)

    #hidden layers for action
    net_actions = layers.Dense(units=32)(actions) #had (states) here instead
    net_actions = layers.BatchNormalization()(net_actions)
    net_actions = layers.LeakyReLU(alpha=0.1)(net_actions)

    net_actions = layers.Dense(units=64)(net_actions)
    net_actions = layers.BatchNormalization()(net_actions)
    net_actions = layers.LeakyReLU(alpha=0.1)(net_actions)

and the output to console shows that it is no longer a None type:

action_gradients [<tf.Tensor 'gradients_3/dense_13/MatMul_grad/MatMul:0' shape=(?, 1) dtype=float32>]

Edit:  The code works for MtnCarContinuous-v0 environment of openai gym with a modified reward function (incremental based on car position because the final reward is so sparse), over 90 seems to be achieved in 11 episodes.  I rendered so actually saw the car make it to the top on the attempts where score is over 100 below.
Here are the results:

episode: 0 score: -41.59034754863541 mean: -41.59 std: 0.0
episode: 1 score: -75.87242108451743 mean: -58.73 std: 17.14
episode: 2 score: 32.01844640263133 mean: -28.48 std: 45.01
episode: 3 score: 132.904319261567 mean: 11.86 std: 80.02
episode: 4 score: 125.81198290946529 mean: 34.65 std: 84.85
episode: 5 score: 84.26163480413017 mean: 42.92 std: 79.63
episode: 6 score: 126.89684490110164 mean: 54.92 std: 79.37
episode: 7 score: 139.18190524840517 mean: 65.45 std: 79.3
episode: 8 score: 100.24481691450521 mean: 69.32 std: 75.56
episode: 9 score: 165.80286734425076 mean: 78.97 std: 77.31
episode: 10 score: 109.29507292352991 mean: 94.05 std: 66.24
episode: 11 score: 209.07900825070152 mean: 122.55 std: 44.84

Source code here:  https://github.com/nyck33/openai_my_implements/tree/master/continuous/mountaincar


When memory errors occur with model.fit(), is it due to GPU RAM or normal RAM?

Yes. The error is for GPU memory. You should look at training in batches options if you haven't already. This thread has short explanation on possible benefits and implications.

Quick batch size tutorial


What is the human performance on ImageNet, the top 1 error? (not top 5 error)

The ImageNet top-1 error rate for humans is unknown.  Russakovsky et al and Andrej Karpathy only discussed the ImageNet top-5 error rate for humans.


There are a few factors to consider in anomaly detection.  A simple method would be to plot a boxplot of the data and calculate outliers this way (boxplot description).

If the time series is seasonal, you could take an approach similar to Twitter's anomaly detection algorithm (Twitter Anomaly Detection).  This uses an outlier detection method (ESD) applied to the seasonal decomposition of the time series.

You could use clustering and SVMs as well.

There is also change-point detection which I understand less but can be very effective.

Really, your answer depends on how complex your data is.  Sometimes, a simple heuristic algorithm works the best.

Here are some similar questions:

Anomaly detection in Time Series Data - Help Required

How to classify and cluster this time series data


Which is the best Machine learning technique for this Load forecasting problem?

I think this is one of the time series forecasting because you want to predict the future load of residence with the past dataset of residence's load by time. By my experience I recommend using LSTM RNN for the solution. An LSTM is well-suited to classify, process and predict time series given time lags of unknown size and duration between important events. Of course, this is supervised learning.

As for difference between supervised learning and unsupervised learning on your case, you don't need to use unsupervised learning for this solution because you have already dataset with input/correct output. if you are training your machine learning task for every input with corresponding target, it is called supervised learning, which will be able to provide target for any new input after sufficient training. Contrary, if you are training your machine learning task only with a set of inputs, it is called unsupervised learning, which will be able to find the structure or relationships between different inputs. It says you should use supervised learning - RNN or CNN.

PS: as for framework, I recommend using Tensorflow or Keras.


Replace the missing value with empty string and add these columns up

Classes.fillna(value='').sum(axis=1)


Product of dot products in neural network

Well, you are basically just creating a big nth degree polynomial with no small degrees.

So yes, fundamentally flawed because you don't have a bias in your weights, and then the optimization will be very complex because each parameter/weight depends on all the others entries in the computation.


K-modes clustering: Estimating which features were most impactful on clustering?

It boils down to counting.

Compute which feature value has the highest probability of predicting a particular cluster. It's a straightforward application of Bayes' formula.


The problem you are suggesting is a text summarization problem. It can be of two types abstractive - understanding the text distribution and then producing a summary, extractive - extracting specific words from the text to produce a summary. Clearly, your problem falls in the latter category. Also, since you have the labels to be generated it is a supervised text summarization problem. Take a look at this paper regarding algorithms for the same. It suggests two solutions for a similar problem, you'll of course have to modify them a bit and see which one works best for you. Good luck.


How are weights organized in Conv2D layers?

If you print those shapes using below for loop

weights_m=model.get_weights()
for i in range(8):
  print(weights_m[i].shape)

you will get output as

(3, 3, 1, 32)
(32,)
(3, 3, 32, 64)
(64,)
(9216, 128)
(128,)
(128, 10)
(10,)

so we will get one layer weight and bias. we have a total of 4 layers(2 conv + 2 dense) so 8 weight vectors.


How to decide for the contamination value (proportion of the outliers) in my dataset?

If you want to experiment something like that I think you can add some outliers into your data for that respective field and see what are its effects.

If you find that these are prone to outliers(most of of the features are), then you can handle situation by removing such outliers before appending to the existing data.

What is your target variable?
Did you do any kind of Correlation analysis/ Predictor Importance?


How to predict the duration of burst given several series

I guess, a nice place to start is seminal work of John Kleinberg:  https://www.cs.cornell.edu/home/kleinber/bhs.pdf
Yes, this paper is outdated, but anyway it's still very influential and simple. Also, digging articles, citing this paper, could be very helpful. By the way, these guys solve very similar task ( and they even use Weibo logs as well!): http://www.csce.uark.edu/~xintaowu/publ/pakdd15.pdf


RandomForest advantage compared to newer GBM models is that it is easy to tune and robust to parameter changes. It is robust for most use cases although the peak performance might not be as good as a properly-tuned GBM. Another advantage is that you do not need to care a lot about parameter. You can compare the number of parameter for randomforest model and lightgbm from its documentation. In sklearn documentation the number of parameter might seem a lot, but actually the only parameter you need to care about(ordered by importance) are max_depth, n_estimators, and class_weight, and the other parameters are better to be left as is. So for me, I would most likely use random forest to make baseline model.

GBM is often shown to perform better especially when you comparing with random forest.
Especially when comparing it with LightGBM. A properly-tuned LightGBM will most likely win in terms of performance and speed compared with random forest.

GBM advantages :

More developed. A lot of new features are developed for modern GBM model (xgboost, lightgbm, catboost) which affect its performance, speed, and scalability.

GBM disadvantages :

Number of parameters to tune
Tendency to overfit easily

Please bear in mind that increasing the number of estimators for random forest and gbm implies different behaviour. High value of n_estimators for random forest will affect it robustness, where as for GBM model will improve the model fit with your training data (which if too high will cause your model to overfit).


Difference between from nltk import word_tokenize and from nltk.tokenize import word_tokenize?

So basically
from nltk import word_tokenize
and
from nltk.tokenize import word_tokenize
from nltk import word_tokenize is basing to the same tokenizeAPI as both are the same with the usage and I don't see any difference in the results which they give.


On what threshold we should resample the data?

First of all, resampling or artificial sampling is not necessary nor panacea in many cases of imbalanced learning. Other methods may yield good results (eg threshold tuning, class weights adjustment and so on..).
That being said, resampling is indeed helpful in many cases of imbalanced learning.
But when do we have imbalance really? Is a 60-40 split imbalanced or not?
This is the heart of your question and cannot be answered fully (except for extreme cases) unless other methods are tried first and then resampling.
In many cases whether imbalance indeed is a problem depends on the case at hand and there are no hard limits (except of course extreme cases as mentioned above).
So first try other methods to adjust balance and then try resampling, is my advice.


How to Scaling Out Artifical Neural Networks?

You might take a look over TensorFrames, a Databricks library which allow running TensorFlow code on top of Apache Spark :

https://github.com/databricks/tensorframes


First of all, the score_samples function of SKLearn's Kernel Density object returns the log of the probability density, not of probability. Therefore, its exponent isn't exactly probability - e.g. in your example you have a probability above 1, which can't be.
Second, apparently the log-probability-density is normalized by the number of points the kernel was trained on (see line 210 here, where log_density -= np.log(N).
This effectively changes the expression $\rho_K(y) = \sum_{i=1}^{N} K(y - x_i; h)$ (from the User Guide) to $\rho_K(y) = {1 \over N} \sum_{i=1}^{N} K(y - x_i; h)$, i.e. turning the sum across points in the train set to an average across the train set.
By the way, I think that this is what they meant to say in this line in the function's description:

This is normalized to be a probability density, so the value will be low for high-dimensional data.

But I agree that it's a bit unclear.


Preparing for a Machine Learning Design Interview

To wrap up you can have the following to prepare for ML interviews:

Machine Learning Engineering Book
Machine Learning Systems
Towards Data Science (Medium): They have a lot of interesting posts there.

And don't forget Data Science Stack Exchange. On this site, you can have a lot of interesting questions and answers. If you don't know the answer for your interview question, feel free to drop it here.


Importance of Variance in Machine Learning

Variance in a feature (defined as the average of the squared differences from the mean) is important in machine learning because variance impacts the capacity of the model to use that feature.

For example, if a feature has no variance (e.g., is not a random variable), the feature has no ability to contribute to task performance. A zero variance feature will be constant for different levels of the target.

Additionally, some machine learning models make strong assumptions about the distribution of features. Variance is one way to check for distributional assumptions.


p1 = (X.dot(X.T))**-1 needs to be p1 = np.linalg.inv((X.T.dot(X)))

X**-1 doesn't invert a matrix, this is equivalent to 1/X (elementwise) and you need to transpose the first not the second matrix.


how does model.score(X_test,y_test)'s behave?

You don't specify the language or library you're using.

Assuming it's sci-kit learn in python then model.score automates the prediction of your data using X_test and compares it with Y_test and by default uses the R-squared metric to so (hence don't need to manually derive y_pred).

If you have derived the predictions anyway (e.g. using model.predict(X_test)) then you can use the metric directly e.g. r2_score(y_true, y_pred).


Higher frequency of time series benefits

You are limited by the Nyquist frequency, which is a theoretical frequency. Let's suppose you want to capture a phenomenon that occurs at $60\,\text{Hz},$ such as standard electrical power from your wall output. The Nyquist frequency says that, theoretically, you should sample at $120\,\text{Hz}$, twice the phenomenon frequency, in order to recreate your signal.

However, in theory there's no difference between theory and practice, but in practice, there is. The usual rule-of-thumb in the data acquisition world is to sample at ten times the frequency of the phenomenon you want to capture. In the example above, therefore, you'd really want to sample at $600\,\text{Hz}$ to get a good picture for your signal.

Higher sampling rates will generally produce a better representation of your phenomenon. The flip side, of course, is that you produce more data, which makes analysis slower. There's definitely a law of diminishing returns going on here, and sampling the living daylights out of your system is unlikely to help you much unless you're in the RF world.


I have found answer for my question here.
If anyone needs it in future: all above likelihoods are assumed to be a gaussian distributions. Likelihood $P(\gamma)$ is assumed to be uniform. In the article it is shown how exactly are they defined and how to compute further is described in detail in article linked above.


Time series prediction without sliding window

Implementing this with a recurrent neural network is not that difficult. My suggestion is close to what you already suggested but instead of sampling different points in time I would just broadcast your target over all time steps. If there are a total of N time steps, and we want to predict on steps t=1 to t=N-1, our X is a matrix of shape [N-1, k] with k the number of features. Y is a sequence of shape [N-1] with all the values being equal to EventN.

This is similar to what you actually want to achieve, namely at every time step you want to predict the final value. Your loss function will be calculated over all time steps and backpropagated through time. It's possible that you might want to weigh your function as a function of the time step, maybe later predictions are more or less important, you'll need to do some analysis on this. Another recommendation would be to add some feature of time in your feature explicitly, that might help the network.


In their paper, mixup: Beyong Empirical Risk Mininization, the authors provide a piece of code how to train an epoch in PyTorch:
# y1, y2 should be one-hot vectors
for (x1, y1), (x2, y2) in zip(loader1, loader2):
    lam = numpy.random.beta(alpha, alpha)
    x = Variable(lam * x1 + (1. - lam) * x2)
    y = Variable(lam * y1 + (1. - lam) * y2)
    optimizer.zero_grad()
    loss(net(x), y).backward()
    optimizer.step()

As you can see, they only use a single loss function defined on the new combined target variable y or, in your terminology, target.


Using a White noise image to minimise the loss in a (convolutional) neural network

The loss functions are not related to the noise image.

The network is run twice to determine parameters to the loss functions. Once for the content image, and once for the style image. The code you are asking about occurs afterwards.

First the network is run against the content image (note no noise mixing here):

sess.run([net['input'].assign(content_img)])

Aside: Uses an assignment function - not a feed_dict - because in order for the style transfer to work, the input has become a tf variable, not a placeholder.

Then the first loss function is defined against some of the network parameters from this run (this extracts the activations of interest from the networks as fixed params to the loss function - because it uses output from sess.run, not references to tensor variables):

cost_content = sum(map(lambda l,: l[1]*
  build_content_loss(sess.run(net[l[0]]),net[l[0]]) , CONTENT_LAYERS))

The same thing is done for the style image:

sess.run([net['input'].assign(style_img)])
cost_style = sum(map(lambda l: l[1]*
      build_style_loss(sess.run(net[l[0]]), net[l[0]]), STYLE_LAYERS))

And a final cost function defined:

cost_total = cost_content + STYLE_STRENGTH * cost_style

Now, after the cost function is defined, the code initialises a new input (which is also our style transfer output):

sess.run(net['input'].assign( INI_NOISE_RATIO* noise_img +
                   (1.-INI_NOISE_RATIO) * content_img))

This line has nothing to do with defining loss. It just initialises the network ready to generate gradients against a starting image and the loss functions that were already defined beforehand. You can start with pure noise or a copy of the content image to start the process, and each will give a slightly different result.

The original paper started from a random image. Quoting from the paper (on page 10):

  Thus we can change the initially random image $\tilde{x}$ until it generates the same response in a certain layer of the CNN [. . .]

. . . however, starting from the content image was tried and used very soon afterwards by many in practice, and it works well.

This mix of both approaches is what the author has chosen, presumably because it gets pleasing results. As there are no lower layers in the network than the input, in fact you can start from an arbitrary image (e.g. all zeros), and you should still get a result, although it may not look as nice.

To (sort of) answer your question as written:

  Why use a mix of the noise image and content image as input for the white noise bit in the content loss?

The code is not doing that. The mixing is occurring as you say. But the line you show is not part of the content loss. The mixed image is the initial solution, and iterations will turn it into the stylised image.


Data Science Career: From Researcher To Data Scientist

Great question! I can appreciate that you have a base of statistics to work from. Most data scientists - especially the ones that go to these data "camps" or whatever - out there do not have this and it should serve you well in your career. Here are some answers to your questions, in order:

Your background will help you because you are used to things like study design and execution and working with data. But I don't seen anything specific to HCI that would help you. When you are a good data scientist, the data itself takes on a muted importance. Right now, when I work on a project, I care that the data was collected in a sustainable way, but what the data is doesn't really matter. Eventually you will reach this too - it's all just numbers and math :-)
You have the statistics side. So now you need the (1) programming skills and the (2) the modeling skills. You should choose to learn a language like Python or R and develop your skills there. From there, you should start looking at the relevant libraries like Tensorflow and Keras to help you build your modeling skills. You should also pick up a textbook or some other resource that talks about what models actually do and how you can tune them, agnostic of the language you choose. Finally, you'll be ready for some research on neural networks and how those work. The measure of success here is being able to write some "traditional" models in Tensorflow (like regression, decision trees, etc) and also being able to write a neural network in Keras.

I think that your base of already working around experiments and (presumably) applying the scientific method will serve you well and already puts you ahead of most data scientists out there. Now, it's just a matter of being able to do all of that with code and, if you follow the steps above, I think you will see those connections and how you can enter the world of data science.


Why leaky relu is not so common in real practice?

Speaking from my experience, the performance of the two is almost the same. It might depend on the problem.

LeakyReLU was introduced to address the vanishing gradient problem, however it introduces yet another hyperparameter, the slope. If you want to squeeze out a little bit more performance of your model you can use LeakyReLU and tune the slope parameter, but that comes again of the cost of potential overfitting.


Catastrophic forgetting in linear semi-gradient RL agent?

The same thing was happening to me with a deep Q Network on the cart-pole  problem. Having a "memory" with past (S,A,R,S) sequences and sampling it to form mini batches with the new observations helped a lot to reduce catastrophic forgetting. Reducing the step size once the agent has improved a certain amount also helped.


Why don't all feature selection methods in sklearn allow specifying desired variance explained?

Because PCA is just one of the feature selection methods, don't expect others to have a tractable computable formula to specify ahead of time the optimal number of new features.
I suggest to start reading on Representation learning, and what we mean by latent variables.


Measuring performance of different classifiers with different sample sizes

You need to look at the confidence interval of the statistic.  This helps measure how much uncertainty in the statistic, which is largely a function of sample size.


Beginner in programming and data science with 100 hours to spend learning the basics

Being good at a subject does not automatically make someone a good teacher, and it looks like Roman's answer on Quora has fallen into a trap of thinking everything he knows is simple and could be picked up quickly. Also making the potential student attempt things outside of a beginner level - effectively just by research on the web following a 2 paragraph pointer - is going to make progress slow and frustrating. Despite no doubt good intentions, the advice there is likely to give a very poor learning experience.

There are full structured tutorials available on free Massive Online Open Courses (MOOCs) available for learning data science topics. These have been put together by professionals who know how to teach, and the effort put into designing any of these courses, providing materials etc, totally dwarfs the effort that went into the advice on Quora.

Ignore the Quora answer, and sign up for one or more of these MOOCs. You can find them at Coursera, Udacity, edX and other similar places. If you want to cram a lot of study into a short time, look for "at your own pace" courses where all the materials are available to you immediately.

The almost canonical start point to test the waters would be Andrew Ng's Machine Learning course on Coursera. Machine Learning is one of the more immediately accessible and fun parts of data science, and the course goes into theory and practice with additional sections for beginners at the start covering necessary maths and programming. Following that entire course is probably around 100 hours total effort. You won't come out of it knowing data science, but you will gain useful practical skills, and get a real taster for the machine learning side of the subject.


How to store a trained Random Forest model in python or matlab as a matrix

Not sure about a matrix, but here's a repo for porting sklearn models into other languages, including C.

https://github.com/nok/sklearn-porter


At what stage are ROC curves used when building machine learning model?

The ROC-AUC curves are used to find the best threshold that optimizes True Positive Rate vs False Positive Rate. Using it in a K-Fold cross-validation is a good practice to determine the best threshold to use.
Then, your final test is here to validate that you did not overfit on some hyperparameters, including this threshold. So ROC-AUC must not be used again in final test. You should compare the results of your final test with the same threshold used in your cross-validation.
Hope it helps.
Note on threshold (EDIT):
The threshold to optimize could be the threshold to use in a binary classification problem that outputs probabilities (for instance, output of a sigmoid or a logistic regression). In that case, various threshold settings gives various the model's predictions (FPR, TPR), and so is built the ROC curve.
You could read further on sklearn guide page.


Future of deep learning (compared to traditional machine learning)

That's very interesting topic. So thumbs up from my side. Now coming to the point. Deep learning may be hot now but some variants of it or something new all together may emerge later. Let me point out the reasons I feel deep learning maybe getting old or will get.

Slow Learner

Slow learner as in, it converges to an optimal solution slowly but with GPU acceleration the training speed can be improved dramatically. The slowness is affected by the learning rate.
Adjusting the learning rate affects the reliability of the resulting deep neural net.

Huge Training Data Requirement

Deep learning requires very huge training data to achieve good performance. The presence of a huge number of parameters to adjust requires some huge example sets. The fact that deep learning requires such a large number of training examples makes it dull in some way, despite such huge training set requirement deep neural nets do have error rates of about 10%.

Overfitting

Deep learning tends to overfit easily but with the new dropout algorithm, this problem can be avoided but with some consequences such as an increase in error rates.

Poor Initial State

Deep neural nets have parameters that need to be initialized. The most used method is random initialization, this results in neural nets with very poor initial states. Compare this to a mammalian brain, the brain is born with some rigid instincts such as basic survival behavioral patterns.

Sensory Data Transformations

For example in visual object recognition tasks, images undergo various geometric and photometric transformations that need to be modeled by a recognition system to rectify new image observations. Deep learning as used today does not take such transformations into account, this is one of the reasons why deep neural nets still suffer from relatively high error rates (relative to a human).
Some other reason can be the slow transformation, algorithms mayb emerge together in future, strcture and the approaches. These are some pin points from my side. You can also go through these posts, they're also on it and to the point-

KDnuggets
Huffington Post

Cheers! :)


The DBSCAN clustering algorithm has a built-in Jaccard distance metric.

from sklearn.cluster import DBSCAN
db = DBSCAN( metric='jaccard' ).fit(X)
labels = db.labels_

# Number of clusters in labels, ignoring noise if present.
n_clusters_ = len(set(labels)) - (1 if -1 in labels else 0)

Where X is your dataset with the related columns you want to use.


Connecting Infusionsoft data to Google data studio

I'm starting to look at this now, to start with simply on batched/CSV export/import process pushing data into Google BigQuery first and using it as a source in DataStudio. I've thought about how to automate and I'm not sure Zapier will cut it - it doesn't seem to have an action for Contact 'updates' for one, which is a showstopper for us (though you don't mention what objects you want to query).

Another option would be to use the Novak Solutions Data Warehouse service to get everything into SQL and either query directly from DataStudio or replicate into BigQuery using StitchData or similar.

If you want a cheaper option still, it could be done via the Infusionsoft API, but you'd have the time and cost of creating the synchronisation tasks and managing changes. If you have a limited subset of objects (eg. Contacts + Tags) this might be cost effective.


What are recommended methods for multi-task prediction?

For the project, do you need to explain exactly how the input variables gave you the predictions? If so, trees based methods (R library 'tree', 'rpart') and Logistic Regression would be easy to explain to your stakeholder / lecturer.

If this explanation is not that important, or you could get by just showing the relative predictor importance, and you'd like to focus on accuracy. XGBoost is my default recommendation for multi-class predictions that I use at work on a daily basis. If XGBoost has too many parameters for your liking, you could try randomForest which only has 2 main parameters to tune. Both XGBoost and randomForest have functions for variable importance measures. (R Library XGBoost and randomForest)


One iteration means that you do one forward pass and one backward pass for a single batch containing batch_size examples. Then you move on to the next batch. Also see this post on SE SO.
Note that this is not identical to an epoch. An epoch is only completed when all examples of your dataset have been passed through your network. And as long as your batch size is less than the number of datapoints you will need $\frac{n}{\text{batch size}}$ (with $n$ being the sample size) iterations to complete an epoch. Also see this answer on SE DS.


Multiple confusion matrix for multiple training instances. Which one to take?

I can't check at the moment (no Matlab at hand), but I suppose the differences come from the different random seeds used to initialize the neural networks (at least this is the only part which i can think of that has a random component). I would suggest predicting class probabilities, averaging those and then viewing the resulting confusion matrix of the "averaged" prediction. This way you - to a degree - mitigating the effect of randomness resulting from different initializations of the weights.


How Int8 (byte) operations can be useful for deep learning?

Actually, recently people have been trying much lower precision in neural nets: 1-2-5 scheme (1 bit weights, 2 bit activations, and 5 bit gradients) seems to work well for easy datasets (MNIST and CIFAR-10). However, on ImageNet the results are significantly lower than those with full-precision (16 or 32 bit).
To achieve state of the art, convnets don't need more than 16 bits for training, but current RNNs might need more. For inference, on ImageNet, 4-5 bit weights (stochastically rounded from full precision) should be enough.


How to apply supervised machine learning when the target label depends on multiple input rows?

What about doing a concatenation of your rows (i.e. Attr1 -> Attr12) , such that you now have 3*4 features (because 4 rows of 3 features) as an input to a multiclass classification model?

For instance, first sample would be described by :

X = [1.1, 1.4, 2.5, 2.3, 2.5, 2.7, 1.1, 1.6, 1.9, 1.5, 1.6, 1.7]
y = "A"

Otherwise, there is no issue in giving 2D or 3D inputs towards a classifier. Take for example convolutional neural networks that do operations on images!


In theory, CPU and GPU should reach same accuracy.
But imo, generally people tend to use bigger batch size on GPU due to high parallelization. As you might know, in stochastic gradient descent a smaller batch size has a higher chance of reaching the global optimum. This could be a reason for the slight difference in accuracy.


Deep learning performance on classifying simple geometric figures

A CNN network should easily achieve close to 100% accuracy on this task. Few aspects make this task easier :

Clean High-contrast images generated from program
Only 1 shape per image
Given few images of a shape, libraries can generate thousands of training samples automatically with various transformations like rotations / scaling / flip
Within few layers network should identify edge-detection patterns for vertices and learn the 3 vertices = triangle and so on

For example, this code can generate training examples based on few input images :

https://keras.io/preprocessing/image/

datagen = ImageDataGenerator(
    featurewise_center=True,
    featurewise_std_normalization=True,
    rotation_range=20,
    width_shift_range=0.2,
    height_shift_range=0.2,
    horizontal_flip=True)

Few articles :

https://www.researchgate.net/publication/312328148_Binary_shape_classification_using_Convolutional_Neural_Networks

https://towardsdatascience.com/object-detection-with-neural-networks-a4e2c46b4491


Tensorflow Conv3D with variable input size

In Keras, you should specify the shape of your inputs and that shape should be fixed. Then you probably have to somehow resize all of your samples to a fix size $m \times n \times 500$.

You can use similar ideas in image resizing (e.g. interpolation, Nearest-neighbor interpolation, Bilinear and bicubic algorithms, Box Sampling, ...).


Git for Deep Learning - what are the best tools for versioning/tracking machine learning experiments?

One such tool is Polyaxon. I'm using it extensively and it's really helping the experimentation pipeline.

It has features like experiment queuing, hyperparameter tuning, clustering, could be self-hosted or on the cloud, plots the metrics, it is independent of the framework and so on ...

I definitely recommend it.


Transfer learning (on pre-trained inception net model) for multi label classification is giving similar probability for all labels

In my experience, the example code for a low number of classes (<200) works well. When moving to more classes the imbalance data makes the network converge to the same numbers. You have imbalance data because now each output is a binary classifier by its own, this doesn't happen with softmax. The way to mitigate the problem is to use weighted_cross_entropy_with_logits and set pos_weight to a positive number > 1 (10 works). But I still don't get very good results.


How to deal with highly skewed (on counts) dependent variables?

What models will you employ?

If you are working with deep learning, you can train it using mini batch gradient descent, and "artificially" build each mini batch so that the classes you want to predict appear with a more balanced frequency.


Why doesn't overfitting devastate neural networks for MNIST classification?

I have replicated your results using Keras, and got very similar numbers so I don't think you are doing anything wrong.

Out of interest, I ran for many more epochs to see what would happen. The accuracy of test and train results remained pretty stable. However, the loss values drifted further apart over time. After 10 epochs or so, I was getting 100% train accuracy, 94.3% test accuracy - with loss values around 0.01 and 0.22 respectively. After 20,000 epochs, the accuracies had barely changed, but I had training loss 0.000005 and test loss 0.36. The losses were also still diverging, albeit very slowly. In my opinion, the network is clearly over-fitting.

So the question could be re-phrased: Why, despite over-fitting, does a neural network trained to the MNIST data set still generalise apparently reasonably well in terms of accuracy?

It is worth comparing this 94.3% accuracy with what is possible using more naive approaches.

For instance, a simple linear softmax regression (essentially the same neural network without the hidden layers), gives a quick stable accuracy of 95.1% train, and 90.7% test. This shows that a lot of the data separates linearly - you can draw hyperplanes in the 784 dimensions and 90% of the digit images will sit inside the correct "box" with no further refinement required. From this, you might expect an overfit non-linear solution to get a worse result than 90%, but maybe no worse than 80% because intuitively forming an over-complex boundary around e.g. a "5" found inside the box for "3" will only incorrectly assign a small amount of this naive 3 manifold. But we're better than this 80% lower bound guesstimate from the linear model.

Another possible naive model is template matching, or nearest-neighbour. This is a reasonable analogy to what the over-fitting is doing - it creates a local area close to each training example where it will predict the same class. Problems with over-fitting occur in the space in-between where the values of activation will follow whatever the network "naturally" does. Note the worst case, and what you often see in explanatory diagrams, would be some highly curved almost-chaotic surface which travels through other classifications. But actually it may be more natural for the neural network to more smoothly interpolate between points - what it actually does depends on the nature of the higher order curves that the network combines into approximations, and how well those already fit to the data.

I borrowed the code for a KNN solution from this blog on MNIST with K Nearest Neighbours. Using k=1 - i.e. choosing the label of the nearest from the 6000 training examples just by matching pixel values, gives an accuracy of 91%. The 3% extra that the over-trained neural network achieves does not seem quite so impressive given the simplicity of pixel-match counting that KNN with k=1 is doing.

I tried a few variations of network architecture, different activation functions, different number and sizes of layers - none using regularisation. However, with 6000 training examples, I could not get any of them to overfit in a way where test accuracy dropped dramatically. Even reducing to just 600 training examples just made the plateau lower, at ~86% accuracy.

My basic conclusion is that MNIST examples have relatively smooth transitions between classes in feature space, and that neural networks can fit to these and interpolate between the classes in a "natural" manner given NN building blocks for function approximation - without adding high frequency components to the approximation that could cause issues in an overfit scenario.

It might be an interesting experiment to try with a "noisy MNIST" set where an amount of random noise or distortion is added to both training and test examples. Regularized models would be expected to perform OK on this dataset, but perhaps in that scenario the over-fitting would cause more obvious problems with accuracy.

This is from before the update with further tests by OP.

From your comments, you say that your test results are all taken after running a single epoch. You have essentially used early stopping, despite writing that you have not, because you have stopped the training at the earliest possible point given your training data.

I would suggest running for many more epochs if you want to see how the network is truly converging. Start with 10 epochs, consider going up to 100. One epoch is not many for this problem, especially on 6000 samples.

Although increasing number of iterations is not guaranteed to make your network overfit worse than it already has, you haven't really given it much of a chance, and your experimental results so far are not conclusive.

In fact I would half expect your test data results to improve following a 2nd, 3rd epoch, before starting to fall away from the training metrics as the epoch numbers increase. I would also expect your training error to approach 0% as the network approached convergence.


How can I reduce the number of dimensions using a Clustering algorithm in a mixed dataset?

Some common techniques to reduce number of features:

Missing Values Ratio. Data columns with too many missing values are unlikely to carry much useful information. Thus data columns with number of missing values greater than a given threshold can be removed. The higher the threshold, the more aggressive the reduction.

Low Variance Filter. Similarly to the previous technique, data columns with little changes in the data carry little information. Thus all data columns with variance lower than a given threshold are removed. A word of caution: variance is range dependent; therefore normalization is required before applying this technique.

Random Forests / Ensemble Trees. Decision Tree Ensembles, also referred to as random forests, are useful for feature selection in addition to being effective classifiers. Personally I will prefer this method as it is easy to implement. Example use is given in the kernel -->
https://www.kaggle.com/prashant111/xgboost-k-fold-cv-feature-importance?scriptVersionId=48823316&cellId=74

High Correlation Filter. Data columns with very similar trends are also likely to carry very similar information. In this case, only one of them will suffice to feed the machine learning model. Here we calculate the correlation coefficient between numerical columns and between nominal columns as the Pearson’s Product Moment Coefficient and the Pearson’s chi square value respectively. Pairs of columns with correlation coefficient higher than a threshold are reduced to only one. A word of caution: correlation is scale sensitive; therefore column normalization is required for a meaningful correlation comparison.

Refer : https://www.kdnuggets.com/2015/05/7-methods-data-dimensionality-reduction.html


Efficient way to search list of items in a text document

I have not tried this before but I will suggest:

Get all unique items from all documents:
Create a dictionary where key is that unique item and values is list of document ids (document names) where this item is present.

Once you are done. Just search your item in the dictionary key, it will return all the documents ids where this item is present.

This algorithm is called Inverted Indexing in Information retrieval.


Predict sinus with keras feed forward neural network

Keep in mind that the Accuracy measure is measuring whether the values are Exactly The Same. I.e. it is a classification measure, whereas approximating a sin curve is much better suited for measurement as a regression problem.

That said:

In evaluating the network's performance, what is the network actually doing? Let's take the network and perform a little visual analysis on its performance:

import matplotlib.pyplot as plt
preds = model.predict(x)
plt.plot(x, y, 'b', x, preds, 'r--')
plt.ylabel('Y / Predicted Value')
plt.xlabel('X Value')
plt.show()

Hmm. The model seems to be minimizing error by simply getting closer and closer to guessing 0 for every value, rather than approximating the function. There are several hypotheses here to explain this. One is that the network is not complex enough to model the function. In order to test this, let's simplify the function- that is, let's bring the range down to one sine cycle:

x = np.arange(0, math.pi*2, 0.1)
y = np.sin(x)

And try to train the network again:

Not wonderful, but a better fit, certainly.

How about with 100 epochs instead of 10?

How about with 1000 epochs?

This is, of course, very interesting. After 1000 epochs, our networks is able to roughly approximate the downward curve from 1:0 ($\pi/2$: $\pi$) of the sine response, but not the initial upward curve 0:1 (0:$\pi/2$) or the region in which the function is negative ($\pi$:$2\pi$).

This result begs the question- what will it look like after 10000 epochs?

Not significantly better. It looks like we'll have to change the architecture of the network (more layers, more neurons, and/or different activation functions) to improve beyond this point.

To inform this architecture change, let's take a look at the sigmoid activation function:

Uh-oh. The value of this sigmoid function can only ever be in the range 0:1, and the range of the sin function is -1:1.

To correct this, let's just normalize the sin response between 0 and 1:

y = (np.sin(x)+1)/2

Now, the network performs much better than before after 1000 epochs:

And 10000:

After 100000 epochs, it's roughly perfect:

Even still, this advancement doesn't help much on the larger sin range (after 1000 epochs):

x = np.arange(0, 100, 1)
y = (np.sin(x)+1)/2

If we, however, take the model trained on a single sin curve and further train it on the larger range, we begin to see progress after 1000 epochs:

x = np.arange(0, 100, .1)
y = (np.sin(x)+1)/2

model_copy = model
model_copy.fit(x, y, epochs=1000, batch_size=8, verbose=0)
model_copy_preds = model_copy.predict(x)

And more so after 10000 epochs:

And more so, well into the third repetition after 100000 epochs:

So, with careful training, our network with a single layer of sigmoid activations appears to be learning to generalize the sin curve. Further investigation could find a limit to that generalization, certainly.

For reproduction:

import numpy as np
from keras.layers import Dense
from keras.models import Sequential
import matplotlib.pyplot as plt
import math

x = np.arange(0, math.pi*2, .1)
y = (np.sin(x)+1)/2

model = Sequential([
    Dense(10, input_shape=(1,)),
    Activation('sigmoid'),
    Dense(1)
])

model.compile(loss='mean_squared_error', optimizer='SGD', metrics=['mean_squared_error'])
model.fit(x, y, epochs=100000, batch_size=8, verbose=0)

preds = model.predict(x)

plt.plot(x, y, 'b', x, preds, 'r--')
plt.ylabel('Y / Predicted Value')
plt.xlabel('X Value')
plt.show()

x = np.arange(0, 100, .1)
y = (np.sin(x)+1)/2

model_copy = model
model_copy.fit(x, y, epochs=10000, batch_size=8, verbose=0)
model_copy_preds = model_copy.predict(x)

plt.plot(x, y, 'b', x, model_copy_preds, 'r--')
plt.ylabel('Y / Predicted Value')
plt.xlabel('X Value')
plt.show()


How to do imbalanced classification in deep learning (tensorflow, RNN)?

This has already been answered both in stackoverflow and crossvalidated.

The suggestion in both cases was to add class weights to the loss function, by multiplying logits:

loss(x, class) = weights[class] * (-x[class] + log(\sum_j exp(x[j])))

For example, in tensorflow you could do:

ratio = 31.0 / (500.0 + 31.0)
class_weight = tf.constant([ratio, 1.0 - ratio])
logits = ... # shape [batch_size, 2]
weighted_logits = tf.mul(logits, class_weight) # shape [batch_size, 2]
xent = tf.nn.softmax_cross_entropy_with_logits(
  weighted_logits, labels, name="xent_raw")


.values coverts the dataframe into an array and remove the underlying structure introduced by pandas. The pandas DataFrame is a higher version of numpy array which gives the data a tabular look and feel which helps in better visualising the data (with the column names). However when we use .values we remove the higher structure and represent the data in a lower numpy array format. The advantage here is that the array format is computationally less expensive than the DataFrame format. In other words it increases the speed and allows faster computations. I hope this clears your question.


When developing machine learning models, is the size of each class in the test set important?

Depends if you are going to do online learning.
Lets say you will do online learning/incremental learning than test set Distribution will make difference. For example because of catastrophic forgetting of neural Networks.
If you are making Batch predictions than it makes no difference whats the test set Distribution. Model knows no difference since it does not Change ist state.


Cost function dependence on size - batch gradient descent

Your second change (calculating the average error) is the correct method. Imagine this: if there are billion examples in your training set, even if you make (very) small error on each, the sum total is going to add up to a large number.
So the cost function in case of least squared is mean squared error, not just sum.


How to apply multiple filter in Data Frame?

data.loc[ data['age'].isin(range_1) & data['height'].isin(range_2) ]


Which probability distribution will you use to model outliers?

I think the answer is Gaussian distribution. This is a famous approach that is used in Anomaly Detection. What you do is to fit your feature to the Gaussian distribution and the samples which have the probability below the specific threshold are labeled as an outlier.
Quoting from the paper Modeling Outlier Score Distributions:

Many existing unsupervised outlier detection algorithms calculate some kind
of score per data object which serves as a measure of the degree of outlier. Scores are used in ranking data points such that the top n points are considered
as outliers. For example, the statistical-based approach proposed in [4], uses
a Gaussian mixture model to represent normal behavior and each datum is
given a score on the basis of changes in the model. A high score indicates a high
possibility of being an outlier.

Example of usage - 1.
Example of usage - 2


Understanding Transformer's Self attention calculations

What is the value of Key, Value in the self attention calculation of Transformer model?

You have the queries matrix Q. Imagined it to a coordinate system. the x-axis is the key and y-axis is the resulting value.

Query vector is embedding vector for the word that is queried, is that right?

Yes, for the word the word or token

Is attention calculated in RNN is different from self attention in Transformer?

RNNs is a family of neural networks. They don't include attention per se.
A resource, that helped me greatly is: https://jalammar.github.io/illustrated-transformer/ maybe it will you also to better understand transformers.


Getting the leave-one-out error on least square regression to fit polynomials

numpy and scikit-learn are quite separate open-source projects. As such, they are compatible in some ways but not in others. All scikit-learn models are variants on the fundamental estimator object (see https://scikit-learn.org/stable/tutorial/statistical_inference/settings.html). The cross_val_score function expects a scikit-learn-style estimator object and there's no easy way to create such an object from the output of numpy.polyfit.

I'd suggest working entirely in scikit-learn, creating a regression estimator that you can cross-validate with cross_val_score. This SO post shows you how to build that polynomial regression estimator.


If you are using Python, you can use pandas. It comes with an expanding standard deviation function.

import pandas as pd
import numpy as np

# Generate some random data
df = pd.DataFrame(np.random.randn(100))

# Calculate expanding standard deviation
exp_std = pd.expanding_std(df, min_periods=2)

# Print results
print exp_std


My model accuracy doesn't change after first epoch

Your metric is accuracy although you are working on a regression problem, this doesn't make sense. You should use instead:

metrics = ['mean squared error']


Total RAM would be - Batch size X RAM to train one image (since backpropagation happens after the batch)

RAM for one training image -

A/ 4 Bytes X Number of parm

B/ Size of input for each layer considering downsampling and number of features map

(Suppose input are 200 × 300 pixels, the first layer’s feature maps might be 100 × 150, the second layer’s feature maps can be 50 × 75, and the third layer’s feature maps can be 25 × 38. The first convolutional layer has 100 feature maps, this first layer takes up 4 × 100 × 150 × 100 = 6 million bytes (6 MB). The second layer will take up 4 × 50 × 75 × 200 = 3 million bytes (3 MB).

C/ Size for the input image


AWS SageMaker Model as endpoint size limit

It doesn't seem that it is about the size of the model. I am no SageMaker expert, but the error message suggests that the model was deployed, but that something went wrong when the health check was run.

This could be caused by many different things, but the most probable would be that there is a bug in the code. Please check the following:

Can the model be loaded properly?
Can the model make a prediction?
Can the model make multiple predictions?


Bagging with Neural Networks Best practices

I think you have the right general idea.

Divide the dataset into training and test.
Divide the training dataset into k folds, call this M.

For each iteration of hyperparameter tuning (whatever you choose to tune);

For each fold i in M;

Set the validation set to be equal to fold i, and the (inner) training set to be equal to all folds that are not i.
Using the inner training set, generate N bootstrap resamples. Fit a single neural network model to each bootstrap resample, generating N models.
Predict the validation set using each of the N neural network models.
Count the votes of each neural network model. Set the final prediction for an observation in the validation set to whatever has the most votes as predicted by your N models. You should have a matrix of m observations by N predictions. Since you are actually classifying objects, try to keep N odd so that there are no ties. Alternatively, don't classify objects, and instead generate probabilities from your N models for each observation in the validation set. Then, calculate the mean predicted probability for each observation in your validation set, and choose a threshold for classification based off the costs of false positives/false negatives (probably superior, but I won't discuss this).
Calculate your error using your final predictions from step 6 and the actual ground truth labels from the validation set. Save these error scores.

end

Calculate the average error from all of the folds. If this average error is lower than a previous set of hyperparameters error, then save these hyperparameters from this iteration of tuning.

end

Generate N bootstrap resamples of the entire training set (the training set created in step 1) and fit a single neural network model to each of the N bootstrap resamples, using the optimal set of hyperparameters that you found above for each neural network model.
Generate predictions on the test set (created in step 1) using each of your N models. Set final predictions for the test set using the exact same methodology you used in step 6.
Calculate the error using your final predictions from step 10 and the ground truth labels found in the test set. This is your final, unbiased estimate of model performance.

In general, the more bootstrap resamples you can generate, the better. However, since you are fitting neural networks you might need to be conservative with how many models you decide to bag due to computational costs. I recommend setting the number of bootstrap samples, N, to be as large as possible while being mindful of your own computational/time constraints.

EDIT: In response to the comments in which we require a faster method, one can replace step 2 (and delete step 3) and instead divide the training dataset into an inner training set and a validation one. We then validate our model to a single validation set rather than k sets, which has the potential to increase the likelihood of overfitting to a specific validation set (though since we are presumably dealing with a large dataset, it probably will be fine).

All other steps should remain the same, though there is clearly no loop over folds anymore. In step 8, there will also be no need to calculate the average error since we will only have one estimate of model error from the single validation set.


How to select optimal threshold which separate different classes?

Rank all your instances by their output value. Then for each instance (or for each different output value, if many instances have the same value), calculate the performance (precision, recall, f-score) considering that the threshold is the output value for this instance. When this is done for all the instances, you can plot the graph of f-score as a function of the threshold for a nice visual result, or simply pick the threshold corresponding to the maximum performance.

For the record, this is also how a ROC curve can be built manually.

Btw your optimal threshold is certainly lower than 0.5, since your precision is 1.0 at 0.5 but you can probably still increase recall.


Number of capsules in the Primary Capsule Layer of Capsule networks

It could be that each capsule is of 2D form (thus 6x6). So each component capsules outputs an 8D vector. The 8D vector is derived by analyzing 8 feature maps propagated from the convolutional layer. So its like placing the feature maps in groups of 8. This means each component capsule represents one capsule therefore in the "dynamic paper", there are actually 32 capsules of form 8D.
NB: The Primary Capsule Layer is actually made up of convolutional layers.
So you can look at the 6x6 as the size of the feature maps after convolution in the primary Capsule layer.
take a look at this site.
Anyway I am still reading wide on capsules, because most of the stuffs in the paper are consolidated. I am open to corrections as well. Thanks


What is the difference between C and lambda in the context of an SVM?

You are right to be confused. What's going on is that the hyperparameters refer to different formulations. On the one hand there is a generic empirical risk minimizer

$$\lambda\lVert \mathbf w \rVert^2 + \frac 1 n \sum_{i=1}^n \max\left(0, 1 - y_i(\mathbf x_i^T \mathbf w + w_0)\right)$$

On the other there is the soft-margin objective

$$\frac{1}{2} \lVert \mathbf w \rVert^2 + C \sum_{i=1}^n \xi_i$$

such that $$\xi_i \geq 0, y_i\left(\mathbf x_i^T \mathbf w + w_0\right) \geq 1-\xi_i, \forall i$$

The first one places SVM in the framework of empirical risk minimization, in which the objective is an expected loss plus the $L_p$-norm of the main parameter. The purpose of introducing slack variables and constraints in the second formulation is to allow a fraction of points (adjustable by $C$) to lie on the wrong side of the margin, and eliminate the non-differentiable $\max$ function. Moreover, it draws attention to the fact that SVMs encode sparsity in the loss function, rather than the prior. This is called the primal form and can be solved using quadratic programming. Wikipedia currently explains this in the Computing the classifier section. You should be able to see that $C=1/{2n\lambda}$. Some texts omit the $1/2$ or the $n$.


Are there any frameworks available that allow for automated large scale supervised machine learning?

I think, Rapidminer can be used for that. However, the free version can only load 10,000 rows of data.


If we limit the question on comparing two graphs, I can propose a way based on adjacency matrices comparison. There is a sample notebook: graph_diff.ipynb

To summarize:

Having two graphs,

   A  B  C  D                  A  B  D  E  F
A  0  2  2  2               A  0  1  2  3  0
B  2  0  1  1               B  1  0  0  1  1
C  2  1  2  0               D  2  0  2  1  0
D  2  1  0  0               E  3  1  1  0  1
                            F  0  1  0  1  0

We can compare them and detect changes, producing result similar to diff output:

   A  B  C  D  E  F
A  1  0 -2  1  2  2
B  0  1 -2 -2  2  2
C -2 -2 -2 -2 -2 -2
D  1 -2 -2  2  2  2
E  2  2 -2  2  2  2
F  2  2 -2  2  2  2

Compare matrix nodes from both graphs. Edges values indicate change:

1 = same edge
0 = changed edge
-2 = removed edge
+2 = added edge

This matrix can be visualized as grid:

Or as graph:

which is more informative because shows also nodes changes:

green: new (added nodes E and F)
red: removed (removed C)
yellow: unaffected (A, B, D)

For simplicity, edges are unidirectional.


Why are only 3-4 features important in my random forest?

RandomForestRegressor has a parameter called max_features, which is the number of features to consider when determining the optimal split. You haven't explicitly specified this, so Python will use the default (auto) and consider all features.

Given that your trees are very shallow and you are considering all of the features to split, it would not surprise me that the strongest 3-4 are consistently popping up (the bagging process in random forests will cause some variation within this).

Decreasing max_features and/or increasing max_depth may yield a greater variety of "important" features.


Overfitting/Underfitting with Data set size

So, the underfitting means that you still have capacity for improving your learning while overfitting means that you have used a capacity more than needed for learning.

Green area is where testing error is rising i.e. you should continue providing capacity (either data points or model complexity) to gain better results. More green line goes, more flat it becomes i.e. you are reaching the point where the provided capacity (which is data) is enough and better to try providing the other type of capacity which is model complexity.

If it does not improve your test score or even reduce it that means that the combination of Data-Complexity was somehow optimal and you can stop training.


Numpy arithmetic operation between two columns

These types of operations can easily be done using both numpy or pandas. However, in this case I would recommend pandas since it is more intuitive. Using the example array we can create a pandas dataframe:

arr = np.array([[78, 3412, 98, 3441], [106, 3412, 127, 3434], [139, 3411, 160, 3434], [170, 3411, 191, 3442]])
df = pd.DataFrame(arr, columns=['a', 'b', 'c', 'd'])

The two new columns can now be added as follows:

df['e'] = df['a'] - df['c']
df['f'] = df['a'].diff(1)

Directly using numpy, one possible way would be to do:

arr = np.c_[ arr, arr[:,0] - arr[:,2], np.append(np.NaN, arr[1:, 0] - arr[:-1, 0]) ]


How to populate pandas series w/ values from another df?

Here is a one solution:

df2['Population'] = df2.apply(lambda x: df1.loc[x['Year'] == df1['Year'], x['State']].reset_index(drop=True), axis=1)

The idea is for each row of df2 we use the Year column to tell us which row of df1 to access, and then State to select the column.  Afterwards we reset the index of the result to prevent pandas from keeping the columns separate.


Multiclass classification with Neural Networks

Indeed, this is the standard interpretation of continuous classifier outputs, not only for neural networks, but for the more general case called Softmax Regression.

Thus, provided that you have used softmax activation on the final layer (in order, among other things, to ensure that your outputs indeed sum up to 1), you can interpret the continuous outputs as the respective probabilities of a particular data sample belonging to each one of your classes.

See also the discussion in this (rather unfortunately titled) discussion at SO:

How to convert the output of an artificial neural network into probabilities?


Representing similar states in reinforcement learning?

Define your biggest suits as suit1 and if lowest is different suit2. Then do the same with the ground.

In your example, it would be king and queen of suit1 in your hand and 4 suit1, and 10 suit2 and 6 suit3 on the ground.


Can Expectation Maximization estimate truth and confusion matrix from multiple noisy sources?

I don't have a full answer to your question, but I wanted to help (and would love to know a complete answer). In a simpler case when you have one source $m=i=1$, it seems to me you are describing a scenario that resembles a hidden Markov model which uses a EM algorithm-based solution (Baum–Welch algorithm) – that is if you make the further hypothesis that a Markov property is satisfied. Because convergence properties of the stationary distribution of Markov chains are exponentially fast the algorithm convergence may be limited to the implemented algorithm. I hope that info helps point you in the right direction.


latent_dim does not become of shape 16*16

x = layers.Dense(128 * 16 * 16)(generator_input)

mean: The input of size 32 (the latent_dim) is connected to a layer of size 16*16*128

These value are decided by the data scientist, they are the hyper-parameters.
There is a lot of research on how to adjust hyper-parameters for GAN.
People use generally a size of 100 for the latent_dim but it may depends of the complexity of the image you are trying to generate.


How to transform a folder of images in a csv file

You are describing a one-time pre-processing step that will crawl through your folder and turn each image into a line of data and then save the entire collection in a CSV file. In turn, that file becomes your gold standard dataset.

If I was in your position, I would look into the Keras pre-processing tools that already provide python libraries to quickly do this task. It's a common need for image processing, the Keras library is very mature and can do this for you.


Is Data Science just a trend or is a long term concept?

The one thing that you can say for sure is: Nobody can say this for sure. And it might indeed be opinion-based to some extent. The introduction of terms like "Big Data" that some people consider as "hypes" or "buzzwords" don't make it easier to flesh out an appropriate answer here. But I'll try.

In general, interdisciplinary fields often seem to have the problem of not being taken serious by either of the fields they are spanning. However, the more research is invested into a particular field, the greater is the urge to split this field into several sub-topics. And these sub-topics sonner of later have to be re-combined in new ways, in order to prevent an overspecialization, and to increase and broaden the applicability of techniques that are developed by the (over?)specialized experts in the different fields.

And I consider "Data Science" as such an approach to combine the expertise and findings from different fields. You described it as

  ...a mix of computer science and statistics techniques

And indeed, several questions here aim at the differentiation between data science and statistics. But a pure statistician will most likely not be able to set up a Hadoop cluster and show the results of his analysis in an interactive HTML5 dashboard. And someone who can implement a nice HTML5 dashboard might not be so familiar with the mathematical background of a Chi-Squared-Test.

It is reasonable to assume that giving students enough knowledge to apply the most important techniques from the different fields that are covered by data science will lead to new applications of these techniques, and be beneficial - also for the "purists" in these fields. The combination of these techniques is not straightforward in many cases, and can justify an own branch of research.

You also asked whether in 10 years, data science will be considered as "just a topic inside computer science". Again: Nobody can say for sure. But I wonder at which point people stopped asking the question whether "Computer Science" will one day only be considered only as a mix of (or a subject of) Electrical Engineering and Mathematics...


Introducing weights into spectral clustering

What happens in Spectral Clustering is simply finding some blocks in the data according to the property for which your matrix is defined. If it's similarity you get the similar data points.

The first one is how to introduce weights when performing spectral clustering. Like in the Shi-Malik, Ng-Jordan-Weiss versions, where only the distances are used.

Do not get confused. All what you need is to introduce your similarity measure which can be weighted. Just insert all the weights into a classical spectral clustering version (Shi-Malik or Ng or whatever) and there you go!

The second one is that my dissimilarity measure does not exists sometimes. In these cases, it has zero weight. How to introduce these cases in the algorithm? I suppose the zero weight solves the problem if I know how to introduce them.

I think this should not be a big issue. Just put zero and try your algorithm with a mock data and see what happens. If didn't work, try to fill missing values by some statistical indicators e.g. mean of the data and see the result.

Please write a comment about how it worked.

Good luck :)


What is the term for data that is too sparse to represent the underlying data model?

Small sample size is probably the concept you're looking for. A common failure in statistics is trying to draw conclusions from data that isn't "big" enough to accurately represent the underlying distribution.

It's worth noting that "small data" increases your chances of overfitting - which is another way of saying that your model is weak to outliers and noise. It's also worth noting that bootstrapping, simulation, and duplication might help (but not always).

There may be some other niche term for this, but if you relate it to basic stats there's a high probability everyone in the room will understand what you're talking about.


How to operate dataset from paperswithcode.com?

This is specified in the dataset description in paperswithcode.com (emphasis mine):

IG-3.5B-17k is an internal Facebook AI Research dataset for training image classification models. It consists of hashtags for up to 3.5 billion public Instagram images.

So the dataset is not public.
In the paper, however, the authors argue that the images and their hashtags are "visible" in Instagram:

Our datasets have two nice properties: public visibility and simplicity. By using publicly accessible images, the data used in our experiments is
visible to everyone. To see what it looks like, the images are browsable by hashtag at https://www.instagram.com/explore/tags/ followed by a specific hashtag;
for example https://www.instagram.com/explore/tags/brownbear shows images tagged with #brownbear. Our data is also taken from the “wild”, essentially as-is, with minimal effort to sanitize it. This makes the dataset construction process particularly simple and transparent.

I understand that they are saying "you can go and query Instagram yourself" to see the images in the dataset, but I don't think this is actually practical or even allowed in their terms of service.


Deep Learning for non-continuous dataset

Deep Learning excels in problems where the data is relatively unstructured. Stacked layers help find conceptual features that can be used to infer rules.

Your dataset seems very structured at first glance. And, as you pose, it doens't look like it needs specialised layers that exploit sequential or spatial relations.

Neural network wise, this would warrant one or two fully connected layers, connected to an output layer (shaped to your wishes). However, typically, problems like these are tackled with a more biased approaches (eg Decision Tree Learners).


RL Advantage function why A = Q-V instead of A=V-Q?

In my understanding, $V(s)$ is always larger than $Q(s,a)$, because the function $V$ includes the reward for the current state $s$, unlike $Q$

This is incorrect. There is not really such a thing as "the reward for current state" in the general case of a MDP. If you mean the $V(S_t)$ should include the value of $R_t$, then this is still wrong, given David Silver's use of the conventions for time step indexing. It is possible to associate immediate reward with either the current time step, leading to sequence $S_0, A_0, R_0, S_1, A_1, R_1$ etc or you can use the convention of immediate reward being on next time step $S_0, A_0, R_1, S_1, A_1, R_2$ etc. David Silver (and Sutton & Barto's book) uses the latter convention.

Under that convention:

$$V(s) = \mathbb{E}_{\pi}[\sum_{k=0}^{\infty}
\gamma^{k}R_{t+k+1}|S_t=s]$$

$$Q(s,a) = \mathbb{E}_{\pi}[\sum_{k=0}^{\infty} \gamma^{k}R_{t+k+1}|S_t=s, A_t=a]$$

You can see that the first term in the expansion of the sum for both Q(s,a) and V(s) is $R_{t+1}$. If you changed the convention, then both would include the equivalent value, but would be labelled $R_{t}$ in any formula.

Q and V do not differ in which time steps they sum reward over. They may differ in the value of $R_{t+1}$ because $V(s)$ assumes following the policy $\pi$ when selecting $A_t$ whilst $Q(s,a)$ uses the value $a$ supplied as a parameter for $A_t$, which can be different.

  how can we be certain what to subtract from what, such that our Advantage is always positive?

Advantage can be negative, that is fine. It means that the action $a$ in $A(s,a)$ is a worse choice than the current policy's.

  I don't see why a value of $(Q−V)$ would be useful. On the other hand, $(V−Q)$ would be useful

Both would be equally useful, it is mainly convention that we work with finding maximum Advantage representing the benefit of choosing a specific action instead of following the current policy, as opposed to finding the minimum "Disadvantage". However, the concept of Advantage in this context is arguably the more natural view.

  because it would tell us the reward we would get on $s_{t+1}$ if we took the action $a$

As explained above, this is wrong. The value in $A(s,a)$ expresses the potential benefit we would get for changing the policy $\pi(s)$. That might include changes to $R_{t+1}$, but is not limited to a single time step.

Some RL approaches do create a predictive function for expected immediate reward $\hat{r}(s,a)$ - typically this is a secondary component, used to help refine parameters for other function approximators.


Softmax: Different output scikit-learn and TensorFlow

The problem turned out to be silly, I just needed more epochs, a smaller learning rate (and for efficiency I turned to AdamOptimizer, results are now equal.

(1681,)
(1681, 2)
SCI-KITLEARN RESULTS:
    Accuracy: 0.822129684711
    Precision: 0.837883361162
    Recall: 0.784522522208
    F1: 0.806251963817

TENSORFLOW RESULTS:
    Accuracy: 0.82213
    Precision: 0.837883361162
    Recall: 0.784522522208
    F1: 0.806251963817


Natural Language Processing (NLP) in .net environment

There are plenty of NLP libraries for .Net if you Google that. Please be more specific as to what you want to accomplish when they query the DB.


Extracting specific data from unstructured text - NER

From your question, I too feel it's a NER problem. And about the dataset, unless there is a data set which tags the reservation numbers and is similar to your application, you WILL have to create your own data set.

I worked on a similar problem before and my dataset looked something like this:

<TEAM>Northern</TEAM>   NNP
<TEAM>Ireland</TEAM>    NNP
man NN
<PLAYER>James</PLAYER>  NNP
<PLAYER>McIlroy</PLAYER>    NNP
is  VBZ
confident   JJ
he  PRP
can MD
win VB
his PRP$
first   JJ
major   JJ
title   NN
at  IN
this    DT
weekend NN
's  POS
<COMPETITION>Spar</COMPETITION> JJ
<COMPETITION>European</COMPETITION> JJ
<COMPETITION>Indoor</COMPETITION>   NNP
<COMPETITION>Championships</COMPETITION>    NNP
in  IN
<LOCATION>Madrid</LOCATION> NNP

You can see that that I have the entity tag and the part of speech tag in the word. When I parse this dataset for training, I also add the IOB tags (Inside, Outside, and Beginning)

[(('Claxton', 'NNP\n'), 'B-PLAYER'),
 (('hunting', 'VBG\n'), 'O'),
 (('first', 'RB\n'), 'O'),
 (('major', 'JJ\n'), 'O'),
 (('medal', 'NNS\n'), 'O'),
 (('.', '.\n'), 'O'),
 (('British', 'JJ\n'), 'O'),
 (('hurdler', 'NN\n'), 'O'),
 (('Sarah', 'NNP\n'), 'B-PLAYER'),
 (('Claxton', 'NNP\n'), 'I-PLAYER')......]

Then I just used the ClassifierBasedTagger(There are other taggers too). I can't find the source but I used this code:

class NamedEntityChunker(ChunkParserI):
def __init__(self, train_sents, **kwargs):
    assert isinstance(train_sents, Iterable), 'The training set should be an Iterable'

    self.feature_detector = features
    self.tagger = ClassifierBasedTagger(
    train = train_sents,
    feature_detector = features,
    **kwargs)

def parse(self, tagged_sents):
    chunks = self.tagger.tag(tagged_sents)

    iob_triplets = [(w, t, c) for ((w, t), c) in chunks]

    return conlltags2tree(iob_triplets)

Here features is a function which returns a dictionary of the features to be used such as the previous word, previous word's pos tag etc. Just features to train the model on.

{
    'word' : word,
    'lemma' : stemmer.stem(word),
    'pos' : pos,
    'allascii' : allascii,

    'next-word' : nextword,
    'next-lemma' : stemmer.stem(nextword),
    'next-pos' : nextpos,

    'prev-word' : prevword,
    'prev-lemma': stemmer.stem(prevword),
    'prev-pos' : prevpos
}

You can find useful theory here

I hope this helps.


I have faced similar challenges. You could try to use the RevolutionR rxDTree function which performs reasonably well on large data sets and can be converted to a rpart object.

I found to achieve a reasonable performance boost by specifying the minimum bin size with rpart.control(minbucket=2). You could also tune the complexity parameter. Note that both may lead to poor performance.


You can still define state value functions $v(s)$, action value functions $q(s,a)$ and policy functions $\pi(s)$ or $\pi(a|s)$ when the state $s$ is from a very large or continuous space. Reinforcement Learning (RL) is still a well-defined problem in that space.

What becomes harder is iterating through the state space. That rules out two simple approaches:

Tabular methods - that store lists of all states with the correct action or value.
Any method that needs to iterate through all states, e.g. the dynamic programming methods Policy Iteration or Value Iteration.

These are important methods for RL. With tabulation and assuming you can iterate through all possibilities, then you can prove that you will find the optimal policy.

However, RL methods can still work with large state spaces. The main method to do so is to use some form of function approximation, which then generalises the space so that knowledge learned about a single state is used to assess similar states.

Function approximation can simply be discretising the space to make the numbers more manageable. Or you can use a parametrisable machine learning approach, such as neural networks. The combination of neural networks with reinforcement learning methods is behind the "deep" reinforcement learning approaches that have been subject of much recent research.

If you use any function approximation with RL, then you are not guaranteed to find the most optimal policy. Instead you will find an approximation of that policy. However, that is often good enough for purpose.

To answer the questions more directly:

  What will be the policy if the state space is continuous in Reinforcement learning

There is no change at the theoretical level. You can express the policy as $\pi(s)$ for a deterministic policy, or $\pi(a|s)$ for a stochastic policy, regardless of the space of $s$.

At the implementation level, you will need to implement a parametric function that takes $s$ as one of its inputs. The function parameters $\theta$ are what is learned. For instance if you use an action value based method such as Q-learning, then you will create an approximation to $Q(s,a)$ - in the literature you may see this directly represented as $\hat{q}(s,a,\theta) \approx Q(s,a)$

Using a neural network for $\hat{q}(s,a,\theta)$ is one common way to achieve this, where the neural network's weight and bias values are in $\theta$.

  What if the state space is continuous, will the agent have information of every possible state in the state space?

Depends what you mean by "have information". The agent cannot possibly store separate data about each state. However, it may have information about similar states, or store its knowledge about states in a more abstract fashion (such as in the parameters $\theta$)

  Also will an RL agent be able to take decision if its in a new state that it has not encountered during training ?

Yes. For this to work well with function approximation, it relies on successful generalisation between similar states. So it is important that the state space representation works towards this. For instance, if two states are close together in the state space representation you use, it should be expected that value function and policy functions are often similar - not always, the function can have arbitrary shape, but trying to learn effectively random mapping would be impossible.


What will happen if almost constant values for features?

If a feature is truly constant across all the instances, then it's useless for classification and it can be removed.
But in any other case, as in your example, a feature can be useful even if constant with most instances. In your example it might turn out that the target class is strongly correlated with type_of_residence, so it would be a mistake to get rid of it since your model can take advantage of it.
Unless you have a good reason to get rid of a feature (expert knowledge), keep all of them. If you must reduce the number of features (for instance to avoid overfitting), then the selection should be based on information about which features are the least informative (e.g. using some correlation measure).
Ideally the best way to know which features are useful is to try to train a classifier and evaluate it with every possible subset of features (but it's rarely possible in practice)


If I understood correctly your question, you want a function that takes a signal (a fixed window) and outputs a 32-bit representation in a way that the correlation between it and any other signal is preserved. Mathematically speaking given signals $s_1$ and $s_2$ in $S$ and a correlation function $corr(s_1, s_2)$ you want some function $f : S \rightarrow B$ (where $B$ is the space of 32-bit binary numbers) that you could use another correlation function for instance $corr_f(b_1, b_2) \approx corr(s_1, s_2)$.

If that is what you want you should at hashing techniques in particular learning to hash. Essentially in hashing want you do is you represent your input by binary numbers in a way that the hamming distance between the binary numbers preserves some target similarity (distance) function such as the correlation. In particular for cross-correlation (inner product) you have some methods based on random projections.

So once you have learned (designed) your hashing function $f$, what I would do is:

b1 = f(s1)
send b1
receive b1
b2 = f(s2)
return h(b2, b1) # this value is going to tell you if the signals are correlated


Probability calibration is worsening my model performance

The probability calibration is just stacking a logistic or isotonic regression on top of the base classifier.  The default is logistic, and since the sigmoid is a strictly increasing function, the rank-ordering of samples will be unaffected, and so AUC should not change at all.

(With isotonic regression, it's actually piecewise-constant, so within a span where the function is constant, all samples will have their scores made equal, and so your ROC curve will become more coarse, which would affect the AUC; but these effects ought to be small, as long as the isotonic fit produces sufficiently many/short constant segments.)

Also, gradient boosting like XGBoost produces scores biased toward the extremes, not away from them like random forests, so the logistic calibration is unlikely to work out well.


Adjusting Probabilities When Using Alternative Cutoffs For Classification

First of all, you cannot always consider what a machine learning algorithm outputs as a "probability". Logistic regression outputs a sigmoid activation on a (0, 1) scale, but that doesn't magically make it so!

We simply often scale things to a (0, 1) scale in ML as a measure of confidence.

Also in your example, if the events are mutually exclusive (like classification), just think of them as "event 1" and "NOT event 1". Something like p(e1) + p(~e1) = 1.

So when your book tells you to lower the threshold, it is simply saying that you require a smaller level of confidence to choose e1 over e2. This doesn't mean you are choosing one with smaller likelihood, you are simply making a conscious choice to adjust your precision-recall curve.

There are other ways to combat class imbalance, but changing the threshold to be more sensitive to any indication of confidence of one class over another is certainly a way to do that.


In CNNs, the convolutional layers are used to extract features in the input in order to reduce the cost function. These extracted convolutional features should be classified using dense layers. Consequently, the use of dense layers is for classifying convolutional extracted features. Based on the complexity of the high level features, extracted features in deep convolutional layers, you can have one or more layers.

Take a look at
How to set the number of neurons and layers in neural networks
 to investigate how to set the number of neurons in dense layers.

  Can I use only one fully connected layer in CNN?

Yes, you can. For instance, you can use only one layer for MNIST data set and get an acceptable learning.

  How to determine the dimension of the fully connected layer output?

The dimension of each fully connected layer is equal to the number of neurons in that layer. For instance, suppose you have a weight matrix $W$ which is $10\times20$. the latter number represents the number of output neurons. Consequently, the output dimension belongs to $\mathbb{R}^{20}$.


Branch of data science that covers event based time series?

"rather than aggregated values at evenly distributed intervals?" I think that part defines your question very well; i.e that is usually what we generally do in usual time-series problems, merging the events in some discrete time intervals.

What you can do is that, you can give the time distance between the events as features, without giving evenly distributed intervals. For example, the exact time distances between events of every disease outbreak or every patient getting sick and etc. Moreover, you can do some feature engineering; squared time differences or percentage changes of time difference between the differences between the sequential doubles (x, x-1) and (x-1, x-2) where x-n,.....,x-2, x-1, x are the sequential events, not the time.

The relevant concept here seems to be "unevenly spaced time-series", the wiki page Wikipedia: Unevenly spaced time series and this researcher's website http://eckner.com/research.html  seems to give enough material to get started with.