The STM32 is a 32bit micro-controller and all of your processing seems to be taking place over a double precision data type.

Doubles are 64bit "numbers" but not in the straightforward sense that 64bit integers are. Therefore, mathematical operations on doubles can take some additional steps (see page 47 onwards) on CPUs that are not equipped with special hardware to handle floating point operations. If the CPU does not have an FPU then mathematical operations are handled in software which means that they take more CPU cycles to complete (For example, you can try double precision multiplications on an Arduino (like Uno), it's going to be fun).

The STM32F767 does have (see page 13) a Floating Point Unit, which means that it can potentially perform double precision operations faster than when these operations are implemented in software and take up CPU cycles. Notice that the FPU is better than not having it but it still has its own limits, for indicative timings see page 16 on this document

BUT, this has to be enabled (see page 20).

Therefore, my guess is that your interrupt routine is striving to fire at the intended 10kHz but it doesn't complete before $\frac{1}{10000}$ seconds. So, your effective sampling frequency is not 10kHz and the whole system is transposed to a different frequency range.

You might want to check if the FPU is enabled at double precision or move to fixed point numbers which every CPU will happily crunch. With fixed point numbers, you have another problem which is that your filter might go out of spec because of the rounding that is inevitable due to the reduction in precision. But these are aspects that you have to manage in your solution.

Hope this helps.


Samples of period to frequency, sampling theorem

No, that is not the reason for aliasing. Due to sampling, a discrete system processes reality at discrete and well defined times. Imagine walking around by rapidly opening and closing your eyes. Obviously, you can navigate the place if you walk around slowly, but if you were to run fast then there comes a point that the environment changes too fast for you to make reliable navigation decisions.

In fact, don't imagine, this is exactly what cinematography cameras are doing. Keep your eyes on the hub bolts of the wheel. Obviously, the wheel turns clockwise but the videoed hub bolts appear to turn clockwise, then go stationary, then anti-clockwise, stop, clockwise, stop, anti-clockwise and so on. Every stopping and reversal is us stradling yet another version of the aliased spectrum as the rate of rotation approaches the camera's frame-rate (The video's Sampling Frequency).

The point is that while your eyes are closed, the camera's shutter is closed, a digital signal processing system's sampling is now "holding", reality does not stop and keeps on happening.

Therefore, it is not that the system "runs out of samples" but more that the system will keep sampling whatever is "in front of it" and in the case of periodic events (such as the case of a spinning wheel), it will start catching the waveform at such points that it would appear that a high frequency is now yet another frequency within the capabilities of the system.

To come back to the "eye flickering" example. Imagine that you are watching a merry-go-round, open eyes it just about crossed 12 o'clock, close your eyes, open: about 3, close, open: about 6, close, open: about 9, close, open: about 11.

Now, imagine that the merry-go-round increases its speed of rotation. There will be a point that the timing will be such that this will happen:

open: 12 o'clock, close (NOW THE MERRY-GO-ROUND MAKES A COMPLETE REVOLUTION BUT ONLY JUST!) open: 11 o'clock, close (NOW THE MERRY-GO-ROUND MAKES A COMPLETE REVOLUTION BUT ONLY JUST!) open: 10 o'clock, close (NOW THE MERRY-GO-ROUND MAKES A COMPLETE REVOLUTION BY ONLY JUST!) open: 9 o'clock, close

While your eyes are closed, you are missing what happens in reality.

This is how the hub appears to roll backwards, or the merry-go-round would appear to run backwards, or an $F_s$ Hz waveform appear as DC (i.e. 0 Hz) when it is sampled bang on on $F_s$ (the sinusoid goes around so fast that we keep sampling it when it is at its positive maximum! -Yes, of course it is phase dependent!-).

In terms of the mathematics behind this, sampling is just a "fancy" modulation (or more generally, the multiplication of two signals, please see this link from this link), because of the "hold". So, if you were to multiply two sinusoids at some $f_1, f_2$, then what you get at the output is one component at $f_1-f_2$ and one component at $f_1+f_2$. This is what creates this folding around $F_s$ if you were to substitute some $f_2$ with $F_s$.

And if we proceed one step further, the $F_s$ waveform is not a sinusoid at $F_s$ but rather a series of spikes at a rate of $F_s$. So, to see how the sampled signal looks like, take your reality signal, say some $f(t)$ (where t is time) and MULTIPLY IT with this spike train.

Multiplication in the time domain equals convolution in the frequency domain and what do you get if you convolve some spectrum with a train of spikes? You get the same spectrum repeated at every spike (in exactly the same reason of why you get echoes, when you convolve with an impulse response of sparse spikes).

And this is how Aliasing emerges. The typical way of controlling for it is to put an analog low-pass filter at the input of the system that attenuates frequencies above $\frac{F_s}{2}$. Therefore, the system doesn't get to "see" (or process) anything (aliased) above $\frac{F_s}{2}$.

Hope this helps.

EDIT:
From the comments to this response, I think that the question was more about whether or not a sampled system runs out of frequency components to represent a waveform because this sampling of the time domain, should be reflected to the frequency spectrum too. In other words, the question is about running out of discrete components to represent a waveform and "could this be the reason for aliasing?"

The answer to this is again no, because, to get rid of the "annoying" half a sample at 28.5 Hz (from the original example) all you have to do is double your observation window. In general, for any non-integer frequency at a given $F_s$, there is an integer number of samples that can contain it. But in that case, we still operate under the same constraints for aliasing.

(When you consider the effect of a window on the components of the Discrete Fourier Transform, then you are thinking about the "resolution" of the transform, but again, in that case, the resolution operates within the constraints of aliasing).


Within the context of Digital Signal Processing...
Upsampling and interpolation are synonyms, expander is not.
Specifically, an upsampler and an interpolator include an expander followed by an anti-imaging lowpass filter. The expander, is the initial stage of an interpolator or an upsampler.
The following is an expander:
$$x[n] \longrightarrow \boxed{ \uparrow 2} \longrightarrow y[n] = \begin{cases} { x[n/2]  ~~, ~~~n = 0,2,4,... \\ ~~~~ 0~~~~~~~ , ~~~n= 1,3,5,...} \end{cases} $$
The following is an upsampler / interpolator:
$$x[n] \longrightarrow \boxed{ \uparrow 2} \longrightarrow w[n] \longrightarrow \boxed{ h[n]}  \longrightarrow y[n] $$
Similarly, downsampler and decimator are synonyms, compressor is not.
The following is a compressor:
$$x[n] \longrightarrow \boxed{ \downarrow 2} \longrightarrow y[n] = x[2n]$$
and the following is a downsampler / decimator:
$$x[n] \longrightarrow \boxed{ h[n]}  \longrightarrow w[n] \longrightarrow \boxed{ \downarrow 2}  \longrightarrow y[n] $$
Specifically, a downsampler and a decimator include an anti-aliasing lowpass filter followed by a compressor. The compressor, is the final stage of a decimator or a downsampler.
Note that you may omit the anti-aliasing lowpass filter when downsampling (or decimating) due to the fact that the input signal is already bandlimited to the band of interest. In such a case, apparently the expander, the downsampler, and the decimator become the same operations; but this is not the general case.
There's no fixed definition of such things but the above is very common.
Authors might prefer using them in different contexts as long they do explicitly describe what they are doing. Informally I may use upsampler and expander in similar places, but the distinction should be clear in my mind.


Confusion implementing chebyshev type 2 band pass filter in matlab

You're using the function cheb2ord in the wrong way. Assuming that your lower stopband edge is $800$ Hz and your upper stopband edge is $2000$ Hz (note that a bandpass filter has two stopband edges, as well as two passband edges), and if the sampling frequency is $6000$ Hz, then the following command should do what you intended to do:
[b, a] = cheby2(10, 60, [4, 10]/15);
Note that for bandpass and bandstop filters, the resulting filter order is twice the number you use in the call to cheby2.


Calculating Delta MFCC, correct formula?

The way you compute MFCCs and derivatives has little-to-no influence on the final result, so just pick one and go ahead. Although when applied to speech recognition and similar problems the same algorithm should be used for training and decoding. Thus if you use a pre-trained model you have to know how features for its training were computed. If not sure, you can always take a look at real-world examples - CMU Sphinx and Kaldi.


Entropy of enhanced image more than the original image

According to me, the above relation should always hold true,

According to the data processing theorem (and here), this is always true.

  how can an enhanced image contain more information than the original image [?]

It cannot. A trivial example is that of interpolation. Suppose that you upsample an image by a factor of 2. This means, inserting a "new" pixel between two known ones. The information remains the same though because no "new light" has been captured. We still do not know what exactly happened BETWEEN the known pixels. All we can do effectively is INTERpolate.

Interpolation works out the values of the unknown pixels based on the values of the KNOWN pixels, therefore re-distributing information across a larger surface. Since all processing takes place over the CAPTURED data it cannot but be inferred that a data processing step can only DECREASE the information content of a dataset.

  I have just seent that E2 will be greater than E1 if we enhance the image using CLAHE technique i.e the entropy of the enhanced image will be greater than the original image.

First of all, you are comparing two entropy measures that appear to have no relationship between them, no shared context.

Having said this, is there any imaginable case in which $E2 > E1$?

Consider the processing step as a black box. It accepts an image ($I$) and it produces an image ($O$), only that in this case, all that this black box does is $O = \alpha \cdot I + (1-\alpha) \cdot N(I)$, where $N(I)$ is a noise process that produces a two dimensional noise image with the same dimensions and overall brightness as $I$. The parameter $\alpha$ determines how much image and how much noise is mixed in the output.

So, at $\alpha = 1$, $O$ is essentially exactly the same as $I$. In this case $E(O) \le E(I)$. BUT, at $\alpha = 0$, $O$ is simply a noise image. An incoherent set of pixels whose entropy is very high. Since there is absolutely no relationship between $I$ and $N$, then it can well be that $E(O) > E(I)$ but this does not mean that anything NEW (AND meaningful) was added to the image.

In the place of $N$ we could also have had extraneous information. For example, image from one camera being cross referenced with an image (of the same scene) from another camera.

But, there is also something else that is at play in your case and this is a "simplistic" view of entropy.

Before we move on to CLAHE, here is a simple experiment. I am using Octave, you can try it in MATLAB or anything else with a bit of re-writing:

n=1500;nBin=16;maxNum=32;
a=round(rand(1,n)*maxNum);
Q = hist(a,nBin)./sum(hist(a,nBin));
H = -sum(Q.*log2(Q))

Where n is the length of a random sequence, nBin is the number of histogram bins, a is a quantised sequence of random numbers up to maxNum, Q is the normalised marginal probability of a and H is the calculated entropy of a.

Now, for a given n,maxNum produce a sequence a and then calculate its Shannon entropy for two different nBin, say for instance nBin=16 and nBin=32.

To recap, SAME sequence of numbers a (absolutely no processing), two DIFFERENT histogram resolutions. What are the two different H?

About 4 for nBin=16 and about 5 for nBin=32. Isn't this a bit suspicious, given that $2^4=16$ and $2^5=32$?

Shannon's entropy gives you the upper bound of the entropy of a source and the way it is derived here, assumes that all the characteristics of the source are captured in Q. Function rand returns numbers from a uniform distribution. If you divide the sequence's histogram at 2 levels, you get 2 equiprobable events and still need 1 bit to describe them (try it, for nBin=2). If you divide the histogram of the SAME sequence at 4 levels, you now get 4 EQUIPROBABLE events for which you need 4 bits to describe them. At 5 levels of the SAME sequence you now get.....and so on.

In this example, a remained the same but the expression of entropy changed, purely as a byproduct of how we calculated it.

This way of calculating the entropy suffers from another problem. That of "empty" bins. If you get a bit more adventurous in the above example and set nBin=500 (for example), you will notice that your H might come back as NaN. Once the histogram contains zero bins, log2(0)*0 produces NaN. Again, we get a result which is purely based on the way entropy is calculated. You might say, "I will omit the zero entries" but then again, you might be omitting them in one image because they did not happen but you would not be able to ignore them in another image because they DID happen.

And THIS is what you see in your case, as a byproduct of how adaptive equalisation works.

Your initial $I$ has a given histogram "shape", a given distribution of grey tones. Most likely this is concentrated around a lobe inside a limited region at the low end. Its entropy will appear low. Yes, you don't need many bits to describe this lobe. AFTER equalisation, the algorithm has now redistributed grey values all over the place, therefore it results in a WIDER histogram (with also differences in its overall shape, with tones that now are not 'negligible') appearing to have a higher entropy. Appearing to require MORE bits to describe the "new" states.

Shannon's entropy is not the only way to calculate entropy. There are other definitions as well, such as those that work over the spectrum of the image or based on data compression.


Question on PCM Sampling and Quantization order

That link provides a very superficial Q & A on signals and systems.
It's not clear which kind of a sampler or quantizer they are referring to.
However, if sampling and quantization are defined as ideal mathematical operators then,
you should get exactly the same result by altering their orders. Hence the sample values should be same (meaning that the same noise also).
May be they're referring to some unstated practical ADC schemes (that I cannot recall) which do not perform ideal operations and might create different results.


NOTE: (Updated noting the sampling rate and desired frequency) Your post describes extracting a 40 Hz signal at a 100,000 Hz sampling rate. Your filtering implementation will be significantly easier (and more effective) if you apply course low pass filtering and decimation to a lower rate before applying the 2nd order tuned resonator derived from an exponential averager as shown in detail below. Please review this post Fast Integer 8 Hz 2nd Order LP for Microcontroller. I suggest decimating to a 200 Hz sampling rate which is a decimate by 500 (in stages such as 5-10-10).

Once decimated, consider using an exponential averager implementation, following this post for converting the lowpass filter to a bandpass, tunable to any frequency including your desired 40Hz:

Transfer function of second order notch filter

The difference between the notch filter in the post above and an exponential averager, is that the exponential averager has the following transfer function:

$$H(z) = \frac{\alpha z}{z+\alpha-1}$$

I think you will see the similarity and be able to follow the process in the linked post above. A diagram of the implementation as an "Exponential Averager" or low pass filter is shown below. This would create a maximum at DC and would need to be converted to a bandpass implementation similar to what was done in the linked post.

UPDATE: In pursuing the transformation myself I came up with the following result:

$$ H(z) = K\frac{z^2}{z^2-2\beta \cos\omega_n z+\beta^2} $$

Where:

(update, adding the closed-form equation for K)

K is a normalizing constant that is dependent on $\omega_n$ and $\alpha$ given as:

$$K= \alpha \sqrt{\beta^2-2\beta cos(2 \omega_n)+1}$$

$\omega_n$: center normalized radian frequency, between 0 and $2\pi$

$\alpha$: bandwidth constant, bandwidth gets tighter as $\alpha$ approaches 0.

$\beta = 1- \alpha$

The resulting implementation as a tuned 2nd order resonator is shown below.

Below are two examples with different bandwidths at center frequency $\omega_n=\pi/3$. The first plot is with $\alpha =0.1$ and the second plot is with $\alpha = 0.001$.

Note that the very long delay this second filter would have as evidenced by the very steep phase slope through the passband (group delay = $d\phi/d\omega$), as required for any tight bandwidth filter. This would also mean a relatively long convergence time. Given the ability to tune bandwidth and center frequency independently, you can envision how this can enable an optimized implementation for certain applications that is initially wide band for fast convergence while the bandwidth tightens over time to maximize SNR for the signal of interest. This structure can also be used as a bandpass loop filter or in a bandpass Sigma Delta implementation for bandpass sampling.

Below is an equivalent form of the exponential averager that only requires one multiplier as Rick Lyons has shown at the site linked below. Here I have chosen $\alpha$ to be similar to what Rick had done, in that $\alpha$ is a small number approaching 0 to make the bandwidth tighter (this is in contrast to my other post linked above for the notch filter implementation where I used $\alpha$ approaching 1 to be a tighter bandwidth, so swapping $\alpha$ with $1-\alpha$ between these two posts. I think the formula is cleaner if you actually use $\alpha$ as the number approaching 1, in that it eliminates my need to use $\beta$ as I have done above without the formulas appearing too cumbersome.

https://www.dsprelated.com/showarticle/182.php


This is what happens when a signal is downsampled: you get a scaling of the frequency axis and the addition of shifted spectra (aliasing). The same happens to the lowpass filtered signal. The spectrum of the downsampled signals is given by Eq. $(4.77)$ in Oppenheim and Schafer's Discrete-time Signal Processing (3rd ed), which for downsampling factor $M=2$ can be written as
$$X_d(e^{j\omega})=\frac12\Big[X\left(e^{j\omega/2}\right)+X\left(e^{j(\omega/2-\pi)}\right)\Big]\tag{1}$$
The second term on the right-hand side of $(1)$ fills up the "empty spaces" in the frequency ranges that are eliminated by the lowpass and highpass filters, respectively.


Is Sum of Absolute Value / $ {L}_{1} $ Norm of Differences Convex?

It is easier to prove it by using atoms.
The 1st atom is the Absolute Value function $ \left| \cdot \right| $ which is convex.
Then you have linear operation by the subtraction which is convex (Also concave).
Then you linear combination which is also Convex.
Hence the function is Convex.


Let Sic be the i-th sample of the c-th channel, you started with stereo (2-channel) 16kHz. Your audio was like this
 s11, s12, s21, s22, s31, s32, s41, s42, s51, s52
When you made it a 4-channel audio you reassigned the same infromation to different channels like this
 s11, s12, s13, s14, s21, s22, s23, s24
In the stereo configuration you were using 2 samples every sampling period, so at 16kHz you play 32000 samples per second. With four channel you use 4 samples every sampling period, if you want to play 32000 samples per second your sampling rate needs to be 32000/4, 8kHz.


Short answer: your conclusion seems ok.

Concept of units make sense only in the world of physics. Mathematical objects such as DFT trasforms do not have any kind of units but just their values, quasi exceptions being the planar angle meausure of radian and volumetric angle mesaure of the steradian. However, when a variable represents a physical quantity (as almost always is the case) then it's customary to associate units with those mathematical variables as well.

Let's do a 1D analysis of those units, from which the 2D version follows straightforwardly. Discrete-time variables are indexed sequences such as $x[n]$ or $X[k]$ which may represent samples of a continuous-time signal $x(t)$ and the DFT of $x[n]$ respectively.

We can assume that a sequence $x[n]$ bears a unit which is the unit of the associated physical variable $x(t)$ from which $x[n]$ was sampled. So if $x(t)$ was a voltage in units of Volt,   then the unit of $x[n]$ will be Volt too.

The index $n$ does not have any units which can be seen by the following uniform sampling relation :

$$ t_n = n T_s $$

where $t_n$ is the sampling time with a unit of second (s), and $T_s$ is the sampling period with a unit of second (s) too; hence the index number $n$ is unitless. However, some like to call $n$ with a unit of samples. Yet this is not only unnecessary but also as meaningless as associating a unit with room numbers in a hotel. Nevertheless it's customary to do so and I do also use it.

The 1D-DFT $X[k]$ associated with a given sequence $x[n]$ is defined as follows:

$$ X[k] = \sum_{n=0}^{N-1} x[n] e^{ j \frac{2\pi}{N} k n} $$

The complex exponential function (as well as other trigonometric functions such as sine, cosine, tangent) produce unitless outputs as pure numbers. The trigonometric functions expects angles in units of radians as their input arguments.

Therefore, the unit of the DFT $X[k]$ will be that of the sequence $x[n]$.

The unit of the DFT frequency bin index $k$ is found from the unit of the complex exponential argument :

$$ \frac{2 \pi }{N} k n = \theta $$

In this expression, the sample index $n$ and the total sample count $N$ (or frame length as some would like to call) have no units (or they both have the unit of samples, but it will cancel neverthelesss) and the angles $\theta$ and $2\pi$ have the unit of radians hence the DFT frequency index $k$ is unitless too.

How about calling the frequency variable $k$ as cycles per frame length ? Ok, it is also preferred and some may find it informative to call $k$ as the number of cycles a sine wave fits into the given frame length of analysis.

With the same reasoning, a 2D intensity image sample sequence $f[n,m]$ will have units of intensity and the spatial index variables $m,n$ are either unitless or given a samples unit. And for the 2D-DFT $X[u,v]$, the frequency index variables $u,v$ are unitless. Nevertheless, again some prefer calling them in the units of cycles per image frame length and you can also do it.

Now coming to the units of the expression

$$ \sqrt{ u^2 + v^2 } < R ,$$

the variable $R$ is either unitless as explained above or can be associated with the unit of cycles per frame length as some would prefer. From your expanded questions, it became clear that you wanted to change the scale of this actually unitless parameter from cycles per frame length to cycles per mm.

Now, to find out the continuous-space frequency in cycles per meter associated with a given DFT frequency bin index $u$ or $v$ , you would actually perform the following conversion :

$$ f_u =  \frac{u}{H} F_s $$

where $u$ is the unitless DFT frequency bin index count, $H$ is the number of samples (frame length in samples or image height in your example) and $F_s$ is the sampling frequency in cycles per meter (I assume). In your project you set $f_s = H$ (assuming $H = \min(H,W)$) and you give a pixel-size parameter of $\gamma$ in mm/pixel units. Then this $\gamma$ is the inverse of sampling frequency $F_s$ in samples per mm. And the above formula becomes :

$$ f_u =  \frac{u}{f_s} F_s = \frac{u}{f_s} \frac{1}{\gamma} = \frac{u}{f_s \gamma} $$

Finally since the ideal circular lowpass filter bandwidth radius parameter $R$ is a same kind of variable with DFT frqeuency bin index variable $u$ or $v$, therefore its conversion from cycles per frame length to cycles per mm will be done using the same formula:

$$ r =  \frac{R}{f_s} F_s = \frac{R}{f_s} \frac{1}{\gamma} = \frac{R}{f_s \gamma} $$

where $r$ has the units of cycles per mm.


What are the practical constraints on designing Sensing matrix in compressed Sensing?

Checking for RIP of a matrix is an NP-Hard problem which means it is not computationally feasible to accomplish. RIP is used in matrix design mostly in theoretical aspects. Stealing @David 's comments, RIP is a tight condition.
To design sensing matrices, another weaker condition named Coherence is used, which its computational complexity is tractable ($O(n^2)$ where $n$ is length of matrix). No matter what type of elements form the matrix, you can always check for its coherence.
A similar question has been asked here:
Compressive sensing: numerical generation of RIP matrices . Check it out for more details.


Resampling won't change the signal duration. It will change the sample rate and the number of samples. And, if you're lowering the sample rate, it will change the frequency components of the signal (if done right, with a proper filter, it will reduce the maximum possible frequency to half the new sample rate).

In order to make your signal "faster" or "slower" you need to use a different technique, time compression/expansion or simply time stretching. The basic idea there is to split the signal into blocks, estimate the frequency components in every block and then replace each block with a block of shorter or longer duration containing synthesized sinewaves of the same frequencies and phase shifts. It appears that currently there's no high quality method for time stretching signals by more than 30%. All cheap methods distort audio signals noticeably to the ear.

What is the problem that you are trying to solve? What's the nature of the sampled signal? Can you expand your question?


What is meant by "Group delay"?in simple words?

Here is my simplest explanation:
The group delay, as the negative derivative of phase, predicts the time delay of the amplitude envelope of a pulse, as shown in the hand-drawn graphic below. The upper part of the sketch shows a sinusoidal waveform varied in amplitude by its envelope. The lower one is showing this same envelope before and after a system that has group delay.
This applies when the phase of the frequency response can be approximated as linear for the "group" of frequencies within the pulse envelope. Thus for non-linear phase systems, this applies to generally narrower band signals such as the pulse I show where the amplitude transition is gradual.
Consider a single sine-wave with the amplitude envelope such as I show. The time delay of the sine wave itself would be predicted directly from the phase of the frequency response (by dividing by the frequency of the sine-wave: with $\phi = angle(H(j\omega))$,  the time delay is $-\phi(\omega)/\omega$), while the time delay of the pulse envelope is predicted from the negative derivative of the phase with respect to frequency ($-d\phi(\omega)/d\omega$)).

These posts and answers are helpful:
Calculate the time delay introduced by group delay for IIR-Filters
https://electronics.stackexchange.com/questions/135475/physical-significance-of-group-delay
And most helpful to what otherwise seems like a paradox of causality for positive group delay is this paper https://www.researchgate.net/publication/253463703_Causality_and_Negative_Group_Delays_in_a_Simple_Bandpass_Amplifier referenced by Max in this post Physical Meaning of Negative Group Delay for causal LTI systems which I bottom line as causality is not violated but due to the bandwidth restrictions above we create a condition that causes the pulse envelope at the output to precede the input: The output pulse does not appear until the input amplitude varies (and if we have gain in the system the output envelope grows faster) and due to the bandwidth constraint and the result of destructive summation of the input pulse the output will start to decrease before the input does. Very cool DSP magic trick. See this post that illustrates this with a specific example.


Solving a Convolution Problem of a 1D Signal

Even though I realize that this is a very late response, I will nevertheless try to answer this question because I find it instructive and also because the number of upvotes suggests that this question is of general interest to the community.

As already suggested in the question, let's define two signals $x(t)$ and $w(t)$ as
$$x(t)=e^{-kt}u(t),\quad k>0\\w(t)=\frac{\sin(\pi t/10)}{\pi t}$$

One possible interpretation of the convolution $(x*w)(t)$ is that an exponentially damped signal $x(t)$ is filtered by an ideal lowpass filter with impulse response $w(t)$. In the question it was also correctly pointed out that convolution in the time domain corresponds to multiplication in the frequency domain. The Fourier integral of $x(t)$ can be easily computed:

$$X(j\omega)=\int_{0}^{\infty}e^{-kt}e^{-j\omega t}dt = \frac{1}{k+j\omega}$$

The Fourier transform of $w(t)$ should be familiar because it is an ideal lowpass filter. In the question there was some confusion concerning the definition of the Sinc function. I suggest to simply remember the impulse response of a unity gain lowpass filter with cut-off frequency $\omega_0=2\pi f_0$ without making use of any of the definitions of the Sinc function:
$$h_{LP}(t)=\frac{\sin \omega_0 t}{\pi t}\tag{1}$$

Comparing (1) with the definition of $w(t)$, we see that $w(t)$ is simply a unity gain lowpass filter with cut-off frequency $\omega_0=\pi/10$:
$$W(j\omega)=u(\omega+\omega_0)-u(\omega-\omega_0)$$ where I've used the step function $u(\omega)$ in the frequency domain.

In order to find the time function $y(t)=(x*w)(t)$ one can compute the inverse Fourier transformation of $Y(j\omega)=X(j\omega)W(j\omega)$:

$$y(t)=\frac{1}{2\pi}\int_{-\infty}^{\infty}X(j\omega)W(j\omega)e^{j\omega t}d\omega=
\frac{1}{2\pi}\int_{-\omega_0}^{\omega_0}\frac{1}{k+j\omega}e^{j\omega t}d\omega$$

Unfortunately, there is no closed form solution of this integral using elementary functions. It can be evaluated numerically using the exponential integral $\text{Ei}(x)$, or, alternatively, the sine and cosine integrals $\text{Si}(x)$ and $\text{Ci}(x)$. So I don't think that the purpose of the exercise was to actually compute the convolution, but its purpose was probably to come up with a qualitative description of what is going on (exponential signal filtered by an ideal lowpass filter).

Nevertheless, I thought it would be instructive to take a look at the signal $y(t)$, so I evaluated it numerically for the parameters $k=0.05$ and $\omega_0=\pi/10$. The following figure shows the result:

The green curve is the input signal $x(t)$ and the blue curve is the filtered signal $y(t)$. Note the (non-causal) ripples of $y(t)$ for $t<0$ caused by the ideal (non-causal) lowpass filter. If we increase the cut-off frequency of the lowpass filter, the distortion of the input signal should become smaller. This is shown in the next figure where I've increased the cut-off frequency by a factor of 10, i.e. $\omega_0=\pi$ (instead of $\pi/10$):


Estimating noise level without 'quiet' segment

If you want to determine if the 2-5 narrowband frequencies are present use a matched filter with a threshold to determine their presence. In matlab the function is xcorr.
If you want to determine what the noise level is when only signal is present, try filtering out the signal. Create a filter for each frequency then convolve the filters together to get a final filter.


How does the PSK demodulator differentiates one carrier from the other in CDMA

To me the transmitters' information is divided in code, but codes can only be separated once they are bits not while they are "pass band symbols", right?

Wrong!

I'm not quite sure how to continue this answer, other than saying, "please re-read the CDMA introduction you've been reading".

The whole idea is that you receive a sequence that is a superposition of the two transmit signals, and by correlation with the known spreading sequences, you can extract the bits from that.

Let's put it more mathematical:

Your spreading sequences are called $c_i$.

Orthogonality is defined via the inner product:

$$a \perp b \iff <a, b> = 0 = <b,a>\text{ .}$$

Now, let's assume perfectly orthonormal sequences, so that $<c_i, c_j> = \delta_{ij}$, i.e. the inner product is 1 only if you multiply a sequence with itself, and 0 for any other sequence.

Corollary, the $c$ span a vector space $C$ which contains all linear combinations of CDMA sequences; its dimension is the same as the number of possible sequences, since these per definition are orthogonal (and hence, especially linearly independent) and form a basis.

Now, let's apply the code. You'll take one symbol $s$, in case of BPSK, one bit, represent it as $\{-1,+1\}=S$, and multiply it to get the transmit sequence $t$. You get the set of possible transmit sequences $t_{i,s}$ for user $i$:

$$T_i = \left\{sc_i \,\forall s\in S \right\}= \left\{ -c_i, +c_i\right\}$$

Notice that $T_i$ spans a vector space that has one dimension less than $C$!

Let's now define a receive signal $r$. We don't know what symbols were sent yet, so we say that user $i$ sent symbol $s_i$, and user $j$ sent symbol $s_j$:

$$r = s_ic_i+ s_jc_j$$

Now, we simply consider $r$ to be a sequence of complex numbers.

What happens now if we multiply $r$ with the individual spreading sequences $c$? Let's do it with user $i$'s code:

$$\begin{align}
<c_i,r> &= <c_i, s_ic_i+ s_jc_j>&\text{linearity of inner product}\\
&= <c_i, s_ic_i>+ <c_i, s_jc_j>&\text{$s_i$, $s_j$ are scalars}\\
&= s_i <c_i, c_i> + s_j <c_i, c_j>&\text{orthonormality}\\
&= s_i \cdot 1 + s_j \cdot 0\\
&= s_i
\end{align}$$

How nice! The same works for every other user, too.

I was a bit pushing the vector space notion here: It's important to realize that all this simply happens due to linear algebra representing our signals. You can now also see why CDMA systems that support a high number of users get slower per user (simply because that dictates a higher number of base vectors of $C$, which dictates a higher dimensionality and thus, longer sequences). Also, it shows that CDMA on this theoretical level works with any linear modulation, since we simply need $S\subset \mathbb C$, i.e. $s$ just needs to be scalar.


You can do something like this in Python (setting data to your original signals):

import numpy as np
data = data1

max_val = np.max(data)
thresh = max_val/2
thresh_crossings = np.where(np.diff(np.signbit(np.array(data) - max_val/2)))[0]
print(len(thresh_crossings)/2)

This code returns 7 for your first signal and 8 for the second one.

First I find the maximum value. Then I arbitrarily set a threshold (in this case, half that value), taking advantage of the fact that the signals are pretty clean and there are not any spurious peaks. Doing so leads to something like this:

Then you just have to find how many times the signal crosses that threshold, and divide it by two (because it will cross it twice for each peak, once when it rises and afterwards when it falls).

If you have any additional information a priori of the frequency of the peaks, you could add some hysteresis to make the algorithm more robust (i.e. if there are small zigzags in the signal, avoid counting them as threshold crossings).


What is the functional difference between DSB and 2SB mixers, and why exactly would 2SB be better for radio astronomy?

Is the separation of the two SBs to two outputs the functional difference between 2SB and DSB? If not, what would it be?

Almost certainly yes, although it seems that the only hits that "2SB mixer" gets is for astronomy sites.  See the figure, from the first paper you cite.  It clearly shows two sidebands being received independently.
Basically, you're getting two SSB channels out of this, which, presumably, can be selected downstream.

The abstract of the second linked paper says that 2SB mixers' "main advantages being to avoid spectral confusion and to diminish effective system temperature by a factor two with respect to double sideband (DSB) receivers." Is it possible explain in fairly simple terms what that is so?

Spectral confusion: DSB mixers take the sideband above the carrier and the sideband below the carrier and mix them equally to baseband.  The output of a DSB mixer is the sum of the downconverted USB and the downconverted LSB.  For uncorrelated signals, the power in the USB and the power in the LSB are added.
Noise temperature: Noise temperature is a function of the ratio of noise power to desired signal power.  Because the DSB mixer output power is the sum of the LSB and USB power, it's output has the noise contribution from both of these sidebands.  Because (based on the context of the article) the desired signal is in only one sideband, you're getting the noise power from two sidebands, and signal from one.
With a SSB mixer (remember you get two SSB channels out of the 2SB mixer) you get the noise power from one sideband, and the signal from one sideband.  So the power/noise ratio is doubled -- this results in the noise temperature being halved.


Detrend data with no clear secular trend prior to Fourier analysis?

How you pre-process data is related to the purpose of the analysis and since you haven't mentioned why you are doing spectral analysis, answering your question can't go beyond making some general remarks.

Detrending is a preprocessing operation. You want to compensate or remove some bias from your data. As an example, many kinds of data are collected in the presence of long term phenomena that is extraneous to what your are interested in finding. Many kinds of data have seasonal, diurnal, weekly, monthly, tidal, .... ect cycles associated with them. Unemployment has a seasonal dependence to it as an example.

I once was tasked with looking for jumps in syndrome data that is reported to health departments and there was strong increases on Mondays and after holidays in reports but that had to do with many doctor's offices being closed on weekends.  This was an artifact of how the data is collected and not how epidemics evolve.

Matlab's dtrend function removes a linear fit from data.  If the collection interval is small compared to some much longer trend in the data, it reduces the bias introduced by taking a section that is less than the period of the longer term trend, particularly for data like your which is strictly positive.  One may conceptualize dtrending as reducing low frequency spectral leakage that would otherwise mask features in your spectra.

Again, without knowing what you are looking for in your data, no specific answer can be recomended for your situation.


Extremely computationally intensive image/video compression schemes which offer better compression than industry standard JPEG/H.264

There are various aspects of compression - whether you are referring to image or video. (May be audio is totally different so i am not referring this here).

If you really look at the history of compression standard, when they were first formed - MPEG1 real time encoders were rare. CPU at that time itself was not enough to make it real time; however, it was not the processing complexity (alone) that stopped more complexity algorithms to be used under codecs. (Probably the primary factor that was responsible for inclusion/exclusion of certain algorithms were a. patents, b. feasibility of implementation).

However, i would like to re-phrase the question a bit more simpler:

  What other research in compression is available which is still to come
  to market (or didn't come at all but might be worth for some
  applications)?

Here are some points to look at:

1. Better transform?

DCT by far has been universal transform across all codecs. However, there are many research papers which suggest that Wavelets could be better. Indeed the only JPEG2000 included wavelets - but that standard itself wasn't big hit. Probably it was better than DCT in some way; specially there are claims that Wavelets outperformce DCT [See: Ref 1]

I think wavelet is computationally expensive over DCT (partly because 8x8 DCT implementations are highly optimized). But i guess i didn't really flew that much because from the market perspective because DCT by the time was far more wide spread and the difference in improvement was not so substantial for people to switch.

2. Multi hypothesis motion compensation
While MPEG 2 was happy enough with $B$ pictures, extensive research from Bern Girod [See Ref 2] proved that - the concept can be extended way beyond this. Indeed H.264 actually has made great generalization on multiple target based motion compensaion - basically one can store past 16 pictures (compare to 2 in MPEG2) and indeed the prediction can be extremely high.

While, this is already part of the standard in H.264 - this is still not fully exploited. There is a need to make such extensively large search for motion estimation practically viable. Most H.264 encoder are still nascent in exploiting full potential of this.

3. Scalable Image/Video compression
Scalable Image and video compression is again a holy grail of how multiple description encoding can be done for image and video. Basically the same Image or video can be encoded at multiple Resolution, quantization (quality), or frame rate. These produces different layers of quality vs. bit rate right within the same stream - and the streaming server can actually adapt the bitrate and quality dynamically to best fit the video in the quality.
See [Ref 3], [Ref 4], [Ref 5]

Of this, only real success is progressive JPEG which is practically useful for websites and indeed useful. Again while the promise of dynamically adapting bit stream is great but it works only if there are practical systems where estimating such bit rate is possible and type of adaptation can serve some real purpose. However, in video domain, in spite the support of this in the standards (MPEG2, MPEG4 and H.264) - there is hardly any usage of this.

4. Object Video
This is perhaps ultimate holy grail of Compression. Object video [Ref 6] as initiated by MPEG standard body itself however, this was also backed by extensive research.

The basic idea is to segment the video in terms of distinct objects and later on each such object can be treated as separate plane. each object and be applied with different bit rate, quantization, frame rate and much efficient prediction can happen. Also, one can manipulate such objects like the way you can do stuff in photoshop/gimp.

However, as it turns out,  the segmentation of real-world image and video with perceptual relevant objects - is a hard problem! So even if i do have encoder/decoders - i need robus and real time segmentation system to make things possible. This is by far a really open problem as to how do you apply video in the form of such layered video to make it work.

5. More dimensions
Apart from this - there are codecs for stereo video (a.k.a multiview) and 3D video (almost similar concept) which additionally try to reduce the redundancy in the additional dimension. While this is still research going on here, H.264 does have a profile for this now. Same is true for the Cinema encoding.

There are many research projects still in the area of this - however, video compression is quite a commodity market by now. And research which doesn't substantially generate new application or if there are feasibility issues in implementation wont survive practically in spite it's ingenuity.

Hope this helps.

References:

A Comparative Study of DCT- and Wavelet-Based Image Coding by Zixiang Xiong, et. al. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 9, NO. 5, AUGUST 1999 pp. 692
Efficiency Analysis of Multihypothesis Motion-Compensated Prediction for Video Coding
Bernd Girod IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 9, NO. 2, FEBRUARY 2000 173
Spatial-domain resolution-scalable video coding by Barry G. Haskell, Hsueh-Ming Hang. Proc. SPIE 2094, Visual Communications and Image Processing November 718 (1993);
The MPEG-4 Fine-Grained Scalable Video Coding Method for Multimedia Streaming Over IP
by Hayder M. Radha, et. al. IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 3, NO. 1, MARCH 2001 53
Advances in Scalable Video Coding by Ohm, J.-R.; Proceedings of the IEEE Jan. 2005 Volume: 93 Issue:1 pp. 42 - 56
Object-Based Texture Coding of Moving Video in MPEG-4, by Andr ́ Kaup, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 9, NO. 1, FEBRUARY 1999 pp. 5


This link implies that .wav files have several standards for storing 32 bit floating point.  It looks like foramt 32-bit 16.8 float (type 1 - 32-bit) would do what you want.

As best I can see, SOX (reference) can work (read it at least) with this type of format, so you could look at the source code to see how to generate/read these files.  CoolEdit claims to be able read and write this format (see first link.)


Polyphase components Spectrum formula in Discrete time

Let's start with your equation.

$$ P_k\left( e^{j\Omega} \right) = \frac{1}{L} e^{jk\Omega/L} \sum_{p=0}^{L-1} e^{-2 \pi j k p / L} H\left( e^{j(\Omega-2\pi  p)/L} \right) $$

Do a little rearranging.

$$ P_k\left( e^{j\Omega} \right) =  e^{j\Omega\frac{k}{L}} \frac{1}{L} \left[ \sum_{p=0}^{L-1} H\left( e^{j\Omega\frac{1}{L} }  e^{-j2\pi \frac{p}{L}} \right) \left( e^{j2\pi/L} \right)^{-kp}   \right] $$

It can be made simpler to understand by a few variable substitutions.

$$ y = e^{j\Omega} $$

$y$ is a complex value on the complex unit circle at $\Omega$ radians around the circumference.

$$ h[p] = H\left( y^{\frac{1}{L}} e^{-j2\pi \frac{p}{L}} \right) $$

$h[]$ is a sequence of values sampled from the $H()$ function.  $y^{\frac{1}{L}}$ is a complex value on the unit circle at $1/L$th the angle of $y$.  The $e^{-j2\pi \frac{p}{L}}$ is a clockwise walk around the unit circle in $L$ even steps.  Conceptually:

$$ e^{-j2\pi \frac{p}{L}} = \left( e^{j2\pi } \right)^{-\frac{p}{L}} = 1^{-\frac{p}{L}} $$

Therefore $h[]$ has a period of $L$.  If $p$ were to go below zero or above $L-1$, it would be a repeat pattern.

With these substitutions, your equation becomes this:

$$ P_k\left( y \right) =  y^{\frac{k}{L}} \frac{1}{L} \left[ \sum_{p=0}^{L-1} h[p] \left( e^{j2\pi/L} \right)^{-pk} \right] $$

The part in brackets is the definition of the DFT.

$$ X[k] = \sum_{n=0}^{N-1} x[n] \left(e^{i2\pi/N}\right)^{-nk} $$

It's arranged the same way as in my answer here Fractional powers of complex numbers (DSPrelated computation)

The $\frac{1}{L}$ is a normalizing factor making the magnitudes of the DFT calculation independent of $L$.

$ y^{\frac{k}{L}} $ is a complex value $ k/L $ of the way to $y$ along the unit circle which means the results of the DFT bin are rotated that far.

So, a long answer to your question.  The purpose of the second exponential $ e^{-2\pi j k p/L} $ is to execute the DFT transform.  It is part of the definition and the reason it works the way it does.  Lots of stuff available on that if you are unfamiliar.


Offset QPSK detection in GNU Radio - Sample delay

Thanks to Marcus' answer, I was able to resolve the problem. See below for the updated code:

IQ Mapping

#ifdef HAVE_CONFIG_H
#include "config.h"
#endif

#include <gnuradio/io_signature.h>
#include "oqpskIQMap_impl.h"

namespace gr {
  namespace oqpsk {

    oqpskIQMap::sptr
    oqpskIQMap::make(int sps)
    {
      return gnuradio::get_initial_sptr
        (new oqpskIQMap_impl(sps));
    }

    /*
     * The private constructor
     */
    oqpskIQMap_impl::oqpskIQMap_impl(int sps)
      : gr::sync_block("oqpskIQMap",
              gr::io_signature::make(1, 1, sizeof(gr_complex)),
              gr::io_signature::make(1, 1, sizeof(gr_complex))),
        d_delay(sps/2)
    {
        //set_history(1);
    }

    /*
     * Our virtual destructor.
     */
    oqpskIQMap_impl::~oqpskIQMap_impl()
    {
    }

    int
    oqpskIQMap_impl::work(int noutput_items,
        gr_vector_const_void_star &input_items,
        gr_vector_void_star &output_items)
    {
      const gr_complex *in = (const gr_complex *) input_items[0];
      gr_complex *out = (gr_complex *) output_items[0];

      for(int i = d_delay; i < noutput_items; i++)
          out[i] = gr_complex(real(in[i]),imag(in[i - d_delay]));

      // Do <+signal processing+>

      // Tell runtime system how many output items we produced.
      return noutput_items - d_delay;
    }

  } /* namespace oqpsk */
} /* namespace gr */

IQ Demapping
Please note that this is a downsampling block and which expects 2 samples per symbol and produces 1 sample per symbol (a single constellation point).
    #ifdef HAVE_CONFIG_H
    #include "config.h"
    #endif

#include <gnuradio/io_signature.h>
#include "oqpskIQDemap_impl.h"

namespace gr {
  namespace oqpsk {

    oqpskIQDemap::sptr
    oqpskIQDemap::make(int delay)
    {
      return gnuradio::get_initial_sptr
        (new oqpskIQDemap_impl(delay));
    }

    /*
     * The private constructor
     */
    oqpskIQDemap_impl::oqpskIQDemap_impl(int delay)
      : gr::sync_decimator("oqpskIQDemap",
              gr::io_signature::make(1, 1, sizeof(gr_complex)),
              gr::io_signature::make(1, 1, sizeof(gr_complex)), 2),
        d_delay(delay)
    {
        set_output_multiple(2);
    }

    /*
     * Our virtual destructor.
     */
    oqpskIQDemap_impl::~oqpskIQDemap_impl()
    {
    }

    int
    oqpskIQDemap_impl::work(int noutput_items,
        gr_vector_const_void_star &input_items,
        gr_vector_void_star &output_items)
    {
      const gr_complex *in = (const gr_complex *) input_items[0];
      gr_complex *out = (gr_complex *) output_items[0];

      //for(int i = 0; i < noutput_items; i+=2)
          //out[i] = gr_complex(real(in[i]),imag(in[i + 1]));
      out[0] = gr_complex(real(in[0]),imag(in[1]));

      // Tell runtime system how many output items we produced.
      return 1;
    }

  } /* namespace oqpsk */
} /* namespace gr */

Regards,


FIR Filter Design - input parameters conversion

As I've mentioned in a comment, the Parks McClellan algorithm is usually used to design frequency selective filters with a fixed maximum stopband error, which results in an equiripple behaviour in the stopband. Note that the algorithm can in principle approximate any desired frequency response shape. However, many implementations just allow for piecewise constant desired responses (which includes all standard frequency selective filters, such as low pass, high pass, etc.).

If you want a frequency response that decays monotonically from the cut-off frequency, you probably want something like a Butterworth filter. This is an IIR filter which is usually designed via the bilinear transform from an analog prototype filter. If you want an FIR filter, then one option would be to truncate (or window) the infinite impulse response of the Butterworth filter.

In Matlab/Octave this can be easily done as follows. First design a digital Butterworth filter with the given specifications, compute its impulse response, and truncate it. Where to truncate depends on the error that you allow. For your example a third order Butterworth filter is sufficient. In the following example I assume a sampling frequency of $8\,\text{kHz}$:

[b,a] = butter(3,1.6/4);    % Butterworth IIR filter
h = impz(b,a,30);           % impulse response (30 coeffs)
f = logspace(0,3.6,400);    % log. frequency grid [1,4000]Hz
H = freqz(b,a,f*pi/4000);   % Butterworth frequency response
Hf = freqz(h,1,f*pi/4000);  % FIR frequency response
semilogx(f,20*log10(abs(H)),f,20*log10(abs(Hf)))
axis([1,4000,-80,5]), grid on

You see just a single curve because both filters (IIR and FIR) have virtually the same magnitude response.


Confusion with data after cross correlation

I had done something like this and ran into similar problems a while ago, using Matlab. The audio to the microphones was speech or whistling in my case. I was estimating the delay between two mics to find the direction sound was arriving from.

The audio data from the mics probably has oscillations of some kind (like a sum of sinusoids plus some noise), at least at certain locations. The cross-correlation of one such waveform with a delayed version of it will be a signal with oscillations. This would look like this example, but possibly decaying slowly. I've seen something similar in this question.

The cross-correlation peak should indicate the delay, but there are multiple peaks so you may be detecting the wrong one. Since your input signal is not a pure sine-wave, the largest peak should be the delay but it would be hard to guarantee that in a hardware system. This may be why you're getting inconsistent results.

Since you know the distance between the two microphones (and the system's sampling rate), you can estimate the maximum expected delay (in a number of samples), as the time it would take for sound to travel the full distance between the mics (0.6 meters). Then you should search for the cross-correlation peak only in the first such number samples, rather than in the full waveform.

Low-pass filtering may also be helpful if you're just interested in the delay. The higher frequency components can create peaks in the cross-correlation sooner (oscillations with a smaller period). In my case, I had found that a low-pass cut-off of about 1.0-1.5 kHz was adequate since my signal (speech) was wide-band and contained enough information at low frequencies for the delay estimation.


Detecting text lines in image by means of Hough Transform

OpenCV now has a text detection module included. You might want to take a look at it:

Detector: http://docs.opencv.org/master/modules/objdetect/doc/erfilter.html
Recognizer: http://docs.opencv.org/trunk/modules/text/doc/ocr.html


How does subpixel image shifting using DFT really work?

There are several key insights you need in order to understand how DFT allows you to shift an image.

First, Fourier's theorum: It's probably easier to look at the continuous (i.e., analog) case first.  Imagine you have some function, call it g(t).  For simplicity, let's say that g(t) is an analog audio recording, so it's a one-dimensional function, which is continuous, and represents the instantaneous pressure as a function of time.

Now, g(t) is one way we can represent our audio recording.  Another is G(f).  G(f) is the Fourier transform of g(t).  So, G(f) == FT(g(t)).   G(f) has all the same information as g(t), but it represents that information in the frequency domain instead of the time domain.  There are some nit-picky details about Fourier Transforms, which I won't mention.

You can sort of think of G(f) as the "distribution of frequencies" contained in g(t).  So, if g(t) is a sine-wave (i.e., a pure tone), then G(f) will be zero everywhere, except at the frequency of that tone.  This is probably a good point to mention that G(f) is in general a complex function--that is to say that it returns complex numbers, which can be thought of having a real and imaginary component or a magnitude and phase.

One small digression here:  Since g(t) is continuous (both in domain and range), G(f) is also continuous. So, how can G(f) be zero everywhere except for the tone frequency?  Well, FT(sin(wt)) = $\delta(w)$.  Where $\delta$ is the Dirac delta function.

Ok, so now we've got continuous FT's under our belt.

Here's the second insight:  A Discrete Fourier Transform is to a Fourier Transform as a sampled signal is to an analog signal.  In this case, the
"discrete" refers to quantization of the function's domain (time or frequency), not it's range.  (The sampled digital signal you get from your sound card is quantized in both domain and range.)

The digital byte-stream you get from your sound card contains "samples" of the original continuous (analog) signal from the microphone.  If we take the DFT of our sampled g(t), we still get a G(f).  G(f), remember, is just a different way of representing the information contained in g(t). If we obeyed Nyquist's theorum, the sampled signal g(t) contains all the "intelligence" of the original continuous signal, so our discrete G(f) must contain all the information from our original continuous signal. Parenthetically, G(f) is still a complex function.

This is where the magic of sub-pixel shifting comes in, but in this case I'm going to write about shifting the audio signal in time by less than a sample, since it's the same thing.

Remember how G(f) is a complex function?  It needs to be complex in order to represent frequencies that aren't zero at t=0.  Remember, sin(0)=0, so sin(2*0)=0, etc.  But what if we started recording a quarter of the way through a cycle of the tone?  That's where the phase part of G(f) comes from.  In this case, the phase would be 90 degrees or pi/2 radians, depending on your preference for representing a quarter of a cycle.  So G(tone_frequency) = 0 + i or $e^{{i \pi}\over{2}}$.

That means we can shift our audio recording in time (by any amount we choose, including a fraction of a sample time) simply by modifying the phase of G(t). Actually, that statement is perhaps a bit too casual.  For an un-quantized, sampled signal the phase can be adjusted arbitrarily (this is part of the reason I made the distinction between quantization of domain and range earlier).  However, for a quantized sampled signal (our byte-stream of audio, for example) the quantization step size (i.e., number of bits) determines the resolution with which we can adjust the phase.  When we Inverse Fourier Transform G(f) (or DIFT it, for this sampled signal), the new set of samples g'(t) = DIFT(G(F)) will all be shifted in time by the amount we pick.

Applying this to your pixels simply means using a 2-dimensional FT instead of the 1-dimensional FT discussed here.


How to recognize an object from a small training set of images?

Few reasons i could think of are:

Size of your training set is very small. Larger training sets have always been the key for accuracy.
Each algorithm will have some drawback like SURF is not good at viewpoint change and illumination change. So if the test images vary from the training images in case of illumination and viewpoint, results on those data will be poor.I don't know how FLANN works.


How to compute the statistics of the DFT of correlated noise?

After many weeks I give the answer to my own question.

There is a limit in which we can solve this problem in a reasonably simple way. Suppose we sum enough points in our DFT that the central limit theorem guarantees that the distribution of the sum's real and imaginary parts are Gaussian distributed. Then we only need to compute the variance. If we specialize the case of the variances of the real part of the DFT we can write

$$
\langle (\textrm{Re} X_k) (\textrm{Re} X_l) \rangle = \frac{1}{N^2}\sum_{n=0}^{N-1}\sum_{m=0}^{N-1}\langle x_n x_m \rangle \cos(2\pi n k / N)\cos(2 \pi m l / N)
$$
The thing to note is that $\langle x_n x_m \rangle$ is just the correlation function of the time domain samples, which we can denote $\rho(n-m)$. Putting this in, we get

$$
\langle (\textrm{Re} X_k) (\textrm{Re} X_l) \rangle = \frac{1}{N^2}\sum_{n=0}^{N-1}\sum_{m=0}^{N-1} \rho(n-m) \cos(2\pi n k / N)\cos(2 \pi m l / N)
$$

This form in terms of the correlation function is useful because the correlation function is frequently known; By the Wiener-Khinchin theorem the correlation function is the Fourier transform of the spectral density.

This sum can be computed numerically, or even analytically for some particular forms of $\rho$. To compute covariance with imaginary parts of $X$ just put $\sin$ instead of $\cos$.

I found the idea for this in J. Schoukens and J. Renneboog, IEEE Transactions on Instrumentation and Measurement, Vol. IM-35, No. 3, September (1986).


Fourier Transform: $\omega$ vs $f$ as frequency variable

You have the obvious relationship $\omega=2\pi f$. So going from a Fourier transform expressed in terms of $\omega$ to a Fourier transform expressed in terms of $f$ is just a matter of substitution.
The problem you've encountered is very likely in the argument of the Dirac delta impulse. Clearly, $\delta(\omega)=\delta(2\pi f)$, but you need to know the following relationship in order to obtain an expression in terms of $\delta(f)$:
$$\delta(ay)=\frac{1}{|a|}\delta(y),\qquad a\neq 0\tag{1}$$
So we have
$$\delta(\omega)=\delta(2\pi f)=\frac{1}{2\pi}\delta(f)\tag{2}$$
E.g., the Fourier transform of $x(t)=e^{j\omega_0t}=e^{j2\pi f_0t}$ is given by
$$\begin{align}\mathcal{F}\big\{e^{j\omega_0t}\big\}&=2\pi\delta(\omega-\omega_0)\\&=2\pi\delta\big(2\pi (f-f_0)\big)\\&=\delta(f-f_0)\tag{3}\end{align}$$
I'm sure that from this you can derive the corresponding expressions for $x(t)=\cos(\omega_0t)$.


Choices of convention and notation for the Fourier transform?

The choice of convention should be the one most appropriate (or familiar) for the audience you are trying to communicate with.


How to learn image processing / computer vision in a practical way?

Digital Image Processing. Rafael C. Gonzalez, Richard E. Woods. - it is the classical book on Image Processing, that is not bound to any library.

To translate theoretical knowledge into practical knowledge you can start with the book Mastering OpenCV with Practical Computer Vision Projects, which can be downloaded here.


The key idea is that the random sampling approach enforces more constraints on the resulting signal than the uniform sampling approach does.

The POCS (projections onto convex sets) algorithm used for the reconstruction of the randomly sampled signal is the key piece: it enforces:

That the signal must be from this spectrum.
That the signal is real-valued.
What we know about the signal's spectrum (the randomly sampled Fourier coefficients).
What we know about the signal's time-domain form.

The uniform sampling approach makes no attempt to enforce the bold constraint.

Here's an example which shows:

Top left: Uniform undersampling and reconstruction using the FFT only.
Top right: Random undersampling and reconstruction using the FFT only.
Bottom left: Reconstruction using POCS

As you can see, that final constrain really improves the reconstruction hugely.

R Code Below

#30219

T <- 128
N <- 5

x <- rep(0, T)
x[sample(T,N)] <- rep(1,N)

X <- fft(x);
Xu <- rep(0, T)
Xu[seq(1,T,4)] <- X[seq(1,T,4)];
xu <- fft(Xu, inverse = TRUE)*4/T;

par(mfrow=c(2,2))
plot(x, type="l", lwd = 5, ylim = c(0,1.2))
lines(abs(xu), col="red")
title('Original (black) & reconstructed\n from uniform undersampling (red)')

Xr <- rep(0,T)
r_ix <- sample(T,T/4)
Xr[r_ix] <- X[r_ix]
xr <- fft(Xr, inverse = TRUE)*4/T

plot(x, type="l", lwd = 5, ylim = c(0,1.2))
lines(abs(xr), col="red")
#lines(Re(xr), col="blue")
#lines(Im(xr), col="green")
title('Original (black) & reconstructed\n from non-uniform undersampling (red)')

#soft thresh function

softThresh <- function(vals_to_threshold, lambda)
{
  ix <- which(abs(vals_to_threshold) < lambda)
  vals_to_threshold[ix] <- rep(0, length(ix))

  ix <- which(vals_to_threshold >= lambda)
  vals_to_threshold[ix] <- vals_to_threshold[ix] - lambda

  ix <- which(vals_to_threshold <= -lambda)
  vals_to_threshold[ix] <- vals_to_threshold[ix] + lambda

  return(vals_to_threshold)
}

# POCS
lambda <- 0.1
Xhat <- Xr
for (iteration in seq(1,100))
{
  # 1. Compute the inverse FT to get estimate
  xhat <- Re(fft(Xhat, inverse = TRUE)/T)
  # 2. Apply Softrhesh in the time domain
  xhat <- softThresh(xhat, lambda)
  # 3. Find the FFT
  Xhat <- fft(xhat)
  # 4. Enforce known values
  Xhat[r_ix] <- X[r_ix]
}

plot(x, type="l", lwd = 5, ylim = c(0,1.2))
lines(xhat, col="red")
title('Reconstructed using POCS')


Estimate Parameters of Linear Combination of Harmonic Signals with Partial Known Frequencies

Bottom Line: The OP is experiencing the effects of spectral leakage and the primary motivation for windowing with the Discrete Fourier Transform. The reason the windowing approach achieves much better estimation accuracy is explained as follows:
The OP's equation is a sum of real sinusoids, and his estimator to determine those sinusoids is a form of the Discrete Fourier Transform (DFT). Without additional windowing (which is a rectangular window) it is well understood that the DFT suffers from a considerable amount of spectral leakage, in that the Kernel of the rectangular window (the Discrete-Time Fourier Transform or DTFT of the rectangular window which is of the form of an aliased Sinc function) convolves in frequency with the underlying two impulses in frequency of each real sinusoid. Since the aliased Sinc function decreases relatively gradually with frequency offset from the location of the impulses representing the tones, and has such high side-lobes, many other frequency bins are affected causing errors in their own measurements (and a significant decrease in dynamic range or the ability to detect a lower level tone in the presence of a strong one). Windows are selected that have the property of much lower side-lobe levels, and roll-off much faster in frequency, which therefore causes any given tone to have much less impact on the estimation of other tones present.
This is made much clearer if we observe the DTFT instead of the DFT, as the DTFT is a continuous function of frequency and the DFT is samples on this continuous function. By observing the DTFT we can more easily get the full intuitive picture of what is being described above. Further we also see how approximating the DTFT (through zero-padding prior to doing the DFT) can also be used to approximate the frequencies). I show this with plots of the OP's original frequencies where we can see the effect of the windowing directly:
First consider the OP's example of 1024 samples with just F1 alone with and without additional windowing. First without additional windowing (rectangular window) The log magnitude plot of the DTFT with the corresponding 1024 DFT samples of just F1 is given in the plot below:
This is specifically the DTFT of
$$sig1 = 1.0\cos(2\pi f_1 n) + 0.5\sin(2\pi f_1 n)$$
for n = 0 to 1023.  And f1 = 0.123
(Note: the above is just a single tone at $f1$ with additional phase shift as can be derived from the relationship $a\cos(x)+b\cos(x) = \sqrt{a^2+b^2}\cos(x-\tan^{-1}(b/a))$
Magnitude Spectrum of F1 Signal Only - Rectangular Window

If we zoom in on one of the spectral peaks, we can clearly see the Sinc shaped Kernel (in this case given the number of samples the aliasing would be very small so this is a very close approximation of an actual Sinc) that has convolved with the underlying impulse that would be at the F1 frequency location, and we see the spectral leakage and how it affects the other frequency locations. F1 as 0.123 is very close to bin number 126 as given by $1024 \times .123 = 125.952$.
Zoom In on F1 Peak Location

Below I show the two frequency components as the OP has used, but still as separate signals to demonstrate how the spectral leakage of each will effect the measurement of the other.
This is specifically
$$sig1 = 1.0\cos(2\pi f_1 n) + 0.5\sin(2\pi f_1 n)$$
$$sig2 = 0.05\cos(2\pi f_2 n) + 0.01\sin(2\pi f_2 n)$$
For n = 0 to 1023 and f1 = 0.123 and f2 = 0.312874
Magnitude Spectrum of F1 and F2 Signals (Separate) - Rectangular Window

Notice if we zoom in on the peak for F2 how significant the leakage from F1 is will effect the estimate of the magnitude for F2, given that the leakage from F1 is only 29 dB lower! If the magnitude of F2 was made lower, the resulting error would be even more pronounced. (Also notice how with the DTFT we can locate the F2 frequency precisely at its fractional bin location of $1024 \times 0.312874 \approx 320.3830$ which is where the peak of the orange curve in the plot below would be located, but once combined with the F2 signal given by the blue curve, this would cause a frequency error as well since the peak of the blue curve happens slightly to the right of the orange F1 curve.
Zoom In on F2 Peak Location - With Rectangular Window

If we then compute the DTFT of the composite waveform with both F1 and F2, we see the resulting amplitude and frequency error in vicinity of the F2 Peak location as shown in the plot below (where Green is the DTFT of the composite waveform summing both F1 and F2 components).

This error is predicted by the coherent sum of the two waveforms and thus can be predicted from rectangular window Kernel and knowledge of the relative amplitude and frequency offsets of the two signals. Given the waveforms are complex the phase of the components will impact the summation result (thus the inteference could feasibly reduce or increase the amplitude of the affected component).
In this example the following were the signal contributions at the F2 peak location (not in dB scale as above to show the summation directly).
F2 alone at bin 320.38: magnitude = 52.10 (given as 52.00 -j3.26)
F1 alone at bin 320.38: magnitude = 1.70 (given as 1.42-j0.94)
F1 + F2 at bin 320.38: magnitude = 53.58 (given as 53.42-j4.19)
Where F1 + F2 as measured is predicted by the sum of F1 and F2 alone when the complex components are added.
Finally if we observe the above plot with the use of windowing (we'll use the same Hann window the OP used) we can see the significant reduction in spectral leakage and then exactly how this would contributed to a better estimation of both the frequency and magnitude of the underlying signals.
Magnitude Spectrum of F1 and F2 Signals (Separate) - Hann Window

Zooming in on the F2 spectral peak now shows that the leakage from F1 is now over 100 dB lower, but we also see the drawback of windowing- the main lobe of the Kernel is wider leading to reduced frequency selectivity. (The rectangular window has the best selectivity with the narrowest possible main lobe, but the worst dynamic range due to the high sidelobe level and slow roll-off).
Zoom In on F2 Peak Location - With Hann Window

For more info on DTFT vs DFT and approximating the DTFT with zero padding see Getting the DTFT from the DFT samples and For 2D signals can it be said that the frequency response is the same as the Fourier transform?
For additional details on windowing see:
Why would one use a Hann or Bartlett window?
For additional details on spectral leakage see: Intuition for sidelobes in FFT
Suggestions for Efficient Processing
Assuming the objective is to only find the magnitudes for each of the tones, if 0.01 dB accuracy is sufficient, a suggestion is to use a flattop window, which provides nearly 100 dB of sidelobe rejection while maintaining a very flat main lobe in frequency extending out to $\pm 0.5$ bins. This means no further interpolation would be necessary to estimate the magnitude of a tone located at a fractional bin frequency to get an estimate within 0.01 dB of the actual magnitude.
Below shows an example of the Kernel (DTFT) of a flattop window with just 128 samples, showing nearly 100 dB of sidelobe rejection reached at a 5 bin offset.

Zooming in to see the flatness, we see that with 128 points we maintain less than 0.01 dB flatness over $\pm 0.5$ bins.

If the $0.01$ dB magnitude accuracy was acceptable, the solution to minimize processing would be to choose the maximum DFT size required to ensure no two tones are spaced closer than 5 bins based on the minimum tone proximimity (a 58.7 dB leakage which occurs at approximately 4.6 bins would cause a maximum amplitude error of 0.01 dB). This would set the minimum number of samples to process, after which can be multiplied by the flattop window and fft'd where the maximum bins would be sufficient to provide the amplitude estimates.
Here is an interesting paper with further information on designing the flattop window to any passband error specification (at the expense of sidelobe performance), as well as other optimized flattop solutions: Design and Improvement of Flattop Windows with Semi Infinite Optimization.


An Interesting Model with Unknown Orthogonal Design Matrix

We find such signal models in radar literature. For example, $\Gamma$ represents the transmitted signal information matrix, $\mu$ is the deterministic unknown vector representing the reflections from the target, $z_i$ is a random vector generated by the clutter surrounding the target, and $e_i$ is the receiver noise. In statistical literature, such models are called linear mixed models. You can make several assumptions, such as orthogonal signal matrix, unknown clutter covariance matrix etc., and solve a hypothesis testing problem where the goal of the detector is to detect if there is a target or not. In other words, $\mu = 0$ vs $\mu \ne 0$.


First you need to interpolate between samples rather than just retaining and discarding samples (which introduces horrible jitter noise during non-integer downsampling).

Then, if the original audio data contained spectrum around or above 4000 Hz (half the sample rate), you will need to low-pass filter before or in conjunction with the interpolation to the lower sample rate, or else you will get aliasing.

A good combined method of low-pass filtering plus interpolation is to use a windowed Sinc function as the interpolation kernel, where the Sinc is scaled to be the transform of an ideal "brick-wall" or rectangular filter with the appropriate low-pass cut-off frequency.  The width of the window on the scaled Sinc function will control the transition width of the frequency cut-off.


getting frequency response from input and output signal

What you are looking for is probably an empirical transfer-function estimate (ETFE). Several methods can be used, but Welch's averaged periodogram [1] is arguably the most used [2].

The plant output data is usually generated using Gaussian white-noise excitation, although more informative input signals can be generated by experiment design, if prior information about the plant is known [3]. The ETFE of the plant $\widehat{G}(k)$ is found as the quotient of the cross power spectral density estimate of the input and the measured output $P_{yu}(k)$, and the power spectral density estimate of the input $P_{uu}(k)$, i.e.,
\begin{equation*}
 \widehat{G}(k) = \frac{P_{yu}(k)}{P_{uu}(k)} .
\end{equation*}
In Welch's method, the time-series data is divided into windowed segments, with an option to use overlapping segments. Then, a modified periodogram of each segment is computed and the results are averaged. Welch's method for generating an ETFE corresponds to the function tfestimate in MATLAB. One of the advantages of Welch's method is the flexibility in terms of the number of frequency samples and excitation signal used.


What is role of BW and frequency in internet speeds?

Are my assumptions correct?

no, sorry.

  (BW) It is the amount of data that can be transferred simultaneously.just like more no. of highway lanes so that many cars can travel side by side simultaneously.

We need to talk about different meanings of the word "Bandwidth".

In networks technology, that often means "number of bits per second that this system transports". To avoid confusion, I call this "data rate" instead of bandwidth.I don't think your "highway number of lanes" analogy works very well, because it breaks down as soon as you start thinking about packets, latencies, congestion. I'd simply work with "number of bits per second", that is simple enough that it doesn't need any analogy, does it?
In signal theory, "bandwidth" means "the size of the frequency range occupied by a signal".
For example, a FM radio station has a bandwidth of ca 75 kHz, and if it's at 100 MHz center frequency, that means it's occupying spectrum from 99.9625 MHz to 100.0375 MHz.Another example: human voice has an audio bandwidth of ca 4.5 kHz, it spans the range from ca 40 Hz to 4.5 kHz (rough numbers, really).

Both definitions aren't interchangeable: For example, you can transport 1 MB/s over a 1 kHz wide channel, if your signal quality is good enough and you've got the right technology to make use of that, or you can at most transport 1 kB/s over a 400 MHz channel, if that channel is bad enough.

  It decides the speed of the data. So high frequency is like each car travelling at high speed.

That is totally wrong. I can't even slightly correct it to make it seem right.

  do higher frequency message signals require higher frequency carrier signals?

Higher frequency message signals typically mean "higher bandwidth message signal", and these need a carrier that is at lease high enough so none of the message signal mixed onto the carrier ends up at a negative frequency. That's all.

  So if we have both high BW and high frequency we can achieve higher data rates than either of them acting alone.

No, it's only the signal bandwidth together with the channel quality (especially: SNR) that matters for the achievable data rate. The whole point of modern communications technology is that the carrier frequency doesn't matter to the message signal.


Signal recovery from multiple noisy sensors

Your ability to achieve coherent gain is going to depend on a number of factors and your upper bound on gain is 10log10(N)dB, where N is the number of sensors.

The essential idea is that you need to align your signals in time and add them resulting in a superposition.  The signals need to be nearly identical at each sensor output.

In beamforming, there is a notion of signal source located at some distant point and plane waves are incident on the sensors.  One can infer the anticipated time delays from geometry.  One can also assume other wavefront geometries such as spherical wave incidence.

Effects such as polarization and sensor orientation complicate the similarity of the received signal at each sensor.

Some signals have multiple propagation paths and these paths aren’t all going to be free space paths. Seismic signals have pressure and sheer wave propagation.

The assumption that noise is independent at each sensor isn’t a given for every application.

These are all general considerations and you need to offer more detail on your problem if you want more help.  The 10log10(N) upper bound should apply in any case.


What is intuitively a syndrome in error correction?

The word syndrome was introduced into error-control coding theory
and practice (by Hagelbarger) very early on. It is a term borrowed from
medicine: a syndrome is a group of symptoms, e.g. stomachache, nausea,
fever, lack of appetite, which together are diagnostic (or highly suggestive)
of the presence of a specific disease or condition -- in this instance, appendicitis.  Note that the presence of a single symptom is usually not
considered diagnostic of a specific disease, it is the presence of the
syndrome or a collection of symptoms.
This notion in medicine is almost exactly mirrored in
the error-control coding world. If the syndrome is identically
zero (there are no symptoms whatsoever), nothing needs to be
done, and the received word is accepted as correct (no disease
is diagnosed and no treatment is prescribed).
A nonzero syndrome indicates that the received word has errors
in it (the presence of symptoms means that something is not right
with the patient), and in most cases, the syndrome is also diagnostic
of the locations of the errors and what needs to be done to correct them
with the (Reed-Solomon) decoder substituting for the physician's
thought processes (from syndrome, find the error locations and values)
and actions such as prescribing medicines or performing surgery
(correct the errors, fill in the erasures, etc.).  In some
cases, the fact that the syndrome is nonzero indicates the presence
of errors but the decoder is not able  to determine the error
locations and values (and so is unable to correct the errors).
This is called decoder failure.  In even fewer cases, the syndrome
indicates a specific error pattern as the most likely cause but in fact the
actual error pattern is something else. Correcting what the decoder
thinks is the error pattern introduces more errors into the
received word, thus making it "wronger" than before, but
this is not detectable at the receiver. This is not called decoder
malpractice but decoder error or undetected error, and is,
fortunately, a very rare event, much rarer than decoder failure.


Why can't the callee whom I am on call with hear the sound coming from my phone's speaker but I can while my phone is on loud speaker?

First of all, welcome to DSP SE.
In general most (if not all) smartphones nowadays have some kind of noise cancellation technology integrated. The simplest of all is to have two microphones, one which is supposed to record the noise and the other one record the voice with the noise (please keep in mind that this is an oversimplification of the problem). Then by subtracting the one from the other you end up with the voice without the noise.
Nowadays, there are some more sophisticated techniques to perform denoising such as echo-cancellation, active-noise-control schemes and many more, or even neural-network-based denoisers. In addition to that, the software running in the background may also perform some kind of speech recognition in order to get the speech components of the signal. Alternatively, some source-separation techniques could be used, but to be honest I am not sure how effective they could be.
All in all, the algorithm running in the background tries to separate your voice from the background (effectively, any other sound) and transmit only your voice. Most of the aforementioned techniques are adaptive filtering techniques, which means that they present a time-variable response. On top of that, they most probably will affect your voice. This is quite easy to experience yourself. Just try to move your phone around while you speak (or the person who you talk to should do that instead for you to hear the difference) or start and stop some sounds in the background abruptly.
I hope that this answer provides some insight into the general scheme of the process running in the background while you perform either phone or video calls. I know that this may not be a direct answer but it is a bit difficult to get an exact answer without providing the exact model and conditions (and to be honest I doubt you would still not be able to know exactly how it is done since this most probably constitutes the "magic" the company integrates into their products).


Does it make sense to use a wavelet that is equal to a sine of one period?

With all due caution, no in both cases (title and body question). I'll start with the second one.

Continuous wavelets use all dilations of the mother wavelet, which are not accessed with the STFT
The STFT is complex in general, and the windowed sine is not.

For the first one: I  never tried it, and do not remember having seen it in use, and one should check first whether it satisfies the admissibility condition (first thought: seems OK on 0 in Fourier, and decays at infinity, I'd expect families of synthesis wavelets). You can compute the effects in Fourier using SE.Math: Fourier transform of 1 cycle of sine wave.

However, the sine falls abruptly at its edges, without differentiability, in other words it is not smooth enough. So I would not expect a  more interesting interpretation  of the scalogram than with other continuous wavelet.

But it could be used as an exploratory tool (analysis) for some specific signals. You can think about  hybrid methods as well: try to fit (in more or less robust ways) a parametric sine model, at different scales.


Based on the spectrum, it looks to me like this was generated by FM synthesis. The mathematics of starting with a sound and working backwards to generate it using FM synthesis can be time consuming. The best reference I know of is Computer Music. I've seen pdf editions of this book online that I thought were legal, but I'm only seeing illegal looking things now, so I'm not linking.

Some simpler approximations that might work, but almost certainly won't sound the same:

white noise with a bandpass filter. (maybe plus a little clipped sine wave)
a clipped sine wave or square wave through a low pass filter. (subtractive synthesis)


How does filter bank work in image processing?

Sorry, I couldn't access the paper behing the paywall, so here is an educated guess about the figure that you linked in the question.

$H_0$ is a low-pass filter, while $H_1$ is a high-pass filter. You can note that the union of both domains correctly yield the full bandwidth of the input signal.
These filters are directional, i.e., they operate on different subbands that correspond to a set of orientations. This is shown in the figure by the 1,2,3,4 triangles: each triangle corresponds to a set of low/high-frequency of a given set of orientations. For example, 1 corrresponds to lower frequencies, and the slope of the lines in 1 is positive.
Finally, the directional filter bank consists in two operations: separatig low/high frequencies (the filter bank part) and applying a transfer function that also depends on the orientation of the current frequency (the directional part).

I'm note sure that you can implement the figure from this explanation though. A good starting point now would be for you to look for an example code that does similar things.


Series Sum Operator Properties and Identities

That equation is almost correct. Tim gave the correct solution in a comment, which is just a scaled version of the given formula. I'd like to show how you can quite easily see this.
First, note that
$$\sum_{m=-N}^N\phi(n-m)=(r\star\phi)(n)\tag{1}$$
where $\star$ denotes convolution, and $r(n)$ is a sequence of ones in the range $n\in[-N,N]$. Second, summing a sequence is also a convolution with the sequence $r(n)$ evaluated at $n=0$. So we can write
$$\begin{align}\sum_{n=-N}^N\sum_{m=-N}^N\phi(n-m)&=(r\star r\star\phi)(n)\Big|_{n=0}\\&=(s\star\phi)(n)\Big|_{n=0}\tag{2}\end{align}$$
where $s(n)$ is the convolution of two rectangular sequences, which is a triangular sequence:
$$s(n)=(2N+1)\left[1-\frac{|n|}{2N+1}\right],\qquad n\in[-2N,2N]\tag{3}$$
Combining $(2)$ and $(3)$ gives the desired result, which equals the formula in your question, scaled by $2N+1$.


Physics-based sound synthesis is a pretty multidisciplinary topic.  Quoting from the preface of Physical Audio Signal Processing:

  [T]he material of this book is multidisciplinary, building on results
  from physics, musical acoustics, psychoacoustics, signal processing,
  control engineering, computer music, and computer science. Such
  diversity is typical of applied research.

It wouldn't really make sense to have it be some kind of subfield of math or physics; computer graphics is in a similar position.  Physics-based sound synthesis is mainly numerically simulating a physical model.  The trick, as usual, is making approximations to arrive at something reasonably efficient while not losing too much of what is important.  This is how fields like psychoacoustics and computer science come in.  There isn't a mathematical theory that will tell you what approximations won't produce unpleasant sounds and will run efficiently on a SHARC DSP.  The constraints are too ad-hoc.  Of course, you can put the constraints of a particular problem into a general mathematical theory of approximation and optimization to help derive solutions.


Limiting bandwidth by adjusting sample rate

Depends on your signal. Typically we try to sample at the Nyquist rate, which is equal to two times the maximum frequency of your signal. If you sample less often than this, you will lose information and your answer will not be correct, I think.


The result if order of two filter are reversed

Formally, (linear) derivatives and convolutions may commute, as explained on Wikipedia-Convolution/Properties/Differentiation. This is a major "operation-saving" property: if one wants to differentiate many images and convolve then with a fixed kernel (here a Gaussian), one can convolve the images with the differentiate kernel.
You don't have to differentiate all images, just the kernel once.
Caveat: what is written above could be taken with care, notably when derivatives don't exist theoretically, or with subtleties in implementing derivatives.


Matlab: White noise with flat/constant power spectrum

You need to average a bunch of FFTs of white noise to approach the average power spectral density.

see this and this.


How can I implement a binary chirp signal containing zeros and ones?

What is a chirp signal?

A chirp signal is a sinusoidal signal of the form $\sin(bt+\frac{1}{2}at^2)$ whose instantaneous radian frequency (the derivative with respect to time of the argument of the sine, viz. $b+ at$, is a linear function of $t$. If the chirp signal is of
duration $T$, then the instantaneous frequency at the end is $B=b+aT$.
If $a > 0$, the frequency increases
with time from the initial value $b$ to its maximum value $B$ at $t = T$,
while if $a<0$, the frequency decreases from
$b$ downwards to the minimum value $B$ at $t = T$.

Is the Fourier transform of a chirp signal constant over
the frequency range $[b,B]$  (or $[B,b]$ for pessimists who choose $a < 0$)?

No, the Fourier transform is not constant over this
frequency range.  Nor does the DFT have equal amplitude
in all bins if we sample the chirp signal and compute its DFT.

What is a fake chirp signal?

A fake chirp signal is one whose instantaneous radian frequency
increases in steps of equal height $\Delta = (B-b)/n$ but decreasing
durations so that it consists of one full period of $\sin(bt)$
followed by one full period of $\sin((b+\Delta)t)$ followed by one
full period of $\sin((b+2\Delta)t)$, $\ldots$, followed by one full period
of $\sin((b+(n-1)\Delta)t) = \sin((B-\Delta)t)$.  Note that the total
duration is
$$T = 2\pi\left[\frac{1}{b} + \frac{1}{b+\Delta}
+ \cdots + \frac{1}{b+(n-1)\Delta}\right]$$
To maintain phase continuity, we insist that each full period start
and end at $0$ amplitude and $0$ phase so that, for example, between
$t=\frac{2\pi}{b}$ and $t=\frac{2\pi}{b}+\frac{2\pi}{b+\Delta}$ when
we have a sinusoid of radian frequency $b+\Delta$,
the exact mathematical expression
for the signal is $\sin((b+\Delta)t+\theta)$ where $\theta$ is chosen
so that $(b+\Delta)\frac{2\pi}{b}+\theta$ is an integer multiple of $2\pi$.

What is a binary chirp signal?

If you create a chirp signal, sample it, and hard-limit it to $\pm 1$
(or, as you prefer, shift the dc level so that the signal is $0$s and $1$s)
then you can call it a binary chirp signal, but you shouldn't
expect it to have the properties one expects of a chirp
signal.  In any case, it is necessary to have $b,B \ll f_s$
where $f_s$ is the sampling rate to get something reasonable.
If you don't want to go through the hassle of creating a sampled
chirp signal, etc. you can get something that can be called
a faker  binary chirp signal very simply.  Here are
two examples.

$$111110000011110000111000110010~\text{or}~1111100001110010$$

The first sequence is effectively one period of a sinusoid with $10$ samples
per period followed by one period of a sinusoid with $8$ samples
per period followed by one period of a sinusoid with $6$ samples per period,
etc.  The second sequence has half periods of sinusoids with decreasing
numbers of samples per period.  I call this a faker binary chirp
signal because instead of the frequency of the sinusoid
increasing in constant steps, it is the period of the sinusoid
that is decreasing in constant steps.  So we don't have the linear
frequency sweep that characterizes chirp signals.  But the signal is
trivial to synthesize, and if $b, B \ll f_s$, the difference between
a fake binary chirp and a faker binary chirp can be small.

I don't read MATLAB but if x = 1:0.1:1e3; means that it is desired
that the frequency sweep from $1$ Hz to $1000$ Hz (or by a factor of $1000$
in general), then the digital sequence (whether sampled from a true chirp
pr a fake chirp or synthesized as a faker chirp) is going to begin with
long strings of $1$s followed by long strings of $0$s.  Matters
will be even worse
if a frequency resolution or step size $\Delta$ of $0.1$ Hz is required.


How to check if a signal is mainly composed of a few impulse peaks?

You can use kurtosis as a measure of how 'peaky' your signal is. Or the flatness measure, which is the ratio of the geometric mean of the signal to its arithmetic mean. Any signal which is not relatively flat will have a flatness value near 0.
$$\text{Flatness} = \frac{\sqrt[N]{\prod_{n=0}^{N-1} x(n)}}{\frac{1}{N} \sum_{n=0}^{N-1} x(n)}$$


generate M random complex QAM symbols. example: (1+j,1-j,-1-j,1-j....)
  Split my M samples into a N 2048 sized arrays
  Take IFFT of each array individually
  Add cyclic prefix to each array

Exactly! In real-world OFDM systems, not all carriers are used; typically, you leave out one or two DC carriers and leave a few guard carriers at the band edges, so you'd have something like

  split my M samples into 1950-sized chunks, and map them to the elements of a 2048-vector, leaving the center ones and edge ones free

The edge guard carriers make the fifth step unnecessary:

  Make N square shaped signals by oversampling my IFFT arrays

so scratch that, usually.

  Pulse shape the individual signals with raised cosine

Don't do that! That's wrong. This is OFDM, and it's only used when you encounter multipath channels; matched filtering does nothing good here; it only makes things worse by making the channel more complicated and reducing average power.

Remember why you'd do raised cosine with single carrier systems: You want things to be

nicely pulse-shaped, which simply doesn't make sense in the context of OFDM, and
you want matched filtering because you want to maximize SNR in an AWGN channel, and OFDM is never used in channels that are purely AWGN, and
because after convolution with the conjugate pulse shaping filter in the receiver, you might have an easier time recovering timing, but that doesn't apply to OFDM either, because timing recovery must, in the wideband channel scenario, be done using other means (keywords: Schmidl-Cox, autocorrelation timing recovery methods).

  Mix up the signals with frequency separation = 1/symbol_period by multiplying them by $e^{j2\pi ft}$

No; your subcarrier separation is what the IFFT does; you just mix up the baseband signal generated by step 4 to bandpass with your $e^{j2\pi ft}$.

Since you're only simulating things, and we know that bandpass-region channels have an equivalent representation in baseband (in fact, only because we know that works is that we can use OFDM at all): don't do this at all, but work with the complex baseband signal directly.


Create Matched Filter Reference Signal when Scaling Unknown

I think the problem is not as bad as you suspect it is.

I wasn't around at the time, but from what I've read, early radar systems essentially connected the matched filter's output to an oscilloscope, and a trained operator would look at the phosphor and decide, from experience and intuition, when the signal raised above the noise ("the grass") indicating a reflection.

In practice, you never know in advance the amplitude of the pulse you're looking for. There is attenuation in the channel that can change over time, even on very short time scales. Sometimes you don't know how much power the transmitter is using.

There are at least two solutions to this problem. One is to use automatic gain control in the receiver, so that you know in advance the power or the peak amplitude of the pulse going into the matched filter. The second is to use a training sequence or a pilot signal, so that the receiver can calibrate the thresholds appropriately.

To add a bit of detail: to simplify the math, assume the signal $p(t)$ is time-symmetric and has energy 1. The transmitter emits signal $kp(t)$, with $k$ unkown at the receiver. The receiver correlates the received signal with a matched filter with impuse response $lp(t)$. When the transmitted signal is present at the receiver, the matched filter's output is $$r=kl+n,$$ which is a Gaussian random variable with mean $kl$ and variance $N_0/2$.

When nothing but noise is present at the receiver, the matched filter's output is just $$r=n,$$ which is a Gaussian random variable with mean 0 and variance $N_0/2$. If the receiver can estimate $l$, then the problem reduces to standard hypothesis testing (see for example these notes).


Convolution of two rectangular pulses intuition

A convolution integral is an overlap integral, i.e., for any given shift of the two aperiodic functions being convolved, the convolution integral is simply the overlap area. McGillem and Cooper [1, p. 58] defined the convolution integral of $x_1$ and $x_2$ as

$$\mathrm {x_3 =x_1*x_2 =\int_{-\infty}^{\infty}x_1(\lambda)x_2(t-\lambda)\,\mathrm d\lambda \tag{1}}$$

As a simple graphical illustration of the defining integral, they considered the following two rectangular pulses:

With $x_1$ and $x_2$ as shown in the above figure, their convolution is shown in the figure below:

This figure is redrawn from [1, p. 59]. The shaded areas are the overlap areas as a function of the shift, $t$, and the resulting convolution has a trapezoidal shape. If the rectangular pulses had had equal width, then the convolution would havec simplified to an isosceles triangular shape.

1  C.D. McGillem, G.R. Cooper, "Continuous and Discrete Signal and System Analysis", 2nd Ed., Holt, Rinehart and Winston, NY, ©1984, pp. 58-59.


What are some efficient algorithms for real-time calculation of eigenvalues?

papers?

why not books

Golub, Gene H., and Charles F. Van Loan. Matrix computations. Vol. 3. JHU Press, 2012.

and if you need something more basic like understanding the difference between the Holder p-norm of a matrix and the Frobenious norm.

Stewart, Gilbert W. Introduction to matrix computations. Elsevier, 1973.

Although there are probably some exceptions, the symmetric square matrix is the most common in application in signal processing.

Typically, you calculate the eigenvectors and the eigenvalues follow.  I'm not aware of any algorithms that just calculate the eigenvalues.

The eigenvalues are sometimes called the matrix spectrum.

It would actaully be hard to not find a numerical linear algebra library that didn't compute eigenvector and SVD of a square matrix.

A popular book for  those who don't want to wade in deeply is

Press, William H., Saul A. Teukolsky, William T. Vetterling, and Brian P. Flannery. Numerical recipes 3rd edition: The art of scientific computing. Cambridge university press, 2007.