UPDATE: My previous response did not answer the OP's question. The following addresses the question directly:

Bottom line: Prior to windowing in time, each sample in frequency is an IID Gaussian random variable since the Fourier Transform of an AWGN waveform in time results in an identically distributed waveform in frequency (Gaussian distributed and to be white meaning each sample is independent of the next). After windowing in time, a dependence is created between the adjacent samples in frequency. But the overall frequency response will still be white (uniform and equal power overall) and Gaussian. The variance of a sine wave in relation to the variance/Hz of the white noise process (variance for an AWGN process must be given as a density in units of power/Hz as a truly white noise process has infinite power) will be unchanged in relation to each other; if the window caused the power of the sinewave to go down by one half, the power of the noise would also go down by on half. The actual values depend on how normalization is done in the computations, but for a straight power computation which is energy/time, reducing the window by one half (for example) would reduce the power by one half independent of what kind of waveform was involved (Sine, AWGN, etc).  This is in contrast to what would happen if we convolved with a rectangular window, which is covered in the second half of the post below (what was my original, but misguided, response).

Details:

For discrete time signals, consider the following from Parseval's Theorem which shows that the energy of the signal in time and frequency is the same:

When time goes from $-\infty$ to $+\infty$ which would be for the DTFT:

$$\sum_{n=-\infty}^{\infty}|x[n]|^2=\frac{1}{2\pi}\int_{-\pi}^{\pi}|X((e^{j\phi})|^2d\phi\tag{1}$$

Note when using normalized frequency (1) becomes the form below that is perhaps easier to follow:

$$\sum_{n=-\infty}^{\infty}|x[n]|^2=\int_{-0.5}^{0.5}|X(f)|^2df$$

When time is limited (windowed) would be for the DFT:

$$\sum_{n=0}^{N-1}|x[n]|^2=\frac{1}{N}\sum_{k=0}^{N-1}|X[k]|^2\tag{2}$$

In the above DFT relationship using Parseval's Theorem we are comparing energy; if we further scale by M where M represents the total observation time in samples, we will then be comparing power under various rectangular window sizes of N samples which we can apply to both sinusoidal tones and white noise:

$$\frac{1}{M}\sum_{n=0}^{N-1}|x[n]|^2=\frac{1}{M}\frac{1}{N}\sum_{k=0}^{N-1}|X[k]|^2\tag{3}$$

The DTFT case will not converge without any window applied (infinite energy) but we can get insight into the answer by considering an arbitrarily large window (the DFT) and then comparing that to what happens when we reduce it with a smaller window.

Sine Wave

Consider a sine wave with an arbitarily long window N with an observation time that also equals N:

If the window is indeed very large compared to a cycle of the sinewave, then the DFT of the sine wave will be well approximated by two impulses (as is the case exactly when the window is an integer number of cycles of the sinewave) each with a magnitude that is N/2 times the peak magnitude of the sine wave in time. Thus for a sine wave with an arbitrarily long window, Parseval's theorem results in the expected variance of a sine wave with peak $A_p$ (using M=N in Equation (3)):

$$\frac{1}{N^2}\sum_{k=0}^{N-1}|X[k]|^2 = \frac{1}{N^2}\left( \left(\frac{N}{2}A_p\right)^2+\left(\frac{N}{2}A_p\right)^2\right)=\frac{A_p^2}{2}=\sigma^2$$

As we reduce the window for the sinewave, the frequency response of the sine wave is indeed "smeared" to other bins; the impulses will become Sinc functions in frequency that will get wider as the window gets narrower, and the total power when considering the squared sum of all bins will go down as the ratio of N/M where M represents the original window size. Note that the total power of the original window size M will change in both domains if the residual fraction of a sine wave cycle becomes significant compared to the integrated area under one cycle squared, as is the case when the window duration is not significantly longer than one cycle of a sine wave. If we were considering a single complex exponential frequency tone, this variation as the window size became significantly reduced would not occur. However to be noted in either case, the power in time is equal to the power in frequency regardless of window duration and frequency of the tone (the power in both is equally effected).

AWGN

An additive Gaussian white noise process in time is an additive Gaussian white noise process in frequency, with the same distribution in both domains. (So therefore as far as a mathematical function it is just a change of variable from time to frequency when using a unitary Fourier transform). Let's also remind ourselves of what AWGN is conceptually: It is white, meaning it has equal power density over ALL frequencies (and therefore unlimited power and therefore not realizable), and Gaussian- meaning the distribution of its magnitude in time takes on a Gaussian shape. The Fourier transform of a Gaussian white process is also a Gausssian white process; what does that mean? In the frequency domain, the distribution in magnitude of the function versus frequency also takes on a Gaussian shape and in this case in terms of it being "white" it means explicitly that the transform of this function (the time domain function) has equal power over ALL time. Bottom line, as far as we are concerned, besides the variable defining the domain, the functions are identical.
With regard to Fourier transforms, multiplying by a window in one domain is convolution of the window kernel (Fourier Transform of the window) in the other domain. When we filter a signal, we convolve the signal with the impulse response of the filter, which is the inverse Fourier transform of the frequency response.  Further to be noted when working with the DFT as we have done above, the convolution itself is a circular convolution.

With that said, consider what would happen to the frequency response of an AWGN process when we window it in time: Prior to windowing, which is the case of an arbitrarily long window N with an observation time equal to N, the frequency response is indeed white, and as we noted above the "time response" is also similarly "white" in this case (meaning it extends over the full length with all the samples having a similar distribution). Also to note, relative to our sample time interval, each sample in time is uncorrelated from the next (therefore the resulting in a spectrum over our digital frequency interval that is indeed white). The variance of our time domain signal is equal to the variance of our DFT when we scale the DFT by N=M as shown in (3).

Just as in the case of the sine wave, if we reduce the rectangular window M to be less than M, the power (variance) will reduce by N/M, but what is interesting and pertinent to the question, is that the frequency response will remain white and Gaussian!  Why is this? By reducing the rectangular window to M, we are convolving the frequency response with a Sinc fucntion (or in our discrete system what well approximates a Sinc function for large M and is actually an "aliased" Sinc function), and as noted this is a circular convolution. Thus the frequency response would still be white, but to be noted we have created a dependence for each sample in frequency on adjacent samples due to the convolution operation. This means in frequency each sample is no longer independent from sample to sample, so in the time domain the transform will no longer be white- but in the frequency domain the amplitude distribution itself will still be Gaussian, and the power density will still be uniform over all frequencies within the digital frequency interval used and so therefore is is indeed still white in frequency.

Thus the impact of a rectangular window in time to the frequency domain is to remove the independence between the adjacent frequency samples, and reduce the overall power proportionally when compared over the same observation interval (equally as is done with a sine wave, so does not change SNR); but it does not change the statistical description of being white (in frequency) and Gaussian distributed.  The dependence between samples in frequency is similar to the effect of a dependence of samples in time: When we have a dependence between samples in time we have a band-limited (low pass filtered) process which we can therefore say is "frequency limited". When we have a dependence between samples in frequency we have a time-limited process; which is what the rectangular window is doing.

As a final point to help see what is going on; sometimes it easier to think in one domain instead of the other, so consider if we applied the rectangular window to any AWGN signal in frequency that is initially white (uniform density over all frequencies). Prior to windowing - the time domain signal would extend over our complete observation interval, and the DFT would extend over the complete frequency space defined by our sampling time interval.  When observing the signal in time, no matter how much we zoomed into the time domain waveform, it would appear as in the first plot below for AWGN, because every sample is independent of the next. And the historgram of the magnitude distribution is Gaussian. If we were to band-limit the frequency response (by multiplying the frequency response with a rectangular window), we would see in the time domain something similar to the second plot below; in that as we zoom in, we can see defined trajectories from one sample to the next! Note that the histogram of the magnitude (as long as we do it over enough samples) does not change and is still Gaussian. And important note that our time domain function still extends over our complete observation time with a uniform power- so it is "white" in time and Gaussian but it is no longer white in frequency. Thus we see directly what would happen to the frequency response in the case of the OP's question. Instead of the waveforms below being time, they would be frequency. The frequency response is still uniform in power (white) and Gaussian, but due to the windowing in time we would now be able to zoom in on the frequency response and observe the sample to sample correlation that would now exist that didn't exist prior to windowing. Prior to windowing each sample in frequency would be independent from adjacent samples so as we zoomed in on the frequency response it would continue to look like the first plot below. But if the time domain function was windowed, it would create dependence bewteen the adjacent samples in frequency and when we zoomed in to the frequency response in that case we would start to observe something like the second plot below: we would see a definite trajectory of the frequency response waveform as we move from one sample to the next- however it is still white (the power on average over all frequencies would be flat) and Gaussian distributed.

White Gaussian Noise (AWGN)

Band-Limited Gaussian Noise

A further way to prove that the frequency response remains white after multiplying the time domain function with a rectangular window is to observe the autocorrelation function in each case: The autocorrelation fucntion for an AWGN signal is an impulse, and the frequency response of an impulse is a uniform function.  Adding zeros to the AWGN fucntion (or equivalently windowing) does not change the result from being an impulse, and therefore the frequency response will still be uniform (white).  Adding zeros does interpolate between the existing samples in frequency, and thus the trajectories previously described are created... and to note from that, for a given window size of length T of an AWGN signal, the samples in frequency separated by 1/T will remain independent, but all samples in between will be dependent on the two adjacent samples separated by 1/T.

Previous post:
The following was initially given as a response but this is specific to convolving with a rectangular window which was not the question asked:

A windows duration and shape effects the spectral density of white noise based on the frequency response of the window directly. While noise will be reduced in power based on the relative length of the window; meaning as a sum of squares or $\int_0^T(x^2)dx$, while a sine wave within the correlation bandwidth of the window (meaning frequency < 1/T where T is the window length) would increase as a summation. I prefer to consider the window as a moving average such that the sine wave (if low enough in frequency) does not change and the noise is proportionally smaller. This just means we normalized the window to its length but is more intuitive that the window would not effect the sine-wave itself but would remove noise. The normalization if not used just results in an arbitrary scaling but the ratio of signal to noise is what is of interest in the end in either case.

Consider an example (digital) white noise process with total variance = 1

If we filtered this with a 10 tap unity gain filter (representing convolving the white noise process with a discrete rectangular window [1 1 1 1 1 1 1 1 1 1]), the noise from tap to tap in the filter would be uncorrelated, so would go up by the sqrt(10) in standard deviation (which represents its magnitude quantity), while a sine-wave that was within the filter bandwidth would be correlated and would increase by a factor of 10 in magnitude.

Observe the frequency response of such a filter, where the DC gain of 20dB represents the factor of 10 described above, as (20Log10(10)). This response shows exactly what would happen to the power level of a single tone at any frequency within the filters spectrum, while the power of multiple tones would be the sum of their individual powers (which is how we handle what happens to the noise, as in $\sum x^2 $ ) :

And the expected effect on the white noise

The noise is now shaped (colored) due to the lowpass nature of the window, and the overall noise after processing through this filter should only go up by 10log10(10) = 10 dB. Thus the SNR has increased 10 dB since the tone (signal) when up by 20 dB while the noise went up by 10dB, or if we normalize to the level of the tone, the noise has gone down by 10 dB or 1/10th in total power.

Testing this experimentally:

noise= randn(2^12,1);
var1 = std(noise);
noisefilt = filter(ones(10,1),1,noise);
var2 = std(noisefilt);
freqz(ones(10,1));  % frequency response

Results in var1 = 1.00355 and var2 = 10.64.

The increase is just a constant (and arbitrary) gain factor so what is important is how the noise is effected relative to a sine wave, in that the window reduces the noise power of white noise proportionally (in this case compare a wider window to one 1/10th in size and the smaller one removes 1/10th of the power) while reduces the sinewave according to a Sinc function with the first null at 1/T where T is the length of the window. (Or for any arbitrary window based on the Fourier transform of the window itself).

Also as I mentioned in the comment under the original posting, I believe fred harris handles the mathematics well in describing coherent vs non-coherent gain, equivalent noise bandwidth etc in windowed systems in this classic paper that I reference often: https://www.utdallas.edu/~cpb021000/EE%204361/Great%20DSP%20Papers/Harris%20on%20Windows.pdf


FFT looking messy after filtering process

If your description is correct, you are seeing noise.

After step 1, everything above 2 kHz should be gone. After your band-bass everything below 8750 Hz is gone, so you basically end up with a null set. Since your filters are not infinitely steep there is still something non-zero left over but it's mostly going to be noise and very poorly defined.

Rectification is also a highly non-linear operation, so chances are you also going to get some aliasing in there as well.


Realtime filtering on a microcontroller, with dynamic cut off frequencies

You can pre-compute the coefficients for a bunch (say $N=1024$) different cutoff frequencies. If your application doesn't need to provide fine control over the cutoff frequency, just read from your table and you're done. This works best if you have a lot of ROM and/or you use short FIR or IIR filters.
If you need finer control than those $N$ values, and if the relationship between your coefficients and cutoff frequency is well-behaved (ideally, monotonic), you can use linear interpolation within the pre-computed table of coefficients. This works best with well-behaved structures like state-variable filters - for which the coefficients are monotonic functions of the cutoff frequencies, and which do not get unstable when modulating them at fast rates. This is a very common problem in musical audio applications (equalizers, synthesizer analog filter emulation...). If you use other filter topologies, make sure that they cannot get unstable when their coefficients are changed "in mid-flight" - to me, this is a harder problem than actually computing them.


Recover signal from its spectral amplitude and envelope

One counterexample is two wave packets of identical spread and amplitude, well separated in time so that their envelopes do not overlap, and of high enough (differing) well-separated frequencies such that their analytic signals can be considered Gaussian-enveloped complex exponentials. The Hilbert envelope will thus consist of two identical well-separated Gaussians. The amplitude spectrum will consist of well-separated Gaussians of identical magnitude. There is not enough information to assign the frequencies to the wave packets.

So in general you cannot resolve the original signal, not even up to an ambiguity of a constant phase shift over all frequencies.

With sufficiently large time spreads of the two wave packets their frequencies need not be far apart to still be well separated in the spectrum, satisfying your constraint for a narrow-banded signal.


LMS algorithm for modeling step-size ambiguity!

I have the first edition of Behrouz Farhang-Boroujeny's Adaptive Filters book.  I found it useful and it was definitely more practical in terms of implementing adaptive filters than other textbooks like those from Haykin and Sayed, primarily because of the included Matlab code.  However, like any topic in the area of adaptive filtering, I would use it with a grain of salt.

There are numerous variations of the LMS algorithm, primarily because it is readily implemented and not computationally expensive compared to other adaptive algorithms such as the Recursive Least Squares (RLS) algorithm.

I think a lot of the LMS variations you can find (particularly in Farhang-Boroujeny's book) come about because people tweak one of the more traditional variations, ending up with something that performs well for their particular problem.  These often make it into journals because they are different and are indeed novel implementations.

I would suggest you read about the standard LMS algorithm and the normalized version of this algorithm, the NLMS.  Once you feel comfortable with your understanding of these algorithms, then implement them and apply them to your problem.  Tuning the step-size can then be done by trial-and-error.  If you are keen to understand the mathematical details of how the step-size is determined, I suggest the Haykin and Widrow textbook.  It has in depth analysis of the convergence behavior of LMS-based algorithms.

Here is an example (my code) of the LMS algorithm in Matlab.

function [ prediction_error, weights ] = LMS_Algorithm( regressive_sequence, ...
    step_size, number_of_taps )

% This script-file implements the Least Mean-squares (LMS) adaptive
% algorithm.
%
%
%% See Also
% LMS_Algorithm, NLMS_Algorithm, RLS_Algorithm, APA_Algorithm

% Author:  Michael R. Wirtzfeld
% Modification Date:  Monday, February 27, 2012
% Creation Date:  Wednesday, February 15, 2012

weights = zeros( number_of_taps, size(regressive_sequence, 2) );
prediction_error = zeros( size(regressive_sequence) );

for index = (number_of_taps + 1):1:numel( regressive_sequence )

    tap_inputs = [ regressive_sequence( (index - 1):-1:(index - ...
        number_of_taps) ) ]';

    prediction = weights(:, index)' * tap_inputs;

    prediction_error(index) = regressive_sequence(index) - prediction;

    weights( :, (index+1) ) = weights( :, index ) + ...
        step_size * tap_inputs * prediction_error(index);

end;  % End:  for index = 1:1:numel( regressive_sequence )

weights = weights( :, 1:1:numel( regressive_sequence ) );

%% References

% Section 9.5, Summary of the LMS Algorithm, Adaptive Filter Theory, Third
% Edition, S. Haykin, Prentice Hall, 1996

Here is an example of the NLMS algorithm in Matlab.

function [ prediction_error, weights ] = NLMS_Algorithm( regressive_sequence, ...
    step_size, number_of_taps )

% This script-file implements the Normalized Least Mean-squares (NLMS)
% adaptive algorithm.
%
%
%% See Also
% LMS_Algorithm, NLMS_Algorithm, RLS_Algorithm, APA_Algorithm

% Author:  Michael R. Wirtzfeld
% Modification Date:  Monday, February 27, 2012
% Creation Date:  Wednesday, February 15, 2012

a = 0.0001;

weights = zeros( number_of_taps, size(regressive_sequence, 2) );
prediction_error = zeros( size(regressive_sequence) );

for index = (number_of_taps + 1):1:numel( regressive_sequence )

    tap_inputs = [ regressive_sequence( (index - 1):-1:(index - ...
        number_of_taps) ) ]';

    prediction = weights( :, index )' * tap_inputs;

    prediction_error(index) = regressive_sequence (index) - prediction;

    normalization_term = a + norm(tap_inputs)^2;

    weights( :, (index+1) ) = weights( :, index ) + ...
        (step_size / normalization_term) * tap_inputs * prediction_error(index);

end;  % End:  for index = 1:1:numel( regressive_sequence )

weights = weights( :, 1:1:numel( regressive_sequence ) );

%% References

% Table 9.2, Summary of the NLMS Algorithm, Adaptive Filter Theory, Third
% Edition, S. Haykin, Prentice Hall, 1996

These functions were used as part of an investigation to see how a standard tap-delay line implementation of the LMS and NLMS algorithms could model a second-order auto-regressive sequence.

As Fat32 said, most of the textbooks on the topic of adaptive filters use different notations and suggest different recommendations regarding how to set parameters, so it's no wonder that you feel uncertain about what to do.

I hope this helps.


is there a use for a changing kernel in image convolution

Yes it's true; there are image filter kernels whose coefficients may change with location of the processing and further based on image data. They're known as shift-varying filters. And data dependency also makes them nonlinear.

One prime application of shift-varying nonlinear filtering is on the edge adaptive, noise reduction where the kernel coefficients are determined based on a statistical analysis of a local group of pixels; according to whether it's an edge dominant or noise dominant block. There are different applications too.


How make FFT when buffer size is not 2^L

The prime factors of 500 are:

2     2     5     5     5

fftw can do this fast.

If you want to roll your own in:

Rabiner, Lawrence R., and Bernard Gold. "Theory and application of digital signal processing." Englewood Cliffs, NJ, Prentice-Hall, Inc., 1975. 777 p. (1975).

there is a section that shows how to do a 1 dimensional dft in two dimensions.

see my answer

Combine FFT's of shorter length than sample data to get spectrum of all data

pasting the Matlab code from there to here, and taking M=20 N=25 for your problem.

clear all
M=3;
N=32;
x=linspace(0,10,M*N);
X=reshape(x,N,M).'; % read in as rows
Twiddle=zeros(size(X));
for i=1:M
for k=1:N
Twiddle(i,k)=exp(-1j*2*pi*(i-1)(k-1)/(NM));
end
end
X=fft(X) % fft on each column
X=X.*Twiddle;% element by element product
X=fft(X.').' ; %fft on each row
y=reshape(X,N*M,1); % read out as columns
figure(1)
plot(abs(y),'linewidth',2)
title('Composite DFT')
figure(2)
plot(abs(fft(x)),'linewidth',2)
title('Direct DFT')
figure(3)
plot(x,'linewidth',2)
title('time series')

FFTW is probably faster.


Where should triangle, sawtooth and square waves start?

If you look at it on the surface, all periodic waves have phases associated with them such as $\sin(\omega t + \phi)$ or $\cos(2\pi f_0 t + \theta)$ which has phase angels of $\phi$ and $\theta$ radians respectively. Therefore, they do not necessarily begin at their peaks at $t=0$ always. Indeed at $t=0$ above waves assume the values of $\sin(\phi)$ and $\cos(\theta)$. And you can therefore shift them by amounts of $\phi$ and $\theta$ left or right to make their phases zero with respect to display graphics plotting systems.

Oscilloscopes make this adjustment by their trigger levels. For example, to set the peak of the signal $A \cos(2\pi f_0 t + \theta)$ at the origin ($t=0$) then you shall adjust your trigger level to about $A$, assuming that the trigger operation will start plotting the wave at that amplitude positioned at the origin of the coordinate grid.

The same should be true for other waveshapes such as sawtooth, triangle, square or many unnamed periodic possibilitied.

I cannot see if you have a deeper question here, but on the surface the the answer is as above.

Plot of $\text{sgn}( \sin( 2\pi t ) ) $ :

And plot of $\text{sgn}( \cos( 2\pi t ) ) $ :


In my opinion, using the STFT (or spectrogram) as you already did is a good start. Try to change the parameters of your STFT, i.e. window size and frame shift until you are able to detect certain frequencies and their repetitions.

This is an example for a decelerating car on the highway. The inertial sensor (not a microphone) was placed at the axle:


If exponentials are eigenfunctions, where does the transient term come from?

There is a transient because the input isn't just a cosine, but a causal cosine; that is, the input isn't a cosine from $-\infty \rightarrow \infty$, only from 0 onwards.  If the input were actually just an infinite length cosine, then you would not get any transients.

I notice that the notes you are working from don't show that the input is causal.  One reason I maintain that it actually is, is that otherwise you have an output that is tending to $\infty$ as $t \rightarrow -\infty$.  And also you don't get transients without a change to the system; the change in this case is that the system starts at time $t=0$.


How to avoid leakage in the following FFT

Since your frequencies f2 and f3 are a multiple of f1=1k, and since your time vector is until 5 ms, that means your signal fits for a whole 5 periods, harmonics and all, so it's just ripe for a FFT. The problem you're having is that you chose the FFT to be of a number of points (1024) different than the number of samples (1000), so now one bin will be Fs/nfft = 2e5/1024 = 195.31 Hz. If you check the first values of the frequency vector xfft(1:10), you'll see that the first frequency falls in between two bins:
            0
   1.9531e+02
   3.9062e+02
   5.8594e+02
   7.8125e+02
   9.7656e+02 % f1 is between this
   1.1719e+03 % ...and this
   1.3672e+03
   1.5625e+03
   1.7578e+03

The other two have the same symptom. This means the amplitude (which is not normalized, that's why it's ~5k) can't be the real one. When the equivalent DFT will calculate the butterflies, the indices will not match f1 at the same spot. If you would have performed the FFT for 1000 points, the bins would have been 2e5/1e3 = 200 Hz in width. That would have made the frequency vector display these frst 10 values:
      0
    200
    400
    600
    800
   1000 % spot on
   1200
   1400
   1600
   1800

Using a 1024 length FFT means now there are more than one butterfly with different values, and they all add up. Which means you have a mild case of spectral leakage, i.e. the frequency that should have been present in one bin, only, is spreading across its neighbours. Also see the Wikipedia article.


How to: pixel packs in an image to representative point coordinates (x,y)

Just a naive suggestion: do you know about component labeling?

The technique is about finding chunks of "touching" pixels and assigning them a label, e.g. an integer number. You can then interrogate each chink separately, looking for the pixel that share the same label.

In MATLAB, here is the function that does it trivially: bwlabel


So, the technical difference between a scrambler and an interleaver:

The interleaver is just a permutation of input symbols; a resorting.
The scrambler is an operation designed to modify the transmitted symbol in such a way that they are not present in the output.

Thus, "scrambler" is a way less precise defined term. I usually define it as "operation that is easy to invert if you know the transmit operation, and typically fulfills more of a statistical purpose than a cryptographic one".

The typical purpose of a scrambler is to break up periodicities in the original symbol stream; this can, for example, be done by combining with a pseudorandom sequence. To de-scramble, the receiver would do the inverse combination (in case of XOR, it's autoinverse) with the same sequence. Often, that sequence is longer than a "block" of transmitted signals.

That way, you're basically applying an uncorrelated signal to your data signal; even if you had spikes in the spectrum of your data signal, because it had repetitiveness in it, these will (in expectation) be gone after scrambling, and you get a nice, white spectrum.

A white spectrum is important:

you will need to adhere to legal restrictions on power, and these usually specify "maximum power within a small bandwidth". So, if you want to use the most of your expensive spectrum, you want to have a nice PSD without peaks.
from frequency-selective channel point of view, having peaks in the spectrum is a dangerous game: if a deep fade happens to fall on the peak, you lose a lot of signal. You can "distribute" your risk better if you evenly use the channel.
from a synchronization point of view, for many synchronization algorithms, it's important that

the spectrum is symmetric (e.g. for the band-edge frequency locked loop (FLL)) or
that all symbols are equally likely (you'd have a peak at 0 Hz in your spectrum if you have an "average != 0" signal) (important for PLLs, otherwise the assumption that all symbols are equally likely breaks, and that breaks the optimality of "correcting" towards the closest correct phase)
that there's no runs of constant symbols, even if that happens likely in the input data (e.g. constant-rate digital broadcasting: when there's no bits to transmit, because e.g. everything is silent or the video codec only needs to compress a still image, then you have to pad with 0s, so that you keep your symbol rate) (extremely crucial for symbol timing recovery to have enough symbol transitions)

The interleaver, on the other hand, doesn't change the fact that your data signal might have periodic components: sure, you're moving them to different positions, but they're still there; you just "moved" the spikes in the spectrum in phase, you have little chance of actually getting rid of them.

On the other hands, if you have a burst error on say symbols Nr. 100 to Nr. 200, your XOR-scrambler would still have that burst error in the same range. After a sufficiently long interleaver, the same burst error would be distributed to more or less isolated singular errors, which might be much better for the channel  decoder down the processing line.

So, all in all:

interleaver against burst errors
scrambler against unwanted regularities in the signal.


What is the difference between y[n] and y(k) (using square brackets)?

The notation for discrete-time signals, $x[n]$, was first noticed by me in the original Oppenheim and Schaffer (1975), even though i didn't see that book until 1981.

So before, discrete-time signals in DSP used the same notation in mathematics used for sequences (infinite or not), with the subscript, $x_n$.  In both notations, the argument or subscript play the same role and are only integers.

$$ x[n] \triangleq x_n \qquad \qquad x\in\mathbb{C} \qquad n\in\mathbb{Z} $$

Usually the signal, $x[n]$, is real but it need not be.  But the argument, $n$, must be an integer which makes this no different than a sequence, $x_n$.

With the DTFT and DFT this notation was common pre-O&S,

$$\begin{align}
 X_k &\triangleq \sum\limits_{n=0}^{N-1} x_n e^{-i 2 \pi \frac{nk}{N}} \\
\\
 x_n &= \frac{1}{N} \sum\limits_{k=0}^{N-1} X_k e^{+i 2 \pi \frac{nk}{N}} \\
\end{align}$$

or they would use that awful $W_N^{nk}$ notation.

Now the problem with the "$x_n$" notation is that you would get too many subscripts to deal with, if you had a signal that was a multi-element vector.  And there was nothing in the notation to denote which subscript was the "time-like" variable.  With analog signals, $v(t)$, this wasn't a problem.  If you had a cable with 8 wires (plus ground), you could label them "$v_1(t)$" to "$v_8(t)$" and looking at $v_m(t)$, there was no doubt that it meant the "$m$th element that is a function of time, $t$."  But with "$x_{m,n}$" (as opposed to the O&S way "$x_m[n]$") there is nothing, other than order, differentiating discrete time from the "channel" number.  Add another dimension (such as the 2-dim pixel radiance as a function of time) and you have a worser mess:

$$ x_{r,c}[n] \qquad \text{vs.} \qquad x_{r,c,n} $$

$r$ is row number, $c$ is column number, and $n$ is discrete-time.

So with this notation, there is a parity between a vector of continuous-time (a.k.a. "analog") signals $x_m(t)$ and discrete-time (a.k.a. "digital") signals $x_m[n]$.  So, likely, whatever relationship one might see between the various analog signals $x_m(t)$ in a multi-conductor cable, I might expect to see between the various elements of the digital vector signal $x_m[n]$ and sampling would be no different than with a single-dimensional signal

$$ x_m[n] \triangleq x_m(t)\bigg|_{t=nT} \qquad \qquad T=\frac{1}{f_\mathrm{s}} $$

So this provides for a consistency of notation along with a way to differentiate arguments that must be integers whereas whatever is contained within parenths $f\big( \cdot \big)$ is considered to have no such restriction (could be a real or complex argument).  That is, to me, helpful.

So, if it's a very old DSP paper (pre-1978, perhaps), the notation you would see would almost certainly be $x_n$ for discrete-time signals and the DFT would be $X_k$ for discrete-frequency.  But this has evolved to

$$\begin{align}
 X[k] &\triangleq \sum\limits_{n=0}^{N-1} x[n] e^{-j 2 \pi \frac{nk}{N}} \\
\\
 x[n] &= \frac{1}{N} \sum\limits_{k=0}^{N-1} X[k] e^{+j 2 \pi \frac{nk}{N}} \\
\end{align}$$

and EEs will use "$j$", instead of "$i$" for the imaginary unit.

Math folks might use this notation:

$$\begin{align}
 \hat{f}_k &\triangleq \frac{1}{\sqrt{n}} \sum\limits_{j=0}^{n-1} f_j \, e^{-i 2 \pi \frac{jk}{n}} \\
\\
 f_j &= \frac{1}{\sqrt{n}} \sum\limits_{k=0}^{n-1} \hat{f}_k \, e^{+i 2 \pi \frac{jk}{n}} \\
\end{align}$$


Whittaker-Shannon ($\mathrm{sinc}$) interpolation for a finite number of samples

An alternative explanation of what the function $g(u)=\sum_{m\in\mathbb{Z}}\operatorname{sinc}(u-mN)$ is.
First, note that
$$g(u)=\sum_{m\in\mathbb{Z}}\operatorname{sinc}(u-mN)=\text{sinc}(u) \ \circledast \ \sum_{m\in\mathbb{Z}}\delta(u-mN)$$.
Let's calculate the Fourier Transform of $g(u)$:
$$G(f) \triangleq \mathscr{F}\{g(u)\}=\operatorname{rect}(f)\cdot\frac{1}{N}\sum_{m\in\mathbb{Z}}\delta(f-\tfrac{m}{N})$$
Here, we have used the convolution theorem. As we see, the spectrum is a sampled version of the rect function. It's clear that it is discrete, since $g(u)$ is periodic. We can now distinguish two cases: $N$ odd and $N$ even.

For the $N$ odd case the result is clear:
$$\begin{align}G(f)&=\frac{1}{N}\sum_{m=-(N-1)/2}^{(N-1)/2}\delta(f-\tfrac{m}{N})\\
g(u)&=\frac{1}{N}\sum_{m=-(N-1)/2}^{(N-1)/2}e^{j2\pi \frac{mu}{N}}
\end{align}$$
which is just a time-scaled Dirichlet kernel. Let $t=2\pi\frac{u}{N}$, then
$$g(\tfrac{N}{2\pi}t)=\sum_{m=-((N-1)/2)}^{(N-1)/2}e^{jmt}=\frac{1}{N}\frac{\sin((N-1)/2+1/2)t)}{\sin(t/2)}=\frac{\sin(tN/2)}{N\sin(t/2)}.$$
Now doing backsubstituting to $u$, we get to
$$g(u)=\frac{\sin(\pi u)}{N\sin(\pi u/N)}$$
which confirms the result from Olli.
For the case of even $N$, the treatment is a bit more difficult: The Diracs appear exactly at the discontinuity of the $\operatorname{rect}()$ function. But, treating the $\operatorname{rect}()$ function as e.g. the limit of a raised cosine where the rolloff goes to zero, we can argue that the rect has value of $\tfrac12$ at the discontinuity. Hence, in frequency domain $G(f)$ can be expressed as
$$\begin{align}G(f)&=\sum_{m=-(N/2-1)}^{N/2-1}\delta(f- \tfrac{m}{N})+\tfrac{1}{2}(\delta(f-\tfrac{1}{2})+\delta(f+\tfrac{1}{2}))\\
g(u)&=\sum_{m=-(N/2-1)}^{N/2-1}e^{j2\pi\frac{mu}{N}}+\cos(2\pi\tfrac{1}{2}u) \end{align}$$
Doing the same substitution as above, we get to
$$\begin{align}g(u)&=\frac{1}{N}\frac{\sin(\pi\frac{N-1}{N}u)}{\sin(\pi u/N)}+\frac{\cos(\pi u)}{N}\\
&=\frac{1}{N}\frac{\sin(\pi u - \pi u/N)+\cos(\pi u)\sin(\pi u/N)}{\sin(\pi u/N)}\end{align}$$
Now, using the trigonometric identity $\sin(\pi u - \pi u/N)=\sin(\pi u)\cos(\pi u/N)-\cos(\pi u)\sin(\pi u/N)$, some parts in the numerator cancel and we get
$$\begin{align}g(u)&=\frac{\sin(\pi u)\cos(\pi u/N)}{N\sin(\pi u/N)}\\&=\frac{\sin(\pi u)}{N\tan(\pi u/N)}\end{align}$$
which again confirms the result from Olli.


How to determine the sine Fourier coefficients of discrete data?

In accordance with the case of a continuous function $f(x)$ where
$$b_n=\frac{2}{L}\int_{0}^Lf(x)\sin\left(\frac{mπx}{L}\right)dx$$
I came with the following expression:
$$A_m=\frac{1}{N-1}\sum_{n=1}^{N}w_n\sin\left(\frac{mπx_n}{L}\right)$$
Can anyone confirm or not that solution?


Peak to peak amplitude of sum of sinusoidals (harmonic frequencies)

Suppose that we have a continuous-time periodic signal
$\displaystyle s(t) = a_0 + \sum_{n=1}^N a_n \cos(n\omega_0 t + \theta_n)$.

What does this mean? Do we have a  trace of $s(t)$ on some recorder chart
and the trace looks periodic?  If we did, the question to be solved would be
trivial since we could simply measure the maximum and minimum values of $s(t)$ on
the chart.  So the problem of interest is:

  Given the values of
  $\omega_0, a_0, a_1, a_2, \cdots, a_N, \theta_1, \theta_2, \cdots, \theta_N$,
  find the values of
  $$M_\max = \max s(t)\quad \operatorname{and} \quad M_\min =  \min s(t).$$

The OP wants to find the peak-to-peak amplitude of $s(t)$ and this is, of
course, just $M_\max-M_\min$.  It is worth noting that since $s(t)$ is periodic
with period $T = \frac{2\pi}{\omega_0}$, the maximum value and minimum value
occur (at least once) in each interval of length $T$ on the time axis.  The key
issue here is since we know the $a_i$ and the $\theta_i$ etc., we can write
down the formula
$$s(t) = a_0 + \sum_{n=1}^N a_n \cos(n\omega_0 t + \theta_n)\tag{1}$$
(whether this constitutes "reconstruction" of the time-domain signal
is a question that I will leave to others to answer) but the
standard method for finding the maxima and minima of $s(t)$
requires us to find solutions to the nonlinear equation $(2)$ below
(that is, values of $t$ for which $(2)$ holds):
$$\frac{ds(t)}{dt} = -\sum_{n=1}^N a_n\cdot n\omega_0 \sin(n\omega_0 t + \theta_n) = 0.
\tag{2}$$
Once we have found the values $t_1, t_2, \ldots, t_k, \ldots$ for which
$\displaystyle \sum_{n=1}^N a_n\cdot n\omega_0 \sin(n\omega_0 t_k + \theta_n) = 0$,
we can simply calculate $s(t_1), s(t_2), \ldots, s(t_k), \ldots$ and search
through this list of numbers (it is not necessary to sort the list
as recommended in hotpaw2's answer) to find $M_\max$ and $M_\min$.  But,
ignoring the difficulty of finding the $t_k$'s makes hotpaw's answer not
particularly useful in solving the OP's problem.  The answer by ethereal
is even worse in this regard since all it boils down to is the assertion
that (for $\alpha > 0$)
the maximum and minimum values of $\alpha\cdot s(t)$ are $\alpha\cdot M_\max$
and $\alpha\cdot M_\min$ respectively. Finding the maximum and minimum
of $s(t)$ from knowledge of its Fourier series is a nontrivial task, not at
all as easy as it is made out to be by ethereal or by hotpaw2.

A related problem has received a lot of research attention in the past
thirty years or so.  For a periodic signal such as $s(t)$, the average power
in the signal is readily computed as
$$\bar{P} = \frac{1}{T}\int_0^T |s(t)|^2\,\mathrm dt = a_0^2 + \frac{1}{2}\sum_{i=1}^N |a_i|^2.$$
On the other hand, the peak power is $P_\max = \max\{M_\max^2, M_\min^2\}$,
and is also of interest, especially to system and power amplifier designers,
and the ratio $\frac{P_\max}{\bar{P}}$, aptly named the _peak-to-average-power
ratio (PAPR) has received much attention. It can be calculated exactly for
a given $s(t)$ but only with great computational effort, and so a lot of effort
has gone into finding bounds on the PAPR.  Some of these could help in getting
bounds for the OP's problem.
only with a great deal of effort.


Cut-off frequency in Fourier-Spectrum of a noise message

I think you consider that pulse in the spectrum as the noise. Then you should remove it by a lowpass filter of cutoff frequency
$$ f_c = 5 ~\text{kHz.}$$

Slightly less than this should be used for guarding against the transition width of the lowpass filter. So you better select something like $4.8$ kHz for the lowpass filter cutoff frequency.

Note that your sampling frequency $F_s$, probably, is $44.1$ kHz and not $44.8$ kHz. Then your digital frequency for cutoff into the Matlab functions will be:
$$f_{cd} = \frac{ 2 f_c }{F_s} $$

with the given values of $F_s$ and $F_c$ your digital cutoff will be:

$$f_{cd} = \frac{ 2 \times 5  \times 10^3 }{44.1 10^3} = 0.2268 $$

or better

$$f_{cd} = \frac{ 2 \times  4.8 \times 10^3 }{44.1 10^3} = 0.2177 $$


The magnitude of taking discrete cosine transform of an image two times is similar to the original

It's beacuse the inverse discrete fourier transform (DCT) is almost identical to the forward DCT. So taking twice the transform will be similar to the original signal. In fact if you provide which DCT type (DCT-I, DCT-II etc) you have used, one can show the effect more explicitly.


How do you calculate RMSE in degrees for a DOA estimator system with multiple incoming signals?

The way to do it is to find peaks and not maximum values. Like the function findpeaks in MATLAB (local peaks) you can define resolution (distance between peaks and more...)


Synthesizing harmonic tones with Karplus Strong

First, note that the behaviour in which high-pitched notes disappear faster than low-pitched notes is found in many musical instruments. You might better not totally compensate for that.

One way I got the behaviour I wanted out of Karplus-Strong was the following:

I add an additional exponential decay for the lowest notes - a "loss" factor so that x[n] = loss * (x[n] + x[n - 1]) / 2
For the highest notes, I randomly decide whether I will do a buffer update or not ; with an increasingly high probability for the "no update" event as pitch increases.

To avoid dealing with fractional delays - and to get any kind of pitch modulation I want (but also to simplify the code), I use a ring buffer of the nearest power of 2 (for example, 128 samples for A4) and then do a linearly interpolated read from it. One can see this as a wavetable oscillator whose lookup table is dynamically modified/refreshed as the phase increment counter moves through it.


Producing Colored Noise from a given PSD Data

Estimating the necessary filter order
There are methods for estimating the needed filter order for low/band/high-pass filters, but the fact that you are talking about "colored noise" suggests that you are not going for relatively simple frequency profiles.  Thus, the answer is "no", I am not aware of an order estimator that will help you.  The best generic advice I can give you is to go as high as you can stand performance-wise and hopefully that will be enough.

Filtered/Colored output has lower amplitude than input
It sounds like you know the frequency profile you want, and are creating a filter that produces that profile.  If that is the case, fixing this should be easy.  Do the following:

Calculate the average of the amplitude of the frequency profile you are creating.
Divide all of your filter coefficients by that average.

That should make the overall gain of the filter "1" with regards to white noise input.


What is the difference between clustering and quantization?

They are essentially the same. The generalized form of the Lloyd's algorithm is LBG which is used for vector quantization (VQ).You may use k-means for the purpose of vector quantization just like the LBG algorithm. In vector quantization, the input space is partitioned into a set of convex areas. They are also referred to as Voronoi regions or cells. If you look at here it says k-means (which is mostly used in computer science) is actually originated from VQ in signal processing as well as pulse code modulation back in fifties.

If you want to implement VQ, you may directly use k-means algorithm. The original LBG algorithm is presented in the following paper if you are interested:

Y. Linde, A. Buzo, and R. M. Gray, ``An Algorithm for Vector Quantizer Design,'' IEEE Transactions on Communications, pp. 702--710, January 1980.


If you sample every 0.2ms you collect 5000 samples per second and hence the sampling rate is 5000 Hz. According to Nyquist, this means the highest frequency the signal can have so we don't experience aliasing would be 2500 Hz.

Though that wasn't the question (it was about the sampling theorem explicitly), it's worth pointing out that that's only a theoretical limit. In practice, we need to go a bit higher (leave some room) for various reasons, including the fact that we can never observe an infinitely long signal, as correctly pointed out by the commenters.


Autocorrelation and Power Spectral Density (Discrete)

First of all lets state more correctly that a discrete-time 1D auto-correlation sequence (ACS), $\phi_{XX}[\kappa]$, of a single parameter $\kappa$, of a discrete-time random process $X[n,s]$ will exist as long as the process is stationary or at least WSS (wide sense stationary).

And therefore the power spectral density (PSD), $\Phi_{XX}(e^{j2\pi fT})$, of the process will also exist iff the random process is stationary (or WSS at least) and its ACS is stable, in the sense that its Fourier transform will exist. (In case ACS is not stable, generalized definition of the Fourier transform is used to represent nonconvergent integrals with impulses or with discontinuties at those particular frequencies...)

Note that the notation changes slightly and I prefer to use $\phi_{XX}[m]$ to denote the ACS of the WSS random process $X[n,s]$ and use $\Phi_{XX}(e^{j\omega})$ to indicate its PSD, which is indeed given by the discrete-time Fourier transform (DTFT) of its ACS :
$$ \Phi_{XX} (e^{j\omega}) = \mathcal{F} \{ \phi_{XX}[m] \} = \sum\limits_{m=-\infty}^{\infty} \phi_{XX}[m] e^{-j\omega m} $$

Since the PSD of a WSS random process $X[n,s]$ is a function returned by the DTFT of a sequence, it must be a periodic and continuous function of frequency $\omega$ , as all other Fourier transforms. We are only interested in the first period from $\omega=-\pi$ to $\omega=\pi$ (or from $\omega=0$ to $\omega=2\pi$)

Coming to your confusion, in most practical applications, a discrete-time random process $X[n,s]$ is represented by a discrete-time random sequence $x[n]$ which is created by the process of sampling a particular realization, $x(t)$, of a continuous-time random process $X(t,s)$ with a sampling period $T_s$. In such a case $\omega = 2\pi f T_s$ could be used as indication of the sampling relationship between the continuous-time and the discrete-time random processes involved. Also in communications terminology, frequency in Hertz $f$ is preferred over frequency in radians per second $\Omega$.

Note also that I've deliberately used a capital letter $X$ , instead of a lowercase $x$, to denote the ownership of those ACS and PSD functions; They belong to the random process $X[n,s]$ and not to a single particular realisation sequence denoted by $x[n]$. The sequence $x[n]$ belongs to the family of all functions that constitude the ensamble of the random process $X[n,s]$ ($n$ denotes the (time) index and $s$ denotes the outcome of the experiement for those indexed random variables $X_n(s)$ associated with the process $X$)

The random process $X[n,s]$ exists as a theoretical entity, an abstract being with its own characterisation that generates all those possible realizations $x[n]$ being observed. In practice we can only observe one or more $x[n]$ and therefore we have to estimate the true (but unobservable) quantities of ACS and PSD that belong to random process $X[n,s]$ from those realisations by means of statistical estimation methods. This process is not exact, hence in practice we can only compute an approximation to those true theoretical entities of ACS and PSD from particular obsrevations of instances $x[n]$ of the random process $X[n,s]$.


Implementing discrete Poisson equation wtih Neumann boundary condition

For image in painting and blending a pixel its new value is dependent on its neighboring pixels. These neighbors may or may not lie on a border. The aim is to interpolate and create smooth transitions. It is thus not only dependent on a border condition, nor does a value on the border get replicated exactly to inside pixels.  Should the area between boundaries be exactly 1 pixel wide, this pixel will most likely be the average of the boundary values.

A simple blender algorithm will work like this. Given:

A background image $\textbf{b}$
A foreground image $\textbf{f}$
A mask $\textbf{m}$

where we will put the foreground image $\textbf{f}$ onto the background $\textbf{g}$, where the mask $\textbf{m}$ is true.

To avoid a visible seam we will make a correction image $\textbf{c}$. The run-of-the-mill patching to form new image $\textbf{g}$ then is:

$g(x, y) = \left \{\begin{array}{cc}
  b(x, y) & \textrm{if } m(x, y) = 0 \\
  f(x, y) + c(x, y) & \textrm{if } m(x, y) = 1
 \end{array} \right .$

This image $\textbf{c}$ is obviously some form of interpolation, but informally we want two things from $\textbf{c}$:

$\textbf{f}$+$\textbf{c}$ and $\textbf{b}$ should match on the edges
It should be 'smooth' as to not create disturbances

Smooth transitions can be attained by simply stipulating that a pixel its new value is the weighted average of its four neighbors (up, down, left and right):

$c(x,y) = \frac{c(x-1, y)+c(x+1, y)+c(x, y-1)+c(x-1, y+1)}{4}$

And for boundary pixels the corresponding values of $\textbf{c}$ are replaced by $\textbf{b}-\textbf{f}$. If for instance for $c(x,y)$ the right neighbour is on de boundary:

$c(x,y) = \frac{c(x-1, y)+c(x+1, y)+c(x, y-1)+(b(x-1, y+1)-f(x-1, y+1))}{4}$

This means that $\textbf{every}$ pixel of $\textbf{f}$ under mask $\textbf{m}$ gives a linear equation, and each pixel in the patch is also a variable.

The result is a square linear system:  $\textbf{A}\textbf{c} = \textbf{d}$, which can be solved for  $\textbf{c}$.

Note that

For boundary pixels, the values of $\textbf{b}$ and $\textbf{f}$ are $\textbf{constants}$ in this system, not variables. They thus go `to the right' of the equals sign.
Pixels outside the masked region, as defined by $\textbf{m}$ are not variables.

See also

J. Jia, J. Sun, C.K. Tang, and H.Y. Shum. Drag-and-drop pasting. In ACM
Transactions on Graphics (TOG), volume 25, pages 631-637. ACM, 2006.
P. Perez, M. Gangnet, and A. Blake. Poisson image editing. In ACM
Transactions on Graphics (TOG), volume 22, pages 313-318. ACM, 2003.


Digital low-pass filters; best practice application

I think a digital low-pass filter, applied after sampling, should be a good solution. One benefit of digital filtering is that you can obtain very sharp transition regions, and large rejection of out-of-band signals. The cost is relatively high computational complexity, in the sense that your processing code will execute many arithmetic operations.

Another benefit of digital filtering is that you can make use of a very large, well tested array of algorithms to design the filer, without necessarily understanding how they work. Obtaining the frequency response of a particular filter is very easy, so you can quickly evaluate if the filter meets your needs.

For illustration, this is how you would design and evaluate the filter in Matlab. I'll assume that the sampling frequency is 20 samples per second, and you want the cutoff frequency at 1 Hz.

% define frequencies:
fs = 20;
fn = fs/2;  % Nyquist frequency
fc = 1;     % cutoff frequency

% define the filter
b = fir1(100, fc/fn);  % filter order is 100; cutoff frequency is 1 Hz

% plot its frequency response
freqz(b);   % this command plots the filter's magnitude and phase responses.

The response obtained is:

You can see that the response is really nice: it is linear in the passband, and the rejection is around 60 dB to 80 dB out of band. Using this filter, though, requires around 100 multiplications and 100 additions per each processed sample.


The FFT algorithm is not usually normalized while giving the output. Hence it is not unity.It gives as
The Fourier transform is taken as
$$
 F(e^{jw})=\sum x[n] e^{-jwn}
$$
Thus you could see the sum of x's when w=0 case instead of normalized one


LTE questions (subcarriers and cyclic prefix)

The FFT size in LTE is defined as 2^(nextpow2(Nsubcarriers/2)+1) or the nearest power of two that is also higher than the number of subcarriers for a given bandwidth. For the 10 Mhz case, for example, this means that 424 FFT bins(1024-600) are zero padded apart including the DC bin. The usual practice is to zero pad the central 423+DC bins and use the rest for filling the data so that you get a continous spectrum of 10 MHz around DC. You can also fill the central 600 FFT bins with data and do a fftshift. See table 1 in this document.
Yes. Each Resource Element(i.e.1 subcarrier and 1 OFDM system in the Time- Frequency Grid) has one QAM Symbol in shared channel.

Check out this visualization to get a better clarity about the LTE resource grid. The resource is allocated both in time and frequency domain and in the case of Multi User MIMO(MU-MIMO) transmission mode, spatial resource allocation of resource elements to different user equipment in the downlink also take place. To keep things simple, let's just focus on a single-antennae transmission. The following diagram will give you a better idea of the whole transmission process.


Meaning of Q and I in all-software QPSK communication systems?

A point in 2D space can be represented by either X and Y coordinates, or a distance and angle (relative to some reference point and axis).  However, the point measurement errors or noise is some common real systems is more likely to be uniform in the the units of an X,Y coordinate system, which may make error analysis and numerical error control in this space more tractable, thus lowering the computational requirement for most types of processing, even in purely digital systems.


A simple bound is just  the number of samples. Indeed, for
$$ X_k = \sum_{n=0}^{N-1} x[n]e^{-2i\pi kn/N}$$
then obviously:

$$ |X_k| \le \sum_{n=0}^{N-1} \left|x[n]\right|\left|e^{-2i\pi kn/N}\right|$$
hence
$$ |X_k| \le \sum_{n=0}^{N-1} \left|x[n]\right|\le N$$
if $-1\le x[n]\le 1$. The $N$ bound is
s attained for a given $k$ when you choose a complex $x[n]=\overline{e^{-2i\pi kn/N}} = e^{2i\pi kn/N}$. If your input allows a constant signal  (equal to one), $x[n]=1$, then this is attained for the zeroth frequency $X_0 =  \sum_{n=0}^{N-1} 1 = N$. As a side note, if the signal length is a power of $2$, this is quite convenient.

For more restricted inputs, like real, zero-average signals only, there could be a tighter bound. A recent paper presented at SampTA 2017 provides DFT bounds for some random sequences (Bernoulli): Bounds on Discrete Fourier Transform of Random Mask (published version).


You made a mistake in this last equality

$$x[n] = \frac{1}{2\pi jn}\left( e^{-j\pi n/4} -e^{-j\pi n} +2e^{j\pi n/4} -2e^{-j\pi n/4} + e^{j\pi n} -e^{j\pi n/4} \right) \neq \frac{\sin(\pi n/4)}{\pi n}$$

The right way to do this would be:

$$\begin{align}
x(n)&=\frac{1}{2\pi jn}\left( e^{-j\pi n/4} -e^{-j\pi n} +2e^{j\pi n/4} -2e^{-j\pi n/4} + e^{j\pi n} -e^{j\pi n/4} \right) \\
&=\frac{1}{\pi n}\left( \frac{e^{-j\pi n/4}-e^{j\pi n/4}}{2j}+ \frac{e^{j\pi n} -e^{-j\pi n}}{2j} +\frac{2e^{j\pi n/4} -2e^{-j\pi n/4}}{2j}  \right) \\
&=\frac{1}{\pi n}\left( -\sin\left(\pi n/4\right)+\sin\left(\pi n\right)+2\sin\left(\pi n/4\right) \right)\\
&=\frac{1}{\pi n}\left( \sin\left(\pi n\right)+\sin\left(\pi n/4\right) \right)
\end{align}$$

Which makes sense as one can see that the DTFT is the sum of two rectangular windows, one of width $2\pi$ and one of width $\pi/2$, each corresponding to each $\mathrm{sinc}()$.

EDIT:

I've just noticed judging by your comment that the mistake you made was to think that

$$\frac{\sin(\pi n)}{\pi n} =0$$

As you already know, the numerator is $0$ for all $n$... except for $n=0$. In that case, the denominator also is zero, so you have an indetermination. The same happens in the continuous case. Remember that we assume

$$\mathrm{sinc}(0)=1$$

by taking the limit when $t\to0$. To respect the fact that the discrete $\mathrm{sinc}$ is a sampled version of the continuous one, then it equals $1$ at the origin too. So we can state that:

$$\frac{\sin(\pi n)}{\pi n} =\delta(n)$$

Notice that the DTFT you wrote can also be expressed (using the fact that convolution is distributive) as:

$$X(\Omega)=\sum_{k=-\infty}^{+\infty}\left(u(\Omega+\pi)-u(\Omega-\pi)\right)\star \delta(\Omega-2k\pi) + \sum_{k=-\infty}^{+\infty}u\left(\left(\Omega+\frac{\pi}{4}\right)-u\left(\Omega-\frac{\pi}{4}\right)\right)\star \delta(\Omega-2k\pi)$$

The first term is a window of width $2\pi$ that is $2\pi$-periodic... So it's basically $1 \ \forall \Omega$.

$$X(\Omega)=1 + \sum_{k=-\infty}^{+\infty}u\left(\left(\Omega+\frac{\pi}{4}\right)-u\left(\Omega-\frac{\pi}{4}\right)\right)\star \delta(\Omega-2k\pi)$$

Now it's easier to see that the IDTFT I got at the beginning of the question corresponds indeed to the given DTFT.


AM is still used worldwide in civil aviation, where there is a large installed base of radios (in small planes, etc.) than can not be easily updated (due to multiple national and International regulatory hurdles), and where you want one transmission to be able to talk over another signal on the same channel in an emergency.

AM radio receivers are simple and cheap using primitive technology (1 discrete diode, etc.), and there are still a few parts of the world where people can barely afford an old one of those.


Sine Wave "Lobes" (Basics, and probably a dumb question)

You're just seeing the amplitude of the individual samples, not the wave that travels between them.    If you generate a high-frequency sine wave, you will see that the samples don't necessarily go anywhere close to the actual peaks of the wave.
At some points, it looks like you might expect, with samples near the peaks of the waveform:

but at others, the samples are not near the actual peak of the waveform:

(These are both from a 12.001 kHz wave viewed in Ocenaudio sampled at 48 kHz)
If you are displaying only the samples (as you are in matplotlib because it does only linear interpolation between them), then you will see this as "lobes" that go up and down as the samples move closer and farther from the actual peaks.  But the actual waveform the samples represent does not go up and down, it's constant.


How to prevent "zipping" effect on a modulated fractional audio delay line? (Flanger)

It's a bug.

M_PI*config->count++

You are processing 2 channels, but incrementing your modulation in the inner loop. So at the end of processing 1 channel, you continue to increment the modulation as you process the second channel.

c1 [ 1,2,3,4]         [9,10,11,12]...
c2           [5,6,7,8]

The modulation has a discontinuity, i.e in my simplifies example - c1 modulation jumps from '4' to '9'. resulting in zipper noise. Hope that makes sense.


Options for DSP hardware for Under Grad Lab?

I think using a Raspberry PI or Arduino (without ADCs or DACs) has no advantage over using just a PC with Matlab or Python. The thing that separates DSP hardware from "normal" processors is that they can do certain operations very fast and/or in parallel. If you want to do something cool, I would use a FPGA with ADCs and DACs like the Red Pitaya for lab experiments.


Detecting a three-tone signal in real-time

The approach I'm familiar with has been used for quite some time in broadcast situations. You mentioned your concern with voice signal "peaking" the frequencies of interest. From that statement, I am assuming that you can transmit your signal in-between periods of voice activity.

Here is the method:

On the generation side:

arrange for the signal to play for twice as long as the analysis.

On the analysis side:

Setup non-overlapped FFTs of a length that is 1/2 of the stimulus. This ensures that one of your FFTs will eventually contain an entire signal.
On each FFT, compute the power sum of the tone bins (including adjacent bins as a result of windowing), the power sum of the non-tone bins, and ratio the result to provide a SNR.
If the ratio is above some threshold, then you have detected your signal. You can determine your threshold using your prior knowledge of the noise floor and transmitted tone levels.

A couple of things you can do to improve:

Adjust the phases of the generated signal to reduce the crest factor.
Choose the 3 signal frequencies and/or the FFT length so that the frequencies fall in the centers of the FFT bins. If your generator and analyzer are synchronous, this would allow you to avoid windowing for FFT altogether.


QAM-spectrum with uneven real & non-odd imaginary part

You are right in that the Fourier Transform (FT) of every real $x(t)$ has even real part and odd imaginary part. However, your figures only show the absolute value of the bandpass spectrum, so you cant see real- and imaginary part separately. Note that the above figure (showing $|S(f)|$ is the baseband signal, which is indeed non-even and hence complex valued. But, after upconversion, the signal becomes real and if you write a small script that does the calculation of the upconversion, you will see, the FT $S_{BP}(f)$ of the bandpass signal $s_{BP}(t)$ will have mentioned properties. I have once written a walkthrough article of up-down-conversion here which might also help in understanding this process.

To clarify a bit more in math. terms:

$\underline{s}(t)=s_I(t)+js_Q(t)$ is a complex baseband signal, which consists of the real part $s_I(t)$ and the imaginary part $s_Q(t)$.
$S(f)=\mathcal{F}\{\underline{s}(t)\}$ is the Fourier Transform of the baseband signal. This $S(f)$ is arbitrary (i.e. not even or odd) since $\underline{s}(t)$ is a complex-valued baseband signal.
$s_{BP}(t)$ is the upconverted version of $\underline{s}(t)$, and is a real-valued signal.
Hence, $S_{BP}(f)=\mathcal{F}\{s_{BP}(t)\}$ has even real part and odd imaginary part.


Finding Fourier transform of a discrete signal from its Z-transform

DTFT is the Z-transform at the unit circle. So if $z=re^{j\omega}$
then for DTFT $r = 1$.

i.e If you have the Z-transform of a signal then plug-in $e^{j\omega}$ for every $z$


Why ignores constant when applying integral into differential-equation.?

The constants would be the initial conditions, such as an initially charged capacitor or an inductor with an initial current. If all constants are set to zero it means that you have zero initial conditions, which is usually the standard assumption unless stated otherwise.


Origin of artifacts 12 Hz away from electrical line noise (50 Hz)?

Probably at exactly $\pm 12.5\text{ Hz}$, ie. some periodic phenomena that happen every four grid frequency cycles. Typical candidates for such things are rectifiers with little load.

At any rate, it's probably coming from your power grid, and you won't be able to get rid of them by anything short of notch filters.


Find continuous signal given a condition on its samples

The condition
$$x(nT)=\delta[n]\tag{1}$$
is called the Nyquist criterion for zero intersymbol interference (ISI). It is important for the design of transmit pulses in digital communication systems.
Condition $(1)$ can be expressed in the frequency domain as
$$\frac{1}{T}\sum_{k=-\infty}^{\infty}X\left(\omega-\frac{2\pi k}{T}\right)=1\tag{2}$$
where $X(\omega)$ is the Fourier transform of $x(t)$. From $(2)$ we see that the shifted spectra need to add up to a constant. This is only possible if the maximum frequency of $x(t)$ satisfies $\omega_m\ge \pi/T$. For the minimum value $\omega_m=\pi/T$, the shifted spectra don't overlap and, consequently, $x(t)$ must be an ideal low pass signal with a flat spectrum. This is the only solution and it corresponds to the solution you came up with. If $\omega_m>\pi/T$, there are infinitely many solutions to $(2)$. One well-known example are the raised-cosine pulses.


Why do we get different imaginary parts of a zero centered Gaussian for the the same number of data points N?

Answer : When $x_2 = [-1024:1:1023]$, then $x_2[n]$ satisfies the condition $x_2[n] = x_2[(N-n)\mod N]$. That is why when $x_2 = [-1024:1:1023]$, then the FFT is real and hence the imaginary part is $0$. If you see the scale of the $y$-axis for imaginary part of $x_2$ plot, it is of the order of $10^{-17}$ which is almost $0$ in MATLAB.

Detailed Explanation:

When $x \in \{-1024, -1023, -1022,..., 0, 1, 2, ..., 1023\}$, then you get the following mapping :
$$\begin{array}{lcl}x[0] &=& {\rm gaussian}(-1024)\\
x[1] &=& {\rm gaussian}(-1023)\\
x[2] &=& {\rm gaussian}(-1022) \\
& \vdots\\
x[1024] &=& {\rm gaussian}(0)\\
x[1025] &=& {\rm gaussian}(1) \\
& \vdots \\
x[2047] &=& {\rm gaussian}(1023)\end{array}$$ Observe that $x[1] = x[2047] = x[(2048 - 1)\mod\ 2048]$, $x[2] = x[2046] = x[(2048 - 2)\mod \ 2048]$ and so on. This makes $x[n]$ real and symmetric mod $N$, which will in turn make $X[k]$ real and that is why you see that the imaginary part as $0$. MATLAB shows $0$ as values of the order of $10^{-17}$.

Do the same mapping for $x = [-1023:1:1024]$, and you will see that $x[n] \ne x[(N-n)\mod \ N]$ and hence imaginary part is not $0$.


How to mix two audio files recorded with different settings to obtain a realistic result

I have recorded Signal_A, and downloaded signal_B from the internet (therefore two different microphones/settings are used for recording)

You might be able to achieve a "seamless" mix (or transition) but the fact that you have two different microphones and settings might add artifacts in the final product. You can tell how this sounds like if you have ever noticed the differences between microphones. Accounting for such differences might be a bit more challenging.

  The sound pressure level of both signals are known( I measured the sound pressure level of Signal_A in 1m distance, and also know the sound pressure level of Signal_B in the same distance).

It is good that you have measured the SPL but most commonly you are unlikely to know the exact relationship that takes you from SPL to converted values.

  As far as I know, the value recorded/represented in each waveform of any signal is relative to that corresponding signal, and since two signals are recorded using different settings we can expect that one might be more amplified than the other. Therefore a simple mixing of the two won’t give us a realistic result.

I am not entirely clear on what you mean by "a simple mixing" but the first approach would be to simply adjust the amplitudes of the two recordings and then "add" them together as per your MergedSignal equation.

  Now the question is: is there a way to manipulate recorded signal_A and signal_B programatically, assuming the fact that the SPL values of the original signals at 1m distance is known, and get the merged signal as they were playing simultaneously and recorded at 1m distance?

Yes. BUT!, you have to make sure that the rest of the signal pathway was the same as well. Now:

Sound waves hit the transducer. The transducer converts them to voltage with a (hopefully) linear relationship of Volts per Pascal of pressure. In addition, this relationship is frequency dependent. But let's say that it is constant across the spectrum and call it $\alpha$.
Behind the transducer there is an amplifier whose job is to take the electrical signal and bring it up to some level. Again, this is achieved with a (hopefully) linear relationship. Let's keep things simple and assume that the job of this amplifier is to take the signal of the transducer and adapt it to a standard line level of approximately 2 Volts (peak-to-peak). Let's call this $\beta$.
The final stage is the Analog to Digital Converter (ADC). It converts the 2volts peak to peak to some range with a given word length (for example 0..255, -127..128 and so on).

So, assuming that $\alpha$ (mic), $\beta$ (amp/signal conditioning) are the same, you can indeed scale the recordings by the ratio of the SPLs. That is:

$$MergedSignal = (1.0 - \frac{SPL_A}{SPL_B}) \times Signal_A + \frac{SPL_A}{SPL_B} \times Signal_B$$

In the more general case and when you don't know the SPL, you can simply "match" the average level of the converted values. So, same relationship, instead of $SPL_A, SPL_B$ you have the averages of the signals.

Hope this helps.


Filtering White Noise to Generate Harmonic Signal

If you filter white noise with a narrow band filter that passes $e^{j\alpha t}$ and $e^{-j\alpha t}$ you will get a sinusoidal output at mean frequency $\alpha$ with an amplitude and phase that varies at the maximum rate according to the bandwidth of your filter.  The narrower the bandwidth, the lower the power (since the power for white noise is a distribution evenly spread over all frequencies), but the closer you will approximate a pure cosine signal at a random initial offset in phase and random initial amplitude (which will then still proceed to vary, although relatively  slowly compared to a winder band filter, in amplitude and phase).  If you could do an infinitely narrow filter (which you can't), you would also get no signal.


Upsampling is basically padding zeros in between signal samples and then interpolation. The interpolation itself is a low pass filter. The rcosdesign is a low pass filter so after zero padding by factor of 3, you need to convolve the resulting zero padded sequence with this filter.

Note that: it is not an ideal filter, so you would get varied attenuation for different frequencies for the discrete time frequency repsonse after convolving with this filter.

Further, remember you would be "drawing in" frequency components from other periodic repetitions of the discrete time frequency response of the original sequence immediately after zero padding, these would appear in the high frequencies of your $-\pi$ to $\pi$ spectrum hence roll off and spectrum of original sequence would decide the amount of alisaing and distortion of the original signal's frequency response.

As an example if your original sequence is bandlimited in $-\pi/4$ to $\pi/4$. Then this would shrink to $-\pi/12$ to $\pi/12$ after upsampling by 3 but you would also get frequency components to the left of $-7\pi/12$ and right of $7\pi/12$. So in this example make sure your roll of begins at $-\pi/12$ and $\pi/12$ and frequency resonse of filter decays very much to 0 before $7\pi/12$ and $-7\pi/12$

Sample MATLAB code below:

Sampling period  implied and ignore the time scale:

close all;
clear all;
clc;

t=-5:0.1:5;
T=5;
RECT=rectpuls(t,T);
FFT_size=1024;

signal_fft=fft(RECT,FFT_size);

signal_up=upsample(RECT,3);
signalup_fft=fft(signalup,FFT_size);
h=rcosdesign(0.25,64,2);
H=fft(h,FFT_size);
Product=H.*signalup_fft;
filtered_sequence=ifft(Product,FFT_size);

figure;
subplot(6,1,1);
plot(t,RECT);
ylim([-2 2]);
xlim([-5 5]);
title('original signal');
subplot(6,1,2);
plot(fftshift(abs(signal_fft)));
title('frequency response of original signal,periodicity implied');
subplot(6,1,2);
plot(fftshift(abs(signal_fft)));
title('frequency response of original signal,periodicity implied');
subplot(6,1,3);
plot(fftshift(abs(signalup_fft)));
title('frequency response of upsampled signal,periodicity implied');

subplot(6,1,4);
plot(fftshift(abs(H)));
title('frequency response of rcosdesign filter,periodicity implied');

subplot(6,1,5);
plot(fftshift(abs(Product)));
title('frequency response of filtered sequence,periodicity implied');

subplot(6,1,5);
plot(filtered sequence);
title('recovered signal, ignore time scale');


This is the common case for receiving spread spectrum signals since we do not know carrier frequency and phase offset but can determine that from the complex output of our correlators.

Each correlator (Early, Prompt and Late) would be complex; the complex Prompt correlator output is used for Carrier Recovery (measure rate of rotation or change in phase from one complex symbol to the next), and the magnitudes of the Early and Late correlators are used for Timing Recovery.  If you are using spread spectrum that once carrier offset is removed only resides on the real axis (BPSK modulation), then the Early and Late Correlators can be simplified to be real only as long as the carrier offset is removed prior to timing recovery and carrier tracking is implemented in such a way to rotate the signal to the real axis. This is not necessary as complex Early and Late correlators can optionally be used in which case you would use the magnitudes to derive timing.

Here is a link to a decent online tutorial showing the use of the complex (I and Q) correlator outputs and further details timing recovery using Early-Late.

http://what-when-how.com/a-software-defined-gps-and-galileo-receiver/carrier-and-code-tracking-gps-and-galileo-receiver-part-3/

From which I clipped this diagram:

Additional methods to determine the error that I didn't see mentioned in the tutorial (after a quick review, perhaps they are detailed there as well):

$err = Re((P)^*(E-L))$

Where $(P)^*$ is the complex conjugate of the complex Prompt correlator output ($I_P-jQ_P$) sampled at the estimated symbol timing location, and E and L are the complex correlator outputs 1/2 a chip earlier ($I_E+jQ_E$) and 1/2 a chip later ($I_L+jQ_L$) than the estimated symbol timing location.  The operation $Re()$ is to take the real part of the result: if there is no carrier offset, the phase of P should be the same as the phase of E-L on average such that the complex conjugate multiplication will rotate the result to the real axis, however due to noise contributions or possible carrier offset the result will have an imaginary component so the Re() operation is to remove that.

Multiplying out this form of the error discriminator based on I and Q outputs of each correlator results in:

$$Re((I_P-jQ_P)(I_E-I_L-jQ_E+jQ_L))$$

$$ = I_P I_E - I_P I_L - Q_P Q_E +Q_P Q_L$$

If the signal is rotated to the real axis first using carrier recovery then this can be $P(E-L)$ were P, E and L are the output of the I (Real) correlator only in this case.

By multiplying by P as in both approaches above we both normalize the result and strip the data modulation from effecting the error result.


Reason of effect of continuity and discreteness on periodicity of signal?

To claim that "sampling changes the continuity and periodicity of a signal", implies that given a signal $x(t)$ and its sampled version $q(n) = x(n \cdot T), n \in \mathbb{N}, T = \frac{1}{Fs}$ there is a way to "transform" $x(t)$ in an aperiodic $q(n)$ just by periodic sampling.

This is impossible, especially for linear systems that cannot go into oscillations out of their own accord.

The only thing that you do with sampling is "tuning in" to a part of a signal's spectrum. Here is why, given a periodic signal at some frequency:

$$x(t) = sin(2 \pi f t)$$

We are going to sample it with a periodic train of pulses:

$$q(t) = \text{III}_T(t) \cdot x(t) \implies \frac{1}{T} \sum_{n=-\infty}^{n=\infty}e^{i 2 \pi n \frac{t}{T}} \cdot x(t) \implies $$

$$\frac{1}{T} \sum_{n=-\infty}^{n=\infty}\left( \cos\left(2 \pi n \frac{t}{T}\right) +i \cdot \sin\left(2 \pi n \frac{t}{T}\right) \right) \cdot x(t)$$

This expression by itself, does not guarantee that $q(t)$ is not going to be aliased and this is why $x(t)$ is first going through a low pass anti-aliasing filter. After this operation, sampling, is putting a "canvas" over that antialiased spectrum (discretises it).

How it is then possible that the operation described above can generate an aperiodic $q(t)$ for a periodic $x(t)$?

The only thing that this operation does is sample from the already existing components of $x(t)$ which are periodic functions. If $x(t)$ contained discontinuities, so would $q(t)$. If $x(t)$ was aperiodic, so would $q(t)$.
At the end of the day, what you do here is a straightforward mapping. You don't change any "type of signal".

The case that robert-bristow-johnson mentions is a result from number theory, transposed to periods of sinusoids. You cannot compose an irrational number out of integers and this tiny little difference that remains after each period presents itself as a "running phase" of the sinusoid.

There are however cases where sampling can have an effect on the signal, BUT that is entirely resonable. In the integration of non-linear differential equations by naive schemes such as Euler's method, the "sampling" that is effected by choosing a fixed step size can lead to large errors in the reconstructed waveform and more often than not in cases like this, the sampling becomes irregular. More dense around areas of large errors and vice versa. But even in that case, periodic regions (in the input) would remain periodic in the output.

Hope this helps.


By taking the absolute value of the derivative of each signal, I was able to get the index of the wobbly signals I desired.

Here are the top 100 wobbliest signals in my wobbly dataset:

Here are the top 100 wobbliest signals in my non-wobbly dataset:


need general resource about image matching

Check out Szeliski's book:
http://szeliski.org/Book/drafts/SzeliskiBook_20100903_draft.pdf

There is also an old book on feature detection:
http://www.amazon.com/Feature-Extraction-Processing-Computer-Edition/dp/0123965497

You can read the sections that you care about. Also, I think it is always a good idea to read about scale space theory if you are to work with such algorithms. For example:

http://www.nada.kth.se/~tony/book.html


OFDM synchronization using training symbols

Claim

In OFDM receivers, special OFDM training symbols for which the data content is known, can be used to perform the synchronization process.

Be a bit careful about that statement.
There's basically four kinds of synchronization that I can think of right now:

Frequency synchronization
Timing synchronization
Phase synchronization
Clock synchronization

... as well as higher-level things such as frame sync, decoder sync.
Frequency sync
Can be done in a number of ways. A very OFDM-typical way of doing so are autocorrelation methods.
With these methods, you don't actually use a known reference symbol, but send OFDM symbols that have a specific temporal structure.
For example, the well-known Schmidl&Cox synchronization is based on the idea that if you have two preamble symbols with certain symmetry, you can do both a fine frequency estimate as well as a timing estimate.
Rough frequency estimates are often done by exploiting the fact that OFDM systems usually don't use all subcarriers, but leave guard bands at the edges. So, for this, too, no known symbol is needed.
Timing sync
... to expand on the above: It's very easy to build a timing estimator based on a fixed lag correlation, when you have a symbol that repeats itself. You just compare $N$ samples with $N$ samples a fixed delay $\Delta T$ later; when that correlation function sees a peak, you might be detecting your preamble symbol. Side effect is that the phase of that correlation (divided by $\Delta T$) serves as fine frequency estimate.
DAB (and DAB+), for example, use a non-Schmidl&Cox autocorrelation based frequency and timing estimator. WiFi uses S&C, for example.
Clock Sync
That's a tough one with OFDM, as it's usually pretty bad if the receiver sampling clock is different from the transmitter's clock. It's solved by limited symbol length, or by coupling the carrier frequency to the sampling clock (so that you can correct the sampling clock by correcting the frequency), or by estimates based on the phase sync.
Phase sync
This is really the only thing that is normally done through known data. But it's commonly not done by known symbols, but only by known pilots within distributed "ressource blocks", i.e. single subcarriers within payload symbols, as that has a lot of advantages by observing the channel as its being used.

Practically, how these OFDM symbols are generated at the receiver?

not at all. They are received, just like every other OFDM symbol. That's why it's called a receiver.
The known pilot subcarrier symbols are typically written in some standard document.

If these reference symbols are sufficient for both symbol timing and frequency offset determination,

They aren't. Frequency synchronization is usually done with autocorrelation methods, see above.

why differential detection is often employed?

Are they? Unbacked claim! I know that DAB does use OFDM with differential PSK internally, but that is kind of an odd duck: It's from the early 90s, so that OFDM was feasible on small systems, but proper channel estimates weren't. Generally, using OFDM says "need to use a wideband channel efficiently", and that usually means that you really want that non-differential modulation BER goodness.

The information bits are modulatedon the subcarriers by differential QPSK.

You're referring to some specific system, not to something that applies to OFDM in general. So, this is, in general, a false statement.

The reference phase and amplitude of each subcarrier have to be known in order to estimate the bits correctly. For this purpose differential detection is used.

This is self-contradictory. Either you need to know the phase of each subcarrier, or you do differential modulation. Never both.


Efficient implementation of FIR comb/decimating filter

"Is there a computationally efficient way to take advantage of the fact that most of the taps are zero?"

is there a pattern to those taps (or to the taps that are potentially non-zero)?  can you skip over those taps that are known in advance to have zero coefficient?


Sine of frequency 0 contains sines of all frequencies at once in it

You are confusing time domain and frequency domain. A constant time domain function (a sine with frequency 0, if you like) corresponds to a delta impulse at the origin (i.e., frequency zero!) in the frequency domain. A delta impulse at frequency zero is zero for all other frequencies $\omega\neq 0$. Consequently, a constant in the time domain is not a combination of all frequencies. What you were probably thinking of is a delta impulse in the time domain, which corresponds to a constant in the frequency domain (i.e., it contains all frequencies).

So, summarizing, a constant in one domain corresponds to a delta impulse in the other domain. A constant in the time domain does not contain any frequencies other than 0 (delta at zero in the frequency domain). A constant in the frequency domain (i.e. a combination of all frequencies) corresponds to a delta impulse in the time domain.


The Difference Between Bilateral Filter and Gaussian Filter

The Gaussian filter corresponds to isotropic diffusion (or the heat equation if you are familiar with it).
Its application to an image will blur it uniformly, whatever the image content, edges, etc.

The bilateral filter is almost like a Gaussian filter, except that the Gaussian is modulated by a function that computes the similarity between the central pixel (where the filter is applied) and a pixel in its neighborhood (that is used in blurring).

If the two pixel values are very close, it multiplies the Gaussian coefficient by something close to 1, and hence it is equivalent to Gaussian filtering.
If the pixel values are very different , it will multiply the Gaussian coefficient by a number close to 0, thus turning off the Gaussian filtering for this pixel.

Intuitively, this behaviour yields the following result: Gaussian filtering in uniform areas of the image, no filtering across object borders.
The bilateral filter will produce a more pleasant results, because it will avoid the introduction of blur between objects while still removing noise in uniform areas.


I tried something that I didn't thought of before:

force $\omega_0$ to be a multiple of $2 \pi \frac{fs}{N}$, with $fs$ my sampling rate.

And of course, this is what happens in the fft.

I only ask myself if I can get the same results for $\omega_0\neq2 \pi \frac{k}{N} fs$


What is the normalization factor for the quincunx lattice wavelet transform, and how do you find it?

I don't think there is any single best number for normalisation because it depends on the structure of the values in your lattice.

In the simplest case where all values are equal, the predict operation zeros the black points and the update does not change the white points.  Because each  predict-update pair halves the number of nonzero points, multiplying the lattice by sqrt(2) after each pair of steps would conserve energy.

With all values independent with zero mean and equal variance, the predict step multiplies the variance of the black points by 5/4 and then the update step multiplies the variance of the white points by 281/256 so energy increases at each step.


MATLAB has amdemod (see MATLAB documentation) which can be used to recover suppressed carrier AM modulated signal. From the documentation, the demodulator uses a low-pass filter generated using [num,den] = butter(5,Fc*2/Fs). This can be a good start.


What framesize is usually used for audio effects?

Typically this requires a trade off between CPU efficiency and latency. Frame based signal processing incurs at least one frame of latency and it you are planning to play live through the effect, you need to keep this quite low otherwise it get's distracting.
On the other hand, frame-based processing is way more efficient than sample by sample processing, since you can use DMA, vector instructions, highly optimized loops etc and you can amortize setup (coefficient calculation and loading, loading and storing of states, etc.) over the entire frame.
For live audio effects which moderate algorithmic complexity, something like 32 or 64 samples at 44.1 kHz or 48 kHz typically is a good starting point.


What are reds and trans in Image processing

The numbers appear to be RGBA values: red, green, blue, and alpha (transparency).

reds appear to be mostly red-tinted colors (except the last entry, which is a completely transparent  item).

trans is the transparent "color".


STFT amplitude normalization, librosa library

I did answer a similar question a few years back, but can't find it. Basically, you are losing the energy because of the windowing. It's true to say that you should multiply the spectra by 2 in order to retrieve the energy from negative frequencies and divide it by the window size. However, this is only true for the rectangular window. The correct normalization should be following:

out = 2 * np.abs(out) / np.sum(window)

where np.sum(window) in case of rectangular (or no window in fact), is equal to the frame size. It kind of makes sense, right? If window attenuates the signal at its ends, then this energy should be recovered somehow.

Now back to your problem, librosa.stft by default uses hann window which looks like:

If you calculate the np.sum(np.hanning(window_size)) you will get $511.5$. If you read up more, the coherent gain of that window is approximately $0.5$. Which is defined as:

$$CG = \frac{\Sigma w(N)}{N} $$

So in order to solve your problem, I suggest to explicitly pass the window vector to the stft function:

import librosa
import numpy as np
import matplotlib.pyplot as plt

frequency = 1000
fs = 4 * frequency
time_vector = 1/fs * np.arange(20000)

sine_wave = np.sin(2 * np.pi * frequency * time_vector)

window_size = 1024
window = np.hanning(window_size)
hop_length = 512

out  = librosa.core.spectrum.stft(sine_wave, n_fft = window_size, hop_length = hop_length, window=window)
out = 2 * np.abs(out) / np.sum(window)

fig = plt.figure()
axes = fig.add_subplot(111)
im = axes.pcolormesh(out)
fig.colorbar(im)

plt.figure()
slice = out[:, 10]
plt.plot(slice)
plt.grid()

plt.show()

Which will return:


Determine the system function H(s) of a system and find out the differential equation

Your transfer function looks correct. Note that you can rewrite $H(s)$ as

$$H(s)=-\frac{3s}{5s+3}\tag{1}$$

or, equivalently,

$$5sY(s)+3Y(s)=-3sX(s)\tag{2}$$

Since multiplication by $s$ corresponds to differentiation in the time domain you obtain from $(2)$

$$5y'(t)+3y(t)=-3x'(t)\tag{3}$$


Using MATLAB to define a function in signal processing 101

Well, this is Octave but should work in Matlab:

function y=Y(t)
  y=ones(size(t));
  y=y+(t<=0)*(-2);

.. and it is vectorised. Calling will look like:

>t=linspace(-T,T,1000);
>y=Y(t);

To make this more general we could also make the time at which the function switches a parameter also:

function y=Y(t,S)
  y=ones(size(t));
  y=y+(t<=S)*(-2);


Easiest Pattern to Recognize with Machine Vision

In case you can shoot a video of the static scene than a blinking light would be the easiest as you could easily detect it by subtracting the n - 1 frame from the n frame until you see something with high values.
If you take a still shot you can use 2 main ideas:

If the colors of the scene are from a given plate, find a color very different in Hue and make sure it is intense (Even active lighting which will saturated).
Use a pattern which is easily detectable like stripes of B/W or pure green and magenta. You may encode data in the stripes.

Resources
Some resources pasted here in comments:

Fiducials / Fiducial Markers.
AprilTag 3 - A visual fiducial system popular in robotics research.
What the Name of This Kind of Computer Vision? (Related question).


Matlab stores all level of Wavelet's Approximation & Details in a single Row

You can use the function dwt instead that does single-level decomposition and use that again for 2-level decomposition. Personally, I use wavedec all the time, I actually find it easy to have it in one array. Boils down to personal preferences I guess.


The green output in the video looks like it's just the blocks of the filtered data, with transients at the beginning of each block. That's probably why the spectrum always looks the same, you're just seeing the blocking artifact / frequency. You are probably starting the filtering from scratch every time, whereas you need to keep the data (state) of the filter from one block to the next.


Best DSP algoritms for ultrasonic background noise cancellation

One way to do this is to look at modeling your signal:
$$
x[n] = x_h[n] + x_n[n]
$$
where $x_h$ is the hissing sound and $x_n$ is the noise.

If you can say that $x_n$ is modeled as:
$$
x_n[n] = \sum_{k=K_1}^{K_2} a_k \sin(k\omega_0 n + \phi_k)
$$
where $K_1$ is the lowest harmonic of frequency $\omega_0$ that makes it through your high pass filter, $K_2$ is the highest harmonic, $a_k$ is the amplitude of the $k^{\rm th}$ harmonic, $\omega_0$ is the fundamental frequency and $\phi_k$ is the absolute phase of the $k^{\rm th}$ harmoninc.

Then you can estimate $(a_k,\omega_0, \phi_k)$ by minimizing
$$
\left| x[n] - \sum_{k=K_1}^{K_2} a_k \sin(k\omega_0 n + \phi_k) \right|^2
$$
to find $(\hat{a}_k,\hat{\omega}_0, \hat{\phi}_k)$ and then look at the residual:
$$
\hat{x}_h[n] = x[n] - \sum_{k=K_1}^{K_2} \hat{a}_k \sin(k\hat{\omega}_0 n + \hat{\phi}_k)
$$

This should allow some cancellation of the harmonic background noise, but won't be real time.  There are algorithms that can do the estimation in real-time, but you're best trying this first on recorded data to see if it works.


FFT: Extracted spectrum depends on number of points sampled

If I use Y=fft( a(1:3:end) ), then I get frequencies at 0 and nonzero frequencies:

Well, yes, but that means you decimate your signal by a factor of 3, which on one hand will shift frequencies to $\frac13$ of their original frequency, but also will lead to aliasing. In other words: unless you know there's no signal above $f=\frac16f_\text{sample}$, you simply can't do that.

EDIT (based on zoomed-in plots):

So, hand-counted from your most-zoomed in plot, it seems there's about 5 peaks in 20 samples, which implies a period of 4 samples.
That is ok by Nyquist's sampling theorem.

Now, if you just look at every third sample, which you do by doing a(1:3:end), then there's only ~1.333 samples per period. Nyquist demands that there's at least 2 samples per period. So, your second attempt aliases the signal from the $\frac15f_\text{sample/2}$ onto a different point in the spectrum, where it actually isn't. You damaged your signal by undersampling; you mustn't do that. It's not very clear to me how the idea to do that happened in the first place, but you should definitely read up on the Sampling Theorem.


Converting Google's text-to-speech voice to, say, Justin Bieber's voice

How we should go about from converting Google Text to Speech's m3 file to Justin Bieber's voice?

Ask Mr. Bieber to repeat what your MP3 says.

I think letting one speech synthesizer first speak something and then hoping to convert it is a bad approach (as you can see if you consider how complicated that would be – first, you need to recognize in some low-level way the things that the speech synth said, then re-synthesize them with a Bieber voice synth).

Instead, "just" implement a Bieber voice synth. That will "only" require you to have a model of the target voice, instead of another model for utterance recognition.

I'm not deep enough into voice synthesis, but common voice models have a metric ton of parameters that you can tweak – maybe you can tweak a male voice model until it resembles Mr. Bieber's voice?

I honestly think you might be widely underestimating the nuances and number of parameters that human voice has.

For example, record you saying something, and then record someone from two generations older than you (or younger than you, whichever is easier to find) say the same thing. Compare rhythm, compare amount of pauses, compare stress on syllables and words – you can't just go ahead and convert your voicing into someone else's voicing just by means of simple transformation.


Part 2: Root Locus, Transfer Functions and Unit Step Response?

From the root locus plot, the open loop transfer function has a real-valued pole at $s_0=0$ and a complex conjugate pole pair $s_1=-1.5+ 2j$ and $s_1^*$. There's also a zero at $s=-2$. Consequently, $G(s)$ is given by
$$G(s)=\frac{s+2}{s(s-s_1)(s-s_1^*)}$$
The closed-loop transfer function is
$$H(s)=\frac{kG(s)}{1+kG(s)}=\frac{k(s+2)}{s(s-s_1)(s-s_1^*)+k(s+2)}\tag{2}$$
The final value of the step response equals $H(0)=1$, which leaves options $A$ and $B$.
Looking at the root-locus plot, we can see that the real-valued pole is (a bit) closer to the imaginary axis than the complex-conjugate pole pair. So that real-valued pole is (slightly) dominant. Based on this observation one can guess that step response $A$ is correct because it clearly shows the influence of the real-valued pole (i.e., a smooth increase with no overshoot), whereas step response $B$ just looks like a step response of a second-order system with a complex-conjugate pole pair.
In sum, apart from looking at the final value of the step response given by $H(0)$, you can check if there are one or more dominant poles that determine the behavior of the step response. You should be familiar with the step responses of first and second-order systems; this will allow you to make an educated guess also for higher order systems. This website gives a good overview of step responses of first, second, and higher order systems.


Mathematically Inclined Signal and Systems / Signal Processing Book Recommendations

Oppenheim and Willsky's Signals and Systems or Lathi's Linear Systems and Signals are intended for Sophomores who have only a single semester of differential equations under their belts, so it is a bit unfair to criticize them for leaving out the functional analysis and the conformal mapping.  At the sophomore level my favorite book is Siebert's Circuits, Signals, and Systems.  It won't give you the mathematical rigor you desire either, but you can see his great love for the mathematics and he has these wonderful, witty, footnotes that provide a great historical perspective.

There is a great book (which I love, but do not recommend to you) by the (applied) mathematician Richard Hamming (of the "Hamming window", "Hamming code", "Hamming distance", "Hamming bound" and "Hamming problem") called Digital Filters.  In it he makes a number of snarky comments like:

  Since we are interested mainly in using mathematics, we are obliged in our turn to be ambiguous with respect to mathematical rigor.  Those who believe that mathematical rigor justifies the use of mathematics in applications are referred to Lighthill and Papoulis for rigor; those who believe that it is the usefulness in practice that justifies the mathematics are referred to the rest of this book.  (1998 Dover edition, page 72.)

So in addition to the book by Papoulis that @Matt_L recommended I will add Hamming's (and Siebert's!) recommendation of M.J. Lighthill, Fourier Analysis and Generalized Functions, Cambridge University Press, 1958.