Power spectral density of sinusoidal signal in noise

It looks like you are looking at the result of a combination of things.

First of all let us observe the image and try to understand it. The shredded horizontal line with value $\frac{N_{in}}{2B}$ represents $n(t)$ in the frequency domain and the signal $s(t)$ is represented by the two vertical arrows valued $\frac{A^2}{4R_L}$. Two arrows one in the sinus's frequency value and its mirror in the negative side is a representation of a sinuous wave in the frequency domain. The cosinuous wave will have the a similar representation as the sinuous wave in the frequency domain.I will reffer you to the table here

A BPF is a band pass filter and it is a combination of a low pass filter (will only pass frequencies up to a specified $f_0$ frequency) and a high pass filter (will only pass frequencies up from a specified $f_0$ frequency). In short, a BPF will only pass frequencies from a specific band bounded between some $f_1$ and $f_2$ as in $f_1\leq f\leq f_2$. The BPF is represented as a window in the frequency domain. You can see in your image the window with $B$ width around the sine arrows. This is also the reason the noise $n(t)$ is shredded, The filter is cutting it off outside the window's boundary. In the time domain a BPF will be represented by a $sinc()$ function, as can be seen in the table in this page.

I think your question needs additional data, since it lacks an explanation of what exactly $R_L$ is. I am assuming it is a resistor and this is some sort of an electronic circuit but if you want any further help you need to supply more detailed on the system.


How to filter OFDM Signal with an analog filter

same as your previous RRC question: you simply apply a digital filter that approximates the analog filter.
There's no analog signals in your PC, and OFDM is a digital signal anyway, so only the digital equivalent of the analog system would matter to the receiver, anyway.
Look at it this way: You're supposed to simulate a digital system. Therefore, the receiver is digital. Therefore, the analog effects that would happen in a real-world transceiver system are only relevant in their digital form.
And if Nyquist has taught you anything, any band-limited analog system can be represented through a digital one.
So, stop worrying about building analog filters, but describe the thing you want to simulate and then implement that digitally.
(I mean, really! Look at the purpose of OFDM: it's a purely digital thing, meant to do the inverse to a purely analog multipath channel, which is an analog filter. It works. You don't have to model anything in analog, unless you're not really understanding what OFDM is about.)


Is the system represented by the equation $y(t) = x(2t)$ time invariant?

From your solution:

I followed the following algorithm:

$$ y(t) =x(2t) $$
$$ y_1(t) = x_1(2t)$$

Let  $$x_2(t) = x_1(t-t_0)  ~~~\text{and}~~~ y_2(t) = x_2(2t) $$

On this following step (time sampling of the shifted argument) you make the usual mistake:

$$\implies y_2(t) = x_1(2(t - t_0))$$ which should instead be :
$$\implies y_2(t) = x_2(2t) = x_1(t - t_0)|_{t=2t} = x_1(2t - t_0)$$

The remaining parts follow as usual to show that the time scaler is a time-varying system...


Why do we use the top left corner as the origin in image processing?

I propose that the raster scan is a Western invention, in analogy with the way we write: left to right, top to bottom. Remember that "rastrum" denotes a  rake. When you rake a surface, or paint a floor, one usually does this such that one does not step on the raked or painted surface. The "raker" or "painter" moves backwards. One wants to avoid the situation described in Painted Into A Corner:

This defines a sort of 2D causality, using data for above and if not, from the same level (row) but left.  Parsing the 2D space top to bottom then left to right could have been an option. However, we know that the human vision is more sensitive to vertical than to horizontal, as one can see from the JPEG quantization matrix.

So perhaps, by going left to right first, the human mind could more easily spot a vertical change in a whole picture. *

However, the ancient slowness of a beam parsing a screen is not less important. For web browsing, it can be more important to first display a global low resolution patch and refine it iteratively, like in the progressive JPEG, JPEG 2000, or the clever Adam7 algorithm used in PNG.

See for instance http://www.libpng.org/pub/png/pngpics.html


Representing a continuous LTI system as a discrete one

The sequence $h[n]$ consists of samples of a low pass filtered version of $h_c(t)$. Define that low pass filtered impulse response as

$$\tilde{h}_c(t)=\int_{-\infty}^{\infty}h_c(\tau)\frac{\sin(\pi(t-\tau)/T)}{\pi(t-\tau)/T}d\tau\tag{1}$$

and

$$h[n]=\tilde{h}_c(nT)\tag{2}$$

The output signal is

$$\begin{align}y(t)&=(x\star h_c)(t)\\&=\int_{-\infty}^{\infty}x(t-\tau)h_c(\tau)d\tau\\&=\int_{-\infty}^{\infty}\sum_{k=-\infty}^{\infty}x[k]\frac{\sin(\pi(t-kT-\tau)/T)}{\pi(t-kT-\tau)/T}h_c(\tau)d\tau\\&=\sum_{k=-\infty}^{\infty}x[k]\int_{-\infty}^{\infty}h_c(\tau)\frac{\sin(\pi(t-kT-\tau)/T)}{\pi(t-kT-\tau)/T}d\tau\\&=\sum_{k=-\infty}^{\infty}x[k]\tilde{h}_c(t-kT)\tag{3}\end{align}$$

Consequently, we have

$$\begin{align}y(nT)&=\sum_{k=-\infty}^{\infty}x[k]\tilde{h}_c((n-k)T)\\&=\sum_{k=-\infty}^{\infty}x[k]h[n-k]\end{align}\tag{4}$$

This results is also obvious from the fact that $(x\star h_c)(t)=(x\star\tilde{h}_c)(t)$ must hold, because $x(t)$ is band-limited. And since $\tilde{h}_c(t)$ is also band-limited, we can simply take samples from $x(t)$ and $\tilde{h}_c(t)$ to obtain the result.


Spectrogram for created signal in Octave

Try this:

fs = 100;
t = 0:1/fs:10;
x = [sin(2*pi*7*t), sin(2*pi*12*t), sin(2*pi*2*t)];
specgram(x,256,1000, [],200);

I have reduced the sampling frequency because the three frequencies in your signal are quite low. In the specgram() function, I have increased the FFT size to 256 and used the default hanning(256) window with 200/256 $\approx$ 80% overlap.


I understand the sampling process is completely handled by the sensor itself, i.e. it delivers digital data. Then you just collect the desired number of samples from the sensor and calculate its FFT. The frequency range depends only on the sampling rate ($f_{\text{max}}=f_s/2)$, the frequency resolution depends only on the number of samples $N$, in your case $N\in[64,2048]$.

There is no sampling done on the uC, as it already receives sampled data. However, what you will want to do is employ a lowpass filter before calculating the FFT, with a cutoff frequency slightly lower than $f_{\text{max}}$, to get rid of high frequency content, that would otherwise cause aliasing in your spectrum.


Preserving integral through downsampling

Increase the gain of your filter by the downsampling factor.

You do this by multiplying the filter taps by the downsampling factor.  This is equivalent to doing exactly what you have done before and then multiplying the results by the downsampling factor.


Finding baseline (reference) of nosiy data and lag/lead between 2 signals

You can try the function 'ccf' in R.
The way I would setup your data is:
df=
time | red | black | red diff | black diff |

** make sure your time interval's are constant. i.e every 5 seconds/10 seconds etc

red diff and black diff being the difference between the adjacent values in the red and black data columns, respectively.
i.e. diff(df$red) = red diff

Then do a ccf on the 2 difference columns and adjust 'lag.max' as necessary to view the plot that is generated. There should be a peak in the graph which is the lag (that occurs the most) between red and black.


To make it more clear, I suppose your question is

  Why it is said that the compressor gain at low input amplitudes is higher, while the step size of a nonuniform quantizer is small in that region. Similarly, Why it is said that the gain of the compressor is higher for high input amplitudes, while the step size is larger for those inputs.

First, notice that nonuniform quantization is used to improve the average SNR by adapting the quantizer to the source probability density function. Of course, when the source pdf is not uniform (e.g. when it is Gaussian), then uniform quantization is not a good choice. To better understand it, assume that you want to quantize a Gaussian source into $K$ levels. Let's first assume we have a uniform quantizer. One way to reduce the distortion caused by qunantization is to increase the number of steps ($K$). Increasing $K$ means the step size is reduced which eventually leads to a higher SNR (finer quantization). Increasing the number of levels means that the total number of bits to represent each sample is increased. For instance if we have 256 levels we need 8 bits to represent each level. But for 1024 levels we will require 10 bits.

Now in the above example assume that we want to work with a fixed $K$ (e.g. 256) and improve the SNR. To do this, notice that a Gaussian source is more likely to produce samples around the mean. So for example, a zero-mean Gaussian source is very likely to generate values close to zero. So we allocate smaller step size in the area around zero, and instead make the step size larger at the tail of the pdf (where it is less likely to happen). So technically, we quantize with a smaller step size on average (but without increasing the number of levels).

Any scalar nonuniform quantizer can be implemented by using a chain of

1- A compressor: a memoryless monotonic nonlinearity $G$

2- A uniform quantizer

3- An expander: inverse of nonlinearity $G$

In your example, the nonlinearity has a large gain at the origin (the characteristic function is steep) while it causes some sort of saturation effect at large amplitudes. But this gain is applied on the input signal not on the quantizer. Still the result should be quantized by a uniform quantizer afterwards. Although it is a uniform quantizer, but the preliminary change caused by the nonlinearity $G$ and then $G^{-1}$ makes it seem as if different step sizes are used: The low amplitude bins are magnified first, then there is a uniform grid, then they are compressed. They seem as they are quantized by a small step size. Something similar also applies to the high amplitude bins, except they are compressed first, then they are expanded. So the overall result is equivalent to the effect of a large step size.


Causal Noise Free 1st Order Derivative in Discrete Domain

Well, Since basically the Derivative Operation is Linear Filter you can chose your own optimal trade off between Noise Sensitivity and Bandwidth.

If you look at Finite Differences Coefficients page at Wikipedia you can see you can chose higher "Accuracy" filters for 1st Derivative.
You can also chose Forward / Central / Backward method.


How do you program a completely transparent crossover?

I'm no expert but I'll explain how I would approach this. In general, crossover design involves selecting your respective cutoff frequency (or frequencies, in your case) and desired attenuation or filter order, and deciding on a filter type. Trivially, a Butterworth filter is a popular choice as they are easy to design and implement. According to Wikipedia, a better choice may be the Linkwitz-Riley filter:

  A Linkwitz–Riley (L-R) filter is an infinite impulse response filter used in Linkwitz–Riley audio crossovers...It is also known as a Butterworth squared filter. A Linkwitz-Riley "L-R" crossover consists of a parallel combination of a low-pass and a high-pass L-R filter. The filters are usually designed by cascading two Butterworth filters, each of which has −3 dB gain at the cut-off frequency. The resulting Linkwitz–Riley filter has a −6 dB gain at the cutoff frequency. This means that summing the low-pass and high-pass outputs, the gain at the crossover frequency will be 0 dB, so the crossover behaves like an all-pass filter, having a flat amplitude response with a smoothly changing phase response. This is the biggest advantage of L-R crossovers compared to Butterworth crossovers, whose summed output has a +3 dB peak around the crossover frequency.

Wikipedia on L-R Filters

I would design these such that the bands of interest are all separated out nicely, either through some sort of series configuration or by explicitly designing bandpass filters instead of lowpass/highpass filters (except for your lowest and highest bands of interest, of course).

This could either be done by designing the filters as analog and then using a bilinear transform to bring them into the Z-domain, resulting in a form that could be implemented in your language of choice, OR could be done digitally from the get-go in Matlab or Python (using SciPy's iirdesign() function).

SciPy's IIR Design Function

Then, you would need to implement them in whatever language/platform you're using (although I would hazard a guess that its C++ on Windows/Macfor a VST?).

Additionally, the concept of a "transparent" filter is something of a oxymoron. Either you are going to affect your signal or you aren't, and everything will involve some compromise. For instance, better filtering in the frequency domain may result in poorer performance in the time domain. Choosing what you can "ignore" is key to a good design.


What is the advantage of multiple antenna in receiver in energy harvesting ?harvested more power?

After thinking,i think when the antenna number of device B become 2,the harvested power of B will not become $0.1W \times 2$,WHY?

Because when there is just one antenna in B device,the beamforming direction just aim that antenna,because just that antenna has to receive or harvest the signal,so that antenna can harvest all the RF energy

However,if the antenna number become two,device A has to let both of two antenna of B receive or harvest the signal,so the beamforming direction has to change,maybe the middle of these two antenna,so now first antenna will harvest about $0.05W$,and the other will receive $0.05W$ as well.Or perhaps $0.06W$ and $0.04W$,but the total power will not be larger than 0.1W


Meaning of Multiplying a complex number with its conjugate

Multiplying a complex number with it's conjugate yields the squared magnitude of that complex number.

However, in your formula, you are multplying the signal with the conjugate of the N samples delayed signal, take the squared magnitude of that (thus dropping phase differences between the signal and delayed signal, which would result from a frequency offset) and average it over Ng sample times. This means you are calculating the autocorrelation of the signal at a time delay of N sample times.

I assume the formula is from an algorithm for coarse symbol timing estimation for OFDM signals. The pseudo-random pilot sequence of length N is repeated twice. It's autocorrelation thus has a peak at N. Now, if you want to estimate the timing offset, you just take the time index of this peak.


Determine impulse response given input and output: which ROC?

The function $H(z)$ without a specified ROC is generally not sufficient for uniquely determining the time domain sequence $h[n]$. You need more information about the desired $h[n]$. There are three types of ROCs:

the ROC is an annulus ($R_1<|z|<R_2$): $h[n]$ is a two-sided sequence (non-causal)
the ROC is defined by $|z|>R$: $h[n]$ is a right-sided sequence
the ROC is defined by $|z|<R$: $h[n]$ is a left-sided sequence

If you're looking for a stable system, then the ROC needs to include the unit circle $|z|=1$. If that does not uniquely define the ROC, then you need to know if you're looking for a left-sided, a right-sided, or a two-sided sequence.


How to implement discretized CWT with FFT?

To be honest, if I want to use a CWT, I'll use the toolbox Matlab provides.

I have every confidence that I could code a CWT, but an efficient, debugged, and tested set of routines is not an interest.

There are a number of open source wavelet codes that a Google search will return, such as

https://stackoverflow.com/questions/9606458/looking-for-a-good-c-c-wavelet-library-for-signal-processing

so there is code available that you can use and/or modify or just get ideas from.

Also, you haven't posted what you have so far produced, code or equations.  No one can offer any concrete suggestions or advice given your description.


Quadrature component - condition on equivalent lowpass signal

Your idea with the Hilbert transform doesn't work. The only signal (apart from $x(t)=0$) for which the Hilbert transform is zero, is a constant signal.

A band pass signal $x(t)$ can be written in terms of its complex envelope $x_{LP}(t)$:

$$\begin{align}x(t)&=\textrm{Re}\left\{x_{LP}(t)e^{j\omega_0t}\right\}\\&=\textrm{Re}\{x_{LP}(t)\}\cos(\omega_0t))-\textrm{Im}\{x_{LP}(t)\}\sin(\omega_0t))\tag{1}\end{align}$$

From $(1)$ it is clear that the quadrature component is zero if the imaginary part of the complex envelope $x_{LP}(t)$ is zero, i.e., if $x_{LP}(t)$ is real-valued. In the frequency domain, this is equivalent with $X_{LP}(f)$ being conjugate symmetric: $X_{LP}(f)=X_{LP}^*(-f)$

Note that your equation involving the Hilbert transform is wrong. The complex band pass signal

$$x_a(t)=x_{LP}(t)e^{j\omega_0t}\tag{2}$$

is called the analytic signal. Its real part equals the (real-valued) band pass signal $x(t)$, and its imaginary part is the Hilbert transform of its real part:

$$x_a(t)=x(t)+j\hat{x}(t)\tag{3}$$

where $\hat{}$ denotes the Hilbert transform. So we simply have

$$x(t)=\textrm{Re}\{x_a(t)\}\tag{4}$$

which is the same as Eq. $(1)$.


Capacity and channel estimation algorithms: explanation of results

The channel capacity expression you cited is the one with CSI at receiver that means the receiver know perfectly the realization of fading channel, denoted $\mathbf{H}$, but not the realization of additive white Gaussian noise.
In that case, the channel capacity is the max (or sup) data rate at which there exists a receiver that achieves decoding error probability smaller than any infinitesimal $\epsilon > 0$, and can be computed using the expression in your links (with adding expected value operator depending assumptions) see chapter 8.
Therefore, I would not be convinced if people say their receiver needs to estimate the channel while it is assumed to be known (that estimation is not illegal though).
That said, it is legit if the channel estimation is not at the receiver. One such case is that the channel estimation is done at transmitter. But in any case, receiver must known perfectly the channel so that you can use the channel capacity expression.
Thus, if your transmitter estimates the channel and calculate precoding vectors to transmit data and your receiver knows perfectly the precoding and the channel, you can use the capacity expression.


You said you were new to DSP, but in reality, anyone new to filtering (analog or digital) has the same problem. In other words, we all learn about filtering in the frequency domain but pay little attention to what is taught about the effects of filtering in the time domain.

The problem you are showing, excessive overshoot and ringing, is most easily solved by choosing a diffrent type of filter. You don't say say what type of filter you are using, but it most certainly has a very sharp transition band, and a flat pass band, or a ripple in the pass band, which allows for a sharper transition band.

An extreme example of just the oppisite of your filter is the Gaussian response, which has roll off in the pass band and a wide transition band. Its time domain response will have no overshoot or ringing, but also has "poor" attenuation characteristics.

The solution then, is to design a filter that gives as much filtering as needed, but doesn't cause any more overshoot and ringing than your application can tolerate. You need to learn that with filters, you can have sharp filters, or you can have minimal distortion (overshoot and ringing), but not both.

I am not a MatLab user, so I can't suggest a specific MatLab function to use. Might I suggest some free FIR and IIR software that will clearly and easily demonstrate what I have said here. Both the FIR and IIR programs allow you to adjust the filter's response in the freqency domain, and quickly see the effects in the time domain. In the IIR program, experiment with the Gauss and Elliptic responses as extreme oppisites of one another. The programs are available at  http://www.iowahills.com/

Also, you need to pay attention to the filter's group delay. For 2 filters with essentially the same magnitude response, the filter with the more linear phase (flatter group delay) will cause less distortion.


Understanding Digital Audio - "From Audio Device" in Matlab

1) The microphone converts the audio signal to an electrical signal, and then that electrical signal often goes through an amplifier and then to the ADC.  The ADC usually samples the voltage of the electrical signal, which is directly analogous to the pressure of the audio signal.

If you want to think of it in audio terms, probably the most useful way to think of it is that your microphone/amp/something saturates at a certain dB level.  Let's call that "X" dB.  That is where your signal maxes out.  If you have 16-bit sampling then you can span a decibel range of $20log_{10}(2^{16})$ = 96 dB.  That means that, in an ideal world, you could cover audio signals from "X" dB to "X" - 96 dB.  Since the world is not ideal you would probably not get that entire range, but you would probably get pretty close.

2) "Consecutive samples" just means the number of samples.  They throw "consecutive" in there just to say that they are not skipping samples.  If you set that value to 1 you will only get one sample per channel, which is probably not what you want to do.  The number of channels refers to the number of audio sources.  For mono you would set that to 1.  For stereo you would set it to 2.


what is silence in audio .wav file and how to compute decibels?

Q1: Sample values in .wav file represent the waveform amplitude?

Yes.

Q2: What is silence?

That depends on how you define it. The signal that you generated actually has a lot of energy but it's all at 0 Hz. Your DAC cannot reproduce 0 Hz, your speaker cannot reproduce 0 Hz, air cannot transmit 0 Hz, and your ear can't hear 0 Hz. There is lots of energy but you can't get it to you ear or perceive it. If you want a loud signal, create a sine wave with a frequency 1000 Hz and an amplitude of 10000. That will be loud.

Q3: How to compute decibels from the .wav amplitude values?

That depends on what type of decibels and for what purpose. You can always calculate dBFS (dB re digital Full Scale) as
$$20 \cdot log(x/x_{max})$$
where $x$ is your sample and $x_{max}$ the clipping point of your AD or DA converter, i.e. 32768 for signed 16-bit integers.
Sound pressure levels, dBSPL, are much more complicated and require calibration and in many cases also a frequency weighting function.


Assuming a digital (discrete-time) sample rate conversion (upsampling) operation carried on old samples of a signal x[n] to upsample it by an integer factor R  by the utilization of a zero-order hold interpolation filter, you would essentially replicate (repeat) every new and empty (R-1) samples with the single old sample that they originate from. I guess the problem you have stems from the fact that you misinterpret the relation:
$$
n_{old}=\frac{n_{new}}{R}
$$

which is correct for continuous variables but not if you consider the fact that both $n_{old}$,$n_{new}$,and $R$ are integers. Therefore it cannot hold true for those samples $n_{new}$ which are not integer multiples of $R$

To remedy your confusion with the ZOH output index calculation, you shall better use the following interpretation as a relation between new and old sample indices:
$$ n_{old} = \lfloor \frac{n_{new}}{R} \rfloor$$

Where floor, with brackets $\lfloor$, $\rfloor$ is the mathematical operation that returns the largest integer that is less than or equal to its argument.

Or you could, equally, define a ruling to map old and new samples as:
index $n_{old}$ is equivalent to the integer $K$, for all those $n_{new}$ that satisfy $K R \leq n_{new} < (K+1)R$


With high probability, the band-pass filter.

DWT are invertible non-redundant discrete transformations that decompose data onto iterated low- and high-pass filters. Hence, the bands limits are mostly inverses of powers of two. So, you cannot chose arbitrary band values with DWT. More details can be found in the answer to What are the pros and cons of wavelet for filtering compared to conventional filters?

DWT satisfies additional properties (like perfect reconstruction, vanishing moments) that linear filters are not subject to. Hence, the choice in admissible DWT  filters is reduced due to these additional constraints.

Hence, for pure band-pass filtering, a carefully designed pure band-filter can always be better than a DWT (and I usually am pro-wavelet). However, one can design a band-pass filter that is way worse than a DWT processing in certain settings.


Your question appears to be about the signal to quantization noise ratio for uniform quantization. You have to count each bit, but that generally doesn't mean that the SNR is $6N$ dB, where $N$ is the number of bits. You normally have a constant offset:

$$\text{SNR}_{\text{dB}}\approx c+6N\tag{1}$$

where the constant $c$ depends on the Crest factor of the signal.

To give you an example, assume that the signal is in the range $[-x_p,x_p]$, and assume uniform quantization with $N$ bits. Then the quantization step size is

$$q=\frac{2x_p}{2^N}\tag{2}$$

If we further assume that the quantization error is uniformly distributed in the interval $[-q/2,q/2]$, the quantization noise power is

$$P_n=\frac{1}{q}\int_{-q/2}^{q/2}\xi^2d\xi=\frac{q^2}{12}=\frac13\frac{x_p^2}{2^{2N}}\tag{3}$$

With $x_{\text{RMS}}$ the RMS value of the signal, the signal to quantization noise ratio is given by

$$\text{SNR}=\frac{x^2_{\text{RMS}}}{P_n}=3\frac{x^2_{\text{RMS}}}{x^2_p}\cdot2^{2N}=\frac{3}{C^2_x}\cdot2^{2N}\tag{4}$$

where $C_x$ is the signal's Crest factor. From $(4)$, the SNR in dB is

$$\text{SNR}_{\text{dB}}=10\log_{10}\left(\frac{3}{C^2_x}\right)+20N\log_{10}2=10\log_{10}\left(\frac{3}{C^2_x}\right)+6.02\cdot N\tag{5}$$

which has the form of Eq. $(1)$.


You can check the area of the same-color region. It should be close to the last one and it should be around some value (you can filter out very small and very big regions).
You can check rectangularity, circularity, convexity, angle, width, height depending on your object to be tracked.
If your false-detected regions are very small, you can try to a bit smooth the image first and then check for values.
Besides that if the false detected objects are very similar to the correct ones, your method seems to be the most feasible way to do it. You can consider making it faster by choosing different platform or optimizations.


Let's look at a simple example: you have speech signal that has a sinusoidal noise component at 7 kHz.
If you sample this at 8 kHz without pre-filtering, the noise will alias down to 1kHz and contaminate your speech signal in a very important frequency region.
If you pre-filter at 4 kHz, the noise will be gone.


This question is old, but for the benefit of the next person stumbling across it:
You have to multiply transfer functions in order to "add" them in the bode plot.


Detecting which digital number an image represents

A simple count of the pixels where the binary color is matching between the image to be recognized and the images of the ten reference digits might be good enough.

For best results, it is advisable to create the reference images using exactly the same method as that for the target image.

If it turns out that binarization doesn't work repeatably enough, you can switch to finer comparison metrics such as Sum of Absolute Differences, Sum of Squared Differences or Normalized Grayscale Correlation.


Why is my discretized transfer function unstable when my discretized state-space model is stable?

You're using a high order model so you might be running into numerical precision issues.

Make sure the realization of model_state_space is minimal. You can do this with

min_model_state_space = minreal(model_state_space)

Also, not all state-space realizations are created equally. Some have better numerical properties, such as a balanced realization:

[balanced_model_state_space, G] = balreal(model_state_space)

If the vector G contains small values it means you can reduce the order of your model without giving up accuracy. Look up the matlab function balred.

Just from looking at your step response plot you probably don't need a 30th order model (1st or 2nd order might even be enough).


What are the relative merits of various upsampling schemes?

For the purposes of this answer I will use Matlab's terminology and define "upsampling" as the process of inserting $m-1$ zeros in between the input samples, and "interpolation" as the combined process of upsampling and filtering to remove the $m-1$ aliases ($m$ being the interpolation factor) that upsampling introduces.  For an explanation of how/why upsampling introduces aliases, please see this thread.

It is important to understand that any low-pass filter can be used to get rid of the aliases and thus complete the interpolation.  Some filters have advantages when used in interpolation, though.  I'll discuss the various flavors of interpolation filtering below.

FIR Filter

Interpolating FIR filters are efficient because they combine upsampling and alias filtering into one step.  This is most easily seen in an example.  Suppose we have a data sequence $x[n]$ and we want to interpolate it by a factor of two.  The first step is to upsample by a factor of two.  This changes the original data sequence from $x_0, x_1, ... x_N$ to $x_0, 0, x_1, 0, ... x_N$.

Now suppose we have a low-pass FIR filter, $h[n]$, that we will use to remove the alias.  When we convolve the upsampled data sequence with the filter, half the filter taps are stimulated by the non-zero samples, and half the taps are inactive because they correspond to the zero samples.  The half that is stimulated and the half that is inactive flips back and forth as the filter goes through the data.  These two sets of taps are sometimes referred to as the filter phases.

This same effect can be achieved implicitly by eliminating the upsampling, and filtering the original data sequence with an interpolating FIR filter.  The interpolating FIR filter produces $m$ outputs for every input sample.  For all $m$ outputs the filter will operate on the same $ceil(K/m)$ input samples (where K is the number of filter taps and "ceil" is the ceiling function).

An example will hopefully illustrate how this works.  Suppose that we have a six tap filter and we are interpolating by a factor of two.  The filter taps are [1 -2 4 4 -2 1].  If we literally interpolated and then filtered the samples and filter taps would line up (once there was full overlap) as follows:

$$
0 : 1\\
x_2 : -2 \\
0 : 4 \\
x_1 : 4 \\
0 : -2 \\
x_0 : 1 \\
$$
Next sample...

$$
x_3 : 1\\
0 : -2 \\
x_2 : 4 \\
0 : 4 \\
x_1 : -2 \\
0 : 1 \\
$$
Next sample...

$$
0 : 1\\
x_3 : -2 \\
0 : 4 \\
x_2 : 4 \\
0 : -2 \\
x_1 : 1 \\
$$
And so on.  The point of the interpolating filter is that it skips actually inserting the zeros and just alternates which set of taps it uses at the moment instead.  Thus, the preceding sequence would now look like the following:

$$
x_2 : -2 \\
x_1 : 4 \\
x_0 : 1 \\
$$

$$
x_3 : 1 \\
x_2 : 4 \\
x_1 : -2 \\
$$

$$
x_3 : -2 \\
x_2 : 4 \\
x_1 : 1 \\
$$

Zero Order Hold

A zero-order hold interpolator is one that simply repeats each sample $m-1$ times.  So a factor of two zero-order hold interpolator converts $x_0, x_1, ... x_N$ into $x_0, x_0, x_1, x_1, ... x_N, x_N$.  This method is attractive because it is exceedingly easy, both in terms of coding and computational load, to implement.

The problem with it is that its low-pass filtering is quite poor.  We can see that when we recognize that the zero-hold interpolator is a special case of FIR interpolation.  It corresponds to upsampling followed by an $m$-wide rectangle filter.  The Fourier transform of a rectangle filter is a sinc function, which is a rather shabby low-pass filter.  It's shabbiness can be fixed with a compensating FIR filter, but if you are going to do that you might as well just use a good low-pass filter to begin with.

First Order Hold

First order hold is a step up from the zero-hold interpolator in that it linearly interpolates the upsamples using the two nearest input samples.  So, a factor of two first-order hold interpolator would convert $x_0, x_1, ... x_N$ into $x_0, \frac{x_0+x_1}{2}, x_1, \frac{x_1+x_2}{2}, ... x_N$.

Like the zero-order hold interpolator, the first-order hold interpolator is a special case of FIR interpolation.  It corresponds to upsampling and filtering with a triangle filter.  For factor-of-two interpolation the filter is $[\frac{1}{2} 1 \frac{1}{2}]$, for factor-of-three interpolation the filter is $[\frac{1}{3} \frac{2}{3} 1 \frac{2}{3} \frac{1}{2}]$, and so on.

The triangle filter is two rectangle filters convolved together, which corresponds to sinc squared in the frequency domain.  This is a definite step up from the zero-order hold, but is still not great.

IIR Filter

I have never used an interpolating IIR filter so I won't say a lot about them.  I assume that the same arguments apply as in regular filtering- IIR filters are more efficient, can be unstable, don't have linear phase, etc.  I do not believe that they can combine the upsampling and filtering steps like a FIR filter can, but I could be wrong about that.

FFT Interpolation

I'll throw this one in even though it's not very common (of course, I don't think the zero-hold is common either).  This thread discusses FFT resampling, where resampling is both interpolation and decimation.

Higher Order Holds

Second-order hold interpolators are usually referred to as "quadratic interpolators".  They are non-linear and thus cannot be implemented as FIR filters, which are linear.  I do not understand the math behind them well, so I won't discuss their performance.  I will say, though, that I believe that they are somewhat common outside of signal processing.

Higher order (three or more) methods also exist.  These are referred to as "polynomial regressions".

EDIT:

Cascade Integrator Comb (CIC) Filters

I forgot to mention CIC Filters.  CIC filters are used for two reasons: they use only adders/subtracters (not as big a deal now that multiplies are fast and cheap), and they can do really large sample rate changes pretty efficiently.  Their down side is that they are essentially an efficient implementation of a cascaded rectangle filter, so they have all of the disadvantages of rectangle filters as discussed above.  CIC interpolators are pretty much always preceded by a compensating FIR filter that pre-distorts the signal to cancel out the distortion introduced by the CIC.  If the sample rate change is large enough the cost of the pre-distortion filter is worth it.


As vaz's answer suggests, your differential encoding is incorrect. In QPSK
regarded as a phase-shift-keyed signaling scheme as opposed to
a 4-QAM scheme (see this previous answer of mine for some details), the input dibit $(b_I,b_Q)$ is treated as being the Gray-coded integer representation of
the integers $\{0, 1, 2, 3\}$ and the phase of the RF carrier in the
corresponding signaling interval is $0$ or $\pi/2$ or $\pi$ or $3\pi/2$. In tabular form, we have
$$
\begin{array}{|c|c|c|c|c|c|}
\hline
(b_I,b_Q) & \text{normal value} ~k & \text{Gray code value} ~\ell  &\text{phase-modulated signal}\\
\hline
(0,0) & 0 & 0
& \sqrt{2}\cos\left(2\pi f_c t - 0\frac{\pi}{2}\right)\\
(0,1) & 1 & 1
& \sqrt{2}\cos\left(2\pi f_c t - 1\frac{\pi}{2}\right)\\
(1,1) & 3 & 2
& \sqrt{2}\cos\left(2\pi f_c t - 2\frac{\pi}{2}\right)\\
(1,0) & 2 & 3
& \sqrt{2}\cos\left(2\pi f_c t - 3\frac{\pi}{2}\right)\\
\hline
\end{array}
$$
That is, we can say that the QPSK modulator input
$(b_I,b_Q)$ is regarded as the Gray code representation
of the integer $\ell \in \{0,1,2,3\}$ and the modulator produces the
output
$$\sqrt{2}\cos\left(2\pi f_c t - \ell\frac{\pi}{2}\right).$$

In differential QPSK (DQPSK), the input dibit Gray-code value
is subtracted (differential, get it?) from the previous transmitted dibit Gray-code value (left-handed folks can use addition if they prefer) to get
the next transmitted dibit Gray code value.  That is, with $\ell$
continuing to stand for Gray code value, we have that
$$a_{\ell}[n] \equiv a_{\ell}[n-1] \mp b_{\ell}[n] \bmod 4. \tag{1}$$
At the demodulator, we have
$$\hat{b}_{\ell}[n]\equiv \hat{a}_{\ell}[n-1] \mp \hat{a}_{\ell}[n] \bmod 4 \tag{2}$$ where the carets denote the demodulated symbols and
it is the job of the project manager to make sure that
the modulator designer and the demodulator designer both use the same
sign ($+$ or $-$) in $(1)$ and $(2)$.  In well-designed DQPSK systems,
there is no explicit (two-bit binary) arithmetic used in $(2)$,
and the DQPSK demodulator determines $b_{\ell}[n]$ from soft-decision
data or more usually, the raw matched-filter/correlator outputs in
the I and Q branches at two successive sampling instants.

Finally, you can apply the same idea to $M$-PSK with $M = 2^n > 4$
with the successive phase shifts being labeled with the Gray-coded representations of the integers $\{0, 1, 2, \ldots, 2^n-1\}$.
Equations $(1)$ and $(2)$ above get modified to be arithmetic modulo
$M$ but the same ideas work. The receivers are more complicated, though.


Finding dependence between two sparse signals

If you have enough sample measurement data, you could reconstruct a time series for each sensor pair that eliminates any time step where either one or both of the sensor pairs are na.  Then you are left with a time series for that pair without any na's and can proceed with standard time series approaches for calculating correlation.  Then repeat those steps for each sensor pair.

For example:

X_1 = [2, 5, 1, na, na, 1, na, na, 89]
X_2 = [na, 4, 9, 8, na, na, na, 2, 9]

after reconstruction becomes:

X_1 = [5, 1, 89]
X_2 = [4, 9, 9]


Most images are inherently low-pass: the object that is displayed in the image generally doesn't change from pixel to pixel.

Most image pixel noise is generated by each individual pixel capturing the image. As a result this noise is generally assumed to be uncorrelated from pixel to pixel.  That means the noise is  (bandlimited) white noise, which has uniform components across all frequencies low to high.

So, when you apply a low pass filter to an image, most of the information about the objects imaged will pass straight through because they are in the low pass filter's pass band.

Conversely, noise will have its high frequency components attenuated by the low pass filter.

That is why the statement low pass filters effect noise more than the real data is generally true.

To answer your explicit question:

  Does this mean that the filter effects noise more due to the noise being correlated from pixel to pixel?

The filter affects the noise more due to the noise being uncorrelated from pixel to pixel.


The difference of downsampling an image and smoothing an image?

Downsampling is the process of properly discarding every $(M-1)$ samples out of $M$ to reduce the signal sample rate by $M$. To avoid aliasing the signal being downsampled is initially lowpass filtered at a proper cutoff frequency before compression. The effect is a change of sample rate. Also the spectral positions of low frequencies are stretched to higher frequencies towards $\omega = \pi$. As a consequence, any spectra that passes beyond $\omega = \pi$ becomes aliased into mistreated bands, hence the requirement of a pre-processing lowpass anti-aliasing filter.

Smoothing is a more general thing. Whereas it may naturally refer to lowpass filtering in general, it may also mean more complicated things such as Kalman estimators which are also defined as smoothing operators. In the former case smoothing refers to getting rid of high frequency signal components, whereas in the latter case it refers to getting rid of randomness (usually of high frequency).


It is difficult to explain the exact reasoning behind this exponent quantitatively but more generally, raising a function to some power $\alpha$, sharpens (for $\alpha>1$) or "dulls", (for $\alpha<1$) the curve's slopes.

In this particular application, raising to a power, changes the slopes of the window function which, multiplied by some signal $x(n)$, will eventually shape its spectrum. This is the purpose of the window function.

(In the application of the Raised cosine filter for example, the "pulse shaping" is more intuitive).

Therefore, the short answer is that the coefficient is 0.85 because that coefficient gave the designers the sort of spectrum lobe shaping they were after.

Now, for a more practical demonstration, here is the exact same curve but at different exponents. On the left it is the window function itself, on the right it is the window's spectrum.

There are a few things worth noticing here:

I have put the windows and their spectra in the same graph because the differences for small variations of the exponent are subtle.
Because the window function is multiplied with the signal in the time domain, the resulting signal (from the application of the window) is the convolution of the signal's spectrum with the window's spectrum. So what you see on the right, the spectrum of the window function, is not multiplied with the signal. It undergoes the slightly more complicated operation of convolution. Given a signal $x(n)$ and a window function $h(n)$, the resulting signal is $y(n) = x(n) \cdot h(n)$. But this will result in $Y = \mathcal{F}(x) * \mathcal{F}(y)$ where $\mathcal{F}$ denotes the Discrete Fourier Transform.
This convolution operator is what causes the shaping of the spectrum. To understand broadly what effect does the window have on the spectrum, you need to understand the relationship between the time domain and the frequency domain. Broadly speaking, a "sharp" function in the time domain has a "wide" frequency domain representation and vice versa. For more information about why this is, consider the shape of the sinc filter. Now, if you convolve your spectrum with a "wide" function, you basically smooth the sharp transitions between harmonics and vice versa. But, to produce a wide convolving function in the frequency domain, you need a sharp function in the time domain (and vice versa). To achieve this, you raise the exponent.
This might be a little confusing at first but once you get the relationship between the two domains, along with the convolution theorem, you will be in a better position to understand what a window function achieves and how.

Hope this helps.


HHT and how to plot the Hilbert spectrum

I figured out an answer.

From here:
https://web.cs.dal.ca/~tt/CSCI690611/papers/HHT.pdf

The Hilbert spectrum x(t) is:

where "a" is the instantaneous amplitude of the resulting IMF's
(i.e.: absolute value of the Hilbert Transform applied to every point of one IMF),
and "w" is the instantanous frequency of the resulting IMF', calculated from the inverse tan of the imaginary divided by real components.

So adding the results using all the IMF's should provide the 3D time-frequency-amplitude information.


How to design a filter that can filter out noise accurately, after setting the parameters of the filter using standard signal?

If you can use a higher precision instrument to measure the signal, then, if the signal maintains it's characteristics, you can build a system's model and use a Kalman filter for the "not so good" input channel.


What does "two signals aliasing to each other" mean?

Aliasing would occur if $f_2 = f_1 + kf_s$, where $k$ is an integer. Consider the following example:

$$
x_1(t) = \cos(j2\pi f_1 t + \phi_1) \\
x_2(t) = \cos(j2\pi f_2 t + \phi_2)
$$

An ideal sampler samples the above at a sample rate of $f_s$ to yield discrete-time signals $x_1[n]$ and $x_2[n]$:

$$
x_1[n] = \cos\left(2\pi f_1 \frac{n}{f_s} + \phi_1\right) \\
x_2[n] = \cos\left(2\pi f_2 \frac{n}{f_s} + \phi_2\right)
$$

We got the above by noting that the sampler takes samples at times $\left[0, \frac{1}{f_s}, \frac{2}{f_s}, \ldots\right]$. To see the aliasing effect, substitute back in the relationship between $f_1$ and $f_2$ that we started with:.

$$
x_2[n] = \cos\left(2\pi (f_1 + kf_s) \frac{n}{f_s}+ \phi_2\right) \\
x_2[n] = \cos\left(2\pi f_1 \frac{n}{f_s} + 2\pi kf_s \frac{n}{f_s}+ \phi_2\right) \\
x_2[n] = \cos\left(2\pi f_1 \frac{n}{f_s} + 2\pi kn + \phi_2\right)
$$

Now we recall a property of sinusoids:

$$
\cos(x + 2\pi m) = \cos(x)\ \forall m \in \mathbb{Z}
$$

That is, if we add any integer multiple of $2\pi$ in the argument of a sinusoid, we get the same value out of the function (stated differently, sinusoids are $2\pi$ periodic). That means that we can drop the $2\pi kn$ out of the above expression for $x_2[n]$:

$$
x_2[n] = \cos\left(2\pi f_1 \frac{n}{f_s} + \phi_2\right)
$$

Now compare the two discrete-time signals again:

$$
x_1[n] = \cos\left(2\pi f_1 \frac{n}{f_s} + \phi_1\right) \\
x_2[n] = \cos\left(2\pi f_1 \frac{n}{f_s} + \phi_2\right)
$$

When we look at the sampled versions of $x_1(t)$ and $x_2(t)$, they appear to be sinusoids at the exact same frequency. In the discrete-time domain, since one or both of them were not sampled at at least the Nyquist rate, and the two frequencies have the relationship that was stated above, we are unable to discriminate between the two frequency components. This is what is called aliasing.


Image Gradient: Light to Dark, Dark to Light

The UINT8 type is limited to integer numbers on the range [0, 1, 2, ..., 255].
Hence negative values are clipped into 0.

A solution could be wither use other types (Floating Points) or scale and shift.

For instance, given a Floating Point image with values imageMax and imageMean you can scale it into the [0, 1, 2, ..., 255] by doing:

Create only positive numbers: image - imageMin.
Scale it into the range: round(255 * (image - imageMin) / (imageMax - imageMin)).

Now the conversion won't loose data.

Others have different approach, set 128 to be zero.
Namely, add 128 to the image, clip it into [0, 255] and then convert to UINT8.

Enjoy.


CTFT to DTFT why can't you always just substitute $\Omega = \omega/T_S$

It is due to the property of unit impulse function

\begin{equation}
\delta(\alpha t)  = \frac{ 1}{|\alpha|} \delta(t)
\end{equation}

(Usually $\omega $ is used for angular frequency and $\Omega $ is used for angle. Interpret my answer accordingly. )

suppose
\begin{equation}
X_a(\omega) = \delta(\omega -x_0 )
\end{equation}
and if we substitute '$\omega$' with $\frac{\Omega}{T_s}$ then
\begin{equation}
X_a(\frac{\Omega}{T_s}) = \delta(\frac{\Omega}{T_s}  -x_0 )
   = \delta(\frac{\Omega - x_0 T_s}{T_s} )
   =  \delta(\frac{ 1}{T_s} (\Omega - x_0 T_s) )
   =  {T_s} *\delta(\Omega - x_0 T_s)
\end{equation}
where '$\alpha$'  is $\frac{ 1}{T_s}$
and
$$
\frac{ 1}{|\alpha|} = T_s
$$

and when you do DTFT calculation on

\begin{equation}
X(\Omega)  = \frac{ 1}{T_s} X_a(\frac{\Omega}{T_s})
   =\frac{ 1}{T_s} * ( T_s * \delta(\Omega - x_0 T_s) )
   = \delta(\Omega - x_0 T_s)
\end{equation}

in the final answer $T_s$ and $\frac{ 1}{T_s}$ cancels

and you will get the expression based on the independent variable '$\Omega - x_0 T_s$'

So there is nothing wrong in your actual answer.


The two oscillator types are equivalent, because the factor PI can be moved to freq and need not be present in the final equation.

The difference in the plots is not due to one oscillator type being better than the other; they are equivalent. The difference seen is spectral analysis artifacts that depend on the frequency of the sine and how that relates to the Discrete Fourier transform (DFT) length. DFT decomposes the time-domain signal into harmonic frequencies (multiples of a fundamental frequency that has a fundamental period equal to the analysis window length). If the sine frequency is not exactly at such a harmonic frequency, the sine "energy" still needs to appear somewhere in the frequency domain and will be spread to multiple frequency bins. The plots in this question on such spectral leakage are very similar to yours. Spectral leakage can be ameliorated by multiplying the data by a suitable window function before DFT.

With the oscillator of type a, if freq is of form some_integer*2*PI/N, where N is the DFT length, then the sine frequency is one of the harmonic frequencies seen by DFT and there will be no spectral leakage.


In smoothing, each data point is replaced by some kind of local average of surrounding data points. When the size of smoothing window is defined in terms of octave, e.g., 1/3th Oct., 1/6th Oct etc., it is called fractional-octave smoothing.

Let $Z[i]$ be the complex valued frequency spectrum, where $i$ is the frequency index (FFT bin) and $0 \le i \le N - 1; N$ is the length of frequency bins. Then smoothed spectrum is given by
$$
Z_s[i] =\sum\limits_{j=i-\Delta/2}^{i+\Delta/2} W[j] * Z[j]
$$
where $\Delta$ is the smoothing window and $W[j]$ is the smoothing coefficients. Here, $Z[j]$ is in complex form ($a + ib$) and $W[j]$ is real. So smoothed spectrum, $Z[i]_s$, contains both magnitude and phase information.

In general smoothing coefficients are normalized, such that, $\sum\limits_{j=0}^{n} W[j] = 1;$ where $n$ number of smoothing coefficients defined by $\Delta$.

There are different methods for choosing these smoothing coefficients. For example, Rectangular, Savitzky-Golay, Gaussian, Binomial, etc. All the methods are some kinds of moving weighted average. The idea is to give the most weight to the data points nearest the point of estimation and the lest weight to the data points that are furthest away.

In signal processing, the aim of smoothing is to enhance the S/N by reducing the noise as much as possible, but distorting the original shape as small as possible. There is a trade-off between reducing the noise and preserve the original shape.

You also have to handle the edging data points. There are several possible ways to deal with the boundary values, e.g., repeating the last value, zero padding, shortening the window size.


A DFT will convert a segment of a periodic waveform of any frequency (uniquely below Nyquist)  into an orthogonal set of frequency results.  The problem is that if the signal is not periodic in the FFT length, the result will not be in one result bin, but be smeared into all the FFT result bins, due to convolution with the transform of the rectangular or other window function, making it difficult to separate the periodic signal from narrowband + wideband noise (or DC + ramp + noise for spectrum below the 1st bin).

Another potential problem with estimating non-periodic real spectrum very near DC (or Fs/2) is interference from its own negative frequency image in the DFT/FFT result, which can be constructive or destructive depending on phase, and thus non-trivial to account for.

So yes, the low frequency, below Fs/N, will "exist" in the DFT/FFT complex result, but be more difficult to impossible to find or estimate, depending on the signal to noise ratio.  But in zero noise, you theoretically need only 3 or 4 non-aliased samples to find a single pure sinusoid of any frequency, so it will be represented in the transformed N samples, for N >= 4, just not clearly in only 1 or 2 DFT/FFT result bins.


Dynamics processor for speech - apply gain on delayed samples?

It sounds better this way.

Let's say you have limiter and you can't exceed a certain amplitude value (risk of clipping, for example). Now there is an audio event (say a snare hit) that peaks at twice that amplitude. If you just reduce this sample by sample instantaneously you create a lot of audible distortion. If you have look ahead, you can ramp down the gain smoothly so that by the time the audio even arrives, the gain is low enough so that you are not clipping any more.

Rapid gain changes tend to create a lot of audible artifacts. The look ahead buffer allows you to slow down the gain changes without the risk of creating an instantaneous out of bounds condition.

It's also useful to look at a bit in the future when determining the gain for the current sample: if that's the highest point in the waveform can slow down the gain reduction, but if it's part of a slope, you probably want to maintain or increase your gain reduction speed.

Of course, you are not actually "looking into the future". Look ahead simply means that you put a delay on the processing path but NOT on the detection path of the dynamic processor.


Detecting lines with houghlines in matlab

The biggest issue is that the pixels for each line are too few (I will explain with more details below). I would consider to stretch, and then dilate your raw image a little bit:
file='http://i.stack.imgur.com/LmIJJ.png';
I=imread(file);
I=imcrop(I,[1 206 size(I,2) size(I,1)]);
I=imresize(I,[size(I,1) 256]);
I=imdilate(I,strel('line',1,0));

I assume you are using the sample code from Mathworks on hough, houghlines, and houghpeaks. The problem is that there are not very many pixels along one slope in your image. You feel like it is a line, but it is actually a curve that consists of many short lines with different orientation. I think one parameter that you need to tune is the 'FillGap' which will connect the very short segments. I tried:
lines = houghlines(BW,T,R,P,'FillGap',30,'MinLength',15);

And the image I obtained:

There is still plenty of room for improvement, yet at least most lines are detected.
Note that the theta-axis in hough peaks indicates the slopes of the lines, you may also check them from lines, T, and R. But since the image has already been stretched along the 2nd dimension of raw data, you may need to scale them back (with some tangent transformations) if you plan to get the slopes of those detected lines.
Hope it helps, thank you.


Aliasing in digital communication system

If the signal breaks the Nyquist criterion we'll only get a part of the signal.

This needs to be clarified: the frequencies above Nyquist will fold back into the Nyquist band, so it's not just that you lose part of the signal; it's that the signal is corrupted (usually) beyond hope of reconstruction.

As an example, consider a 4 kHz real signal sampled at 4 kHz -- the frequencies between 2 and 4 kHz will fold back into the 0 to 2 kHz range; not a single "part" of the signal will survive.

You're correct that this can't be solved by modifying the constellation order $M$ or the carrier frequency. As you correctly assume, you have two options. You can reduce the bandwidth through pulse shaping; if the symbol rate is $R$, then the minimum bandwidth $B=R/2$ is achieved with sinc pulses. The second option is to reduce the rate.

(As a side note, recently there have been proposals to increase the rate without increasing the bandwidth (search for "Faster than Nyquist signaling"), but I think these ideas are outside the scope of your studies at the moment.)


What DSP techniques can be used to change the timbre of complex tone?

Phase vocoders with custom resynthesizers can be used for this, as in time pitch stretching code that attempts to preserve the spectral formant envelope.  But phase vocoder analysis/resynthesis is not artifact-free.


ReLu in the frequency domain becomes convolution, NOT multiplication.
See chapter 4.5 of the paper: "...in frequency domain, ReLu() acts as a convolution with the function of a known form. "
But you don't need to calculate it at every point because of the pooling.


In this case, images/frames are moving horizontally or vertically?

Neither. They just appear; usually, they stand still for as long as a frame is displayed.

  But what is the minimum rate of frame/images so that still images appear as video?

That depends on a lot of factors like brightness, amount of motion in the material, and mental state of the observer.

Typically, upwards of roughly 20 frames a second, things are perceived as fluid motion, but there seems to be some benefits to going higher in frame rate, especially in high-movement scenes.

Traditionally, cinema and TV used to be 24 or 25 frames a second, but this number has changed in recent years, with digital TV producing whatever frame rate the video codec used does.


Video: Detecting a digital timer and when it ticks

To OCR or not?

If the timer is about $h$ pixels height, you could segment each frame into lines of height $h/2$, run an OCR on each line, and scan the result with a regexp for anything that looks like a timer, eg. \d+[\.:,;-]\d+[\.:,;-]. This gives you $x=time_{real}$ vs. $y=time_{video}$ data pairs. Sure, most of them are wrong, but imposing the restriction $y=1*x+n$ (unless the video speed was changed) eliminates most of them. Just group all data points according to whether they lie on a line of slope m=1.

However, this works only if there are at least some good data points. If we cannot make any assumptions about the color, font, or size (or even script!), OCR doesn't perform well.

Instead, making use of some intrinsic properties of all timers works much better and is agnostic to the exact font or script used. Let us make the following basic assumptions about a timer:

overlayed on a static background, ie. there's at least one bounding box that contains only the timer and no other changing objects
it remains on a fixed position for the entire duration of the video
a ticking timer is changing periodically
it contains a seconds digit with a frequency of 0.1 Hz
it is not animated, ie. the seconds digit remains static and changes only every second

We can use these assumptions to recognize the timer automatically.

Downsampling

Scanning every frame, pixel, and every color channel is a lot of work, takes time, and also completely unnecessary. Let's say we've got a 2 hour video, 720p60, yuv420p. That's 2*3600 seconds * 60fps * 1280*720 pixels/frame * 3 channels/pixel =~ 1200 billion data values to examine.

Grayscale. The color channels are down-sampled already anyway.
1-0.1 fps is usually enough, the timer's frequency is only 0.1Hz.
A resolution of 320x180 pixels is more than enough, unless the timer had been really small to begin with, that still leaves us with ~20-60 pixels to analyze.

This reduces the number of data points by about 3*600*4*4~=30000, for a total of about 40 million data points.

Locating the timer(s)

This is the hard part.

We employ several methods to check each pixel whether it might belong to the timer, then we combine them to larger areas. The video is split into intervals of length dt, and for each interval,n frames get analyzed. 60 frames per interval works pretty nicely.

Check for changes. At the very least, the timer must be changing constantly. We could use the standard deviation of each pixel, but I found that randomizing the sample order and taking the average difference between adjacent pixels gives better results sometimes. The standard deviation tends to be lower.
Simple check for 10-seconds periodicity. Group pixels into those that are pixels 10 seconds apart of each other, take the average difference for each group, then average all groups. Basically the auto-correlation for $\tau=10s$ This is simple, fast, and tends to work well.
Compute the full auto-correlation (via FFT for speed). Sounds better, but you need to find a way to analyze it. What I found works is checking at 10 seconds, 20 seconds etc. whether there is a peak (higher value than at nearby samples), and checking that there is no higher value between two adjacent peaks.
Check step-behavior. Take samples every 0.2 or 0.1 seconds and check when the pixel changes. As each digit remains on screen 1 second, it should do so every 5 or 10 frames. Some digits may look alike, but the difference between two changes should be an integer multiple of 5 or 10.

I found that for the most part, combing the first two methods yields quick, fast, and accurate results.

Now we've got a mask of pixels that possibly belong to the timer. Apply a close operation on them (with a threshold of ~0.5), label connected areas, compute the bounding boxes, eliminate very small or very large ones, and areas that haven't got many pixels that were classified as belonging to the timer.

After doing this for all intervals, group bounding boxes together that are close to each other, eliminate groups with only a few members, and take only those groups with the most members. These are most likely the bounding boxes for the seconds digit of the timer. Average each group and return the result.

(Left: Mask, Right: Image with Mask)

(Top: Raw image, 2nd: checked for changes, 3rd: simple check for 10s periodicity, 4th: auto correlation diagram analyzed, 5th: all masks combinded, Bottom: Bounding box computed)

Note that the close step is not shown above.

Scanning for activity (start/stop)

Now that we found a timer, or rather the seconds digit, we scan the video to analyze when it is ticking and when not. If it is ticking, the digit will keep changing.

extract n frames at an interval of dt , crop to the timer rectangular area
for each pixel, take the squared difference between adjacent frames, and average over all pixels
threshold to 0 (stopped) or 1 (ticking), a good value is about 1.0-1.5
apply a gaussian smooth of about 2-4 samples against rapid changes, and threshold again to 0.5
take difference between adjacent frames again to get the changes between ticking and paused

Also, the timer may not be visible at some parts of the video, add the following step(s) before applying the gaussian smooth

for each timer region found during the scanning process, crop all scanned frames to the timer region
group these images by similarity (squared difference), and select only the ~10-15 groups with the highest frequency, these are most likely the actual digits
average all images of each group to get an idealized sample for each digit
if the timer does is not rectangular (eg. italic text), save the original mask for the timer region and compare only these pixels

Now, after taking the difference between adjacent frames, set all values to 0 (stopped) where the image does not match any of these digits (again, squared difference).

(left: all images from intervals classified as containing the timer extracted from the timer area; right: images grouped by sum of squared difference)

Considerations

There are a few things to consider when applying these methods.

Acquiring frames. Sounds simple, but you don't want to dump them all uncompressed to your HDD. I'm using ffmpeg with fast seeking -ss time -i input.mp4, which seeks to the nearest key frame and decodes frames until it reaches the requested time. Except it won't work some times, especially for bad video files that are common when you are saving a network stream. After transcoding the video file (with ffmpeg), most files that didn't work before were recognized perfectly. If you scale with ffmpeg, encode to 320x180, set a low key frame interval and set -movflags faststart, reading frames from the resulting file is accurate and fast. And at 320x180, it doesn't use much space either. Avoid downsampling the frame rate, as that might destroy your time stamps.
Choosing the right threshold values, time intervals and other parameters. This is what took by far most of my time, especially since testing takes some time.
In particular, it helps to choose a time step such that each digit appears. Choosing 5 seconds for example is a bad choice, as it will give you only 0's and 5's. Choosing a time step lower than 1 second won't work well when trying to analyze how much each pixel changes, as the consecutive digits will be the same.
Related to this, each interval needs to contain enough frames. If the period is 10s and you choose 60 frames for the interval length and a time step of 1.2 seconds, the length after which the digit repeats is $lcm(1.2s,10s)=60s$, ie. 50 frames out of 60. And that is not enough. I found that, in order to eliminate unwanted noise, the number of frames should be least 3-6 times as high as the period length in frames. So a time step of, say 3s, works quite well, as lcm(3,10)=30, which is 10 frames, so you get 6 periods.

Does it work, though?

I made a simple ruby script that implements the method described above.

It's probably full of bugs and the code is a mess, but it seems to work. I tested it on 200 videos of different length (5 minutes to 6 hours, most around 30 minutes). They all contained the same kind of timer (color, font), but at possibly different sizes and positions. After transcoding those video files where ffmpeg didn't manage to seek accurately, the script got the correct result for pretty much all of them, it got only 4 wrong (detected stopped when it had been ticking).

It's possible I might have fine-tuned the parameters for that specific testing set, but at least we can say it does work rather well if you can make some general assumptions about the timer. Also, I tested a few other videos with different font sizes/colors, and they worked as well. I also tested it on a video with two timers (one that kept ticking the entire video), and the script recognized them as well.

As far is accuracy is concerned, it usually determines the starting/ending point at $\pm 1...3$ seconds, (because of some smoothing and averaging to eliminate noise), which is more than enough for me. I guess you could get an even better estimate if, after determining there is a starting/ending point at a certain frame interval, you check exactly when the digit starts/stops changing.

Sample output:

Transcoding video (110.89 min) file before analysis...

ffmpeg -y -loglevel quiet -i "dump_000000038.mp4" -c:v libx264 -an -vf scale=320:180 -preset ultrafast -sws_flags bilinear -g 20 -movflags faststart -crf 12 -f mp4 ./transcode.mp4 < /dev/null
Transcoding took 423.545512367s, 16x real-time.

Analyzing <transcode.mp4>, duration 1:50:53.684...

Scanning for timers...

Trying timestep 11.0, method simple...
Analyzing 0:00:00.000-0:11:00.000...
 - got 0 connected area(s)
Found 0 candidates.

Analyzing 0:11:00.000-0:22:00.000...
 - got 1 connected area(s)
Found 1 candidates.

Analyzing 0:22:00.000-0:33:00.000...
 - got 1 connected area(s)
Found 1 candidates.

Analyzing 0:33:00.000-0:44:00.000...
 - got 0 connected area(s)
Found 0 candidates.

Analyzing 0:44:00.000-0:55:00.000...
 - got 1 connected area(s)
Found 1 candidates.

Analyzing 0:55:00.000-1:06:00.000...
 - got 1 connected area(s)
Found 1 candidates.

Analyzing 1:06:00.000-1:17:00.000...
 - got 1 connected area(s)
Found 1 candidates.

Analyzing 1:17:00.000-1:28:00.000...
 - got 1 connected area(s)
Found 1 candidates.

Analyzing 1:28:00.000-1:39:00.000...
 - got 1 connected area(s)
Found 1 candidates.

Analyzing 1:39:00.000-1:50:00.000...
 - got 0 connected area(s)
Found 0 candidates.

Extracting timer digit images...
Grouping timer digit images...

Timer(s) found:
Timer 1: (176,150), size 5x10

Analyzing timer(s) for event(s)...

Trying coarse timestep 11.0, fine 1.75...
Scanning timers for activity...
Timer 0: Found 4 event(s).
Checking at the beginning of the video at a higher resolution for the timer 0...
Checking at the end of the video at higher a resolution for the timer 0...
Refining estimate for timer 0, event 0...
Refining estimate for timer 0, event 1...
Refining estimate for timer 0, event 2...
Refining estimate for timer 0, event 3...
Reading frame 50/50...

This is what they're doing:
Timer 1:
  - starts off paused.
  - starts at 0:04:53.000
  - stops at 0:37:32.000
  - starts at 0:38:57.000
  - stops at 1:45:26.000
  - ends up paused.

Scanning took 25s, 16265 fps.


Is it possible to compute the wavelet transform of a unit step function?

Finite energy functions is a class of choice, but wavelet transforms can exists in more generic cases.

For the continuous case: let us call $H(t)$ the Heaviside step function. Then $|H(t)\psi(t)|$ is dominated by $|\psi(t)|$. If $\psi(t)$ is $L_1$ or integrable (in addition to being  $L_2$), then  $\int_{-\infty}^{\infty}H(t)\psi(\frac{t-a}{b})\mathrm{d}t$ is defined. Moreover, since admissible wavelets have zero-average, the continuous wavelet coefficients can decay at infinity. With wavelet of finite support, they will be zero for all $\psi(\frac{t-a}{b})$ with support in $[0,\;+\infty[$.

For discrete wavelets, this would be about the same. As long as the filters are summable (or $\ell_1$), and this happens  especially for FIR wavelet filters, you can compute coefficients. Special note: if a discrete filter is $\ell_1$, then it is $\ell_2$ as well, so the condition is milder than in the continuous case.


Negative frequency of a sinusoidal wave on the magnitude spectrum: Physical explanation?

Regarding your first comment, I have a figure that might help, in terms of intuition. It's a bit badly scaled but it might do the work.


Using IFFT for the purpose of "Multiple Orthogonal Frequency Shift Keying"

Yes, basically this is OFDM with an on/off constellation for each subcarrier instead of a QAM constellation as often used. So you would map your data to FFT values (each FFT bin is either 1 or 0), then take the IFFT to create the transmit waveform, with the cyclic prefix etc as in standard OFDM implementations.


Embedded Image Processing System that sends photos to Smartphone via Bluetooth

This question is more of a hardware setup. Books like Gonzalez will be more relevant when you have images as pixels and process them to transform to more desirable images.

Here are the few tips:

Setup a hardware like Raspberry PI - a small general purpose computer
Add hardware I/O interface like Camera Module
You can use standard USB Dongles and make it work on your pi.
Write your application which will now collect pics and send across to the app on your smart phone!

Thats' it, really!


Does it make sense to express a given FT from 0 to 1 as a convolution?

As for the why, it's always good to come up with alternatives, even if one doesn't immediately see their benefit. In this case it's about efficiency, especially if the FFT length is a prime. The resulting algorithm can also be used to solve the more general problem of computing (samples of) the $\mathcal{Z}$-transform on circles or spirals in the complex plane. You can also zoom into a certain frequency region without the need to compute the DFT over the whole range.
The resulting algorithm is called Bluestein's algorithm or Chirp Z-transform. The basic trick to represent the DFT (or $\mathcal{Z}$-transform) by a convolution is to write
$$nk=\frac{n^2+k^2-(n-k)^2}{2}\tag{1}$$
and use this expression in the original formula. This results in the representation of the original DFT sum by $(i)$ a multiplication of the original sequence with a chirp sequence, $(ii)$ a convolution, and $(iii)$ another multiplication with a chirp sequence. The details are explained in the link cited above.


Programmatically (e.g. in Python) adding clipping to an audio file?

Simple clipping :
threshold = 0.5
If x > threshold
  x = threshold
elseif x < -threshold
  x = -threshold
end

Real-world clipping can be significantly more complex than this, involving various time-constants, asymmetry, heating-effects,....


The sinc function actually represents an ideal (brickwall) lowpass filter that's used to complete the interpolation process after the data has been expanded (zero stuffed) properly. So let me outline the time domain approach here:
Assume you have data samples $x[n]$ of length $N$, and you want to upsample this by the integer factor of $L$, yielding a new interpolated data $y[n]$ of length $M = L \times N $ samples.
The first stage is to zero stuff the input $x[n]$; i.e., expand it by $L$ by the expander block :
$$ x[n] \longrightarrow \boxed{ \uparrow L } \longrightarrow x_e[n] $$
where $x_e[n]$ is related to $x[n]$ by the following:
$$ x_e[n] = \begin{cases} { x[n/L] ~~~,~~~ n = 0, \pm L, \pm 2L... \\ ~~~ 0 ~~ ~~~,~~~\text{otherwise} } \end{cases} $$
Then, to complete the interpolation process and fill in the empty (zeroed) samples of $x_e[n]$, one has to lowpass filter $x_e[n]$ by an ideal lowpass filter $h[n]$ with the following frequency domain definition $H(\omega)$ :
$$ H(\omega) = \begin{cases} { ~ L ~ ~~~,~~~ |\omega| < \frac{\pi}{L}  \\ ~ 0 ~ ~~~,~~~\text{otherwise} } \end{cases} $$
The impulse response $h[n]$ of this ideal filter is computed by the inverse discrete-time Fourier transform of $H(\omega)$ and is given by
$$ h[n] = L \frac{ \sin( \frac{\pi}{L} n) } {\pi n} $$
This is an infitely long and non-causal filter, and thus cannot be implemented in this form. (See Hilmar's comments) Practically it's truncated and weighted by a window function, for example by a Hamming or Kaiser window.
The following MATLAB / OCTAVE code represents designing the filter and applying it into data in time domain:
L = 5;      % interpolation factor
N = 500;    % data length

x = hamming(N)'.*randn(1,N);  % generate bandlimited data...

% expanded signal
xe = zeros(1,N*L);
xe(1:L:end) = x;          % generate th expanded signal

% interpolation sinc filter
n = -32:32;               % timing index
h = L*sin(pi*n/L)./(pi*n); % ideal lowpass filter
h(33) = L;                % fill in the zero divison sample
h = hamming(65)'.*h;      % apply weighting window on h[n]

% interpolate:
y = filter(h,1,xe);     % y[n] is the interpolated signal


Fourier coefficients of product of two periodic signals

You did everything right, and you're almost there. Your last equation becomes

$$\sum_lc_l\sum_md_m\delta[n-l-m]\tag{1}$$

Due to the Kronecker delta the sum over $m$ reduces to a single element:

$$\sum_md_m\delta[n-l-m]=d_{n-l}\tag{2}$$

because $\delta[n-l-m]$ is non-zero for $m=n-l$ and zero otherwise. Plugging $(2)$ into $(1)$ gives the desired result:

$$\sum_lc_ld_{n-l}$$


OK, so suppose your transmitted signal is $s(t)$ and the received signal is $r_i(t)$ in continuous (analog) time .

Then the received signal at the $i^\textrm{th}$ microphone is
$$
r_i(t) = h_{d,i} s(t-\tau_{d,i}) + h_{r,i} s(t-\tau_{r,i})
$$

  How does the received signal look in analog and digital time domain and also how does the FFT look like?

Well if $s(t) \stackrel{\textrm{FT}}{\longleftrightarrow} S(\omega)$ then
$$
r(t) \stackrel{\textrm{FT}}{\longleftrightarrow} R(\omega) = \left [ h_{d,i} e^{-j\omega\tau_{d,i}} + h_{r,i} e^{-j\omega\tau_{r,i}} \right ] S(\omega)
$$
which is simple application of the time shift property.

  Do we still perceive only one spike in a particular bin in FFT? Or should you sample appropriately to achieve this?

Once you start talking about 'bins' they we move to discrete-time. In general, unless you are very careful with sampling, it's almost impossible to sample a single sinusoid and achieve a spike in just two bins (positive and negative frequencies).  Achieving a spike in just one bin requires a complex exponential which is not really possible with real-world sampling unless you use a specific modulation scheme (which you don't mention).

  How does the FFT look if we have two tones?

Again FFT implies discrete time. If you send two tones and sample them, then unless you choose your sampling scheme very cleverly, you will end up with energy in many bins. See this discussion for more details.

  he math of this is what I am seeking. Assume baseband audio such as speech.

@Turbo If the signal is baseband audio, then there will never be just a single tone. Speech is generally modelled as an autoregressive (LPC) model driven by either white noise or a periodic pulse train. See figure 2.40 of this reference (screenshot included below for clarity).


When might looking at rising edges perform better than cross-correlation?

Using the rising edge is better than cross-correlation when there is the physical equivalent of an inductor in the system, which will introduce inertia into the system.  That inertia will cause the output to rise gradually, then fall back down.  Even in that case the cross-correlation might be more useful, but if you wanted to know when the input starts to affect the output the rise would be more accurate.

The figure above is illustrative of what I'm talking about.  The inductor (or "inertia introducing object") resists any change in the current flow, so initially it resists the flow causing an exponential rise (as opposed to a sharp rise that matches the presumed non-ideal impulse at the input), and then opposes the current stopping when the input ends, causing the exponential fall at the output.

If the input looks more-or-less like an impulse function, it will correlate most strongly with the output at the output's peak, at time 100.  It's clear, though, that the input starts to affect the output at time 1.  In different situations, either measurement could be the more useful one.


Finding a good inverse for an ill-conditioned matrix transformation

The usual way to approach inverting a matrix that is rank deficient is to use a generalized inverse.

This means if your matrix equation is:
$$
y = \mathbf{A}x
$$
where $\mathbf{A}$ is an $n \times m$ matrix, then you can use, for example, the Moore-Penrose pseudo-inverse, $\mathbf{A}^{+}$:
$$
\mathbf{A}^{+} = (\mathbf{A}^*\mathbf{A})^{-1} \mathbf{A}^*
$$
if a left inverse is required or
$$
\mathbf{A}^{+} = \mathbf{A}^*(\mathbf{A}\mathbf{A}^*)^{-1}
$$
for the right inverse.

If $n=m$ then either be used.

However, as per the link above, there are many different possibilities for selecting an inverse in the rank-deficient / non-square case because the system of equations is underdetermined.

As per @MBaz's comment, you can calculate this using the singular value decomposition of $\mathbf{A}$.


How to estimate ALL time-delays across MULTIPLE signals?

There are papers describing a method called multichannel cross-correlation, that recursively estimates time delay of each signal.

However, If I have not missunderstood you problem you can also select one of your signal to be your reference signal and then repeatedly calculate cross-correlation against that compared to the other signals. that way you can align every signal to your reference point.

EDIT: based on your latest comments:

  So far I've only had a chance to look at the original paper linked (microphones in a sound room). It's very interesting, however it seems to require that you know something about the geometry of the microphones. I had been hoping to find a generalized, system-agnostic method to do multi-channel cross-correlation. Something with which you can just input a set of arbitrary signals or patterns and it outputs the "best fit" delay vector.

What it appears to me that you need then is  blind system identification methods which I can link you some examples. Common methods for this task utilizes cross correlation error minimization but there are some limitations based on your model selection.

Under-Modelled Blind system identification - cross correlation based

Blind system Identification for acoustic source localization - Eigenvalue decomposition method

Blind system identification using sparse leaning method - LMS adaptive algorithm based on cross correlation formulation


What is difference between the terms bit rate,baud rate and data rate?

Bit rate and data rate are somewhat ambiguous, and their exact definitions vary from one field or application to another.

Bit rate is often used to measure the information rate, since information is measured in bits. Sometimes it is used to measure the number of actual 1s and 0s transmitted per second. When the information is compressed (source-coded), the two definitions are equal.

Data rate can be used to mean the same as bit rate. Sometimes it is used to measure the rate at which data, as opposed to overhead, is transmitted. For instance, an audio compact disc system has a data rate of $2\cdot44100\cdot16=1411200$ bits (zeros and ones) per second, but the actual transmission rate is approximately double that, once you include framing, metadata, and FEC.

The baud rate (or symbol rate, or pulse rate) is the number of actual pulses (symbols) transmitted per second. If a pulse "carries" several bits, then the baud rate can be significantly smaller than the bit rate. This is important because the bandwidth required to transmit the information is determined by the baud rate.

So, consider an encoded video stream with 1000 bits per frame, at 24 frames per second, with 10% overhead per frame, and 4 bits per symbol:

From an information-theory perspective, the bit rate is $24\cdot 1000=24000$ (information) bits/sec.
You can also say, from a more practical perspective, that the bit rate required is $1.1\cdot 24 \cdot 1000 = 26400$ (ones and zeros) bits per second, but you could still define the data rate as $24000$ bits per second, since the remaining 2400 bits are overhead.
The pulse rate on the transmission link will be $26400/4=6,600$ pulses per second.


Downconverting real signal to IQ baseband

The allowed bandwidth of the data created and used to modulate the IQ signal needs to be below the cutoff frequency of the receiver post-heterodyne low pass filter, which needs to be below the carrier frequency, which needs to be below half the HF sampling rate to avoid aliasing.

So try a lower data rate and/or a much higher frequency carrier, with a suitably corresponding higher frequency sample rate.  You can lower your data rate by using a slower changing rand statement, and filtering that to remove harmonics of the squarish pulses.


How to generate such sounds as in example?

https://ccrma.stanford.edu/

There is a Center focused on Music and Acoustics at Stanford University and their website has many resources. There are several papers on computer generation of musical instrument sounds.


In what case do we need to increase the length of the signal compared to zero padding when using FFT?

Zero-padding before an FFT results in a form of high quality (Sinc kernel) interpolation of the resulting upsampled spectrum.

The interpolation of frequency peaks works well when the level of the surrounding spectrum, both the noise floor and any adjacent frequency energy, is low enough to not distort the interpolated peak.  (The examples shown in the question have a fairly high S/N).

But an interpolated spectral peak is very rounded after high ratios of zero padding (the result of convolution with the added rectangular window caused by zero padding).  Thus, any surrounding spectral energy can change the shape of this wider spectral hump enough to change the location of its peak in relation to the frequency of any actual underlying sinusoidal or other narrow band energy.  Two peaks close together can either get spread apart or end up appearing to be just one wide spectral peak in the upsampled interpolated FFT spectrum.  This problem becomes even worse when trying to find or measure a spectral notch by interpolation.

So if you are looking for more information about a signal with a relatively high local S/N (signal-to-adjacent-noise ratio) around the signals of interest, zero-padding before an FFT may be helpful (providing a computationally efficient wide band interpolator).

If you are looking for more information about spectrum just above the noise floor, or looking for information about closely spaced sinusoids or spectral peaks or notches, then more samples will be needed to avoid distortions due to interpolation of too little information about the spectrum caused by just zero-padding an FFT.

Also, the Sinc interpolation done by zero-padding a DFT of strictly real data is actually a dual Dirichlet (periodic Sinc) kernel interpolation, which causes even greater distortion in the interpolated spectrum near the DC and Fs/2 bins, or when used with very short amounts of actual data in relation to the size of the FFT.  More actual data is likely to be needed to get a more accurate and detailed spectrum near the DC or Fs/2 result bins of a DFT.


You apply the same formula, but instead of using a scalar $x(n)$, you will have $x(n) \in \mathbb{C}^{M \times 1}$, where $M$ is the number of channels, and $x^{H}(n)$ is the $1 \times M$ matrix whose entries are the complex conjugate of the entries in $x(n)$. As a result $R \in \mathbb{C}^{M \times M}$ is an Hermitian matrix, and its eigenvalues are all real.


First of all, your graph does not match the equation you gave for the function. From the hint of msm, we should use the following property of the Fourier Transform:

$$
\mathcal{F}\left\{\frac{d}{dt}x(t)\right\}=j2\pi fX(f)
$$

with

$$
\mathcal{F}\{x(t)\} = X(f)
$$

Now, what's the derivative of your function in the graph? Its given by

$$
x'(t)=\frac{1}{4}\text{rect}\left(\frac{t-2}{4}\right) - \text{rect}(t-\frac{9}{2})
$$

Now, let's do the Fourier Transform of the derivative:

$$
j2\pi fX(f)=\text{sinc}(4f)\exp(-j2\pi 2f) - \text{sinc}(f)\exp(-j2\pi \frac{9}{2}f)
$$

with $\text{sinc}(x)=\frac{\sin(\pi x)}{\pi x}$.

Now, finally dividing by $j2\pi f$ you get

$$
X(f) = \frac{1}{j2\pi f}\left(\text{sinc}(4f)\exp(-j2\pi2f)-\text{sinc}(f)\exp(-j2\pi\frac{9}{2}f)\right).
$$

You can try to simplify this expression, but it might not yield something that is better to understand.


Is my answer incomplete? Checking the stability of a system

Did you mention the region of convergence?  The system can be stable, but anti-causal depending on the ROC.


Why is $\int^\infty _{0^-}\delta(t-nT)e^{-st}dt = e^{-nsT}$?

Remember the definition of the delta function: it integrates $1$ all over the t-axis but it is zero for any $t\neq 0$. This means that if we multiply any function by $\delta (t)$, then the function is going to be zero everywhere except for $t=0$. In this case, we have

$$ \delta (t-nT)e^{-st}=
\left\{
 \begin{array}{ll}
  \delta (t-nT)e^{-snT}  & \mbox{if } t = nT \\
  0 & \mbox{if } t\neq nT
 \end{array}
\right.$$

So if we integrate this function, we have that the integral is zero for every point except for $t=nT$. So

$$\int_0^\infty \delta (t-nT)e^{-st}dt=\int_0^\infty \delta (t-nT)e^{-snT}dt=e^{-snT}\int_0^\infty \delta (t-nT)dt$$

But by definition, $$\int_0^\infty \delta (t-nT)dt=\int_{-\infty}^\infty \delta (t-nT)dt=1$$

So $$\int_0^\infty \delta (t-nT)e^{-st}dt=e^{-snT}$$

Sidenote: For this to be true, $n$ must be positive. If it is negative, then the integral will be $0$ (as the function inside the integral will be zero for all the integration interval).


$N$ point moving average filters in state space

Yes sure they are LTI. Let $A$ be the $(L-1)\times (L-1)$ shift matrix
$$
A := \begin{pmatrix}0 & 1 & 0 && \dots & 0\\0 & 0 & 1 & 0 &\dots &0\\\vdots &&& \ddots && 0\\0&&\dots&&1&0\\0 &&\dots&&0&1\\0&0&\dots&0&0&0\end{pmatrix}
$$
and
$$
B = \begin{pmatrix}0\\0\\\vdots\\0\\1\end{pmatrix},\qquad C=\begin{pmatrix}1/L&1/L&\dots&1/L\end{pmatrix},\quad D=1/L
$$
Call $z$ the state variables, then a state space model is
\begin{align*}z(k+1) &= Az(k) + Bx(k)\\y(k) &= Cz(k)+Dx(k)\end{align*}
Infact one has
\begin{align*}
z_1(k) &= z_2(k-1) = z_3(k-2) = ... = z_{L-1}(k-L) = x(k-(L-1))\\
z_2(k) &= x(k-(L-2))\\
&\vdots\\
z_i(k) &= x(k-(L-i))\\
&\vdots\\
z_{L-1} &= x(k-1)
\end{align*}
thus
\begin{align*}
y(k) &= \dfrac{1}{L}\left(\sum_{i=1}^{L-1} x(k-L+i) + x(k)\right)= \dfrac{1}{L}\left(\sum_{m=1}^{L-1} x(k-m) + x(k)\right) \\&= \dfrac{1}{L}\sum_{m=0}^{L-1}x(k-m)
\end{align*}


Calibration (intrinsic) parameters of a virtual camera with some known information

Since your camera is simulated and the scenes are synthetic, you already have this information: Check out my updated answer to this question.


In FSK, you don't care about the frequency-varying phase properties of your channel – they simply don't matter to your signal, so a (relative) amplitude estimate for all mark frequencies would technically suffice.

Most FSK receivers don't even do that – it's only helpful to know that "mark frequency A gets through stronger than frequency B" if you use that to adjust your decision boundaries (e.g. for an MAP decision, you'd need that knowledge).

I don't have access to any DASH7 phy layer documentation, but if it's really derived from RFID tag technology, then you can be 100% certain the thing works without channel state information. Complexity wouldn't allow that. That doesn't mean you can't make it work better at further distances – but it does imply that if you're well within the specified ranges and environments, getting a channel estimate will not help you significantly much.
Wikipedia says "range up to 2 km, less than 200 kHz bandwidth", and that basically means your second observable multipath component would have to have a 1.5 km longer way to travel than the first one you receive – that is physically practically irrelevant, as that delayed version of the signal will be so much weaker than the quicker path that it can't lead to significant frequency-selectivity.

So the answer to your question is "you don't do that at all".

The rest of this answer, hence, is of merely academic interest:

Now, if your channel is actually strongly frequency-selective, so that knowing the relative amplitude becomes desirable, you can also be sure that the transition between different frequencies (which is Gauss shaped, transmitter-side) isn't nice anymore – and that's bad for your Gauss-expecting receiver.

Now, the only thing that could be FSK-specifically done here is some form of nonlinear emphasis on some values – that really doesn't sound very tempting, because estimating the necessary distortion to counteract the nonlinear memory-affected effect of a frequency selective channel on the FM demodulation of a frequency transition sounds pretty hard. And whenever possible, we try to compensate systems where they are linear – that gives us at least some guarantees about invertibility/well-posedness of the inverse problem.

So, don't try to deal with your FSK in any special way. Instead, just do what works for any transmission type:

transmit something you know – for example, a preamble, or a known data packet – that is as white as you get with the given scheme (in this case: transmit pseudorandom white data through your GFSK transmitter)
Correlate with that at the receiver
Since that correlation will be mathematically equivalent to (noisy) convolution of your transmit signal's autocorrelation function and the channel impulse response: It's your channel estimate. You don't need it any better than that, because it excites exactly the frequencies present in your signal.

The first step – transmission of something you know – is in many systems done via a decision feedback equalizer, where "something you know" is what you received and decided for, but that means block-wise operation in an FSK case, since your system is not linear in modulation pulses, but has memory.

TL;DR: Don't. Your system design implies the channel is flat, and hence you don't need a channel estimate. Spend your time on something else!


FFT- based frequency offset estimation algorithm

Given the links in the question, the OP is interested in more details on implementing a squaring algorithm for carrier recovery. A squaring algorithm is a popular analog option for the recovery of BPSK and QPSK modulations based on its simplicity and is completely applicable to digital implementations as well, but it is not the only algorithm nor most efficient. Specific to squaring algorithm for carrier recovery, the following will help understand how this approach works and its limitations.
If the modulation is BPSK, then the symbols each have a carrier phase of either 0 or 180 degrees relative to the unmodulated carrier (which the OP is trying to recovery). A squaring loop strips the modulation by using the property:
$$\cos(\alpha)cos(\beta) = \frac{1}{2}cos(\alpha - \beta) + \frac{1}{2}cos(\alpha + \beta)$$
From above observe what happens if we square a BPSK signal given as
$$f(t) = \cos(2\pi f t+\phi[m])$$
Where $\phi[m] = 0, \pi$ for each symbol $m$
$$f^2(t) = \frac{1}{2}\cos(0) +\frac{1}{2}cos(4 \pi ft + 2\phi[m])$$
The first term is a DC offset value and is ignored or filtered out. Notice what happens in the second term from which we can now derive a constant carrier signal: The frequency has doubled (so squaring a tone in general will double its frequency, and frequency doublers are implemented by doing this) AND importantly the modulation has been completely removed, since doubling a signal's frequency will also double its phase. Since the phase was going between $0$ and $\pi$ (180 degrees) before being doubled, it will go between $0$ and $2\pi = 0$ after being doubled.  This is a common analog recovery scheme due to the simplicity of implementing a frequency doubler (using a mixer, or even an XOR gate), and a frequency divider (with a flip-flop) to divide the recovered tone without the modulation by 2 to recover the coherent BPSK carrier.  Pulse shaping will introduce other spectral artifacts, but they will all be sufficiently lower than the strong recovered carrier such that a PLL can be used to clean-up and provide a clean recovered coherent carrier for demodulation.
The same approach can be used for QPSK, but in this case we have four phase states, so the doubling needs to occur twice (resulting in $f^4(t)$).  Doubling once will reduce the QPSK signal to a BPSK signal at double the carrier frequency, and doubling that will then strip the remaining BPSK modulation for a recovered carrier at four times the original frequency. (We also see that for any four quadrature phase states, if we raise it to the fourth power it will reduce to one phase state, meaning the modulation will be stripped: for example, for all phases $(0, \pi/2, \pi, 3\pi/2)$, any of these phases multiplied by 4 will be modulo $2\pi$ which is a phase equal to 0.
This approach could be done using an FFT such as first multiplying the QPSK signal by four and then use the FFT to find the strongest carrier that is at approximately four times the original carrier frequency, but that is highly inefficient and an approach I would not recommend following (this is basically breaking fred harris' rule "Don't copy the analog!" given we have so much more ability to provide direct optimized solutions with high efficiency with more direct DSP approaches targeting the core problem, where analog implementations are often much more constrained by the physical components themselves). Further it immediately requires a sampling rate 4 times higher to handle the 4x carrier increase (even for any given carrier offset). Further once the strongest FFT bin is found, the actual frequency will inevitably be at an smaller offset between the bins, so further resolution will be required to maintain actual coherence. I suggest alternate carrier recovery approaches at the links below, but if it is necessary to pursue a squaring approach, this would typically be done in a loop such that a PLL is used to clean-up the strongest single tone signal (at double the actual carrier for BPSK or four times the actual carrier for QPSK).
For a suggested carrier recovery approach please also see these other links:
High modulation index PSK - carrier recovery
BPSK and QPSK demodulation computational complexity
Demodulation of 4-QAM


Downsampling and Convolution: When Can one Swap Between Them?

I thought this was because you were getting aliasing in your signal during one of the downsampling operations. However, I re-wrote your code in R (included below) and I did not see the same effect  I suspect conv is doing something odd. You may want to look at using the shape parameter and see if that makes a difference.

Below are four plots of different variants of yours:

Top left: Yours.
Top right: a variation that uses slightly different x values.
Bottom left: Yours at twice the frequency.
Bottom right: a variation that uses slightly different x values at twice the frequency.

As you can see, in all four examples the red and the black overlay exactly.

Yes, in general, these will not be the same. Below is an example with a sawtooth wave, and the two do not line up.

Here is the specific example you ask about with the triangle wave.

R Code Only Below

#43794
#N = 100;
N <- 100
#dsfactor = 3;
dsfactor <- 3
#H = dsfactor*3;
H <- dsfactor * 3
#x = linspace(0,10,N);
x1 <- seq(0,10,10/(N-1))
x2 <- seq(0,10*(N-1)/N,10/N)
#y = sind(18*x);
y1 <- sin(18*x1/180*pi)
y2 <- sin(18*x2/180*pi)
y3 <- sin(36*x1/180*pi)
y4 <- sin(36*x2/180*pi)
#h = ones(H,1);
h <- rep(1,H)

#convyh = conv(y,h);
convy1h <- filter(y1,h)
convy2h <- filter(y2,h)
convy3h <- filter(y3,h)
convy4h <- filter(y4,h)

downsample <- function(signal, factor)
{
  return(signal[seq(1,length(signal), factor)])
}

#convyh_ds = downsample(convyh,dsfactor)./H;
convy1h_ds <- downsample(convy1h, dsfactor)/H
convy2h_ds <- downsample(convy2h, dsfactor)/H
convy3h_ds <- downsample(convy3h, dsfactor)/H
convy4h_ds <- downsample(convy4h, dsfactor)/H
#convyh_ds3 = conv(downsample(y,dsfactor),downsample(h,dsfactor))./(ceil(H/dsfactor));
convy1h_ds3 <- filter(downsample(y1,dsfactor), downsample(h,dsfactor))/ceiling(H/dsfactor)
convy2h_ds3 <- filter(downsample(y2,dsfactor), downsample(h,dsfactor))/ceiling(H/dsfactor)
convy3h_ds3 <- filter(downsample(y3,dsfactor), downsample(h,dsfactor))/ceiling(H/dsfactor)
convy4h_ds3 <- filter(downsample(y4,dsfactor), downsample(h,dsfactor))/ceiling(H/dsfactor)
#figure;
#plot(convyh_ds,'DisplayName','downsample(conv)','LineWidth',1);
#hold on;
#plot(convyh_ds3,'DisplayName','conv(downsample)','LineWidth',1);
#grid minor
#legend('show');
#line([0 length(convyh_ds2)],[0 0],'LineStyle','--');

par(mfrow=c(2,2))
plot(convy1h_ds)
lines(convy1h_ds3, col='red')

plot(convy2h_ds)
lines(convy2h_ds3, col='red')

plot(convy3h_ds)
lines(convy3h_ds3, col='red')

plot(convy4h_ds)
lines(convy4h_ds3, col='red')


Digtial FIR Impulse response & transfer function

You can solve this with high-school algebra and the time delay property
$$ \mathcal{Z}(x(n-k)) = z^{-k} X(z) $$
Applying this to your expression
$$y(n)=2x(n)-1x(n-1)+3x(n-2)$$
we get
$$Y(z) = 2 X(z) - X(z)z^{-1} + 3X(z)z^{-2}$$
$$\frac{Y(z)}{X(x)} = 2z^{-2} - z^{-1} + 3$$
The denominator is null because the filter is not recursive