Distribution of the output of AWGN channel

I see this question is open for a long time, I am not sure if I understand it correctly.
PDF of the symbols
The first probability is a $P_{\mathbf{Y}}: \mathbb{C}^n \to \mathbb{R}$, you are not considering phase noise, the probability  of the $n$ symbols are identically distributed, let's say with $P_{\mathbf{Y}_i}: \mathbb{C}^n \to \mathbb{R}$, since the joint probability of independent variables are the product of the individual variables this distribution can be written as
$$P_{\mathbf{Y}}(y) = \prod_{i=1}^{N}P_{\mathbf{Y}_i}(y_i)$$
Where $P_{\mathbf{Y}_i}(y_i) = \sum_{x\in \mathbf{X}_i}p_{\mathbf{X}}(x_i) e^{-||y_i - x_i||^2}/\pi$ which leads to
$$P_{\mathbf{Y}}(y) = \frac{1}{\pi^N}\prod_{i=1}^{N} \sum_{x\in \mathbf{X}_i} p_{\mathbf{X}}(x_i) e^{-||y_i - x_i||^2}$$
The term $p_{\mathbb{X}}(x_i)$ can be used to model probabilistic shaping, but for the original quesiton it will probably be a the inverse of the number of symbols in the constellation $1/c = 1/|\mathbf{X}_i|$, that by it's turn is the same for all symbols, thus
$$P_{\mathbf{Y}}(y) = \frac{1}{c^N\pi^N}\prod_{i=1}^{N} \sum_{x\in \mathbf{X}_i} p_{\mathbf{X}}(x_i) e^{-||y_i - x_i||^2}$$
The complex of evaluating this will be $c N$ exponentials. The number of additions and multiplications will be linear in $cN$ as well, differently from the initial formula that would compute $c^N$ terms.
For some applications you may want the logarithm of that expression, in such situations it is preferable to use the sum of logarithms
$$log\left(P_{\mathbf{Y}}(y)\right) = \sum_{i=1}^{N}\log\left(\frac{1}{c \pi} \sum_{x\in \mathbf{X}_i} p_{\mathbf{X}}(x_i) e^{-||y_i - x_i||^2}\right)$$
All of this is valid for QAM modulation as well, and don't have the exponential number of terms in the initial formula.
PDF of the energy
In the second part you mention the PDF of the energy of the symbols
That is the probability distribution of
$$ \sum_{i=0}^{N} \sum_{x_i \in \mathbf{X}_i} p_{\mathbf{X}_i}(x_i) ||x_i + w_i||^2$$
Assuming $w$ having a variance $\sigma$ for both real and imaginary parts, since the norm of a complex number $||a + 1j b||^2 = a^2 + b^2$, assuming uniform probability for the symbols in the constelation, we can divide the whole expression the symbols $\sigma$ (since the the formula is given for a covariance matrix being the identity). And the distribution of the energy will be of the form $\sum_{k=1}^{2N} X_{i}^2$ as assumed for the noncentral $\chi^2$ distribution with $2N$ degrees of freedom.
The noncentrality parameter given by
$$\lambda =\frac{N}{c \sigma^2} \sum_{x \in \mathbf{X}} ||x||^2$$
The number of degrees of freedom $k=2N$, because we separated real and imaginary parts.
Let $z$ be the energy of one sample, taking in to account that the $\chi^2$ distribution was normalized by dividing by the variance, the PDF of $z$ will be
$$p_{||\mathbf{Y}||^2}(z) =
\frac{1}{2\sigma} e^{\left(\frac{z}{2\sigma} + \frac{\lambda}{2}\right)}
\left(\frac{z}{\lambda \sigma}\right)^{(N-1)/2}
I_{N-1}\left( \sqrt{\frac{\lambda z}{\sigma}}\right)
$$
Note:
In spite of being a closed formula it depends on the computation of $I_{\alpha}(x)$, a modified Bessel function, that for positive integer $\alpha$ and positive $x$ reduces to a series with rational coefficients in $x$, thus
$$ I_{N-1}\left( \sqrt{\frac{\lambda z}{\sigma}}\right) = \sum_{m=0}^{\infty} \frac{\left(\sqrt{\frac{\lambda z}{\sigma}}\right)^{2m+N-1}}{m!(m+N)!} $$


Is there a scientific name for this technique?

You can call $C(k_f)$ the "angular coherence", or "angular correlation coefficient" at frequency $k_f$.

Alright, I just made that up if you just need a name for it! If you need some interpretation, here is some detail:

A complex number can be represented by a vector in the polar system.  So we have $M$ vectors here. The numerator is a vector sum and the denominator is a normalizing factor. From the triangle inequality, the magnitude of $C$ is always in the interval $[0,1]$. $C$ is zero if the vectors are anti-symmetric pairs or if they completely cancel eachother, and is one if all are equal. So the remaining possibilities represent how close their composition are to either these two extremes.

I suppose you asked for the name to find more resources. Alternatively, you can ask your possible related questions directly, and probably there are some people here to answer.


When channels are in a cascade and are independent is the capacity the sum?

Let the first channel, channel A, have capacity $C_A=1-\varepsilon$ and the second channel, channel B, have capacity $C_B=1-\varphi$. Now put them in cascade, that is, first channel A then channel B. The resulting capacity of the cascaded channel will be $\text{min}(C_A, C_B)$. You can figure this out by thinking about the information flow like water through pipes. For example, if you're able to push $X$ bits through channel A but only $X-10$ bits through channel B, well then the total capacity will be limited by the smallest capacity channel in the cascaded system. Hope this helps!


FFT of a pure sine wave is not a single peak

Just as I wanted to click the "ask" button, I realized that I had run into a fencepost problem.

My time vector is too long by 1 point. This means that the first and last point of my voltage vector are the same, which in turn means that my periodic signal has a phase shift as it repeats. This also explains why the phenomenon went (almost) away as I decreased the frequency.

If I instead define my time vector properly as

t = 0:1/f_sample:T_total-1/f_sample;

then I get the result that I expect: A single peak and a tiny bit of noise around it.


System Properties; Linear, Causal, Time-Invariant, Stable?

Such questions have been asked here before, and the concepts are simple and important. Dilip Sarwate is right to ask you to add details as to where you get stuck, so we can give better answers (instead of just solving the problem for you).

I'll try to explain the basics in a clear way so that this answer may also be useful for other users, and to help you understand and solve the problem yourself. Assuming you understand linearity, let's move on to the other properties:

time-invariance: if $y[n]$ is the response to input $x[n]$, $y[n-m]$ must be the response to $x[n-m]$ for any (integer) $m$. By inspection, this is obviously the case for systems A, B, and D. System C can get slightly confusing for beginners, but just use the criterion to see that for $x[n-m]$ the output is $x[-n-m]$ which does not equal $y[n-m]=x[-(n-m)]=x[-n+m]$. So system C is NOT time-invariant. For system E a simple substitution of the summation index shows you that the system is indeed time-invariant.
causality: this is actually very simple. Just answer the question "does the output signal at any time depend on future values of the input signal?" If the answer is no, then the system is causal, otherwise it isn't. Systems A, B, and D only use the current input value to compute the output, so consequently they are all causal. System E uses the current and past input values, so it's also causal. Only system C is not causal because for negative values of $n$ the output depends on future values of the input.
stability: usually the concept of bounded-input-bounded-output (BIBO) stability is used, which means that if the input signal is bounded, i.e. $|x[n]|\le M$, then also the output signal is bounded, i.e. $|y[n]|\le N$, for some arbitrary positive constants $M$ and $N$. For systems A, B, C, and D this is obviously the case, because the output can never be unbounded for bounded $x[n]$ (assuming that the constants $A$ and $B$ are finite, of course). For system E that's different, because you have an infinite sum which needs to converge for the output to be finite. But for a bounded input signal you can always construct a case where the output is unbounded. Just choose a constant input signal with any finite value, and the sum will grow without bounds. Hence, system E is not BIBO stable.


Upsampling requires two steps:

Inserting zeros. This does indeed replicate the spetcrum in the frequency domain. For N zeros inserted you get N copies of the original spectrum
Lowpass filtering to remove the mirror spectra. That's often also called an "interpolation filter".

The choice of interpolation filter depends a lot on the requirements of your specific application (signal to noise ratio, spectral suppression, shape of original spectrum, transient behavior, phase distortion, latency, MIPS, etc.). There is no "one size fits all" solution


Cyclic prefix vs. guard interval. Why use one over the other?

A silent  guard interval would likely increase PAPR (peak to average power), which requires a sender capable of more dynamic range (without clipping or generating power supply noise, etc.)

A long enough cyclic prefix prevents multipath echoes from one symbol interfering with the portion of the next symbol after the prefix.  And a symbol interfering with itself from only its own multipath echo can be modeled as a flat fading channel, which may be easier to equalize or to error correct over a far wider range of multipath channel responses.


Is a cubic Lagrange interpolation tensor product the same as bicubic interpolation?

i don't need the points, but the Community keeps resurrecting unanswered questions (or answered questions that have no accepted answer) until i go batty seeing them on the list, time and time-again.

:-\

i think the bicubic interpolation is more likely 3rd-order Hermite polynomial than 3rd-order Lagrange polynomial interpolation. the former guarantees continuity of both the zeroeth and first derivative. the latter only guarantees continuity of the zeroeth derivative (the interpolated function itself).

this previous answer deals pretty completely with the necessary math of it.


Non Uniform DFT Algorithm by Dutt & Rokhlin

You may download the NFFT Library - Non Equispaced Fast Fourier Transform.
It has interfaces for MATLAB, Octave and Julia.


The transfer function is

$$H(s)=\frac{g}{\prod_k(s-p_k)}\tag{1}$$

where $g$ is a scaling constant which is usually chosen such that the maximum value of the magnitude of $H(j\omega)$ equals $1$.

For a third order (or any odd-order) Chebyshev filter, a maximum occurs at $\omega=0$, so the gain constant can be derived from the requirement $H(0)=1$:

$$g=\prod_k(-p_k)=-\prod_kp_k$$

For your example the gain constant is $g=0.49134$.


Image Processing Prior to Feature Detection

Do you really have to use Harris corners? There is many features developed after Harris corners, with better properties. A good overview can be found in this article:

A.L. Dahl, H. Aanæs, and K.S. Pedersen. Finding the Best Feature Detector-
Descriptor combination

Based on that article as well as my personal experience, I would suggest switching either to MSER (Maximally Stable Extermal Regions), or even combine them with DoG (Difference of Gaussians) -- the features first presented as a part of the SIFT pipeline.

If the problem really is in low contrast, then MSER features should really make you happy: they are (fairly) invariant to changes in lighting. In short, they are connected regions of the image stable through a series of different threshold binarizations.

The feature extraction process is independent from calculating the descriptors, so it shouldn't be too hard to integrate new ways of feature extraction in to your process.

Also, I've heard of (but never actually worked with) Multiscale Harris corners as an extension to Harris corners. I don't know much about them and personally can not recommend any reading materials on this topic, so I leave article search and picking the most interesting materials to you.

Furthermore, might I suggest that the image you posted might have other problems than low contrast. In my personal experience, vegetation like bushes or possibly the field you have, as well as the lovely bubbly clouds tend to produce "generic features" -- features which tend to have equally similar (or dissimilar) descriptors as a lot of other features.

Practically, this means that when doing feature matching on two images from a different perspective, features extracted from these kinds of surfaces tended to be falsely matched.  I have done a Master thesis that in a large part deals with feature extraction to be used in feature matching further used to calculate a homography transformation between two images when I came across this problem. I didn't find any other articles describing this problem at the time, but my thesis might be helpful for your overall approach.

Lastly, as you have set, thresholds and techniques that work just fine on most images extract to little features in this kind of images, because of its mostly homogeneous areas. This kind of images present problems in feature matching (which can be extended to image stitching), content based image retrieval, and I would presume tracking as well as similar applications. No method currently works quite well on them.

Methods that work good on this kind of images as well as the typical cases are being explored and researched currently, such as an approach I started working on briefly described in this answer.


Frequency analysis to determine low-pass cut off frequency

I think you will get almost the same amount of frequency content for both the cases (fft of 60 ms and fft of 2 sec). The time series indicates that only from 0.6 to 1.2 sec there is vibration. But it's better to use the full signal while doing the FFT as there is no fear of losing any data. If you cut some portion from the time series, you have to be careful about choosing the window (use cosine tapering not rectangular window) as edging effect may be introduced into your frequency content.
Sorry, I didn't get the question. Could you please rephrase it?
I think that for all the trials, the time series would be different. Therefore, it is not recommended to average all the time series. It is better to analyze all the time sequences and obtain the frequency contents. If dominant frequencies for all the trials are similar, then the average frequency can be calculated. If not, you can try to find out the reason e.g. inconsistency in the sample material etc.


Interpreting the FFT results of a structural vibration problem

If the beam is under the excitation of a sweeping wave, then the peak(s) in the FFT of the response is not necessarily the natural frequency of the beam. As an extreme, if you are exciting the beam with a constant frequency omega, your FFT of the acceleration time history will have a dominant peak at omega.

However, if the beam is under random excitation, or an impulse, the first peak in you FFT will be your natural frequency. And there is a method named "peak-picking method" to find out the natural frequency. Here is a reference for that: Google books


Given an impulse response of
$$h(t) =
\begin{cases} \frac{1}{t^2} &,& \text{ for } t < 1/2 \\ 0 &,& \text{ otherwise } \\ \end{cases}
$$

Its associated CTFT (frequency response) is
$$H(\Omega) = \int_{-\infty}^{\infty} h(t) e^{-j \Omega t} dt $$

Now look at its value at $\Omega = 0$ which is $H(0)$;
$$H(0) = \int_{-\infty}^{\infty} h(t) dt = \int_{-\infty}^{\infty} \frac{1}{t^2} dt = 2 $$

Conclusion: since $H(0)$ is $2$ , then $1$ or $1/2$ or any number $x$ less than $2$; i.e., $x < 2$, cannot be an upper bound for $H(\Omega)$ as it already passes that value...


How is loop information encoded in FLAC files?

The simple answer is, "the same way it's kept in .wav files."  FLAC uses the same RIFF "chunk" structure as WAV.  The WAV file format has a chunk for use by samplers, which includes looping information.  I found I could simply copy that chunk from the WAV file and append it to the FLAC file and it worked (for anything that supported loops in FLAC files, which isn't everything -- for example linuxsampler does not, due to a library it uses not supporting it yet.)
Better yet, the flac application has a --keep-foreign-metadata option that does the trick without my having to code anything.


How significant is the time lag between inertial IMU data in a inertial analisys?

It depends on your algorithm, and on whether the timing is known.
Assuming that the timing of each measurement is known, you can incorporate measurements at unsynchronized moments by executing a Kalman update step for the measurements you have, at the moment the measurements happened -- just use a measurement matrix ($H$ in most Kalman literature) that matches the measurement you have, and do your update.
This does mean that you need to use the appropriate state update equations, which makes for the filter design being a royal pain in the behind -- but it's more a matter of dotting a bunch of 'i's and crossing a bunch of 't's (and not crossing 'i's or dotting 't's) than it is a matter of the basic math getting worse.


Finding the memory polynomial model coefficients

Memory polynomial system identification is a linear regression problem.  Suppose you sent a $N$-element sequence $x(n)$ through a system and obtained an $N$-element output sequence $y(n)$.  If the system were described by a memory polynomial, then your model would be...

$y(n) = \sum_{q=0}^{Q}[x(n-q)a(q)+x(n-q)|x(n-q)|^2 b(q)] + \epsilon(n)$

Here $a(q)$ and $b(q)$ are coefficient of the models and $\epsilon(n)$ is an error term.  This is a linear model in $a$ and $b$, and can be expressed in matrix form as...

$\begin{bmatrix}y(Q)\\y(Q+1)\\...\\y(N-1)\end{bmatrix} = \begin{bmatrix}x(Q)&...&x(0)&x(Q)|x(Q)|^2&...&x(0)|x(0)|^2\\x(Q+1)&...&x(1)&x(Q+1)|x(Q+1)|^2&\ldots&x(1)|x(1)|^2\\\ldots&\ldots&\ldots&\ldots&\ldots&\ldots\\x(N-1)&\ldots&x(N-1-Q)&x(N-1)|x(N-1)|^2&\ldots&x(N-1-Q)|x(N-1-Q)|^2\\\end{bmatrix}\begin{bmatrix}a(0)\\...\\a(Q)\\b(0)\\...\\b(Q)\end{bmatrix}+\begin{bmatrix}\epsilon(0)\\\epsilon(1)\\...\\\epsilon(N-1)\end{bmatrix}$

You can solve for $a$ and $b$ using any of the linear least squares functions in MATLAB, Python, etc.


If a time-series has odd number of samples does it have no energy at Nyquist frequency?

No, that's not what it means. You just don't get a frequency bin at Nyquist if the DFT length is odd. If you want you can always append one or more zeros to the time domain sequence such that the total length becomes even.
If you're just interested in the energy at Nyquist, you could also simply compute
$$E_{Nyq}=\left|\sum_{n=0}^{N-1}x[n](-1)^n\right|^2\tag{1}$$
where $N$ is the length of the sequence, no matter if it's even or odd.


kernel to calculate second order partial derivative of digital image

Your five-point derivative kernel is a 1D kernel. Applied along the x axis it gives the partial derivative for x, applied along the y axis it gives the partial derivative for y.
The $\frac{\partial^2}{\partial x \partial y}$ derivative would need a 2D square kernel. It is more efficient to compute this by applying two first order partial derivatives.
$$\frac{\partial^2}{\partial x \partial y} f = \left( \frac{\partial}{\partial x} \frac{\partial}{\partial x} \right) f = \frac{\partial}{\partial x} \left(  \frac{\partial}{\partial x} f \right)$$
Since $\frac{\partial}{\partial x} f$ is a convolution,
$$\frac{\partial}{\partial x} f = k_x \ast f$$
we can write the earlier equation as
$$\frac{\partial^2}{\partial x \partial y} f = \left( k_x \ast k_y \right) \ast f = k_x \ast \left(  k_y \ast f \right)$$
So we can say that the 2D square convolution kernel that you are looking for is $k_x \ast k_y$ (with $k_x$ the kernel that computes the x-derivative and $k_y$ the one that computes the y-derivative). Note that, if you think of your 1D kernels as vectors, this convolution is equivalent to the outer product of the two vectors.
However, it is more expensive to compute a convolution with one 5x5 kernel (25 multiplication and additions per pixel) than to compute two convolutions with 5x1 and 1x5 kernels (10 multiplications and additions per pixel). I would compute two first order derivatives in sequence.


It is a common value because at $-3 \ \mathrm{dB}$ the power of the signal is reduced to half its value. I'll show a brief example to make it clearer.

Suppose that you have a signal whose amplitude is $1 \ \mathrm{V}$. If we reduce its amplitude by $-3 \ \mathrm{dB}$, the new amplitude would be close to $ 0.707 \ \mathrm{V} $.

The power of a signal is proportional to the square of its amplitude. For simplicity, let's assume a resistance of $1 \ \Omega$ on which we are going to calculate the dissipated power. Therefore, originally we would have had a power of $$(1 \ \mathrm{V})^2\cdot\frac{1}{1 \ \Omega}=1 \ \mathrm{W}$$

But after the reduction, we have a power of

$$(0.707 \ \mathrm{V})^2\cdot\frac{1}{1 \ \Omega}=0.5 \ \mathrm{W}$$

The power has decreased to half its original value. That is why the $-3 \ \mathrm{dB}$ point is commonly used to characterize frequency-related properties.


Chromatogram peak detection - bunching vs others?

peak(s) determination tends to be application specific.  There are many heuristics and one usually needs to try and test a lot.

It helps a lot if there is a reliable statistical model for your data.

One trick is to note that peak is larger than its adjacent neighbors.  Peaks on boundaries can be determined by reflection.

Another issue is the set that you are finding peaks on.  Some data consists of multiple scans that can be averaged to reduce noise. One can also do what you refer to as bunching and bunching has numerous variations such as summing, weighted summing, taking a median and combinations.

Another concern may be strong peaks bleeding into weak peaks or masking them entirely.  Many applications have this bin bias problem and there are ways to approach that which are beyond what can be answered in a simple answer given here.

One can also use thresholds to isolate peaks and again that has many variations.

Usually, a community has conventions on how to approach problems like finding peaks and there is probably some documents that constitute a standard. You might want to look for those.


Is there such a thing as a continuous phase BPSK?

You're right that the transitions can be abrupt, but you can always choose to use pulse shaping, i.e., instead of using a rectangular pulse, use some smoother function $p(t)$. The corresponding baseband signal is then

$$s(t)=\sum_kA_kp(t-kT)\tag{1}$$

where $A_k$ are the symbols, and $T$ is the symbol length. A smooth function $p(t)$ will avoid abrupt transitions, and will therefore reduce spectral sidelobe levels. I think this paper uses that idea.


Problem 1:

As pointed by learner, awgn adds noise to the sequence y. So you are not plotting the autocorrelation of white gaussian noise, but the autocorrelation of white gaussian noise plus a constant.

Use zeros instead of ones ; or analyze the difference $z - y$ ; or since $y$ is a constant signal, remove the mean of $z$.

Problem 2:

Once point 1 is solved, and since your sequence is of finite length, you will be plotting the autocorrelation of white noise multiplied by a square window. What you will see is thus the autocorrelation of white noise convolved by the autocorrelation of a square window. The autocorrelation of a square window has a triangular shape.

Another way to look at it: the further you move from 0, the less data is present in your input vector. For example, in the sequence:

$-1, 4, 5, 6, -2$

There are 4 pairs of samples distant by a lag of 1, 3 pairs of samples distant by a lag of 2 (-1 and 5 ; 4 and 6 ; 5 and -2), and only two pairs of samples distant by a lag of 3 (-1 and 6, 4 and -2). Thus, there won't be as much data to estimate correctly the autocorrelation function for the larger values of the lag.

You can artificially compensate for the windowing by providing the 'unbiased' argument to xcorr.

Problem 3:

Keep in mind that the theory is about infinite length sequences (or for finite length sequences, about the expected value of the result).


Neural Networks and Complex Valued Inputs

The power of complex representations remains an open topic to me. I still do strive the understand Fourier transformations.

An underlying question is, to me: why would complex transformations be useful for real data? More generally, when data dwell in a set $S$, is $S$ the most appropriate set of analysis, or is it more appropriate to resort to a bigger set $S^*$? For instance, for real-valued polynomials, we know that the field of complex numbers provides a more elegant extension. This might not be so distant, as $z$-transforms (extended polynomials) are tools of choice for real time-series analysis. Real linear-time invariant systems are root signals (here eigenvectors) that are complex (cisoids). To better separate frequencies, time-frequency analyses often employ analytic signals and the Hilbert transform, or analytic, dual-tree multiscale tools like wavelets. The recent paper Complex-Valued Signal Processing: The Proper Way to Deal With Impropriety deals with more stochastic observations:

  Complex-valued signals occur in many areas of science and engineering
  and are thus of fundamental interest. In the past, it has often been
  assumed, usually implicitly, that complex random signals are proper or
  circular. A proper complex random variable is uncorrelated with its
  complex conjugate, and a circular complex random variable has a
  probability distribution that is invariant under rotation in the
  complex plane. While these assumptions are convenient because they
  simplify computations, there are many cases where proper and circular
  random signals are very poor models of the underlying physics. When
  taking impropriety and noncircularity into account, the right type of
  processing can provide significant performance gains. There are two
  key ingredients in the statistical signal processing of complex-valued
  data: 1) utilizing the complete statistical characterization of
  complex-valued random signals; and 2) the optimization of real-valued
  cost functions with respect to complex parameters. In this overview
  article, we review the necessary tools, among which are widely linear
  transformations, augmented statistical descriptions, and Wirtinger
  calculus. We also present some selected recent developments in the
  field of complex-valued signal processing, addressing the topics of
  model selection, filtering, and source separation.

But going back in time (Oppenheim's works for instance), one knows that complex phase can capture non-stationarity, discontinuity (edges and jumps) and oscillatory behaviors (textures). My present belief is that, at a given scale, a complex feature computed as a whole is more efficient at capturing local behavior that a pair of real and imaginary parts computed somewhat separately, incorporating some invariances with respect to translation or rotation.

As for neural networks, and of course deep learning, the recent theory of scattering networks, and subsequent works, have provided a solid ground for understanding how deep learning works, from a solid mathematical point of view, based on complex wavelet frames and non-linear operators. Another interesting paper for your question is:

M. Tygert et al., 2016, A Mathematical Motivation for Complex-Valued Convolutional Networks, Neural Computation

  Courtesy of the exact correspondence, the remarkably rich and rigorous
  body of mathematical analysis for wavelets applies directly to
  (complex-valued) convnets.

  Convolutional networks (convnets) have become increasingly important
  to artificial intelligence in recent years, as reviewed by LeCun,
  Bengio, and Hinton (2015).This note presents a theoretical argument
  for complex-valued convnets and their remarkable performance.
  Complex-valued convnets turn out to calculate data-driven multiscale
  windowed spectra characterizing certain stochastic processescommonin
  the modeling of time series (such as audio) and natural images
  (including patterns and textures).We motivate the construction of such
  multiscale spectra using local averages of multiwavelet absolute
  values or, more generally, nonlinear multiwavelet packets.

And maybe, complex numbers  are not enough, and we should turn to quaternions...


The spectrum of a pulse train with a defined pulse with T that is indeed random in duration of zeros and ones would have the magnitude envelope of a Sinc function (PSD with envelope Sinc^2) with first null at 1/T.  Since the data is random the spectrum will be continuous. Similar to this is something like GPS where we transmit Pseudo-random patterns, but the pattern repeats (assuming no data modulation which again adds randomness). In this case, whenever there is a repeating pattern, you would see discrete tones at the repetition rate and every harmonic of the repetition rate, and the envelope would still be the Sinc function as described with nulls at 1/T. (Thus with doing forensics on such a spectrum you can discern the length and repetition rate of the sequence based on the spacing of the spectral lines, and the number of "chips" in the sequence based on the location of the null, etc.).

Finally multiplying by the carrier just moves this spectrum (which is best viewed in my opinion as a positive and negative spectrum even at baseband, as then you can translate it from there to any frequency with no changes to its features).  The fact that this is on-off keying means that it will have a DC component, so the carrier will come through very strong at the RF output. In comparsion if the signal was bipodal (which would then be BPSK modulation), with an equi-probable stream of 1's and 0's you would have no DC at baseband (avg =0) leading to supressed carrier at RF.  The carrier power in the former case is wasted (no information added) so is less efficient in terms of BER vs power than the latter case.

The reason for this is the base square pulse has a FT that is the Sinc function, and your random data is represented as a stream of impulses (with random positive and negative sign) that is convolved in time with your pulse (effectively a zero order hold). If the impulses are random in a pattern that never repeats, the spectrum of that pulse stream will be a continuous flat spectrum: white noise.  Therefore in frequency you multiply white noise with the Sinc to get the spectrum before being upconverted. The mixer translates this spectrum to the carrier.

Your picture just shows the main lobe, but if your pulse is indeed rectangular, you will have all the sidelobes of the Sinc function out to the effective BW (based on the rise/ fall time of your pulse).

I hope this clears it up for you!


Periodic signal checking when using $\Sigma$

Let's define two new signals:

$$x_1(t)=\cos\left(\frac{π}{75}t\right)$$

$$x_2(t)=\sum_{κ=-\infty}^{\infty}\left[\mathrm{rect}\left(\frac{t+2κ}{10}\right)\right]$$

It's easy to see that the signal in your question can be calculated as

$$x(t)=x_1(t)+x_2(t)$$

The period of $x_1(t)$ is $T_1=150$. That can be seen easily: when $t=T_1$, the argument of the cosine becomes $2\pi$, so the function "resets".

Now let's see what happens with $x_2(t)$. Each term of the summation is a rectangular pulse of width $10$ centered at $t=-2\kappa$. As the magnitude of the shift factor (in this case, $2$) is less than the width, some pulses will overlap with others when performing the sum.

In fact, for almost every $t$, there will be exactly five rectangular pulses adding up. I say "almost" because there are special cases. When $t\in\mathbb{Z}$ and is odd, there will be only four pulses adding there, but there will also be two "half-pulses" corresponding to the value the $\mathrm{rect}()$ function takes at the discontinuity (every pulse will be discontinuos at two odd integers only), which is $0.5$. Because the values of $\kappa$ extend all over the natural numbers, it is valid to redefine $x_2(t)$ so that:

$$x_2(t)=5 \ \forall t$$

You can try that using Octave. I ran this script:

kMin = -500;
kMax = 500;
shiftFactor = 2;
tMax = shiftFactor*kMax + 10/2;
tMin = shiftFactor*kMin - 10/2;
t = tMin:tMax;
x2 = zeros(length(t),1);

for k = kMin:kMax;
  for ii = 1 : length(t)
    if t(ii) > (-shiftFactor*k - 10/2) && t(ii) < (-shiftFactor*k + 10/2)
      x2(ii) = x2(ii) + 1;
    elseif t(ii) == (-shiftFactor*k - 10/2) || t(ii) == (-shiftFactor*k + 10/2)
      x2(ii) = x2(ii) + 0.5;
    end
  end
end

plot(t,x2,'r','Linewidth',1.5);
xlim([tMin-1 tMax+1]);
ylim([0 5.5]);
grid minor;

The values kMin and kMax are the limit values $\kappa$ takes. Ideally, they should be infinite, but that's obviously not possible on a computer. shiftFactor is the constant that multiplies $\kappa$ in the summation. You can play a little bit with that number to see how the signal changes. I hardcoded the width of the pulses ($10$). I used the convention that the pulse equals $0.5$ in the discontinuity. If you plot the result, you get something like this:

Note that the ramps appearing by the sides do not happen in the infinite case. They show up here because the pulses that should be adding up for those values of $t$ were cut off when letting $\kappa$ take values between $500$ and $-500$ only.

So $x(t)$ is basically a cosine with an offset. Therefore, the sum of the functions we defined at the beginning is periodic. Even more, the period of $x(t)$ is the same as that of $x_1(t)$. Ergo,

$$T=T_1=150$$


Are two signals are the same if their auto-convolution functions are the same?

Almost. Look at the autoconvolution in the frequency domain where the autoconvolution of $x$ (with itself) gives us $(X(f))^2$ in the frequency domain while the autoconvolution of $-x$ (with itself) gives us $(-X(f))^2 = (X(f))^2$. So, given an autoconvolution function, there are two (very related) signals $x$ and $-x$ that have the same autoconvolution function.

  Are two signals are the same if their auto-correlation functions are the same?

Not quite.  Now we are given $|X(f)|^2$ in the frequency domain and there are many different factorizations possible. For example, if $y(t)$ is a signal such that values taken in by its Fourier transform always lie on the unit circle in the complex plane (for every $f$, $|Y(f)|=1$) then $|X(f)Y(f)|^2 =  |X(f)|^2$ and so $x\star y$ has the same auto-correlation function as $x$. Note that $x(t)$ and $x(t-\tau)$ (which is a delayed version of $x(t)$) have the same autocorrelation function ($Y(f)$ happens to be $\exp(-j2\pi f \tau)$ here). Another factorization replaces $X(f)$ by $X^*(f)$ which tells us that $x(t)$ and $x(-t)$ (which is just $x(t)$ running backwards in time) have the same autocorrelation function.


How up sampling of a signal is done in real world?

If padding L-1 zeros were the only thing that's done for upsampling, nobody would actually store the end result with zeros that way, because the zeros would be redundant.
In practice, padding with L-1 zeros is always followed by low pass filtering. If that filtering is done with an FIR filter, a dumb implementation would have many coefficients being multiplied by 0. A smart implementation will instead use a polyphase approach where only those coefficients that won't see a guaranteed 0 will be multiplied and summed.
So that partially answers your question: most hardware or software implementations won't even do any padding. They will instead cycle through a set of polyphase filter coefficients, and do something with the result of these successive FIR filter outputs.
Things might be different when doing Matlab or a NumPy implementation: in that case, quick prototyping is often more important than the computational or storage efficiency of the implementation, and rather than using a polyphase filter, people will indeed insert zeros in between samples and convolve the result with the non-polyphase version of the FIR filter.


How do I use MATLAB to obtain a continuous state equation from a transfer function and ode45 to solve it?

You're doing pretty much everything right. The only problem is that your system is a third order system (s^3) and hence, the state x in

dx = A*x + B*u

is 3x1 but your initial condition xnot is a scalar. Replace

xnot = 100

with

xnot = [0 0 0].';

(or whatever initial condition you have) and it should work.


Yes, you can bandpass the QPSK signal before matched filtering and demodulation if you wish to do so, but what you need to devote some thought to is whether such an action is a wise decision. The answer might depend on various parameters that you have left unspecified.

Bandwidth of bandpass filter $<$ QPSK signal bandwidthIn this case, the filtered QPSK signal now has intersymbol interference (ISI) in it whereas the incoming signal did not, which makes the subsequent matched filtering and demodulation more problematical. Equalization might be needed to ameliorate the ISI which will add to the cost/weight/power consumption of the receiver, and error probability will likely suffer.
I didn't bring up the issue of the center frequency of the bandpass filter in the previous paragraph, but if the center frequency is different from the carrier frequency, then there is a whole 'other can of worms that you will have opened up.
Bandwidth of bandpass filter $>$ QPSK signal bandwidthThis is a much better notion that the previous one. The issue of mismatch between center frequency of the filter and the carrier frequency is still there, though. But, is this a good idea? Maybe, maybe not. Remember that the matched filter is also a bandpass filter. Indeed, as pointed out here, the matched filter has transfer function $S^*(f)$ where $S(f)$ is the Fourier transform of the QPSK signal, and so the matched filter has large gain at frequencies where the signal is strong and low gain at frequencies where the signal is weak   -- a resume'-writer, no less, since it is emphasizing the strong points and playing down the weak ones. Thus, additional bandpass filtering ahead of the matched filter is not really necessary from a purely DSP perspective. However, your circuit designer colleagues might well insist on the extra bandpass filtering on the grounds that it helps with dynamic range issues in low SNR situations.

So, explore all the ramifications before jumping into bandpass filtering. Do you really need to use a bandpass filter, or can you get by without it?


Circular wrapping of an asymmetric function (in DFT calculations)

The key is being even, notice that $f_{N+k} = f_{k}$, and when you do a circular wrapping of an infinite function you should have $f_{k} = \sum_{i=-\infty}^\infty f((i \cdot N + k) T_s)$, then naturally $f_{N-k} = f_{k}$, in practice you want to choose $f$ and $N$ so that $f$ vanishes before $N/2$, then you are safe to use only one term of the summation that define $f_{k}$.
This holds true for asymmetric signals, the important point is that if the convolution result has more than $N$ non-zero points then then you can only recover the samples that are less than $N$ indices from both ends of the signal.
In other words if you have a signal that is non-zero from $L \le i \le H$ then you can only recover the samples at $i$ such that $H - N < i < L + N$. If you have a response of length not greater than $N$ then you can recover all the outputs. If your response has length $N + d$ you can only recover $N - d$ samples from the output. And by circularly shifting by the inputs by different number of samples you can control which samples of the output will be usable. you can also split your operators in multiple chunks to process an infinite duration signal as in overlap-save method.
An example in python
There is a pair of useful function fftshift and ifftshift to perform these operations without unintended code obfuscation.
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import skewnorm;
a = 3

plt.figure(figsize=(6, 6))

# circularly shifted equally spaced grid
t = np.fft.ifftshift(np.linspace(-10, 10, 2048));

# filter
h = skewnorm.pdf(t, a)
# signal without circular shift
x = np.zeros((2048))
x[500:1300] = np.arange(-400, 400);
# apply filter to signal (result is circularly shifted, you can
# recover the signal in the natural order with ifftshift)
y = np.fft.irfft(np.fft.rfft(h) * np.fft.rfft(x)) / len(t)


Smooth (not) equally-spaced signal strength data

Between these I only have experience with Kalman filters. They don't need equally spaced measurements but you can make it a lot easier with it. Missing measurements are really easy to cope with.

If you want a simple solution and you think that the most likely outcome of your next measurement is the same as the last one (there are no 'dynamics') you could also interpolate your missing values and use a linear filter.


Two different definitions of cross-correlation

Start with:

$$ R_{XY}(\tau) = \int_{-\infty}^{+\infty} x(t) y(t+ \tau) dt $$

Then it follows:

$$ R_{YX}(\tau) = \int_{-\infty}^{+\infty} y(t) x(t+ \tau) dt $$

Let $s = t+ \tau $:

$$ R_{YX}(\tau) = \int_{-\infty}^{+\infty} y(s -\tau ) x(s ) ds $$

Reverse the product and substitute $t$ for $s$:

$$ R_{YX}(\tau) = \int_{-\infty}^{+\infty}  x(t ) y(t -\tau ) dt $$

Now it is pretty clear that

$$ R_{XY}(\tau) = R_{YX}(-\tau) $$

The question of odd or even is irrelevant because they are not the same function.

Note:  Your second function is assumed to be $ R_{YX}(\tau) $, otherwise your are simply giving contradictory definitions to the same function.


What are True Positive and False Positive with reference to image processing

A ,,true'' positive would be an image that has a property (in the context of face detection maybe: ,,contains a face'') and that is recognized by a program as such. A ,,false'' positive then is one that does not have the property but is recognized anyway.


Using reference objects to estimate the point spread function?

As you probably know, there is a developed literature on deblurring. Regarding the evaluation of the PSF, you may find these useful:

Understanding and evaluating blind deconvolution algorithms
Modeling the Performance of Image Restoration From Motion Blur

If the known object $X_0$ is going to be shot head on in subsequent frames $Y_i$, you can find the blur kernel $K$ by maximizing some image similarity metric $I$ for basic (say, procrustes) transformations $T$ of $X_0$:

$K_i = \underset{K, T}{\arg \max} \; I \left( K \ast T X_0 ; Y_i \right)$

In a more sophisticated formulation, you could enforce temporal smoothness priors on the $K_i$'s and $T$'s. I would also suggest using the last frame's optimal estimates as initial values for the next frame.

If $X_0$ appears at an angle in $Y_i$ (since it is moving, after all) I conjecture that you will have a hard time finding a good match (that is, $X_0$ will not be of much use). Obviously, there is some information to be leveraged, even when shot at an angle, but I suspect the effort will not be worth it. This leads me to wonder what sort of a scenario you would be able to use this technique in. You know what they say: there's one way to find out.


Why Frequency Domain conversion is Important in Digital Image Processing?

Not sure if frequency domain is necessarily better for image processing but frequency domain does give you an advantage for certain types of problems because it is more quantitive.

First you need to understand what frequency domain of an image means as it is intuitively difficult to look at frequency domain of an image and make sense out of it. Topic is beautifully handled here : What does frequency domain denote in case of images?

Basically frequency domain represents the rate of change of spatial pixels and hence gives an advantage when the problem you are dealing with relates to the rate of change of pixels which is very important in image processing. For example :high frequency in the frequency domain represents rapidly or sharply changing pixels such as boundaries or edges in an image. A high pass filter can be extremely helpful in identifying or removing these edges easily but the same problem is much more difficult in spatial domain (x-y domain).
Similarly a simple low pass filter can be used to get a smoother image. As you can guess images can be easily made sharper or smoother by manipulating different parts of frequency domain.

Some points on why it is so powerful :

Frequency domain gives you control over the whole images, where you can enhance(eg edges) and suppress (eg smooth shadow) different characteristics of the image very easily.
Frequency domain has a established suit of processes and tools that be borrowed directly from signal processing in other domains.
Some tools used for even image recognition such as correlation , convolution etc are much simpler and computationally cheaper in frequency domain.


What Are the Differences in Four Different Types of Third Order Image Moments (Skewness)?

It it based on the definition of Central Moments on 2D grid given in the Wikipedia Page.

$$ {\mu}_{pq} = \int\limits_{-\infty}^{\infty} \int\limits_{-\infty}^{\infty} (x - \bar{x})^p(y - \bar{y})^q f(x,y) \, dx \, dy $$

Since we're dealing with 2 random variables, a multiplication which the powers sum to 3 can be achieved in 4 different methods.

It's up to you to decide how to calculate the third moment.

Usually I'd use the Rotation Invariant Moments.
You can find the definition on the same page.
Enforcing the Rotation (And Scale) Invariance yields single definition.


Channel Capacity - is this only for discrete/digital signals?

The definition of channel capacity can be applied to either digital or analog cases. The meaning depends on how you calculate it. The definition is:

$$C=\max_{p_X(x)}\ I(X;Y)$$

In the digital world, $p_X(x)$ is a mass probability function and the mutual information is calculated as $H(Y)-H(Y|X)$, where $H(\cdot)$ denotes the entropy.

In the analog case, then $p_X(x)$ is a probability density function and the mutual information is calculated as $h(Y)-h(Y|X)$, where $h(\cdot)$ denotes the differential entropy.

Regarding your other question, channel capacity and bandwidth are not the same thing. Take for example a Gaussian channel, where $$C=B\log_2(1+\mathrm{SNR})$$

You can see that the capacity depends on the bandwidth, but they are clearly not the same.


Employing MIMO under the two-ray propagation model

The Channel Impulse Response (CIR) for a beamforming vector $\mathbf{w} \in \mathbb{C}^{N_T \times 1}$ (which corresponds to your spatial streams) is the following

$$\mathbf{c}(n) = \sum\limits_{k=1}^q \alpha_k \mathbf{a}_{R,k}\mathbf{a}_{T,k}^\top \mathbf{w} \delta(n - \tau_k) \tag{1}$$
where $q$ is the number of paths (is $2$ due to the two-ray propagation model), $\alpha_k$ is the complex gain of the $k^{th}$ path, $ \mathbf{a}_{R,k}$ and $\mathbf{a}_{T,k}$ are the receive and transmit steering vectors of the $k^{th}$ path (depending on the antenna placement and the Angles-of-Arrival and Angles-of-Departures of the path) and finally $\tau_k$ is the propagation delay of the $k^{th}$ path.

I can see that in your case $\mathbf{a}_{T,k} \in \mathbb{C}^{2 \times 1},\mathbf{a}_{R,k} \in \mathbb{C}^{2 \times 1},\mathbf{w} \in \mathbb{C}^{2 \times 1}$. Note that equation $1$ could be written as
$$\mathbf{c}(n) = \Big(\sum\limits_{k=1}^q \alpha_k \mathbf{a}_{R,k}\mathbf{a}_{T,k}^\top  \delta(n - \tau_k) \Big) \mathbf{w}$$
So in order to "equalize" to get $\mathbf{w}$, you need to know the matrix $\mathbf{H} = \sum\limits_{k=1}^q \alpha_k \mathbf{a}_{R,k}\mathbf{a}_{T,k}^\top  \delta(n - \tau_k) $.


Analysis Received Signal at receiver side

Constructive interference just increases the gain of the signal at the receiver, which is designed (using automatic gain control) to adjust the signal level over a very wide range to that desired for reception. So the issue the OP refers to is of no real consequence as long as the received signal is in the (relatively wide) range of the receiver between maximum and minimum signal levels allowed.
Multipath distortion can come in different forms based on how fast the multipath is changing (such as in a mobile environment) resulting in "fast" or "slow" fading, and the relative distance between the paths compared the symbol rate, resulting in "flat" or "frequency selective" fading.  These are the real issues with multipath distortion which are corrected with equalizers in the receiver.
Please see this post which expands on these different fading models:
Rayleigh fading with frequency selective fading channel


What is the default carrier frequency of matlab's pskmod?

The function pskmod.m returns a vector of complex baseband symbols. Each element of that vector has the form $y_k=e^{j(2\pi m(k)/M+\phi) }$, where $M$ is the alphabet size, $m(k)$ is an integer between $0$ and $M-1$ which depends on the message symbol at time $k$, and $\phi$ is some initial phase (which defaults to $0$).

There is no carrier involved, and for simulation purposes it's usually also not necessary to use one because a digital communication system can be simulated by using the equivalent baseband system.


Can we see the Fourier transform as a filtering operation?

Hint:

The DFT matrix only contains powers of a complex root of unit. Every row or column holds increasing powers of a common power ($\omega^{mn}=(\omega^m)^n$); hence the real and imaginary parts follow (co)sinusoids of increasing frequencies.

https://www.dsprelated.com/freebooks/mdft/DFT_Sinusoids.html

Every component of the DFT of a signal is a convolution of the signal with those sinusoids.


Error probability for orthogonal signaling with unequal symbol probabilities

I haven't checked the details fully but the OP's revised solution is correct. Once upon a time, I had posted a to a class webpage from 1998 where I had some lecture notes giving the answer
but the link is now defunct because my University seems to have deleted that webpage from its servers.  Anyway, the answer that I came with to the OP's question is
$$P_e = \pi_0 Q\left(\mathsf{SNR} -(2\cdot\mathsf{SNR})^{-1}\ln\left(\frac{\pi_1}{\pi_0}\right)\right) + \pi_1 Q\left(\mathsf{SNR} +(2\cdot\mathsf{SNR})^{-1}\ln\left(\frac{\pi_1}{\pi_0}\right)\right) \tag{1}$$
where $\mathsf{SNR} = \dfrac{\Vert s_0-s_1\Vert}{\sqrt{2N_0}} = \sqrt{\dfrac{E_b}{N_0}}$ in this case. Note that when $\pi_1 > \pi_0$, $\ln\left(\frac{\pi_1}{\pi_0}\right) > 0$, and so $$Q\left(\mathsf{SNR} +(2\cdot\mathsf{SNR})^{-1}\ln\left(\frac{\pi_1}{\pi_0}\right)\right) < Q\left(\mathsf{SNR} -(2\cdot\mathsf{SNR})^{-1}\ln\left(\frac{\pi_1}{\pi_0}\right)\right),$$
that is, in the weighted sum of two error probabilities in $(1)$, the larger weight $\pi_1$ weights the smaller error probability, which is exactly how it should be.


Processing for vibration analysis can be quite involved, depending on what type of machinery you're testing, the kind of faults you're looking for, the technique(s) you're using, and the test equipment you have available.  Without more specific information, I presume you're doing something like that described in the 'Envelope Detection' section of:

http://www.bksv.com/doc/bo0501.pdf

Since your spectrum is not clear, there could be any number of things wrong, such as: 1) improper placement of accelerometer, 2) improper synchronization, 3) improper analysis bandwidth, etc., etc., etc.  A lot of things can go wrong, and it's not an area for amateurs. No company wants their million dollar engine or generator to burn out or blow up because some technician didn't detect an impending failure.

There is an extensive field of literature, and a host of companies that train people to do vibration analysis (e.g.:

http://www.mobiusinstitute.com/site2/landing.asp?LinkID=106
http://www.mobiusinstitute.com/site2/detail.asp?LinkID=53

A brief bit of googling can easily lead to hundreds of thousands of hits, some of which may be quite specific to your machinery, measurement set-up, etc.


The article Handling Spectral Inversion in Baseband Processing that you refer to in your question is not about simple inversion of the frequency axis, but it is about inversion of the frequency axis and conjugation in the frequency domain. This is necessary because due to different mixing conventions in several conversion stages, the upper and lower side-bands can be exchanged, and the two side-bands exhibit conjugate symmetry, because the transmitted signal is real-valued. So one side-band can be obtained from the other by inverting the frequency axis and by additional complex conjugation (in the frequency domain).

This process of inversion and conjugation in the frequency domain corresponds to complex conjugation of the complex baseband signal in the time domain, which is equivalent to simply inverting the $Q$ component. Since a phase shift of $180$ degrees is irrelevant, one can equivalently invert the $I$ component. A phase shift of $90$ degrees corresponds to a multiplication with $j$ in the time domain, and if this phase shift can be tolerated, you get the additional option of swapping the $I$ and $Q$ channels.

If you want pure inversion of the frequency axis without conjugation, then you indeed need to invert the time axis of the corresponding time domain signal (as pointed out in a comment by @curiousStudent). Luckily this is usually not needed in practice because this can't be done easily in a real-time system.

As a final note, all three options in your question do flip the magnitudes of the spectra around DC, it is just the phases that behave differently in all three cases.


Uncertainty principle - Duration bandwidth principle

An important  theorem, known as Weyl's, 1931, is:
if  function $s(t)$ and  related functions $ts(t)$, $s'(t)$ are in $L^2$ (square integrable) with the related $\|\cdot\|$ $L_2$ norm symbol then:
$$ \| s(t) \|^2 \leq 2\| ts(t) \| \| s'(t) \|$$
Equality is attained when  $s(t)$ is a modulated Gaussian/Gabor elementary function defined as:
$$ s'(t) / s(t)  \propto t $$
or practically as:
$$s(t) = C \exp [-\alpha(t - t_m)^2 + \imath 2 \pi \nu_m (t - t_m)]$$
found by integration by part + Cauchy–Bunyakovsky–Schwarz.
If one defines  time or frequency location, as a center of mass related to energy as:
$$ E = \int |s(t)|^2 dt = \int |S(f)|^2 df$$
and
$$\overline{t} = 1/E \int t|s(t)|^2 dt \qquad \overline{f} = 1/E \int f|S(f)|^2 df$$
and  energy dispersion as:
$$\Delta t = \sqrt{1/E \int (t - \overline{t})^2 |s(t)|^2 dt }$$
$$\Delta f = \sqrt{1/E \int (f - \overline{f})^2 |S(f)|^2 df }$$
then for  finite-energy every signal  $s(t)$,  with $\Delta t$  and $\Delta f$ finite, one gets:
$$\Delta t\Delta f \geq \frac{1}{4\pi}$$
The limit is attained for some Gaussian variations. My interpretation of the question is is more about the term 'time/frequency dispersion' than the fuzzy concept of 'time/frequency resolution'.


Sampling and the creation of multiple images at integer multiples of the sampling frequency

Below is an example of two sine waves at 11 Hz and 1 Hz, both sampled with a 10 Hz sampling clock.  The sampled locations are drawn as circles. If you only had the samples, and had no knowledge of any filtering prior to the sampling process, you would not be able to know for certain if the samples came from a 1 Hz analog signal or an 11 Hz analog signal. In fact, any signal at N fs ± 1 Hz for any integer N would produce the same result. (9 Hz, 11 Hz, 19 Hz, 21 Hz, etc).

Thus, with appropriate analog bandpass filtering prior the sampling, we can directly sample ("undersample") a bandpass signal, as long as we follow Nyquist's criteria that the sampling rate be > 2x the signal bandwidth. Do not confuse bandwidth with highest frequency content!  The sine wave shown for example has 0 bandwidth in that it occupies an infinitely thin line in the spectrum. Actual bandpass waveforms of interest would have some bandwidth based on their modulation content.

What this picture also helps clarify is the increased sensitivity to any sampling clock jitter when performing bandpass sampling.  Note specifically the slope of the red lower frequency signal as compared to that of the blue higher frequency signal in vicinity of the sample locations, and imagine small variations due to clock jitter and how these variations would transform to amplitude error.

Please see this answer for specific details related to a conceptual understanding of having multiple frequency "images": Aliasing after downsampling


Resampling and removing high frequency noise?

Sample playback

The basic idea of sample playback in musical applications is to keep track of each voice's playback position, to form an output sample by reading the source sample data at the playback position, to add a possibly time-varying playback step to the playback position, and to repeat this in a program loop until we have accumulated enough output samples for that voice. For multiple voices, their output samples are summed in a mixing buffer. If the output sampling frequency differs from the source sampling frequency, or if we want to play a different note than the source note, then playback step may be not equal to 1 (one source sample per one output sample), using your example scenario:

                source sampling frequency * desired note frequency
playback step = --------------------------------------------------
                output sampling frequency * source note frequency

                44100 Hz * 146.83 Hz
              = -------------------- = 0.56121239919
                44100 Hz * 261.63 Hz

If the playback step is not an integer, then the playback position will sometimes also be not an integer and we need to interpolate between the samples. It is also possible to do vibrato and tone portamento by varying the desired note frequency with time and consequently also varying the playback step, which will inevitably lead to a mostly non-integer playback position and the need for arbitrary sample rate conversion.

Interpolation, spectral images and aliasing

In this context, sampling is best understood (see Andreas Franck (2012) Efficient Algorithms for Arbitrary Sample Rate Conversion with Application to Wave Field Synthesis, section 3.4.1 Sample Rate Conversion as an Analog Resampling Process) as multiplication of a band-limited continuous-time audio signal by a Dirac comb and storing the integral over each scaled Dirac pulse in the resulting impulse train as the discrete-time sample data. The Fourier transform of the impulse train is periodic with a period of $2\pi$ in angular frequency. The chosen interpolation method is characterized by its continuous-time impulse response, and interpolation can be seen as convolution (filtering) of the impulse train by the continuous-time impulse response. The impulse response of a reasonable interpolation method is approximately low-pass, meaning that the convolution attenuates the spectral images (the duplicate spectra centered at multiples of $2\pi$). Then sampling of the approximately low-pass signal, at the playback positions encountered, will hopefully not lead to significant aliasing of the spectral images into the audio band.

If the bandwidth of the audio signal is half its sampling frequency, then a playback step > 1 will lead to aliasing of not only the spectral images, but also of some of the base-band audio frequencies. It helps to have an intermediate output sampling frequency where there is headroom for aliasing into a band above the audio frequencies that can be filtered away at an additional sample rate conversion to the final output sampling frequency. It is also possible to have multiple versions of the sample data that have been pre-filtered to reduce their bandwidth, to avoid aliasing when playback step > 1, similar to MIP mapping in computer graphics, see Laurent de Soras (2005) The Quest For The Perfect Resampler.

A time-varying playback step may further increase the continuous-time bandwidth, but usually the variation is slow enough that the playback step can be considered locally constant when it comes to bandwidth considerations.

Looping

Traditionally samplers have a number of ways of playing back sample data:

no loop (one-shot),
forward loop, with integer loop start and integer loop length (in sampling periods equal to 1 / sampling frequency, or the time difference between successive samples), and
ping-pong loop (forward-backward-...), with integer loop start and integer loop length (in one direction).

The forward loop requires a change in the program loop. If the playback position goes to the loop end (the first sample not included in the loop) or beyond it, then the loop length is subtracted from it. This works assuming that the playback step is smaller than the loop length. Ping-pong loop requires a bit more program logic, or the ping-pong loop can be unrolled into a forward loop. Upon note release, it is also possible to let the playback position escape the loop and to play the source sample data until the end.

Considering forward loops, when playing back the source sample data at its sampling frequency, what the loop length being an integer number of sampling periods means is that the period of the output waveform will be quantized to an integer multiple of the source sampling period. So it typically won't be possible to produce exactly the correct frequency, unless the source note frequency is a submultiple of the source sampling frequency. With your example, a loop length of 1000 sampling periods gives 261.63 Hz * 1000 / 44100 Hz = 5.932653 periods of the sine wave, so the last period will be cut short and give you a wide-band buzzing noise no matter what interpolation method you use.

Because of this inflexibility, we can bring into question whether the loop length should be an integer number of sampling periods. The benefit of a floating point (or a fixed point) loop length would be that we get a more correct note frequency, but the danger is that we introduce a discontinuity in the interpolated signal. With an integer loop length this discontinuity can be avoided by doing a short cross-fade of the loop start and end so that they have a few identical samples around them, as many as needed by the interpolation method. This way the interpolated curve will be exactly the same at the beginning and at the end of the loop. An alternative to cross-fading is to duplicate a few samples from the beginning of the loop at the loop end and in the following samples and to move the the loop slightly forward in time so that the interpolation method sees the same samples at the start and at the end of the loop. In case of a floating point loop start and loop length, they can be adjusted until no discontinuity can be heard, although this ties high loop quality to the present choice of the interpolation method.


Conceptual Question on equalization technique in rayleigh fading channel based on a paper

The model they have used is general and you can apply it to QAM as well.
In wireless communication systems, channel and noise are two different impairments affecting the overall transmission. Here as everywhere else, the noise is AWGN (additive white Gaussian; if you want to know what these terms mean, see this article). And the channel is a linear filter with taps whose magnitudes are modeled as Rayleigh random variables.
Yes, you can use the expression in that paper. The authors didn't give the expression because they might not have known it and simply generated it from Monte Carlo simulations.


Can a linear reconstruction in compressive sensing perform well?

This is a good idea, I was pondering upon awhile ago. I drop some thoughts.

A Linear(no activations) MLP with a single hidden layer performs the
same or sometimes better than a Multi-layer model with ReLU
activations. What might be the cause of this?

The dimension reduction transform, i.e. taking CS measurements is a linear operation,  a measurement is weighted linear combination of input "sparse" vector. Are you reconstructing the same image that is used for training the network? The results could be different if the training image and test image are drastically different.

In the literature, to guarantee construction X has to be sparse in
some domain. But i tried removing all transformations from my model
and just worked in spatial domain and the model performs exactly the
same. shouldn't it struggle to reconstruct the signal?

There is actually no need to add that. Transformation is like any other function that network can "learn". It does not matter if you map measurement to "sampling" domain or the transform domain.
Maybe you might be interested in seeing Baraniuk's work on similar thing as well:
https://patentimages.storage.googleapis.com/7d/cc/57/9f8ebb657d6fb7/US10985777.pdf


Why do we observe band when recording CRT monitors or Flouroscent lamp via video recorder?

This effect is most obvious in light generators that brighten and dim cyclicly over time, as in an AC current powered photon generator (bulb), or the phosphors on an old CRT periodically hit by a refresh beam.

In some ways this is similar to audio beating between two different but close audio frequency signals or pitched notes.  If the frequencies are slightly different (including the fundamental of one to a strong harmonic of another), then the phase will drift in and out over time.  In audio, this creates changing constructive or destructive interference.  In video recording this might capture the brightest portion of the beam or lamp when in phase, or not capture it when out of phase, or a capture over time while the phase is continually changing in between, resulting in rolling of the brightest band.

Note that the NTSC frame refresh rate is 29.97 Hz, which will beat against half an AC line frequency of 30 Hz.


Image Shadow Removal Using OpenCV and Python

fgbg = cv2.createBackgroundSubtractorMOG2(128,cv2.THRESH_BINARY,1)
masked_image = fgbg.apply(image)

in masked_image shadow will be grey color(pixel value= 127)
just replace 127 to 0, to convert grey pixel to black.

masked_image[masked_image==127]=0


which programming language is the best for image processing?

Most image processing code in image signal processors
(ISP) in the industry are written in C++ and assembly, which I would say have more syntax and language gibberish than MATLAB.

So if you are considering to write code for educational purposes than MATLAB and python have good libraries and easier to code.

Else if you are writing code to be run on real HW, C++ and assembly mostly.

Open CV is a good cross platform library with C++ and other languages supported as well


Here is a detour over the frequency domain. We'll use index $k$ for frequency and index $n$ for time. Capital letters for Frequency Domain variables

The Fourier Transform (FT) of an impulse is $R(k)=1$
FT of the autocorrelation is the magnitude squared of the FT of the original signal, i.e $R(k) = |S(k)|^2$
Combining 1 and 2 we get $|S(k)| = 1 $,i.e. $S(k)$ must be an allpass filter
An allpass filter has zeros that are inverse of the poles
Any finite length sequence can be viewed as the impulse response of an FIR filter
An FIR filter has all the poles at $z=0$
An FIR allpass filter can therefore have only zeros at $z = \infty$
Zeros at infinity correspond to pure integer delays.
Pure delays have only one non-zero sample

And that's basically it. This says if $s[n]$ is a finite length sequence, the only way to get a unit impulse as auto correlation is to have only one non-zero sample. Any sample will work though, it doesn't have to be at $n=0$. For a length of 3 you get 4 solutions: $[1,0,0,0],[0,1,0,0],[0,0,1,0],[0,0,0,1]$


4th order high-pass filter on a DSP: standard or biquads?

The two solutions in a floating point implementation are assumed to be identical, with the two BiQuads being a factored version of the standard difference equation. The BiQuad is the better way to go for fixed point as you isolate two 2nd order systems and in doing so will be easier to keep stable under variations due to the quantization involved.

For more details on this, see the responses to this post:

How does cascading biquad sections for higher order filters work?

and

https://en.wikipedia.org/wiki/Digital_biquad_filter

Note too for those less familiar: cascading two filters is the same mathematically as convolving their coefficients; multiplying two polynomials is done by convolving their coefficients. The transfer function for filters is described by polynomials so it is all the same thing; a 4th order filter can be factored into two 2nd order filters.


Linear approximation of a system described by a logarithm - how?

Functions like $\sqrt{\cdot}$ or $\log{(1+\cdot)}$ may emulate nonlinear system behavior like saturation. They are although used as companding methods, such as the $\mu$-law (or the A-law):
$$ x\to \mathrm{sgn}x\frac{\log{(1+\mu|x|})}{\log{(1+\mu)}}\,. $$
This can be used prior to quantization for signals with high-dynamics, and allows one to use uniform quantization and maintain a similar amount of relative errors for weak as well as strong signals (see for instance Digital filtering using logarithmic arithmetic (I glad to discover this work by Nick Kingsbury).
For telephony, people developed segment ("piecewise linear", but actually piecewise affine to be correct) companding versions of such laws, for instance H. Kaneko, "A Unified Formulation of Segment Companding Laws and Synthesis of Codecs and Digital Compandors". Basically, $\log(x+y)$ can be approximated by some affine law in its proper interval. I am aware of such approximations  for square roots: if $a\ge b$, then for instance with about 4% precision:
$$\sqrt{a^2+b^2} \sim 0.96a+0.4b  $$
yet I have to check for logarithms. In the same framework, as already said by @ben, if $a$ is a fixed quantity:

if $a>b$, then $\log(a+b)\sim b/a+\log a$

which is  an affine approximation.


Noise is random, but like most random phenomena, it follows a certain pattern. Different patterns are given different names.

Consider rolling a die. This is clearly random. Roll the die 1000 times, keeping track of each result. Then, calculate the histogram of the result; you'll find that you got each of 1, 2, 3, 4, 5 and 6 approximately the same number of times. This pattern is called "uniform", and throwing a die can be modeled by a "uniform random variable".

The same experiment can be repeated with thermal noise. Heat up a resistor, amplify the resulting voltage and measure its instantaneous power multiple times. Then calculate the histogram. This time, you won't find a uniform histogram; it will be shaped like a bell curve, with values near zero more common than values far away from zero. This kind of histogram is called Gaussian, after K. F. Gauss.

Gaussian random phenomena are very common in nature. It turns out that whenever the random thing you observe is the aggregate of many, independent random events, the overall random variable is Gaussian (this is technically called the central limit theorem). In the case of thermal noise, you're measuring the aggregate effect of millions or billions of randomly oscillating electrons, excited by the heat.

There's an easier way to create Gaussian randomness at home (or simulated in a computer): take many dice, say 100, throw them all many times, and keep track of the total sum of each throw. If you find the histogram again, you'll see it follows a bell curve. The reason is intuitively easy to grasp: with 100 dice, you're very unlikely to get a total of 100 (all dice would have to land in 1), but it's very easy to get a number around 350, because many different combinations add up to such a number.

To summarize, there are many different kinds of noise that can affect a signal or an image, each with different statistical properties. Gaussian noise is a particularly important kind of noise because it is very prevalent. It is characterized by a histogram (more precisely, a probability density function) that follows the bell curve (or Gaussian function). As you study it more, you'll find that it also has several other important statistical properties.

See also:

why Gaussian noise is usually used?
what's gaussian noise look like?

and many other related questions on this website.


What do I with the sampling period when I inverse $\mathcal Z$-transform?

There should probably be a $z^{-2}$ term with the $0.3$. It looks like a typo.

Once you've transformed to the $z$ domain (or discrete-time), and are not interested in transforming back to the continuous-time domain, there is no real need to carry the $T_s$. Just refer to $e[k], y[k]$, etc.


In very general terms, pre-distortion means altering a signal's shape, so that after it goes through the channel, the output looks like the original signal. So you basically alter the signal by the inverse of the distortion.

This is a separate issue from coding.


Fourier Transform for frequency analysis

That's to be expected. If the input of your FF transform is 30 (real) points, you also get 30 real amplitudes out, for 30 different frequency bins. Each bin is (44100/2)/30 Hz wide, or about 700 Hz. That is quite wide. Add to that the effect of spectral leakage, which may quite well extend for 5 bins in either direction. 10 out of 30 bins will contain the ground frequency. Any higher harmonics will add to that.

Normally, people use much longer FFT's. 256 is quite a common choice, and 1024 isn't unheard of either.


Relationship between wavelet shape and wavelet filters

I will start with an averaging filter analogy. The simplest (non trivial) and cheapest way to average a discrete signal $x[n]$ is to  compute the mean of two adjacent samples, and remplace the two samples by this average.  This amounts to compute the discrete system:

$$\frac{x[2n]+x[2n+1]}{2}\,.$$

This can be recast as the convolution $$y[n] = x[n]\ast \frac{ [1\; 1]}{2}$$

You might remember that $\frac{ [1\; 1]}{2}\ast \frac{ [1\; 1]}{2} =  \frac{ [1\;2\; 1]}{4}$. And that  $\frac{ [1\;2\; 1]}{4}\ast \frac{ [1\; 1]}{2} =  \frac{ [1\;3\;3\; 1]}{8}$

This should remind you of the Pascal or binomial triangle, that evolve to a Gaussian shape:

Here, starting with a crude, and a simple finite-impulse response two-point averaging filters,  via iterations, you can end up with an IIR, not recursive, infinite-support Gaussian filter. This example is more a metaphor of how discrete filters generate a continuous function.

To be  able to add downsampling within the filters, without losing information, contraints the data smoothing picture. In the discrete context, this requires to have a downsampled smoothed version, and a difference  version that records when the data is not smooth enough to look like  its smooth version. This is the root to most practical discrete wavelet transforms. The Multi Resolution Analysis relates an (yet unknown)  scaling function $\phi$ at two-scales, via a linear combination, with weights corresponding to the filter $H$ coefficients. This is explained for instance in Signal Processing: Fourier and Wavelet Representations, 6.1. Introduction, Scaling Function and its Properties, page 191. Finally, the spectrum of the scaling function is given by:
$$\Phi^{(\infty)}(\omega)= \prod_{k=1}^\infty \frac {1} {\sqrt 2} H\left( \frac {\omega} {2^k}\right) \Phi^{(\infty)}(0)$$
where iterated half-cut-off frequencies are apparent. The take-away message is: the spectrum of the scaling function is related to the that of the low-pass filter. And in the same manner, the spectrum of the wavelet function is related to the that of the high-pass filter. Some more pointers are given in:

Why is a wavelet transform implemented as a filter bank?
Discrete Wavelet Transform (DWT) Filter Bank

The limit is that, if you start from continuous wavelets, very few of them can be faithfully implemented as discrete schemes as filters, and those filters, when they exist, don't look like wavelets. Because they don't have too. Their resemblance with wavelets only make sense when they are repeated over and over at different scales, across many dyadic sub-sampling.

And most discrete wavelets (like Daubechies') don't have an analytic formula. But if you look at the bottom line, from afar, the filters and the wavelets look similar in terms of envelope (red line below) and zero-crossings. So the global shape is not so different, as shown below with appropriate scaling of the wavelets and  filters:

This similarity seems common for smooth-enough wavelets. Honestly, I have not investigated this into more details, because the actual ressemblance  of filters and wavelets is  not so important, their properties are way more of interest.


Calculation of autocorrelation - does calculation "loop"?

You didn't specify for which values of $n$ your signal is not zero, so I'll assume that happens from $0$ to $4$. You have a finite sequence $x(n)$ such that

$$x(n) = 0 \qquad \forall n\in\mathbb{Z}:n<0 \ \mathrm{or} \ n>4$$

This means that $x(5)$, $x(6)$, etc. are equal to $0$. The same happens for $x(-1)$, $x(-2)$, and so on. Thus, products involving these quantities will be zero as well.


Is phase and amplitude information necessarily lost when undersampling?

From an RF perspective... since you specified just the one sine wave then yes, Nyquist holds because you are operating at baseband. Any sampling rate less than 2B will result in aliasing, because you are undersampling. In this case your bandwidth is the frequency of the sine wave, since it is the highest frequency you want to recover.

However if you have a low frequency signal modulated onto a high frequency carrier signal, you can get away with undersampling because the carrier has no information. Can use spectral folding to recover the baseband signal and discard the carrier.

Now about phase recovery, there are multiple methods to accomplish it. One is to use complex signals, where you create an I/Q signal by complex mixing. With an I/Q signal you can recover instantaneous phase because you have a reference, the other copy of the signal.


Basic dithering without noise shaping

Basic dithered quantization without noise shaping works like this:

Figure 1. Basic dithered quantization system diagram. Noise is zero-mean triangular dither with a maximum absolute value of 1. Rounding is to nearest integer. Residual error is the difference between output and input, and is calculated for analysis only.

The triangular dither increases the variance of the resulting residual error by a factor of 3 (from $\frac{1}{12}$ to $\frac{1}{4}$) but decouples the mean and variance of the net quantization error from the value of the input signal.  That means that the net error signal is uncorrelated with the input but higher moments are not decoupled, so it's not truly completely independent random error, but no one has determined that people can hear any dependency of the higher moments in the net error signal on the input signal in an audio application.

With independent additive residual error we would have a simpler model of the system:

Figure 2. Approximation of basic dithered quantization. Residual error is white noise.

In the approximate model the output is simply input plus independent white noise residual error.

Dithering with noise shaping

I can't read Mathematica very well so instead of your system I'll analyze the system from Lipshitz et al. "Minimally audible noise shaping" J. Audio Eng. Soc., Vol.39, No.11, November 1991:

Figure 3. Lipshitz et al. 1991 system diagram (adapted from their Fig. 1). Filter (italicized in the text) includes in it a one sample delay so that it can be used as an error feedback filter. Noise is triangular dither.

If the residual error is independent from current and past values of signal A, we have a simpler system:

Figure 4. An approximate model of the Lipshitz et al. 1991 system. Filter is the same as in Fig. 3 and includes in it a one sample delay. It is no longer used as a feedback filter. Residual error is white noise.

In this answer I will work with the more easily analyzed approximate model (Fig. 4). In the original Lipshitz et al. 1991 system, Filter has a generic infinite impulse response (IIR) filter form that covers both IIR and finite impulse response (FIR) filters. In the following we will assume that Filter is a FIR filter, as I believe based on my experiments with your coefficients that that is what you have in your system. The transfer function of Filter is:

$$H_\mathit{Filter}(z) = -b_1z^{-1} - b_2z^{-2} - b_3z^{-3} - \ldots$$

The factor $z^{-1}$ represents a one-sample delay. In the approximate model there is also a direct summation path to output from residual error. This gets summed with the negated output of Filter, forming the full noise shaping filter transfer function:

$$H(z) = 1 - H_\mathit{Filter}(z) = 1 + b_1z^{-1} + b_2z^{-2} + b_3z^{-3} + \ldots.$$

To go from your Filter coefficients, which you list in order $\ldots, -b_3, -b_2, -b_1$, to the full noise shaping filter transfer function polynomial coefficients $1, b_1, b_2, b_3, \ldots$, the sign of the coefficients is changed to account for the negation of Filter output in the system diagram, and the coefficient $b_0 = 1$ is appended to the end (by horzcat in the Octave script below), and finally the list is reversed (by flip):

pkg load signal
b = [-0.16, 0.51, -0.74, 0.52, -0.04, -0.25, 0.22, -0.11, -0.02, 0.31, -0.56, 0.45, -0.13, 0.04, -0.14, 0.12, -0.06, 0.19, -0.22, -0.15, 0.4, 0.01, -0.41, -0.1, 0.84, -0.42, -0.81, 0.91, 0.75, -2.37, 2.29];
c = flip(horzcat(-b, 1));
freqz(c)
zplane(c)

The script plots the magnitude frequency response and the zero locations of the full noise shaping filter:

Figure 5. Magnitude frequency response of the full noise-shaping filter.

Figure 6. Z-plane plot of poles ($\times$) and zeros ($\circ$) of the filter. All the zeros are inside the unit circle, so the full noise-shaping filter is minimum-phase.

I think the problem of finding the filter coefficients can be reformulated as the problem of designing a minimum-phase filter with a leading coefficient of 1. If there are inherent limitations to the frequency response of such filters, then these limitations are transferred to equivalent limitations in noise shaping that uses such filters.

Conversion from all-pole design to minimum-phase FIR

A procedure for design of different but in many ways equivalent filters are described in Stojanović et al., "All-Pole Recursive Digital Filters Design Based on Ultraspherical Polynomials", Radioengineering, vol 23, no 3, September 2014. They calculate denominator coefficients of the transfer function of an IIR all-pole low-pass filter. Those always have a leading denominator coefficient of 1 and have all poles inside the unit circle, a requirement of stable IIR filters. If those coefficients are used as the coefficients of the minimum-phase FIR noise shaping filter, they will give an inverted high-pass frequency response compared to the low-pass IIR filter (transfer function denominator coefficients becoming numerator coefficients). In your notation one set of coefficients from that article is {-0.0076120, 0.0960380, -0.5454670, 1.8298040, -3.9884220, 5.8308660, -5.6495140, 3.3816780}, which could be tested for the noise shaping application although it isn't exactly to the specification:

Figure 7. Magnitude frequency response of the FIR filter using coefficients from Stojanović et al. 2014.

Figure 8. Pole-zero plot of the FIR filter using coefficients from Stojanović et al. 2014.

The all-pole transfer function is:

$$H(z) = \frac{1}{1 + a_1z^{-1} + a_2z^{-2} + a_3z^{-3} + \ldots}$$

So, you can design a stable all-pole IIR low-pass filter and use the $a$ coefficients as $b$ coefficients to get a minimum-phase high-pass FIR filter with a leading coefficient of 1.

To design an all-pole filter and to convert that to a minimum-phase FIR filter, you will not be able to use IIR filter design methods that start from an analog prototype filter and map the poles and zeros into the digital domain using bilinear transform. That includes cheby1, cheby2, and ellip in Octave and Python's SciPy. These methods will give zeros away from z-plane origin so the filter will not be of the required all-pole type.

Answer to the theoretical question

If you don't care how much noise there will be at frequencies above quarter of the sampling frequency, then Lipshitz et al. 1991 addresses your question directly:

  For such weighting functions, which go to zero over part of the band, there is no theoretical limit to the weighted noise-power reduction obtainable from the circuit of Fig. 1. This would be the case if, for example, one assumes that the ear has zero sensitivity between, say, 20 kHz and the Nyquist Frequency, and chooses the weighting function to reflect this fact.

Their Fig 1. shows a noise shaper with a generic IIR filter structure with both poles and zeros, so different to the FIR structure that you have at the moment, but what they say applies also to that, because a FIR filter impulse response can be made arbitrarily close to the impulse response of any given stable IIR filter.

Octave script for filter design

Here is an Octave script for coefficient calculation by another method that I think is equivalent to the Stojanovici et al. 2014 method parameterized as $\nu=0$ with the right choice of my dip parameter.

pkg load signal
N = 14; #number of taps including leading tap with coefficient 1
att = 97.5; #dB attenuation of Dolph-Chebyshev window, must be positive
dip = 2; #spectrum lift-up multiplier, must be above 1
c = chebwin(N, att);
c = conv(c, c);
c /= sum(c);
c(N) += dip*10^(-att/10);
r = roots(c);
j = (abs(r(:)) <= 1);
r = r(j);
c = real(poly(r));
c .*= (-1).^(0:(N-1)); #if this complains, then root finding has probably failed
freqz(c)
zplane(c)
printf('%f, ', flip(-c(2:end))), printf('\n'); #tobalt's format

It starts with a Dolph-Chebyshev window as the coefficients, convolves it with itself to double the transfer function zeros, adds to the middle tap a number that "lifts up" the frequency response (considering the middle tap as being at zero time) so that it is everywhere positive, finds the zeros, removes zeros that are outside the unit circle, converts the zeros back to coefficients (the leading coefficient from poly is always 1), and flips the sign of every second coefficient to make the filter high-pass. Results from (an older but almost equivalent version of) the script look promising:

Figure 9. Magnitude frequency response of the filter from (an older but almost equivalent version of) the above script.

Figure 10. Pole-zero plot of the filter from (an older but almost equivalent version of) the above script.

The coefficients from (an older but almost equivalent version of) the above script in your notation: {0.357662, -2.588396, 9.931419, -26.205448, 52.450624, -83.531276, 108.508775, -116.272581, 102.875781, -74.473956, 43.140431, -19.131434, 5.923468}. The numbers are large which could lead to numerical problems.

Octave implementation of noise shaping

Finally, I did my own implementation of noise shaping in Octave and don't get problems like you did. Based on our discussion in comments, I think the limitation in your implementation was that the noise spectrum was evaluated using a rectangular window a.k.a. "no windowing", which spilled the high-frequency spectrum to the low frequencies.

pkg load signal
N = length(c);
M = 16384; #signal length
input = zeros(M, 1);#sin(0.01*(1:M))*127;
er = zeros(M, 1);
output = zeros(M, 1);
for i = 1:M
  A = input(i) + er(i);
  output(i) = round(A + rand() - rand());
  for j = 2:N
    if (i + j - 1 <= M)
      er(i + j - 1) += (output(i) - A)*c(j);
    endif
  endfor
endfor
pwelch(output, max(nuttallwin(1024), 0), 'semilogy');

Figure 11. Quantization noise spectral analysis from the above Octave implementation of noise shaping for constant zero input signal. Horizontal axis: Normalized frequency. Black: no noise shaping (c = [1];), red: your original filter, blue: the filter from section "Octave script for filter design".

Figure 12. Time domain output from the above Octave implementation of noise shaping for constant zero input signal. Horizontal axis: sample number, vertical axis: sample value. Red: your original filter, blue: the filter from section "Octave script for filter design".

The more extreme noise shaping filter (blue) results in very large quantized output sample values even for zero input.


What's the maximum allowable amplitude modulation frequency for a given carrier frequency?

In the frequency domain, an AM modulated sinusoid looks like a carrier plus two side-bands in complex conjugate symmetry above and below the carrier.  If you permit the lower sideband to cross into the negative frequency spectrum, then there is no limit to maximum modulating frequency.  Of course that may not look like your typical AM modulated signal in the time domain.


pdf of the sum of gaussian distributions using fft Ask

The FFT algorithm expects the origin of the signal to be on the leftmost sample of the signal. You are convolving two shifted Gaussian together, yielding a Gaussian that is even more shifted. Because of the FFT imposes periodicity, when the curve is shifted past the right edge it comes back in on the left edge.

Whit an odd number the shift is exactly the full width of the signal, so you don’t notice it.

A solution would be to use a zero-mean Gaussian distribution, which you can obtain from yours by ifftshift. After performing the convolution, you can shift it back to where it was using fftshift.

np.fft.fftshift(ifft(fft(np.fft.ifftshift(pdf))**2))


What exactly is a 90 degree phase shift of a digital signal in FM demodulation appraoches?

The OP asked about a phase shift specifically but from the written details without seeing the actual demodulator implementation, I suspect he may possibly be asking how to implement the delay that is used for some simple FM demodulation approaches. I want to mention the possibility that this may actually be a delay element and not a phase shift in the sense that phase shift implies a shift of that phase at all frequencies, while a true delay has a phase shift that is proportional to frequency. It is is this property that is exploited to form simple FM demodulator structures.

For example, a very simple non-coherent digital (or analog!) FM discriminator is implemented by a delay and multiply. As introduced above, this is truly a delay resulting in a phase shift of 90 degrees only at the center frequency and NOT a phase shift of 90 degrees at all frequencies within the signal bandwidth (as would be provided by the Hilbert Transform).

As the OP commented in another answer to this question, a delay of 1.176 samples (which is indeed conveniently a true delay!) would work for this approach, the details and trade space involved will be clearer once the demodulation process of converting frequency change to amplitude change is understood.

To understand the operation, first it is useful to review how a multiplication in time between two signals results in a phase measurement:

In the case of a real signal (which I assume you may be doing), the product creates the sum and difference of the two input frequencies (and phase), and once low pass filtered, the difference signal results in a voltage that is proportional to the cosine of the phase. The complex case is also shown (and in this case either the real or the imaginary output as shown could be used, or both for non-ambiguous phase from 0 to 360 degrees).

The ideal operating point for a sensitive phase to voltage (or magnitude regardless of units) for the case of a real signal is when the two input signals are in quadrature phase; at this point the output signal is crossing through zero, but importantly, has the highest rate of change to any change of phase in the input. This is particularly true for analog demodulation, where we desire a linear phase to voltage translation to minimize harmonic distortion (so not as critical in FSK demodulation).

The next important item to understand is how a fixed delay is a "frequency to phase converter". A fixed delay passes all frequencies with equal magnitude, but the phase at the output (relative to the input) is linearly proportional to the frequency (in a fixed delay a higher frequency signal may take several cycles to transition the delay while a lower frequency signal may only shift fractions of a cycle). Therefore with regards to a signal that is changing its frequency over time (the FM modulated signal including FSK), the delay and multiply will first convert frequency variation to phase variation via the delay and then phase variation to amplitude variation via the multiplier!

It can be further determined from this understanding what delay would be optimum for two FSK frequency choices that will provide a peak maximum and minimum signal at each location (refer to the figure below which shows the complex multiplier approach but applies equally to the real signal approach with the additional low pass filter as given in the figure above.)

So for the OP’s case specifically with a 1 sample delay as the delay element:

Frequency    Phase Shift for 1 sample delay

1200 Hz:      360*1.2/8 = 54°   cos(54°) = 0.59

2200 Hz:      360*2.2/8 = 99°    cos(99°) = -0.16

This will work in the sense that it provides a bipolar output ranging from -0.16 to +0.59 out of the possible range of +/-1. So not ideal in terms of best SNR but will certainly work.

Interestingly consider the case where a 4 sample delay is used for this example:

1200 Hz:      360*1.2/8 * 4 = 216°   cos(216°) = -0.8

2200 Hz:      360*2.2/8 * 4 = 396°    cos(396°) =  +0.8

This is much closer to the ideal result for this non-coherent demodulation approach of +/-1 output and very simple to implement given the sampling rate and FSK frequencies used. As an even further simplification in a mixed digital/analog approach, a single XOR gate could be used at the multiplier, followed by an RC low pass filter (and then a Schmidt trigger as your decision slicer)! It is important in all of these implementations that the input signal is hard limited (after bandpass filtering) to remove any incidental AM modulation/noise on the signal prior to demodulation, since these demodulation approaches are equally sensitive to amplitude variation on the input.

To add to this because I like the simplicity and the similar design completely in the analog domain, below shows a commonly used non-coherent FM demodulator circuit. This is not an optimized FM demodulation solution but commonly used due to it simplicity.

Observe from the sketch below the expected magnitude and phase response of just the LC resonator portion of the circuit (the capacitor and inductor in parallel and connected to ground):

At low frequencies the inductor impedance ($j\omega L$) dominates resulting in a positive 90° phase shift (+j), and at high frequencies the capacitor impedance dominates ($1/(j\omega C)$, resulting in a -90° phase shift (1/j). The important thing to observe is what the phase shift is doing as the response passes through resonance at $f_c$. Near this location the phase change versus frequency is nearly linear with a slope defined by the Q of the resonator ($\sqrt{L/C}$). The slope, or derivative of phase with regard to frequency is time delay! So the tank is creating a delay line in proximity of this location (resonance), and conveniently at resonance the impedances cancel resulting in no loss (in the ideal circuit). However unfortunately the phase shift at this frequency, as shown in the response, is 0°. This is solved by adding another smaller capacitor (smaller to dominate in impedance) in series which serves to shift the entire phase response down an additional -90°!  So the circuit gives us the desired delay line with a 90° phase shift at the center frequency (carrier) of our modulated signal. We choose the slope (Q) of the tank circuit based on the discriminator sensitivity needed, and possible linearity specs that we need to meet (as for larger frequency excursions relative to the bandwidth of the tank circuit used the phase versus frequency will no longer be linear and there will be FM to AM translation effects due to the amplitude response of the tank).


Discrete Fourier Transform in Signal Processing - Interpreting graphs of transformed signals

(iii) seems be to correct. Spectrum of two frequencies.

The other choices can be verified,

(i) Fourier transform of $f(t=0)=1$ is 1. So it cannot be graph (iii)

(ii) Signal (iii) is sine wave and there is no spectrum associated with that.

(iv) It's Fourier transform of sine wave (iii) which is

$$f(t)=0.5\cdot \sin(2\pi Ft)$$

and it's transform is

$$\frac{\pi}{i}\left[\delta\left(\omega-2\pi F\right)-\delta\left(\omega+2\pi F\right)\right]$$

Imaginary part is not zero


Why is the lowpass-filter coming right after the local oscillator when mixing an RF?

Today I received an answer from the authors of the book. It turns out that figure 1.7a was wrong, and they sent me the updated one.

The corrected diagram will be added in this Summer's update!


audio analog input + usb data cable = sound noises

The problem was resolved by adding ground loop noise filter like this


What is the impulse response used in an interpolation filter when upsampling?

The dot in that summation is just scalar multiplication.  And yes, it's a convolution -- you're convolving the input signal by the filter.