I've never really heard of "warp" regarding ADCs or DACs, but I imagine it's because the analog-to-digital (or vise versa) mapping function, which is supposed to be linear, a straight line, is curved slightly due to variation in parts.  I don't see how this would happen with sigma-delta ADCs or DACs (unless the summer at the input with the delta feedback is non-linear).  but with an R-2R DAC, if the R's aren't all perfectly the same, the digital-to-analog mapping function will not be a straight line.  For a flash ADC, if all of the resistors in the voltage divider line are not all perfectly equal, that analog-to-digital mapping function will also not be linear and would be "warped".

If, somehow, one can measure or determine the "warpness" of the mapping function, I suppose it can be compensated in the DSP software.


Under the white Gaussian noise assumption, the optimal decoder would be to minimize each window of $7$ bits, that is take you first 7 symbols
$$s_1 = [\:\:1,−1,\:\:1,\:\:1,\:\:1,−1,\:\:1]$$
and compute two metrics
$$c_0 = \Vert s_1 - b_0 \Vert_2^2 = 2^2 + 2^2 + 2^2 + 2^2 + 2^2 = 20$$
$$c_1 = \Vert s_1 - b_1 \Vert_2^2  = 2^2 + 2^2 = 4$$
where $b_1 = [−1,−1,−1,1,1,−1,1]$ is the Barker code corresponding to $1$ and $b_0 = [1,1,1,-1,-1,1,-1]$ is the Barker code corresponding to $0$. So if $c_0 < c_1$ then the first received bits is $0$, else $1$.

Next, grab the other $7$ symbols
$$s_2 = [1,-1,1,-1,-1,1,-1]$$
and do the same thing.

PS: You could have also computed a Hamming distance as the criterion chooses the minimum number of errors.


If I want to know the power of a certain frequency in the signal (not in a range of frequencies), can we say that the power of each frequency in the signal is exactly No/2?

No, BUT: You mean the right thing, you just say it wrongly:
The Power Spectral Density is constant – a single frequency doesn't have any power; it has a "power per bandwidth"! To arrive at a power, you need to integrate the density over a non-zero mass of frequencies.
(This is kind of an important distinction to make – only infinitely long periodic signals, e.g. sine waves, have power at a single frequency; everything else has a "power distributed over frequencies".)

The total of power of additive white Gaussian noise is infinity, what does this mean? Is it reasonable to assume that the noise added to the signal have an infinite power?

Yes. Notice that you're never dealing with a truly white Gaussian noise in continuous-time systems (luckily for the universe, I might add); it's always approximately white for some bandwidth. Everything else is physically impossible – but rarely matters. Example: the thermal noise you can measure over a resistor is the classical example of white Gaussian noise in systems. However, it's not really white – the power density decreases at very high frequencies. But that's totally irrelevant to your observation – your measurement doesn't go into the terahertzes.
In time-discrete systems, things look different: for a sampled time-continuous stochastic signal (noise) to be white, it's sufficient that the original time-continuous signal had a constant PSD over a bandwidth. So, there's no physical problem in the time-continuous world. Since a discrete signal is just a sequence of numbers, there's no concern for "physicality" anyways.


Histogram of signal with a time shift added

In your example, $x_3[n]$ is the sum of two iid random variables, $x_1[n]$ and $x_1[n-1]$. Since each is taken from the uniform distribution, the probability density function (pdf) of each is:

$$
f_{x_1}(x) = \begin{cases}
\frac{1}{b-a}, & x \in [a,b] \\
0, & \text{otherwise}
\end{cases}
$$

This is just a rectangle function that is scaled and shifted to lie on the interval $[a,b]$. One property of the sum of independent random variables is that the pdf of the sum is equal to the convolution of the individual random variables' pdfs. Therefore:

$$
f_{x_3}(x) = f_{x_1}(x) * f_{x_1}(x)
$$

If you convolve two rectangles together, you get a triangle function. Therefore, if you histogram enough samples of $x_3[n]$, you should see the plot converge to a triangular shape.


measuring how much signal corrupted by passing from a medium

The most typical measure of a signal corruption (or degradation) would be an error squarred integral (or sum for discrete case), preferably normalized such as:

$$ \epsilon = \frac {\int (x(t)-x_r(t))^2 dt}{\int (x(t))^2 dt} $$

Where $x_r(t)$ is identified as the received and corrupted signal. The interval of integration choosen to fit into signal duration of interest.

This is an unweighted (or equiweighted) calculation. If you think that certain features of the error is more impotrant, more relevant, more effective than the others, then you could reflect this by modifications such as simple linear weighting or onlinear processing of the error accordingly, which would produce more meaningful results.


Evaluate DFT-downsampler-upsampler-IDFT block diagram

Given a finite length input sequence $x[n]$ of $N=64$ points, its 64-point DFT be $X[k]$. Then according to the block diagram you've provided, the signal $r[n]$ will be of $N/2$ points long and its $N/2$ - point DFT $R[k]$ will be related to $X[k]$ as :

$R[k] = X[2k]$, which will be manipulated as:

$$
\begin{align}
X[k] &= \sum_{n=0}^{N-1} x[n] e^{-j \frac{2\pi}{N} kn} ~~~,~~~ k=0,1,...,N-1 \\
R[k] & = X[2k] = \sum_{n=0}^{N-1} x[n] e^{-j \frac{2\pi}{N} 2kn} ~~~,~~~ k=0,1,...,N/2-1\\
&=\sum_{n=0}^{N-1} x[n] e^{-j \frac{2\pi}{N/2} kn} \\
&=\sum_{n=0}^{\frac{N}{2}-1} x[n] e^{-j \frac{2\pi}{N/2} kn} + \sum_{n=\frac{N}{2}}^{N-1} x[n] e^{-j \frac{2\pi}{N/2} kn} \\
&=\sum_{n=0}^{\frac{N}{2}-1} x[n] e^{-j \frac{2\pi}{N/2} kn} + \sum_{n=0}^{\frac{N}{2}-1} x[n+\frac{N}{2}] e^{-j \frac{2\pi}{N/2} kn} \\
&=\sum_{n=0}^{\frac{N}{2}-1} \left( x[n]+x[n+\frac{N}{2}] \right) e^{-j \frac{2\pi}{N/2} kn} \\
&= \text{N/2-point DFT} \{ r[n] \}\\
&= \sum_{n=0}^{\frac{N}{2}-1} r[n] e^{-j \frac{2\pi}{N/2} kn} \\
\end{align}
$$

hence we see that $r[n]$ is the sequence whose length (or period) is reduced to $N/2$ and is generated by adding the second half of $x[n]$ onto the first half.


Help with finishing this integral, to obtain the power spectral density of a pure cosine wave

You don't need to make it so complicated as to use two variables $t$ and $t^\prime$.

\begin{align} X_T(f) &= \int_{-T/2}^{T/2} x(t)\exp(-j2\pi ft)\,\mathrm dt\\
&= \int_{-T/2}^{T/2}\frac{\exp(j2\pi f_0t)+\exp(-j2\pi f_0t)}{2}\exp(-j2\pi ft)\,\mathrm dt\\
&= \frac 12\int_{-T/2}^{T/2} \exp(j2\pi (f_0-f)t)+\exp(-j2\pi (f_0+f)t)\,\mathrm dt\\
&= \frac 12 \left[\frac{\exp(j2\pi (f_0-f)t)}{j2\pi(f_0-f)}~ + ~\frac{\exp(j2\pi (f_0+f)t)}{j2\pi(f_0+f)}\right\vert_{-T/2}^{T/2}\\
&= \frac 12 \left[\frac{\exp(j\pi (f_0-f)T)-\exp(-j\pi (f_0-f)T)}{j2\pi(f_0-f)}\right.\\
&\qquad\qquad + \left.\frac{\exp(j\pi (f_0+f)T)-\exp(-j\pi (f_0+f)T)}{j2\pi(f_0-f)}\right]\\
&= \frac T2 \big(\operatorname{sinc}((f_0+f)T) ~+ ~\operatorname{sinc}((f_0-f)T)\big)\\
&= \frac T2 \big(\operatorname{sinc}((f-f_0)T) ~+ ~\operatorname{sinc}((f+f_0)T)\big).
\end{align}
So we get \begin{align}\frac 1T\big|X_T(f)\big|^2 &= \frac T4 \big(\operatorname{sinc}^2((f-f_0)T) ~+ ~(\operatorname{sinc}^2((f+f_0)T)\\
&\qquad\qquad + 2\operatorname{sinc}((f-f_0)T)(\operatorname{sinc}((f+f_0)T)\big)
\end{align}
In the frequency domain, the two $\operatorname{sinc}^2$ functions are of height $\frac T4$, are centered at $\pm f_0$ with central lobes of width $\frac 2T$ Hz. As $T \to \infty$, the central lobe widths shrink to $0$ and it is straightforward to show that for any value of $f$ other than $\pm f_0$,
$$\lim_{T\to\infty} \frac 1T\big|X_T(f)\big|^2 =0, ~ f \neq \pm f_0.$$
Now, $\frac 1T|X_T(\pm f_0)|^2$ diverges to $\infty$ and with the usual trick of pulling a Dirac delta out of the hat in such cases, we get that it must be that
$$S_{xx}(f) = \lim_{T\to\infty} \frac 1T|X_T(f)|^2 = \frac 14 \big(\delta(f-f_0) + \delta(f+f_0)\big), ~~ \text{when} ~x(t) = \cos(2\pi f_0 t).$$


Discrete wavelet transform disadvantages

A $2$-channel stage of a wavelet transform, combines two filters in parallel, followed by a down-sampling by two. The later is the cause for shift-invariance, as the filters are time-invariant. Signals $$x_0[n] = \{\ldots,0,1,0,1,0,1,\ldots\}$$ and  $$x_1[n] = \{\ldots,1,0,1,0,1,0,\ldots\}$$ which are shifted by only one sample, yield respectively  $$y_0[n] = \{\ldots,0,0,0,0,0,0,\ldots\}$$ and $$y_1[n] = \{\ldots,1,1,1,1,1,1,\ldots\}$$

Answers to down-sampling a thus not shift-invariant. This happens even with low-pass or high-pass filters. In an $L$-level wavelet decomposition, the overall wavelet filter bank is invariant to multiples of $2^L$ shift, not the intermediate one.

When applied in 2D, classical wavelet schemes, by simplicity, apply 1 DWT on rows and columns of the image, in a separate way. The resulting 2D wavelet filter is of rank one, very poor at separating directions other than vertical or horizontal. However, genuine 2D wavelets with better  directional selectivity exist. The paper A panorama on multiscale geometric representations, intertwining spatial, directional and frequency selectivity, Signal Processing, 2011, is devoted to that topic.


When a stochastic process would be a beneficial model in terms of noise

When would it be beneficial to model the signal as an outcome of a stochastic process?

When the process that generates the image/signal has a strong element of chance.

This is not related to noise necessarily. Sometimes, a deterministic model for the behaviour of a quantity simply does not exist.

For example, modern supermarkets have barcode readers that go "beep" when customers scan an item. Suppose now that we want to create a model of that "beep" sequence from which we can derive the rate at which customers are bying products from the supermarket.

What is a sequence of "beep"s? It is Beep-Random Pause-Beep-Random Pause-Beep and so on.

There is no deterministic equation for the inter-beep pauses. Deterministic models of the form $y = \alpha t + \beta x \dots$ return the same output $y$ for the same parameters $\alpha, \beta, \dots$.

But in the case of the supermarket, we have no idea about how customers arrive at the barcode scanner to scan their baskets. They just arrive...at random.

So, in this case, the inter-beep random interval is governed by a Poisson distribution which only requires a rate parameter. This rate denotes average rate of events per unit of time. In our case, the rate at which customers scan their products.

As this model is stochastic, it generates a different series of interbeep intervals everytime that you "run it" because there are infinite ways by which you can generate a sequence of events at some average rate $\lambda$...at random.

The exact same mathematics govern radioactive decay, the output of a suitable detector (that detects byproducts of the decay) and after a few more parameters are taken into consideration, the progressive formation of a gamma camera image.

Now, a stochastic process does not have to produce discrete events at some rate $\lambda$ only. It might represent the random fluctuation of some other quantity. What is the birth rate of a society at a given year?.....I don't know....But let's say that it is a random variable with characteristics governed by some probability distribution. Random variables represent quantities that either are inherently uncertain (e.g. radioactive decay) or, our understanding during the construction of the model is uncertain.

With this in mind, you might find exploring the rest of the examples provided in Stanley Pawlukiewicz's post really entertaining too. Drunken sailor walks are everywhere around us and they are the starting point of a lot of wonderful generative art and graphics effects today.

  How significant would noise have to be in for example an image for it to be beneficial to estimate the power spectrum density instead of taking the fourier transform directly?

The choice between obtaining the Power Spectral Density (PSD) or Discrete Fourier Transform (DFT) is not exactly based on the amount of noise present in the image but rather, what are you trying to model.

  And would it matter what kind of noise that was present in the image/signal, if it is additive or multiplicative?

Yes it would. It is the difference between an additive noise component, like background crowd murmur in a shopping centre when you are trying to hear a person talking to you and a fault at the cable of the microphone recording the person. In the first case, you record two sources simultaneously and both are present. In the second case, when one is present the other is not. If your microphone cable is faulty, it kind of makes and does not make contact so your rBZZZord is fuZZZBZ f artifacBZZZSPAK!ZZZ ike thiZZZZ. You either have the person's voice or not. Well, when do I have the person's voice? I don't know...what is the rate at which the microphone connection fails at? Another manifestation of multiplicative noise is wow and flutter in record players. That is, the variation that is inserted in the recording because of the variable rate of rotation of the disc platter. There, the spectrum of the playback is the modulation (or product) of the song's spectrum with the variation of the rate of turn of the disc platter. What could that be? I don't know, but let's say that it's a random variable that is $0$ (that is, bang on the right speed), plus or minus $\sigma$ turns per minute.

In fact, if we were talking about disc players that use an elastic middle wheel to couple the motor to the disc platter, this "model" would directly represent the variation in the radius of the elastic wheel that progressively gets worn out and inserts this "modulation" of the player's speed.

I am mentioning this last part more to show that there is a direct link between reality and how you choose to represent it in a model.

So, the real question here is "What are you trying to model"? And one of the best places to look for experience in creating models of the real world is Automatic Control.

Hope this helps.


Region of root locus not inside the feasible region for a given performance specification

The root locus will start at the open loop poles and move toward the open loop zeros as the gain is varied. Your pole cannot get past the zero that is located in the yellow region. This pole that is close to the imaginary axis will be dominant, meaning it will dominate your response times, slowing down the entire system. One approach is to cancel that zero with an additional open loop pole at it's location, unless that zero was introduced by your controller in which you have the ability to move it to be out in the white area, or eliminate it entirely.
It appears that the zero that terminates the closed loop pole as shown is due to $(s+A)$ in the numerator, in which case choosing a value for A that is within the white region should allow the pole to extend further to that point as the gain is increased. Also, the pole and zero due to the Pade approximation does not appear to be visible in the root locus.
To see how poles close to the imaginary axis are dominant, consider the simple case of $x(t) = e^{-at}$ which has a Laplace Transform $X(s)=\frac{1}{s+a}$  (which is a pole at $s= -a$. The time domain equation is a decaying exponential with time constant $a$, if a is very small this will take a very long time to decay. If $a$ is very large, the decay rate will be very fast.


How do I deal with pixel intensity overflow when performing discrete convolution on an image and the result is greater than 255?

There is no perfect solution to the problem you face. You must make a choice and every choice has pros and cons. For example you can apply saturation thresholding. Or you can use normalization. Saturation threshold clipping is very simple to realize and provide excellent performance provided that intermediate processing stages will not magnify the intensities too much above 255 (or below 0). Otherwise, it will severely distort the image. Normalization guarantees a loss free output but may severely degrade the algorithm quality. Besides it requires you to know in advance the normalization scale, which may not be possible always.


Why different smartphones with same Megapixel camera have different quality of image?

I am not sure this is the right place to ask (Perhaps photo.stackexchange.com?) but this is my view on the subject. The term 'Megapixel' has little if any to do with image quality nowadays, since all the popular devices have sufficient amount of pixels. Way more important is the image processing algorithms, optics and sensor size. All digital cameras have huge advantage on the last two while smartphones usually 'make up' for this with image processing algorithms, but not enough to achieve similar quality.


Perform inverse distance weighting interpolation using multiple images matlab

imresize will enlarge your image, and it gives you a choice of nearest neighbor, bi-linear, or bi-cubic interpolation. As to using the other images for interpolation, you would have to write that yourself.


How to Find Shift Between Two Signals with Cross Correlation?

Well, All in there is a simple play with the indices of the Auto Correlation Function.

Have a look at my solution for Finding Reference Audio Signal in Test Audio Signal and Cropping Accordingly.
You'll find there full MATLAB Code for the task you mention.

Enjoy...


2D-otsu algorithm in Digital image processing

The Otsu algorithm most responders are referring to, is the 1D algorithm indeed, exactly for the reason mentioned above. The main drawback is that it does not work very well in images with low s/N ratio or low contrast. Also, when you only have a small object with a big background, Otsu will incorrectly select background pixels.

The 2D Otsu provides a solution to this. I don't know the maths involved, but it performs '2D histogram projection', I found it was well documented in:
J. Zhang, J. Hu, 2008 International Conference on Computer Science and Software Engineering
From what I understand it projects the histograms in such way and then applies some filters to it so that the peaks relating to the 'object' will appear more profound in the resulting histogram, enabling better segmentation.

Both algorithms can be applied in 2D and 3D ( to add some more confusion).


Why would we simulate baseband rather than passband

One very obvious advantage of baseband simulation of communication systems is the low sampling rate it requires.

Considering a narrowband communication system having a few MHz of bandwidth to a few GHz of carrier frequency, it would be much easier (in terms of computation) to simulate it in the baseband sampling rate.

The reliability and the quality of a baseband simulation depends mostly on how accurate and realistic your mathematical modeling of the actual physical channel and the modulation system using it.


Intersymbol interference due to limited channel bandwidth

Here is a graphic I have that demonstrates ISI due to multipath propogation:

Notice the eye diagram in the upper right hand corner showing what we should receive if there was no ISI, or other noise for that matter. (This eye diagram is showing the waveform vs time over 2 symbols again and again so that we can see the effects of ISI clearly. If you have any questions about eye diagrams in general, ask it as another question and we can go into further details on that).

This waveform reaches our receiver over multiple paths, each with their own delay and attenuation as shown by the multiple waveform vs time plots in the lower right hand corner of the figure. The eye diagram as received looks like the horrible mess due to this interference that is depicted in the lower left hand corner. (Luckily this can easily be undone through channel equalization in the receiver).

The variation in time of all the different propagation paths is known as the delay spread. Below is a plot I have shown in other posts showing the typical delay spreads for macro-cellular channels, as tabulated by Professor Paul Walter Baier of the University of Kaiserslautern, Germany (Vielen Dank, Herr Professor Baier!):

The significance of delay spread is whether we end up with Frequency Selective Fading (which means deep selective frequency nulls within the passband), or Flat Fading (which means the entire passband power level will vary, equally in frequency across the passband).  In the figure below this is demonstrated by two dominant paths with a Delay between the paths before summing together at the receiver. On the right is the resultant magnitude of the received signal versus frequency for two different delay conditions, the red plot is the result of a relatively long delay, while the blue plot is with a relatively short delay. The shaded region is our example channel at a particular frequency of interest. Notice for the red (long delay) given our channel bandwidth that we have the possibility of a deep null, selectively within our passband, while for the blue (short delay) the power level across the passband is generally much flatter. If you are familiar with the characteristics of an FIR comb filter then you know exactly what is going on; a fixed delay is a phase that varies linearly with frequency, so at some frequencies the phase will be $\pi$ radians so that the receive signals will cancel each other, while other frequencies the phase will $0$ (or equivalently multiples of $2\pi$) so that the signals will coherently add. Since the phase of a fixed delay varies linearly with frequency we see the repetition of this over frequency as the delayed path passes repeatably through $0$ and $\pi$ conditions as we vary the frequency of interest. A longer delay will increase more in phase for a given change in frequency, hence the tighter frequency response for that case.

Plot showing delay vs fading:
RED: Frequency response for a long delay.
BLUE: Frequency response for a relatively shorter delay.
SHADED AREA: Example frequency channel of interest.

Limited bandwidth induced ISI is easily understood in the time domain: A relatively tight filter in the frequency domain MUST have a long impulse response in the time domain (consider filtering as a low pass filter, and in order to really smooth the signal, you must average for a long time). So if you have a filter with a long impulse response, it goes to say that it's response to one symbol will extend into other symbols, causing Inter-Symbol-Interference. The trick is to provide relatively tight filtering while not causing ISI, and this is done with the class of Nyquist filters. They do this by having zero crossings in the impulse response at the symbol spacing. Thus the response from one symbol is always zero at the decision-sampling location of all other symbols. In fact the clean eye diagram in the first plot was created with such a Nyquist filter (the raised cosine filter specifically). All the deviations of the waveform are the result of trailing impulse responses of earlier symbols.

For further details on that see: Why root raised cosine filter can eliminate intersymbol interference (ISI) ?


What does filtering do to a signal in time domain? Does the signal gets distorted?

It's always important to keep in mind that more than 80% of signal processing is filtering. However, not every filtering operation should be "viewed" in the same way.

Let us take the example you mention, of an "ideal known channel" (assuming it's not in deep fade). In wireless communications that would mean perfect knowledge of channel to the receiver. Typically, you could invert the channel in a very basic and often used scheme known as Zero forcing i.e. if $$y=hx + n$$ where y is received signal then the equalization is simply $y/h$ assuming real signals and noise. Similarly an MMSE equalizer ( optimal for most SNRs) has a typical mathematical expression that you would find in standard textbooks. The point is it would be less intuitive to look at this equalizer from the perspective of it's frequency response. It might be more intuitive to look at it from the perspective of vector spaces in this case. Especially as the degree of freedom of a network grow. For ex: a MIMO system.

Filtering is essential to iradicate BER, simply because if you don't filter ( as mentioned above), the time domain signal has been essentially "changed" by the channel.

We are always trying "mitigate" the effects of the channel. I would say "mitigate" rather than "invert" because it is sometimes not even possible or doesnt make sense to simply invert. For ex: in the scheme above $y/h$, notice that the noise is also devided by $h$. If $h$ is small, then you boost the noise as well. So in such a case a MMSE equalizer would make sense. When viewed in line of probability theory, the MMSE equalizer is calculating the expected value of the posterior distribution of $x$ (your signal) after observing $y$. So in that sense it is not inverting the channel.

Long story short, we try to do what's best for the system depending on bit rates, how close the bit rate is to channel capacity, SNR, modulation schemes etc. Rather than simply invert the channel.

There are other scenarios such as an antialsiaing filter, in such cases it is best viewed from it's frequency response, like cut off frqeuency and phase responses. You would get a better idea of the time domain filtered sequence looking at it from the frequency domain perspective. For ex: the cut off frqeuency would tell us what maximum gradients are tolerated in the input signal to the filter and hence are part of the filtered sequence or are smoothen out. The phase response would tell us which freqencies of the filtered sequence, would be delayed by what amount. Whether delay would be constant for the whole signal (linear phase) or there would be different group delays (non linear phase).

Indeed the filtering will change the shape of the input signal, otherwise usually nothing interesting is happening, but the filter itslef is not designed to "distort" the signal, it is designed to "better" the signal according to application, indeed the shape /magnitude of time domain filtered signal will change after filtering, but for the better of the system.


Use of pseudo-random binary sequence (Gold, Kasami, etc.) in frequency hopped code-division multiple access (FH/CDMA)

There are many ways to do this. A simple method is to take the current value of the register (not just its output) as the carrier frequency selector. For instance, a (6,1) LFSR with 63 states can select between 63 different channels.

Other methods are based on cubic and quadratic congruence codes. I don't know much about them myself, but look up the papers by Titlebaum et al.


(1) Yes, if the technology permits it for the power handling requirements of the design, the strategy is to filter as much as possible immediately at the front-end of the receiver prior to the LNA. The primary reason is jamming from stronger signals. Depending on receiver topology used we would be concerned with 2nd and 3rd order intermodulation products and receiver phase noise that can be induced on out of band signals if significantly stronger all creating noise in band that would limit our receiver sensitivity.

FBAR technology is an excellent choice for front-end filters combining low loss and exceptional rejection. I have used FBAR filters in the design of femtocell cellular base-stations where the constraint was to receive the entire band (in this case it was a duplexer so would be used to receive 1850-1910 MHz and transmit 1930-1990 MHz --- notice the 20 MHz spacing between these two bands that this technology was able to provide exceptional isolation up at 2 GHz). We use the low loss band-selecting filter due to linearity/intermodulation concerns with the LNA, however with FBAR one must also be very careful with inter-modulation effects especially when using these filters in multi-carrier designs. Here is a good online article with more details about FBAR and other filtering considerations.

http://www.linleygroup.com/mpr/article.php?id=11440

(2) The distortion you are referring to is concern with frequency offset. Frequency offset will always exist since the typical transmitter/receiver set do not use a shared clock. The two dominant sources of frequency offset are the offset of the clocks themselves and additional offsets under dynamic conditions due to Doppler.  The relevant standards will specify the maximum allowable clock offset for the design, but < 5 KHz is a reasonable maximum possible frequency offset at 2 GHz for lower cost technologies (this is driven by the ppm specification for the master oscillator used).  If I calculated correctly the Doppler at this frequency when the transmitter and receiver are moving apart from each other at 60 mph would only be 179 Hz. The combination of both of these numbers will be well within the passband of whatever front-end filter is used, as no such filter would have a brick-wall response.

(3) I think you are confusing the carrier (2 GHz) with the bandwidth at that carrier (20 MHz). The actual carrier frequency is irrelevant to how much data we can transmit (without consideration to path loss effects on signal power for the SNR component of capacity).  The theoretical maximum symbol rate is given by the Shannon-Hartley equation for maximum channel capacity which accounts for both SNR and BW:

$$C = B \log_2(1+ SNR)$$

Where:

C is the maximum capacity in B/sec
B is the bandwidth of the channel is Hz
SNR is the signal to noise power ratio (not in dB but ratio of power)

Note MIMO technologies exceed this by utilizing multipath to send multiple "channels" simultaneously.

For more info on Shannon-Hartley equation see:

https://www.gaussianwaves.com/2008/04/channel-capacity/


3dB and 6dB Per Octave Rolloff Digital Filters / Generating Pink and Brown Noise

-6 dB / octave is trivial.

a common reference for a pinking filter is http://www.firstpr.com.au/dsp/pink-noise/


The Parameter Devil — How to set them when no validation against groundtruth is possible

Assuming that there is a ground truth, (at least theoretically) one of the possible ways to overcome the "tediousness" problem is a "bootstrap" ground truth creation. If you already have a decent algorithm that does the job in about, say 80%-90% of the cases, you can run your algorithm on a large set of instances and ask a user to mark only the mistakes. This approach has its own flaws, such as bias towards your algorithm.

However, there are some cases in which there is no ground truth at all, only different system trade-offs. For instance, an image processing system is required to output a sharp, color-accurate, non-noisy image. Obviously, you can't have them all at the same time. In such case you should use objective metrics that can be calculated on the result of your system. (See Imatest, DXO analyzer for image processing).

Once you have these, there are methods of multi-objective optimization that can create a mapping from the trade-offs (which are clear to the user) to the intrinsic parameters.

In any case, you should never give the user a parameter that he cannot understand. If all fails, simply hard-code the parameter.


How to calculate auto-correlation of a bpsk modulated signal or how to calculate expectation value of complex exponential function

This question is essentially unanswerable because too many details are left out.
Here is an answer to the following simplified version of the question

  What is the autocorrelation function of the random process
  $$X(t) = A\exp(j2\pi f_ct), -\infty < t < \infty$$
  where $A$ is a random variable equally likely to have values
  $+1$ and $-1$, and the autocorrelation function is defined as
  $$R_X(t,\tau) = E[X(t+\tau/2)X^*(t-\tau/2)], -\infty < t,\tau < \infty.$$

This is easy since the only random variable involved is $A$, and
it is a discrete random variable.  Thus, we have
$$\begin{align}
E[X(t)]&= E[A]\cdot \exp(j2\pi f_ct)\\
&= 0 ~~\scriptstyle{\text{since }} E[A]=0.\\
R_X(t,\tau) &= E[X(t+\tau/2)X^*(t-\tau/2)],\\
&= E[A^2]\cdot \exp\left(j2\pi f_c(t+\tau/2)-j2\pi f_c(t-\tau/2)\right)\\
&= \exp(j2\pi f_c\tau)~~\scriptstyle{\text{since }} E[A^2]=1.
\end{align}$$

What would be needed to make this question worth answering? First,
one usually has a baseband pulse shape, usually time-limited,
that is nowhere mentioned by the OP. The autocorrelation function
of a BPSK signal is usually a scaled version of the
autocorrelation of the baseband pulse shape (and so the power
spectral density is determined by the shape of the baseband pulse).
To get to this result, one needs to consider not just
one bit but rather the stream of bits that modulate the BPSK signal,
and assume that these bits are i.i.d. random variables (such as
$A$ above). Second, something needs to be said about the noise
$n(t)$ that the OP casually includes in her question.


How can frequency dispersion in a vibrating string be modelled using an LTI filter?

From your question, it sounds like you think the all-pass filter is an independent post-processing step that takes as its input the output of the classic Karplus-Strong model (delay line, LP filter, close the loop). If that would be the case, yes, it would be very surprising indeed that new harmonics would appear! But here, the allpass filter is part of the closed loop. You can intuitively think about what it does in these terms: it is a short delay, with a delay value which is frequency-dependent (phase shifter). This gets added to the delay provided by the main $N$ samples delay line, and causes the lower frequencies to "see" a longer loop. So the highest harmonics will have a period close to $N$, and the lower harmonics will have a lower period. The overall transfer function of the loop is no longer a comb with regular spacing between teeth, but a comb with a slowly increasing gap between teeth.

What you are forgetting also is that this whole system (delay line, all-pass filter combined with loss LP filter, close the loop) which is a LTI, is excited by a burst of white noise. So the input signal contains all frequencies, and there's nothing surprising or contradictory with the fact that altering the transfer function of a LTI can change its frequency response so that its resonant frequencies are shifted, or so that new bumps appear.

If you need to convince yourself... System A is a band-pass filter with a frequency of 1kHz. System B is a band-pass filter with a frequency of 1kHz ; in parallel with a band-pass filter with a frequency of 2kHz. Both are excited by white noise. In the second case you have "added" a LTI filter in the system; and a new frequency has appeared. There's no magic in this - you have just shaped the frequency response of the system differently to sculpt the input in new ways.


Implemenation of Length N Moving average filter using block covolution algorithm(overlap and save algorithm)

in psudo pyhon code
const tail = 5

'inside loop where you feed it values

list.push(entry)
list.rotate.left(1)
if len(list)>tail list.pop()
avg = sum(lst) / len(lst)
print avg


What is a $BT$ (Bandwidth-Time) product with reference to modulation?

The $BT$ product is the bandwidth-symbol time product where $B$ is the $-3\textrm{ dB}$(half-power) bandwidth of the pulse/filter and $T$ is the symbol duration. For different applications you will find varying recommended values. In GSM telephony for instance, a $BT=0.3$ is recommended. In satellite communications with GMSK, for near-earth missions the CCSDS recommends a $BT = 0.25$ whilst for deep-space/interplanetary missions, the use $BT=0.5$ is recommended. You find more details in this CCSDS report on bandwidth-efficient modulations. See page 2-2 and page 2-3 for the mentioned recommended values.

What does that mean ? Let's say we have $1$ bit per symbol ($T$ then correspond to bit time). For a $BT = 1$, the pulse shaping the symbol spreads over one bit period duration. For $BT = 0.25$, the spread is over $4$ bit periods, for $BT = 0.3$ the spread is over approximately $3$ bit periods, and for $BT = 0.5$ the spread is over $2$ bit periods.

This means that a smaller $BT$ product results into higher ISI and a compact spectrum. Measures need to be taken for the introduced ISI in this case much more than in the case of a higher $BT$ product where less ISI is introduced and we have much less compact spectrum.

In GMSK, one of its properties is it maintains a constant envelope and that's because of the Gaussian pulse applied prior to modulation. The GMSK pulse can be defined as in equation $(1)$ below$^1$:
$$
g(t) = \frac{1}{2T}\left[Q\left(2\pi B\cdot\frac{t-\frac T2}{\sqrt{\ln(2)}}\right)-Q\left(2\pi B\cdot\frac{t+\frac T2}{\sqrt{\ln(2)}}\right)\right]\tag{1}
$$

Where $Q(t)$ is the complementary cumulative distribution function defined as:
$$
Q(t)=\int_{t}^{\infty}\frac{1}{\sqrt{2\pi}}\exp\left(-\frac 12 x^2\right)dx\tag{2}
$$
The figure below shows the pulse shapes for different values of the $BT$ product:

The spread of this pulse is inversely proportional to the $BT$ product and its peak amplitude directly proportional to the product. Again, this means that lower $BT$ results into a wider spread (over bit symbol period) and with lower peak amplitude and a high $BT$ results into a narrower spread with a higher peak amplitude. In conclusion, the pulse duration increases as the bandwidth of the pulse decreases.

Here they show a link between the $BT$ product, the filter’s $-3\textrm{ dB}$ cutoff frequency and the bit rate $R_b$ as:
$$
BT = \frac{f_{-3\rm dB_{\rm cutoff}}}{R_b}\tag{3}
$$
You can say that the $BT$ somehow determines the degree of filtering. More on fundamentals and properties of GMSK can be found in this paper $^2$ and this paper $^3$. In both papers discussions in relation to equation $(3)$ and the variations in the eye pattern as a result of $BT$ product values are given.

You can find extra stuff here, here, and here.

$[1]$: John G. Proakis, Digital Communications, 4th Edition, McGraw-Hill, 2000.

$[2]$: A. Linz and A. Hendrickson, "Efficient implementation of an I-Q GMSK modulator," in IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, vol. 43, no. 1, pp. 14-23, Jan 1996.

$[3]$: K. Murota and K. Hirade, "GMSK Modulation for Digital Mobile Radio Telephony," in IEEE Transactions on Communications, vol. 29, no. 7, pp. 1044-1050, Jul 1981.


Radar terminology - Specific meaning of "video signal"

You also see this on Spectrum Analyzers; "video bandwidth" for the same reason: the video is the output voltage from the power detector used to drive the vertical on a display (when the horizontal is the sweep), or in the case of the polar displays used to drive the radius where the sweep drives the angle. Since it is used for the display signal it is properly called the "video" signal. As I introduced, this is the same on a spectrum analyzer where we can adjust both "resolution bandwidth" and "video bandwidth": Resolution Bandwidth is the bandwidth of the effective bandpass filter prior to the power detector, while Video Bandwidth is the bandwidth of the effective low pass filter after the power detector. The first filter will reduce the total power incident on the detector when the input is spread as a spectral density, thus reducing the voltage out of the detector, while the second filter (video filter) will only smooth (average) the measured noise.

Here is a perfect picture showing what I describe in terms of the Spectrum Analyzer, credit Erik Diez of Agilent (now Keysight).
http://electronicdesign.com/test-amp-measurement/fundamentals-spectrum-analysis

So for the case of radar, at least the original analog systems, the video output was a similar power detector used to drive one axis (either magnitude or as was common radius on a polar plot for a rotating system) of the radar output display.

(image credit: http://www.linuxwolfpack.com/swapshop.php)


What is relationship between entropy and frequency in image

If I understand correctly, you are taking the entropy in the image from the distribution of black and white pixels. In image (a) you have 50% black, 50% white, so the entropy is 1 bit. In image (b) you have 85% black, 15% white, so the entropy is 0.6 bits.

If you make another image (c) with black and white stripes equal in width to the black stripes in image (a), you will have 50% black, 50% white again, i.e. entropy 1. But the frequency will be less than image (a).

So for this notion of entropy, the answer is: No, in general there is no relation between entropy and frequency in an arbitrary image.

However, in real world images energy is more concentrated in low frequencies than in high frequencies. So if you define entropy in terms of the distribution of such images, you will find a relationship between frequency and entropy.


How does a wavelet help in image compression?

A common wavelet based standard is JPEG 2000 and a common DCT based standard is JPEG.

JPEG 2000 uses wavelets, but a good portion of the better compression it achieves than JPEG is due to the fact that JPEG uses a much much simpler entropy coder (JPEG does context-dependent Huffman codes and run length coding, JPEG 2000 does arithmetic coding with some extra tricks). You can gain a lot by tweaking the entropy coding stage. The particular wavelet transform you choose will make a difference as well. And how you decide to compress the coefficients makes a huge difference as well (JPEG reads its dct block coefficients in a zig-zag pattern and uses run length coding to compress long runs of zeros in the higher freq components due to the nature of the quantizers it uses).

The primary advantage of wavelets as used in JPEG 2000 versus rounding the DCT coefficients in the manner of JPEG is that it reduces blocking artifacts.

If you're not familiar with wavelets, you can look at an introductory text on wavelets, like Mallat's Wavelet tour of Signal Processing or Vetterli's new book (freely available online). Then, you can read the JPEG 2000 standard. These notes are kinda nice at a high level as well (and it has good pointers to the references).

For a quick review of JPEG, you can look at Gonzalez & Woods' Digital Image Processing (second or third edition should be fine).


Decoupling two signals where one is a delayed version of the other

You can employ cross correlation of the received signal with the signal you know that is contained.

Fs = 1000;
t = np.arange(0, 2, 1./Fs)
s = lambda t: np.cos(2*np.pi*10*t*t)

t0 = 0.5

single = s(t)
signal = s(t) + s(t-t0)

plt.subplot(1,2,1)
plt.plot(t, signal)
plt.plot(t, single)

plt.subplot(1,2,2)
plt.plot(t, np.correlate(signal, single, 'same'));

The delay between the peaks is the time distance between both signals.


For a generic value of the group delay $\alpha$, the filter has a 2-sided infinite impulse, and is therefore not causal.

Causal linear phase systems are (almost) always FIR systems with any of the 4 types of symmetry.

In these cases, the group delay is multiple of 1/2 (0, 1/2, 1, 3/2, etc.).

You can have causal linear phase IIR filters, but these are not of finite-order (cannot be implemented with a difference equation), so they are of little use in practice.


MSK signal recoverd though FM Discriminator?

It is possible that the sample data has a pulse-shaping filter before the frequency modulator. That would explain the first plot.

Look at the power spectrum of the sample data and check the sidelobe levels as compared to textbook MSK and your modulator. A pulse shaping filter will reduce the sidelobe levels.

Square waves are expected in the second plot, because the data entering the modulator has no pulse shaping on it. The spectrum should be textbook MSK.


Why is privacy scrambling a standalone block in a communications transmitter?

There are two possible scramblers, one meant for security and
privacy of the information, and the other for making sure that
the sequence of symbols transmitted over the channel has
properties that are desirable for digital transmission purposes.
Either or both might be present in a system, or both could be
absent too. Furthermore, the scramblers might well be
non-interchangeable since the scrambling needed for privacy
need not be the same as that required for transmission purposes.

The question then is, where should these scramblers be placed
in the transmitter.  The channel scrambler obviously should be
between the error-correction encoder (whose output is to be
transmitted) and the modulator that creates the actual signals
that are broadcast over an antenna or transmitted down a cable
or twisted pair of wires (or recorded on a magnetic medium or
CD or DVD or what have you).  The question of where the
privacy scrambler should be placed is more complicated.
@JasonR has contended in comments that putting it before
the data compression unit makes the input to the compressor
less compressible.  This is certainly something to think
about, but I am not completely sure about that it is always
a major concern
because the scrambler for security and privacy might be doing
other things than breaking up long sequences of repeated
symbols.  In any case, the location of the privacy scrambler
before the data compression unit may also be dictated by
privacy and security concerns that may have nothing to do with
efficient communication.  If I were in charge of privacy and
security, I would make darn sure that the chain looked like

data --> ENCRYPTION -- > scrambling -->

in my domain before the dsp and comm people even get to see
the stream of symbols.  Let them worry about
how to transmit the stream I give them as efficiently as
possible; my job is security and so I make sure that things are
nailed down tight when they leave my shop!


The random phase $\phi$ makes the carrier signal $c(t)=A\cos(2\pi f_0+\phi)$ wide-sense stationary with the autocorrelation function

$$R_c(\tau)=\frac{A^2}{2}\cos(2\pi f_0\tau)\tag{1}$$

The Fourier transform of $(1)$ is the power spectrum of the carrier:

$$S_c(f)=\frac{A^2}{4}\left[\delta(f-f_0)+\delta(f+f_0)\right]\tag{2}$$

where $\delta(f)$ is the Dirac impulse.

Since the carrier signal $c(t)$ and the message signal $m(t)$ are assumed to be uncorrelated, the autocorrelation function of the modulated signal $s(t)=c(t)m(t)$ equals the product of the autocorrelation functions of $c(t)$ and $m(t)$:

$$R_s(\tau)=R_c(\tau)R_m(\tau)\tag{3}$$

From $(3)$ it follows that the power spectrum $S_s(f)$ of $s(t)$ is the convolution of the power spectra of $c(t)$ and $m(t)$:

$$S_s(f)=S_c(f)\star S_m(f)=\frac{A^2}{4}\left[S_m(f-f_0)+S_m(f+f_0)\right]\tag{4}$$


MPEG Layer I: use of scalefactor and quantizers

Let's get back to the official MPEG standard document and avoid your confusing literature for a moment!

So, you get audio as time samples. You use a very specifically crafted 32-band filterbank, which takes these input sample stream and generates 32 sample streams of each 1/32 of the original rate (that's the "time/frequency mapping" in that diagram you show, and the QMF in this diagram). All these streams are time-domain!¹

Now, you take a frame – that is, 12 "parallel" samples from each of these 32 streams. The codec's job is to figure out how "important" they are, and only use as much bits as needed to represent these.

The "importance" step is done by the psychoacoustic model (which might do arbitrary complex things, but that's not the point of discussion here), which will give us some info of the type "use $n_0$ bits for the first subband, $n_1$ for the second,…".

The Quantizer's job is now to take the 12 bits of "its" subband and save them with the $n$ bits depth it got. That's pretty straightforward. You just take the highest value in your 12 samples, scale it so that it is $2^n$, scale the other 11 samples as well, round them to integers, and save the scalefactor for later reconstruction.

You do that to all 32 subbands, and send the resulting streams of bits together with the scalefactors as the data part of an MPEG 1 Layer I frame.

At the decoder, you multiply with 1/scalefactor, then you transform the 32 subbands back into a single time signal.

Your confusion stems from the fact that you took the intro to that chapter as a kind of "step before" encoding happens, whereas it just was a slightly misleading (and claiming the samples were transformed to frequency domain) overview.

Also, the author of your book is wrong about the usage of the FFT in MPEG 1.

Be very precise about what you're exactly considering. We're here talking about MPEG 1 Layer I, not Layer III. In layer III ("MP3"), there's a kind-of-a-frequency-domain-transform happening on the subband samples (a modified Discrete Cosine Transform). That might have contributed to confusion.

¹ the "frequency mapping" is actually the mapping of signals to the frequency-disjunkt subbands. There's a bit of a mathematical caveat to that – the QMF filterbank is pretty close to being a wavelet transform, but I really wouldn't call it a time-frequency transformation. The original standard wouldn't want to do that, either, so they intentionally used the word "mapping". By the way, you'll find more info in the standard, which you can find by following the picture source that the author of your list (presumably) has in its table of figures.


When an object is Red, we mean that it only reflects the portion of the electromagnetic spectrum representing this color, about 650nm.

For a red object, we generally consider that only the color red is reflected, like the XKCD-like diagram below.

Source:

Color and Color Vision
Quora

EDIT:

This is a wavelength-intensity graph for various colored objects (anodised aluminum to be exact). The intensity is maximum for the wavelength associated with the color we see, but it still can reflect other colors, although at a much lower intensity as well.

Source:

Chen, Yuting, et al. "On the generation of interferometric colors in high purity and technical grade aluminum: an alternative green process for metal finishing industry." Electrochimica Acta 174 (2015): 672-681.


Image derivative vs Image gradient.

Same thing.

It tells you how much local contrast there is between adjacent regions in an image. The image gradient gives you the slope of the contrast towards its greatest magnitude.

  Image directional derivative/gradient.

Same thing. In a simple $f(x)$, direction is implied. The derivative is always taken with respect to $x$ and the $x$ axis points to the right (if we are talking about a spatial dimension). In an $f(x,y)$ or more generally, any $f$ of more than one variables, the derivative can be obtained towards (or parallel to) a direction. If you don't specify the direction of differentiation, you get the  derivative (as above) which tells you the direction of higher slope.

  Image Laplacian vs image second derivative vs image divergence.

You need to understand Divergence. Divergence tells you how much "outgoing flow" there is in a vector field. Imagine a fountain, the water comes up through a pipe in a small pool. "New" water comes in to the pool and pushes the existing water out of its way. Assuming that the pool is perfectly levelled, "new" water is pushing existing water outwards without preference to a specific direction.Isotropically. If we were to represent the velocity field as a vector field, at the point of the fountain we would see vectors going out, radially, towards all directions, like a little star. The Divergence of that point (the source) would have positive maximum value. If the exact opposite happened (i.e. the water was flowing IN a sink), then the divergence at that point would have negative maximum value.

So, how do you estimate the Divergence at some point? You take a "circle" (or sphere or hypersphere in higher dimensions), you position it at a particular point in space and then you estimate how much flow goes in and out through its boundary. Why circle / sphere / hypersphere? Because Isotropically. How do you do this "boundary" thing? With a surface integral. Where do you get the "flow" from? From the vector field.

The Laplacian is the Divergence of the gradient. Remember that Divergence expects a vector field at its input.

The Laplacian of an image depends to an extent on what the image depicts. If we are talking about simple visible light images, the Laplacian gives you areas that either reflect light or absorb light. It can be used to find specular reflections for example or spherical regions that bounce light towards all directions (concave or convex).

The second derivative is the second derivative (please see earlier answers). It tells you the slope of the slope of "brightnes" and it can be used to find local maxima and local minima.

  Image curvature (how it is related to the other terms?).

Which image curvature? You might have curvature of features ON the image (e.g. the curvature of a fence).

Given the discussion so far, you probably mean "curvature" as it emerges in the third dimension (reflectivity or brightness). In that case, curvature is more related to the Laplacian (but it could also be related to gradient towards a direction, anything that implies curvature).

  Image normal vs image gradient magnitude.

A Normal usually refers to a vector. A vector has direction and magnitude.

The image normal is a vector that is perpendicular to some notion of curvature of the image.

Usually, normals are made to have a magnitude of one because we are only interested in the direction they are pointing towards. But that is not necessarily the case with any normal.

The image gradient tells you the highest slope departing or arriving at a given point in the image. Therefore, the image gradient is a vector field which could be used to derive the normal vector field. But if you simply take the magnitude of it, all that you have is an estimate of local contrast towards the direction of maximum difference.

My advice would be for you to revisit vector calculus and linear algebra because these are the fundamental concepts that image processing is re-using. Divergence is not redefined in image processing. This will also help you in terms of "code". All major scientific computing platforms have these operators implemented, it is just a matter of re-using them rather than re-writing them.

The other thing to note is that imaging has context. Divergence over visible light photography is different in interpertation than infrared photography or X-Ray photography or other types of imaging. In visible photography there are certain assumptions that are often made to simplify the mathematics (e.g. in the way that light reflects off of objects). These assumptions make it "easy" to use something like Divergence to estimate curvature but there are still limitations. (And you might end up requiring more than one images to truly estimate "curvature"). In visible light photography, bright / dark, usually means light/shadow, but in infrared photography, bright/dark could also mean near/far.

Hope this helps.


what is meant by biased and unbiased Autocovariance sequence estimate?

In spectral estimation, one estimates "spectra" and other values. And one is interested in knowing whether the estimate $\hat{x}$ converge to the "true" but unknown value  $x$. It is unknown, for instance because one has only access to a finite quantity $N$ of samples to compute with, a limited number of realization, because ergodicity is not assured, etc. So we want to know whether  $\hat{x}_N\to x$ as $N\to \infty$. The convergence can be thought in different ways, and the most common is the mean square sense: does

$$\lim_{N\to \infty} E(|\hat{x}_N-x|^2)=0$$
where $E$ is the expectation? This can be really complicated to prove in practice. Thus, often people are satisfied when simpler conditions are met. And there are two necessary conditions for the above:

$$\lim_{N\to \infty} E(\hat{x}_N)=x$$

$$\lim_{N\to \infty} E(|\hat{x}_N|^2)=0$$

The first is the asymptotic unbiasedness, the second the vanishing variance. Together, they define consistent estimators. So, a biased autocorrelation estimate won't converge to the true autocorrelation. And derived estimators (like periodograms) are unlikely to converge as well.


What is it called if a system is "in rest" until time $t=t_0$?

There are two things to consider here. First of all, a non-causal system can react to input that hasn't arrived yet. And, secondly, there could be non-zero initial conditions that would contribute to the output signal without any input signal.

So what you probably mean is a "causal system initially at rest" or "a causal system with zero initial conditions".


IDTFT of convolution in the frequency domain

It seems to me that this exercise is meant to combines  the basic properties of the DTFT (gathered here: Table of DTFT Properties). Matt described the product/convolution property. You also get the time-shift/complex modulation. I suspect (thought I did not do the calculations) that the issue with the factor $10$ could be tackled with a change in variable: $10\Omega = 5\times (2\Omega)$, and the use of the time-scaling (expansion) property (see Time scaling of discrete-time sequences and the DTFT):
$$ S(c\Omega) \Longleftrightarrow \left\{
                \begin{array}{ll}
                  x[n/c] \textrm{ if } n/c \textrm{ is an integer } \\
                  0 \textrm{ otherwise.}
                \end{array}
              \right.$$
Other hints hinge around Dirichlet kernels:
$$D_N(x) =\frac{\sin\left(\left(N +1/2\right) x \right)}{\sin(x/2)}$$
They are also called asinc or psinc (aliased or periodic cardinal sine or sinc), and related to finite-support discrete windows. If $*$ is the Convolution sign, the resolution may use the product/convolution properties of Fourier.
And more generally (at the bottom of the page on Dirichlet kernels), you have the identity:
$$\sum_{n=0}^{N-1} e^{jn\Omega} = e^{j(N-1)\Omega/2}\frac{\sin(N \, \Omega/2)}{\sin(\Omega/2)}\,,$$
which relates the DTFT of a discrete time window $w_{[0,N-1]}$ (from index $n=0$ to $n=N-1$) to ratios of sines with a phase correcting term. You can check details at Discrete-time Fourier transform of a window function.


Conversion from stationarity to non-stationarity

You've asked, if it's possible to convert a non-stationary signal into a stationary one and vice versa; the answer is yes it's possible to some extend as outlined below.

Indeed it forms the basis of a family of transform based signal compression algorithms. Specifically, for transform based signal compression, a block of time domain samples $x=\left[x_0,x_1,...,x_{N-1} \right]$ are taken, and transformed into a block of frequency domain coefficients $c=\left[c_0,c_1,...,c_{N-1}\right]$ practically via a DCT (discrete cosine transform) or similar and are re-quantized for furher processing.

Note that the time domain signal in the block is assumed to be WSS and correlated, as each sample $x[n]$ has the same variance $\sigma_x^2$, whereas the frequency domain coefficients are non-stationary with each coefficient $c[k]$ having a different variance, $\sigma_{c_k}^2$. This change in variance results in change in bit-resolution of the quantizer, hence compression. Furthermore, the compression efficieny is a direct consequence of the strength of the correlation in the time domain samples.

This non-stationary frequency transform coefficients $c[k]$ are made use of opimal quantization of those coefficients with larger variances (typically the lower frequency coefficients) assisted by the human visual system perceptual characteristics as well. That's a lossy compression. However, you don't have to compress anything, if all you need to get is non-stationary signal.

So this provides an answer for your question, but whether a sutiable method for your financial signal exists is up to you.

Have a look at the following books:

A Primer for Financial Engineering: Financial Signal Processing and Electronic Trading_AKANSU

Financial Signal Processing and Machine Learning_AKANSU


Integration with infinite/finite limits as a form of summation in DSP notation for discrete signals

In the absence of any further information, we must assume that tradition plays a role in the numerical implementation.  The language of mathematics implies that an integration is an integration, so we can assume that the trapezoidal rule holds:

$\int_a^b f (x){\mkern 1mu} dx \approx \frac{1}{2}\sum\limits_{k = 1}^{N} {\left( {{x_{k + 1}} - {x_k}} \right)} \left( {f({x_{k + 1}}) + f({x_k})} \right)$

However, the intent of the author is another mitigating factor, so careful reading is required.  There are often some pathological issues of notation, but these are strange.

After some numerical experimentation, I've found that in general, an integration is an integration, no matter which field of inquiry.  The language of mathematics is universal.


Correlating a sequence or a code with itself at QPSK symbol rate

The answer is yes, absolutely 1 sample per symbol is sufficient if your symbol timing and frequency offsets are sufficiently accurate (they need not be perfect).

Regarding frequency accuracy: The magnitude of correlation vs frequency offset is a Sinc function with first null at 1/T where T is the length of the sequence being correlated.

I started to develop this point in another recent post (Derivation of the Optimal Matched Filter - Convolution vs. Correlation) showing the first three samples of a correlation when a frequency offset exists. The results in this post showing correlation without a frequency offset should make it very clear why 1 sample (the correct sample) per symbol is sufficient.  In the plot we see the resulting Sinc response of correlation vs frequency offset with the first null at 1/T = 1 Hz. If we needed a wider tolerance to frequency offset, we could easily correlation over 100 samples in the case to then push the null out to 1/.1 = 10 Hz. Of course we would lose 10 dB in processing gain (see prior post referenced) so we see the trade of SNR needed versus the frequency inaccuracy that can be tolerated.

This next graphic continues that example to show the case of a correlation over 1000 samples vs frequency offset when the sampling rate is 1 KHz. This means that T = 1 second in this case, and we see that if we considered correlating a 3 Hz signal to a 4 Hz signal, that after summing 1000 products, the summation will be back at the origin (correlation null).

The exact relationship for the correlation versus timing offset depends very much on the pulse shape of each chip in your signal (the change in magnitude for each sample due to timing offset), and the concern therefore would be both with a timing offset in time, and a timing offset in frequency (this is different from the frequency offset above in that an offset in frequency of the timing clock will cause it's sample position within the symbol to roll, while a frequency offset in the carrier will cause the complex IQ correlation to rotate on the IQ plane.).

Both cases are easily computed by knowing that correlation is a sum of products, and correlation sequences typically have the same number of 1's and 0's (or close to the same number, often they are an odd number so in that case would have one greater 1 than 0 or vice versa).


How does the quantization error generate noise?

"Noise" in this context refers to anything unwanted added to the signal, it doesn't necessarily mean it is gaussian noise, white noise, or any random well-described process.

In the context of quantization, it is a purely algebraic argument. One can view quantization as the addition of an unwanted signal ("noise") equal to... the difference between the original signal and the quantized signal. Note that this quantification noise is not random, and is correlated with the input signal. For example, if a signal is periodic, the quantization noise introduced when quantizing it will be periodic too.


It seems to me that the only way in which one of those solutions is possible is without aliasing. Probably there is a typo error, and the actual sampling rate is greater than 36 kHz. In such a case, for example, if the sampling rate was actually 37 kHz, we can interpolate with a factor I = 4, and then decimate with a factor D = 3 (in that order):
$$
f' = f / I = 18.0~kHz / 4 = 4.5~kHz; \\
f'' = f' \times D = 4.5~kHz \times 3 = 13.5~kHz
$$
Where $f'$ is the frequency component after interpolation, and $f''$ is the frequency component after decimation. Therefore, my final solution is (d) if, and ONLY if, there is a typo and the sampling rate ($f_s$) is actually greater than the Nyquist frequency: $$f_s > f_{Nyquist} = 2\times f = 2\times 18.0~kHz = 36.0~kHz$$.


Difference in Interpretation: $ω$ (rads/s) vs. $ω$ (rads) and $X(ω)$ vs. $X(e^{jω})$

Most people learn (or try) DSP from this book

  Oppenheim, and Schaefer, (1999). Discrete-time signal processing. Pearson Education.

and there $\Omega$ is used for continuous time and $\omega$ is for discrete time, which is how they keep their notation consistent. Usually both are continuous, which can lead to some confusion when you consider the DFT and/or Fourier series.

In the Olden Days, Yea DSP was primarily practiced by people who had a background in continuous time signals derived from some physical problem, and notational conventions for $\omega$ were typically $2\pi f$ (in yonder rads/sec).

Most books back then, tended to start with some general Linear Systems theory, and then go to sampling.  I think the most recent version of Oppenheim and Schaefer wait until Chapter 4 to cover sampling.

People who taught and wrote about DSP, recognized that including a sample rate wasn't really necessary.  My first DSP class was taught by a guy who did adaptive beamforming and he introduced the concept of unity sampling, $f_s=1 \text{ sample/second}$ so all our filter frequency responses were plotted between -1/2 to 1/2 Hz.  You just needed to scale everything by your actual sample rate when you got back to your physical problem.

It became clearer over time that there were time series, like Stock Market time series that were inherently discrete in time and a lot of DSP didn't really require reference to time, and things like image processing really didn't either, so to make things more general, $\omega$ as radians seems to have become the most common notation. You can also sell your books to finance majors now.

The notation $X(e^{j\omega})$ is handy when you plug $e^{j\omega}$ into a Z transform (if the unit circle is in the region of convergence), and reinforces the concept that a function or composition of a periodic function $e^{j\omega}$, is periodic.

The notation $X(\omega)$ needs to be interpreted in the context of where it is, could be discrete time, could be continuous time.

So, the answer to your question is: Context.

As an American English speaker, it would be natural for me to assume that this is the complete story but someone who learned from the Russian (or French, or ....) DSP literature would most likely have a different explanation.


Cross correlation to compute delay between two lines does not work

If you plot the cross correlation instead of taking the maximum, then I expect you'd see the problem.

The cause is that your signals aren't centered around zero.  They have an offset.  The cross correlation of two signals with an offset is a kind of triangle looking thing with the peak at zero.

As far as the cross correlation is concerned, your two signal are identical except for the offset - and you normally use a high pass filter or subtract the average of the data points to remove the offset.

Your result is correct and to be expected, it's just not what you wanted.


frequency spectrum of the conjugate of $x(t)$

If $X(f)$ is the Fourier transform of $x(t)$, then the Fourier transform of $x^*(t)$ is given by $X^*(-f)$. That's why for real signals $X(f)=X^*(-f)$ holds.


Why is the ROC of Laplace transform independent of imaginary part of s?

I think your misunderstanding is that you manipulate an expression, which is only valid for $\text{Re}\{s\}>0$, in order to show for which values of $s$ it might be valid.

Note that the integral

$$\mathcal{L}\{u(t)\}=\int_0^{\infty}e^{-st}dt$$

only converges for $\text{Re}\{s\}>0$, so the result $1/s$ does not make any sense for other values of $s$.

Only the real part of $s$ in the term $e^{-st}$ inside the integral can provide the necessary damping such that the integrand decays sufficiently fast for the integral to converge. That's why the region of convergence only depends on the real part of $s$.


I will try to be intuitive more than rigorous.
Low Pass Filter means a filter which limits the ability of the output data to change fast (Relative to a parameter called the Low Pass Cut Off Frequency).
Namely it make the signal transient to take longer (Smoother).

How do we do that?
We, make adjacent samples have similar values.

So for applying a Moving Mean Filter (Output is the mean value of few adjacent samples) means we applied a Low Pass Filter.

Another way is Auto Regressive model, which you pointed.
In that mode the output sample is the previous samples with a small addition which again, with proper ratio between the weight of the previous output to the new input means the speed of changes is limited.

This is the intuition.
If your needs are basic this might work (Just apply some weighing with non negative values, Let's say Gaussian Filter).
If you need more advance control you'll need to learn the frequency Representation of the sampled data.


Time-frequency representation of sound signal using Matlab

I believe that this "color graph" you are looking for is a spectrogram (although it looks to me more like a scalogram, but you did not mentioned wavelets). Let me give you an example in MATLAB of obtaining such plot:

load handel
nfft = 512;
noverlap = 128;
win = hamming(nfft);
spectrogram(y, win, noverlap, nfft, Fs, 'yaxis')
colormap('jet')

So first line is loading of some standard test recording, it gives you two variables: Fs (sampling frequency) and y (your signal). Next you define length of your analysis: nfft in your Short Time Fourier Transform and by how many samples you want to shift your analysis window. When defining window you can use variety of them (simply check in MATLAB help), I've used most common one: Hamming window. Last line is for setting colour representation.

Computation of spectrogram is very straightforward, please refer to help for more details: click. The last argument 'yaxis' tells MATLAB to use horizontal time axis and vertical for frequency.

Having that you can play with length of your analysis window and the overlap. You must though understand one, most important thing: better resolution in time (short window) yields poorer resolution in frequency domain and vice versa - that's a trade-off.

In the end you should get something like:

EDIT:

Because you would like to obtain the scalogram, here's how to do it in MATLAB. Most important thing - you must have Wavelet Toolbox installed!

% Definition of signal consisting of two sinusoids
f1 = 10;
f2 = 40;
Fs = 1000;
t  = 0:1/Fs:1;
y =  sin(2*pi*t*f1) + sin(2*pi*t*f2);

wname = 'morl'; % Choosing the wavelet
scales = 1:1:128; % Defining scales
coefs = cwt(y, scales, wname); % Get wavelet coefficients

wscalogram('image', coefs, 'scales', scales, 'ydata', y); % Get the scalogram, together with time domain signal overlayed
colormap(jet) % Set the colormap

As a result you will get something like:


Calculating the SNR of Audio Signal (Recommended Libraries)

It is really hard to create something to estimate the SNR in all cases. Each case requires its own treatment according to model of the acquisition of the signal.

For instance, if the model is AWGN noise an the audio is of human voice and data is samples in high sample rate (Let's say above 44.1 [KHz]) then you can use a lot of the bins in the DFT of the signal to estimate the Noise STD.
Then according to the energy in the bins of the data you can estimate the SNR.

Regarding a library, you might find general tools in Mozilla's Deep Speech library.


Document image vanishing point detection and evaluation

why use OCR techniques that are sensitive to vanishing points?

projective distortion just skews the letters- right?  So, why not just model those expected skews into the OCR templates for each letter?  Also, the nearby letters would skew similarly, so you could probably use correlated parallel lines among neighboring letters to estimate the amount of local/global skew (and avoid dealing with VPs).

also, I assume you mean the phone will take a pic of the document, so don't you also have to worry about the paper not being flat and having curve linear distortions too, which seems to me would completely confound, and supersede any vanishing point issue you are worried about.


I think linear predictive coding (otherwise known as an auto-regressive moving average) is what you are looking for.  LPC extrapolates a time series by first fitting a linear model to the time series, in which each sample is assumed to be a linear combination of previous samples.  After fitting this model to the existing time series, it can be run forward to extrapolate further values while maintaining a stationary(?) power spectrum.

Here is a little example in Matlab, using the lpc function to estimate the LPC coefficients.

N = 150;    % Order of LPC auto-regressive model
P = 500;    % Number of samples in the extrapolated time series
M = 150;    % Point at which to start predicting

t = 1:P;

x = 5*sin(t/3.7+.3)+3*sin(t/1.3+.1)+2*sin(t/34.7+.7); %This is the measured signal

a = lpc(x, N);

y = zeros(1, P);

% fill in the known part of the time series
y(1:M) = x(1:M);

% in reality, you would use `filter` instead of the for-loop
for ii=(M+1):P
    y(ii) = -sum(a(2:end) .* y((ii-1):-1:(ii-N)));
end

plot(t, x, t, y);
l = line(M*[1 1], get(gca, 'ylim'));
set(l, 'color', [0,0,0]);
legend('actual signal', 'extrapolated signal', 'start of extrapolation');

Of course, in real code you would use filter to implement the extrapolation, by using the LPC coefficients a as an IIR filter and pre-loading the known timeseries values into the filter state; something like this:

% Run the initial timeseries through the filter to get the filter state
[~, zf] = filter(-[0 a(2:end)], 1, x(1:M));

% Now use the filter as an IIR to extrapolate
y((M+1):P) = filter([0 0], -a, zeros(1, P-M), zf);

Here is the output:

It does a reasonable job, though the prediction dies off with time for some reason.

I don't actually know much about AR models and would also be curious to learn more.

--

EDIT: @china and @Emre are right, the Burg method appears to work much better than LPC.  Simply by changing lpc to arburg in the above code yields the following results:

The code is available here: https://gist.github.com/2843661


Centre channel derivation using correlation

Audio up-mixers are fairly complicated and the "better" ones are either protected IP or trade secrets.
To extract a center channel you would look at the correlation between the left and right input channels and dynamically steer the correlated content towards the center and the uncorrelated towards left and right. The devil is in the details though: it's tricky to do this without creating undesirable steering artifacts. There are lot of implementation issues tow work out: time constants, energy detectors details, correlation metrics, steering curves, frequency band splicing, etc.
Here is on old patent you can plow through if you are interested: https://www.freepatentsonline.com/7016501.pdf


How to estimate the frequency at certain time of linear sine sweep?

Frequency is the first derivative of the phase so in this case that's f0+(f0-f1)*t/T. If you plug in the numbers you get 110Hz, which is what your measurement shows. Not sure how you got 65Hz for that ??

It's intuitively clear as well. The sweep starts at 20 Hz and ends at 200 Hz after 60s. After 30s you should be at the half way point between 20 and 200, which is 110 Hz.

$$\varphi (t) = 2 \pi \cdot (f_{0}+\frac{f_{1}-f_{0}}{2T}\cdot t)\cdot t = 2 \pi \cdot (f_{0}\cdot t+\frac{f_{1} -f_{0}}{2T}\cdot t^{2})$$

$$f(t)=\frac{1}{2 \pi}\cdot \frac{\partial \phi}{\partial t}=f_{0}+\frac{f_{1}-f_{0}}{T}t$$


Analytical Expression for Convolution of Two 2D DFTs

Convert your PSF to OTF. Then, multiply your OTF by the image (in which it should be in the frequency domain as well as your OTF).


How to measure a lung nodule in a CT Scan DICOM image?

Aside from the whole discussion of pure signal processing:
What exactly do you define as "the nodule". This is usually a biological entity with difficult to define borders. The nature of the nodule is sometimes invasive growth and therefore even in histological sections poorly defined. The CT itself has therefore a threshold higher than histology and therefore the true border of a nodule cannot defined easily. On the other hand, healthy tissue around the nodule might be compressed and appear dense in CT. This might be depending on the respiration phase the scan was triggert on (best results in midinspiration, according to newer data). Or inflammation might obscure the true border of the nodule.

Another aspect is interpolation of the images. The technique is usually a spiral-CT, so you don´t miss any lesions. The sections are therefore calculated. This results less defined borders of objects. If the focus would be on border detection or near histology resolution, high-resolution scans are usually required. This is what you do for interstitial lung disease.
Unfortunately, they slice through the lung with a pretty large distance in between. Radiation dose would be extremely high, if you try to get a "full volume scan". But in this type of scan, you have to be aware, that You´s miss lesions between slices.

To come back to the initial question: I think you will have to validate your technique - whatever it is - with the gold standard. Which is histology sections. (Unfortunately, the lung isn´t easily sectioned...). Another option is an additional technique: like PET-CT (combining positron emission tomography with computer tomography), but the alignment algorithms are sometimes tricky.


You need to add another criteria for the tie breaking.

Maybe how coherent each threshold area is after the we apply the threshold.

So for example lets say you have a tie between thresholds T1 and T2. You can apply T1 and than find the number of connected component C1 it gave you than apply T2 and find the number of connected component C2 it gave you and than decide to take the threshold that gave less connected components.


I'm reading a blog about extracting MFCCs features for Machine Learning applications, but I didn't understand the following points about the mean normalization:

Do not read random blog posts, read books instead. At least they are reviewed by the publisher.

  What does he mean by "balance the spectrum and improve the Signal-to-Noise (SNR)"?

Not spectrum is balanced but features. Signal to noise ratio remains the same after this operation.

  Before feeding the MFCCs features to my neural network, I'm doing min-max normalization for the whole features in the dataset instead of normalizing each file alone (like what the author does above), and I'm not sure which method gives more accurate results.

Per-file normalization is often better if per-band amplitude of files in the database is different. At the same time if files are small per-file estimate might be inaccurate, then per-dataset normalization might be better.

  Why did he add the epsilon value 1e-8, and didn't divide by the standard deviation?

He just does mean normalization, not variance normalization, so he doesn't divide. As per 1e-8, it is totally meaningless.


Why use windowing function instead of truncating the signal to integer amount of periods?

When an integer number of periods is known, truncation when followed by FFT (assuming periodicity) is equivalent to windowing it with a uniform window. Using another window may induce some leakage.

But, if your signal was sampled this way, this either means that you were very lucky, or that you already have a lot of knowledge about your signal. And thus that you have dmaller need for Fourier analysis.

The point of using specific windows is indeed to help analysis when periods are unknown.


How can I know if these two signal are orthogonal to each other

The inner product of two continuous-time signals is defined by

$$\langle\mathbf{s}_1,\mathbf{s}_2\rangle=\int_{-\infty}^{\infty}s_1(t)s^*_2(t)dt\tag{1}$$

where $*$ denotes complex conjugation. In your example the conjugation is irrelevant because both signals are real-valued. So you just need to multiply the signals and determine the area of the result.

With $(1)$ it should be easy for you to show that the two given signals are indeed orthogonal.


How to detect algebraic loop in a system

It's not that loops end up adding things recursively. Think of them like wired connections. Loops in wired connections don't lead to recursive loops do they? Now, at each adder the sum of all that's incoming and outgoing must be equal, much like Kirchhoff's current law. This leads to a linear system of equations. The solution of which tells you in what state the system settles.

Let's consider the case that both $H$ and $G$ are completely memoryless so that the output of $G$ for an input $x[n]$ is $G \cdot x[n]$. Further, let's call the outputs of the two adders $s_1[n]$ and $s_2[n]$. Then, your network layout leads to the following equations

$$\begin{align}
s_1[n] & = x[n] + G \cdot s_2[n] \quad \mbox{(first adder)} \\
s_2[n] & = H \cdot s_1[n] + G \cdot y[n] \quad \mbox{(second adder)}  \\
y[n] & = H^2 \cdot s_1[n] \quad \mbox{(output)}
\end{align}$$

That's a linear system of equations. For every $n$, you are given $x[n]$ (your input), your unknowns are the output $y[n]$ and the two intermediate variables $s_1[n]$ and $s_2[n]$. Solving the three equations with three unknowns gives you the value of $y[n]$ to expect. Since there is no memory here, you can do this for each $n$ independently.

The $z$-transform helps you to do this easily even in the case where there is memory.


What is limit on performance of edge-preserving filters for low signal-to-noise regimes?

This is Gibb's phenomenon (just not for the common edge case).

Hence, with a linear filter, the better you remove noise (which is, low-pass filtering), the more you remove the edges (which inherently have energy everywhere in the spectrum).

There's no minimum case – the noise bandwidth you allow is always the same as the signal bandwidth you allow, and thus, complementary, the amount of signal energy (and hence, edge precision) you cut off.

Among the things you can do is apply a clever signal model. Does your signal always look like your $x(t)$? Great, so you only need to estimate one parameter ($\alpha$) to fully describe it!


Why do these 2 methods give different solutions?

The problem is that you took the derivative of the function

$$\hat{x}_u(t)=2e^{-3t}-e^{-4t}\tag{1}$$

whereas using the Laplace transform you implicitly assumed that $x_u(t)$ equals zero for $t<0$:

$$x_u(t)=\hat{x}_u(t)u(t)=(2e^{-3t}-e^{-4t})u(t)\tag{2}$$

where $u(t)$ is the unit step function.

If you take the derivative of $(2)$ then you'll get the same result as with the Laplace transform, taking into account that the derivative of the step function $u(t)$ is the Dirac delta impulse $\delta(t)$:

$$\frac{dx_u(t)}{dt}=\frac{d\hat{x}_u(t)}{dt}u(t)+\hat{x}_u(t)\delta(t)=\frac{d\hat{x}_u(t)}{dt}u(t)+\hat{x}_u(0)\delta(t)\tag{3}$$

where I've used the fact that for any function $f(t)$ that is continuous at $t=0$ we have $f(t)\delta(t)=f(0)\delta(t)$. I trust that you can take it from here.


How can I make a mmWave frequency selective channel in matlab?

Your code appears to be summing up amplitudes over all the paths to simulate Rayleigh (flat) fading.  The assumption of flat fading is that the delay spread of the channel is much less than the symbol period, so all the symbols add up roughly at the same time (1).

To make the leap to frequency-selective fading, you need to account for the pulse shape in your summation, i.e. you need to have some vector of sample indices n, a function p(n) that returns the pulse amplitudes at times n*Ts, and multipath delay ndelay. Something like this (pseudo-MATLAB code):

n = 0:maxn;
Npath = 4;
ndelay = [2, 5, 3, 7]; % length(ndelay) == Npath
sumval = 0;
for i = 1:Npath
    sumval = sumval + otherstuff*p(n - ndelay(i));
end

You must specify ndelay ahead of time manually or generate it according to some probability distribution.

References:
(1) Goldsmith, Wireless Communications, chapters 2-3, 2005.


Numerically generating noise time series from spectral density

As mentioned by @MBaz in comments, you could use the fact that filtering white noise with unitary power spectral density (for example from independent Gaussian samples zero mean and unitary standard deviation) with a filter having a frequency response $H(f)$ gives you colored noise with power spectral density $S_{xx}(f) = |H(f)|^2$.

Then comes the question of designing such a filter. A flexible approach would be to use a Frequency sampling-based FIR filter design. Note that this is readily available as FrequencySamplingFilterKernel in Mathematica (or similarly fir2 in matlab for future readers), using an evenly spaced sampling of $|H(f)| = \sqrt{S_{xx}(f)}$ up to the Nyquist frequency (i.e. $f_{max}$ which would correspond to half your time-domain sampling rate). Once you have the FIR filter coefficients, you can use those to filter your white noise with ListConvolve.

Note that the steeper the slopes in your power spectral density, the more frequency samples (resulting in a higher number of FIR filter coefficients) you'll need to get an accurate representation of your power spectral density.


What does Smoothness Prior mean in the context of image processing?

A prior is something that you know a priori (beforehand) about how the system you're looking at works.

A smoothness prior means that you know that the image you're looking at will be smooth in a particular way.

That paper goes on to say:

  The term diffusion comes from the idea of propagating local information with smoothness constraints, by analogy with physical phenomena like heat propagation in physical structures. These phenomena can be formalized with PDEs, and diffusion is therefore performed using PDE-based regularization.

So in this case the smoothness that you know about beforehand is enforced using a partial differential equation.


If you can demodulate then it's probably correct. Perhaps it's just the fact that the carrier is not very fast that it looks weird. Can you increase the symbol period and the carrier frequency. Otherwise, my guess is that you are not convolving correctly. Just double-check if your RC has the correct width (or zero crossings).
For reference I generated this:

Pulse-shaped BPSK with rect() through convolution:

Modulated by multiplying with the carrier tone:

Now with the raised-cosine pulse shape:

Note that by sampling at the correct times (each symbol period), we can recover the BPSK sequence perfectly.

Now modulated:


Complex numbers in a frequency domain of a 2D image

The real and imaginary plane are also the symmetric and anti-symmetric planes.  Image content that is symmetric around the center lines will show up in the real plane.  Image content that is opposite in brightness across the center lines will appear in the imaginary plane.  Data that is neither or some combination of both will end up in both, with the phase angle indicating the mix or a shift (or rotation wrapping around the edges) from symmetric to anti-symmetric.


Can a pole at the origin be considered a dominant pole?

You could get away with defining it as such (or extending the concept to the pole with the most positive real part).

But typically the concept of a dominant pole is applied to systems that are truly stable, and thus actually settle out.  A system with singular poles on the imaginary axis (and no poles in the right-half plane) is considered "metastable", because with the right input it has a bounded response, but with the wrong input its response is unbounded.


Root raised cosine pulse shaping peak value

You can derive an upper bound for the peak value of the output signal depending on the filter's impulse response. The output is given by

$$y[n]=\sum_{k=0}^{N-1}h[k]x[n-k]\tag{1}$$

where $h[k]$ is the length $N$ impulse response, and $x[k]$ is the input signal. From $(1)$ we have

$$|y[n]|=\left|\sum_{k=0}^{N-1}h[k]x[n-k]\right|\le\sum_{k=0}^{N-1}|h[k]||x[n-k]|\le |x[n]|_{\text{max}}\sum_{k=0}^{N-1}|h[k]|\tag{2}$$

Since in your case $|x[n]|_{\text{max}}=1$, the magnitude of the peak value of the output signal is guaranteed to be smaller than the sum of the absolute value of all filter coefficients.

However, since your input signal is upsampled by a factor $L$ (i.e., it has $L-1$ zeros between two signal values), we can come up with an improved (i.e., tighter) bound:

$$|y[n]|\le\max_{l=0,1,\ldots L-1}\sum_{k=0}^{\lfloor \frac{N-1}{L}\rfloor}|h[l+kL]|$$

because in the sums $(1)$ and $(2)$ only every $L^{th}$ filter coefficient is multiplied by a non-zero input value.

The following Matlab/Octave script shows the computation of the bound:

h = randn(50,1);    % some impulse response
x = randi([-1,1],1,20); % input signal
L = 10;         % interpolation factor
x = [x;zeros(L-1,20)]; x=x(:);  % interpolate x
y = filter(h,1,x);  % compute output signal
p = max(abs(y));    % peak value

% compute bound
b = zeros(L,1);
for i = 1:L, b(i) = sum(abs(h(i:L:end))); end
b = max(b);

% show peak value and bound
[p,b]

A few runs of the script show that the bound is pretty tight.


Filter design with zero - pole placement method

Here let me show you a simple procedure very similar to pole zero placement which will be helpful for your notch filter design.

First, lets analyse the frequency response of a single zero and let
$$ H(z) = 1 - b z^{-1} $$
be a first order system with a single zero at $z = b$ where $b$ is a complex constant with a radius $r$ and phase angle $\phi$ radians; i.e.,
$$ b = r e^{j\phi} $$

Lets see this zero on the z-plane and the corresponding frequency response magnitude for the values of $r = 0.9$ and $\phi = \pi/4$ radians:

Note that since there is a single zero, this is a non-symmetric frequency response (due to a complex impulse response $h[n]$). To get rid of this non-symmetric frequency response, we shall force the impulse response to be real and the way to achieve this is to add a second zero at the complex-conjugate location resulting in the filter:
$$ H_{az}(z) = (1 - bz^{-1})(1 - b^{*} z^{-1}) $$
as the second figure shows:

Now we have a pair of complex-conjugate zeros, whose frequency response is conjugate symmetric (and therefore the magnitude response is even symmetric as displayed)

next, we display the same figures, for a pair of complex-conjugate poles with a radius of $r=0.85$ and the same angle $\phi = \pi/4$ radians:
$$ H_{ap}(z) = \frac{1}{(1 - az^{-1})(1 - a^{*} z^{-1})} $$

So far we have displayed the pole-zero locations and corresponding frequency responses for individual pairs of poles and zeros at the same frequency. Lets combine them into a single filter and display the result:
$$ H(z) = \frac{(1 - bz^{-1})(1 - b^{*} z^{-1})}{(1 - az^{-1})(1 - a^{*} z^{-1})} $$

Nice! We have reached a system whose frequency response now resembles that of a notch, but a weak one? The solution comes by recognizing that we should better put the zero on the unit circle (i.e., set its radius $r=1$) for an infinetely deep nulling at that frequency;

Now this system is what can be called as a 2nd order notch filter with a pole radius of $r=0.85$ and radian frequency of $\omega = \pi/4$ radians per sample. This is a quite satisfactory notch filter.

Finally lets display a much sharper notch by moving the pole closer to the zero (closer to the unit circle) by setting its radius to $r = 0.99$. But note that it's very dangerous for a pole to wander about unit circle, as it can easily fall over it, thereby making the system unstable...

For your convenience, choose the pole radius between 0.9 and 0.99 depending on your numerical accuracy and how sharp a notch you need. And adjust the frequency of the pole-zero pair according to which frequency you want to null.


MATLAB's $\tt filter()$ function for complex valued data

As @msm said, there's no difference. You can refer to Matlab's documentation here: https://www.mathworks.com/help/matlab/ref/filter.html#inputarg_x which says "Complex Number Support: Yes".


Can I apply Fourier Transform to a non-time-indexed signal?

Yes you can. The unit of the "frequency" axis after the transform will be $m^{-1}$, and is known as spatial frequency.

For example, if there is a strong peak in the Fourier transform at $3.10^{-4} m^{-1}$, it means that your original curve exhibits a strong pattern that repeats at a scale of every $3.3km$, and from that you could infer that maybe the signal was recorded from a vehicle doing laps at the Monaco Grand Prix. The harmonics of this spatial frequency would contain a "signature" of the shape of the circuit.

A practical application of this is handwriting recognition - looking at shapes in the Fourier domain yields representations invariant to scaling, rotations, or more robust to deformations than the original data.


Instantaneous usually means the value of something right at this time. I think what this means in your case is that the function $f(t)$ is complex and you are taking the argument (finding the angle):
$$
\theta(t) = \arg\left( f(t) \right) = {\rm atan}(\Im[f(t)],\Re[f(t)])
$$
where $\Im[ \cdot ]$ indicates taking the imaginary part and $\Re[ \cdot ]$ indicates taking the real part.


Mean delay for a M tap harmonically weighted filter

The coefficients of this FIR filter do not posses any type of symmetry, hence it cannot be a linear phase filter. However, you can still compute its phase response either analytically (or numerically) as:

$$\phi(\omega) = \tan^{-1}( \frac{ \mathcal{Im} \{ H(e^{j\omega}) \} }{\mathcal{Re} \{ H(e^{j\omega}) \} } ) $$

Then you can compute its derivative (again either analytically or numerically) to obtain the group delay function

$$ \tau(\omega) = -\frac{d\phi(\omega)}{d\omega} $$

from which you can compute the mean delay according to its definition.


Algorithm to estimate coherence when measuring a transfer function with a swept sine

For each frequency, divide the measurement time into chunks.  For each chunk, do your estimation of the amplitude at that frequency in both the input and output channels.  Then compute the coherence for this ensemble of measurements at each frequency.

For example, if you measure at each frequency for Ncyc cycles, then divide these into Navg measurements each of Ncyc/Navg cycles.

A nice reference on this subject is Bendat and Piersol.


SNR estimation: signal with unknown amplitude and Gaussian noise

A common technique relies on filter banks and robust statistics. The idea is to isolate some frequency subbands where the transformed signal is sparse, and the rest is (filtered) noise. From here, you can use a median estimator.

In the context of orthogonal wavelets (a form of dyadic filter banks), if the signal is sufficiently sampled, the highest frequency subband often satisfies the above requirements. Or one can choose a suitable union of subbands where the signal is considered sparse.

Then, if $c_i$ denote the subband coefficients, an estimator of the Gaussian noise variance $\sigma$ is:

$$ \hat{\sigma} = \frac{\textrm{median}|c_i|}{0.6745}\,.$$

The following picture displays the average noise estimation, with standard deviation, from several realizations of a Gaussian noise added to a line of an image.

This is described in Penalized threshold for wavelet 1-D or 2-D de-noising. More details are given in S. Mallat, A wavelet tour of signal processing, section 11.3 Thresholding Sparse Representations,
Noise Variance Estimation. The denominator factor is discussed in Rousseeuw and Croux, Alternatives to median absolute deviation, 1993.

A very basic avatar consists in differentiating the data with the basic $[1,-1]$ finite difference, and using the above estimator, further divided by $\sqrt{2}$ to account for its energy normalization.


How is this was derived? DTFT for symmetric pulse

First, the following trick will be useful:
$$ e^{j a} - e^{j b} =  e^{j \frac{a+b}{2}} \left( e^{j \frac{a-b}{2}} - e^{j \frac{b-a}{2}} \right) $$

Then simply expand the sum as:
$$ X(e^{j\omega}) = \sum_{n=-N}^{N}e^{-j\omega n} = \frac{ e^{-j\omega (-N)} - e^{-j\omega (N+1)} }{1 - e^{-j\omega}} = \frac{ e^{j\omega N} - e^{-j\omega (N+1)} }{1 - e^{-j\omega}} $$

Apply the trick to the numerator and denominator by recognising $a$ and $b$ carrefully: for the numerator $a=\omega N$ and $b=-\omega (N+1)$ whereas for the denominator $a=0$ and $b=-\omega$ which yields:

$$ X(e^{j\omega}) = \frac{ e^{j\omega N} - e^{-j\omega (N+1)} }{1 - e^{-j\omega}} = \frac{ e^{j \frac{\omega N -\omega (N+1) }{2}} \left( e^{j\frac{\omega N +\omega (N+1) }{2}} - e^{-j\frac{\omega N +\omega (N+1) }{2}} \right) }{ e^{-j\omega/2} \left( e^{j\omega/2}  - e^{-j\omega/2} \right) }$$

which simplifies to:
$$ X(e^{j\omega}) = \frac{ e^{-j\omega/2}  \left( e^{j\frac{\omega N +\omega (N+1) }{2}} - e^{-j\frac{\omega N +\omega (N+1) }{2}} \right) }{ e^{-j\omega/2} \left( e^{j\omega/2}  - e^{-j\omega/2} \right) } = \frac{  e^{j \omega \frac{2N+1 }{2}} - e^{-j \omega \frac{2N+1}{2}} }{   e^{j\omega/2}  - e^{-j\omega/2}  }  $$

which is:
$$ X(e^{j\omega}) = \frac{  \sin( \omega (N+\frac{1}{2}) )}{   \sin( \omega \frac{1}{2}) }  $$


Is there a simpler way to calculate the amplitude response of the following filter

The given transfer function has a specific form. With the denominator polynomial $A(z)=2+bz^{-2}$, you can write the transfer function as

$$H(z)=\frac{z^{-2}}{2}\frac{A\left(\frac{1}{z}\right)}{A(z)}\tag{1}$$

Since for $z=e^{j\omega}$ (i.e., on the unit circle, where we evaluate the frequency response), we have $|A(z)|=|A(z^{-1})|$, the magnitude of $H(e^{j\omega})$ is given by

$$\left|H(e^{j\omega})\right|=\frac12\tag{2}$$

The given filter is a (scaled) all-pass filter. It's easy to recognize all-pass filters by noting that their numerator and denominator polynomials have mirrored coefficients.