Since you have a measurement microphone, it should come with a manual that contains all necessary information to characterise it (if not with all this information in digital format too).

One crucial piece of information is its Sensitivity which is expressed in V/Pa (Volts per Pascal), at a specific frequency. This figure allows you to convert VOLTAGE (which is what you are measuring at the output of the mic) to PRESSURE (which is what the transducer responds to).

The sensitivity of the microphone, in combination with its frequency response will tell you how much VOLTAGE does the microphone produce for a reference PRESSURE level at a given frequency.

So, this mic states 36mV/Pa and is relatively flat from 10Hz to 10kHz. This means that a sound wave of Sound Pressure Level (SPL) of 1 Pascal falling on the mic's transducer will cause a 36mV signal to be generated at its output at any frequency between 10Hz and 10kHz.

Outside of these ranges, the curve is slightly lowered which means that the sound wave's SPL will still be 1Pa, but the output of the mic will not be 36mV but 36mV times the factor that is obtained from the frequency response.

So, now we have a way of converting between SOUND PRESSURE LEVEL (in Pascal) and VOLTAGE. What about what is hapenning in the sound card?

The sound card will have a pre-amplifier / amplifier to amplify the signal from the microphone and this amplifier will have a frequency response on its own which will modulate the frequency response of the mic. In other words, because of its construction, the amplifier will amplify different frequencies by slightly different factors, just as the microphone did.

To cut a long story short: A sound card's LINE level is 1.736V. The amplifier of the sound card is set to 1 and you are recording with 16bit resolution. This means that the positive peak of 1.736V is converted to the digital value of $2^{15}=32768$.

What value does 36mV produces? $\frac{36 \cdot 10^-3 V}{1.736 V} \cdot 32768 = 680$

Right, so when the WAV file reads a sample of 680 at an amplification level of 1, then we know that the microphone was producing 36mV and through its sensitivity we can infer that 1Pa of sound pressure level was falling on it.

So, to convert between your sample value and the sound pressure level that caused this sample value you need:

The microphone's sensitivity
The microphone's frequency response
The sound card's LINE IN Spec
The sound card's amplification level
The wordlength that was used during sampling.

Hope this helps.


Testing for Linearity and Shift-Invariance

Your demonstration of shift-invariance is not correct.
You time-shift the input signal by $n_0$, which is correct. But you also time-shift the sinusoidal, and this only happens if you were time-shifting the output.
The description of the system contains a signal that explicitly depends on time ($\sin(2\pi/7 n \cdots)$). In general this is a red-flag for non time-invariance.


Questions on Weighted Least Square Estimation

The linear model is $b=Ax+r$, so you can see the errors $r$ as the noise added to the measurements $b$.
The covariance $\Sigma_b=E\{(b-\bar{b})(b-\bar{b})^T\}=E\{(b-Ax)(b-Ax)^T\}=E\{rr^T\}$
$E\{rr^T\} \neq E\{bb^T\}$ unless $Ax=0$


Given a continuous time signal, does the minimum Nyquist sampling rate depend on the choice of the set of basis functions?

In the most general case, if you want to sample a continuous-time signal without loss of information, the minimum sampling rate is independent of any choice basis functions. The faster the signal changes with respect to the independent variable (which doesn't need to be time), the faster you have to sample. And if the signal is not band-limited or if it can't be parameterized in some other way, sampling will be lossy.

Note, however, that the (Nyquist-Shannon) sampling theorem is only a sufficient criterion for sampling a continuous-time signal without loss of information. In general you have to choose the sampling frequency greater than twice the highest frequency component in the signal. However, under certain conditions you can get away with sampling at a lower rate.

One example are bandpass signals, which can often be sampled at rates much lower than their highest frequency, especially if their bandwidth is small compared to their center frequency. But note that it is generally not sufficient to sample at a rate equal to twice the signal's bandwidth. See this answer for more details on bandpass sampling.

Another special case are sparse signals. The sparsity can be exploited to represent these signals with fewer samples than required by the sampling theorem. The corresponding technique is called compressed sensing.

In general, if a given signal has a "finite rate of innovation (FRI)" (such signals are called "FRI-signals"), they can be sampled at or above their rate of innovation. This subject treated in this important paper.

I also recommend reading this overview paper: Sampling - 50 Years After Shannon.


Libraries for Voice Activity Detection (Not Speech Recognition)

In my answer to your that question, I had mentioned that Voice Activity Detection is a standard feature for codecs like G.729 and such others.

You should look for reference encoders and decoders for algorithms that applies this.

One such example is - http://www.voiceage.com/openinit_g729.php

Another possible source is Speex codec. Which implements VAD

BTW: You should google "Voice Activity Detection" or "Talk Spurt" rather than "Speech Detection".


Is there any way to get intensity value of a certain wavelength for given pixel of a normal  image?

If I understand correctly - you want to know what wavelength a given RGB value will show?

Only some RGB values represent a pure wavelength, those on the edge of CIE diagram (there is no wavelength of brown for example) see http://www.fourmilab.ch/documents/specrend/ for details and code


Cannot understand the attenuation in signal as the audio bit depth of adc changes

From NI myRIO-1900 User Guide and Specifications:

  The audio inputs are left and right stereo line-level inputs with a
  ±2.5 V full-scale range

  • For Audio In/Out,
  LSB Weight = 5 V ÷ 212 = 1.221 mV
  Maximum Positive Reading = +2047 * 1.221 mV = 2.499 V
  Maximum Negative Reading = -2048 * 1.221 mV = -2.500 V

Your computer's head phone output probably has another constant of proportionality between voltage and numerical value. Perhaps you can find manufacturer's data, but it is probably safer to simply measure with the help of your NI device.


The Mathematical structure of all Transforms can be understood by simply first seeing what a Fourier Series Representation is. In Continuous Time Domain, Fourier established that all periodic waveforms can be expressed as sum of Complex exponentials of some magnitude(which are the coefficients in a Fourier Series). The same can actually be applied for an aperiodic waveform assuming a periodic extension of it. So what the underlying Transform Relationships give us are nothing but the Analysis and Synthesis equations for what frequencies are present in a signal and what are their magnitudes. Rather, you can view these magnitudes as the Projections of the signal on the complex exponentials as to trying to figure out how much of a particular frequency is present in a signal. (You might want to remember the Analysis and Synthesis equations to see what I've said here).

Now for a certain set of functions(L2 functions or rather Limited Energy functions $\int_{-\infty}^{\infty}|x^{2}(t)|dt$ being finite and a set of conditions called Dirchlet's conditions being satisfied) have a special and easily identifiable Analysis equation called the Fourier Transform which comes straight from Fourier Representation of Series coefficients. Now in real world, we rarely encounter functions that violate these properties. If a function were to blow up to infinity in Energy we can never use it anyway.

Caution must be exercised because the Fourier representations involve a lot of intricacies which are rather ignored in daily life problem solving. One might learn a lot by looking at Existence, Measurability, Riemann Integrability etc. of functions and their energies.

Once you've tackled the problem of continuous time(both in terms of Series and Transforms) Discrete is just extrapolations right from Nyquist Shannon's sampling theorem. Now what exactly is a digital signal? It is just the sampled version of it. How quickly sampled is what determines the frequency content that you get  in the Sampled version. If you look at the formula of a DFT it is nothing but the Analysis equation of Fourier Series represented in $0,1...N-1$ points which are called "Bins". It is nothing but the Discrete version of the Fourier Series(think of Integral in the Analysis equation being replaced by Summation and the $\exp(-j\omega t)$ term in the transform being written as $exp(-2*pi*j*k/N)$ where k represents the number of points that you have from the sampled signal and $2*pi/N$ is nothing but the analogous term in the series which is the Fundamental Angular Frequency. I hope this explains much about how exactly DFT is able to get the Frequencies from a Time domain wave.

And for the last part, for 2 files to have exactly the same amplitude values at exactly the same points of time, it is impossible for them to differ in their frequency content because the Math behind it sees just the values. Maybe I'm misinterpreting your question or your phrasing is not proper.


SNR Signal to noise ratio of Pseudo random bit sequence (PRBS)

To estimate the Signal to Noise Ratio you can use the normalized correlation coefficient to the reference time-aligned sequence. I detail this approach further at this post here including the relationship between correlation coefficient and SNR. Note that "noise" in this context is any deviation of the received signal from the reference sequence such as multipath effects and band-limiting. To negate those effects modify the reference sequence to also include that or use equalization techniques on the received sequence to match the reference. In many cases these effects are second order and may be neglected.

Noise detection

If you can assume the noise is White, and you can assume the PRBS is a rectangular waveform (with a Sinc function spectrum) then the equivalent 2-sided brick-wall noise bandwidth of the Sinc with nulls at 1/T (where T is the pulse width) is 1/T. If your signal level is sufficiently in far excess of the total noise power (such as 15 dB or more) then you can estimate the signal power from total power received and compute the noise power over that equivalent bandwidth (1/T) using the estimate of the noise floor far from the signal. Pay attention to the resolution bandwidth setting on the spectrum analyzer to then determine power values / Hz (for example if you are using a 1 KHz resolution band width and a pixel displays a power level of -70 dBm, this is -100 dBm/Hz since $10Log10(1000) = 30$ dB.) For lower signal powers you can resolve the signal to noise by knowing the total power is the additive power of signal plus noise (for example if the signal is only 10 dB higher the noise, the total power measured will be 0.4 dB higher than the signal alone given $10Log(1+10^{-10/10})= 0.4$, and for 15 dB higher will be 0.14 dB from  $10Log(1+10^{-15/10}) = 0.14$)


How to add awgn noise to a signal in Fourier domain for given SNR

Since the Fourier transform (and its discrete version) is linear, there is literally no difference in the way you do this in time and frequency domain:

You observe the energy in your signal, set the Gaussian's variance according to the desired SNR, and add the noise. Not a single flop was given about in which domain you were that day!


Can the Hanning Window be represented in the time domain

So what you mean is that you want the continuous-time Hann window instead of the discrete-time window?

$$w_{Hann}(t) = 1 - \cos \left(2\pi \frac{t}{T} \right)$$

is not correct, since it goes to 0 at 0.

Blackman & Tukey give the continuous time window as:

$$D_2(\tau)=\frac 1 2 \left (1+\cos \frac{\pi \tau}{T_m} \right )$$

for $|\tau| < T_m$ and 0 elsewhere.

Wolfram Alpha

https://en.wikipedia.org/wiki/Hann_function

https://archive.org/details/TheMeasurementOfPowerSpectra/page/n58


Estimating exterior camera parameters by matching lines using least square estimations (Lowe, 1991)

I am not aware of an open implementation of this paper. However, there are other methods to do what you like:

One of them is SoftPosIT is one of them. It has many open implementations available: (1), (2), (3). The downside is that one needs to give some sufficiently close initialization.
If you can live with planar models, then you could use cross ratios for identification. Line-sweep gives one such nice algorithm.
Birdal et. al. also give a simple method for extrinsic camera pose estimation from 2d points not assuming that correspondences are known, but for the moment the open implementation is not available.
Line3D++ gives ability to perform line reconstruction from multi-view images: https://github.com/manhofer/Line3Dpp . This doesn't only solve the extrinsic estimation problem, but camera poses should be an intermediate stage in this work. Maybe one could benefit from such reconstruction and can then calibrate the cameras automatically using the available 3D geometry.

There are also many other 2d-3d pose estimation methods (especially for textureless objects), but for your scenario, where simple geometric primitives suffice, I would go for simpler ones, as aforementioned.


Difference: Ergodic channel capacity and spectral efficiency

Loosely speaking:

Capacity is the supremum of data rate that one can send data with arbitrarily small error probability over a given channel;
ergodic capacity is also the supremum of rate with arbitrarily small error probability, but for fading channels, in assuming the fading process is ergodic, as the term "ergodic" suggests;
spectral efficiency is data rate divided by occupied bandwidth.

Therefore,

Ergodic capacity is not capacity. Capacity is not ergodic capacity. They are channel-specific and we do not necessarily know how to achieve them.
Spectral efficiency is system-specific, i.e. we have already had a system, as said MarcusMuller in his comment. We ignore error probability in using this. Nonetheless, a spectral efficiency is usually implicitly linked with an "acceptable" error probability.

In short, they are very different performance indicators with very different assumptions. In academy, you are free to choose any indicator that supports your research.


Motion detection one threshold over dataset

you should filter the data before you use the threshold. Run a low pass filter on the 1D signal (convolution with a Gaussian) and than normalize the data to the range between 0-1 after that i think you will be able to find a global threshold.


How can I tell if an image is part of another image?

This problem can be treated as signal/image detection or image registration. From the image registration perspective, there is a good tutorial that is exactly what you need: Registering an Image Using Normalized Cross-Correlation


When simulating a voltage divider using wave digital filters I get inversed polarity outputs

The problem that you are facing is one of polarity. When the 3-port series adaptor is defined in the wave-domain certain polarities are inherently assumed. If I am not mistaken the polarity of the 3-port series adaptor you are using is the following:

The circuit that you have shown has the following explicit polarities (shown in both Kirchoff- and wave-domain).

This miss-match is not a big deal for linear circuits (just invert the output). However, if you move up to nonlinear circuits polarity is crucial to the correctness of the derived algorithm. To remedy issues related to polarity, one needs to be mindful of the polarities at play and act accordingly. For the above circuit you can for example reverse the polarities of the resistors by placing a 2-port series adaptor, also known as an inverter, to invert the polarity so that it is correct with respect to the polarities as you define them.


Deriving a Kalman Filter Equation for a Linear Gaussian Filtering Model with Non Zero Mean Noise

Since the model is linear taking care of known mean is pretty simple.
Measurement
Since the "Bias" of the measurement is known all you need is to define $ \hat{y}_{k} = {y}_{k} - {m}_{r} $. Then use it as you'd use a measurement with no bias.
Model
The model is used in the prediction step. What you should do is use the regular prediction step and add to it $ {m}_{q} $ after the prediction.
All other steps only include the covariance matrices and are the same.


what is the possible solution for the Intra-class variations poblems?

It boils down to the features.  You have shown some images of the same class. You also need to look at images of the other class(s), and find a set of features that maximize inter-class variance and minimize intra-class variance. What features those might be depends on your data. You need to look at the types of variation that exists within each class, and try to find features, transformations, or distance measures that account for it. It is very hard to tell you anything more specific based on the information you have provided.

Edit:
One thing you can do is post some images of a different class.

Also, are the images aligned? Maybe before you compute the features you should translate all your images so that the white pixels coincide as much as possible.  Then you can define a distance measure between two images as the percentage of white pixels that co-locate, and then use a k-nearest-neighbor classifier.

Some other random thoughts:
Shape context features may be suitable for your problem.

You also might want to search the literature on recognizing hand-written characters.


Confusion in Normalised Bandwidth of a signal

You can take the FFT on any signal after modulation, but would be simplest (fewer samples required) to do it immediately after modulation. With proper upsampling the FFT of your signal prior to upsampling would be identical as long as you used proportionally more points to keep the time length identical. In your example of 100 KHz and 30 MHz, you could take a N point FFT at 100 KHz or a $300 N$ point FFT at 30 MHz and the result should look identical (with just more points interpolated in the FFT due to the upsampling). Given the 100 KHz sampling rate and room for your interpolation filter, I would assume that your GMSK modulated signal BW you are trying to view is less than 70 KHz (as a complex signal)?


Power spectral density of $\left(x(t)\right)^2$?

Since the question has been raised as to whether the hint that I had given to the OP in a comment on the original question was appropriate for a newcomer to signal processing, here goes.

Stripped of extraneous baggage and notation, the question is whether it is possible to determine the value of $E[X^2Y^2]$ straightforwardly where   $X$ and $Y$ are zero-mean unit variance jointly Gaussian random variables with correlation coefficient $\rho$, or is it necessary to resort
to advanced methods or results such as the one cited in the answer by @MattL or generalizations thereof such as Price's Theorem (cf. the
book cited in MattL's answer).  One straightforward method involves
conditional expectations and the result that is known to be a LIE:

$$E[X^2Y^2] = E\big[E[X^2Y^2\mid Y]\big]$$

Sorry, I couldn't resist: LIE is an acronym for the Law of Iterated Expectation which asserts that the expectation of $A$ is the
same as the expectation of the random variable $E[A\mid B]$. Note that $E[A\mid B]$ is a function of $B$, not $A$ as one might naively believe. We already squeezed out all of the randomness in $A$ when we took its
expectation. The LIE is
that the expected value of $E[A\mid B]$, a function of $B$) works out, by a miracle of
modern mathematics, to be the same as the expected value of $A$.

Now, for any choice of real number $y$, given that $Y=y$,

$$E[X^2Y^2\mid Y = y] = E[X^2y^2\mid Y = y]
= y^2 E[X^2\mid Y = y].$$
But, given that $Y = y$, we know that the conditional distribution
of $X$ given $Y=y$ is a Gaussian distribution with mean $\rho y$
and variance $1-\rho^2$. Consequently,
$$E[X^2\mid Y = y] = \operatorname{var}(X\mid Y=y)
+ \left(E[X\mid Y = y]\right)^2 = (1-\rho^2) + (\rho y)^2$$
leading to
$$E[X^2Y^2\mid Y = y] = y^2((1-\rho^2) + (\rho y)^2))
= y^2(1-\rho^2) + \rho^2y^4 $$
giving us that the random variable $E[X^2Y^2\mid Y]$ is just
$Y^2(1-\rho^2) + \rho^2Y^4$. Since $E[Y^2] = 1$ and $E[Y^4] =  3$,
we get that
$$E[X^2Y^2] = E\big[E[X^2Y^2\mid Y]\big] = E[Y^2(1-\rho^2) + \rho^2Y^4]
= (1-\rho^2) + 3\rho^2 = 1 +2\rho^2.$$

Finally, noting that $\rho = e^{-a|t|}$, we can write
$$R_y(t) = 1 + 2e^{-2a|t|}$$
just as has been explained in MattL's answer.


Negative values from High Pass filtering

A high pass filter will remove a large portion to all of the 0 Hz component or DC offset from a signal.  That's similar to subtracting the mean average, which for a normally distributed signal will make around half of the signal negative (since it was below the original average mean).

If you want the mean to be the same after a high pass filter, you could always add it back.

If you don't care about the actual mean value, you could also add an arbitrarily large positive offset to make sure the sum was always positive.


After working through some possible solutions, we figured out it would be easier to design argitrary magnitude and group delay separately in MATLAB:

fdesign.arbmag
fdesign.arbgrpdelay


Quantization step-size for a given signal based on its PDF

You have a uniform input PDF, and the optimal quantizer for a uniform inpu will be a uniform quantizer. Note that the your uniform PDF has two pieces and the interval from $-4$ to $4$ is completely untreated.

Then, assuming a 2-bit (4 level) uniform midrise quantizer with the given PDF your decision intervals $I_k$ and reconstruction levels $y_k$ will be as follows:

$$ x \in I_1 = [-6,-5] \implies y_1 = -5.5 $$
$$ x \in I_2 = [-5,-4] \implies y_2 = -4.5 $$
$$ x \in I_3 = [4,5] \implies y_3 = 4.5 $$
$$ x \in I_4 = [5,6] \implies y_4 = 5.5 $$


Algorithm(s) to mix audio signals without clipping

The physically "correct" way to do this is summing the samples. However when you add two arbitrary samples, the resulting value could be up to twice the maximum value. ... The naive solution here is to divide by N, where N is the number of channels being mixed.

That's not the "naive" solution, its the only solution.  That's what every analog and digital mixer does, because it's what the air does, and it's what your brain does.

Unfortunately, this appears to be a common misconception, as demonstrated by these other incorrect non-linear "mixing" (distortion) algorithms:

Mixing digital audio (the wrong way)
A quick-and-dirty audio sample mixing technique to avoid clipping (don't do this)

The "dividing by N" is called headroom; the extra room for peaks that's allocated above the RMS level of the waveform.  The amount of headroom required for a signal is determined by the signal's crest factor.  (Misunderstanding of digital signal levels and headroom is probably partially to blame for the Loudness war and Elephunk.)

In analog hardware, the headroom is maybe 20 dB.  In a hardware DSP, fixed-point is often used, with a fixed headroom; AD's SigmaDSP, for instance, has 24 dB of headroom.  In computer software, the audio processing is usually performed in 32 bit floating point, so the headroom is enormous.

Ideally, you wouldn't need to divide by N at all, you'd just sum the signals together, because your signals wouldn't be generated at 0 dBFS in the first place.

Note that most signals are not correlated to each other, anyway, so it's uncommon for all the channels of a mixer to constructively interfere at the same moment.  Yes, mixing 10 identical, in-phase sine waves would increase the peak level by 10 times (20 dB), but mixing 10 non-coherent noise sources will only increase the peak level by 3.2 times (10 dB).  For real signals, the value will be between these extremes.

In order to get the mixed signal out of a DAC without clipping, you simply reduce the gain of the mix.  If you want to keep the RMS level of the mix high without hard clipping, you will need to apply some type of compression to limit the peaks of the waveform, but this is not part of mixing, it's a separate step.  You mix first, with plenty of headroom, and then put it through dynamic range compression later, if desired.


Contour sharpening: Optimal direction for derivation

Yes, you are correct.

The direction of the gradient vector (dx,dy) is (pretty much by definition) the "optimal direction of derivation".
The magnitude of the gradient is the "intensity" of the edge, the steepness of the derivative.


What is the maximal frequency resolution for Matlab's STFT implementation spectrogram()?

The length of the FFT is a tradeoff between frequency and time resolution. Spectrograms are often generated by calculating overlapping FFTs on the signal of interest. If you make the FFT longer, then the effective bandwidth of each output bin becomes smaller, so the resolution along the frequency axis improves. The only limiting factor to the frequency resolution that you can obtain is the total observation time that you have from the signal.

At the same time, however, your ability to resolve a feature that is localized in time diminishes. An intuitive way to think about this is to view the FFT as a complex downconversion followed by an integrate-and-dump operation:

$$
X[k] = \sum_{n=0}^{N-1} (x[n] e^{\frac{-j2 \pi n k}{N}})
$$

Viewing it in this way makes the loss of time resolution more apparent. The product in parentheses shifts $x[n]$ down in frequency by $\frac{2 \pi n k}{N}$, and the resulting signal is integrated over a window of $N$ samples. If there is a feature in $x[n]$ that is only located in a limited time span, then as $N$ becomes larger, more of the overlapped FFTs will contain that time period inside their integration time windows. Therefore, the feature will appear in more rows of the spectrogram image (assuming time is along the Y axis). If you then make a cut down the columns (i.e. the frequency bins) of the spectrogram that the feature is located in, you would notice a broader, smeared out peak. You therefore have a lesser ability to resolve the actual time location of the feature's onset.

You are also right that increasing the FFT length does require more computations, which might be relevant for real-time applications.


How many types of Bandpass filters are there for image processing?

That article discusses just 3 types of filters (honestly a 2D Butterworth filter is the first time I hear)  and is clearly lacking any generality. 1D analog and digital filter design is a mature subject. However at some points 2D filter design dramatically differs from a 1D filter design,  as reflected by the active research on the area.

Fundamentally image data is exceedingly sensitive to phase distortion and any operations (including filtering) must therefore preserve the waveform shape as much as possible. (i.e. they have minimum, if not zero, phase distortion.)

Both 1D and 2D FIR filters are prime candidates for this required linear phase property. Their extension to 2D is straightforward.

A Butterworth filter is basicaly an IIR type and hence it has nonlinear phase response especially at the transition band edges. However, butterworth filters do have the smoothest passband characteristics among the basic filter types.

In contrast Chebyshev types I-II and elliptic filters are IIR filter types, with Equiripple characteristics in the passband and/or stopband. They have much sharper transitions than attainable via any FIR or Butterworth filter given the same order N.Elliptic filters are considered as the optimal IIR filter which will have smallest transition width for a given order N, at an accepted maximum passband and stopband deviation. However I have not seen any of these 2D IIR filter types to be preferred over a carefully optimized 2D FIR filter.

For 2D filter design, there are also other methods such as frequency sampling or 1D to 2D circular transforms. A particular class of such mappings can produce practically optimal 2D filters.

Finally a variation of Parks-McClellan minimax filters can also be adopted to 2D , however with more difficulties, since fundemantal theorems of 2D polynomials are more complex and less fruitful than the 1D case.


Nyquist rate determination and oversampling

you actually have to sample at more than twice the highest frequency.  if $B$ is the bandwidth or highest frequency and $f_\text{s}$ is the sampling frequency, then to satisfy the Sampling Theorem (for normal baseband sampling):

$$ 2B < f_\text{s} $$

it is not

$$ 2B \le f_\text{s} $$

sampling at exactly twice the frequency of a sinusoid is "critically sampling" and loses amplitude or phase.  it can't get both.  in your case, it is sampling the sinusoid when it crosses zero.


First of all, since the system A is not LTI, then the ordering of systems is critical in determining the output, and you can weakly conclude that $V_1[n] \neq V_2[n]$ in general, but is it so for all possible inputs? The following is true: if the orders of $N$ LTI systems are interchanged, the resulting outputs would be the same for all possible inputs. But the following is not (necessarily) true: if at least one system were not LTI, then outputs corresponding to different orderings must be different for all possible inputs... It's not true because there can exist some inputs for which the coprresponding outputs could be same in different orderings...

You have solved the two outputs correctly and reached the condition that $V_1[n]=x[-n+1]$ and $V_2[n]=x[-n-1]$. Can you find some inputs $x[n]$  whose transform under these two systems might yield the same output? i.,e., are there any $x[n]$ for which $x[-n-1] = x[-n+1]$ for all $n$ ?

As @loudnoises has exemplified a trivial input of the form $x[n] = 0$ for all $n$ provides a single example to the option b. Indeed all inputs of the form $x[n] = K$ (the constant signal) satisfy the relation that $x[-n+1] = x[-n-1]$ hence there exists infinitely many (for each different K) inputs of the form $x[n]=K$ for which $V_1[n] = V_2[n]$ .


Is it an excessive pre-processing step in image processing?

I cannot access the raw images, so I did not test the algorithm. Note the aim of pre-processing is to increase the cell contrast to the background. You can remove one step from 2-6 each time to observe the background segmentation effect. By this means you may determine what steps play critical roles on the accurate cell boundary extraction and what others are not necessary. Mostly morphological operations plus sobel filters should be sufficient.


False negative probability of cross-correlation

"Suppose $D$ is a string...".

No it is not.

Because if this were a string (a discrete sequence), then its content would take values from an alphabet of symbols, say $A \in \left[0 \dots (b-a) \right]$ where $b,a \in \mathbb{N}, b>a$ with some uniform discrete probability $\mathcal{U}(0,|A|)$, where $|\cdot|$ denotes the length of the sequence. Then, the operation of correlation would not make sense and we would have to use an appropriate string "similarity" function. For example, the hamming distance. Then, we would accept an error rate, which would represent the number of symbols set in "error" by some noise process. In ideal conditions, the normalised hamming distance (the sum of all symbols in error in a sequence divided by the length of the sequence) would be $0$ and as the error rate increased, so would the hamming distance all the way to $1$ in which case we would miss the entire sequence.

But that is not what is implied here. What is implied here is that $D \in \mathbb{R^k}$ (where $k$ here denotes the length of $D$), it takes values from a continuous $\mathcal{U}(a,b)$ and these represent the "message". This message is then contaminated by additive white gaussian noise to produce $X$. Then, we try to correlate a rolling window (of finite width) of $D$ over $X$ and observe the amplitude of the correlation at each "window shift" ($C_n$).

So, given that former framework where $D \in \mathbb{R}$:

The first step in modelling this is to look at what is the output exactly. The output is something like "What is the probability that the amplitude of correlation makes (or does not make) the threshold?". Without knowing anything else about the amplitude of each correlation ($C_n$), its value can swing anywhere $\in \left[-1 \dots 1 \right]$. So the probability there is some $p \in \mathcal{U}(-1,1)$. Not very useful yet.

Under ideal conditions (no noise), we would expect the correlation to correctly recognise the subsequence every time. So, if $P$ was repeating in $X$, $C_n=1$.

In the presence of additive white gaussian noise, the correlation amplitude is reduced (+). And, not only $C_n$ is reduced but any estimate of $n$ (where the sequence was detected, the $n$) is becoming uncertain too.

Additive white Gaussian noise is kind of special because its correlation with anything is $0$. So, if you had something like $Z_{\alpha} = \alpha \cdot X + (1-\alpha) \cdot \mathcal{N}(0,\sigma)$, then the correlation of $Z_1, Z_0$ is $0$. Another way to look at $\alpha$ is as (a variable proportional to) the signal to noise ratio (SNR). The lower the $\alpha$ the more noise you get in the signal, the more noise you get in the signal, the lower the correlation amplitude gets, the lower the correlation amplitude gets, the harder it is for it to reach the threshold.

What is the lowest value the correlation can attain?

It is zero. (If it gradually went towards $-1$, ....that would be spooky).

When will the correlation be zero?

When the noise will be at its maximum.

When is the noise at its maximum?

...it depends on the relative power between the noise and the signal. In other words, all things being equal, $\alpha$ depends on the length of $D$ to achieve the same SNR (but scaled up, to account for the fact that a sequence is now longer). For more details about this, you might want to have a look at this.

Long story short, this now brings us to the bit error rate curves. The theoretical bit error rate curves tell you how many symbols in error you are expected to get given the power SNR. These are also relevant here because at the end of the day, you are also making a binary decision. You either detect $P$ or you don't.

But, all that you have to do here is to adjust your $E_b/N_0$ for the lengths of the sequences you are looking for because in your case, your "bit" is the whole sequence $P$.

What is the theoretical expression for bit error rate?

$$BER = \frac{1}{2} \text{erfc}\left(\sqrt{\frac{E_b}{N_0}}\right)$$

Where $\text{erfc}$ is the error function. For more details on BER, see here

This $BER$ will tell you what you are after, that is, what is the probability that I miss a $P$ when it is there, given a particular SNR.

Hope this helps.

Notes:

+: The correlation has a finite range $\in \left[-1 \dots 1\right]$. Therefore, saying "...the standard deviation of the correlation signal..." is a misconception. This would be valid for something like $\mathcal{N}(0,0.2)$. But as the correlation tends to the edges of its range, this "...plus or minus..." of the standard deviation becomes invalid. When the conditions are such that the correlation hits $1$, there is no way its deviation will be anything else than $0$. So, perhaps this definition of the threshold as $K$ times the standard deviation of the correlation signal needs a bit more thinking depending on the problem at hand (?). I am assuming that it was defined in this way to then be able to determine a suitable value that guarantees detection.


Yes; what you describe doesn't mathematically look any different than a single sampler at twice the rate. But they sensors need to sense the same, band-limited thing!

Actually, that's how some very high-speed ADCs work.

If you're not actually doing GS/s, getting a faster ADC would probably be easier than staggering multiple ADCs.


What is a good example of an ergodic process?

Just suppose I give you a series of numbers, and I tell you they were picked randomly. And you know I am not trying to deceive you. Numbers are: $3$, $1$, $4$, $1$,  $5$, $3$, $2$, $3$, $4$, $3$.

I now propose you to predict the next one, or at least, to be as close as possible. Which number would you pick?

[Think]

[Compute]

I bet most of the readers are likely to choose a number between $0$ and $6$. Because of the limited span.
Perhaps an integer. Who is likely to propose $\pi$ (even thinking of the first digits)?
Possibly  $2$, $3$, or $4$. Maybe even $3$.

Basically, you are assuming that I provided numbers with some unknown rule. And perhaps, you may think (or make the hypothesis) that the series of given numbers, if long enough,  can provide you with a good understanding of the rules that I have in mind.  If you do so, you hypothesize that  my mental process is ergodic:

  a process in which every sequence or sizable sample is equally representative of the whole (as in regard to a statistical parameter) (Merriam-Webster)

Here, there is no way to be sure that my series follows an ergodic process. 3432 is my card PIN, 3 a mistake (I intended 6, but I am clumsy), 4, 3, 1 and 5 are first digits of $\pi$ that I use quite often. My next "number" would have been C (in hexadecimal). I do not believe this process is ergodic. Each number comes out from  different laws. But honestly, I do not know. Maybe I am subject to some higher order forces that drive me under ergodicity rules.

So, ergodicity is a hypothesis of a sort of "simplicity" in the rules of a process. Like stationarity or sparsity.
 Cast a regular die with $6$ faces. Toss a normal coin. If nothing outside tries to influence the result (an invisible being that catches the die and shows some face of its choice), you are likely to produce an ergodic process.

Instead of being able to toss an infinite number of coins, with your   infinite number of thumbs, precisely at the same second, you toss one coin every second, and believe the final result is about the same.

The Brownian motion possesses ergodic properties too.


How can noise be added to a signal to obtain a specific SNR?

Easiest way is to scale energy of noise. Nevertheless you must understand that for some files it is not that simple. Let's say that you have the recording of crying baby in silence. What is then your point of reference? Are you going to add white noise based on ratio between it's energy and average energy across the whole recording (silence+baby)? Or maybe you should calculate the energy in sliding window and then use only sound of interest as point of reference? That is why you should not add noise based on energy of your whole file.

When it comes to some particular examples, then I suggest you to take a look here for mathematical derivation. If you are looking for functions, then please check out the MATLAB awgn function. Nevertheless if you want something even more sophisticated, then you might want to check the Audio Degradation Toolbox. You can find a dedicated routine for adding various types of noise with a given SNR.


I dont think there is a common framework for defining all those operators. Each of them were developed by different people, at several moments of the history. They are being integrated day by day onto a common discipline called DSP.

As clarification, an operator (like a moment or any other) and the "redundancy" of a signal are very different and unrelated concepts.

IF you want a deeper answer, list all the moments you are considering, and all the different math theories you are reviewing for each of them.


Question about Hilbert transform of a cosine signal

Make sure that you understand the conditions under which

$$\mathcal{H}\big\{A(t)\cos(\omega_0 t)\big\}=A(t)\mathcal{H}\big\{\cos(\omega_0 t)\big\}=A(t)\sin(\omega_0t)\tag{1}$$

holds. Eq. $(1)$ holds if $A(t)$ is a low-pass signal with a cut-off frequency smaller than $\omega_0$. This implies that $A(t)\cos(\omega_0t)$ is a band-pass signal with no energy at DC. Eq. $(1)$ is a special case of Bedrosian's theorem.

In your example, $A(t)$ is not band-limited, and, consequently, $(1)$ doesn't hold. However, if you increase the modulation frequency $\omega_0$, the overlap of the spectra of $A(t)$ and of the carrier becomes smaller, because $A(t)$ does have a low pass character. So for large $\omega_0$, Eq. $(1)$ is approximately satisfied, which is exactly what you see in your plots.

Also take a look at this related answer.


What Is the Bilateral Filter Category: LPF, HPF, BPF or BSF?

Neither. To me, filter classes using the notion of frequency bands (low-pass, high-pass, etc.) can be used  safely in the linear case. And the bilateral filter is nonlinear. Edges are not really high-frequency: they often have sharp variations across the edge, but slow variation along it. Hence linear directional filters often have a derivative part and and orthogonal smoothing part.
I would consider the bilateral filter as an edge-preserving smoother, a broad and somewhat imprecise class.
Suggested reading: Fast and Provably Accurate Bilateral Filtering, 2016

The bilateral filter is a non-linear filter that uses a range
filter along with a spatial filter to perform edge-preserving
smoothing of images. A direct computation of the bilateral filter
requires O(S) operations per pixel, where S is the size of the support
of the spatial filter. In this paper, we present a fast and provably
accurate algorithm for approximating the bilateral filter when the
range kernel is Gaussian. In particular, for box and Gaussian spatial
filters, the proposed algorithm can cut down the complexity to O(1)
per pixel for any arbitrary S. The algorithm has a simple
implementation involving N+1 spatial filterings, where N is the
approximation order. We give a detailed analysis of the filtering
accuracy that can be achieved by the proposed approximation in
relation to the target bilateral filter. This allows us to estimate
the order N required to obtain a given accuracy. We also present
comprehensive numerical results to demonstrate that the proposed
algorithm is competitive with the state-of-the-art methods in terms of
speed and accuracy.


Number 4 : Comparing narrow-band energy to the energy of the total spectrum is a commonly described method of DTMF detection.  This may involve using more than one FFT result bin and/or interpolation between bins.

The other methods depend on whether one has an accurate estimate of the signal level, noise floor, and how well the tone frequency is centered in a single FFT bin (related to frequency, sample rate, FFT length and window).


Eigenfunction property for LTI sinusoidal and the sinusoidal steady-state response

It can be shown mathematically that the output of an LTI system when a complex exponential is applied is just a scaled version of the input, with no transient terms.

that's for an input that has been going on since forever ago (and will continue forever to the future):

if
$$ x(t) = e^{j \omega_0 t} \qquad \forall t \in \mathbb{R} $$

then

$$ y(t) = H(j \omega_0) e^{j \omega_0 t} = H(j \omega_0) x(t) $$

  What’s going on here?

problem is, sometimes we have to apply an input at some time we might call "$t=0$", and before the LTI settles down to the $y(t)$ shown above, there are transients.  but if the LTI is stable (and not "marginally stable"), the transients will go away eventually.


Continuous Wavelet Transform with Scipy.signal: what is parameter "widths" in cwt() function? How do time-frequency?

complex morlet was added Aug 10, 2007
ricker and cwt were added Sep 20, 2011

There's no indication that cwt is meant to be compatible with morlet.  As cwt docstring says:

  Wavelet function, which should take 2 arguments. ... second is a width parameter, defining the size of the wavelet (e.g. standard deviation of a gaussian).

The morlet function takes 4 arguments, the second of which is not a width parameter, it's a frequency parameter, so I don't think it is meant to be used with cwt. Using ricker as a template, you are supposed to define your own functions to use with cwt. In addition, cwt cannot handle complex wavelets (as of v0.18).

I'd suggest you try a different implementation that has this done for you:

Pyscellania has a Wavelets module (now mirrored at github) which implements real/complex Morlet, MexicanHat, Paul order 2, Paul order 4, 1st Derivative Of Gaussian, 4th Derivative Of Gaussian, Unnormalised version of continuous Haar transform, Normalised Haar
aaren/wavelets supports Morlet, Paul, DOG, Ricker
Machine Learning PYthon (mlpy) has mlpy.wavelet.cwt that supports Morlet, Paul, and Derivative Of Gaussian wavelets
Dapid/fast-pycwt supports Morlet and Ricker
ObsPy (seismological observatories) has a cwt module and "for now only 'morlet' is implemented"
pyCWT has Morlet wavelet


Are Computer Stereo Vision techniques appropriate for sub-millimetre measurements?

Stereo imaging

Given the large field of view you need in relation to the accuracy you want, and how close you want to be, I think that stereo imaging may be a challenging, so you need to somehow amplify the differences you are trying to measure.

Structured lighting

If you are essentially trying to measure the profile of an object, have you considered a single high resolution camera and structured lighting?

Thanks to looptechnology for this image, used without permission, but hopefully attribution will be enough.

Note, the shallower the grazing angle, the the greater accuracy you can measure, but the lower the supported depth of field would be, so for your application you would need to optimise for your needs or make your system adjustable (one laser angle for 0-500um, another for 500-1500um and so on). In this case though, you would probably have to calibrate each time you changed laser position.

Incidentally, a very cheap way to try this out would be to pick up a pair of Laser Scissors which include a basic line laser LED.

Finally, you can remove the vibration problem by sampling multiple times, rejecting outliers and then averaging. A better solution though would be to mount the whole test apparatus on a block of granite. This worked well for laser micro-machining tools I've worked with in the past, which require micron level position and depth of focus accuracy, even when located in factories.

Some back of the envelope calculations.

Lets assume an incident angle of 10 degree from horizontal, and a camera with a 640x480 resolution and a field of view of 87 x 65mm. If we place the beam so that it is right at the bottom of the portrait frame with no sample, and then place the sample with the beam crossing it, this should give us a maximum height of around 15mm and thus an uncorrected resolution of around 24um for each pixel the line walks up the screen. With this setup, a 0.1mm variation should be visible as a 4 pixel variation in position.

Similarly, if we use an incident angle of 2 degrees from horizontal then this should give us a maximum height of around 3mm (Tan(2deg)*87mm) and thus an uncorrected resolution of around 4.7um per pixel, for a much more noticeable 20 pixel jump. This would probably require a much more accurate line laser however.

Note, if the camera is close enough then the you may need to do a second trig calculation, using the camera height, to determine the the true position of the line relative to the base line.

Also note that if you don't need absolute accuracy, and local repeatability is enough (say you are profiling the flatness of a sample to ensure it is within given tolerances) then just being able to see the relative position of the laser line might be enough.


Continuous-time vs Discrete-time Fourier transform

This is pretty straightforward to show. Let $x[n]$ be a sampled version of a continuous-time signal $x_c(t)$:

$$x[n]=x_c(nT)\tag{1}$$

The DTFT of $x[n]$ is defined by

$$X(e^{j\omega})=\sum_{n=-\infty}^{\infty}x[n]e^{-jn\omega}\tag{2}$$

The CTFT of the signal

$$x_c(t)\cdot\sum_{n=-\infty}^{\infty}\delta(t-nT)=\sum_{n=-\infty}^{\infty}x_c(nT)\delta(t-nT)\tag{3}$$

is given by

$$\begin{align}\mathscr{F}\left\{\sum_{n=-\infty}^{\infty}x_c(nT)\delta(t-nT)\right\}&=\sum_{n=-\infty}^{\infty}x_c(nT)\mathscr{F}\left\{\delta(t-nT)\right\}\\&=\sum_{n=-\infty}^{\infty}x_c(nT)e^{-jn\Omega T}\tag{4}\end{align}$$

which equals $(2)$ with $x[n]=x_c(nT)$ and $\omega=\Omega T$ being the normalized angular frequency.

And now I found this question, which is pretty similar but also asks about some background.


oversampled coefficient for existing exponential smoothing

I think it's instructive to take a look at the impulse responses, taking absolute time into account. I'm assuming the following recursion:

$$y_n=\alpha x_n+(1-\alpha)y_{n-1}\tag{1}$$

The impulse response corresponding to the recursive system described by (1) is

$$h_n=\alpha(1-\alpha)^n$$

Assume now that we have a discrete-time system with sample interval $T$, and another one with sample interval $T^{\prime}=T/L$. Let $\beta=1-\alpha$, and let $\alpha^{\prime}$ and $\beta^{\prime}$ be the constants of the system with sample interval $T^{\prime}$. In order for the two systems to have the same time constant and the same scaling we require (considering a point in time $t_n=nT=nLT^{\prime}$)

$$\alpha\beta^{n}=\alpha^{\prime}{\beta^{\prime}}^{nL}\tag{2}$$

From (2) we see that

$$\alpha=\alpha^{\prime},\quad \beta=\beta^{\prime L}\quad
\text{(i.e. $\beta^{\prime}\neq 1-\alpha^{\prime}$)}$$

In this case the impulse response of the system with the higher sampling rate is simply an interpolated version of the other system's impulse response. Note that the condition on $\beta^{\prime}$ is the one that you came up with intuitively, and it is also the more important of the two because it affects the time constant of the system. The choice $\alpha=\alpha^{\prime}$ only affects the scaling and in this sense is secondary. Choosing $\alpha^{\prime}=1-\beta^{\prime}$ is of course also possible and just changes the scaling.


Usually before processing the speech signals, they are passed through a high pass filter with cutoff frequency of around 150 Hz to remove such artefacts.


How to find the energy of a rect signal?

Here's a definition of the energy of a signal from Wikipedia: https://en.wikipedia.org/wiki/Energy_(signal_processing)
For your specific signal, I recommend to draw the original signal and then the transformation. Finally, just calculate the squared magnitude and the area under it.
These are plots of the steps I mentioned (from Matlab):

As you can see in the last plot, the energy is basically the area under the rectangle, which is in this case 4*4=16.


To answer your first question, what they mean is that the first training symbol only encodes data on the even-numbered subcarriers. The other subcarriers are set to zero. That is, the frequency-domain,

$$
X[k] =
\begin{cases}
s_k, &k \text{ mod } 2 = 0 \\
0, &\text{otherwise}
\end{cases}
$$

The symbols to encode on the even-numbered subcarriers $s_k$ are chosen from a pseudorandom noise (PN) sequence. They assert that this results in a time-domain symbol that has two identical half-symbols in it (i.e. the first half of the symbol period is equal to the second half). Why is that?

Recall one of the properties of the discrete-time Fourier transform: if you zero-stuff a signal in the time-domain (by inserting zeros between each pair of samples), then in the frequency domain, the spectrum is repeated periodically. You might have seen this before in a description of discrete-time interpolation: insert zeros, then add a lowpass filter to eliminate the spectrum duplicates.

The property is dual in the sense that it works the other way also. When assigning symbols to subcarriers, you're working in the frequency domain. An inverse DFT is used to generate the time-domain signal for transmission. If you set all of the odd-numbered subcarriers to zero, then you're generating a frequency-domain signal that is zero-stuffed by a factor of two. When you inverse transform that to the time domain, you get the same effect: the resulting symbol consists of two periodic half-symbol waveforms.

I haven't read the paper in detail, but I'm assuming their synchronization technique takes this redundancy into account in some way.


Synchronization with a specified sequence

I've found that a good sequence should have an autocorrelation function similar to a pulse...is it correct? If it is...why?

You want to pinpoint one specific delay – the right time synchronization "point". From that, it directly follows that the ACF should be as "dirac-y" as possible to minimize ambiguities and maximize energy in the one right time.


Been there, done that.  One of our on-going projects is to create a process for processing nutrition labels from iPhone camera for a health-tracking app.  My company decided to develop a solution for this particular application - extracting data from US nutrition labels.  This solution will be used for this client, but we decided to pre-package a few other flexible capabilities along the way, to be used by a wider audience.  The solution will be available to general public in about 3 weeks.  (I'll come back here and post an update.)

Main goal that will make or break the entire idea is to get usable images.  Curved images, heavy shading, low resolution, and blur will all substantially decrease OCR quality, often to no quality.  In apps, we found that user training and guidance is the most helpful method.  Then there are some technical tools such as quickly detecting quality of image and suggesting to re-take it if needed.  In general, if you can achieve "high quality fuel" (images) you can expect high performance from your "machine".

Next, the OCR.  Tesseract can do good job reading clear and simple test such as pages from books.  Commercial products such as ABBYY distinguish themselves by working well on tough images - shading, distortions, small prints, etc.  And unfortunately if mobile images are used, they are more often on bad side than good quality side.

Next, pick your approach to locating and extracting data.  Please see my answer on text parsing vs specialized test extraction tools:
https://stackoverflow.com/questions/3070732/processing-ocred-text
For our project we'll be using ABBYY FlexiCapture for targeted nutrition data.  It has special tools even when OCR makes mistakes to still find appropriate data (sort of controlled fuzzy search), and that was an important factor in selecting it for the task.


How Does the RMS of White Noise Change with Sampling Frequency?

It changes according to the bandwidth of the LPF Filter in the sampling chain.
Namely if you use the same LPF yet with two different samplings rates the RMS of the noise is the same.

Now, the theoretical RMS of White Noise is given by its PSD which is a constant.
The RMS of the sampled noise is the constant value multiplied by the BW of the LPF in the sampling chain.


Best averaging to convert 50Hz Chi data to one hour averages?

Essentially, you want to take your original signal and downsample it. Since you need to take care of aliasing before downsampling, you'd need to perform low-pass filtering of the signal.

Your original sampling frequency is $F_S=50Hz$, you want to downsample it to $1/h$ (1 per hour), which is $F_S/(60*60*50)$. So, your anti-aliasing filter needs to have cut-off frequency of $0.5/h$. After filtering you can just take every (50*60*60)th sample of the filtered sequence to have hourly averaged data. Note that downsampling to hourly averages requires, that the actual information in the signal does not vary too quick over time.

Code for some rudimentary filter design is below. Since your cutoff-frequency is so low, you need a very long filter to perform a good filtering. This is in tradeoff against calculation time.

Fs = 50
T = 24*60*60  # data of one day

cutoff = 0.5/(60*60)  # cutoff frequency

# Create some simulated measurement data
t = np.arange(0, T, 1/Fs)
x = np.sin(2*np.pi*t*3/(60*60*24))  # frequency: 3 periods/24 hours
y = x + 0.1*np.random.randn(*x.shape)

b = signal.firwin(15*3611, cutoff, nyq=Fs/2)
a=1

plt.subplot(1,2,1)
plt.plot(b)
plt.subplot(122)
w, h = signal.freqz(b, a, 32*4096)
plt.plot(w, 20*np.log10(abs(h)), '-x')
plt.xlim((0, 3/(60*60)))
plt.axvline(cutoff)

r = np.convolve(y, b)
r = r[:len(y)]

plt.plot(t, y)
plt.plot(t, r)
r_down = r[::50*60*60]
t_down = t[::50*60*60]

plt.stem(t_down, r_down)

As you can see, I get accurate downsampled data.


Difference between causality and memorylessness

A causal system does not need to know the future in order to compute its output. A memoryless system computes the output only from the current input. A memoryless system is always causal (as it doesn't depend on future input values), but a causal system doesn't need to be memoryless (because it may depend on past input or output values).

The system $$y[n]=x[n]+2x[n+1]$$ is non-causal because it needs to look into the future (by $1$ sample) to compute its output. The system $$y[n]=3\big(x[n]\big)^2$$ is memoryless (and necessarily causal) because it only needs the current input sample $x[n]$ to compute the output.

The systems in your question are both causal and have memory (if $n_0>0$).


Minimum bit-width for fixed point FFT outputs

it really depends on how the DFT is defined.  usually we define the DFT and inverse DFT as:

$$ X[k] = \sum\limits_{n=0}^{N-1} x[n] \ e^{-j 2 \pi \tfrac{nk}{N}} $$

$$ x[n] = \frac{1}{N} \sum\limits_{k=0}^{N-1} X[k] \ e^{+j 2 \pi \tfrac{nk}{N}} $$

but they could just as well be defined as

$$ X[k] = \frac{1}{N} \sum\limits_{n=0}^{N-1} x[n] \ e^{-j 2 \pi \tfrac{nk}{N}} $$

$$ x[n] = \sum\limits_{k=0}^{N-1} X[k] \ e^{+j 2 \pi \tfrac{nk}{N}} $$

all of the theorems work exactly the same, except there will be a scaling difference in the convolution theorem and Parseval's theorem.

and, my suggestion, for fixed-point arithmetic is splitting the difference:

$$ X[k] = \frac{1}{\sqrt{N}} \sum\limits_{n=0}^{N-1} x[n] \ e^{-j 2 \pi \tfrac{nk}{N}} $$

$$ x[n] = \frac{1}{\sqrt{N}} \sum\limits_{k=0}^{N-1} X[k] \ e^{+j 2 \pi \tfrac{nk}{N}} $$

now, let's assume for shits and grins that $N$ is a power of 2, so there will be $\log_2(N)$ passes in the fixed-point FFT code.

in the first definition (at top), in the inverse FFT, there will be a division by 2 (an arithmetic shift right) for the results of the FFT butterflies in each inverse FFT pass (and none of this in the forward FFT).  $X[k]$ will have a mean square that is $N$ times bigger than the mean square of $x[n]$.  so $x[n]$ might be expected to padded with $\log_2(N)$ extended sign bits on the left to keep $X[k]$ from getting too big for the word width.

in the second definition (in the middle), the roles are reversed and in the forward FFT, there will be a division by 2 for every FFT butterfly.  so, since the magnitude of $x[n]$ is expected to grow for each FFT pass, then $X[k]$ should be padded with $\log_2(N)$ guard bits on left.

in the third definition, $X[k]$ and $x[n]$ both have the same mean square or the same energy.  and in both the forward and inverse FFT, for every alternate FFT pass, the outputs of the butterflies are divided by 2.  now having the same mean square does not guarantee that word width limits are not exceeded.  if the words going into the first FFT (i'll assume the first FFT is the forward FFT) are padded with $\tfrac12 \log_2(N)$ guard bits, then i would not expect any overflow.  and if the processing in the frequency domain preserves the mean-square of $X[k]$, then i would not expect any overflow in the inverse FFT.

one totally different trick, if you have the code to efficiently do this, is to do "block-floating-point" arithmetic for the FFT.  but that might not be a very fast FFT because it requires testing the magnitude of the outputs of each butterfly.  the Mot DSP56002 and later chips (from the 1990s) had a sticky-bit in the status register that automatically detected this bit-width growth, and a selectable "divide-by-2" mode on the outputs, which both facilitated a very efficient block-floating-point FFT in the fixed-point processor.


Your "mistake" if it can be called that is to assume that your input signal was zero for $t\le0$.

When talking about the ZI-response and the ZS-response of a (typically causal) LTI system, "zero input" means "zero input from now on".

In that case, the past input's influence on the future signal is fully expressed with the ZI-response, while the future input's influence on the future signal is fully determinable by the ZS-response.


Would it be possible to extract the "smudge" (i.e., the equivalent of an image filter applied to the physical photo that results in the scanned image), and then apply the inverse of this filter to all my images (with appropriate rotation and offset)?

It is difficult to reply Yes or No to this question. The results depend very much to what you are working with and for what purpose.

There are two things that are happening here:

The smudge absorbs some light
The smudge refracts and scatters some light

Dealing with 1 is easier than dealing with 2. Light is absorbed and therefore the smudge leaves behind it a shadow. If you had a "mask" of how much light passes through each point of the smudge, then you could get rid of it by adjusting the image pixel values (i.e. the comment suggestion).

If the smudge is caused by a deposit (e.g. finger markings, droplets, gel, etc) then it not only has diminished the light that went through it, it also has changed its path. Therefore, even if you correct the shadow, the "focus shadow" will still be there.

It is exactly like having an extra lens on top of the existing focusing mechanism.

This second bit could be corrected with a "filter", but it's easier said than done.

The main problem here is that you will not be able to recover an exact "image" of the smudge, so that you can then invert it. This is even worse if the smudge has been changing slightly with every image acquisition.

Short of obtaining an "empty" scan, you could try to obtain the average frame over N (grayscale) images. This is assuming that the N images have the smudge at exactly the same position. If not, then you will have to align them.

If N is high enough (at least 20 but it depends on image content too) then you will start to see the smudge shape over a (mostly) average background.

This can be used to diminish the shadow (at least) or deconvolve the affected regions to diminish the "focus shadow" too.

But, whether this is successful also depends on the use of the images. If you had to do measurements on the affected pixels, or look for patterns, this filtering operation could distort your results.

  I don't know if tools already exist that could do this, but any guidance as to algorithms/approaches would be much appreciated

These operations can be applied with low level image processing tools (e.g. Octave or skimage ) but for a quick test you could use:

Hugin to align the images
Gimp to stack the (aligned) images (in layers) and obtain the average image that would reveal the smudge.

The deconvolution bit is a bit more involved. There is ImageJ which has an astrophotography plugin that would be very similar to your use case except that in astrophotography the "smudge" is assumed symmetric and spatially invariant (same across the image)....which yours might not be. There are however other plugins you might find useful in this.

Hope this helps.

(If a subset of the images could be shared, we could have a quick go at it).


Frequency estimation on chaotic waveform

Here is a stab.

Start with:

$$ \ddot{u}  - 2\beta \dot{u}  +(\omega^2+\beta^2)(u-s)=0 $$

Take the derivative:

$$ \dddot{u}  - 2\beta \ddot{u}  +(\omega^2+\beta^2)(\dot{u}-\dot{s})=0 $$

Now, $ \dot{s} = 0 $ Wherever it is defined.  So:

$$ \dddot{u}  - 2\beta \ddot{u}  +(\omega^2+\beta^2)(\dot{u})=0 $$

Let $ v = \dot{u} $, and you get

$$ \ddot{v}  - 2\beta \dot{v}  +(\omega^2+\beta^2)v=0 $$

Which is an ordinary second degree linear differential equation.  v will have the same frequency as u.

The most basic (good) differential approximation is:

$$ d[n] = \frac{u[n+1]-u[n-1]}{2} $$

So run that over your data and see what you get and please add a chart to your question with the results.

I think you may best solve this one by analyzing the zero crossing of those results.  The transitions are going to cause weirdness as your system "falls" towards the new equilibrium.

Followup:

A more traditional approach perhaps.

$$ \ddot{u}  - 2\beta \dot{u}  +(\omega^2+\beta^2)(u-s)=0 $$

Your homogenuous equation is:

$$ \ddot{u}  - 2\beta \dot{u}  +(\omega^2+\beta^2) u=0 $$

with solutions:

$$ u_h = \left[ C_1 \cos( \omega t ) + C_2 \sin( \omega t ) \right] e ^ { \beta t } $$

And your particular equation is:

$$ \ddot{u}  - 2\beta \dot{u}  +(\omega^2+\beta^2) s=0 $$

Assume:

$$ u = a t^2 + b t + c $$

$$ \dot{u} =  2 a t + b $$

$$ \ddot{u} =  2 a  $$

$$ (2a)  - 2\beta ( 2 a t + b )  + (\omega^2+\beta^2) s=0 $$

$$ (2 - 4 \beta  t)  a  - 2\beta  b   + (\omega^2+\beta^2) s=0 $$

$$ a = 0 $$

$$ b = \frac{\omega^2+\beta^2}{2\beta} \cdot s $$

So :

$$ u_p = \left( \frac{\omega^2+\beta^2}{2\beta} \cdot s \right) t $$

For any range of consistent $s$ you now have your general and particular solutions.

Therefore you should only need five points to solve for your five unknowns:  $ \omega, \beta, C_1, C_2,$ and $ c $.

With noise, that goes out the window.  If you take the second derivative, your particular solution will be squashed completely.  Then you should only have to worry about the $s$ transition points.

Line crossings will give you $\omega$, and the envelope will give you $\beta$.


Colored noises: Stationary or non-stationary?

Stationary processes have a spectrum that's time-independent. As the Wikipedia link shows, the colored noises each have a characteristic spectrum, are therefore not time-dependent, and thus stationary.


systematic way to determine best bit mapping policy

For $2^{2n}$-QAM, Gray code labeling often assigns the odd-numbered bits to one axis (I or Q) and the even-numbered bits to the other axis (Q or I) and uses $n$-bit Gray code labeling along each axis. Alternatively, the first $n$ bits could be assigned to I (or Q) and the last $n$ bits to Q (or I). Depending on the structure of the modulator (e.g. 8 bits entering the modulator on a serial line versus 8 bits entering the modulator simultaneously on a 8-bit-wide bus), one method might be preferable over the other. It is important to note that it is not necessary to convert the data into Gray code; just to use the Gray code labeling.  An extensive discussion of this last point can be found here on Google Groups.


Matlab Neural Networks: input size vs. number of inputs

This question is better fit in stackoverflow or cross validated site, yet I am trying to illustrate a simple example.

Suppose you have three measurements $a$, $b$, and $c$, with each measurement only $1$ time reading, the hidden layer node and output are scalar. Compose an input vector is shown below:

        w1
(a,b,c) => (H1)-------(O1) => Result

The value at output layer is $ \alpha ((a+b+c)w_1)$, where $\alpha$ is the activation function.

When you treat the measurements separately at the input layer, the neural network is:

  w1
a =>
     \
  w2
b => (H1)----------(O1) => Result
     /
  w3
c =>

and the output is $ \alpha (aw_1 + bw_2 + cw_3)$. Now you are dealing with three independent neurons, which indicates that updating one of the weights won't change the 'learned' weights on other input nodes.

As far as I know, a single neuron usually take in scalars rather than vector values (for the case of one time reading), or take in vectors rather than matrix arrays (for the case of a series of time readings).


Suggestions for creating 3D Bounding Boxes of Books on a Shelf from RGB-D Images

Exploiting the 3D geometry is important here. From the depth image, you could get the 3D point cloud by backprojection. Once the point cloud (set of 3D points) is given, you might benefit from VoteNet:

Deep Hough Voting for 3D Object Detection in Point Clouds, Charles R.
Qi, Or Litany, Kaiming He, Leonidas J. Guibas, ICCV 2019
https://arxiv.org/abs/1904.09664

If, in addition, the color (RGB) information is available, we could utilize within the same framework, this time maybe using ImVoteNet:

ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image
Votes, Charles R. Qi, Xinlei Chen, Or Litany, Leonidas J.
Guibas, CVPR 2020
https://arxiv.org/abs/2001.10692

Both of the papers provide open source implementations. See here for VoteNet and this repository for ImVoteNet. Note that both of these are methods that require heavy supervision, i.e. large amounts of annotated training data. If such data is not available at your disposal, I would start looking into hand-crafted algorithms, e.g. descriptors or point pair feature detectors.
Hope this helps.


Set of all images with a certain MSE between themselves and a given image

Well, you know, there are infinitely many. Start with $D=I$, change one particular pixel to $I[m,n]+\sqrt{\epsilon}$. You have your first candidate that satisfies ${\rm MSE}(I,D)=\epsilon$. Now you can generate a new image $D$ by adding a perturbation to an arbitrary subset of pixels (say, add $\Delta$ to a single pixel) and correct the resulting difference in another pixel (say, subtract $\Delta$ from another pixel). As this works for any subset and any perturbation $\Delta$, the number of images you can generate is infinite.

Of course, if your images are INT8, this makes the set finite, yet still impractibly large. Only the amount of perturbations you can apply to single pairs of pixels is in the order of $2^{7MN}$, considering larger groups of pixels makes this even more larger.

Another way of looking at it: an $M \times N$ INT8 image has $256^{MN}$ degrees of freedom. ${\rm MSE}(I,D)=\epsilon$ is a single contraint. It's a quadratic one yes, but still it does not significantly reduce the degrees of freedom you have.


Coefficients of a Zero Forcing linear equalizer for infinite length input

The following are hints as this is reads as a homework question. Please note that these are not ground breaking hints and can be found in a good text or summary on zero forcing somewhere I'm sure.
Hint 1
The first step would be to look at the model and be able to pick out what the channel is (you have channel convolved with signal plus noise). It seems you have identified the channel as, $h_k = \alpha^{-k}u_k$, where $u_k$ is the unit step function, and the z-transform can be found by directly applying the formula.
\begin{align}
H(z) &= \sum_{k=-\infty}^{\infty}h_k z^{-k} \\
&= \sum_{k=-\infty}^{\infty} \big(\alpha^{-k}u_k \big)z^{-k} \\
&= \sum_{k=0}^{\infty} \alpha^{-k} z^{-k} \\
&= \alpha^0z^0+\alpha^{-1}z^{-1}+\alpha^{-2}z^{-2}+\cdots
\end{align}
Just be careful when simplifying...
Hint 2
Zero forcing (ZF) means that the equalizer is trying to completely eliminate intersymbol interference (ISI), and ZF does achieve this in the noise free case. For the noise free case, in the z-domain your model becomes:
$$ Y(z) = H(z)A(z) $$
The ZF equalizer should be $W_{\text{ZF}}(z)=\frac{1}{H(z)}$ so that the equalized signal will have eliminated the ISI caused by the channel.
Hint 3
This should become clear when you consider what happens to each symbol as it goes through the channel. Take a single symbol being sent over the channel at $k=0$. The received noise free version is $a_0+\alpha^{-1}a_{-1}+\cdots$, or in words this is, $\text{Original symbol } + \text{ Previous symbol } + \cdots$. Can you pick an $\alpha$ to ensure the only non-zero value is the original symbol?


"What do you think" is the wrong question here: It's not a matter of opinion. Your second approach is mathematically simply not calculating the mean of a series of complex numbers. The mean of a complex sequence is w.l.o.g. complex, the mean of your IQIQIQIQ is real, so it can't possibly be the mean/DC component.


BIBO stability and input at system's pole

The signal $x[n] = (\frac{1}{3})^n$ is not bounded. As $n \to -\infty$, $x[n] \to \infty$; therefore, the system's response to your candidate signal $x[n]$ has no bearing on its BIBO stability.

A bounded, causal version of the signal (and likely what you were thinking) is $x_b[n] = (\frac{1}{3})^n u[n]$, where $u[n]$ is the unit step function. The response of the system to this signal is bounded as you would expect.


Analytical design of a bandstop Butterworth filter

it looks like you are designing a parallel highpass and lowpass filter.  i don't think that's the best way to do it.  instead, take your butterworth LPF prototype

$$\Big|H(j\Omega)\Big|^2 = \frac{1}{1+\Omega^{2N}}$$

and make this substitution for normalized $\Omega$:

$$ j\Omega = s \leftarrow \frac{Q}{\frac{s}{\Omega_0} + \frac{\Omega_0}{s}} $$

$Q$ is inversely proportional to the bandwidth of the bandstop filter  (if $N=1$, then $Q$ is the actual "Q" of the second-order filter.  $\Omega_0$ is the angular frequency of where the bandstop is tuned to.


Other time-frequency-plane tiling than STFT, DWT, ConstantQ-Transform: multiresolution STFT?

Your time-frequency grid is Mondrian-shaped: essentially, rectangles supported on dyadic splits of the time or the frequency axes. Hence, you can easily start for any reversible time-frequency tiling, and further slit any of its rectangle onto another invertible tiling. Some call that hierarchical, or nested time-scale/time-frequency decompositions. As long as each is invertible, the combination remains revertible.

Furthermore, dyadic and power-of-two uniform decompositions can be turned into the other, only by grouping coefficients. This was used for instance in A DCT-based embedded image coder, 1996: an eight (or $2^3$) multiband filter-bank DCT was turned into a 3-level dyadic structure.

So, combining dyadic wavelets and $2^K$ windowed filter-bank, you can get invertible schemes as wished.

One issue though: how does one choose how to nest those different embedded decompositions? Seen as wavelet or local cosine, lapped-transform packets, entropy concepts could be used, but the diversity can be huge.


Understanding Pitch Detection with Autocorrelation Methods

For higher resolution periodicity estimation, given properly band-limited data, the data can be upsampled (interpolated) before autocorrelation, and/or, the autocorrelation results can be interpolated (Sinc kernel or some order of polynomial).

This, of course, assumes the signal has a high enough S/N, and that the periodicity itself is unmodulated in timbre(harmonic content), phase, frequency, or amplitude, which isn’t always true of real-world pitched sounds.  Thus a larger number of periods may be needed for a some desired statistical error bound on the pitch estimation.


How to achieve image straightening as cam scanner does?

Geometric Transformation

Supose that $f$ is an image, defined over $(w,z)$ coordinate system, undergoes geometric distortion to produce an image $g$, defined over $(x,y)$ coordinate system. To perform this operation is used this

$$(x,y) = T\{(w,z)\} $$

So, one of most commmonly used forms of spatial transformation is the affine transform (Wolberg [1990]). It can be written in the matrix form as

$$ [x \ y \ 1] = [w \  z \ 1] \ \mathbf{T} = [w \  z \ 1]
\left[ { \begin{array}{ccc}
         t_{11} & t_{12} & 0 \\
         t_{21} & t_{22} & 0 \\
         t_{31} & t_{32} & 1 \\
        \end{array}} \right]$$

This transformation can scale, rotate, translate or shear a set of points, depending on the values choosen for the elements of $\mathbf{T}$

Your case: Perspective Transformation

For perspective transformation, you need a 3x3 transformation matrix. Straight lines will remain straight even after the transformation. To find this transformation matrix, you need 4 points on the input image and corresponding points on the output image. Among these 4 points, 3 of them should not be collinear.

Implementations

Python

OpenCV

OpenCV has a lot of methods to perform image warping and geometric transformation an you can get the examples in the docs page

scikit-image

scikit-image also have many examples in the documation page.

For python just choose the module more comfortable to you or use both.

MATLAB ® Image Processing Toolbox

As the others examples the Math Work's software have a lot of functions to perform geometric transformation. Beware that is an commercial software and it not have a strong programming language if you want make a little software.

References

Wolberg [1990] G. Wolberg. Digital Image Warping . IEEE Computer Society Press, 1990.

Gonzalez [1987] Gonzalez, R.C., P. Wintz, 1987. Digital Image Processing. 2nd Ed., Addison-Wesley, Reading:MA.

Futher Reading

This lecture is a good introduction to this process


Real time numerical differentiation of signals

Your original differentiator, which should be $x(n)-x(n-1)$, is called a "first difference" differentiator. That differentiator amplifies high-frequency noise. As a next step I suggest you try what's called the "central difference" differentiator defined by:

$$ \mathit{Diff} = \frac{x(n)-x(n-2)}{2} $$

which does not amplify high-frequency noise.


Transfer function of double exponential smoothing?

I would use the $\mathcal{Z}$-transform. Let me change the notation slightly by using $n$ as the discrete time index, lower case letters for time domain quantities, $x[n]$ as the observed input data sequence, and $y[n]$ as the output sequence, which is an estimate of $x[n+1]$. With this notation, the time domain equations are

$$\tag{1}\begin{align}
l[n]&=ax[n]+(1-a)(l[n-1]+t[n-1])\\
t[n]&=b(l[n]-l[n-1])+(1-b)t[n-1]\\
y[n]&=l[n]+t[n]
\end{align}$$

Taking the $\mathcal{Z}$-transform of these 3 equations gives

$$\begin{align}\tag{2}
L(z)&=aX(z)+(1-a)z^{-1}(L(z)+T(z))\\
T(z)&=b(1-z^{-1})L(z)+(1-b)z^{-1}T(z)\\
Y(z)&=L(z)+T(z)
\end{align}$$

From the second equation in (2) we can express $T(z)$ in terms of $L(z)$:

$$T(z)=\frac{b(1-z^{-1})}{1-(1-b)z^{-1}}L(z)\tag{3}$$

Plugging (3) into the first equation of (2), we can express $L(z)$ in terms of $X(z)$:

$$L(z)\left[1-(1-a)z^{-1}\left(1+\frac{b(1-z^{-1})}{1-(1-b)z^{-1}}\right)\right]=aX(z)$$

from which, after some algebra, you get

$$L(z)=\frac{a\left[1-(1-b)z^{-1}\right]}{1+[a(1+b)-2]z^{-1}+(1-a)z^{-2}}X(z)\tag{4}$$

Finally, plugging (3) into the last equation of (2) gives

$$Y(z)=\left[1+\frac{b(1-z^{-1})}{1-(1-b)z^{-1}}\right]L(z)=\frac{1+b-z^{-1}}{1-(1-b)z^{-1}}L(z)$$

and combining with (4) results in a relation between the output $Y(z)$ and the input $X(z)$:

$$Y(z)=\frac{a(1+b-z^{-1})}{1+[a(1+b)-2]z^{-1}+(1-a)z^{-2}}X(z)=H(z)X(z)\tag{5}$$

where $H(z)$ is the desired transfer function, which is of course a second order IIR filter.


A 48k sample rate is too low to reliably reproduce your upper side of 20k+4k = 24kHz.  Thus you end up with a LSB (lower sideband SSB) signal, not an AM signal.


I've had a look at the problem in the book you've cited (Figure 10.7, Adaptive Filter for Noise Cancellation).  As Dan has stated, a linear combination can filter out a specific frequency, but it depends on the specific noise frequency, the sample rate, and the length of the adaptive filter delay-line (i.e. the number taps or weights used in the adaptive filter).

In general, however, a given adaptive filter arrangement used for noise cancellation (i.e. sample rate, length of delay-line, adaptive algorithm) will dictate what specific frequencies of the noise will be attenuated.


it's not a Dirac delta, it's a Kroenecker (discrete-time) delta.

$$ \begin{align} \frac{1}{N} \sum\limits_{k=0}^{N-1} \left(-W_N^n \right)^k \ & = \ \frac{1}{N} \sum\limits_{k=0}^{N-1} \left(-e^{j 2 \pi n/N} \right)^k \\
& = \ \frac{1}{N} \sum\limits_{k=0}^{N-1} \left(e^{-j\pi}e^{j 2 \pi n/N} \right)^k \\
& = \ \frac{1}{N} \sum\limits_{k=0}^{N-1} \left(e^{j 2 \pi (n-N/2)/N}\right)^k  \\
& = \ \frac{1}{N} \frac{1 - e^{j 2 \pi (n-N/2)}}{1 - e^{j 2 \pi (n-N/2)/N}} \\
& = \ \begin{cases} \frac{1}{N} N \text{  for } n = N/2 + mM \text{  where } m \in \text{integers} \\ \frac{1}{N} 0 \text{  otherwise} \\ \end{cases}
\end{align} $$

so it's not exactly $ \delta[n-N/2] $ but more like discrete kronecker comb.

$$ \frac{1}{N} \sum\limits_{k=0}^{N-1} \left(-e^{j 2 \pi n/N} \right)^k  =  \sum\limits_{m=-\infty}^{+\infty} \delta[n-N/2+mN]$$


LTI system $y(t)=x(t-T)$ with or without memory

From Signals and Systems/Time Domain Analysis -- Memory

  A system is said to have memory if the output from the system is
  dependent on past inputs (or future inputs) to the system. A system is
  called memoryless if the output is only dependent on the current
  input.

According to some textbooks, the notion of memory is stricter, and requires causality of the system. To that respect:

if $T\ge 0$, the system is causal. If $T=0$, it only use the current time (memoryless).   If $T>0$, the output need a memory of the past of input $x$
if $T< 0$, the system is non-causal.  The output needed a memory of the future of input $x$. You can either say "non applicable", or still term it "memory".

References (especially the first, with many practical exemples):

Lecture III: Systems and their properties by Maxim Raginsky
$y(n)=x(-n)$, causal or not, memory or memoryless?


Emotional Tone detection using audio on mobile phones

I dont know how bound your problem is... Is this dealing only with words that are already detected?

This is a very very hard problem, as @JimClay mentions. For such problems, one cannot simply say "pick highest "x"", because the features are not obvious to us. (Even though they might be obvious to non-observable states within our brains, whose signal processing paradigms are unfortunately inaccessible to us via introspection).

So in problems such as those, I would first collect a large sample of "angry" tones, and a large sample of "happy" tones to contrast. You now have two options:

1) I would 'brute force' it and start looking for features to extract that separate them amicably in feature space. It is honestly hard to say where to begin without looking at real data. VERY coarsely speaking, I imagine angry tones have more low-frequency information, VS higher-pitched happy ones. This then lends itself to including the DFT of the data as 1 feature in DFT_length dimensional space. Of course, not all 'low pitchy' sounds are angry, and not all high-pitchy sounds are happy, so you need another feature vector(s) on which to discriminate, lest you inadvertently create a male-female detector...

2) I would use a different approach and simply label my data vectors, (here, each data vector is the time domain snippet of detected words, all of which are re-sampled so that they have the same pre-determined length). I would then start off with a simple linear perceptron to try and create a hyper-plane to best separate the two classes. (You can add an extra dimension to your classifier to make linearly un-separable clusters separable by taking your hyper-plane up one more dimension). Once adequately trained, you can use that weight vector as your classifier and project all data vectors unto it to determine which class it falls under. Bear in mind you need A LOT of training data for this to work properly, because there are many situations where emotions can manifest itself in words and phrases. This is why I asked you how bound your problem is. (Words? Phrases? Sentences?, etc).


DFT and FFT application in signal processing

I prefer to give tips by comments but i don't have the right jet…
on the questions a, c and e you have look to the delta between 2 frequencies.
what is the space between each frequency and how much is the delta frequency of you're sampled time.
Probably questions B,D and f also but i'm not for shure because that a bit more complicated and had never done it myself...


Why real part of FFT converts image into rotation + original?

Let's say your image is given by $I(x,y)$. Then its Fourier transform is given by
$$
I^f(\omega_x,\omega_y) = \int_x\int_yI(x,y)e^{j\omega_xx}e^{j\omega_yy}dxdy
$$

Now you take the real part and perform the inverse:

\begin{align}
I_m(\alpha,\beta) &= \int_{\omega_x}\int_{\omega_y}\Re\left\{I^f(\omega_x,\omega_y)\right\} e^{j\omega_x\alpha}e^{j\omega_y\beta}d\omega_xd\omega_y
\\
&= \int_{\omega_x}\int_{\omega_y}\Re\left\{\int_x\int_yI(x,y)e^{j\omega_xx}e^{j\omega_yy}dxdy\right\} e^{j\omega_x\alpha}e^{j\omega_y\beta}d\omega_xd\omega_y\\
&= \int_x\int_yI(x,y)\int_{\omega_x}\int_{\omega_y}\Re\left\{e^{j\omega_xx}e^{j\omega_yy}\right\} e^{j\omega_x\alpha}e^{j\omega_y\beta}d\omega_xd\omega_ydxdy
\end{align}

You can clearly see that the inner integral is the 2D Fourier transform of
$$
\cos(\omega_xx)\cos(\omega_yy) + \sin(\omega_xx)\sin(\omega_yy)
$$
which is
$$
\frac{1}{2}\left[\delta(x-\alpha)\delta(y-\beta) + \delta(x+\alpha)\delta(y+\beta)\right]
$$

Substituting the result to $I_m$ yields
$$
I_m(x,y) = \frac{1}{2}\left[I(x,y)+I(-x,-y)\right]
$$

Of course in your case $x,y>0$, however the discrete Fourier transform assumes your signal is $N$-periodic and you get
$$
I_m(x,y) = \frac{1}{2}\left[I(x,y)+I(N-x,M-y)\right]
$$
where $N,M$ are the dimensions of your image. I think you can see now why you got that result.


Calculating the numerical gradient of a volume in one direction at a time

The only difference I can see between your gradient3 function and MATLAB's gradient is that the latter returns the horizontal derivative as the first output, and your code returns as "x" derivative the vertical derivative.

Note that MATLAB arrays are stored such that the first index is vertical, and the second index is horizontal. Therefore, the code under case 'x', D(2:k-1,:,:) = (F(3:k,:,:)-F(1:k-2,:,:))/2 computes the vertical difference, which intuitively we'd call y derivative.

Comparing:

F = randn(20,20,20);
[FX,FY,FZ] = gradient(F);
isequal(FX,gradient3(F,'y')) % returns true

By the way, using convn you can speed up the computation of the finite difference derivative by about a factor 2x:

D = convn(F,[0.5,0,-0.5],'same');

This produces a different result at the edges of the image, which might or might not be a problem for you but is typically unimportant.


Relationship between $E_b/N_0$ and $\rm SNR$ with channel coding

The relationship between SNR and $E_b/N_0$ is independent of the code rate. Note that $E_b$ is the energy per data bit (not the coded bits), and $R_b$ is the (uncoded) data bit rate. As long as you keep using these values, you can use the formula given in your question. Of course, when going from an uncoded system to a coded system, the values of $E_b$ and $R_b$ will usually change, so the resulting SNR will generally be different.


Calculate transfer function of two parallel transfer functions in a feedback loop

Your two transfer functions are in parallel, i.e they simply add up. So your feedback transfer function is simply $G(z) = H_1(z)+H_2(z)$. You want to makes sure that the magnitude of $G(z)$ is smaller than one.

and the overall closed loop transfer function is

$$H(z) = \frac{Y(z)}{X(z)} = \frac{1}{1+H_1(z)+H_2(z)}$$


How to produce color image using edge detected images?

First of all, thanks for sharing this problem. I hope it can foster some good discussions here at stack-exchange. I guess this problem is commonly called Multi-Focus Multi-Image Fusion (MF-MIF).

Let's look at all the images together with their grayscale histograms and the gradient magnitude (information content in each image):

As the original question requires an edge-based (or rather gradient as I take it) method, I will follow the idea presented in the following paper:

  S. Paul, I.S. Sevcenco, P. Agathoklis, "Multi-exposure and Multi-focus
  image fusion in gradient domain", Journal of Circuits, Systems, and
  Computers, Vol. 25, No. 10 (2016), DOI: 10.1142/S0218126616501231
  https://pdfs.semanticscholar.org/114b/835c5ea86e93efff1027cd6da7188c0cc92f.pdf

Essentially, the method aims to blend the gradients of the luminance components of the
input images using the maximum gradient magnitude at each pixel location and then to obtain the fused luminance using a Haar wavelet-based image reconstruction technique. The algorithm also includes a Poisson solver at each resolution to eliminate artifacts that may appear due to the non-conservative nature of the resulting gradient.

The official source code of the method is given in this github repository and in your, kind of challenging dataset, the output looks like the following:

I am not sure if this is a sufficiently good output, but I bet further investigations can quickly lead to improved results. A small observation: The first image seems to be quite sharp. It seems like it has a global focus and already looks (only looks though) better than the resulting image.


Is it impossible to determine the inverse Z-transform without any other information?

It's true that the algebraic expression for the $\mathcal{Z}$-transform is generally not sufficient for computing the corresponding time-domain sequence. The additional information we need is the region of convergence (ROC). The ROC together with the expression for the $\mathcal{Z}$-transform uniquely determines the corresponding time-domain sequence.

For the given example

$$H(z)=\frac{1}{1-az^{-1}}\tag{1}$$

it is clear that the expression $(1)$ can be obtained in two ways. First,

$$\begin{align}-\sum_{n=-\infty}^{-1}a^nz^{-n}&=-\sum_{n=1}^{\infty}a^{-n}z^n\\&=-\frac{\frac{z}{a}}{1-\frac{z}{a}}\\&=\frac{1}{1-az^{-1}},\quad |z|<|a|\end{align}$$

corresponding to the sequence $h[n]=-a^nu[-n-1]$, and, secondly,

$$\sum_{n=0}^{\infty}a^nz^{-n}=\frac{1}{1-az^{-1}},\quad |z|>|a|$$

corresponding to $h[n]=a^nu[n]$. In both cases I've used the formula for the geometric series.

The only difference is the ROC, so it's crucial to know the ROC in addition to the expression for the $\mathcal{Z}$-transform. The ROC $|z|>|a|$ indicates a right-sided sequence, whereas the ROC $|z|<|a|$ corresponds to a left-sided sequence. For higher order systems, you can also have an annulus $|a|<|z|<|b|$ as ROC, corresponding to a two-sided sequence.

If the sequence is interpreted as the impulse response of a system, then the system is stable if the ROC contains the unit circle $|z|=1$. It is causal if the sequence is zero for $n<0$.


How to generate a signal which frequency are obtained?

Your code is wrong because when you want to get a signal whose frequency changes with time, then the argument of the sine you generate is given by the integral of the function you want to generate in frequency (see https://en.wikipedia.org/wiki/Chirp). That's why, in your case, your f function should be:

$f(t)=\int_0^t800+500\sin{2\pi t}=\phi_0 + 800t -\frac{500}{2\pi} \cos(2\pi t)$

If you set the zero phase $\phi_0=0$, then you would obtain the signal you are looking for. Apart from that, take into account that if you just want that signal in the positive part of the spectrum of your double sided spectrum, then you should do $e^{if(t)}$, because if you just take the sine of $f(t)$ then the signal would be replicated in your negative side of the spectrum.


Coloring a waveform with spectral centroid or by other means

You can try first the following (no segmentation):

Process the signal in small chunks (say 10ms to 50ms in duration) - if necessary with a 50% overlap between them.
Compute the spectral centroid on each chunk.
Apply a non-linear function to the spectral centroid value to get a uniform distribution of the palette color used. Logarithm is a good start. Another option is to compute first the distribution of the centroid value other the entire file, and then color according to percentiles of this distribution (CDF). This adaptive approach guarantees that each color in the palette will be used in equal proportion. The drawback is that what is plotted in blue in one file won't be similar sounding to what is plotted in blue in another file if this adaptive approach is used!

It is not clear from the picture if Serato does this, or indeed goes one step further and attempts to segment the signal - it might not be surprising that it does since knowing the instants at which notes are present in a music signal can help synchronizing them! The steps would be:

Compute a Short-Term Fourier Transform (spectrogram) of the signal - using the FFT on short overlapping segments (start with a FFT size of 1024 or 2048 with 44.1kHz audio).
Compute an onset-detection function. I recommend you to look at this paper - the proposed approach is very effective and even a C++ implementation takes less than 10 lines. You can find an implementation in Yaafe - as ComplexDomainOnsetDetection.
Detect peaks in the onset-detection function to get the position of note onsets.
Compute the spectral centroid over all the time segments delimited by the detected onsets (don't forget to window and/or zero-pad!)
And don't forget the non-linear map! Note that the gradient-effect that appear between each note on the Serato waveform might be artificially generated.

Once you get this working, a "low hanging fruit" would be to compute a few other features on each segment (moments, a handful of MFCCs...), run k-means on these feature vectors, and decide on the color of the segment using the cluster index. See section II of Ravelli's paper.


Applying a mathematical function to a 16-bit Python array

If you're doing ANY audio signal processing in python, I heartily recommend librosa. Your scipy route will work (read as float, process, playback), but it is less direct. Also, I recommend sticking with float, as fixed-point work is more relevant if your code is running on an embedded/mobile system sans-FPU.

You can pip install librosa and get applying math functions quickly. Check out the introduction ipython notebook. Additionally, if you're doing more serious work, there is great integration with scikit-learn machine learning pipelines.

To get you started your code might look like this:

import librosa
import numpy as np

# Load the example track
y, sr = librosa.load("myfile.wav")

# apply mathematical functions or processes to 'y' here:

# If you do this in an ipython notebook (highly recommended as a interactive python env),
# you can play back in one line!
IPython.display.Audio(data=y, rate=sr)


Expressing 2N point DFT in terms of N point DFT

Consider a sequence $x[n]$ of length $N$ whose $N$-point DFT is $X[k]$. Then let $X_2[k]$ be the $2N$-point DFT of $x[n]$.

As you have stated, the even indexed samples of $X_2[k]$ will be easily shown to be: $$X_2[k] = X[k/2] ~~~,~~~k = 2m, m=0,1,...,N-1$$

Then the odd indexed samples of $X_2[k]$ will be given by the $N$-point DFT of the signal  $x[n] e^{-j \frac{\pi}{N} n } $, $n=0,1,...,N-1$.

$$X_{2}[k] = \sum_{n=0}^{N-1} x[n] e^{-j \frac{\pi}{N} n} e^{-j \frac{2\pi}{N} k n}  = X[k + 0.5]  ~~~~ ,~~~ k =2m+1, m=0,1,...,N-1 $$


[Beginning of the story] Remember the discrete wavelet curse: in 1D, with 2-scale or dyadic wavelets, you cannot have finite support, realness, orthogonality and linear phase (symmetry/antisymmetry) at the same time, except for the Haar wavelet, which lacks of regularity and overlap. You have to lift one constraint to have the other fulfilled. For instance :

if you lift symmetry, you get Daubechies wavelets
if you lift the dyadic, 2-scale, you get FIR $M$-band filter banks, etc.

When extending discrete wavelets to 2D, many options appear. Genuine two-dimensional dyadic wavelets exist (e.g. Design of Regular Nonseparable Bi-dimensional Wavelets Using Gröbner Basis Techniques, 1998), but their lack for separability make them cumbersome (and not often used in practice).

So, most people stick to the large background of known 1D dyadic wavelet designss and apply them on rows an columns of cartesian sampled data, like images. Thus is clearly a lack of imagination, and more genuine 2D, oriented, geometrical wavelets are possible, but the SE margin is too small to wrote about it.

Meanwhile, sticking to 1D wavelet designs, the order of operations on rows and columns matter.

Two main schemes are classics, but depending on the literature, they are more or less known, and more or less used, and often under different names, so:

process all rows then all columns (or reverse): this scheme seems better known in numerical analysis and partial differential equations. It can be found under many names: separable, standard, S-form, rectangular, anisotropic, tensor, hyperbolic, separated.
process rows and columns alternatively: this scheme seems better known in image/video processing. It can be found under many names: non-separable wavelet transform, non-standard form, NS-form, square wavelet transform, isotropic wavelet transform, Mallat decomposition, isotropic wavelet transform, combined wavelet transform.

Some papers are given  here, and it is detailed in "Chapter 3. Oriented and geometrical multiscale representations" of the 2011 review paper on 2D wavelets: A panorama on Multiscale Geometric Representations, intertwining spatial, directional and frequency selectivity.

What is the best? I sure do not know, I I am still working on this. But it's geeting late, and I am attending these days a conference in honor of Alexandre Grossmann and Yves Meyer on wavelets in Paris, and I shall wake up early.

[EDIT: Added references to related questions]

2D DWT Image Issue
What is the correct order of operations for a 2D Haar wavelet decomposition?
2D DWT computation order

Let me backlet... rrr


Solving for frequency deviation, $\Delta f $ (FM modulation)

If you use the standard definition of frequency deviation as the

  "maximum instantaneous difference between an FM modulated frequency and the nominal carrier frequency"

then for the given example you indeed have $\Delta f=50\;\text{kHz}$, because the nominal carrier frequency is $f_c=10\;\text{MHz}$, the maximum frequency is $f_c+\Delta f=10.05\;\text{MHz}$, and the minimum frequency is $f_c-\Delta f=9.95\;\text{MHz}$.


You can see by definition that your system is LTI System.
Moreover it is casual as it is only depends on input from the past.

The quaestion asks for response on the Unit Pulse signal on time 0.
In LTI Casual System it is the coefficient which responds to to input at time $ n $ -> Its value is $ \frac{1}{3} $.

I'm not sure where $ h \left[ 1 \right] + h \left[ 5 \right] $ is from though.
As clearly this system can be represented by 3 Coefficients which you found in Z Domain (Though they are also given from the finite differences equation).