Topics

Guided arcs, and the standalone explorations around them

The standard kit for a topic is theory plus a clean, importable Python module with tests; a few older pages are prose with inline code only, and some topics add a notebook of runnable experiments. Topics marked [HW] include embedded C/C++ implementations, by default for the ESP32-S3 and STM32F4 (NUCLEO-F446RE) pair; some range wider, from an 8-bit AVR to a Cortex-M33 with an NPU, where crossing capability tiers is itself the lesson.

The tone here is research-flavored. Open questions are flagged, not hidden. Where something doesn’t work well, we say so.

Looking for biquads, smoothing, multirate, zero-phase filtering, zero-crossing detection, or detrending? Those are chapters of the Basics learning path.

Guided arcs

Three groups of pages are written as sequences rather than as separate articles: each page assumes the one before it and adds exactly one new idea. The arc overview is the page that explains why that order, and it is the right place to start; the pages below it are listed in reading order.

Estimation & detection

Every measurement in noise is a number or a verdict, both have a complete theory, and the theory’s speed limit is computable before you build anything. Estimation & detection threads foundations first, then five instruments, each adding one unknown the previous page assumed away, then the theory that lets an estimate keep up with streaming data, and finally the whole arc wired into one running instrument. Entry level: best read after the noise arc, with some comfort in basic probability.

  1. Estimation basics: what makes one estimator better than another: bias, variance, MSE, the √N law, weighted least squares, and the Cramér-Rao speed limit. Opens the arc as the bridge from probability to likelihood
  2. Estimating a sinusoid: amplitude, phase, and frequency of a tone in noise; why frequency accuracy improves as N^-3/2 rather than N^-1/2, and the threshold SNR where every estimator breaks [HW]
  3. Lock-in detection: measuring microvolts under a mountain of 1/f noise by modulating up and demodulating back down, and an LED/photodiode photometer that resolves a fortieth of an ADC step [HW]
  4. Detection theory: hypotheses, thresholds, ROC curves, and Neyman-Pearson; why the matched filter is optimal, what the energy detector pays for not knowing the waveform, and a CFAR detector that holds its false-alarm rate without calibration [HW]
  5. Sequential change detection: the detector that never stops: Wald’s sequential test, Page’s CUSUM and the one-line recursion that is secretly a maximisation over every possible change time, the run-length trade-off that buys quiet almost for free, and the GLR for changes whose size nobody can name in advance [HW]
  6. Time-delay estimation: every “where” is a “when”: sonar ranging and two-microphone bearings from a correlation peak, the bandwidth-times-√ENR precision limit, GCC-PHAT for echoes, and a centroid method for channels where “the delay” is not one number [HW]
  7. Recursive estimation: the same least-squares answer, computed as the data arrives: Sherman-Morrison to RLS, why forgetting costs a variance floor, the Kalman filter as recursive MMSE, and total least squares for when the regressors lie
  8. A voice pitch estimator: the arc’s capstone: bandpass front end, streaming periodogram, cepstral f0 with the octave guard, a two-feature VAD with measured distributions, and a Hampel-gated pitch track, shipped on an ESP32 wearable prototype for speech-therapy feedback [HW]

Noise & stochastic processing

Noise as a first-class DSP subject: where it comes from, how to measure it, how to build it, and when adding more of it actually helps. Noise & stochastic processing is the guided arc that threads them in order, from the generative processes behind every noise type to the closing case where noise improves detection. Entry level: noise and SNR and the frequency domain from Basics.

  1. Random processes: the six process classes behind every noise type, and what breaks when you assume Gaussian [HW]
  2. PSD estimation: from the broken periodogram to Welch’s method, with confidence intervals [HW]
  3. Noise generation: building noise to a specification: Box-Muller, Voss-McCartney, Perlin, and Simplex [HW]
  4. Dither: deliberately adding noise to linearise quantisation; the TPDF that makes digital audio work [HW]
  5. ADC noise: beyond the 6.02 formula: jitter, ENOB, oversampling, and sigma-delta noise shaping [HW]
  6. Image noise: Gaussian, salt-and-pepper, speckle, and photon shot noise in two dimensions, including a real-time 3×3 median filter on an ESP32 camera [HW]
  7. Stochastic resonance: when adding noise improves signal detection; the counterintuitive closer

Feature extraction

Reducing a signal to a handful of numbers, arranged from the richest spectral front end to the cheapest time-domain one. Feature extraction is the guided arc that threads all five in order and explains when to reach for which. Entry level: comfortable with the frequency domain from Basics; the wavelets page additionally leans on multirate filter banks.

  1. Short-Time Fourier Transform: the sliding-window DFT behind every spectrogram, the time-frequency resolution trade-off, and inverting it with overlap-add, plus a real-time block FFT on the metal [HW]
  2. Spectral features: describing a spectrum’s shape in five numbers (centroid, spread, rolloff, flux, flatness), the cheap front end of classifiers and voice-activity gates, all from the same FFT bins on the metal [HW]
  3. Wavelets: when one window size is not enough, a transform whose tiles change shape with frequency, from the scalogram to a filter-bank DWT, denoising, and integer lifting on the metal [HW]
  4. Mel-frequency cepstral coefficients: the speech feature front end, from pre-emphasis through the mel filterbank to the DCT, and a training-free wakeword detector on the metal [HW]
  5. Statistical features: summarising a window of samples with a handful of statistics (mean, variance, RMS, skew, kurtosis, ZCR), the cheapest feature front end there is, and an on-device activity classifier and voice gate from an 8-bit AVR up [HW]

Standalone topics

These do not belong to an arc and can be read in any order; each states its own prerequisites at the top. The one exception is the outlier-detection and noise-whitening pair, written as two parts of one argument.

  • Outlier detection: streaming anomaly detection using robust statistics
  • Noise whitening: characterising and whitening 1/f^α noise; part two of the outlier-detection pair
  • Adaptive filtering: LMS, NLMS, and RLS for system identification and noise cancellation [HW]
  • Matched filtering: detecting known signals in noise, from bat echolocation to LIGO [HW]
  • Model-based filtering: optimal estimation when you model the signal and noise statistics, from the Wiener filter to a Kalman tracker on the metal [HW]
  • Beamforming: direction-of-arrival estimation with sensor arrays, inspired by the scorpion [HW]
  • Gammatone filters: the cochlea as a bank of bandpass filters, one auditory channel per four biquads [HW]
  • Gabor filters: oriented visual-cortex receptive fields, from the uncertainty principle to 2-D convolution on hardware [HW]
  • PSO for filter design: particle swarm optimization for the non-convex IIR designs where gradients get stuck, plus on-device adaptation [HW]
  • Finite word-length effects: what quantization, round-off noise, coefficient sensitivity, and limit cycles do to a fixed-point filter, and why second-order sections survive where direct forms go unstable
  • The Goertzel algorithm: evaluating a single DFT bin with a second-order IIR filter, from the math to a real-time DTMF decoder on the metal [HW]
  • Pitch detection: estimating fundamental frequency from autocorrelation to real-time embedded systems [HW]
  • Empirical mode decomposition: data-driven decomposition into intrinsic mode functions
  • PPG signal processing: a complete DSP pipeline for heart rate extraction, case study with ESP32 and MAX30102 [HW]