Estimation & Detection

From signals to decisions: eight pages on extracting numbers, and verdicts, from noise

How well can you measure this?

The noise arc spent seven pages understanding the enemy, and it kept asking that question at every turn: the PSD page put confidence intervals on its spectra, the ADC page counted effective bits, the dither page measured what a quantiser does to a small signal. This arc answers the question with the tools it deserves. Every measurement in noise is one of two problems in disguise: estimation (the answer is a number: a frequency, an amplitude, a delay) or detection (the answer is a verdict: present or absent, normal or anomalous). Both have a complete, quantitative theory, the theory has a speed limit, and the limit is computable before you build anything.

That last clause is the arc’s engineering payoff, and its recurring move. The Cramér-Rao lower bound turns “how well can you measure this?” from a rhetorical sigh into a design equation: the lock-in page sizes a photometer from $S(f_0)/T$ before soldering it, the detection page picks a CFAR threshold from an F-distribution and then counts its false alarms on the bench, and the time-delay page budgets a direction finder’s degrees from bandwidth and SNR. Math to metal, with the bound as the contract between them.

The arc

The order is deliberate: one page of foundations, then five instruments, each adding one unknown the previous page assumed away, then the page that sets the whole ladder in motion, sample by sample, and a capstone that runs the whole arc in one instrument.

Estimation basics: what makes an estimator good: bias, variance, MSE, consistency, and the √N law; weighted least squares when samples are not equally trustworthy; Fisher information and the CRLB as the speed limit; maximum likelihood as the recipe that approaches it. Capstone: the periodogram of the noise arc is an approximate ML estimator, derived on the PSD page.
Estimating a sinusoid: the theory pointed at the most common measurement there is: amplitude, phase, and frequency of a tone in noise. The coherent single-bin DFT, the startling $1/N^3$ frequency law, interpolating the FFT peak off its grid, and the threshold SNR below which every estimator falls off a cliff. [HW]: a tone tracker sized by its CRLB before it is built.
Lock-in detection: what to do when the signal is under the noise floor and the noise is 1/f: modulate up, demodulate synchronously back down. The averaging wall, the $\sqrt{S(f_0)/T}$ noise floor, and the Rayleigh floor that hands off to detection theory. [HW]: an LED/photodiode photometer that resolves a fortieth of an ADC step.
Detection theory: when the answer is a verdict, not a number. The likelihood ratio as the only statistic anyone needs, Neyman-Pearson and the ROC, the matched filter’s optimality upgraded to a theorem, the energy detector’s price for ignorance, and CFAR, a detector whose false-alarm rate survives an unknown noise level. Re-reads the workshop’s ad-hoc detectors (VAD rules, Hampel fences, Goertzel thresholds) as instances of the theory. [HW]: a self-calibrating burst detector, verified by counting alarms.
Sequential change detection: the detector that never stops. Fixed windows decide about a block; monitoring asks whether the process itself has changed, with no window, no false-alarm probability (only a rate), and no single guilty sample. Wald’s sequential test, Page’s CUSUM and the one-line recursion that is a maximisation over every possible change time, the run-length trade that costs only $2/\delta^2$ samples of delay per e-fold of quiet, and the GLR for when nobody can say how large the change will be. [HW]: a drift monitor riding the CFAR detector’s own block-energy stream, in integer arithmetic.
Time-delay estimation: every “where” is a “when”. Sonar ranging and microphone-pair bearings from a correlation peak, the bandwidth-times-√ENR precision limit (and why accuracy beats resolution by orders of magnitude), GCC-PHAT when the room echoes, and the centroid method when dispersion smears “the” delay into a question. [HW]: a two-microphone direction finder whose dominant error is not noise but a half-sample of channel skew.
Recursive estimation: everything so far waited for the whole record; this page computes the same answers as the data arrives. One algebraic identity (Sherman-Morrison) turns batch least squares into RLS, forgetting buys tracking at the price of a variance floor, and two more steps reveal the Kalman filter as recursive MMSE, giving model-based filtering’s Kalman page the theory it deserves. Plus total least squares, for when the regressors themselves are noisy and ordinary LS converges confidently to the wrong answer.
A voice pitch estimator: the capstone: every page above, wired into one running system. A speaking-voice pitch tracker for speech-therapy feedback: bandpass front end, streaming-averaged periodogram, cepstral $f_0$ readout (quefrency read the right way around, with the octave guard), a two-feature VAD whose thresholds come with measured distributions, and a Hampel gate on the pitch track; closed by an honest account of where classical stops and deep-learning estimators take over. [HW]: the one piece of the arc where the metal predates the page: the design shipped on an ESP32 wearable prototype in 2022, and the embedded page walks its real-time architecture.

One table: what is unknown, what it costs

The arc’s inner logic is a ladder of ignorance. Each rung leaves one more quantity unknown, and the theory prices every step.

Page	You know	You don’t know	The price, quantified
Estimating a sinusoid	frequency (coherent case)	amplitude, phase	the CRLB baseline
Estimating a sinusoid	waveform class	also the frequency	threshold effect; $1/N^3$ is the consolation
Lock-in detection	modulation frequency & phase	a signal under the 1/f floor	integration time: depth costs bandwidth
Detection theory	the waveform	whether it is there	vs knowing nothing of the waveform: ~$\sqrt{2N}$ more energy (energy detector)
Detection theory	(as above)	even the noise level	CFAR loss: 0.2 dB with 16 reference blocks
Change detection	how large a change matters	when it happened, or whether it has yet	delay $\approx 2h/\delta^2$; each e-fold of quiet costs only $2/\delta^2$ more samples
Change detection	(as above)	also how large the change is	the GLR’s $O(W)$ arithmetic, and a threshold with no usable closed form: 3.6x hotter than the naive one
Time-delay estimation	the waveform	when it arrives	$1/(\bar\beta^2 \cdot \mathrm{ENR})$
Time-delay estimation	(as above)	also the reference arrival	the pairwise bound: exactly double
Recursive estimation	the model	whether the parameter stays put	tracking’s variance floor: $\sigma^2\frac{1-\lambda}{1+\lambda}$ instead of $\sigma^2/k \to 0$
Voice pitch estimator	the pitch band	$f_0$, and whether anyone is speaking at all	quefrency grid resolution $f_0^2/f_s$; the voicing threshold priced by its noise distribution ($\max$-of-$M$ z-score $\approx \sqrt{2\ln M}$)

The verification thread

This arc has a second, quieter storyline: almost every closed form in it was caught wrong, or nearly wrong, at least once during the build: a bound quoted in the wrong SNR convention, a chi-square with one degree of freedom too many, a frequency sum over the wrong half of the spectrum. The working rules that emerged are on every page: any CRLB is checked against a numerically built Fisher matrix; any detection-statistic distribution is checked by Monte Carlo and an independent special case; any claimed relationship between plotted quantities is asserted, not eyeballed. The change-detection page pushed the habit one step further, computing every run length three independent ways and requiring them to agree, which promptly caught the checking apparatus itself being subtly wrong before it could referee anything. The first-party archive sources got the same treatment, and the scorecard is part of the story: a 2003 derivation carried three slips, a 2002 lecture chapter carried none, a 2000s research note ended at an apparent paradox that one correct matrix inversion resolves (the full account), and a 2001 thesis’s code folder needed an authorship audit before anything in it could be quoted at all (three lineages, only one of them the author’s; the stratigraphy). Every one of those checks is written down as a verification record kept alongside the source.

The math-to-metal thread

Six of the eight pages carry embedded.qmd companions on the ADR-005 platforms, and they escalate deliberately:

Estimating a sinusoid: the CRLB used forward: sizing an instrument’s record length from the precision spec before the firmware exists.
Lock-in detection: timing as arithmetic: the carrier integrated over whole cycles ($f_0 = f_s/9$) and the integration length a whole number of mains cycles, so the 50 Hz comb and ambient light land on exact nulls by construction, resolving 20 µV behind a 12-bit ADC.
Detection theory: statistics as a self-test: a detector whose designed false-alarm rate is counted on the bench, where both too many alarms and silence are failures.
Change detection: arithmetic cheap enough to be a passenger: an integer CUSUM riding the CFAR detector’s existing block-energy stream, recovering the sensor-health information that detector is designed to throw away, for 276 bytes and a few tens of cycles per block. Also the page where the theory openly runs out: a threshold set for one false alarm per hour is a rare-event threshold, and the Gaussian design misses it by a factor of two.
Time-delay estimation: simultaneity as the spec: dual-simultaneous ADC conversion and shared-clock I2S because a fixed half-sample skew is a bearing bias no averaging can touch.
Voice pitch estimator: the thread run in the other direction: the hardware existed first (an ESP32/TinyPico wearable prototype), and the arc supplies the theory its design decisions were waiting for; the embedded page documents the real system’s FreeRTOS architecture, I2S capture, and cycle budget, with a NUCLEO-F446RE port priced as a feasibility note.

Estimation basics has no hardware page; its metal is everyone else’s. Recursive estimation deliberately borrows its metal too: the Kalman tracker on model-based filtering’s embedded page already runs this page’s recursion on the ADR-005 hardware.

Where to start

New to the subject: read pieces 1 through 8 in order: each instrument leans on the foundations page, and the later pages reuse the earlier ones’ machinery (the sinusoid page’s threshold effect returns in time-delay estimation; the lock-in’s Rayleigh floor is closed by detection theory; the foundations page’s weighted least squares comes back to life, one sample at a time, in recursive estimation; and the capstone spends everything at once). Here for a specific problem: the ignorance table above is the index: find the row that matches what you don’t know.

--- title: "Estimation & Detection" subtitle: "From signals to decisions: eight pages on extracting numbers, and verdicts, from noise" --- > *How well can you measure this?* The [noise arc](noise-and-stochastic-processing.qmd) spent seven pages understanding the enemy, and it kept asking that question at every turn: the PSD page put confidence intervals on its spectra, the ADC page counted effective bits, the dither page measured what a quantiser does to a small signal. This arc answers the question with the tools it deserves. Every measurement in noise is one of two problems in disguise: **estimation** (the answer is a number: a frequency, an amplitude, a delay) or **detection** (the answer is a verdict: present or absent, normal or anomalous). Both have a complete, quantitative theory, the theory has a speed limit, and the limit is computable *before you build anything*. That last clause is the arc's engineering payoff, and its recurring move. The Cramér-Rao lower bound turns "how well can you measure this?" from a rhetorical sigh into a design equation: the lock-in page sizes a photometer from $S(f_0)/T$ before soldering it, the detection page picks a CFAR threshold from an F-distribution and then *counts* its false alarms on the bench, and the time-delay page budgets a direction finder's degrees from bandwidth and SNR. Math to metal, with the bound as the contract between them. ## The arc The order is deliberate: one page of foundations, then five instruments, each adding one unknown the previous page assumed away, then the page that sets the whole ladder in motion, sample by sample, and a capstone that runs the whole arc in one instrument. 1. **[Estimation basics](estimation-basics/index.qmd)**: what makes an estimator good: bias, variance, MSE, consistency, and the √N law; weighted least squares when samples are not equally trustworthy; Fisher information and the CRLB as the speed limit; maximum likelihood as the recipe that approaches it. Capstone: the periodogram of the noise arc *is* an approximate ML estimator, [derived on the PSD page](psd-estimation/index.qmd#why-the-periodogram-it-is-almost-maximum-likelihood). 2. **[Estimating a sinusoid](estimating-a-sinusoid/index.qmd)**: the theory pointed at the most common measurement there is: amplitude, phase, and frequency of a tone in noise. The coherent single-bin DFT, the startling $1/N^3$ frequency law, interpolating the FFT peak off its grid, and the threshold SNR below which every estimator falls off a cliff. **[HW]**: a tone tracker sized by its CRLB before it is built. 3. **[Lock-in detection](lock-in-detection/index.qmd)**: what to do when the signal is *under* the noise floor and the noise is 1/f: modulate up, demodulate synchronously back down. The averaging wall, the $\sqrt{S(f_0)/T}$ noise floor, and the Rayleigh floor that hands off to detection theory. **[HW]**: an LED/photodiode photometer that resolves a fortieth of an ADC step. 4. **[Detection theory](detection-theory/index.qmd)**: when the answer is a verdict, not a number. The likelihood ratio as the only statistic anyone needs, Neyman-Pearson and the ROC, the matched filter's optimality upgraded to a theorem, the energy detector's price for ignorance, and CFAR, a detector whose false-alarm rate survives an unknown noise level. Re-reads the workshop's ad-hoc detectors (VAD rules, Hampel fences, Goertzel thresholds) as instances of the theory. **[HW]**: a self-calibrating burst detector, verified by counting alarms. 5. **[Sequential change detection](change-detection/index.qmd)**: the detector that never stops. Fixed windows decide about a block; monitoring asks whether the process itself has changed, with no window, no false-alarm probability (only a rate), and no single guilty sample. Wald's sequential test, Page's CUSUM and the one-line recursion that *is* a maximisation over every possible change time, the run-length trade that costs only $2/\delta^2$ samples of delay per e-fold of quiet, and the GLR for when nobody can say how large the change will be. **[HW]**: a drift monitor riding the CFAR detector's own block-energy stream, in integer arithmetic. 6. **[Time-delay estimation](time-delay-estimation/index.qmd)**: every "where" is a "when". Sonar ranging and microphone-pair bearings from a correlation peak, the bandwidth-times-√ENR precision limit (and why accuracy beats resolution by orders of magnitude), GCC-PHAT when the room echoes, and the centroid method when dispersion smears "the" delay into a question. **[HW]**: a two-microphone direction finder whose dominant error is not noise but a half-sample of channel skew. 7. **[Recursive estimation](recursive-estimation/index.qmd)**: everything so far waited for the whole record; this page computes the *same* answers as the data arrives. One algebraic identity (Sherman-Morrison) turns batch least squares into RLS, forgetting buys tracking at the price of a variance floor, and two more steps reveal the Kalman filter as recursive MMSE, giving [model-based filtering](model-based-filtering/index.qmd)'s Kalman page the theory it deserves. Plus total least squares, for when the regressors themselves are noisy and ordinary LS converges confidently to the wrong answer. 8. **[A voice pitch estimator](voice-pitch-estimator/index.qmd)**: the capstone: every page above, wired into one running system. A speaking-voice pitch tracker for speech-therapy feedback: bandpass front end, streaming-averaged periodogram, cepstral $f_0$ readout (quefrency read the right way around, with the octave guard), a two-feature VAD whose thresholds come with measured distributions, and a Hampel gate on the pitch track; closed by an honest account of where classical stops and deep-learning estimators take over. **[HW]**: the one piece of the arc where the metal predates the page: the design shipped on an ESP32 wearable prototype in 2022, and the [embedded page](voice-pitch-estimator/embedded.qmd) walks its real-time architecture. ## One table: what is unknown, what it costs The arc's inner logic is a ladder of ignorance. Each rung leaves one more quantity unknown, and the theory prices every step. | Page | You know | You don't know | The price, quantified | |---|---|---|---| | Estimating a sinusoid | frequency (coherent case) | amplitude, phase | the CRLB baseline | | Estimating a sinusoid | waveform class | also the frequency | threshold effect; $1/N^3$ is the consolation | | Lock-in detection | modulation frequency & phase | a signal under the 1/f floor | integration time: depth costs bandwidth | | Detection theory | the waveform | whether it is there | vs knowing nothing of the waveform: ~$\sqrt{2N}$ more energy (energy detector) | | Detection theory | (as above) | even the noise level | CFAR loss: 0.2 dB with 16 reference blocks | | Change detection | how large a change matters | *when* it happened, or whether it has yet | delay $\approx 2h/\delta^2$; each e-fold of quiet costs only $2/\delta^2$ more samples | | Change detection | (as above) | also how large the change is | the GLR's $O(W)$ arithmetic, and a threshold with no usable closed form: 3.6x hotter than the naive one | | Time-delay estimation | the waveform | when it arrives | $1/(\bar\beta^2 \cdot \mathrm{ENR})$ | | Time-delay estimation | (as above) | also the reference arrival | the pairwise bound: exactly double | | Recursive estimation | the model | whether the parameter stays put | tracking's variance floor: $\sigma^2\frac{1-\lambda}{1+\lambda}$ instead of $\sigma^2/k \to 0$ | | Voice pitch estimator | the pitch band | $f_0$, and whether anyone is speaking at all | quefrency grid resolution $f_0^2/f_s$; the voicing threshold priced by its noise distribution ($\max$-of-$M$ z-score $\approx \sqrt{2\ln M}$) | ## The verification thread This arc has a second, quieter storyline: **almost every closed form in it was caught wrong, or nearly wrong, at least once during the build**: a bound quoted in the wrong SNR convention, a chi-square with one degree of freedom too many, a frequency sum over the wrong half of the spectrum. The working rules that emerged are on every page: any CRLB is checked against a numerically built Fisher matrix; any detection-statistic distribution is checked by Monte Carlo *and* an independent special case; any claimed relationship between plotted quantities is asserted, not eyeballed. The [change-detection page](change-detection/index.qmd) pushed the habit one step further, computing every run length three independent ways and requiring them to agree, which promptly caught the *checking* apparatus itself being subtly wrong before it could referee anything. The first-party archive sources got the same treatment, and the scorecard is part of the story: a 2003 derivation carried three slips, a 2002 lecture chapter carried none, a 2000s research note ended at an apparent paradox that one correct matrix inversion resolves ([the full account](time-delay-estimation/index.qmd#two-noisy-sensors-the-bound-doubles)), and a 2001 thesis's code folder needed an authorship audit before anything in it could be quoted at all (three lineages, only one of them the author's; [the stratigraphy](recursive-estimation/index.qmd)). Every one of those checks is written down as a verification record kept alongside the source. ## The math-to-metal thread Six of the eight pages carry `embedded.qmd` companions on the [ADR-005](../docs/adr/005-embedded-platforms.md) platforms, and they escalate deliberately: - **Estimating a sinusoid**: the CRLB used *forward*: sizing an instrument's record length from the precision spec before the firmware exists. - **Lock-in detection**: timing as arithmetic: the carrier integrated over whole cycles ($f_0 = f_s/9$) and the integration length a whole number of mains cycles, so the 50 Hz comb and ambient light land on exact nulls by construction, resolving 20 µV behind a 12-bit ADC. - **Detection theory**: statistics as a self-test: a detector whose designed false-alarm rate is *counted* on the bench, where both too many alarms *and silence* are failures. - **Change detection**: arithmetic cheap enough to be a passenger: an integer CUSUM riding the CFAR detector's existing block-energy stream, recovering the sensor-health information that detector is designed to throw away, for 276 bytes and a few tens of cycles per block. Also the page where the theory openly runs out: a threshold set for one false alarm per hour is a rare-event threshold, and the Gaussian design misses it by a factor of two. - **Time-delay estimation**: simultaneity as the spec: dual-simultaneous ADC conversion and shared-clock I2S because a fixed half-sample skew is a bearing bias no averaging can touch. - **Voice pitch estimator**: the thread run in the other direction: the hardware existed first (an ESP32/TinyPico wearable prototype), and the arc supplies the theory its design decisions were waiting for; the [embedded page](voice-pitch-estimator/embedded.qmd) documents the real system's FreeRTOS architecture, I2S capture, and cycle budget, with a NUCLEO-F446RE port priced as a feasibility note. Estimation basics has no hardware page; its metal is everyone else's. Recursive estimation deliberately borrows its metal too: the Kalman tracker on [model-based filtering's embedded page](model-based-filtering/embedded.qmd) already runs this page's recursion on the ADR-005 hardware. ## Where to start New to the subject: read pieces 1 through 8 in order: each instrument leans on the foundations page, and the later pages reuse the earlier ones' machinery (the sinusoid page's threshold effect returns in time-delay estimation; the lock-in's Rayleigh floor is closed by detection theory; the foundations page's weighted least squares comes back to life, one sample at a time, in recursive estimation; and the capstone spends everything at once). Here for a specific problem: the ignorance table above is the index: find the row that matches what you don't know.