Feature Extraction

Five ways to turn a signal into a handful of numbers

A raw signal is too many numbers. A second of audio is sixteen thousand samples; a minute of accelerometer data is thousands more. Almost nothing downstream wants them all. A classifier, a voice-activity gate, a music-similarity search, an activity recogniser: each wants a short, descriptive summary of a window of signal, a few numbers per frame that capture what is happening without storing every sample. Producing those numbers is feature extraction, and it is the bridge between a signal and a decision.

Five Topics pages cover the feature front ends worth knowing, from the richest and most expensive to the cheapest. They were written as a deliberate sequence, and this page is the thread that ties them together: what each one is for, how they build on each other, and when you would reach for which. If you only read the Topics section as a reference, read these five in order at least once.

The arc

The order is load-bearing. Each page leans on the one before it.

The Short-Time Fourier Transform is the foundation. A single Fourier transform assumes the spectrum never changes; the STFT chops the signal into short frames and transforms each one, giving a spectrum that evolves in time. That time-frequency picture, the data behind every spectrogram, is what the next three pages describe, summarise, or improve on. Start here.
Spectral features collapse each frame’s spectrum into five interpretable numbers: where the energy sits (centroid), how spread out it is (spread), where it tails off (rolloff), how fast it is changing (flux), and how tonal versus noise-like it is (flatness). This is the cheap, general-purpose front end of music information retrieval and the simplest viable timbre and voice classifiers. It depends directly on the STFT.
Wavelets step sideways. The STFT picks one frame length and lives with it: fine in frequency or fine in time, never both. The wavelet transform analyses the signal at many resolutions at once, wide windows for low frequencies and narrow ones for high, which is the trade-off most real signals actually want. It reads best after the STFT because it is defined by contrast with it, and it ties back to the multirate filter banks from the basics.
Mel-frequency cepstral coefficients specialise the spectral idea for speech. The pipeline warps the spectrum to match human pitch perception (the mel scale), takes a log, and decorrelates the result with a discrete cosine transform, yielding about thirteen numbers that describe the spectral envelope of a sound. It is built on the STFT and reads as the speech-flavoured sibling of spectral features. Its embedded companion, a training-free wakeword detector, is the running example the whole track was pointing at.
Statistical features drop the transform entirely. They describe the window of raw samples directly with first-year statistics: mean, variance, RMS, energy, peak-to-peak, skewness, kurtosis, and the zero-crossing rate. No FFT, no filterbank, just sums and a few powers. This is the cheapest feature front end there is, and it closes the arc by being both the baseline the others must beat and, very often, all you actually need.

Two ways to read the five

The arc runs along two axes at once. One is what domain you work in: the first four describe the spectrum (transform first, then summarise), while statistical features describe the time-domain window straight. The other is richness versus cost: an MFCC pipeline captures the fine shape of a vowel but needs an FFT every frame, while a windowed variance captures “is anything happening” for a few additions per sample.

Page	Works on	Output per frame	Cost	Typical job
STFT	the signal	a full spectrum	one FFT/frame	the time-frequency picture everything else reads
Spectral features	the STFT spectrum	~5 numbers	one FFT + a few sums	timbre, genre, music similarity
Wavelets	the signal	multi-resolution coefficients	a filter-bank cascade	transients, denoising, non-stationary signals
MFCC	the STFT spectrum	~13 numbers	FFT + mel + log + DCT	speech and keyword spotting
Statistical features	the raw window	a handful of numbers	a few ops/sample	activity recognition, the first VAD gate

A useful instinct: reach down the table first. Try statistical features before spectral ones, and spectral ones before MFCC. The cheaper feature is often enough, and when it is not, you have a baseline to measure the expensive one against.

The thread to the metal

Feature extraction is where the workshop’s “math to metal” pipeline pays off, because features are what actually run on a microcontroller in the field. Every page in the arc carries an embedded.qmd companion that takes the algorithm to hardware, and together they sketch a capability ladder:

Statistical features run on an 8-bit AVR. A windowed variance feeding a decision tree is a complete activity classifier in a few hundred bytes, and the same energy and zero-crossing rate make a voice gate on any part with an ADC.
Spectral features and the STFT want a Cortex-M4F or an ESP32-S3: one real FFT per hop, then the five features straight from its bins, comfortably inside a hop’s worth of time.
MFCC turns the STFT into a working wakeword detector on the ESP32-S3, the front end in C and a dynamic-time-warping classifier matching a few stored utterances, training-free and frugal enough for a coin cell when it is duty-cycled between wakewords.
Wavelets map cleanly onto multirate filter banks, with an integer lifting path for parts without a floating-point unit.

The lesson the ladder teaches is the one the whole site is about: the right feature is the one that fits the silicon you have, and the cheapest feature that does the job is usually the right one.

Closing the track

These five pages complete the feature-extraction extension planned for the workshop, the bounded scope the project set itself: stay in DSP, and extend through DSP-flavoured feature extraction rather than into standalone machine-learning topics. That line still holds: the arc ends at classical, training-free pattern recognition, which is why the wakeword detector uses dynamic time warping and the activity classifier uses a hand-built decision tree rather than a learned model. Each page’s closing section points at where the machine-learning continuation begins (TinyML, Edge Impulse, a learned back end fed these same features), but that is deliberately out of scope: the workshop’s job is to deliver clean, understandable features to the doorstep of a learning system, not to cross it.

If you have read the basics and want to see DSP earn its keep on real problems, this arc is the place the theory turns into numbers you can act on. There is also a sibling arc, noise & stochastic processing, that gives noise the same first-class treatment across seven pages: where it comes from, how to measure it, how to build it, and when adding more of it actually helps.

--- title: "Feature Extraction" subtitle: "Five ways to turn a signal into a handful of numbers" --- A raw signal is too many numbers. A second of audio is sixteen thousand samples; a minute of accelerometer data is thousands more. Almost nothing downstream wants them all. A classifier, a voice-activity gate, a music-similarity search, an activity recogniser: each wants a *short, descriptive summary* of a window of signal, a few numbers per frame that capture what is happening without storing every sample. Producing those numbers is **feature extraction**, and it is the bridge between a signal and a decision. Five Topics pages cover the feature front ends worth knowing, from the richest and most expensive to the cheapest. They were written as a deliberate sequence, and this page is the thread that ties them together: what each one is for, how they build on each other, and when you would reach for which. If you only read the Topics section as a reference, read these five in order at least once. ## The arc The order is load-bearing. Each page leans on the one before it. 1. **[The Short-Time Fourier Transform](stft/index.qmd)** is the foundation. A single Fourier transform assumes the spectrum never changes; the STFT chops the signal into short frames and transforms each one, giving a spectrum that evolves in time. That time-frequency picture, the data behind every spectrogram, is what the next three pages describe, summarise, or improve on. Start here. 2. **[Spectral features](spectral-features/index.qmd)** collapse each frame's spectrum into five interpretable numbers: where the energy sits (centroid), how spread out it is (spread), where it tails off (rolloff), how fast it is changing (flux), and how tonal versus noise-like it is (flatness). This is the cheap, general-purpose front end of music information retrieval and the simplest viable timbre and voice classifiers. It depends directly on the STFT. 3. **[Wavelets](wavelets/index.qmd)** step sideways. The STFT picks one frame length and lives with it: fine in frequency or fine in time, never both. The wavelet transform analyses the signal at many resolutions at once, wide windows for low frequencies and narrow ones for high, which is the trade-off most real signals actually want. It reads best after the STFT because it is defined by contrast with it, and it ties back to the multirate filter banks from the basics. 4. **[Mel-frequency cepstral coefficients](mfcc/index.qmd)** specialise the spectral idea for speech. The pipeline warps the spectrum to match human pitch perception (the mel scale), takes a log, and decorrelates the result with a discrete cosine transform, yielding about thirteen numbers that describe the spectral envelope of a sound. It is built on the STFT and reads as the speech-flavoured sibling of spectral features. Its embedded companion, a training-free wakeword detector, is the running example the whole track was pointing at. 5. **[Statistical features](statistical-features/index.qmd)** drop the transform entirely. They describe the window of raw samples directly with first-year statistics: mean, variance, RMS, energy, peak-to-peak, skewness, kurtosis, and the zero-crossing rate. No FFT, no filterbank, just sums and a few powers. This is the cheapest feature front end there is, and it closes the arc by being both the baseline the others must beat and, very often, all you actually need. ## Two ways to read the five The arc runs along two axes at once. One is **what domain you work in**: the first four describe the spectrum (transform first, then summarise), while statistical features describe the time-domain window straight. The other is **richness versus cost**: an MFCC pipeline captures the fine shape of a vowel but needs an FFT every frame, while a windowed variance captures "is anything happening" for a few additions per sample. | Page | Works on | Output per frame | Cost | Typical job | |---|---|---|---|---| | STFT | the signal | a full spectrum | one FFT/frame | the time-frequency picture everything else reads | | Spectral features | the STFT spectrum | ~5 numbers | one FFT + a few sums | timbre, genre, music similarity | | Wavelets | the signal | multi-resolution coefficients | a filter-bank cascade | transients, denoising, non-stationary signals | | MFCC | the STFT spectrum | ~13 numbers | FFT + mel + log + DCT | speech and keyword spotting | | Statistical features | the raw window | a handful of numbers | a few ops/sample | activity recognition, the first VAD gate | A useful instinct: reach down the table first. Try statistical features before spectral ones, and spectral ones before MFCC. The cheaper feature is often enough, and when it is not, you have a baseline to measure the expensive one against. ## The thread to the metal Feature extraction is where the workshop's "math to metal" pipeline pays off, because features are what actually run on a microcontroller in the field. Every page in the arc carries an `embedded.qmd` companion that takes the algorithm to hardware, and together they sketch a capability ladder: - **Statistical features** run on an 8-bit AVR. A windowed variance feeding a decision tree is a complete activity classifier in a few hundred bytes, and the same energy and zero-crossing rate make a voice gate on any part with an ADC. - **Spectral features and the STFT** want a Cortex-M4F or an ESP32-S3: one real FFT per hop, then the five features straight from its bins, comfortably inside a hop's worth of time. - **MFCC** turns the STFT into a working **wakeword detector** on the ESP32-S3, the front end in C and a dynamic-time-warping classifier matching a few stored utterances, training-free and frugal enough for a coin cell when it is duty-cycled between wakewords. - **Wavelets** map cleanly onto multirate filter banks, with an integer lifting path for parts without a floating-point unit. The lesson the ladder teaches is the one the whole site is about: the right feature is the one that fits the silicon you have, and the cheapest feature that does the job is usually the right one. ## Closing the track These five pages complete the feature-extraction extension planned for the workshop, the bounded scope the project set itself: stay in DSP, and extend through DSP-flavoured feature extraction rather than into standalone machine-learning topics. That line still holds: the arc ends at classical, training-free pattern recognition, which is why the wakeword detector uses dynamic time warping and the activity classifier uses a hand-built decision tree rather than a learned model. Each page's closing section points at where the machine-learning continuation begins (TinyML, Edge Impulse, a learned back end fed these same features), but that is deliberately out of scope: the workshop's job is to deliver clean, understandable features to the doorstep of a learning system, not to cross it. If you have read the [basics](../basics/index.qmd) and want to see DSP earn its keep on real problems, this arc is the place the theory turns into numbers you can act on. There is also a sibling arc, [**noise & stochastic processing**](noise-and-stochastic-processing.qmd), that gives noise the same first-class treatment across seven pages: where it comes from, how to measure it, how to build it, and when adding more of it actually helps.