Image Noise on Hardware

A 3×3 median filter for salt-and-pepper noise on an ESP32 camera

The main page shows a 3×3 median filter removing salt-and-pepper noise in Python. On a microcontroller with a camera, you run the same filter in real time (every pixel, every frame) and the result is visibly cleaner images from a sensor with dead or hot pixels. The algorithm is simple enough to run at QVGA (320×240) on any Cortex-M4 or ESP32, and it is the textbook example of a filter matched to its noise model: the median ignores outliers; salt-and-pepper pixels are outliers by definition.

This page delivers both default platforms (per ADR-005): the ESP32-S3 with an OV2640 camera (the most common embedded vision setup), and the NUCLEO-F446RE, where the interesting part is fitting the frame in 128 KB of SRAM; see the STM32F4 section below. The filter code itself is platform-independent C.

Why median and not Gaussian

A Gaussian (averaging) filter removes Gaussian noise but smears salt-and-pepper pixels across their neighbours: a single dead pixel becomes a blurry blob. The median replaces each pixel with the middle value of its neighbourhood, and an outlier at 0 or 255 is never the middle of a sorted list that contains mostly valid values. The trade-off: fine single-pixel detail (text, thin lines) is also removed because it looks like an outlier to the filter. Whether that matters depends on the downstream task.

3×3 median filter in C

The core loop: for each pixel in a greyscale image, extract the 3×3 neighbourhood and select the middle value. A full sort is overkill for 9 elements: a hardcoded compare-and-swap network that only guarantees the median position is faster and has deterministic execution time.

// 3x3 median filter on a greyscale image, single-pass, zero-copy output.
// Input:  src[row][col] = uint8_t greyscale pixel.
// Output: dst[row][col] = median of the 3x3 neighbourhood.
// Border pixels (row=0, row=H-1, col=0, col=W-1) are copied unchanged.

#include <stdint.h>

static inline void swap_if_greater(uint8_t *a, uint8_t *b) {
    if (*a > *b) { uint8_t t = *a; *a = *b; *b = t; }
}

static uint8_t median_3x3(const uint8_t src[][320], int row, int col) {
    // Read the 3x3 neighbourhood.
    uint8_t v[9] = {
        src[row-1][col-1], src[row-1][col], src[row-1][col+1],
        src[row  ][col-1], src[row  ][col], src[row  ][col+1],
        src[row+1][col-1], src[row+1][col], src[row+1][col+1],
    };

    // Median-of-9 selection network: 24 compare-and-swaps.  Only v[4]
    // (the median) is guaranteed correct afterwards; the rest of the
    // array is left partially ordered.  (A full 9-element sort needs 25;
    // Devillard's minimal median-of-9 gets by with 19.)
    swap_if_greater(&v[0], &v[1]); swap_if_greater(&v[3], &v[4]);
    swap_if_greater(&v[6], &v[7]); swap_if_greater(&v[1], &v[2]);
    swap_if_greater(&v[4], &v[5]); swap_if_greater(&v[7], &v[8]);
    swap_if_greater(&v[0], &v[1]); swap_if_greater(&v[3], &v[4]);
    swap_if_greater(&v[6], &v[7]); swap_if_greater(&v[0], &v[3]);
    swap_if_greater(&v[1], &v[4]); swap_if_greater(&v[5], &v[8]);
    swap_if_greater(&v[3], &v[6]); swap_if_greater(&v[2], &v[5]);
    swap_if_greater(&v[1], &v[3]); swap_if_greater(&v[4], &v[7]);
    swap_if_greater(&v[3], &v[6]); swap_if_greater(&v[2], &v[4]);
    swap_if_greater(&v[2], &v[3]); swap_if_greater(&v[4], &v[6]);
    swap_if_greater(&v[3], &v[4]); swap_if_greater(&v[5], &v[7]);
    swap_if_greater(&v[2], &v[3]); swap_if_greater(&v[4], &v[5]);

    return v[4];  // the median
}

void median_filter_3x3(const uint8_t *src, uint8_t *dst,
                       int width, int height, int stride) {
    // Copy border rows/columns unchanged.
    for (int c = 0; c < width; c++) {
        dst[c] = src[c];                          // top row
        dst[(height-1)*stride + c] = src[(height-1)*stride + c];  // bottom row
    }
    for (int r = 0; r < height; r++) {
        dst[r*stride] = src[r*stride];            // left column
        dst[r*stride + width-1] = src[r*stride + width-1];  // right column
    }

    // Filter interior pixels.
    for (int r = 1; r < height - 1; r++) {
        for (int c = 1; c < width - 1; c++) {
            // Use the 2-D row/col accessor for readability.
            // In production, inline the neighbourhood reads for speed.
            uint8_t v[9];
            int k = 0;
            for (int dr = -1; dr <= 1; dr++)
                for (int dc = -1; dc <= 1; dc++)
                    v[k++] = src[(r+dr)*stride + (c+dc)];
            // Sorting network (same as above, abbreviated).
            swap_if_greater(&v[0], &v[1]); swap_if_greater(&v[3], &v[4]);
            swap_if_greater(&v[6], &v[7]); swap_if_greater(&v[1], &v[2]);
            swap_if_greater(&v[4], &v[5]); swap_if_greater(&v[7], &v[8]);
            swap_if_greater(&v[0], &v[1]); swap_if_greater(&v[3], &v[4]);
            swap_if_greater(&v[6], &v[7]); swap_if_greater(&v[0], &v[3]);
            swap_if_greater(&v[1], &v[4]); swap_if_greater(&v[5], &v[8]);
            swap_if_greater(&v[3], &v[6]); swap_if_greater(&v[2], &v[5]);
            swap_if_greater(&v[1], &v[3]); swap_if_greater(&v[4], &v[7]);
            swap_if_greater(&v[3], &v[6]); swap_if_greater(&v[2], &v[4]);
            swap_if_greater(&v[2], &v[3]); swap_if_greater(&v[4], &v[6]);
            swap_if_greater(&v[3], &v[4]); swap_if_greater(&v[5], &v[7]);
            swap_if_greater(&v[2], &v[3]); swap_if_greater(&v[4], &v[5]);
            dst[r*stride + c] = v[4];
        }
    }
}

Performance budget

For a QVGA frame (320×240 = 76,800 pixels) with a 3×3 median, the filter processes the 318×238 = 75,684 interior pixels (the one-pixel border is copied unchanged, per the code):

Operation	Per interior pixel	Per frame
Neighbourhood reads	9	681,156
Compare-and-swap ops	24	1,816,416
Stores	1	75,684 (+1,116 border copies)
Total ops	~34	~2.6 M

At 240 MHz on an ESP32-S3, 2.6 M operations is ~11 ms per frame, fast enough for 30 fps with room for the camera readout and other processing. On a Cortex-M4 at 180 MHz it is ~15 ms per frame.

The bottleneck is memory access: 9 reads and 1 write per interior pixel, all to external PSRAM (ESP32-S3 camera buffer). At 30 fps that is roughly 20 MB/s of memory bandwidth, fine for the ESP32-S3’s octal PSRAM interface (~80 MB/s). If the camera DMA is writing to the same buffer at the same time, double-buffer the frames.

STM32F4 (NUCLEO-F446RE)

The F446RE has no PSRAM, so the constraint flips from compute to memory: a QVGA greyscale frame is 76,800 bytes, and the chip has 128 KB of SRAM. One frame fits; a second full destination buffer (150 KB total) does not. The fix is to filter in place with a two-row history buffer: keep copies of the two source rows the network still needs before the output overwrites them.

// In-place 3x3 median for the F446RE's single-frame SRAM budget.
// median9() is the same 24-swap selection network as median_3x3() above,
// factored to take the 9 values directly.
#define W 320
#define H 240

static uint8_t prev_row[W], curr_row[W];   // 640 B of history

void median_filter_3x3_inplace(uint8_t *img) {
    memcpy(prev_row, img, W);                       // row 0: stays unfiltered
    for (int r = 1; r < H - 1; r++) {
        memcpy(curr_row, &img[r * W], W);           // preserve row r
        for (int c = 1; c < W - 1; c++) {
            uint8_t v[9] = {
                prev_row[c-1], prev_row[c], prev_row[c+1],
                curr_row[c-1], curr_row[c], curr_row[c+1],
                img[(r+1)*W + c-1], img[(r+1)*W + c], img[(r+1)*W + c+1],
            };
            img[r * W + c] = median9(v);
        }
        memcpy(prev_row, curr_row, W);
    }
}

Getting pixels in: the STM32F446 has a DCMI peripheral, so an OV7670/OV2640-class parallel camera wires directly to it (the Nucleo board has no camera connector; use the DCMI pins on the morpho headers). One snapshot lands in SRAM via DMA:

extern DCMI_HandleTypeDef hdcmi;      // 8-bit parallel, snapshot mode
static uint8_t frame[H * W];          // 76,800 B of the 128 KB SRAM

void grab_and_filter(void) {
    HAL_DCMI_Start_DMA(&hdcmi, DCMI_MODE_SNAPSHOT,
                       (uint32_t)frame, sizeof(frame) / 4);   // length in words
    while (HAL_DCMI_GetState(&hdcmi) == HAL_DCMI_STATE_BUSY) { }
    median_filter_3x3_inplace(frame);
}

For bench work without a camera, stream a frame over UART into frame[] and back out after filtering; the filter does not care where the pixels came from. At 180 MHz the ~2.6 M ops take ~15 ms per frame (the figure quoted above), comfortably real-time for snapshot processing, marginal for continuous 60 fps video.

Verifying

The self-test: capture a frame, inject a few known dead pixels (set to 0 or 255), run the median filter, and check that the injected pixels are restored to the neighbourhood median. On hardware:

Point the camera at a uniform surface (a well-lit sheet of paper).
Capture one frame. Compute the pixel variance (should be low, the surface is uniform).
Inject 1% salt-and-pepper noise into the buffer (flip random pixels to 0 or 255).
Run the median filter. The variance should return to near the original value.
A Gaussian filter on the same corrupted frame will leave visible dark/light smears; the median removes them completely.

This is a genuinely practical thing: real camera sensors accumulate hot pixels over their lifetime, and a median filter in the capture pipeline is cheaper than replacing the sensor.

References

--- title: "Image Noise on Hardware" subtitle: "A 3×3 median filter for salt-and-pepper noise on an ESP32 camera" bibliography: ../../references.bib --- The [main page](index.qmd) shows a 3×3 median filter removing salt-and-pepper noise in Python. On a microcontroller with a camera, you run the same filter in real time (every pixel, every frame) and the result is visibly cleaner images from a sensor with dead or hot pixels. The algorithm is simple enough to run at QVGA (320×240) on any Cortex-M4 or ESP32, and it is the textbook example of a filter matched to its noise model: the median ignores outliers; salt-and-pepper pixels are outliers by definition. This page delivers both default platforms (per [ADR-005](../../docs/adr/005-embedded-platforms.md)): the ESP32-S3 with an OV2640 camera (the most common embedded vision setup), and the NUCLEO-F446RE, where the interesting part is fitting the frame in 128 KB of SRAM; see the [STM32F4 section](#stm32f4-nucleo-f446re) below. The filter code itself is platform-independent C. <hr> ## Why median and not Gaussian A Gaussian (averaging) filter removes Gaussian noise but smears salt-and-pepper pixels across their neighbours: a single dead pixel becomes a blurry blob. The median replaces each pixel with the middle value of its neighbourhood, and an outlier at 0 or 255 is never the middle of a sorted list that contains mostly valid values. The trade-off: fine single-pixel detail (text, thin lines) is also removed because it looks like an outlier to the filter. Whether that matters depends on the downstream task. <hr> ## 3×3 median filter in C The core loop: for each pixel in a greyscale image, extract the 3×3 neighbourhood and select the middle value. A full sort is overkill for 9 elements: a hardcoded compare-and-swap network that only guarantees the median position is faster and has deterministic execution time. ```c // 3x3 median filter on a greyscale image, single-pass, zero-copy output. // Input: src[row][col] = uint8_t greyscale pixel. // Output: dst[row][col] = median of the 3x3 neighbourhood. // Border pixels (row=0, row=H-1, col=0, col=W-1) are copied unchanged. #include <stdint.h> static inline void swap_if_greater(uint8_t *a, uint8_t *b) { if (*a > *b) { uint8_t t = *a; *a = *b; *b = t; } } static uint8_t median_3x3(const uint8_t src[][320], int row, int col) { // Read the 3x3 neighbourhood. uint8_t v[9] = { src[row-1][col-1], src[row-1][col], src[row-1][col+1], src[row ][col-1], src[row ][col], src[row ][col+1], src[row+1][col-1], src[row+1][col], src[row+1][col+1], }; // Median-of-9 selection network: 24 compare-and-swaps. Only v[4] // (the median) is guaranteed correct afterwards; the rest of the // array is left partially ordered. (A full 9-element sort needs 25; // Devillard's minimal median-of-9 gets by with 19.) swap_if_greater(&v[0], &v[1]); swap_if_greater(&v[3], &v[4]); swap_if_greater(&v[6], &v[7]); swap_if_greater(&v[1], &v[2]); swap_if_greater(&v[4], &v[5]); swap_if_greater(&v[7], &v[8]); swap_if_greater(&v[0], &v[1]); swap_if_greater(&v[3], &v[4]); swap_if_greater(&v[6], &v[7]); swap_if_greater(&v[0], &v[3]); swap_if_greater(&v[1], &v[4]); swap_if_greater(&v[5], &v[8]); swap_if_greater(&v[3], &v[6]); swap_if_greater(&v[2], &v[5]); swap_if_greater(&v[1], &v[3]); swap_if_greater(&v[4], &v[7]); swap_if_greater(&v[3], &v[6]); swap_if_greater(&v[2], &v[4]); swap_if_greater(&v[2], &v[3]); swap_if_greater(&v[4], &v[6]); swap_if_greater(&v[3], &v[4]); swap_if_greater(&v[5], &v[7]); swap_if_greater(&v[2], &v[3]); swap_if_greater(&v[4], &v[5]); return v[4]; // the median } void median_filter_3x3(const uint8_t *src, uint8_t *dst, int width, int height, int stride) { // Copy border rows/columns unchanged. for (int c = 0; c < width; c++) { dst[c] = src[c]; // top row dst[(height-1)*stride + c] = src[(height-1)*stride + c]; // bottom row } for (int r = 0; r < height; r++) { dst[r*stride] = src[r*stride]; // left column dst[r*stride + width-1] = src[r*stride + width-1]; // right column } // Filter interior pixels. for (int r = 1; r < height - 1; r++) { for (int c = 1; c < width - 1; c++) { // Use the 2-D row/col accessor for readability. // In production, inline the neighbourhood reads for speed. uint8_t v[9]; int k = 0; for (int dr = -1; dr <= 1; dr++) for (int dc = -1; dc <= 1; dc++) v[k++] = src[(r+dr)*stride + (c+dc)]; // Sorting network (same as above, abbreviated). swap_if_greater(&v[0], &v[1]); swap_if_greater(&v[3], &v[4]); swap_if_greater(&v[6], &v[7]); swap_if_greater(&v[1], &v[2]); swap_if_greater(&v[4], &v[5]); swap_if_greater(&v[7], &v[8]); swap_if_greater(&v[0], &v[1]); swap_if_greater(&v[3], &v[4]); swap_if_greater(&v[6], &v[7]); swap_if_greater(&v[0], &v[3]); swap_if_greater(&v[1], &v[4]); swap_if_greater(&v[5], &v[8]); swap_if_greater(&v[3], &v[6]); swap_if_greater(&v[2], &v[5]); swap_if_greater(&v[1], &v[3]); swap_if_greater(&v[4], &v[7]); swap_if_greater(&v[3], &v[6]); swap_if_greater(&v[2], &v[4]); swap_if_greater(&v[2], &v[3]); swap_if_greater(&v[4], &v[6]); swap_if_greater(&v[3], &v[4]); swap_if_greater(&v[5], &v[7]); swap_if_greater(&v[2], &v[3]); swap_if_greater(&v[4], &v[5]); dst[r*stride + c] = v[4]; } } } ``` <hr> ## Performance budget For a QVGA frame (320×240 = 76,800 pixels) with a 3×3 median, the filter processes the 318×238 = 75,684 interior pixels (the one-pixel border is copied unchanged, per the code): | Operation | Per interior pixel | Per frame | |---|---|---| | Neighbourhood reads | 9 | 681,156 | | Compare-and-swap ops | 24 | 1,816,416 | | Stores | 1 | 75,684 (+1,116 border copies) | | **Total ops** | ~34 | **~2.6 M** | At 240 MHz on an ESP32-S3, 2.6 M operations is ~11 ms per frame, fast enough for 30 fps with room for the camera readout and other processing. On a Cortex-M4 at 180 MHz it is ~15 ms per frame. The bottleneck is memory access: 9 reads and 1 write per interior pixel, all to external PSRAM (ESP32-S3 camera buffer). At 30 fps that is roughly 20 MB/s of memory bandwidth, fine for the ESP32-S3's octal PSRAM interface (~80 MB/s). If the camera DMA is writing to the same buffer at the same time, double-buffer the frames. <hr> ## STM32F4 (NUCLEO-F446RE) {#stm32f4-nucleo-f446re} The F446RE has no PSRAM, so the constraint flips from compute to memory: a QVGA greyscale frame is 76,800 bytes, and the chip has 128 KB of SRAM. One frame fits; a second full destination buffer (150 KB total) does not. The fix is to filter **in place** with a two-row history buffer: keep copies of the two source rows the network still needs before the output overwrites them. ```c // In-place 3x3 median for the F446RE's single-frame SRAM budget. // median9() is the same 24-swap selection network as median_3x3() above, // factored to take the 9 values directly. #define W 320 #define H 240 static uint8_t prev_row[W], curr_row[W]; // 640 B of history void median_filter_3x3_inplace(uint8_t *img) { memcpy(prev_row, img, W); // row 0: stays unfiltered for (int r = 1; r < H - 1; r++) { memcpy(curr_row, &img[r * W], W); // preserve row r for (int c = 1; c < W - 1; c++) { uint8_t v[9] = { prev_row[c-1], prev_row[c], prev_row[c+1], curr_row[c-1], curr_row[c], curr_row[c+1], img[(r+1)*W + c-1], img[(r+1)*W + c], img[(r+1)*W + c+1], }; img[r * W + c] = median9(v); } memcpy(prev_row, curr_row, W); } } ``` Getting pixels in: the STM32F446 has a DCMI peripheral, so an OV7670/OV2640-class parallel camera wires directly to it (the Nucleo board has no camera connector; use the DCMI pins on the morpho headers). One snapshot lands in SRAM via DMA: ```c extern DCMI_HandleTypeDef hdcmi; // 8-bit parallel, snapshot mode static uint8_t frame[H * W]; // 76,800 B of the 128 KB SRAM void grab_and_filter(void) { HAL_DCMI_Start_DMA(&hdcmi, DCMI_MODE_SNAPSHOT, (uint32_t)frame, sizeof(frame) / 4); // length in words while (HAL_DCMI_GetState(&hdcmi) == HAL_DCMI_STATE_BUSY) { } median_filter_3x3_inplace(frame); } ``` For bench work without a camera, stream a frame over UART into `frame[]` and back out after filtering; the filter does not care where the pixels came from. At 180 MHz the ~2.6 M ops take ~15 ms per frame (the figure quoted above), comfortably real-time for snapshot processing, marginal for continuous 60 fps video. <hr> ## Verifying The self-test: capture a frame, inject a few known dead pixels (set to 0 or 255), run the median filter, and check that the injected pixels are restored to the neighbourhood median. On hardware: 1. Point the camera at a uniform surface (a well-lit sheet of paper). 2. Capture one frame. Compute the pixel variance (should be low, the surface is uniform). 3. Inject 1% salt-and-pepper noise into the buffer (flip random pixels to 0 or 255). 4. Run the median filter. The variance should return to near the original value. 5. A Gaussian filter on the same corrupted frame will leave visible dark/light smears; the median removes them completely. This is a genuinely practical thing: real camera sensors accumulate hot pixels over their lifetime, and a median filter in the capture pipeline is cheaper than replacing the sensor. ## References ::: {#refs} :::