Mel spectrogram is a visual representation of an audio signal that shows how its energy is distributed across frequency and time. It is a specialised version of the classic spectrogram, but with the frequency axis warped into the Mel scale – a perceptual scale that mimics how the human ear perceives pitch.
Unlike the linear frequency scale in Hertz, the Mel scale is logarithmic – it compresses frequency bands in the low range and expands them in the high range. This reflects the fact that humans are much better at distinguishing small changes in low frequencies (e.g., between 100 Hz and 200 Hz) than in high frequencies (e.g., between 10,000 Hz and 10,100 Hz).
The conversion from frequency \( f \) in Hertz to the Mel scale is given by:
and the inverse conversion – from Mel back to Hertz – is:
Figure 1: Mel spectrogram (40 Mel bands) of a dynamic synthetic signal. The y‑axis shows frequency in Hertz (Mel-warped). Оне can see a frequency sweep from 200 Hz to 3000 Hz (curved stripe), impulsive bursts, and amplitude variations over time.
The plot above shows a Mel spectrogram of a time-varying signal:
- A frequency sweep (chirp) from 200 Hz to 3000 Hz between 0.1 s and 0.4 s.
- A burst of broadband noise around 0.5 s (like a percussive hit).
- A pulsing low-frequency tone (150 Hz) that turns on and off between 0.6 s and 0.9 s.
- A high‑frequency tone (4000 Hz) that appears briefly at 0.7 s.
- All components have different amplitudes, creating a rich, dynamic texture.
This dynamic behaviour is what makes spectrograms so powerful – one can spot events (onsets, offsets), glides (pitch changes), and rhythmic patterns at a glance.
Creating a Mel spectrogram involves several steps:
- Spectral analysis: The signal is split into short overlapping windows, and for each window the spectrum is computed using the short-time Fourier transform (STFT).
- Mel filter bank: A set of triangular filters, placed along the Mel scale, is applied to the linear frequency spectrum. Each filter covers a specific frequency region and sums the energy within it.
- Logarithmic scaling: Often, the logarithm is applied to the Mel spectrogram to produce a Log-Mel spectrogram, which further boosts low-amplitude information and matches the logarithmic perception of loudness.
The result is a representation that is significantly more compact and informative than the standard spectrogram – instead of hundreds or thousands of frequency bins, a Mel spectrogram typically uses between 40 and 128 Mel bands.
Mel spectrograms are widely used in modern audio technologies:
- Automatic speech recognition (ASR): Nearly all modern ASR systems – from Siri and Google Assistant to transcription services – use Mel spectrograms as input to their neural networks.
- Speaker identification: Mel spectrograms capture timbre and individual voice characteristics, making them indispensable for biometric systems.
- Music analysis and classification: Genre recognition, music recommendation, and instrument separation rely on Mel-frequency representations.
- Audio enhancement: Noise reduction and dereverberation algorithms work directly on Mel spectrograms.
- Medical diagnostics: Voice disorder analysis and early detection of neurological conditions using Mel spectrograms of speech.
The Mel spectrogram is closely related to MFCCs (Mel-Frequency Cepstral Coefficients) – features derived by applying a cosine transform to the logarithm of the Mel spectrogram. MFCCs are among the most widespread audio descriptors in speech processing.
Despite its advantages, the Mel spectrogram has limitations. It is a lossy representation – the filter bank averages energy within frequency bands, which discards fine spectral details. Also, it cannot recover phase information, which is essential for applications like sound synthesis. In such cases, other representations are used – for example, complex spectrograms or wavelet transforms.
In the deep learning era, the Mel spectrogram has become the standard "image-like" input for audio models. It turns sound into a 2D matrix (time × Mel frequencies), which can be processed by convolutional neural networks (CNNs) in the same way as images. This makes it a bridge between the world of audio signals and the world of computer vision – two fields that increasingly converge in modern AI systems.
The Mel spectrogram is one of the most practical and widely used tools in modern audio processing. It combines mathematical rigour with a deep understanding of human perception, turning complex sound waves into a clear, information-rich, and machine-readable picture of sound.