hilbert transform - Compression algorithms specific to complex signals

I am looking for (lossy or lossless) compression algorithms dedicated to complex signals. The latter could be composite data (like the left and right for stereo audio), a Fourier transformation or an intermediate step of a complex processing, an Hilbert pair of generic signals, or complex measurements like some NMR (Nuclear magnetic resonance ) or mass spectrometry data.

A little background: I have been working on real signal and image coding, and mildly on video and geological mesh compression. I have practiced adaptive lossless and lossy compression (RLS, LPC, ADPCM), Huffman and arithmetic coding, transforms (DCT, wavelets), etc. Using standard compression tools separately on each of the real and the imaginary parts is not the main aspect of the question. I am interested in compression-related ideas specific to complex data, for instance:

• sampling of the complex plane,


• joint quantization of module and phase,


• optimal quantization of 2D complex values: Loyd-Max is "optimal" for 1D data. I remember that 2D optimal quantization is generally more complicated. Are there 2D binnings dedicated to complex, or for instance Gauss integers?


• entropy coding methods, arranging along complex "features" (angle, magnitude),


• entropy calculation for complex data,


• use of specific statistical tools (e.g. from Statistical Signal Processing of Complex-Valued Data: The Theory of Improper and Noncircular Signals)


• integer transformations for Gauss integers.

I have not found much tools. My scarce references are:

I would be interested in more authoritative references. A compression file format dedicated to complex data would be a plus.

Answer

Personally, I'd assume that coders that work well on real-valued data would work well on the separated imaginary and real parts too – and after those, another round of trying to compress by exploiting the fact that I and Q very often will have very similar derivatives.

Thus, my first action would be to try a DPCM coder on the separated real and imaginary parts. If that doesn't fit your needs, then do DPCM on the interleaved results, then do the huffman. Alternatively, go first from $\Re\{z\}+j\,\Im\{z\}$ to $|z|+e^{j\angle z}$ and compress $|z|$ based on a simple DPCM, and $\angle z$ with an elegantly wrapping linear predictor (reminds me a bit of the title of the '97 paper you cite).

In fact, I'm trying to remember all the details, but we had an interesting discussion on the topic on the GNU Radio mailing list not so long ago (SDR users sometimes run into problems like having a hard time finding storage that can write 100MS/s of int16 complexes…). I remember looking at FLAC for the purpose, and then not writing any code. I wonder why that is. Anyway, I'd really try to convince flac to compress your samples as audio (maybe just simply stereo) and see how well linear prediction gets the most of your signal's entropy :)

---

EDIT I remember why I didn't look into FLAC: zstd came up and distracted me.

So: I went ahead and generated a sum of sinusoids and noise:

and saved everything as interleaved signed complex int16.

Then, I did a quick

flac --force-raw-format --channels 2 --bps 16 --sample-rate $((50*1000)) --endian little --sign signed -8 -o samples.flac samples.complex_sc16

(faking a 50 kHz sampling rate... but as far as I can tell, the FLAC coder doesn't care – the sampling rate info is just used to tell potential audio players what sampling rate to use).

The result was a 31% compression over the 191 MB of original data. Not that bad!

Zstd, on noise-ridden data, of course isn't that effective - a mere 5% compression was achieved (which is really meagre, considering that of the 16 bit per I and Q, most of the time, maybe 10 are used, and very few samples even came close to the maximum integer values, indicating that there should be significant runs of 0 and 1 in the bitstream. Anyway, as soon as I switched of noise, the 191 MB were immediately reduced to 12 MB – strength of a dictionary-based compressor on periodic data, especially since zstd uses a "modern PC-sized" dictionary size (and not the "oh, 16kB is soooo much" of gzip). Flac didn't do much better than on the noise-ridden signal.