wire / wire-format / lz4-frame

LZ4 frame format

LZ4 is an extremely fast compression algorithm. The LZ4 frame format: 4-byte magic (0x184D2204), frame descriptor (flags + optional content size + dictionary ID + content checksum flag), then blocks (each with a 4-byte size, compressed or uncompressed), then an end mark (0x00000000) and optional content checksum.

wire-format kind wire-format status standard verification verified tier B wire-format@1

aka: LZ4 frame format

wire format

spec: LZ4 frame format (github.com/lz4/lz4)

LZ4 is an extremely fast compression algorithm. The LZ4 frame format: 4-byte magic (0x184D2204), frame descriptor (flags + optional content size + dictionary ID + content checksum flag), then blocks (each with a 4-byte size, compressed or uncompressed), then an end mark (0x00000000) and optional content checksum.

fieldsizemeaning
Magic number4 bytes0x184D2204 (little-endian).
FLG1 byteBit 7-6: version (01), bit 5: block independence, bit 4: block checksum, bit 3: content size, bit 2: content checksum, bit 1-0: reserved (0).
BD1 byteBlock max size: 4 (64KB), 5 (256KB), 6 (1MB), 7 (4MB).
Content size0 or 8 bytesSize of uncompressed content (if FLG content size flag set).
Dictionary ID0 or 4 bytesID of the dictionary (if FLG reserved/dict flag set).
HC1 byteHeader checksum (XXH32 of the header bytes, low byte).
Blockvariable4-byte block size (MSB = uncompressed flag) + block data. Size 0 = end mark.
Content checksum0 or 4 bytesXXH32 of the uncompressed content (if FLG content checksum flag set).

example:

04 22 4d 18 40 40 00 00 00 00

LZ4 magic + frame descriptor (version 1, 64KB blocks, no content size/checksum)


provenance

agent: curl -H 'accept: application/json' wire.phall.io/wire-format/lz4-frame or /wire-format/lz4-frame.json