wire / wire-format / zstd-format
Zstandard is a fast lossless compression algorithm developed by Facebook. A zstd frame: 4-byte magic (0x28B52FFD), frame header (window size, dictionary ID, frame content size), compressed blocks (literals + sequences), and optional checksum. Blocks are either compressed (literals Huffman + FSE sequences) or raw/RLE.
aka: Zstandard (zstd) frame format
spec: RFC 8878
Zstandard is a fast lossless compression algorithm developed by Facebook. A zstd frame: 4-byte magic (0x28B52FFD), frame header (window size, dictionary ID, frame content size), compressed blocks (literals + sequences), and optional checksum. Blocks are either compressed (literals Huffman + FSE sequences) or raw/RLE.
| field | size | meaning |
|---|---|---|
| Magic number | 4 bytes | 0x28B52FFD (little-endian: FD 2F B5 28). |
| Frame header descriptor | 1 byte | Bit 7-6: Frame_Content_Size flag, bit 5: Single_Segment, bit 4: unused, bit 3-2: reserved, bit 1-0: Dictionary_ID flag. |
| Window descriptor | 0 or 1 byte | Window size (if not single segment): exponent (5 bits) + mantissa (3 bits) -> window size = (1 << (exp+10)) + (mantissa << exp). |
| Dictionary ID | 0/1/2/4 bytes | ID of the dictionary used (if any). |
| Frame content size | 0/1/2/4/8 bytes | Size of the uncompressed data (if present). |
| Block (repeated) | variable | Block header (3 bytes): last_block (1 bit) + block_type (2 bits: 0 compressed, 1 raw, 2 RLE, 3 reserved) + block_size (29 bits). Then block content. |
| Compressed block | variable | Literals section (Huffman-coded) + sequences section (FSE-coded: literal lengths, offsets, match lengths). |
| Content checksum | 4 bytes | Optional XXH64 checksum of the uncompressed data (if checksum flag set). |
example:
28 b5 2f fd 20
zstd magic + frame header descriptor (single segment, FCS 1 byte)
agent: curl -H 'accept: application/json' wire.phall.io/wire-format/zstd-format
or /wire-format/zstd-format.json