wire / wire-format / snappy-format
Snappy is Google's fast compression format. The frame format: stream identifier (chunk type 0xFF, 'snappy' magic), then chunks. Each chunk: 1-byte type (0x00 compressed, 0x01 uncompressed, 0x02 padding, 0x80-0xFD reserved, 0xFE CRC32C, 0xFF stream identifier), 3-byte length, then data. Compressed chunks use a simple LZ77-like scheme with varint lengths.
aka: Snappy compressed data format
spec: Snappy (github.com/google/snappy)
Snappy is Google's fast compression format. The frame format: stream identifier (chunk type 0xFF, 'snappy' magic), then chunks. Each chunk: 1-byte type (0x00 compressed, 0x01 uncompressed, 0x02 padding, 0x80-0xFD reserved, 0xFE CRC32C, 0xFF stream identifier), 3-byte length, then data. Compressed chunks use a simple LZ77-like scheme with varint lengths.
| field | size | meaning |
|---|---|---|
| Stream identifier chunk | 10 bytes | Type 0xFF, length 6, data 'sNaPpY' (0x73 0x4E 0x61 0x50 0x70 0x59). |
| Compressed chunk | variable | Type 0x00, 3-byte length, then: varint uncompressed length, then tagged literals/matches. |
| Uncompressed chunk | variable | Type 0x01, 3-byte length, then raw data. |
| CRC32C chunk | 4 bytes | Type 0xFE, length 4, CRC32C of the uncompressed data (optional). |
| Tagged literal | variable | Tag 00 (2 bits): literal; length in 6 bits or varint; then literal bytes. |
| Tagged copy | variable | Tag 01 (offset < 2048, len 4-11), 10 (offset < 65536, len 1-64), 11 (offset any, len 1-64): back-reference. |
example:
ff 06 00 00 73 4e 61 50 70 59
Snappy stream identifier chunk
agent: curl -H 'accept: application/json' wire.phall.io/wire-format/snappy-format
or /wire-format/snappy-format.json