makimono

巻物 · a hand scroll

makimono compresses enwik9, the first 109 bytes of English Wikipedia and the test file of the Large Text Compression Benchmark, into a self-extracting archive of 95,793,872 bytes. It is a modified build of zmix 1.0, a context-mixing compressor with a small transformer network.

enwik9 after compression, drawn to scale

enwik9, original
gzip -9
322,591,988
zstd -19
230,825,074
xz -9e
211,776,220
makimono
95,793,872 · 0.766 bits/char

Full width = 1,000,000,000 bytes. gzip, zstd and xz measured by us on one core of a desktop PC; makimono's size includes its decompressor and network weights.

Result

archive9.exe
95,793,872 Bdecompressor + weights + data
vs zmix 1.0
−302,389 B−0.31% (96,096,261 B)
compression
63 h 29 minpeak RSS 9,970,240 KB
decompression
61 h 19 minpeak RSS 9,573,516 KB

One core of a KVM virtual machine on an AMD EPYC 9755 (16 GB RAM, Ubuntu 22.04), shared with other work, so the times are pessimistic. The decompressed file was compared byte for byte with enwik9.

Download

Direct downloads from this site:

Download archive9.exe 95.8 MB Download source 16.9 MB

archive9.exe sha256 90211d15748be01c6eb487d365c654f511877b14ec5f1cf300cff360f108abb9
makimono-src.tar.gz sha256 13e60e4361de5ed2fb8d8ac7ca49c85b6cb5714a19e384e0d8b252f3db379a23
SHA256SUMS · mirror: GitHub release v1.0 · code: github.com/krapuchino83/makimono

Decompress it yourself

archive9.exe is a Linux x86-64 program (the .exe name comes from zmix). It needs AVX2 and about 10 GB of free memory, and runs for about two and a half days.

mkdir scroll && cd scroll
cp ~/Downloads/archive9.exe . && chmod +x archive9.exe
./archive9.exe          # writes data9, identical to enwik9
sha256sum data9         # 159b85351e5f76e60cbe32e04c677847a9ecba3adc79addab6f4c6c7aa3744bc

To rebuild the compressor from source, run ./build-makimono.sh in the source package. It reproduces the submitted comp9 byte for byte.

What makimono changes in zmix

  1. New weights for the same network. The 6-million-parameter fx2 transformer keeps its shape. Its weights were fine-tuned with distillation from fx2-cmix-transformer's full-precision model, stored as 5- and 7-bit integers instead of 4-bit. The network is a little larger to store and predicts Wikipedia text better.
  2. Kernels that read them. The inference code reads the mixed-precision weights and scales each token's activations separately. It keeps zmix's determinism rules: no reciprocal-estimate instructions and no system maths library, the two things that make Intel and AMD processors disagree.
  3. Engine options zmix turned off for time. LTCB has no time limit, so 28 model inputs zmix drops for speed are kept, and adaptive depth is off.

Every change is listed in CHANGES-vs-zmix-1.0.patch, and the training scripts for the weights are in weights-training/.

Built on