A researcher at Tufa Labs has compressed one gigabyte of Wikipedia text into a 105.39MB file, with no data lost, using a small transformer model as the only predictor. The package, called tufazip, is the smallest entry on the Large Text Compression Benchmark from any submission that relies on a neural network alone, according to the team's research post.

Tommy He built it. The method does not run a chatbot in the cloud. It trains a language model on the text as it compresses, and rebuilds an identical model on the other side to unpack the result.

The model learns while it reads

The predictor is a 163-million-parameter transformer with 12 layers. Before compression starts, a smaller 34-million-parameter model is trained offline. Call it a starter kit: it holds two already-trained layers, which are copied and copied again to fill all 12 slots when the full model is assembled in memory.

Each copy then continues learning on its own. As the compressor reads token by token, it asks the model to guess what comes next and then trains on the answer. The decompressor performs the same steps in the same order, rebuilding its copy of the model as it goes. That is why the archive only needs to carry the compressed text, the unpacking program and the 4.33MB starter model, stored at low precision to save space, rather than the full trained network.

Because the output has to match the input byte for byte, every one of those training updates has to be fully deterministic. A difference of one rounding step between the two machines would scramble everything that follows.

Where it lands on the benchmark

The first version, which learned only online, placed second when it was submitted in early September. The 105.39MB version, after optimization, ranked 13th on the benchmark's November 19 update. All 12 entries above it combine a transformer with conventional hand-built compressors. Tufazip does not.

That distinction is the point of the exercise. Traditional compressors such as those in zip and 7-Zip look for repeated patterns with fixed rules. A neural predictor builds a statistical model of the text and improves as it reads, which lets it capture structure that a dictionary of repeated strings cannot represent.

The price is compute. Compressing or decompressing this archive takes two Nvidia B200 GPUs about a week. A laptop can unpack a zip file in seconds because the method is cheap; this one trades electricity for a smaller file. Graphics cards already write gigabytes to their own SSDs while idle, so the hardware to do this kind of work is increasingly common, but nobody is going to compress a photo library this way.

What the result does and does not show

The benchmark is text only, which matters for anyone imagining video or images. The result also says nothing about how the method behaves on a corpus that differs from Wikipedia, where the model has an enormous amount of consistent, well-edited prose to learn from.

Still, it moves a line. Compression has long been a place where hand-tuned algorithms beat learned ones on size, with learned methods winning only when the model itself was excluded from the file size. Here the decoder had to be carried inside the archive and the entry still finished in the top 15, which is why the team's note emphasizes the absence of any hand-built model rather than the ranking alone.

Practical storage work is heading in the other direction for now. Memory standards are adding bandwidth and shrinking modules rather than chasing ratio, and data centers measure their bills in megawatts. A compression method that needs a week of GPU time per gigabyte is a research result, not a product. It does suggest, though, that the boundary between a model and a file is getting thinner.