Open Source C Archiving Software

Library and command line tools for XZ and LZMA compressed files

LZ4 is lossless compression algorithm, providing compression speed > 500 MB/s per core (>0.15 Bytes/cycle). It features an extremely fast decoder, with speed in multiple GB/s per core (~1 Byte/cycle). A high compression derivative, called LZ4_HC, is available, trading customizable CPU time for compression ratio. LZ4 library is provided as open-source software using a BSD license. This benchmark simulates simple "static content transfer" scenario such as OS Kernel compression or video game's static assets (text/images/tables/scripts/etc) which loading from Flash Memory / HDD / SSD. In this case, compression time is completely ignored. Because only content developers compress the data at once and usually they don't care about its computational cost. But they always care end user's experience a.k.a. "loading time" and bandwidth. Please pay attention to "LZ4HC -9" which is quite faster than other methods.

Zstandard is a fast compression algorithm, providing high compression ratios. It also offers a special mode for small data, called dictionary compression. The reference library offers a very wide range of speed / compression trade-off, and is backed by an extremely fast decoder (see benchmarks below). Zstandard library is provided as open source software using a BSD license. Its format is stable and published as IETF RFC 8478. The negative compression levels, specified with --fast=#, offer faster compression and decompression speed in exchange for some loss in compression ratio compared to level 1, as seen in the table above. Zstd can trade compression speed for stronger compression ratios. It is configurable by small increment. Decompression speed is preserved and remain roughly the same at all settings, a property shared by most LZ compression algorithms, such as zlib or lzma.

JDIFF is a program that outputs the differences between two binary files, either in binary format or in human readable format (detailed or summarized) and then allows to reconstruct the second file from the first one and the diff-file.

minidjvu is a DjVu encoder for black-and-white images.

LZO is a compression library which is widely used around the world. The main problem with LZO is that it is absolutely not human readable. People have done crazy stuff to get LZO to run in their language. Usually it implies inline assembly or trying to execute data which actually contains machine code. This is sick. Whoever is responsible for this sorry situation ought to be ashamed. So I'm going to deobfuscate LZO and provide a ISC implementation of this algorithm in Python and C. In addition, I will provide a textual description of the algorithm so that it can be easily ported to any programming language. I expect a severe performance degradation, but I leave optimizing for speed to other people.

PileWorks provides the organizational structure for coordinating several different projects which approach some aspect of Pile Technology. PileWorks defines a set of interfaces and implements some basic infrastructure for Pile engines and agents.

Command-line tool for identifying and uncompressing a wide array of archive formats. It aims to unlock old and hard-to-find archive types.