README

CXC Compression

CXC Compression is a compression project written in Free Pascal, focused on strong practical compression, fast decompression, deterministic output, and efficient multithreaded compression.

CXC is being developed as an independent compression engine and SDK. It is intended to be usable on its own or as a compression backend for other applications and archive formats.

The project is actively developed and continues to evolve as new compression strategies, parsers, entropy coders, and performance improvements are explored.

Goals

• Good compression ratio on practical data
• High compression throughput and fast decompression
• Deterministic output in both single-threaded and multithreaded operation
• Controlled, explainable memory use
• Efficient multithreaded compression
• Portable Free Pascal source
• Stable decoding compatibility across supported archive versions

These are engineering goals rather than universal performance claims. Results depend on the data, build, machine, and selected compression settings.

Current status

CXC Compression is currently version 1.0 and writes archive format 0.26 at level 9. The current release has been validated with Free Pascal Compiler 3.2.2 on Linux x86-64. The reader compatibility matrix covers archive versions 0.19 through 0.26.

Single-threaded and multithreaded compression produce deterministic archives for the same input and settings. Windows and other operating systems have not yet been included in the formal release-validation matrix.

Features

• LZ-based block compression with multiple compression levels
• BT32 match finding in the production level-9 path
• A retained 4 MiB history window for current level-9 archives
• FSE sequence coding and direct LZ reconstruction for method 27
• Deterministic ordered output from multithreaded compression
• Per-block CRC32 integrity checking
• Structural archive inspection with the info command
• Huffman, Huffman4X, rANS, and range-coding infrastructure
• Compatibility decoders for supported historical archive methods

Not every coding component is selected by every compression level. In particular, Huffman/Huffman4X and rANS support includes infrastructure and compatibility paths as well as active selections in specific block methods; the production level-9 path uses its own method-27 FSE sequence pipeline.

Command line

cxc c INPUT OUTPUT.cxc [LEVEL] [BLOCK_MIB] [OPTIONS]
cxc d INPUT.cxc OUTPUT [--force]
cxc i INPUT.cxc
cxc profile-presets
cxc profile-policies

Examples:

cxc c input.bin output.cxc
cxc c input.bin output.cxc 9 5 --threads auto
cxc d output.cxc restored.bin
cxc i output.cxc

Existing output files are rejected unless --force is supplied. CXC always rejects a lexically identical normalized input and output path. See docs/CLI.md for options and safety behavior.

Building

From the repository root:

fpc -B -O3 -g- -Si -Sh -Sc cxc.lpr

Free Pascal Compiler 3.2.2 is the currently validated compiler version; this does not imply that other FPC versions cannot compile the project. See BUILDING.md for details.

Compression levels

CXC accepts levels 1 through 9. The named presets are 1 (fast), 3 (good and the default), 5 (better), 7 (strong), and 9 (maximum). Higher levels generally spend more work searching for a smaller representation. Levels 1 through 8 write the established 0.19 format; the current production level-9 path writes format 0.26 and uses BT32 matching, retained history, and method 27 where selected.

The optional block size is expressed in MiB. If it is omitted, CXC selects a level-appropriate default; the validated level-9 command uses 5 MiB blocks.

Deterministic compression

Parallel workers compress blocks independently, while completed blocks are committed in their original order. For identical input, options, and build, the validated one-thread, fixed-thread, and automatic-thread configurations produce byte-identical archives. Determinism does not mean that builds with future algorithm changes will necessarily produce the same archive bytes.

Archive compatibility

Product and archive versions are separate. CXC Compression 1.0 currently writes archive 0.19 for levels 1 through 8 and archive 0.26 for level 9. Its validated reader compatibility range is 0.19 through 0.26. Supported older methods remain readable, while unknown methods and unsupported future archive versions are rejected.

See docs/FORMAT.md and docs/COMPATIBILITY.md.

Performance

CXC is developed with both compression ratio and runtime performance in mind. Benchmark results depend on corpus, machine, compiler, thread configuration, and competing codec presets.

In one authenticated 44-file benchmark over 302,601,834 source bytes, the current CXC level-9 archive was 136,419,851 bytes (45.082294842%). 7-Zip at mx1 produced 136,445,437 bytes, so CXC was 25,586 bytes smaller in that specific test. Stronger 7-Zip presets, zstd level 19, and RAR5 produced smaller archives in the same benchmark. These figures describe that workload and are not a general ranking.

Using CXC as a compression engine

CXC decides how to compress bytes. The application using CXC decides what should be compressed with it.

The Pascal units can be embedded or adapted as a compression backend for a future application or archive format. The command-line program in cxc.lpr also provides a concrete integration example. Conceptually:

Application / archive format
        |
        +-- CXC
        +-- other codecs

This repository ships the CXC compression engine and its single-file command line interface. It does not ship a multi-file archive application, and the current source-level integration surface should not be treated as a frozen binary ABI.

Known limitations

Formal validation currently covers Linux x86-64. The memory limit is a soft planner rather than a hard RSS cap, and filesystem safety checks have the usual lexical-path, race, and power-loss boundaries. See docs/KNOWN_LIMITATIONS.md for the precise list.

About the project

CXC Compression started as a personal project focused on experimenting with compression ratio, decompression speed, deterministic output, and practical archive workloads.

I decided to make the project public in the hope that it may be useful to other developers interested in compression, archiving, or Free Pascal development. You are welcome to study the code, use it in your own projects, modify it, or adapt it to your particular needs under the terms of the included license.

The project is developed primarily with Free Pascal (FPC). One of the motivations for publishing CXC is to contribute another modern compression-oriented codebase to the Free Pascal ecosystem, particularly for developers interested in high-performance binary processing, archive formats, and compression research.

CXC is an actively evolving project. The current implementation reflects a long series of experiments, profiling, validation, and compatibility work, and there is still plenty of room for further improvement. Contributions, testing on different systems and workloads, and constructive technical feedback are welcome.

License

CXC Compression is licensed under the Apache License 2.0.

Copyright (c) 2026, Ivan Cipee Milovanovic. All rights reserved.

ccRangeCoder.pas retains its Igor Pavlov / LZMA SDK public-domain provenance notice.