Skip to content

Update FastLZ to b1342da #13230

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
phongn wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Update FastLZ to b1342da #13230

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 41 additions & 23 deletions lib/fastlz/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,39 +34,57 @@ For [Vcpkg](https://github.com/microsoft/vcpkg) users, FastLZ is [already availa

A simple file compressor called `6pack` is included as an example on how to use FastLZ. The corresponding decompressor is `6unpack`.

FastLZ supports any standard-conforming ANSI C/C90 compiler, including the popular ones such as GCC, Clang, Intel C++ Compiler, Visual Studio and even Tiny CC. FastLZ works well on a number of architectures (32-bit and 64-bit, big endian and little endian), from Intel/AMD, ARM, and MIPS.
FastLZ supports any standard-conforming ANSI C/C90 compiler, including the popular ones such as [GCC](https://gcc.gnu.org/), [Clang](https://clang.llvm.org/), [Visual Studio](https://visualstudio.microsoft.com/vs/features/cplusplus/), and even [Tiny CC](https://bellard.org/tcc/). FastLZ works well on a number of architectures (32-bit and 64-bit, big endian and little endian), from Intel/AMD, PowerPC, System z, ARM, MIPS, and RISC-V.

The continuous integration system runs an extensive set of compression-decompression round trips on the following systems:


For more details, check the corresponding [GitHub Actions build logs](https://github.com/ariya/FastLZ/actions).

| | | | |
|--------------|---------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------|
| **amd64** | **Linux** | **Windows** | **macOS** |
| GCC | ![amd64_linux_gcc](https://github.com/ariya/FastLZ/workflows/amd64_linux_gcc/badge.svg) | ![amd64_windows_gcc](https://github.com/ariya/FastLZ/workflows/amd64_windows_gcc/badge.svg) | ![amd64_macos_gcc](https://github.com/ariya/FastLZ/workflows/amd64_macos_gcc/badge.svg) |
| Clang | ![amd64_linux_clang](https://github.com/ariya/FastLZ/workflows/amd64_linux_clang/badge.svg) | ![amd64_windows_clang](https://github.com/ariya/FastLZ/workflows/amd64_windows_clang/badge.svg) | ![amd64_macos_clang](https://github.com/ariya/FastLZ/workflows/amd64_macos_clang/badge.svg) |
| Intel CC | ![amd64_linux_icc](https://github.com/ariya/FastLZ/workflows/amd64_linux_icc/badge.svg) | | |
| TinyCC | ![amd64_linux_tcc](https://github.com/ariya/FastLZ/workflows/amd64_linux_tcc/badge.svg) | | | |
| VS 2017 | | ![amd64_windows_vs2017](https://github.com/ariya/FastLZ/workflows/amd64_windows_vs2017/badge.svg) | |
| VS 2019 | | ![amd64_windows_vs2019](https://github.com/ariya/FastLZ/workflows/amd64_windows_vs2019/badge.svg) | |
| **i686** | **Linux** | **Windows** | **macOS** |
| GCC | ![i686_linux_gcc](https://github.com/ariya/FastLZ/workflows/i686_linux_gcc/badge.svg) | | |
| Clang | ![i686_linux_clang](https://github.com/ariya/FastLZ/workflows/i686_linux_clang/badge.svg) | | |
| VS 2017 | | ![i686_windows_vs2017](https://github.com/ariya/FastLZ/workflows/i686_windows_vs2017/badge.svg) | |
| VS 2019 | | ![i686_windows_vs2019](https://github.com/ariya/FastLZ/workflows/i686_windows_vs2019/badge.svg) | |
| **arm64** | **Linux** | **Windows** | **macOS** |
| GCC | ![arm64_linux_gcc](https://github.com/ariya/FastLZ/workflows/arm64_linux_gcc/badge.svg) | | |
| **mips64** | **Linux** | **Windows** | **macOS** |
| GCC | ![mips64_linux_gcc](https://github.com/ariya/FastLZ/workflows/mips64_linux_gcc/badge.svg) | | |
| | | | |
|----------------------|--------------------------------------------------------------------------------------------------------:|--------------------------------------------------------------------------------------------------:|--------------------------------------------------------------------------------------------:|
| **amd64** | **Linux** | **Windows** | **macOS** |
| GCC | ![amd64_linux_gcc](https://github.com/ariya/FastLZ/workflows/amd64_linux_gcc/badge.svg) | ![amd64_windows_gcc](https://github.com/ariya/FastLZ/workflows/amd64_windows_gcc/badge.svg) | ![amd64_macos_gcc](https://github.com/ariya/FastLZ/workflows/amd64_macos_gcc/badge.svg) |
| Clang | ![amd64_linux_clang](https://github.com/ariya/FastLZ/workflows/amd64_linux_clang/badge.svg) | ![amd64_windows_clang](https://github.com/ariya/FastLZ/workflows/amd64_windows_clang/badge.svg) | ![amd64_macos_clang](https://github.com/ariya/FastLZ/workflows/amd64_macos_clang/badge.svg) |
| TinyCC | ![amd64_linux_tcc](https://github.com/ariya/FastLZ/workflows/amd64_linux_tcc/badge.svg) | ![amd64_windows_tcc](https://github.com/ariya/FastLZ/workflows/amd64_windows_tcc/badge.svg) | |
| VS 2019 | | ![amd64_windows_vs2019](https://github.com/ariya/FastLZ/workflows/amd64_windows_vs2019/badge.svg) | |
| **i686** | **Linux** | **Windows** | **macOS** |
| GCC | ![i686_linux_gcc](https://github.com/ariya/FastLZ/workflows/i686_linux_gcc/badge.svg) | | |
| Clang | ![i686_linux_clang](https://github.com/ariya/FastLZ/workflows/i686_linux_clang/badge.svg) | | |
| TinyCC | | ![i686_windows_tcc](https://github.com/ariya/FastLZ/workflows/i686_windows_tcc/badge.svg) | |
| VS 2019 | | ![i686_windows_vs2019](https://github.com/ariya/FastLZ/workflows/i686_windows_vs2019/badge.svg) | |
| **i586** | **Linux** | **DOS** | |
| GCC | | ![i586_dos_gcc_cross](https://github.com/ariya/FastLZ/workflows/i586_dos_gcc_cross/badge.svg) | |
| | **Linux** | | |
| **powerpc** | | | |
| GCC | ![powerpc_linux_gcc](https://github.com/ariya/FastLZ/workflows/powerpc_linux_gcc/badge.svg) | | |
| **ppc64(le)** | | | |
| GCC | ![ppc64_linux_gcc](https://github.com/ariya/FastLZ/workflows/ppc64_linux_gcc/badge.svg) | | |
| GCC | ![ppc64le_linux_gcc](https://github.com/ariya/FastLZ/workflows/ppc64le_linux_gcc/badge.svg) | | |
| **s390x** | | | |
| GCC | ![s390x_linux_gcc](https://github.com/ariya/FastLZ/workflows/s390x_linux_gcc/badge.svg) | | |
| **armhf** | | | |
| GCC | ![armhf_linux_gcc](https://github.com/ariya/FastLZ/workflows/armhf_linux_gcc/badge.svg) | | |
| **arm64** | | | |
| GCC | ![arm64_linux_gcc](https://github.com/ariya/FastLZ/workflows/arm64_linux_gcc/badge.svg) | | |
| **mips(el)** | | | |
| GCC | ![mipsel_linux_gcc](https://github.com/ariya/FastLZ/workflows/mipsel_linux_gcc/badge.svg) | | |
| GCC | ![mips_linux_gcc](https://github.com/ariya/FastLZ/workflows/mips_linux_gcc/badge.svg) | | |
| **mips64(el)** | | | |
| GCC | ![mips64el_linux_gcc](https://github.com/ariya/FastLZ/workflows/mips64el_linux_gcc/badge.svg) | | |
| GCC | ![mips64_linux_gcc](https://github.com/ariya/FastLZ/workflows/mips64_linux_gcc/badge.svg) | | |
| **riscv** | | | |
| GCC | ![riscv_linux_gcc](https://github.com/ariya/FastLZ/workflows/riscv_linux_gcc/badge.svg) | | |
| **riscv64** | | | |
| GCC | ![riscv64_linux_gcc](https://github.com/ariya/FastLZ/workflows/riscv64_linux_gcc/badge.svg) | | |



## Block Format

Let us assume that FastLZ compresses an array of bytes, called the _uncompressed block_, into another array of bytes, called the _compressed block_. To understand what will be stored in the compressed block, it is illustrative to demonstrate how FastLZ will _decompress_ the block to retrieve the original uncompressed block.

The first 5-bit of the block, i.e. the 5 most-significant bits of the first byte, is the **block tag**. Currently the block tag determines the compression level used to produce the compressed block.
The first 3-bit of the block, i.e. the 3 most-significant bits of the first byte, is the **block tag**. Currently the block tag determines the compression level used to produce the compressed block.

|Block tag|Compression level|
|---------|-----------------|
Expand All @@ -77,16 +95,16 @@ The content of the block will vary depending on the compression level.

### Block Format for Level 1

FastLZ Level 1 impements LZ77 compression algorithm with 8 KB sliding window and up to 264 bytes of match length.
FastLZ Level 1 implements LZ77 compression algorithm with 8 KB sliding window and up to 264 bytes of match length.

The compressed block consists of one or more **instructions**.
Each instruction starts with a 1-byte opcode, 2-byte opcode, or 3-byte opcode.

| Instruction type | Opcode[0] | Opcode[1] | Opcode[2]
|-----------|------------------|--------------------|--|
| Literal run | `000`, L₅-L₀ | -|- |
| Literal run | `000`, L₄-L₀ | -|- |
| Short match | M₂-M₀, R₁₂-R₈ | R₇-R₀ | - |
| Long match | `111`, R₁₂-R₈ | M₇-R₀ | R₇-R₀ |
| Long match | `111`, R₁₂-R₈ | M₇-M₀ | R₇-R₀ |

Note that the _very first_ instruction in a compressed block is always a literal run.

Expand Down
Loading