Fast, GPU-based decompressor for standard bzip2 streams. Except for the initial parser, all stages of the bzip2 pipeline (Huffman decoding, RLE2, inv. Move-to-Front, inv. Burrows-Wheeler Transform, RLE1) are executed on the GPU. Uncompressed output is kept in device memory.
This current implementation supports bzip2 streams of up to 2^31 - 1 bytes (uncompressed).
For further information, please refer to our conference paper.
Note: This program serves as auxiliary material for our publication and is not suitable for production use.
- CUDA-enabled GPU (compute capability 7.0 or higher)
- GNU/Linux
- GNU compiler version 11.4.1 or higher
- CUDA SDK 12.4.1 or higher
- latest proprietary graphics drivers
To compile the demo program, run:
make
./bin/bz2gpu <input_path> <output_path> <GPU to be used> (default 0)
- batched execution for supporting larger streams
- port parser to CUDA
- stream error detection (parsing, checksums,...)
- recursive variant of Helman-Jájá for iBWT
- random selection of splitters for list ranking
- automatic parameter tuning
- unit tests
This project is distributed under the MIT License (see LICENSE).
The files
src/rle.cu include/thrust_custom_alloc.cuh
contain modified sourcecode from the Thrust library. Thrust is distributed under the Apache License v2.0 with LLVM Exceptions (see thrust_license).
test/lichess_puzzles_10M.bz2
contains the first 10 million bytes from the lichess.org puzzle dataset and can be used for quick tests.