๐ฆ CHONK docs with Chonkie โจ โ The lightweight ingestion library for fast, efficient and robust RAG pipelines
๐ฆ CHONK your texts with Chonkie โจ Type-friendly, light-weight, fast and super-simple chunking library
FastCDC implementation in Python https://pypi.org/project/fastcdc/
SmartChunk is a lightweight, structure-aware semantic chunking toolkit designed to supercharge RAG (Retrieval-Augmented Generation) and LLM pipelines. Unlike naive splitters that break text arbitrarily, SmartChunk respects document structure (headings, lists, tables, code blocks) and semantic flow, ensuring cleaner, more coherent chunks.
Go implementation of the AE chunking algorithm.
Chunkpad is designed to prepare documents for Retrieval-Augmented Generation (RAG) pipelines and AI applications.
A Rust library for Content-Defined Chunking (CDC).
Android Resumable Uploads SDK from Fastpix
Implementation of an interactive chatbot for summarizing legal and policy documents. Includes data preprocessing (cleaning, tokenization, hierarchical chunking), extractive TF-IDF baselines, and fine-tuned abstractive models (DistilBART, LED). Integrates a retrieval layer for document relevance and uses ROUGE, BLEU, and cosine similarity metrics.
A nodejs chunking system
Explore and benchmark the world of data chunking algorithms in 'ChunkingChampions' - a competitive arena to determine the most efficient and effective chunking strategies for varied data sizes.
A high-performance FastCDC 2020 implementation written in Python + Cython
[2024-2] AI ๊ธฐ๋ฐ ํ์ ์ง์ ํ๋ซํผ ์๋น์ค "Clerker"
A smol Go package for splitting text into chunks while preserving semantic meaning.
Lightweight Semantic Chunking Library. Plug any embedding provider/API. Batch embeddings for efficiency and handling API rate limits.
MS PowerPoint extension created using Yoman generator and React JS
Application to assist book editors with locating key information across a series for continuity
a simple utility to split given array into chunks of input size with array reverse option
Add a description, image, and links to the chunking-algorithm topic page so that developers can more easily learn about it.
To associate your repository with the chunking-algorithm topic, visit your repo's landing page and select "manage topics."