#deduplication Repositories
Discover 26 active open source GitHub repositories tagged with #deduplication, accumulating 133,156 total stars (averaging 5,121 stars per project). Leading this topic is restic/restic with 37,333 stars.
26 repositories total
Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.
The RedPajama-Data repository contains code for preparing large datasets for training large language models.
A C library for parsing/normalizing street addresses around the world. Powered by statistical NLP and open geo data.
Windows disk usage analyzer and cleanup tool with interactive treemaps, large-file discovery, duplicate detection, and file-system monitoring.
Ephemeral Workspaces, durable shared history. LayerFS gives every agent a complete, writable filesystem workspace over shared committed history, without copying the base. Agents work in parallel, commit what changed as deduplicated checkpoints, then branch, rewind, or merge. Try it with LayerFS ร DSH.
Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends
Mirror of borgmatic: Simple, configuration-driven backup software for servers and workstations
Replace complex Borg Backup terminal commands with a beautiful web UI. Create, schedule, and restore backups with just a few clicks.
AI-based entity resolution, identity resolution, master data management, and deduplication at scale for humans and agents
A powerful and modular toolkit for record linkage and duplicate detection in Python
Collection of ready-made SQL queries for PostgreSQL for common tasks (retrieving and modifying data, speeding up queries, database maintenance)
Straightforward fuzzy matching, information retrieval and NLP building blocks for JavaScript.
Deduplicating archiver with encryption and paranoid-level tests. Swiss army knife for the serious backup and disaster recovery manager. Ransomware neutralizer. Win/Linux/Unix
Frequently Asked Questions
What is the most popular #deduplication repository on GitHub?
The top ranked open source #deduplication project on GitHub is restic/restic with 37,333 stars. Fast, secure, efficient backup program
Which programming languages are most used in #deduplication repositories?
Among the top ranked #deduplication projects on GitHub, the most common programming languages are Python, Go, C++, C.
How is the #deduplication repository ranking calculated?
We track 26 active open source repositories tagged with #deduplication (500+ stars), totaling 133,156 stars with an average of 5,121 stars per repository. Rankings update daily from the GitHub API.