near-duplicate-detection

Image similarity in Golang. Version 4 (LATEST)

ISCC: International Standard Content Code

A Simple Image Clustering Script using CLIP and Hierarchial Clustering

Test-contamination detector - finds train/eval overlap via deterministic shingle matching, per-source rollups, CI-ready exit codes. Standard library only.

Fast image similarity search with hash tables (Golang). Version 2 (LATEST)

Python library for detecting near duplicate texts in a corpus at scale.

Simple library for finding duplicate and near-duplicate text documents in massive sets/libraries/databases

Fast image similarity search with hash tables (Golang). Version 1

Multi module project focused on near-duplicate search for images.

Holds code for near-duplicate image parser using optimized image classifiers.

Production-oriented file deduplication and data-protection platform with tenant isolation, SHA-256 exact deduplication, MinHash/LSH and dHash near-duplicate detection, DLP, AES-256-GCM encryption, RBAC, and audit logging.

Scalable near-duplicate image detection using Vision Transformers, Siamese Networks, Triplet Loss, Deep Hashing, and FAISS.

an application for comparing images using various image hashing algorithms

Bachelor's Thesis on Near-Duplicate Image Detection. This repo contains all resources, code, and documentation developed during the process.

Perceptual video deduplication on Backblaze B2: hash a video library with videohash, cluster near-duplicates by Hamming distance, and reclaim storage. Local, open-source, keyless compute — B2 holds the library, the hash index, and every cluster report.

Perceptual image hashing in Python, with the libphash C library inside: find duplicate and near-duplicate images with pHash, dHash, aHash, wHash and five more algorithms.

Code similarity search by compression - find near-duplicate files, clone families, and where a pasted snippet came from. No embeddings, no model, no vector database.

Dependency-free 64-bit perceptual hashes (pHash/DHash/AHash) plus a measured map of what they are and are not invariant to

Where did this text come from? Exact copies, near-duplicates and paraphrases with highlighted evidence and Wayback first-seen dates — in the browser, plus a FastAPI server.

Find duplicated files including permutations where underscores replace spaces. Allow a tolerance and use a signature for audio files to ignore metadata variations.