🥁🥁Undouble - Python library for detecting duplicate images using hash functions
Finding identical or similar photos manually is a long and tedious task. ITS can not be solved simply by comparing the size and file name, because. photos are taken from different sources (mobile devices, social networking applications, etc.), which results in differences in these attributes and creates differences in resolution, scaling, compression, and brightness. Hash functions are ideal for detecting identical and similar photos due to their resistance to minor changes. This idea is the basis of Undouble - the Python library that works using a multi-stage image preprocessing process (grayscale, normalization and scaling), image hash calculation, and image grouping. Threshold 0 will group images with identical image hash. The results can be easily examined using the plotting function, and the images can be moved using the move function. When moving images, the image from the group with the highest resolution is copied, and all other images are moved to the "undouble" subdirectory.
To try this open source library (https://github.com/erdogant/undouble), you first need to install it: pip install undouble, then import the package into your project: from undouble import Undouble. Then, by setting the hash method and hash size, duplicates can be detected using undouble. In this case, the following steps are performed: recursively reading all images from the directory with the specified extensions, computing the hash, and grouping similar images.
See an example with explanations here:
https://towardsdatascience.com/detection-of-duplicate-images-using-image-hash-functions-4d9c53f04a75