66 lines
2.4 KiB
Markdown
66 lines
2.4 KiB
Markdown
|
# dedupe.py — Duplicate File Finder / Deleter
|
||
|
|
|
||
|
|
Scans a directory for duplicate files, keeps one copy of each, and reports or deletes the rest.
|
||
|
|
|
||
|
|
## What it does
|
||
|
|
|
||
|
|
It groups files by size as a fast first pass, then confirms real duplicates by comparing **SHA-256 content hashes**. For each set of identical files it keeps one copy and reports (or deletes) the others.
|
||
|
|
|
||
|
|
By default the script runs in **dry-run mode** — it only tells you what it *would* delete. Nothing is removed unless you explicitly pass `--delete`.
|
||
|
|
|
||
|
|
## ⚠️ Important
|
||
|
|
|
||
|
|
Two files with the same **size** are *not* necessarily the same file. That's why the script hashes file contents before deleting. Only use size-only matching (`--size-only`) if you fully understand the risk of deleting unrelated files.
|
||
|
|
|
||
|
|
## Requirements
|
||
|
|
|
||
|
|
- Python 3.6 or newer
|
||
|
|
- No third-party packages (standard library only)
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
|
||
|
|
```
|
||
|
|
python dedupe.py DIRECTORY [options]
|
||
|
|
```
|
||
|
|
|
||
|
|
`DIRECTORY` is the folder to scan.
|
||
|
|
|
||
|
|
### Options
|
||
|
|
|
||
|
|
| Option | Description |
|
||
|
|
| --- | --- |
|
||
|
|
| `--delete` | Actually delete the duplicate copies. Without this flag the script only prints what it would do (dry run). |
|
||
|
|
| `--size-only` | Match files on size alone and skip content hashing. **Dangerous:** same size does not mean same content. |
|
||
|
|
| `--no-recursive` | Only scan the top level of `DIRECTORY`; do not descend into subfolders. |
|
||
|
|
| `-h`, `--help` | Show the built-in help message. |
|
||
|
|
|
||
|
|
### Examples
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# See what would be deleted in ~/Downloads (safe, nothing removed)
|
||
|
|
python dedupe.py ~/Downloads
|
||
|
|
|
||
|
|
# Actually delete verified duplicates
|
||
|
|
python dedupe.py ~/Downloads --delete
|
||
|
|
|
||
|
|
# Only look at the top level, not subfolders
|
||
|
|
python dedupe.py ~/Downloads --no-recursive
|
||
|
|
|
||
|
|
# Match purely on size (risky) — dry run first
|
||
|
|
python dedupe.py ~/Downloads --size-only
|
||
|
|
```
|
||
|
|
|
||
|
|
## How it decides what to keep
|
||
|
|
|
||
|
|
Within each set of duplicates, the files are sorted alphabetically by path and the **first** one is kept. All others in the set are the candidates for deletion.
|
||
|
|
|
||
|
|
## Output
|
||
|
|
|
||
|
|
For each duplicate set the script prints the file it keeps and each file it would delete (or has deleted). At the end it prints a summary with the number of files affected and the amount of disk space reclaimed.
|
||
|
|
|
||
|
|
## Safety tips
|
||
|
|
|
||
|
|
1. Always run once **without** `--delete` and review the output first.
|
||
|
|
2. Make sure you have a backup before deleting anything important.
|
||
|
|
3. Avoid `--size-only` unless you're certain size alone is enough.
|