281 lines
9.8 KiB
Markdown
281 lines
9.8 KiB
Markdown
# PST Email Extractor
|
||
|
||
A command-line tool that scans a directory for Outlook PST files and extracts the metadata of every email — subject, sender, recipients, date, size, attachments count, and message class — into individual CSV files.
|
||
|
||
---
|
||
|
||
## Table of Contents
|
||
|
||
1. [Features](#features)
|
||
2. [Requirements](#requirements)
|
||
3. [Installation](#installation)
|
||
4. [Usage](#usage)
|
||
5. [Output Format](#output-format)
|
||
6. [How It Works](#how-it-works)
|
||
7. [Performance Notes](#performance-notes)
|
||
8. [Troubleshooting](#troubleshooting)
|
||
9. [Platform-specific pypff Installation](#platform-specific-pypff-installation)
|
||
10. [Project Structure](#project-structure)
|
||
11. [FAQ](#faq)
|
||
|
||
---
|
||
|
||
## Features
|
||
|
||
- **Batch processing** – drop any number of `.pst` files in a folder and run once
|
||
- **Resilient extraction** – corrupt messages are skipped gracefully; processing continues
|
||
- **Rich metadata** – exports 8 fields per message including attachment count and message class
|
||
- **Progress bars** – real-time per-file progress via `tqdm`
|
||
- **UTF-8 output** – fully compatible with Excel, Google Sheets, and any modern CSV tool
|
||
- **Iterative folder traversal** – avoids Python recursion limits on deeply nested PST archives
|
||
- **Human-friendly summaries** – per-run statistics on success / failure counts
|
||
|
||
---
|
||
|
||
## Requirements
|
||
|
||
| Dependency | Version | Purpose |
|
||
|---|---|---|
|
||
| Python | ≥ 3.8 | Runtime |
|
||
| `tqdm` | ≥ 4.66 | Progress bars |
|
||
| `pypff` / `libpff-python` | any | PST file reading |
|
||
|
||
> **Note:** `pypff` wraps the native C library **libpff**. Installation varies by platform — see [Platform-specific pypff Installation](#platform-specific-pypff-installation).
|
||
|
||
---
|
||
|
||
## Installation
|
||
|
||
### Step 1 – Clone or download the project
|
||
|
||
Place `Extract.py` and `install_dependencies.py` in the same directory.
|
||
|
||
### Step 2 – Run the dependency installer
|
||
|
||
```bash
|
||
python install_dependencies.py
|
||
```
|
||
|
||
This script will:
|
||
|
||
1. Upgrade `pip`, `setuptools`, and `wheel`.
|
||
2. Install `tqdm` via pip.
|
||
3. Attempt to install `libpff-python` / `pypff` via pip.
|
||
4. Print platform-specific instructions if the automatic install fails.
|
||
|
||
### Step 3 – Verify
|
||
|
||
```bash
|
||
python -c "import pypff; import tqdm; print('All good!')"
|
||
```
|
||
|
||
---
|
||
|
||
## Usage
|
||
|
||
### Basic usage (process PST files in the script's own directory)
|
||
|
||
```bash
|
||
python Extract.py
|
||
```
|
||
|
||
### Specify a directory
|
||
|
||
```bash
|
||
python Extract.py /path/to/pst/folder
|
||
```
|
||
|
||
```powershell
|
||
# Windows PowerShell
|
||
python Extract.py "C:\Users\You\Desktop\PST_Files"
|
||
```
|
||
|
||
### Example output
|
||
|
||
```
|
||
📂 Working directory: /data/pst_files
|
||
|
||
================================================================================
|
||
🔍 SEARCHING FOR PST FILES
|
||
================================================================================
|
||
✓ Found 3 PST file(s)
|
||
|
||
📁 PST Files:
|
||
1. archive_2021.pst (1,204.3 MB)
|
||
2. archive_2022.pst (876.1 MB)
|
||
3. personal.pst (312.7 MB)
|
||
|
||
================================================================================
|
||
🚀 PROCESSING
|
||
================================================================================
|
||
|
||
────────────────────────────────────────────────────────────────────────────────
|
||
📊 1/3 | ✓ Done: 0 | ❌ Failed: 0 | ⏳ Remaining: 2
|
||
🔄 archive_2021.pst
|
||
📊 Counting messages in archive_2021.pst …
|
||
✓ 42,817 messages found
|
||
Processing archive_2021.pst: 100%|████████| 42817/42817 [02:14<00:00]
|
||
✓ Written 42,817 rows | Errors skipped: 3
|
||
✅ Saved → archive_2021_email_list.csv (42,817 rows)
|
||
```
|
||
|
||
---
|
||
|
||
## Output Format
|
||
|
||
For each input file `<name>.pst`, a corresponding `<name>_email_list.csv` is created in the **same directory**.
|
||
|
||
### CSV columns
|
||
|
||
| Column | Description | Max Length |
|
||
|---|---|---|
|
||
| `Folder` | Full folder path inside the PST (e.g. `Inbox\Projects\2022`) | 200 chars |
|
||
| `Subject` | Email subject line | 500 chars |
|
||
| `From` | Sender name and/or email address | 200 chars |
|
||
| `To` | Semicolon-separated recipient list | 1000 chars |
|
||
| `Date` | Delivery or submit time in `YYYY-MM-DD HH:MM:SS UTC` | — |
|
||
| `Size` | Estimated message size in bytes (sum of body and headers) | — |
|
||
| `Attachments` | Number of attachments | — |
|
||
| `MessageClass` | MAPI message class (e.g. `IPM.Note`, `IPM.Appointment`) | 100 chars |
|
||
|
||
### Example rows
|
||
|
||
```csv
|
||
Folder,Subject,From,To,Date,Size,Attachments,MessageClass
|
||
Inbox,Q3 Budget Review,Alice Smith <alice@example.com>,bob@example.com,2023-09-01 14:22:10 UTC,18432,2,IPM.Note
|
||
Sent Items,Re: Proposal,Bob Jones <bob@example.com>,alice@example.com,2023-09-02 09:05:44 UTC,4210,0,IPM.Note
|
||
Calendar,,Alice Smith <alice@example.com>,,2023-10-10 09:00:00 UTC,1024,0,IPM.Appointment
|
||
```
|
||
|
||
---
|
||
|
||
## How It Works
|
||
|
||
1. **Discovery** – `find_pst_files()` uses `glob` to locate all `.pst` files in the given directory.
|
||
2. **Counting** – Before extraction, `count_total_messages()` recursively counts all messages so `tqdm` can show an accurate progress bar.
|
||
3. **Traversal** – `process_folder()` uses an **iterative stack** (not Python recursion) to walk the full folder tree, guarding against deep nesting and corrupt nodes.
|
||
4. **Extraction** – `extract_email_info()` wraps every individual attribute access in a `try/except` so one corrupt property never aborts the whole message.
|
||
5. **Writing** – Results are streamed directly to a UTF-8 CSV file via `csv.writer`, keeping memory usage flat regardless of PST size.
|
||
6. **Error recovery** – Failed messages are counted and reported in the summary but do not interrupt processing.
|
||
|
||
---
|
||
|
||
## Performance Notes
|
||
|
||
- **Memory** – The extractor streams rows directly to disk, so even a 50 GB PST file won't exhaust RAM.
|
||
- **Speed** – Processing speed depends on your disk I/O. SSDs can process ~5,000–15,000 messages/min; spinning disks are 3–5× slower.
|
||
- **Large recipient lists** – Recipient lists are capped at 100 entries and 1,000 characters to avoid giant rows.
|
||
- **Iterative traversal** – The folder walker uses an explicit stack instead of recursion, making it safe for PST archives with hundreds of nested folders.
|
||
|
||
---
|
||
|
||
## Troubleshooting
|
||
|
||
### `ModuleNotFoundError: No module named 'pypff'`
|
||
|
||
Run `python install_dependencies.py`. If the automatic install fails, see [Platform-specific pypff Installation](#platform-specific-pypff-installation).
|
||
|
||
### `Failed to open PST file`
|
||
|
||
- The file may be in use by Outlook. Close Outlook completely and try again.
|
||
- The file may be corrupted. Try running the Outlook Inbox Repair Tool (`scanpst.exe` on Windows).
|
||
|
||
### Progress bar reaches 100% but fewer rows are written than expected
|
||
|
||
This is normal. Some items in a PST folder hierarchy are not email messages (calendar events, contacts, tasks). They are counted in the total but may fail extraction silently. The `Errors skipped` count in the summary reflects this.
|
||
|
||
### CSV opens with garbled characters in Excel
|
||
|
||
The CSV is UTF-8. In Excel, use **Data → From Text/CSV** and choose **UTF-8** encoding, or open via `File → Open` and select the encoding in the import wizard.
|
||
|
||
### Very slow processing on large PST files
|
||
|
||
Ensure the PST file is on a local drive, not a network share or cloud-synced folder. Network latency multiplies enormously across hundreds of thousands of small reads.
|
||
|
||
---
|
||
|
||
## Platform-specific pypff Installation
|
||
|
||
### Linux (Debian / Ubuntu)
|
||
|
||
```bash
|
||
sudo apt-get install python3-libpff
|
||
# or
|
||
pip install libpff-python
|
||
```
|
||
|
||
### Linux (Fedora / RHEL)
|
||
|
||
```bash
|
||
sudo dnf install libpff-devel python3-libpff
|
||
```
|
||
|
||
### macOS (Homebrew)
|
||
|
||
```bash
|
||
brew install libpff
|
||
pip install libpff-python
|
||
```
|
||
|
||
### Windows
|
||
|
||
Automatic pip installation may fail because libpff requires compilation.
|
||
|
||
**Option 1 – Pre-built wheel** (if available for your Python version):
|
||
|
||
```powershell
|
||
pip install libpff-python
|
||
```
|
||
|
||
**Option 2 – Build from source** – follow the official guide:
|
||
https://github.com/libyal/libpff/blob/main/documentation/Building.md
|
||
|
||
**Option 3 – WSL (recommended)** – Install Windows Subsystem for Linux, then follow the Ubuntu steps above. This is the easiest path for most Windows users.
|
||
|
||
---
|
||
|
||
## Project Structure
|
||
|
||
```
|
||
.
|
||
├── Extract.py # Main extractor – run this
|
||
├── install_dependencies.py # One-time dependency installer
|
||
└── README.md # This file
|
||
```
|
||
|
||
Outputs are placed alongside the input PST files:
|
||
|
||
```
|
||
/your/pst/folder/
|
||
├── archive_2021.pst
|
||
├── archive_2021_email_list.csv ← generated
|
||
├── archive_2022.pst
|
||
└── archive_2022_email_list.csv ← generated
|
||
```
|
||
|
||
---
|
||
|
||
## FAQ
|
||
|
||
**Does the tool modify the PST files?**
|
||
No. PST files are opened read-only. The tool never writes to them.
|
||
|
||
**Can I run it on a single PST file instead of a whole directory?**
|
||
Put the PST file in its own folder and pass that folder path. Alternatively, the function `export_pst_to_csv(pst_path, output_csv_path)` can be imported and called directly from your own script.
|
||
|
||
**What happens if I run it twice on the same directory?**
|
||
Existing CSV files are overwritten without warning. If you want to preserve previous results, move or rename them first.
|
||
|
||
**Does it export email bodies or attachments?**
|
||
No — only metadata is exported. This keeps the output small and avoids legal / privacy issues associated with exporting full message content.
|
||
|
||
**What message types are included?**
|
||
All MAPI message objects found in the PST: emails (`IPM.Note`), meeting requests (`IPM.Schedule.*`), appointments (`IPM.Appointment`), contacts (`IPM.Contact`), tasks (`IPM.Task`), etc. The `MessageClass` column lets you filter by type in your spreadsheet.
|
||
|
||
**Will it work with OST files?**
|
||
libpff has partial OST support. Rename the file to `.pst` and try — results may vary depending on the OST version and whether the file is currently synced.
|
||
|
||
---
|
||
|
||
*For bug reports and feature requests, open an issue in your project repository.*
|