# Martinhal ISDSS — v1.5 A self-hosted, secure data-access portal with an admin approval workflow. Built with Node.js + Express + SQLite. No external database or cloud service required. > © 2026 Martinhal IT - Joao Vaz - Version 1.5 Patch 0.2 --- ## What it does | # | Requirement | Where it lives | |---|-------------|----------------| | 1 | Login page for users | `public/login.html` + `routes/auth.js` (first boot: `public/setup.html` + `routes/setup.js`) | | 2 | Users can add MFA to their account | **My Account** page → `routes/auth.js` (TOTP via authenticator app) | | 3 | "View Data" page listing downloadable data | `public/view-data.html` (folder contents hidden until access is granted) | | 4 | Admin-only "Data Management" page: create/rename/delete/move folders & files, upload | `public/data-management.html` + `routes/manage.js` | | + | Admin user management (add the people who request data) | Data Management → **Users** tab → `routes/users.js` | | 5 | Items created in Data Management appear on View Data | Same folder/file tables power both pages | | 6 | Every access request emails **all administrators** that an approval is pending; approval/denial emails the requester | `routes/data.js` (`/request`, `/requests/:id/approve`) | | 7 | All email text is configurable, including uploading & using images | Data Management → **Email Templates** & **Brand Images** tabs → `routes/settings.js` | | + | Mail server configuration in the interface | Data Management → **Email Server** tab → `routes/mail.js` | | + | Log of every message sent/received, exportable & emailable | Data Management → **Email Log** tab → `routes/mail.js` | | 8 | "Logs" page recording every action on the other pages | `public/logs.html` + `lib/audit.js` | | + | Admin "Storage" page: server disk plus per-folder usage | `public/storage.html` + `routes/storage.js` | | 9 | Logs can be exported (CSV) and/or emailed | Logs page → `routes/logs.js` | | 10 | Footer on every page after login: `© 2026 Martinhal IT - Joao Vaz - Version 1.5 Patch 0.2` | Rendered by `public/js/app.js` | | + | ISDSS logo in the sidebar of signed-in pages | `public/assets/isdss-logo.png` | | + | Per-user profile picture, shown by the username | `routes/auth.js` avatar endpoints | | + | Copy of every sent email placed in the mail server's Sent folder | `lib/mailer.js` (IMAP) | | + | "Version Control" page for all users, managed by admins | `public/version-control.html` + `routes/versions.js` | | + | "Legislation" page for all users, managed by admins | `public/legislation.html` + `routes/legislation.js` | | + | Folder dates: Recorded Date + derived Legal Validity | `lib/dates.js`, shown on View Data | --- > **Already running an earlier version?** See **[UPGRADE.md](UPGRADE.md)** — > upgrading in place keeps all users, folders, files, approvals, and logs. ## Run with Docker (recommended) Everything runs in a single container. You only need Docker (and, for the one-command path, the Compose plugin). ### Option A — Docker Compose ```bash # 1. (Optional but recommended) set a session secret cat > .env <<'ENV' SESSION_SECRET=replace-with-a-long-random-string # COOKIE_SECURE=true # when served over HTTPS # SMTP_HOST=smtp.yourprovider.com # SMTP_PORT=587 # SMTP_USER=... # SMTP_PASS=... # MAIL_FROM=ISDSS ENV # 2. Build and start docker compose up -d --build ``` Now open **http://localhost:3000**. On this first visit the site shows a **setup wizard** asking you to create the administrator account — no credentials in environment files. Fill it in and you are signed in straight away. The database, uploaded files, and brand images persist in named volumes (`datahub-data`, `datahub-uploads`, `datahub-brand`), so they survive restarts and rebuilds. ```bash docker compose logs -f # follow logs docker compose down # stop (keeps volumes/data) docker compose down -v # stop and DELETE all data (setup runs again) ``` ### Option B — plain Docker ```bash docker build -t martinhal-datahub:1.3 . docker run -d --name datahub -p 3000:3000 \ -e SESSION_SECRET="replace-with-a-long-random-string" \ -v datahub-data:/app/data \ -v datahub-uploads:/app/uploads \ -v datahub-brand:/app/brand \ martinhal-datahub:1.3 ``` Notes: - The administrator account is created through the browser on first boot, so no password ever needs to live in an env file, a shell history, or your compose file. Until it is created, every page redirects to the wizard and the APIs are locked; once created, the wizard can never be opened again. - Set `COOKIE_SECURE=true` when you put the container behind an HTTPS reverse proxy (nginx/Caddy/Traefik). - The image runs as a non-root user and includes a container `HEALTHCHECK`. --- ## Run without Docker ## Requirements - **Node.js 18 or newer** (tested on Node 22) - npm ## Setup ```bash # 1. Install dependencies npm install # 2. Configure environment cp .env.example .env # → edit .env: set a long random SESSION_SECRET, and (optionally) your SMTP settings. # 3. Prepare the database (creates tables + default email templates) npm run init # 4. Start the server npm start ``` Then open **http://localhost:3000** and the **setup wizard** will ask you to create the administrator account. > `npm run init` no longer asks for credentials — it only prepares the > database, and it is safe to re-run at any time. --- ## First boot The very first time the site is opened it presents a one-step setup wizard: 1. Choose an **administrator username**, **email**, and **password** (minimum 10 characters, with at least one letter and one number). 2. The account is created and you are signed in immediately. 3. The wizard closes permanently — from then on `/setup.html` just redirects to the login page, and a second attempt to call the setup API is rejected. While no administrator exists, every page redirects to the wizard and the rest of the API returns `503 Setup required`, so the instance cannot be used in a half-configured state. Creating the account is recorded in the audit log as `SETUP_COMPLETED`. Right after setup, it's worth visiting **My Account** to switch on two-factor authentication for the administrator. --- ## How access works Access is granted **per folder**, and it cascades: - A regular user browsing **View Data** sees only the **names of top-level folders**. The contents — sub-folders and files — stay hidden until access is granted, so folder names are the only thing disclosed up front. - Requesting access to a folder notifies the administrators. Once approved, the user can open the folder and everything inside it: sub-folders, and all files, at any depth. **No second request is ever needed for the contents.** - Loose files sitting at the root (not inside any folder) are still requested individually, since they have no folder to inherit from. - Administrators see and download everything without requesting. The server enforces this independently of the interface: the tree endpoint filters out anything the user may not see, and the download route re-checks folder access on every request. --- ## Version Control page Every signed-in user gets a **Version Control** entry in the left menu showing the release history. The content is entirely admin-managed from **Data Management → Version Control**, where entries can be added, edited, and deleted. Each entry has a version, an optional title, release date (`YYYY-MM-DD`), free-text notes, and a display-order number (higher appears first). All three actions are audited (`VERSION_ENTRY_CREATED`, `VERSION_ENTRY_UPDATED`, `VERSION_ENTRY_DELETED`). --- ## Profile pictures Each user can set a profile picture from **My Account → Profile picture** (PNG, JPEG, WebP or GIF, up to 4 MB). It appears next to their name at the bottom-left of the sidebar on every signed-in page, and updates there immediately without a reload. Users with no picture show their initials on a disc instead. Pictures are stored under the `avatars/` directory (a persistent Docker volume, `AVATAR_DIR`), served only to signed-in users, and one picture per user: uploading a replacement deletes the previous file, and removing it deletes the file too. Uploads and removals are audited (`AVATAR_UPDATED`, `AVATAR_REMOVED`). The `avatar` / `avatar_mime` columns are added to existing databases automatically on the first start after upgrading. --- Uploading files into a folder shows a **progress bar** while the transfer runs, and the dialog stays open until it completes. --- ## Copying sent mail to the Sent folder When the platform sends an email it can also drop a copy into the mail server's **Sent** folder over IMAP, so messages the system sends appear alongside those sent by a person. Configure it under **Data Management → Email server** in the IMAP section: host, port, security, username, password, and optionally an exact Sent-folder name (left blank, the server's own Sent mailbox is detected, falling back to common names like `Sent`, `Sent Items`, `INBOX.Sent`, `[Gmail]/Sent Mail`). The IMAP username and password default to the SMTP ones, which is the usual case, and **Test IMAP** confirms the connection and which folder will be used. This is entirely optional: leave the IMAP host blank and nothing changes. Crucially, filing the copy is best-effort — if IMAP is unreachable or misconfigured the email is still sent normally, and only a `MAIL_SENT_COPY_FAILED` note is written to the log. A failed copy never turns a delivered message into an error. --- ## Branding Signed-in pages show the ISDSS logo at the top of the sidebar, from `public/assets/isdss-logo.png`. Because the artwork is dark-on-white and the sidebar is dark, it sits on a light rounded panel so it stays legible. The logo already contains the ISDSS wordmark, so it replaces the previous badge and text rather than sitting beside a duplicate label. The **login and first-run setup screens are deliberately left as they were**, with the original monogram. To change the logo, replace `public/assets/isdss-logo.png` (a wide landscape image around 3:1 works best; it is displayed about 184px wide, so roughly 520px wide keeps it crisp on high-resolution screens). If the file is ever missing the sidebar falls back to the original wordmark rather than showing a broken image. --- ## Date format Dates are **shown and entered as DD-MM-YYYY** throughout: folder dates, the Version Control and Legislation pages, the access-request table, the audit and email logs, and the CSV exports. Internally they are stored in ISO form (`YYYY-MM-DD`) so they sort and compare correctly, and converted for display. Date fields accept DD-MM-YYYY and validate it as a real calendar date. --- ## Folder dates Every folder carries a **Recorded Date**, chosen when the folder is created. From it a second date is derived: - **Recorded Date** — the date the material was recorded (`YYYY-MM-DD`). - **Legal Validity** — always the recorded date **plus 30 days**. Both are shown under each folder on **View Data**, for locked folders as well as open ones. Legal Validity is never stored: it is calculated from the recorded date on every request, so the two can never drift apart. A validity date in the past is marked *expired*. The date is required when creating a folder, is validated as a real calendar date, and can be corrected later with the calendar button on a folder in Data Management. It cannot be cleared once set. Folders created before this release have no date. They keep working and are listed with a *no recorded date* marker plus a banner in Data Management, so an administrator can fill them in; until then their Legal Validity shows as *not set*. --- ## Browsing folders in Data Management The folder tree collapses. Each folder has an arrow to show or hide what is inside it, plus a count of its sub-folders and files and its two dates at a glance. **Expand all** and **Collapse all** act on the whole tree. The open/closed state is kept as you work, so uploading or renaming does not reset your place. --- ## Legislation page Every signed-in user gets a **Legislation** entry in the left menu listing the laws, regulations, and internal rules governing the data. The content is admin-managed from **Data Management → Legislation**, where entries can be added, edited, and deleted. Each entry has an optional reference (e.g. `GDPR Art. 6`), a required title, an in-force date, a summary, an optional link to the full text, and a display-order number. All three actions are audited. --- ## Access validity and the countdown When an administrator approves a request they choose how long the access lasts: | Option | Effect | |--------|--------| | Valid for 24 hours | expires 24 hours after approval | | Valid for 15 days | expires 15 days after approval | | Valid for 30 days | expires 30 days after approval | | Valid forever | never expires | Every window is measured from **the moment the approval is confirmed**, not from the request date. On View Data each folder shows an **Access** field: - a live countdown (`14d 03h 21m 07s`) for timed access, turning amber in the final hour; - *Access granted — no expiry* for permanent access; - *Full access (administrator)* for administrators; - **Access Denied** when the user has no access, or once the window has lapsed. The countdown runs against the server's clock (the server sends its time with the folder list), so a wrong clock on someone's machine cannot make access look longer than it is. When a window lapses in front of the user the page refreshes itself, the folder's contents disappear again, and the user can submit a fresh request. Expiry is enforced on the server, not just in the interface: an expired approval stops the folder listing *and* blocks the download route. Access granted on a folder still cascades to everything inside it for as long as it is valid. --- ## Access request notifications The moment a user requests access, **every administrator** is emailed that a request is waiting for approval. The message states who asked and for what, the request reference, how many requests are currently pending, and carries a **Review pending requests** button linking straight to Data Management. Set the portal address under **Data Management → Email Server → Portal address** so that button points at your real hostname. Without it the link falls back to the address the request came in on. A few deliberate behaviours: - If no mail server is configured the notification is recorded as **queued** in the Email Log, and Data Management shows a warning banner saying administrators are not receiving these emails. - If delivery fails the failure is recorded in the Email Log and the audit log, but **the access request itself is still created** — mail problems never lose a request. - The wording lives in the `request_to_admin` template and is fully editable. Upgrades never overwrite a template you have customised; use **Restore default** in the template editor if you want to adopt the version shipped with a new release. --- ## Storage page (administrators) **Storage** in the Administration section of the menu shows where space is going. **Server disk** — total, used, and free space on the filesystem holding the uploads directory, with a bar that turns amber past 75% and red past 90%, and a warning when the disk is nearly full. Free space is reported the way `df` does (excluding blocks reserved for root), so the numbers match what you see on the host. **Used by ISDSS** — how much of that space this application accounts for, split into uploaded files, brand images, and the database (including its write-ahead log), each with the path it lives at. **Space used per folder** — every folder with two figures: **total** (the folder and everything nested inside it) and **own** (only the files sitting directly in it), plus a file count and a relative bar. Sort by size or by name. Files stored outside any folder are listed separately. **Consistency** — cross-checks the database against the disk and reports: - files recorded in the database but missing from disk (they appear in listings but cannot be downloaded); - files on disk that no record points at, with a **Reclaim space** button to delete them. That action is audited as `STORAGE_CLEANUP`. In Docker the figures describe the filesystem backing the mounted volume, which is what determines whether uploads will succeed. --- ## Mail server & email log ### Configuring the mail server **Data Management → Email Server** configures SMTP from inside the application — host, port, username, password, from-address, implicit TLS, and certificate verification. Two buttons help you check it before relying on it: - **Test connection** opens a connection and authenticates without saving. - **Send test message** sends a real message using the saved settings. Settings saved here take precedence over the `SMTP_*` environment variables. If you have an existing `.env`-based setup, it keeps working untouched — the page simply shows the values as coming from the environment until you override them. The stored password is never sent back to the browser; the field shows a mask, and saving the form with the mask left alone keeps the existing password. Note that the password is stored in the database in plain text, so treat `data/datahub.db` as a secret and keep file permissions tight. ### Email log **Data Management → Email Log** records every message the system sends: access requests, approvals and denials, log exports, and test messages. Each entry records time, direction, status (`delivered`, `queued`, or `failed`), sender, recipient, subject, a body preview, attachments, the triggering context, and the failure reason when delivery fails. Messages produced while no mail server is configured are recorded as **queued** rather than lost. The log can be filtered by text, direction, and status, then **exported as CSV** or **emailed** (the filtered set is attached as a CSV) to a specific address or to all administrators. **On received mail:** ISDSS sends mail but has no mailbox of its own, so there is nothing to poll for incoming messages. The log stores direction as a first class field and accepts inbound entries via `POST /api/mail/log/inbound`, so a forwarder or mail-hook can record replies alongside outgoing mail. Automatic collection would need IMAP credentials and a polling job, which is not part of this release. --- ## Managing users The wizard creates only the first administrator. Everyone else is added from **Data Management → Users**: - **New user** — username, **email (required)**, role, and an initial password (minimum 10 characters with a letter and a number). - **Edit** — change email or role, or set a new password. - **Reset MFA** — turns two-factor authentication off for someone who has lost their authenticator app, so they can set it up again from *My Account*. - **Delete** — removes the account. Data they published and their entries in the audit log are kept; only their pending access requests are removed. Two safeguards are enforced by the server: you cannot delete the account you are signed in with, and the last remaining administrator can be neither deleted nor demoted. **Email is mandatory on every account.** It is how approval notifications reach people, so an account without one cannot do its job. The rule is applied in three places: the browser form, the API (creating *or* updating an account), and — for databases created by this release onwards — a database constraint. An existing account can have its address changed but never cleared. If a database created by an earlier release contains an account with no address, the upgrade does **not** lock that user out. Instead the account is listed with a *missing — required* badge, a banner appears above the user list, and a warning is written to the server log at startup, so an administrator can add the address. Every one of these actions is written to the audit log (`USER_CREATED`, `USER_UPDATED`, `USER_DELETED`, `USER_MFA_RESET`). --- ## Email (SMTP) Email is optional for evaluation. If you don't configure SMTP, every outgoing message is written to the **Logs** as a `MAIL_OUTBOX` entry so nothing is lost and you can see exactly what *would* have been sent. To send real email, set these in `.env`: ``` SMTP_HOST=smtp.yourprovider.com SMTP_PORT=587 SMTP_SECURE=false SMTP_USER=your-username SMTP_PASS=your-password MAIL_FROM=ISDSS ``` Admin notification recipients are simply **every user whose role is `admin`**. --- ## How the approval flow works 1. A user browses **View Data** and clicks **Request access** on a file or folder. 2. An email (using the configurable *"Request received"* template) goes to all admins. 3. An admin opens **Data Management → Access Requests** and clicks **Approve** or **Deny**. 4. The requester receives an email (the *"approved"* or *"declined"* template). 5. Approved users can now **Download** the item from View Data. Admins can always download. --- ## Configurable emails & images In **Data Management**: - **Email Templates** — edit the subject and HTML body of each notification. Insert placeholders like `{{username}}`, `{{target_name}}`, `{{email}}`, `{{target_type}}`, `{{created_at}}` (click a chip to insert). Use **Preview** to see the rendered result with sample data. The footer is added automatically. - **Brand Images** — upload logos/images. Each uploaded image shows up as an insertable chip in the template editor, dropping a ready-to-use `` tag into the body. --- ## Logs Every meaningful action is recorded: logins, MFA changes, folder/file create·rename·move·delete, uploads, access requests, approvals/denials, downloads, template edits, image uploads, and email dispatch. From the **Logs** page you can filter by text/action/page, **Export CSV**, or **Email logs** (sends the filtered set as a CSV attachment to an address or to all admins). --- ## Project layout ``` datahub/ ├── UPGRADE.md # in-place upgrade guide for existing installs ├── Dockerfile # container image (multi-stage, non-root, healthcheck) ├── docker-compose.yml # one-command run with persistent volumes ├── docker-entrypoint.sh # ensures dirs, runs DB init, starts the server ├── .dockerignore ├── server.js # app entry, session + route wiring, page gating ├── db.js # SQLite schema & connection ├── init-db.js # create first admin / seed demo users ├── lib/ │ ├── audit.js # log() helper │ ├── mailer.js # nodemailer transport, mail logging, no-SMTP fallback │ ├── settings.js # key/value settings helper │ ├── dates.js # recorded date + derived legal validity (+30 days) │ ├── validate.js # shared account field rules (email is mandatory) │ └── templates.js # email template store, render, defaults ├── middleware/auth.js # requireAuth / requireAdmin ├── routes/ │ ├── setup.js # first-boot wizard API (creates the first admin) │ ├── users.js # admin user management (create/edit/reset/delete) │ ├── versions.js # Version Control entries (read: all, write: admin) │ ├── legislation.js # Legislation entries (read: all, write: admin) │ ├── mail.js # SMTP configuration + mail log (export / email) │ ├── storage.js # disk usage + per-folder space (admin) │ ├── auth.js # login, logout, session, MFA │ ├── manage.js # admin folder/file CRUD + upload (Data Management) │ ├── data.js # View Data tree, requests, approvals, downloads │ ├── settings.js # email templates + brand images │ └── logs.js # list, CSV export, email export ├── public/ # setup, login, view-data, version-control, legislation, │ # data-management, storage, account, logs + css/js │ └── assets/ # isdss-logo.png (sidebar logo on signed-in pages) ├── avatars/ # per-user profile pictures (Docker volume) ├── data/ # SQLite database (created at runtime) ├── uploads/ # stored files (created at runtime) └── brand/ # uploaded brand images (created at runtime) ``` --- ## Security notes for production - Serve behind HTTPS and set `COOKIE_SECURE=true` in `.env`. - Set a strong, unique `SESSION_SECRET`. - Passwords are hashed with bcrypt; MFA uses TOTP (RFC 6238). - Consider putting the app behind a reverse proxy (nginx/Caddy) and adding rate-limiting and backups of the `data/`, `uploads/`, and `brand/` folders. - Uploaded files are served only through the authenticated, approval-checked download route — never from a public static path. --- ## License MIT.