PRIVATE_SELF_DEV/Infosec
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v_2.3

Browse Source
This commit is contained in:
jpmvaz
2026-09-13 20:10:17 +01:00
parent 9b5cc30fb4
commit b192fb3f17
482 changed files with 0 additions and 48205 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+601
View File
@@ -0,0 +1,601 @@
# Infosec — v2.3
A self-hosted, secure data-access portal with an admin approval workflow.
Built with Node.js + Express + SQLite. No external database or cloud service required.
> © 2026 Martinhal IT - Joao Vaz - Version 2.3
---
## What it does
| # | Requirement | Where it lives |
|---|-------------|----------------|
| 1 | Login page for users | `public/login.html` + `routes/auth.js` (first boot: `public/setup.html` + `routes/setup.js`) |
| 2 | Users can add MFA to their account | **My Account** page → `routes/auth.js` (TOTP via authenticator app) |
| 3 | "View Data" page listing data to **Open** (view in a new tab) or **Download** | `public/view-data.html` (folder contents hidden until access is granted) |
| + | "Calendar View" page: month grid marking days that have events (folder Incident Dates); click a day to list them | `public/calendar-view.html` (reuses `/api/data/tree`) |
| 4 | Admin-only "Data Management" page: create/rename/delete/move folders & files, upload | `public/data-management.html` + `routes/manage.js` |
| + | **Data Creation** tab (Data Management): build files from scratch — a text-and-image **Document** or an event **Timeline** with branches — saved into any folder | `public/data-management.html` + `routes/datacreate.js` |
| + | Admin user management (add the people who request data) | Data Management → **Users** tab → `routes/users.js` |
| 5 | Items created in Data Management appear on View Data | Same folder/file tables power both pages |
| 6 | Every access request emails **all administrators** that an approval is pending; approval/denial emails the requester | `routes/data.js` (`/request`, `/requests/:id/approve`) |
| 7 | All email text is configurable, including uploading & using images | Data Management → **Email Templates** & **Brand Images** tabs → `routes/settings.js` |
| + | Mail server configuration in the interface | Data Management → **Email Server** tab → `routes/mail.js` |
| + | Log of every message sent/received, exportable & emailable | Data Management → **Email Log** tab → `routes/mail.js` |
| 8 | "Logs" page recording every action on the other pages | `public/logs.html` + `lib/audit.js` |
| + | Admin "Storage" page: server disk plus per-folder usage | `public/storage.html` + `routes/storage.js` |
| 9 | Logs can be exported (CSV) and/or emailed | Logs page → `routes/logs.js` |
| 10 | Footer on every page after login: `© 2026 Martinhal IT - Joao Vaz - Version 2.3` | Rendered by `public/js/app.js` |
| + | Infosec logo in the sidebar of signed-in pages | `public/assets/infosec-logo.png` |
| + | Per-user profile picture, shown by the username | `routes/auth.js` avatar endpoints |
| + | Copy of every sent email placed in the mail server's Sent folder | `lib/mailer.js` (IMAP) |
| + | Admin-only "Version Control" page (release history), linked below Logs | `public/version-control.html` + `routes/versions.js` |
| + | "Legislation" page for all users, managed by admins | `public/legislation.html` + `routes/legislation.js` |
| + | Folder dates: Incident Date + derived Legal Validity | `lib/dates.js`, shown on View Data & Data Management |
---
> **Already running an earlier version?** See **[UPGRADE.md](UPGRADE.md)** —
> upgrading in place keeps all users, folders, files, approvals, and logs.
## Run with Docker (recommended)
Everything runs in a single container. You only need Docker (and, for the
one-command path, the Compose plugin).
### Option A — Docker Compose
```bash
# 1. (Optional but recommended) set a session secret
cat > .env <<'ENV'
SESSION_SECRET=replace-with-a-long-random-string
# COOKIE_SECURE=true # when served over HTTPS
# SMTP_HOST=smtp.yourprovider.com
# SMTP_PORT=587
# SMTP_USER=...
# SMTP_PASS=...
# MAIL_FROM=Infosec <no-reply@yourdomain.com>
ENV
# 2. Build and start
docker compose up -d --build
```
Now open **http://localhost:3000**. On this first visit the site shows a
**setup wizard** asking you to create the administrator account — no
credentials in environment files. Fill it in and you are signed in straight
away.
The database, uploaded files, and brand images persist in named volumes
(`datahub-data`, `datahub-uploads`, `datahub-brand`), so they survive restarts
and rebuilds.
```bash
docker compose logs -f # follow logs
docker compose down # stop (keeps volumes/data)
docker compose down -v # stop and DELETE all data (setup runs again)
```
### Option B — plain Docker
```bash
docker build -t martinhal-datahub:1.3 .
docker run -d --name datahub -p 3000:3000 \
-e SESSION_SECRET="replace-with-a-long-random-string" \
-v datahub-data:/app/data \
-v datahub-uploads:/app/uploads \
-v datahub-brand:/app/brand \
martinhal-datahub:1.3
```
Notes:
- The administrator account is created through the browser on first boot, so
no password ever needs to live in an env file, a shell history, or your
compose file. Until it is created, every page redirects to the wizard and the
APIs are locked; once created, the wizard can never be opened again.
- Set `COOKIE_SECURE=true` when you put the container behind an HTTPS reverse
proxy (nginx/Caddy/Traefik).
- The image runs as a non-root user and includes a container `HEALTHCHECK`.
---
## Run without Docker
## Requirements
- **Node.js 18 or newer** (tested on Node 22)
- npm
## Setup
```bash
# 1. Install dependencies
npm install
# 2. Configure environment
cp .env.example .env
# → edit .env: set a long random SESSION_SECRET, and (optionally) your SMTP settings.
# 3. Prepare the database (creates tables + default email templates)
npm run init
# 4. Start the server
npm start
```
Then open **http://localhost:3000** and the **setup wizard** will ask you to
create the administrator account.
> `npm run init` no longer asks for credentials — it only prepares the
> database, and it is safe to re-run at any time.
---
## First boot
The very first time the site is opened it presents a one-step setup wizard:
1. Choose an **administrator username**, **email**, and **password**
(minimum 10 characters, with at least one letter and one number).
2. The account is created and you are signed in immediately.
3. The wizard closes permanently — from then on `/setup.html` just redirects
to the login page, and a second attempt to call the setup API is rejected.
While no administrator exists, every page redirects to the wizard and the rest
of the API returns `503 Setup required`, so the instance cannot be used in a
half-configured state. Creating the account is recorded in the audit log as
`SETUP_COMPLETED`.
Right after setup, it's worth visiting **My Account** to switch on two-factor
authentication for the administrator.
---
## How access works
Access is granted **per folder**, and it cascades:
- A regular user browsing **View Data** sees only the **names of top-level
folders**. The contents — sub-folders and files — stay hidden until access is
granted, so folder names are the only thing disclosed up front.
- Requesting access to a folder notifies the administrators. Once approved, the
user can open the folder and everything inside it: sub-folders, and all files,
at any depth. **No second request is ever needed for the contents.**
- Loose files sitting at the root (not inside any folder) are still requested
individually, since they have no folder to inherit from.
- Administrators see and download everything without requesting.
The server enforces this independently of the interface: the tree endpoint
filters out anything the user may not see. Files can be **opened in a new tab**
(rendered in place) as well as downloaded; both the view and download routes
re-check folder access on every request, and the view route additionally serves
the file hard-sandboxed (`Content-Security-Policy: sandbox`, `nosniff`) so an
opened file cannot run scripts or reach the app session.
---
## Calendar View
**Calendar View** (in the left menu, next to View Data — available to every signed-in user) gives a month-by-month overview of which days have events. An *event* is a folder's **Incident Date**, so the calendar highlights each day that has one or more folders recorded on it and shows a count. Clicking a highlighted day lists those folders with their incident date, Legal Validity (marked when expired), and access state; folders you can open link through to **View Data**. The page reuses the same access-filtered `/api/data/tree` endpoint as View Data, so a user only ever sees the events they are allowed to see, and administrators see them all. No data is stored specifically for the calendar — it is derived from the folders that already exist.
## Data Creation
The **Data Management** page has a **Data Creation** tab (admin only) for producing files without uploading them. Two kinds are supported:
- **Document** — an ordered mix of free-text blocks and images (with optional captions). Images are embedded directly into the output.
- **Timeline** — a list of main events, each with **Name**, **Date**, **Time** and **Additional Data**, plus optional **branches** off any event that carry the same four fields.
You pick a **destination folder** and a **file name**, optionally **Preview** the result, then **Save**. The output is written as a single self-contained `.html` file (no scripts) and stored through the same `files` table and `uploads/` directory as uploaded files — so a created file appears in its folder and behaves like any other file on View Data, subject to the usual access control. Creation is audited as `FILE_CREATE`.
## Version Control page
Administrators get a **Version Control** entry in the sidebar's Administration
section (directly below **Logs**) showing the release history. The page and its
data are admin-only. Entries are managed from
**Data Management → Version Control**, where entries can be added, edited, and
deleted. Each entry has a version, an optional title, release date
(`YYYY-MM-DD`), free-text notes, and a display-order number (higher appears
first). All three actions are audited (`VERSION_ENTRY_CREATED`,
`VERSION_ENTRY_UPDATED`, `VERSION_ENTRY_DELETED`).
---
## Profile pictures
Each user can set a profile picture from **My Account → Profile picture**
(PNG, JPEG, WebP or GIF, up to 4 MB). It appears next to their name at the
bottom-left of the sidebar on every signed-in page, and updates there
immediately without a reload. Users with no picture show their initials on a
disc instead.
Pictures are stored under the `avatars/` directory (a persistent Docker volume,
`AVATAR_DIR`), served only to signed-in users, and one picture per user: uploading
a replacement deletes the previous file, and removing it deletes the file too.
Uploads and removals are audited (`AVATAR_UPDATED`, `AVATAR_REMOVED`).
The `avatar` / `avatar_mime` columns are added to existing databases
automatically on the first start after upgrading.
---
Uploading files into a folder shows a **progress bar** while the transfer runs,
and the dialog stays open until it completes.
---
## Copying sent mail to the Sent folder
When the platform sends an email it can also drop a copy into the mail server's
**Sent** folder over IMAP, so messages the system sends appear alongside those
sent by a person. Configure it under **Data Management → Email server** in the
IMAP section: host, port, security, username, password, and optionally an exact
Sent-folder name (left blank, the server's own Sent mailbox is detected, falling
back to common names like `Sent`, `Sent Items`, `INBOX.Sent`,
`[Gmail]/Sent Mail`). The IMAP username and password default to the SMTP ones,
which is the usual case, and **Test IMAP** confirms the connection and which
folder will be used.
This is entirely optional: leave the IMAP host blank and nothing changes.
Crucially, filing the copy is best-effort — if IMAP is unreachable or misconfigured
the email is still sent normally, and only a `MAIL_SENT_COPY_FAILED` note is
written to the log. A failed copy never turns a delivered message into an error.
---
## Branding
Signed-in pages show the Infosec logo at the top of the sidebar, from
`public/assets/infosec-logo.png`. Because the artwork is dark-on-white and the
sidebar is dark, it sits on a light rounded panel so it stays legible. The logo
already contains the Infosec wordmark, so it replaces the previous badge and text
rather than sitting beside a duplicate label.
The **login and first-run setup screens are deliberately left as they were**,
with the original monogram.
To change the logo, replace `public/assets/infosec-logo.png` (a wide landscape
image around 3:1 works best; it is displayed about 184px wide, so roughly 520px
wide keeps it crisp on high-resolution screens). If the file is ever missing the
sidebar falls back to the original wordmark rather than showing a broken image.
---
## Date format
Dates are **shown and entered as DD-MM-YYYY** throughout: folder dates, the
Version Control and Legislation pages, the access-request table, the audit and
email logs, and the CSV exports. Internally they are stored in ISO form
(`YYYY-MM-DD`) so they sort and compare correctly, and converted for display.
Date fields accept DD-MM-YYYY and validate it as a real calendar date.
---
## Folder dates
Every folder carries an **Incident Date**, chosen when the folder is created.
From it a second date is derived:
- **Incident Date** — the date the material was recorded (`YYYY-MM-DD`); labelled **Incident Date** on both View Data and Data Management (the underlying field is unchanged).
- **Legal Validity** — always the incident date **plus 30 days**.
Both are shown under each folder on **View Data**, for locked folders as well as
open ones. Legal Validity is never stored: it is calculated from the incident
date on every request, so the two can never drift apart. A validity date in the
past is marked *expired*.
The date is required when creating a folder, is validated as a real calendar
date, and can be corrected later with the calendar button on a folder in Data
Management. It cannot be cleared once set.
Folders created before this release have no date. They keep working and are
listed with a *no incident date* marker plus a banner in Data Management, so an
administrator can fill them in; until then their Legal Validity shows as
*not set*.
---
## Browsing folders in Data Management
The folder tree collapses. Each folder has an arrow to show or hide what is
inside it, plus a count of its sub-folders and files and its two dates at a
glance. **Expand all** and **Collapse all** act on the whole tree. The open/closed
state is kept as you work, so uploading or renaming does not reset your place.
---
## Legislation page
Every signed-in user gets a **Legislation** entry in the left menu listing the
laws, regulations, and internal rules governing the data. The content is
admin-managed from **Data Management → Legislation**, where entries can be
added, edited, and deleted. Each entry has an optional reference (e.g.
`GDPR Art. 6`), a required title, an in-force date, a summary, an optional link
to the full text, and a display-order number. All three actions are audited.
---
## Access validity and the countdown
When an administrator approves a request they choose how long the access lasts:
| Option | Effect |
|--------|--------|
| Valid for 24 hours | expires 24 hours after approval |
| Valid for 15 days | expires 15 days after approval |
| Valid for 30 days | expires 30 days after approval |
| Valid forever | never expires |
Every window is measured from **the moment the approval is confirmed**, not from
the request date.
On View Data each folder shows an **Access** field:
- a live countdown (`14d 03h 21m 07s`) for timed access, turning amber in the
final hour;
- *Access granted — no expiry* for permanent access;
- *Full access (administrator)* for administrators;
- **Access Denied** when the user has no access, or once the window has lapsed.
The countdown runs against the server's clock (the server sends its time with
the folder list), so a wrong clock on someone's machine cannot make access look
longer than it is. When a window lapses in front of the user the page refreshes
itself, the folder's contents disappear again, and the user can submit a fresh
request.
Expiry is enforced on the server, not just in the interface: an expired approval
stops the folder listing *and* blocks the download route. Access granted on a
folder still cascades to everything inside it for as long as it is valid.
---
## Access request notifications
The moment a user requests access, **every administrator** is emailed that a
request is waiting for approval. The message states who asked and for what, the
request reference, how many requests are currently pending, and carries a
**Review pending requests** button linking straight to Data Management.
Set the portal address under **Data Management → Email Server → Portal address**
so that button points at your real hostname. Without it the link falls back to
the address the request came in on.
A few deliberate behaviours:
- If no mail server is configured the notification is recorded as **queued** in
the Email Log, and Data Management shows a warning banner saying
administrators are not receiving these emails.
- If delivery fails the failure is recorded in the Email Log and the audit log,
but **the access request itself is still created** — mail problems never lose
a request.
- The wording lives in the `request_to_admin` template and is fully editable.
Upgrades never overwrite a template you have customised; use **Restore
default** in the template editor if you want to adopt the version shipped
with a new release.
---
## Storage page (administrators)
**Storage** in the Administration section of the menu shows where space is going.
**Server disk** — total, used, and free space on the filesystem holding the
uploads directory, with a bar that turns amber past 75% and red past 90%, and a
warning when the disk is nearly full. Free space is reported the way `df` does
(excluding blocks reserved for root), so the numbers match what you see on the
host.
**Used by Infosec** — how much of that space this application accounts for, split
into uploaded files, brand images, and the database (including its write-ahead
log), each with the path it lives at.
**Space used per folder** — every folder with two figures: **total** (the folder
and everything nested inside it) and **own** (only the files sitting directly in
it), plus a file count and a relative bar. Sort by size or by name. Files stored
outside any folder are listed separately.
**Consistency** — cross-checks the database against the disk and reports:
- files recorded in the database but missing from disk (they appear in listings
but cannot be downloaded);
- files on disk that no record points at, with a **Reclaim space** button to
delete them. That action is audited as `STORAGE_CLEANUP`.
In Docker the figures describe the filesystem backing the mounted volume, which
is what determines whether uploads will succeed.
---
## Mail server & email log
### Configuring the mail server
**Data Management → Email Server** configures SMTP from inside the application —
host, port, username, password, from-address, implicit TLS, and certificate
verification. Two buttons help you check it before relying on it:
- **Test connection** opens a connection and authenticates without saving.
- **Send test message** sends a real message using the saved settings.
Settings saved here take precedence over the `SMTP_*` environment variables. If
you have an existing `.env`-based setup, it keeps working untouched — the page
simply shows the values as coming from the environment until you override them.
The stored password is never sent back to the browser; the field shows a mask,
and saving the form with the mask left alone keeps the existing password. Note
that the password is stored in the database in plain text, so treat
`data/datahub.db` as a secret and keep file permissions tight.
### Email log
**Data Management → Email Log** records every message the system sends: access
requests, approvals and denials, log exports, and test messages. Each entry
records time, direction, status (`delivered`, `queued`, or `failed`), sender,
recipient, subject, a body preview, attachments, the triggering context, and
the failure reason when delivery fails. Messages produced while no mail server
is configured are recorded as **queued** rather than lost.
The log can be filtered by text, direction, and status, then **exported as CSV**
or **emailed** (the filtered set is attached as a CSV) to a specific address or
to all administrators.
**On received mail:** Infosec sends mail but has no mailbox of its own, so there
is nothing to poll for incoming messages. The log stores direction as a first
class field and accepts inbound entries via `POST /api/mail/log/inbound`, so a
forwarder or mail-hook can record replies alongside outgoing mail. Automatic
collection would need IMAP credentials and a polling job, which is not part of
this release.
---
## Managing users
The wizard creates only the first administrator. Everyone else is added from
**Data Management → Users**:
- **New user** — username, **email (required)**, role, and an initial password
(minimum 10 characters with a letter and a number).
- **Edit** — change email or role, or set a new password.
- **Reset MFA** — turns two-factor authentication off for someone who has lost
their authenticator app, so they can set it up again from *My Account*.
- **Delete** — removes the account. Data they published and their entries in
the audit log are kept; only their pending access requests are removed.
Two safeguards are enforced by the server: you cannot delete the account you
are signed in with, and the last remaining administrator can be neither deleted
nor demoted.
**Email is mandatory on every account.** It is how approval notifications
reach people, so an account without one cannot do its job. The rule is applied
in three places: the browser form, the API (creating *or* updating an account),
and — for databases created by this release onwards — a database constraint.
An existing account can have its address changed but never cleared.
If a database created by an earlier release contains an account with no
address, the upgrade does **not** lock that user out. Instead the account is
listed with a *missing — required* badge, a banner appears above the user list,
and a warning is written to the server log at startup, so an administrator can
add the address.
Every one of these actions is written to the audit log (`USER_CREATED`,
`USER_UPDATED`, `USER_DELETED`, `USER_MFA_RESET`).
---
## Email (SMTP)
Email is optional for evaluation. If you don't configure SMTP, every outgoing
message is written to the **Logs** as a `MAIL_OUTBOX` entry so nothing is lost
and you can see exactly what *would* have been sent.
To send real email, set these in `.env`:
```
SMTP_HOST=smtp.yourprovider.com
SMTP_PORT=587
SMTP_SECURE=false
SMTP_USER=your-username
SMTP_PASS=your-password
MAIL_FROM=Infosec <no-reply@yourdomain.com>
```
Admin notification recipients are simply **every user whose role is `admin`**.
---
## How the approval flow works
1. A user browses **View Data** and clicks **Request access** on a file or folder.
2. An email (using the configurable *"Request received"* template) goes to all admins.
3. An admin opens **Data Management → Access Requests** and clicks **Approve** or **Deny**.
4. The requester receives an email (the *"approved"* or *"declined"* template).
5. Approved users can now **Download** the item from View Data. Admins can always download.
---
## Configurable emails & images
In **Data Management**:
- **Email Templates** — edit the subject and HTML body of each notification.
Insert placeholders like `{{username}}`, `{{target_name}}`, `{{email}}`,
`{{target_type}}`, `{{created_at}}` (click a chip to insert). Use **Preview**
to see the rendered result with sample data. The footer is added automatically.
- **Brand Images** — upload logos/images. Each uploaded image shows up as an
insertable chip in the template editor, dropping a ready-to-use `<img>` tag
into the body.
---
## Logs
Every meaningful action is recorded: logins, MFA changes, folder/file
create·rename·move·delete, uploads, access requests, approvals/denials,
downloads, template edits, image uploads, and email dispatch.
From the **Logs** page you can filter by text/action/page, **Export CSV**, or
**Email logs** (sends the filtered set as a CSV attachment to an address or to
all admins).
---
## Project layout
```
datahub/
├── UPGRADE.md # in-place upgrade guide for existing installs
├── Dockerfile # container image (multi-stage, non-root, healthcheck)
├── docker-compose.yml # one-command run with persistent volumes
├── docker-entrypoint.sh # ensures dirs, runs DB init, starts the server
├── .dockerignore
├── server.js # app entry, session + route wiring, page gating
├── db.js # SQLite schema & connection
├── init-db.js # create first admin / seed demo users
├── lib/
│ ├── audit.js # log() helper
│ ├── mailer.js # nodemailer transport, mail logging, no-SMTP fallback
│ ├── settings.js # key/value settings helper
│ ├── dates.js # recorded date + derived legal validity (+30 days)
│ ├── validate.js # shared account field rules (email is mandatory)
│ └── templates.js # email template store, render, defaults
├── middleware/auth.js # requireAuth / requireAdmin
├── routes/
│ ├── setup.js # first-boot wizard API (creates the first admin)
│ ├── users.js # admin user management (create/edit/reset/delete)
│ ├── versions.js # Version Control entries (admin only)
│ ├── legislation.js # Legislation entries (read: all, write: admin)
│ ├── mail.js # SMTP configuration + mail log (export / email)
│ ├── storage.js # disk usage + per-folder space (admin)
│ ├── auth.js # login, logout, session, MFA
│ ├── manage.js # admin folder/file CRUD + upload (Data Management)
│ ├── datacreate.js # Data Creation: build Document/Timeline HTML files (admin)
│ ├── data.js # View Data tree, requests, approvals, downloads
│ ├── settings.js # email templates + brand images
│ └── logs.js # list, CSV export, email export
├── public/ # setup, login, view-data, version-control, legislation,
│ # data-management, storage, account, logs + css/js
│ └── assets/ # infosec-logo.png (sidebar logo on signed-in pages)
├── avatars/ # per-user profile pictures (Docker volume)
├── data/ # SQLite database (created at runtime)
├── uploads/ # stored files (created at runtime)
└── brand/ # uploaded brand images (created at runtime)
```
---
## Security notes for production
- Serve behind HTTPS and set `COOKIE_SECURE=true` in `.env`.
- Set a strong, unique `SESSION_SECRET`.
- Passwords are hashed with bcrypt; MFA uses TOTP (RFC 6238).
- Consider putting the app behind a reverse proxy (nginx/Caddy) and adding
rate-limiting and backups of the `data/`, `uploads/`, and `brand/` folders.
- Uploaded files are served only through the authenticated, approval-checked
download route — never from a public static path.
---
## License
MIT.