Architecture

Request flow

From the browser to the kernel, every dashboard view follows the same path:

The same process also runs four background threads started at boot — they don't serve HTTP, they push state into SQLite or into the notification queue while the host is up:

systemd unit

The installer drops a unit at /etc/systemd/system/proxmenux-monitor.service. Default content:

[Unit]
Description=ProxMenux Monitor - Web Dashboard
After=network.target

[Service]
Type=simple
User=root
WorkingDirectory=/opt/proxmenux-monitor
ExecStart=/opt/proxmenux-monitor/ProxMenux-Monitor.AppImage
Restart=on-failure
RestartSec=10
Environment="PORT=8008"

[Install]
WantedBy=multi-user.target

• User=root — required: SMART, pvesh, journal scopes, ZFS commands and the web terminal all need root.
• Restart=on-failure with a 10-second back-off — non-zero exits relaunch automatically.
• After=network.target — waits for the host network stack to be online.

systemctl cat proxmenux-monitor.service       # show the unit content
systemctl status proxmenux-monitor.service    # state + recent log
journalctl -u proxmenux-monitor.service -f    # follow live

What the AppImage contains

The AppImage is a self-mounting filesystem. AppRun at the root sets up the environment and execs flask_server.py:

AppDir/
├── AppRun                              # entrypoint: sets PATH/LD_LIBRARY_PATH, exec flask_server
├── usr/
│   ├── bin/
│   │   ├── flask_server.py             # main process
│   │   ├── flask_*_routes.py           # Flask blueprints (auth, health, terminal, …)
│   │   ├── auth_manager.py             # JWT + TOTP + API tokens
│   │   ├── health_monitor.py           # 10-category checker
│   │   ├── health_persistence.py       # SQLite layer
│   │   ├── notification_manager.py     # orchestrator
│   │   ├── notification_channels.py    # Telegram, Discord, Email, …
│   │   ├── notification_templates.py   # message rendering + AI hook
│   │   ├── notification_events.py      # JournalWatcher, TaskWatcher, …
│   │   ├── ai_providers/               # OpenAI · Anthropic · Gemini · Groq · Ollama · OpenRouter
│   │   ├── proxmox_storage_monitor.py  # storage pool inspection
│   │   ├── hardware_monitor.py         # CPU/PCIe/GPU enumeration
│   │   ├── ipmitool, sensors, upsc     # vendored hardware tools
│   │   └── …
│   ├── lib/python3/                    # vendored Python deps (Flask, JWT, psutil, …)
│   └── share/                          # icons + .desktop file
└── web/                                # Next.js static export
    ├── index.html
    ├── _next/                          # JS / CSS chunks
    └── manifest.json                   # PWA manifest

Two consequences of this layout:

• No host Python pollution. The vendored interpreter and packages are isolated inside the AppImage — upgrading the host's system Python doesn't affect the Monitor and vice-versa.
• Hardware tools are bundled too. ipmitool, lm-sensors and upsc ship inside the AppImage so the dashboard can read out-of-band sensors and UPS state without forcing the user to install Debian packages.

Flask app structure

flask_server.py creates a single Flask(__name__) instance, enables CORS, and registers six blueprints plus a WebSocket initializer:

The full endpoint list with request / response shapes is in API Reference.

Data sources

Nothing is collected from a custom agent — the Monitor reads the same files and runs the same commands a human admin would:

Persistence

Two filesystem locations split state by sensitivity:

Health Monitor cycle

Every 5 minutes health_monitor.py runs a deterministic cycle across the ten categories shown on the dashboard:

1. Critical PVE services (pveproxy, pvedaemon, pvestatd, pve-cluster).
2. Proxmox storage pools (pvesh get /storage + per-storage availability).
3. Disks and filesystems: SMART, dmesg I/O errors, ZFS pool health, mountpoint capacity.
4. VMs and CTs: failed starts, crashed guests, QMP errors, shutdown failures.
5. Network: bridge / bond status, link state, latency to the gateway.
6. Updates: pending package upgrades and security patches.
7. Logs: persistent / spike / cascade pattern detection in the system journal.
8. Memory: OOM killer activity, sustained high pressure.
9. Temperature: CPU / chassis sensors against vendor thresholds.
10. Security: authentication failures, ban events, fail2ban jail status.

Each finding is normalised into a stable error_key + category + severity. The persistence layer deduplicates against the existing errors table — repeated events update last_seen and the occurrence counter without spamming notifications.

The cycle also auto-resolves stale errors using the per-category Suppression Duration setting, cleans up errors for resources that no longer exist (deleted VMs / removed disks / unmounted storages), and prunes the events log older than 30 days. The full catalogue of categories and the dashboard view that surfaces them is documented in Dashboard → Health Monitor.

Notification engine

notification_manager.py is the orchestrator. It loads the configured channels, owns the delivery queue, and exposes both a Python API (for Flask routes and the Health Monitor cycle) and a CLI entrypoint (for the .sh hook scripts shipped with ProxMenux).

• Watchers push events: JournalWatcher tails the system journal, TaskWatcher polls the Proxmox task list, ProxmoxHookWatcher reacts to backup / replication / snapshot hooks, and PollingCollector handles slow data sources.
• Templates turn an event into a (title, body) pair. The same template can run through the configured AI provider (OpenAI / Anthropic / Gemini / Groq / Ollama / OpenRouter) to produce a plain-language rewrite; both versions are stored in notification_history.
• Channels deliver messages: Telegram, Discord, Email, Gotify and Apprise (multi-channel). Each is implemented in notification_channels.py behind the same create_channel() / send() interface, so adding a new channel is a single class.
• Encryption. Sensitive settings (telegram.token, discord.webhook_url, ai_api_key_*, email.password) are XOR-encrypted with the key in .notification_key before being written to the DB. Plaintext never touches disk.

Per-event toggles, channel overrides and AI configuration are surfaced in Settings → Notifications and Settings → AI Assistant.

WebSocket terminal

The Terminal tab in the dashboard is a thin xterm.js client wired to a server-side PTY through a WebSocket. Two transport modes:

• HTTP mode (default): Flask's development server with flask-sock handles upgrade requests. Good enough for LAN / direct access.
• HTTPS / WSS mode: when an SSL certificate is configured, the process switches to gevent.pywsgi.WSGIServer with geventwebsocket.handler.WebSocketHandler, so WebSockets work over TLS without polyfills.

The PTY is a child of the Flask process, so it inherits User=root from the unit. Every terminal request goes through JWT auth; the user must already be logged in to the dashboard before a PTY is allocated.

If you access the Monitor through a reverse proxy, make sure WebSocket forwarding is enabled (the Upgrade and Connection headers). Without it the terminal won't work.

Reverse proxy & Fail2Ban

Two safeguards make sure security works the same way whether the dashboard is hit directly or through a reverse proxy:

1. Real client IP recovery. A before_request hook reads X-Forwarded-For and X-Real-IP in that order, falling back to request.remote_addr. The recovered address is what auth logging and rate limiting see. This is always on.
2. Application-level Fail2Ban check (optional). When the dashboard sits behind a proxy, the kernel firewall can't block the real attacker IP — the connection always comes from the proxy. To plug that gap, the same hook above queries the proxmenux Fail2Ban jail every 30 seconds, caches the banned IP set, and short-circuits requests from those IPs with HTTP 403 inside Flask.

Reverse-proxy snippets (Nginx / Caddy / Traefik) and the Fail2Ban jail walkthrough are in Access & Authentication and Security → Fail2Ban.

Where to next

• Access & Authentication — first-launch setup, password + TOTP 2FA, reverse-proxy snippets, Fail2Ban jail.
• API Reference — every endpoint, token management, security best-practices.
• Settings → ProxMenux Monitor — the in-menu service toggle and status verification flow inside the ProxMenux TUI.