Devices and acceleration

OCI manager Apps~9 min

GPU, NVIDIA, Coral, USB, FUSE and block devices become validated native Proxmox VE resources of the container.

The device the application needs, not the whole host

A device requested by the Compose file or by the application profile becomes a concrete devN entry or LXC mount. Asking for a GPU, a USB device or a Coral does not make the container privileged.

Host inventory

/dev/dri/renderD128 GID 993 · Intel

ProxMenux

vendor and permissions checked

LXC

same device effective GID

Where the device request comes from

Source	What is read	What the installer does
Docker Compose	`devices`, `group_add`, `deploy.resources` and NVIDIA requests	Each requirement becomes a device request shown for review
Catalog profile	The GPU, Coral, OpenCL, USB or FUSE support the application actually has	Only the options validated for that image are offered
Image metadata and documentation	VA-API, Selkies, LinuxServer mods or the NVIDIA runtime	The documented variables and preparation are added
User selection	CPU only, Intel/AMD, OpenCL, NVIDIA or an optional device	The selection is stored in the instance contract

Detected devices are not attached on their own

The host is inventoried, but only devices declared by the Compose file or by a compatible profile are offered and attached. A GPU, USB dongle or Coral present on the host is not exposed to every LXC.

How the host device is identified

Before the LXC is modified, the device is read on the host and matched against the chosen profile.

Type	Identity	Validation
Intel/AMD DRM	`/dev/dri/renderD*` and `/sys/class/drm/NODE/device/vendor`	A character device with vendor `0x8086` (Intel) or `0x1002` (AMD)
AMD OpenCL	The render node, plus `/dev/kfd` when the profile needs it	Existence, type, vendor, permissions and declared compatibility
AMD ROCm	The render node, `/dev/kfd` and the generation the compute driver reports in `/sys/class/kfd/kfd/topology/nodes/*/properties`	A GPU of a generation the application's ROCm image supports (the RDNA2 GPUs it names, and newer ones) is offered as it is. Other GPUs of those families, such as the Radeon 680M and 780M, are offered as experimental and are never the proposed option. ROCm does not support them officially: the application presents them as the generation of their family (`HSA_OVERRIDE_GFX_VERSION`), and the installer asks for confirmation before downloading the larger image. An older GPU, such as the integrated graphics of a Ryzen 5000U, is not offered, and Immich keeps recognition on the CPU
NVIDIA	`nvidia-smi` and `nvidia-container-cli`	GPU, UUID, PCI bus, driver version, Toolkit, `/dev/nvidia*` nodes, binaries and libraries
Coral PCIe/M.2	`/dev/apex_N` and its link in `/sys/dev/char/MAJOR:MINOR`	Character node, major/minor, owner, GID and permissions
USB and serial	`/dev/ttyUSB`, `/dev/ttyACM` or `/dev/bus/usb/BBB/DDD`	Character node; for USB also vendor, product and serial when sysfs publishes them
KVM, TUN, FUSE, video and generic SCSI	`/dev/kvm`, `/dev/net/tun`, `/dev/fuse`, `/dev/videoN` or `/dev/sgN`	Supported path, node type and effective permissions
Optical drive	`/dev/srN`	A block device

What happens during the installation

1
The template offers its profiles
For example CPU only, Intel/AMD VA-API, AMD OpenCL, Intel OpenCL or NVIDIA. The options belong to the image, not to a common menu.
2
A profile is chosen
It defines the device nodes, environment, mods or runtime the application needs.
3
A path is proposed
For DRM, /dev/dri/renderD128, which can be changed on a host with several render nodes. For USB or serial, the concrete node is selected.
4
Validation
Existence, type, allowed vendor, permissions and GID are checked. A mismatch stops the operation.
5
The contract is written
Path, mode, GID, write access and profile are recorded for updates and recreations.
6
Attach and test
pct set adds the devN entry and access is then checked inside the LXC. LinuxServer images are also checked as user abc.

How it appears in the LXC configuration

Illustrative values: dev0 and dev1 are the free slots Proxmox VE assigns, and renderD128, apex_0 and the GID depend on the hardware of the node.

Intel/AMD VA-API

dev0: path=/dev/dri/renderD128,mode=0660,gid=993,deny-write=0

Coral PCIe/M.2

dev0: path=/dev/apex_0,mode=0660,gid=GID,deny-write=0

A specific USB device

dev0: path=/dev/bus/usb/003/004,mode=0660,gid=GID,deny-write=0

AMD OpenCL

dev0: path=/dev/dri/renderD128,mode=0660,gid=GID,deny-write=0
dev1: path=/dev/kfd,mode=0660,gid=GID,deny-write=0

The GID is read with stat on the host and written to the devN entry; the render and video groups are not assumed to have a fixed number. The device keeps the same /dev path inside the LXC, where the application's own mechanisms look for it.

Profiles an image can offer

Profile	Translation	Offered when
Intel/AMD VA-API	`/dev/dri` render node	The application supports video acceleration
OpenCL	Render node, `/dev/kfd` when needed and the official mod	The image or profile documents it
NVIDIA	Driver devices and libraries of the host	The host has a working driver and the NVIDIA Container Toolkit
Coral	`/dev/apex_0` or the USB bus	The profile declares Coral support (Frigate)
USB, serial, FUSE	A single device, a validated tree or an LXC feature	The contract asks for it

NVIDIA

Node requirement: NVIDIA Container Toolkit

A working driver on Proxmox VE is not enough to give an NVIDIA GPU to an OCI image. OCI manager Apps uses nvidia-container-cli, from the NVIDIA Container Toolkit, to identify the devices and to obtain the binaries and libraries that match the loaded driver.

The ProxMenux NVIDIA installer installs the NVIDIA Container Toolkit from the official NVIDIA repository together with the driver. It checks its four packages, validates nvidia-container-cli and records the result in the change journal of Audit & Report.

These two commands on the host show whether the driver and the Toolkit are available:

nvidia-smi -L
nvidia-container-cli --version

On a host where the driver was installed by other means, the Toolkit is installed from the official stable repository:

apt-get update
apt-get install -y --no-install-recommends ca-certificates curl gnupg2

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
  | gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg

curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
  | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \
  > /etc/apt/sources.list.d/nvidia-container-toolkit.list

apt-get update
apt-get install -y nvidia-container-toolkit libnvidia-container-tools

The inventory OCI manager Apps uses is the output of:

nvidia-container-cli list --device all --libraries --binaries --firmwares --ipcs

Docker runtime configuration is not involved

The containers are native LXCs and no Docker daemon is used, so nvidia-ctk runtime configure --runtime=docker plays no part: ProxMenux queries nvidia-container-cli directly and writes the LXC devices and mounts. The commands and supported platforms are maintained in the NVIDIA Container Toolkit installation guide.

Inventory from the driver

nvidia-container-cli lists the device nodes, binaries, firmware and libraries of the installed driver.

No fixed version

The template does not name library files. The profile is generated from the current host.

Read-only mounts

The host libraries are mounted read-only instead of being copied into the container.

Driver changes

After a driver change the inventory is generated again before the affected LXCs start.

NVIDIA result (simplified)

devN: path=/dev/nvidia0,...
devN: path=/dev/nvidiactl,...
devN: path=/dev/nvidia-uvm,...
lxc.mount.entry: HOST_LIBRARY CONTAINER_LIBRARY none ro,bind,create=file 0 0

Passing only /dev/nvidia0 is not enough. The user-space components of the loaded driver are mounted read-only, and nvidia-smi then runs inside the LXC to compare GPU, UUID, PCI bus and version with the host inventory.

USB, serial and USB Coral

USB numbering can change

A path such as /dev/bus/usb/003/004 can change when the device is reconnected or the host restarts. The profile records vendor, product and serial when they are available, but a new bus address is not remapped automatically.

A peripheral is given by its concrete node: /dev/ttyUSB0, /dev/ttyACM0, /dev/apex_0 or /dev/bus/usb/BBB/DDD. Passing the whole of /dev is not accepted. Coral is offered only to applications whose profile declares it.

Device trees and LXC features

Request	Translation	Scope
`/dev/dvb`, `/dev/snd` or `/dev/bus/usb`	Each character node of the tree gets its own `devN` entry with the host mode and GID	Only the requested tree, not the rest of `/dev`
`/dev/fuse`	The node and, when the profile needs it, the `fuse=1` feature	FUSE alone does not publish mounts to other LXCs
`/dev/net/tun`	A `devN` entry at the same path inside the LXC	The VPN or network configuration stays in the application
`/dev/kvm`	A validated `devN` entry	Offered only when the contract asks for it

Confirmations by level of risk

Concrete devices, optional privilege, required privilege, AppArmor or seccomp relaxation and access to the host PID namespace are treated as separate cases, not under one generic privileged label. Each option with a risk is explained and confirmed during the installation.

OCI manager Apps: Previous

Data, paths and networking

OCI manager Apps: Next

Multi-container applications