Hardware Discovery via SSH¶

The Proxmox REST API does not expose chassis hardware (serial number, manufacturer, product name) or per-NIC link details (negotiated speed, duplex, link state). Operators that need those fields populated in NetBox historically had to hand-edit each dcim.Device and dcim.Interface.

In the netbox-proxbox 0.0.15 line, the plugin added per-node SSH credentials, a token-gated credential endpoint, and an opt-in SSH-driven discovery pass. Current releases store the reflected results in typed sync-state sidecars. The released proxbox-api 0.0.11 backend remains compatible and simply does not run this discovery pass; discovery activates with a backend build that includes proxbox-api PR #80. That backend runs dmidecode and ethtool on each Proxmox node and reflects the parsed values onto the matching NetBox records.

This page is the operator-facing setup guide.

When to enable it¶

You manage more than a handful of Proxmox hosts and want NetBox's Device.serial, Device.manufacturer, and chassis sync-state fields populated automatically.
You want per-NIC negotiated speed / duplex / link state visible in the NetBox interface detail view.
You can dedicate a least-privilege SSH user on each node (recommended: proxbox-discovery — see Node-side setup).

When NOT to enable it¶

You cannot pin host keys (the discovery driver refuses TOFU; every node's SHA-256 fingerprint must be stored before discovery runs).
You cannot dedicate an SSH user — running discovery as root is technically possible but not supported by the documentation.
Your management network does not reach the node SSH port.

High-level architecture¶

proxbox-api                NetBox plugin                Proxmox node
─────────────              ─────────────                ─────────────
1. Fetch credentials ────►  /api/plugins/proxbox/
                            ssh-credentials/by-node/
                            <node_id>/credentials/
                            (NetBox API token)

2. Open SSH session ─────────────────────────────────►  sshd (port 22)
   (pinned fingerprint,                                  ↓
   sudo -n)                                              ↓
                                                         dmidecode -t 1
                                                         dmidecode -t 3
                                                         ip -o link show
                                                         ethtool <iface>

3. Parse outputs ◄───────────────────────────────────── stdout

4. Reflect to NetBox ────►  ProxboxDeviceSyncState /
                            ProxboxInterfaceSyncState (drift-detect)

All SSH primitives — host-key pinning, sudo handling, output capping, command allow-listing — live in proxmox-sdk.ssh.RemoteSSHClient. proxbox-api is a thin consumer; it does not import paramiko.

One-time setup¶

1. Select the credential storage backend¶

Node SSH credentials follow the effective storage selection of their ProxmoxEndpoint: an explicit endpoint selection overrides the plugin setting, and an explicit plugin setting overrides Automatic. Automatic selects OpenBao only when netbox_openbao is enabled; otherwise it selects legacy Fernet.

For OpenBao storage, configure the default SecretEngine, the exact openbao_policy_slug, and an OpenBao service user for automated reveals. The node credential form stores password or keypair material through netbox-openbao's audited transaction and keeps only opaque UUID references in NodeSSHCredential. A Fernet key is not required for these node secrets.

For Legacy encrypted (Fernet) storage:

Open Plugins → Proxbox → Settings in the NetBox UI.
Tick Enable encryption and paste a 32-byte Fernet key (or a raw 32-byte secret — the plugin will base64-encode it).
Save.

This is the plugin-at-rest key for ciphertext in NetBox. It is separate from proxbox-api's own database-encryption key and from the FastAPI endpoint API key. Once any plugin ciphertext exists, ordinary settings saves cannot clear or replace it; use the verified rotation workflow on the same page. An explicit OpenBao selection never falls back to these Fernet columns when its reference, provider, policy, or access is unavailable.

2. Enable the feature flag¶

In Plugins → Proxbox → Settings, tick Enable SSH-based hardware discovery and save. The flag is off by default; until you flip it, no SSH sockets are opened during a sync.

Physical-NIC MAC reflection has a second, independent checkbox on the same card: Sync physical NIC MAC addresses. It is also off by default. Enable it only when you want discovery to create native dcim.MACAddress rows and assign them as each physical interface's primary_mac_address. Both checkboxes must be on; enabling the MAC checkbox alone never opens SSH or writes MAC data.

3. Node-side setup¶

For each Proxmox node you want to discover, create a dedicated discovery user. Example for Debian / Proxmox VE 8:

# 1. Create the user (no shell, no home password)
adduser --disabled-password --gecos "" proxbox-discovery

# 2. Drop your proxbox-discovery ed25519 public key into authorized_keys.
#    Use the locked-down command= prefix so only the discovery script runs:
mkdir -p /home/proxbox-discovery/.ssh
cat > /home/proxbox-discovery/.ssh/authorized_keys <<'EOF'
command="/usr/local/bin/proxbox-discover",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty ssh-ed25519 AAAA... proxbox-discovery
EOF
chown -R proxbox-discovery:proxbox-discovery /home/proxbox-discovery/.ssh
chmod 700 /home/proxbox-discovery/.ssh
chmod 600 /home/proxbox-discovery/.ssh/authorized_keys

# 3. Install the discovery dispatch script
cat > /usr/local/bin/proxbox-discover <<'EOF'
#!/bin/sh
case "$SSH_ORIGINAL_COMMAND" in
  "dmidecode -t 1"|"dmidecode -t 3")
    sudo -n /usr/sbin/$SSH_ORIGINAL_COMMAND ;;
  "ip -o link show")
    /usr/sbin/$SSH_ORIGINAL_COMMAND ;;
  ethtool\ *)
    /usr/sbin/$SSH_ORIGINAL_COMMAND ;;
  *)
    exit 1 ;;
esac
EOF
chmod 755 /usr/local/bin/proxbox-discover

# 4. Grant only the two dmidecode calls under sudo
cat > /etc/sudoers.d/proxbox-discovery <<'EOF'
proxbox-discovery ALL=(root) NOPASSWD: /usr/sbin/dmidecode -t 1, /usr/sbin/dmidecode -t 3
EOF
chmod 440 /etc/sudoers.d/proxbox-discovery
visudo -cf /etc/sudoers.d/proxbox-discovery

Only dmidecode needs sudo; ip and ethtool work without it on Proxmox.

4. Pin the host-key fingerprint¶

The discovery driver refuses to connect unless the node's host key matches the stored SHA-256 fingerprint exactly — no TOFU.

# Get the canonical fingerprint of the node's ed25519 host key
ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub -E sha256
# Output: 256 SHA256:abc123… root@node1 (ED25519)

Copy the SHA256:<base64> segment.

5. Add the credential in NetBox¶

Browse to Plugins → Proxbox → SSH Credentials → Add.
Pick the Proxmox node (one credential per node).
Fill in:
- Username: proxbox-discovery
- Port: 22 (or your custom port)
- Auth method: SSH private key (recommended)
- Private key: paste the ed25519 PEM whose public counterpart you placed in the node's authorized_keys.
- Pinned host-key fingerprint: paste SHA256:abc123…
- Sudo required: ✅ (the dispatch script uses sudo for dmidecode)
Save. With OpenBao selected, the form writes an ssh-keypair or ssh-password credential through netbox-openbao and persists its opaque UUID reference. With Legacy encrypted selected, it encrypts the selected material with the configured Fernet key before persisting.

When the Proxmox node is linked to a dcim.Device, the selected authentication credential is also assigned to that device as the primary credential for the login purpose. Material may be saved before the node has a device link; in that state the OpenBao credential and UUID reference exist, but assignment is deferred. Creating, changing, or removing ProxmoxNode.netbox_device automatically creates, moves, or removes the owned primary login assignment. Unrelated assignments and provider material are preserved.

Automation can inspect the credential metadata endpoint before attempting a connection. For OpenBao-backed nodes it returns openbao_assignment_ready, a sanitized openbao_assignment_detail, and an openbao_assignment_lookup containing the generic assignment selectors assigned_object_type=dcim.device, the linked device ID, and purpose=login. These selectors are compatible with netbox-openbao assignment filters and disclose neither credential UUIDs nor material. The authenticated hardware credential endpoint remains the only Proxbox API path that returns the selected password or private key to an authorized consumer.

Do not use ORM bulk writes or parent-object cascades to unlink this graph. Proxbox refuses raw update, bulk-create/update, raw-delete, and deletion of the linked Device, node, or endpoint while OpenBao references or assignments remain. Clean up the node credential explicitly first. Storage cannot be changed from OpenBao to legacy until that cleanup is complete; if the provider package was removed too early, restore it so the owned assignment can be removed safely.

This local NodeSSHCredential plus public netbox-openbao assignment boundary supersedes every unrelated credential-provider fallback. Hardware discovery and terminal access do not query another private plugin for node SSH material.

Browser terminal endpoint SSH¶

The Proxmox endpoint detail page also has a browser SSH Terminal tab. Node targets use the per-node NodeSSHCredential rows described above. The endpoint target uses Proxbox-native fields on ProxmoxEndpoint; it does not use an unrelated companion plugin's SSH binding.

Before upgrading an installation that previously sourced node credentials from another plugin, run python manage.py audit_node_ssh_credentials --fail-on-missing. Create a local NodeSSHCredential for every enabled API + SSH node listed as blocking, rerun the command, and continue only after it reports blocking_missing_local_credentials=0. Disabled and API-only nodes are listed separately as informational; they do not block the upgrade because the plugin must not open SSH sessions to them.

On the Proxmox endpoint add/edit form, operators can configure SSH credential source:

Dedicated SSH credential keeps the existing encrypted endpoint ssh_* fields. Store an SSH username, port, auth method, pinned host-key fingerprint, and either a password or private key. Saving a secret requires ProxboxPluginSettings.encryption_key.
Reuse endpoint username/password uses the endpoint's own Proxmox username/password for SSH. The plugin strips the Proxmox realm before sending the credential to proxbox-api (root@pam becomes root) and always sends auth_method=password. Token-only endpoints cannot use this mode because no endpoint password exists to reuse. A pinned SSH host-key fingerprint is still required.

The endpoint API consumed by proxbox-api (/api/plugins/proxbox/ssh-credentials/by-endpoint/<id>/credentials/) keeps the same response keys in both modes: host, username, port, auth method, host-key fingerprint, password/private-key presence flags, password, and private_key.

Reuse mode exposes the Proxmox API password

In Reuse endpoint username/password mode the secrets endpoint returns the endpoint's own Proxmox API password (reused as the SSH password). That endpoint is gated by a NetBox API token holding view and open_ssh_terminal on the Proxmox endpoint, and requires HTTPS in non-DEBUG — but because such a token can therefore retrieve the Proxmox API password, scope open_ssh_terminal to the operators already trusted with that endpoint's credentials. The dedicated ssh_password / ssh_private_key fields are ignored while reuse is selected.

Operations¶

What the discovery pass writes¶

NetBox object	Typed sidecar field	Source
`dcim.Device`	`ProxboxDeviceSyncState.hardware_chassis_serial`	`dmidecode -t 3 → Serial Number`
`dcim.Device`	`ProxboxDeviceSyncState.hardware_chassis_manufacturer`	`dmidecode -t 1 → Manufacturer`
`dcim.Device`	`ProxboxDeviceSyncState.hardware_chassis_product`	`dmidecode -t 1 → Product Name`
`dcim.Interface`	`ProxboxInterfaceSyncState.nic_speed_gbps`	`ethtool <iface> → Speed: 10000Mb/s` → `10`
`dcim.Interface`	`ProxboxInterfaceSyncState.nic_duplex`	`ethtool <iface> → Duplex: Full`
`dcim.Interface`	`ProxboxInterfaceSyncState.nic_link`	`ethtool <iface> → Link detected: yes`

Migration 0049 originally registered these names as hidden custom fields. Migrations 0086 and 0087 remove those legacy definitions and values after the typed writer cutover. 0087 covers these six specifically: 0086's ownership check compares labels, and these had been reconciled with different ones, so it failed closed and skipped them. 0087 selects candidates by data type and ui_editable="hidden", then leaves alone any of the six that still holds a value on any device or interface, whoever wrote it. The discovery pass now writes only the device and interface sync-state sidecars, which remain read-only reflection data.

Idempotency¶

A second consecutive successful sync emits zero ObjectChange rows for the six sidecar fields. Drift detection lives in the proxbox-api reflection helper introduced by issue #357.

Disabling the feature¶

Untick Enable SSH-based hardware discovery in Settings. The next sync opens zero SSH sockets — the discovery orchestrator returns early when the flag is False. Stored credentials remain encrypted at rest; nothing is deleted.

To keep chassis and link facts enabled while stopping only physical-NIC MAC writes, untick Sync physical NIC MAC addresses. Existing MAC rows are left untouched; subsequent discovery runs simply stop reconciling them.

Troubleshooting¶

`host_key_mismatch` warning frame¶

The node's host key changed since you pinned the fingerprint. Either:

The node was reinstalled — re-pin the fingerprint after verifying the new key out-of-band.
A MITM is intercepting the SSH session — stop, audit, and only re-pin once you have confirmed the new key matches what is on the node console.

`hardware_discovery_timeout` warning frame¶

The SSH connection or command run exceeded the timeout (connect 10 s, exec 30 s). The orchestrator runs nodes sequentially, so a stalled node only delays itself — the run continues.

`sudo: a password is required`¶

The discovery user is missing the sudoers entry, or the entry is wider than the two dmidecode commands listed above. Re-check /etc/sudoers.d/proxbox-discovery and confirm sudo -n dmidecode -t 1 succeeds when run as the discovery user on the node.

503 from the secrets endpoint¶

/api/plugins/proxbox/ssh-credentials/by-node/<id>/credentials/ returns 503 Service Unavailable when the selected credential material cannot be resolved. For OpenBao, verify the UUID reference, plugin configuration, exact policy, provider availability, and actor access; the resolver fails closed and never reads legacy Fernet ciphertext as a downgrade. For Legacy encrypted, verify ProxboxPluginSettings.encryption_key and that the row is decryptable with that key. Use verified rotation when the old key is available, or have a separately authorized operator destructively reset only the affected SSH family when it is not. Re-enter reset credentials before the next sync.

403 from the secrets endpoint¶

The NetBox API token used by proxbox-api was rejected, the request was not HTTPS while DEBUG=False, or the token's user lacks netbox_proxbox.view_nodesshcredential. Confirm proxbox-api is configured against this NetBox, sends Authorization: Token <key> (or NetBox v2's Bearer <key.secret> form), and uses a service account with the credential-view permission. This endpoint does not use FastAPIEndpoint.token.

Security boundary recap¶

Browser users with the standard view permission on the credential see only metadata (has_password, has_private_key, fingerprint).
Plaintext is only returned to the NetBox API-token-gated …/credentials/ endpoint, which rejects browser sessions and refuses non-HTTPS when DEBUG=False.
All SSH primitives are pinned to modern AEAD ciphers, ETM MACs, and curve25519 kex inside proxmox-sdk.ssh.RemoteSSHClient.
The discovery user runs a command=-locked script, has no shell, no agent forwarding, no PTY, and sudo access only to two dmidecode invocations.
hardware_discovery_enabled=False (default) results in zero SSH sockets opened — pinned by tests/test_hardware_discovery_flag_off.py in proxbox-api.
hardware_discovery_sync_nic_macs=False (default) prevents native physical NIC MAC writes even when the discovery pass itself is enabled. The backend requires both opt-ins and treats a missing field from an older plugin as False.