Streaming Protocol¶

Proxbox uses Server-Sent Events (SSE) as the primary transport for real-time sync progress. A secondary WebSocket channel carries broadcast messages for dashboard updates. This page documents both protocols in detail.

Two Sync Transport Modes¶

SSE Streaming (primary)JSON Polling (legacy)

The plugin opens a long-lived HTTP GET to proxbox-api/full-update/stream. The response is text/event-stream and carries SSE frames for every stage and every object processed. The plugin proxies this stream to the browser via a Django StreamingHttpResponse.

Used by: ProxboxSyncJob.run() via run_sync_stream(), browser-facing StreamingHttpResponse proxy views
Advantages: real-time per-object progress, no polling needed, works through proxies with X-Accel-Buffering: no
Timeout: HTTP between-chunk read timeout is 3600 s (_SYNC_STREAM_READ_TIMEOUT). RQ job wall-clock limit is 7200 s.

The plugin sends a GET to proxbox-api/full-update and waits for a single JSON response containing all sync results.

Used by: sync_full_update_resource(), some older dashboard views
Advantages: simple request/response, easy to debug
Disadvantages: no real-time progress, times out on large clusters

SSE Event Types¶

Every SSE event has the form:

event: <event_type>
data: <json_payload>

(Note the blank line between events — this is the SSE message boundary.)

`discovery`¶

Emitted once at the start of a full-update stream. Lists all stages that will be processed.

{
  "event": "discovery",
  "phase": "full-update",
  "status": "discovered",
  "message": "Discovered 12 sync stage(s) for full update",
  "count": 12,
  "items": [
    {"name": "devices", "type": "stage"},
    {"name": "storage", "type": "stage"},
    ...
  ],
  "progress": {"current": 0, "total": 12, "percent": 0},
  "metadata": {"operation_id": "550e8400-e29b-41d4-a716-446655440000"}
}

`step`¶

Emitted twice per stage: once when the stage starts, once when it finishes.

// Stage started
{"step": "virtual-machines", "status": "started", "message": "Starting virtual machines synchronization."}

// Stage completed
{"step": "virtual-machines", "status": "completed", "message": "Virtual machines synchronization finished.", "result": {"count": 47}}

`substep`¶

Emitted by individual sync services for sub-stage progress (e.g., per-VM progress within the VM stage).

{"substep": "vm_create", "status": "processing", "message": "Creating VM web-server-01", "vmid": 100}

`item_progress`¶

Carries per-object sync result for fine-grained frontend progress bars.

{"event": "item_progress", "name": "web-server-01", "status": "created", "type": "virtual_machine", "id": 42}

`phase_summary`¶

Emitted by some stages to summarize a batch of objects processed.

{"event": "phase_summary", "phase": "backups", "created": 12, "updated": 3, "skipped": 0, "errors": 0}

`error_detail`¶

Structured error event with category, suggestion, and detail text.

{
  "event": "error_detail",
  "phase": "virtual-machines",
  "category": "internal",
  "message": "Virtual machine sync failed",
  "detail": "Connection refused to Proxmox VE",
  "suggestion": "Check Proxmox endpoint connectivity and retry"
}

`error`¶

Short-form error event used alongside error_detail.

{"step": "full-update", "status": "failed", "error": "Error while syncing virtual machines.", "detail": "..."}

`complete`¶

Always the last event in a stream. ok: true on success, ok: false on failure.

// Success
{"ok": true, "message": "Full update sync completed.", "result": {"devices_count": 3, "virtual_machines_count": 47, ...}}

// Failure
{"ok": false, "message": "Error while syncing virtual machines.", "errors": [{"detail": "..."}]}

SSE Proxy Chain¶

The SSE stream passes through two hops before reaching the browser:

sequenceDiagram
    participant Browser
    participant NB as NetBox Plugin<br/>(Django view)
    participant RQ as RQ Worker<br/>(ProxboxSyncJob)
    participant API as proxbox-api<br/>(FastAPI)

    Browser->>NB: GET /proxbox/sync/stream/
    NB->>NB: Create StreamingHttpResponse
    NB->>API: GET /full-update/stream (via iter_backend_sse_lines)
    API-->>NB: SSE frames (chunked)
    NB-->>Browser: SSE frames (proxied)

    note over RQ,API: Background job path (separate)
    RQ->>API: GET /full-update/stream (via run_sync_stream)
    API-->>RQ: SSE frames consumed by on_frame callback
    RQ->>RQ: Write progress to Job.log

The iter_backend_sse_lines() function in netbox_proxbox/services/backend_proxy.py opens a streaming requests.get() with stream=True and yields each raw SSE line. The Django view wraps this in a StreamingHttpResponse with Content-Type: text/event-stream.

For background jobs, run_sync_stream() consumes the SSE stream to completion using an on_frame callback that writes progress data to the NetBox Job record.

WebSocketSSEBridge¶

proxbox-api uses an internal bridge class to decouple async sync work from SSE frame emission:

proxbox_api/utils/streaming.py (simplified)

class WebSocketSSEBridge:
    """Connects an async sync function to SSE frame output."""

    async def send(self, event: str, data: dict) -> None:
        """Called by sync service to emit a progress event."""
        await self._queue.put(sse_event(event, data))

    async def close(self) -> None:
        """Signal that no more events will be sent."""
        await self._queue.put(None)

    async def iter_sse(self) -> AsyncIterator[str]:
        """Yields SSE frames for the HTTP response generator."""
        while True:
            frame = await self._queue.get()
            if frame is None:
                break
            yield frame

Each stage gets its own bridge instance. The full_update.py event generator runs the sync function as a background asyncio.Task, iterates the bridge's SSE output, then awaits the task result:

async def _run_vms_sync():
    try:
        return await create_virtual_machines(..., websocket=vm_bridge, use_websocket=True)
    finally:
        await vm_bridge.close()          # always signal completion

vms_task = asyncio.create_task(_run_vms_sync())
async for frame in vm_bridge.iter_sse():
    yield frame                          # proxy to HTTP response
sync_vms = await vms_task                # collect result

Stream Lifecycle State Machine¶

stateDiagram-v2
    [*] --> Connected: GET /full-update/stream

    Connected --> Discovering: event:discovery

    Discovering --> Stage_1: event:step devices started
    Stage_1 --> Stage_1: event:substep / item_progress
    Stage_1 --> Stage_2: event:step devices completed

    Stage_2 --> Stage_2: event:substep / item_progress
    Stage_2 --> Stage_N: event:step ... (stages 3–11)

    Stage_N --> Complete: event:step backup-routines completed\nevent:complete ok=true
    Stage_N --> Failed: event:error_detail\nevent:error\nevent:complete ok=false

    Complete --> [*]
    Failed --> [*]

    note right of Stage_N: Cancellation also\nleads to Failed state

WebSocket Channel¶

In addition to SSE, the plugin supports a WebSocket channel for broadcast messages. This is used by the home.js dashboard to receive sync-end notifications without polling.

Browser ──── ws://netbox-host/plugins/proxbox/ws/ ──── websocket_client.py ──── proxbox-api WS

The websocket.js script exposes:

onSyncEnd(listener)    // register a callback for sync completion events
notifySyncEnd(obj)     // trigger all registered listeners (called by sync.js)

Timeout Architecture¶

Proxbox involves three distinct timeout layers that must all be longer than the expected sync duration:

Timeout	Value	Location	Controls
RQ job wall-clock	7200 s	`PROXBOX_SYNC_JOB_TIMEOUT` in `jobs.py`	Max time for an RQ worker to hold the job before killing it
HTTP read (between chunks)	3600 s	`_SYNC_STREAM_READ_TIMEOUT` in `backend_proxy.py`	Max time to wait for the next SSE chunk from proxbox-api
NetBox API request	120 s	`PROXBOX_NETBOX_TIMEOUT` env var	Per-request timeout for netbox-sdk calls inside proxbox-api

Stuck jobs

pending forever: no RQ worker is running, or the worker does not listen to the default queue.
running for a long time: proxbox-api is still processing — check the Job log for SSE progress events.
errored with JobTimeoutException: the RQ wall-clock limit was hit — increase PROXBOX_SYNC_JOB_TIMEOUT.

See Backend Integration for diagnosis steps.

Streaming Protocol¶

Two Sync Transport Modes¶

SSE Event Types¶

discovery¶

step¶

substep¶

item_progress¶

phase_summary¶

error_detail¶

error¶

complete¶