Skip to content

Commit 833dc2e

Browse files
authored
Multiple app registrations per runner (#44)
Accept atomic app snapshots with independent routes, versions, capacity, prices, and sessions. Keep authentication and O2R channels runner-scoped. Preserve admitted work while apps drain and pin session configuration. Adapt scalar heartbeats to the same app model. Generate short app- IDs with collision retries and validate IDs before committing snapshots. Add lifecycle, routing, payment, collision, and SDK compatibility coverage.
1 parent e2b3319 commit 833dc2e

21 files changed

Lines changed: 1360 additions & 434 deletions

‎docs/http-api.md‎

Lines changed: 26 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -10,29 +10,36 @@ See [orchestrator setup](orchestrator.md), [signer setup](signer.md), and
1010

1111
| Method and route | Purpose | Auth / envelope |
1212
| --- | --- | --- |
13-
| `POST /runners/heartbeat` | Register or reconcile dynamic runner | `Authorization` bootstrap credential on first call, derived heartbeat credential thereafter; JSON `runner_id`, interval, TTL, optional new credential, `session_ids`, optional `o2r`. |
13+
| `POST /runners/heartbeat` | Register or reconcile dynamic runner | `Authorization` bootstrap credential on first call, runner-scoped heartbeat credential thereafter; JSON `runner_id`, `version`, and `apps` snapshot; response includes interval, TTL, optional new credential, app reconciliation and `o2r`. |
1414
| `POST /runners/{runner_id}/unregister` | Remove dynamic runner | Scoped heartbeat credential. |
1515
| `GET /discovery` | Find available runners | JSON discovery entries with app, URL, mode, capacity and price. |
16-
| `POST /apps/{runner_id}/session` | Reserve persistent session | JSON `session_id`, `app_url`, `control_url`; paid mode may first return a payment challenge. |
17-
| `POST /apps/{runner_id}/session/{session_id}/stop` | Stop client session | Session authorization and callback reconciliation. |
18-
| `POST /apps/{runner_id}/session/{session_id}/payment` | Submit a ticket payment | On-chain only. |
16+
| `POST /apps/{app_id}/session` | Reserve persistent session | JSON `session_id`, `app_url`, `control_url`; paid mode may first return a payment challenge. |
17+
| `POST /apps/{app_id}/session/{session_id}/stop` | Stop client session | Session authorization and callback reconciliation. |
18+
| `POST /apps/{app_id}/session/{session_id}/payment` | Submit a ticket payment | On-chain only. |
1919
| `POST /refresh-payment` | Refresh ticket parameters for the same payer and manifest | On-chain only; JSON `sender` and `manifest_id`. |
20-
| `* /apps/{runner_id}/session/{session_id}/app/{app_path...}` | Persistent proxy | Preserve HTTP method, streaming, SSE and WebSocket semantics and Livepeer session headers. |
21-
| `* /apps/{runner_id}/app/{app_path...}` | Single-shot proxy | Transient capacity-limited reservation; identical session/control headers and cancellation ownership. |
22-
| `POST /runner/{runner_id}/session/{session_id}/channels` | Create generic trickle channels | Scoped session callback token; JSON `channels`. |
23-
| `DELETE /runner/{runner_id}/session/{session_id}/channels` | Delete generic trickle channels | Scoped session callback token. |
24-
| `POST /runner/{runner_id}/session/{session_id}/proxy` | Create session proxy URL | Scoped session callback token; JSON `target_url`; destination policy applies. |
25-
| `POST /runner/{runner_id}/session/{session_id}/stop` | Runner callback to release session | Scoped session callback token. |
20+
| `* /apps/{app_id}/session/{session_id}/app/{app_path...}` | Persistent proxy | Preserve HTTP method, streaming, SSE and WebSocket semantics and Livepeer session headers. |
21+
| `* /apps/{app_id}/app/{app_path...}` | Single-shot proxy | Transient capacity-limited reservation; identical session/control headers and cancellation ownership. |
22+
| `POST /runner/{app_id}/session/{session_id}/channels` | Create generic trickle channels | Scoped session callback token; JSON `channels`. |
23+
| `DELETE /runner/{app_id}/session/{session_id}/channels` | Delete generic trickle channels | Scoped session callback token. |
24+
| `POST /runner/{app_id}/session/{session_id}/proxy` | Create session proxy URL | Scoped session callback token; JSON `target_url`; destination policy applies. |
25+
| `POST /runner/{app_id}/session/{session_id}/stop` | Runner callback to release session | Scoped session callback token. |
2626
| `* /ai/trickle/{channel...}` | Generic channel traffic | Use the returned channel URL; see the [Trickle protocol](../trickle/README.md). |
2727

28-
SDK revisions, fixture setup, and compatibility test coverage are documented in
28+
SDK revisions and fixture setup are documented in
2929
[development](development.md#sdk-integration-fixtures).
3030

31-
Registration defaults capacity to one and trims/lowercases price units and
32-
currencies. See [public URLs](orchestrator.md#public-urls-and-tls) for
33-
`RunnerServiceURL`, and [sessions and proxies](orchestrator.md#sessions-and-proxies)
34-
for pricing, release behavior, and proxy routes. O2R channels send
35-
`{"keep":"alive"}` periodically while the runner is registered.
31+
Each heartbeat supplies a full `apps` snapshot for the owning runner. Each app
32+
has `id`, `app`, `version`, and `runner_url`, plus `mode`, `proxy`, `capacity`,
33+
`price_info`, `status`, `metadata`, `gpu`, and `session_ids` as needed. Top-level
34+
`version` is the runner SDK version; `apps[].version` is the application version.
35+
Capacity defaults to one; price units and currencies are trimmed and lowercased.
36+
37+
App IDs are unique per node; names may repeat. Omit `id` to generate one, then
38+
retain the returned ID in later snapshots. Omitted apps finish existing work
39+
before removal. O2R events use `app_id`.
40+
See [public URLs](orchestrator.md#public-urls-and-tls) for `RunnerServiceURL`, and
41+
[sessions and proxies](orchestrator.md#sessions-and-proxies) for pricing, release
42+
behavior, and proxy routes.
3643

3744
## Discovery HTTP
3845

@@ -49,12 +56,13 @@ with no `address`:
4956
{"url": "https://orch.example.com:8935", "queried_at": "2026-10-22T18:05:00Z"}
5057
]},
5158
{"address": "https://orch.example.com:8935", "runners": [
52-
{"url": "https://orch.example.com:8935/apps/r1/app", "app": "example", "observed_at": "2026-10-22T18:05:00Z"}
59+
{"url": "https://orch.example.com:8935/apps/a1/app", "app": "example", "observed_at": "2026-10-22T18:05:00Z"}
5360
]}
5461
]
5562
```
5663

57-
Runner records also carry mode, capacity, prices, and optional metadata/GPU fields.
64+
Runner records represent ready apps and carry mode, capacity, prices, and optional
65+
metadata/GPU fields.
5866
Discovery preserves complete runner records. `nodes` lists successfully queried,
5967
non-evicted sources, and `queried_at` records each source's last successful query
6068
time. Empty node and runner lists are omitted. With neither to advertise, discovery

‎docs/metrics.md‎

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -26,13 +26,14 @@ Failed collectors omit unavailable statistics and report
2626

2727
## Identity and counting
2828

29-
New Livepeer families carry `component`; legacy and Go/process collectors retain
30-
their existing labels. App/runner labels use registered `app`, `runner_id`, and
31-
`mode`; aggregate runners for app totals. Sessions and Trickle retain ownership.
29+
Livepeer families carry `component`; aggregate and Go/process collectors retain
30+
their existing labels. App identity labels are `app` (name), `app_id`,
31+
`runner_id` (owner), and `mode`. Inventory families measure app registrations;
32+
the aggregate `runners` gauge counts owners. Sessions and Trickle retain ownership.
3233
Inventory exists before traffic; unregister/expiry removes it, retaining counters.
3334

3435
Counters reset on restart; durable queue gauges recover from SQLite. ETH values
35-
are approximate operational measurements. Legacy aggregate runner/session, up,
36+
are approximate operational measurements. Aggregate runner/session, up,
3637
and outbox names remain available.
3738

3839
Operations use route patterns or fixed names; unknown routes/methods use
@@ -74,13 +75,13 @@ gauges; timestamps use Unix seconds. Most event families appear on first use.
7475
| Area | Families and dimensions |
7576
| --- | --- |
7677
| Runtime | `build_info{version,revision}`, standard `go_*`, `process_*`, and `go_build_info`. |
77-
| Legacy | `runners`, `sessions`, `signer_up`, and `discovery_up` retain their existing labels. |
78+
| Aggregate | `runners`, `sessions`, `signer_up`, and `discovery_up` retain their existing labels. |
7879
| HTTP | `http_requests_total{operation,method,status}`, `http_request_duration_seconds{operation,method}`, `http_requests_in_flight`, `http_admission_rejections_total{pool}`. Probe/scrape requests are excluded. |
79-
| Apps | `app_calls_total{app,runner_id,mode,protocol,outcome}`, `app_call_duration_seconds`, `app_response_header_duration_seconds`, `app_calls_in_flight`. Histograms use identity/protocol; in-flight uses identity. |
80-
| Upstreams | `upstream_requests_total{app,runner_id,mode,destination,outcome}`, `upstream_response_header_duration_seconds`. Destination is `runner` or `session_proxy`; HTTP failures use `http_4xx`/`http_5xx`. |
80+
| Apps | `app_calls_total{app,app_id,runner_id,mode,protocol,outcome}`, `app_call_duration_seconds`, `app_response_header_duration_seconds`, `app_calls_in_flight`. Histograms use identity/protocol; in-flight uses identity. |
81+
| Upstreams | `upstream_requests_total{app,app_id,runner_id,mode,destination,outcome}`, `upstream_response_header_duration_seconds`. Destination is `runner` or `session_proxy`; HTTP failures use `http_4xx`/`http_5xx`. |
8182
| Inventory | `runner_registered`, `runner_ready`, `runner_capacity`, `runner_capacity_available`, `runner_sessions{state="pending\|active"}`, `runner_last_heartbeat_timestamp_seconds`, all with app identity. Unavailable runners expose zero available capacity; heartbeat timestamps apply to dynamic runners. |
8283
| Lifecycle | `runner_events_total{event}`, `runner_health_checks_total{outcome}`, `session_reservations_total{outcome}`, `session_ends_total{reason}`, `session_duration_seconds`, with app identity. Reservations include allocations pending payment; only activated sessions contribute lifetime observations. |
83-
| Streams | `trickle_channels{kind="control\|session"}`, `trickle_buffered_bytes` for retained published segment bytes; `stream_transfer_bytes_total{app,runner_id,mode,protocol,direction}`, `stream_errors_total{app,runner_id,mode,protocol,outcome}`. Trickle requests are transport activity, not extra app invocations. |
84+
| Streams | `trickle_channels{kind="control\|session"}`, `trickle_buffered_bytes` for retained published segment bytes; `stream_transfer_bytes_total{app,app_id,runner_id,mode,protocol,direction}`, `stream_errors_total{app,app_id,runner_id,mode,protocol,outcome}`. Trickle requests are transport activity, not extra app invocations. |
8485
| Payments | `payment_operations_total{operation,outcome}` for challenge/receive/refresh/charge; `payment_tickets_total{kind="accepted\|winning"}`, `payment_credit_eth_total`, `payment_charge_eth_total`, `payment_billed_seconds_total`, `payment_fixed_charges_total`, with app identity. Credit is accepted expected ticket value; charges are successfully debited service usage. |
8586
| Signer | `signer_payments_total{type,outcome}`, `signer_tickets_total{type}`, `signer_sign_operations_total{kind,outcome}`. Signing kinds are ticket, segment, state, and orchestrator_info. Partial signing before a failed request remains visible. Successful payment/ticket totals follow required outbox persistence, even if response delivery is interrupted. |
8687
| Authorization | `signer_auth_webhook_requests_total{status,outcome}`, `signer_auth_webhook_duration_seconds`, `signer_auth_cache_hits_total`. Status zero means no webhook response; outcomes distinguish success, denied, and error. |

‎docs/orchestrator.md‎

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -75,23 +75,24 @@ its HTTPS address. The separate metrics listener must remain on loopback; see
7575

7676
## Sessions and proxies
7777

78-
Each session keeps its reserved price and paid status; heartbeat and exchange-rate
79-
updates affect new sessions. Releasing a session immediately frees capacity,
80-
revokes its token, and removes its channels and proxies.
78+
Each session keeps its reserved endpoint, app/mode settings, price, and paid status;
79+
heartbeat and exchange-rate updates affect new sessions. Releasing a session
80+
immediately frees capacity, revokes its token, and removes its channels and proxies.
8181
A runner stop gives in-flight HTTP, SSE, and WebSocket requests up to one second
8282
to finish; all other releases cancel them immediately. This applies to both
8383
persistent and single-shot sessions.
8484

85-
Enabling a runner's proxy creates an automatic proxy. Without a URL
86-
template, generated URLs use `ServiceURL/run/{proxy}`. The earlier
85+
Enabling an app's proxy creates an automatic proxy belonging to that app; use its
86+
returned proxy ID and URL. Without a URL template, generated URLs use
87+
`ServiceURL/run/{proxy}`. The earlier
8788
`/proxy/{proxy}` path remains an alias for active proxies. A runner can also
8889
create a session proxy through its callback route with a `target_url`.
8990

9091
`ProxyURLTemplate` accepts a hostname label or final path segment, for example
9192
`https://{proxy}.apps.example` or `https://apps.example/proxy/{proxy}`. Configure
9293
DNS, TLS, and forwarding to this listener for those URLs. Hostname routing uses
93-
the actual Host header. Single-shot domain proxies require a runner ID that is
94-
a lowercase DNS label of at most 63 characters.
94+
the actual Host header. Single-shot domain proxies require an app registration ID
95+
that is a lowercase DNS label of at most 63 characters.
9596

9697
## Enable payments
9798

@@ -137,7 +138,8 @@ Paid `/readyz` returns 503 until payment dependencies and the redemption worker
137138
are ready. See [metrics and operational events](metrics.md) for requirements,
138139
app/runner measurements, queue monitoring, and scrape/query examples.
139140

140-
The registry accepts up to 256 runners and 10,000 sessions, with 25 generated
141+
The registry accepts up to 256 runners, 128 retained apps per runner, 512 apps
142+
globally (including draining apps), and 10,000 sessions, with 25 generated
141143
proxies per session. Control requests have 64 admission slots and a five-second
142144
body deadline. Streaming requests have 256 separate slots.
143145

‎internal/observability/metrics.go‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,10 +38,10 @@ func (l Labels) With(key, value string) Labels {
3838
return copy
3939
}
4040

41-
type Identity struct{ App, Runner, Mode string }
41+
type Identity struct{ App, AppID, Runner, Mode string }
4242

4343
func (i Identity) Labels() Labels {
44-
return Labels{"app": i.App, "runner_id": i.Runner, "mode": i.Mode}
44+
return Labels{"app": i.App, "app_id": i.AppID, "runner_id": i.Runner, "mode": i.Mode}
4545
}
4646

4747
// Metrics owns a registry; services and tests never share global collectors.

‎orchestrator/app_test.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -380,7 +380,7 @@ Unit = "fixed"
380380
registry := NewRegistry("", "https://public.example", time.Second, time.Minute)
381381
registry.SetWeiPerUSD(big.NewRat(1, 1))
382382
require.NoError(t, loadStatic(path, registry))
383-
runner := registry.runners["r"]
383+
runner := registry.apps["r"]
384384
require.Equal(t, 204, runner.HealthCode)
385385
require.Equal(t, "1.5", runner.USDQuote.Price.String())
386386
require.Equal(t, &runnerGPU{ID: "0", Name: "H100", VRAMMB: 80000}, runner.GPU)

‎orchestrator/discovery_test.go‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,8 @@ func TestDiscoveryLocalInventory(t *testing.T) {
2424
s.ServeHTTP(w, httptest.NewRequest(http.MethodGet, "/deployment/discovery", nil))
2525
require.Equal(t, 200, w.Code)
2626
require.NotContains(t, w.Body.String(), `"nodes"`)
27+
require.NotContains(t, w.Body.String(), `"app_id"`)
28+
require.NotContains(t, w.Body.String(), `"runner_id"`)
2729
var entries []discoveryEntry
2830
require.NoError(t, json.Unmarshal(w.Body.Bytes(), &entries))
2931
require.Len(t, entries, 1)

0 commit comments

Comments
 (0)