NAT adapter: relay and operator trials

Result

The isolated Iroh adapter passed a finite private trial across Fly machines in ORD, IAD and LAX. The two endpoints initiated relay connections; direct IP transports were disabled and no public Fly services were configured.

• The initial query succeeded on its first attempt and returned one origin-verified synthetic Beacon through the relay.
• Stopping the relay caused the query to reach its five-second deadline. A separate command confirmed that the client machine remained reachable.
• After restarting the relay, the first query timed out; the second succeeded. The server retained the same transport identity and signed introduction.
• All three temporary machines were destroyed. Four pre-existing machine IDs and states were unchanged.

Each query has a five-second deadline. Cloud orchestration and endpoint shutdown add time outside that query budget. Recovery permits at most four explicit attempts; this result does not establish immediate or guaranteed reconnection.

What was tested

The adapter authenticates endpoint identities and verifies both operator and transport signatures on expiring, scope-bound introductions. Returned Beacons retain origin authentication. The finite listener permits four active handshake/ request tasks and 128 incoming attempts per run. The test relay permits eight active authenticated connections from explicitly listed identities, with a 128 KiB/s receive limit per client connection.

Earlier Linux testing used two separate internal networks, with the relay attached to both and no published host ports. It also refused an unlisted relay identity and recovered after relay restart.

Two earlier Fly attempts exposed harness errors: remote exit-status interpretation, and attempting to invoke a standalone kill executable absent from the minimal image. Neither was counted as a successful outage trial. The corrected harness uses a shell and an explicit remote-status marker. All attempts completed cleanup.

Reproduction and artifacts

The approved image is registry.fly.io/skeinnet:nat-relay-v1, registry digest sha256:d6ce1d5919ea2629ec685162402d0d56f483d681262b275a8956371035b79732. The local image ID is sha256:902c8472e31867f33c1db35e268563cacc023263242e87145a03db507a275abe. Node identities are generated at runtime, not embedded in the image.

python3 scripts/test_nat_process_fly.py --app skeinnet --image registry.fly.io/skeinnet:nat-relay-v1 --expected-digest sha256:d6ce1d5919ea2629ec685162402d0d56f483d681262b275a8956371035b79732 --output outputs/nat-fly-reproduction

This command requires authorized Fly access and creates billable temporary machines. Results and cleanup evidence are retained in docs/results/nat-process-v1/fly/. The runtime source lives in experiments/nat-transport/; it is separate from Core and now powers the v2 operator helper.

Public HTTPS and operator integration

The follow-up trial connected this local Mac to a Fly peer through a dedicated public HTTPS relay. Certificate verification passed. Direct IP/UDP transports were disabled, and the local computer required no inbound port or router changes. The first query returned one origin-verified synthetic Beacon.

The integrated native operator and Linux x86_64 Docker image each passed the same bidirectional exchange. Docker published no host ports. Both paths preserved identities across setup, refused an incorrectly bound plan, refused duplicate runtimes and live plan replacement, produced sanitized reports, and expired after 45 seconds. A probe used the already-running endpoint through its private control socket. Native socket cleanup was checked; Docker socket removal was not separately inspected. Temporary Fly machines and the Docker test volume were removed.

The exact packaged Apple Silicon executable also passed the HTTPS trial. A Fly provisioning race interrupted an earlier attempt before the relay started; all its machines were cleaned up, and the harness now retries that specific conflict. The source snapshot builds with locked dependencies; its Docker image exactly matches the image used in the successful Docker HTTPS trial.

Installer tests passed for Apple Silicon, Intel under Rosetta, and static Linux x86_64. They covered repeat setup, private-key permissions, corrupted-key refusal, archive corruption and replacement of a modified cached executable. Source and Docker remain optional alternatives to the native download.

A machine-readable results summary is published alongside the site. Detailed evidence is retained in docs/results/operator-nat-v2/. Reproduce the public trial with authorized Fly access and the configured HTTPS test app:

python3 scripts/test_nat_https_fly.py --operator-binary experiments/nat-transport/target/release/skein-operator-nat --output outputs/nat-https-reproduction

The test provisions finite machines using the approved relay image, and removes them on completion. The empty HTTPS app and its shared address remain reserved for coordinated trials; there is no continuously running relay.

Limits and next gates

These are same-account tests, including one real local network and Docker Desktop via OrbStack. They do not establish arbitrary NAT compatibility, independent administration, unrestricted public DoS resistance, or survival after losing every usable relay. Iroh encrypts endpoint payloads; public relay access uses HTTPS through Fly's edge. The relay still observes connection metadata and can interrupt delivery.

The public test configured Fly connection concurrency at soft 12/hard 16 and retained the authenticated relay identity/connection limits above. Pre-authentication resource pressure remains unmeasured. Upstream relay connection-rate settings are unimplemented and are not relied upon. These bounds do not justify unrestricted deployment.

Persistent identities recovered through an explicitly selected replacement relay in local tests. Automatic relay-provider discovery/failover and independently administered relay replacement remain open. The released onboarding profile is a finite, identity-allowlisted, relay-only trial; direct hole punching is disabled. The next milestone is participation by independent operators using the updated native, Docker, or source instructions.

Coordinator sequence

Collect 2–4 v2 public enrollment files over an authenticated channel. Confirm which person controls each identity; an enrollment file alone is not proof of ownership. Prepare individual plans shortly before the scheduled session:

python3 scripts/prepare_nat_operator_trial.py --relay https://YOUR-RELAY/ --seconds 60 --output outputs/cohort-plans alice-enrollment.json bob-enrollment.json

Distribute each participant's matching plan privately. The script writes public plans and the relay's identity allowlist; it neither contacts participants nor starts infrastructure. Start the configured HTTPS relay for 120 seconds just before participants join, using that exact allowlist:

skein-nat-process relay 0.0.0.0:8792 120 /state/ready.json "ALLOWED_TRANSPORT_IDS_COMMA_SEPARATED"

The relay backend must sit behind the HTTPS endpoint named in the plans. The Fly harness documents the tested service configuration. Do not expose its backend HTTP port publicly. Exchange signed peer records, have both nodes probe each other while running, collect sanitized reports, and verify shutdown. A missed window requires restarting the finite relay; expired plans must be regenerated. This manual coordinator step remains necessary until independently operated standing relays have passed their own availability and resource tests.