Internal APIs
Run three instances of an API at one overlay address, 10.128.0.50, and spread clients across them without a gateway or load balancer.
- Needs
- Three API hosts and at least one client, with root access and outbound internet access
- Uses
- conflux proxy with one shared --ipv4, conflux up on clients
Before and After VeilNet
Use this setup when an internal API runs on several hosts and you want clients to reach it at one address, without a gateway or load balancer.
Today, an API is published through a gateway on the internet, with a load balancer in front of its instances. Clients connect over TLS with X25519 and ECDSA, and a quantum computer can break both.
With VeilNet, every instance takes the same private IPv4 address on the overlay. Clients see one destination with three instances behind it, and each client is sent to one of them. The API keeps listening on loopback, and nothing is published to the internet.
Today, TLS 1.3, X25519 and ECDSA protect these links. A quantum computer can break all of them. With VeilNet, ML-DSA-87, ML-KEM-1024 and AES-256-GCM protect every link.
Sharing One Address
Follow Installation on each machine. The API itself does not change. In this guide, it listens on
127.0.0.1:8080on each host.
On a sovereign network, the guardian commissions each host and issues its manifest. Install it with sudo conflux enrol --manifest FILE, then run each command without --taint, because the credential carries the taints. Keep --ipv4, so that every instance takes the shared address and not an address that the guardian allocated. See Joining a sovereign network.
Distributing Clients
The client's anchor sends each flow to one instance. A flow is everything that one client host sends to the shared address, so traffic from different client hosts spreads across the instances.
| Rule | What it means for the API |
|---|---|
| A flow keeps its instance | A flow stays with its instance while it keeps sending, so a session never moves to another host partway through. After 15 minutes without traffic, the flow may move to another instance. |
| Nearer instances get more flows | New flows are weighted by the cost of reaching each instance. A nearer instance receives more flows, but every instance receives some. |
| Only a leaving instance's flows move | An instance leaves the group when it is dropped, or when traffic to it has failed for 2 seconds. Its flows move to the other instances, and they do not move back when it returns. |
| Taints decide who is in the group | A client chooses only among the instances that its taints allow it to exchange data with. |
Balancing works per client host, not per request. A busy client sends all its requests to one instance, so load evens out across many clients but not within one client.
Verifying the Setup
On a client, run sudo conflux routes. You should see the shared address as one route, with all three instances in the VIA column:
$ sudo conflux routes NETWORK VIA STATE WHY 10.128.0.50/32 (64:ff9b:1::a80:32/128) dfyspsc5lk,qh5av2ricp,z7kok3zv2m accepted - 10.128.0.101/32 (64:ff9b:1::a80:65/128) - served -
On an instance, the shared address shows as served, because that instance answers at it:
$ sudo conflux routes NETWORK VIA STATE WHY 10.128.0.50/32 (64:ff9b:1::a80:32/128) - served -
To reach one particular instance, use the IPv6 address from its overlay row. Each instance answers at that address as well as at the shared address:
$ curl -s -o /dev/null -w '%{http_code}\n' 'http://[fd80:c4b9:99ae:d4ad:1a8a:1eca:5e42:c961]/health'
200