VeilNet

Anchors

Every machine on a VeilNet network runs an anchor, the node that connects it to the overlay and carries its traffic.

One Kind of Node

An anchor is a node on the overlay, the virtual network that anchors build on top of your existing network, the underlay. Every node on the overlay is an anchor, and there are no separate classes of node. Any reachable anchor can act as:

  • a peer that exchanges traffic with other members
  • a relay that carries traffic for other members
  • a starting point for anchors that join
  • an observer that tells other anchors which address it sees them at

What sets one anchor apart from another is its credential, the signed statement that admits it to a realm. On a sovereign network, your bootstrap nodes and the guardian's root node in each realm are ordinary anchors.

The bootstrap nodes are reachable from the internet. The root node holds the powers it needs to issue the realm's commands. See Powers and Boundaries.

Realms form a tree, and the tree decides which anchors may relay for which realms. An anchor relays for its own realm and the realms beneath it, but never for a realm on another branch. See Permissions Between Realms.

One Ethernet Segment

To the programs on your machines, the overlay looks like one Ethernet segment that every member is plugged into. Each anchor runs its own network stack, the virtual kernel, so your host needs no Ethernet device for the overlay. Ordinary IP traffic crosses it unchanged.

The segment differs from a physical one in four ways:

  • No flooding. The anchor already knows every member's MAC address from the realm's membership. A frame for an address that no member holds is dropped, never flooded.
  • No ARP. The segment is IPv6 only, and neighbour discovery never goes out on the wire. The anchor carries IPv4 traffic inside IPv6. See IPv4 Addresses.
  • Broadcast and multicast travel one hop. The anchor also limits their rate.
  • Only data peers appear. The segment lists only the peers that this anchor may exchange data with. Any other anchor is absent from it, not unreachable.

Sending a Packet

When a program sends a packet to another machine's overlay IPv6 address, the two anchors carry it in these steps:

Fig. 01 — One packet, from one host to another
ANCHOR Athe senderANCHOR B · 07 · REVERSES EVERY STEPthe receiver01 · INTERFACE anchor0The host routes the packet in02–03 · NETWORK STACKEthernet header added04 · SWITCHMAC mapped to AnchorID05 · END-TO-END ENCRYPTIONEncrypted for B aloneQUIC · A STREAM BY DEFAULTOn the connection A dialledINTERFACE anchor0Delivered to the hostNETWORK STACKEthernet header removedSWITCHSwitched to the local portDECRYPTEDOnly B holds the keyQUICArrives on B's connectionOVERLAY PREFIX → anchor0NO ARP · NO SOLICITATIONUNKNOWN MAC: DROPPEDA RELAY CANNOT READ ITOR DIRECT06 · RELAY, IF NEEDEDUP TO FIVE · NONE CAN READ IT
Swipe to see the full chart

A relay forwards the frame, but holds no key that can decrypt it. When a direct connection is possible, the frame takes it and no relay is involved.

StepWhereWhat happens
1HostYour host routes the packet to the overlay interface, anchor0.
2Network stackThe anchor accepts the packet for forwarding. It forwards to an address outside the overlay only if a member offers a route to it.
3Network stackThe anchor adds an Ethernet header. It already knows the destination's MAC address, so it sends no request on the wire.
4SwitchThe anchor maps the destination MAC address to the AnchorID of the peer that owns it. It drops a frame for an address that no member owns.
5EncryptionThe anchor encrypts the frame end to end, with an ML-KEM session key that only the two anchors hold.
6NetworkThe frame travels directly, or through up to five relays. A relay cannot read the frame or tell what kind of traffic it carries.
7Receiving anchorThe receiving anchor reverses each step. The program on that machine receives an ordinary IP packet.

Transport

Anchors carry traffic over QUIC. The transport has four properties:

  • One UDP port. An anchor listens on one UDP port, on all interfaces, for as long as it runs. By default, the operating system chooses the port. To choose it yourself, run conflux up --port N.
  • A stream by default. Frames travel on a QUIC stream, so a lost packet costs one round trip instead of a timeout in your application. If you would rather lose a frame than wait for it, pass --low-latency to send frames as datagrams.
  • A connection each way. Two directly connected anchors hold two connections, one dialled by each. Each anchor sends on the connection it dialled, so the sender owns its route. It chooses the route, measures it and repairs it.
  • One address to start from. A joining anchor needs only one bootstrap address. Gossip tells it about the rest of the realm.

Ways onto the Overlay

Conflux runs the anchor on your machine, and gives you two ways to connect the host to the overlay:

ModeCommandWhat you get
Interfacesudo conflux upA network interface, anchor0 by default. The host routes overlay traffic into it, so ping, ssh and every other program can use overlay addresses. It needs root or Administrator rights.
Userspace proxysudo conflux proxy 8080=127.0.0.1:3000No interface. The anchor publishes the port of a local service at its overlay address, from inside its own process.

In either mode, the anchor can also offer its realm a network it is attached to (--subnet), or a route to the internet (--serve-exit). Members learn of these offers through gossip. Each member decides for itself whether its default route goes through an exit (--use-exit). See Subnet Routers and Exits.

Inspecting an Anchor

Conflux runs the anchor as a service, in the anchor daemon, anchord. You control it with anchorctl, which talks to the daemon over a local control socket. The socket is protected by file permissions and a token, and it is never exposed on the network.

Conflux passes anchorctl commands through. For example, sudo conflux peers runs anchorctl's peers command, and sudo conflux anchorctl … runs any anchorctl command by name. To check the anchor, run:

$ sudo conflux status

The anchor row shows this machine's full AnchorID. The underlay line shows the address and port that the anchor listens on in your existing network.

To list the anchor's peers, run:

$ sudo conflux peers
ANCHOR      OVERLAY                                  IPV4        HARDWARE           STATE      CONN     DATA  REACH     RTT
i46dpakhug  fd80:c4b9:99ae:df9b:5261:d178:18f1:783c  10.128.0.2  ae:70:60:82:85:68  connected  out*,in  yes   nat-cone  990µs
s5g3oefl2s  fd80:c4b9:99ae:a015:58f1:5b6e:b140:5e8e  -           5a:f0:a9:da:4e:32  connected  out      no    unknown   1.2ms
Reading the peers table
ColumnWhat it shows
ANCHORThe peer's AnchorID, shortened to ten characters. Commands that name an anchor need all 58 characters, as status prints them.
OVERLAY, IPV4The peer's overlay IPv6 address, and the IPv4 address it advertises, if any. The IPV4 column appears only when at least one peer advertises an IPv4 address.
HARDWAREThe peer's MAC address on the overlay.
STATEconnected means a live, authenticated connection. known means the anchor learned of the peer through gossip but has no connection to it. Both are normal.
CONNout is the connection this anchor dialled, and in is the one the peer dialled. A star marks the connection that this anchor sends on.
DATAWhether the two anchors may exchange data. no means that a block, a realm boundary or a taint separates them.
REACHThe peer's reachability, as the peer advertises it.
RTTThe round-trip time, measured every five seconds.

To watch peers arrive and leave, and to see renewals and blocks as they happen, stream the anchor's events:

$ sudo conflux anchorctl events