Routing

How Babel picks a route, and how the reconciler puts it into the kernel.

Babel (RFC 8966) decides which neighbor carries each prefix. ranet3 forwards by those decisions in its own table, keyed by source and destination prefix. The route reconciler copies the same decisions into a kernel routing table, out of the tun, and each lookup the host makes ends as where a lookup ends on Linux and on macOS give. Without cap.table, the reconciler does not run, and the host routes into the tun however it is configured to. On macOS, link.underlay.bind still writes a default scoped to the underlay's interface for a family whose zero address another program's route takes. Sharing a host describes that route.

The configuration reference lists every key under cap.route, cap.babel and cap.table.

The Babel speaker

ranet3 runs a Babel speaker over its ESP sessions. Each ESP session is a point-to-point Babel link. Babel packets use UDP port 6696 and the group ff02::1:6, sent from a random link-local address. They travel inside ESP and never pass through the tun.

Every Hello interval, each neighbor gets a Hello and an IHU. The Hello carries a timestamp (RFC 9616). The IHU carries this node's receive cost and echoes the neighbor's timestamps. A full route update goes out every update interval. Each Babel packet other than a path MTU probe fits in a 1280-byte inner IPv6 packet.

The router id is random. The speaker keeps it in the state directory (--state-dir) together with a limit on the seqnos it has sent. A restart carries on with the same router id, past every seqno the last run sent.

Link cost

A link's cost has 2 parts, and a third where cap.babel.cost.extra names the neighbor. The first is the receive cost the neighbor reports in its IHU, scaled by the share of the neighbor's last 16 Hellos this node received (ETX, RFC 8966 Appendix A.2.2). Each node computes the receive cost it reports the same way, as cap.babel.cost.rx (96 by default) scaled by its own reception.

The second part is a round-trip penalty (RFC 9616). The penalty is 0 at or below cap.babel.cost.rtt.min (10 milliseconds). It is cap.babel.cost.rtt.weight (1024) at or above cap.babel.cost.rtt.max (1024 milliseconds), and linear between them. Each new sample moves the smoothed round trip 16.4 percent of the way toward it. This node adds the penalty to its own cost of the link and does not advertise it.

A third part is the amount cap.babel.cost.extra charges the neighbor, chosen by the identity its session authenticated: by its node, by its organization, or for any organization other than this node's own (configuration reference). This node adds it last to its own cost of the link, and also to the receive cost in every IHU it sends that neighbor, so the neighbor's cost of the link back rises too. Each direction of the session pays it once, at the end that sends into it, and the neighbor needs no setting of its own: a BIRD neighbor takes the receive cost from the IHU as any Babel speaker does. ranet3 neighbors shows the amount in its Extra column, apart from the cost that holds it.

On this node both sums stop at 65534, so the amount alone never makes a usable link unreachable here, and a cost that is already infinite stays infinite. The neighbor holds no such bound. One that scales by its reception, as ranet3 does with cap.babel.quality at its default etx, multiplies the receive cost it is sent by its own loss and takes a product past 65534 as infinite: a receive cost of 30000 becomes infinite at a neighbor that heard 7 of this node's last 16 Hellos. At either end, a route's metric through the link is the link cost plus the metric advertised over it, and that sum stops at infinity too. A large amount can therefore leave a path with no finite metric, and so no fallback.

The amount suits a deployment where some paths cost more than their round trip says. Another organization's nodes may price every hop at a fixed 96 while this node prices a hop by its round trip, so a path across their nodes undercuts a direct session. With foreign = 1024, the most the round-trip penalty can add, a direct session to a node of this organization wins over a path through another organization's node, which stays the fallback, since 1024 is far from either bound above. A node name is only as strong as its organization's key, since every holder of that key can claim any of its names.

With cap.babel.quality set to "none", this node's own reception does not scale the cost, and the receive cost a neighbor reports, which that neighbor may scale by its own loss, is taken as it comes. A route's metric is the link cost, at least 1, plus the metric the neighbor advertises. A metric of 65535 is infinite and means unreachable.

What a node announces

cap.route.announce lists the prefixes this node originates. It advertises them with metric 0, under its own router id and seqno. An entry is a prefix, or a prefix with the source it serves ({ prefix, from }). A source-specific entry is IPv6 only (RFC 9079).

The node holds each prefix it originates as its own in its table. It holds each address cap.table assigns the same way, as a host route that it announces only if it originates that route too. A route a neighbor offers for the same destination and source prefix as one of these never replaces it. A host address of this node's, a /32 or /128 it originates or cap.table assigns, is delivered whatever the source, as the host's kernel delivers it, so a route a neighbor offers for that address is dropped whatever its source prefix. For a wider prefix, a source-specific route a neighbor offers for the same destination is another route: the node learns it and prefers it for packets from its source, by RFC 9079 section 4's destination-first ordering. A peer's packet for an address inside a prefix this node originates goes to the tun, and the host decides it. A read off the tun for such an address is dropped with no answer. Neither follows a covering route such as a default.

With transit on, the node relays every route it selects to its other neighbors and forwards the packets they bring, as transit describes. With transit off, it advertises only the prefixes it originates, and still learns and installs every route. Transit is off unless cap.transit writes enable = true, and a change to it takes a restart, as reloading lists.

cap.egress adds the prefixes it is ready to carry, without a source prefix. Exit nodes describes when it does. A node with an egress segment also originates each egress SID as a /128 while the exit is ready, as exit nodes describes.

Route selection

For each prefix, the speaker selects the feasible route with the lowest metric. A tie goes to the neighbor whose peer ID sorts first. The feasibility condition of RFC 8966 section 3.5.1 keeps the mesh free of loops, and an unfeasible route is never selected.

Selection has hysteresis (RFC 8966 Appendix A.3) with a time constant of 3 Hello intervals. The speaker leaves the selected route only for one better on both the current and the smoothed metric. An equal metric keeps the current next hop.

When only unfeasible routes remain, the speaker sends a seqno request. It repeats the request up to 4 times, the first after 3 seconds. A route is sent back toward its own next hop as a retraction, which is split horizon.

A route that appears or goes away sends an update at once. So does a change of next hop, origin or seqno, and a metric rise of at least rx. Any other change waits for the next full update.

Retracted routes

A retracted or expired route is held at infinity for the hold time of the Update it replaced, then flushed. While it is held, the forwarding entry answers "unreachable" (RFC 8966 section 3.5.4). A packet for that prefix is dropped instead of following a shorter covering route, such as a default another node carries. The forwarding decision answers one a peer sends with Destination Unreachable, and ranet3 answers one the tun hands it the same way, back into the tun, unless a steering policy takes it or the checks at the mesh's edge drop it, as transit says. The reconciler installs the same hold in the kernel, but for a route that counts as a default. Once the route is flushed, the prefix stays held until the latest time any update this node sent naming it reachable expires at a neighbor, if that is later, as the next paragraph says. Neither hold waits for every neighbor to acknowledge the retraction, as the protocol deviations say.

A prefix loses its last route when that route is flushed, when the session of the neighbor that offered it closes or is replaced, or when this node stops announcing it while cap.table does not assign it, or, for the host route of an address, when cap.table stops assigning it while this node does not announce it. A neighbor this node told the prefix is reachable may still send packets for it here until that update expires there, even after a retraction, which is sent once and never acknowledged. So the prefix is held the same way until the latest time any update this node sent naming it reachable expires at a neighbor, unless a neighbor's route for it is selected first. Each update's time is the route expiry of RFC 8966 Appendix B, "3.5 times the advertised update interval", counted from when that update was sent and taken from the interval it carried, plus 1 Hello interval for it to arrive. That covers a neighbor that expires a route at the multiple Appendix B suggests, as ranet3 does. RFC 8966 section 3.5.3 lets a neighbor choose "a small multiple". A shorter update interval set later does not shorten the hold, since a neighbor that missed the update sent under it still counts toward the one before. The prefix stays unreachable for that time even when every neighbor read the retraction, since this node cannot know that they did. A prefix that no update from this node named reachable in that time is not held, and a covering route serves it at once.

A hold refuses transit for as long as it lasts. The kernel's copy of a hold refuses the traffic this host sends or forwards into it only where the lookup ends at it, as rows L4 and L9 of where a lookup ends on Linux and row D3 of where a lookup ends on macOS give, and any other lookup ends where those tables give. A retracted prefix covered by a shorter route is answered unreachable until the latest update this node sent for it expires, 60 seconds at the defaults. An exit whose readiness withdraws its default holds that default the same way. While it originates its own default it holds no other exit's, since it drops every update for a prefix it originates. With another exit's default on offer, its transit for the internet moves to that exit once that exit's next update arrives, within 16 seconds at the defaults. With none, it answers that transit unreachable, IPv6 code 0 and IPv4 Network Unreachable, until the hold ends. The protocol deviations list the departure. Where the hold counts as a default the kernel keeps none, and that traffic ends as where a lookup ends on Linux and on macOS give for the routes left: by the host's own routes, or by a shorter route of the mesh back into the tun, where ranet3 answers it, or drops it at the mesh's edge, as a read off the tun. A leaf holding no other feasible default holds its own for its own hold time, 56 seconds at the defaults.

Transit

ranet3 decides each packet a session delivers in its own table, after this node's segments take the ones addressed to them. Nothing of the host takes part: its forwarding settings, its firewall's forward chain and its kernel tables do not. The tun carries what the mesh delivers to this host and the networks behind it, what the host sends or its kernel forwards into it from its other links, which the checks at the mesh's edge below meet, and, on an exit, the egress it hands the host to carry out, as exit nodes describe. The checks run in RFC 1812's order for both families, each before any answer, so a malformed packet draws none:

1. A frame that holds no whole IP packet is dropped with no answer.
2. A multicast, limited broadcast or IPv6 link-local destination goes to the tun, whatever the table holds. An IPv4 link-local destination does not, since RFC 3927 section 7 says "A router MUST NOT forward a packet with an IPv4 Link-Local source or destination address", and the next check drops it.
3. A source or destination no router forwards is dropped, an IPv4-mapped destination in ::ffff:0:0/96 among them, which RFC 6890 section 2.2.3 marks not forwardable. An IPv6 link-local source with a wider destination a router forwards is answered on the session it came from.
4. The table answers the packet as route selection leaves it. An IPv4 header failing RFC 1812 section 5.2.2's checks is then dropped with no answer, unless an entry of this node's own holds the destination, in which case the host's kernel checks the header. This node's own destination goes to the tun. A destination no entry holds, and one inside a held prefix, is answered unreachable. A destination this node holds through a route that counts as a default is egress, which an exit carries out as exit nodes describe: one inside cap.border.overlay is answered as one no entry holds, and any other goes to the exit, or to the tun through a route cap.route announces, only where the mesh routes a reply to its source, through a peer, a held prefix, or a network or address of this node's own.
5. With cap.transit off, a packet for another node is answered administratively prohibited, unless an End.DT46 SID of this node delivering to the mesh took it out of its outer header.
6. A hop limit or TTL of 1 or less is answered Time Exceeded, before the size is looked at.
7. An IPv4 header with options has them processed. A source route is dropped, Record Route and Timestamp record the IPv4 router-id, and options that cannot be read are answered Parameter Problem.
8. A packet larger than the next session carries is answered Packet Too Big, or Fragmentation Needed where it sets DF, naming that session's size. An IPv4 packet without DF is cut into fragments, or dropped with no answer and counted too_big where its data, from its fragment offset, runs past 65535 octets. Any other packet has its hop limit or TTL taken down and a TCP SYN's MSS lowered to the next session's, and goes on.

An answer comes from the node's router-id of the family. With no IPv4 router-id it comes from 192.0.0.8, and with no IPv6 one from the answer SID, the first End SID an announced prefix covers as path MTU describes, or not at all. A Packet Too Big with no IPv6 router-id and no SID, and a Fragmentation Needed with no IPv4 router-id, come from the packet's destination. An answer to a packet from this host or a network behind it, a source an entry of this node's own reaches that does not count as a default, goes into the tun instead, from the answered packet's destination, the one source every reverse path check of the host takes. A source only this node's own default reaches gets no answer, but for the beyond scope answer to an IPv6 link-local source, which goes back on the session the packet came by. Path MTU gives the rate limits the answers draw on, and the protocol deviations say where each departs from the RFCs.

A read off the tun to a multicast destination, link scope included, or to the limited broadcast 255.255.255.255 is dropped and counted as multicast before its route is looked up or it is steered. The mesh routes no multicast, and RFC 1812 section 5.3.5.1 says "Limited broadcasts MUST NOT be forwarded", where a covering route such as a default would carry either to the one peer it names.

A read off the tun is also checked at the mesh's edge before it enters, since a node that forwards for others can put a packet into the tun with a source or a header of their choosing. The decision drops a read from a source the table routes to another node or holds unreachable, through a route that does not count as a default, or from an overlay source this node does not own (forged), one carrying an IPv6 routing header the mesh refuses (routing_header), one to this node's own SIDs (sid), or one bound toward an exit's default from a source this node does not own (unowned_source). Every IPv6 read whose walk ends at an inner IPv4 or IPv6 packet has that packet checked one level deep, whatever its destination, since the mesh cannot tell another node's SID from any other address, as the limits say. An inner packet that is itself encapsulated is dropped (nested). A read the table answers as this node's own, which the mesh could only hand back to the tun, is dropped (looped). None of these is answered, as the border filter and segment routing describe for the mesh's edge.

A read off the tun that passes those checks and that no steering policy takes, whose destination no entry holds or that lies inside a held prefix, is answered as the table above gives for that destination. The answer goes back into the tun from the packet's destination, whatever the packet's source, since the tun is the one link the packet came by: an uplink address the mesh routes nothing to is answered too, unless the destination lies under an entry that counts as a default, where the check above drops the read as unowned_source. It draws on the same rate limits. A steered packet goes by its first segment, and one whose first segment has no route is counted and gets no answer.

ranet3 status counts each outcome on its Transit line, and ranet3_transit_packets_total exports the same counts. A dropped packet also records the event netstack.transit.dropped with its reason, at most once every 30 seconds.

The route reconciler

Writing a cap.table block turns the reconciler on. It mirrors the mesh's selected routes and its holds into a kernel table. Each prefix this node originates is a hold there inside cap.border.overlay, and outside it a throw on Linux and nothing on macOS, but for one that counts as a default, which gets nothing. An address cap.table assigns gets no route there unless this node originates it too, since the host's kernel holds it as one of its own already. On Linux a hold or a throw names no device, though an IPv6 one reads back as dev lo, and every other route leaves through the tun. On macOS every route it installs, a hold among them, is on the utun. The reconciler also assigns the tun's addresses. On Linux it also installs the policy rules it derives.

What it installs

Each entry of the mesh's table becomes:

A throw ends the table's lookup, and the rules after the one that looked it up decide, as row L10 of where a lookup ends gives. A hold of an overlay prefix refuses an unmarked lookup bound to no device for an overlay destination no route of the mesh holds, row L4, rather than letting it follow another route, such as an exit's default, or leave by main. A route for a longer prefix inside the overlay still wins by its length. A route of the mesh wider than an overlay prefix carries a peer's packet for an address inside that prefix, since the forwarding decision keeps no hold of an overlay prefix, while the host's own packet for the same address ends as rows L2, L4 and B1 to B4 give, by its mark and the device its lookup is bound to.

A node is an exit when its cap.route announces, or its cap.egress advertises, a prefix that counts, or its cap.segment holds an egress segment. An exit installs no route that counts, its own or learned, whatever the sessions, since with another exit's default in its table its own traffic from its own prefixes would end at row L11 of where a lookup ends, into the tun and out through that exit.

Policy rules on Linux

The reconciler derives its policy rules from cap.border.overlay and this node's own prefixes, and installs them in two blocks at fixed priorities, each rule in both families unless it names a prefix, and each carrying cap.table.proto. At Tailscale's default base the first block sits below its rules at 5210 to 5270, and the second above them, as sharing a host notes. <table> is cap.table.id.

A skipped length skips only a route the lookup found, never an unreachable, blackhole or prohibit one.

This node's own prefixes are every address cap.table.addresses writes, and every prefix this node originates, such as one cap.route announces, that does not count as a default, but a prefix the mesh cannot carry, such as a link-local one, a prefix cap.egress advertises for the hosts behind this node, and an announcement with a source, D from S. Under the destination-first ordering of RFC 9079 section 4 the mesh routes D to this node only for packets from S, so a reply to D from outside S would never come back through the mesh. With no cap.border, no 5110 rule is installed.

Where a lookup ends on Linux

These tables are where a lookup ends under the derived rules, and the other pages link here for it. They hold where cap.table is written and the reconciler is enabled, with every rule of the reconciler's last pass in place, no rule of another program in the policy database, and no route of another writer in cap.table.id's table.

Where that does not hold, the reconciler does this:

• A pass adds each nop before the goto that lands on it. Where a nop's add fails, or another program deletes it, the kernel passes its goto over, as it would a missing one (net/core/fib_rules.c:324-330 at Linux 940de590, where the walk passes a goto whose target is unresolved, and :936-944 and 1078-1098, where an add and a delete leave it so), so a marked lookup meets that block's lookups as an unmarked one does until a later pass adds the nop.
• The reconciler never removes another writer's route from its table. It warns once at startup where another writer's unicast route is in its table, and once for each route it would install whose key another writer holds. The rows below leave such a route out.
• The gotos jump from 5100 to the first rule at 5190 and from 5400 to the first rule at 5490, so another program's rule at 5101 to 5189 or 5401 to 5489 meets only unmarked lookups, and one at 5190 or 5490 meets marked ones, ahead of the reconciler's nop where it was added first. Another program's goto that lands among the reconciler's rules goes on from the rule it lands on.
• Another program's rule ahead of 5100 can take any lookup it selects, a marked one among them, before the reconciler's rules see it.

Sharing a host gives how another program's rule changes where a lookup ends, with the layouts measured.

• A lookup is marked when its mark is exactly 0x72, which ranet3's underlay sockets carry where the process may set it, as sharing a host says. Any other mark, such as kube-proxy's 0x4000, counts as unmarked.
• A lookup carries a source where its socket is bound to an address, or an earlier lookup gave it one, such as at a connect. A datagram of a socket bound to no address carries none, but over IPv6 the kernel looks a lookup with no source that the walk refuses up again from the source it then picks (ip6_dst_lookup_tail, net/ipv6/ip6_output.c:1147-1188 at Linux 940de590), and that lookup ends by the rows as one from that source: at L11 where the source is one of this node's own prefixes and no row before L11 answers it, and refused again where L4, L7 or L9 refused it. A forwarded packet is an unmarked lookup carrying its own source.
• The mesh's longest match is the longest route of cap.table.id covering the destination, a hold or a throw among them. Over IPv6 a source-specific route matches only a lookup from its source.
• Within a table the first row that matches gives where the lookup ends. Each row holds for both families unless it names one.

A lookup bound to no device:

• L3 and L5: every overlay prefix has a hold of its own but one that holds its family's unspecified address and one the mesh cannot carry, so only inside those can the mesh's longest match be a route shorter than the prefix, a throw, or nothing. Without cap.border no destination is inside the overlay.
• L4, L7 and L9: a skipped length never skips an unreachable, blackhole or prohibit route. That is why the reconciler installs no hold of a route that counts.
• L10: the throw is this node's own prefix outside the overlay. Main's route out of the uplink for the host's subnet, its LAN or a prefix cap.egress advertises has already answered at L6.
• L11: an exit installs no route that counts, so on an exit no lookup ends there.

A lookup bound to a device, such as by a socket bound with SO_BINDTODEVICE, ends by the rows above with these changes. An address of the host's own ends at L1 only for a lookup bound to the device that holds it.

B1 and B2 are a declared limit, also listed in limits and the protocol deviations. Such a packet never enters the mesh, and an application that binds the uplink's device asked for the uplink. Over IPv4 a packet B1 puts on the uplink's link for a held destination went by main's route before ranet3, and a host of that link holding the address receives it.

A pass derives the rules again from what the mesh holds, so a prefix this node starts announcing, or a reload moving the overlay, moves them in the next pass. Each nop goes in before the goto that lands on it. A rule another program deletes comes back within a second, as sharing a host describes. ranet3 debug rules lists the rules the reconciler derives beside every rule the kernel holds.

Protocol 155 differs from BIRD's 12 and babeld's 42. The reconciler asks for each route exclusively. A key another writer already holds stays as it is, and the reconciler warns once. Linux logs kernel is leaving a route that another writer holds. macOS logs kernel is leaving a route that another program holds, or for a member of the zero-free cover the line the cover gives.

Some routes are never installed:

• multicast, link-local and loopback prefixes, and 255.255.255.255/32, with the warning kernel is not installing a prefix the mesh cannot carry
• source-specific IPv4 routes, with the warning kernel cannot install source-specific IPv4 route

The mesh's own table still holds both kinds and forwards a source-specific IPv4 route, while a destination of the first kind meets the checks transit lists.

When a pass runs

A pass applies, in order, the addresses, the routes and the rules. It removes what the mesh no longer wants before it installs.

• A change in the mesh, or a kernel notification, starts a pass 250 milliseconds later. Changes in that window join the same pass.
• A periodic pass runs every cap.table.reconcile (30 seconds by default) and corrects drift.
• A failed pass is retried after 1 second, doubling up to the periodic interval.
• A reload that changes cap.table, the addresses it assigns or cap.border.overlay, or whose cap.route announcements make this node an exit or stop making it one, ranet3 enable reconciler and ranet3 disable reconciler start a pass too.

ranet3 disable reconciler withdraws every route, address and rule the reconciler installed. ranet3 enable reconciler puts them back. Both are refused on a node with no cap.table.

Routes that count as a default

Some routes would carry the host's own traffic into the mesh, and they count as a default. Such a route lies outside cap.border.overlay and is shorter than /8 for IPv4 or /16 for IPv6, or contains its family's unspecified address however long it is. A default and its /1 halves count, and so do a split internet such as four IPv4 /2s or 2000::/3, and a wide prefix such as fd00::/8, fc00::/7 or an IPv4 /7. A route at or longer than the floor, or inside the overlay, is a subnet, unless it contains the unspecified address. A route lies inside the overlay when each of its addresses lies in some entry, whether one entry holds it or several together do. Without cap.border, which macOS refuses at startup, no route lies inside the overlay.

The floors are /8 and /16 because a route shorter than either takes a share of a host's traffic as a default does, a split internet above all, while the subnets a mesh routes outside its overlay are longer. The unspecified address counts at any length because macOS falls back to it for a socket bound with IP_BOUND_IF, as sharing a host describes.

The reconciler installs a route that counts only after a session has been live. It withdraws them all once no session has been live for cap.table.capture_grace (10 seconds by default, 1 second at least), and restores them on the first live session. On macOS with link.underlay.bind a plain /0 goes in as its zero-free cover only once the host has a default of its own of that family, and another route it installs unscoped holding its family's zero address only once its family's fallback is in place, as the zero-free cover describes.

A session is live for 20 seconds after the last authenticated IKE message from its peer, and for 20 to 30 seconds after the last authenticated ESP packet, as the design's limits list. No session counts as live while the underlay socket is not ready. The withdrawal logs kernel is withdrawing the routes that would carry this machine's own traffic, and the restore logs kernel is carrying this machine's own traffic over the mesh again.

The reconciler never installs a hold of a route that counts, nor this node's own prefix that counts, whatever the sessions. An unreachable route of that width in the kernel would answer with an error every lookup of the host's own traffic that ends at it, as where a lookup ends on Linux and on macOS give, while other sessions stay live, and a retracted split internet would cut the host off there. The mesh's own table keeps the hold, and a packet the tun hands ranet3 for an address inside it is answered unreachable rather than following a shorter route (RFC 8966 section 3.5.4), unless a steering policy takes it or the checks at the mesh's edge drop it, as transit says. A peer's packet for such an address never reaches the host, and the forwarding decision answers it unreachable. A packet the host sends or forwards for one ends as where a lookup ends on Linux and on macOS give for the routes left: by the host's own routes, or by a shorter route of the mesh back into the tun, where ranet3 answers or drops it the same way. The protocol deviations list the departure.

A node whose cap.route announces, or whose cap.egress advertises, a prefix that counts and is not a default makes itself an exit for that family, as if it announced a default: every other node's reconciler treats a route to the prefix as above. The file still loads, and the daemon logs a warning at startup and on each reload it applies, such as config: cap.route announces fd00::/8, outside the overlay and shorter than /8 for IPv4 or /16 for IPv6 or holding the unspecified address, which counts as a default and makes this node an exit for its family, as routing.md says. A gateway to a network numbered from fd00::/8 that should stay a subnet router announces the prefixes it carries, each /16 or longer, instead.

ranet3 exit list lists a /0 or the two /1 halves one node advertises together, and nothing else. A node whose other advertisement counts, such as 2000::/3, carries its family's traffic with no row there: ranet3 routes shows its route, and ranet3 whois names the peer an address inside it takes.

Ownership

The reconciler reads back, deletes and withdraws only what it owns. On Linux it owns the routes in its own table that carry its protocol and leave by its tun, and its holds and throws there. It also owns the rules that carry its protocol and the addresses it added. On macOS it is the routes of its own shape out of its own utun. Sharing a host describes the rules in full.

Linux and macOS

On both platforms a route that counts as a default, the /0 default among them, goes in only once a session has been live, and never on an exit, as what it installs says. On macOS a route scoped to the utun reaches a socket bound to the utun or to an address of it, as where a lookup ends on macOS says. A listed endpoint address is a literal address of a node's endpoint in the trust document. An endpoint written as a hostname does not count.

iOS and Android have no reconciler. Platforms covers each platform in full.

Where a lookup ends on macOS

macOS has no policy rules, and the reconciler writes into the one forwarding table every program shares. A socket bound to no interface and no address of the utun takes the longest match among the table's unscoped routes, the host's and the reconciler's alike. A route scoped to an interface answers a socket bound to that interface, and one scoped to the utun also answers a socket bound to an address of the utun, which the source-specific row below relies on. A route that counts as a default goes in only once a session has been live, and never on an exit, as what it installs says. The first row that matches gives how an entry of the mesh's table goes in:

A destination takes one scoped route. The reconciler offers the entry with the most specific source first, and leaves another entry that would go in scoped at the same destination uninstalled, such as ::/0 beside ::/0 from a prefix of the utun, and logs it once. With link.underlay.bind, a plain /0 goes in as the members of its zero-free cover. Where a lookup ends:

D5's hostname endpoint is a declared limit, listed in limits.

The zero-free cover on macOS

With link.underlay.bind set, a plain /0 that Babel selects goes into the kernel as 8 routes per family that together hold every address but 0.0.0.0/8 and ::/8:

An unscoped /0 out of the utun would collide with the host's own default, and macOS has no route replace. No member holds its family's zero address, so a socket bound with IP_BOUND_IF, ranet3's own underlay socket among them, falls back to the host's own default for every destination. The members are therefore unscoped even where a listed endpoint address lies in one, and nothing is written outside the utun. That was measured on a Mac with a bound socket connecting inside every member. The Mac had no unscoped IPv6 default, so the IPv6 half was measured against a stand-in default on a test utun.

Only the kernel's table holds the cover. Babel's table, the updates it sends and ranet3 exit list keep the /0, so no route's prefix is replaced by a longer one inside the protocol (RFC 8966 Appendix C).

A member is withheld while the kernel holds another route of the reconciler's, unscoped and in the same family, of length 1 up to the member's own, that contains it. The kernel then forwards every address outside the hole by the route Babel's longest match would, or by a member where that match is the /0. A route the kernel refuses or fails to install withholds nothing, and the members it would withhold are tried in its place, but for a route refused against this node's own member, described below. A route that another program's route refuses at a member's own key leaves that member refused too, which takes the family's cover out as described below.

The hole, 0.0.0.0/8 and ::/8, stays on the host's own route. ::/8 holds 64:ff9b::/96, the NAT64 well-known prefix, so NAT64 traffic does not take the exit.

The cover of a family goes in and out whole, with the /0. The capture gate judges the /0, so the cover waits for a live session and leaves when no session has been live for capture_grace. Its family also waits for the host's own default of that family, and while it waits the reconciler logs kernel is holding back a route that would carry this machine's own traffic. Each family's cover a pass puts in or takes out logs kernel changed the cover a plain default is installed as and records a kernel.cover event, which the withdrawal of ranet3 down or ranet3 disable reconciler does not, as debugging says. The Kernel line of ranet3 status names the covered families.

A member that another program's route at its key refuses, or that fails to install, takes its family's cover out whole, since part of a cover would split that family's traffic between the mesh and the other route. Each of the family's members that no route withholds counts as skipped, but for one that failed to install, which the pass reports as an error. The refusal logs kernel is leaving a default's cover out, since another program holds a route at one of its members once, with held_by naming that route. The family's traffic then takes the routes already in the host's table.

A mesh route at a member's own destination, such as a real 128.0.0.0/1 a node advertises, takes that member's key. The reconciler deletes the member before it installs the route. When the delete fails, the pass reports it as an error, and macOS refuses the route against this node's own member, which counts as skipped. Each later pass deletes the member again and tries the route, so the error and the skipped count last until a delete succeeds. Meanwhile the member leaves by the utun as the route would, and traffic is forwarded as Babel selects.

Until exit selection exists, a Mac with bind set takes whichever /0 Babel selects. A source-specific default and a real /1 half that a node advertises are not covered. They are routes like any other, and a hold of a default is not installed at all.

A route the reconciler installs unscoped that is not a member but holds a zero address, such as a real 0.0.0.0/1 or a 0.0.0.0/24, needs a default scoped to the underlay's interface for that family before it is installed. The reconciler asks for it on every pass and asks again after a pass deletes such a route, so the scoped default is withdrawn in that same pass, once the route is out of the table. Sharing a host describes it.

Watching it

These commands show the routes. Command line lists their flags.

• ranet3 routes shows Babel's table, every prefix, including the ones held unreachable. The prefixes this node announces and the addresses cap.table assigns show this node as their next hop. It is not the kernel's table.
• ranet3 neighbors shows each neighbor with its link cost, the amount cap.babel.cost.extra charges it, the cost it reports, its round trip and the routes it offers.
• ranet3 whois <address> shows the longest prefix covering an address first, then the shorter ones, each with the neighbor that is its next hop, or this node for one of its own.
• The Kernel line of ranet3 status shows where the reconciler writes, the routes installed and skipped, and the last pass.

SIGUSR1 writes the mesh's routes to the log, this node's own entries as via this node and the holds as via unreachable.

The 2 route lines share 1 limit of 1 line per second. Each carries changes_since_last, the number of changes since the last line. The events babel.neighbor.up, babel.neighbor.down, babel.route.selected and babel.route.retracted record each change. kernel.pass records each pass that changed the kernel, met a newly refused route, failed or followed a reload, and a withdrawal only when it fails, as debugging says.

The neighbor cost includes the amount cap.babel.cost.extra charges, which no series reports alone. ranet3 neighbors shows it apart. Metrics describes the scrape.