Setting up a gateway on OpenBSD - Revision history

KI5QKX: mw push

2026-04-30T18:52:55Z

mw push

← Older revision		Revision as of 18:52, 30 April 2026
Line 270:		Line 270:
	* Logging and error checking can always be improved.		* Logging and error checking can always be improved.

			[[Category:How-To]]
	[[Category:How-To Guides]]		[[Category:How-To Guides]]
	[[Category:Infrastructure]]		[[Category:Infrastructure]]
	[[Category:Gateways]]		[[Category:Gateways]]

KI5QKX: mw push

2026-03-01T00:14:54Z

mw push

← Older revision		Revision as of 00:14, 1 March 2026
Line 272:		Line 272:

	[[Category:How-To Guides]]		[[Category:How-To Guides]]
			[[Category:Infrastructure]]
			[[Category:Gateways]]

KI5QKX: mw push

2026-02-28T23:20:40Z

mw push

← Older revision		Revision as of 23:20, 28 February 2026
Line 34:		Line 34:
	We can repeat this process for each AMPRNet tunnel, creating interfaces and adding routes for each subnet.		We can repeat this process for each AMPRNet tunnel, creating interfaces and adding routes for each subnet.

	== Handling ~~Encapsulated Inbound Traffic Without~~ a ~~Reciprocal Tunnel~~ ==		== Handling encapsulated inbound traffic without a reciprocal tunnel ==

	When an inbound IPENCAP datagram arrives on our external interface, the network stack in the OpenBSD kernel recognizes it by examining the protocol number in the IP header: IPENCAP is protocol number 4 (not to be confused with IP version 4). Any such packets are passed to the packet input function in the ''gif'' implementation, which searches all configured ''gif'' interfaces trying to match the configured tunnel source and destinations addresses with the corresponding addresses in the inbound packet. If such an interface is found, the packet is enqueued to the interface, which will strip the IPENCAP header and route the resulting "de-encapsulated" IP packet. This works for tunnels that are configured bidirectionally between any two sites. That is, if site A has a tunnel to site B, and B has a corresponding tunnel to A, they can send each other traffic.		When an inbound IPENCAP datagram arrives on our external interface, the network stack in the OpenBSD kernel recognizes it by examining the protocol number in the IP header: IPENCAP is protocol number 4 (not to be confused with IP version 4). Any such packets are passed to the packet input function in the ''gif'' implementation, which searches all configured ''gif'' interfaces trying to match the configured tunnel source and destinations addresses with the corresponding addresses in the inbound packet. If such an interface is found, the packet is enqueued to the interface, which will strip the IPENCAP header and route the resulting "de-encapsulated" IP packet. This works for tunnels that are configured bidirectionally between any two sites. That is, if site A has a tunnel to site B, and B has a corresponding tunnel to A, they can send each other traffic.
Line 48:		Line 48:
	One this interface is configured, IPENCAP traffic from remote systems that have defined tunnels to us will flow, regardless of whether we have created a tunnel to them.		One this interface is configured, IPENCAP traffic from remote systems that have defined tunnels to us will flow, regardless of whether we have created a tunnel to them.

	== Policy-based ~~Routing Using Routing Domains~~ ==		== Policy-based routing using routing domains ==

	The configuration explored so far is sufficient to make connections to AMPRNet subnets we have manually configured tunnels for, but suffers from a number of deficiencies. In particular, the following two issues that we will discuss now.		The configuration explored so far is sufficient to make connections to AMPRNet subnets we have manually configured tunnels for, but suffers from a number of deficiencies. In particular, the following two issues that we will discuss now.
Line 90:		Line 90:
	With this in place, routing works as expected for all of the cases mentioned above.		With this in place, routing works as expected for all of the cases mentioned above.

	== Persistent ~~Configuration Across Router Restarts~~ ==		== Persistent configuration across router restarts ==

	We now have enough information that we can set up tunnels between our router and arbitrary AMPRNet subnets. However, doing so manually is tedious and not particularly robust. We would like the system to automatically configure our tunnels and default routes at boot time. Fortunately, the OpenBSD startup code can do this easily. For each network interface <code>$if</code> on the system, we can configure it automatically at boot time by putting configuration commands into the file <code>/etc/hostname.$if</code>.		We now have enough information that we can set up tunnels between our router and arbitrary AMPRNet subnets. However, doing so manually is tedious and not particularly robust. We would like the system to automatically configure our tunnels and default routes at boot time. Fortunately, the OpenBSD startup code can do this easily. For each network interface <code>$if</code> on the system, we can configure it automatically at boot time by putting configuration commands into the file <code>/etc/hostname.$if</code>.
Line 198:		Line 198:
	block in on $extif		block in on $extif

	# Pass ~~44net~~ traffic		# Pass 44Net traffic
	pass in on $extif inet proto ipencap from any to $extamprgate		pass in on $extif inet proto ipencap from any to $extamprgate

Line 232:		Line 232:
	A site can add more elaborate rules as desired.		A site can add more elaborate rules as desired.

	== Maintaining ~~Mesh Routes~~ with 44ripd ==		== Maintaining mesh routes with 44ripd ==

	The above shows how to set up AMPRNet tunnel interfaces, set routes to them, use routing domains for policy-based routing, and set firewall rules. However, so far all of these steps have been manual. This works for a handful of tunnels and routes, but there are hundreds of subnets AMPRNet in the AMPRNet mesh; maintaining all of these manually is not reasonable.		The above shows how to set up AMPRNet tunnel interfaces, set routes to them, use routing domains for policy-based routing, and set firewall rules. However, so far all of these steps have been manual. This works for a handful of tunnels and routes, but there are hundreds of subnets AMPRNet in the AMPRNet mesh; maintaining all of these manually is not reasonable.

KI5QKX at 20:19, 12 December 2024

2024-12-12T20:19:50Z

← Older revision		Revision as of 20:19, 12 December 2024
Line 269:		Line 269:
	* Exporting and/or parsing an encap file would be nice.		* Exporting and/or parsing an encap file would be nice.
	* Logging and error checking can always be improved.		* Logging and error checking can always be improved.


			[[Category:How-To Guides]]

Cross: Specify use of the `-cloning` flag when creating routes.

2024-04-28T02:19:56Z

Specify use of the `-cloning` flag when creating routes.

Cross: Minor grammar tweaks

2021-06-30T02:17:14Z

Minor grammar tweaks

← Older revision		Revision as of 02:17, 30 June 2021
Line 81:		Line 81:
	Now traffic on our AMPRNet subnet will be routed through the UCSD gateway, while traffic on the external interface will be routed through our ISP's router.		Now traffic on our AMPRNet subnet will be routed through the UCSD gateway, while traffic on the external interface will be routed through our ISP's router.

	Another piece of functionality ~~allows~~ us to dispense with much of the complexity of routing between domains: the tunnel assigned to a ''gif'' can be in a different routing domain than the interface itself. Going back to our example, if we place each of our ''gif'' interfaces into routing domain 44 then traffic routed out to on coming in from the tunnel will be routed with our AMPRNet-specific routing table. But if we place the tunnel on that interface into routing domain 23, then the encapsulated traffic that we send to and from the Internet (e.g., to other tunnel end points) will be routed in that domain, thus routing through our ISP's network. Critically, matching incoming datagrams to ''gif'' interfaces as described above happens in the routing domain associated with the tunnel, so inbound traffic coming through our external interface will be directed to the correct interface.		Another piece of functionality lets us dispense with much of the complexity of routing between domains: the tunnel assigned to a ''gif'' can be in a different routing domain than the interface itself. Going back to our example, if we place each of our ''gif'' interfaces into routing domain 44 then traffic routed out to or coming in from the tunnel will be routed with our AMPRNet-specific routing table. But if we place the tunnel on that interface into routing domain 23, then the encapsulated traffic that we send to and from the Internet (e.g., to other tunnel end points) will be routed in that domain, thus routing through our ISP's network. Critically, matching incoming datagrams to ''gif'' interfaces as described above happens in the routing domain associated with the tunnel, so inbound traffic coming through our external interface will be directed to the correct interface.

	We specify the routing domain of a tunnel via the <code>tunneldomain</code> parameter to <code>ifconfig</code> when configuring the route on an interface:		We specify the routing domain of a tunnel via the <code>tunneldomain</code> parameter to <code>ifconfig</code> when configuring the route on an interface:

Cross: Mainly grammar and typographical errors fixed.

2021-06-16T13:00:16Z

Mainly grammar and typographical errors fixed.

Cross at 13:20, 21 January 2021

2021-01-21T13:20:23Z

← Older revision		Revision as of 13:20, 21 January 2021
Line 236:		Line 236:
	The above shows how to set up AMPRNet tunnel interfaces, set routes to them, use routing domains for policy-based routing, and set firewall rules. However, so far all of these steps have been manual. This works for a handful of tunnels and routes, but there are hundreds of subnets AMPRNet in the AMPRNet mesh; maintaining all of these manually is not reasonable.		The above shows how to set up AMPRNet tunnel interfaces, set routes to them, use routing domains for policy-based routing, and set firewall rules. However, so far all of these steps have been manual. This works for a handful of tunnels and routes, but there are hundreds of subnets AMPRNet in the AMPRNet mesh; maintaining all of these manually is not reasonable.

	However, Dan Cross (KZ2X) has written a daemon specific to OpenBSD called <code>44ripd</code> that maintains tunnel and route information as distributed via the [https://wiki.ampr.org/wiki/RIP AMPRNet RIP] variant sent from <code>44.0.0.1</code. To use this, make sure that multicasting is enabled on the host by setting <code>multicast=YES</code> in <code>/etc/rc.conf.local</code>, and then retrieve the 44ripd software from Github at https://github.com/dancrossnyc/44ripd. The software is built with the <code>make</code> command. For example:		However, Dan Cross (KZ2X) has written a daemon specific to OpenBSD called <code>44ripd</code> that maintains tunnel and route information as distributed via the [https://wiki.ampr.org/wiki/RIP AMPRNet RIP] variant sent from <code>44.0.0.1</code>. To use this, make sure that multicasting is enabled on the host by setting <code>multicast=YES</code> in <code>/etc/rc.conf.local</code>, and then retrieve the 44ripd software from Github at https://github.com/dancrossnyc/44ripd. The software is built with the <code>make</code> command. For example:

	<nowiki>git clone https://github.com/dancrossnyc/44ripd		<nowiki>git clone https://github.com/dancrossnyc/44ripd

Cross at 17:45, 11 January 2021

2021-01-11T17:45:07Z

← Older revision		Revision as of 17:45, 11 January 2021
Line 250:		Line 250:
	fi</nowiki>		fi</nowiki>

	Note that the <code>-s</code> options instruct the daemon not to try and allocate the <code>gif0</code> and <code>gif1</code> interfaces, as these are manually configured.		Note that the <code>-s</code> options instruct the daemon not to try and allocate the <code>gif0</code> and <code>gif1</code> interfaces, as these are manually configured. The <code>-D</code> and <code>-T</code> options set the routing domains for ''gif'' interfaces and their associated tunnels, respectively.

	=== Set the Default AMPRNet Tunnel and Route Manually ===		=== Set the Default AMPRNet Tunnel and Route Manually ===

Cross at 17:32, 11 January 2021

2021-01-11T17:32:22Z

← Older revision		Revision as of 17:32, 11 January 2021
Line 115:		Line 115:
	inet 44.44.107.1 255.255.255.255</nowiki>		inet 44.44.107.1 255.255.255.255</nowiki>

	</code>/etc/hostname.gif1</code>:		<code>/etc/hostname.gif1</code>:

	<nowiki>tunnel 23.30.150.141 169.228.34.84 tunneldomain 23		<nowiki>tunnel 23.30.150.141 169.228.34.84 tunneldomain 23

← Older revision		Revision as of 02:19, 28 April 2024
Line 21:		Line 21:
	ifconfig gif1 tunnel 23.30.150.141 169.228.34.84		ifconfig gif1 tunnel 23.30.150.141 169.228.34.84
	ifconfig gif1 inet 44.44.107.1 netmask 255.255.255.255		ifconfig gif1 inet 44.44.107.1 netmask 255.255.255.255
	route add -host 44.0.0.1 -link -iface gif1 -~~llinfo~~</nowiki>		route add -host 44.0.0.1 -link -iface gif1 -cloning</nowiki>

	The first command creates the interface, causing the kernel to synthesize it into existence. The second configures the tunnel itself: that is, the the IP addresses that will be put into the IPENCAP datagram that the tunnel creates: the first address is the ''local'' address, which will serve as the source address for the IPENCAP packet, while the second is the ''remote'' address, to which the packet will be sent. The third sets an IP address for the local endpoint of the interface: this exists solely so that traffic that is generated by the router, such as ICMP error messages (host or port unreachable, for example), have a valid source address. Note that despite the fact that this is a point-to-point interface, we do not specify the IP address of the remote end.		The first command creates the interface, causing the kernel to synthesize it into existence. The second configures the tunnel itself: that is, the the IP addresses that will be put into the IPENCAP datagram that the tunnel creates: the first address is the ''local'' address, which will serve as the source address for the IPENCAP packet, while the second is the ''remote'' address, to which the packet will be sent. The third sets an IP address for the local endpoint of the interface: this exists solely so that traffic that is generated by the router, such as ICMP error messages (host or port unreachable, for example), have a valid source address. Note that despite the fact that this is a point-to-point interface, we do not specify the IP address of the remote end.

	The fourth and final command creates a host route and associates it with the tunnel interface. The <code>-link</code>, <code>-iface</code> and <code>-~~llinfo~~</code> flags indicate that this is an interface route, ~~and~~ that traffic for the route should go directly to the given interface (<code>gif1</code>) instead of identifying the gateway via an IP address. We can examine this route from the ~~commandline~~. E.g.,		The fourth and final command creates a host route and associates it with the tunnel interface. The <code>-link</code>, <code>-iface</code> and <code>-cloning</code> flags indicate that this is an interface route, that traffic for the route should go directly to the given interface (<code>gif1</code>) instead of identifying the gateway via an IP address, and that routes should be dynamically cloned when used. We can examine this route from the command line. E.g.,

	<nowiki>$ route -n show -inet \| grep '44\.0\.0\.1 '		<nowiki>$ route -n show -inet \| grep '44\.0\.0\.1 '
	44.0.0.1 link#7 ~~UHLSh~~ 1 2 - 8 gif1</nowiki>		44.0.0.1 link#7 UHCSh 1 2 - 8 gif1</nowiki>

	Consult the manual page for [https://man.openbsd.org/netstat.1 ''netstat''(1)] for details on what the <code>~~UHLSh~~</code> flags mean.		Consult the manual page for [https://man.openbsd.org/netstat.1 ''netstat''(1)] for details on what the <code>UHCSh</code> flags mean.

	We can repeat this process for each AMPRNet tunnel, creating interfaces and adding routes for each subnet.		We can repeat this process for each AMPRNet tunnel, creating interfaces and adding routes for each subnet.
Line 119:		Line 119:
	<nowiki>tunnel 23.30.150.141 169.228.34.84 tunneldomain 23		<nowiki>tunnel 23.30.150.141 169.228.34.84 tunneldomain 23
	inet 44.44.107.1 255.255.255.255		inet 44.44.107.1 255.255.255.255
	!route -qn -T 44 add 44.0.0.1/32 -link -iface gif1 -~~llinfo~~		!route -qn -T 44 add 44.0.0.1/32 -link -iface gif1 -cloning
	!route -qn -T 44 add default 44.0.0.1</nowiki>		!route -qn -T 44 add default 44.0.0.1</nowiki>

Line 171:		Line 171:
	Destination Gateway Flags Refs Use Mtu Prio Iface		Destination Gateway Flags Refs Use Mtu Prio Iface
	default 44.0.0.1 UGS 0 54718 - 8 gif1		default 44.0.0.1 UGS 0 54718 - 8 gif1
	44.0.0.1 link#7 ~~UHLSh~~ 1 2 - 8 gif1		44.0.0.1 link#7 UHCSh 1 2 - 8 gif1
	44.44.107/24 44.44.107.1 UCn 16 4 - 4 cnmac2		44.44.107/24 44.44.107.1 UCn 16 4 - 4 cnmac2
	44.44.107.1 44:d9:e7:9f:a7:66 UHLl 0 4825 - 1 cnmac2		44.44.107.1 44:d9:e7:9f:a7:66 UHLl 0 4825 - 1 cnmac2

← Older revision		Revision as of 13:00, 16 June 2021
Line 36:		Line 36:
	== Handling Encapsulated Inbound Traffic Without a Reciprocal Tunnel ==		== Handling Encapsulated Inbound Traffic Without a Reciprocal Tunnel ==

	When an inbound IPENCAP datagram arrives on our external interface, the network stack in the OpenBSD kernel recognizes it by examining the protocol number in the IP header: IPENCAP is protocol number 4 (not to be confused with IP version 4). Any such packets are passed to the packet input function in the ''gif'' implementation, which searches all configured ''gif'' interfaces trying to ~~find~~ match the configured tunnel source and destinations addresses with the corresponding addresses in the inbound packet. If such an interface is found, the packet is enqueued to the interface, which will strip the IPENCAP header and route the resulting "de-encapsulated" IP packet. This works for tunnels that are configured bidirectionally between any two sites. That is, if site A has a tunnel to site B, and B has a corresponding tunnel to A, they can send each other traffic.		When an inbound IPENCAP datagram arrives on our external interface, the network stack in the OpenBSD kernel recognizes it by examining the protocol number in the IP header: IPENCAP is protocol number 4 (not to be confused with IP version 4). Any such packets are passed to the packet input function in the ''gif'' implementation, which searches all configured ''gif'' interfaces trying to match the configured tunnel source and destinations addresses with the corresponding addresses in the inbound packet. If such an interface is found, the packet is enqueued to the interface, which will strip the IPENCAP header and route the resulting "de-encapsulated" IP packet. This works for tunnels that are configured bidirectionally between any two sites. That is, if site A has a tunnel to site B, and B has a corresponding tunnel to A, they can send each other traffic.

	Now consider the case where site A has a tunnel configured to send traffic to site B, but B has no tunnel configured to A: in this case, the datagram arrives as before and is presented to the ''gif'' implementation, but the search above fails since B has no tunnel to A, so nothing matches the source ''and'' destination addresses on the incoming packet. In this case, the system might be responsible for routing such packets to another computer or network, so the packet is not ~~de-encapsulated~~ and processed. However, in an AMPRNet context, we very well may want to process that packet. Accordingly, the ''gif'' implementation has a mechanism for describing an interface that accepts encapsulated traffic from any source destined to a local address. If we configure a ''gif'' interface where the distant end of the ''tunnel'' set to <code>0.0.0.0</code>, then any incoming datagram where the destination address is the same as the local address on the interface, will be accepted ~~and de-encapsulated~~ and processed as before. Using this, we can set up an interface specifically for accepting traffic from systems to which we have not defined a tunnel:		Now consider the case where site A has a tunnel configured to send traffic to site B, but B has no tunnel configured to A: in this case, the datagram arrives as before and is presented to the ''gif'' implementation, but the search above fails since B has no tunnel to A, so nothing matches the source ''and'' destination addresses on the incoming packet. In this case, the system might be responsible for routing such packets to another computer or network, so the packet is not decapsulated and processed. However, in an AMPRNet context, we very well may want to process that packet. Accordingly, the ''gif'' implementation has a mechanism for describing an interface that accepts encapsulated traffic from any source destined to a local address. If we configure a ''gif'' interface where the distant end of the ''tunnel'' set to <code>0.0.0.0</code>, then any incoming datagram where the destination address is the same as the local address on the interface will be accepted, decapsulated and processed as before. Using this, we can set up an interface specifically for accepting traffic from systems to which we have not defined a tunnel:

	<nowiki>ifconfig gif0 create		<nowiki>ifconfig gif0 create
Line 50:		Line 50:
	== Policy-based Routing Using Routing Domains ==		== Policy-based Routing Using Routing Domains ==

	The configuration explored so far is sufficient to make connections to AMPRNet ~~directly-~~configured ~~AMPRNet subnets~~, but suffers from a number of deficiencies. In particular, ~~there are~~ two issues that we will discuss now.		The configuration explored so far is sufficient to make connections to AMPRNet subnets we have manually configured tunnels for, but suffers from a number of deficiencies. In particular, the following two issues that we will discuss now.

	First, there is a problem with exchanging traffic with non-AMPRNet systems on the Internet. Presumably, these systems are not aware of AMPRNet tunneling, so traffic from them goes to the gateway at UCSD, where it will be encapsulated and sent through a tunnel to the external interface on our router. There, it will be ~~de-encapsulated~~ and delivered into our subnet. However, return traffic will be sent to the router, but since the destination is generally a tunnel, it will be sent via the default route, but ~~from~~ an AMPRNet source address. Since most ISPs will not pass AMPRNet traffic, the result will likely be lost before it reaches the destination. We may think it would be possible to work around that using a firewall rule to NAT the source address to something provided by our ISP, but even if the resulting datagram made it to the destination, for a protocol like TCP it would no longer match the 5-tuple for the connection, and would thus be lost.		First, there is a problem with exchanging traffic with non-AMPRNet systems on the Internet. Presumably, these systems are not aware of AMPRNet tunneling, so traffic from them goes to the gateway at UCSD, where it will be encapsulated and sent through a tunnel to the external interface on our router. There, it will be decapsulated and delivered into our subnet. However, return traffic will be sent to the local router, but since the destination is generally not a tunnel, it will be sent via the default route, but with an AMPRNet source address. Since most ISPs will not pass AMPRNet traffic, the result will likely be lost before it reaches the destination. We may think it would be possible to work around that using a firewall rule to NAT the source address to something provided by our ISP, but even if the resulting datagram made it to the destination, for a protocol like TCP it would no longer match the 5-tuple for the connection, and would thus be lost.

	The second problem is ~~how~~ reaching AMPRNet systems for which we have not configured a tunnel. Without a tunnel, and thus a route, we cannot send traffic to those systems.		The second problem is reaching AMPRNet systems for which we have not configured a tunnel. Without a tunnel, and thus a route, we cannot send traffic to those systems.

	We can solve both of these problems by sending all of our traffic through a tunnel interface to the UCSD gateway by default, e.g., by setting the default route:		We can solve both of these problems by sending all of our traffic through a tunnel interface to the UCSD gateway by default, e.g., by setting the default route:
Line 60:		Line 60:
	<nowiki>route add default 4.0.0.1</nowiki>		<nowiki>route add default 4.0.0.1</nowiki>

	However, how does the encapsulated traffic from the tunnel interface get sent to our ISP's ~~router~~? We can add a host route for the UCSD gateway in our local routing table, but we have to do this for every tunnel, which is unwieldy. Further, connecting to our external interface becomes complicated: suppose someone <code>ping</code>s our external interface. Assuming we permit this, the response would be routed through the UCSD gateway tunnel, but even if the gateway passed arbitrary traffic back onto the Internet, the packet might be lost as upstream ISPs would refuse to route it.		However this creates a new problem: how does the encapsulated traffic from the tunnel interface get sent to our ISP's gateway? We can add a host route for the UCSD gateway in our local routing table, but we have to do this for every tunnel, which is unwieldy. Further, connecting to our external interface becomes complicated: suppose someone <code>ping</code>s our local router's external interface. Assuming we permit this, the response would be routed through the UCSD gateway tunnel, but even if the gateway passed arbitrary traffic back onto the Internet, the packet might be lost as upstream ISPs would refuse to route it and even if delivered, it wouldn't match the destination address of the original ICMP echo request packet anyway.

	The solution to all of these problems is to use [https://en.wikipedia.org/wiki/Policy-based_routing ''policy-based routing'']. Specifically, we would like to make routing decisions based on the source IP address of our traffic. We might be able to do this with firewall rules, but the edge cases get complicated very quickly. Fortunately, there is another way: [https://man.openbsd.org/rdomain.4 routing domains].		The solution to all of these problems is to use [https://en.wikipedia.org/wiki/Policy-based_routing ''policy-based routing'']. Specifically, we would like to make routing decisions based on the source IP address of our traffic. We might be able to do this with firewall rules, but the edge cases get complicated very quickly. Fortunately, there is another way: [https://man.openbsd.org/rdomain.4 routing domains].