IPIP Mesh/MikroTik RouterOS 7 Container

Setting up a gateway in a ROS7 Mikrotik router running in a container on arm and arm64 models and x86-64 CHR

This is an experimental software build for the 'enthusiasts' out there.

Info

These are the steps for setting up a fully functional AMPR gateway on an arm/arm64 Mikrotik router Tested and found working on CRS2116 and RB3011 for now.

NOTE: THE SETUP SCRIPT DOES NOT SECURE YOUR ROUTER. YOU NEED TO SET UP FIREWALL RULES YOURSELF.

General concept

Mikrotik routers running ROS 7 (7.15.3 being current at the time of writing) based on arm and arm64 processor, as well as CHR setups are able to run software containers (similar to docker). This opens the possibility to host a virtualized gateway in such a container, allowing a simple and efficient setup on modern systems.

The gateway will be hosted in a VRF on the router, providing gateway services using policy routing.

As a concept, the container has a single VETH interface which will decapsulate all incoming IPIP traffic from the tunnels, and encapsulate all outgoing traffic towards them. The container itself is isolated behind a bridge and offers some basic filtering function (e.g. restrict access from internet hosts). It will receive the RIPv2 broadcasts from the AMPR gateway and provide the obtained routes as RIP broadcasts to the router itself inside the mentioned VRF.

The container does not save anything to disk (which would be a bad idea on the router's flash memory), so the AMPR routes are lost on container or router restart, and you need to wait the now classical 5 minutes. But this should be no problem on a 24/7 on router.

Initial steps

The steps to be taken depend on the fact if you set up a new router or want to add the container to an existing running one.

Limitations

The router does not forward multicast frames at all, nor does it send out broadcasts. Incoming broadcasts are accepted and forwarded to the local VRF.

The container itself needs to sit behind a bridge due to a kernel bug in the version used by Mikrotik which sends out "Port unreachable" ICMP messages on incoming IPIP traffic if it is handled in user space (The same thing causing the need of a kernel filter in amprd. This is fixed in newer kernel releases but it will take a while for it to make its way into ROS). Bridge filtering is used to mask those messages.

New router set up

As a prerequisite, get your internet connection working based on the default mikrotik configuration. Basically set up your ISP uplink either via DHCP or by setting up a PPPoE or similar connection. Leave the firewall rule as they are.

First you need to enable container support according to the info provided by Mikrotik. In a console type in:

/system/device-mode/update container=yes

The device will ask you to reset it by hand (you can not do this remotely).

Next we need to install the container according to your hardware. Please chose the correct setup script variant:

ARM32 -  ampr_arm32.rsc
ARM64 -  ampr_arm64.rsc
CHRx86 - ampr_x86_64.rsc

Unfortunately, containers are not available on Mips, Tile or PowerPC devices.

The example assumes you use an arm32 device. Please use the proper one...

Open a route console window.

1. Check is the remote server is available:

[admin@MikroTik] > ping yo2loj.ro
 SEQ HOST                                     SIZE TTL TIME       STATUS                    
   0 89.33.44.100                               56  58 10ms574us 
   1 89.33.44.100                               56  58 9ms141us  
   2 89.33.44.100                               56  58 9ms5us    
   sent=3 received=3 packet-loss=0% min-rtt=9ms5us avg-rtt=9ms573us max-rtt=10ms574us

2. Download the configuration script

[admin@MikroTik] > /tool fetch url="http://yo2loj.ro/containers/ampr_arm32.rsc"
     status: finished
 downloaded: 5KiBC-z pause]
      total: 5KiB
   duration: 1s

3. Run the configuration script

[admin@MikroTik] > import ampr_arm32.rsc
AMPR: Creating bridge and VRF
AMPR: Setting up RIP
AMPR: Creating container envs
AMPR: Setting up firewall rules
AMPR: Creating container update script
AMPR: Creating routing rules
AMPR: Installing container
No container is installed
     status: finished
 downloaded: 366KiB
      total: 366KiB
   duration: 1s
AMPR: Script finished successful

Your container is now installed. You need to configure its environment variables according to the description give.

After configuration is complete, go to "containers" and star it up. It should show "running" and you should see it's messages in the log window.

After at most 5 minutes, you should get the tunnel routes in your vrf, and your gateway should be fully up and running.

Container configuration parameters

You need to adapt the pre-existing container environment variables to your particular gateway before starting it again. The following ENV parameters are preset in Container-> Envs:

AMPR_SUBNETS - holds your local subnets as defined in the portal, as comma separated list of <SUBNET>/<MASK> tupples, e.g. "44.128.0.0/24,44.128.1.0/24"
ALL_VIA_AMPRGW - enables forwarding of all AMPR destinations via AMPRGW, values are "0" or "1"
FORWARD_INTERNET - enables forward of traffic from/to internet hosts, values are "0" or "1"
IGNORED_SUBNETS - allows you to ignore specific subnets provided by RIP, by <SUBNET>/<MASK> or gateway address e.g. "44.128.0.0/16"
CALL_HOME - the classic string, <CALLSIGN>@<LOCATOR> to show up on the map. You will get a yellow dot. e.g. "YO2LOJ@KN05OR". Leaving the field empty disables call home.

Please note that the provided default will allow you to play around, but will not provide a working set up.

After finishing the configuration, add your internet interface to the interface list called "Internet":

/interface list member add interface=ether1 list=Internet

Now edit the Route->filter rip-ampr-in to point to your preferred source address (your router's AMPR IP address:

(Winbox is your friend)

...and enable the 2 disabled routing rules under Routing->Rule:

[admin@MikroTik] /routing/rule> set 0 disabled=no
[admin@MikroTik] /routing/rule> set 1 disabled=no

This should do it... Start the container, sit back and wait 5 minutes for the routes to show up in your vrf.

Of course you need to set up firewall rules & stuff, but if you do not enable internet forward, you should be pretty safe.

Configuration on an existing working router

Basically you need to do 6 steps by snooping around in the provided rsc files:

1 - Bridge, VETH and VRF setup: http://yo2loj.ro/containers/1_ampr_bridge_vrf.rsc
2 - RIP setup: http://yo2loj.ro/containers/2_rip.rsc
3 - Firewall rules, Filter, NAT and Mangle: http://yo2loj.ro/containers/3_firewall.rsc
4 - Container environment setup: http://yo2loj.ro/containers/4_container_env.rsc
5 - Container installation, architecture dependent. Files hold the download and update script:
 ARM32:  http://yo2loj.ro/containers/5_container_arm32.rsc
 ARM64:  http://yo2loj.ro/containers/5_container_arm64.rsc
 x86_64: http://yo2loj.ro/containers/5_container_x86_64.rsc
6 - final routing rules: http://yo2loj.ro/containers/6_rules.rsc

All available files are here: http://yo2loj.ro/containers/

(Details are coming...)

Rip Rip Hurray! de YO2LOJ