DragonflyBSD Home Router

DragonflyBSD Home Router

My trusty PCEngines APU2 is giving-up-the-ghost. Sadly the AMD SoC it uses is EOL, and PCEngines is thus winding up. So I need to replace it. Fortunately, I have a Beelink SER5 kicking about (a second one). It only has one NIC, but 802dot1q never killed anyone.

Base requirements

For this to work, I need a few things set up on this new router:

In addition, I need my switch to have at least 8x1000BASE-T ports, and support for configuring VLANs. What I ended up buying well overshot that - the Extreme Networks X440-G2-12T-10GE4. Silent unless running unusually hot (the fans only kick in past a threshold), it's a nice edge switch with 4x10GBASE-X uplinks and 12x1000BASE-T switchports.

I bought it from etb-tech, who I've used professionally in the past. They're alright; quite good at sorting out problems that pop up (dead PSUs, misconfigured units, and so on).

Extreme X440-G2-12T-10GE4

I've never used Extreme Networks kit. Some Cisco, a lot of Dell, but not Extreme. But - it's purple, shouldn't have fans spinning during normal operation - and it seems to be a bit more than what I need. So I figured I'd give it a go.

Intial configuration was done over a serial cable ("rollover", RJ45 to DB9). ETB were good enough to properly reset things before shipping it out. I wont cover the commands I ran (alas, I didn't jot them down). To summarise:

DragonflyBSD

rc.conf

Aside from typical host configuration that is generally populated as part of the installation process, a few settings need changing in order to prepare the box to function as a router:

gateway_enable="YES"
ipv6_enable="YES"
ipv6_network_interfaces="re0 tun0"
ipv6_gateway_enable="YES"
ipv6_router_enable="YES"

VLANs

VLANs aren't much of a bother on DragonflyBSD. I don't know if it supports (for example) 802.1ad (q-in-q), but it certainly does support typical use. The syntax to create a named VLAN interface in rc.conf is interesting:

vlans_re0="vlan10"
create_args_vlan10="vlan 10"

While this works (and is suggested by rc.conf), the same thing can be done a different way (as found in examples for FreeBSD online). This alternative has the benefit of also creating the tun0 interface, which I'll need later.

cloned_interfaces="vlan10 tun0"
ifconfig_vlan10="vlan 10 vlandev re0"

So - simple enough. This VLAN is used to push tagged frames carrying PPP over the switch (configured above) and onto the modem, so it doesn't need further configuration.

PPPoE

PPP is less simple. For reasons unclear to me, pppd took umbrage at the thought of creating its tun interface, and would instead complain it doesn't exist and exit. Thus the cloned_interfaces change I made above.

I'm not sure how this will interract with wireguard, which also uses a tun interface (I'm presuming when kernel support lands in 6.6, it'll just work). Aside from that, things seemed quite standard. The FreeBSD documentation out there helped a lot. For PPP, in rc.conf:

ppp_enable="YES"
ppp_profile="aaisp"
ppp_mode="ddial"
rtsold_enable="YES"
rtsold_flags="-d tun0"

And in ppp.conf:

default:
 set log Phase Chat LCP IPCP CCP tun command
 set device PPPoE:vlan10:aaisp
 set ctsrts off
 set timeout 0
 set redial 0 0
 set mtu 1492
 set mru 1492
 enable ipv6cp
 enable ipcp
 enable dns
 enable mssfixup
 enable lqr
 shell /usr/sbin/ndp -i re0 -accept_rtadv
 shell /usr/sbin/ndp -i tun0 accept_rtadv
 add! default HISADDR
 add! default HISADDR6

aaisp:
 set authname my_username
 set authkey my_password

You'll note the addition of ndp commands. This is simply to ensure that the local network interface doesn't accept router adverts, while ensuring the ppp interface does. Do I expect spurious router adverts on my local network? No, but it would be an interesting attack to perform on a local network. There's no harm in making sure we're not accepting those packets.

PF

PF is a large topic. The only thing I'll comment on here, is that the version shipping with DragonflyBSD is not the same as the version in OpenBSD, nor FreeBSD. Both in terms of code, and in terms of version. For this reason, you can really only depend on the man-page for pf.conf (well, or old threads on the FreeBSD forums). The best example are the changes to nat and rdr, which are now done with match rules upstream.

IPv4 addresses with dhcpd

dhcpd is a dog. But I've always felt odd about using dnsmasq when I don't actually need any of its DNS functionality. Configuration for my network, specified in dhcpd.conf (which I mixed up with dhcpcd.conf at least once), is as follows:

option domain-name "piconet.co.uk";
option domain-name-servers 192.168.1.254;
default-lease-time 600;
max-lease-time 7200;
authoritative;
log-facility local7;

subnet 192.168.1.0 netmask 255.255.255.0 {
  range 192.168.1.20 192.168.1.120;
  option routers 192.168.1.254;
}

IPv6 RA's with radvd

The easiest part - telling radvd what interface to run on is enough. It'll pick up my prefix from the interface's own address allocation. Again in rc.conf:

rtadvd_enable="YES"
rtadvd_interfaces="re0"

DNS with Unbound

I'd already been running unbound on a different box, which OPNSense had been providing to clients. So this was a case of copying that configuration over and ensuring everything was running properly. In unbound.conf:

server:
	interface: re0@53
        prefer-ip4: no
	prefer-ip6: yes
        access-control: 192.168.1.0/24 allow
        access-control: 2001:8b0:ca70:32b2:: allow
        access-control: fe80:: allow
        cache-max-ttl: 3600
        cache-max-negative-ttl: 15

forward-zone:
        name: "."
        forward-addr: 2001:8b0::2020
        forward-addr: 2001:8b0::2021

Monitoring

So, one thing OPNSense does really well is its dashboard. I mean, yes, php-fpm does sometimes die, and restarting the service doesn't always work. But it's a nice interface that provides status at a glance. I wanted something like that for both my new router and my other DFly-based server.

Munin is one option. But is backed by a PostgreSQL instance, which is a bit much. Netdata is interesting, in that they specifically support FreeBSD in both their open-source and paid/hosted product. But they don't support Dragonfly, and I really don't want to go through the rigmarole of building from source and figuring out how to add support. Icinga is an option. I have considerable experience using it, but again, it requires a dedicated database (Maria or Postgres).

Whereupon I stumbled across Monit. It's in the package repos, it has one configuration file, and it has built-in support for monitoring disks, network interfaces, and remote hosts (via ICMP/TCP/UDP). The only downside is that to monitor multiple hosts from one monit instance, you seem to need to pay for m/monit.

I'm not against paying for software. Especially not when we're talking about free (as in freedom) software. But the initial pricing is too steep for me. One host costs 41EUR, thereafter each additional host costs +9EUR. I only need to monitor two hosts, so that initial cliff is quite large.

Still! The open-source single-host version works fine. It just means I have one interface per host.

Closing Notes

It's been fun setting this up. If not for that one weird issue with PPP not creating tun0 on start, I'd have said it was painless. IPv6, as always, was a bit fiddly. Seeing EXOS and comparing it to whatever Dell ships on their PowerEdge switches was fun (Dell wins on functionality in their HTTP interface, Extreme win on how they present functionality).

Will it last? There's part of me that feels I should probably just slap OPNSense on it and be done with it. But on the other hand, revisiting this stuff is probably helpful for my job. If anything changes, I imagine it'll get a post.

Monit dashboard