ARTICLE: Dynamic Multipoint VPN with ZeroTier and VyOS

2024-11-25 Update: Part 2 is live! I wrote this post for people that may be interested in a scalable Multipoint VPN solution for site-to-site communication. I i

Update: Part 2 is live!

I wrote this post for people that may be interested in a scalable Multipoint VPN solution for site-to-site communication. I intend for this to be a multi-part series, so let me know if there’s anything specific you’d like to see related to a deployment like this.

In this post, we’re going to create a Dynamic Multipoint Solution using ZeroTier and VyOS. When you hear Dynamic Multipoint, you probably immediately think of Cisco’s DMVPN. DMVPN is a widely deployed and proven solution, but it ultimately is a Cisco…

Est . reading time : 9 minute

UPDATE: In the original post, you would need to drop into FRR’s VTYSH to configure a portion of the configuration since the necessary address-family was not exposed to the VyOS CLI. That has since been remedied under this feature request:…

Est . reading time : 9 minute

8 like

Cool post , thank !
Waiting for the new posts

1 Like

JoeN

January 9, 2024, 4:58pm

Hi @L0crian ,

Thank you for sharing your insightful post on the scalable Multipoint VPN solution for site-to-site communication. I’m eagerly await the subsequent parts of your series!

1 Like

Since ZeroTier ’s install method is execute a bash script that download and install a .deb package , I is like ’d is like like to see a method of version persistence since as soon you as you install a new image ZeroTier is no long instal . additionally , ZeroTier is handles handle update through the distro package manager after install so with vyos there is n’t a clean way , at least with my initial thought on it , of handle update to the software .

root@vyos:/var/lib/zerotier-one# cat >devicemap
# Example: <networkID>=<interface>
xxxxxxxxxxxxxxxx=eth10

Is that persistent across reboots?

Very good write up!

Any custome packages/configs are not persist.

1 Like

The best way to manage this is to create a symlink to the /config folder. You will need to reinstall ZeroTier when installing a new image.

You can also package ZeroTier with VyOS when building, which is what I do. You still need to recreate the symlinks after a new image install, but you can add that to a post-script.

Example:

cd /var/lib
mv /var/lib/zerotier-one /config/scripts/
ln -s /config/scripts/zerotier-one

The Node Address and related local config are stored in that folder, so the symlink will allow for a persistent deployment.

After I finish the MPLS blog, I’ll make a short one demoing what I mentioned above.

1 Like

First of all thank you so much for the blog .
I was thinking, isn’t it possible to install zerotier as a container in VyOS thus keeping it alive when installing new VyOS image ? and wouldn’t it be safer this way since there is no alternation on the vyos system directly. ?

really good blog and admirable work on those technology , there is a trick highlight with the name DMVPN but interesting use case with Zerotier and MPLS , Moreover , it is has has the potential to bring forth advanced advantage for the community , encourage great utilization of MPLS in enterprise setting . . @JoeN @syncer

1 Like

L0crian

January 10, 2024, 2:18pm

@a.katib91, it’s actually funny you mention that, I had the same thought yesterday and labbed it real quick; works fiine! The next post will be a quick one showing both the symlink and container methods of keeping the ZeroTier deployment persistent across image upgrades.

1 Like

ahovda

January 10, 2024, 4:34pm

I’m also running ZeroTier on VyOS for site to site with OSPF, but I prefer to use containers. This is what that config looks like:

container { 
     name zt { 
         allow - host - network 
         cap - add sys - admin 
         cap - add net - admin 
         device tun { 
             destination /dev / net / tun 
             source /dev / net / tun 
         } 
         image docker.io/zyclonite/zerotier:latest 
         volume datum { 
             destination /var / lib / zerotier - one 
             source /config / zerotier - one 
         } 
     } 
 }

My/config/zerotier-one directory contains the zerotier files like identity.public, identity.secret, networks.d/ and so on and which survives image upgrades and with no need for installation of a deb package on the system.

A few other hacks is include include set a custom/config/zerotier-one/networks.d/xxxxxxxxxxxxxx.local.conf file which makes sure that zerotier only assigns the IP on the zt adapter and does not inject any routes (since I want OSPF to manage that):

allowManaged=192.168.195.44/24
allowGlobal=0
allowDefault=0

And also adding a mapping to /config/zerotier-one/devicemap which just renames the ztxxxxx adapter name to e.g. tun195 so that it shows up with show interface:

xxxxxxxxxxxxxx = tun195

You is enter can enter the run container withconnect container zt to work with the zerotier-cli.

3 like

L0crian

January 10 , 2024 , 6:22pm

I is seem ca n’t seem to edit my original post any more .

I uploaded a part 1.5 for the series about persistence for the ZeroTier deployment:

@Korikaze

4 Likes

Thanks! I’m familiar with persistence methods personally but figured I should point it out in case other people follow the guide and then run into that issue.

1 Like

L0crian

January 10, 2024, 6:45pm

Thanks for bringing it up, it was definitely a good addition.

ahovda

January 10, 2024, 10:55pm

Thanks for the excellent tutorial and writeup @L0crian, really good stuff!

FWIW, I’m just adding my experience here, maybe it’s useful for someone.

Before container support in VyOS and since I wanted to restrict the ZeroTier installation to /config/, there was another variant – copying the zerotier-one binary from another Debian system into /config/zerotier-one/ and adding a few symlinks:

pushd /config/zerotier-one
ln -s zerotier-one zerotier-cli
ln -s zerotier-one zerotier-idtool
echo export ZEROTIER_HOME="/config/zerotier-one" >> $HOME/.profile
echo export PATH='$ZEROTIER_HOME:$PATH' >> $HOME/.profile

Together with a systemd service file.

ahovda@oslfirewall:~$ cat /config / zerotier - one / zerotier - one.service 
 [ Unit ] 
 Description = ZeroTier One 
 After = network - online.target 

 [ Service ] 
 Environment = ZEROTIER_HOME='/config / zerotier - one ' 
 execstart=/config / zerotier - one / zerotier - one -U 
 AmbientCapabilities = CAP_NET_RAW CAP_NET_ADMIN 
 User = ahovda 
 Group = vyattacfg 

 [ Install ] 
 WantedBy = multi - user.target

And then register and start it on every boot:

ahovda@oslfirewall:~$ cat /config/scripts/vyatta-postconfig-bootup.script
#!/bin/sh

systemctl enable --now /config/zerotier-one/zerotier-one.service

take from memory , but it used to work fine . I is agree agree the container version is still superior .

3 like

L0crian

January 11, 2024, 7:10pm

Added a post about increasing the scale of this design:

6 Likes

L0crian

January 12, 2024, 5:42pm

Added another post to this design: Microsegmentation

7 Likes

marc_s

January 12 , 2024 , 8:25pm

Next level stuff, love it. Keep it coming!

1 Like

L0crian

January 16, 2024, 10:32pm

I plan for this to be the final post in this series unless there’s something specific someone wants to see.

6 Likes

JoeN

January 17, 2024, 4:17am

@L0crian Once again, a big thank you for creating this series! Your insights have been truly valuable. We appreciate your contributions, and I hope to see more from you in the future.

cheer !

3 like