Yarmo - wireguardBlog of an Open Source developerZola2022-04-16T12:00:12+00:00https://yarmo.eu/tags/wireguard/atom.xmlWireguard and docker: providing VPN access to arbitrary containers2022-04-16T12:00:12+00:002022-04-16T12:00:12+00:00
Unknown
https://yarmo.eu/blog/wireguard-docker/<h2 id="Your_container_might_benefit_from_VPN_access">Your container might benefit from VPN access</h2>
<p>Some containers just aren't meant to be connected directly to the internet. After all, you wouldn't want your ISP knowing which Linux distribution you download and share.</p>
<p>If like me you have your BitTorrent client installed as a container on a homeserver to make sure it's always connected but you don't want to route your other containers through a VPN, you'll probably want to use a VPN-in-a-container and route your BitTorrent client through it.</p>
<p>I already had a similar solution using OpenVPN but it was time for an upgrade. Oh yes, it's <a href="https://www.wireguard.com/">Wireguard</a> time.</p>
<p>As VPN provider, I use <a href="https://mullvad.net/">Mullvad</a>.</p>
<h2 id="The_solution">The solution</h2>
<p>Our situation is this: our homeserver (could be a Linux machine, a Raspberry Pi…) runs two docker containers, one which is fine to be directly connected to the internet and one which would benefit from VPN access.</p>
<p>One could install the Wireguard client straight on the machine and route both containers through the VPN, but for various reasons, that's now what we want here.</p>
<p>Our solution will be to add another container which connects to the VPN and route our sensitive container through the VPN container.</p>
<p>With some experimenting, I got it working 90%. The only issue was that while the BitTorrent client was perfectly shielded by the VPN, I could no longer access the client myself. Not great.</p>
<p>After two days of trying stuff out and searching the internet, I found the working solution on a blog post from 2021 which sadly already no longer exists. But thanks to the Web Archive, <a href="https://web.archive.org/web/20210207170757/https://bookstack.almueti.com/books/wireguard/page/docker-compose-with-mullvad-wireguard-arbitrary-service">its wisdom is lost no more</a>.</p>
<h2 id="PostUp_and_PreDown">PostUp and PreDown</h2>
<p>The reason I didn't get it working myself is because I knew the problem lay in the <code>PostUp/PreDown</code> commands of the Wireguard configuration. And I don't know how to read or write those :/ Mullvad provides their own but they do not work in this situation.</p>
<p>I must therefore warn you that I sadly do not fully understand the solution. I probably could fiddle with it and get it working on a different system, but I don't <em>understand</em> it. I simply took my 90%-functional implementation, copy-pasted the <code>PostUp/PreDown</code> commands from the linked blog post and voilà, success!</p>
<p>Not proud of it, and I hope I'll gain understanding of these commands in the near future, but that's the situation.</p>
<h2 id="The_implementation">The implementation</h2>
<p>You must have Wireguard installed on your system but it doesn't need to be running any connection.</p>
<h3 id="docker-compose.yml"><code>docker-compose.yml</code></h3>
<pre data-lang="yml" style="background-color:#212733;color:#ccc9c2;" class="language-yml "><code class="language-yml" data-lang="yml"><span style="color:#73d0ff;">version</span><span style="color:#ccc9c2cc;">: </span><span style="color:#bae67e;">'2.3'
</span><span>
</span><span style="color:#73d0ff;">services</span><span style="color:#ccc9c2cc;">:
</span><span> </span><span style="color:#73d0ff;">wireguard</span><span style="color:#ccc9c2cc;">:
</span><span> </span><span style="color:#73d0ff;">image</span><span style="color:#ccc9c2cc;">: </span><span style="color:#bae67e;">linuxserver/wireguard
</span><span> </span><span style="color:#73d0ff;">hostname</span><span style="color:#ccc9c2cc;">: </span><span style="color:#bae67e;">wireguard
</span><span> </span><span style="color:#73d0ff;">container_name</span><span style="color:#ccc9c2cc;">: </span><span style="color:#bae67e;">wireguard
</span><span> </span><span style="color:#73d0ff;">cap_add</span><span style="color:#ccc9c2cc;">:
</span><span> - </span><span style="color:#bae67e;">net_admin
</span><span> - </span><span style="color:#bae67e;">sys_module
</span><span> </span><span style="color:#73d0ff;">ports</span><span style="color:#ccc9c2cc;">:
</span><span> - </span><span style="color:#bae67e;">8112:8112
</span><span> - </span><span style="color:#bae67e;">58846:58846
</span><span> </span><span style="color:#73d0ff;">volumes</span><span style="color:#ccc9c2cc;">:
</span><span> - </span><span style="color:#bae67e;">/lib/modules:/lib/modules
</span><span> - </span><span style="color:#bae67e;">./data/wireguard:/config
</span><span> </span><span style="color:#73d0ff;">sysctls</span><span style="color:#ccc9c2cc;">:
</span><span> - </span><span style="color:#bae67e;">net.ipv4.conf.all.src_valid_mark=1
</span><span>
</span><span> </span><span style="color:#73d0ff;">deluge</span><span style="color:#ccc9c2cc;">:
</span><span> </span><span style="color:#73d0ff;">image</span><span style="color:#ccc9c2cc;">: </span><span style="color:#bae67e;">linuxserver/deluge
</span><span> </span><span style="color:#73d0ff;">container_name</span><span style="color:#ccc9c2cc;">: </span><span style="color:#bae67e;">deluge
</span><span> </span><span style="color:#73d0ff;">network_mode</span><span style="color:#ccc9c2cc;">: </span><span style="color:#bae67e;">service:wireguard
</span><span> </span><span style="color:#73d0ff;">volumes</span><span style="color:#ccc9c2cc;">:
</span><span> - </span><span style="color:#bae67e;">./data/deluge:/config
</span><span> - </span><span style="color:#bae67e;">./data/downloads:/downloads
</span></code></pre>
<p>In this docker-compose setup, we use the <a href="https://hub.docker.com/r/linuxserver/wireguard">linuxserver/wireguard</a> and <a href="https://hub.docker.com/r/linuxserver/deluge">linuxserver/deluge</a> container images. Please have a look at their respective documentation for more information on their configuration.</p>
<p>A few interesting notes:</p>
<pre data-lang="yml" style="background-color:#212733;color:#ccc9c2;" class="language-yml "><code class="language-yml" data-lang="yml"><span> </span><span style="color:#73d0ff;">cap_add</span><span style="color:#ccc9c2cc;">:
</span><span> - </span><span style="color:#bae67e;">net_admin
</span><span> - </span><span style="color:#bae67e;">sys_module
</span><span> [</span><span style="color:#bae67e;">…</span><span>]
</span><span> </span><span style="color:#73d0ff;">volumes</span><span style="color:#ccc9c2cc;">:
</span><span> - </span><span style="color:#bae67e;">/lib/modules:/lib/modules
</span></code></pre>
<p>The <a href="https://hub.docker.com/r/linuxserver/wireguard">linuxserver/wireguard</a> image uses the system's Wireguard module and this configuration allows the container to access it.</p>
<pre data-lang="yml" style="background-color:#212733;color:#ccc9c2;" class="language-yml "><code class="language-yml" data-lang="yml"><span> </span><span style="color:#73d0ff;">sysctls</span><span style="color:#ccc9c2cc;">:
</span><span> - </span><span style="color:#bae67e;">net.ipv4.conf.all.src_valid_mark=1
</span></code></pre>
<p>This is important but sadly, I do not know what it does.</p>
<pre data-lang="yml" style="background-color:#212733;color:#ccc9c2;" class="language-yml "><code class="language-yml" data-lang="yml"><span> </span><span style="color:#73d0ff;">ports</span><span style="color:#ccc9c2cc;">:
</span><span> - </span><span style="color:#bae67e;">8112:8112
</span><span> - </span><span style="color:#bae67e;">58846:58846
</span></code></pre>
<p>This is the interesting part. We assign those ports to the <code>wireguard</code> container, but they are the <a href="https://raw.githubusercontent.com/linuxserver/docker-deluge/58ce900bc33b06d3c9cec24e7f17ac9d8b4433cf/Dockerfile">ports exposed by the <code>deluge</code> container</a>! Indeed, since the <code>deluge</code> container's network flows through the <code>wireguard</code> container, we can only access the <code>deluge</code> container through the <code>wireguard</code> container's network.</p>
<p>By the way, port <code>8112</code> is used for the <a href="https://deluge.readthedocs.io/en/latest/reference/web.html">Deluge WebUI</a> and port <code>58846</code> is used by <a href="https://dev.deluge-torrent.org/wiki/UserGuide/ThinClient">Deluge Thin Clients</a>. Your BitTorrent client of choice will most likely use different ports!</p>
<pre data-lang="yml" style="background-color:#212733;color:#ccc9c2;" class="language-yml "><code class="language-yml" data-lang="yml"><span> </span><span style="color:#73d0ff;">network_mode</span><span style="color:#ccc9c2cc;">: </span><span style="color:#bae67e;">service:wireguard
</span></code></pre>
<p>The trick that makes it all work: make sure that the <code>deluge</code> container connects to the internet through the <code>wireguard</code> container.</p>
<h3 id="wg0.conf"><code>wg0.conf</code></h3>
<p>This Wireguard configuration file is based on the one provided by Mullvad, but with the <code>PostUp/PreDown</code> commands found in the <a href="https://web.archive.org/web/20210207170757/https://bookstack.almueti.com/books/wireguard/page/docker-compose-with-mullvad-wireguard-arbitrary-service">blog post mentioned earlier</a>.</p>
<pre data-lang="conf" style="background-color:#212733;color:#ccc9c2;" class="language-conf "><code class="language-conf" data-lang="conf"><span style="color:#ffa759;">[Interface]
</span><span style="color:#ffcc66;">PrivateKey </span><span style="color:#f29e74;">= <</span><span>private key</span><span style="color:#f29e74;">>
</span><span style="color:#ffcc66;">Address </span><span style="color:#f29e74;">= <</span><span>ip address</span><span style="color:#f29e74;">>
</span><span style="font-style:italic;color:#f29e74;">DNS </span><span style="color:#f29e74;">= <</span><span>ip address</span><span style="color:#f29e74;">>
</span><span>
</span><span style="color:#ffcc66;">PostUp </span><span style="color:#f29e74;">= </span><span style="font-style:italic;color:#f29e74;">DROUTE</span><span style="color:#f29e74;">=</span><span>$(ip route </span><span style="color:#f29e74;">|</span><span> grep default </span><span style="color:#f29e74;">|</span><span> awk </span><span style="color:#bae67e;">'{print $3}'</span><span>); </span><span style="font-style:italic;color:#f29e74;">HOMENET</span><span style="color:#f29e74;">=</span><span style="color:#ffcc66;">192.168.0.0/16</span><span>; </span><span style="font-style:italic;color:#f29e74;">HOMENET2</span><span style="color:#f29e74;">=</span><span style="color:#ffcc66;">10.0.0.0/8</span><span>; </span><span style="font-style:italic;color:#f29e74;">HOMENET3</span><span style="color:#f29e74;">=</span><span style="color:#ffcc66;">172.16.0.0/12</span><span>; ip route add </span><span style="color:#ffa759;">$HOMENET3</span><span> via </span><span style="color:#ffa759;">$DROUTE</span><span>;ip route add </span><span style="color:#ffa759;">$HOMENET2</span><span> via </span><span style="color:#ffa759;">$DROUTE</span><span>; ip route add </span><span style="color:#ffa759;">$HOMENET</span><span> via </span><span style="color:#ffa759;">$DROUTE</span><span>;iptables </span><span style="color:#f29e74;">-</span><span style="font-style:italic;color:#f29e74;">I OUTPUT </span><span style="color:#f29e74;">-</span><span>d </span><span style="color:#ffa759;">$HOMENET </span><span style="color:#f29e74;">-</span><span>j </span><span style="font-style:italic;color:#f29e74;">ACCEPT</span><span>;iptables </span><span style="color:#f29e74;">-</span><span style="font-style:italic;color:#f29e74;">A OUTPUT </span><span style="color:#f29e74;">-</span><span>d </span><span style="color:#ffa759;">$HOMENET2 </span><span style="color:#f29e74;">-</span><span>j </span><span style="font-style:italic;color:#f29e74;">ACCEPT</span><span>; iptables </span><span style="color:#f29e74;">-</span><span style="font-style:italic;color:#f29e74;">A OUTPUT </span><span style="color:#f29e74;">-</span><span>d </span><span style="color:#ffa759;">$HOMENET3 </span><span style="color:#f29e74;">-</span><span>j </span><span style="font-style:italic;color:#f29e74;">ACCEPT</span><span>; iptables </span><span style="color:#f29e74;">-</span><span style="font-style:italic;color:#f29e74;">A OUTPUT </span><span style="color:#f29e74;">! -</span><span>o </span><span style="color:#ffa759;">%i </span><span style="color:#f29e74;">-</span><span>m mark </span><span style="color:#f29e74;">! --</span><span>mark $(wg show </span><span style="color:#ffa759;">%i</span><span> fwmark) </span><span style="color:#f29e74;">-</span><span>m addrtype </span><span style="color:#f29e74;">! --</span><span>dst</span><span style="color:#f29e74;">-</span><span>type </span><span style="font-style:italic;color:#f29e74;">LOCAL </span><span style="color:#f29e74;">-</span><span>j </span><span style="font-style:italic;color:#f29e74;">REJECT
</span><span style="color:#ffcc66;">PreDown </span><span style="color:#f29e74;">= </span><span style="font-style:italic;color:#f29e74;">HOMENET</span><span style="color:#f29e74;">=</span><span style="color:#ffcc66;">192.168.0.0/16</span><span>; </span><span style="font-style:italic;color:#f29e74;">HOMENET2</span><span style="color:#f29e74;">=</span><span style="color:#ffcc66;">10.0.0.0/8</span><span>; </span><span style="font-style:italic;color:#f29e74;">HOMENET3</span><span style="color:#f29e74;">=</span><span style="color:#ffcc66;">172.16.0.0/12</span><span>; ip route del </span><span style="color:#ffa759;">$HOMENET3</span><span> via </span><span style="color:#ffa759;">$DROUTE</span><span>;ip route del </span><span style="color:#ffa759;">$HOMENET2</span><span> via </span><span style="color:#ffa759;">$DROUTE</span><span>; ip route del </span><span style="color:#ffa759;">$HOMENET</span><span> via </span><span style="color:#ffa759;">$DROUTE</span><span>; iptables </span><span style="color:#f29e74;">-</span><span style="font-style:italic;color:#f29e74;">D OUTPUT </span><span style="color:#f29e74;">! -</span><span>o </span><span style="color:#ffa759;">%i </span><span style="color:#f29e74;">-</span><span>m mark </span><span style="color:#f29e74;">! --</span><span>mark $(wg show </span><span style="color:#ffa759;">%i</span><span> fwmark) </span><span style="color:#f29e74;">-</span><span>m addrtype </span><span style="color:#f29e74;">! --</span><span>dst</span><span style="color:#f29e74;">-</span><span>type </span><span style="font-style:italic;color:#f29e74;">LOCAL </span><span style="color:#f29e74;">-</span><span>j </span><span style="font-style:italic;color:#f29e74;">REJECT</span><span>; iptables </span><span style="color:#f29e74;">-</span><span style="font-style:italic;color:#f29e74;">D OUTPUT </span><span style="color:#f29e74;">-</span><span>d </span><span style="color:#ffa759;">$HOMENET </span><span style="color:#f29e74;">-</span><span>j </span><span style="font-style:italic;color:#f29e74;">ACCEPT</span><span>; iptables </span><span style="color:#f29e74;">-</span><span style="font-style:italic;color:#f29e74;">D OUTPUT </span><span style="color:#f29e74;">-</span><span>d </span><span style="color:#ffa759;">$HOMENET2 </span><span style="color:#f29e74;">-</span><span>j </span><span style="font-style:italic;color:#f29e74;">ACCEPT</span><span>; iptables </span><span style="color:#f29e74;">-</span><span style="font-style:italic;color:#f29e74;">D OUTPUT </span><span style="color:#f29e74;">-</span><span>d </span><span style="color:#ffa759;">$HOMENET3 </span><span style="color:#f29e74;">-</span><span>j </span><span style="font-style:italic;color:#f29e74;">ACCEPT
</span><span>
</span><span style="color:#ffa759;">[Peer]
</span><span style="color:#ffcc66;">PublicKey </span><span style="color:#f29e74;">= <</span><span>public key</span><span style="color:#f29e74;">>
</span><span style="color:#ffcc66;">AllowedIPs </span><span style="color:#f29e74;">= </span><span style="color:#ffcc66;">0.0.0.0/0
</span><span style="color:#ffcc66;">Endpoint </span><span style="color:#f29e74;">= <</span><span>ip address </span><span style="color:#ffa759;">with</span><span> port</span><span style="color:#f29e74;">>
</span></code></pre>
<h2 id="Verification">Verification</h2>
<p>We need to make sure we are in fact connected safely to Mullvad! To do this, let's use Mullvad's <code>https://am.i.mullvad.net/connected</code> API endpoint.</p>
<pre data-lang="bash" style="background-color:#212733;color:#ccc9c2;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#ffd580;">docker</span><span> exec</span><span style="color:#ffcc66;"> -t</span><span> wireguard curl https://am.i.mullvad.net/connected
</span><span style="font-style:italic;color:#5c6773;"># You are connected to Mullvad (server XXYY-wireguard). Your IP address is XYZ.XYZ.XYZ.XYZ
</span></code></pre>
<p>Success! But wait, that's the <code>wireguard</code> container, this just checks whether our config is working. What about the <code>deluge</code> container?</p>
<pre data-lang="bash" style="background-color:#212733;color:#ccc9c2;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#ffd580;">docker</span><span> exec</span><span style="color:#ffcc66;"> -t</span><span> deluge curl https://am.i.mullvad.net/connected
</span><span style="font-style:italic;color:#5c6773;"># You are connected to Mullvad (server XXYY-wireguard). Your IP address is XYZ.XYZ.XYZ.XYZ
</span></code></pre>
<p>Victory! Have fun sharing Linux distributions!</p>