Jekyll2023-11-07T21:40:43+00:00https://vercas.com/feed.xmlVercas.comNot a good source of randomnessZeroTier Hooks on Linux2023-10-28T00:00:00+00:002023-10-28T00:00:00+00:00https://vercas.com/2023/10/28/zerotier-hooks-linux<p>ZeroTier on Linux lacks any facilities for running hooks when it connects to or disconnects from a network.<br />
Other VPN software provides this feature so the user can do things like setting up and tearing down firewall rules.<br />
Luckily, systemd provides a trivial way to solve this problem by starting and stopping a service when a device shows up or disappears.</p>
<hr />
<h1 id="the-code">The Code</h1>
<p>The following <code class="language-plaintext highlighter-rouge">systemd</code> unit should be placed in a file like <code class="language-plaintext highlighter-rouge">/etc/systemd/system/network-hooks@.service</code></p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="pi">[</span><span class="nv">Unit</span><span class="pi">]</span>
<span class="s">Description=Hooks for Network Interfaces</span>
<span class="s">BindsTo=sys-subsystem-net-devices-%i.device</span>
<span class="s">After=sys-subsystem-net-devices-%i.device</span>
<span class="pi">[</span><span class="nv">Service</span><span class="pi">]</span>
<span class="s">Type=oneshot</span>
<span class="s">ExecStartPre=-/usr/local/lib/%i.down</span>
<span class="s">ExecStart=/usr/local/lib/%i.up</span>
<span class="s">ExecStop=/usr/local/lib/%i.down</span>
<span class="s">RemainAfterExit=yes</span>
<span class="pi">[</span><span class="nv">Install</span><span class="pi">]</span>
<span class="s">WantedBy=sys-subsystem-net-devices-%i.device</span>
</code></pre></div></div>
<p>An example “up” hook for an example <code class="language-plaintext highlighter-rouge">ztFooBar</code> interface for a network with subnet <code class="language-plaintext highlighter-rouge">10.11.12.0/24</code> would be placed in <code class="language-plaintext highlighter-rouge">/usr/local/lib/ztFooBar.up</code> and look like:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/usr/bin/env bash</span>
<span class="c"># This forwards TCP and UDP traffic on port 1234 to 10.11.12.13:1234. It's assumed the forwardee is in the ZeroTier network.</span>
iptables <span class="nt">-t</span> nat <span class="nt">-I</span> PREROUTING 1 <span class="nt">-i</span> eno1 <span class="nt">-p</span> tcp <span class="nt">--dport</span> 1234 <span class="nt">-j</span> DNAT <span class="nt">--to-destination</span> 10.11.12.13:1234
iptables <span class="nt">-t</span> nat <span class="nt">-I</span> PREROUTING 1 <span class="nt">-i</span> eno1 <span class="nt">-p</span> udp <span class="nt">--dport</span> 1234 <span class="nt">-j</span> DNAT <span class="nt">--to-destination</span> 10.11.12.13:1234
<span class="c"># These rules are placed in the DOCKER-USER chain, which is where forwarding rules need to go if you have Docker installed. Without Docker installed, these would go in the FORWARD chain instead.</span>
iptables <span class="nt">-I</span> DOCKER-USER 1 <span class="nt">-i</span> eno1 <span class="nt">-p</span> tcp <span class="nt">-d</span> 10.11.12.13 <span class="nt">--dport</span> 1234 <span class="nt">-m</span> state <span class="nt">--state</span> NEW,ESTABLISHED,RELATED <span class="nt">-j</span> ACCEPT
iptables <span class="nt">-I</span> DOCKER-USER 1 <span class="nt">-i</span> eno1 <span class="nt">-p</span> udp <span class="nt">-d</span> 10.11.12.13 <span class="nt">--dport</span> 1234 <span class="nt">-m</span> state <span class="nt">--state</span> NEW,ESTABLISHED,RELATED <span class="nt">-j</span> ACCEPT
<span class="c"># This allows all traffic to be forwarded from the ZeroTier network into eno1</span>
iptables <span class="nt">-I</span> DOCKER-USER 1 <span class="nt">-i</span> ztFooBar <span class="nt">-o</span> eno1 <span class="nt">-m</span> state <span class="nt">--state</span> NEW,ESTABLISHED,RELATED <span class="nt">-j</span> ACCEPT
<span class="c"># This allows established and related packets to flow from eno1 into ZeroTier, so packets in the other direction belonging to connections allowed by the rule above can reach the right host.</span>
iptables <span class="nt">-I</span> DOCKER-USER 1 <span class="nt">-i</span> eno1 <span class="nt">-o</span> ztFooBar <span class="nt">-m</span> state <span class="nt">--state</span> ESTABLISHED,RELATED <span class="nt">-j</span> ACCEPT
<span class="c"># This changes the source IP address of the traffic leaving through eno1, forwarded from the ZeroTier network.</span>
iptables <span class="nt">-t</span> nat <span class="nt">-I</span> POSTROUTING 1 <span class="nt">-o</span> eno1 <span class="nt">-s</span> 10.11.12.0/24 <span class="nt">-j</span> MASQUERADE
</code></pre></div></div>
<p>The corresponding <code class="language-plaintext highlighter-rouge">/usr/local/lib/ztFooBar.down</code> file would look like:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/usr/bin/env bash</span>
iptables <span class="nt">-D</span> DOCKER-USER <span class="nt">-i</span> eno1 <span class="nt">-o</span> ztFooBar <span class="nt">-m</span> state <span class="nt">--state</span> ESTABLISHED,RELATED <span class="nt">-j</span> ACCEPT
iptables <span class="nt">-D</span> DOCKER-USER <span class="nt">-i</span> ztFooBar <span class="nt">-o</span> eno1 <span class="nt">-m</span> state <span class="nt">--state</span> NEW,ESTABLISHED,RELATED <span class="nt">-j</span> ACCEPT
iptables <span class="nt">-t</span> nat <span class="nt">-D</span> POSTROUTING <span class="nt">-o</span> eno1 <span class="nt">-s</span> 10.11.12.0/24 <span class="nt">-j</span> MASQUERADE
iptables <span class="nt">-t</span> nat <span class="nt">-D</span> PREROUTING <span class="nt">-i</span> eno1 <span class="nt">-p</span> tcp <span class="nt">--dport</span> 1234 <span class="nt">-j</span> DNAT <span class="nt">--to-destination</span> 10.11.12.13:1234
iptables <span class="nt">-t</span> nat <span class="nt">-D</span> PREROUTING <span class="nt">-i</span> eno1 <span class="nt">-p</span> udp <span class="nt">--dport</span> 1234 <span class="nt">-j</span> DNAT <span class="nt">--to-destination</span> 10.11.12.13:1234
iptables <span class="nt">-D</span> DOCKER-USER <span class="nt">-i</span> eno1 <span class="nt">-p</span> tcp <span class="nt">-d</span> 10.11.12.13 <span class="nt">--dport</span> 6881 <span class="nt">-m</span> state <span class="nt">--state</span> NEW,ESTABLISHED,RELATED <span class="nt">-j</span> ACCEPT
iptables <span class="nt">-D</span> DOCKER-USER <span class="nt">-i</span> eno1 <span class="nt">-p</span> udp <span class="nt">-d</span> 10.11.12.13 <span class="nt">--dport</span> 6881 <span class="nt">-m</span> state <span class="nt">--state</span> NEW,ESTABLISHED,RELATED <span class="nt">-j</span> ACCEPT
<span class="nb">exit </span>0
</code></pre></div></div>
<p>It’s best if the <code class="language-plaintext highlighter-rouge">down</code> hook is idempotent.</p>
<p>You can enable hooks for the example network interface by running this command:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>systemctl <span class="nb">enable </span>network-hooks@ztFooBar.service
</code></pre></div></div>ZeroTier on Linux lacks any facilities for running hooks when it connects to or disconnects from a network. Other VPN software provides this feature so the user can do things like setting up and tearing down firewall rules. Luckily, systemd provides a trivial way to solve this problem by starting and stopping a service when a device shows up or disappears.Running The Lounge Torified with Docker2021-02-22T00:00:00+00:002021-02-22T00:00:00+00:00https://vercas.com/2021/02/22/thelounge-docker<p>For half a decade I’ve been using a headless Quassel Core as my IRC client, but its shortcomings have become too much to bear.<br />
I’ve looked for better alternatives that can meet my requirements (that have changed since I’ve started using Quassel), and the closest to my needs seems to be The Lounge.<br />
Sadly, it <del>is</del>was missing one very important feature: the ability to connect through a SOCKS5 proxy.<br />
In this post I show how I’ve overcome this limitation, and explain how and why.</p>
<hr />
<h2 id="reasoning">Reasoning</h2>
<p>Allow me to start with a rant.<br />
Quassel is incredible. A lot of features packed into one application.<br />
One outdated, slow, and bloated application.<br />
So many features yet no automatic cleaning of message backlogs.<br />
After years of usage, it renders some operations so incredibly slow, that doing them blocks the daemon for so long, the client loses connection to the daemon, and the daemon loses connection to every IRC server.<br />
It literally hasn’t been updated since I’ve started using it, and it’s by no means stable or bug-free.</p>
<p>It worked really well for me because I was running it on a rented dedicated server. I didn’t have servers at home back then.<br />
I was running the Quassel client on Windows, where it worked really well and it looked well.<br />
But then I’ve also started using the client on Linux, where the experience is significantly worse.<br />
It looks awful no matter what I do. Its integration with system themes is plainly and simply hostile to the eye.<br />
It never remembers the layout of the interface, it gets reset to absurd values every time I start the client. Channel list and channel member list are unreadable because they are too small.<br />
Timestamps in the message list are cut off and the right side fades off when this happens, making less than half the timestamp actually readable. Nicknames are often cut off too, and also faded…</p>
<p>Also the inability to make link previews in Quassel use a proxy was a major deal-breaker. <strong>Just by accidentally moving the mouse over a link, it makes Quassel access it for generating the preview.</strong><br />
<strong>This is a major privacy issue.</strong><br />
Unacceptable.
(this can be fixed by torifying the client, but then you need to connect to your Core via Tor as well)</p>
<p>As replacement, I’ve been looking specifically for either web-based options first, and terminal-based options second.<br />
The terminal-based clients (many of which are headless) have all the features I could possibly want, except for the basic QoL ones: great looks, link previews, and notifications.<br />
Why web-based or terminal-based specifically? Well, because they are the most portable interfaces amongst my platforms of choice.</p>
<p>In the end, I was left with two clients that had feature parity (relative to what I need): IRCCloud and The Lounge.<br />
I’ve decided to go with The Lounge because it’s self-hosted, putting me in control of my data, and because it simply looks nicer.</p>
<p>As I’ve said in the first paragraph, its only shortcoming was the inability to connect to an IRC server via proxy. I have solved this by forcing all of the traffic through a transparent Tor proxy.<br />
Tor’s SOCKS5 proxy is the only proxy I wanted to connect to anyway.</p>
<h2 id="show-and-tell">Show and Tell</h2>
<p>First, here’s a <code class="language-plaintext highlighter-rouge">docker-compose.yml</code> file you can look at:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">version</span><span class="pi">:</span> <span class="s2">"</span><span class="s">3.8"</span>
<span class="na">services</span><span class="pi">:</span>
<span class="na">tor-router</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">tor-router</span>
<span class="na">container_name</span><span class="pi">:</span> <span class="s">tor-router</span>
<span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
<span class="na">cap_add</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">NET_ADMIN</span>
<span class="na">dns</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">127.0.0.1</span>
<span class="na">npm</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">jc21/nginx-proxy-manager:latest</span>
<span class="na">container_name</span><span class="pi">:</span> <span class="s">npm</span>
<span class="na">restart</span><span class="pi">:</span> <span class="s">unless-stopped</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">80:80"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">443:443"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">81:81"</span>
<span class="na">environment</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">DB_MYSQL_HOST=npm-db</span>
<span class="pi">-</span> <span class="s">DB_MYSQL_PORT=3306</span>
<span class="pi">-</span> <span class="s">DB_MYSQL_USER=npm</span>
<span class="pi">-</span> <span class="s">DB_MYSQL_PASSWORD=npm</span>
<span class="pi">-</span> <span class="s">DB_MYSQL_NAME=npm</span>
<span class="pi">-</span> <span class="s">DISABLE_IPV6=true</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">/srv/npm/data:/data</span>
<span class="pi">-</span> <span class="s">/srv/npm/certs:/certs</span>
<span class="pi">-</span> <span class="s">/srv/npm/letsencrypt:/etc/letsencrypt</span>
<span class="na">depends_on</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">npm-db</span>
<span class="na">npm-db</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">jc21/mariadb-aria:10.4</span>
<span class="na">container_name</span><span class="pi">:</span> <span class="s">npm-db</span>
<span class="na">restart</span><span class="pi">:</span> <span class="s">unless-stopped</span>
<span class="na">environment</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">MYSQL_ROOT_PASSWORD=npm</span>
<span class="pi">-</span> <span class="s">MYSQL_DATABASE=npm</span>
<span class="pi">-</span> <span class="s">MYSQL_USER=npm</span>
<span class="pi">-</span> <span class="s">MYSQL_PASSWORD=npm</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">/srv/npm/mysql:/var/lib/mysql</span>
<span class="na">thelounge</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">ghcr.io/linuxserver/thelounge</span>
<span class="na">container_name</span><span class="pi">:</span> <span class="s">thelounge</span>
<span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
<span class="na">network_mode</span><span class="pi">:</span> <span class="s">service:tor-router</span>
<span class="na">environment</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">PUID=9002</span>
<span class="pi">-</span> <span class="s">PGID=9002</span>
<span class="pi">-</span> <span class="s">TZ=Europe/London</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">/srv/thelounge:/config</span>
<span class="na">depends_on</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">tor-router</span>
<span class="na">socat-thelounge-in-tor</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">alpine/socat</span>
<span class="na">container_name</span><span class="pi">:</span> <span class="s">socat-thelounge-in-tor</span>
<span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
<span class="na">network_mode</span><span class="pi">:</span> <span class="s">service:tor-router</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">/srv/.sockets:/sockets</span>
<span class="na">depends_on</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">tor-router</span>
<span class="na">command</span><span class="pi">:</span> <span class="s">UNIX-LISTEN:/sockets/thelounge,fork,mode=700 TCP:127.0.0.1:9000</span>
<span class="na">socat-thelounge-out</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">alpine/socat</span>
<span class="na">container_name</span><span class="pi">:</span> <span class="s">socat-thelounge-out</span>
<span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">/srv/.sockets:/sockets</span>
<span class="na">command</span><span class="pi">:</span> <span class="s">TCP-LISTEN:80,fork UNIX-CONNECT:/sockets/thelounge</span>
</code></pre></div></div>
<p>This might look unholy to someone who doesn’t quite know what is going on.<br />
NodeProxyManager is there to provide SSL (e.g. via Let’s Encrypt) to The Lounge. This is required for push notifications, and for registering the <code class="language-plaintext highlighter-rouge">irc://</code> and <code class="language-plaintext highlighter-rouge">ircs://</code> URI handlers.<br />
<code class="language-plaintext highlighter-rouge">socat</code> is necessary to access The Lounge across the different network namespaces of the containers. It uses a Unix domain socket, simply.<br />
Set up NPM to forward traffic to hostname <code class="language-plaintext highlighter-rouge">socat-thelounge-out</code> port 80, enable websocket support. Use Let’s Encrypt for SSL. Boom, quick maths.<br />
Keep in mind your hostname (e.g. <code class="language-plaintext highlighter-rouge">thelounge.example.com</code>) needs to be accessible from the internet, and port 80 on your router needs to be forwarded to your Docker host, if you want to use Let’s Encrypt. You can use the features that require HTTPS with self-signed certificates too.</p>
<p>The magic here is in the <code class="language-plaintext highlighter-rouge">tor-router</code> image, which is a <em>slightly</em> altered version of <a href="https://github.com/flungo-docker/tor-router">this one</a>.<br />
Simply edit the Dockerfile and change the first line to:</p>
<div class="language-Dockerfile highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">FROM</span><span class="s"> alpine:latest</span>
</code></pre></div></div>
<p>You absolutely want to have the latest Tor, both to get all the security fixes, and to get the new features, such as hidden service protocol v3.<br />
After making the edit, build it like this:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker build <span class="nt">-t</span> tor-router src
</code></pre></div></div>
<p>Just change all the volumes to suit your filesystem structure, change the database passwords, and you can run it.<br />
<strong>DO NOT CHANGE THE UID/GID OF THELOUNGE TO 9001</strong>, see why down below.<br />
And you are done.</p>
<h2 id="details">Details</h2>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="na">network_mode</span><span class="pi">:</span> <span class="s">service:tor-router</span>
</code></pre></div></div>
<p>This tells Docker that a container is to be run in the network namespace of another (in this case, <code class="language-plaintext highlighter-rouge">tor-router</code>).<br />
This means it sees the same network interfaces, it can connect to all the listening ports (even if they are in another container), and it is subject to the same <code class="language-plaintext highlighter-rouge">iptables</code> rules.<br />
<a href="https://github.com/flungo-docker/tor-router/blob/master/src/iptables.rules">This file</a> in the <a href="https://github.com/flungo-docker/tor-router">tor-router repo</a> is where the magic happens, and the author has generously documented what each rule does.<br />
Brief overview: all TCP and DNS (UDP port 53) traffic is forwarded to a specific port that the Tor daemon listens to for transparent proxying, and only the Tor daemon (well, anything running with UID 9001) can send traffic to anything else.<br />
As long as you don’t use UID 9001 for a torified service, and you make sure nothing <em>could</em> do that without your knowledge, it will work perfectly fine.</p>
<h2 id="bottom-line">Bottom Line</h2>
<p>I am quite happy with this setup so far. It just works! (famous last words, I know)</p>For half a decade I’ve been using a headless Quassel Core as my IRC client, but its shortcomings have become too much to bear. I’ve looked for better alternatives that can meet my requirements (that have changed since I’ve started using Quassel), and the closest to my needs seems to be The Lounge. Sadly, it iswas missing one very important feature: the ability to connect through a SOCKS5 proxy. In this post I show how I’ve overcome this limitation, and explain how and why.OpenWrt apu4c4 Router - OS & Software2020-01-18T00:00:00+00:002020-01-18T00:00:00+00:00https://vercas.com/2020/01/18/apu4c4-openwrt-2<p>This is a continuation to my previous post about the assembly of hardware for my <code class="language-plaintext highlighter-rouge">apu4c4</code> OpenWrt router.<br />
Here I will discuss a bit about my journey with the software side, builds, configurations, drivers, various packages.</p>
<hr />
<p>The first thing I tried was, of course, the default build for AMD64 generic targets.<br />
Writing it to the SSD was trivial, just used <code class="language-plaintext highlighter-rouge">dd</code>.<br />
However, when booting it, after a bit of trial and error, I realized that its default serial console is configured with a baud rate of 38,400 or so - unlike the SeaBIOS which uses 115,200.<br />
I found this rather puzzling because the default baud rate in the OpenWrt build system is, in fact, 115,200…</p>
<p>This image has a few problems:</p>
<ul>
<li>It’s missing the <code class="language-plaintext highlighter-rouge">igb</code> driver required for the NICs on the <code class="language-plaintext highlighter-rouge">apu4c4</code>.</li>
<li>The kernel is built without the features required for some packages, such as LXC.</li>
<li>Funky serial port settings.</li>
<li>Tiny rootfs size (I’ve got plenty of room on my SSD).<br />
Therefore, I had no doubt that I need to build OpenWrt myself.<br />
This turned out to be quite easy, because the build system automatically fetches all its dependencies! Kudos to the developers for that.</li>
</ul>
<p>One small detail wasn’t clear to me, though. They use the Kbuild system, used by the Linux kernel.<br />
Therefore, a lot of configuration options are actually tri-state: <code class="language-plaintext highlighter-rouge">Y</code>, <code class="language-plaintext highlighter-rouge">N</code>, or <code class="language-plaintext highlighter-rouge">M</code>.<br />
Normally in Linux <code class="language-plaintext highlighter-rouge">Y</code> means <em>compile this module into the kernel</em>, <code class="language-plaintext highlighter-rouge">N</code> means <em>do not compile this module</em>, and <code class="language-plaintext highlighter-rouge">M</code> means <em>compile this module as a separate executable</em> (to be later put into the initrd/initramfs).<br />
With OpenWrt, it wasn’t so obvious to me at first. For packages (<strong>including kernel modules/drivers</strong>), <code class="language-plaintext highlighter-rouge">Y</code> means <em>compile into a package and include in the resulted image</em>. It does <em>not</em> compile modules into the kernel itself.<br />
<code class="language-plaintext highlighter-rouge">N</code> means <em>do not build</em> and <code class="language-plaintext highlighter-rouge">M</code> means <em>compile</em>… And that’s it.<br />
At first I assumed <code class="language-plaintext highlighter-rouge">Y</code> means <em>compile and install</em> and <code class="language-plaintext highlighter-rouge">M</code> means <em>compile and package in image, but do not install</em>. I was very wrong.</p>
<p>Another thing that wasn’t obvious to me, was that the build system doesn’t actually have all the packages by default.<br />
If you want to build OpenWrt, you definitely need to read the instructions on their website. I didn’t at first, because I am a dum dum.<br />
There is a script you need to execute in order to add the rest of the package sources to the build system.<br />
This will substantially increase the number of options available, which might be extremely overwhelming for a beginner. It also makes it very easy to hoard packages…<br />
The kernel modules and drivers + a few basic things that are selected by default anyway are really the only packages you need to worry about. You can use <code class="language-plaintext highlighter-rouge">opkg</code> to get the rest after you log into OpenWrt - except kernel modules.<br />
Just remember to grab the ones mentioned <a href="https://openwrt.org/toh/pcengines/apu2#kernel_modules">on the OpenWrt wiki</a> as well.</p>
<p>Once you boot it with the right drivers, it will create two interfaces by default: <code class="language-plaintext highlighter-rouge">lan</code> and <code class="language-plaintext highlighter-rouge">wan</code>.<br />
<code class="language-plaintext highlighter-rouge">eth0</code> was the LAN and <code class="language-plaintext highlighter-rouge">eth1</code> was the WAN for me. I swapped them. You don’t really need to do this.<br />
As stated in the previous post, <code class="language-plaintext highlighter-rouge">eth0</code> is the RJ-45 port next to the serial port. <code class="language-plaintext highlighter-rouge">eth1</code> is the one adjacent to <code class="language-plaintext highlighter-rouge">eth0</code>.</p>
<p>The SSH server used by default in OpenWrt is Dropbear, which seems to be a-okay. I might replace this later if I need more features.<br />
If I remember correctly, it’s not enabled by default. I highly recommend using public key authentication here instead of password authentication.</p>
<p>LuCI, the default web front-end to OpenWrt, is absolutely amazing. You can use it to configure pretty much anything and it majorly improves the experience.<br />
I know y’all think using command-line interfaces makes you look cool, but LuCI provides you with excellent overviews of your configuration, as well as names and descriptions of all the options you can (or cannot) configure for a specific system/item. It even shows default values!<br />
<code class="language-plaintext highlighter-rouge">opkg install luci</code> should be all you need. You can use LuCI itself to install more packages. I personally use the Material theme, it is quite smexy and it comes with a few nice features such as auto-refreshing certain pages.</p>
<p>If you want to try out my build configuration (tailored to my hardware specifically), then you can find it <a href="TODO">here</a>.<br />
Just put this into <code class="language-plaintext highlighter-rouge">.config</code> in your OpenWrt local repo. <strong>If you are using my config, please read the rest of the information below.</strong></p>
<h2 id="experience-so-far">Experience so far</h2>
<p>I’ve experimented with quite a few features and packages. Here are some of my findings.</p>
<h3 id="wireguard">WireGuard</h3>
<p>This one performs incredibly well. I can saturate a 1 Gbps link with WireGuard traffic and it only pushes <strong>one</strong> CPU core (when transmitting <em>or</em> receiving) to <60% usage, the rest hover around 10%.<br />
The LuCI app for this is quite janky:</p>
<ul>
<li>It won’t tell you if you have an invalid character in the interface name. <code class="language-plaintext highlighter-rouge">-</code> is not a valid character apparently.</li>
<li>When you make modifications and save the settings, it commits the changes but doesn’t actually apply them. It fails to add new peers and you need to restart the <code class="language-plaintext highlighter-rouge">network</code> service manually.</li>
<li>The status page is difficult to read because it’s chaotic and extremely unpolished. The icons look so faded, one might think they’re a placeholder for an image that failed to load.
To use WireGuard, you will need to have enabled quite a few crypto modules, and some of them come with AVX2 variants by default. Remove them with <code class="language-plaintext highlighter-rouge">sed -i '/avx2/d' /mnt/openwrt-root/etc/modules.d/10-crypto-misc</code>.</li>
</ul>
<h3 id="ath10k--firmware">ath10k & firmware</h3>
<p>The <strong>vendor</strong> firmware and driver work fine.<br />
The community-supported ones, though, don’t. (these are the <code class="language-plaintext highlighter-rouge">ath10k-firmware-qca988x-ct</code> and <code class="language-plaintext highlighter-rouge">kmod-ath10k-ct</code> packages)<br />
Some devices were getting kicked out of the network randomly and my router had to be restarted every 3-4 days when using the community firmware and driver.<br />
I was also getting continuous spam in my system log about some queue being full, coming from the driver.</p>
<p>YMMV but I recommend staying away from the community packages if you have my specific hardware.</p>
<h3 id="lxc">LXC</h3>
<p>This one is a PITA to use and I couldn’t get it to start containers properly.<br />
As far as I can tell, it doesn’t run <code class="language-plaintext highlighter-rouge">/sbin/init</code> in the containers at all.<br />
You can’t do anything in them like this. Also making networking work is a chore. Default config comes with no networking support.<br />
What even is the point of containers without networking?</p>
<h3 id="docker">Docker</h3>
<p>This isn’t packaged for OpenWrt but I tried to install it anyway.<br />
<strong>Don’t do this.</strong></p>
<p>Docker sets up some bizzare firewall rules that will break everything.<br />
What happened to me was that, suddenly, I could route traffic from any zone to any other zone - this is extremely bad.</p>
<p>If I could get LXC to work, I would try to run Docker inside LXC…</p>
<h3 id="filesystem-and-block-devices">Filesystem and Block Devices</h3>
<p>Grab the <code class="language-plaintext highlighter-rouge">block-mount</code> package and you can configure filesystems and swap devices/files in <code class="language-plaintext highlighter-rouge">/etc/config/fstab</code>.</p>
<p>If you want to use OpenWrt’s ability to overlay a filesystem on top of your root, then you need to mount something to <code class="language-plaintext highlighter-rouge">/overlay</code> and the rest will be taken care of for you.</p>
<p>Note that, if you want to do this automatically via <code class="language-plaintext highlighter-rouge">/etc/config/fstab</code>, you need to put that file in the rootfs, not the <code class="language-plaintext highlighter-rouge">/overlay/upper/etc/config/fstab</code> file.<br />
As far as I can tell, every other config file can sit in the overlay.</p>
<h3 id="adblock">Adblock</h3>
<p>Makes YouTube videos load extremely slowly at the beginning.</p>
<h3 id="btrfs">BTRFS</h3>
<p>Way too slow in this kernel version, file I/O will grind to a halt.<br />
Tried to use it mainly for the sake of Docker, to use its snapshots.</p>This is a continuation to my previous post about the assembly of hardware for my apu4c4 OpenWrt router. Here I will discuss a bit about my journey with the software side, builds, configurations, drivers, various packages.WireGuard on Pinebook Pro2019-11-05T00:00:00+00:002019-11-05T00:00:00+00:00https://vercas.com/2019/11/05/wireguard-pbp<p>WireGuard is the new and hip VPN protocol that all the cool kids are using these days.<br />
It’s only natural that I want to use it as well, and the only client I really need is my Pinebook Pro.<br />
As of right now (November 1st 2019) when I’m writing this, it’s not trivial to make use of the <code class="language-plaintext highlighter-rouge">wireguard-dkms</code> package on the PBP.<br />
Luckily, alternatives exist, and I will explain what and how.</p>
<p>Now, WireGuard does have official implementations in userland, that should be far easier to use, but right now there’s some issues…<br />
First one is written in Go, and it performs awfully.<br />
Second one is written in Rust, and it’s very new and lacking in features.</p>
<p>However, the good folks at Cloudflare have written their own implementation in Rust: <a href="https://github.com/cloudflare/boringtun">BoringTun</a><br />
This seems to have all the features I want/need. I literally don’t know of any feature that’s missing.</p>
<h2 id="compiling-on-pbp">Compiling on PBP</h2>
<p>Ideally this would be as easy to install as <code class="language-plaintext highlighter-rouge">cargo install boringtun</code>, but it’s not…<br />
Sadly, the versions of <code class="language-plaintext highlighter-rouge">cargo</code> and <code class="language-plaintext highlighter-rouge">rustc</code> in Debian are too old to compile (at least) a dependency.<br />
We need a more recent compiler.</p>
<p>The first option I tried was, of course, installing latest Rust on the PBP.<br />
The script they offer on their website is not going to work. It uses <code class="language-plaintext highlighter-rouge">uname -a</code> to find the current architecture - which fails.<br />
The issue here is that <code class="language-plaintext highlighter-rouge">uname -a</code> reports the kernel’s architecture, which is AArch64 on the PBP.<br />
But the userland is 32-bit, therefore <code class="language-plaintext highlighter-rouge">armhf</code>.<br />
The right compiler should be <a href="https://static.rust-lang.org/rustup/dist//armv7-unknown-linux-gnueabihf/rust-init">armv7-unknown-linux-gnueabihf</a>. This link is the script that installs the compiler.</p>
<p>But once again, this doesn’t work.<br />
Every time I run <code class="language-plaintext highlighter-rouge">cargo</code> it says:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>error: command failed: 'cargo'
error: caused by: No such file or directory (os error 2)
</code></pre></div></div>
<p>And I don’t want to waste my time fixing this one, because there is a far better solution!</p>
<h2 id="cross-compiling">Cross-compiling</h2>
<p>Rust makes it ezpz to cross-compile so this is exactly what we’re going to do.<br />
First, install Rust using the instructions on their website. It just works.</p>
<p>Then you need to grab a toolchain for our target. On Ubuntu 18.04 you’d use:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>apt <span class="nb">install </span>gcc-arm-linux-gnueabihf
</code></pre></div></div>
<p>And then grab the tools to compile Rust for your target architecture:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>rustup target add armv7-unknown-linux-gnueabihf
</code></pre></div></div>
<p>You’re almost done! The code will now compile <strong>but</strong> it will fail to link. To fix this, you need this config file:<br />
<code class="language-plaintext highlighter-rouge">$HOME/.cargo/config</code></p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[target.armv7-unknown-linux-gnueabihf]
linker = "arm-linux-gnueabihf-gcc"
</code></pre></div></div>
<p>Now, in the BoringTun directory (that I assume you checked out locally on an x86 machine or something), do the build:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cargo build <span class="nt">--bin</span> boringtun <span class="nt">--release</span> <span class="nt">--target</span> armv7-unknown-linux-gnueabihf
</code></pre></div></div>
<p>Your result is <code class="language-plaintext highlighter-rouge">target/armv7-unknown-linux-gnueabihf/release/boringtun</code>, and this is the only file you need to copy over to your Pinebook Pro.</p>
<h2 id="wireguard-setup">WireGuard Setup</h2>
<p>WireGuard uses a peer-to-peer model and it is incredibly easy to set up compared to literally every other VPN protocol/software I’ve used before.<br />
However, without the kernel module, the easy way to set it up (<code class="language-plaintext highlighter-rouge">wg-quick</code>) doesn’t work.</p>
<p>Install the <a href="http://ftp.uk.debian.org/debian/pool/main/w/wireguard/wireguard-tools_0.0.20191012-1_armhf.deb">wireguard-tools</a> package (luckily available for armhf):</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cd</span> /tmp
wget http://ftp.uk.debian.org/debian/pool/main/w/wireguard/wireguard-tools_0.0.20191012-1_armhf.deb
<span class="nb">sudo </span>dpkg <span class="nt">-i</span> wireguard-tools_0.0.20191012-1_all.deb
</code></pre></div></div>
<p>The package might need a dependency: <code class="language-plaintext highlighter-rouge">sudo apt install libmnl0</code>.<br />
Feel free to use another Debian mirror.</p>
<p>I assume that you’ve set up the other peer(s) properly and you’re reasonably sure that they work.<br />
The way I’ve organized the files for my WireGuard instance is the following:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>/srv
└── wireguard
├── boringtun
└── wg0
├── check.sh
├── conf
├── err
├── log
├── start.sh
└── stop.sh
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">/srv/wireguard/boringtun</code> is just the executable built previously.<br />
<code class="language-plaintext highlighter-rouge">/srv/wireguard/wg0/{err,log}</code> are used by BoringTun.</p>
<p><code class="language-plaintext highlighter-rouge">/srv/wireguard/wg0/start.sh</code>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/sh</span>
<span class="nv">NAME</span><span class="o">=</span>wg0
<span class="nv">CMD</span><span class="o">=</span><span class="s2">"/srv/wireguard/boringtun </span><span class="nv">$NAME</span><span class="s2"> --err /srv/wireguard/</span><span class="nv">$NAME</span><span class="s2">/err --log /srv/wireguard/</span><span class="nv">$NAME</span><span class="s2">/log --disable-drop-privileges 1"</span>
wg show <span class="nv">$NAME</span> <span class="o">></span> /dev/null 2> /dev/null
<span class="k">if</span> <span class="o">[</span> <span class="nv">$?</span> <span class="nt">-eq</span> 0 <span class="o">]</span><span class="p">;</span> <span class="k">then
</span><span class="nb">echo</span> <span class="s2">"Stopping existing instance."</span> 1>&2
ip <span class="nb">link set</span> <span class="nv">$NAME</span> down
pkill <span class="nt">-f</span> <span class="s2">"</span><span class="k">${</span><span class="nv">CMD</span><span class="k">}</span><span class="s2">"</span>
<span class="nb">sleep </span>0.3
<span class="k">fi</span>
<span class="nv">$CMD</span>
wg setconf <span class="nv">$NAME</span> /srv/wireguard/<span class="nv">$NAME</span>/conf
ip addr add 192.168.254.2/24 dev <span class="nv">$NAME</span>
ip <span class="nb">link set</span> <span class="nv">$NAME</span> up
</code></pre></div></div>
<p>Here <code class="language-plaintext highlighter-rouge">192.168.254.2/24</code> should be the IP you want within your WireGuard subnet.<br />
If you want to add access to more IP ranges (e.g. the <em>local</em> network in your home), add <code class="language-plaintext highlighter-rouge">ip route add 192.168.1.0/24 via 192.168.254.1 dev $NAME</code> where <code class="language-plaintext highlighter-rouge">192.168.1.0/24</code> is your subnet and <code class="language-plaintext highlighter-rouge">192.168.254.1</code> is the peer that will be your gateway.</p>
<p><code class="language-plaintext highlighter-rouge">srv/wireguard/wg0/stop.sh</code>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/sh</span>
<span class="nv">NAME</span><span class="o">=</span>wg0
<span class="nv">CMD</span><span class="o">=</span><span class="s2">"/srv/wireguard/boringtun </span><span class="nv">$NAME</span><span class="s2"> --err /srv/wireguard/</span><span class="nv">$NAME</span><span class="s2">/err --log /srv/wireguard/</span><span class="nv">$NAME</span><span class="s2">/log --disable-drop-privileges 1"</span>
wg show <span class="nv">$NAME</span> <span class="o">></span> /dev/null 2> /dev/null
<span class="k">if</span> <span class="o">[</span> <span class="nv">$?</span> <span class="nt">-eq</span> 0 <span class="o">]</span><span class="p">;</span> <span class="k">then
</span>ip <span class="nb">link set</span> <span class="nv">$NAME</span> down
pkill <span class="nt">-f</span> <span class="s2">"</span><span class="k">${</span><span class="nv">CMD</span><span class="k">}</span><span class="s2">"</span>
<span class="k">else
</span><span class="nb">echo</span> <span class="s2">"No instance to stop."</span> 1>&2
<span class="k">fi</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">srv/wireguard/wg0/check.sh</code>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/sh</span>
<span class="nv">NAME</span><span class="o">=</span>wg0
wg show <span class="nv">$NAME</span> <span class="o">></span> /dev/null 2> /dev/null
<span class="k">if</span> <span class="o">[</span> <span class="nv">$?</span> <span class="nt">-eq</span> 0 <span class="o">]</span><span class="p">;</span> <span class="k">then
</span><span class="nb">exit </span>0
<span class="k">else
</span><span class="nb">exit </span>1
<span class="k">fi</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">/srv/wireguard/wg0/conf</code>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[Interface]
PrivateKey = INSERT_PRIVATE_KEY_HERE
[Peer]
PublicKey = INSERT_PEER_PUBLIC_KEY_HERE
AllowedIPs = 192.168.254.0/24
Endpoint = HOST:PORT
PersistentKeepalive = 25
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">INSERT_PRIVATE_KEY_HERE</code> should be replaced with a private key, as generated by <code class="language-plaintext highlighter-rouge">wg genkey</code>.<br />
<code class="language-plaintext highlighter-rouge">INSERT_PEER_PUBLIC_KEY_HERE</code> should be the public key of your peer.<br />
<code class="language-plaintext highlighter-rouge">192.168.254.0/24</code> should be your VPN subnet. Append <code class="language-plaintext highlighter-rouge">, 192.168.1.0/24</code> after it if you also want LAN access, for example.<br />
<code class="language-plaintext highlighter-rouge">HOST:PORT</code> should be the host (IP or domain) and WireGuard port of the peer.</p>
<p>You can add as many peers as you need there. If you do, I assume you already know how to set them up.</p>
<p>Once you’ve started your interface with the script above, you can find your public key with the <code class="language-plaintext highlighter-rouge">wg show</code> command (or just <code class="language-plaintext highlighter-rouge">wg</code>):</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># wg
interface: wg1
public key: YOUR_PUBLIC_KEY_HERE
private key: (hidden)
listening port: 42069
peer: YOUR_PEER_PUBLIC_KEY_HERE
endpoint: HOST:PORT
allowed ips: 192.168.1.0/24, 192.168.254.0/24
latest handshake: 20 seconds ago
transfer: 10.66 KiB received, 10.15 KiB sent
persistent keepalive: every 25 seconds
</code></pre></div></div>
<h2 id="plumbing">Plumbing</h2>
<p>Naturally, I’d like to connect to WireGuard when I’m at work and disconnect when I am not.<br />
I don’t need to connect to WireGuard while I’m at home because my router will route traffic from LAN to the WireGuard subnet.</p>
<p>So I’ve written two scripts for this. Which are kind of untested, because I’m writing something far more complex in Lua to be able to handle wireless virtual interfaces.</p>
<p><code class="language-plaintext highlighter-rouge">/etc/network/if-up.d/wg0</code>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/bash</span>
<span class="k">if</span> <span class="o">[</span> <span class="s2">"</span><span class="k">${</span><span class="nv">IFACE</span><span class="k">}</span><span class="s2">"</span> <span class="o">==</span> <span class="s2">"wlan0"</span> <span class="nt">-a</span> <span class="s2">"</span><span class="k">${</span><span class="nv">ADDRFAM</span><span class="k">}</span><span class="s2">"</span> <span class="o">==</span> <span class="s2">"inet"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then
</span><span class="nv">SSID</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span>iwgetid <span class="s2">"</span><span class="k">${</span><span class="nv">IFACE</span><span class="k">}</span><span class="s2">"</span> <span class="nt">-r</span><span class="si">)</span><span class="s2">"</span>
<span class="k">case</span> <span class="s2">"</span><span class="k">${</span><span class="nv">SSID</span><span class="k">}</span><span class="s2">"</span> <span class="k">in
</span>work-network<span class="p">)</span>
/srv/wireguard/wg0/start.sh
<span class="p">;;</span>
<span class="k">*</span><span class="p">)</span>
<span class="c"># nothing</span>
<span class="p">;;</span>
<span class="k">esac</span>
<span class="k">fi</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">/etc/network/if-down.d/wg0</code>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/sh</span>
<span class="k">if</span> <span class="o">[</span> <span class="s2">"</span><span class="k">${</span><span class="nv">IFACE</span><span class="k">}</span><span class="s2">"</span> <span class="o">==</span> <span class="s2">"wlan0"</span> <span class="nt">-a</span> <span class="s2">"</span><span class="k">${</span><span class="nv">ADDRFAM</span><span class="k">}</span><span class="s2">"</span> <span class="o">==</span> <span class="s2">"inet"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then</span>
/srv/wireguard/wg0/stop.sh
<span class="k">fi</span>
</code></pre></div></div>
<h2 id="observations">Observations</h2>
<p>BoringTun fails to drop privileges for some reason. That’s why I added <code class="language-plaintext highlighter-rouge">--disable-drop-privileges 1</code>.<br />
No idea how to fix this.</p>
<p>I still haven’t set up DNS properly. I will solve this issue. Sometime.</p>
<p>If you connect to your home VPN from within the home network itself, it’s gonna be icky. Ergo the scripts above.</p>
<h2 id="performance">Performance</h2>
<p>Using <code class="language-plaintext highlighter-rouge">iperf</code> from my home network (the peer is my gateway’s WAN IP, not the LAN IP), it maxes out my upload speed.<br />
I get roughly 14 Mbps and BoringTun uses 25-40% CPU time of a Cortex-A53.<br />
This was from my PBP to one of the servers in my LAN.</p>
<p>However, if I change my settings to connect to my gateway’s LAN IP, and do <code class="language-plaintext highlighter-rouge">iperf</code> to the gateway itself over the WireGuard network, results are a bit different.<br />
I get 44 Mbps and BoringTun uses multiple processes (threads?) to do its work. None of the threads go above 40% or so usage.<br />
And I get 46 Mbps to my gateway over plain 802.11n</p>
<p>So, I am very satisfied with the performance and efficiency so far.</p>
<p>When idling, BoringTun uses no CPU time at all.</p>
<h2 id="sources">Sources</h2>
<p>First of all, there’s the <a href="https://www.wireguard.com/">WireGuard website</a> with quite a bit of information.<br />
And of course, the ArchWiki has <a href="https://wiki.archlinux.org/index.php/WireGuard">a page on WireGuard</a> as well.</p>WireGuard is the new and hip VPN protocol that all the cool kids are using these days. It’s only natural that I want to use it as well, and the only client I really need is my Pinebook Pro. As of right now (November 1st 2019) when I’m writing this, it’s not trivial to make use of the wireguard-dkms package on the PBP. Luckily, alternatives exist, and I will explain what and how.OpenWrt apu4c4 Router - Hardware Assembly2019-10-06T00:00:00+00:002019-10-06T00:00:00+00:00https://vercas.com/2019/10/06/apu4c4-openwrt-1<p>I’ve decided that I need to build my own router, because I cannot find anything on the market that meets my requirements.<br />
Anything that gets close is also very expensive, and my budget is rather limited.<br />
So I’ve ended up building my own using PC Engines <code class="language-plaintext highlighter-rouge">apu4c4</code> and OpenWrt.<br />
In this post, I will talk a bit about putting the hardware together and some of the decisions I’ve made along the way.</p>
<p>Before starting a build like this, there are some things that need to be done first.<br />
Most important one is building and flashing the OS, which in my case is OpenWrt.<br />
The “generic” OpenWrt profile for x86_64 comes with an extremely minimalistic kernel.<br />
So minimalistic, in fact, that it lacks the features needed for some OpenWrt packages.<br />
The image does not even contain the driver required to use the NICs, which is <code class="language-plaintext highlighter-rouge">igb</code>.</p>
<p>A way to connect to the serial port is going to be convenient. If you know how to set this up without serial port access, then you don’t need my help at all.<br />
On this note, the OpenWrt image for x86_64 is set up with a baud rate of 38,400…</p>
<p>I will talk about the challenges with software in another post.</p>
<p><img src="https://u.vercas.com/apu4c4-openwrt-build/IMG_0042.png" alt="All the parts" /></p>
<p>Right here are all the components for version 1 of my router (and extras).<br />
Version 2 will have more antenna holes in the case plus a 2.4 GHz radio, and version 3 will have an LTE modem.</p>
<p>The (only) case for <code class="language-plaintext highlighter-rouge">apu4c4</code> sold by PC Engines comes with only two SMA holes. My build will need 6, but right now I do not have the tools to drill holes.<br />
This isn’t so bad, though, because I need time to pick a good (and compatible) LTE modem.</p>
<p>I installed two pigtails in the case prior to taking a picture. I later learned the hard way that those need to be tightened.<br />
The silly shape of the SMA holes is just right for stripping threads…</p>
<p>Below I will explain my choices and show the items up close.</p>
<p><img src="https://u.vercas.com/apu4c4-openwrt-build/IMG_0043.png" alt="Wireless Module and SSD" /></p>
<p>The wireless adapter I chose is WLE900VX, which does 3x3 SU-MIMO 802.11ac/b/g/n dual band (2.4 GHz and 5 GHz), but <strong>single radio</strong>.<br />
What this means is that only one band can actually be used at the same time, which is alright for stations (client devices), but not really ideal for access points (routers).<br />
This is not a big deal for me, because 2.4 GHz radios are dirt cheap and I can just add another one <strong>via USB</strong>.<br />
This one will be set to 5 GHz mode because I didn’t just pay 40 quid for a glorified 2.4 GHz radio…<br />
In version 1 I will just use two antennas (CH0 and CH1). It works fine like that.<br />
This is 802.11ac Wave 1, therefore it has no MU-MIMO, only SU-MIMO. <em>ELI5: Can only talk to one station at a time.</em></p>
<p>So there are 802.11ac Wave 2 adapters out there. But I did not choose one, because:</p>
<ul>
<li>802.11ax (aka Wi-Fi 6) has just been finalized. I actually purchased the adapter before that but I knew it was coming.</li>
<li>Twice as expensive for the same bandwidth.</li>
<li>Not going to max out the bandwidth over Wi-Fi, ever.</li>
<li>Most of my devices can only do 2.4 GHz anyhow.</li>
</ul>
<p>I choose an SSD instead of just using a microSD card for a few reasons:</p>
<ul>
<li>120 GB SSD costs 22 quid, 32 GB Samsung PRO Endurance microSD card costs 25 quid.</li>
<li>I need space to make sure I can put in everything I need, e.g. a caching proxy.</li>
<li>Higher capacity means more blocks that can be used to level wear.</li>
<li>I’d like to use the SD slot for data transfers.</li>
<li>Not planning on using failover WWAN.</li>
</ul>
<p><img src="https://u.vercas.com/apu4c4-openwrt-build/IMG_0044.png" alt="Bits and Bobs" /></p>
<p>Screws, rubber pads, thermal pads, heat spreader.</p>
<p>In the bag where the heat spreader (chunk of aluminium) is, there’s also <strong>two</strong> thermal pads.<br />
The thermal pads have two films on them, one is clear and the other is blue.<br />
The two pads have their own blue film <strong>but the clear film is shared</strong>.</p>
<p>Also with the board came a bag with 6 screws for the mPCIe/mSATA slots. They’re gray and have some blue sealant on them.<br />
1.5mm Philips head.</p>
<p>The bag that came <em>inside</em> the case contains 4 gray screws with wide heads, 4 black screws, 4 rubber pads, and two SMA plugs.<br />
I used 2.5mm Philips head for these.</p>
<p><img src="https://u.vercas.com/apu4c4-openwrt-build/IMG_0045.png" alt="apu4c4 Board" /></p>
<p>The board in all its glory.</p>
<p>The mPCIe slots all have different connections and purposes:</p>
<ol>
<li>Left: mSATA <em>or</em> modem (USB + SIM)</li>
<li>Middle: modem (USB + SIM)</li>
<li>Right: Wi-Fi Adapter (PCIe)</li>
</ol>
<p>PCI enumeration (and, therefore, interface number assignment) is pretty straight forward: left to right as seen in this picture.<br />
<code class="language-plaintext highlighter-rouge">eno0</code> is next to the serial port, <code class="language-plaintext highlighter-rouge">eno3</code> is next to the USB 3.0 ports.</p>
<p><img src="https://u.vercas.com/apu4c4-openwrt-build/IMG_0046.jpeg" alt="M5 denutter" /></p>
<p>The first thing you need to do is unscrew the standoffs from the serial port. M5 is their size.<br />
Behold my glorious screwdriver, for it gets the job done.</p>
<p><img src="https://u.vercas.com/apu4c4-openwrt-build/IMG_0047.jpeg" alt="Serial Port Denutted" /></p>
<p>Righty tighty, lefty loosey.</p>
<p><img src="https://u.vercas.com/apu4c4-openwrt-build/IMG_0050.png" alt="SoC" /></p>
<p>Turn the board around, you will find the SoC. It’s a naked die, no integrated heat spreader.</p>
<p><img src="https://u.vercas.com/apu4c4-openwrt-build/IMG_0051.png" alt="Thermal Pad and Heat Spreader" /></p>
<p>I do not have one of those tools for positioning the heat spreader, so I did all this my own way.<br />
The official instructions make it very difficult to actually slot the board into the case.</p>
<p>The thermal pad that goes on the die should be peeled off the clear film, then applied to the die, and then the blue film can be removed. I did not remove it yet because I needed to position the heat spreader.</p>
<p>I centered the slab on top of the die and measured the distance between the edges of the board and the edges of the heat spreader.<br />
My measurements say that the heat spreader needs to be roughly 35mm away from the side of the case and 10mm away from the front of it.</p>
<p>Then I peeled off the blue film <strong>from the heat spreader</strong> and stuck it to the case roughly according to my measurements.<br />
Finally, I peeled off the blue film from the thermal pad.</p>
<p>It might be handy to have some good high quality tweezers to grab onto the edges/corners of the films. I did not have any, therefore I suffered.</p>
<p><img src="https://u.vercas.com/apu4c4-openwrt-build/IMG_0048.png" alt="Board Seated in Case" /></p>
<p>Fitting the board into the bottom case is a bit tricky.<br />
The RJ45 port housing thing-a-magik has some bits on the sides that push in when the board goes into the case.<br />
Those things are tough, and the only sane way to get the board into the case is by positioning it against the back at a shallow angle (so the thermal pad doesn’t stick to the heat spreader <em>yet</em>), make sure the edges are <strong>perfectly aligned</strong> so that both sides of the RJ45 housing get equal pressure, then push the board in.<br />
After it’s in, slowly lower the other end of the board, making sure that everything stays aligned, especially the screw holes. Those pushy bits <em>will</em> push the board out while it’s being lowered.</p>
<p><img src="https://u.vercas.com/apu4c4-openwrt-build/IMG_0049.png" alt="Board Seated in Case from Above" /></p>
<p>Glorious progress.</p>
<p>Now it’s time to add the screws. Worry not, the SIM slots remain accessible.<br />
The gray screws from the bag that came in the case are the ones for the board.</p>
<p><img src="https://u.vercas.com/apu4c4-openwrt-build/IMG_0052.png" alt="Wireless Module in Board" /></p>
<p>Installing mSATA/mPCIe (or even M.2/NGFF) modules can be tricky for people who’ve never done it before.<br />
The socket’s pins are designed to apply pressure on the module’s pads to achieve better contact.<br />
This means that modules have to be installed at an angle. In this case, about 30 degrees.<br />
After it fit snuggly in, press it down gently until it sits on the standoffs and screw it down.<br />
The tiny screws with blue stuff on them that came with the board is what you’re looking for.</p>
<p>The U.FL connectors are designed by the Devil himself. Good luck with that.<br />
The only help I can give you is this: they will absolutely not plug in at an angle. You need to make sure that you apply pressure equally on the whole circle.</p>
<p><img src="https://u.vercas.com/apu4c4-openwrt-build/IMG_0056.png" alt="SSD in Board" /></p>
<p>The SSD is installed in the same way. With mine, the flash chips are facing down… Odd but okay.<br />
Make sure you’ve got the OS set up prior to installation, otherwise you’re going to have to do a lot of (un)screwing.</p>
<p><img src="https://u.vercas.com/apu4c4-openwrt-build/IMG_0055.png" alt="Wireless Module and SSD in Board" /></p>
<p>And here are all the internals for version 1.</p>
<p><img src="https://u.vercas.com/apu4c4-openwrt-build/IMG_0057.jpeg" alt="Assembled Router Top" /></p>
<p>Positioning the top of the case over the bottom one is a bit tricky because it wants to slide forward.<br />
But it’s not really rocket surgery. The black screws that came with the case are the ones meant to hold the case together.</p>
<p><img src="https://u.vercas.com/apu4c4-openwrt-build/IMG_0058.jpeg" alt="Assembled Router Bottom" /></p>
<p>Put the rubber pads where you want them (in my case, over the threaded inserts for the board) and it’s done.</p>
<hr />
<p>Software will be discussed in another post.</p>I’ve decided that I need to build my own router, because I cannot find anything on the market that meets my requirements. Anything that gets close is also very expensive, and my budget is rather limited. So I’ve ended up building my own using PC Engines apu4c4 and OpenWrt. In this post, I will talk a bit about putting the hardware together and some of the decisions I’ve made along the way.