This is a continuation to my previous post about the assembly of hardware for my apu4c4 OpenWrt router.
Here I will discuss a bit about my journey with the software side, builds, configurations, drivers, various packages.

---

The first thing I tried was, of course, the default build for AMD64 generic targets.
Writing it to the SSD was trivial, just used dd.
However, when booting it, after a bit of trial and error, I realized that its default serial console is configured with a baud rate of 38,400 or so - unlike the SeaBIOS which uses 115,200.
I found this rather puzzling because the default baud rate in the OpenWrt build system is, in fact, 115,200…

This image has a few problems:

• It’s missing the igb driver required for the NICs on the apu4c4.
• The kernel is built without the features required for some packages, such as LXC.
• Funky serial port settings.
• Tiny rootfs size (I’ve got plenty of room on my SSD).
  Therefore, I had no doubt that I need to build OpenWrt myself.
  This turned out to be quite easy, because the build system automatically fetches all its dependencies! Kudos to the developers for that.

One small detail wasn’t clear to me, though. They use the Kbuild system, used by the Linux kernel.
Therefore, a lot of configuration options are actually tri-state: Y, N, or M.
Normally in Linux Y means compile this module into the kernel, N means do not compile this module, and M means compile this module as a separate executable (to be later put into the initrd/initramfs).
With OpenWrt, it wasn’t so obvious to me at first. For packages (including kernel modules/drivers), Y means compile into a package and include in the resulted image. It does not compile modules into the kernel itself.
N means do not build and M means compile… And that’s it.
At first I assumed Y means compile and install and M means compile and package in image, but do not install. I was very wrong.

Another thing that wasn’t obvious to me, was that the build system doesn’t actually have all the packages by default.
If you want to build OpenWrt, you definitely need to read the instructions on their website. I didn’t at first, because I am a dum dum.
There is a script you need to execute in order to add the rest of the package sources to the build system.
This will substantially increase the number of options available, which might be extremely overwhelming for a beginner. It also makes it very easy to hoard packages…
The kernel modules and drivers + a few basic things that are selected by default anyway are really the only packages you need to worry about. You can use opkg to get the rest after you log into OpenWrt - except kernel modules.
Just remember to grab the ones mentioned on the OpenWrt wiki as well.

Once you boot it with the right drivers, it will create two interfaces by default: lan and wan.
eth0 was the LAN and eth1 was the WAN for me. I swapped them. You don’t really need to do this.
As stated in the previous post, eth0 is the RJ-45 port next to the serial port. eth1 is the one adjacent to eth0.

The SSH server used by default in OpenWrt is Dropbear, which seems to be a-okay. I might replace this later if I need more features.
If I remember correctly, it’s not enabled by default. I highly recommend using public key authentication here instead of password authentication.

LuCI, the default web front-end to OpenWrt, is absolutely amazing. You can use it to configure pretty much anything and it majorly improves the experience.
I know y’all think using command-line interfaces makes you look cool, but LuCI provides you with excellent overviews of your configuration, as well as names and descriptions of all the options you can (or cannot) configure for a specific system/item. It even shows default values!
opkg install luci should be all you need. You can use LuCI itself to install more packages. I personally use the Material theme, it is quite smexy and it comes with a few nice features such as auto-refreshing certain pages.

If you want to try out my build configuration (tailored to my hardware specifically), then you can find it here.
Just put this into .config in your OpenWrt local repo. If you are using my config, please read the rest of the information below.

Experience so far

I’ve experimented with quite a few features and packages. Here are some of my findings.

WireGuard

This one performs incredibly well. I can saturate a 1 Gbps link with WireGuard traffic and it only pushes one CPU core (when transmitting or receiving) to <60% usage, the rest hover around 10%.
The LuCI app for this is quite janky:

• It won’t tell you if you have an invalid character in the interface name. - is not a valid character apparently.
• When you make modifications and save the settings, it commits the changes but doesn’t actually apply them. It fails to add new peers and you need to restart the network service manually.
• The status page is difficult to read because it’s chaotic and extremely unpolished. The icons look so faded, one might think they’re a placeholder for an image that failed to load. To use WireGuard, you will need to have enabled quite a few crypto modules, and some of them come with AVX2 variants by default. Remove them with sed -i '/avx2/d' /mnt/openwrt-root/etc/modules.d/10-crypto-misc.

ath10k & firmware

The vendor firmware and driver work fine.
The community-supported ones, though, don’t. (these are the ath10k-firmware-qca988x-ct and kmod-ath10k-ct packages)
Some devices were getting kicked out of the network randomly and my router had to be restarted every 3-4 days when using the community firmware and driver.
I was also getting continuous spam in my system log about some queue being full, coming from the driver.

YMMV but I recommend staying away from the community packages if you have my specific hardware.

LXC

This one is a PITA to use and I couldn’t get it to start containers properly.
As far as I can tell, it doesn’t run /sbin/init in the containers at all.
You can’t do anything in them like this. Also making networking work is a chore. Default config comes with no networking support.
What even is the point of containers without networking?

Docker

This isn’t packaged for OpenWrt but I tried to install it anyway.
Don’t do this.

Docker sets up some bizzare firewall rules that will break everything.
What happened to me was that, suddenly, I could route traffic from any zone to any other zone - this is extremely bad.

If I could get LXC to work, I would try to run Docker inside LXC…

Filesystem and Block Devices

Grab the block-mount package and you can configure filesystems and swap devices/files in /etc/config/fstab.

If you want to use OpenWrt’s ability to overlay a filesystem on top of your root, then you need to mount something to /overlay and the rest will be taken care of for you.

Note that, if you want to do this automatically via /etc/config/fstab, you need to put that file in the rootfs, not the /overlay/upper/etc/config/fstab file.
As far as I can tell, every other config file can sit in the overlay.

Adblock

Makes YouTube videos load extremely slowly at the beginning.

BTRFS

Way too slow in this kernel version, file I/O will grind to a halt.
Tried to use it mainly for the sake of Docker, to use its snapshots.