Fake SNRs

With the addition of per-link signal levels that I added over the weekend, my wmediumd fork leveled up from “mere curiosity” to “potentially useful someday.” For the case of mesh, this means you can use signal levels to inform how HWMP will create the mesh paths.

For example, as a test I was able to validate this fix, by setting up a virtual 4-node mesh with a bad path and a good path. With the patch reverted, the bad path was almost always selected due to its PREQs being received at the target first. [In actuality, this test will exhibit frequent path swapping because the order in which the PREQs are received is essentially random, a finding in “Experimental evaluation of two open source solutions for wireless mesh routing at layer two” by Garroppo et al. Wmediumd doesn’t show this yet because frames are mostly received and queued in-order. At the time of the patch, I validated it in an actual 15-node mesh.]

There are still a couple of things that would be nice to have here. Today, we base the decision on whether a multicast frame is received by the signal level from the transmitter to us and the multicast rate. However, this means that with a low multicast rate, there is basically zero frame loss. In real life, loss happens much more frequently, and so we cannot test the effects of lost path request frames in wmediumd, which is the subject of at least one pending HWMP patch. Another problem is that the current setup works only with static setups; we might be interested in what happens with mobile nodes, for example. For that we’d need to be able to change the signal level periodically; how to easily specify that is a bit of a question mark.

tmux + wmediumd

Wmediumd gained the ability to do a simple contention simulation a while ago. It turned out to be a small change to the existing code: just ensure that any new frames are scheduled after any other queued frames of equal or higher priority from any other station.

Assuming the simulation is accurate, we might use this to gather some information about different kinds of wireless network topologies. For example: what is the throughput and latency like for a mesh network, as a function of hops?

The one sticking point is that it’s a bit of a pain to set up a bunch of mesh nodes with hwsim with their own IPs and routing tables. I’ve previously scripted this with send-to-self routing, but it’s a bit ugly. So I looked into doing this with network namespaces and controlling it all with tmux. The result is this fairly minimal script to launch a number of mesh nodes in a linear toplogy. From there one can easily run ping and iperf to gather some data, as in this chart:

This image shows the result, and is in line with measurements that Javier Cardona had done on actual hardware. We can see that throughput is roughly inversely proportional to the number of nodes, while latency is directly proportional.

This may seem pretty bad at first, but makes sense when you consider that a radio transceiver can only listen or talk at once — it is all about radio physics, nothing to do with mesh specifically (which is not to say that mesh has no inefficiencies). Also this level of performance is when all the nodes are in range of each other; in such a case you’d be unlikely to have so many hops because the nodes would instead just peer directly with each other. So we might design our networks to avoid many hops, reduce the number of nodes in a given interference area, use fancy phy algorithms to enhance spatial reuse, or use multiple channels.

My plan with wmediumd is to use it in a bit more automated fashion to evaluate things like changes to HWMP — I think if we can identify topologies that people care about then it’s a bit stronger to say “this change always makes things better” if we can show repeatable before-and-after results from wmediumd.

VHT mesh

…is a thing now.

# iw dev
    Interface wlan0
		ifindex 4
		wdev 0x1
		addr 30:b5:c2:fb:34:d8
		type mesh point
		channel 149 (5745 MHz), width: 80 MHz, center1: 5775 MHz

# iperf -c
Client connecting to, TCP port 5001
TCP window size: 43.8 KByte (default)
[  3] local port 34175 connected with port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   148 MBytes   124 Mbits/sec


In response to my previous post about UART access on the Alfa AWUS036NHA, I got a nice email from Jim Ewing about the DWM-W034, an ath9k_htc device that apparently is embedded inside HDTVs and readily available for $10. He found the TX/RX pins on the board, and it looks like it would be a bit easier to put a socket on one of these compared to the Alfa since there are no pesky chips nearby.

With his permission, and so that this information doesn’t get lost to the sands of time, here’s the pic he sent me. Enjoy!

mesh on ath10k

I’ve had my ath10k AP (TP-Link Archer C7 v2) since last October or so, with the goal of having a VHT-capable device with which to test (currently non-existent) VHT mesh. Unfortunately, for nearly all of that time, I’ve been stuck on a firmware crash shortly after bringing up the device. Not that I’ve spent a whole lot of time on it, but there’s only so much one can do when getting to the point of “firmware crashes and it has something to do with peers but that’s all I know and I don’t have the time, tools, or code to dig deeper.”

I think there’s some variant on rubber duck debugging where you complain publicly about some issue, and doing that makes you think about it more, and then the way forward is magically revealed. That, plus some helpful hints from the residents of the ath10k ML, got me over the hump, so now it works!

Next up, finding a spare mini PCIe slot for the other ath10k device I have, and getting the VHT bits done…


router += serialMy ath10k-based router now has serial headers, after I managed to crash it a few times over the weekend while testing out some patches. Thank you, TP-Link, for making the pins so easily accessible, to the extent that it’s easier to whip out the soldering iron than remind oneself how to setup kexec/kdump.

I do need to get one of these nifty usb-serial cables so that the JTAG ribbon cable + FTDI-breakout-on-a-breadboard monstrosity can go back into the
parts bin, but it is working fine otherwise. [The pictured 7-segment display and TTL chips are just misdirection, by the way.]

There was a momentary bit of confusion on my part when the console showed the router stuck in a loop requesting recovery firmware over TFTP instead of the normal boot process…until I realized that the reset button was wedged in the depressed state by the case. Whoops.


The ath9k and later chips support a spectral scan feature for measuring channel occupancy. I played with it for the first time last week over the Thanksgiving holiday. Simon Wunderlich’s FFT_eval is a great tool to look at the captured RF spectrum, but I found myself wanting something more real-time, and the various other things I found on github with that aim didn’t work for me. So, this too-ugly-to-live python hack happened.

It turns out that the samples are rather coarse and infrequent, so the result is not as dynamic as I’d hoped. However, I made the pretty heatmap below with a few hours’ worth of samples, and although I didn’t yet label frequencies, you can see a couple of channels in active use. There’s still plenty of room for improvement in the visualization.

functional bitrate sim

My wmediumd rewrite is a bit further along thanks to getting a few hours to hack on it this weekend. It can now accurately simulate throughput between a pair of radios using legacy rates. For example, if we set the SNR between two devices to 20 dB, then they can communicate at a nominal 54 mbps rate, yielding about 26 Mbps achieved in iperf:

[  3]  0.0-10.0 sec  31.2 MBytes  26.1 Mbits/sec

At 15 dB, we can send between 24 and 36 Mbps nominal rates, which yields:

[  3]  0.0-10.1 sec  21.0 MBytes  17.5 Mbits/sec

Note that achieved throughput is quite a bit lower than nominal, as in real life — if aggregation were implemented then they would be closer.

The basic architecture is pretty simple: frames are queued on a per-sender management or data queue depending on type, and delivery time is computed based on whether or not there is loss and the contention window parameters of the queues. A timerfd is used to schedule reporting of frame delivery back to the kernel at appropriate times. The delivery time does not take into account actual contention, although this could be done in principle by looking at all the queued frames for all stations.

I haven’t really decided what to do about configuration. I stripped out the jamming and probability matrix configurations, as I feel like doing things on a signal level basis are simpler. But at this point there’s no real way to specify signal levels either (other than hardcoding), and some scenarios probably want something dynamic (e.g. mobile stations).

Changes are in my wmediumd master branch. Unfortunately, I won’t have much time to work on this for the next two months, but patches for the many TODOs are welcome.

wmediumd speed test

Thanks to some inquries on linux-wireless, I took a look at wmediumd recently. The code could use a bit of work, and there are some features I’ve been meaning to add since forever, so I started gutting it with an eye towards sprucing up the architecture and feature set (changes can be found here).

One of the questions from the mailing list was whether wmediumd adds a lot of overhead compared to mac80211_hwsim. It is of course doing more work, with additional memory copies, context switches, etc — but is it enough to make wmediumd unworkable?

So I did a quick TCP iperf test on my laptop with an open mesh, and get the following numbers.

hwsim without wmediumd:

    [  3]  0.0-10.0 sec  1.36 GBytes  1.16 Gbits/sec

hwsim with wmediumd:

    [  3]  0.0-10.0 sec  1.27 GBytes  1.09 Gbits/sec

It looks like wmediumd is doing fine. This is with monitors running, the non-monitor case does about twice that. Actually, I think this is a bit lower than it should be, but considering both cases are close, and a good deal faster than your typical wifi connection, it’s probably good enough for some level of bandwidth simulation.

wpas mesh

Continuing where I left off with my OpenWRT mesh nodes, after installing the OS, the next step is to get a mesh-enabled userspace on them.

One can use iw to create an open mesh, and the authsae daemon for secure mesh, and OpenWRT already ships both of those, so just installing those packages is really all that is required.

However, I’m currently working on a patchset to add mesh support to wpa_supplicant, which could be useful for platforms where wpa_s is already present and running yet another daemon just for secure mesh is unpalatable. Here’s the recipe I’m using to keep the latest version on the device and use it for day-to-day activities.

Since OpenWRT can use git as a package source and already does so for hostapd, building a custom wpa_supplicant is mainly a matter of just changing the git repository url and config. I made the following changes in the package/network/services/hostapd directory:

diff --git a/package/network/services/hostapd/Makefile b/package/network/services/host
index 6872742..5985339 100644
--- a/package/network/services/hostapd/Makefile
+++ b/package/network/services/hostapd/Makefile
@@ -10,10 +10,10 @@ include $(TOPDIR)/rules.mk
diff --git a/package/network/services/hostapd/files/wpa_supplicant-full.config b/packa
index bbfaa73..4d9e00e 100644
--- a/package/network/services/hostapd/files/wpa_supplicant-full.config
+++ b/package/network/services/hostapd/files/wpa_supplicant-full.config
@@ -407,3 +407,9 @@ CONFIG_NO_RANDOM_POOL=y

(Offhand, I don’t know if P2P and TDLS are really required, but as it matches my existing config, we’ll go with that.)

You’ll also need to enable CONFIG_WPA_SUPPLICANT_OPENSSL=y in the OpenWRT menuconfig in order for SAE to link properly.

Rebuilding from scratch looks like this:

rm dl/hostapd-*.tar.bz2
make package/hostapd/{download,prepare,clean,compile,install} V=s

Once built, I have a simple script which copies over the bin/x86_64/packages/{hostapd*,wpa-s*} files and then runs opkg install on each of the nodes.

To start the mesh, I use the following script:

pubip=`ip route get | awk 'NR==1 {print $NF}'`
last8=`echo $pubip | awk -F . '{print $4}'`

cat<<__EOM > wpa_s.conf
ip addr flush $iface
ip link set $iface down
iw dev $iface set type mp
ip link set $iface up
ip addr add $meship/24 dev $iface

killall wpa_supplicant
wpa_supplicant -dd -i $iface -c wpa_s.conf >wpa_s.log 2>&1 &

In response to the previous blog post, Johannes Berg pointed out that running nfsroot and PXE booting these devices would be even easier than futzing with USB sticks and copying binaries back and forth. Unfortunately, the BIOS on these machines doesn’t appear to support netboot, and at least for now, I can’t be bothered to figure out how to do it from within grub. At any rate, I find this setup makes for a fairly painless compile / deploy / test cycle.