Ath5k AP Mode

People seem to keep asking about this, despite there being a quality page on kernel.org on how to run an AP with any mac80211 driver. So for what it’s worth, here’s how my setup works. Note if you are seeing flakiness with certain clients, e.g. it works fine with a computer but not with your cell phone, it is likely there is some bug with the power saving handling. I’m currently working on a few such issues, so it may be fixed soon enough.

To get started, you need the following:

  • some sort of network uplink, like a wired ethernet port
  • a kernel with ap mode support for ath5k (2.6.31+) and netfilter (for NAT)
  • dnsmasq
  • hostapd

You have two basic options for interfacing the wireless network with your wired network. One is by using a bridge directly to the wired network. Another is to use NAT so the wireless network is on its own subnet. The former is more typical of an embedded device, but I prefer the latter on my home LAN, so that’s what I’ll describe.

Turn off any wireless daemon (such as NetworkManager) while you experiment with hostapd to ensure that nothing else has the device open.

The hostapd.conf is large but self-explanatory. One can get away with a rather small config file if using the defaults. This example sets up an AP on wlan0 with the SSID “hostapd_ath5k”. It has a wlan-facing IP address 192.168.10.1, it’s on channel 11, supports 802.11g, and has the WPA pre-shared key “my_password”. I have also set up EAP, it works but requires making a lot of certs and such, so Google is your friend here.

hostapd.conf:

interface=wlan0
driver=nl80211
ssid=hostapd_ath5k
hw_mode=g
channel=11
auth_algs=3
own_ip_addr=192.168.10.1
wpa=1
wpa_passphrase=my_password
wpa_key_mgmt=WPA-PSK
wpa_pairwise=CCMP

For DHCP and DNS forwarding, I use dnsmasq. This couldn’t be easier. Just put something like the following in /etc/dnsmasq.conf:

   dhcp-range=192.168.10.50,192.168.10.150,12h

This will hand out addresses in the 192.168.10.X subnet. Then I use a small script to enable IP masquerading before launching hostapd (note, it will flush all iptables rules, which may not be what you want, so use with caution).

run.sh:

#!/bin/sh

DEV=wlan0
GW_DEV=eth0

# set IPs
ifconfig $DEV down
ifconfig $DEV up
ifconfig $DEV 192.168.10.1

# setup ip masq
echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -F
iptables -t nat -F
iptables -A FORWARD -i $DEV -j ACCEPT
iptables -A FORWARD -o $DEV -j ACCEPT
iptables -t nat -A POSTROUTING -o $GW_DEV -j MASQUERADE

./hostapd hostapd.conf

Running the script will launch hostapd, and if all goes well, you’ll see it show up in scans from other computers.

If anything goes wrong, make sure:

  • you’ve started dnsmasq
  • you have a valid backhaul connection
  • you aren’t using power save mode on client (iwconfig wlan0 power off — see previous comment about PS bugs)
  • you haven’t found a bug

Constantly folded

I was poking around with the disassembler the other day, which is never a good idea. One of the things I looked at was the following bit in the RX hotpath for ath5k:

rxs->qual = rs.rs_rssi * 100 / 35;

Now, that’s not a very significant calculation, and I doubt it will show up in profiles, but divides automatically trigger my “can we do better?” reflex. I was interested to see how gcc compiled this, because we have division by a constant, but we first have to multiply by a variable due to order of operations.

We can generally remove a division by multiplying by its reciprocal. Interestingly, gcc already does that. I couldn’t quite puzzle out all of the steps, but here’s the disassembly with my best guess:

; multiply eax by 100, store in ecx
 80483d0:	6b c8 64             	imul   $0x64,%eax,%ecx

; load (32 / 35) * 2**32 into edx
 80483d3:	ba eb a0 0e ea       	mov    $0xea0ea0eb,%edx

; multiply 100 * argc by 32/35 and store in edx:eax
 80483d8:	89 c8                	mov    %ecx,%eax
 80483da:	f7 ea                	imul   %edx

; take top 32 bits of result in edx (when sign extended, the
; complement of the final answer?) and add it back to the numerator
 80483dc:	8d 04 0a             	lea    (%edx,%ecx,1),%eax
 80483df:	89 c2                	mov    %eax,%edx

; divide by 32 to remove pre-multiply
 80483e1:	c1 fa 05             	sar    $0x5,%edx

; subtract one if we need to round
 80483e4:	89 c8                	mov    %ecx,%eax
 80483e6:	c1 f8 1f             	sar    $0x1f,%eax
 80483e9:	29 c2                	sub    %eax,%edx


So then I went hunting for the constant folding code in gcc, and there are all kinds of tricks like this. Very neat. Along the way I also found a link to the book Hacker’s Delight, now wish-listed.

In the original code, the denominator is a bit arbitrary, we could pick a different number that is more amenable to shifts and adds and save a multiply, but it’s hardly worth it.

wl1251 performance

After fixing the remaining ifup bug (as expected, it was easy), I have some initial numbers on the new kernel driver versus the stock vendor driver on the G1:

driver avg ping ms netperf mbit/s
tiwlan 65.231 7.53
wl1251 8.565 3.82

So, better on latency, worse on throughput. wl1251 is also quite a lot larger when taking all of cfg80211/mac80211 into account, though I didn’t spend any time trying to tweak the size in the build. Well, at least the code doesn’t make you want to poke your own eyes out.

Hang fixed

These sorts of bugs can ruin your weekend, just ask Ange who had to listen to me mope yesterday. Of course, I spotted it right away when looking at it freshly during this morning’s commute. Now, ifconfig wlan0 up; ifconfig wlan0 down; ifconfig wlan0 up still fails with wl1251, but it doesn’t hang and the rest looks tractable.

wl1251_sdio merged

The SDIO patches for TI 1251 (Android wifi chipset) are finally merged into wireless-testing, so they should be a lot easier to hack on now. That means the driver should make it into 2.6.32, though at a rather experimental stage. I did fix some crashes on ifup/ifdown since last posting, but there’s always more work to do. Current todo list includes better behavior for non-polling controllers (make the irq have a top-half), tracking down a device hang on reinitialization, pushing the msm_wifi.ko module, and on and on.

But I need to spend spare cycles on ath5k in the near term. John Linville recently remarked that he was sick of seeing bug reports that say “it works fine in madwifi,” and frankly, I agree. There’s little excuse for having a sub-standard driver given that we have had two fully open HALs for almost a year. Of course, that can be laid at my feet as much as anyone’s, so my plan is to install madwifi side-by-side with ath5k and do a lot of performance testing to see where we stand. ANI is the big missing feature; it will be useful to see how madwifi performs with and without it.

In other nerdy news, yesterday I scored a copy of Kernighan and Pike’s The Practice of Programming at the local used book store for $3. I’ve read the first five chapters so far. While I’ve been at this long enough to have already learned the book’s best practices (some the hard way), I really wish it was required reading at many of the places I’ve worked. You could do away with a lot of stupid coding standards documents by instead saying “read tpop, oh and please no studly caps.”

Now bionic

Kalle has posted a rebased version of my SDIO patches on top of the latest wireless-testing. I’ve done a little bit of testing with it — the driver starts up and loads fine, but once the interface goes down, it’s busted:


<6>[ 980.884302] wl1251_sdio mmc0:0001:1: firmware: requesting wl1251-fw.bin
<6>[ 981.036448] wl1251_sdio mmc0:0001:1: firmware: requesting wl1251-nvs.bin
<3>[ 981.045449] init: untracked pid 354 exited
<3>[ 981.074408] init: untracked pid 355 exited
<7>[ 981.331853] wl1251: 151 tx blocks at 0x3b788, 35 rx blocks at 0x3a780
<7>[ 981.339300] wl1251: firmware booted (Rev 4.0.4.3.5)
<7>[ 984.702740] wlan0: direct probe to AP 00:1a:70:da:a9:cd (try 1)
<7>[ 984.901621] wlan0: direct probe to AP 00:1a:70:da:a9:cd (try 2)
<7>[ 985.101583] wlan0: direct probe to AP 00:1a:70:da:a9:cd (try 3)
<7>[ 985.301617] wlan0: direct probe to AP 00:1a:70:da:a9:cd timed out
<7>[ 1024.802837] wl1251: down
<3>[ 1044.856640] init: untracked pid 358 exited
<3>[ 1050.033886] mmc0: Data timeout
<3>[ 1050.037701] wl1251: ERROR sdio write failed (-110)
<3>[ 1051.045674] mmc0: Data timeout
<3>[ 1051.049519] wl1251: ERROR sdio write failed (-110)
<3>[ 1052.057673] mmc0: Data timeout
<3>[ 1052.061518] wl1251: ERROR sdio write failed (-110)

Exploring this further turned into a rather long yak shaving session because I wanted to get iw on the phone to do more manual tests, and I wanted to link it dynamically against bionic, the Android’s fork of a BSD libc. Here were a few interesting discoveries found along the way:

$ ./iw
iw: not found

Right away, I intuited this was a problem finding libnl.so, but I was setting LD_LIBRARY_PATH. So I looked at the bionic linker source to find:


/* TODO: Need to add support for initializing the so search path with
* LD_LIBRARY_PATH env variable for non-setuid programs. */

So libnl.so had to go into the hard-coded library search path of /system/lib. That alone didn’t help, however. I eventually remembered that dynamically linked executables include the name of the dynamic linker in ELF headers, and on Android this is called “/system/bin/linker” instead of “/usr/lib/ld.so.1.” After fixing that (-Wl,-dynamic-linker,/system/bin/linker), the program just SEGVed at startup, so I added Android linker scripts and other random crap to the linker command line (much of it furnished by the very cool agcc wrapper script).


# ./iw
Usage: ./iw [options] command
Options:
--debug enable netlink debugging
--version show version (0.9.14-2-g5286851)
Commands:
help

Almost there, but no available commands? A glance at the iw source revealed that iw sticks all that stuff into an ELF section which mostly disappears when you link with –gc-sections. So with that exorcised from my linker command line, I finally have a functioning Android iw that is dynamically linked against bionic libc and my own cross-compiled libnl.

One wonders if the pain of bionic is worth its benefits, unless a benefit was curing ennui: “I’m bored, let’s write our own libc!” Anyway, I should be able to produce the various wireless tools natively compiled for Android in short order now. After that I’ll pop my Android TODO stack to the task, “make it easier to build custom images in my build environment.” I’d like to customize the filesystem images to have all of these utilities and relevant drivers in the normal places rather than hanging out in weird locations on the sdcard.

Wifi UI Part 0

Work on the Android wireless driver is stalled for now as kvalo is rebasing the patches. I believe the next step will be to take the platform code, which I have temporarily built into a module called msm_wifi¸ and turn it into a platform rfkill device. Or something like that — this is what he and I came up with at the wireless summit. Then there’s an annoying stack trace because free_irq() happens in common code, but sdio irqs are somewhat different so that needs to be pushed down into the interface layer.

Meanwhile, my project for the weekend was to get wl12xx integrated into the UI. This is an area that is likely to change anyway, so I didn’t want to spend a lot of time to do the whole thing properly, but I did want to get the plumbing pieces done correctly, which means libnl-tiny plus wpa_supplicant with netlink (-Dnl80211) support. As it turned out, this was a somewhat frustrating ordeal of mismatching wpa_s and libnl versions and general cross compiling headaches, so I didn’t finish this yet.

I did, however, also try the wext driver with wpa_s which I already compiled using the Android port. There are a few special commands that the Android UI sends to the wpa_s control interface, so I stubbed out a few of those in the wext driver. Then I moved wlan.ko out of the way, replacing it with msm_wifi (the hardware library inserts the wlan.ko module when you enable wireless through the UI). With these changes, wl12xx initializes when you click the checkbox, but wpa_supplicant gets confused somehow so there’s still a little work to be done here. I suppose I should just patch some of the library code to know about the other driver in order to reduce the number of hacks needed to get it started.

ath5k performance history

I was curious how well ath5k has been improving since its release, so I hacked up a little script to associate and do five minute iperf runs from /etc/rc.local. Looks pretty good so far (w-t is wireless-testing plus 5 or 6 extra patches I have brewing). The script is pretty kludgy, but may form the basis for some automatic build/regression testing in the future.
ath5k iperf chart

Kernel 2.6.30

Linux kernel 2.6.30 is out, featuring the best mainline support for ath5k yet. I recommend if you do use wireless with 2.6.30 (or .29), that you also grab this patch which fixes a nasty memory corruption issue. That one haunted me for months but it was a great feeling to finally nail it down. Of course the fix turned out to be quite simple. It should be hitting stable in short order.

What to look for in 2.6.31:

  • AP mode (finally!)
  • better transmission power settings
  • enhanced support for regulatory domains
  • rfkill support

There are still a couple of bugs in AP mode and rfkill, but hopefully they’ll get ironed out in the next two weeks or so. It’s too early to make predictions for 2.6.32, but improved power saving is a possibility, and of course lots of bug fixes. I also hope to set up some automated testing now that I have more than one piece of hardware (thanks Luis!) and will be interested in seeing performance statistics since ath5k was introduced.

Next week, Ange and I will be heading off to Berlin, where I’ll be attending the Linux Wireless Mini-summit at FUDCon for a couple of days, and after that we’ll be staying a few days for vacation. I’m sure to return with a head full of ideas about 802.11 networking, and a belly full of beer and sausage.

wl12xx howto

So, wl12xx definitely works including transmission, but I still haven’t fixed the rough edges in the driver. However, I did bother to make a semi-permanent writeup on how to build and use wl12xx on the G1 as it currently stands. I hope to clean up the warts and submit the patchset next week. Meanwhile, I have a lot of ath5k bugs to work on. Enough for today though.