Android rebuilt

While stuck inside for days due to snow, I once again downloaded the Android source trees from the repos. I plan to update my Android wifi page with better, step-by-step instructions to go from nothing to a working set of wireless utilities on the phone. In the meantime, I’ve re-familiarized myself with the current Android build environment (for various values of current — this uses donut since as far as I know, eclair doesn’t have a definition for the Dream (G1/ADP1) phone yet).

I do have to say that things in this area have improved a lot. While it still took some google-fu to figure out which branch to check out to get the correct software (tag donut-plus-aosp), buildling the entire image from scratch was rather straightforward (. build/envsetup.sh; lunch aosp_dream_us-eng; make). My main complaint is that the build system is far too slow to use for day-to-day work. Is Google still not eating their dogfood here?

So far I have libnl, iw, and packetspammer now integrated properly via Android.mk makefiles, and a custom kernel plus the support modules for wl1251. Unfortunately, space is really tight on the system partition, so the TI driver had to go to make room for everything. I’m still waiting for someone to port the Android runtime to emdebian or maemo with /usr on the sdcard, but perhaps I’ll mess around with bind mounts until then.

It seems wl1251 SDIO doesn’t like the new power-saving code, so that is something else I’ll look into soon. For now, one can disable that in the driver or possibly via “iwconfig power off”.

wl1251: cmd set ps mode
wl1251: cmd configure
mmc0: Data timeout
wl1251: ERROR sdio write failed (-110)
...
wl1251: ERROR elp wakeup timeout

wl1251 performance

After fixing the remaining ifup bug (as expected, it was easy), I have some initial numbers on the new kernel driver versus the stock vendor driver on the G1:

driver avg ping ms netperf mbit/s
tiwlan 65.231 7.53
wl1251 8.565 3.82

So, better on latency, worse on throughput. wl1251 is also quite a lot larger when taking all of cfg80211/mac80211 into account, though I didn’t spend any time trying to tweak the size in the build. Well, at least the code doesn’t make you want to poke your own eyes out.

Hang fixed

These sorts of bugs can ruin your weekend, just ask Ange who had to listen to me mope yesterday. Of course, I spotted it right away when looking at it freshly during this morning’s commute. Now, ifconfig wlan0 up; ifconfig wlan0 down; ifconfig wlan0 up still fails with wl1251, but it doesn’t hang and the rest looks tractable.

wl1251_sdio merged

The SDIO patches for TI 1251 (Android wifi chipset) are finally merged into wireless-testing, so they should be a lot easier to hack on now. That means the driver should make it into 2.6.32, though at a rather experimental stage. I did fix some crashes on ifup/ifdown since last posting, but there’s always more work to do. Current todo list includes better behavior for non-polling controllers (make the irq have a top-half), tracking down a device hang on reinitialization, pushing the msm_wifi.ko module, and on and on.

But I need to spend spare cycles on ath5k in the near term. John Linville recently remarked that he was sick of seeing bug reports that say “it works fine in madwifi,” and frankly, I agree. There’s little excuse for having a sub-standard driver given that we have had two fully open HALs for almost a year. Of course, that can be laid at my feet as much as anyone’s, so my plan is to install madwifi side-by-side with ath5k and do a lot of performance testing to see where we stand. ANI is the big missing feature; it will be useful to see how madwifi performs with and without it.

In other nerdy news, yesterday I scored a copy of Kernighan and Pike’s The Practice of Programming at the local used book store for $3. I’ve read the first five chapters so far. While I’ve been at this long enough to have already learned the book’s best practices (some the hard way), I really wish it was required reading at many of the places I’ve worked. You could do away with a lot of stupid coding standards documents by instead saying “read tpop, oh and please no studly caps.”

Now bionic

Kalle has posted a rebased version of my SDIO patches on top of the latest wireless-testing. I’ve done a little bit of testing with it — the driver starts up and loads fine, but once the interface goes down, it’s busted:


<6>[ 980.884302] wl1251_sdio mmc0:0001:1: firmware: requesting wl1251-fw.bin
<6>[ 981.036448] wl1251_sdio mmc0:0001:1: firmware: requesting wl1251-nvs.bin
<3>[ 981.045449] init: untracked pid 354 exited
<3>[ 981.074408] init: untracked pid 355 exited
<7>[ 981.331853] wl1251: 151 tx blocks at 0x3b788, 35 rx blocks at 0x3a780
<7>[ 981.339300] wl1251: firmware booted (Rev 4.0.4.3.5)
<7>[ 984.702740] wlan0: direct probe to AP 00:1a:70:da:a9:cd (try 1)
<7>[ 984.901621] wlan0: direct probe to AP 00:1a:70:da:a9:cd (try 2)
<7>[ 985.101583] wlan0: direct probe to AP 00:1a:70:da:a9:cd (try 3)
<7>[ 985.301617] wlan0: direct probe to AP 00:1a:70:da:a9:cd timed out
<7>[ 1024.802837] wl1251: down
<3>[ 1044.856640] init: untracked pid 358 exited
<3>[ 1050.033886] mmc0: Data timeout
<3>[ 1050.037701] wl1251: ERROR sdio write failed (-110)
<3>[ 1051.045674] mmc0: Data timeout
<3>[ 1051.049519] wl1251: ERROR sdio write failed (-110)
<3>[ 1052.057673] mmc0: Data timeout
<3>[ 1052.061518] wl1251: ERROR sdio write failed (-110)

Exploring this further turned into a rather long yak shaving session because I wanted to get iw on the phone to do more manual tests, and I wanted to link it dynamically against bionic, the Android’s fork of a BSD libc. Here were a few interesting discoveries found along the way:

$ ./iw
iw: not found

Right away, I intuited this was a problem finding libnl.so, but I was setting LD_LIBRARY_PATH. So I looked at the bionic linker source to find:


/* TODO: Need to add support for initializing the so search path with
* LD_LIBRARY_PATH env variable for non-setuid programs. */

So libnl.so had to go into the hard-coded library search path of /system/lib. That alone didn’t help, however. I eventually remembered that dynamically linked executables include the name of the dynamic linker in ELF headers, and on Android this is called “/system/bin/linker” instead of “/usr/lib/ld.so.1.” After fixing that (-Wl,-dynamic-linker,/system/bin/linker), the program just SEGVed at startup, so I added Android linker scripts and other random crap to the linker command line (much of it furnished by the very cool agcc wrapper script).


# ./iw
Usage: ./iw [options] command
Options:
--debug enable netlink debugging
--version show version (0.9.14-2-g5286851)
Commands:
help

Almost there, but no available commands? A glance at the iw source revealed that iw sticks all that stuff into an ELF section which mostly disappears when you link with –gc-sections. So with that exorcised from my linker command line, I finally have a functioning Android iw that is dynamically linked against bionic libc and my own cross-compiled libnl.

One wonders if the pain of bionic is worth its benefits, unless a benefit was curing ennui: “I’m bored, let’s write our own libc!” Anyway, I should be able to produce the various wireless tools natively compiled for Android in short order now. After that I’ll pop my Android TODO stack to the task, “make it easier to build custom images in my build environment.” I’d like to customize the filesystem images to have all of these utilities and relevant drivers in the normal places rather than hanging out in weird locations on the sdcard.

Wifi UI Part 0

Work on the Android wireless driver is stalled for now as kvalo is rebasing the patches. I believe the next step will be to take the platform code, which I have temporarily built into a module called msm_wifi¸ and turn it into a platform rfkill device. Or something like that — this is what he and I came up with at the wireless summit. Then there’s an annoying stack trace because free_irq() happens in common code, but sdio irqs are somewhat different so that needs to be pushed down into the interface layer.

Meanwhile, my project for the weekend was to get wl12xx integrated into the UI. This is an area that is likely to change anyway, so I didn’t want to spend a lot of time to do the whole thing properly, but I did want to get the plumbing pieces done correctly, which means libnl-tiny plus wpa_supplicant with netlink (-Dnl80211) support. As it turned out, this was a somewhat frustrating ordeal of mismatching wpa_s and libnl versions and general cross compiling headaches, so I didn’t finish this yet.

I did, however, also try the wext driver with wpa_s which I already compiled using the Android port. There are a few special commands that the Android UI sends to the wpa_s control interface, so I stubbed out a few of those in the wext driver. Then I moved wlan.ko out of the way, replacing it with msm_wifi (the hardware library inserts the wlan.ko module when you enable wireless through the UI). With these changes, wl12xx initializes when you click the checkbox, but wpa_supplicant gets confused somehow so there’s still a little work to be done here. I suppose I should just patch some of the library code to know about the other driver in order to reduce the number of hacks needed to get it started.

wl12xx howto

So, wl12xx definitely works including transmission, but I still haven’t fixed the rough edges in the driver. However, I did bother to make a semi-permanent writeup on how to build and use wl12xx on the G1 as it currently stands. I hope to clean up the warts and submit the patchset next week. Meanwhile, I have a lot of ath5k bugs to work on. Enough for today though.

First scan

Success! Scanning with wl12xx on my ADP1 works. I’ll post more later but here’s a teaser. That’s a real SSID, however apt.

# ./iwlist wlan0 scan
wlan0     Scan completed :
          Cell 01 - Address: 00:0F:B5:63:30:4E
                    Channel:11
                    Frequency:2.462 GHz (Channel 11)
                    Quality=14/70  Signal level=-96 dBm
                    Encryption key:on
                    ESSID:"SERENDIPITY"
                    Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 18 Mb/s
                              24 Mb/s; 36 Mb/s; 54 Mb/s
                    Bit Rates:6 Mb/s; 9 Mb/s; 12 Mb/s; 48 Mb/s
                    Mode:Master
                    Extra:tsf=000001dec907e41c
                    Extra: Last beacon: 160ms ago
                    (Unknown Wireless Token 0x8C05)

wl12xx update

I won’t claim success until I actually transfer data, but so far, it looks like wl12xx SDIO support is baked:

[  111.247063] cfg80211: Using static regulatory domain info
[  111.253472] cfg80211: Regulatory domain: US
[  111.262810] 	(start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
[  111.271568] 	(2402000 KHz - 2472000 KHz @ 40000 KHz), (600 mBi, 2700 mBm)
[  111.279472] 	(5170000 KHz - 5190000 KHz @ 40000 KHz), (600 mBi, 2300 mBm)
[  111.287254] 	(5190000 KHz - 5210000 KHz @ 40000 KHz), (600 mBi, 2300 mBm)
[  111.294975] 	(5210000 KHz - 5230000 KHz @ 40000 KHz), (600 mBi, 2300 mBm)
[  111.302482] 	(5230000 KHz - 5330000 KHz @ 40000 KHz), (600 mBi, 2300 mBm)
[  111.310264] 	(5735000 KHz - 5835000 KHz @ 40000 KHz), (600 mBi, 3000 mBm)
[  111.317894] cfg80211: Calling CRDA for country: US
[  112.684807] lib80211: common routines for IEEE802.11 drivers
[  112.690972] lib80211_crypt: registered algorithm 'NULL'
[  112.744713] lib80211_crypt: registered algorithm 'TKIP'
[  112.787467] lib80211_crypt: registered algorithm 'WEP'
[  112.845267] lib80211_crypt: registered algorithm 'CCMP'
[  113.251671] wl12xx: registering driver
[  113.256462] wifi_probe
[  113.259208] trout_wifi_power: 1
[  113.569452] trout_wifi_reset: 0
[  113.623681] trout_wifi_set_carddetect: 1
[  113.628290] mmc0: card_present 1
[  113.631921] mmc0: Slot status change detected (0 -> 1)
[  113.653283] mmc0: Command timeout
[  113.664788] mmc0: card claims to support voltages below the defined range. These will be ignored.
[  113.678429] mmc0: SDIO card claims to support the incompletely defined 'low voltage range'. This will be ignored.
[  113.692773] mmc0: new SDIO card at address 0001
[  113.702813] wl12xx: in probe
[  113.728844] phy0: Selected rate control algorithm 'minstrel'
[  113.738915] wl12xx: loaded
[  113.743919] wl12xx: initialized

Unfortunately, I can’t really test it yet because the wpa_supplicant already on the phone doesn’t support -Dwext or -Dnl80211, and there are no wireless-tools. So I need to cross-compile those before attempting to associate. In the meantime, here’s a very rough patch that implements the goods from wireless-testing wl12xx. I need to work on read() and write() – right now they are a copy of what is in the google driver because it appears the host driver has issues with RW_EXTENDED. I don’t believe the readb/writeb fallbacks should be strictly necessary. Ditto with the platform code, that needs a slightly better solution.

wl12xx merged

commit 00d8979d598d5461a06d800c779f15e03888d9d8
Author: Kalle Valo 
Date:   Wed Apr 29 23:33:31 2009 +0300

    wl12xx: add driver

    wl12xx is a driver for TI wl1251 802.11 chipset designed for embedded
    devices, supporting both SDIO and SPI busses. Currently the driver
    supports only SPI. Adding support 1253 (the 5 GHz version) should be
    relatively easy. More information here:

    http://focus.ti.com/general/docs/wtbu/wtbuproductcontent.tsp?contentId=4711

    (Collapsed original sequence of pre-merge patches into single commit for
    initial merge. -- JWL)

    Signed-off-by: Kalle Valo 
    Signed-off-by: Bob Copeland 
    Signed-off-by: John W. Linville 

Hooray, wl12xx is now in wireless-testing! As the commit log states, SDIO support isn’t there yet, because I haven’t written it yet. Well, I wrote some of it, but nothing worth even compile-testing yet.

As for the Android itself, I bought a snazzy serial breakout board for the phone so now I can do some real work on it. It works just fine with screen(1) as the terminal program.

Also, I solved the problem with booting: apparently the init program for 2.6.25 doesn’t work with 2.6.27 — when I put my own compiled init in the initramfs, everything mounted fine. The input devices now do not work, but I suspect it’s just a difference in paths and that an upgrade to 1.5 userland will fix all of that.