bjørn / beorn / 白熊

Public wifi - how could it get so bad?

Warning: This is a rant.

With wifi came the promise of being online (almost) anywhere, but due to incompetent or misdirected implementation and management, it’s pretty much a patchwork of extremely unreliable networks. My experience is that there’s a 30-40% chance of actually being able to get online at an access point.

In the case of wireless access points (AP) owned and operated by cafes/restaurants, they are:

• often connected through a patchwork of bad power, telephone, and network cables, usually piled on top of each other next to the cash register to ensure a reboot every cash register kachink
• using a cheap and buggy access point that freezes up every 30 minutes
• unencrypted/open, and long since discovered and overloaded by P2P freeloaders
• configured to the same channel as the other 10 access points around, causing “mysterious” packet loss of 50%+
• configured with the same SSID (e.g., linksys) as the other 10 access points around, making actual selection a stroke of luck
• practically unmanaged. It’s always better to just go over and try to fix it yourself if something’s wrong; if you ask the staff they’re usually completely clueless, and will try to call someone almost as clueless, before finally you’ve spent a good hour helping them just get the thing working.

In the case of commercial access points, while they’re more professionally installed and run, the access and management systems kill the experience. Here are typical problems:

• Filtering of everything but port 80 (web) - goodbye IMAP, IRC, etc. I can’t do any work when all I can do is browse (yeah, I know I could set up a tunnel over port 80, and I may have to start doing that sigh).
• You have to first go through a “home made” web application to register and sign in, which you often can’t due to bugs on the web application preventing you from registering;
  • failure to localize phone numbers or addresses
  • missing web pages
  • 500 server errors
  • failure to send SMS or email registration messages
• Worst of all, there doesn’t seem to be anyone monitoring or receiving these server errors; if there were, I wouldn’t be hitting them nearly as often.
• If you’ve managed to sign in, you can expect to be suddenly locked out if you close your laptop and open it again since your old session hasn’t expired, and they have “smart” sessions.

I have yet to find a good web application like this. In my opinion they should be banned altogether. The internet is not the same as the web, so just requiring you to go through such a manual and unreliable system to get online is insane.

If I have to guess, I would have to guess that these systems are designed, implemented, bought, and managed by incompetent people as well, people that:

• Don’t understand or value simplicity and reliability, always erring on making things too complicated and thus unreliable
• Think the web is the Internet, so they don’t see anything wrong in “breaking” the Internet by disabling everything but the web
• Think you can just build-and-forget, relying on (expensive) customer bug reports (that often have an unhappy ending) instead of proactively managing and fixing problems

What works?

I have been using wifi extensively in a few places around the world (Beijing, Hong Kong, Budapest, Singapore, San Francisco), and the only wifi access points I’ve seen that work are those that are free and run by clueful coffee shop or restaurant owners that know to:

• invest in robust cabling, and put the AP out of harms way
• invest in proper training of staff, and howto material
• invest in proper installation that
  • sets the access point SSI to something unique, the name of the venue is perfect as well as good advertisement
  • configures the access point to use a channel that doesn’t conflict with other nearby access points
  • changes the admin password to non-default one (most amateur access points do not)
  • adds encryption (WPA is the only secure thing these days)
• change the WPA password every day (using a hex key), and has it written down on a piece of paper they can hand out to guests
• advertise wifi availability and instructions for getting online prominently

In addition, for laptop users, it’s also always good to have power sockets around.

Python 3000 alpha 1 released!

Guido just announced it Time to experiment.

Blueprint, a CSS framework

Tired of fighting with browsers on CSS issues regarding layouts, styling of buttons, or generating print versions? Then you’d probably be interested in the Blueprint CSS Framework which I discovered via Mark Boulton. It’s getting great reviews, looks to have a lot of promise… and by a fellow Norwegian to top it off. Check the tutorial for a good introduction.

Finding Python packages

Although easy_install might be sufficient to install a Python package (aka egg) onto your system, it doesn’t help you find available packages or see what you have on your system. Most package managers does, so this was a bit surprising. I was frustrated enough that I started looking at what it would take to make one. I came so far as to create a little utility built on top of setuptools that downloaded the Python Package Index (aka cheeseshop, aka PyPI) using httplib2 with caching.

Searching for pypi using this utility I saw a couple of packages, including one called yolk that promised to be a “Command-line tool querying PyPI and Python packages installed on your system.” Great! Saved me a lot of work.

Yolk not only allows you to search PyPI using yolk -S <name>, it also gives you a way to query the packages that you have installed on your system using yolk -l. I’ve NEVER seen this before for Python eggs, so it was quite fun. Yolk doesn’t cache stuff from the PyPI, however, so that’s a bit of a downer for us living in bandwidth-challenged areas like China.

If you wonder, yolk is a word play on “egg yolk“, which is the yellow part of the egg. Python packages as done by setuptools are called eggs, thus the name.

By the way, httplib2, although not part of the standard library, is quite a bit better than urllib2 or httplib, and apparently the only way to easily make HTTP requests that use HTTP methods besides GET and POST; i.e., PUT, DELETE, HEAD, etc, as required by REST. It also exposes a very feature-complete client-side cache, so you can do:

I hope the standard library cleanup project that is planned for Python 3000 includes this module, and updates it so that it can support file-like interface to reading, not just returning a string like it does now. A compatibility module with the old http/url modules would be cool too.

Slides from “Django Master Class” at OSCON

At OSCON 2007 Jacob Kaplan-Moss, Simon Willison, and Jeremy Dunck held a Django Master Class. Jacob just posted the slides on the web, and it contains a few goodies, like:

• testing strategies
• testing support in Django
• dumpdata
• templatetags introduction
• custom model fields
• OpenID (which seems to be on Simon’s mind a lot lately)
• a cool example using jQuery to do Ajax-style client-side form validation based on conditions set in the Django model(!)
• deployment options for high-performance sites
• Geo Django + PostGIS for geo-coordinate manipulation

RESTful Web Services, a book review

After hearing about it on an ITC podcast (also mentioned in previous post) I ordered the “RESTful Web Services” book from Amazon (and got into an argument with Sam Ruby regarding Python pain points; he seems to be a Ruby-convert these days). It arrived last week, and I spent a lot of this weekend browsing through the book, and this is a review of sorts.

This book has a star rating of 5 out of 5 on Amazon, and I agree it’s worth it. It covers a lot of ground and includes material rarely found elsewhere.

The first part covers REST basics, like the meaning and usage of the HTTP methods GET, PUT, DELETE, POST, as well as HEAD. I’m pretty familiar with that part of REST and HTTP, but people that don’t know the basic premise of REST, and haven’t heard of conditional GETs, partial GETs, caching, content negotiation, compression, and the various HTTP return codes should read the first part carefully; the whole premise of REST is to use HTTP the way it was designed, so you’d better know HTTP. The REST Wikipedia entry gives a good basic intro to REST. I find many web site optimization sites give a good introduction to HTTP features.

The authors also show how to use existing best-practice web services such as Amazon’s S3 and Google.

The meat of the book is in the middle where the authors show how to design a read-only and read-write web application in a RESTful way. There were a few new and interesting things for me:

1. A step-by-step guide to designing a RESTful web services app
2. A discussion of URI design conventions
3. An overview of hypermedia formats (e.g., XML and microformats) that can be used to represent state
4. Discussions about the tricky situations where REST just doesn’t seem to fit, e.g., when you need transactions

I’ll summarize the first three.

A guide for building RESTful web services apps

Or, as they say in the book, a procedure for building resource-oriented architectures:

1. Figure out the data set
2. Split the data set into resources
   For each kind of resource:
3. Name the resources with URIs (see below)
4. Expose a subset of the uniform interface (GET, PUT, DELETE, POST, HEAD)
5. Design the representation(s) accepted from the client (see below)
6. Design the representation(s) served to the client (see below)
7. Integrate this resource into existing resources, using hypermedia links and forms
8. Consider the typical course of events: what’s supposed to happen?
9. Consider error conditions: what might go wrong?

URI design conventions

This is important, and fortunately easy to get right once you know how to align yourself with the biggest conventions:

1. use path variables to encode hierarchy, e.g., /parent/child
2. use punctuation characters in path variables to avoid implying hierarchy where non exists
   1. Use semicolon when the order is not important, e.g., /parent/child1;child2
   2. Use comma where the order is important, e.g., /coordinate/123.12,42.5
3. use query variables to imply inputs into an algorithm, e.g., /search?q=jellyfish&lang=zh
4. if you need key=value pairs in the middle of the URI you can use matrix URIs, e.g., /demographic/age=30;gender=m/persons/

Google’s GData Reference explains how GData adds some more URI requirements:

1. categories should be encoded in the URI prepending them with /-/, for example /classifieds/-/jobs/available

2. arguments in the query string that have special meaning include:

   1. q for full-text queries
   2. author for author names
   3. updated-min/max to specify ranges which documents must have been updated in
   4. published-min/max is similar to updated-min/max but for the publication time instead of updated time
   5. max-results limits number of documents per page
   6. start-index sets which page to start from

Hypermedia formats

There are many data/representation formats in the world. XML lead the last revolution, going from obscure binary formats to standardized textual formats.

But although XML is good as an interchange format, it’s not exactly very readable by humans. For that we have a more specific XML document type, HTML. But HTML the way it’s been used before is mostly geared towards visual presentation for humans. Isn’t there a format that can satisfy both humans and computers?

Enter microformats, or the method of using specific HTML tags and CSS class names to mark up normal HTML so that it’s both easy to visually render nicely, and to parse and semantically understand by programs. Most formats you would like to represent are either defined or being defined already: events, contact information (people, organizations, companies, places), reviews, resumes, geographic locations, Atom (weblog) entries, voting links, and listings and classifieds.

Often standardization is just agreeing on names for concepts, their meaning, and their relationships. There is an important work called the Dublin Core that defines common attributes for all content objects. For lists of content objects, RSS and Atom have defined the terms.

I was intrigued by how the different standards built on top of each other, and how it would be possible to achieve maximum compatibility and richness.

• Atom seems to be the base for many initiatives
• OpenSearch is Atom with some more information useful for search results, like total number of results
• Google’s GData includes OpenSearch and adds even more attributes
• The Atom Publishing Protocol defines a web service for the typical case of listings and publishing Atom entries; any blog service that supports this could be used by 3rd party blog publishing applications

A normal web application consists of list pages and detail pages, and it should be possible to make most detail pages use microformats, and most list pages use microformats and provide a GData-compatible feed and query mechanisms.

Painful…

My good friend Glyn sent me this link:

Although I’m guilty of my share of broken-and-painfully-hard-to-understand Chinese, and there’s plenty of Chinglish around in Beijing, this HDD installation instruction manual is in a league of its own.

Do we really need separate fields for first and last names?

This post shows how name components, which one is sorted on, and which one is normally used, vary around the world. The Chinese section is a bit weak as it doesn’t mention common titles such as 老毛 and 小毛, and the common use of nicknames, often by doubling given name characters like 东东, but the author gets his point across. Maybe it’s time to just use a full name field, and perhaps a field for an alias or preferred shorter name?

“Zope is back” movie trailer

Martijn Faassen (whom apparently started EuroPython years back) did the funniest lightning talk at this year’s EuroPython, promoting his new web framework Grok, built on Zope:

Talking to him, he seemed very passionate to reinvigorate Zope through Grok. I must say I think both Zope 3 and Grok is a big step forward compared to Zope 2.x, and I’m sure they’ll pick up more developers over time. It’s going to take some time for me, though, who got pretty burnt by the Zope/ZODB combination, to give it another try again. And I don’t think I’ll ever trust a database that doesn’t explicitly enforce a schema; it’s just a recipe for maintenance hell.