fíam

As some of you may be aware of, Google has launched a framework for developing and publishing web applications. Currently, it only supports Python, altough they say they're working on integrating more languages. The interesting thing is their framework it's really similar to Django - it even includes Django's template engine - diffing only on a few points:

• The database layer: Google App Engine apps can't use a standard database, they use the Datastore API.
• User authentication: Google provides a standard API for handling authentication based on Google/Gmail accounts. However, you can still use the Django auth system, provided we have some glue for the database layer.
• Deployment: Google App Engine runs only under CGI, which Django doesn't support. However, there's a (refused) patch in Trac for addressing this (I haven't tested it yet, it may not work).
• Static files: Every file or directory containing static files needs to be listed into a manifest-style file.

Looking at this, running Django apps in Google App Engine seems really easy. We only need a wrapper translating calls from Django ORM to Google Datastore, write a manifest and, perhaps, a bit of tweaking to the mentioned patch. In fact, I'm planning to start working on all these things as soon as I have time. On the other hand, I'm finally launching ffloat.it on April 15th, so I'm a bit busy this week.

As I promised yesterday, here is the next Django-Geonames release. Changes from 0.2 are:

• Incremental updates support (database independant)
• Application renamed from geo to geonames
• Added LICENSE

However, MySQL is still the only database backend supported, because of the geonames-import script. Using the Django ORM for pushing all those records to the database would make it even slower, so I'm most likely to add support for PostgreSQL using it's native API. As for other database backends supported by Django, I don't think you're going to use this database with SQLite and I don't feel like installing Oracle for now (anyway, I'd be happy to accept patches to support it).

django-geonames-0.3.zip

I've made some important changes to django-geonames, fixing get_parent() and get_children(), which were failing on some corner cases. For example: if a third level administrative division wasn't inside a second level administrative division, get_parent() was returning the same object passed as self.

I'm currently working on improving import time, as well as experimenting with compressed tables for faster retrieval. Expect another release by tomorrow.

django-geonames-0.2.zip

I've been using the Geonames database with Django for a while, as part of the ffloat.it project (which will be launched soon, stay tuned ;)). I've done some preprocessing to the data, normalizing some bits and denormalizing others, but always aiming for the best speed. My main needs were:

• Searching locations by name
• Finding the parent for a given location
• Finding children for a given location

I don't need to find locations by coordinates, so this first release is not optimized for that. In addition, this release has been tested with MySQL only, but I think I don't use any MySQL-specific features, so you may be able to use it with PostgreSQL changing the database driver in the geoimport.py script.

Next release, planned for somewhen in the next two weeks, will feature incremental updates support (currently, you'll need to import the full database at once) as well as PostgreSQL support.

Keep in mind this database is really huge, containing more than 6000000 (yep, six millions) locations and more than 2000000 internationalized names. Importing it takes more than an hour in my laptop and it needs about 1.5 GiB of disk space (I've massively indexed it).

Install instructions

Just unzip django-geonames-0.1.zip and drop the geo application inside your project. Add it to your INSTALLED_APPS, syncdb and then go to the geo directory and run the geoimport.py script (you'll need to specify database credentials and db name, run geoimport.py -h for help). I haven't included any documentation for the models, but i think, more or less, they should be self explanatory. Don't hesitate to ask any question, posting a comment to this entry.

I hope you find it usefull.

Acabo publicar mi árbol git de Blango, así como los parches necesarios que hay que aplicar sobre Django para que funcione. Es necesario aplicar el 0003 y el 0005.