fíam

Well, let me start by explaining all those mails about byNotes I haven't answered in the last few months. Around mid November I got contacted by the recruiting staff of a social network. At that time, I was still working in the project for my degree (computer science students in Spain are required to write a project for the university in order to get the degree) and I wanted to concentrate on it, but I decided to just take the interviews just in case. They finally told me they wanted me to join them and we agreed to continue the negotiations once I had finished my project.

Fast forward to January, my project is done and christmas holidays are gone, time to reestablish negotiations. Salary is good, schedule is very good (telecommuting 99% of the time) but there's something bad: I must leave byNotes, since management considers it's competing with them.

It wasn't an easy decision, but it was the best option. byNotes has been very fun, but also such a time drain to me, and I haven't got any single eurocent from it. Plus, I'm a coder, not a designer nor a business guy, and I don't have too much money to spend on it. So I really don't think I could make it profitable.

As of today, I've been already working for my new employer for some weeks. My name still figures as the domain holder for bynotes.com and in some about pages, but that should change soon. I'm sure the new maintainers will do a good job at keeping the project alive.

And now comes the interesting part for you, developers. What happens with my free software projects? Well, I'm keeping all of them, but I won't be able to spend some much time working on them. Since I won't be owning byNotes anymore, I'm moving them to GitHub and using LightHouse for ticket tracking. The following is the list of projects I'll be moving to GitHub in the next few days:

• Django projects

  • Blango
  • wapi
  • django-bundles
  • django-geonames
  • django-geocoding
  • django-storage
  • django-mediafiles
  • django-oauthsp

• Python projects

  • ffmpeg bindings

• Cocoa projects

  • My fork of OAuthConsumer (fixing some bugs, implementing some goodies)
  • BNMaps (map framework)

Since I started writing byNotes I wanted to release all the code under a free software license. After all, would you trust a service you cannot audit?

However, since it all started as a small project in my free time (it's now about 30k sloc), the source code was all in one big repository and preparing it for release took some time. First, I needed to evaluate some SCM systems because I wanted to split the code into subprojects and I wasn't sure how good git-submodule was. Finally, after some analysis and some advice from newman on #django-hotclub, I went with git for the SCM and Redmine for bugtracking.

Not all code is ready for release yet, but there are some subprojects inmediately available at the byNotes Source Code Repository. Let me introduce them:

• django-bundles: Provides an easy way to create JS and CSS bundles, with optional support for compression and validation using YUI and rhino.
• django-geocoding: Uses the Google APIs and django-geonames for geocoding and reverse geocoding.
• django-geonames: Models for working with the geonames.org database.
• django-storage: Application for storing user-uploaded files
• django-dboembed: oEmbed client using the database for caching.

wapi and django-oauthsp, have also been released and, the best news for some of you, all these apps are licensed under MIT license (wapi and django-oauthsp were under AGPL3), so you can use all this code with your closed source projects.

There are more releases coming soon, including all the source code for byNotes as well as the source code for the iPhone client.

I've just finished my first iPhone application: a client for byNotes. If you're interested in betatesting it, just send me an email with your device identifier.

This entry puts and end to "Building a website API with Django". I think I've covered everything you need to build a website API with Wapi, so it's now time for you to start playing with it. I hope you've enjoyed reading this series as much as I enjoyed writing it. I've finally pylinted and documented almost all the code, so Wapi 0.2 should be released in the next few hours. From now, I'll try to keep the current API and every backwards incompatible change will be documented.

Documenting your API with Wapi

Documenting your functions

As we've seen on part 4 on this series, every API parameter takes an optional third argument which documents it. In addition, every function docstring is also passed to the documentation template. Furthermore, validators applied to parameters are also autodocumented. Let's see an example:

@readonly
@required_parameter('type', int, _('Admin division type, ranging from 1 to 4.'), validators=RangeValidator(min_value=1, max_value=4))
@required_parameter('id', int, _('The administrative division id.'))
def geo__admin(self, request, dct):
    """Returns information for the given administrative division type and id. Keep
    in mind that administrative divisions identificators are not guaranteed to
    be permanent, as it happens with geoname identifiers."""

If you want to check how it looks when passed to the documentation, check the byNotes API documentation.

urls.py

You'll also need to plug your documentation view into your urls.py, but it's pretty simple:

from wapi.documentator import ApiDocumentator
from notes.api import BynotesApi

urlpatterns = ...
(r'^api/1.0/doc/', ApiDocumentator(api=BynotesApi()))

Grouping functions into namespaces

By default, Wapi orders your functions alphabetically. However, usually you'll want to reorder this namespaces and given them friendly names. In Wapi, you can do this using the NAMESPACES class attribute in your API class:

class BynotesApi(object):
    NAMESPACES = ( 
        (_('User information'), 'user'),
        (_('Positions'), 'position'),
        (_('Fetching notes'), 'notes'),
        (_('Friends'), 'friends'),
        (_('Posting notes'), 'post'),
        (_('Geographic functions'), 'geo'),
    )

The format of the tuples is (namespace_friendly_name, namespace_prefix).

Example template

     {% for namespace in namespaces %}
    <div class="api-ns" id="ns-{{ namespace.short_name }}">
        <h2>{{ namespace.name }}</h2><ol>
        {% for function in namespace %}
        <li id="{{ function.name }}" class="function">
            <h3>
                {{ function.endpoint }}.{ format }
                {% if function.requires_login %}
                    <span class="warn">{% trans "Requires login" %}</span>
                {% endif %}
            </h3>
            <span>{% trans "allowed methods" %}:</span>{% if function.is_read %} GET{% endif %}{% if function.is_write %} POST{%endif %}
            <p class="fdoc">{{ function.doc }}</p>
            {% if function.required_parameters %}
                <h4>{% trans "Required parameters" %}:</h4>
                {% for parameter in function.required_parameters %}
                    <p class="parameter">
                        <span>{{ parameter.name }} ({{ parameter.type_name }}):</span>
                        {{ parameter.doc }}
                    </p>
                    {% if parameter.doc_info %}
                        {% for name, value in parameter.doc_info %}
                            <p class="parameter-info">
                                <span>{{ name }}:</span>
                                {{ value }}
                            </p>
                        {% endfor %}
                    {% endif %}
                {% endfor %}
            {% endif %}
            {% if function.optional_parameters %}
                <h4>{% trans "Optional parameters" %}:</h4>
                {% for parameter in function.optional_parameters %}
                    <p class="parameter">
                        <span>{{ parameter.name }} ({{ parameter.type_name }}):</span>
                        {{ parameter.doc }}
                    </p>
                    {% if parameter.has_default %}
                        <p class="parameter-info">
                            <span>{% trans "Default" %}:</span>
                            {{ parameter.default }}
                        </p>
                    {% endif %}
                    {% if parameter.doc_info %}
                        {% for name, value in parameter.doc_info %}
                            <p class="parameter-info">
                                    <span>{{ name }}:</span>
                                {{ value }}
                            </p>
                        {% endfor %}
                    {% endif %}
                {% endfor %}
            {% endif %}
        </li>
        {% endfor %}
        </ol>
    </div>
{% endfor %}

Current limitations

As you may have noticed, translating function documentation is currently not supported in Wapi, since the docstring must be a bare string. Some work has been done on this, but the current solution requires modifications to makemessages, so I think it was better to leave this changes for Wapi 0.3, which will be released post Django 1.0.

Let me start this entry announcing the first beta releases of wapi-0.2 and django-oauthsp-0.2. Keep in mind this code is released under AGPLv3, as per the reasons detailed in this entry. Note also that the API is subject to change before reaching 0.2, this code should be considered as a preview release.

Now let's talk about <serializations />!

Principles

Wapi serializers, as opposed to the Django serializers, let you fully customize how an object is serialized. You can even specify multiple ways of serializing an object, depending on multiple things. However, this flexibility comes with a price: complexity. Declaring serializers in Wapi is not an easy task. So let's start explaining the basics.

Class based approach (again)

Serializations are specified as a class, which inherits from wapi.serializers.Serializer, defining multiple methods which return a dictionary. Every method is a serialization "preset", which you can specify at the time you serialize the objects. Let's see a brief example:

from wapi.serializers import Serializer
from wapi import serializers
from notes.models import UserProfile

class UserProfileSerializer(Serializer):
    serializes = UserProfile
    @serializers.objname('user')
    def default(self, obj, **kwargs):
        return {
            'username': obj.user.username,
            'display_name': obj.display_name,
            'avatar32': obj.get_avatar_url(32),
            'avatar128': obj.get_avatar_url(128),
        }

As you see, we've defined a class with the serializes attribute. This is very important, since serializers are registered via metaclass and this attribute tells wapi the object type this serializer applies to. The default method is called when you serialize an object without specifying any presets. On the other hand, the objname decorator tells wapi how the object should be named. By default, the default method will use the class name specified in serializes, while other methods will use the method name. Finally, keep in mind this code needs to be imported, but you can put whereever you want. I usually add a serializers.py file to every application and then I import it from the models. Let's see how this would look when serialized:

In [1]: from notes.models import UserProfile

 In [2]: from wapi.responses import XmlResponse, JsonResponse, YamlResponse

In [3]: print XmlResponse(UserProfile.objects.filter(pk=1))
     Content-Type: application/xml

     <?xml version='1.0' encoding='utf8'?>
     <objects><userprofile><username>fiam</username><avatar32>http://django:8000/_m/avatars/32/e12/eq4/nrxz.jpg</avatar32><display_name>Alberto G.</display_name><avatar128>http://django:8000/_m/avatars/128/e12/eq4/nrxz.jpg</avatar128></userprofile></objects>

 In [4]: print JsonResponse(UserProfile.objects.filter(pk=1))
      Content-Type: application/json

      [{"username": "fiam", "avatar32": "http://django:8000/_m/avatars/32/e12/eq4/nrxz.jpg", "display_name": "Alberto G.", "avatar128": "http://django:8000/_m/avatars/128/e12/eq4/nrxz.jpg"}]

  In [5]: print YamlResponse(UserProfile.objects.filter(pk=1))
       Content-Type: application/x-yaml

       userprofile: {avatar128: 'http://django:8000/_m/avatars/128/e12/eq4/nrxz.jpg', avatar32: 'http://django:8000/_m/avatars/32/e12/eq4/nrxz.jpg', display_name: Alberto G., username: fiam}

How Wapi saves you from DRY

When eating my own dogfood (id est, using Wapi when building byNotes), I ended up repeating a lot of code in multiple serializers, so I added multiple methods for including and extending another serializations.

Extending

extends is the most simple method for extending a serialization. Given method foo and method bar, which extends the former, the dictionary returned by bar updates the dictionary returned by foo before passing it the formatters. So, for example, that's how I add the date to a serialization:

class PositionSerializer(Serializer):
   serializes = Position
   def default(self, obj, **kw):
        return {
            'latitude': obj.latitude,
            'longitude': obj.longitude,
         }

    @serializers.objname('position') # I want the object to be named "position", not "when"
    @serializers.extends('default')
    def when(self, obj, **kw):
         return {
            'when': localdatetime(obj.when),
          }

  # From a response:
  # JsonResponse(Position.objects.all(), method='when')
  # From a Wapi function:
  # SerializableResponse(Position.objects.all(), method='when')

Chaining and including

Sometimes an object includes objects of other types. chain will call the specified serialization method over the object and return the serialization dictionary. You can use it as a return value from a serialization method or merge it with the current serialization before returning. Another approach is including, which includes the serialization value inside an object field. For example, in the following code I include the profile of the note submitter in the user field and then I merge it with the position:

# Using chain as a return value
class EventSerializer(Serializer):
    serializes = Event
    def add_when(self, obj, **kw):
        return {
            'event_date': obj.localized_when(),
        }

    def add_place(self, obj, **kw):
        note = kw['note']
         return chain(note.position)

    @extends('add_when')
    @extends('add_place')
    def default(self, obj, **kw):
        return proplist(obj, ('what', 'place_name'))

class NoteSerializer(Serializer):
    serializes = Note
    def default(self, obj, **kw):
        base = {
            'permalink': 'http://%s%s' % (settings.SITE_NAME, obj.get_absolute_url()),
            'id': obj.id,
            'type': obj.note_type,
            'message': obj.message,
            'user': serializers.include(obj.submitter.get_profile()),
            'pubdate': localdatetime(obj.submitted).strftime('%F %T %z'),
            'public': obj.public,
        }
        return merge(base, chain(obj.position))

Passing arguments to serializations

As you may have noticed, every serialization methods receives a **kw parameter. This allows for a great flexibility, since all the responses (XmlResponse, JsonResponse, YamlResponse, SerializableResponse and SingleSerializableRespose) as well as the helper functions (chain and include) accept **kwargs and pass it to the serialization method. For example, take a look at the add_place method in the EventSerializer class:

def add_place(self, obj, **kw):
    note = kw['note']
     return chain(note.position)

So, when we serialize, chain or include an event, I do it in the following way:

chain(note.event, note=note)

proplist

After writing some serialization methods, I found myself repeating this pattern a lot:

class FooSerializer(Serializer):
    serializers = Foo
    def default(self, obj, **kw):
         return {
             'bar': obj.bar,
             'baz': obj.baz,
         }

So I added a new method, which takes a list of properties and returns a serialization formed by the properties and their values:

class FooSerializer(Serializer):
    serializers = Foo
    def default(self, obj, **kw):
         return proplist(obj, 'bar', 'baz')

Feed back

As I have mentioned, the API in Wapi and django-oauthsp is subject to change. I wrote this articles hoping to get some feedback for improving the API before the release, but I haven't received any. So, now that the code is available, download it and play with it a bit. I'm sure there's a lot of room for improvement.

Next articles in this series

In the next (and final) article in this series I will talk about wapi.documentator, which takes care of documenting your API. If you want to see how it looks, check the byNotes API documentation.