fíam

It's been a few days since I posted my decision to release all my code related to Django under the AGPLv3 license. It generated quite a few reactions I didn't expect, so let me tell you the full story.

It all started when I needed to support, let's call it technology X - which is pretty hot right now -, in byNotes. I first searched if someone had done it and I found there was a generic and abstract Python implementation, but it needed a lot of work to be adapted to Django. So I decided to build my own solution from scratch. The same day I released the first version, I found an announcement posted the previous day from another developer who had also coded technology X support for Django. It's a pity when that happens, since one of us could have spent that time implementing technology Y. However, it's difficult to avoid those situations, so we must live with it.

The bad thing (TM) happened a few days later, when I was chatting with a friend on freenode. I told him I've written support for technology X and it could be useful for a project he's working on. He told me about a relatively popular site - I didn't know it was built on Django - which had been supporting technology X for a few months. I checked the site and I found nothing about the source code for technology X, even when it's something potentially useful for a lot of sites.

I don't blame them for not releasing their technology X implementation, after all they're free to do what they want with their code as long as they respect the licenses for other projects or frameworks they may rely on. However, the Django community lost something in this scenario. If relatively popular site had released their implementation of technology X, the other developer would have worked on Y while I would have worked on Z. Django would now have support for X, Y and Z, but thanks to the decision of relatively popular site to keep their implementation of X closed, we now have 2 free implementations of X. And that hurts the community, since now someone will have to spend time working on Y and Z instead of doing another things.

Are you starting to see my point? Now, with my code released under AGPLv3 instead of BSD, any site using it would have to contribute their changes back. So, if they improve my implementation of technology X, I can spend more time working on technology Y and that benefits the Django community.

I fail to see how this is being greedy. And if, as someone suggested, my code is so trivial, there's no problem reimplementing it. Anyway, I didn't claim my code does something extraordinary. In fact, I'm sure anyone could implement it. When I release something I've written, my intention is not rocking your world, but saving you from wasting time. If I've already written support for X, you're better working on Y instead of reinventing the wheel.

However, I'm not totally happy with the AGPLv3, because of its viral nature. I would be fine with something like the LGPL but with the Affero clause and I'm sure I'm not alone. Id est, you may link closed source code with my software, but you must release any modifications you make to my code. However, there's no such license for now, so I have to use AGPLv3. Do you want this to change? Spread the word, I'm sure a new license by the FSF combining the LGPL with the Affero clause it's going to benefit us all.

I've just finished preparing the Wapi code for the 0.2 release, so you can now download it. oauthsp will be ready in a few days (probably this weekend).

Simon has recently featured this snippet which shrinks a SHA1 hash from 40 to 27 characters using base65 encoding. I've been using another approach for some time and, modestly, I think mine is better, so let me tell you how I do it before you start using the suboptimal approach.

Python does it for us

First, the base64 module already provides a pair of functions, aptly named urlsafe_b64encode and urlsafe_b64decode, which do quasi-safe url encoding using base64. For example:

>>> base64.urlsafe_b64encode(sha1('foobar').digest())
'iEPX-SQWIR3p67lj_0zigSWTKHg='

The problem here is the equal sign, since we can't put it safely in a URL. However, since we are using this to store a hash and we're going to generate the hash again when we verify the data, we can encode the hash again instead of decoding the base64-encoded string, which lets us safely remove the equal sign. Furthermore, we are also using 27 characters, the same as the other more complex implementation.

Can we decode it? Sure!

Sometimes, you'll want to decode the base64 encoded data. However, once we remove the equal sign, we can't no longer pass the base64 encoded string to urlsafe_b64decode:

>>> base64.urlsafe_b64decode('iEPX-SQWIR3p67lj_0zigSWTKHg')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.5/base64.py", line 112, in urlsafe_b64decode
    return b64decode(s, '-_')
File "/usr/lib/python2.5/base64.py", line 76, in b64decode
     raise TypeError(msg)
 TypeError: Incorrect padding

Let's examine how base64 works. The input string is processed in 3-byte blocks which are split into 6-bits pieces. Hence, we have 2^6 == 64 possible values per piece which translates to a single character into our base64 alphabet. So, every three byte block is translated to a 4 characters string. When the last block has less than three bytes, it's padded with zeros and one equal sign is added for every padding byte. This leads us to the conclusion that base64 encoded strings always have a length divisible by four. Let's take advantage of this:

>>> s = 'iEPX-SQWIR3p67lj_0zigSWTKHg'
>>> base64.urlsafe_b64decode(s + '=' * (4 - len(s) % 4))
'\x88C\xd7\xf9$\x16!\x1d\xe9\xeb\xb9c\xffL\xe2\x81%\x93(x'

So now we have our original string back.

Putting it all together

from base64 import urlsafe_b64encode, urlsafe_b64decode

def uri_b64encode(s):
     return urlsafe_b64encode(s).strip('=')

def uri_b64decode(s):
     return urlsafe_b64decode(s + '=' * (4 - len(s) % 4))

I think there are lots of people which misunterstand the Affero GPL v3, so I decided to write this entry to clarify some things.

Keep in mind I'm not a lawyer, so you should read my advice with a grain of salt. I've been using free software licenses for some years and I always take the time to read and understand them, but that doesn't mean I cannot make mistakes or misunderstand some sections.

You can

• Sell commercial software under an AGPLv3 license: Nobody prevents you from selling AGPLv3 licensed code, even when you didn't write it. However, customers buying the software from you must abide by the license terms. Paying for an AGPLv3 product doesn't mean you can avoid the license terms. However, if you're the copyright owner, you can sell the same software under a different license.
• Build another software on top of it: AGPLv3 grants you permission to download and modify any software which you can use, even if it's over a network (this an important difference with the GPLv3). You're free to modify it or use it into another project, making your project a derivative work of the original software.
• Link to or use BSD licensed code: 3-clause BSD licensed code (don't confuse it with 4-clause BSD license) let's you use or import it into AGPLv3 projects. You can also distribute both codebases as a single package. However, keep in mind that you are not allowed to change the license in the file headers nor claim copyright over the external project you imported.
• Look at code under AGPLv3 license, even when you work on projects with more restrictive licenses: AGPLv3 is about code freedom, not about patents nor algorithms. You're free to look at AGPLv3 code and then do a clean-room reimplementation of it.
• Use GPLv3 libraries: Using a GPLv3 library doesn't impose additional restrictions over the library nor over the AGPLv3 code, so there's no problem here.
• Use LGPLvx libraries: No version of the LGPL imposes any restrictions over the code using it.

You can't

• Write BSD or closed source software on top of it: AGPLv3 is designed to keep the code always free, so you can't use nor import AGPLv3 code into a closed source project.
• Copy AGPLv3 code into another project under a non-AGPLv3 license: Even a single line. It's no allowed, because you're bypassing the license term under the code was released.
• Mix GPLv3 and AGPLv3 code: This may seem counterintuitive, but it's true. You can't make a GPLv3 derivate work and an AGPLv3 software, since you're removing the obligation to distribute the source to users over a network. On the other hand, you can't add the Affero clause to a GPLv3 codebase, since you're imposing additional restrictions on it. The same applies for GPLv2.
• Use GPLv2 libraries: GPLv2 only libraries don't allow you to use them in GPLv3 projects, so you're in the same case when using AGPLv3.

You must

• License your derivative works under AGPLv3: If you modify AGPLv3 code or use it in another project, the resulting code must be under an AGPLv3 license.
• Provide a way to download the code over the same medium used to deliver the software service to the user: If you run a website which uses AGPLv3 code, your pages must provide a link to the software you're running in the webserver (zipfile or version control system, it doesn't matter).

For Django developers

Django is released under a 3-clause BSD license, so you're free to build AGPLv3 software on top of it.

This entry puts and end to "Building a website API with Django". I think I've covered everything you need to build a website API with Wapi, so it's now time for you to start playing with it. I hope you've enjoyed reading this series as much as I enjoyed writing it. I've finally pylinted and documented almost all the code, so Wapi 0.2 should be released in the next few hours. From now, I'll try to keep the current API and every backwards incompatible change will be documented.

Documenting your API with Wapi

Documenting your functions

As we've seen on part 4 on this series, every API parameter takes an optional third argument which documents it. In addition, every function docstring is also passed to the documentation template. Furthermore, validators applied to parameters are also autodocumented. Let's see an example:

@readonly
@required_parameter('type', int, _('Admin division type, ranging from 1 to 4.'), validators=RangeValidator(min_value=1, max_value=4))
@required_parameter('id', int, _('The administrative division id.'))
def geo__admin(self, request, dct):
    """Returns information for the given administrative division type and id. Keep
    in mind that administrative divisions identificators are not guaranteed to
    be permanent, as it happens with geoname identifiers."""

If you want to check how it looks when passed to the documentation, check the byNotes API documentation.

urls.py

You'll also need to plug your documentation view into your urls.py, but it's pretty simple:

from wapi.documentator import ApiDocumentator
from notes.api import BynotesApi

urlpatterns = ...
(r'^api/1.0/doc/', ApiDocumentator(api=BynotesApi()))

Grouping functions into namespaces

By default, Wapi orders your functions alphabetically. However, usually you'll want to reorder this namespaces and given them friendly names. In Wapi, you can do this using the NAMESPACES class attribute in your API class:

class BynotesApi(object):
    NAMESPACES = ( 
        (_('User information'), 'user'),
        (_('Positions'), 'position'),
        (_('Fetching notes'), 'notes'),
        (_('Friends'), 'friends'),
        (_('Posting notes'), 'post'),
        (_('Geographic functions'), 'geo'),
    )

The format of the tuples is (namespace_friendly_name, namespace_prefix).

Example template

     {% for namespace in namespaces %}
    <div class="api-ns" id="ns-{{ namespace.short_name }}">
        <h2>{{ namespace.name }}</h2><ol>
        {% for function in namespace %}
        <li id="{{ function.name }}" class="function">
            <h3>
                {{ function.endpoint }}.{ format }
                {% if function.requires_login %}
                    <span class="warn">{% trans "Requires login" %}</span>
                {% endif %}
            </h3>
            <span>{% trans "allowed methods" %}:</span>{% if function.is_read %} GET{% endif %}{% if function.is_write %} POST{%endif %}
            <p class="fdoc">{{ function.doc }}</p>
            {% if function.required_parameters %}
                <h4>{% trans "Required parameters" %}:</h4>
                {% for parameter in function.required_parameters %}
                    <p class="parameter">
                        <span>{{ parameter.name }} ({{ parameter.type_name }}):</span>
                        {{ parameter.doc }}
                    </p>
                    {% if parameter.doc_info %}
                        {% for name, value in parameter.doc_info %}
                            <p class="parameter-info">
                                <span>{{ name }}:</span>
                                {{ value }}
                            </p>
                        {% endfor %}
                    {% endif %}
                {% endfor %}
            {% endif %}
            {% if function.optional_parameters %}
                <h4>{% trans "Optional parameters" %}:</h4>
                {% for parameter in function.optional_parameters %}
                    <p class="parameter">
                        <span>{{ parameter.name }} ({{ parameter.type_name }}):</span>
                        {{ parameter.doc }}
                    </p>
                    {% if parameter.has_default %}
                        <p class="parameter-info">
                            <span>{% trans "Default" %}:</span>
                            {{ parameter.default }}
                        </p>
                    {% endif %}
                    {% if parameter.doc_info %}
                        {% for name, value in parameter.doc_info %}
                            <p class="parameter-info">
                                    <span>{{ name }}:</span>
                                {{ value }}
                            </p>
                        {% endfor %}
                    {% endif %}
                {% endfor %}
            {% endif %}
        </li>
        {% endfor %}
        </ol>
    </div>
{% endfor %}

Current limitations

As you may have noticed, translating function documentation is currently not supported in Wapi, since the docstring must be a bare string. Some work has been done on this, but the current solution requires modifications to makemessages, so I think it was better to leave this changes for Wapi 0.3, which will be released post Django 1.0.