The Corruption of Python

I enjoy doing silly things with code - having an idea that makes me chuckle and then figuring out how to make it happen. These are often fun diversions, an exercise in pushing my limits and Python's - but sometimes they turn into proper projects which I release, other people use, and I then need to maintain. As a result, a lot of my projects harbour a dark secret or two, and I've used a lot of techniques to hide them away and protect my users from the troubles they can bring. I'm going to talk about these techniques in this and the next few posts.

The Zen of Python says that simple is better than complex, explicit is better than implicit, and beautiful is better than ugly. I sometimes say that my projects follow The Corruption of Python: complex makes the simple possible, implicit is simpler than explicit, and beauty is in the eye of the beholder.

The most obvious way for a library to move code away from the user is to put it in a function for them to call. Python takes this a step further and gives us decorators - not controversial or particularly complicated, but it's a good place to start. They're a quick and easy way to move boilerplate functionality away from the code you work on regularly. A contrived example would be checking an argument is a string:

def write_to_screen(var):
    if not isinstance(var, str):
        raise ValueError("Expected a string")
    print(var)

Simple enough, but if you do that 10 times, you need to write those 2 lines 10 times. It starts to get messy - the complexity is now with the person writing this code. Decorators let you shift that complexity to somewhere else in your codebase, like a function call but separate from your code, with a neater syntax:

@enforce_string
def write_to_screen(var):
    print(var)

Nice and clean, nothing cluttering your function logic, difficult to mess up. But this comes at a cost - those 2 lines of checking logic are now 6:

def enforce_string(fn):
    def wrap(var):
        if not isinstance(var, str):
            raise ValueError("Expected a string")
        return fn(var)
    return wrap

And this is a contrived example - things can get much worse in the real world.

Mara is my asynchronous networking library, which uses decorators to register event handlers:

@server.on(events.Receive)
async def echo(event: events.Receive):
    event.connection.write(event.data)

Here we have an echo function which receives an event and sends the same data back to the same client. We bind that to the server using the server.on decorator, which says "listen for the Receive event and pass it to this function". It's about as simple and clear as networking can get.

But it comes at a cost - the code behind that simple decorator involves two classes and multiple functions. I didn't need to do it like this - I could have exposed the event lookup dictionary on the server object, and told Mara users to append their listener callbacks to that directly.

By doing it this way I've made the library easier to use, but that one line decorator is now hiding significant complexity, and if there's a problem in there it's going to be pretty difficult for someone using Mara to figure out what has gone wrong. By making the choice to simplify Mara's API, I've raised the barrier to entry for any potential contributors.

I'm happy with that - my goal is to write a library which makes networking easier, and this achieves that. With tests I can mitigate the risk and minimise the hassle. But the point I'm making is that this has increased my maintenance burden - and this burden is only going to get worse as we progress through this series of posts.

So as I said, decorators are neither controversial nor all that complicated, but this does set the scene for where we're going. Next time, we'll talk about metaclasses.

Leave a comment