The Corruption of Python

Wed, 01 Jan 2025 13:02:57 +0000

I enjoy doing silly things with code - having an idea that makes me chuckle and then figuring out how to make it happen. These are often fun diversions, an exercise in pushing my limits and Python's - but sometimes they turn into proper projects which I release, other people use, and I then need to maintain. As a result, a lot of my projects harbour a dark secret or two, and I've used a lot of techniques to hide them away and protect my users from the troubles they can bring. I'm going to talk about these techniques in this and the next few posts.

The Zen of Python says that simple is better than complex, explicit is better than implicit, and beautiful is better than ugly. I sometimes say that my projects follow The Corruption of Python: complex makes the simple possible, implicit is simpler than explicit, and beauty is in the eye of the beholder.

The most obvious way for a library to move code away from the user is to put it in a function for them to call. Python takes this a step further and gives us decorators - not controversial or particularly complicated, but it's a good place to start. They're a quick and easy way to move boilerplate functionality away from the code you work on regularly. A contrived example would be checking an argument is a string:

def write_to_screen(var):
    if not isinstance(var, str):
        raise ValueError("Expected a string")
    print(var)

Simple enough, but if you do that 10 times, you need to write those 2 lines 10 times. It starts to get messy - the complexity is now with the person writing this code. Decorators let you shift that complexity to somewhere else in your codebase, like a function call but separate from your code, with a neater syntax:

@enforce_string
def write_to_screen(var):
    print(var)

Nice and clean, nothing cluttering your function logic, difficult to mess up. But this comes at a cost - those 2 lines of checking logic are now 6:

def enforce_string(fn):
    def wrap(var):
        if not isinstance(var, str):
            raise ValueError("Expected a string")
        return fn(var)
    return wrap

And this is a contrived example - things can get much worse in the real world.

Mara is my asynchronous networking library, which uses decorators to register event handlers:

@server.on(events.Receive)
async def echo(event: events.Receive):
    event.connection.write(event.data)

Here we have an echo function which receives an event and sends the same data back to the same client. We bind that to the server using the server.on decorator, which says "listen for the Receive event and pass it to this function". It's about as simple and clear as networking can get.

But it comes at a cost - the code behind that simple decorator involves two classes and multiple functions. I didn't need to do it like this - I could have exposed the event lookup dictionary on the server object, and told Mara users to append their listener callbacks to that directly.

By doing it this way I've made the library easier to use, but that one line decorator is now hiding significant complexity, and if there's a problem in there it's going to be pretty difficult for someone using Mara to figure out what has gone wrong. By making the choice to simplify Mara's API, I've raised the barrier to entry for any potential contributors.

I'm happy with that - my goal is to write a library which makes networking easier, and this achieves that. With tests I can mitigate the risk and minimise the hassle. But the point I'm making is that this has increased my maintenance burden - and this burden is only going to get worse as we progress through this series of posts.

So as I said, decorators are neither controversial nor all that complicated, but this does set the scene for where we're going. Next time, we'll talk about metaclasses.

Mara - a Python network service framework

Sun, 13 Dec 2015 13:34:44 +0000

I've released a new version of Mara, my network service framework written in Python. It aims to make it easy to build TCP/IP services, such as echo servers, flash policy servers, chatrooms, talkers and MUDs.

It's event-based; that is to say you write event listener functions which you bind to events that your service raises - like Connect, Receive or Disconnect.

Mara is on pypi, so you can pip install mara, then start writing your service. An echo server in Mara looks like this:

from mara import Service
service = Service()

@service.listen(mara.events.Receive)
def receive(event):
    event.client.write(event.data)

if __name__ == '__main__':
    service.run()

You can then save it as echo.py and run the service by calling it:

python echo.py
* Server listening on 127.0.0.1:9000

That's a pretty simple example, but Mara can do a bunch more. Its core has support for things like telnet negotiation, timers, a storage system, and seamless restarts (where client connections and storage objects persist, but your code is reloaded cleanly), and it ships with a contrib module which has a lot of optional extras, such as a command manager and dispatcher, basic natural language tools, user accounts and rooms.

Although there's a focus on talkers and muds in the contrib modules at the moment, Mara should be a reasonable base for writing any network service. To get a feel for what you can do with it, take a look at the examples, which include an IRC-style chat server, a simple talker, and the start of a basic mud. There's also fairly comprehensive documentation.

I've always enjoyed writing this sort of thing, so this is a fun side project for me. At its heart Mara is a rewrite of my old perl chat server Cletus, which I wrote in 2001 - in fact if you dive back a few months through git, you'll see Mara was called Cletus until I realised that name was taken on pypi.

It's still missing a few glaringly obvious features at the moment - most notably unicode and python 3 support, an example of using threads through events and timers, and more contrib modules for the mud like items, combat and NPCs. That said, it should make a solid starting point for any network service that you'd want to write, and as always, contributions are welcome.

Radiac's Blog: mara

The Corruption of Python

Mara - a Python network service framework