Extending time and details

For the fun of it I added an "Extended Room" contrib to Evennia the other night.

("Contribs" are optional code snippets and systems that are not part of the actual codebase. They are intended for you to use or dissect as you like in your game development efforts).

The ExtendedRoom is a room typeclass meant to showcase some more advanced features than the default one. Its functionality is by all means nothing revolutionary in MUD-world, but it was fun and very simple to do using only Evennia's basic building blocks - the whole thing took me some two hours to code, document and clean up for a contrib push. The "ExtendedRoom" contribution has the following features:

• Season-based descriptions. The new Room typeclass will change its overall description based on the time of year (the contrib supports the four seasons, you can hack this as you please). It's interesting from an implementation point of view since it doesn't require any Script or ticker at all - it just checks on-demand, whenever it is being looked at, only updating if the season has actually changed. There is also a general description used as a fallback in case of a missing seasonal one.
• Time-of-day-based descriptions. Within each Season-based description you can embed time-of-day based ones with special tags. The contrib supports four time slots out of the box (morning, afternoon, evening, night). In the description, you just embed time-dependent text within tags, like <morning>Morning sunlight is shining through the windows</morning>. Only time-relevant tags will be shown. This is a simple regular expression substitution, should be easy to expand on if one wants more fine-grained time slots.
• Details. I took the inspiration of these from a MOO tutorial I read a long time ago. "Details" are "virtual" look-targets in the room. It allows you to add visual interest without having to add a bunch of actual objects for that purpose. Details are simply stored in a dictionary on the room. Details don't change with Season in this implementation, but they are parsed for time-of-day based tags!
• Custom commands. The room is supported by extending two of the custom commands. The Details require a slightly modified version of the look command. There is also a new @desc for setting/listing details and seasonal descriptions. The new time command, finally, simply shows the current game time and season in the room.

Installing and testing the snippet is simple - just add the new commands to the default cmdset (they will dynamically replace the same-named default ones), dig a few rooms of the new typeclass and play around! Especially the details do make building interesting rooms a lot more fun (I got hung up playing with them way too long last night).

Coding from the inside

Some time ago, a message on the Evennia mailing list asked about "softcode" support in Evennia. Softcode, a defacto standard in the MUX/MUCK/MUSH/MOO world, is conceptually a "safe" in-game scripting language that allows Players to extend the functionality of things without having access to the server source.

Now, Evennia is meant to be extended by normal Python modules. For coding game systems and advanced stuff, there is really no reason (in my opinion) for a small development team to not use a modern version control system and proper text editors rather than entering things on a command line without formatting.

But there is a potential fun aspect of having an online scripting language - and that is player content creation. Crafters wanting to add some pizazz to their objects, builders getting an extra venue of creativity with their rooms - that kind of thing. I didn't plan to add softcode support to Evennia, but it "sounded like an interesting problem" and one thing led to another.

Python is of course an excellent scripting language from the start. Problem is that it's notoriously tricky to make it run safely with untrusted code - like that inserted by careless or even potentially malignant Players. Scanning the Internet on this topic is a pretty intimidating experience - everywhere you hear that it shouldn't be done, and that the various suggested solutions of a "sandbox" are all inherently unsafe. Python's awesome introspection utilities is its own enemy in this particular case.

For Evennia we are however not looking for a full sandbox. We want a Python-like way for Players to influence a few determined systems. Moreover, we expect short, simple scripts that can do without most of Python's functionality (since our policy is that if it's too complex or large, it belongs in an external Python module). We could supply black-box "safe" functions to hide away complex functionality while still letting people change things we expect them to want to script. This willingness to accept heavy restrictions to the language should work to our advantage, I hope.

Evennia actually already has a safe "mini-language" in the form its "lock system", and thus it was a natural way for me to start looking. A "lock string" has a very restricted syntax - it's basically function calls optionally separated by boolean operators, like this:

lockfunc1(*args) and lockfunc(*args, **kwargs) and not lockfunc2()

The result of this evaluation will always be a boolean True/False (if the lock is passed or not). Only certain functions are available to use (controlled by the coder). The interesting thing is that this string can be supplied by the Player, but it is not evaluated - rather it's manually parsed, from left to right. The function names and arguments are identified (as for the rest, only and/or/not are allowed). The functions are then called explicitly (in Python code, not evaluated as a string) and combined to get a result. This should be perfectly safe as long as your functions are well-defined.


For the potential softcode language, I first took this hands-on approach - manually parsing the string into its components. I got a pretty decent demo going, but the possibilities are much larger than in the simple lockstring case. Just parsing would be one thing, but then to also make sure that each part is okay to use too is another matter ... It would probably be doable, but then I got to supplying some sort of flow-control. The code starts to become littered with special cases which is never a good sign.

So eventually I drifted off from the "lock-like" approach and looked into Python's ast module. This allows you to view Python code as an "abstract syntax tree" (AST).  This solves the parsing issues but once you start dealing with the AST tree you are sort of looking at the problem from the other end - rather than parsing and building the script from scratch it more becomes a matter of removing what is already there (an AST tree can be compiled directly back into Python code after all). It nevertheless seemed like the best way forward.

Testing a few different recipes from the web, I eventually settled on an approach which (with some modifications compared to the original) uses a whitelist (and a blacklist for some other things) to only allow a given set of ast nodes and items in the execution environment. It walks the AST tree before execution and kills dangerous Python structures in one large swath. I expanded on this a fair bit, cutting away a lot of Python functionality for our particular use case. Stuff like attribute acces and assignments, while loops and many other Pythonesque things went out the window.

Around this highly stunted execution system I then built the Evennia in-game scripting system. This includes in-game commands as well as scriptable objects with suitable slots defining certain functionality the Player might want to change. Each Evennia developer can also supply any set of "safe" blackbox functions to offer more functionality to their Player-coders.

A drawback is the lack of a functional timeout watchdog in case of a script running too long. I'm using Twisted's deferToThread to make sure the code blocks as little as possible, but alas, whereas I can check timeouts just fine,  the problem lies in reliably killing the errant sub-thread. Internet experts suggest this to be tricky to do safely at the best of times (with threads running arbitrary code at least), and not wanting to kill the Twisted server is not helping things. I pondered using Twisted's subprocess support, but haven't gotten further into that at this point. Fact is that most of the obvious DOS attack vectors (such as the while loop and huge powers etc) are completely disabled, so it should hopefully not be trivial to DOS the system (famous last words).

I've tentatively dubbed the softcode system "Evlang" to differentiate it from our normal database-related "Scripts".
So is Evlang "safe" to use by untrusted evil Players? Well, suffice to say I'm putting it up with a huge EXPERIMENTAL flag, with plenty of warnings and mentions of "on your own risk". Running Evennia in a chroot jail and with minimum permissions is probably to recommend for the security paranoid. Hopefully Evennia coders will try all sorts of nasty stuff with it in the future and report their finding in our Issue tracker!

But implementation details aside, I must admit it's cool to be able to add custom code like this - the creative possibilities really do open up. And Python - even a stunted version of it - is really very nice to work with, also from inside the game.