The last few months has been dominated by bug-fixing and testing in Evennia-land. A lot more new users appears to be starting to use Evennia, especially from the MUSH world where the Evennia-based Arx, After the Reckoningis, while still in alpha, currently leading the charge.
With a new influx of users comes the application of all sorts of use- and edge-cases that stretch and exercise the framework in all the places where it matters. There is no better test of code than new users trying to use it unsupervised! Evennia is holding up well overall but there are always things that can be improved.
I reworked the way on-object Attributes was cached (from a stupid but simple way to a lot more sophisticated but harder way) and achieved three times faster performance in certain special cases people had complained about. Other issues also came to view while diving into this, which could be fixed.
I reworked the venerable batch command and batchcode processors (these allow to create a game world from a script file) and made their inputs make more sense to people. This was one of the older parts of Evennia and apart from the module needing a big refactoring to be easier to read, some parts were pretty fragile and prone to break. Especially when passing it file names tended to be confusing since it understood only certain relative paths to the files to read in and not even I could remember if one should include the file ending or not. This was cleaned up a lot.
Lots of changes and updates were made to the RPSystem contrib, which optionally adds more advanced roleplaying mechanics to Evennia. The use of this in Evennia's demo game (Ainneve, being separately developed) helps a lot for ironing out any remaining wrinkles.
Lots and lots of other fixes and smaller feature updates were done (About 150 commits and 50 Issues closed since end of summer).
A fun thing with a growing user base is that we are also starting to see a lot more Pull requests and contributions. Thanks a lot, keep 'em coming!
Map system contrib (merged), for creating a world based on ASCII map. Talking about maps, users contributed not just one but two new tutorials for implementing both static and dynamic maps with Evennia.
Webclient notifications (pending), for making the webclient show us in a clearer way when it gets updated in a different tab. A more advanced implementation awaits the webclient being expanded with a proper client-side option window; there is currently a feature request for this if anyone's interested in taking it on.
AI system contrib (pending). This is a full AI backend for adding complex behaviors to game agents. It uses Behavioral trees and is designed to be modified both in code and from inside the game.
Action system contrib (pending). This contrib assigns the actions of Characters a time cost and delays the results of commands the given time. It also allows players to go into turn-based mode to enforce a strict action order.
Lots of now closed PRs were contributed by the Arx lead developer to fix various bugs and edge-cases as they came up in live use.
The fixing and tightening of the nuts and bolts will likely continue the remainder of the year. I'm currently working on a refactoring of the way command sets are merged together (see the end of my blog post on Evennia in pictures for a brief summary of the command system). But with so much new blood in the community, who can tell where things will turn from here!
Since it is no longer available to read on OR, I'm reposting it in full here.
Figure 1: The parts of the Evennia library
Evennia is a game development library. What you see in Figure 1is the part you download from us. This will not run on its own, we will soon initialize the missing “jigsaw puzzle” piece on the left. But first let’s look at what we’ve got.
Looking at Figure 1you will notice that Evennia internally has two components, the Portal and the Server. These will run as separate processes.
The Portal tracks all connections to the outside world and understands Telnet protocols, websockets, SSH and so on. It knows nothing about the database or the game state. Data sent between the Portal and the Server is protocol-agnostic, meaning the Server sends/receives the same data regardless of how the user is connected. Hiding behind the Portal also means that the Server can be completely rebooted without anyone getting disconnected.
The Server is the main “mud driver” and handles everything related to the game world and its database. It's asynchronous and uses Twisted. In the same process of the Server is also the Evennia web server component that serves the game’s website. That the Server and webserver are accessing the database in the same process allows for a consistent game state without any concerns for caching or race condition issues.
Now, let’s get a game going. We’ll call it mygame. Original, isn’t it?
Figure 2: The full setup for mygame
After installing evennia you will have the evennia command available. Using this you create a game directory - the darker grey piece in Figure 2 that was missing previously. This is whereyou will create your dream game!
During initialization, Evennia will create Python module templates in mygame/ and link up all configurations to make mygame a fully functioning, if empty, game, ready to start extending. Two more commands will create your database and then start the server. From this point on, mygame is up and running and you can connect to your new game with telnet on localhost:4000 or by pointing your browser tohttp://localhost:4001.
Now, our new mygame world needs Characters, locations, items and more! These we commonly refer to as game entities. Let’s see how Evennia handles those.
Figure 3: The Database Abstraction of Evennia entities
Evennia is fully persistent and abstracts its database in Python using Django. The database tables are few and generic, each represented by a single Python class. As seen in Figure 3, the example ObjectDB Python class represents one database table. The properties on the class are the columns (fields) of the table. Each row is an instance of the class (one entity in the game).
Among the example columns shown is the key (name) of the ObjectDB entity as well as a Foreign key-relationship for its current “location”. From the above we can see that Trigger is in the Dungeon, carrying his trusty crossbow Old Betsy!
The db_typeclass_path is an important field. This is a python-style path and tells Evennia which subclass of ObjectDB is actually representing this entity.
Figure 4: Inheriting classes to customize entities
In Figure 4 we see the (somewhat simplified) Python class inheritance tree that you as an Evennia developer will see, along with the three instanced entities.
ObjectDB represents stuff you will actually see in-game and its child classes implement all the handlers, helper code and the hook methods that Evennia makes use of. In your mygame/ folder you just import these and overload the things you want to modify. In this way, the Crossbow is modified to do the stuff only crossbows can do and CastleRoom adds whatever it is that is special about rooms in the castle.
When creating a new entity in-game, a new row will automatically be created in the database table and then “Trigger” will appear in-game! If we, in code, search the database for Trigger, we will get an instance of the Character class back - a Python object we can work with normally.
Looking at this you may think that you will be making a lot of classes for every different object in the game. Your exact layout is up to you but Evennia also offers other ways to customize each individual object, as exemplified by Figure 5.
Figure 5: Adding persistent Attributes to a game entity.
The Attribute is another class directly tied to the database behind the scenes. Each Attribute basically has a key, a value and a ForeignKey relation to another ObjectDB. An Attribute serializes Python constructs into the database, meaning you can store basically any valid Python, like the dictionary of skills seen in Figure 5. The “strength” and “skills” Attributes will henceforth be reachable directly from the Trigger object. This (and a few other resources) allow you to create individualized entities while only needing to create classes for those that really behave fundamentally different.
Figure 6: Sessions, Players and Objects
Trigger is most likely played by a human. This human connects to the game via one or more Sessions, one for each client they connect with. Their account on mygame is represented by a PlayerDB entity. The PlayerDB holds the password and other account info but has no existence in the game world. Through the PlayerDB entity, Sessions can control (“puppet”) one or more ObjectDB entities in-game.
In Figure 6, a user is connected to the game with three Sessions simultaneously. They are logged in to their Player account Richard. Through these Sessions they are simultaneously puppeting the in-game entities Trigger and Sir Hiss. Evennia can be configured to allow or disallow a range of different gaming styles like this.
Now, for users to be able to control their game entities and actually play the game, they need to be able to send Commands.
Figure 7: Commands are Python classes too
Commands represent anything a user can input actively to the game, such as the look command, get, quit, emote and so on.
Each Command handles both argument parsing and execution. Since each Command is described with a normal Python class, it means that you can implement parsing once and then just have the rest of your commands inherit the effect. In Figure 7, the DIKUCommand parent class implements parsing of all the syntax common for all DIKU-style commands so CmdLook and others won’t have to.
Figure 8: Commands are grouped together in sets and always associated with game entities.
Commands in Evennia are always joined together in Command Sets. These are containers that can hold many Command instances. A given Command class can contribute Command instances to any number of Command Sets. These sets are always associated with game entities. In Figure 8, Trigger has received a Command Set with a bunch of useful commands that he (and by extension his controlling Player) can now use.
Figure 9: Command Sets can affect those around them
Trigger’s Command Set is only available to himself. In Figure 8 we put a Command Set with three commands on the Dungeon room. The room itself has no use for commands but we configure this set to affect those inside it instead. Note that we let these be different versions of these commands (hence the different color)! We’ll explain why below.
Figure 10: The name Command “Set” is not just a name
Command Sets can be dynamically (and temporarily) merged together in a similar fashion as Set Theory, except the merge priority can be customized. In Figure 10 we see a Union-type merger where the Commands from Dungeon of the same name temporarily override the commands from Trigger. While in the Dungeon, Trigger will be using this version of those commands. When Trigger leaves, his own Command Set will be restored unharmed.
Why would we want to do this? Consider for example that the dungeon is in darkness. We can then let the Dungeon’s version of the look command only show the contents of the room if Trigger is carrying a light source. You might also not be able to easily get things in the room without light - you might even be fumbling randomly in your inventory!
Any number of Command Sets can be merged on the fly. This allows you to implement multiple overlapping states (like combat in a darkened room while intoxicated) without needing huge if statements for every possible combination. The merger is non-destructive, so you can remove cmdsets to get back previous states as needed.
… And that’s how many illustrations I have the stamina to draw at this time. Hopefully this quick illustrated dive into Evennia helps to clarify some of the basic features of the system!
As 2015 is slowly drawing to an end, I looked back through Evennia's repository to see just what was going on this year. And it turns out it was a lot! I honestly didn't remember some things happened as recently as they did.
Note: For those reading this not familiar with Evennia, it's a Python library for creating MUDs (text-based multiplayer games).
Making Evennia into a library
In February of 2015 we merged what was likely the biggest change happening for a good while in Evennia - the complete refactoring of the Evennia repository into a library. It used to be that when you cloned the Evennia repo, it would come with a pre-made game/ folder where you were supposed to put your custom files. Mixing the stuff you downloaded from us with your own files (which you might want to keep under version control of your own) was not a very clean solution.
In the big "library update", we instead created a stand-alone evennia program which you use to create your own "game dir" with pre-created templates. Not only does this allow you to treat the cloned evennia repo as a proper library, you can also use the same evennia install for multiple games and makes it a lot clearer just what comes from us and what is your custom code.
Below is a fun Gource representation (courtesy of Youtuber Landon Wilkins) of how the Evennia repository structure has changed over the years. You can see the latest library-change a little after the 3minute, 20 second mark.
Typeclasses into proxies
At the same time as the library change, I also completely overhauled the way typeclasses are represented in Evennia: Typeclasses are Python classes that links back to a database model under the hood. They used to use a custom overloading of varioust get/set methods, now they instead utilize django proxy models extended to support multiple inheritance. T
his radically increased performance and made the code considerably cleaner, as well as completely hid the core django model from the end user (no longer a need to track if you were dealing with a model or its typeclass - you only ever run into typeclasses). This, together with the library-model led to some changes in people's source codes.
Even so, this change led to a lot less problems and edge-cases than I had anticipated: it seems more people had issues with upgrading django than with adopting their codes to the new typeclass changes ...
Evennia Autodocs
Following the big library merger I sat down to write a more comprehensive autodoc utility. We had been distributing a Doxygen config file with the repo for a long time, but I wanted something that integrated with our github wiki, using markdown in the source (because frankly, while Sphinx produces very pretty output, ReST markup looks really ugly in source code, in my opinion).
The result was the api2md program, which is now a part of our wiki repository. It allows our source code to be decorated in "Google style", very readable output:
def funcname(a, b, c, d=False): """ This is a brief introduction to the function/class/method
Args: a (str): This is a string argument that we can talk about over multiple lines. b (int or str): Another argument c (list): A list argument d (bool, optional): An optional keyword argument
Returns: e (str): The result of the function
Notes: This is an example function. If `d=True`, something amazing will happen.
"""
This will be parsed and converted to a Markdown entry and put into the Github wiki, one page per module. The result is the automatically generated Evennia API autodocs, reachable as any other wiki page.
The convertion/prettification of all core functions of Evennia to actually use the Google-style docstrings took almost all year, finishing late in autumn. But now almost all of Evennia uses this style. Coincidentally this also secures us at a 45% comment/code ratio. This places us in the top 10% of well-documented open-source projects according to openhub (gotta love statistics).
Imaginary realities / Optional Realities
Spring of 2015 saw some more articles for the Imaginary Realities e-zine as well as some for the newly-opened Optional Realities web forum (unrelated, despite the similar name). The latter was made by a team working on a commercial Evennia-based sci-fi game, but the forums were open for other games (of any engine) and general discussion on mud and mud design.
Optional Realities published an impressive range of articles (one every week for several months) and organized several very interesting mud-related contests. It did, I think, a lot for bringing some new life to the mud-development scene. Unfortunately Optional Realities suffered a complete database loss towards the end of the year, forcing it to sort of reboot from scratch. I hope it will rebound and that the articles can be put back online again!
Ainneve
Over summer I put out a general roll call for developers willing to lead the development of a small but fully functioning Evennia demo game - a project separate from the main Evennia development. The idea is to have something working for people to rip out and start from. The response to my request was very good and we relatively quickly ended up with two devs willing to lead and direct the effort.
The work-in-progress demo game they started is called Ainneve. It uses the base Open Adventure RPG rules and is distributed using the same liberal licence as Evennia.
Ainneve has been sputtering along throughout autumn and winter, and while progress has been a bit ... sporadic, it seems to attract new volunteers to help out. As more of the base systems gets finalized there is going to be even more "low hanging fruit" for people to jump in and help with.
EvMenu, EvMore and EvEditor, RPSystem
In July I merged some new systems into Evennia's utilities library. EvMenu is a class that builds an in-game menu. It uses multiple-choice questions for the user to navigate its nodes. I had a "menusystem" in our contrib/ folder since many years, but it was showing its age and when I found I myself didn't really understand how it was working I found it time to make something more flexible.
The EvMore is a page-scroller that allows the server to pause long texts and wait for the user to press a key before continuing. It's functionality is similar to the unix more program.
For its part, EvEditor has existed for a while as well, but it was moved from the contrib folder into the main utilility library as it turned out to be an important and common resource (it's basically a mud-version of the classic vi editor).
Finally, some months later, in September, I added the rpsystem and rplanguage contribution modules. The former is an author-stance recognition system similar to what is seen in some rp-heavy muds (where you don't know/see people's name off the bat, but only their description, until you manually assign a name to them).
The rplanguage module is used for handling foreign (fantasy) languages by obfuscating words heard by different parties depending on their relative language skills.
Python 3 ... some day
As we know, Evennia uses Python 2. In autumn of 2015 first one and then two of our users took upon themselves to help make Evennia a little more ready for running also under Python 3. After a lot of work, Evennia's core is now using code syntax that is compatible with both Python 2 and 3.
We don't run under Python 3 at this point though. This is not something under our control, but is due to Twisted not supporting Python 3 yet. But when it becomes possible, we are now better prepared for transitioning to a point where we can (hopefully) support both main Python versions.
EvCast
In September, user whitenoise started his "EvCast" video tutorial series on Evennia. There are so far two episodes, one on installing Evennia and the second on general Python concepts you need to use Evennia efficiently:
Python.__init__ Podcast
In the end of September I was interviewed by the hosts of the python.__init__ podcast. Turned into more than an hour of discussion about Evennia. T'was fun! You can listen to it here.
Nested inlinefuncs
Towards the end of the year I got (via Optional Realities) lured to participate in another text-gaming related website, musoapbox. This is a forum primarily populated by players from the MUSH-side of text-gaming. After some back and forth they made a strong case for increasing the functionality of Evennia's inlinefunctions.
An inlinefunction is, in Evennia, a function call embedded in a string sent to the server. This is parsed and executed (no, there is no dangerous eval happening, the call is parsed as text and then called as a normal function call). It allows for text to be dynamically replaced at run-time. What the MUSHers suggested was to also allow those inlinefunc's to be nestable - that is, to implement a call stack. The nestable form of inlinefuncs was merged with Evennia master at the end of November.
Patreon
This actually happened already back in February: after some requests for more ways to support Evennia development, I opened a Patreon page. And here at the end of the year, a whole ten patrons have signed up for it. Very encouraging!
Next year
In 2016 the first upcoming thing that comes to mind is push out the changes to the webclient infrastructure. That has been a long time coming since I've been unsure of how to proceed on it for quite some time. The goal is to make the webclient a lot more "pluggable" and modular than it is, and to clean up its API and way of communicating with the server. Overall I think the web-side of things need some love.
I'll likely put together some more general-use contribs as well, I have some possibles in mind.
We'll continue squashing bugs and work down our roadmap.
I'll also try to get an Evennia Wikipedia.org page together, if you want to look at how it's progressing and help editing it, see the Evennia mailing list for the link.
... And a lot more I don't know yet, no doubt! On towards a new year!
.
Griatch
Evennia was recently used as a test bed to train an AI system to attempt to play a MUD as a human would - by only reading and understanding the text on the screen.
Researchers at MIT (Massachusetts Institute of Technology) recently presented the paper Language understanding for Text-based games using Deep reinforcement learning (PDF)at a conference on natural language processing. A summary is in the MIT press release.
I was contacted by these fine folks some time ago so I knew they had plans to use Evennia for their research. It's great to see they now have an article out on it! Evennia devs are also mentioned in the acknowledgements - so something for the Evennia dev community to be proud of!
MUDs are tricky
The main complication for an AI playing a MUD is that the computer has no access to the actual game state but must try to surmise how well it's doing only from the text given (same as a human would). The researchers compare the results from a range of deep-learning neural network algorithm that they train to play.
To test their AI, the researchers first used Evennia to build a simple training "Home World": a 4-room "house" where the simple goal is to find and eat an apple while refraining to go to sleep. The room descriptions used here were pretty formulaic although not trivial to give a challenge. This they used to train their AI system.
They then took this trained neural network and applied it to the real challenge, playing the Evennia Tutorial World. You can yourself try this out in our demo install or by just running a single command when starting Evennia. They call it "Fantasy World" in the article.
The tutorial world has hand-written descriptions and often describes the exits as part of the room text. The article actually makes a comprehensive analysis of the tutorial world, including the available game states and transitions as well as the number of words and number of commands per state. Interesting stuff in itself. I presume the scientists have modified their copy of the tutorial world to provide better metrics for their analysis.
A bridge too far
As far as I understand from the article, the AI does understand to use commands with one or two arguments (like eat apple or the move red-root right), but they note that actually finding the tomb of the fallen hero (the main quest of the tutorial) is too hard for the AI:
[...]However, this is a complex quest that requires the player to memorize game events and perform high-level planning which are beyond the scope of this current work.
So instead they evaluate the AI's performance on a more mundane task: Getting across the bridge to the castle. It's not clear to me if the AI actually plays more of the game too or if their test just exposes the AI to the bridge itself. I suspect it does play more due to the examples they use from other rooms; evaluating the bridge-crossing is just a clear-cut metric to use for "success".
The MIT press release claims that the AI is also scored on how much health/magic it has, but I don't see that mentioned in the article itself (and the tutorial world only has magic if you find the hero's tomb which they claim they cannot do).
The bridge in Evennia's tutorial world is actually a single "room" that takes multiple steps to cross. At every step the room description changes to describe the progress. Random texts will appear as the bridge sways in the wind and various environmental cues are heard and seen. There is also a small chance of falling off the bridge if one lingers too long on it.
So although all you really need to do is to walk east repeatedly, I can see why this can be a challenge to a neural network having no mental image of what a bridge is. It can only work off the text it's given at any given time.
In the paper, the algorithms are evaluated both on their ability to actually cross the bridge and on how optimal their solution was, for example by not issuing invalid commands to the situation.
Beyond the bridge
The results are that after being trained on the training house setup, the AI will eventually be able to cross the bridge. The particular algorithm proposed also perform slightly better than the comparison ones (and a lot better than simple randomness).
So from the perspective of the researchers this seems to be a success. Even so, this reinforces the fact that quite some way to go before an AI can *actually* play a real MUD successfully. Using MUDs for this type of research is a good idea though, and I do hope they expand and continue this line work in the future.
Who knows, maybe the AI will even find that ancient tomb eventually!
I recently made an article presenting the infrastructure and main coding features of Evennia using a series of nifty diagrams.
It can hopefully help newcomers get a feeling for the grand structure of an Evennia server/game a little easier. It's published as a feature on Optional Realities and you can find the article here: Evennia Illustrated.
I also recently joined MU Soapbox, a forum predominantly discussing MUSH games, to answer some technical questions on Evennia. Unsurprisingly, some (although so far surprisingly few) MUSHers express concern about Evennia's explicit lack of softcode (that is, the ability for players to use a safe in-game language to code their game rather than to use external Python modules). Their argument is sound: They are used to in-game coding as a way to easily express creatitivy and easily garner help from players.
Evennia's stance here clash a bit with those opinions: Our philosophy is that our command system is powerful enough to offer players any complexity of build commands they want. The design/coding of the game itself should be done using proper coding IDEs and modern version control tools.
There is no denying that compared to a softcode-coded game, a player-level contributor to an Evennia game needs some extra tools to create and contribute code over version control. The admin also needs to check such contributions for malicious code before merging it into their running game. Now, these are differences I actually consider advantages from a code-quality perspective. And for finding help, people on average are more likely to know Python than a custom softcode language. But here opinions differ and in a given game community those language adoption numbers can be severely skewed.
So far, the MUSHers that have adopted Evennia seems to have done so very much to get away from softcode. It will be interesting to see if things like Kelketek's in-development Group building Evennia contrib will be stirring some interest from those on the fence, or if coding their entire game in softcode is indeed an irreplaceable source of their gaming fun.
