Friday, July 1, 2016

The art of sharing nicks and descriptions

In the month or so since the merger of Evennia's development branch and all its web-client updates, we have been in bug-fixing mode as more people use and stress the code.

There have been some new features as well though - I thought it could be interesting to those of you not slavishly following the mailing list.

 

Shared web login

When you are logged into the website you will now also auto-login to your account in the web client - no need to re-enter the login information! The inverse is also true. You still need to connect to game at least once to create the account, but after that you will stay connected while the browser session lasts.

Behind the scenes the shared login uses cookies linked to server-side Django sessions which is a robust and safe way to manage access tokens. Obviously browser sessions are irrelevant to telnet- or ssh connections.

Extended Nicks 

Evennia's nick(name) system is a way to create a personal alias for things in game - both to on-the-fly replacing text you input and for referring to in-game objects. In the old implementation this replacement was simply matched from the beginning of the input - if the string matched, it was replaced with the nick.

In this new implementation, the matching part can be much more elaborate. For example you can catch arguments and put those arguments into the replacement nick in another order.

Let's say we often use the @dig command this limited way:

@dig roomname;alias = exit;alias, backexit;alias
 
Let's say we find this syntax unintuitive. The new nick system allows to change this by catching the arguments in your nick and put it into the "real" command. Here is an example of a syntax putting the aliases in parentheses and separating all components with commas:

> nick newroom $1($2), $3($4), $5($6) = @dig $1;$2 = $3;$4, $5;$6

From here on you can now create your rooms with entries like this: 

> newroom The great castle(castle), to the castle(castle), back to the path(back)

Multidescer contrib

I have added a new "multidescer" to the contrib folder. A multidescer is (I think) a MUSH term for a mechanism managing more than one description. You can then combine any of these various descriptions into your "active" description.

An example of usage:

desc hat = a blue hat.
desc basic = This is a tall man with narrow features.
desc clothing = He is wearing black, flowing robes.

 These commands store the description on the Character and references them as unique keywords. Next we can combine these strings together in any order to build the actual current description: 

> desc/set basic + |/ + clothing + On his head he has + hat
> look self
This is a tall man with narrow features. 
He is wearing black, flowing robes. On his head he has a blue hat.

This allows for both very flexible and easy-to-update descriptions but also a way to handle freeform equipment and clothing. And you can of course use the nick system to pre-format the output

> nick setdesc $1 $2 $3 $4 = $1 + |/ + clothing + On his head he has a $4

This way you can clothe yourself in different outfits easily using the same output format:

> setdesc basic clothing hat 
 
 The multidescer is a single, self-contained command that is easy to import and add to your game as needed.



... There's also plenty of bug fixes, documentation work and other minor things or course.

Anyway, summer is now upon us here in the northern hemisphere so things will calm down for a bit, at least from my end. Have a good 'un!
.
Griatch

Image by ryancr (released under Creative Commons)

Tuesday, May 31, 2016

Evennia in pictures

Evennia in pictures
by Griatch, September 2015

This article was originally written for Optional Realities
Since it is no longer available to read on OR, I'm reposting it in full here.

The MU* development system Evennia works quite differently from many traditional MU* code bases/servers and we get questions about this from time to time. This article will try to give an overview for outsiders - using pictures!  

Figure 1: The parts of the Evennia library

evennia_infrastructureA.png

Evennia is a game development library. What you see in Figure 1 is the part you download from us. This will not run on its own, we will soon initialize the missing “jigsaw puzzle” piece on the left. But first let’s look at what we’ve got.

Looking at Figure 1 you will notice that Evennia internally has two components, the Portal and the Server. These will run as separate processes.

The Portal tracks all connections to the outside world and understands Telnet protocols, websockets, SSH and so on. It knows nothing about the database or the game state. Data sent between the Portal and the Server is protocol-agnostic, meaning the Server sends/receives the same data regardless of how the user is connected. Hiding behind the Portal also means that the Server can be completely rebooted without anyone getting disconnected.

The Server is the main “mud driver” and handles everything related to the game world and its database. It's asynchronous and uses Twisted. In the same process of the Server is also the Evennia web server component that serves the game’s website. That the Server and webserver are accessing the database in the same process allows for a consistent game state without any concerns for caching or race condition issues.

Now, let’s get a game going. We’ll call it mygame. Original, isn’t it?

Figure 2: The full setup for mygame
evennia_infrastructure.jpg.png

After installing evennia you will have the evennia command available. Using this you create a game directory - the darker grey piece in Figure 2 that was missing previously. This is where you will create your dream game!

During initialization, Evennia will create Python module templates in mygame/ and link up all configurations to make mygame a fully functioning, if empty, game, ready to start extending. Two more commands will create your database and then start the server. From this point on, mygame is up and running and you can connect to your new game with telnet on localhost:4000 or by pointing your browser to http://localhost:8000.

Now, our new mygame world needs Characters, locations, items and more! These we commonly refer to as game entities. Let’s see how Evennia handles those.

Figure 3: The Database Abstraction of Evennia entities

evennia_typeclassesA.png
Evennia is fully persistent and abstracts its database in Python using Django. The database tables are few and generic, each represented by a single Python class. As seen in Figure 3, the example ObjectDB Python class represents one database table. The properties on the class are the columns (fields) of the table. Each row is an instance of the class (one entity in the game).

Among the example columns shown is the key (name) of the ObjectDB entity as well as a Foreign key-relationship for its current “location”. From the above we can see that Trigger is in the Dungeon, carrying his trusty crossbow Old Betsy!

The db_typeclass_path is an important field. This is a python-style path and tells Evennia which subclass of ObjectDB is actually representing this entity.

Figure 4: Inheriting classes to customize entities

evennia_typeclassesB.png

In Figure 4 we see the (somewhat simplified) Python class inheritance tree that you as an Evennia developer will see, along with the three instanced entities.

ObjectDB represents stuff you will actually see in-game and its child classes implement all the handlers, helper code and the hook methods that Evennia makes use of. In your mygame/ folder you just import these and overload the things you want to modify. In this way, the Crossbow is modified to do the stuff only crossbows can do and CastleRoom adds whatever it is that is special about rooms in the castle.

When creating a new entity in-game, a new row will automatically be created in the database table and then “Trigger” will appear in-game! If we, in code, search the database for Trigger, we will get an instance of the Character class back - a Python object we can work with normally.

Looking at this you may think that you will be making a lot of classes for every different object in the game. Your exact layout is up to you but Evennia also offers other ways to customize each individual object, as exemplified by Figure 5.

Figure 5: Adding persistent Attributes to a game entity.

evennia_typeclassesC.png

The Attribute is another class directly tied to the database behind the scenes. Each Attribute basically has a key, a value and a ForeignKey relation to another ObjectDB. An Attribute serializes Python constructs into the database, meaning you can store basically any valid Python, like the dictionary of skills seen in Figure 5. The “strength” and “skills” Attributes will henceforth be reachable directly from the Trigger object. This (and a few other resources) allow you to create individualized entities while only needing to create classes for those that really behave fundamentally different.


Figure 6: Sessions, Players and Objects

evennia_session_player_character.png

Trigger is most likely played by a human. This human connects to the game via one or more Sessions, one for each client they connect with. Their account on mygame is represented by a PlayerDB entity. The PlayerDB holds the password and other account info but has no existence in the game world. Through the PlayerDB entity, Sessions can control (“puppet”) one or more ObjectDB entities in-game.

In Figure 6, a user is connected to the game with three Sessions simultaneously. They are logged in to their Player account Richard. Through these Sessions they are simultaneously puppeting the in-game entities Trigger and Sir Hiss. Evennia can be configured to allow or disallow a range of different gaming styles like this.

Now, for users to be able to control their game entities and actually play the game, they need to be able to send Commands.

Figure 7: Commands are Python classes too

evennia_commandsA.png

Commands represent anything a user can input actively to the game, such as the look command, get, quit, emote and so on.

Each Command handles both argument parsing and execution. Since each Command is described with a normal Python class, it means that you can implement parsing once and then just have the rest of your commands inherit the effect. In Figure 7, the DIKUCommand parent class implements parsing of all the syntax common for all DIKU-style commands so CmdLook and others won’t have to.

Figure 8: Commands are grouped together in sets and always associated with game entities.

evennia_commandsB.png

Commands in Evennia are always joined together in Command Sets. These are containers that can hold many Command instances. A given Command class can contribute Command instances to any number of Command Sets. These sets are always associated with game entities. In Figure 8, Trigger has received a Command Set with a bunch of useful commands that he (and by extension his controlling Player) can now use.

Figure 9: Command Sets can affect those around them

evennia_commandsC.png

Trigger’s Command Set is only available to himself. In Figure 8 we put a Command Set with three commands on the Dungeon room. The room itself has no use for commands but we configure this set to affect those inside it instead. Note that we let these be different versions of these commands (hence the different color)! We’ll explain why below.

Figure 10: The name Command “Set” is not just a name

evennia_commandsD.png

Command Sets can be dynamically (and temporarily) merged together in a similar fashion as Set Theory, except the merge priority can be customized. In Figure 10 we see a Union-type merger where the Commands from Dungeon of the same name temporarily override the commands from Trigger. While in the Dungeon, Trigger will be using this version of those commands. When Trigger leaves, his own Command Set will be restored unharmed.

Why would we want to do this? Consider for example that the dungeon is in darkness. We can then let the Dungeon’s version of the look command only show the contents of the room if Trigger is carrying a light source. You might also not be able to easily get things in the room without light - you might even be fumbling randomly in your inventory!

Any number of Command Sets can be merged on the fly. This allows you to implement multiple overlapping states (like combat in a darkened room while intoxicated) without needing huge if statements for every possible combination. The merger is non-destructive, so you can remove cmdsets to get back previous states as needed.



… And that’s how many illustrations I have the stamina to draw at this time. Hopefully this quick illustrated dive into Evennia helps to clarify some of the basic features of the system!

Sunday, May 22, 2016

Evennia 0.6 !


As of today, I merged the development branch to make version 0.6 of the MU* development system and server Evennia.

Evennia 0.6 comes with a lot of updates, mainly in the way Evennia talks to the outside world. All communication is now standardized, so there are no particular treatment of things like text - text is just one of any standardized commands being passed between the server the client (whether over telnet, ssh, websockets or ajax/comet).

For example the user can now easily plug in "inputfuncs" to handle any data coming from the client. If you want your client to offer some particular functionality, you just need to plop in a python function to handle it, server-side. We also now offer a lot of utility functions for things like monitoring change (tell the client whenever your health status changes so it can update a health bar or flash the screen).

The HTML5 webclient has itself updated considerably. Most is happening behind the scenes though. Notably the webclient's javascript component is split into two:

  •  evennia.js, acts as a library for handling all communication with the server part of Evennia. It offers events for a gui library to plug into and send/receive. It will also gracefully degrade from a websocket connection to AJAX/COMET long-polling if the player uses an older browser. 
  • evennia_gui.js is the default front-end and implements a traditional and stable "telnet-like" interface. The html part uses uses Django templating to make it easy to customize and expand. Since this simply makes use of the resources offered by evennia.js, one could pretty easily slip in some other gui library here, or set up calls to get all sorts of interesting information from the server (which talks back using inputfuncs). 
There are a truckload of more updates and features that are covered on the mailing list.
.
Griatch

Thursday, March 24, 2016

Technical stuff happening

Hi folks, a bit more technical entry this time. These usually go onto the Evennia mailing list but I thought it would be interesting to put it in the dev-blog for once.

So, I'm now halfway through the TODO list issue of the wclient development branch as alluded to in the last post. The wclient branch aims to rework and beef up the web client infrastructure of Evennia.

The first steps, which has been done a while was converting the SSH/SSL and IRC input/output protocols to use the new protocol infrastructure (telnet and websockets was done since before). That's just under-the-hood stuff though. Today I finished the changes to the Monitor/TickerHandlers, which may be of more general interest.

With the changes to the the way OOB (Out-Of-Band) messages are passing through Evennia (see this mailing list post for more details), the OOBHandler is no more. As discussed there, the handling of incoming data is becoming a lot freer and will be easily expandable to everyone wanting to make for a custom client experience. The idea is thus for Evennia to offer resources for various input commands to make use of, rather than prescribing such functionality in a monolothic way in the OOBHandler. There were three main functionalities the OOBHandler offered, and which will now be offered by separate components:

  1. Direct function callback. The instruction from the client should be able to trigger a named server-side function. This is the core of the inputfunc system described previously.
  2. Field/Attribute monitoring. The client should be able to request monitoring of an object's database fields or Attributes. For example, the client may request to be notified whenever the Character's "health" Attribute changes in some way. This is now handled by the new monitorhandler. See below.
  3. Non-persistent function repeats. One should be able to set up a repeating ticker that survives a server reload but does not survive a cold shutdown - this mimics the life cycle of server Sessions. Scripts could do this already but I wanted to be able to use the TickerHandler for quick assignment. Problem was that the Tickerhandler in master branch is not only always-persistent, it also only calls database object methods. So I have now expanded the tickerhandler to also accept arbitrary module functions, without any connection to a database object.
The MonitorHandler 

evennia.MONITOR_HANDLER is the new singleton managing monitoring of on-object field/attribute changes. It is used like this:


MONITOR_HANDLER.add(obj, field_or_attrname, callback, **kwargs)

Here obj is a database entity, like a Character or another Object. The field_or_attrname is a string giving the name of a db_* database field (like "db_key", "db_location" etc). Any name not starting with db_ is assumed to be the name of an on-object Attribute (like "health"). Henceforth, whenever this field or attribute changes in any way (that is, whenever it is re-saved to the database), the callback will be called with the optional kwargs, as well as a way to easily get to the changed value. As all handlers you can also list and remove monitors using the standard MONITOR_HANDLER.remove(), .all() etc.


The TickerHandler

evennia.TICKER_HANDLER should be familiar to Evennia users from before - it's been around for a good while. It allows for creating arbitrary "tickers" that is being "subscribed" to - one ticker will call all subscribers rather than each object or function having its own timer.

Before, the syntax for adding a new ticker required you specify a typeclassed entity and the name of the method on it to call every N seconds. This will now change. This is the new callsign for creating a new ticker:


TICKER_HANDLER.add(interval, callback, idstring="", persistent=True, *args, **kwargs)

Here, interval, like before, defines how often to call callback(*args, **kwargs).

The big change here is that callback should be given as a valid, already imported callable, which can be either an on-entity method (like obj.func) or a global function in any module (like world.test.func) - the TickerHandler will analyze it and internally store it properly.

idstring works as before, to separate tickers with the same intervals. Finally persistent=False means the ticker will behave the same way a Script with persistent=False does: it will survive a server reload but will not survive a server shutdown. This latter functionality is particularly useful for client-side commands since the client Session will also not survive a shutdown.

... So this is a rather big API change to the TickerHandler, which will mean some conflicts for those of you relying heavily on tickers. Easiest will definitely be to simply stop the old and start new ones. It's not clear yet if we'll offer some automated way to convert old tickers to new ones. Chime in if this is something important to you.

Happening Next

The next steps involves making use of these new utilities to implement the basic OOB commands recommended by the MSDP and GMCP protocols along with some recommended functionality. We'll see how long that takes, but progress is being made. And if you are a web guy, do consider helping out.
.
Griatch

Sunday, February 14, 2016

Climbing up Branches

Today I pushed the latest Evennia development branch "wclient". This has a bunch of updates to how Evennia's webclient infrastructure works, by making all exchanged data be treated equal (instead of treating text separately from other types of client instructions).

It also reworks the javascript client into a library that should be a lot easier to expand on and customize. The actual client GUI is still pretty rudimentary though, so I hope a user with more web development experience can take upon themselves to look it over for best practices.

A much more detailed description of what is currently going on (including how to check out the latest for yourself) is found in this mailing list post. Enjoy!

Thursday, December 17, 2015

A summary of a year

As 2015 is slowly drawing to an end, I looked back through Evennia's repository to see just what was going on this year. And it turns out it was a lot! I honestly didn't remember some things happened as recently as they did.

Note: For those reading this not familiar with Evennia, it's a Python library for creating MUDs (text-based multiplayer games).
 

Making Evennia into a library

In February of 2015 we merged what was likely the biggest change happening for a good while in Evennia - the complete refactoring of the Evennia repository into a library. It used to be that when you cloned the Evennia repo, it would come with a pre-made game/ folder where you were supposed to put your custom files. Mixing the stuff you downloaded from us with your own files (which you might want to keep under version control of your own) was not a very clean solution.

In the big "library update", we instead created a stand-alone evennia program which you use to create your own "game dir" with pre-created templates. Not only does this allow you to treat the cloned evennia repo as a proper library, you can also use the same evennia install for multiple games and makes it a lot clearer just what comes from us and what is your custom code.

Below is a fun Gource representation (courtesy of Youtuber Landon Wilkins) of how the Evennia repository structure has changed over the years. You can see the latest library-change a little after the 3minute, 20 second mark.



Typeclasses into proxies

At the same time as the library change, I also completely overhauled the way typeclasses are represented in Evennia: Typeclasses are Python classes that links back to a database model under the hood. They used to use a custom overloading of varioust get/set methods, now they instead utilize django proxy models extended to support multiple inheritance. T

his radically increased performance and made the code considerably cleaner, as well as completely hid the core django model from the end user (no longer a need to track if you were dealing with a model or its typeclass - you only ever run into typeclasses). This, together with the library-model led to some changes in people's source codes.

Even so, this change led to a lot less problems and edge-cases than I had anticipated: it seems more people had issues with upgrading django than with adopting their codes to the new typeclass changes ...

Evennia Autodocs

Following the big library merger I sat down to write a more comprehensive autodoc utility. We had been distributing a Doxygen config file with the repo for a long time, but I wanted something that integrated with our github wiki, using markdown in the source (because frankly, while Sphinx produces very pretty output, ReST markup looks really ugly in source code, in my opinion).

The result was the api2md program, which is now a part of our wiki repository. It allows our source code to be decorated in "Google style", very readable output:

def funcname(a, b, c, d=False):
   
"""
    This is a brief introduction to the 

    function/class/method

    Args:
        a (str): This is a string argument that 

          we can talk about over multiple lines.
        b (int or str): Another argument
        c (list): A list argument
        d (bool, optional): An optional 

           keyword argument

    Returns:
        e (str): The result of the function

    Notes:
        This is an example function. 

        If `d=True`, something amazing will happen.

    """



This will be parsed and converted to a Markdown entry and put into the Github wiki, one page per module. The result is the automatically generated Evennia API autodocs, reachable as any other wiki page.

The convertion/prettification of all core functions of Evennia to actually use  the Google-style docstrings took almost all year, finishing late in autumn. But now almost all of Evennia uses this style. Coincidentally this also secures us at a 45% comment/code ratio. This places us in the top 10% of well-documented open-source projects according to openhub (gotta love statistics).

Imaginary realities / Optional Realities

Spring of 2015 saw some more articles for the Imaginary Realities e-zine as well as some for the newly-opened Optional Realities web forum (unrelated, despite the similar name). The latter was made by a team working on a commercial Evennia-based sci-fi game, but the forums were open for other games (of any engine) and general discussion on mud and mud design.

Optional Realities published an impressive range of articles (one every week for several months) and organized several very interesting mud-related contests. It did, I think, a lot for bringing some new life to the mud-development scene.  Unfortunately Optional Realities suffered a complete database loss towards the end of the year, forcing it to sort of reboot from scratch. I hope it will rebound and that the articles can be put back online again!

Ainneve

Over summer I put out a general roll call for developers willing to lead the development of a small but fully functioning Evennia demo game - a project separate from the main Evennia development. The idea is to have something working for people to rip out and start from. The response to my request was very good and we relatively quickly ended up with two devs willing to lead and direct the effort.

The work-in-progress demo game they started is called Ainneve. It uses the base Open Adventure RPG rules and is distributed using the same liberal licence as Evennia.

Ainneve has been sputtering along throughout autumn and winter, and while progress has been a bit ... sporadic, it seems to attract new volunteers to help out. As more of the base systems gets finalized there is going to be even more "low hanging fruit" for people to jump in and help with.

EvMenu, EvMore and EvEditor, RPSystem

In July I merged some new systems into Evennia's utilities library. EvMenu is a class that builds an in-game menu. It uses multiple-choice questions for the user to navigate its nodes. I had a "menusystem" in our contrib/ folder since many years, but it was showing its age and when I found I myself didn't really understand how it was working I found it time to make something more flexible.

The EvMore is a page-scroller that allows the server to pause long texts and wait for the user to press a key before continuing. It's functionality is similar to the unix more program.

For its part, EvEditor has existed for a while as well, but it was moved from the contrib folder into the main utilility library as it turned out to be an important and common resource (it's basically a mud-version of the classic vi editor).

Finally, some months later, in September, I added the rpsystem and rplanguage contribution modules. The former is an author-stance recognition system similar to what is seen in some rp-heavy muds (where you don't know/see people's name off the bat, but only their description, until you manually assign a name to them).

The rplanguage module is used for handling foreign (fantasy) languages by obfuscating words heard by different parties depending on their relative language skills.

Python 3 ... some day

As we know, Evennia uses Python 2. In autumn of 2015 first one and then two of our users took upon themselves to help make Evennia a little more ready for running also under Python 3. After a lot of work, Evennia's core is now using code syntax that is compatible with both Python 2 and 3.

We don't run under Python 3 at this point though. This is not something under our control, but is due to Twisted not supporting Python 3 yet. But when it becomes possible, we are now better prepared for transitioning to a point where we can (hopefully) support both main Python versions.

EvCast

In September, user whitenoise started his "EvCast" video tutorial series on Evennia. There are so far two episodes, one on installing Evennia and the second on general Python concepts you need to use Evennia efficiently:




 Python.__init__ Podcast

In the end of September I was interviewed by the hosts of the python.__init__ podcast. Turned into more than an hour of discussion about Evennia. T'was fun! You can listen to it here.

Nested inlinefuncs

Towards the end of the year I got (via Optional Realities) lured to participate in another text-gaming related website, musoapbox. This is a forum primarily populated by players from the MUSH-side of text-gaming. After some back and forth they made a strong case for increasing the functionality of Evennia's inlinefunctions.

An inlinefunction is, in Evennia, a function call embedded in a string sent to the server. This is parsed and executed (no, there is no dangerous eval happening, the call is parsed as text and then called as a normal function call). It allows for text to be dynamically replaced at run-time. What the MUSHers suggested was to also allow those inlinefunc's to be nestable - that is, to implement a call stack. The nestable form of inlinefuncs was merged with Evennia master at the end of November.

Patreon

This actually happened already back in February: after some requests for more ways to support Evennia development, I opened a Patreon page. And here at the end of the year, a whole ten patrons have signed up for it. Very encouraging!

Next year

In 2016 the first upcoming thing that comes to mind is push out the changes to the webclient infrastructure. That has been a long time coming since I've been unsure of how to proceed on it for quite some time. The goal is to make the webclient a lot more "pluggable" and modular than it is, and to clean up its API and way of communicating with the server. Overall I think the web-side of things need some love.

I'll likely put together some more general-use contribs as well, I have some possibles in mind. 

We'll continue squashing bugs and work down our roadmap.

I'll also try to get an Evennia Wikipedia.org page together, if you want to look at how it's progressing and help editing it, see the Evennia mailing list for the link.


... And a lot more I don't know yet, no doubt! On towards a new year!
.
Griatch