Tuesday, October 20, 2020

On using Markdown with Sphinx - onward to Evennia 0.9.5

Last post I wrote about the upcoming v1.0 of Evennia, the Python MU* creation engine. We are not getting to that 1.0 version quite yet though: The next release will be 0.9.5, hopefully out relatively soon (TM).

Evennia 0.9.5 is, as you may guess, an intermediary release. Apart from the 1.0 roadmap just not being done yet, there is one other big reason for this - we are introducing documentation versioning and for that a proper release is needed as a base to start from. Version 0.9.5 contains everything already in master branch, so if you have kept up-to-date you won't notice too much difference. Here are some highlights compared to version 0.9:

  • EvMore will paginate and properly handle both EvTables and database query output. For huge data sets, pagination can give a 100-fold speed-increase. This is noticeable e.g. in the scripts and spawn/list commands, once you have a lot of items.
  • EvMenu templating language, to make it easier to create simpler menus. 
  • Webclient improvements: Cleanup of interface and the ability for players to save/load their pane layouts from the client. The developer can still provide a default for them to start out with.
  • MUD/Evennia Intro wizard to the tutorial world to explain basic game controls in an interactive way. 
  • Default channels can now be defined in settings instead of having to do so from in-game. 
  • New documentation system (see below).
  • Many, many bug fixes and optimizations!

Many contributors helped out along the way. See the changelog where contributors of the bigger new features are listed.

The path to a new documentation

For many years we've used the Github wiki as our documentation hub. It has served us well. But as mentioned in my previous post, it has its drawbacks, in particular when it comes to handling documentation for multiple Evennia versions in parallel.

After considering a bunch of options, I eventually went with sphinx, because it has such a good autodoc functionality (parsing of the source-code docstrings). This is despite our wiki docs are all in markdown and I dislike restructured text quite a bit. Our code also uses friendly and in-code-readable Google-style docstrings instead of Sphinx' hideous and unreadable format. 

Luckily there are extensions for Sphinx to handle this: 

  • Napoleon to convert Google-style docstrings to reST on the fly
  • recommonmark to convert our markdown wiki pages to reST on compile-time
  • sphinx-multiversion to merge docs from one or more GIT branches into a documentation where you can select between the versions.

What could go wrong? Well, it's been quite a ride.

Getting Markdown into reST

Linking to things in recommonmark turned out to be very flaky. I ended up forking and merging a bunch of PRs from the project but that was not enough: Clearly this thing was not built to convert 200 pages of technical markdown from a github wiki.

My custom fork of recommonmark had to be tweaked a bit for my needs, such as not having to specify the .md file ending in every link and make sure the url-resolver worked as I expected. There were a bunch of other things but I will probably not merge this back, the changes are pretty Evennia-specific.

Even so, many of my wiki links just wouldn't work. This is not necessarily recommonmark's fault, but how sphinx works by grouping things into toctrees, something that the Evennia wiki doesn't have. 

Also, the recommonmark way to make a toctree in Markdown is to make a list of links - you can't have any descriptive text, making the listing quite useless (apparently people only want bland lists of link-names?). After trying to figure out a way to make this work I eventually capitulated - I make pretty lists in Markdown while using a "hidden" toctree to inform sphinx how the pages are related.

Getting the wiki into the new doc site

This required more custom code. I wrote a custom importer that reads the wiki and cleans/reformats it in places where recommonmark just dies on them. I also made a preprocessor that not only finds orphan pages but also builds a toctree and remaps all links in all documents to their actual location on compilation. The remapper makes it a lot easier to move things around. The drawback is that every page needs to be uniquely named. Since this was already the case in the wiki, this was a good tradeoff. So with a lot of custom code the wiki eventually could port automatically.   

The thing is, that even with all this processing, recommonmark doesn't support stuff like Markdown tables, so you still have to fall back to reST notation for those. And Napoleon, while doing a good job of parsing google docstrings, do not expect Markdown. So the end result is mostly markdown but we still have to fall back to reST for some things. It's probably as far as we get.

Deploying the docs 

Figuring out how to build and deploy these components together was the next challenge. Sphinx' default Makefile was quite anemic and I also wanted something that regular contributors could use to test their documentation contributions easily. I ended up having to expand the Makefile quite a lot while also adding separate deploy scripts and interfaces to github actions (which we recently started using too).

Finally, the versioning. The sphinx-multiversion plugin works by extracting the branches you choose from git and running the sphinx compiler in each branch.  The plugin initially had a bug with how our docs are located (not at the root of the package) but after I reported it, it was quickly fixed. The result is a static document site where you can select between the available versions in the sidebar.

I've not gotten down to trying to make LaTeX/PDF generation work yet. I'm dreading it quite a bit... 

Where we are

The github wiki is now closed for external contributions. The v0.9.5 of the new documentation will pretty much be an import of the last state of the wiki with some minor cleanup (such as tables). While we'll fix outright errors in it, I don't plan to do many fixes of purely visual glitches from the conversion - the old wiki is still there should that be a problem.

The main refactoring and cleanup of the documentation to fit its new home will instead happen in v1.0. While the rough structure of this is already in place, it's very much a work in progress at this point.

Conclusions

Evennia 0.9.5 has a lot of features, but the biggest things are 'meta' changes in the project itself. After it is out, it's onward towards 1.0 again!


Tuesday, April 14, 2020

Spring updates while trying to stay healthy

So, spring grows nearer for those of us on the Northern hemisphere. With everyone hopefully hunkered down and safe from the Covid-19 pandemic, I thought it overdue to make another dev blog for the progress of Evennia, the Python MU*-creation system.


The last few months have seen primarily bug fixing on the Evennia front, but it also has seen an uptick of PRs from the community and the re-opening of the develop branch in earnest. There is still quite a lot of work to do before we can add that extra 0.1 and go from version 0.9 to 1.0.

What's in a version?


For me personally, I never put much stock in the notion of versions. Evennia didn't even have versions until a few years back: We used to just have a rolling git release. But eventually it became clear that our user base was big enough that we needed to more clearly separate major (and possibly breaking) updates from what came before. So I started versioning at Evennia 0.5 and have had roughly a new release every year since (not a plan or a promise, it just happened to turn out that way).

Evennia has been useful (and been used) for game development for many years already. But there is no denying that a 1.x label tends to convey more confidence in a system than a 0.x label, that's just the way things are. So while the new version is still quite some way off, there are a bunch of changes and improvements that we want to do in this release to mark the version change in a good way.

Documentation changes


Our documentation will move away from our trusty Github wiki. Instead we will convert the wiki into a static github page built from sources inside evennia/docs/.

The advantage of the wiki is that it is a very low entry for people to contribute and fix things using Github's editing system. We have had a lot of use of this over the years and the wiki has served us well. The drawbacks are starting to get ever more noticeable, however:
  • Whereas the wiki is itself version-controlled, we cannot show multiple versions of the wiki at the same time. This makes it hard to update the documentation at the same time as non-released code is being written. This is probably my main reason for doing the change.
  • The wiki today consists of some 200+ pages. It is hard to get an overview of what is where and what needs to be updated. 
  • The wiki word-search functionality is not really great.
  • It's impossible to review changes before they go live, to check consistency and style. This has led to some documentation pages overlapping.
  • Building the documentation to local HTML or PDF is an archaic process that I doubt anyone but me has done with any regularity. 
The change so far planned is to switch to the Sphinx documentation build-system (same as Python/Django etc is using). We will use it with extensions that allows us to still use Markdown like in the old wiki. This also allows us to build a more comprehensive (and pretty) API documentation of the entire library. We have more options to add comprehensive online search functionality in this solution as well.

Furthermore, will hopefully be able to set it up so that we can maintain and publish separate documentations for each forthcoming release. That is, you should be able to read the docs for 1.0, 1.1 or the latest master development as you like (similarly to how Django does it, although probably not as fancy from the onset). 

This means that contributions to the documentation will be done as PRs through GitHub, just like when contributing any other code. While this does add a little more of a hurdle to contributions, hopefully the benefits will far outweigh those. Building the docs locally will not require a running Evennia server (unless you want the api docs) and we will try to set everything up for to make it easy to contribute.

Many of the details around the docs are still up in the air. This is still very much work-in-progress, like everything else.

Work with this has started in the static-file-docs branch of Evennia. But we have not closed the wiki either - the two will exist in parallel for now.

PyPi


As mentioned before,  we will finally start to distribute Evennia via PyPi (the Python Package Index) - that is, you will be able to run `pip install evennia`. Using GIT will no longer be a requirement to get started.

Considering how quickly people in open-source throw up their three lines of code on PyPi these days, it may be surprising Evennia is not already on PyPi. I have however felt that reading and referencing the highly-commented code is a big part and requirement for getting the most out of the library.

With the new documentation system, this would improve. And you can of course still use git and follow master branch like the good ol' days if you want!

Web Admin improvements


For the longest time, the Django-admin component has been somewhat on the back-burner. With the help of community contributors, this is improving so you will be able to do more work the Admin GUI related to creating and managing objects, tie puppets to Accounts etc.

API improvements 


Whereas the last few months have been mostly spent fixing lingering bugs, one thing planned for version 1.0 is a general cleanup of legacy strangeness in the API. For example, certain methods can return a list or a single object depending situation, which makes it hard to generalize. There are a lot of small quirks like that which we hope to address and make more consistent.

There has also been a recent flurry of contributor PRs intended to help decouple Evennia's systems and make them easier to replace for those inclined to do so. Many of this is still being worked on, but it's likely you'll be able to replace many more "core" components for 1.0 with your own variations without doing any hacking in upstream code at all.
 ... Needless to say, this is an advanced feature that most developers will never need. But Evennia was always intended to be heavily customizable and having the option is a good thing!

Another feature that will come for 1.0 is a REST-API, added by one of our contributors. This uses Django-REST-Framework and makes it easier for external programs to authenticate and fetch data out of the Evennia database (great both for external apps, websites or custom what-have-you).
At this time you can only fetch database objects via this interface, you cannot perform Command-calls or "play the game" this way (REST is a stateless concept and Evennia Commands tend to retain state).

Many other fixes and contributions


There's a truckload of stuff already available in master branch, but with the latest contributions of bigger code changes, we have started to use the Evennia develop branch again in earnest again. For a summary of the changes so far, check out the Changelog

However, unless you want to contribute to Evennia itself (or really, really want to be on the bleeding edge), you are still recommended to use the master branch for now. A lot of work still to do, as said.


Image ©George Hodan, released under CC0 Public Domain

Monday, September 30, 2019

Blackifying and fixing bugs

Since version 0.9 of Evennia, the MU*-creation framework, was released, work has mainly been focused on bug fixing. But there few new features also already sneaked into master branch, despite technically being changes slated for Evennia 1.0.




On Frontends

Contributor friarzen has chipped away at improving Evennia's HTML5 web client. It already had the ability to structure and spawn any number of nested text panes. In the future we want to extend the user's ability to save an restore its layouts and allow developers to offer pre-prepared layouts for their games. Already now though, it has gotten plugins for handling both graphics, sounds and video:

Inline image by me (griatch-art.deviantart.com)


A related fun development is Castlelore Studios' development of an Unreal Engine Evennia plugin (this is unaffiliated with core Evennia development and I've not tried it, but it looks pretty nifty!): 

Image ©Castlelore Studios
 
On Black

Evennia's source code is extensively documented and was sort of adhering to the Python formatting standard PEP8. But many places were sort of hit-and-miss and others were formatted with slight variations due to who wrote the code.
 
After pre-work and recommendation by Greg Taylor, Evennia has adopted the black autoformatter for its source code. I'm not really convinced that black produces the best output of all possible outputs every time, but as Greg puts it, it's at least consistent in style. We use a line width of 100.

I have set it up so that whenever a new commit is added to the repo, the black formatter will run on it. It may still produce line widths >100 at times (especially for long strings), but otherwise this reduces the number of different PEP8 infractions in the code a lot.

On Python3

Overall the move to Python3 appears to have been pretty uneventful for most users. I've not heard almost any complaints or requests for help with converting an existing game.
The purely Python2-to-Python3 related bugs have been very limited after launch; almost all have been with unicode/bytes when sending data over the wire.

People have wholeheartedly adopted the new f-strings though, and some spontaneous PRs have already been made towards converting some of Evennia existing code into using them.

Post-launch we moved to Django 2.2.2, but the Django 2+ upgrades have been pretty uneventful so far.Some people had issues installing Twisted on Windows since there was no py3.7 binary wheel (causing them to have to compile it from scratch). The rise of the Linux Subsystem on Windows have alleviated most of this though and I've not seen any Windows install issues in a while.

On Future

For now we'll stay in bug-fixing mode, with the ocational new feature popping up here and there. In the future we'll move to the develop branch again. I have a slew of things in mind for 1.0. 

Apart from bug fixing and cleaning up the API in several places, I plan to make use of the feedback received over the years to make Evennia a little more accessible for a new user. This means I'll also try reworking and consolidating the tutorials so one can follow them with a more coherent "red thread", as well as improving the documentation in various other ways to help newcomers with the common questions we hear a lot. 

The current project plan (subject to change) is found here. Lots of things to do!



Top image credit: https://www.goodfreephotos.com/ (public domain)

Thursday, July 4, 2019

Evennia 0.9 released

Last week we released Evennia 0.9, the next version of the open source Python MU* creation system.

This release is the result of about 10 months of development, featuring 771 commits, 70 closed pull requests from the community and something like 80 issues and feature/requests closed. Thanks everyone!



The main feature of Evennia 0.9 is that we have finally made the move to Python3. And we burn the bridges behind us; as announced in previous posts we completely drop Python2 support and move exclusively to only support the latest Python3.7.

Overall the move to Python3 was not too bloody (and much work towards a never published py2+3 version was already done by Evennia contributors in a separate branch earlier). The main issues I ran into were mainly in the changes in how Python3 separates strings from bytes. This became crticial since Evennia implements several connection protocols; there were a lot of edge cases and weird errors appearing where data went to and from the wire.

A regular user has it a lot easier though. So far people have not had too much trouble converting their games from 2.7 to 3.7. The biggest Linux distros don't all have Py3.7 out of the box though, so that may be a concern for some, we'll see.

... but Py3 is nowhere all there is to find in this release though! There are a plethora of more features in the latest Evennia, all to make it easier to make the text-based multiplayer game of your dreams.

You can see a summary of new features in the ML announcement and even more details in the actual CHANGELOG file.


So what's up next?

Now follows a period of bug-fixing and stabilizing. Maybe resolve some of those long-standing "tech-dept" issues and overall make Evennia more stable.

Eventually work will then commence (in the develop branch) on version 1.0 of Evennia. For this next release I think I'll step back from new features a bit and focus on refactoring and cleanup of the API as well as other things around the library's distribution, documentation and presentation.

But for now, onward to summer vacations.

Sunday, May 26, 2019

Creating Evscaperoom, part 2


Jester smiling oh so sweetly
The Jester, your 'adversary'


This is part two of my post-mortem dev-blog about Evscaperoom, the multiplayer, text-based 'escape room' I wrote in Python and Evennia. You can read the first part of the dev blog here.

This was a game-jam entry I created in a month for the Mud Coder's guild's Game Jam. The theme was One Room. You can play the game for free in your browser or with a traditional MUD client. There are no spoilers in these blog posts. 

Update: These days you can play the Evscaperoom by logging into the Evennia demo game at https://demo.evennia.com. It's just one of the exits you can get through when you enter.

The first part dealt with the overall game-design aspects. This second, final part will go into details of the code and the systems I built to quickly create all the content. The code referenced here is released under the BSD license and is available on github.

At the time of this post, players have played Evscaperoom for a little more than a week. At the end I'll share some observations and things I learned along the way.

Ease of building

Over the one-month game jam, I spent about four days making the game's 'engine' and toolset with Evennia. The rest of the time was spent using those tools to actually create game content (the story, puzzles etc).

An important thing was that I didn't want to do any traditional in-game 'building'. That is - no logging into the game and running various commands to build objects and rooms. This is partly because I wanted rooms to be buildable on-demand, but also because I didn't want my game to only exist in the database but in actual version-controllable python modules.

So all of the Evscaperoom is created in code (notably in the game states discussed below). This made it so that I could add unit tests to quickly find bugs and edge cases. It also made it easy to just clone the full game to an online server, init a database and run Evennia on it in a docker when time came to make it public.


Overall game structure 

The main Evscaperoom menu, showing the option to create a new room or join one of two existing rooms.
Main menu
The game loop is simple: When you log in to the game, you get into a menu where you can create a new room to solve or join an existing one. Quitting a room brings you back to that menu. Quitting again leaves the game entirely. In between, you remain inside a single game location ('room').

To make it easier for people to agree to meet up in a room, i made a little 'fantasy name generator' to make unique random names of the rooms. It felt more thematic than showing room id's. The generator combines phonemes together with some very simple logic. Not all comes out easy-to-pronounce, but the result is at least identifiable, like the Sheru and Uyoha above.

I decided that I should not keep empty rooms around, so whenever a room has no more players in it, it's deleted from the database along with all its content. This means players can't really log off and come back to the same room unless a friend stays behind. I felt it was worth keeping things clean and avoid a growing backlog of empty, unsolved rooms. It is, unfortunately, quite common for players to log in, create a room and then immediately log off.

I distribute Evscaperoom as an Evennia 'game dir'. Once you've installed Evennia you can just clone the evscaperoom repo and start a new multiplayer server with it. While the game dir has some Evennia templates in it by default, Almost all the custom logic for this game is in the evscaperoom/ folder. The only other modification I did was to make sure Evennia rerouted new players into the Evscaperoom menu when they connect.


Room class

Since all of the gameplay happens in a single room, it made sense to center all of the data-storage around a new, custom Evennia Room class. This "EvscapeRoom" class holds all resources for this room. Evennia makes sure to persist it all to the database for you.

The Evennia API provides a lot of powerful but game-general functions. Since our use-case for this game is very strictly defined, I made a slew of helper functions to cut down on boiler plate and pre-set options I wanted to always use.

For example, I added helper methods both for creating and finding objects in the room. On creation, all objects are tagged with the room's unique hash, meaning that one can be sure to never have any cross-over between rooms (like accidentally finding the object in another room (that of course has the exact same name). Since I also decided to never have more than one object with a given name per room, I could make these methods very simple.

The room class also has helpers for finding all players in the room and for sending messages to them. It also catches players leaving so that eventual on-character variables can be cleaned.

Importantly, the very action of deleting the room will automatically clean all resources tied to it, keeping things tidy.


Commands and Objects

This shows a list of all commands useful in the room.
The help screen, show all top-level commands
As discussed in part one of this blog, the Evscaperoom, uses a 'focus' mode: The player must examine/focus on an object first in order to operate or use it.

The basic command syntax is:
> command [target]
The parsing I made actually allows for a more complex syntax, but in the end this was all that was really needed, since the currently 'focused' object does not need to be specified. This is the process of using one object with another:

> examine key

~~ key (examining) ~~ 
This is a brass key.

(To unlock something with it, use insert into <target>)

> insert into door

You unlock the door with the key!

(the into is optional). Here, we focus on the key. We get the key's description and a hint that you can insert it into things. We then insert it into the door, which is another object in the room. The insert command knows that we are focusing on the key already and that it should look into the room for an object door to use this with.

Technically, these on-object 'actions' (like insert above), are dynamically generated. Here is an example of the key object:

class Key(EvscaperoomObject):
    def at_focus_insert(self, caller, **kwargs):
        target = kwargs['args']
        obj = caller.search(obj)
        if not obj: 
            return 
        if obj.check_flag("can_use_key"):
            obj.handle_insert(self)

Not shown here is that I made a wrapper for the "no-match" command of Evennia. This command fires when no other commands match. I made this instead analyze the currently 'focused' object to see if it had a method at_focus_<command_name> on it. If so, I inject the supplied arguments into that method as a keyword argument args.

So when you focus on the key and give the insert command, the at_focus_insert method on the key will be called with a target to insert the key into. We search for the target (the door in the example), check if it even accepts keys and then pass the key to that object to handle. It would then be up to the door to figure out if this particular key unlocks it.

I created a library of base objects that I can just use as mixins for the object I want to create. Here's an example:

from evscaperoom import objects

class Box(objects.Openable, 
          objects.CodeInput, 
          objects.Movable):
     # ...
    
This class will offer actions to open, insert a code and move the object around. It will need some more configuration and addition of messages to show etc. But overall, this method-to-command solution ended up being very stable and extremely easy to use to make complex, interactive objects.


Room states

I think of the escape room as going through a series of states. A change of state could for example be that the user solved a puzzle to open a secret wall. That wall is now open, making new items and puzzles available. This means room description should change along with new objects being created or old ones deleted.

I chose to represent states as Python modules in a folder. To be a state, each module needs to have a global-level class State inheriting from my new BaseState class. This class has methods for initializing and cleaning up the state, as well as was for figuring out which state to go to next. As the system initializes the new state, it gets the current room as argument, so it can modify it.

This is a (simplified) example of a state module:  

# module state_001_start.py
       
from evscaperoom.state import BaseState
from evscaperoom import objects


MUG_DESC = """
A blue mug filled with a swirling liquid. 
On it is written "DRINK ME" with big letters.
"""

class Mug(objects.EvscapeRoomObject):
    def at_focus_drink(self, caller, **kwargs): 
        caller.msg(f"You drink {self.key}.")      
        self.next_state() # trigger next state

class State(BaseState):

    hints = ["You are feeling a little thirsty...",
             "Drink from the mug, dummy."]

    next_state = "state_002_big_puzzle"

    def init(self): 
        mug = self.create_object(
            Mug, key="wooden mug", aliases=["mug"])
        mug.db.desc = MUG_DESC.strip()

In this simple state, a mug is created, and when you drink from it, the next state is triggered. The base object has a helper function to trigger the next state since I found that interactive with an object is almost always the reason for switching states.

The state-class has a lot of useful properties to set, such as which the next state should be (this can be overridden in case of branching paths). You can also store
a sequence of hints specific for that state.



Informing the room

I wrote the content in second-person perspective ("You open the door"). This is however a multiplayer game and I didn't intially appreciate how many texts must also exist in a third-party form for the rest of the room to see ("Griatch opens the door").

As the amount of text grew (the Evscaperoom has close to 10 000 lines of code, a lot of which is content strings), it became clear that it would not be feasible to manually supply third-persion version strings as well.

The solution was to add parsing and translation of pronouns and verbs (a concept I first saw on the game Armageddon).

I write the string like this:

OPEN_TEXT = "~You ~open the *door."

The ~ marks text that should be parsed for second/third-person use (I'll discuss the *door marking in the next section). This I then send to a helper method that either sends it only to you (which means it comes back pretty much the same, but without the special markers) or to you and to the room, in which it will look different depending on who receives it:

     I see "You open the [door]."
     Others see "Griatch opens the [door]."

English is luckily pretty easy to use for this kind of automatic translation - in general you can just add an "s" to the end of the verb. I made a simple mapping for the few irregular verbs I ended up using.

Overall, this made it quick to present multiple viewpoints with minimal extra text to write.

Shows the various accessibility options for showing items.
The option menu
The *door -style marking allowed me to generalize how target-able objects in the room were displayed. This meant that users can customize how objects are shown to them. The default is to mark them both with colors and square brackets (this makes it clear also for people with screen readers). But one can also use only colors or turn off the marking completely (hard mode).

Bringing it online

Evennia is both a mud framework and mudserver as well as a webserver based on Twisted. It runs the game's website (with the help of Django) and also provides its own HTML5 webclient. I tweaked the default website text and played a little with CSS but otherwise didn't spend too much time on this bit.

I got a $5/month DigitalOcean droplet with Ubuntu. I made a new, unprivileged "evennia" user on it and cloned the evscaperoom repo to it. I then started a tmux session and ran the Evennia docker image in there. Getting the game online took maybe thirty minutes, most of which was me figuring out where to open the droplet and DigitalOcean firewalls.

I then pointed http://experimental.evennia.com at the droplet's IP and that was it!

Updating the online server is now only a matter of pushing changes to my github repo, pulling it to the server and reloading Evennia; Before release, I used a private github repo for this, afterwards I simply made it public. Pretty straightforward.

Some lessons learned

I have gotten pretty positive reviews on Evscaperoom so far. In the first two days people stumbled on some edge-case bugs, but after that it has been very stable. Mostly I've had to make small typos and grammar corrections as I (or players) spot them. 

There were nevertheless some things I learned, some of which led to more real improvements post-launch.

No amount of help is too much help

Shows focus on the 'bed', with an example of the header telling how to leave the 'focus' mode.
The header shows how to get out of focus mode
Firstly, the focus-system (examine, then do stuff) is a little un-orthodox and needs to be explained. I saw people logging in, examining exactly one thing and then logging out. Eventually I found out (because a user told me), that this was likely because they could not figure out how to leave the focus mode. They'd just flounder about, think the game was buggy and log off. 

The answer (just run examine again) is found with the help command, but clearly this was not intuitive. The solution was to add an explicit help text to the top every time you examine something. After this, the confusion seems to have gone away. 

Make it easy to connect for all tastes

Another example - a commenting user had pretty strong opinions about the fact that you used to have to supply a username and password to play the game. They suggested this was a 'huge hurdle'. Not sure if that's true. But unless you want to use a particular name, there is also no actual gameplay reason to formally authenticate for Evscaperoom. 

This was easy to fix. Evennia has guest-player support out of the box so I just activated that and supplied some more fantasy-sounding names than the default "Guest 1", "Guest 2" etc. Since then, maybe 40% of players connecting have chosen to do so as an anonymous guest. I don't know if those would have left completely if the option was not available, but it's at least a convenient shortcut for getting into the game.

Everything takes longer than expected

I already knew this one, but still I fell into the trap of thinking that things were going well and that there would be plenty of time to finish before deadline. 

Creating text content is a lot faster than creating graphical assets, but it's still a lot of work. Just the ending 'cinematics' took me almost two days to finish and clean up at the end. 

For once I did pick a reasonable scale for this project though. So while the last few days of the Jam was more intense than I would have liked, I never truly felt like I would not be able to finish in time.


Building a MU* game in pure code is awesome

Evennia tries to not instil and specific game type, hence its tools are very general. Wrapping these general tools as a highly opinionated and game-specific toolbox enforced to me just how easy it is to do things when you don't need to cover the general case.

Using the tools and creating content purely in-code was great and ended up leading to a very fast content creation. Python works perfectly as a scripting language and I don't think there is a reason for using in-game building at all for your game, especially not when you are working on your own like this.

I made a few admin-only commands to jump between states and to set flags, but otherwise most bugs were caught by a generic unit test that just looped over all founds states and tried to initialize them, one after another.


Conclusions

For all my work on the Evennia library/server, I've not actually used it for games of my own very much. This was a great opportunity for doing so. It also gave me the opportunity to test the Python3-development branch of Evennia in a production setting.

I found a few edge-case library bugs which I fixed, but overall things worked very smoothly, also for this kind of game which is certainly far away from the normal MU*-mold that most use Evennia for. I am a bit biased, but overall I felt Evennia to be very mature for the use of making a game from scratch.

In the future I will most likely break out the 'engine' and tools of the Evscaperoom into an Evennia contrib so that others can make similar games with it easily.

Looking forward to future game jams now!

Saturday, May 18, 2019

Creating Evscaperoom, part 1

Over the last month (April-May 2019) I have taken part in the Mud Coder's Guild Game Jam "Enter the (Multi-User) Dungeon". This year the theme for the jam was One Room.

The result was Evscaperoom, an text-based multi-player "escape-room" written in Python using the Evennia MU* creation system. You can play it from that link in your browser or MU*-client of choice. If you are so inclined, you can also vote for it here in the jam (well, no more).

This little series of (likely two) dev-blog entries will try to recount the planning and technical aspects of the Evscaperoom. This is also for myself - I'd better write stuff down now while it's still fresh in my mind!

Update: The next part of this blog is here.

Update 2: The Evscaperoom can these days be played by connecting to the Evennia game demo at https://demo.evennia.com.


Inception 

When I first heard about the upcoming game-jam's theme of One Room, an 'escape room' was the first thing that came to mind, not the least because I just recently got to solve my my own first real-world escape-room as a gift on my birthday. 

If you are not familiar with escape-rooms, the premise is simple - you are locked into a room and have to figure out a way to get out of it by solving practical puzzles and finding hidden clues in the room. 

While you could create such a thing in your own bedroom (and there are also some one-use board game variants), most escape-rooms are managed by companies selling this as an experience for small groups. You usually have one hour to escape and if you get stuck you can press a button (or similar) to get a hint.

I thought making a computer escape-room. Not only can you do things in the computer that you cannot do in the real world, restricting the game to a single room limits so that it's conceivable to actually finish the damned thing in a month. 

A concern I had was that everyone else in the jam surely must have went for the same obvious idea. In the end that was not an issue at all though.


Basic premises
 
I was pretty confident that I would technically be able to create the game in time (not only is Python and Evennia perfect for this kind of fast experimentation and prototyping, I know the engine very well). But that's not enough; I had to first decide on how the thing should actually play. Here are the questions I had to consider:

Room State 

 An escape room can be seen as going through multiple states as puzzles are solved. For example, you may open a cabinet and that may open up new puzzles to solve. This is fine in a single-player game, but how to handle it in a multi-player environment?

My first thought was that each object may have multiple states and that players could co-exist in the same room, seeing different states at the same time. I really started planning for this. It would certainly be possible to implement.

But in the end I considered how a real-world escape-room works - people in the same room solves it together. For there to be any meaning with multi-player, they must share the room state.

So what I went with was a solution where players can create their own room or join an existing one. Each such room is generated on the fly (and filled with objects etc) and will change as players solve it. Once complete and/or everyone leaves, the room is deleted along with all objects in it. Clean and tidy.

So how to describe these states? I pictured that these would be described as normal Python modules with a start- and end function that initialized each state and cleaned it up when a new state was started. In the beginning I pictured these states as being pretty small (like one state to change one thing in the room). In the end though, the entire Evscaperoom fits in 12 state modules. I'll describe them in more detail in the second part of this post. 

Accessibility and "pixel-hunting" in text

When I first started writing descriptions I didn't always note which objects where interactive. It's a very simple and tempting puzzle to add - mention an object as part of a larger description and let the player figure out that it's something they can interact with. This practice is sort-of equivalent to pixel-hunting in graphical games - sweeping with the mouse across the screen until you find that little spot on the screen that you can do something with.

Problem is, pixel-hunting's not really fun. You easily get stuck and when you eventually find out what was blocking you, you don't really feel clever but only frustrated. So I decided that I should clearly mark every object that people could interact with and focus puzzles on better things.


In fact, in the end I made it an option:

Option menu ('quit' to return)   1: ( ) No item markings (hard mode)  2: ( ) Items marked as item (with color)  3: (*) Items are marked as [item] (screenreader friendly)  4: ( ) Screenreader mode

As part of this I had to remind myself never to use colors only when marking important information: Visually impaired people with screen readers will simply miss that. Not to mention that some just disable colors in their clients.

So while I personally think option 2 above is the most visually pleasing, Evscaperoom defaults to the third option. It should should start everyone off on equal footing. Evennia has a screen-reader mode out of the box, but I moved it into the menu here for easy access.

Inventory and collaboration

In a puzzle-game, you often find objects and combine them with other things. Again, this is simple to do in a single-player game: Players just pick things up and use them later.

But in a multi-player game this offers a huge risk: players that pick up something important and then log off. The remaining players in that room would then be stuck in an unsolvable room - and it would be very hard for them to know this.

In principle you could try to 'clean' player inventories when they leave, but not only does it add complexity, there is another issue with players picking things up: It means that the person first to find/pick up the item is the only one that can use it and look at it. Others won't have access until the first player gives it up. Trusting that to anonymous players online is not a good idea.

So in the end I arrived at the following conclusions:
  • As soon as an item/resource is discovered, everyone in the room must be able to access it immediately.
  • There can be no inventory. Nothing can ever be picked up and tied to a specific player.
  • As soon as a discovery is made, this must be echoed to the entire room (it must not be up to the finder to announce what they found to everyone else).  
As a side-effect of this I also set a limit to the kind of puzzles I would allow:
  • No puzzles must require more than one player to solve. While one could indeed create some very cool puzzles where people collaborate, it's simply not feasible to do so with random strangers on the internet. At any moment the other guy may log off and leave you stuck. And that's if you even find someone logged in at the same time in the first place! The room should always be possible to solve solo, from beginning to end.

Focusing on objects

So without inventory system, how do you interact with objects? A trademark of any puzzle is using one object with another and also to explore things closer to find clues. I turned to graphical adventure games for inspiration:

Hovering with mouse over lens object offers action "look at lens"
Secret of Monkey Island ©1990 LucasArts. Image from old-games.com

A common way to operate on an object in traditional adventure games is to hover the mouse over it and then select the action you want to apply to it. In later (3D) games you might even zoom in of the object and rotate it around with your mouse to see if there are some clues to be had.

While Evennia and modern UI clients may allow you to use the mouse to select objects, I wanted this to work the traditional MUD-way, by inserting commands. So I decided that you as a player would be in one of two states:
  • The 'normal' state: When you use look you see the room description.
  • The 'focused' state: You focus on a specific object with the examine <target> command (aliases are ex or just e). Now object-specific actions become available to you. Use examine again to "un-focus". 
A small stone fireplace sits in the middle of the wall opposite the [door]. On the chimney hangs a small oil [painting] of a man. Hanging over the hearth is a black [cauldron]. The piles of [ashes] below are cold.  (It looks like fireplace may be suitable to [climb].)

In the example above, the fireplace points out other objects you could also focus on, whereas the last parenthesis includes one or more "actions" that you can perform on the fireplace only when you have it focused. 

This ends up pretty different from most traditional MUD-style inputs. When I first released this to the public, I found people logged off after their first examine. It turned out that they couldn't figure out how to leave the focus mode. So they just assumed the thing was buggy and quit instead. Of course it's mentioned if you care to write help, but this is clearly one step too many for such an important UI concept. 

So I ended up adding the header above that always reminds you. And since then I've not seen any confusion over how the focus mode works.

For making it easy to focus on things, I also decided that each room would only ever have one object named a particular thing. So there is for example only one single object in the game named "key" that you can focus on. 

Communication

I wanted players to co-exist in the same room so that they could collaborate on solving it. This meant communication must be possible. I pictured people would want to point things out and talk to each other.

In my first round of revisions I had a truckload of individual emotes; you could

      point at target

 for example. In the end I just limited it to  

     say/shout/whisper <message>

and 

     emote <whatever>

And seeing what people actually use, this is more than enough (say alone is probably 99% of what people need, really). I had a notion that the shout/whisper could be used in a puzzle later but in the end I decided that communication commands should be strictly between players and not have anything to do with the puzzles.

I removed all other interaction: There is no fighting and without an inventory or requirement to collaborate on puzzles, there is no need for other interactions than to communicate.

First version you didn't even see what the others did, but eventually I added so that you at least saw what other players were focusing on at the moment (and of course if some major thing was solved/found).

In the end I don't even list characters as objects in the room (you have to use the who command to see who's in there with you).

Listing of commands available in the Evscaperoom (output of the help-command in game)
The main help command output.
Story

It's very common for this type of game to have a dangerous or scary theme. Things like "get out before the bomb explodes", "save the space ship before the engines overheat", "flee the axe murderer before he comes back" etc). I'm no stranger to dark themes, but for this I wanted something friendlier and brighter, maybe with a some dark undercurrents here and there.

My Jester character is someone I've not only depicted in art, but she's also an old RP character and literary protagonist of mine. Who else would find it funny to lock someone into a room only to provide crazy puzzles and hints for them to get out again? So my flimsy 'premise' was this: 


The village Jester wants to win the pie eating contest. You are one of her most dangerous opponents. She tricked you to her cabin and now you are locked in! If you don't get out in time, she'll get to eat all those pies on her own and surely win!

That's it - this became the premise from which the entire game flowed. I quickly decided that it to be a very "small-scale" story: no life-or-death situation, no saving of the world. The drama takes place in a small village with an "adversary" that doesn't really want to hurt you, but only to eat more pies than you.

From this, the way to offer hints came naturally - just eat a slice of "hintberry pie" the jester made (she even encourage you to eat it). It gives you a hint but is also very filling. So if you eat too much, how will you beat her in the contest later, even if you do get out?

To further the rustic and friendly tone I made sure the story took place on a warm summer day. Many descriptions describe sunshine, chirping birds and the smell of pie. I aimed at letting the text point out quirky and slightly comedic tone of the puzzles the Jester left behind. The player also sometimes gets teased by the game when doing things that does not make sense.

I won't go into the story further here - it's best if you experience it yourself. Let's just say that the village has some old secrets. And and the Jester has her own ways of doing things and of telling a story. The game has multiple endings and so far people have drawn very different conclusions in the end.

Scoring

Most often in escape rooms, final score is determined by the time and the number of hints used. I do keep the latter - for every pie you eat, you get a penalty on your final score.

As for time - this background story would fit very well with a time limit (get out in X time, after which the pie-eating contest will start!). But from experience with other online text-based games I decided against this. Not only should a player be able to take a break, they may also want to wait for a friend to leave and come back etc. 

But more importantly, I want players to explore and read all my carefully crafted descriptions! So I'd much rather prefer they take their time and reward them for being thorough. 

So in the end I give specific scores for actions throughout the game instead. Most points are for doing things that drive the story forward, such as using something or solving a puzzle. But a significant portion of the score comes from turning every stone and trying everything out. The nice side-effect of this is that even if you know exactly how to solve everything and rush through the game you will still not end up with a perfect score. 

The final score, adjusted by hints is then used to determine if you make it in time to the contest and how you fare. This means that if you explore carefully you have a "buffer" of points so eating a few pies may still land you a good result in the end.
 

First sketch

I really entered the game 'building' aspect with no real notion of how the Jester's cabin should look nor which puzzles should be in it. I tried to write things down beforehand but it didn't really work for me. 

So in the end I decided "let's just put a lot of interesting stuff in the room and then I'll figure out how they interact with each other". I'm sure this is different from game-maker to game-maker. But for me, this process worked perfectly. 


Scribbles on my notebook, sketching up the room's main items
My first, very rough, sketch of the Jester's cabin

The above, first sketch ended up being what I used, although many of the objects mentioned never ended up in the final game and some things switched places. I did some other sketches too, but they'd be spoilers so I won't show them here ...


The actual game logic

The Evscaperoom principles outlined above deviate quite a bit from the traditional MU* style of game play. 

While Evennia provides everything for database management, in-game objects, commands, networking and other resources, the specifics of your game is something you need to make yourself - and you have the full power of Python to do it!

So for the first three days of the jam I used Evennia to build the custom game logic needed to provide the evscaperoom style of game play. I also made the tools I needed to quickly create the game content (which then took me the rest of the jam to make). 

In part 2 of this blog post I will cover the technical details of the Evscaperoom I built. I'll also go through some issues I ran into and conclusions I drew. I'll link to that from here when it's available!

Continue to part 2.

Thursday, May 9, 2019

Podcast about Evennia

I was interviewed on the (pretty grandiosely named) podcast Titans of Text the other day.

In the interview, which are run by people from the MUD Coder's Guild (a great initiative!), I talk a bit about the history of Evennia, the text-based multiplayer game engine I'm working on, and go into some various technical aspects of the engine as well. Check it out and support the podcast!

https://www.titansoftext.com/4