Evennia 0.9.5 released!

 

As of today, Evennia 0.9.5 is out. Evennia is a Python based library and framework for creating text-based multiplayer games (MUD/MU*). 

This is a gradual improvement halfway between 0.9 and the upcoming 1.0. So if you have been keeping up-to-date with the master branch of Evennia over the last year you will not notice much difference from this release (time to upgrade if you haven't been keeping up though!). 

While an interim release, there are still a lot of things that has happened since v0.9: 

Webclient improvements

• Big web client improvements (courtesy of contributor friarzen) - players can now save and restore pane layouts directly in the client (so you could have a separate pane for channel chatter, another for look-returns, two input panes etc etc). 
• The layout changes makes it easier for devs to create default layouts to offer to players of their game. People in the Evennia community have already started doing very cool stuff with this, I'll try to gather screenshots for a future blog.
  
• Allow to redirect video/music to separate panes.
• Many other fixes, such as improving the input-history behavior.
  

EvMenu improvements

EvMenu is a powerful system for creating in-game text menus.

• The EvMenu class was refactored to be easier to override. For example, all messages now go through EvMenu.msg which allows for easy customization. It also defaults to sending with a type of "menu", making it easier to redirect menus to seprate panes in the webclient.
• In a node, EvMenu is now accessed via the much more logically named .ndb._evmenu instead of .ndb._menutree (the old name still works for backwards compatibility, but is deprecated).
• New optional EvMenu template system for quickly building simpler EvMenus without needing so much code. This makes it easy to catch and parse arbitrary input from the user and redirect to the correct node as needed. Creating menu nodes as functions still work (and is a lot more powerful), this can be mixed with templating to create different effects.
  

New settings

• INLINEFUNC_STACK_MAXSIZE is an integer that allows to control how big the inlinefunc nesting stacksize is.
  
• New DEFAULT_CHANNELS setting to allow customization of which channels should be initialized on startup. This can be modified after initial server start.
• CHANNEL_HANDLER_CLASS allows for specifying an alternative to the default ChannelHandler if wanting to change how Channels behave. 
• SERVER_LOG_DAY_ROTATION defines how many days the server log should run before being force-rotated (default is seven days).
• SERVER_LOG_MAX_SIZE specifies how big the log must be before it auto-rotates (even if SERVER_LOG_DAY_ROTATION days has not passed yet). 
• PORTAL_LOG_DAY_ROTATION, PORTA_LOG_MAX_SIZE - equivalent for the Portal.
  

 Other improvements

• The EvMore pager saw big performance speedups, making the viewing of large numbers of entries much snappier. You can now also paginate EvTables directly and create custom pagers by override the EvMore class (useful if you want to e.g. do a EvTable per-page).   
• Improvement to the multi-match parser: Trying to get for example 3-box will now fail with a no-found if there are only two boxes in the room (before it would show the multi-match menu).  
• New inside_rec lockfunc to recursively check if an object is inside another. Putting this on a room will thus also check the contents of any objects in the room, not only the contents themselves. Or if you had something in your wallet (a container).
• New $random inlinefunc for producing a random number in strings.
  
• TickerHandler.add() now returns a store_key to uniquely describe the ticker just added. The TickerHandler.remove() accepts a new kwarg store_key for removing the ticker - this makes it easier to manage tickers instead having to insert the full specifications of the ticker to remove it.
• Many fixes to the spawn command and prototype functionality. The new spawn/raw flag will now return the prototype-dict so one can manually edit and copy&paste it.
• The evennia.GLOBAL_SCRIPTS container will now contain all global scripts, not only those explicitly created with the GLOBAL_SCRIPTS setting. 
• The list_to_string utility converts a list to a nice string-representation, such as ["a", "b", "c", "d"] -> "a, b, c and d". The function is renamed to iter_to_string (but old name still works) and now also works with generators and will not crash even when provided a single value.
• A lot of bug fixes and stability fixes!
  

See the changelog here.

 

New documentation system 

The bigger change with 0.9.5 is that we are moving to a new documentation system. The details of the long road to do this is documented in my prervious post. The point is that we are stopping the use of the Github wiki in favor of statically generated documentation hosted on github pages. At the same time we also move the old evennia.com website from Google-sites to Github. 

Check it out: 

• New evennia.com: https://evennia.com
• New static documentation:  https://evennia.com/docs/latest
  

 As for the docs, they will be maturing for a long time still. The old wiki will not be updated anymore, but it will also not be going anywhere in the short term. Version 0.9.5 of the docs is pretty much a copy of the wiki and I hope to not have to spend too much more work maintaining it since the wiki is still around.

 

New updates and documentation features will primarily be happening in the 1.0-dev version of the documentation. This will include refactoring all pages as well as a new intro-tutorial and many other things. 

 

But that's for future blogs ...

 

On using Markdown with Sphinx - onward to Evennia 0.9.5

Last post I wrote about the upcoming v1.0 of Evennia, the Python MU* creation engine. We are not getting to that 1.0 version quite yet though: The next release will be 0.9.5, hopefully out relatively soon (TM).

Evennia 0.9.5 is, as you may guess, an intermediary release. Apart from the 1.0 roadmap just not being done yet, there is one other big reason for this - we are introducing documentation versioning and for that a proper release is needed as a base to start from. Version 0.9.5 contains everything already in master branch, so if you have kept up-to-date you won't notice too much difference. Here are some highlights compared to version 0.9:

• EvMore will paginate and properly handle both EvTables and database query output. For huge data sets, pagination can give a 100-fold speed-increase. This is noticeable e.g. in the scripts and spawn/list commands, once you have a lot of items.
• EvMenu templating language, to make it easier to create simpler menus. 
• Webclient improvements: Cleanup of interface and the ability for players to save/load their pane layouts from the client. The developer can still provide a default for them to start out with.
  
• MUD/Evennia Intro wizard to the tutorial world to explain basic game controls in an interactive way. 
• Default channels can now be defined in settings instead of having to do so from in-game. 
  
• New documentation system (see below).
  
• Many, many bug fixes and optimizations!

Many contributors helped out along the way. See the changelog where contributors of the bigger new features are listed.

The path to a new documentation

For many years we've used the Github wiki as our documentation hub. It has served us well. But as mentioned in my previous post, it has its drawbacks, in particular when it comes to handling documentation for multiple Evennia versions in parallel.

After considering a bunch of options, I eventually went with sphinx, because it has such a good autodoc functionality (parsing of the source-code docstrings). This is despite our wiki docs are all in markdown and I dislike restructured text quite a bit. Our code also uses friendly and in-code-readable Google-style docstrings instead of Sphinx' hideous and unreadable format. 

Luckily there are extensions for Sphinx to handle this: 

• Napoleon to convert Google-style docstrings to reST on the fly
• recommonmark to convert our markdown wiki pages to reST on compile-time
  
• sphinx-multiversion to merge docs from one or more GIT branches into a documentation where you can select between the versions.
  

What could go wrong? Well, it's been quite a ride.

Getting Markdown into reST

Linking to things in recommonmark turned out to be very flaky. I ended up forking and merging a bunch of PRs from the project but that was not enough: Clearly this thing was not built to convert 200 pages of technical markdown from a github wiki.

My custom fork of recommonmark had to be tweaked a bit for my needs, such as not having to specify the .md file ending in every link and make sure the url-resolver worked as I expected. There were a bunch of other things but I will probably not merge this back, the changes are pretty Evennia-specific.

Even so, many of my wiki links just wouldn't work. This is not necessarily recommonmark's fault, but how sphinx works by grouping things into toctrees, something that the Evennia wiki doesn't have. 

Also, the recommonmark way to make a toctree in Markdown is to make a list of links - you can't have any descriptive text, making the listing quite useless (apparently people only want bland lists of link-names?). After trying to figure out a way to make this work I eventually capitulated - I make pretty lists in Markdown while using a "hidden" toctree to inform sphinx how the pages are related.

Getting the wiki into the new doc site

This required more custom code. I wrote a custom importer that reads the wiki and cleans/reformats it in places where recommonmark just dies on them. I also made a preprocessor that not only finds orphan pages but also builds a toctree and remaps all links in all documents to their actual location on compilation. The remapper makes it a lot easier to move things around. The drawback is that every page needs to be uniquely named. Since this was already the case in the wiki, this was a good tradeoff. So with a lot of custom code the wiki eventually could port automatically.   

The thing is, that even with all this processing, recommonmark doesn't support stuff like Markdown tables, so you still have to fall back to reST notation for those. And Napoleon, while doing a good job of parsing google docstrings, do not expect Markdown. So the end result is mostly markdown but we still have to fall back to reST for some things. It's probably as far as we get.

Deploying the docs 

Figuring out how to build and deploy these components together was the next challenge. Sphinx' default Makefile was quite anemic and I also wanted something that regular contributors could use to test their documentation contributions easily. I ended up having to expand the Makefile quite a lot while also adding separate deploy scripts and interfaces to github actions (which we recently started using too).

Finally, the versioning. The sphinx-multiversion plugin works by extracting the branches you choose from git and running the sphinx compiler in each branch.  The plugin initially had a bug with how our docs are located (not at the root of the package) but after I reported it, it was quickly fixed. The result is a static document site where you can select between the available versions in the sidebar.

I've not gotten down to trying to make LaTeX/PDF generation work yet. I'm dreading it quite a bit... 

Where we are

The github wiki is now closed for external contributions. The v0.9.5 of the new documentation will pretty much be an import of the last state of the wiki with some minor cleanup (such as tables). While we'll fix outright errors in it, I don't plan to do many fixes of purely visual glitches from the conversion - the old wiki is still there should that be a problem.

The main refactoring and cleanup of the documentation to fit its new home will instead happen in v1.0. While the rough structure of this is already in place, it's very much a work in progress at this point.

Conclusions

Evennia 0.9.5 has a lot of features, but the biggest things are 'meta' changes in the project itself. After it is out, it's onward towards 1.0 again!

Spring updates while trying to stay healthy

So, spring grows nearer for those of us on the Northern hemisphere. With everyone hopefully hunkered down and safe from the Covid-19 pandemic, I thought it overdue to make another dev blog for the progress of Evennia, the Python MU*-creation system.


The last few months have seen primarily bug fixing on the Evennia front, but it also has seen an uptick of PRs from the community and the re-opening of the develop branch in earnest. There is still quite a lot of work to do before we can add that extra 0.1 and go from version 0.9 to 1.0.

What's in a version?

For me personally, I never put much stock in the notion of versions. Evennia didn't even have versions until a few years back: We used to just have a rolling git release. But eventually it became clear that our user base was big enough that we needed to more clearly separate major (and possibly breaking) updates from what came before. So I started versioning at Evennia 0.5 and have had roughly a new release every year since (not a plan or a promise, it just happened to turn out that way).

Evennia has been useful (and been used) for game development for many years already. But there is no denying that a 1.x label tends to convey more confidence in a system than a 0.x label, that's just the way things are. So while the new version is still quite some way off, there are a bunch of changes and improvements that we want to do in this release to mark the version change in a good way.

Documentation changes

Our documentation will move away from our trusty Github wiki. Instead we will convert the wiki into a static github page built from sources inside evennia/docs/.

The advantage of the wiki is that it is a very low entry for people to contribute and fix things using Github's editing system. We have had a lot of use of this over the years and the wiki has served us well. The drawbacks are starting to get ever more noticeable, however:

• Whereas the wiki is itself version-controlled, we cannot show multiple versions of the wiki at the same time. This makes it hard to update the documentation at the same time as non-released code is being written. This is probably my main reason for doing the change.
• The wiki today consists of some 200+ pages. It is hard to get an overview of what is where and what needs to be updated. 
• The wiki word-search functionality is not really great.
• It's impossible to review changes before they go live, to check consistency and style. This has led to some documentation pages overlapping.
• Building the documentation to local HTML or PDF is an archaic process that I doubt anyone but me has done with any regularity. 

The change so far planned is to switch to the Sphinx documentation build-system (same as Python/Django etc is using). We will use it with extensions that allows us to still use Markdown like in the old wiki. This also allows us to build a more comprehensive (and pretty) API documentation of the entire library. We have more options to add comprehensive online search functionality in this solution as well.

Furthermore, will hopefully be able to set it up so that we can maintain and publish separate documentations for each forthcoming release. That is, you should be able to read the docs for 1.0, 1.1 or the latest master development as you like (similarly to how Django does it, although probably not as fancy from the onset). 

This means that contributions to the documentation will be done as PRs through GitHub, just like when contributing any other code. While this does add a little more of a hurdle to contributions, hopefully the benefits will far outweigh those. Building the docs locally will not require a running Evennia server (unless you want the api docs) and we will try to set everything up for to make it easy to contribute.

Many of the details around the docs are still up in the air. This is still very much work-in-progress, like everything else.

Work with this has started in the static-file-docs branch of Evennia. But we have not closed the wiki either - the two will exist in parallel for now.

PyPi

As mentioned before,  we will finally start to distribute Evennia via PyPi (the Python Package Index) - that is, you will be able to run `pip install evennia`. Using GIT will no longer be a requirement to get started.

Considering how quickly people in open-source throw up their three lines of code on PyPi these days, it may be surprising Evennia is not already on PyPi. I have however felt that reading and referencing the highly-commented code is a big part and requirement for getting the most out of the library.

With the new documentation system, this would improve. And you can of course still use git and follow master branch like the good ol' days if you want!

Web Admin improvements

For the longest time, the Django-admin component has been somewhat on the back-burner. With the help of community contributors, this is improving so you will be able to do more work the Admin GUI related to creating and managing objects, tie puppets to Accounts etc.

API improvements 

Whereas the last few months have been mostly spent fixing lingering bugs, one thing planned for version 1.0 is a general cleanup of legacy strangeness in the API. For example, certain methods can return a list or a single object depending situation, which makes it hard to generalize. There are a lot of small quirks like that which we hope to address and make more consistent.

There has also been a recent flurry of contributor PRs intended to help decouple Evennia's systems and make them easier to replace for those inclined to do so. Many of this is still being worked on, but it's likely you'll be able to replace many more "core" components for 1.0 with your own variations without doing any hacking in upstream code at all.
 ... Needless to say, this is an advanced feature that most developers will never need. But Evennia was always intended to be heavily customizable and having the option is a good thing!

Another feature that will come for 1.0 is a REST-API, added by one of our contributors. This uses Django-REST-Framework and makes it easier for external programs to authenticate and fetch data out of the Evennia database (great both for external apps, websites or custom what-have-you).
At this time you can only fetch database objects via this interface, you cannot perform Command-calls or "play the game" this way (REST is a stateless concept and Evennia Commands tend to retain state).

Many other fixes and contributions

There's a truckload of stuff already available in master branch, but with the latest contributions of bigger code changes, we have started to use the Evennia develop branch again in earnest again. For a summary of the changes so far, check out the Changelog. 

However, unless you want to contribute to Evennia itself (or really, really want to be on the bleeding edge), you are still recommended to use the master branch for now. A lot of work still to do, as said.


Image ©George Hodan, released under CC0 Public Domain