Thursday, April 25, 2019

Steaming on, eating jam

Image credit: The Smithsonian
In the last few months, development of the upcoming Evennia 0.9 has been steaming on. Evennia is, as you may know, a Python library for creating text-based multiplayer games (MUDs, MUSH etc).
But it's not all backend work! There is also some sweet game-jamming going on, I get to that at the end. 


Evennia progress towards Python3 

The regular Evennia develop branch is now running completely in Python 3. Since we are using some new features of this Python release, we will be aiming for Python 3.7 as a minimum version once Evennia 0.9 goes stable. We will also use Django 2.1 and likely Twisted 19 - so we'll be pretty much up-to-date on all our main dependencies.

Now, while the release of Evennia 0.9 is still some time away (there are a bunch of regular bug fixes and minor features that I want to get in there too (see the progress here on the github 0.9 project page), it's worth to consider how much work it'll be for you to migrate and if you should wait or jump in right now.

If you are new, I still recommend you use regular master branch Evennia (using Python 2.7). This is for which all wiki articles and documentation online is currently written after all. Once we move to python3, you'll need to convert your code ... but syntactically the two are really not that different and conversion should not be much of an issue.

Not only are there automatic converters for most stuff, you should only need to do one pass to make sure things work and then you'll be done. This article is pretty old but it serves well to identify the main differences. Later Py3 versions just adds new stuff which you would just not have had access to in Python2.7. Once 0.9 is released, we'll also make guides for how you go about converting existing code (apart from the wealth of info on this topic online).
 
That said, if you are feeling more adventurous, or really want to use Python3 right away, many of the Evennia developer regulars have been running and testing develop branch for months now. It's overall been a pretty painless transition - as said, py3 is not that different from py2.

However, develop branch has other features beyond the py3 jump. And those are not yet documented in the main docs. So you'll have to contend with that (but asking in chat/forum works of course). However while it works fine for development already, as a matter of principle I do not recommend ever using Evennia's develop branch for a production game - it can change quickly and may occationally be broken. You have been warned! 

Game Jam

We are now a little more than a week into the Mud Coders Guild's second yearly Game Jam. This year's theme is "One Room".

I really hope some more Evennia devs jump on this one because Evennia is perfect for this kind of experimental games. This is because Evennia is not imposing a game style on you - while there are default commands you are free to replace or customize the commands, objects and every aspect of your game to be as specific as you want to your game.

For example, I have a small Game Jam contribution in the works, where I very quickly reworked the default Evennia setup to pretty much make it into a different genre of game.

Usually the systems I make for Evennia are generic and intended for any game-type. By contrast, making somehing highly niched is super-fast to do: Building a whole new framework for game mechanisms along with build helpers for me to quickly make content took no more than three days.

There will no doubt be some iteration, but I hope to spend the rest of the jam time on content and gameplay. I have some RL things happening in the coming weeks (including work on Evennia proper) but if I can get the time to tie my jam entry together, I'll likely make one or two blog posts about how it was developed and my reasons for making the choices i did. Most likely the code will appear as an Evennia contribution in case people want to do something similar for their own projects.

So, busy days.

Wednesday, January 2, 2019

Into 2019!

A new year has come around and it's time to both look back at the old and onward to the future of Evennia, the Python MUD creation system!

 

 

 

 

 

Last year 

Last year saw the release of Evennia 0.8. This version of Evennia changes some fundamental aspects of the server infrastructure so that the server can truly run in daemon mode as you would expect (no more running it in a GnuScreen session if you want to see logging to the terminal). It also adds the new Online Creation System, which lets builders create and define prototypes using a menu system as well as big improvements in the web client, such as multiple window-panes (allows the user to assign text to different windows to keep their client uncluttered) as well as plenty of fixes and features to help ease life for the Evennia developer. Thanks again to everyone who helped out and contributed to the release of Evennia 0.8!

On a personal note, I spoke about Evennia at PyCon Sweden this December, which was fun. I might put up my talk and make a more detailed blog post about that in the future, but my talk got a surprising amount of attention and positive feedback. Clearly many people have fond memories of MUDs and enjoy seeing they are not only still around but are possible to create in Python!

This year

Now we are steaming ahead towards Evennia 0.9! Already we have had a large number of contributions towards this release. A coming change is a much improved central way to make typeclass information available to a website as well as central ways to do various common operations like creating characters and linking them to Accounts. Much of this functionality is today hidden in the various default commands. In Evennia 0.9 they will be moved into more easily-findable locations as library functions everyone can just trigger without having to copy functionality.

The biggest change for Evennia 0.9 is however that we will now finally move Evennia over to Python 3. This is now possible as all our dependencies are now sufficiently ported. As discussed in the community for a long time, this will be a clean break - we will not offer any mid-way solution of Python2/3 but will drop Python2 support entirely. We will at the same time move to Django 2.+. In Django 0.9 we will however probably not use any special Python3 features yet. It should hopefully not be too difficult for our users to upgrade, but we'll of course publish extensive instructions and help when that time comes.

We will likely support a minimum of Python 3.6 and maybe 3.7.  This work is currently happening in a separate branch develop-py3 which will soonish merge into the current Python2.7-based develop branch and become Evennia 0.9 when it merges into master branch at an unknown time later this year.

There are a slew of other things planned for Evennia 0.9 and you can follow the progress from our project page. If you want to help out you are of course also very welcome. If you are new and are interested in getting your feet wet in helping out with Open-source development, we have a list of good first issues you could jump into.

Onward into the new year!



Fireworks image courtesy: U.S. Air Force photo by Zachary Hada

Thursday, October 4, 2018

Evennia in Hacktoberfest 2018

Like last year, Evennia, the Python MUD creation system, takes part in Hacktoberfest, a yearly event run by Digitalocean in collaboration with GitHub.

The premise is simple: Sign up at their website and then contribute with 5 GitHub pull requests during the month of October. If you do, you'll win a unique T-shirt!

You can help out any OSS project to win, if you want to help out Evennia, I have marked a bunch of suitable issues with the Hacktoberfest label for you to sink your teeth in. 

Code on!

Sunday, September 30, 2018

Evennia 0.8 released

After about a year of work and almost 540 commits from close to 20 contributors, Evennia 0.8 is out! Evennia is a Python game server for creating text-based multiplayer games (MUDs, Mushes, etc) using Django and Twisted.

Some of the upcoming improvements have been covered by previous dev blogs, such as the completely reworked server infrastructure:




as well as the new Online Creation System that allows for editing in-game prototypes using a powerful menu system:



Other improvements are in the web client, which supports split-panes out of the box. The user can split the output area in any number of panes, resize as desired and then assign different types of content to each pane. You can also have panes that absorb "all the rest" or "all" of the content.


There are still some bits which are a bit shaky and there is still much to do with the web client (for example, not that many outgoing messages yet defaults to being tagged in a way that the webclient recognizes). But it's a step forward!

There are many other improvements for developers, such as easier tools for running debuggers and a lot of new utilities and helper functions. The menu-creation tool (EvMenu) has seen a lot of improvements and also sport a few new decorators for quickly creating multi-page menus with full functionality to flip through and select large numbers of options.

The community has also chipped in with a large number of optional contributions for developers to use for their own game, such as a full turn-based combat system (divided into convenient chunks for the dev to pick and choose support for everything from magic and potions to equipment and ranged attacks).  There are also a range of helper functions for creating simpler menus and build commands as well as auditing tools and additions making better use of Django's very mature security features. 

The more detailed list of improvements and changes can be found in the announcement here. As usual, please report any issues to the issue tracker on github. 

Future

In the immediate future, we'll focus on resolving any bugs that may have slipped through the cracks and also resolve some other issues in the pipeline. 

But beyond that, work on Evennia 0.9 will begin. And before you ask - yes Evennia 0.9 is the version where we'll move fully to Python3. Our dependencies have now reached a point where this is possible and there will be no intermediary Python2/3 version. There is no timeline for the 0.9 release but it should hopefully not be too tricky for the community to make the jump when the time comes.


Saturday, August 18, 2018

Inline building in upcoming Evennia 0.8


Evennia, the Python MUD-server game development kit, is slowly creeping closer to its 0.8 release.

In our development branch I've just pushed the first version of the new OLC (OnLine Creator) system. This is a system to allow builders (who may have limited coding knowledge) to customize and spawn new in-game objects more easily without code access. It's started with the olc command in-game. This is a visual system for manipulating Evennia Prototypes.


Briefly on Prototypes

The Prototype is an Evennia concept that has been around a good while. The prototype is a Python dictionary that holds specific keys with values representing properties on a game object. Here's an example of a simple prototype:

{"key": "My house",
 "typeclass": "typeclasses.houses.MyHouse"}

By passing this dict to the spawner, a new object named "My house" will be created. It will be set up with the given typeclass (a 'typeclass' is, in Evennia lingo, a Python class with a database backend). A prototype can specify all aspects of an in-game object - its attributes (like description and other game-specific properties), tags, aliases, location and so on. Prototypes also support inheritance - so you can expand on an existing template without having to add everything fresh every time.

There are two main reasons for the Prototypes existing in Evennia: 
  • They allow you to customize individual objects easier. For example you could have a 'Goblin' base prototype with inheriting prototypes  Goblin Wizard" and "Goblin Chieftain" - all using the same Typeclass, but with different Attributes, equipment etc. 
  • Prototypes can be manipulated and scripted by builders without needing full Python access. This means that while the Typeclasses are designed and supplied by the Python developer, the builders can then use that typeclass to make very different types of object instances in-game.

 

What's new

As said, Prototypes have been around for a good while in Evennia. But in the past they were either manually entered directly as a dict on the command line, or created in code and read from a Python module. The former solution is cumbersome and requires that you know how to build a proper-syntax Python dictionary. The latter requires server code access, making them less useful to builders than they could be.

Note: If you are visually impaired, each image is also a link to a text-only version.

OLC index

In Evennia 0.8, while you can still insert the Prototype as a raw dict, spawn/menu or the new olc command opens a new menu-driven interface.

Select a prototype to load. This will replace any prototype currently being edited! ___________________________________________________________________________________________________  Select with <num>. Other actions: examine <num> | delete <num> Back (index) | Validate prototype | Quit   1: goblin_archer      5: goblin_archwizard                2: goblin_wizard                                          3: goblin                                                 4: archwizard_mixin
 
More importantly, builders can now create, save and load prototypes in the database for themselves and other builders to use. The prototypes can be tagged and searched as a joint resource. Builders can also lock prototypes if others are not to be able to read or use them to spawn things. Developers can still supply module-based "read-only" prototypes (for use as starting points or examples to their Builders, for example).

Found 1 match.   (Warning: creating a prototype will overwrite the current prototype!) ____________________________________________________________________________________  Actions: examine <num> | create prototype from object <num> Back (index) | Quit   1: Griatch(#1)

You can now also use the menu to search for and create a new Prototype based on an existing object (if you have access to do so). This makes it quick to start up a new prototype and tweak it for spawning other similar objects. Of course you could spawn temporary objects without saving the prototype as well.

The Typeclass defines what 'type' of object this is - the actual working code to use.  All spawned objects must have a typeclass. If not given here, the typeclass must be set in one of the prototype's parents.  [No typeclass set] ______________________________________________________________________________________________________________________________________________  Back (prototype-parent) | Forward (key) | Index | Validate prototype | Quit   1: evennia.contrib.tutorial_world.mob.Mob                 7: evennia.contrib.tutorial_world.objects.TutorialObject    2: evennia.contrib.tutorial_world.objects.Climbable       8: evennia.contrib.tutorial_world.objects.Weapon            3: evennia.contrib.tutorial_world.objects.CrumblingWall   9: evennia.contrib.tutorial_world.objects.WeaponRack        4: evennia.contrib.tutorial_world.objects.LightSource     10: evennia.contrib.tutorial_world.rooms.BridgeRoom         5: evennia.contrib.tutorial_world.objects.Obelisk         current: (1/3)                                              6: evennia.contrib.tutorial_world.objects.Readable        next page

Builders will likely not know which typeclasses are available in the code base. There are new a few ways to list them. The menu display makes use of Evennia 0.8's new EvMenu improvements, which allows for automatically creating multi-page listings (see example above).

There is also a new switch to the typeclass command, /list, that will list all available typeclasses outside of the OLC.

 

Protfuncs

Another new feature are Protfuncs. Similarly to how Inlinefuncs allows for calling for the result of a function call inside a text string, Protfuncs allows for calling functions inside a prototype's values. It's given on the form $funcname(arguments)where arguments could themselves contain one or more nested Protfuncs.

As with other such systems in Evennia, only Python functions in a specific module or modules (given by settings) are available for use as Protfuncs in-game. A bunch of default ones are included out of the box. Protfuncs are called at the time of spawning. So for example, you could set the Attribute 
Strength = $randint(5, 20) 
to automatically spawn objects with a random strength between 5 and 20.

prototype-key: goblin, -tags: [], -locks: spawn:all();edit:all() -desc: Built from goblin prototype-parent: None      key: goblin aliases: monster, mob attrs:  desc = You see nothing special.  strength = $randint(5,20)  agility = $random(6,20)  magic = 0 tags:  mob (category: None) locks:  call:true();control:id(1) or perm(Admin);delete:id(1) or perm(Admin);edit:perm(Admin);examine:perm(Builder);get:all();puppet:pperm(Developer) ;tell:perm(Admin);view:all() location: #2 home: #2   No validation errors found. (but errors could still happen at spawn-time) ______________________________________________________________________________________________________________________________________________  Actions: examine <num> | remove <num> Back (index) | Validate prototype | Quit   1: Spawn in prototype's defined location (#2)       2: Spawn in Griatch's location (Limbo)              3: Spawn in Griatch's inventory                     4: Update 2 existing objects with this prototype

When spawning, the olc will validate the prototype and run tests on any Protfunc used. For convenience you can override the spawn-location if any is hard-coded in the prototype.

https://pastebin.com/raw/K0a1z23h

The system will also allow you to try updating existing objects created from the same-named prototype earlier. It will sample the existing objects and calculate a 'diff' to apply. This is bit is still a bit iffy, with edge cases that still needs fixing.

 

Current status

The OLC is currently in the develop branch of Evennia - what will soon(ish) merge to become Evennia 0.8.

It's a pretty big piece of code and as such it's still a bit unstable and there are edge cases and display issues to fix. But it would be great with more people trying it out and reporting errors so the childhood issues can be ironed out before release!




Building Image: Released as Creative Commons here

Saturday, January 27, 2018

Kicking into gear from a distance

The last few weeks I have reworked the way Evennia's startup procedure works. This is now finished in the develop branch so I thought I'd mention a little what's going on.

Evennia, being a server for creating and running text-games (MU*s), consists of two main processes:
  • The Portal - this is what players connect to with their clients.
  • The Server - this is the actual game, with the database etc. This can be shutdown and started again without anyone connected to the Portal getting kicked from the game. This allows for hot-adding new Python code into the running Server without any downtime. 
Since Evennia should be easy to set up and also run easily on Windows as well as on Linux/Mac, we have foregone using the linux process management services but instead offered our own solution. This is how the reload mechanism currently looks in master branch:

Here I've excluded connections irrelevant to reloading, such as the Twisted AMP connection between Portal and Server. Dashed lines suggest a more "temporary" connection than a solid line.

The Launcher is the evennia program one uses to interact with the Server in the terminal/console. You give it commands like evennia start/stop/reload.

  • When starting, the Launcher spawns a new program, the Runner, and then exits. The Runner stays up and starts the Portal and Server. When it starts the Server, it does so in a blocking way and sits waiting in a stalled loop for the Server process to end. As the Server and Portal start they record their current process-ids in .pid files
  • When reloading, the Launcher writes a flag in a little .restart file. The Launcher then looks up the Server's .pid file and sends a SIGINT signal to that process to tell it to gracefully shut down. As the Server process dies, the Runner next looks at the Server's .restart file. If that indicates a reload is desired, The Runner steps in its loop and starts up a new Server process. 
  • When stopping, everything happens like when reloading, except the .restart file tells the Runner that it should just exit the loop and let the Server stay down. The Launcher also looks at the Portal's .pid file and sends a SIGINT signal to kill it. Internally the processes catch the SIGINT and close gracefully.
The original reason for this Server-Portal-Runner setup is that the Portal is also reloadable in the same way (it's not shown above). But over time I've found that having the Portal reloadable is not very useful - since players get disconnected when the Portal reloads one can just as well stop and start both processes. There are also a few issues with the setup, such as the .pid files going stale if the server is killed in some catastrophic way and various issues with reliably sending signals under Windows. Also, the interactive mode works a little strangely since closing the terminal will actually kill the Runner, not the Server/Portal - so they will keep on running except they can no longer reload ...
It overall feels a little ... fiddly.

In develop branch, this is now the new process management setup:


The Portal is now a Twisted AMP server, while the Evennia Server and Launcher are AMP clients. The Runner is no more.

  • When starting, the Launcher spawns the Portal and tries to connect to it as an AMP client as soon as it can. The Portal in turn spawns the Server. When the Server AMP client connects back to the Portal, the Portal reports back to the Launcher over the AMP connection. The Launcher then prints to the user and disconnects. 
  • When reloading, the Launcher connects to the Portal and gives it a reload-command. The Portal then tells the Server (over their AMP connection) to shutdown. Once the Portal sees that the Server has disconnected, it spawns a new Server. Since the Portal itself knows if a reload or shutdown is desired no external .restart (or .pid) files are needed. It reports the status back to the Launcher that can then disconnect.
  • When stopping, the Launcher sends the "Stop Server" command to the Portal. The Portal tells the Server to shut down and when it has done so it reports back to the Launcher that the Server has stopped. The Launcher then sends the "Stop Portal" command to also stop the Portal.  The Launcher waits until the Portal's AMP port dies, at which point it reports the shutdown to the user and stops itself.
So far I really like how this new setup works and while there were some initial issues on Windows (spawning new processes does not quite work they way you expect on that platform) I think this should conceptually be more OS-agnostic than sending kill-signals. 

This solution gives much more control over the processes. It's easy to start/stop the Server behind the portal at will. The Portal knows the Server state and stores the executable-string needed to start the Server. Thus the Server can also itself request to be reloaded by just mimicking the Launcher's instructions.
 The launcher is now only a client connecting to a port, so one difference with this setup is that there is no more 'interactive' mode - that is the Server/Portal will always run as daemons rather than giving log messages directly in the terminal/console. For that reason the Launcher instead has an in-built log-tailing mechanism now. With this the launcher will combine the server/portal logs and print them in real time to easily see errors etc during development.

The merger of the develop branch is still a good bit off, but anyone may try it out already here: https://github.com/evennia/evennia/tree/develop . Report problems to the issue tracker as usual.

Friday, January 5, 2018

New year, new stuff

Happy 2018 everyone! Here's a little summary of the past Evennia year and what is brewing.

(Evennia is a Python server- and toolbox for creating text-based multiplayer games (MU*)).

The biggest challenge for me last year Evennia-wise was the release of Evennia 0.7. Especially designing the migration process for arbitrary users migrating the Django auth-user took a lot of thought to figure out as described in my blog post here. But now 0.7 is released and a few initial minor adjustments could be made after feedback from daring pilot testers. The final process of migrating from 0.6 to 0.7 is, while involved, a step-by-step copy&paste list that has worked fine for most to follow. I've gotten far fewer questions and complains about it than could be expected so that's a good sign.

Working away on the boring but important behind-the-scenes stuff made me less able to keep up with more "mundane" issues and bugs popping up, or with adding new "fun" features to existing code. Luckily the Evennia community has really been thriving this year; It feels like new users pop up in the support channel all the time now. The number of pull requests both fixing issues and offering new features and contribs have really picked up. A bigger part of my time has been spent reviewing Pull Requests this year than any other I think. I would like to take the opportunity to thank everyone contributing, it's really awesome to see others donating their time and energy adding to Evennia. The Hacktoberfest participation was also surprisingly effective in getting people to create PRs - I have a feeling some were just happy to have an "excuse" for getting started to contribute. We should attend that next year too.

One thing we added with 0.7 was a more formal branching structure: Evennia now uses fixed master and develop branches, where master is for bug-fixes and develop is for new features (things that will eventually become evennia 0.8). This is simple but enough for our needs; it also makes it easier to track new from old now that we are actually doing releases.

Now that Twisted is at a point where this is possible for us to do, we also now have a sort-of plan for finally moving Evennia to Python 3. I won't personally be actively working on it until after 0.8 is out though. I don't expect both Evennia 0.8 and 0.9 (which will be pure py3) to get released this year, but we'll see - so far contributors have done all the work on the conversion.

At any rate, this coming year will probably be dominated by catching up on issues and edge cases that are lining our Issue tracker. One side effect of more newcomers is more eyes on the code and finding the creaky-bits. At least for me, most of my Evennia-time will be spent resolving bugs and issues. The fun thing is that unlike previous years this is not only up to me anymore - hopefully others will keep helping to resolve issues/bugs to broaden our bandwidth when it comes to keeping Evennia stable. The faster we can handle the backlog of issues the faster we can focus on new shiny features after all.

Finally, a continued great thank you to those of you contributing to the Patreon. Even small donations have a great encouraging value when working on something as niche as a Python MU* game server in 2018 - thanks a lot!