The start of the October event has seen a burst of activity which led to some very laggy servers last night, which I'm sure most of you encountered.

Last night we called an emergency admin meeting and spent some time consulting with server experts to try and figure out what the best way to stamp the lag out was. We've identified a number of bottlenecks which we think are strangling the servers in places when we have a high number of users like this all on at one time, and we're putting a plan of action in place to try and iron those out as soon as possible.

One of the most frequently asked questions is, "Why can't you just add another server?" While in the short term this may seem like the best solution, it is a bit more involved than that.

There are essentially two pieces to Lioden — what we refer to as the "frontend" and the "backend". The frontend is the portion of the site that handles running all of the scripts and render out the pages to your browsers. This is the easiest area to add more servers to. Unfortunately it is the least effective, and perhaps even detrimental, in our current situation.

Lioden's code base has grown monumentally since its inception 5 years ago. During this time, much of the core of the code has remained the same. Although this worked perfectly when we peaked at a couple hundred users, scaling up to a couple thousand is another feat entirely.

Over time we have slowly been updating pieces of this foundation (user caching, rollover procedure, energy system, etc.). These are no trivial feats that, while important, can also introduce new problems into the system.

Above it was mentioned that there were two pieces, the other being the "backend". While servers can be added to this portion of the site, it is unfortunately not as straight forward as adding them to the frontend.

This is why our first focus is on addressing legacy code to increase efficiency and add optimization. With that being said, we are looking at extending the backend with an additional server as well. This, however, will take more time and careful planning.

We're beginning with search, and implementing a new piece of software that will really optimise and speed that up, as individual searches can cause some unfortunate server pain. Alongside that our next step is to tweak and improve how inventory is handled, as this too can be a hefty weight on the servers.

This will all be going on in the background so hopefully you shouldn't notice any downtime, though if there are to be any we'll let you know as far in advance as possible. We appreciate your patience throughout all of this and just want to let you guys know how awesome you are!

Thanks for sticking by us!