Existing players used to logging in with their character name and moo password must signup for a website account.
- Slyter 2m
- Burgerwolf 18s PANCAKES
- Veleth 1m
- cata 58s
- Dale 6m
- xXShadowSlayerXx 1m
- BubbleKangaroo 2m
- spungkbubble 26s
- JanekSembilan 24m
- meero619 10s
- SmokePotion 10s Right or wrong, I'm getting high.
- Rillem 1m Make it personal.
- LadyLogic 10m
- Vanashis 4h
- Sivartas 9m
- zxq 39s Blackcastle was no ordinary prison.
- NightHollow 5m
And 28 more hiding and/or disguised
Connect to Sindome @ moo.sindome.org:5555 or just Play Now

More CPU for the MOO

The CPU utilization seems to be hitting 100% from time to time.

Please consider taking advantage of some of the optimizing and tuning resources available via AWS.

Let's get the fastest / most performant CPU available. Last I checked, there were hundreds of extra dollars just sitting in the fund.

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-recommendations.html

https://stratusgrid.com/blog/aws-ec2-benchmarking

It's only in the hundreds? Wow, I'm glad I keep giving. I thought the balance would be higher than that. Where do you check that, Hek?
As far as I'm aware, LambdaMOO/ToastStunt is single threaded, so there's a capacity to how much CPU can efficiently be used. Buying more CPU power on the server won't necessarily help. It might a little bit. I could also be 100% wrong, I've just looked at the schema manuals out of curiosity.

If the MOO is hitting 100% at any time, there's some runaway code somewhere that is causing performance issues, most of the time. Adding more resources would be like putting a bandaid on a gunshot wound.

Batko is correct.
Then it seems like the runaway code should really be looked into, since it sometimes is causing CPU to hit 100% every 15 minutes or so. Not always so bad obviously, but it does get that bad once in a while.
The fact that the MOO IS single threaded is exactly why we should re-size the VM once or twice a year to make sure that it is running on whatever CPU gives the best single threaded performance.

I say once or twice a year so that we can lock in reserved instance pricing.

How many cores does the VM have? The MOO can't be the only thing running there, the operating system and everything else going will also eat a bit of CPU.
Additional CPU and additional memory are a benefit to the MOO due to its single threaded nature, increasing the # of CPUs is not, as the MOO is single threaded. I haven't heard a lot of complaints of lag and such. Is there some persistent issue that I'm unaware of?

As for 100% CPU usage, and runaway code – it's not runaway code causing the CPU to hit 100%. It's many, many, many complex actions being taken at the same time. Combat, for example, is complex, and requires a lot of different code and execution to happen concurrently (or fake concurrently due to being single threaded).

For example, when the autopruner runs, it requires doing a lot of heavy lifting at the same time, and some of that is blocking at the C (programming language of the server) level. The MOO has builtin functions that make up the MOO code itself, which are implemented in C, and these functions take up CPU but do not deplete 'ticks' that verbs run with. A verb can run out of ticks if it tries to do too much, but executing a single builtin function only takes 1 tick, but that builtin function, might do a huge amount of work.

For instance running this built-in 'sort()' function on a giant list, will take 1 tick, and may take 3 seconds and during those 3 seconds all other reads/writes are blocked because the MOO being single threaded is doing ONLY that one task.

Most of the time we avoid certain built-in tasks and instead rely on implementations of things like 'sort' that are written in MOO code itself, which may at times be slightly sub optimal from an execution time standpoint, but it keeps all the execution within the MOO and thus, using ticks, and pauses/suspends which means the verbs will periodically pause to let other verbs from other system functions of players process, instead of blocking everyone until they are done.

Also, technically, ToastStunt is mutli-threaded, in that we can delegate certain tasks or builtins to run in a separate thread, but this functionality isn't used often as it can cause unintended consequences such as race conditions. We use it sparingly.

At any rate, if the CPU is spiking, it usually means a bunch of stuff is happening and there is a bottleneck. I'm interested to hear more about when this is happening and if there are specific things (combat, etc) happening at the same time that people are noticing, as that would give me a good place to look at optimizing code.

Lag has seemed to be a pretty significant issue recently. There are times when the CPU will hit 100% every 15-30 minutes or so.

I know it's not particularly useful but here's a picture of the CPU usage I got on January 3rd, which was a particularly laggy day.

There was a time when any time gang fights would happen and NPCs responded, the CPU would pretty much instantly hit 100%. More recently I've seen it hitting 100% for a while when there is no gang fights, so while I suspect NPC fights is one issue, I don't think it's the only one.

Not strictly related to the server collapsing into the void right now, but since the point was raised here I would also describe overall performance and responsiveness as getting worse and worse over the last year or two and many of the pre-Toast Stunt problems like commands during critical moments getting lost.

There were events over the holidays that had a handful of players involved with them during overall low activity and simple things like movement and sneaking and basic interactions were just never registering for me often enough to give up on it. After the Toast Stunt upgrade everything was extremely fast and precise and felt so responsive so this does appear to be something that changed over time, if slowly.

I don't know if it's NPC scripts run amok or newer features weighing everything down but I for one would happily trade away complex NPC interactions or other high overhead systems for command responsiveness which is basically the most important feature of a competitive game in my opinion.

Right now it just seems to be out of memory - at least if the little graph in the webclient is correct. Swapping causes pretty extreme performance issues.
As I mentioned, I mean this separately from the server breaking now, it just happened to give me time to add to the discussion that the game's performance has been degraded for some reason for some months at least.
Kinda related to what 0x1mm said, I've been finding a lot of the time I open a door, and then try to go through it, only to be told the door is closed. Like a full second after getting the "the door leading x swings open" message. I'm assuming it's sending the "door opened" message before actually setting the door to open, or something, and just getting tangled up in the One Thread?
I haven't gotten any commands disappearing and stuff like that. But I have noticed more lag than usual. In the past, it'd only be bad lag when the little cloud turned orange, but now I get bad lag sometimes even when it's green. I sound like a damn boomer right now but, it does feel like lag's become more frequent the past couple of months.
I've had the same issue so it's not just you, also lately (maybe related) using a look command will return next room before I've actually moved to it.

Other syncing issues are macro commands firing out of order, something that only seemed to happen when the game was really lagging but now seems to be fairly often. This was a super common problem before the ToastStunt upgrade I seem to recall, lots of players had automatic door closing macros because how unreliable and slow moving into another room could be, that briefly seemed to be a thing of the past but it does seem like its trending back in that direction.

That does remind me though, sometimes when I did 'ca' right before entering a new room, it'd show me my alias if the next room has signal though the one I shouldn't have left yet doesn't.
Re: New Game Features and Improvements: Instance Upgrade.

I didn't realize there was a known memory leak, sounds like increasing the server's memory and eventually patching it out could address the issues I've noticed so that's so great to hear!

(Edited by Mench at 6:59 pm on 2/1/2025)

That link didn't work for me.
Link should work now