[Opensim-users] How to get ROBUST to notify that it has finished setup?

[Gwyneth Llewelyn]

Hi all,

I've been tinkering with my automation scripts under Ubuntu Linux
20.04.3 LTS, trying to get them fully integrated with systemd. It's
tougher than I imagined!

My question is rather simple. OpenSim.ini lists a few options to run
some scripts and/or send some notifications when the instance is fully
loaded and operational (for instance, once the instance is fully
loaded, you could check for the statistics API. These can be used for
a variety of purposes, from simple notifications to a sysadmin to let
them know that an instance has rebooted, to let users get some sort of
feedback on which regions are up, etc. and so forth. These can also be
used for system maintenance purposes as well.

I can't find anything similar for ROBUST, though — at least, not on
the configuration files. The closest I could find was a reference to
the 'console'. I'm assuming that this would technically allow a bash
script to connect to ROBUST and perform some sort of check...? A bit,
uh, 'clunky' but... I guess it's a possibility?

What are you using to signal that ROBUST has finished loading?

Thanks in advance!

   - Gwyn

P. S. Some background notes, for those interested in understanding
what I'm trying to accomplish and why I've been having some trouble.
One of the great things about systemd (arguably one of the few...) is
that it launches everything in parallel, as much as possible; the
theory being that services will not need to block each other, which is
what happened in early systems (which relied on a serial sequence of
steps, each having to finish before the next one was launched).

This is great for launching all the OpenSim instances for the whole
grid — they will load in parallel, and, since they're pretty much
self-contained, they will happily get what they need from the database
server, and — in theory! — finish faster than launching each instance,
one by one (in practice, it's not so rosy, since the database server
becomes the bottleneck... although it ought to be possible to
fine-tune it to deal with so many requests in parallel).

However, there are two catches with this approach.

Firstly, if the MySQL database is not ready before ROBUST and/or the
instances launch, OpenSim will assume a 'broken' or non-existing
database connection, and gracefully fail, by asking for the Estate
name and so forth — i.e. basically the instances will be up, but
blocked. The good news is that there are several ways to check that
MySQL is up and running (using some external scripts — ), so this can
be checked before ROBUST or any of the OpenSim instances are launched.

Secondly — and the reason for this message to the list! —  _if_ ROBUST
hasn't launched yet, then none of the OpenSim instances will register
themselves with the core grid services (including the asset server).
I'm not quite sure if each instance, after failing their attempts in
contacting ROBUST, will do any attempt at a later stage to re-check-in
with it. If not, it effectively means a broken grid, where sections of
it, on individual instances, will simply be isolated from the rest of
the grid.

ROBUST is quite fast in loading everything — compared with the OpenSim
instances, at least — which means that there is a good chance that it
launches before the instances. But we cannot be sure that this
actually happens.

Now, systemd has a way to generate a list (rather, a directed
graph...) of dependencies. One can, indeed, make sure that ROBUST has
already been launched *before* launching any of the instances. But
this won't help much in this case, because systemd is only able to
check that the *process* has been launched — not if it's ready to
accept requests. There are some tricks to achieve that, but most
require some changes in the ROBUST code, and I'm not even sure that,
running inside Mono, the C# code has any access to system calls. The
alternative is to use scripts that check for other things — such as,
say, a status page or a file that has been written somewhere — in
order to deduce that something has not only been launched but is
actively accepting requests. I know how to do that inside an OpenSim
instance, but not on ROBUST.

--
"I'm not building a game. I'm building a new country."
-- Philip "Linden" Rosedale, interview to Wired, 2004-05-08