[Opensim-users] Detecting full robust startup, one option
Sara Payne
sarapayne.uk at gmail.com
Thu Nov 18 12:28:29 UTC 2021
One way you can manage this is by handling the ConsoleHistoryFile. I am
assuming in the example below you have the main log file and then console
history giving limited 2nd logging; additionally, your shutdown script will
remove the console history file from the last session.
Firstly, Robust services give consistent outputs. Therefore it is possible
to set the number of lines in the console history file to the expected
output length. You can then check the number of lines in the file using a
bash script against the predicted size. If the number of lines matches
the expected number of lines, then do a deeper check.
Check the last few lines to make sure they match the expected output.
If the console history file is the correct length and the last lines of the
log match what you expect, then robust has fully started.
It is possible to approach long tasks like creating OAR and IAR files in
the same way.
Sara
On Thu, Nov 18, 2021 at 12:00 PM <opensim-users-request at opensimulator.org>
wrote:
> Send Opensim-users mailing list submissions to
> opensim-users at opensimulator.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
> or, via email, send a message with subject or body 'help' to
> opensim-users-request at opensimulator.org
>
> You can reach the person managing the list at
> opensim-users-owner at opensimulator.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Opensim-users digest..."
>
>
> Today's Topics:
>
> 1. How to get ROBUST to notify that it has finished setup?
> (Gwyneth Llewelyn)
> 2. Re: How to get ROBUST to notify that it has finished setup?
> (Leal Duarte)
> 3. Re: How to get ROBUST to notify that it has finished setup?
> (Dahlia Trimble)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 17 Nov 2021 21:03:58 +0000
> From: Gwyneth Llewelyn <gwyneth.llewelyn at gwynethllewelyn.net>
> To: opensim-users at opensimulator.org
> Subject: [Opensim-users] How to get ROBUST to notify that it has
> finished setup?
> Message-ID:
> <CAME74xE89amAV-edp9Lw3i2sYSMnwt2T+QXBm9r=
> cVpSXAYYqg at mail.gmail.com>
> Content-Type: text/plain; charset="UTF-8"
>
> Hi all,
>
> I've been tinkering with my automation scripts under Ubuntu Linux
> 20.04.3 LTS, trying to get them fully integrated with systemd. It's
> tougher than I imagined!
>
> My question is rather simple. OpenSim.ini lists a few options to run
> some scripts and/or send some notifications when the instance is fully
> loaded and operational (for instance, once the instance is fully
> loaded, you could check for the statistics API. These can be used for
> a variety of purposes, from simple notifications to a sysadmin to let
> them know that an instance has rebooted, to let users get some sort of
> feedback on which regions are up, etc. and so forth. These can also be
> used for system maintenance purposes as well.
>
> I can't find anything similar for ROBUST, though ? at least, not on
> the configuration files. The closest I could find was a reference to
> the 'console'. I'm assuming that this would technically allow a bash
> script to connect to ROBUST and perform some sort of check...? A bit,
> uh, 'clunky' but... I guess it's a possibility?
>
> What are you using to signal that ROBUST has finished loading?
>
> Thanks in advance!
>
> - Gwyn
>
> P. S. Some background notes, for those interested in understanding
> what I'm trying to accomplish and why I've been having some trouble.
> One of the great things about systemd (arguably one of the few...) is
> that it launches everything in parallel, as much as possible; the
> theory being that services will not need to block each other, which is
> what happened in early systems (which relied on a serial sequence of
> steps, each having to finish before the next one was launched).
>
> This is great for launching all the OpenSim instances for the whole
> grid ? they will load in parallel, and, since they're pretty much
> self-contained, they will happily get what they need from the database
> server, and ? in theory! ? finish faster than launching each instance,
> one by one (in practice, it's not so rosy, since the database server
> becomes the bottleneck... although it ought to be possible to
> fine-tune it to deal with so many requests in parallel).
>
> However, there are two catches with this approach.
>
> Firstly, if the MySQL database is not ready before ROBUST and/or the
> instances launch, OpenSim will assume a 'broken' or non-existing
> database connection, and gracefully fail, by asking for the Estate
> name and so forth ? i.e. basically the instances will be up, but
> blocked. The good news is that there are several ways to check that
> MySQL is up and running (using some external scripts ? ), so this can
> be checked before ROBUST or any of the OpenSim instances are launched.
>
> Secondly ? and the reason for this message to the list! ? _if_ ROBUST
> hasn't launched yet, then none of the OpenSim instances will register
> themselves with the core grid services (including the asset server).
> I'm not quite sure if each instance, after failing their attempts in
> contacting ROBUST, will do any attempt at a later stage to re-check-in
> with it. If not, it effectively means a broken grid, where sections of
> it, on individual instances, will simply be isolated from the rest of
> the grid.
>
> ROBUST is quite fast in loading everything ? compared with the OpenSim
> instances, at least ? which means that there is a good chance that it
> launches before the instances. But we cannot be sure that this
> actually happens.
>
> Now, systemd has a way to generate a list (rather, a directed
> graph...) of dependencies. One can, indeed, make sure that ROBUST has
> already been launched *before* launching any of the instances. But
> this won't help much in this case, because systemd is only able to
> check that the *process* has been launched ? not if it's ready to
> accept requests. There are some tricks to achieve that, but most
> require some changes in the ROBUST code, and I'm not even sure that,
> running inside Mono, the C# code has any access to system calls. The
> alternative is to use scripts that check for other things ? such as,
> say, a status page or a file that has been written somewhere ? in
> order to deduce that something has not only been launched but is
> actively accepting requests. I know how to do that inside an OpenSim
> instance, but not on ROBUST.
>
> --
> "I'm not building a game. I'm building a new country."
> -- Philip "Linden" Rosedale, interview to Wired, 2004-05-08
>
>
> ------------------------------
>
> Message: 2
> Date: Wed, 17 Nov 2021 21:25:08 +0000
> From: Leal Duarte <ajlduarte at sapo.pt>
> To: opensim-users at opensimulator.org
> Subject: Re: [Opensim-users] How to get ROBUST to notify that it has
> finished setup?
> Message-ID: <adcdcc6f-cb62-6fc9-9f22-fdf6a2d872b5 at sapo.pt>
> Content-Type: text/plain; charset=UTF-8; format=flowed
>
> Hi,
>
> ??? in absolute terms, It is just not possible to know when robust is
> fully loaded, due to opensim heavy multi tasked nature.
>
> ??? In some configurations it can even be split into several processes
> even on different machines.
>
> ??? Even Regions.
>
> ??? What we have is just is end of scripts loading. When that ends
> physics engine may still be processing meshes, for example
>
> Ubit
>
>
> On 17-Nov-21 21:03, Gwyneth Llewelyn wrote:
> > Hi all,
> >
> > I've been tinkering with my automation scripts under Ubuntu Linux
> > 20.04.3 LTS, trying to get them fully integrated with systemd. It's
> > tougher than I imagined!
> >
> > My question is rather simple. OpenSim.ini lists a few options to run
> > some scripts and/or send some notifications when the instance is fully
> > loaded and operational (for instance, once the instance is fully
> > loaded, you could check for the statistics API. These can be used for
> > a variety of purposes, from simple notifications to a sysadmin to let
> > them know that an instance has rebooted, to let users get some sort of
> > feedback on which regions are up, etc. and so forth. These can also be
> > used for system maintenance purposes as well.
> >
> > I can't find anything similar for ROBUST, though ? at least, not on
> > the configuration files. The closest I could find was a reference to
> > the 'console'. I'm assuming that this would technically allow a bash
> > script to connect to ROBUST and perform some sort of check...? A bit,
> > uh, 'clunky' but... I guess it's a possibility?
> >
> > What are you using to signal that ROBUST has finished loading?
> >
> > Thanks in advance!
> >
> > - Gwyn
> >
> > P. S. Some background notes, for those interested in understanding
> > what I'm trying to accomplish and why I've been having some trouble.
> > One of the great things about systemd (arguably one of the few...) is
> > that it launches everything in parallel, as much as possible; the
> > theory being that services will not need to block each other, which is
> > what happened in early systems (which relied on a serial sequence of
> > steps, each having to finish before the next one was launched).
> >
> > This is great for launching all the OpenSim instances for the whole
> > grid ? they will load in parallel, and, since they're pretty much
> > self-contained, they will happily get what they need from the database
> > server, and ? in theory! ? finish faster than launching each instance,
> > one by one (in practice, it's not so rosy, since the database server
> > becomes the bottleneck... although it ought to be possible to
> > fine-tune it to deal with so many requests in parallel).
> >
> > However, there are two catches with this approach.
> >
> > Firstly, if the MySQL database is not ready before ROBUST and/or the
> > instances launch, OpenSim will assume a 'broken' or non-existing
> > database connection, and gracefully fail, by asking for the Estate
> > name and so forth ? i.e. basically the instances will be up, but
> > blocked. The good news is that there are several ways to check that
> > MySQL is up and running (using some external scripts ? ), so this can
> > be checked before ROBUST or any of the OpenSim instances are launched.
> >
> > Secondly ? and the reason for this message to the list! ? _if_ ROBUST
> > hasn't launched yet, then none of the OpenSim instances will register
> > themselves with the core grid services (including the asset server).
> > I'm not quite sure if each instance, after failing their attempts in
> > contacting ROBUST, will do any attempt at a later stage to re-check-in
> > with it. If not, it effectively means a broken grid, where sections of
> > it, on individual instances, will simply be isolated from the rest of
> > the grid.
> >
> > ROBUST is quite fast in loading everything ? compared with the OpenSim
> > instances, at least ? which means that there is a good chance that it
> > launches before the instances. But we cannot be sure that this
> > actually happens.
> >
> > Now, systemd has a way to generate a list (rather, a directed
> > graph...) of dependencies. One can, indeed, make sure that ROBUST has
> > already been launched *before* launching any of the instances. But
> > this won't help much in this case, because systemd is only able to
> > check that the *process* has been launched ? not if it's ready to
> > accept requests. There are some tricks to achieve that, but most
> > require some changes in the ROBUST code, and I'm not even sure that,
> > running inside Mono, the C# code has any access to system calls. The
> > alternative is to use scripts that check for other things ? such as,
> > say, a status page or a file that has been written somewhere ? in
> > order to deduce that something has not only been launched but is
> > actively accepting requests. I know how to do that inside an OpenSim
> > instance, but not on ROBUST.
> >
> > --
> > "I'm not building a game. I'm building a new country."
> > -- Philip "Linden" Rosedale, interview to Wired, 2004-05-08
> > _______________________________________________
> > Opensim-users mailing list
> > Opensim-users at opensimulator.org
> > http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>
>
> ------------------------------
>
> Message: 3
> Date: Wed, 17 Nov 2021 20:19:01 -0800
> From: Dahlia Trimble <dahliatrimble at gmail.com>
> To: opensim-users at opensimulator.org
> Subject: Re: [Opensim-users] How to get ROBUST to notify that it has
> finished setup?
> Message-ID:
> <
> CAAQTD4VuTQKFOEYwixcAS9Dfrq-Q7xxZpOCTCN0_LT_8FNwzpQ at mail.gmail.com>
> Content-Type: text/plain; charset="UTF-8"
>
> You might be able to detect it by having a bash script check to see if the
> associated Robust sockets are active. I'm not sure if it means Robust is
> finished it's startup but it's probably at least pretty close.
>
> On Wed, Nov 17, 2021 at 1:25 PM Leal Duarte <ajlduarte at sapo.pt> wrote:
>
> > Hi,
> >
> > in absolute terms, It is just not possible to know when robust is
> > fully loaded, due to opensim heavy multi tasked nature.
> >
> > In some configurations it can even be split into several processes
> > even on different machines.
> >
> > Even Regions.
> >
> > What we have is just is end of scripts loading. When that ends
> > physics engine may still be processing meshes, for example
> >
> > Ubit
> >
> >
> > On 17-Nov-21 21:03, Gwyneth Llewelyn wrote:
> > > Hi all,
> > >
> > > I've been tinkering with my automation scripts under Ubuntu Linux
> > > 20.04.3 LTS, trying to get them fully integrated with systemd. It's
> > > tougher than I imagined!
> > >
> > > My question is rather simple. OpenSim.ini lists a few options to run
> > > some scripts and/or send some notifications when the instance is fully
> > > loaded and operational (for instance, once the instance is fully
> > > loaded, you could check for the statistics API. These can be used for
> > > a variety of purposes, from simple notifications to a sysadmin to let
> > > them know that an instance has rebooted, to let users get some sort of
> > > feedback on which regions are up, etc. and so forth. These can also be
> > > used for system maintenance purposes as well.
> > >
> > > I can't find anything similar for ROBUST, though ? at least, not on
> > > the configuration files. The closest I could find was a reference to
> > > the 'console'. I'm assuming that this would technically allow a bash
> > > script to connect to ROBUST and perform some sort of check...? A bit,
> > > uh, 'clunky' but... I guess it's a possibility?
> > >
> > > What are you using to signal that ROBUST has finished loading?
> > >
> > > Thanks in advance!
> > >
> > > - Gwyn
> > >
> > > P. S. Some background notes, for those interested in understanding
> > > what I'm trying to accomplish and why I've been having some trouble.
> > > One of the great things about systemd (arguably one of the few...) is
> > > that it launches everything in parallel, as much as possible; the
> > > theory being that services will not need to block each other, which is
> > > what happened in early systems (which relied on a serial sequence of
> > > steps, each having to finish before the next one was launched).
> > >
> > > This is great for launching all the OpenSim instances for the whole
> > > grid ? they will load in parallel, and, since they're pretty much
> > > self-contained, they will happily get what they need from the database
> > > server, and ? in theory! ? finish faster than launching each instance,
> > > one by one (in practice, it's not so rosy, since the database server
> > > becomes the bottleneck... although it ought to be possible to
> > > fine-tune it to deal with so many requests in parallel).
> > >
> > > However, there are two catches with this approach.
> > >
> > > Firstly, if the MySQL database is not ready before ROBUST and/or the
> > > instances launch, OpenSim will assume a 'broken' or non-existing
> > > database connection, and gracefully fail, by asking for the Estate
> > > name and so forth ? i.e. basically the instances will be up, but
> > > blocked. The good news is that there are several ways to check that
> > > MySQL is up and running (using some external scripts ? ), so this can
> > > be checked before ROBUST or any of the OpenSim instances are launched.
> > >
> > > Secondly ? and the reason for this message to the list! ? _if_ ROBUST
> > > hasn't launched yet, then none of the OpenSim instances will register
> > > themselves with the core grid services (including the asset server).
> > > I'm not quite sure if each instance, after failing their attempts in
> > > contacting ROBUST, will do any attempt at a later stage to re-check-in
> > > with it. If not, it effectively means a broken grid, where sections of
> > > it, on individual instances, will simply be isolated from the rest of
> > > the grid.
> > >
> > > ROBUST is quite fast in loading everything ? compared with the OpenSim
> > > instances, at least ? which means that there is a good chance that it
> > > launches before the instances. But we cannot be sure that this
> > > actually happens.
> > >
> > > Now, systemd has a way to generate a list (rather, a directed
> > > graph...) of dependencies. One can, indeed, make sure that ROBUST has
> > > already been launched *before* launching any of the instances. But
> > > this won't help much in this case, because systemd is only able to
> > > check that the *process* has been launched ? not if it's ready to
> > > accept requests. There are some tricks to achieve that, but most
> > > require some changes in the ROBUST code, and I'm not even sure that,
> > > running inside Mono, the C# code has any access to system calls. The
> > > alternative is to use scripts that check for other things ? such as,
> > > say, a status page or a file that has been written somewhere ? in
> > > order to deduce that something has not only been launched but is
> > > actively accepting requests. I know how to do that inside an OpenSim
> > > instance, but not on ROBUST.
> > >
> > > --
> > > "I'm not building a game. I'm building a new country."
> > > -- Philip "Linden" Rosedale, interview to Wired, 2004-05-08
> > > _______________________________________________
> > > Opensim-users mailing list
> > > Opensim-users at opensimulator.org
> > > http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
> > _______________________________________________
> > Opensim-users mailing list
> > Opensim-users at opensimulator.org
> > http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
> >
>
>
> ------------------------------
>
> _______________________________________________
> Opensim-users mailing list
> Opensim-users at opensimulator.org
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>
>
> End of Opensim-users Digest, Vol 89, Issue 1
> ********************************************
>
More information about the Opensim-users
mailing list