[Opensim-dev] multiple, empty, root inventory folders

[Kyle Hamilton]

On Mon, Apr 21, 2008 at 8:34 AM, Justin Clark-Casey
<jjustincc at googlemail.com> wrote:
>
>  Multiple roots were the result of various code bugs.  I actually hope
>  now that we have these nailed, and that all the current multiple
>  inventories are now historical stock.
>
>  If this doesn't prove to be the case, I think it would be worth adding a
>  multiple inventory check on inventory service startup.  I'm loathe to
>  add anything more than this due to the code and startup cost, which
>  would only be going to counter a situation which should in theory never
>  occur.

One of the most important things in any service structure is the
ability to gracefully degrade in the presence of constraint
assumptions which are not met by the data set.  The constraint here is
"there must be only one root inventory for each avatar/account"; in
OSgrid's case, that constraint is currently violated.

I appreciate that you think/hope that you've got these bugs squashed.
But, it's important to recognize that the constraints can easily
become violated through nobody's fault... especially in a
constantly-changing codebase.  Since the inventory service is
restarted only rarely, any instances of root inventory
duplication/inventory-db constraint violation will in the case of
startup-check only be determined at the time of restart.

It's extremely easy to envision that startup is the worst possible
time for these constraint violations to be detected (i.e., the
inventory server has crashed and, during restart, duplicates are found
that cause the startup to abort).

Instead of once-per-run-at-startup, I'd suggest a timer throw to do a
kind of read-only dbck (i.e., scan just to make sure that basic
constraints are met) once an hour or so, eventually moving it to once
every 24 hours.  Since it can be done within a transaction, it can be
guaranteed that the state of the database as it is seen by this check
is consistent, and it would provide for an early-warning system to
show when things start to degrade and the constraints start to be
broken.  (so that maintenance can be performed at that time, instead
of in an urgent situation.)

I've been called at 3am -- and called others at 3am -- too many times
(due to unexpected sanity violations on databases and poor handling of
it by codebases).  I personally don't like unplanned failure modes.
I'd like to see some sort of degradation strategy built into the
infrastructure to help provide a highly-available system.

-Kyle H