[Opensim-dev] Inventory loss

Tue Jul 28 05:38:32 UTC 2009

I don't recall OSgrid really getting many complaints about it either, infact
i do not recall any, I am not saying its never occured, but its so minimal
its not something i ever remember occuring, even when our whole asset table
corrupted, we were able to get 100% recovery, i think we lost 1 asset. We
run on MySQL, but run fragstore for blob storage, any grids still storing
blobs inside the database is going to be in major trouble if they don't
switch, storing blobs inside of any of the currently available database
systems OpenSim's core is really a terrible idea and should not be done for
anything that is considered a production level type grid.

Neb

On Mon, Jul 27, 2009 at 6:39 AM, Chris Hart <Chris at codetorque.co.uk> wrote:

> From MSSQL side of the grid fence, I haven't had anyone complain of lost
> inventory to me - we did until recently have some lost assets after SI
> sessions due to asset server crashing - previously the code would attempt to
> store assets with descriptions bigger than database field length. This
> seemed to happen mostly with SI uploaded items but also was 100%
> reproducible if you had a sim with a long name and a parcel with a long name
> and attempted to create a landmark - there are checks now to stop this from
> happening that Justin kindly committed (and he was good enough to add
> similar checks for MySQL) and we've not had a crash since applying that
> patch over a week and a half ago. In terms of actual inventory, no-one has
> reported loss of specific items. Sometimes slow to load, but not loss.
>
> Is this something that has happened since 0.6.6 tag?
>
> Chris
>
> -----Original Message-----
> From: opensim-dev-bounces at lists.berlios.de [mailto:
> opensim-dev-bounces at lists.berlios.de] On Behalf Of Melanie
> Sent: 27 July 2009 14:32
> To: opensim-dev at lists.berlios.de
> Subject: Re: [Opensim-dev] Inventory loss
>
> Those MySQL issues listed below and others like it are corner cases
> that OpenSim would never trigger. OpenSim is a very simplistic
> database user, it doesn't even really use any relational features of
> the database.
> So, I don't think that MySQL errors are something we need to concern
> ourselves with at this point. Especially, your suggestion would not
> help in those cases, as they are all about complete loss of rows.
> That is unaffected by a "deleted" type flag.
>
> That is neither here nor there anyway, since such a "deleted" flag
> would have to live under OpenSim.Data.XXXX.
>
> It has no business being in core at all.
>
> Melanie
>
>
> Colin B. Withers wrote:
> > Reason I wondered is simply the number of websites dedicated to
> recovering from MYSQL database corruption, and offering tools for the same.
> Google returns over 250,000 sites for 'mysql database corruption'.
> >
> > MYSQL has had numerous bug fixes to fix database corruption, introduced
> by differing versions of MYSQL, eg
> >
> > Fixed in 4.0.18
> > INSERT DELAYED ... SELECT ... could cause table corruption because tables
> were not locked properly. This is now fixed by ignoring DELAYED in this
> context. (Bug #1983)
> >
> > Fixed in 4.0.16
> > Fixed bug in overrun check for BLOB values with compressed tables. This
> was a bug introduced in 4.0.14. It caused MySQL to regard some correct
> tables containing BLOB values as corrupted. (Bug #770, Bug #1304, and maybe
> Bug #1295)
> >
> > Fixed in 4.0.15
> > Fixed rare bug in MYISAM introduced in 4.0.3 where the index file header
> was not updated directly after an UPDATE of split dynamic rows. The symptom
> was that the table had a corrupted delete-link if mysqld was shut down or
> the table was checked directly after the update.
> >
> > Fixed in 4.0.14
> > Comparison/sorting for latin1_de character set was rewritten. The old
> algorithm could not handle cases like "sä" < "ßa". See section 5.6.1.1
> German character set. In rare cases, it resulted in table corruption.
> >
> > and so on..
> >
> > My own experience with Opensim is that in the last 12 months there has
> been three occasions where residents complained of stuff 'missing' (not a
> single resident, but several at once) and a roll-back to the last database
> backup cured the problems.
> >
> > Rock
> >
> > -----Original Message-----
> > From: opensim-dev-bounces at lists.berlios.de [mailto:
> opensim-dev-bounces at lists.berlios.de] On Behalf Of Melanie
> > Sent: Monday, July 27, 2009 2:07 PM
> > To: opensim-dev at lists.berlios.de
> > Subject: Re: [Opensim-dev] Inventory loss
> >
> > IMHO, inventory loss due to MySQL errors and/or corruption are below
> > the radar.
> > Any losses would occur in OpenSim code, I believe.
> >
> > Melanie
> >
> > Colin B. Withers wrote:
> >> I wonder what proportion of inventory items that go astray are the
> result of the success/failure of an operation, or are due to database
> corruption issues.
> >>
> >> Rock
> >>
> >> -----Original Message-----
> >> From: opensim-dev-bounces at lists.berlios.de [mailto:
> opensim-dev-bounces at lists.berlios.de] On Behalf Of Melanie
> >> Sent: Monday, July 27, 2009 12:30 PM
> >> To: opensim-dev at lists.berlios.de
> >> Subject: Re: [Opensim-dev] Inventory loss
> >>
> >> There is a question here of why inventory loss occurs at all. At
> >> what stage do we actually lose (as opposed to failing to bump the
> >> folder serial) inventory items at all, and why?
> >>
> >> While a "deleted" flag and an undelete function do make an admin's
> >> life easier, I believe the real focus should be on the inventory
> >> code. It will be redesigned anyway and once that happens, I think a
> >> strong focus needs to be placed on data integrity preservation.
> >>
> >> That would then mae the undelete functionality largely unnecessary.
> >> Current inventory code often doesn't check for success of an
> >> operation at all. That needs to be revisited.
> >>
> >> Melanie
> >>
> >> Thomas Grimshaw wrote:
> >>> Hey folks.
> >>>
> >>> Been thinking a lot about inventory loss in OpenSim, something that I
> >>> think we should really do as much as possible to avoid. We've been
> >>> experiencing numerous cases of lost inventory in K-Grid recently.
> >>>
> >>> What i'd like to implement, is..
> >>>
> >>> When an item is removed from inventory (deleted, or rezzed if it's
> >>> no-copy), it is not actually deleted by instead an "available" flag is
> >>> set in the inventory database.
> >>> All inventory queries will check for the flag and thus it will appear
> as
> >>> deleted to the user, but it can be restored easily by an admin if
> >>> needed.  A timestamp should also be set which indicates when the item
> >>> was made unavailable, so that routine cleanup can be performed on items
> >>> which were made unavailable a long time ago.
> >>>
> >>> I wanted to get people's opinons of this before I implemented it in
> >>> code. Can anyone think of any drawbacks or possible issues? Any further
> >>> room for improvement?
> >>>
> >>> Cheers
> >>>
> >>> Thomas Grimshaw
> >>> (RemedyTomm)
> >>> _______________________________________________
> >>> Opensim-dev mailing list
> >>> Opensim-dev at lists.berlios.de
> >>> https://lists.berlios.de/mailman/listinfo/opensim-dev
> >>>
> >>>
> >> _______________________________________________
> >> Opensim-dev mailing list
> >> Opensim-dev at lists.berlios.de
> >> https://lists.berlios.de/mailman/listinfo/opensim-dev
> >> _______________________________________________
> >> Opensim-dev mailing list
> >> Opensim-dev at lists.berlios.de
> >> https://lists.berlios.de/mailman/listinfo/opensim-dev
> >>
> >>
> > _______________________________________________
> > Opensim-dev mailing list
> > Opensim-dev at lists.berlios.de
> > https://lists.berlios.de/mailman/listinfo/opensim-dev
> > _______________________________________________
> > Opensim-dev mailing list
> > Opensim-dev at lists.berlios.de
> > https://lists.berlios.de/mailman/listinfo/opensim-dev
> >
> >
>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev
>
> No virus found in this incoming message.
> Checked by AVG - www.avg.com
> Version: 8.5.392 / Virus Database: 270.13.29/2261 - Release Date: 07/27/09
> 05:58:00
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev
>

-- 
Michael Emory Cerquoni - Nebadon Izumi @ http://osgrid.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20090727/96549c65/attachment-0001.html>