MantisBT - opensim
View Issue Details
0008195opensim[REGION] OpenSim Corepublic2017-06-23 02:182017-06-26 01:37
master (dev code) 
master (dev code) 
Grid (Multiple Regions per Sim)
.NET / Windows64
0008195: Save OAR - New errors at early stage of UUID GATHERER now confusing
When a save oar is done with the new UUID gatherer there are red message indications of assets not found or with errors. The summary at the end of the save OAR is then very confusing. A message mid way along (and often well scrolled up by the time the save oar finishes) says in one example I tried that 13 assets were not found and will be ignored. Then later the summary says 3 assets not found. I assume there are really 16 assets not found (??) and that this was 13 in a first pass and 3 at a later stage. But the final message needs to be clearer. Possibly simply saying the TOTAL number not found or that have errors and were ignored as two separate summary figures.


Note also typo of "aditional" which should be "additional" in the new message added in the patches...

[ARCHIVER]: aditional 13 assets have problems and will be ignored


Counts are also odd, out by one (?) or some other problem which makes the stats more confusing along with the above list of red errors appearing...

[ARCHIVER]: 466 scene objects to serialize requiring save of 1231 assets
[ARCHIVER]: ad(d)itional 13 assets have problems and will be ignored
[ARCHIVER]: Saving 1232 assets
[ARCHIVER]: Successfully added 1229 assets (3 assets not found)

Sample Trace for a typical save oar...

Region (OpenVCE) # save oar OpenVCE.oar
10:06:16 - [ARCHIVER]: Writing archive for region OpenVCE to OpenVCE.oar
10:06:16 - [ARCHIVER]: Creating version 0.8 OAR
10:06:16 - [ARCHIVER]: Added control file to archive.
10:06:16 - [ARCHIVER]: Writing region OpenVCE
10:06:17 - [UUID GATHERER]: asset with id 44fccae2-4ae7-7ffd-2a83-67c6109e16e9 not found
10:06:17 - [UUID GATHERER]: asset with id c38bab1a-6304-f8eb-f6e3-42c96a454f7c not found
10:06:17 - [UUID GATHERER]: asset with id 4e3a538e-8287-e224-b1d5-28a95fdaa4ab not found
10:06:17 - [UUID GATHERER]: asset with id 87694dec-38a9-a6bc-d2ec-287259cc6d72 not found
10:06:17 - [UUID GATHERER]: asset with id 587e7e4f-0485-72cc-809a-ccc40e88aded not found
10:06:17 - [UUID GATHERER]: asset with id df694455-f0c5-fc2d-94a1-00c6745f7470 not found
10:06:17 - [UUID GATHERER]: asset with id 71b8ff26-087d-5f44-285b-d38df2e11a81 not found
10:06:17 - [UUID GATHERER]: asset with id 62dc73ca-265f-7ca0-0453-e2a6aa60bb6f not found
10:06:17 - [UUID GATHERER]: asset with id 12788ca0-f20e-9af3-53f6-4be79fa19252 not found
10:06:17 - [UUID GATHERER]: asset with id 69743cb2-e509-ed4d-4e52-e697dc13d7ac not found
10:06:17 - [UUID GATHERER]: asset with id 58da0f9f-42e5-8a8f-ee51-4fac6c247c98 not found
10:06:17 - [UUID GATHERER]: asset with id ee871042-e272-d8ec-3d40-0b0cb3371346 not found
10:06:17 - [UUID GATHERER]: asset with id 8efecbac-35de-4f40-89c1-2c772b83cafa not found
10:06:17 - [ARCHIVER]: 466 scene objects to serialize requiring save of 1231 assets
10:06:17 - [ARCHIVER]: aditional 13 assets have problems and will be ignored
10:06:17 - [ARCHIVER]: Adding region settings to archive.
10:06:17 - [ARCHIVER]: Adding parcel settings to archive.
10:06:17 - [ARCHIVER]: Adding terrain information to archive.
10:06:17 - [ARCHIVER]: Adding scene objects to archive.
10:06:19 - [ARCHIVER]: Saving 1232 assets
10:06:19 - [ARCHIVER]: AssetsRequest executed looking for 1232 possible assets
10:06:19 - [ARCHIVER]: Added 50 assets to archive
10:06:19 - [ARCHIVER]: Added 100 assets to archive
10:06:20 - [ARCHIVER]: Added 150 assets to archive
10:06:20 - [ARCHIVER]: Added 200 assets to archive
10:06:21 - [ARCHIVER]: Added 250 assets to archive
10:06:21 - [ARCHIVER]: Added 300 assets to archive
10:06:21 - [ARCHIVER]: Added 350 assets to archive
10:06:22 - [ARCHIVER]: Added 400 assets to archive
10:06:22 - [ARCHIVER]: Added 450 assets to archive
10:06:22 - [ARCHIVER]: Added 500 assets to archive
10:06:23 - [ARCHIVER]: Added 550 assets to archive
10:06:23 - [ARCHIVER]: Added 600 assets to archive
10:06:23 - [ARCHIVER]: Added 650 assets to archive
10:06:23 - [ARCHIVER]: Added 700 assets to archive
10:06:24 - [ARCHIVER]: Added 750 assets to archive
10:06:24 - [ARCHIVER]: Added 800 assets to archive
10:06:25 - [ARCHIVER]: Added 850 assets to archive
10:06:25 - [ARCHIVER]: Added 900 assets to archive
10:06:25 - [ARCHIVER]: Added 950 assets to archive
10:06:36 - [ARCHIVER]: Added 1000 assets to archive
10:06:36 - [ARCHIVER]: Added 1050 assets to archive
10:06:36 - [ARCHIVER]: Added 1100 assets to archive
10:06:36 - [ARCHIVER]: Added 1150 assets to archive
10:06:37 - [ARCHIVER]: Added 1200 assets to archive
10:06:37 - [ARCHIVER]: Successfully added 1229 assets (3 assets not found)
10:06:37 - [ARCHIVER]: Finished writing out OAR for OpenVCE
No tags attached.
Issue History
2017-06-23 02:18aiaustinNew Issue
2017-06-23 02:18aiaustinStatusnew => assigned
2017-06-23 02:18aiaustinAssigned To => UbitUmarov
2017-06-23 02:20aiaustinDescription Updatedbug_revision_view_page.php?rev_id=6257#r6257
2017-06-23 02:25aiaustinDescription Updatedbug_revision_view_page.php?rev_id=6258#r6258
2017-06-23 02:27aiaustinNote Added: 0032043
2017-06-23 04:47aiaustinNote Edited: 0032043bug_revision_view_page.php?bugnote_id=32043#r6260
2017-06-23 06:05aiaustinNote Added: 0032044
2017-06-23 06:05aiaustinNote Edited: 0032044bug_revision_view_page.php?bugnote_id=32044#r6262
2017-06-23 06:09UbitUmarovNote Added: 0032045
2017-06-23 06:10UbitUmarovNote Edited: 0032045bug_revision_view_page.php?bugnote_id=32045#r6264
2017-06-23 06:13UbitUmarovNote Edited: 0032045bug_revision_view_page.php?bugnote_id=32045#r6265
2017-06-23 07:52UbitUmarovNote Added: 0032046
2017-06-23 07:55aiaustinNote Added: 0032047
2017-06-23 07:57aiaustinNote Added: 0032048
2017-06-23 08:02aiaustinNote Added: 0032049
2017-06-23 08:03aiaustinNote Edited: 0032049bug_revision_view_page.php?bugnote_id=32049#r6267
2017-06-23 08:03aiaustinNote Edited: 0032047bug_revision_view_page.php?bugnote_id=32047#r6269
2017-06-23 08:07aiaustinNote Added: 0032050
2017-06-23 08:07aiaustinNote Edited: 0032048bug_revision_view_page.php?bugnote_id=32048#r6271
2017-06-23 08:09UbitUmarovNote Added: 0032051
2017-06-23 08:27UbitUmarovNote Added: 0032052
2017-06-23 09:32UbitUmarovNote Added: 0032053
2017-06-23 09:49UbitUmarovNote Added: 0032054
2017-06-23 10:50aiaustinNote Edited: 0032048bug_revision_view_page.php?bugnote_id=32048#r6272
2017-06-23 10:50aiaustinNote Edited: 0032048bug_revision_view_page.php?bugnote_id=32048#r6273
2017-06-23 10:57aiaustinNote Added: 0032055
2017-06-23 10:57aiaustinNote Edited: 0032055bug_revision_view_page.php?bugnote_id=32055#r6275
2017-06-23 10:58aiaustinNote Edited: 0032055bug_revision_view_page.php?bugnote_id=32055#r6276
2017-06-23 11:27aiaustinNote Edited: 0032055bug_revision_view_page.php?bugnote_id=32055#r6277
2017-06-23 11:48aiaustinNote Edited: 0032055bug_revision_view_page.php?bugnote_id=32055#r6278
2017-06-24 01:36aiaustinNote Added: 0032058
2017-06-24 01:36aiaustinNote Edited: 0032058bug_revision_view_page.php?bugnote_id=32058#r6280
2017-06-24 01:37aiaustinNote Edited: 0032058bug_revision_view_page.php?bugnote_id=32058#r6281
2017-06-24 01:42UbitUmarovNote Added: 0032059
2017-06-24 10:36aiaustinNote Added: 0032064
2017-06-25 07:52aiaustinNote Deleted: 0032064
2017-06-26 01:02aiaustinNote Added: 0032072
2017-06-26 01:16aiaustinNote Added: 0032073
2017-06-26 01:22aiaustinNote Deleted: 0032073
2017-06-26 01:30aiaustinNote Added: 0032074
2017-06-26 01:32aiaustinNote Edited: 0032074bug_revision_view_page.php?bugnote_id=32074#r6290
2017-06-26 01:36aiaustinNote Edited: 0032074bug_revision_view_page.php?bugnote_id=32074#r6291
2017-06-26 01:37UbitUmarovNote Added: 0032075

2017-06-23 02:27   
(edited on: 2017-06-23 04:47)
In the example given, I am not sure what the 13 "additional assets" mean. Are these ones that SHOULD be in the archive and are not? What types are they if so? The 3 assets not found message at the end... is that really (13+3) = 16 assets not found?

2017-06-23 06:05   
Also... at the end of Commit 66c8b7 (2017-06-22 18:38) in OpenSim/Region/Framework/Scenes/UuidGatherer.cs

m_log.ErrorFormat("[UUID GATHERER]: Failed to uuids ...

"gather" seems to have got lost after "Failed to"

2017-06-23 06:09   
(edited on: 2017-06-23 06:13)
This is evolutionary code, so those incoherencies.
Operations happen on totally different code places and each only sees a limited set on information.

UUID GATHERER only knows and cares about IDs. Only in same cases it does need to fetch and decode assets that reference others, and those are the only ones it can report and not include in the list to save.
Other assets, and the ones on the end of those reference chains are simple ones, so their IDs are only added to the list without any check.

You get a report at this stage, because the bad assets will be forgotten

Assets save takes that list of IDs and does fetch and decode all (some again)
so more problems can be detected.

In your case you have 13 not found plus 3 not found or broken.

and you should notice that this information is actually almost useless ( and none on the save stage), because it does not tell you the object (or inventory item) that is broken, because assets save has no information about those, only knows about assets. And a broken asset may be in use in several objects or items.

hope this makes it more clear
( i will add the not found or broken 'd', and in time try to make this more coherent and clear )

2017-06-23 07:52   
made a few changes, but far from happy. Not knowing the damaged objects or items is bad
2017-06-23 07:55   
(edited on: 2017-06-23 08:03)
Thanks for the explanation @Ubut. For the minor corrections, note the dropped "gather" in one message (see note above) as well as the "ad(d)itional" spelling. You may already have corrected that one now I see.

2017-06-23 07:57   
(edited on: 2017-06-23 10:50)
Watch out @Ubit... spelling error on a variable in your fix just now.. In commit 3c6790

m_previusErrorsCount = previusErrorsCount;

previous is the correct spelling not previus

2017-06-23 08:02   
(edited on: 2017-06-23 08:03)
This is probably a spelling mistake that was already in before your changes...

At line 105 of OpenSim/Region/Framework/Scenes/UuidGatherer.cs

/// Gathered UUIDs will be collected in this dictinaory.

spelling of "dictionary" is wrong

2017-06-23 08:07   
Noting another English correction in the latest fix...

In InventoryArchiveWriteRequest.cs

("[INVENTORY ARCHIVER]: {0} of this assets have problems and will be ignored",

"this assets" should be "these assets"

Occurs twice.
2017-06-23 08:09   
ok ok thx.. still breaking this code
2017-06-23 08:27   
ahhh dictinaory was not mine!! i think ;)
2017-06-23 09:32   
more changes...
commented out some those assets ids error messages
On IAR save and if option to save assets is active, display information about items that where detected to have bad references to assets.
as explained above, this will not detect all problems
actual assets save will detect more, but then there is not information about the items :(
the items are still saved as before...
2017-06-23 09:49   
added identical information on OAR saves
2017-06-23 10:57   
(edited on: 2017-06-23 11:48)
Spotting another inconsistency in this batch of UUIDGatherer and Archiver commits. In commit 11d386 one of the error messages in OpenSim/Region/Framework/Scenes/UuidGatherer.cs

uses a console message module tag of [UUIDgatherer] which I think should be [UUID GATHERER]

This may be used elsewhere too?

Did the "previusErrorsCount" variable mispelling reported above get fixed in one of the patches?

2017-06-24 01:36   
(edited on: 2017-06-24 01:37)
@Ubit, I see more changes being made in ths code area... the warning messages that were already in there and that you have been changing still have "module" tage that show in the console as:


[ARCHIVER Warning]:

The addition of warning is not usual and the message comes out in warning colour anyway... should "Warning" we removed from such messages?

Note previous note too if you missed that amongst all these Mantis changes.

2017-06-24 01:42   
I do want it to be very clear it is a warning, and color does not work everywhere
think I did fix previusErrorsCount
did not found [UUIDgatherer] only [HGUUIDgatherer] and guess that is ok
2017-06-26 01:02   
@Ubit... testing the archiving again with opensim-0.9.0-1543-g0c5f412 (2017-06-26 01:49)...

Note in the following message "where" should be "were" (three occurrences) and there is no closing parenthesis.

08:57:50 - [ARCHIVER]: Successfully added 1229 assets (16 of total possible assets requested where not found, where damaged or where not assets
2017-06-26 01:30   
(edited on: 2017-06-26 01:36)
Interesting... the many new yellow warnings and red errors on save oar and save iar appear to be in NPC appearance notecards, normal notecards which mention UUIDs and commented out UUIDs in scripts. It is good to have the detailed item names and positions to check on things... but I would not be surprised if there are complaints from upgrading grid owners about the large volume of messages any significant region OAR save or avatar inventory save may throw up.

2017-06-26 01:37   
yes, the uuid gather is blind relative to the nature of the uuids, just detects them by their text format
the reason why the split btw errors and warnings
notecards and scripts should not show a error, unless a asset was found and as problems.