Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0008439opensim[REGION] OpenSim Corepublic2019-01-01 07:582020-08-10 01:55
Reporteraiaustin 
Assigned To 
PrioritynormalSeveritytweakReproducibilityalways
StatusnewResolutionopen 
PlatformPCOSWindowsOS Version10
Product Versionmaster (dev code) 
Target Versionmaster (dev code)Fixed in Version 
Summary0008439: Region maptiles - can previous tiles be removed from database as they are replaced by later versions
DescriptionRegion maptiles are stored in the opensim.assets data table each time they are produced. But, as far as I understand, previous copies will never be used again. A good time to tidy the data base would seem to be at exactly the time that a new tile is stored, with older versions for that region UUID being deleted at that time.

Note that at present I occasionally do the following mysql console command while the grid is stopped and sometimes thousands of these are deleted... even on a simple grid.

DELETE FROM opensim.assets WHERE name LIKE "terrainImage_%";
TagsNo tags attached.
Git Revision or version numberopensim-0.9.0.1-685-g9c043fe.zip
Run Mode Grid (Multiple Regions per Sim)
Physics EngineBulletSim
Script Engine
Environment.NET / Windows64
Mono VersionNone
ViewerN/A
Attached Files

- Relationships

-  Notes
(0034325)
tampa (reporter)
2019-02-05 13:37

Actually a very good question. The map tiles seem to be entered into the database as new asset even if their hash is identical. I believe they are in there for robust to be able to serve them directly, but why it creates new entries each time... strange.

I have fields for the creation date in my tables, perhaps a query to clear all but the newest tile would make more sense, then again ideally the entry in the database would be updated instead of new ones being created.
(0034326)
UbitUmarov (administrator)
2019-02-05 14:58
edited on: 2019-02-05 14:59

commands to delete old terrain image and parcel image are actually issued on map generation.
but they are currently ignored unless you set AllowRemoteDelete = true on robust.ini.
AllowRemoteDeleteAllTypes should stay false

(0034327)
tampa (reporter)
2019-02-05 15:10

Doesn't AllowRemoteDelete also enable deletion of other assets though or just the maptiles?

Why do we still store them that way if they are stored in maptiles folder, just for v1 viewers? Should those still be supported at this point?

I just issued a delete on all these entries I have on my assets table, so far nothing has caught fire, you sure these are actively used still?
(0034328)
UbitUmarov (administrator)
2019-02-05 15:12

AllowRemoteDeleteAllTypes should stay false <--- or all types can be deleted
(0034330)
tampa (reporter)
2019-02-05 15:20

What was the reasoning for setting the default to false?
(0034331)
UbitUmarov (administrator)
2019-02-05 15:20

reason lost in time..
(0034332)
tampa (reporter)
2019-02-05 15:28

So then why not change this and prevent blowing up asset tables with useless older versions of maptiles that are never even going to be used even if they happen to be requested by a v1 viewer, because I suppose that also only fetches the most current version?
(0034334)
aiaustin (developer)
2019-02-06 02:06

Thanks for pointing out this Robust[.HG].ini setting which I did not know about. I changed it to AllowRemoteDelete = true on the Openvue and Ailand grid.

Would it be worth setting this parameter to true by default now as these data base entries do build up.
(0034337)
paela argus (reporter)
2019-02-06 07:56

because if you use fsassets is not needed AllowRemoteDelete = true
use old db think for assets at this years why ?
(0036457)
Ferd Frederix (reporter)
2020-05-09 22:01

I had 7657 maptiles. I set AllowRemoteDelete = true in Robust.HG.ini. After clearing maptiles and restarting and making the highest version tiles, I still had 7654 maptiles in robust after all maps were regenned. AllowRemoteDelete = true had no effect.

mysql> select count(*) FROM robust.assets WHERE name LIKE "terrainImage_%";
+----------+
| count(*) |
+----------+
| 7657 |
+----------+
1 row in set (25.36 sec)
(0036458)
Ferd Frederix (reporter)
2020-05-09 22:03

Also very time consuming:

mysql> delete FROM robust.assets WHERE name LIKE "terrainImage_%";
Query OK, 7657 rows affected (23 min 35.96 sec)
(0036465)
aiaustin (developer)
2020-05-10 03:03
edited on: 2020-08-10 01:10

Fred, I just checked my Openvue and AiLand grids and the following SQL gives exactly the number of tiles I would expect, one per region, so the deletes are working correctly in my 0.9.2.0 dev master setup and have been for some time.

SELECT * FROM opensim.assets WHERE name LIKE "terrainImage_%";

One difference I can see may be relevant. I run with a single data base "opensim" for each grid. Each grid does have several servers for the add on regions, so that is not a factor. Could it be that robust.assets old map tiles are not being cleared out and that the clear out only works if its in the same database as used for the region?

(0036678)
tampa (reporter)
2020-08-09 22:21

I think austin is right, at least in regards to the database setup it seems only on standalone-like setups that setting actually deletes anything. I tried the other setting as well to delete all types, when purging inventory nothing happened. It seems those settings don't work for grid setups with connectors.

I had a look in my logs, I have about 3 months worth of logins from about 900 unique users, none of which used a v1 viewer. Probably going to patch v1 maptiles out to stop my database filling up. That's at least my opinion.

The system is not really thought out, as mentioned, it runs on region startup and has to if you want to keep maptile generation at all. That's hardlocked though and simply disallowing generation at startup while keeping the ability to manually issue a generate map is possible.

Not sure what direction is best for this, but the current setup is a bit of a mess, sorry to be so blunt.
(0036679)
aiaustin (developer)
2020-08-10 01:09

The phrase “standalone-like setups” might confuse some. as we use the term “standalone” for non-Robust installations. The old map tiles are properly deleted on Robust grids too... but my example grids where that applies use a single MySQL database for core service and assets, these are not split into separate data bases as allowed by OpenSim configuration options.
(0036680)
tampa (reporter)
2020-08-10 01:55

I use a connector for assets rather than saving them to database, I don't see the deletions causing the error they should cause if they were sent, as the asset server itself has no delete it should produce a 404, but there is nothing. Guessing the connector does not support deletion only the localservice does?

I'll deal with this with some bash and do a deep clean manually, but that's not a long term option :S

- Issue History
Date Modified Username Field Change
2019-01-01 07:58 aiaustin New Issue
2019-01-01 07:59 aiaustin Description Updated View Revisions
2019-01-01 10:51 aiaustin Description Updated View Revisions
2019-02-05 13:37 tampa Note Added: 0034325
2019-02-05 14:58 UbitUmarov Note Added: 0034326
2019-02-05 14:59 UbitUmarov Note Edited: 0034326 View Revisions
2019-02-05 15:10 tampa Note Added: 0034327
2019-02-05 15:12 UbitUmarov Note Added: 0034328
2019-02-05 15:20 tampa Note Added: 0034330
2019-02-05 15:20 UbitUmarov Note Added: 0034331
2019-02-05 15:28 tampa Note Added: 0034332
2019-02-06 02:06 aiaustin Note Added: 0034334
2019-02-06 07:56 paela argus Note Added: 0034337
2020-05-09 22:01 Ferd Frederix Note Added: 0036457
2020-05-09 22:03 Ferd Frederix Note Added: 0036458
2020-05-10 03:03 aiaustin Note Added: 0036465
2020-05-10 04:15 aiaustin Note Edited: 0036465 View Revisions
2020-05-10 04:16 aiaustin Note Edited: 0036465 View Revisions
2020-08-09 22:21 tampa Note Added: 0036678
2020-08-10 01:09 aiaustin Note Added: 0036679
2020-08-10 01:10 aiaustin Note Edited: 0036465 View Revisions
2020-08-10 01:55 tampa Note Added: 0036680


Copyright © 2000 - 2012 MantisBT Group
Powered by Mantis Bugtracker