[Opensim-dev] V1 maptiles and asset spam, possible solution

Sun Oct 1 14:25:03 UTC 2023

Interesting points.

To use these old tiles for a map is quite the roundabout way when 
robust supplies far easier tiles for display on the web, without the 
need to convert from jp2 or making calls to asset server. I know that, 
because I have done exactly that here: https://zetaworlds.com/map 

Inworld
 scripts showing region tiles are a thing, though I have not seen one 
using the direct asset for showing the texture. I suppose there is a way
 to do that, once the uuid is known, but I am not sure how one would 
retrieve that from another region. Can't quite find anything in LSL that
 has the capability of getting that information. OSSL might, but at that
 point might as well go dynamic texture + robust maptiles, seems easier.

It
 would be an issue for a viewer using the V1 tiles to build a world map,
 that is true. However, that viewer would also struggle displaying var 
regions properly as a result unless it does some image scaling there. 
Viewers that still rely for this on not just the minimap are likely to 
encounter other issues with OpenSim as well given all the changes over 
the years. At some point progress can't be made without breaking some 
compatibility for things not maintained for a decade. This has already 
happened a number of times and can't really be avoided.

In
 regards to viewers using it for the minimap the problem does exist, that
 the RegionSettings are transmitted from adjacent regions, but very likely the locally defined
 asset connector is used to make the fetch request for their tile, which would 
then fail. However, as the image data is stored on RegionSettings as 
well, it could be retrieved via that through the neighbor service also. That 
shouldn't be that difficult to do, just need to make another call for that.

Not making changes to a system that ultimately causes unnecessary strain
 and basically kills itself in the process of its operation is poor 
design at best and a disaster waiting to happen at worst. At the moment 
the way this works is to create a new asset with each new generation, 
which will add up in storage space given that images are not exactly 
tiny.

There are already pieces of code 
currently unused for doing at least some cleaning of the tiles, which 
leads me to think this was at some point up on the table. The 
"requirement" for leaving dead tiles on the map is something that highly
 depends on the usecase and isn't something everyone wants. In any case it 
should be an option for a user to select whether they wish to keep their
 spot "reserved" never mind that unless you process kill deregistration 
will vacate the spot on the map in the regions table, it just looks like it is still there as
 the maptile is retained in both systems. (For that the robust tiles are actually better given they stick around even on normal shutdown)

The state the 
entire system is in is quite poor, but baby steps. The most pressing 
issue is the asset spam the V1 tiles generate if one is not careful or 
cleans them regularly, the other being the zoomlevels not generating 
properly due to a hardcoded delay and poor design of the routine. These 
things can be resolved and I have been testing the changes to the 
zoomlevel routine for months now without hiccups. Removing the burden of
 dealing with the V1 tiles all the time is the next point on the testing
 agenda and knowing beforehand what to expect is part of why I am 
writing here.

Ultimately not doing anything
 about this seems like a bad idea given the current state of affairs. If
 that was applied to everything where would we be by now. Breaking 
changes are inevitable, dotnet6 just nuked XEngine for example. Can't 
really be avoided in progress to make some compromises with backwards 
compatibility for things that aren't even maintained anymore themselves.

I'll
 have a look at how neighbor service operates to see if it can be 
utilized to show minimap of adjacent regions properly in case a viewer 
uses the V1 tiles for that(don't actually know which viewer does that 
though still, only know they supposedly exist). That part I did forget might be relevant.

Regards,

Vincent Sylvester

---- On Sun, 01 Oct 2023 14:48:41 +0200 Melanie <melanie at t-data.com> wrote ---

Hi, 

the reason the tiles need to be assets as well as stored in the v2 map system is that both in world LSL scripts and existing 3rd party websites that display grid maps would break. The system is already allowing old maptiles on the asset server to be removed, so they don't build up over time. Of course the "asset" request that is generated by setting a region image asset ID to a prim could be short-circuited locally, and an interface for retrieving map images from some central repository could be agreed between grid operators, but is it worth it? After all, one requirement for displaying a world map is that the tile still be displayable even if the region is crashed, so directly querying the region in question is not an option, also, that would make the world map too slow to be usable on the website. 

The path of least resistance is to let these sleeping dogs lie, I hope I have been able to shed some light on the asset mystery. 

PS: OSGrid's map also uses this system of retrieval. 

- Melanie 

---- On Sun, 01 Oct 2023 13:35:50 +0100 Zetamex Network <mailto:contact at zetamex.com> wrote --- 

I have spent some time over the last couple months working on the entire maptile system, from generation of region tiles, to robust generation of zoomlevels required for the map itself. 

Through that I noticed quite a number of things, but primarily that we still create and store a copy of the region tile to the asset server as "terrainImage_". 

Searching around the parts that utilize said asset I could only find one part about sending the uuid of the asset to the viewer as part of the RegionSettings. 

Given the ultimate request for the image is done via the region asset connector anyways, we could just store the image locally in memory and supply it when the requested asset uuid matches. 

Adding a check in the region asset connector that if no other match returns from the actual asset server(or cache), the requested uuid is checked if it matches a stored region tile uuid. 

Storing the image data on RegionSettings and simply supplying that as "new" asset whenever a request is made to the region asset connector. 

Making sure to cache the request as well should it match as that's the first thing the connector looks for. 

The result is a bit more memory usage for storing the tile at runtime and a faster response in getting the tile asset data. 

No longer storing the image as an asset removes that spam vector from the asset server, which also removes the need for manually cleaning the database entries and or files out of it as robust does not do this on its own. 

I asked a while ago whether we even still need that tile data, but apparently there are still viewers that use it over directly requesting the tiles from robust based on region coordinates. 

I suspect this is because of var regions in some way, although using the tiles from robust and merging them in the viewer, which has a gpu available for doing image processing, would be a better use of resources than having a cpu do it on the server end. 

I did consider using that already existing option to provide the image, but I think the tile required is of the entire region as one image so it would require merging var region tiles into one image and potentially scaling them, which is a lot more processing than just storing the image and matching a uuid. 

Also considered giving robust some more braincells in cleaning up the mess this currently creates, but this would require making assumptions about what data to keep or storing even more information about each tile in order to figure out which ones are no longer required. 

I'm looking for some input on this idea. Are there potential issues I missed? Should the tile be stored as file rather than in memory? Should the V1 tile just be removed entirely and viewers fix it on their end? 

As mentioned, this is part of a larger project to overhaul the 
maptile system on both ends. Removing the spam vectors, reducing 
processing time and increasing the stability and accuracy of the system 
itself. 

Eventually, once testing concludes, this code will 
 go up on mantis for comments and hopefully integration. So far it has 
been working quite well and especially in regards to the zoomlevel 
generation being a lot more accurate than before. 

There 
are quite a few areas that are currently far more rigid and slow than 
they really need to be, but that's a topic for another time. 

Regards, 
Vincent Sylvester 
_______________________________________________ 
Opensim-dev mailing list 
mailto:mailto:Opensim-dev at opensimulator.org 
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev 
_______________________________________________ 
Opensim-dev mailing list 
mailto:Opensim-dev at opensimulator.org 
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev