Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0007363opensim[GRID] Asset Servicepublic2014-10-30 01:292014-12-15 13:06
Reporterwbalazic 
Assigned To 
PriorityhighSeveritymajorReproducibilityrandom
StatusnewResolutionopen 
Platform.NET region / Linux GridOSOS Version
Product Versionmaster (dev code) 
Target VersionFixed in Version 
Summary0007363: Items in assetcache do not propagate to the asset server
DescriptionUsers upload texture on region A. On TP to region B on another server, texture does not exist. Upon return to region A texture returns. Additionally, when clearing assetcache of a region, user experiences loss of content and textures randomly on restart. Restoring an OAR taken right before the assetcache clear does not resolve the issue, the assets still are non-existent which means they never went to the asset server. To resolve the issue, if I restore the original assetcache the item returns. This occurs on ODE and Bullet regions, VAR's or standard regions.
Steps To ReproduceThis happens roughly 30% of the time, and is cannot always be reproduced effectively.
Additional InformationAll servers in the environment including grid server, asset server and databases are on a GB fiber internal backbone communicating with each other (no network latency). All of the servers have SSD drives for assetcache (no disk latency). CPU of regions is at 1% consistently (no CPU latency).
TagsNo tags attached.
Git Revision or version number0.8.0.2 (released on 16th October 2014)
Run Mode Grid (Multiple Regions per Sim)
Physics EngineODE
Script Engine
Environment.NET / Windows64
Mono VersionNone
ViewerAll
Attached Files

- Relationships

-  Notes
(0026982)
aiaustin (developer)
2014-10-30 14:32
edited on: 2014-10-30 14:37

Walter, assets are written to persistent storage after a delay... which
by default can be up to 5 minutes. See

MinimumTimeBeforePersistenceConsidered = 60
MaximumTimeBeforePersistenceConsidered = 600

backup command on the console forces the assets to be made persistent and a,ways is wise befire a shutdown anyway... put it in shutdown_commands.txt

(0026983)
wbalazic (reporter)
2014-10-30 14:39

As part of my shutdown command when restarting the regions I run the backup command. It has no effect on this issue.
(0026984)
wbalazic (reporter)
2014-10-30 14:44
edited on: 2014-10-30 14:55

Copy of my shutdown_commands.txt:


; Copy this file to shutdown_commands.txt to execute region console commands when the simulator is asked to shut down
; e.g. show stats
; Lines that start with ; are comments
show uptime
backup

Trying setting = 0 for the following:

    ;# {MinimumTimeBeforePersistenceConsidered} {} {Time before un-changed object may be persisted} {} 60
    ;; Objects will be considered for persistance in the next sweep when they
    ;; have not changed for this number of seconds.
    MinimumTimeBeforePersistenceConsidered = 0

    ;# {MaximumTimeBeforePersistenceConsidered} {} {Time before changed objects may be persisted?} {} 600
    ;; Objects will always be considered for persistance in the next sweep
    ;; if the first change occurred this number of seconds ago.
    MaximumTimeBeforePersistenceConsidered = 0

According to the remark, setting zero makes it persistent immediately.

(0026985)
aiaustin (developer)
2014-10-31 05:33
edited on: 2014-10-31 05:35

I am going to try min 60 seconds and max 120 seconds on Openvue grid and see how we get on. Having a short delay helps make sure the asset data base does not get cluttered with rezzed boxes to open for their content and which are then immediately deleted or returned to owner I assume.

(0027101)
BlueWall (administrator)
2014-12-15 11:31

You can upload an item then you should see a message in your region console similar to this:

14:24:10 - [BUNCH OF CAPS]: Uploaded asset 9cbcc451-8ff0-4eab-b1b4-9292392e3373 for inventory item 739f286b-399e-4596-9bfb-e6f165992751, inv type texture, asset type texture

Then, in your Robust console of your asset server, you can issue a `show asset <Asset UUID>` command, using the uuid of the asset given in the region's console to verify that the item was saved to the asset server.
(0027102)
wbalazic (reporter)
2014-12-15 11:34

What about for items created in world?
(0027103)
BlueWall (administrator)
2014-12-15 11:56

The agent uuid is: 9d3a2998-3d65-41a0-8e24-14574bfc0fec
The new item is named 'Test Item'
It was placed in the 'Objects' folder when take/take copy

In your Roobust console of your Inventory server use this query to get the asset uuid of 'Test Item':

select assetID from inventoryitems where inventoryName = 'Test Item' and parentFolderID in (select folderID from inventoryfolders where agentID = '9d3a2998-3d65-41a0-8e24-14574bfc0fec' and folderName = 'Objects');

Next, issue the `show asset <asset uuid>` to see the item in your asset server.
(0027104)
wbalazic (reporter)
2014-12-15 11:58

Interestingly I just did a test on this. I see nothing on the region console saying "uploaded asset -xxxxxxx etc". Do I need to have logging set at a higher level for that?
(0027105)
BlueWall (administrator)
2014-12-15 12:01

set your log to level debug to see it.
(0027106)
wbalazic (reporter)
2014-12-15 12:26

That part appears to be working properly:

12:19:48 - [BUNCH OF CAPS]: Uploaded asset e25a16a8-e769-42b6-900f-44101b176137 for inventory item 0f40b695-f396-4a09-a085-b1a19660f
3c2, inv type texture, asset type texture

R.O.B.U.S.T. # show asset e25a16a8-e769-42b6-900f-44101b176137
Name: woodbeam
Description:
Type: Texture (type number = 0)
Content-type: image/x-j2c
Size: 12285 bytes
Temporary: no
Flags: Normal
0000: FF-4F-FF-51-00-2F-00-00-00-00-00-20-00-00-04-00
0010: 00-00-00-00-00-00-00-00-00-00-00-20-00-00-04-00
0020: 00-00-00-00-00-00-00-00-00-03-07-01-01-07-01-01
0030: 07-01-01-FF-52-00-0C-00-00-00-04-01-05-04-04-00
0040: 00-FF-5C-00-23-22-77-1E-76-EA-76-EA-76-BC-6F-00

Let me add that based on my initial remarks, we are able many times to recreate the following issue. Let's say Avi picks up item A at region A then teleports to region B. When they arrive at region B they try to access the item and it says "unable to load asset xxxxxx from database" or some similar dialog. However, when Avi A returns to region A the item is able to be resolved. Which is why I consider this to be an assetcache issue. Now obviously this isn't happening all the time, but it is happening enough that it causes major disruption.
(0027107)
BlueWall (administrator)
2014-12-15 12:55

The asset cache is for objects rezzed in-world and serves to fill requests of entering avatars w/o needing to call the asset server. If the object is created in region A, then it doesn't exist in region B, or it's asset cache until the avatar rezzes it into the region. The object's information is recorded into the grid's Inventory and Asset servers when it is picked up - or copied by the avatar. Any information about the object in Region B will be coming from the Inventory and Asset servers (via the region except for http delivery of textures, I think) and would also depend, somewhat, on the user's network connection speed and quality to resolve the item details within the viewer.

Setting the log levels up to debug might display some sort of related messages whey you experience this. You might examine your Cap* settings in the region ini.
(0027108)
wbalazic (reporter)
2014-12-15 13:06

Actually I'm experimenting with Cap* settings right now in fact. It's not that well documented which will help but alot of this started quite some time ago when throttling was implemented. I have it turned off but I don't think that it is in fact disabled.

- Issue History
Date Modified Username Field Change
2014-10-30 01:29 wbalazic New Issue
2014-10-30 14:32 aiaustin Note Added: 0026982
2014-10-30 14:33 aiaustin Note Edited: 0026982 View Revisions
2014-10-30 14:34 aiaustin Note Edited: 0026982 View Revisions
2014-10-30 14:35 aiaustin Note Edited: 0026982 View Revisions
2014-10-30 14:37 aiaustin Note Edited: 0026982 View Revisions
2014-10-30 14:39 wbalazic Note Added: 0026983
2014-10-30 14:44 wbalazic Note Added: 0026984
2014-10-30 14:55 wbalazic Note Edited: 0026984 View Revisions
2014-10-31 05:33 aiaustin Note Added: 0026985
2014-10-31 05:35 aiaustin Note Edited: 0026985 View Revisions
2014-12-15 11:31 BlueWall Note Added: 0027101
2014-12-15 11:34 wbalazic Note Added: 0027102
2014-12-15 11:56 BlueWall Note Added: 0027103
2014-12-15 11:58 wbalazic Note Added: 0027104
2014-12-15 12:01 BlueWall Note Added: 0027105
2014-12-15 12:26 wbalazic Note Added: 0027106
2014-12-15 12:55 BlueWall Note Added: 0027107
2014-12-15 13:06 wbalazic Note Added: 0027108


Copyright © 2000 - 2012 MantisBT Group
Powered by Mantis Bugtracker