Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0007752opensim[REGION] OpenSim Corepublic2015-11-23 10:442016-06-16 05:52
Reporterkenvc 
Assigned To 
PriorityhighSeveritycrashReproducibilitysometimes
StatusnewResolutionopen 
PlatformDual Quad core Xeon Server 3 ghzOSWindows 64 bit 32GB ram OS VersionServer 2012
Product Versionmaster (dev code) 
Target VersionFixed in Version 
Summary0007752: Teleports out of var regions with a lot of prims/textures often fails with viewer timeout and viewer crash. Viewer problem???
DescriptionTeleports out of var regions with a lot of prims/textures often fails with viewer timeout and viewer crash. It doesnt seem to matter if the destination sim is a var or not. What appears to trigger this issue is when the departure sim is a var. It seems to be worse when the departing sim is loaded with a lot of prims and textures.
Steps To Reproduce1. TP to a sim called Commonwealth Estate and stay there a few minutes.
2. Next, TP between Commonwealth Estate, Commonwealth South, and Commonwealth North staying at each sim for a few minutes before teleporting.
3. After a few TPs, the Viewer will freeze while the TP progress bar is on the screen and eventually timeout and crash the viewer, or it will freeze shortly after arriving at the destination sim amd eventually timeout and crash.
Additional InformationThe attached opensim.log file contains the last teleport during this test that resulted in the viewer freeze and crash. This doesn't happen on 100% of the teleports when doing this test, but it normally doesn't take more than 5-10 minutes to make this happen at the most.
TagsNo tags attached.
Git Revision or version number2015-11-22 17:30:08
Run Mode Grid (Multiple Regions per Sim)
Physics EngineBulletSim
Script Engine
Environment.NET / Windows64
Mono VersionNone
ViewerSingularity & FS
Attached Fileslog file icon OpenSim.log [^] (31,661 bytes) 2015-11-23 10:44

- Relationships
related to 0007389confirmed Teleports between 2 vars is broke for some AVs & TPs away from some vars to normal sim often causes long viewer freeze or crash. 

-  Notes
(0029608)
kenvc (reporter)
2015-11-23 10:50
edited on: 2015-11-23 10:52

Looking at the log file, it appears the destination sim isn't getting all the packets it should be receiving, so the TP never completes.

We are also noticing that if you log out using the viewer after being on one of these vars for 15 minutes or so, the viewer often crashes during the logout process.

(0029617)
UbitUmarov (administrator)
2015-11-23 16:46

I was able to do 6 teleports on those 3 regions with no crash, using Singularity Viewer (64 bit) 1.8.6. and 600m view range.
Considering one regions have 71460, 56457 and 27337 prims, I think this may be more related to viewer capabilities (on each individual hardware) than regions.
Firestorm does crash a few tps after being on a large region, ( possible related to parcel overlay size)
(0029618)
UbitUmarov (administrator)
2015-11-23 17:08

Did also some tps around those 3 regions, with FS 4.6.5 with view range of 1024m (regions reduce it to 512 or value in setting) with no crash.
The log shows no region side issue, just that viewers stopped sending information to regions.
(0029663)
Jim Tarber (reporter)
2015-11-25 08:56

There's a (somewhat) known viewer problem with the way it purges allocations when changing regions and a previously-visible region is no longer within draw distance. The viewer will attempt to purge all related allocations (and I suspect one at a time) in a synchronous blocking manner that can sometimes block it for many seconds, long enough for crossing/teleport timeouts and even as long as 30-60 seconds depending on the viewer machine and amount of former content. I call this "the viewer purge problem".

If the user attempts to interact with the viewer on a Windows machine, Windows will indicate "not responding" since the main Windows event loop is blocked.

Even in smaller allocation counts, this problem causes a temporary delay (lockup) of the viewer on crossings and teleports, but especially visible on crossings, shortly after a successful crossing. If it is a vehicle crossing, it looks as if the user has lost control, or there was a big "lag" in updates after crossing, however it is really more a matter that the viewer locked up for some amount of time after crossing. If it's long enough, the teleport or crossing will fail completely.
(0029664)
kenvc (reporter)
2015-11-25 09:10
edited on: 2015-11-25 09:12

What Jim is describing sounds like the exact issue everyone is complaining about. This is happening mostly when you have been in a var with tons of prims and then teleport out. It also happens a lot when crossing the border between 2 vars that have a lot of prims. It seems to happen less often or with a shorter freeze time on computers with really expensive graphics cards.

Is there no solution for these long viewer freezes which often results in a viewer? Would the solution need to be in opensim or the viewers... or both?

(0029665)
Jim Tarber (reporter)
2015-11-25 09:14
edited on: 2015-11-25 09:16

I think it *needs* to be solved at the viewer end, because at the server end the problem is simply a lack of contact/communications in a timely manner from the viewer. Timeouts could be longer, but that just lowers the number of cases, masking the problem, rather than fixing it. And making a somewhat horrible experience for the other crossing/teleport failure cases.

Even if the teleport/crossing succeeded, the viewer problem remains anyway, since it is still locked up and unresponsive for some extensive period of time. And in the vehicle crossing case, that means the user loses control of the vehicle for some period of time. Also, if there is a viewer processing delay, the server is likely to get a flood of packets from the viewer or vice versa after the crossing, further tying up the viewer response to the user.

ETA: I may be wrong about this, but as far as I know, this viewer purge problem is in all viewers, all versions, including the current upstream LL code.

(0029666)
Jim Tarber (reporter)
2015-11-25 09:20

Ideally, it could be resolved by the viewer using a private heap for these allocations in each region, so that when it purges a region it just makes one single heap call to free the *entire* region-specific heap.

If that was too intrusive or not available, it could just pass the info for the purge to a separate thread that freed the allocations over a longer time or in a non-blocking manner, although that could introduce concurrency issues if the data being freed was in viewer content lists rather than just heap allocations.

A third option might be to use fixed-sized heaps for everything so that deallocations were Order(1) regardless of the number of existing allocations.
(0029724)
kenvc (reporter)
2015-11-27 08:17

I have a normal sim (not var) with 24,000 prims and lots of textures where this is also happening sometimes.
(0030539)
kenvc (reporter)
2016-06-16 05:52

This is still happening. Latest info about this seems to point to the Viewers and the way they all clear the previous scene from the display card's memory. Apparently this is a slow process when the prim count on the var is high and the AV has been on the sim long enough to load a lot of graphics info from the sim into the video card's memory.

 It would be nioce to see a fix for this, but at this point it appears like it has to be done at the viewer end, and I hear it would mean a major change in the way the Viewer handles clearing the previous scene when you TP somewhere.

- Issue History
Date Modified Username Field Change
2015-11-23 10:44 kenvc New Issue
2015-11-23 10:44 kenvc File Added: OpenSim.log
2015-11-23 10:50 kenvc Note Added: 0029608
2015-11-23 10:52 kenvc Note Edited: 0029608 View Revisions
2015-11-23 15:24 kenvc Steps to Reproduce Updated View Revisions
2015-11-23 15:24 kenvc Steps to Reproduce Updated View Revisions
2015-11-23 16:46 UbitUmarov Note Added: 0029617
2015-11-23 17:08 UbitUmarov Note Added: 0029618
2015-11-25 08:56 Jim Tarber Note Added: 0029663
2015-11-25 09:10 kenvc Note Added: 0029664
2015-11-25 09:11 kenvc Note Edited: 0029664 View Revisions
2015-11-25 09:12 kenvc Note Edited: 0029664 View Revisions
2015-11-25 09:14 Jim Tarber Note Added: 0029665
2015-11-25 09:16 Jim Tarber Note Edited: 0029665 View Revisions
2015-11-25 09:20 Jim Tarber Note Added: 0029666
2015-11-25 09:25 kenvc Summary Teleports out of var regions with a lot of prims/textures often fails with viewer timeout and viewer crash. => Teleports out of var regions with a lot of prims/textures often fails with viewer timeout and viewer crash. Viewer problem???
2015-11-27 08:16 kenvc Relationship added related to 0007389
2015-11-27 08:17 kenvc Note Added: 0029724
2016-06-16 05:52 kenvc Note Added: 0030539


Copyright © 2000 - 2012 MantisBT Group
Powered by Mantis Bugtracker