Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0008628opensim[REGION] OpenSim Corepublic2019-11-14 04:442019-11-19 08:09
Reporteraiaustin 
Assigned To 
PrioritynormalSeveritymajorReproducibilityalways
StatusconfirmedResolutionopen 
PlatformPCOSWindowsOS Version10
Product Versionmaster (dev code) 
Target Versionmaster (dev code)Fixed in Version 
Summary0008628: Viewer Disconnect when TP between two grids from a region with same name as destination
DescriptionI am observing an issue where the viewer disconnects (every time) to its B&W screen on HG TP between two grids from a region with the same name and on the same x,y map grid coordinates as the destination region on the other grid. At least that seems to be the common cause, as I can TP fine to regions on the destination grid with unique region names.

Actual cases are between OSGrid regions (e.g. "Vue" or "Far North") to Openvue grid regions (again e.g. "Vue" or "Far North").

OSGrid is on latest OSGrid release from 1-Nov-2019. Openvue grid is on latest dev master as at 14-Nov-2019. Avatar has no Bakes on Mesh. Viewer is standard Firestorm 6.0.2 OS version.

Disconnects as described using both hop:// in the title bar of the viewer or using domain:port:region in Map Tool for TPs.

Clearing Viewer cache and restrating has no effect.
Additional InformationThis is a partial trace of the destination region of "Vue" o Openvue grid coming from OSGrid "Vue region on one failed attempt... the trace at the OSGrid console end looks normal.

12:33:10 - [CompleteMovement]: Missing COF for e858df02-a860-4b92-937a-2b87e4ebcd6d is ae3d94fa-122e-f1d7-7773-29ce654e7144
12:33:10 - [CompleteMovement]: HG
12:33:10 - [CompleteMovement]: end: 360ms
12:33:10 - [HGUUIDGatherer]: Failed to fetch asset 9a728b41-4ba0-4729-4db7-14bc3d3df741 from http://assets.osgrid.org:80/ [^]
12:34:10 - [LLUDPSERVER]: No packets received from root agent of Ai.Austin @hg.osgrid.org for 60000ms in Vue. Disconnecting.
12:34:10 - [CLIENT]: Close has been called for Ai.Austin @hg.osgrid.org attached to scene Vue
12:34:11 - [JobEngine]: Stopping AsyncInUDP-e858df02-a860-4b92-937a-2b87e4ebcd6d
12:34:11 - [SCENE]: Removing root agent Ai.Austin @hg.osgrid.org e858df02-a860-4b92-937a-2b87e4ebcd6d from Vue
12:34:11 - [CAPS]: Remove caps for agent e858df02-a860-4b92-937a-2b87e4ebcd6d in region Vue
TagsNo tags attached.
Git Revision or version number0.9.1.1 Dev 36 45625a0
Run Mode Grid (Multiple Regions per Sim)
Physics EngineBulletSim
Script EngineXEngine
Environment.NET / Windows64
Mono VersionNone
ViewerFirestorm 6.0.2 OS
Attached Fileslog file icon Firestorm.log [^] (241,695 bytes) 2019-11-14 07:06
log file icon Ai-Austin-Firestorm.log [^] (229,127 bytes) 2019-11-14 07:23
jpg file icon 2019-11-14-Viewer-Disconnected-BandW-Screen.jpg [^] (201,398 bytes) 2019-11-14 07:25


jpg file icon 2019-11-14-Viewer-Contacing-New-Region-BandW-Screen.jpg [^] (189,067 bytes) 2019-11-17 03:48

- Relationships

-  Notes
(0035862)
aiaustin (developer)
2019-11-14 04:52
edited on: 2019-11-14 04:55

Another trace... from OSGrid "Far North" -> Openvue "Far North"

Viewer crashes to B&W screen which then shows the QUIT dialogue.

OSGrid end... looks normal "avatar has left the building".

Openvue end...
11:02:14 - [HELO SERVICE]: Unable to perform HELO request to http://inventory.osgrid.org:80/helo/: [^] The remote server returned an error: (404) Not Found.
11:02:14 - [HG INVENTORY SERVICE]: HELO returned
11:02:14 - [CompleteMovement]: Missing COF for e858df02-a860-4b92-937a-2b87e4ebcd6d is ae3d94fa-122e-f1d7-7773-29ce654e7144
11:02:14 - [CompleteMovement]: HG
11:02:14 - [HELO SERVICE]: Unable to perform HELO request to http://assets.osgrid.org/helo/: [^] The remote server returned an error: (404) Not Found.
11:02:14 - [HG ASSET SERVICE]: HELO returned
11:02:14 - [CompleteMovement]: end: 547ms
11:02:14 - [HGUUIDGatherer]: Failed to fetch asset 9a728b41-4ba0-4729-4db7-14bc3d3df741 from http://assets.osgrid.org:80/ [^]

then after 1 minute

11:03:13 - [LLUDPSERVER]: No packets received from root agent of Ai.Austin @hg.osgrid.org for 60000ms in Far North. Disconnecting.
11:03:16 - [CLIENT]: Close has been called for Ai.Austin @hg.osgrid.org attached to scene Far North
11:03:16 - [JobEngine]: Stopping AsyncInUDP-e858df02-a860-4b92-937a-2b87e4ebcd6d
11:03:16 - [SCENE]: Removing root agent Ai.Austin @hg.osgrid.org e858df02-a860-4b92-937a-2b87e4ebcd6d from Far North
11:03:16 - [CAPS]: Remove caps for agent e858df02-a860-4b92-937a-2b87e4ebcd6d in region Far North
...
11:03:16 - [Scene]: The avatar has left the building

(0035863)
tampa (reporter)
2019-11-14 05:58
edited on: 2019-11-14 06:02

I made a "Far North" on ZetaWorlds, teleported over to virtual.aiai.ed.ac.uk:8002/Far North via worldmap just fine in FS 602

Just went back fine as well. The errors there are timeouts which could happen for all sorts of reasons.

(0035864)
UbitUmarov (administrator)
2019-11-14 06:11
edited on: 2019-11-14 06:21

Ai check if regions uuids are diferent (simple silly things check first)

(0035865)
aiaustin (developer)
2019-11-14 06:17
edited on: 2019-11-14 06:20

Thanks Tampa and Ubit… I did check the UUIDs of the regions as I also immediately suspected that... and they are different in all cases.

That's also why I cleared the viewer cache in case there was some viewer side content caching going on. That had no effect.

I also made sure I was not using any Bakes on Mesh. Used the standard Firestorm 6.0.2 OS viewer, etc.

The 60 second time out is a bit irrelevant, as that is just the time out of the attempted login to the destination grid/region which crashes the viewer immediately.

I placed local avatars on the destination region and on the source region and I see the avatar go away cleanly, and see it arrive on the destination grid, even as the viewer of the teleporting user crashes, and as expected it clears away after the timeout.

I wonder if this might be something OSGrid side?

(0035866)
aiaustin (developer)
2019-11-14 06:21
edited on: 2019-11-14 06:27

Ah, let me check is any ADJACENT (child) region has a UUID that matches....

No they don't - they were a bit similar so I had a scare for a moment.

(0035867)
tampa (reporter)
2019-11-14 06:23

Seems like if the UUID were the same it would not even teleport you, at least I cannot get there now having set the region UUID the same as yours, just teleports me back to the region center.
(0035868)
aiaustin (developer)
2019-11-14 06:28

The regions are at the same x,y location on both grids too.. though that has not caused issues before.
(0035869)
tampa (reporter)
2019-11-14 06:31

Otherwise most welcome regions would have trouble too, remember most of them are either 9000,9000 or 7000,7000 for example, named the same too. Whatever is going on hints more toward something losing connection for some reason.

To clarify you are getting an actual crash aka program close and not a simple "you have been disconnected" ?
(0035870)
aiaustin (developer)
2019-11-14 06:33
edited on: 2019-11-14 06:37

Tampa, if you still have your "Far North" region up locally on you grid, can you see if you can teleport back and forth to

hop://login.osgrid.org/Far%20North/128/128/30 [^]

You can also use a space rather than the %20 in the name in the top navigation bar. But if I do that on mantis it does not render properly.

I was able to crash straight away when I did that from far North on the Openvue grid...

so if you can also go here...

hop://virtual.aiai.ed.ac.uk:8002/Far%20North/128/128/30 [^]

Then TP to hop://login.osgrid.org/Far%20North/128/128/30 [^]

and then back to your grid Far North. Does that all work?

(0035871)
aiaustin (developer)
2019-11-14 06:35
edited on: 2019-11-14 07:26

Correct Tampa. To clarify.. the viewer is immediately crashing to its Black and White style crash screen with the "View IM & Chat" and "Quit" buttons as usual for a Firestorm viewer crash.

See attached 2019-11-14-Viewer-Disconnected-BandW-Screen.jpg

(0035872)
aiaustin (developer)
2019-11-14 06:53

"debug http all 6" at each end does not show anything unusual beyond the messages shown above.
(0035873)
tampa (reporter)
2019-11-14 07:06

Added firestorm log from a logout vue to osg
(0035874)
aiaustin (developer)
2019-11-14 07:24

Added Ai-Austin-Firestorm.log of Openvue to OSGrid crash as a comparison.
(0035876)
aiaustin (developer)
2019-11-17 03:53

When the TP occurs from grid 1/region to grid 2/region the view almost instantly goes to its Black and White state. But it does not immediately shows the disconnected from region/Quit popup. It shows the "Contacting New Region" progress bar which continues to move across to about 75%...

See attached image 2019-11-14-Viewer-Contacting-New-Region-BandW-Screen.jpg

After a while at around 75% it then shows then shows the popup saying disconnected from region and providing the two View IM and Quit buttons as the only things you can do.

See attached image 2019-11-14-Viewer-Disconnected-BandW-Screen.jpg
(0035877)
UbitUmarov (administrator)
2019-11-17 05:01
edited on: 2019-11-17 05:04

As I said, main issue on this is region's position on grid map.

Region positions are used as the main region identification in viewers as the so called region handle, a long number created from map position X and Y.

So HG will just fail if there is an overlap of handlers of regions seen at start grid, and the handlers of regions seen at destination grid

For viewers same handler (position on map) means SAME region.

(0035878)
aiaustin (developer)
2019-11-17 05:06

Thanks Ubit. Fix can be to shift X,Y positions of regions on one of the grids then. Easy enough for specific commonly used HG TPs or identified conflicts.

But I do now have a concern for the many grids based on packaged distributions like Diva D2 and Fred Beckhusen's DreamGrid as people will likely use the default x,y position as regions are created. We used to have many grids that base their regions on and around 1000,1000.
(0035879)
UbitUmarov (administrator)
2019-11-17 05:18

yes it is a big HG issue.. no idea if it ever did seem to work, because things are as i said, and we can't disconect viewer from all regions during TPs, they need to stay connected to sender until the TP is done. So very confusing if they think sender and receiver are the same...
(0035880)
tampa (reporter)
2019-11-17 09:06

"Region positions are used as the main region identification in viewers as the so called region handle"

Another rather stupid part of the protocol then. Can this be worked around? Perhaps tricking the viewer? Else I suppose we need to talk to the viewer devs to account for situations where the overlap causes issues.

That said, all the grids I run put their Welcome at 7000,7000 and teleporting between them has not caused issues, same with Far North being located the same position on ZetaWorlds and on Vue, yet the logout only happens when I try to go to OSG. Am I missing something?
(0035881)
aiaustin (developer)
2019-11-17 10:41

Good point Tampa... one end of my example HG TP fails is OSGrid. Does not matter if it is source or destination.

Maybe worth some more testing to see if It is an issue beyond OSGrid... which it ought to be given what Ubit had said about how viewers work. Makes me wonder if we got away with it working in some way before viewer side caching was enabled? Or if changes... speed ups... in HG TP gave shown the Underlying issue more clearly.
(0035882)
aiaustin (developer)
2019-11-18 03:19

Just as a check, I have established that the issue occurs between any two grids I have tested on.. and is a NOT specific to one of the locations being on OSGrid.

Simply moving the x,y location of one end of the HG TP makes it work.
(0035883)
tampa (reporter)
2019-11-18 04:10

Confirmed across 5 grids, though what to do about it is another page entirely. Changing this to confirmed though since it's kind of a big deal for HG.
(0035889)
tampa (reporter)
2019-11-18 13:21

Seems like Ubit made changes to master, I can pull in the changes and apply them to Far North if you want to test there.
(0035890)
UbitUmarov (administrator)
2019-11-18 13:28
edited on: 2019-11-18 13:49

I just fixed the block of those teleports.
We had it on one code path, not other.
Nothing new. This teleport never worked.
There may still be crashes if there are adjacent regions around both ends if the teleport.
Will check that code.

(0035891)
aiaustin (developer)
2019-11-18 13:48
edited on: 2019-11-19 04:00

I made changes on my grids to ensure none of my regions overlap in terms of map grid coordinates already.

Good thought about adjacent visible (child) regions triggering the problem as well as the specific source and destination.

I made a suggestion for a minor rewording of the message Ubit put into the commit.

Try going other region first
——>
Try going to another region first

(0035892)
melanie (administrator)
2019-11-18 14:25

could we not teleport the user to a "pseudo region" at another position on the local grid, which isn't a region but just like a "holding pen" pretending to be different coordinates, that then just sends the viewer on the real hg teleport from there, if this condition is detected?
(0035893)
tampa (reporter)
2019-11-18 14:47
edited on: 2019-11-18 14:47

I like that idea, kinda like old HG you needed to use the links and then you could jump on the local grid just automatic without the user seeing it. Weren't there coordinates 0 to 32 that are reserved for something anyways? Could use those for that.

(0035894)
piusnoel (reporter)
2019-11-19 08:09

Sometimes I experience another issue, where I get an "Unable to verify identity" message when I try to TP from my test grid to some other grids (e.g. OSG never fails, Metropolis fails) or when I try to TP back to my grid without providing a region. The interesting and maybe related thing, is that it works if I TP first to a working grid and from there to the blocking grid (e.g. first TP to OSG then to Metropolis). I will investigate more time in exploring this later this week.

- Issue History
Date Modified Username Field Change
2019-11-14 04:44 aiaustin New Issue
2019-11-14 04:45 aiaustin Description Updated View Revisions
2019-11-14 04:48 aiaustin Description Updated View Revisions
2019-11-14 04:52 aiaustin Note Added: 0035862
2019-11-14 04:53 aiaustin Note Edited: 0035862 View Revisions
2019-11-14 04:54 aiaustin Note Edited: 0035862 View Revisions
2019-11-14 04:55 aiaustin Note Edited: 0035862 View Revisions
2019-11-14 05:58 tampa Note Added: 0035863
2019-11-14 06:02 tampa Note Edited: 0035863 View Revisions
2019-11-14 06:11 UbitUmarov Note Added: 0035864
2019-11-14 06:17 aiaustin Note Added: 0035865
2019-11-14 06:19 aiaustin Note Edited: 0035865 View Revisions
2019-11-14 06:20 aiaustin Note Edited: 0035865 View Revisions
2019-11-14 06:21 aiaustin Note Added: 0035866
2019-11-14 06:21 aiaustin Note Edited: 0035866 View Revisions
2019-11-14 06:21 aiaustin Note Edited: 0035864 View Revisions
2019-11-14 06:23 tampa Note Added: 0035867
2019-11-14 06:24 aiaustin Note Edited: 0035866 View Revisions
2019-11-14 06:27 aiaustin Note Edited: 0035866 View Revisions
2019-11-14 06:28 aiaustin Note Added: 0035868
2019-11-14 06:31 tampa Note Added: 0035869
2019-11-14 06:33 aiaustin Note Added: 0035870
2019-11-14 06:35 aiaustin Note Added: 0035871
2019-11-14 06:36 aiaustin Note Edited: 0035870 View Revisions
2019-11-14 06:36 aiaustin Note Edited: 0035870 View Revisions
2019-11-14 06:37 aiaustin Note Edited: 0035870 View Revisions
2019-11-14 06:53 aiaustin Note Added: 0035872
2019-11-14 07:06 tampa File Added: Firestorm.log
2019-11-14 07:06 tampa Note Added: 0035873
2019-11-14 07:23 aiaustin File Added: Ai-Austin-Firestorm.log
2019-11-14 07:24 aiaustin Note Added: 0035874
2019-11-14 07:24 aiaustin Additional Information Updated View Revisions
2019-11-14 07:25 aiaustin File Added: 2019-11-14-Viewer-Disconnected-BandW-Screen.jpg
2019-11-14 07:26 aiaustin Note Edited: 0035871 View Revisions
2019-11-17 03:48 aiaustin File Added: 2019-11-14-Viewer-Contacing-New-Region-BandW-Screen.jpg
2019-11-17 03:53 aiaustin Note Added: 0035876
2019-11-17 05:01 UbitUmarov Note Added: 0035877
2019-11-17 05:04 aiaustin Note Edited: 0035877 View Revisions
2019-11-17 05:06 aiaustin Note Added: 0035878
2019-11-17 05:18 UbitUmarov Note Added: 0035879
2019-11-17 09:06 tampa Note Added: 0035880
2019-11-17 10:41 aiaustin Note Added: 0035881
2019-11-18 00:46 aiaustin Severity crash => major
2019-11-18 00:46 aiaustin Summary Viewer Crash when TP between two grids from a region with same name as destination => Viewer Disconnect when TP between two grids from a region with same name as destination
2019-11-18 00:46 aiaustin Description Updated View Revisions
2019-11-18 03:19 aiaustin Note Added: 0035882
2019-11-18 04:10 tampa Note Added: 0035883
2019-11-18 04:10 tampa Status new => confirmed
2019-11-18 13:21 tampa Note Added: 0035889
2019-11-18 13:28 UbitUmarov Note Added: 0035890
2019-11-18 13:42 aiaustin Note Edited: 0035890 View Revisions
2019-11-18 13:48 aiaustin Note Added: 0035891
2019-11-18 13:49 aiaustin Note Edited: 0035890 View Revisions
2019-11-18 14:25 melanie Note Added: 0035892
2019-11-18 14:47 tampa Note Added: 0035893
2019-11-18 14:47 tampa Note Edited: 0035893 View Revisions
2019-11-19 04:00 aiaustin Note Edited: 0035891 View Revisions
2019-11-19 08:09 piusnoel Note Added: 0035894


Copyright © 2000 - 2012 MantisBT Group
Powered by Mantis Bugtracker