Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0008455opensim[GRID] Hypergridpublic2019-01-16 01:462019-02-01 09:00
Reporteraiaustin 
Assigned To 
PrioritynormalSeverityminorReproducibilityalways
StatusnewResolutionopen 
PlatformPCOSWindowsOS Version10
Product Version 
Target VersionFixed in Version 
Summary0008455: Child agents not closed cleanly when HG traveller exits grid - closes only after 60000ms no packets received timeout
DescriptionWhen a HG traveller left the grid and was (I believe) properly returned to their home grid, all the adjacent regions on two separate servers showed yellow messages after 5 minutes (60000ms) timeout... such as...

23:37:29 - [LLUDPSERVER]: No packets received from child agent of Brian.. @hg.viewtwo.net:8654 for 60000ms in Edinburgh Uplands. Disconnecting.

There have been changes to Dev Master in recent days in the area of child agent clean up for the teleport handling. However, the Git commits indicate that HG teleports are handled in the old way at present. It is possible this issue has been present for longer, but I am specifically monitoring teleport attempts due to the code changes and thought it worth opening this Mantis issue to allow inputs on any similar issues from those using Dev master versions after the [2019-01-14 17:25] commit.
Additional InformationFull example log... avatar left grid at 22:53:41 then no console messages or activity until 60000ms (approx.) timeout at 23:40:50.

22:53:40 - [HG ENTITY TRANSFER MODULE]: Teleporting agent via landmark to http://hg.viewtwo.net:8654 [^] region 33f15d55-b426-4e70-a942-26606cbe49cf position <121.6659, 119.8572, 22.16383>
22:53:40 - [GATEKEEPER SERVICE CONNECTOR]: contacting http://hg.viewtwo.net:8654/ [^]
22:53:40 - [ENTITY TRANSFER MODULE]: Teleporting Brian.. @hg.viewtwo.net:8654 af4d1b2d-1427-4aef-9234-4a57077873e9 from Openvue to http://hg.viewtwo.net:8654/ [^] (http://hg.viewtwo.net:8654/ [^]) hrethra/<121.6659, 119.8572, 22.16383>
22:53:40 - [REMOTE SIMULATION CONNECTOR]: QueryAccess to http://hg.viewtwo.net:8654/agent/af4d1b2d-1427-4aef-9234-4a57077873e9/33f15d55-b426-4e70-a942-26606cbe49cf/ [^] returned True, reason , version 0.7/0.7
22:53:40 - [ENTITY TRANSFER MODULE]: Openvue transfer protocol version to hrethra is 0.7 / 0.7
22:53:40 - [ENTITY TRANSFER MODULE]: Determined that region hrethra at 8654,8654 size 256,256 needs new child agent for agent Brian.. @hg.viewtwo.net:8654 from Openvue
22:53:40 - [HG ENTITY TRANSFER MODULE]: CreateAgent http://hg.viewtwo.net:8654/ [^] http://hg.viewtwo.net:8654/ [^]
22:53:40 - [REMOTE SIMULATION CONNECTOR]: Creating agent at http://82.31.173.203:8654/ [^]
22:53:40 - [ENTITY TRANSFER MODULE]: Sending new CAPS seed url http://hg.viewtwo.net:8654/CAPS/5158ff76-6fd7-4292-877d-0531665fee2b0000/ [^] from Openvue to Brian.. @hg.viewtwo.net:8654
22:53:41 - [SCENE PRESENCE]: Making Brian.. @hg.viewtwo.net:8654 a child agent in Openvue from root region 9515173628988928
23:40:50 - [LLUDPSERVER]: No packets received from child agent of Brian.. @hg.viewtwo.net:8654 for 60000ms in Sandbox. Disconnecting.
23:40:50 - [CLIENT]: Close has been called for Brian.. @hg.viewtwo.net:8654 attached to scene Sandbox
23:40:50 - [JobEngine]: Stopping AsyncInUDP-af4d1b2d-1427-4aef-9234-4a57077873e9
23:40:50 - [SCENE]: Removing child agent Brian.. @hg.viewtwo.net:8654 af4d1b2d-1427-4aef-9234-4a57077873e9 from Sandbox
23:40:50 - [CAPS]: Remove caps for agent af4d1b2d-1427-4aef-9234-4a57077873e9 in region Sandbox
23:40:50 - [Scene]: The avatar has left the building
TagsNo tags attached.
Git Revision or version numberopensim-0.9.0.1-720-g75d3e7a.zip
Run Mode Grid (Multiple Regions per Sim)
Physics EngineBulletSim
Environment.NET / Windows64
Mono VersionNone
ViewerN/A
Attached Filestxt file icon 2019-02-01-OSGrid-TP-Issue.txt [^] (9,973 bytes) 2019-02-01 09:00 [Show Content]

- Relationships

-  Notes
(0033819)
tampa (reporter)
2019-01-16 05:26

Forgive me for saying, but maybe it is time for teleportsV2. This back and forth with timeouts and small patches here and there does not really seems very scientific nor problem orientated. Without knowing specifically if this is, again, a viewer thing maybe it really is time to handle all this sort of stuff over http/tcp where possible, because it just occurs to me that the vast majority of connection and teleport issues arise from slightly less perfect connections and people using wireless.

It just seems like we are fighting symptoms of a bigger issue rather than fixing the actual issue underneath. Just my opinion though.
(0033820)
aiaustin (developer)
2019-01-16 05:28

Perhaps Ubit’s new teleport protocol version 0.7 is that?
(0033833)
aiaustin (developer)
2019-01-18 07:32
edited on: 2019-02-01 05:49

Child agents remain on the

15:33:18 - [LLUDPSERVER]: No packets received from child agent of Ai.Austin @virtual.aiai.ed.ac.uk:8002 for 60000ms in OpenVCE. Disconnecting.

So HG travellers are definitely not being tidied up for 60000ms and only then due to an inactivity timeout. The root agent has been tidied up, so I assume the information is available server side to tidy up.

(0034092)
aiaustin (developer)
2019-02-01 05:49
edited on: 2019-02-01 07:12

I am seeing more TP issues even on one grid that appears to follow any failure to TP between regions which are running but seem not to latch onto the TP. After the failure, I seem to be stuck in a mode where I have lots of child agents (on regions that were attempted but failed TP targets and are definitely more than draw distance way regions) and no root agent. Viewer (eventually) returns avatar to original region before TP attempt, but canno tmove and need to relog. The child agents seem to stay in the target regions which were being tried and "show users full" shows them as still present for a long time afterwards (way more than 60000ms as I left things longer than that to test if they timed out). The child agents in region consoles only disappeared after AFTER the viewer was stopped.

I observed this red error on the target region on one TP attempt if it helps...

13:43:30 - [BASE HTTP SERVER]: HandleRequest() threw exception System.NullReferenceException: Object reference not set to an instance of an object.
   at OpenSim.Region.Framework.Scenes.SceneObjectPart.get_GroupPosition() in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Region\Framework\Scenes\SceneObjectPart.cs:line 764
   at OpenSim.Region.Framework.Scenes.SceneObjectGroup.set_AbsolutePosition(Vector3 value) in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Region\Framework\Scenes\SceneObjectGroup.cs:line 600
   at OpenSim.Region.CoreModules.Avatar.Attachments.AttachmentsModule.AttachToAgent(IScenePresence sp, SceneObjectGroup so, UInt32 attachmentpoint, Vector3 attachOffset, Boolean silent) in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Region\CoreModules\Avatar\Attachments\AttachmentsModule.cs:line 1072
   at OpenSim.Region.CoreModules.Avatar.Attachments.AttachmentsModule.AttachObjectInternal(IScenePresence sp, SceneObjectGroup group, UInt32 attachmentPt, Boolean silent, Boolean addToInventory, Boolean resumeScripts, Boolean append) in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Region\CoreModules\Avatar\Attachments\AttachmentsModule.cs:line 654
   at OpenSim.Region.CoreModules.Avatar.Attachments.AttachmentsModule.AttachObject(IScenePresence sp, SceneObjectGroup group, UInt32 attachmentPt, Boolean silent, Boolean addToInventory, Boolean append) in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Region\CoreModules\Avatar\Attachments\AttachmentsModule.cs:line 541
   at OpenSim.Region.Framework.Scenes.Scene.AddSceneObject(SceneObjectGroup sceneObject) in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Region\Framework\Scenes\Scene.cs:line 2960
   at OpenSim.Region.CoreModules.Framework.EntityTransfer.EntityTransferModule.HandleIncomingSceneObject(SceneObjectGroup so, Vector3 newPosition) in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Region\CoreModules\Framework\EntityTransfer\EntityTransferModule.cs:line 2693
   at OpenSim.Region.CoreModules.Framework.EntityTransfer.HGEntityTransferModule.HandleIncomingSceneObject(SceneObjectGroup so, Vector3 newPosition) in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Region\CoreModules\Framework\EntityTransfer\HGEntityTransferModule.cs:line 647
   at OpenSim.Region.Framework.Scenes.Scene.IncomingCreateObject(Vector3 newPosition, ISceneObject sog) in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Region\Framework\Scenes\Scene.cs:line 2889
   at OpenSim.Region.CoreModules.Avatar.Attachments.AttachmentsModule.CopyAttachments(AgentData ad, IScenePresence sp) in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Region\CoreModules\Avatar\Attachments\AttachmentsModule.cs:line 356
   at OpenSim.Region.Framework.Scenes.ScenePresence.CopyFrom(AgentData cAgent) in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Region\Framework\Scenes\ScenePresence.cs:line 4859
   at OpenSim.Region.Framework.Scenes.ScenePresence.UpdateChildAgent(AgentData cAgentData) in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Region\Framework\Scenes\ScenePresence.cs:line 4638
   at OpenSim.Region.Framework.Scenes.Scene.IncomingUpdateChildAgent(AgentData cAgentData) in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Region\Framework\Scenes\Scene.cs:line 4545
   at OpenSim.Region.CoreModules.ServiceConnectorsOut.Simulation.LocalSimulationConnectorModule.UpdateAgent(GridRegion destination, AgentData cAgentData, EntityTransferContext ctx) in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Region\CoreModules\ServiceConnectorsOut\Simulation\LocalSimulationConnector.cs:line 218
   at OpenSim.Server.Handlers.Simulation.AgentPutHandler.UpdateAgent(GridRegion destination, AgentData agent) in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Server\Handlers\Simulation\AgentHandlers.cs:line 766
   at OpenSim.Server.Handlers.Simulation.AgentPutHandler.DoAgentPut(Hashtable request, Hashtable responsedata) in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Server\Handlers\Simulation\AgentHandlers.cs:line 735
   at OpenSim.Server.Handlers.Simulation.AgentPutHandler.ProcessRequest(String path, Stream request, IOSHttpRequest httpRequest, IOSHttpResponse httpResponse) in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Server\Handlers\Simulation\AgentHandlers.cs:line 668
   at OpenSim.Framework.Servers.HttpServer.BaseStreamHandler.Handle(String path, Stream request, IOSHttpRequest httpRequest, IOSHttpResponse httpResponse) in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Framework\Servers\HttpServer\BaseStreamHandler.cs:line 72
   at OpenSim.Framework.Servers.HttpServer.BaseHttpServer.HandleRequest(OSHttpRequest request, OSHttpResponse response) in K:\OSGRID\opensim-0.9.0.1-760-g695d807\OpenSim\Framework\Servers\HttpServer\BaseHttpServer.cs:line 694

(0034093)
aiaustin (developer)
2019-02-01 08:59
edited on: 2019-02-01 09:16

Maybe I should shift the "on one grid" TP issue to another Mantis.. but it just seems to have very similar symptoms to do with missing a connect between a TP to a region even within one grid and then it getting in a mess with leaving child agents around and not allowing TP back and jamming up the avatar after the timeouts. The terrain and water is also missing.. Remember this is between two regions within ONE grid (OSGrid) both on 26-Jan-2019) version.. no HG involved.

I attach a full log of the OSGrid regions TP attempt (Vue-Port to VRLand both on the same server in this test in different openSim.exe instances).

2019-02-01-OSGrid-TP-Issue.txt

The main issue seems to be that I appear to connect (Completing movement) and then the next log line is 10 seconds later when it says it did not receive agent update. After that its jammed u, avatar on original region cannot move, and child agents remain on destination region console using "show users full" until relog. I left it well over 5 minutes to see if it timed out after 60000ms and it did not.

16:55:39 - [SCENE PRESENCE]: Completing movement of Ai Austin into region VRLand in position <128, 128, 1.5>
16:55:49 - [SCENE PRESENCE]: Update agent Ai Austin at VRLand did not receive agent update

After about 17 minutes, with the avatar still stuck on the original source region and unable to move, etc, I logged out of the viewer.. then after about another minute I get the no packets received message... but this did NOT happen until the viewer was stopped.

17:12:17 - [LLUDPSERVER]: No packets received from child agent of Ai Austin for 60000ms in VRLand. Disconnecting.
17:12:17 - [CLIENT]: Close has been called for Ai Austin attached to scene VRLand.

On logging back I the viewer usually thinks I am still logged in.. and it needs a second attempt.


- Issue History
Date Modified Username Field Change
2019-01-16 01:46 aiaustin New Issue
2019-01-16 01:47 aiaustin Summary Child agents not closed cleanly when HG traveller exits grid - closes only after 60000ms no packets received timeout. => Child agents not closed cleanly when HG traveller exits grid - closes only after 60000ms no packets received timeout
2019-01-16 01:47 aiaustin Description Updated View Revisions
2019-01-16 01:51 aiaustin Description Updated View Revisions
2019-01-16 01:51 aiaustin Additional Information Updated View Revisions
2019-01-16 05:26 tampa Note Added: 0033819
2019-01-16 05:28 aiaustin Note Added: 0033820
2019-01-16 07:24 aiaustin Description Updated View Revisions
2019-01-18 07:32 aiaustin Note Added: 0033833
2019-02-01 05:49 aiaustin Note Added: 0034092
2019-02-01 05:49 aiaustin Note Edited: 0033833 View Revisions
2019-02-01 05:51 aiaustin Note Edited: 0034092 View Revisions
2019-02-01 05:52 aiaustin Note Edited: 0034092 View Revisions
2019-02-01 07:12 aiaustin Note Edited: 0034092 View Revisions
2019-02-01 08:59 aiaustin Note Added: 0034093
2019-02-01 09:00 aiaustin File Added: 2019-02-01-OSGrid-TP-Issue.txt
2019-02-01 09:03 aiaustin Note Edited: 0034093 View Revisions
2019-02-01 09:03 aiaustin Note Edited: 0034093 View Revisions
2019-02-01 09:05 aiaustin Note Edited: 0034093 View Revisions
2019-02-01 09:06 aiaustin Note Edited: 0034093 View Revisions
2019-02-01 09:09 aiaustin Note Edited: 0034093 View Revisions
2019-02-01 09:10 aiaustin Note Edited: 0034093 View Revisions
2019-02-01 09:16 aiaustin Note Edited: 0034093 View Revisions


Copyright © 2000 - 2012 MantisBT Group
Powered by Mantis Bugtracker