Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0004085opensim[REGION] OpenSim Corepublic2009-09-01 11:392011-08-15 01:42
Reporterjhurliman 
Assigned To 
PrioritynormalSeveritycrashReproducibilitysometimes
StatusclosedResolutionsuspended 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version 
Summary0004085: [PATCH] Asynchronous calls in OpenSim are leaking thread handles on some versions of Mono
DescriptionSee https://bugzilla.novell.com/show_bug.cgi?id=320983 [^] and https://bugzilla.novell.com/show_bug.cgi?id=321772 [^] for the back story. Many versions of Mono, even up to the very recent 2.4.2.3 depending on what threading backend you use, have this problem. Part of it is technically not a bug because the MSDN docs state that a call to BeginInvoke() must conclude with a call to EndInvoke(), since some CLR implementations use the EndInvoke() call to trigger a cleanup of the thread resources. However, I have seen the dreaded "System.ExecutionEngineException: Couldn't create thread" from calls to ThreadPool.QueueUserWorkItem(), which means there is an additional Mono bug when the thread pool is starved on top of the leaky thread handle problem.

I will be attaching a set of patches to work around this issue. The first patch adds a new method to the OpenSim Util class called FireAndForget(), which should replace all .BeginInvoke() calls in the OpenSim codebase.
Additional InformationIf you have been running UGAIM services or one or more regions on a machine for at least a few days, try running:

mono --wapi=hps

On the command line (running as the same user that OpenSim is running as). If this list seems significantly longer than three times the number of running Mono processes, it's likely you are leaking thread handles and will eventually run out of system resources. If this happens, make sure to rm -rf ~/.wapi/ before restarting the servers.
Tagsthread
Git Revision or version number0
Run ModeStandalone (1 Region) , Standalone (Multiple Regions) , Grid (1 Region per Sim) , Grid (Multiple Regions per Sim)
Physics EngineBasicPhysics, PhysicsOfSimplicity, ODE, BulletX, PhysX
Script Engine
EnvironmentMono / Linux32, Mono / Linux64
Mono Version2.4.2
Viewer
Attached Filespatch file icon Added-Util.FireAndForget-to-replace-leaking-calls-to.patch [^] (1,390 bytes) 2009-09-01 11:39 [Show Content]
patch file icon Fixes-seven-leaky-.BeginInvoke-calls.patch [^] (5,192 bytes) 2009-09-01 12:08 [Show Content]

- Relationships

-  Notes
(0013107)
jhurliman (manager)
2009-09-01 12:11

Just uploaded the second half of the patch which fixes all of the leaky BeginInvoke calls I could find. I tried to touch as little as possible with this patch, so most of them have not been converted to the Util.FireAndForget() interface. However, in the future it would be a good idea to use that function to avoid scattering lots of little functions like:

private void PurgeFolderCompleted(IAsyncResult iar)
{
    PurgeFolderDelegate d = (PurgeFolderDelegate)iar.AsyncState;
    d.EndInvoke(iar);
}

around the codebase, which don't always get created or filled out properly leading to threading problems.
(0013522)
Fly-Man- (developer)
2009-09-19 07:01

This issue was automatically checked by Mantis Issue Check:

Patch is available on this issue
(0014399)
justincc (administrator)
2009-12-04 10:20

John, is this patch relevant any more?
(0019580)
makopoppo (manager)
2011-08-15 01:42

Since no reply for over an year, and Mono 2.4.2 is no longer supported, closing it for now.

- Issue History
Date Modified Username Field Change
2009-09-01 11:39 jhurliman New Issue
2009-09-01 11:39 jhurliman File Added: Added-Util.FireAndForget-to-replace-leaking-calls-to.patch
2009-09-01 11:39 jhurliman Git Revision => 0
2009-09-01 11:39 jhurliman SVN Revision => 0
2009-09-01 11:39 jhurliman Run Mode => Standalone (1 Region) , Standalone (Multiple Regions) , Grid (1 Region per Sim) , Grid (Multiple Regions per Sim)
2009-09-01 11:39 jhurliman Physics Engine => BasicPhysics, PhysicsOfSimplicity, ODE, BulletX, PhysX
2009-09-01 11:39 jhurliman Environment => Mono / Linux32, Mono / Linux64
2009-09-01 11:39 jhurliman Mono Version => 2.4.2
2009-09-01 12:08 jhurliman File Added: Fixes-seven-leaky-.BeginInvoke-calls.patch
2009-09-01 12:11 jhurliman Note Added: 0013107
2009-09-01 12:16 jhurliman Status new => patch included
2009-09-01 12:17 jhurliman Tag Attached: thread
2009-09-19 07:01 Fly-Man- Summary Asynchronous calls in OpenSim are leaking thread handles on some versions of Mono => [PATCH] Asynchronous calls in OpenSim are leaking thread handles on some versions of Mono
2009-09-19 07:01 Fly-Man- Note Added: 0013522
2009-12-04 10:20 justincc Note Added: 0014399
2009-12-04 10:20 justincc Status patch included => patch feedback
2011-08-15 01:42 makopoppo Note Added: 0019580
2011-08-15 01:42 makopoppo Status patch feedback => closed
2011-08-15 01:42 makopoppo Resolution open => suspended


Copyright © 2000 - 2012 MantisBT Group
Powered by Mantis Bugtracker