[Opensim-dev] Lockless Lists?

[Hurliman, John]

> -----Original Message-----
> From: opensim-dev-bounces at lists.berlios.de [mailto:opensim-dev-
> bounces at lists.berlios.de] On Behalf Of Hurliman, John
> Sent: Tuesday, November 25, 2008 5:19 PM
> To: opensim-dev at lists.berlios.de; homer.horwitz at gmail.com
> Subject: Re: [Opensim-dev] Lockless Lists?
>
>
>
> -----Original Message-----
> From: opensim-dev-bounces at lists.berlios.de [mailto:opensim-dev-
> bounces at lists.berlios.de] On Behalf Of Christopher Yeoh
> Sent: Tuesday, November 25, 2008 3:26 PM
> To: homer.horwitz at gmail.com; opensim-dev at lists.berlios.de
> Subject: Re: [Opensim-dev] Lockless Lists?
>
> On Mon, 24 Nov 2008 20:43:09 +0100
> "Homer Horwitz" <homerhorwitz at googlemail.com> wrote:
>>
>> So, I'm not sure if we really should do that move. If at all, I'm
>> for a very slow move to lock-free versions from a rather stable
>> software base (which we currently don't have in trunk), so errors
>> that are introduced during that move are more easily identifiable,
>> with much testing in-between. Even then, I'm absolutely sure we will
>> get a lot of Heisenbugs in the process, which will take us weeks to find.
>>
>
> Perhaps what is really needed here is some performance benchmarks
> which highlight existing problems? So for individual changes to
> lockless versions we better see what improvements we'd get on both
> small and large SMP machines and whether its worth the increase in complexity.
>
> I did some debugging of a deadlock a couple of weeks ago and found it
> already pretty complicated. Any suggestions on how other people approach
> these problems with OpenSim? I sprinkled lots of console messages around
> as mdb doesn't seem to work for me, but in retrospect it would have been
> really handy to have been able to just turn on a lock debugging flag and
> have debug output when locks are taken and released.
>
> Regards,
>
> Chris
>
>
> One idea I've tossed around is using Mono.Cecil to inject some code
> for profiling locks. Since
>
> lock (lockObject)
> {
>       ...
> }
>
> Gets expanded out by the compiler to:
>
> Monitor.Enter(lockObject);
> try
> {
>       ...
> }
> finally
> {
>       Monitor.Exit(lockObject);
> }
>
> It shouldn't be too difficult to follow up any Monitor.Enter() calls
> by storing current ticks, and do a check in the finally if current
> ticks - stored ticks > lock_warn_const. Then we could scan any binary
> for long- held locks.
>
> John


Turns out that it was as easy as I had thought. Download at http://www.jhurliman.org/download/LockTime-1.0.zip (I'll have a blog post describing it in more detail up on Monday). I ran it on OpenSim.Region.Environment.dll and while there are definitely a lot of locks (approximately 350, with at least one lock being held and released multiple times per second while the system is idle) I didn't find any that were being held for >20ms. There are a lot of assemblies in OpenSim to test though.

John