[Opensim-dev] BlockingQueue

[Stefan Andersson]

This is all true. What CAN and HAVE happened, is that somebody not aware that a method calls a method that calls a method that reads from the Blocking Queue suddenly finds herself waiting for somebody else to put an item into the queue. This kind of thread interdependency can lead to VERY hard-to-debug scenarios.

Which, incidentally, gets even worse since it's only one thread that is released at a time.

/Stefan




To: opensim-dev at lists.berlios.de
From: alan_webb at us.ibm.com
Date: Wed, 26 Mar 2008 11:03:21 -0400
Subject: Re: [Opensim-dev] BlockingQueue



The BlockingQueue uses Monitor.Pulse
which only releases ONE thread for execution. The others remain suspended.
PulseAll is needed if you want to awaken all the waiting threads. So the
race condition that you describe should not happen (if Mono corrcetly implements
Monitor anway).



The queue AND the lock are private to
the BlockingQueue class, so the only unsynchronized mechanism for  examining
the queue is GetQueueArray which returns a copy of the content. In principle
It isn't possible for anyone to damage the queue. The lock cannot be obtained
by anyone not using the proper BlockingQueue accessors.



Best regards

Alan

-------------------

T.J. Watson Research Center, Hawthorne, NY

1-914-784-7286

alan_webb at us.ibm.com








Justin Clark-Casey <jjustincc at googlemail.com>


Sent by: opensim-dev-bounces at lists.berlios.de
03/26/2008 10:43 AM



Please respond to

opensim-dev at lists.berlios.de






To
opensim-dev at lists.berlios.de


cc



Subject
Re: [Opensim-dev] BlockingQueue













Stefan Andersson wrote:

>

> > > Preparing for the bug-a-thon friday, I want us to come together
on 

> how

> > > we will write this out of the core; I think it will give
us much

> > > better control over threading and deadlocking.

> > For people relatively new to the project, could you give any


> examples of

> > particular problems here?

>

> Well, the thing with the BlockingQueue is that it functions this way:

>

> if something _Reads_ the queue, and it happens to be empty, the thread


> is BLOCKED until something else _Adds_ to the queue. Of course, if


> another thread comes along and reads from it, it is blocked as well,


> and all of them are released in one merry explosion when an item is


> Added. Which probably leads to all kinds of weird race conditions.

That was my reading of it.  At least in the case of assets, there's
only 

one thread which actually reads the blocking queue atm (AssetCacheThread).

>  

> > >

> > > (The PROBLEM with the BlockingQueue is that threads might
LOCK

> > > unknowingly because a sub-call felt like taking a peek at
some

> > > blocking queue. Really really bad form.)

> > Could you be more specific? Looking at OpenSim.Framework.BlockingQueue,

> > I can't see where deadlock could occur - none of the methods
hold on to

> > the m_queueSync lock for long. However, I'm relatively new to

> > synchronization in dot net, though I have done some in Java -
I 

> could be

> > missing something.

>

> I don't mean deadlock as in ordinary lock{ } constructs, I mean the


> use of mutexes can cause a 'wrong' thread to lock by (coder) mistake.


> It's just a dumb effing construct to begin with.

>

> > >

> > > The BlockingQueue is used in the following contexts:

> > >

> > > * Serving AssetRequests

> > > - This should be solved with a specialized ThreadStart queue
that

> > > news up threads whenever something tries to add to it or
if there's

> > > something in the queue, up to a specified max of concurrent
requests.

> > Isn't starting up threads an expensive process? Perhaps a ThreadPool

> > instead if we wanted to go down that route?

>

> Yes and No. Starting up threads is indeed 'expensive', but only in


> relation to how big the work item is. For example, when transferring


> an Asset, the cost of creating the thread is a fraction of the cost
of 

> actually transferring it.

>

> It's exactly the over-excessive use of thread pools for long running


> work items that's exhausting the thread pool as it is right now.

>

> Thread pools should be used only when you KNOW the work item is a


> (relatively) small one.

>

> If we WANT thread re-use for this special case depicted above, we
can 

> simply track and reuse the threads as they are finished.

afair, at the moment the thread processing the asset BlockingQueue 

doesn't send the packets directly, but rather chops them up and places


them on the on PacketQueue.  I don't know if this is expensive or
not.



--

justincc

_______________________________________________

Opensim-dev mailing list

Opensim-dev at lists.berlios.de

https://lists.berlios.de/mailman/listinfo/opensim-dev



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20080326/e1df6b60/attachment-0001.html>