[Opensim-dev] BlockingQueue

[Justin Clark-Casey]

Stefan Andersson wrote:
>
> > > Preparing for the bug-a-thon friday, I want us to come together on 
> how
> > > we will write this out of the core; I think it will give us much
> > > better control over threading and deadlocking.
> > For people relatively new to the project, could you give any 
> examples of
> > particular problems here?
>
> Well, the thing with the BlockingQueue is that it functions this way:
>
> if something _Reads_ the queue, and it happens to be empty, the thread 
> is BLOCKED until something else _Adds_ to the queue. Of course, if 
> another thread comes along and reads from it, it is blocked as well, 
> and all of them are released in one merry explosion when an item is 
> Added. Which probably leads to all kinds of weird race conditions.
That was my reading of it.  At least in the case of assets, there's only 
one thread which actually reads the blocking queue atm (AssetCacheThread).
>  
> > >
> > > (The PROBLEM with the BlockingQueue is that threads might LOCK
> > > unknowingly because a sub-call felt like taking a peek at some
> > > blocking queue. Really really bad form.)
> > Could you be more specific? Looking at OpenSim.Framework.BlockingQueue,
> > I can't see where deadlock could occur - none of the methods hold on to
> > the m_queueSync lock for long. However, I'm relatively new to
> > synchronization in dot net, though I have done some in Java - I 
> could be
> > missing something.
>
> I don't mean deadlock as in ordinary lock{ } constructs, I mean the 
> use of mutexes can cause a 'wrong' thread to lock by (coder) mistake. 
> It's just a dumb effing construct to begin with.
>
> > >
> > > The BlockingQueue is used in the following contexts:
> > >
> > > * Serving AssetRequests
> > > - This should be solved with a specialized ThreadStart queue that
> > > news up threads whenever something tries to add to it or if there's
> > > something in the queue, up to a specified max of concurrent requests.
> > Isn't starting up threads an expensive process? Perhaps a ThreadPool
> > instead if we wanted to go down that route?
>
> Yes and No. Starting up threads is indeed 'expensive', but only in 
> relation to how big the work item is. For example, when transferring 
> an Asset, the cost of creating the thread is a fraction of the cost of 
> actually transferring it.
>
> It's exactly the over-excessive use of thread pools for long running 
> work items that's exhausting the thread pool as it is right now.
>
> Thread pools should be used only when you KNOW the work item is a 
> (relatively) small one.
>
> If we WANT thread re-use for this special case depicted above, we can 
> simply track and reuse the threads as they are finished.
afair, at the moment the thread processing the asset BlockingQueue 
doesn't send the packets directly, but rather chops them up and places 
them on the on PacketQueue.  I don't know if this is expensive or not.

--
justincc