[Opensim-dev] Behaviour of adaptive throttles under high load

[Mic Bowman]

Right... but if the problem is that the simulator is overloaded... then the
solution is not to worry about the adaptive throttles but to look at the
overall bw cap on the simulator. If all connections are suffering
degradation then *all* connections are dropping packets and that means the
problem is the simulator can't keep up. In that situation... no matter what
you do with the adaptive throttles, you're screwed.

My point is... its easy to point fingers but it sounds like we don't have
much information about the root cause of the problem. Turning off adaptive
throttles for high load is fine. But they are there for a purpose. Long
haul links to simulators were pretty much unusable before we added the
throttles (which was 4? years ago).

--mic


On Tue, Nov 25, 2014 at 9:37 PM, Mister Blue <misterblue at misterblue.com>
wrote:

> Could be that network congestion is throwing away a block of UDP packets
> which becomes a block of un-acked messages which causes multiple divide
> bandwidth by two? Most measurements I've seen show that UDP packet
> discarding happens in bursts.
>
>
> On Tue, Nov 25, 2014 at 6:39 PM, Mic Bowman <cmickeyb at gmail.com> wrote:
>
>> As you mention... cutting the throttle by 50% was modeled on the TCP
>> congestion control approach. It is very aggressive as a congestion control
>> mechanism and certainly could be tuned.
>>
>> That being said... do you know why the packets were considered un-acked?
>> If its because the simulator is having problems (which given your
>> description that it happens under load seems to be the case) then we can
>> probably do something more intelligent about throttling over all simulator
>> BW. That is... maybe the problem is that the top end of the overall
>> simulator bw is the problem, not the per connection throttles.
>>
>> Manual throttles & adaptive throttles are not exclusive. You can use
>> both. Adaptive manages the top end, but the manual throttles set an
>> absolute max.
>>
>> --mic
>>
>>
>> On Tue, Nov 25, 2014 at 5:15 PM, Justin Clark-Casey <
>> jjustincc at googlemail.com> wrote:
>>
>>> Hi Mic (primarily),
>>>
>>> Two years ago [1] we had a discussion about the
>>> enable_adaptive_throttles setting.  Just for background, this is a setting
>>> that adapts the amount of data sent to the viewer depending on whether
>>> reliable packets sent from the simulator are acked or not.  As such, it
>>> looks to make sure that a viewer which sets a downstream bandwidth higher
>>> than its network connection can cope with is not permanently hosed with too
>>> much data.  We enabled it on an experimental basis [2].
>>>
>>> As you said at the time, this is modelled on the congestion approach
>>> used in TCP.  I see that for TCP, the rate is halved on every unacked
>>> segment.  In OpenSimulator, it's halved on every unacked reliable packet.
>>>
>>> However, under fairly modest load conditions in the conference grid, I
>>> saw a behaviour where sometimes for a connection a sequence of packets
>>> would expire for some connections in a very short time period (< 1 sec).
>>> This would halve the throttle many times, in my observations right down to
>>> the absolute minimum.  This caused the behaviour from the user's point of
>>> view to degrade considerably for an extended period of time.  The throttles
>>> takes quite a long period to grow again.
>>>
>>> I didn't get much further with the diagnostics since a lack of time
>>> forced us to switch back to manual throttling instead (with a 1 mbit per
>>> viewer and 400 mbit total on the keynotes).  This seemed to work okay in
>>> testing and in the event itself.  However, this leaves one vulnerable to
>>> the problem adaptive_throttles looks to tackle in the first place.
>>>
>>> I'm still reading up about this stuff, but it strikes me that halving
>>> the throttle on every missed packet is much harsher than the TCP approach,
>>> as with UDP a whole sequence can expire at once rather than a single
>>> segment that is subsequently retried before another segment can be missed.
>>>
>>> One idea is to ignore all expiries in a certain period (e.g. next 2
>>> seconds) if an expired packet has already caused the throttle to be
>>> halved.  Of course, this is a bit more complicated to do but hopefully not
>>> too much so.  What do you think?  Any other ideas?
>>>
>>> [1] http://opensimulator.org/pipermail/opensim-dev/2011-
>>> October/023017.html
>>> [2] http://opensimulator.org/pipermail/opensim-dev/2011-
>>> October/023063.html
>>>
>>> Best Regards,
>>>
>>> --
>>> Justin Clark-Casey (justincc)
>>> OSVW Consulting
>>> http://justincc.org
>>> http://twitter.com/justincc
>>> _______________________________________________
>>> Opensim-dev mailing list
>>> Opensim-dev at opensimulator.org
>>> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
>>>
>>
>>
>> _______________________________________________
>> Opensim-dev mailing list
>> Opensim-dev at opensimulator.org
>> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
>>
>>
>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at opensimulator.org
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20141126/17b05e64/attachment.html>