[Opensim-dev] Rate Limit DOS Protection

Tue Oct 8 16:43:18 UTC 2013

I had a look at the Robust.ini.example earlier and I found the instructions
to be clear and straight forward, I am glad you did infact allow this to be
completely disabled, at OSgrid we use nginx as a reverse proxy and also to
do DoS protection and load balancing, without the options to disable this
completely it would have been a major problem moving forward.

On Tue, Oct 8, 2013 at 12:27 PM, Teravus Ovares <teravus at gmail.com> wrote:

> Just to be clear, and comment/answer some questions.
>
> I did add the DOS protection variables to the Robust.ini.example in
> the [LoginService] section.  The Rate Limit code does 'per connection'
> velocity counts.   The default rate setting is 5 requests in 10
> seconds.    If you have a transparent proxy, such as squid, you need
> to set DOSAllowXForwardedForHeader to true so that the code trusts the
> X-Forwarded-For header that your proxy adds to the headers.    If you
> want to turn it off, set DOSMaxRequestsInTimeFrame = 0.       If you
> want to allow individual clients to do 20 connections in 5 seconds,
> set DOSMaxRequestsInTimeFrame = 20 and   DOSRequestTimeFrameMS = 5000.
>   If you want to change how long individual connections are blocked
> when they go over the rate limit, change DOSForgiveClientAfterMS.
>
> Hopefully this helps get you started with the config options.
>
> One more thing, this DOS protector is configured and implemented 'per
> service',    So, if it's implemented in the login service, it's not
> going to get triggered by the Group Service.   If there's DOS
> protection on the Group Service and the login service, and a
> connection gets blocked on the login service, they'll still be able to
> access the Group Service because they're individually rate limited...
>  This is for flexibility.    Choosing what the best rate limit is
> cannot really be done server wide, it has to be based on the needs of
> the individual service and that's why it was implemented this way.
> At this point, the only major service that has rate limiting on by
> default is the login service.
>
> I'll be happy to answer more questions and discuss default settings.
>
>
>  Best Regards
>
> Teravus
>
>
> On Tue, Oct 8, 2013 at 8:04 AM, James Stallings II
> <james.stallings at gmail.com> wrote:
> > As I have often been told by you, Melanie, if I am upgrading and not
> > auditing, adjusting, and testing my configs in accordance with changes,
> I'm
> > doing it wrong.
> >
> > I think that statement probably applies more to those running larger
> > concerns than anyone else.
> >
> > Honestly, Teravus typically does a better job of coding than to produce
> work
> > that does not take such matters of scale into consideration.
> >
> > I'm comfortable with whatever he chooses to do, and if it turns out it
> isn't
> > a case of 'one size fits all', I'll make adjustments accordingly.
> >
> >
> > On Tue, Oct 8, 2013 at 7:52 AM, Melanie <melanie at t-data.com> wrote:
> >>
> >> I'm worried that people with larger installations will see service
> >> failures because legit traffic is seen as abusive. This could cause
> issues
> >> for the lerger grids out there. I don't believe that whatever tenuous
> >> protection this may offer for small grids and standalones outwieghs the
> >> potential service impairment it may cause for unsuspecting larger
> grids. Not
> >> every grid operator reads this list,
> >>
> >> So I'd again suggest that we stick to the way we've always done it and
> >> make the default for new features be "off".
> >>
> >> Melanie
> >>
> >> On 8 Oct 2013, at 09:31, Teravus Ovares <teravus at gmail.com> wrote:
> >>
> >> I understand what you're saying.      It's hard to argue to leave
> >> people unprotected from attacks, though.    I'm certainly open to
> >> making the defaults less protective, and, I'm concerned enough about
> >> it that I'd prefer to leave some protection in place there.
> >>
> >> What are your thoughts on that?
> >>
> >> Best Regards
> >>
> >> Teravus
> >>
> >> On Tue, Oct 8, 2013 at 12:41 AM, Melanie <melanie at t-data.com> wrote:
> >> > Hi,
> >> >
> >> > in keeping with our SOP, the defaults provided should be emulating
> >> > the previous behavior, e.g. NO rate limiting.
> >> >
> >> > I would much appreciate if that procedure would be adhered to,
> >> > unless we vote to abandon it. Users could suffer because they don't
> >> > expect the default config to change on them.
> >> >
> >> > Cheers,
> >> >
> >> > Melanie
> >> >
> >> > On 08/10/2013 05:42, Teravus Ovares wrote:
> >> >> Hi there,
> >> >>
> >> >> I just wanted to inform -dev that I added some rate limiting DOS
> >> >> protection classes to use to protect your opensim based services from
> >> >> rapid calling.      At the moment, this will be most noticeable in
> the
> >> >> Login Service.    I have, both as an example, and good practice,
> >> >> applied the Rate limit protection to the login service.    There are
> >> >> new Configuration options in StandaloneCommon.ini and Robust.ini that
> >> >> control how the connections are rate limited and if trusts the
> >> >> X-Forwarded-For header.    Just for the sake of getting something up
> >> >> there, I set the defaults to something sane, however they may not
> work
> >> >> for everyone, so it may be wise to take a look at the new
> >> >> configuration options in the [LoginService] section of your
> >> >> bin/Robust.ini.example and
> >> >> /bin/config-include/StandaloneCommon.ini.example AND/OR have
> >> >> discussions on what would be more sane default options.   There's a
> >> >> chance that this could affect anyone, so don't neglect to take a look
> >> >> at it.
> >> >>
> >> >> You may also notice messages on your console and in your logs like:
> >> >> 21:56:29 - [LOGINDOSPROTECTION]: client: 192.168.1.213 is blocked for
> >> >> 120000 milliseconds, X-ForwardedForAllowed status is False,
> >> >> endpoint:192.168.1.213
> >> >>
> >> >> This is an example of the DOS Protection blocking a connection
> because
> >> >> the client went beyond the rate limit.
> >> >>
> >> >> The rate limit is defined by X requests in Y period of time and is
> >> >> implemented in a rolling Y fashion.   It also has a 'forget' period
> of
> >> >> time that will unblock the blocked user.
> >> >>
> >> >> At this point, there's one implemented for XMLRPC handlers, one for
> >> >> GenericHTTPHandlers and a base class for StreamHandlers based on
> >> >> BaseStreamHandler.
> >> >>
> >> >> If you are interested in the code changes, you can check the diff:
> >> >>
> >> >>
> http://opensimulator.org/viewgit/?a=commitdiff&p=opensim&h=f76cc6036ebf446553ee5201321879538dafe3b2
> >> >>
> >> >> There's still more to do, and, here's a start to providing some
> >> >> modicum of protection on the services.
> >> >>
> >> >> If you have any questions, feel free to reply and ask..  or send me
> an
> >> >> e-mail personally.
> >> >>
> >> >> Thanks and Best Regards
> >> >>
> >> >> Teravus
> >> >> _______________________________________________
> >> >> Opensim-dev mailing list
> >> >> Opensim-dev at lists.berlios.de
> >> >> https://lists.berlios.de/mailman/listinfo/opensim-dev
> >> > _______________________________________________
> >> > Opensim-dev mailing list
> >> > Opensim-dev at lists.berlios.de
> >> > https://lists.berlios.de/mailman/listinfo/opensim-dev
> >> _______________________________________________
> >> Opensim-dev mailing list
> >> Opensim-dev at lists.berlios.de
> >> https://lists.berlios.de/mailman/listinfo/opensim-dev
> >>
> >>
> >> _______________________________________________
> >> Opensim-dev mailing list
> >> Opensim-dev at lists.berlios.de
> >> https://lists.berlios.de/mailman/listinfo/opensim-dev
> >
> >
> >
> >
> > --
> > ===================================
> > http://osgrid.org/
> > http://simhost.com
> > http://twitter.com/jstallings2
> >
> >
> > _______________________________________________
> > Opensim-dev mailing list
> > Opensim-dev at lists.berlios.de
> > https://lists.berlios.de/mailman/listinfo/opensim-dev
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev
>

-- 
Michael Emory Cerquoni
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20131008/a5f275eb/attachment-0001.html>