[Opensim-dev] some scalability tests...

[Mic Bowman]

Grid mode. Connected to SciSim. Thanks to some help from Brian, we put the
Yellowstone National Park terrain on it today. I think Brian said it works
out to about a 1:10 scale. With the far view distance its pretty impressive
(terrain textures are borked). The performance is terrible, but good enough
to look around some. Even a completely empty region consumes resources and
1000 of them consume a LOT of resources. If you come over to sciencesim,
look for "geography11 00". I'll probably be restarting the regions to get
the map tiles updated & i'm not making any promises to keep it up very long.
But it does make for a very neat demonstration and its been a very useful
experiment.

--mic


On Fri, Jan 22, 2010 at 3:27 AM, Impalah Shenzhou <impalah at gmail.com> wrote:

> Just a stupid silly question: standalone or grid mode? If grid: where were
> the ugaim servers? Same machine?
>
> And a comment: 1024 regions? 8 hours for booting? Weird!
>
>
> 2010/1/22 Mic Bowman <cmickeyb at gmail.com>
>
>> this is just fyi... and a very positive comment about how far opensim has
>> come in recent months!
>>
>> as part of sizing the hw requirements for a mirror world project we're
>> exploring... we wanted to do some scalability tests on the capacity of
>> individual simulators in terms of the total number of regions. we wanted
>> baseline numbers that focus on just the most simple region configuration: a
>> completely empty region with default terrain. that is... this is JUST
>> simulator overhead.
>>
>> all tests are done on one of our blades... dual proc, quad core with 16G
>> ram running 64bit ubuntu. mono 2.6. and the tests are hosting 1024 regions
>> in a 32x32 grid. the simulator configuration was our standard sciencesim
>> config (XEngine, ODE, groups, wind, sun, etc).
>>
>> the first configuration ran all 1024 regions in *one* simulator. i
>> honestly figured this would crash quickly. it didn't. we managed to get all
>> 1024 regions created and running well enough to walk around. it did take
>> almost 8 hours to start. the first couple regions were created in 1-2
>> seconds each. by the end, it was taking 4-5 minutes per region. clearly
>> there is something quadratic in there (stop using linear lists, they are
>> evil! :-). but it could have been the mono garbage collector. who knows...
>> not sure its worth too much investigation because i can't imagine ever
>> running a config like this for real. the simulator did crash when we opened
>> the map in the viewer. the crash was caused because we ran out of sockets.
>> while it was running, the simulator used just over 10G of ram and was
>> running at about 700% CPU utilization (kind of scary to see load averages in
>> the 900 range!).
>>
>> the second configuration ran 16 simulators each with 64 regions. startup
>> took about 30 minutes (each of the 16 simulators avoided the quadratic
>> "knee" we hit with the one big simulator). consumed about 11G of memory and
>> was again consuming essentially all cycles on the machine (completely idle
>> regions aren't very idle). all 16 simulators died just after startup with a
>> "too many open files" error. not sure what caused it, but all of them died
>> loading the same terrain dll as part of some wildcard function looking for
>> dlls. that one is an interesting bug.
>>
>> the final configuration and the one we're shooting for in the short term
>> runs 16 simulators, each with an 8x8 megaregion. again startup was around 40
>> minutes. we might be able to cut that time down by starting all 16
>> simulators simultaneously rather than 4 at a time. i really just wanted to
>> make sure that some of the spikes we see in startup didn't cause failures
>> (some race condition causes all threads to consume maximum processor cycles
>> randomly on startup right now). and, well... it just worked. i figured the
>> viewer would die horribly (it can't handle 250K "real" prims very well) but
>> it survived just fine. turn off far clip with 8x8 megaregions providing
>> neighbors and you are "capable" of contacting simulators in a 24x24 region
>> range!. the view is pretty cool though it seems to not go beyond 12x12. :-)
>> there are a bunch of problems (like you can't see the terrain in the region
>> you're standing in)... but there are a LOT of little humps of terrain in
>> view. oh... and it takes a lot of patience to get everything loaded.
>>
>> so... the conclusion... this opensim thing is pretty amazing! good work
>> everyone.
>>
>> --mic
>>
>>
>> _______________________________________________
>> Opensim-dev mailing list
>> Opensim-dev at lists.berlios.de
>> https://lists.berlios.de/mailman/listinfo/opensim-dev
>>
>>
>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20100122/f72416e6/attachment-0001.html>