[Opensim-dev] Building a test grid

[Justin Clark-Casey]

On 20/07/11 17:52, Wes wrote:
> I have three servers at my disposal. A server with 48GB RAM and two quad core Intel processors, a server with 12GB RAM
> and two quad core Intel processors, and a server with 8GB RAM and two Intel Xeon processors. I have enough publicly
> routable IPs to put them all out on the Internet for testing and decent bandwidth. With these three servers to work
> with, which one should do what role in an OpenSim grid mode install that is hypergrid enabled? How many users could I
> potentially support if there was enough bandwidth? Thanks!

A question with a lot of possible answers.  Make the most powerful server run the robust instances.  To be honest, the 
built-in C# http server doesn't handle heavy traffic at all well, you will end up needing multiple instances of asset 
and inventory and user services as the grid grows, with a load balancer frontend such as nginx.

Ultimately, for a large long term grid one might want to consider something like sras (https://github.com/coyled/sras) 
for asset service, mainly because it doesn't store dupe assets, stores things on the filesystem (mysql gets squirrely 
with a huge number of asset blobs) and runs within apache, which performs far far better.  However, this kind of set up 
is right on the edge and open-source work to increase capacities is in its infancy.

Ultimately, simply running all services out of a PHP + apache instance might be the best solution for performance, with 
load balancing again over multiple servers if needed.  Bottleneck becomes backend database or filesystem.  People have 
done this in the past but systems have not been maintained due to rapidly changing services, though this has settled 
down a lot recently.

Problems are similar to running large scale websites (if not more extreme).  These problems receive a lot of work from a 
lot of clever people.

Max users really depends on how they are spread out throughout regions.  Everybody on one simulator is one bottleneck, 
not sure exactly where it is but may well be scene update and physics loops.  Intel's distributed scene graph work aims 
to distribute this kind of load.  Another bottle neck is user services - short term solution here is to spawn multiple 
services.

-- 
Justin Clark-Casey (justincc)
http://justincc.org/blog
http://twitter.com/justincc