[Opensim-users] Load Testing OpenSim

Thu Jun 24 21:02:06 UTC 2010

While I agree with your point that there are many aspects of performance
that need to be tested, your quote raises in interesting issue:

 "I don't have good scenarios of what large numbers of avatars
realistically do in a region."

This would seem to be the essential key to this kind of "stress test".

The large group of users congregating in one place does tend to have a lot
of visibility especially if perceived performance (e.g. lag) is observed to
degrade.

I have to wonder if there is not some way to observe and gather this
information in some objective way, if not for the current round of testing
then for some future use.

Karen

On Thu, Jun 24, 2010 at 11:05 AM, dz <dz at bitzend.net> wrote:

> Aloha!
>
> As someone who also uses bots to do a lot of my testing, I thought I'd
> share my $.02....
>
> I'm not going to comment a lot about the overall usefulness of small scale
> Load Tests.....  MrBlue had it right a couple posts ago when he said...
>
> Testing the load of a large number of clients requires a lot of
> infrastructure -- multiple client systems and a lot of networking. That
> makes stress testing difficult to set up and to create a portable, automated
>
> stress tester.
>
> Proper tests don't just pick one aspect of performance ( how many agents
> does my console report in the region), but are designed to see how many
> pieces in the complex system interact under stress. Another difficult aspect
> of generalized tests is identifying which level of performance is
> acceptable.   Load testing Currency Transfer routines is vastly different
> from making sure all 4500 of your particles are properly rendered at 48
> meters....  In this same way,  bots are NOT graphical viewers,  but they are
> usefull tools for interacting with virtual spaces...so efforts to make them
> better help us all.
>
> That being said,  I use both *TestClient*  (
> http://lib.openmetaverse.org/wiki/TestClient ) and  *Radegast* (
> http://radegastclient.org/wp/ ) based bots for my testing.   Radegast is
> my favorite for a light weight client that lets me access a bot when I want
> to interact with others or inventory.  All of the scripted testing I do is
> via a TestClient based bot.  There are a number of reasons for this...
> First.  it is simple to approximate loads close to "real" avatars.   Like
> MrBlue says...
>
> I have found that TestClient bots that do something (walk around, chat), in
> aggregate, give a load pretty close to a "normal" viewer load.  I have been
> using scripts that start up groups of bots that walk around (waypoint to
>
> waypoint) but I don't have good scenarios of what large numbers of avatars
> realistically do in a region.
>
> Second..  The scripting option makes it easier to devise simple tests.
> When I have a script that uploads 100 textures, it can be repeated, timed,
> and its success documented as the number of failed uploads.  It can also be
> combined with scripts designed to stress other capabilities we expect as
> users.
>
>
> Here are some of the "tests"  I have designed and used.
>
> Logins -  automated login and logout of clients.  I have found that the
> TestClient command /console gets cumbersome with more than 10 bots per
> client.   I have scripts to log in groups of 10 and groups of 5.  Testing
> involves logging in n groups of 5 or 10 to reach a specified avatar count.
> Once they log in they TP to a particular place in the region and rebake
> thier appearance. Performance is measured in % of successful logins
> completed, Time for all the logins to complete, and the % of appearance
> rebakes that complete.  If you just do THIS test to a remote server, you
> will understand quickly that the only thing really stressed is your modem
> and the network traffic police at your ISP,  Unless they charge you by the
> Byte...
>
> Random Chat - Login 4 groups of 10 bots.   Position the groups in 4 corners
> of the region.  Seat each of the bots on a poseball designed to switch
> between poses every 25 seconds starting from when the bot sits down.  Every
> time the pose changes, have the pose ball chat the name of the new pose.
> This simulates groups of avatars changing animations and the load associated
> with delivering and filtering chat messages.  Since it is difficult to
> assign particular Success/Failure rates to this, I use it as "background
> noise"...  How many bots can I have in region doing this while I can still
> do "my stuff"
>
> Texture uploads - Create a scripted object in world to act as a timer.
> Add loop in the timer event to send an IM to each bot, requesting it do a
> texture upload.  Clone 1 for each of the bot groups.  Login 4 groups of 10
> bots, sit them on the Random Chat poseballs  and have them accept IM
> commands from the scripted prim.   Set the timer for 15 seconds,  this
> should result in each bot getting an IM to upload a texture ( remember each
> IM takes a second ).  Run this test over time...  It is simulating Asset
> Database Additions,  Inventory updates,  In region IMs, inter-agent chats.
> If you code your upload scripts right,  this can also be one way to get your
> huge collection of textures loaded to a new grid :)
>
> Walking paths -  login groups of various sizes to corners of the region,
> script them to walk in sync with their group from corner to corner in loops
> while either adding more walking avatars or loads of other types.  I usually
> start with groups of 10 in each corner.  They are lined up on spots 1 meter
> apart, their HOME location set, and then the whole group is commanded in
> sequence to walk the same path.  after I get 40 avatars walking around the
> outer edge of the region, I add groups of 5 to a path closer to the center
> of the region and send the next 20 avatars walking around the loop.  There
> is something "just right" about seeing the lines of bots flowing around the
> edge of the region, and its easy to see when things "break down"...
> commands get delayed, and bots get off-track.  You start to lag and watch
> them walk off the edge of the region into never never land.   The lag gets
> so severe that it takes bots 5-10 times the amount of time it should to walk
> from corner to corner...    Just makes you want to be a Linden >wink<
>
>
> As a final word...   There is important work to be in the "Capabilities"
> testing space that might be more productive at this point in the OpenSim
> development cycle.   IMHO, individuals are a bigger help to the development
> community if they spend more time building things they want in empty
> regions, and pushing the platform capabilities, rather than testing the
> capacity of an architecture that is undergoing revolutionary change..
>
>
> _______________________________________________
> Opensim-users mailing list
> Opensim-users at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-users/attachments/20100624/1cce67fc/attachment.html>