[Opensim-dev] Creation of 100+ users and TestClient Logins!
dz
dz at bitzend.net
Wed May 25 18:30:34 UTC 2011
Umar,
I have been using pretty standard TestClient bots to do "performance
testing"..
I have also used the existing command set to pre-program complex activities,
like a continuous 6 model runway fashion show for in-world video recording.
Existing TestClient
commands<http://lib.openmetaverse.org/wiki/TestClient_0.7.0_commands>
The existing commands I have found most useful to facilitate individual
control of the bot behavior are
setmaster - Lets you assign an inworld object that can send IM's to
the bot to initiate commands.
script - lets you maintain your command scripts in a directory
relative to the testclient executable.
Load testing generally consisted of 3 stages....
Generating and equipping the avatars.... The login credentials need to
be available, and the avatars need to represent the kind
of population you expect. Performance differences between default
avatars, and those with prim and/or HUD attachments are significant.
Logging in the Test population.. The avatar creation sequence is very
load intensive... TestClient bots are most prone to operation timeout
failures.
Logging in significant numbers of avatars at once will prevent any
significant number of successful logins.. Scripting this activity is the
fastest/easiest
way I have found to pre-load the test regions.. My experience has
been that running 10 bots per TestClient "console" provides an efficient
way to
balance the need to get bots logged in at a reasonable rate, and being
able to keep most of the logged bots active if a TestClient "console"
fails.
The final step of the login process is to have the bot touch an inworld
object... This object is scripted to use the touch_start event to
"register" the UUID
of the bot and act as its "master", by sending the "performance
action" commands via IM or initiating scripts to direct the avatar
Syncronizing the test... Once all the bots have been registered to the
controller devices... I log in to the region using a standard viewer.
By scripting the "controllers' to provide me with menu access to
commands, I can initiate and terminate group activities.
Combining the in-world object ability to send IM commands with the
TestClient access to remote scripts has allowed complex load testing
scenarios
to be designed quickly and repeated across multiple platform releases.
Some examples.....
Using 4 testclient "consoles" running on my home desktop, i have logged
in 40 testclient avatars to an opensim region hosted on a remote server and
integrated into another remote grid. I built a "reasonable" central
building of approximately 8000 prims. I divided the region into 4 quadrants
and logged
10 avatars into the approximate center of each quadrant.
Initially I had each of the 10 bots randomly chat, or perform an animation.
By varying the time between each of the operations I was able to get a feel
for the performance by attempting to move between the groups and
listen/watch for the success rates of the activites. Additional activities
like uploading images, sending IM messages grid wide, transferring and
collecting inventory items, and (now) teleporting to specific locations
around the grid were easily included in the scripts and provided excellent
simulations of real user loads.
With more experience with the available commands I was able to build
"walking" paths for the avatar groups to follow.. Adapting the existing
scripts I set a home point in respective corners for each of the avatars
and designed a simple square for them to walk in. It was then a simple
measure to watch as groups of 5 avatars would walk from one corner to the
next in relative synchronization until the region became "overloaded".. At
that point the avatars would either "miss" command and stop, or loose
connections with the region and I would see them walking off the edges or
disappearing from the scene.
I did not spend any time developing standardized "performance measurement"
mechanisms... Tests evolved from "How many can I get logged in and able to
register on a controller" to how long can I walk 40 bots around a region
before "something" fails. Of course my results are also specific to the
configuration of my desktop/home network/region server/grid service
connections.... so I can guarantee... Your mileage WILL vary..
On Wed, May 25, 2011 at 9:29 AM, Umar Farooq <u.farooq at uea.ac.uk> wrote:
> Dear Dan,
>
> Thanks! I tested the teleports (goto command) and it looks fine now.
>
> We are trying to determine the overall load of the system with the
> introduction of users, contents and their activities starting with an
> unpopulated world on a single simulator simulating a number of regions. I
> just need to know what methods could be used to determine the load based
> on these activities. I saw somewhere you mentioned about a ScienceSim
> patch that collects statistics, could you please let me know if I can get
> and use it.
>
> We would be waiting for the exciting commands to be incorporated into
> libomv TestClient.
>
> Best wishes and have a good day.
>
> Umar
>
> > Umar,
> >
> > There was a bug in TestClient where no region crossings would work. That
> > goes for the 'goto' command for teleporting as well as just walking
> across
> > a border with 'forward' or other movement commands. That bug was fixed
> > yesterday by lkalif and is in the latest public libomv svn source. I've
> > only tested it with walking across borders, not with the 'goto'
> teleports.
> >
> > For metrics, what question do you want to answer about OpenSim that you
> > are scaling the clients? That will partially determine which metrics you
> > need to capture. We have tried to find the bottlenecks to scaling a scene
> > (objects, scripts, physics, clients, etc) and wanted to know if CPU or
> > network bandwidth was the limiting factor. We have measured CPU and
> > bandwidth as well as line-level profiling of various OpenSim modules.
> >
> > Rather than releasing our TestClient patches directly, I am working with
> > the libopenmetaverse team to commit them to their public SVN. For now,
> the
> > public client has the best available peformance until I can merge our
> > commands to automate the wandering movements.
> >
> > ~Dan
> >
> >
> >
> > -----Original Message-----
> > From: opensim-dev-bounces at lists.berlios.de
> > [mailto:opensim-dev-bounces at lists.berlios.de] On Behalf Of Umar Farooq
> > Sent: Monday, May 23, 2011 2:13 AM
> > To: opensim-dev at lists.berlios.de
> > Subject: Re: [Opensim-dev] Creation of 100+ users and TestClient Logins!
> >
> > Dear Dan,
> >
> > I am using Robust, and running a private grid for my experiments. I have
> > noticed the issue of sequential execution of actions, and am using a
> > number of different TestClient instances using different connections.
> >
> > I have noticed that when we use goto or teleport command for bots and
> then
> > apply a command, the server is responds that an action is performed but
> > the bots are not updating in the scene, any comment on that?
> >
> > Which metric would be best to check load of a system? CPU usage, Network
> > Utilisation and anything else? Sorry if that is too stupid to ask!
> >
> > It would be great to have the code available whenever it is possible.
> > Thanks again for your help.
> >
> > Best wishes and have a great day.
> >
> > Umar
> >
> >> Umar,
> >> Are you using Simian or Robust to create your accounts? If the grid
> >> required authentication (email, captcha, etc) you will probably need to
> >> work with the grid operator to get test accounts. On Simian, it's easy
> >> to create test accounts if you can disable the authentications
> >> temporarily.
> >>
> >> As for logging them in, I suggest not putting more than about 25 bots
> >> in
> >> a single TestClient instance. There are many functions in TestClient
> >> which iterate over all bots sequentially performing an action and you
> >> will find them not responding to scene updates in time if you have too
> >> many in one executable.
> >>
> >> We are planning to make our modified TestClient available but it is
> >> not
> >> at this time. It's actually older than the released TestClient from
> >> libomv but it has a few commands for making bots perform interesting
> >> actions independently rather than all bots doing the same thing.
> >>
> >> ~Dan
> >>
> >>
> >> -----Original Message-----
> >> From: opensim-dev-bounces at lists.berlios.de
> >> [mailto:opensim-dev-bounces at lists.berlios.de] On Behalf Of Umar Farooq
> >> Sent: Wednesday, May 18, 2011 2:29 AM
> >> To: opensim-dev at lists.berlios.de
> >> Subject: [Opensim-dev] Creation of 100+ users and TestClient Logins!
> >>
> >> Dear All,
> >>
> >> To use TestClient for adding bots, it requires that the users are first
> >> created as authentication is required during login. Is there any simple
> >> way to create lots of users and then login them via a single command to
> >> the simulator? Thanks alot for your help and guidance!
> >>
> >> Best wishes,
> >>
> >> Umar
> >>
> >> --
> >> Umar Farooq
> >>
> >> Research Scholar,
> >> Virtual Humans Group,
> >> School of Computing Sciences,
> >> University of East Anglia,
> >> Norwich, United Kingdom.
> >>
> >> Cell: +4497853398184
> >>
> >> _______________________________________________
> >> Opensim-dev mailing list
> >> Opensim-dev at lists.berlios.de
> >> https://lists.berlios.de/mailman/listinfo/opensim-dev
> >> _______________________________________________
> >> Opensim-dev mailing list
> >> Opensim-dev at lists.berlios.de
> >> https://lists.berlios.de/mailman/listinfo/opensim-dev
> >>
> >
> >
> > --
> > Umar Farooq
> >
> > Research Scholar,
> > Virtual Humans Group,
> > School of Computing Sciences,
> > University of East Anglia,
> > Norwich, United Kingdom.
> >
> > Cell: +4497853398184
> >
> > _______________________________________________
> > Opensim-dev mailing list
> > Opensim-dev at lists.berlios.de
> > https://lists.berlios.de/mailman/listinfo/opensim-dev
> > _______________________________________________
> > Opensim-dev mailing list
> > Opensim-dev at lists.berlios.de
> > https://lists.berlios.de/mailman/listinfo/opensim-dev
> >
>
>
> --
> Umar Farooq
>
> Research Scholar,
> Virtual Humans Group,
> School of Computing Sciences,
> University of East Anglia,
> Norwich, United Kingdom.
>
> Cell: +4497853398184
>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20110525/a6ae5e92/attachment-0001.html>
More information about the Opensim-dev
mailing list