[Opensim-dev] Creation of 100+ users and TestClient Logins!
Umar Farooq
u.farooq at uea.ac.uk
Fri May 27 18:18:43 UTC 2011
Thanks DZ for your detailed guidance!
> Umar,
>
> I have been using pretty standard TestClient bots to do "performance
> testing"..
>
> I have also used the existing command set to pre-program complex
> activities,
> like a continuous 6 model runway fashion show for in-world video
> recording.
>
> Existing TestClient
> commands<http://lib.openmetaverse.org/wiki/TestClient_0.7.0_commands>
>
> The existing commands I have found most useful to facilitate individual
> control of the bot behavior are
>
> setmaster - Lets you assign an inworld object that can send IM's
> to
> the bot to initiate commands.
>
> script - lets you maintain your command scripts in a directory
> relative to the testclient executable.
>
> Load testing generally consisted of 3 stages....
>
> Generating and equipping the avatars.... The login credentials need
> to
> be available, and the avatars need to represent the kind
> of population you expect. Performance differences between default
> avatars, and those with prim and/or HUD attachments are significant.
>
> Logging in the Test population.. The avatar creation sequence is
> very
> load intensive... TestClient bots are most prone to operation timeout
> failures.
> Logging in significant numbers of avatars at once will prevent any
> significant number of successful logins.. Scripting this activity is
> the
> fastest/easiest
> way I have found to pre-load the test regions.. My experience has
> been that running 10 bots per TestClient "console" provides an efficient
> way to
> balance the need to get bots logged in at a reasonable rate, and
> being
> able to keep most of the logged bots active if a TestClient "console"
> fails.
>
> The final step of the login process is to have the bot touch an
> inworld
> object... This object is scripted to use the touch_start event to
> "register" the UUID
> of the bot and act as its "master", by sending the "performance
> action" commands via IM or initiating scripts to direct the avatar
>
> Syncronizing the test... Once all the bots have been registered to
> the
> controller devices... I log in to the region using a standard viewer.
> By scripting the "controllers' to provide me with menu access to
> commands, I can initiate and terminate group activities.
> Combining the in-world object ability to send IM commands with the
> TestClient access to remote scripts has allowed complex load testing
> scenarios
> to be designed quickly and repeated across multiple platform
> releases.
>
>
> Some examples.....
>
> Using 4 testclient "consoles" running on my home desktop, i have logged
> in 40 testclient avatars to an opensim region hosted on a remote server
> and
> integrated into another remote grid. I built a "reasonable" central
> building of approximately 8000 prims. I divided the region into 4
> quadrants
> and logged
> 10 avatars into the approximate center of each quadrant.
>
> Initially I had each of the 10 bots randomly chat, or perform an
> animation.
> By varying the time between each of the operations I was able to get a
> feel
> for the performance by attempting to move between the groups and
> listen/watch for the success rates of the activites. Additional
> activities
> like uploading images, sending IM messages grid wide, transferring and
> collecting inventory items, and (now) teleporting to specific locations
> around the grid were easily included in the scripts and provided excellent
> simulations of real user loads.
>
> With more experience with the available commands I was able to build
> "walking" paths for the avatar groups to follow.. Adapting the existing
> scripts I set a home point in respective corners for each of the avatars
> and designed a simple square for them to walk in. It was then a simple
> measure to watch as groups of 5 avatars would walk from one corner to the
> next in relative synchronization until the region became "overloaded"..
> At
> that point the avatars would either "miss" command and stop, or loose
> connections with the region and I would see them walking off the edges or
> disappearing from the scene.
>
>
> I did not spend any time developing standardized "performance measurement"
> mechanisms... Tests evolved from "How many can I get logged in and able
> to
> register on a controller" to how long can I walk 40 bots around a region
> before "something" fails. Of course my results are also specific to the
> configuration of my desktop/home network/region server/grid service
> connections.... so I can guarantee... Your mileage WILL vary..
>
>
>
>
>
>
>
>
>
> On Wed, May 25, 2011 at 9:29 AM, Umar Farooq <u.farooq at uea.ac.uk> wrote:
>
>> Dear Dan,
>>
>> Thanks! I tested the teleports (goto command) and it looks fine now.
>>
>> We are trying to determine the overall load of the system with the
>> introduction of users, contents and their activities starting with an
>> unpopulated world on a single simulator simulating a number of regions.
>> I
>> just need to know what methods could be used to determine the load based
>> on these activities. I saw somewhere you mentioned about a ScienceSim
>> patch that collects statistics, could you please let me know if I can
>> get
>> and use it.
>>
>> We would be waiting for the exciting commands to be incorporated into
>> libomv TestClient.
>>
>> Best wishes and have a good day.
>>
>> Umar
>>
>> > Umar,
>> >
>> > There was a bug in TestClient where no region crossings would work.
>> That
>> > goes for the 'goto' command for teleporting as well as just walking
>> across
>> > a border with 'forward' or other movement commands. That bug was fixed
>> > yesterday by lkalif and is in the latest public libomv svn source.
>> I've
>> > only tested it with walking across borders, not with the 'goto'
>> teleports.
>> >
>> > For metrics, what question do you want to answer about OpenSim that
>> you
>> > are scaling the clients? That will partially determine which metrics
>> you
>> > need to capture. We have tried to find the bottlenecks to scaling a
>> scene
>> > (objects, scripts, physics, clients, etc) and wanted to know if CPU or
>> > network bandwidth was the limiting factor. We have measured CPU and
>> > bandwidth as well as line-level profiling of various OpenSim modules.
>> >
>> > Rather than releasing our TestClient patches directly, I am working
>> with
>> > the libopenmetaverse team to commit them to their public SVN. For now,
>> the
>> > public client has the best available peformance until I can merge our
>> > commands to automate the wandering movements.
>> >
>> > ~Dan
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: opensim-dev-bounces at lists.berlios.de
>> > [mailto:opensim-dev-bounces at lists.berlios.de] On Behalf Of Umar Farooq
>> > Sent: Monday, May 23, 2011 2:13 AM
>> > To: opensim-dev at lists.berlios.de
>> > Subject: Re: [Opensim-dev] Creation of 100+ users and TestClient
>> Logins!
>> >
>> > Dear Dan,
>> >
>> > I am using Robust, and running a private grid for my experiments. I
>> have
>> > noticed the issue of sequential execution of actions, and am using a
>> > number of different TestClient instances using different connections.
>> >
>> > I have noticed that when we use goto or teleport command for bots and
>> then
>> > apply a command, the server is responds that an action is performed
>> but
>> > the bots are not updating in the scene, any comment on that?
>> >
>> > Which metric would be best to check load of a system? CPU usage,
>> Network
>> > Utilisation and anything else? Sorry if that is too stupid to ask!
>> >
>> > It would be great to have the code available whenever it is possible.
>> > Thanks again for your help.
>> >
>> > Best wishes and have a great day.
>> >
>> > Umar
>> >
>> >> Umar,
>> >> Are you using Simian or Robust to create your accounts? If the
>> grid
>> >> required authentication (email, captcha, etc) you will probably need
>> to
>> >> work with the grid operator to get test accounts. On Simian, it's
>> easy
>> >> to create test accounts if you can disable the authentications
>> >> temporarily.
>> >>
>> >> As for logging them in, I suggest not putting more than about 25
>> bots
>> >> in
>> >> a single TestClient instance. There are many functions in TestClient
>> >> which iterate over all bots sequentially performing an action and you
>> >> will find them not responding to scene updates in time if you have
>> too
>> >> many in one executable.
>> >>
>> >> We are planning to make our modified TestClient available but it is
>> >> not
>> >> at this time. It's actually older than the released TestClient from
>> >> libomv but it has a few commands for making bots perform interesting
>> >> actions independently rather than all bots doing the same thing.
>> >>
>> >> ~Dan
>> >>
>> >>
>> >> -----Original Message-----
>> >> From: opensim-dev-bounces at lists.berlios.de
>> >> [mailto:opensim-dev-bounces at lists.berlios.de] On Behalf Of Umar
>> Farooq
>> >> Sent: Wednesday, May 18, 2011 2:29 AM
>> >> To: opensim-dev at lists.berlios.de
>> >> Subject: [Opensim-dev] Creation of 100+ users and TestClient Logins!
>> >>
>> >> Dear All,
>> >>
>> >> To use TestClient for adding bots, it requires that the users are
>> first
>> >> created as authentication is required during login. Is there any
>> simple
>> >> way to create lots of users and then login them via a single command
>> to
>> >> the simulator? Thanks alot for your help and guidance!
>> >>
>> >> Best wishes,
>> >>
>> >> Umar
>> >>
>> >> --
>> >> Umar Farooq
>> >>
>> >> Research Scholar,
>> >> Virtual Humans Group,
>> >> School of Computing Sciences,
>> >> University of East Anglia,
>> >> Norwich, United Kingdom.
>> >>
>> >> Cell: +4497853398184
>> >>
>> >> _______________________________________________
>> >> Opensim-dev mailing list
>> >> Opensim-dev at lists.berlios.de
>> >> https://lists.berlios.de/mailman/listinfo/opensim-dev
>> >> _______________________________________________
>> >> Opensim-dev mailing list
>> >> Opensim-dev at lists.berlios.de
>> >> https://lists.berlios.de/mailman/listinfo/opensim-dev
>> >>
>> >
>> >
>> > --
>> > Umar Farooq
>> >
>> > Research Scholar,
>> > Virtual Humans Group,
>> > School of Computing Sciences,
>> > University of East Anglia,
>> > Norwich, United Kingdom.
>> >
>> > Cell: +4497853398184
>> >
>> > _______________________________________________
>> > Opensim-dev mailing list
>> > Opensim-dev at lists.berlios.de
>> > https://lists.berlios.de/mailman/listinfo/opensim-dev
>> > _______________________________________________
>> > Opensim-dev mailing list
>> > Opensim-dev at lists.berlios.de
>> > https://lists.berlios.de/mailman/listinfo/opensim-dev
>> >
>>
>>
>> --
>> Umar Farooq
>>
>> Research Scholar,
>> Virtual Humans Group,
>> School of Computing Sciences,
>> University of East Anglia,
>> Norwich, United Kingdom.
>>
>> Cell: +4497853398184
>>
>> _______________________________________________
>> Opensim-dev mailing list
>> Opensim-dev at lists.berlios.de
>> https://lists.berlios.de/mailman/listinfo/opensim-dev
>>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev
>
--
Umar Farooq
Research Scholar,
Virtual Humans Group,
School of Computing Sciences,
University of East Anglia,
Norwich, United Kingdom.
Cell: +4497853398184
More information about the Opensim-dev
mailing list