Umar,<br><br>I have been using pretty standard TestClient bots to do "performance testing".. <br><br>I have also used the existing command set to pre-program complex activities, like a continuous 6 model runway fashion show for in-world video recording.<br>
<br><a href="http://lib.openmetaverse.org/wiki/TestClient_0.7.0_commands">Existing TestClient commands</a><br><br>The existing commands I have found most useful to facilitate individual control of the bot behavior are<br>
<br> setmaster - Lets you assign an inworld object that can send IM's to the bot to initiate commands. <br><br> script - lets you maintain your command scripts in a directory relative to the testclient executable.<br>
<br>Load testing generally consisted of 3 stages....<br><br> Generating and equipping the avatars.... The login credentials need to be available, and the avatars need to represent the kind<br> of population you expect. Performance differences between default avatars, and those with prim and/or HUD attachments are significant.<br>
<br> Logging in the Test population.. The avatar creation sequence is very load intensive... TestClient bots are most prone to operation timeout failures.<br> Logging in significant numbers of avatars at once will prevent any significant number of successful logins.. Scripting this activity is the fastest/easiest<br>
way I have found to pre-load the test regions.. My experience has been that running 10 bots per TestClient "console" provides an efficient way to <br> balance the need to get bots logged in at a reasonable rate, and being able to keep most of the logged bots active if a TestClient "console" fails.<br>
<br> The final step of the login process is to have the bot touch an inworld object... This object is scripted to use the touch_start event to "register" the UUID <br> of the bot and act as its "master", by sending the "performance action" commands via IM or initiating scripts to direct the avatar<br>
<br> Syncronizing the test... Once all the bots have been registered to the controller devices... I log in to the region using a standard viewer. <br> By scripting the "controllers' to provide me with menu access to commands, I can initiate and terminate group activities. <br>
Combining the in-world object ability to send IM commands with the TestClient access to remote scripts has allowed complex load testing scenarios<br> to be designed quickly and repeated across multiple platform releases.<br>
<br><br>Some examples.....<br><br> Using 4 testclient "consoles" running on my home desktop, i have logged in 40 testclient avatars to an opensim region hosted on a remote server and integrated into another remote grid. I built a "reasonable" central building of approximately 8000 prims. I divided the region into 4 quadrants and logged<br>
10 avatars into the approximate center of each quadrant. <br><br>Initially I had each of the 10 bots randomly chat, or perform an animation. By varying the time between each of the operations I was able to get a feel for the performance by attempting to move between the groups and listen/watch for the success rates of the activites. Additional activities like uploading images, sending IM messages grid wide, transferring and collecting inventory items, and (now) teleporting to specific locations around the grid were easily included in the scripts and provided excellent simulations of real user loads.<br>
<br>With more experience with the available commands I was able to build "walking" paths for the avatar groups to follow.. Adapting the existing scripts I set a home point in respective corners for each of the avatars and designed a simple square for them to walk in. It was then a simple measure to watch as groups of 5 avatars would walk from one corner to the next in relative synchronization until the region became "overloaded".. At that point the avatars would either "miss" command and stop, or loose connections with the region and I would see them walking off the edges or disappearing from the scene.<br>
<br><br>I did not spend any time developing standardized "performance measurement" mechanisms... Tests evolved from "How many can I get logged in and able to register on a controller" to how long can I walk 40 bots around a region before "something" fails. Of course my results are also specific to the configuration of my desktop/home network/region server/grid service connections.... so I can guarantee... Your mileage WILL vary..<br>
<br><br><br><br><br><br><br><br><br><div class="gmail_quote">On Wed, May 25, 2011 at 9:29 AM, Umar Farooq <span dir="ltr"><<a href="mailto:u.farooq@uea.ac.uk">u.farooq@uea.ac.uk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Dear Dan,<br>
<br>
Thanks! I tested the teleports (goto command) and it looks fine now.<br>
<br>
We are trying to determine the overall load of the system with the<br>
introduction of users, contents and their activities starting with an<br>
unpopulated world on a single simulator simulating a number of regions. I<br>
just need to know what methods could be used to determine the load based<br>
on these activities. I saw somewhere you mentioned about a ScienceSim<br>
patch that collects statistics, could you please let me know if I can get<br>
and use it.<br>
<br>
We would be waiting for the exciting commands to be incorporated into<br>
libomv TestClient.<br>
<br>
Best wishes and have a good day.<br>
<font color="#888888"><br>
Umar<br>
</font><div><div></div><div class="h5"><br>
> Umar,<br>
><br>
> There was a bug in TestClient where no region crossings would work. That<br>
> goes for the 'goto' command for teleporting as well as just walking across<br>
> a border with 'forward' or other movement commands. That bug was fixed<br>
> yesterday by lkalif and is in the latest public libomv svn source. I've<br>
> only tested it with walking across borders, not with the 'goto' teleports.<br>
><br>
> For metrics, what question do you want to answer about OpenSim that you<br>
> are scaling the clients? That will partially determine which metrics you<br>
> need to capture. We have tried to find the bottlenecks to scaling a scene<br>
> (objects, scripts, physics, clients, etc) and wanted to know if CPU or<br>
> network bandwidth was the limiting factor. We have measured CPU and<br>
> bandwidth as well as line-level profiling of various OpenSim modules.<br>
><br>
> Rather than releasing our TestClient patches directly, I am working with<br>
> the libopenmetaverse team to commit them to their public SVN. For now, the<br>
> public client has the best available peformance until I can merge our<br>
> commands to automate the wandering movements.<br>
><br>
> ~Dan<br>
><br>
><br>
><br>
> -----Original Message-----<br>
> From: <a href="mailto:opensim-dev-bounces@lists.berlios.de">opensim-dev-bounces@lists.berlios.de</a><br>
> [mailto:<a href="mailto:opensim-dev-bounces@lists.berlios.de">opensim-dev-bounces@lists.berlios.de</a>] On Behalf Of Umar Farooq<br>
> Sent: Monday, May 23, 2011 2:13 AM<br>
> To: <a href="mailto:opensim-dev@lists.berlios.de">opensim-dev@lists.berlios.de</a><br>
> Subject: Re: [Opensim-dev] Creation of 100+ users and TestClient Logins!<br>
><br>
> Dear Dan,<br>
><br>
> I am using Robust, and running a private grid for my experiments. I have<br>
> noticed the issue of sequential execution of actions, and am using a<br>
> number of different TestClient instances using different connections.<br>
><br>
> I have noticed that when we use goto or teleport command for bots and then<br>
> apply a command, the server is responds that an action is performed but<br>
> the bots are not updating in the scene, any comment on that?<br>
><br>
> Which metric would be best to check load of a system? CPU usage, Network<br>
> Utilisation and anything else? Sorry if that is too stupid to ask!<br>
><br>
> It would be great to have the code available whenever it is possible.<br>
> Thanks again for your help.<br>
><br>
> Best wishes and have a great day.<br>
><br>
> Umar<br>
><br>
>> Umar,<br>
>> Are you using Simian or Robust to create your accounts? If the grid<br>
>> required authentication (email, captcha, etc) you will probably need to<br>
>> work with the grid operator to get test accounts. On Simian, it's easy<br>
>> to create test accounts if you can disable the authentications<br>
>> temporarily.<br>
>><br>
>> As for logging them in, I suggest not putting more than about 25 bots<br>
>> in<br>
>> a single TestClient instance. There are many functions in TestClient<br>
>> which iterate over all bots sequentially performing an action and you<br>
>> will find them not responding to scene updates in time if you have too<br>
>> many in one executable.<br>
>><br>
>> We are planning to make our modified TestClient available but it is<br>
>> not<br>
>> at this time. It's actually older than the released TestClient from<br>
>> libomv but it has a few commands for making bots perform interesting<br>
>> actions independently rather than all bots doing the same thing.<br>
>><br>
>> ~Dan<br>
>><br>
>><br>
>> -----Original Message-----<br>
>> From: <a href="mailto:opensim-dev-bounces@lists.berlios.de">opensim-dev-bounces@lists.berlios.de</a><br>
>> [mailto:<a href="mailto:opensim-dev-bounces@lists.berlios.de">opensim-dev-bounces@lists.berlios.de</a>] On Behalf Of Umar Farooq<br>
>> Sent: Wednesday, May 18, 2011 2:29 AM<br>
>> To: <a href="mailto:opensim-dev@lists.berlios.de">opensim-dev@lists.berlios.de</a><br>
>> Subject: [Opensim-dev] Creation of 100+ users and TestClient Logins!<br>
>><br>
>> Dear All,<br>
>><br>
>> To use TestClient for adding bots, it requires that the users are first<br>
>> created as authentication is required during login. Is there any simple<br>
>> way to create lots of users and then login them via a single command to<br>
>> the simulator? Thanks alot for your help and guidance!<br>
>><br>
>> Best wishes,<br>
>><br>
>> Umar<br>
>><br>
>> --<br>
>> Umar Farooq<br>
>><br>
>> Research Scholar,<br>
>> Virtual Humans Group,<br>
>> School of Computing Sciences,<br>
>> University of East Anglia,<br>
>> Norwich, United Kingdom.<br>
>><br>
>> Cell: +4497853398184<br>
>><br>
>> _______________________________________________<br>
>> Opensim-dev mailing list<br>
>> <a href="mailto:Opensim-dev@lists.berlios.de">Opensim-dev@lists.berlios.de</a><br>
>> <a href="https://lists.berlios.de/mailman/listinfo/opensim-dev" target="_blank">https://lists.berlios.de/mailman/listinfo/opensim-dev</a><br>
>> _______________________________________________<br>
>> Opensim-dev mailing list<br>
>> <a href="mailto:Opensim-dev@lists.berlios.de">Opensim-dev@lists.berlios.de</a><br>
>> <a href="https://lists.berlios.de/mailman/listinfo/opensim-dev" target="_blank">https://lists.berlios.de/mailman/listinfo/opensim-dev</a><br>
>><br>
><br>
><br>
> --<br>
> Umar Farooq<br>
><br>
> Research Scholar,<br>
> Virtual Humans Group,<br>
> School of Computing Sciences,<br>
> University of East Anglia,<br>
> Norwich, United Kingdom.<br>
><br>
> Cell: +4497853398184<br>
><br>
> _______________________________________________<br>
> Opensim-dev mailing list<br>
> <a href="mailto:Opensim-dev@lists.berlios.de">Opensim-dev@lists.berlios.de</a><br>
> <a href="https://lists.berlios.de/mailman/listinfo/opensim-dev" target="_blank">https://lists.berlios.de/mailman/listinfo/opensim-dev</a><br>
> _______________________________________________<br>
> Opensim-dev mailing list<br>
> <a href="mailto:Opensim-dev@lists.berlios.de">Opensim-dev@lists.berlios.de</a><br>
> <a href="https://lists.berlios.de/mailman/listinfo/opensim-dev" target="_blank">https://lists.berlios.de/mailman/listinfo/opensim-dev</a><br>
><br>
<br>
<br>
</div></div>--<br>
<div><div></div><div class="h5">Umar Farooq<br>
<br>
Research Scholar,<br>
Virtual Humans Group,<br>
School of Computing Sciences,<br>
University of East Anglia,<br>
Norwich, United Kingdom.<br>
<br>
Cell: +4497853398184<br>
<br>
_______________________________________________<br>
Opensim-dev mailing list<br>
<a href="mailto:Opensim-dev@lists.berlios.de">Opensim-dev@lists.berlios.de</a><br>
<a href="https://lists.berlios.de/mailman/listinfo/opensim-dev" target="_blank">https://lists.berlios.de/mailman/listinfo/opensim-dev</a><br>
</div></div></blockquote></div><br>