[Opensim-dev] Still on Sim and Phys Frames per Second (FPS)

[Sean M]

Greetings everyone,

Please allow me to provide some background and motivation on why the MOSES
Team submitted corrections to the simulation's statistics gathering. When
we first became interested in determining the scalability of OpenSim, very
little information could be found on the web, publications, and through the
developers' IRC chat. Our investigation determined that we must do an
exhaustive study on our own because the information was not available. To
our surprise, we noticed early on that several questionable and incorrect
implementations of metrics resided in the code; this was a bit concerning
because grid owners, researchers, and curious users all relied on the
accurate statistics reporting. The biggest concern to us was that
researchers have published work containing these invalid statistics without
knowing that the gathered statistics were "fudged" and incorrect. [It
should be noted that in academia and research communities, researchers
depend on and refer to previous publications as the basis of their work. If
the referenced data misleads conclusions and reporting, an entire research
thread can be deemed false, wasting time and money and doing serious harm
to the community.]

To make OpenSim's statistics more accurate and valid to measure, the MOSES
Team dedicated financial support and development hours to improve the
simulator for everyone. To do this, we first provided the core developers
with a statement-of-work that was a preview of the statistics development
that we anticipated to make and welcomed feedback. We then followed the
community's process to submit the code back to the project in three code
patches. The first phase corrected the frame rate reporting, which was
originally multiplied by a static/hard-coded value of 5, noted in the code
to be a "hack" that must be corrected in the future, and was not
acknowledged anywhere on the OpenSim website or any other documentation to
be artificially boosted. From our development, the "fudge" factor was
removed, other noted invalid metrics were corrected, and the simulator was
thoroughly tested for both operational correctness and user scalability.
After the first phase of work was verified, we submitted the code, provided
further details of what was submitted, and listened to your feedback to
ensure acceptance.

With the metrics we added and corrected, the community now has enhanced and
valid statistics gathering. From the MOSES Team alone, we have provided 7
peer-reviewed publications (listed below) that uses the statics we have
given back to the community since July. From the work, you now know how
OpenSim's user scalability is affected with increased vertical hardware
scaling: various hardware configurations, allocations, and limitations. We
have also provided the methodology to generate predictive models to allow
grid owners know what hardware is needed to support a target amount of
simultaneous users on a single region. Without correcting the invalid
metrics that resided inside of OpenSim, we could not have given back to the
OS community this type of detailed analysis. More broadly, from our talks
at conferences, workshops, and through our journal publications, we have
brought the attention of OpenSim to other simulation enthusiasts by
spreading the word of this extensive, research-able open-sourced project.
All of this published research and OS awareness stems from the work that
the MOSES team has contributed back to the OpenSim community.

[1] Sean C. Mondesire, Jonathan Stevens, Rebecca Leis, and Douglas B.
Maxwell, “Resource Allocation Predictive Modeling to Optimize Virtual World
Simulator Performance,” In Proceedings of the IEEE ICMLA’15 Workshop on
Machine Learning for Predictive Models in Engineering Applications
(MLPMEA), Miami, FL, December 9-11, 2015.

[2] Sean C. Mondesire, Jonathan Stevens, and Douglas B. Maxwell, "Network
Bandwidth's Effect on Virtual World Simulator Performance Optimization," In
Proceedings of the Interservice/Industry Training, Simulation and Education
Conference (IT/TSEC '15), December 2015.

[3] Sean C. Mondesire, Jonathan Stevens, and Douglas B. Maxwell, "An
Analysis of Increased Vertical Scaling in Three-Dimensional Virtual World
Simulation," In Proceedings of the 8th EAI International Conference on
Simulation Tools and Techniques 2015 (SimuTools '15), August 2015.

[4] Jonathan Stevens, Sean C. Mondesire, Rebecca Leis, and Douglas B.
Maxwell, “An Empirical Analysis of Virtual World Fidelity’s Impact on
Simulator Network Performance,” Journal of Advanced Research in Modeling
and Simulation, Vol. 2, No. 1, August 2015.

[5] Jonathan Stevens, Sean C. Mondesire, Rebecca Leis, and Douglas B.
Maxwell,  “Human Entities' Effect on Server Performance in Distributed
Virtual World Training,” In Proceedings of the 2015 Fall Simulation
Interoperability Workshop (SIW), Orlando, FL, USA, Aug. 31-Sept. 4, 2015.

[6] Sean C. Mondesire, Rebecca Leis, Jonathan Stevens, and Douglas B.
Maxwell, "Analyzing Virtual World Region Fidelity on Scalability and
Simulation Performance," Open Journal of Modeling and Simulation (OJMSi),
Vol. 3, No. 3, July 27, 2015.

[7] Sean C. Mondesire, Jonathan Stevens, and Douglas B. Maxwell, "Vertical
Scalability Benchmarking in Three-Dimensional Virtual World Simulation," In
Proceedings of the 47th Summer Computer Simulation Conference 2015
(SummerSim '15), July 2015.


Best regards,
Sean Mondesire, Ph.D.
MOSES Team

On Thu, Nov 12, 2015 at 5:04 AM, GarminKawaguichi <
garmin.kawaguichi at magalaxie.com> wrote:

> Well done !
>
> Le 12/11/2015 03:43, Nicky Perian a écrit :
>
> Returned lag meter to Kokua with adjusted values
>
> when on opensim grids.
>
> https://bitbucket.org/NickyP/kokuant/commits/c9c2099513d4ee0e2b023199efaff4a049a7cc05
>
> Comment message follows if you don't care to follow the link.
>
> [OPENSIM] Return Lag Meter. Fudge factor added for server section of Lag Meter to compensate for the removal of a server side fudge factor. The trigger on SL grids is 20 for red, between 20 and 30 for yellow and above 30 for green.
>
> On SL grids with nominal 45 fps 20 is 44.44 % and warning point is 66.67 %.On OS grids with nominal 55 fps 20 is 36.3 % and warning point is 54.5 %. On OS there was a bias to not turn red or yellow until performance was worse
>
> than SL points. Maybe that is one reason why the fudge factor was put in the first place. With this change the bias to let performnace get worse than SL is still present and the value for red is 4 and yellow is between 4 and 6.
>
> While on OS 20 and 30 are multipled by (11/55).
>
> --
>
>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at opensimulator.org
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20151112/36b78548/attachment-0001.html>