<div dir="ltr">Sorry for the late response just looked at my email.<div><br></div><div>The multiplier originally had something to do with matching up the actual physics frames with the times physics reported it's own frames since the reporter was reporting at different times then the world was stepping and the values were fluctuating improperly. In the end, I ended up setting it to be 'similar' to the rate reported on Second Life mostly because when it was implemented, I received repeated questions about it and caused more questions then actually serving it's purpose.. so a 'relative' value that someone could look at and feel comfortable with and be able to tell when there was an issue was chosen so that the differences in the way the physics engines and scene graph were run were not repeatedly and erroneously targeted as the cause of bugs. I couldn't really alter the main simulation scene loop frequency. The simulator simply couldn't handle it and there were no process queues in the main simulation that we can skip and process in the next frame if there was not enough time to process them all in a frame. <div><br></div><div>The goal of the stats reporting was to help identify what the cause of a slowdown was and people were seeing <11fps when they were used to 45. As a result they often squarely put the blame on the physics layer when in fact the issue was somewhere else. Early adopters were sensitive about that number mostly because they are taught to look at it as a Simulator Administrator/Estate manager in SL. <div><div><br></div><div>It sounds like since then, someone has done the physics step 5 times, which is good. I'm not sure it would have supported that in the past.</div><div><br></div><div>Before you get the out the torches and pitch forks(Your 'curious' comment), take a look at the following letter from 2009 and understand that you are not the only proponent of using stats to try and identify the bottlenecks.</div><div>------------------------------------</div><div><div>Teravus Ovares <<a href="mailto:teravus@gmail.com">teravus@gmail.com</a>></div><div>11/27/09</div><div><br></div><div>to opensim-dev </div><div>Hey there,</div><div><br></div><div>A while back, we had somewhat reasonable statistics being generated and presented to the client. They were not always accurate, but based on what I saw, I could, pretty much pin certain parts of the simulator as the limiting factor during load tests. </div><div><br></div><div>I'd say, the number 1 reason that they were semi-accurate and not accurate.. in the past.. is because nobody ever thought about instrumentation during the functionality design. It was always 'tacked on later'. One example of this.. is the current AssetCache implementation. There's no way, currently, to know, at a glance.. how many external requests it has open. Additionally, it will be extremely difficult to put one in because of the way the objects are designed and accessed. To put one in, an event needs to be added to the IAssetService interface and each AssetCache implementation will need an interlocked int to count how many gets and puts it currently has open to the external data source as well as it's own event calling schedule. Then, the IAssetService property in Scene, (AssetService) will need an event handler.. which updates the values in SimStatsReporter in Scene (StatsReporter). This idea of external access resource instrumentation should really have been built in to the design of the AssetService. </div><div><br></div><div>This last recent load test, there were no real statistics that I could use to determine what the limiting factor was. Time Dilation was pegged at 1.0.. even when the simulator was obviously struggling. Total Frame time (MS) was -50ms even when the simulation MS was 850ms and the Physics ms was 250ms, so the inconsistencies made it impossible to know what part of the simulator was struggling. Agent Updates were erratic.. sometimes high..</div><div>sometimes low when the simulator was fine and when it was struggling. Pending Uploads and Downloads were always 0, so there was no way to know how well the simulator was downloading and uploading assets to and from the grid. Packet stats were non-existant, so there was no way to know how well the UDP handlers were faring under the load. When it crashed, it crashed with a mono based stack trace which pointed to out of memory errors, so the only way that you could, scientifically, find out what the issue is.. is to run a load test under a memory profiler. We know, that running a public load test under a memory profiler is quite impractical. </div><div><br></div><div>To make something better, I need to know two things, where it is, and where I want it to be. How can we make OpenSimulator better if we don't have statistics that point to where we are currently?</div><div><br></div><div>On that note, I propose that, when designing objects for functionality in OpenSimulator, that we also consider if the objects should be instrumented and, what would be the best way to go about instrumenting the objects. We should incorporate instrumentation into the design of the objects. Some of that instrumentation is appropriate for a client to see, some of it might not be. Consider that, many of them should be client facing and be included in the SimStats that get sent to the client.. so that we can have a reasonable idea of what's going on with a simulator at a glance. Also, in the design of the instrumentation, we make sure that the instrumentation is accurate and</div><div>lightweight. </div><div><br></div><div>The load test went reasonably... but, we didn't get half of the information on the simulator that we needed to be able to improve it.</div><div><br></div><div><br></div><div>Please comment :) I look forward to hearing your responses.</div><div><br></div><div>Regards</div></div><div><br></div><div>Teravus</div><div>--------------------------------------------------------------------</div></div><div><br></div><div> That said, changing it now, I would be careful of the code that does time dilation calculation. Time Dilation is 'supposed to be' a factor calculated by dividing the physics scene time passed by real time passed. This factor guides interpolation through velocity multiplication (Example: Time Dilation:0.5, Linear Velocity: <1,1,1>m/s. would cause the viewer to move the object <1*0.5,1*0.5,1*0.5>m/s until it gets a new update about the object's position from the server) telling the viewer what to do with moving objects between the times that it gets updates on the actual positions of the objects. If you cause the Time Dilation calculation to change to a more inaccurate result then be aware, objects will rubber-band and stutter more often. Some of the time, objects will not visibly come to a stop and fly off forever. It should be easy to test. Make a 30 0.5m cubes and stack them up in 3 groups of ten 2x2x2 with 2 on the top with each group about 2 meters above the ground. Make them physical and gauge the amount of rubber banding as they bobble around. If it's worse after then change then before the change then you have altered the calculation. I use the term 'more inaccurate result' mostly because I'm not sure that OpenSimulator is doing that calculation right as it is now. At one point it was set to 1.0 regardless of what was happening so they didn't have to think about it while working on things... and then someone different set it to the scene graph time instead of the physics time which may or may not be better then 1.0 all the time depending on how the updates actually get out to the viewers.</div></div><div><br></div><div>If you wanted to be really sure, you could artificially add time to the main physics loop and verify that rubber banding is minimal and that time dilation goes down to reduce rubber banding.</div></div><div><br></div><div>Hopefully this was a helpful trip around OpenSimulator memory lane.</div><div><br></div><div>Teravus</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Mar 3, 2015 at 6:55 AM, Diva Canto <span dir="ltr"><<a href="mailto:diva@metaverseink.com" target="_blank">diva@metaverseink.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>Again, for people interested in
learning more about these design issues, and the tradeoffs of
different choices, I highly recommend this tutorial and associated
code:<br>
<a href="http://buildnewgames.com/real-time-multiplayer/" target="_blank">http://buildnewgames.com/real-time-multiplayer/</a><br>
<br>
It's a wonderful little example that explains how interactive
multi-user client-server simulations work. 100x simpler than
OpenSim, but essentially the same concepts. It doesn't explain
time dilation, but it's not hard to see where that comes in: in
varying the time window of the physics loop. The update
(simulation) loop is mainly about notifying clients about changes
in the state of the simulation. We want the physics simulation to
be high fidelity (meaning high loop rate), and the client updates
to be as low as possible (low update loop rate). <br><div><div class="h5">
<br>
On 3/3/2015 2:09 AM, Tom Bess wrote:<br>
</div></div></div><div><div class="h5">
<blockquote type="cite">
<div>Thanks. You have clarified much. I realised the rift is
viewer control but was not so clear about the
impact/relationship between viewer and server frame rates and
the interpolation needed.</div>
<div><br>
</div>
<div>Thanks again.</div>
<div><br>
<span style="font-family:Helvetica;font-size:medium">Tom Willans BSc(Hons) MBCS
CITP</span>
<div>Chartered IT Professional<br style="font-family:Helvetica;font-size:medium">
<span style="font-family:Helvetica;font-size:medium"></span><br style="font-family:Helvetica;font-size:medium">
<span style="font-family:Helvetica;font-size:medium">Managing
Director Bessacarr Publications Ltd</span><br style="font-family:Helvetica;font-size:medium">
<span style="font-family:Helvetica;font-size:medium">+44 (0)121
288 0281</span><br style="font-family:Helvetica;font-size:medium">
<span style="font-family:Helvetica;font-size:medium">email: <a href="mailto:tom.willans@bessacarr.com" target="_blank">tom.willans@bessacarr.com</a></span><br style="font-family:Helvetica;font-size:medium">
<span style="font-family:Helvetica;font-size:medium">skype:
tom.willans</span><br style="font-family:Helvetica;font-size:medium">
<span style="font-family:Helvetica;font-size:medium">Second
Life and OSGrid: Tom Tiros</span>
<div><span style="font-family:Helvetica;font-size:medium"><br>
</span></div>
<div><span style="font-family:Helvetica;font-size:medium">Sent
from my mobile</span></div>
<div><span style="font-family:Helvetica;font-size:medium"><br>
</span></div>
</div>
</div>
<div><br>
On 3 Mar 2015, at 09:08, Dahlia Trimble <<a href="mailto:dahliatrimble@gmail.com" target="_blank">dahliatrimble@gmail.com</a>>
wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>If my memory is correct, SL sims run at a
default of 45 frames/second. OpenSimulator runs at
11. I'm not certain exactly why 11 was chosen but
I do know that increasing it increases the amount
of work the simulator must do. E.g., if you go
from 11 to 45 you quadruple the work the simulator
must do<br>
<br>
</div>
The only signal I'm aware of that affects the viewer
is "time dilation". This value varies from 0.0 to
1.0 where 1.0 indicates the simulator is running at
full speed. If for some reason the physics engine
cannot keep up with the simulation it can signal the
viewers to slow movement. This value is sent with
many UDP packets that are involved with the scene
and moving objects, so the viewers will know this
value as timely as possible.<br>
<br>
</div>
Running the simulation at 11 fps shoudl *not* affect
devices such as the Rift as the camera is controlled
entirely viewer side except under special
circumstances such as a user surrendering camera
control to a script. The viewer frame rate is
*entirely independent* of the simulator frame rate and
will run as fast as the hardware allows if configured
to do so.<br>
<br>
</div>
Choosing an appropriate simulation frame rate would
involve weighing the tradeoffs between simulation
workload and responsiveness to user controls such as
those that control avatar position movement. Bear in
mind that there are also network delays involved; a
simulation running at 11 frames/second will respond to a
user control every 89 ms but there will also be
round-trip network delays which may typically be 100-200
ms but as high as 1.5 seconds or more. Increashing the
simulation frame rate will likely not produce a
perceivable difference in performance but will
*significantly reduce* the capacity of the simulator to
do things like host more avatars and scripted objects.<br>
<br>
</div>
Also bear in mind that much of the code assumes 11
frames/second and raising that may create motion-related
and other bugs that may not be apparent until much later.
OpenSimulator is tuned to function properly at 11
frames/second.<br>
<br>
</div>
To recap: the user camera is for the most part entirely
controlled within the viewer and is unaffected by simulation
frame rate. As such devices such as the Rift which
manipulate the camera will not be affected.<br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Mar 3, 2015 at 12:09 AM,
Tom Bess <span dir="ltr"><<a href="mailto:tom.willans@bessacarr.com" target="_blank">tom.willans@bessacarr.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="auto">
<div><br>
I am reporting a comparison between sl and opensim
and did not realise this. Does sl run at a true
55fps? If so why bother? Presumably the viewer
needs 55fps sent to it to get its calculations
correct as at 11fps opensim does the same at sl
albeit in larger steps. </div>
<div><br>
</div>
<div>Would a faster fps improve the accuracy of
devices such as the rift by having to interpolate
over a shorter period of time? Admittedly I suspect
viewer rendering needs to be improved as well as
this aspect is holding my experience back. </div>
<div><br>
</div>
<div>I understand that other aspects may assume that
11fps is a fixed constant and not allow for this to
change presumably that can be changed but you guys
know more here.</div>
<div><br>
</div>
<div>Thanks for the guide btw. </div>
<div><br>
<span style="font-family:Helvetica;font-size:medium">Tom
Willans BSc(Hons) MBCS CITP</span>
<div>Chartered IT Professional<br style="font-family:Helvetica;font-size:medium">
<span style="font-family:Helvetica;font-size:medium"></span><br style="font-family:Helvetica;font-size:medium">
<span style="font-family:Helvetica;font-size:medium">Managing
Director Bessacarr Publications Ltd</span><br style="font-family:Helvetica;font-size:medium">
<span style="font-family:Helvetica;font-size:medium"><a href="tel:%2B44%20%280%29121%20288%200281" value="+441212880281" target="_blank">+44
(0)121 288 0281</a></span><br style="font-family:Helvetica;font-size:medium">
<span style="font-family:Helvetica;font-size:medium">email:
<a href="mailto:tom.willans@bessacarr.com" target="_blank">tom.willans@bessacarr.com</a></span><br style="font-family:Helvetica;font-size:medium">
<span style="font-family:Helvetica;font-size:medium">skype:
tom.willans</span><br style="font-family:Helvetica;font-size:medium">
<span style="font-family:Helvetica;font-size:medium">Second
Life and OSGrid: Tom Tiros</span>
<div><span style="font-family:Helvetica;font-size:medium"><br>
</span></div>
<div><span style="font-family:Helvetica;font-size:medium">Sent
from my mobile</span></div>
<div><span style="font-family:Helvetica;font-size:medium"><br>
</span></div>
</div>
</div>
<div>
<div>
<div><br>
On 3 Mar 2015, at 05:18, Sean M <<a href="mailto:mondesire.sean@gmail.com" target="_blank">mondesire.sean@gmail.com</a>>
wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>Thanks Justin, Dahlia, Michael, and
everyone. I now have a better understanding of
the way FPS is calculated and of the
correction factor.
<div><br>
</div>
<div>-Sean M. <span></span><br>
<br>
On Monday, March 2, 2015, Justin Clark-Casey
<<a href="mailto:jjustincc@googlemail.com" target="_blank">jjustincc@googlemail.com</a>>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I think this
has already been said but just to
summarize.<br>
<br>
* The 11 fps are the number of scene
frames processed - opportunities where
avatars may be notified about changes in
the scene.<br>
<br>
* Each of these scene frames is associated
with a physics process where 5 physics
frames are processed in each frame. Hence
11 * 5 = 55 fps.<br>
<br>
* Why this number? Others may know better
but my guess is that it's related to the
frequency of updates expected by the
viewer. Teravus may well know more if
he's still around.<br>
<br>
* Changing m_reportedFpsCorrectionFactor
will do nothing except change the server
FPS stat.<br>
<br>
* Changing MinFrameTime will change the
number of scene frames. From my work in
the past, you would also need to adjust
other parameters like physics frames to
stop things going haywire (this was with
ODE, Bullet might work differently). I
expect you also wouldn't gain much if
anything in scene fidelity.<br>
<br>
On 02/03/15 18:23, Sean M wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Greetings,<br>
<br>
We at the MOSES project have noticed
Simulation and Physics frames per second
(FPS) have a few issues that we are
trying<br>
to resolve. The issues are producing
suspicious performance statistics for
the analysis of the current version of<br>
OpenSim that we are running.<br>
<br>
First, there is a correction factor (m_reportedFpsCorrectionFactor)
that the raw SimFPS is multiplied by.
The comment in<br>
the following line is a bit curious
because it indicates that the FPS is
artificially inflated to "lie" about the
actual<br>
FPS being so low:<br>
<br>
OpenSim/Region/Framework/__Scenes/SimStatsReporter.cs:
Line 317<br>
// We're going to lie about the FPS
because we've been lying since 2008.
The actual FPS is currently<br>
// locked at a maximum of 11. Maybe at
some point this can change so that we're
not lying.<br>
int reportedFPS = (int)(m_fps *
m_reportedFpsCorrectionFactor);<br>
<br>
Also, lines 174 and 227 mention the use
of this correction factor.<br>
<br>
Second, this multiplier also comes into
play in the Scene where there is a
MinFrameTime, which seems to be the
minimum<br>
reported amount of time to process a
frame:<br>
OpenSim/Region/Framework/__Scenes/Scene.cs:Line
723<br>
<br>
Both of these variables, the correction
factor and MinFrameTime, are concerning
from a statistics view point as they are<br>
generating skewed and massaged numbers;
therefore, I have a few questions:<br>
<br>
1) Is it commonly known that Sim and Phy
FPSs are inflated to maintain the "lie"?
And if so, will it be corrected to be<br>
an accurate reporting of processed
frames per second?<br>
<br>
2) What exactly are the definitions for
OpenSim's Simulation (Sim) FPS, Physics
(Phy) FPS and a frame (I have found<br>
conflicting and vague definitions on the
wiki)?<br>
<br>
3) What are the known performance
consequences of setting the
m_reportedFpsCorrectionFactor to 1 and
MinFrameTime to 0?<br>
<br>
Thanks,<br>
Sean M.<br>
<br>
<br>
_______________________________________________<br>
Opensim-dev mailing list<br>
<a>Opensim-dev@opensimulator.org</a><br>
<a href="http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev" target="_blank">http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev</a><br>
<br>
</blockquote>
<br>
<br>
-- <br>
Justin Clark-Casey (justincc)<br>
OSVW Consulting<br>
<a href="http://justincc.org" target="_blank">http://justincc.org</a><br>
<a href="http://twitter.com/justincc" target="_blank">http://twitter.com/justincc</a><br>
_______________________________________________<br>
Opensim-dev mailing list<br>
<a>Opensim-dev@opensimulator.org</a><br>
<a href="http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev" target="_blank">http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev</a><br>
</blockquote>
</div>
</div>
</blockquote>
<blockquote type="cite">
<div><span>_______________________________________________</span><br>
<span>Opensim-dev mailing list</span><br>
<span><a href="mailto:Opensim-dev@opensimulator.org" target="_blank">Opensim-dev@opensimulator.org</a></span><br>
<span><a href="http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev" target="_blank">http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev</a></span><br>
</div>
</blockquote>
</div>
</div>
</div>
<br>
_______________________________________________<br>
Opensim-dev mailing list<br>
<a href="mailto:Opensim-dev@opensimulator.org" target="_blank">Opensim-dev@opensimulator.org</a><br>
<a href="http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev" target="_blank">http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<blockquote type="cite">
<div><span>_______________________________________________</span><br>
<span>Opensim-dev mailing list</span><br>
<span><a href="mailto:Opensim-dev@opensimulator.org" target="_blank">Opensim-dev@opensimulator.org</a></span><br>
<span><a href="http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev" target="_blank">http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev</a></span><br>
</div>
</blockquote>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Opensim-dev mailing list
<a href="mailto:Opensim-dev@opensimulator.org" target="_blank">Opensim-dev@opensimulator.org</a>
<a href="http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev" target="_blank">http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev</a>
</pre>
</blockquote>
<br>
</div></div></div>
<br>_______________________________________________<br>
Opensim-dev mailing list<br>
<a href="mailto:Opensim-dev@opensimulator.org">Opensim-dev@opensimulator.org</a><br>
<a href="http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev" target="_blank">http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev</a><br>
<br></blockquote></div><br></div>