[Opensim-dev] Still on Sim and Phys Frames per Second (FPS)
dz
dz at bitzend.net
Mon Nov 9 03:04:01 UTC 2015
Call me confused...... But don't tell me I can't read history!
This discussion is about a patch that was submitted in March and that
patch was based on questions that were raised in February.
According to my reading of the discussions that Google has so kindly
archived in my Opensim-dev folder the ONLY technical objection to the
proposed patch dealt with an issue on the accuracy of time dilation
factors.
There were numerous calls for people who might be affected to speak
up... There were repeated calls for opinions and I see +1's from
Diva, BlueWall, Nebadon, and others who saw fit to participate and voice
opinions. The ONLY objection raised to changing to an accurate
reporting was the assertion that "significant numbers of monitoring tools
and bots" might need to be reworked. Divas call for additional
discussion delayed the implementation of the patch for over 2 months
and NO ONE objected to modifying their "numerous monitoring scripts"
or even commented on a potential negative impact.
Its been months since the patch was applied and the world hasn't stopped
turning. Until just a few days ago there was nary a PEEP about any
adverse impact on the mailing lists... WHERE is this horde of angry
users??? They don't seem to be participating in any of the OpenSim
communities I track... then after almost a WHOLE DAY a patch is
introduced into the next GIANT ( read Impossible to parse through)
Update to reverse the agreement that was achieved. Pardon me if I
wonder WTF ???? This sure makes me confident!
Of course there are always delicious tidbits of perspective that a
look in the history books provides.... these 2 both made me laugh....
Melanie <melanie at t-data.com>
Apr 25
to opensim-dev
I had been under the impression that the "fudge factor" on these stats was
common knowledge.
Good arguments have been brought for changing them to provide accurate
metrics and I find I can't sustain an objection to progress, especially
since SL appears to have a limited shelf life these days.
Announcing this well enough should be sufficient, because I somehow can't
see how anyone using advanced monitoring tools could not be subscribed to
one of the mailing lists.
Whats changed ???
When did the consensus achieved in the discussion group become so
unimportant?
Now we hear that "new reporting statistics" will be implemented to
provide "Accurate reporting" ...
....and that brings me to the last bit of history that sums this
whole thing up nicely.... a letter to the core devs from teravus dated
from 11/27 2009
( if you don't feel like tearing through the whole thing... It is a call
to start designing accurate performance measurement metrics into the
fabric of OpenSim rather than relying on fudged stats that might make
users "feel good " about what is reported by the viewer. It also
discusses the absolute NEED for accuracy so performance progress can
be measured, and closes with the fact that the load tests were
ultimately FUTILE without efforts to move forward and CORRECT the
made up numbers)
Teravus Ovares <teravus at gmail.com>
11/27/09
to opensim-dev
Hey there,
A while back, we had somewhat reasonable statistics being generated and
presented to the client. They were not always accurate, but based on
what I saw, I could, pretty much pin certain parts of the simulator as the
limiting factor during load tests.
I'd say, the number 1 reason that they were semi-accurate and not
accurate.. in the past.. is because nobody ever thought about
instrumentation during the functionality design. It was always 'tacked
on later'. One example of this.. is the current AssetCache
implementation. There's no way, currently, to know, at a glance.. how
many external requests it has open. Additionally, it will be extremely
difficult to put one in because of the way the objects are designed and
accessed. To put one in, an event needs to be added to the IAssetService
interface and each AssetCache implementation will need an interlocked int
to count how many gets and puts it currently has open to the external data
source as well as it's own event calling schedule. Then, the
IAssetService property in Scene, (AssetService) will need an event
handler.. which updates the values in SimStatsReporter in Scene
(StatsReporter). This idea of external access resource instrumentation
should really have been built in to the design of the AssetService.
This last recent load test, there were no real statistics that I could use
to determine what the limiting factor was. Time Dilation was pegged at
1.0.. even when the simulator was obviously struggling. Total Frame
time (MS) was -50ms even when the simulation MS was 850ms and the Physics
ms was 250ms, so the inconsistencies made it impossible to know what part
of the simulator was struggling. Agent Updates were erratic.. sometimes
high..
sometimes low when the simulator was fine and when it was struggling.
Pending Uploads and Downloads were always 0, so there was no way to know
how well the simulator was downloading and uploading assets to and from the
grid. Packet stats were non-existant, so there was no way to know how
well the UDP handlers were faring under the load. When it crashed, it
crashed with a mono based stack trace which pointed to out of memory
errors, so the only way that you could, scientifically, find out what the
issue is.. is to run a load test under a memory profiler. We know,
that running a public load test under a memory profiler is quite
impractical.
To make something better, I need to know two things, where it is, and where
I want it to be. How can we make OpenSimulator better if we don't have
statistics that point to where we are currently?
On that note, I propose that, when designing objects for functionality in
OpenSimulator, that we also consider if the objects should be instrumented
and, what would be the best way to go about instrumenting the objects. We
should incorporate instrumentation into the design of the objects. Some
of that instrumentation is appropriate for a client to see, some of it
might not be. Consider that, many of them should be client facing and be
included in the SimStats that get sent to the client.. so that we can
have a reasonable idea of what's going on with a simulator at a glance.
Also, in the design of the instrumentation, we make sure that the
instrumentation is accurate and
lightweight.
The load test went reasonably... but, we didn't get half of the
information on the simulator that we needed to be able to improve it.
Please comment :) I look forward to hearing your responses.
Regards
Teravus
I guess it should be no surprise that the current call to improve and
provide ACCURATE performance statistics reporting should be derided and
dismissed. ( apologies to those members of core who voted +1 ad
helped us push this forward) Not only are members of the communities
calls ( AND contributions) to improve this area of OpenSim ignored, so
are the calls from fellow core devs about what is needed and how it
should be implemented... Forgive me if I seem JUST A BIT CYNICAL that
these corrected stats are forthcoming...
If you are going to *accede to user demands, *maybe you should consider
the effort some of us users put into to getting this patch approved in the
first place.... As far as I can tell we are the ones contributing to
the project by participating in this forum... Please feel free to
forward me some names and email addresses of these clamoring hordes of
unhappy users so I can search for their outrage in my other OpenSim
related groups and invite them to participate in future discussions
here...
On Sat, Nov 7, 2015 at 8:55 PM, AJLDuarte <ajlduarte at sapo.pt> wrote:
> The fudge factor is now a configuration option on the avinationmerge
> branch.
> It can be found in OpenSimDefaults.ini under the name StatisticsFPSfactor,
> and can be set in OpenSim.ini as usual.
> Its default in code is the legacy value of 5.0.
> Current setting on file is temporary 1.0, until we decide on a "final"
> default.
>
> Regards,
> Ubit
>
>
>
> -----Original Message-----
> From: opensim-dev-bounces at opensimulator.org
> [mailto:opensim-dev-bounces at opensimulator.org] On Behalf Of Melanie
> Sent: Sunday, November 08, 2015 02:35
> To: opensim-dev at opensimulator.org
> Subject: Re: [Opensim-dev] Still on Sim and Phys Frames per Second (FPS)
>
> There are too many viewers in the wild, having too many users that are
> unwilling to switch or update, yet complain about "lag" which they do not
> perceive, but which is indicated by a "lag meter" that is geared to measure
> against constants provided by "that grid".
>
> It is a given that the data sent to viewers WILL be changed to allow viewer
> features to work properly again. It is also a given that control over this
> will be given to users of OpenSim, allowing them to see true performance
> data instead of expected data. However, that option can't be the default in
> a world where the primary use of OpenSim is to provide a social virtual
> world.
>
> I had already suggested and here suggest it again to add more data to the
> stats reporting that will track accurate and unfudged data, but doesn't do
> so in fields currently interpreted in accordance to SL standards by ALL
> mainstream viewers.
>
> This will allow viewers which become aware of the new data to use it to
> provide accurate stats and, for instance, make an adaptive "lag meter" in
> place of the current, constants driven one.
>
> The situation where viewer report an ERROR CONDITION because of the desire
> of some to see "accurate" stats can not be sustained because it undermines
> user confidence.
>
> The choices are to accede to user demands while creating a way for viewers
> to get "smarter" or to live in a world where the change is introduced at
> source code level by grid operators without an adequate correct replacement
> stat, therefore locking in the current situation forever.
>
> Please understand that core exists to guide this project in a way that
> allows it's users to work, not in a way that upholds principles over
> people.
>
> - Melanie
>
> On 08/11/2015 02:53, dz wrote:
> > The issue is promoting accurate reporting of basic performance
> > measurement statistics. ( something that has not achieved nearly
> > enough serious attention )
> >
> > Significant money and manpower is currently being directed at efforts
> > to improve simulator performance.
> > It is a simple fact that the continued funding of these efforts
> > relies on documenting the ACTUAL improvement against the ACTUAL
> > original performance characteristics.
> > It is impossible to justify these efforts when the reported numbers
> > are "made up" and THAT fact is not documented except in some
> > obscure comment left behind in the source code.
> >
> > It is unfortunate that the original decision to include a "Fudge
> > factor multiplier" has created a pool of mis-informed users ( including
> myself
> > and the viewer developers ) .
> > This mistake was complicated by the fact that until very recently
> > there was a philosophical divide that prevented OpenSim and viewer
> > developers from cooperating on issues like these.
> > This decision to "play pretend" with performance stats effectively
> > damaged the reporting credibility of everyone who published these
> > inaccurate results, It also created a rift between the OpenSim and
> > viewer developers over the decision to NOT discuss the impact of
> implementing the change.
> > The fact is, there are numerous places in the OpenSim framework
> > where numbers are "made up" just so that a number appears in
> > performance reports. That an effort is being made to correct those
> > sources of mis-information should be welcomed.
> >
> > It seems to me that the decisions made by core should be made in
> > favor of supporting the ongoing efforts to accurately document and
> > improve simulator performance.
> > Justin realized this and lead many of the efforts to add some
> measurement
> > metrics. Even with those efforts, we still cannot measure basic
> > statistics like Events per Second sent to the script engine, or tie
> > those events to whatever script is handling them. This makes
> > identifying the scripts ACTUALLY responsible for "lagging" a region
> > impossible using the traditional TOP SCRIPTS report in region manager
> window.
> >
> > I would agree that a simple solution might be to allow grid managers
> > to add back the Fudge Factor to appease their vocal users, but would
> > disagree that the PROPER decision should be to continue to report
> > inaccurate results. It would be just as easy to implement a
> > multiplier in the viewer code "Lag Meter", This would also allow
> > the accurate reporting of statistics in the Advanced Statistics
> > window and administrative reporting. I believe it was also one of
> > the suggested resolutions put forth by the viewer developers... It
> > should be clear to anyone who has spent time in world that the "lag
> meter" is incorrect...
> > You can walk, build, chat and TP with the same level of sim
> > performance as you could before the numbers were changed. We've
> > overlooked the fact that viewers have behaved differently in OpenSim
> and
> "that other grid"
> > for years. Why is it "all of a sudden" CRITICAL that this one
> viewer
> > feature HAS to be the same? In these days when core developers are
> > releasing viewers, I cannot understand the urgency of accommodating a
> > minor feature of one viewer whose developers have already
> > demonstrated a willingness to work with OpenSim to tailor a
> > configuration to meet our needs.
> >
> >
> >
> > On Sat, Nov 7, 2015 at 1:19 PM, Melanie <melanie at t-data.com> wrote:
> >
> >> The issue here is the so-called "lag meter". Since removal of the
> >> multiplier, this reports all opensim regions as laggy, without
> >> exception. Users' trust in the "lag meter" is damaging OpenSim
> >> reputation. This is not a value that is merely for display; the
> >> viewer uses this value for computations that are then used to "judge"
> >> a sim to be "laggy" if it's below 35 or so fps. OpenSim now always
> >> reports a lesser value. This is damaging and needs to be made
> >> configurable and by default match the viewer's expectations.
> >>
> >> - Melanie
> >>
> >> On 07/11/2015 16:38, Seth Nygard wrote:
> >> > While I understand the arguments surrounding the original decision
> >> > to report values closely matching "the other grid", IMHO doing so
> >> > created an incorrect understanding in many users' minds of how
> >> > things work and/or behave. We are not that other grid and should
> >> > never pretend to be. Had figures been reported correctly in the
> >> > beginning then there would be no confusion now surrounding this
> >> > subject. However avoiding confusion is a poor reason to roll back and
> once again report the
> >> > artificially inflated values. It is better to simply educate and
> make
> >> > it clear that the value of 11fps is indeed the correct value to
> >> > expect, and is in fact the true value things always have ran at
> >> > despite what any inflated reported value said.
> >> >
> >> > It is true that many scripts and tools have already been written to
> >> > use the inflated values but they can all be changed with relative
> >> > ease. The viewers already have many aspects that are different for
> >> > Open Simulator so they can be changed easily as well for new
> >> > versions also with relative ease. All we need to do as a community
> >> > is establish what the correct and expected values are and then
> document
> and communicate them.
> >> >
> >> > As a user, scripter, tool developer, and grid manager, I for one
> >> > want to see true and accurate values for any and all metrics
> >> > regardless of where they are shown or how they may be used. I
> >> > therefore am firmly against rolling back to any older artificially
> inflated values.
> >> >
> >> > Regards
> >> > -Seth
> >> >
> >> >
> >> > _______________________________________________
> >> > Opensim-dev mailing list
> >> > Opensim-dev at opensimulator.org
> >> > http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
> >>
> >> _______________________________________________
> >> Opensim-dev mailing list
> >> Opensim-dev at opensimulator.org
> >> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
> >>
> >
> >
> >
> > _______________________________________________
> > Opensim-dev mailing list
> > Opensim-dev at opensimulator.org
> > http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at opensimulator.org
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at opensimulator.org
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20151108/1dc23012/attachment-0001.html>
More information about the Opensim-dev
mailing list