Introduction

The OpenSimulator stats manager system is the most modern stats implementation in OpenSimulator and will eventually replace all legacy systems. It allows statistics to be added, updated and removed at runtime. Statistics can be viewed via the console "show stats" command and recorded periodically to file for later analysis.

General Concepts

StatsManager acts as the interface between code that have statistics and modules that want to access and display those statistics. At startup, code register their statistics with StatsManager. Then, any code that wishes to access the statistics (whether to display them on the console, put them in databases, or make them available over the network) requests the values from StatsManager. This architecture allows multiple statistic display and analysis systems while also removing the burden of code instrumentation from the display systems. Also, once code is instrumented and registering statistics, those statistics are available to all display systems.

At initialization, code registers its statistics with StatsManager. The full name of a stat has three components, separated by periods (.). In order, these are

• category - The category of the stat (e.g. server, scene)
• container - The containing object for this stat. For example, on scene this is the scene name (e.g. region 1). For server, this is the component to which the stat relates (e.g. memory, processor).
• name - The stat itself. For example, ClientLogoutsDueToNoReceives or SimFPS.

Full names must be unique.

Registered statistics are generally of two types:

• pull - the value of the statistic is "pulled" when the value is requested (using a closure to access the true value location); and
• push - the value is periodically stored in the statistic by the monitored code. There are two types of "push" statistics - the kind that just store a value and those that record events.

In the namespace Opensim.Framework.Monitoring, there is the basic Stats class (invocation described below) that provides the basic push or pull statistic. There is also a CounterStat class designed for routines that want to track events. CounterStats can have EventHistogram classes added to keep histograms of events over time. This latter feature is good for tracking statistics that can change more quickly than any external sampling.

Programmatic Interface

Creating a stat

A statistic is represented by an instance of the OpenSim.Framework.Monitoring.Stat class or one of its descendant classes. Here's an example that creates a stat for recording all the HTTP requests served by the webserver embedded in OpenSimulator.

m_requestsProcessedStat 
    = new Stat(
        "HTTPRequestsServed",
        "Number of inbound HTTP requests processed",
        "The number of inbound HTTP requests processed by the builtin HTTP server.",
        "requests",
        "httpserver",
        Port.ToString(),
        StatType.Pull,
        MeasuresOfInterest.AverageChangeOverTime,
        stat => stat.Value = RequestNumber,
        StatVerbosity.Debug);

This breaks down as follows

• HTTPRequestsServed - The short name of this stat. This is the name used when the stat is displayed or recorded.
• Number of inbound HTTP requests processed - The full name of this stat.
• The number of inbound HTTP requests processed by the builtin HTTP server. - A description of this stat.
• requests - The unit name for this stat.
• httpserver - The category for this stat.
• container - The container for this stat within the category. In this case, we use the port number as the "container" since this distinguishes this data from similar stats on other ports. Another example of a distinguishing container is the region name.
• StatType.Pull - The statistic type. This is either Pull or Push. If Pull, then when a stat is required (e.g. if the user requests the information or it needs to be recorded), then it is gathered by the given pullAction (of which more below). If a statistic if push, then the pull action is null (or will be ignored) and the code registering the stat is responsible for updating the value directly on an ongoing basis.
• MeasuresOfInterest.AverageChangeOverTime - This can be None or ChangeOverTime. If None, then only the raw stat is recorded. If ChangeOverTime, then StatsManager also derives the moving average over the last 24 samples.
• stat => stat.Value = RequestNumber - This is the anonymous function used to generate the stat on a pull request. In this case, the RequestNumber within the HTTP management code is inserted into the stat value.
• StatVerbosity.Debug - StatVerbosity can be Debug or Info. Info should be used for stats that the user will always be interested in seeing (e.g. number of users on a region) and Debug for debugging stats (such as HTTP requests served). However, right now stat display is not filtered on verbosity.

Registering a stat

A created stat must then be registered with the StatsManager. In the case of our example above, this is done as follows.

StatsManager.RegisterStat(m_requestsProcessedStat);

Deregistering a stat

If your code is never disabled during runtime then you don't need to worry about deregistering the stat. However, if it can be removed (e.g. because it is in a region module) then you can deregister a stat as follows.

StatsManager.DeregisterStat(m_requestsProcessedStat);

Fetching stats

There are several interfaces for fetching stats. The easiest is:

StatsManager.TryGetStats(string category, string container, string shortStatName, out Stat)

which accepts the name of a category, container, and the short version of the stat name and returns the statistic. Null is returned if the stat is not found.

There are also methods to return all of the stats within a category (TryGetStatsForCategory) and within a container (GetStatsForEachContainer).

Statistics from the console

From the console, there are several ways to display the stats:

• show stats returns simulator statistics in a legacy format
• show stats all displays all the registered stats one line per stat.
• show stats category container displays all the stats within a particular category/container

Web access to stats

If [Startup]ManagedStatsRemoteFetchURI is defined and set to some value, the simulator statistics are available over the web via HTTP GET operations. The URL is the simulator's HTTP server URL appended with the value in ManagedStatsRemoteFetchURI.

The URL fetched accepts four different query parameters:

• cat which specifies a category to fetch. If no category is given, the special category "all" is use which return all categories;
• cont specified a container. If not specified, the special container "all" is used to return all containers
• stat specifies the name of the particular stat. If not specified, the stat "all" is used to return all stats.
• callback specifies if the returned JSON data is to be wrapped in a callback function.

For instance, for a simulator at IP address 10.1.1.4 with the INI settings:

[network]
   http_listener_port=9000
[Startup]
   ManagedStatsremoteFetchURI=ManagedStats

example data fetches are:

wget http://10.1.1.4:9000/ManagedStats/

returns all stats registered in the simulator

wget http://10.1.1.4:9000/ManagedStats/?cat=server

returns all stats registered with the category "server"

wget http://10.1.1.4:9000/ManagedStats/?cat=server&cont=threadpool

returns all stats for the "threadpool" container within the "server" category

wget http://10.1.1.4:9000/ManagedStats/?cat=server&cont=threadpool&stat=STPMinThreads

returns the value for the specific stat

The data is returned in JSON format grouped as you might expect with maps of categories containing maps of containers containing maps of stats. For instance, the response to http://10.1.1.4:9000/ManagedStats/?cat=server&cont=threadpool might look like:

{"server":
    {"threadpool":
        {"BuiltinThreadpoolIOCPThreadsAvailable":
            {"StatType":"Stat",
            "Category":"server",
            "Container":"threadpool",
            "ShortName":"BuiltinThreadpoolIOCPThreadsAvailable",
            "Name":"BuiltinThreadpoolIOCPThreadsAvailable",
            "Description":"BuiltinThreadpoolIOCPThreadsAvailable",
            "UnitName":"threads",
            "Value":1000.0
            },
        "BuiltinThreadpoolWorkerThreadsAvailable":
            {"StatType":"Stat",
            "Category":"server",
            "Container":"threadpool",
            "ShortName":"BuiltinThreadpoolWorkerThreadsAvailable",
            "Name":"BuiltinThreadpoolWorkerThreadsAvailable",
            "Description":"BuiltinThreadpoolWorkerThreadsAvailable",
            "UnitName":"threads",
            "Value":499.0
            },
        "STPActiveThreads":
            {"StatType":"Stat",
            "Category":"server",
            "Container":"threadpool",
            "ShortName":"STPActiveThreads",
            "Name":"STPActiveThreads",
            "Description":"STPActiveThreads",
            "UnitName":"threads",
            "Value":15.0
            },
        "STPConcurrency":
            {"StatType":"Stat",
            "Category":"server",
            "Container":"threadpool",
            "ShortName":"STPConcurrency",
            "Name":"STPConcurrency",
            "Description":"STPConcurrency",
            "UnitName":"threads",
            "Value":45.0
            },
        "STPInUseThreads":
            {"StatType":"Stat",
            "Category":"server",
            "Container":"threadpool",
            "ShortName":"STPInUseThreads",
            "Name":"STPInUseThreads",
            "Description":"STPInUseThreads",
            "UnitName":"threads",
            "Value":0.0
            },
        "STPMaxThreads":
            {"StatType":"Stat",
            "Category":"server",
            "Container":"threadpool",
            "ShortName":"STPMaxThreads",
            "Name":"STPMaxThreads",
            "Description":"STPMaxThreads",
            "UnitName":"threads",
            "Value":45.0
            },
        "STPMinThreads":
            {"StatType":"Stat",
            "Category":"server",
            "Container":"threadpool",
            "ShortName":"STPMinThreads",
            "Name":"STPMinThreads",
            "Description":"STPMinThreads",
            "UnitName":"threads",
            "Value":15.0
            },
        "STPWorkItemsWaiting":
            {"StatType":"Stat",
            "Category":"server",
            "Container":"threadpool",
            "ShortName":"STPWorkItemsWaiting",
            "Name":"STPWorkItemsWaiting",
            "Description":"STPWorkItemsWaiting",
            "UnitName":"threads",
            "Value":0.0
            }
        }
    }
}