Collectd

From OpenSimulator

Revision as of 12:56, 2 July 2013 by Ssm2017 (Talk | contribs)

Jump to: navigation, search

Contents

What is collectd ?

Collectd is a daemon which collects system performance statistics periodically and provides mechanisms to store the values in a variety of ways, for example in RRD files. Website

Collectd is a low level system writen in C that is very low consumption. You can use it to read local or distant data or to send data to a central system.

As we are using monitorstats, we do not necessary need to install collectd on the server side and we can use it to track distant simulators.

Collectd can track a lot of formats like database records, parse xml values or json values...

Collectd config for OpenSim

As stated here : Monitoring Module, we can get region statistics using the url : http://sim:port/monitorstats/UUID of the region or region name. We can use the collectd curl_xml plugin to get xml values and store them in a rrd file. To do this, we need to enable the curl_xml plugin and define some dataset types.

Add data types

  • Create a file /etc/collectd/my_types.db
  • Edit the file and add inside (i have put only generic limits but feel free to edit this page and adjust the ranges) : 
opensim_agent_count	value:GAUGE:0:U
opensim_child_agent_count	value:GAUGE:0:U
opensim_gc_memory	value:GAUGE:0:U
opensim_object_count	value:GAUGE:0:U
opensim_physics_frame	value:GAUGE:0:U
opensim_physics_update	value:GAUGE:0:U
opensim_pws_memory	value:GAUGE:0:U
opensim_thread_count	value:GAUGE:0:U
opensim_total_frame	value:GAUGE:0:U
opensim_event_frame	value:GAUGE:0:U
opensim_land_frame	value:GAUGE:0:U
opensim_last_frame	value:GAUGE:0:U
opensim_time_dilatation	value:GAUGE:0:U
opensim_sim_fps	value:GAUGE:0:U
opensim_physics_fps	value:GAUGE:0:U
opensim_agent_updates_per_sercond	value:GAUGE:0:U
opensim_object_updates_per_second	value:GAUGE:0:U
opensim_active_object_count	value:GAUGE:0:U
opensim_active_scripts	value:GAUGE:0:U
opensim_script_events_per_second	value:GAUGE:0:U
opensim_in_paquets_per_second	value:GAUGE:0:U
opensim_out_paquets_per_second	value:GAUGE:0:U
opensim_unacked_bytes	value:GAUGE:0:U
opensim_pending_downloads	value:GAUGE:0:U
opensim_pending_uploads	value:GAUGE:0:U
opensim_total_frame_time	value:GAUGE:0:U
opensim_net_frame_time	value:GAUGE:0:U
opensim_physics_frame_time	value:GAUGE:0:U
opensim_simulation_frame_time	value:GAUGE:0:U
opensim_agent_frame_time	value:GAUGE:0:U
opensim_images_frame_time	value:GAUGE:0:U
  • Edit the file /etc/collectd/collectd.conf :
Uncomment this line 
TypesDB "/usr/share/collectd/types.db" "/etc/collectd/my_types.db"

Add plugin options

  • Edit the file /etc/collectd/collectd.conf :
Add these lines 
LoadPlugin curl_xml
<Plugin "curl_xml">
  <URL "http://localhost:9000/monitorstats/myregion">
    Instance "opensim"
 
    <XPath "/data">
      Type "opensim_agent_count"
      ValuesFrom "AgentCountMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_child_agent_count"
      ValuesFrom "ChildAgentCountMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_gc_memory"
      ValuesFrom "GCMemoryMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_object_count"
      ValuesFrom "ObjectCountMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_physics_frame"
      ValuesFrom "PhysicsFrameMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_physics_update"
      ValuesFrom "PhysicsUpdateFrameMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_pws_memory"
      ValuesFrom "PWSMemoryMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_thread_count"
      ValuesFrom "ThreadCountMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_total_frame"
      ValuesFrom "TotalFrameMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_event_frame"
      ValuesFrom "EventFrameMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_land_frame"
      ValuesFrom "LandFrameMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_last_frame"
      ValuesFrom "LastFrameTimeMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_time_dilatation"
      ValuesFrom "TimeDilationMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_sim_fps"
      ValuesFrom "SimFPSMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_physics_fps"
      ValuesFrom "PhysicsFPSMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_agent_updates_per_sercond"
      ValuesFrom "AgentUpdatesPerSecondMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_active_object_count"
      ValuesFrom "ActiveObjectCountMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_active_scripts"
      ValuesFrom "ActiveScriptsMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_script_events_per_second"
      ValuesFrom "ScriptEventsPerSecondMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_in_paquets_per_second"
      ValuesFrom "InPacketsPerSecondMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_out_paquets_per_second"
      ValuesFrom "OutPacketsPerSecondMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_unacked_bytes"
      ValuesFrom "UnackedBytesMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_pending_downloads"
      ValuesFrom "PendingDownloadsMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_pending_uploads"
      ValuesFrom "PendingUploadsMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_total_frame_time"
      ValuesFrom "TotalFrameTimeMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_net_frame_time"
      ValuesFrom "NetFrameTimeMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_physics_frame_time"
      ValuesFrom "PhysicsFrameTimeMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_simulation_frame_time"
      ValuesFrom "SimulationFrameTimeMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_agent_frame_time"
      ValuesFrom "AgentFrameTimeMonitor/text()"
    </XPath>
 
    <XPath "/data">
      Type "opensim_images_frame_time"
      ValuesFrom "ImagesFrameTimeMonitor/text()"
    </XPath>
 
  </URL>
</Plugin>
  • Save the file and restart collectd

Other options

Local config folder

You can create a folder at /etc/collectd/conf.d and then add a line : 

Include "/etc/collectd/conf.d/*.conf"

at the end of the /etc/collectd/collectd.conf file.

With this option, you can move all the opensimulator config inside a file like /etc/collectd/conf.d/opensimulator.conf

Interval

The default sampling interval is 10 seconds but you can change the value at the begining of the /etc/collectd/collectd.conf file.

Syslog

By default, the collectd daemon is writing its logs in the syslog file but you can set a custom log file like this : 

LoadPlugin logfile
#LoadPlugin syslog
 
<Plugin logfile>
        LogLevel "info"
#       File STDOUT
        File "/var/log/collectd.log"
        Timestamp true
        PrintSeverity false
</Plugin>

Small bug correction

Depending on the collectd version installed, you can have some update time errors set in the log output. To correct this, you can create a file /etc/collectd/conf.d/df_debug.conf and add this inside : 

#source : http://emacstragic.net/collectd-causing-rrd-illegal-attempt-to-update-using-time-errors/
<Plugin df>
        FSType "rootfs"
        IgnoreSelected true
</Plugin>

How to read the values

All the values are stored in a folder at /usr/lib/collectd/rrd in "rrd" files. You can use any rrd compatible reader. You can also set a web server to read the values in real time (there are some ruby or java or perl or php systems for that). You can also zip all the rrd folder content and take it home then read it with an rrd reader like "kcollectd".

Example output :

This output is not beautifull because it was made using an old OpenSimulator version on an empty test machine but you can see some values anyway.

Example output

Retrieved from "http://opensimulator.org/index.php?title=Collectd&oldid=32909"
Personal tools
General
About This Wiki