Collectd
From OpenSimulator
(Difference between revisions)
m |
m (→Collectd config for OpenSim) |
||
| (6 intermediate revisions by one user not shown) | |||
| Line 1: | Line 1: | ||
== What is collectd ? == | == What is collectd ? == | ||
| − | Collectd is a daemon which collects system performance statistics periodically and provides mechanisms to store the values in a variety of ways, for example in RRD files. [http://collectd.org Website] | + | Collectd is a daemon which collects system performance statistics periodically and provides mechanisms to store the values in a variety of ways, for example in RRD files. [http://collectd.org Website] |
| − | + | Collectd is a low level system writen in C that is very low consumption. You can use it to read local or distant data or to send data to a central system. | |
| − | As stated here : [[Monitoring Module]], we can get region statistics using the url : http://sim:port/monitorstats/UUID of the region or region name. We can use the collectd curl_xml plugin to get xml values and store them in a rrd file. To do this, we need to enable the curl_xml plugin and define some dataset types. | + | As we are using monitorstats, we do not necessary need to install collectd on the server side and we can use it to track distant simulators. |
| + | |||
| + | Collectd can track a lot of formats like database records, parse xml values or json values... | ||
| + | |||
| + | == Collectd config for OpenSimulator == | ||
| + | |||
| + | As stated here : [[Monitoring Module]], we can get region statistics using the url : http://sim:port/monitorstats/UUID of the region or region name. We can use the collectd curl_xml plugin to get xml values and store them in a rrd file. To do this, we need to enable the curl_xml plugin and define some dataset types. | ||
=== Add data types<br> === | === Add data types<br> === | ||
| Line 66: | Line 72: | ||
ValuesFrom "AgentCountMonitor/text()" | ValuesFrom "AgentCountMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_child_agent_count" | Type "opensim_child_agent_count" | ||
ValuesFrom "ChildAgentCountMonitor/text()" | ValuesFrom "ChildAgentCountMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_gc_memory" | Type "opensim_gc_memory" | ||
ValuesFrom "GCMemoryMonitor/text()" | ValuesFrom "GCMemoryMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_object_count" | Type "opensim_object_count" | ||
ValuesFrom "ObjectCountMonitor/text()" | ValuesFrom "ObjectCountMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_physics_frame" | Type "opensim_physics_frame" | ||
ValuesFrom "PhysicsFrameMonitor/text()" | ValuesFrom "PhysicsFrameMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_physics_update" | Type "opensim_physics_update" | ||
ValuesFrom "PhysicsUpdateFrameMonitor/text()" | ValuesFrom "PhysicsUpdateFrameMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_pws_memory" | Type "opensim_pws_memory" | ||
ValuesFrom "PWSMemoryMonitor/text()" | ValuesFrom "PWSMemoryMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_thread_count" | Type "opensim_thread_count" | ||
ValuesFrom "ThreadCountMonitor/text()" | ValuesFrom "ThreadCountMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_total_frame" | Type "opensim_total_frame" | ||
ValuesFrom "TotalFrameMonitor/text()" | ValuesFrom "TotalFrameMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_event_frame" | Type "opensim_event_frame" | ||
ValuesFrom "EventFrameMonitor/text()" | ValuesFrom "EventFrameMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_land_frame" | Type "opensim_land_frame" | ||
ValuesFrom "LandFrameMonitor/text()" | ValuesFrom "LandFrameMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_last_frame" | Type "opensim_last_frame" | ||
ValuesFrom "LastFrameTimeMonitor/text()" | ValuesFrom "LastFrameTimeMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_time_dilatation" | Type "opensim_time_dilatation" | ||
ValuesFrom "TimeDilationMonitor/text()" | ValuesFrom "TimeDilationMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_sim_fps" | Type "opensim_sim_fps" | ||
ValuesFrom "SimFPSMonitor/text()" | ValuesFrom "SimFPSMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_physics_fps" | Type "opensim_physics_fps" | ||
ValuesFrom "PhysicsFPSMonitor/text()" | ValuesFrom "PhysicsFPSMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_agent_updates_per_sercond" | Type "opensim_agent_updates_per_sercond" | ||
ValuesFrom "AgentUpdatesPerSecondMonitor/text()" | ValuesFrom "AgentUpdatesPerSecondMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_active_object_count" | Type "opensim_active_object_count" | ||
ValuesFrom "ActiveObjectCountMonitor/text()" | ValuesFrom "ActiveObjectCountMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_active_scripts" | Type "opensim_active_scripts" | ||
ValuesFrom "ActiveScriptsMonitor/text()" | ValuesFrom "ActiveScriptsMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_script_events_per_second" | Type "opensim_script_events_per_second" | ||
ValuesFrom "ScriptEventsPerSecondMonitor/text()" | ValuesFrom "ScriptEventsPerSecondMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_in_paquets_per_second" | Type "opensim_in_paquets_per_second" | ||
ValuesFrom "InPacketsPerSecondMonitor/text()" | ValuesFrom "InPacketsPerSecondMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_out_paquets_per_second" | Type "opensim_out_paquets_per_second" | ||
ValuesFrom "OutPacketsPerSecondMonitor/text()" | ValuesFrom "OutPacketsPerSecondMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_unacked_bytes" | Type "opensim_unacked_bytes" | ||
ValuesFrom "UnackedBytesMonitor/text()" | ValuesFrom "UnackedBytesMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_pending_downloads" | Type "opensim_pending_downloads" | ||
ValuesFrom "PendingDownloadsMonitor/text()" | ValuesFrom "PendingDownloadsMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_pending_uploads" | Type "opensim_pending_uploads" | ||
ValuesFrom "PendingUploadsMonitor/text()" | ValuesFrom "PendingUploadsMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_total_frame_time" | Type "opensim_total_frame_time" | ||
ValuesFrom "TotalFrameTimeMonitor/text()" | ValuesFrom "TotalFrameTimeMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_net_frame_time" | Type "opensim_net_frame_time" | ||
ValuesFrom "NetFrameTimeMonitor/text()" | ValuesFrom "NetFrameTimeMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_physics_frame_time" | Type "opensim_physics_frame_time" | ||
ValuesFrom "PhysicsFrameTimeMonitor/text()" | ValuesFrom "PhysicsFrameTimeMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_simulation_frame_time" | Type "opensim_simulation_frame_time" | ||
ValuesFrom "SimulationFrameTimeMonitor/text()" | ValuesFrom "SimulationFrameTimeMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_agent_frame_time" | Type "opensim_agent_frame_time" | ||
ValuesFrom "AgentFrameTimeMonitor/text()" | ValuesFrom "AgentFrameTimeMonitor/text()" | ||
</XPath> | </XPath> | ||
| − | + | ||
<XPath "/data"> | <XPath "/data"> | ||
Type "opensim_images_frame_time" | Type "opensim_images_frame_time" | ||
| Line 223: | Line 224: | ||
*Save the file and restart collectd | *Save the file and restart collectd | ||
| − | <br> | + | == Other options == |
| + | |||
| + | === Local config folder === | ||
| + | |||
| + | You can create a folder at /etc/collectd/conf.d and then add a line : | ||
| + | |||
| + | <source lang="apache"> | ||
| + | Include "/etc/collectd/conf.d/*.conf" | ||
| + | </source> | ||
| + | at the end of the /etc/collectd/collectd.conf file. | ||
| + | |||
| + | With this option, you can move all the opensimulator config inside a file like /etc/collectd/conf.d/opensimulator.conf | ||
| + | |||
| + | === Interval === | ||
| + | The default sampling interval is 10 seconds but you can change the value at the begining of the /etc/collectd/collectd.conf file. | ||
| + | |||
| + | === Syslog === | ||
| + | By default, the collectd daemon is writing its logs in the syslog file but you can set a custom log file like this : | ||
| + | <source lang="apache"> | ||
| + | LoadPlugin logfile | ||
| + | #LoadPlugin syslog | ||
| + | |||
| + | <Plugin logfile> | ||
| + | LogLevel "info" | ||
| + | # File STDOUT | ||
| + | File "/var/log/collectd.log" | ||
| + | Timestamp true | ||
| + | PrintSeverity false | ||
| + | </Plugin> | ||
| + | </source> | ||
| + | |||
| + | === Small bug correction === | ||
| + | Depending on the collectd version installed, you can have some update time errors set in the log output. To correct this, you can create a file /etc/collectd/conf.d/df_debug.conf and add this inside : | ||
| + | <source lang="apache"> | ||
| + | #source : http://emacstragic.net/collectd-causing-rrd-illegal-attempt-to-update-using-time-errors/ | ||
| + | <Plugin df> | ||
| + | FSType "rootfs" | ||
| + | IgnoreSelected true | ||
| + | </Plugin> | ||
| + | </source> | ||
| + | |||
| + | == How to read the values == | ||
| + | All the values are stored in a folder at /usr/lib/collectd/rrd in "rrd" files. You can use any rrd compatible reader. You can also set a web server to read the values in real time (there are some ruby or java or perl or php systems for that). You can also zip all the rrd folder content and take it home then read it with an rrd reader like "kcollectd". | ||
| + | |||
| + | == Example output :<br> == | ||
| + | |||
| + | This output is not beautifull because it was made using an old OpenSimulator version on an empty test machine but you can see some values anyway. | ||
| + | |||
| + | [[Image:Monitoring_Module-collectd.jpg|Example output]] | ||
Latest revision as of 12:57, 2 July 2013
Contents |
[edit] What is collectd ?
Collectd is a daemon which collects system performance statistics periodically and provides mechanisms to store the values in a variety of ways, for example in RRD files. Website
Collectd is a low level system writen in C that is very low consumption. You can use it to read local or distant data or to send data to a central system.
As we are using monitorstats, we do not necessary need to install collectd on the server side and we can use it to track distant simulators.
Collectd can track a lot of formats like database records, parse xml values or json values...
[edit] Collectd config for OpenSimulator
As stated here : Monitoring Module, we can get region statistics using the url : http://sim:port/monitorstats/UUID of the region or region name. We can use the collectd curl_xml plugin to get xml values and store them in a rrd file. To do this, we need to enable the curl_xml plugin and define some dataset types.
[edit] Add data types
- Create a file /etc/collectd/my_types.db
- Edit the file and add inside (i have put only generic limits but feel free to edit this page and adjust the ranges) :
opensim_agent_count value:GAUGE:0:U opensim_child_agent_count value:GAUGE:0:U opensim_gc_memory value:GAUGE:0:U opensim_object_count value:GAUGE:0:U opensim_physics_frame value:GAUGE:0:U opensim_physics_update value:GAUGE:0:U opensim_pws_memory value:GAUGE:0:U opensim_thread_count value:GAUGE:0:U opensim_total_frame value:GAUGE:0:U opensim_event_frame value:GAUGE:0:U opensim_land_frame value:GAUGE:0:U opensim_last_frame value:GAUGE:0:U opensim_time_dilatation value:GAUGE:0:U opensim_sim_fps value:GAUGE:0:U opensim_physics_fps value:GAUGE:0:U opensim_agent_updates_per_sercond value:GAUGE:0:U opensim_object_updates_per_second value:GAUGE:0:U opensim_active_object_count value:GAUGE:0:U opensim_active_scripts value:GAUGE:0:U opensim_script_events_per_second value:GAUGE:0:U opensim_in_paquets_per_second value:GAUGE:0:U opensim_out_paquets_per_second value:GAUGE:0:U opensim_unacked_bytes value:GAUGE:0:U opensim_pending_downloads value:GAUGE:0:U opensim_pending_uploads value:GAUGE:0:U opensim_total_frame_time value:GAUGE:0:U opensim_net_frame_time value:GAUGE:0:U opensim_physics_frame_time value:GAUGE:0:U opensim_simulation_frame_time value:GAUGE:0:U opensim_agent_frame_time value:GAUGE:0:U opensim_images_frame_time value:GAUGE:0:U
- Edit the file /etc/collectd/collectd.conf :
TypesDB "/usr/share/collectd/types.db" "/etc/collectd/my_types.db"
[edit] Add plugin options
- Edit the file /etc/collectd/collectd.conf :
LoadPlugin curl_xml <Plugin "curl_xml"> <URL "http://localhost:9000/monitorstats/myregion"> Instance "opensim" <XPath "/data"> Type "opensim_agent_count" ValuesFrom "AgentCountMonitor/text()" </XPath> <XPath "/data"> Type "opensim_child_agent_count" ValuesFrom "ChildAgentCountMonitor/text()" </XPath> <XPath "/data"> Type "opensim_gc_memory" ValuesFrom "GCMemoryMonitor/text()" </XPath> <XPath "/data"> Type "opensim_object_count" ValuesFrom "ObjectCountMonitor/text()" </XPath> <XPath "/data"> Type "opensim_physics_frame" ValuesFrom "PhysicsFrameMonitor/text()" </XPath> <XPath "/data"> Type "opensim_physics_update" ValuesFrom "PhysicsUpdateFrameMonitor/text()" </XPath> <XPath "/data"> Type "opensim_pws_memory" ValuesFrom "PWSMemoryMonitor/text()" </XPath> <XPath "/data"> Type "opensim_thread_count" ValuesFrom "ThreadCountMonitor/text()" </XPath> <XPath "/data"> Type "opensim_total_frame" ValuesFrom "TotalFrameMonitor/text()" </XPath> <XPath "/data"> Type "opensim_event_frame" ValuesFrom "EventFrameMonitor/text()" </XPath> <XPath "/data"> Type "opensim_land_frame" ValuesFrom "LandFrameMonitor/text()" </XPath> <XPath "/data"> Type "opensim_last_frame" ValuesFrom "LastFrameTimeMonitor/text()" </XPath> <XPath "/data"> Type "opensim_time_dilatation" ValuesFrom "TimeDilationMonitor/text()" </XPath> <XPath "/data"> Type "opensim_sim_fps" ValuesFrom "SimFPSMonitor/text()" </XPath> <XPath "/data"> Type "opensim_physics_fps" ValuesFrom "PhysicsFPSMonitor/text()" </XPath> <XPath "/data"> Type "opensim_agent_updates_per_sercond" ValuesFrom "AgentUpdatesPerSecondMonitor/text()" </XPath> <XPath "/data"> Type "opensim_active_object_count" ValuesFrom "ActiveObjectCountMonitor/text()" </XPath> <XPath "/data"> Type "opensim_active_scripts" ValuesFrom "ActiveScriptsMonitor/text()" </XPath> <XPath "/data"> Type "opensim_script_events_per_second" ValuesFrom "ScriptEventsPerSecondMonitor/text()" </XPath> <XPath "/data"> Type "opensim_in_paquets_per_second" ValuesFrom "InPacketsPerSecondMonitor/text()" </XPath> <XPath "/data"> Type "opensim_out_paquets_per_second" ValuesFrom "OutPacketsPerSecondMonitor/text()" </XPath> <XPath "/data"> Type "opensim_unacked_bytes" ValuesFrom "UnackedBytesMonitor/text()" </XPath> <XPath "/data"> Type "opensim_pending_downloads" ValuesFrom "PendingDownloadsMonitor/text()" </XPath> <XPath "/data"> Type "opensim_pending_uploads" ValuesFrom "PendingUploadsMonitor/text()" </XPath> <XPath "/data"> Type "opensim_total_frame_time" ValuesFrom "TotalFrameTimeMonitor/text()" </XPath> <XPath "/data"> Type "opensim_net_frame_time" ValuesFrom "NetFrameTimeMonitor/text()" </XPath> <XPath "/data"> Type "opensim_physics_frame_time" ValuesFrom "PhysicsFrameTimeMonitor/text()" </XPath> <XPath "/data"> Type "opensim_simulation_frame_time" ValuesFrom "SimulationFrameTimeMonitor/text()" </XPath> <XPath "/data"> Type "opensim_agent_frame_time" ValuesFrom "AgentFrameTimeMonitor/text()" </XPath> <XPath "/data"> Type "opensim_images_frame_time" ValuesFrom "ImagesFrameTimeMonitor/text()" </XPath> </URL> </Plugin>
- Save the file and restart collectd
[edit] Other options
[edit] Local config folder
You can create a folder at /etc/collectd/conf.d and then add a line :
Include "/etc/collectd/conf.d/*.conf"
at the end of the /etc/collectd/collectd.conf file.
With this option, you can move all the opensimulator config inside a file like /etc/collectd/conf.d/opensimulator.conf
[edit] Interval
The default sampling interval is 10 seconds but you can change the value at the begining of the /etc/collectd/collectd.conf file.
[edit] Syslog
By default, the collectd daemon is writing its logs in the syslog file but you can set a custom log file like this :
LoadPlugin logfile #LoadPlugin syslog <Plugin logfile> LogLevel "info" # File STDOUT File "/var/log/collectd.log" Timestamp true PrintSeverity false </Plugin>
[edit] Small bug correction
Depending on the collectd version installed, you can have some update time errors set in the log output. To correct this, you can create a file /etc/collectd/conf.d/df_debug.conf and add this inside :
#source : http://emacstragic.net/collectd-causing-rrd-illegal-attempt-to-update-using-time-errors/ <Plugin df> FSType "rootfs" IgnoreSelected true </Plugin>
[edit] How to read the values
All the values are stored in a folder at /usr/lib/collectd/rrd in "rrd" files. You can use any rrd compatible reader. You can also set a web server to read the values in real time (there are some ruby or java or perl or php systems for that). You can also zip all the rrd folder content and take it home then read it with an rrd reader like "kcollectd".
[edit] Example output :
This output is not beautifull because it was made using an old OpenSimulator version on an empty test machine but you can see some values anyway.
