Console-less OpenSim

From OpenSimulator

(Difference between revisions)
Jump to: navigation, search
(Added "screen" description)
m (Background)
 
(8 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
__NOTOC__
 +
{{Quicklinks}}
 +
<br />
 +
== Background ==
 +
OpenSimulator uses consoles for each service it's running. It was already possible to run the services(UGAIMR) in the background, while still being able to access the consoles. On Linux we used the convenient screen tool for this. The ideal situation would be though, to run most commands from an external source like a webportal. The region service has an XMLRPC interface, that's called remote admin. While remote admin offers controls to some extent, it's still impossible to let's say shutdown the region service, and restart it from a webpage. This would require a seperate xmlrpc daemon, which has control over the OpenSimulator services. In python, it's quite easy to make such a xmlrpc daemon as you will see below.
  
normally opensim will only run in console mode: whenever you start OpenSim you will end up at a prompt inside OpenSim's very own console &mdash; which is very convenient if you want to manage it that way. it is rather inconvenient if you want to start OpenSim as a daemon in the background (e.g., from <code>/etc/init.d/opensim</code>) and control it via the <code>RemoteAdminPlugin</code>, for example.
+
== Concepts ==
 +
XMLRPC is a powerful, yet simple protocol that's supported by most scripting languages. From all scripting languages, python has the best options to actually create a standalone daemon, that can handle remote xmlrpc calls. With such a standalone daemon, it's simple to link systemcommands to xmlrpc calls. Each simulator would have such a daemon, allowing the controling webportal(like WiXTD) to issue commands to each daemon. This would allow total control from a webportal, like service rebooting, rolling restarts, rolling updates, ugaim on/off, etc. Currently the code below doesn't have any security, but it can be extended to be really secure(SSL, host-based dispatching). It's focussed on a linux platform at the moment, but can be easily modified to work on a windows platform as well.
  
As the suggested backgrounding below doesn't work for everyone, the option of using the package screen is a great alternative. Once screen is installed, use the following command to start the simulator:
+
== MooRPC ==
 +
This code is part of the moo-package on the forge. The script depends on the following custom libraries: [http://pycurl.sourceforge.net/ pycurl], [http://twistedmatrix.com/trac/ twisted], [http://www.jejik.com/articles/2007/02/a_simple_unix_linux_daemon_in_python/ daemon] and [http://www.voidspace.org.uk/python/configobj.html confObj]. It also uses linux-commands, screen and the directory-structure from Moo currently. You're free to modify the code.
  
    screen -S OpenSim -d -m mono OpenSim.exe
+
== Source ==
 
+
<source lang="python">
Once started, you can view the active session(s) listed with <code>screen -ls</code>. To attach a screen, use <code>screen -r Name</code>. From the example above that would be <code>screen -r OpenSim</code>. Once in the screen session, you can detach it by using Ctrl-A,D.
+
 
+
Killing a session without attaching it can be achieved by issuing:
+
 
+
    screen -S OpenSim -r -m  -X quit
+
 
+
{{obsolete}}
+
 
+
as of OpenSim's subversion release r4400 the basic support for running OpenSim without such a console is in the code base and all you need to do is invoke OpenSim with the <code>-background True</code> commandline parameter:
+
 
+
    % mono --debug OpenSim.exe -background True
+
 
+
'''note: that is just a single dash for the <code>-background</code> option!'''
+
 
+
to shutdown a console-less OpenSim instance, use the following <code>shutdown.py</code> script (also located in <code>share/python/console</code>):
+
 
+
<pre>
+
 
#!/usr/bin/python
 
#!/usr/bin/python
# -*- encoding: utf-8 -*-
+
# MooRPC, XMLRPC controller for WiXTD
 
+
# BSD-Licensed
import ConfigParser
+
# Copyright (c) <2009>, <Jeroen van Veen, OS-Networks>
import xmlrpclib
+
# All rights reserved.
import optparse
+
#
import os.path
+
# Redistribution and use in source and binary forms, with or without
 
+
# modification, are permitted provided that the following conditions are met:
if __name__ == '__main__':
+
# * Redistributions of source code must retain the above copyright
    parser = optparse.OptionParser()
+
# notice, this list of conditions and the following disclaimer.
    parser.add_option('-c', '--config', dest = 'config', help = 'config file', metavar = 'CONFIG')
+
# * Redistributions in binary form must reproduce the above copyright
    parser.add_option('-s', '--server', dest = 'server', help = 'URI for the grid server', metavar = 'SERVER')
+
# notice, this list of conditions and the following disclaimer in the
    parser.add_option('-p', '--password', dest = 'password', help = 'password for the grid server', metavar = 'PASSWD')
+
# documentation and/or other materials provided with the distribution.
    (options, args) = parser.parse_args()
+
# * Neither the name of the <organization> nor the
 
+
# names of its contributors may be used to endorse or promote products
    configFile = options.config
+
# derived from this software without specific prior written permission.
    if not configFile:
+
#
        if os.path.isfile(os.path.expanduser('~/.opensim-console.rc')):
+
# THIS SOFTWARE IS PROVIDED BY <OS-Networks> ``AS IS'' AND ANY
            configFile = os.path.expanduser('~/.opensim-console.rc')
+
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
    if not configFile:
+
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
        parser.error('missing option config')
+
# DISCLAIMED. IN NO EVENT SHALL <OS-Networks> BE LIABLE FOR ANY
        sys.exit(1)
+
# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 +
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
 +
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 +
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 +
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 +
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  
    config = ConfigParser.ConfigParser()
+
from twisted.web import xmlrpc, server
    config.readfp(open(configFile))
+
from twisted.internet import reactor
 +
import socket, os, sys, time, xmlrpclib, pycurl
 +
import xmlrpclib, pycurl
 +
import subprocess
 +
sys.path.append("./lib")
 +
from daemon import Daemon
 +
from configobj import ConfigObj
  
     server = config.get('opensim', 'server')
+
try:
     password = config.get('opensim', 'password')
+
     import signal
   
+
    from signal import SIGPIPE, SIG_IGN
     if options.server: server = options.server
+
     signal.signal(signal.SIGPIPE, signal.SIG_IGN)
     if options.password: password = options.password
+
except ImportError:
 +
     pass
 +
try:
 +
     from cStringIO import StringIO
 +
except ImportError:
 +
    from StringIO import StringIO
  
    gridServer = xmlrpclib.Server(server)
+
class CURLTransport(xmlrpclib.Transport):
     res = gridServer.admin_shutdown({'password': password})
+
     xmlrpc_h = [ "Content-Type: text/xml" ]
  
     if res['success'] == 'true':
+
     def __init__(self, username=None, password=None):
         print 'shutdown of %s initiated' % server
+
         self.c = pycurl.Curl()
    else:
+
        self.c.setopt(pycurl.POST, 1)
         print 'shutdown of %s failed' % server
+
         self.c.setopt(pycurl.NOSIGNAL, 1)
 +
        self.c.setopt(pycurl.CONNECTTIMEOUT, 30)
 +
        self.c.setopt(pycurl.HTTPHEADER, self.xmlrpc_h)
 +
        if username != None and password != None:
 +
            self.c.setopt(pycurl.USERPWD, '%s:%s' % (username, password))
 +
        self._use_datetime = False
  
</pre>
+
    def request(self, host, handler, request_body, verbose=0):
 +
        b = StringIO()
 +
        self.c.setopt(pycurl.URL, 'http://%s%s' % (host, handler))
 +
        self.c.setopt(pycurl.POSTFIELDS, request_body)
 +
        self.c.setopt(pycurl.WRITEFUNCTION, b.write)
 +
        self.c.setopt(pycurl.VERBOSE, verbose)
 +
        self.verbose = verbose
 +
        try:
 +
          self.c.perform()
 +
        except pycurl.error, v:
 +
            raise xmlrpclib.ProtocolError(
 +
                host + handler,
 +
                v[0], v[1], None
 +
                )
 +
        b.seek(0)
 +
        return self.parse_response(b)
  
you can either pass in the server URI and password to use via commandline parameters, or you can create <code>.opensim-console.rc</code> in your home directory and set default values:
+
class OSAdmin(xmlrpc.XMLRPC):
 +
    config = ConfigObj('settings.ini')
 +
    radminhost = config['RPC']['radminhost']
 +
    radminpw = config['RPC']['radminpw']
 +
    radminGW = xmlrpclib.ServerProxy(radminhost,transport=CURLTransport())   
 +
    def xmlrpc_admin_broadcast(self,message):
 +
    return self.radminGW.admin_broadcast({"password": self.radminpw,   "message": message})     
 +
    def xmlrpc_admin_create_region(self,region_id,region_name,region_x,region_y,listen_port,region_master_first,region_master_last,region_master_password,listen_ip,external_address,allow_alternate_ports,lastmap_uuid,lastmap_refresh):   
 +
    return self.radminGW.admin_create_region({"password": self.radminpw,"region_id": region_id,"region_name": region_name,"region_x":region_x,"region_y":region_y,"listen_port":listen_port,"region_master_first":region_master_first,"region_master_last":region_master_last,"region_master_password":region_master_password,"listen_ip":listen_ip,"external_address":external_address,"allow_alternate_ports":allow_alternate_ports,"lastmap_uuid":lastmap_uuid,"lastmap_refresh":lastmap_refresh})     
 +
    def xmlrpc_admin_delete_region(self,region_name):   
 +
    return self.radminGW.admin_delete_region({"password": self.radminpw,"region_name": region_name})     
 +
    def xmlrpc_admin_shutdown(self):   
 +
    return self.radminGW.admin_shutdown({"password": self.radminpw})
 +
    def xmlrpc_admin_region_query(self,region_name):   
 +
    return self.radminGW.admin_region_query({"password": self.radminpw,"region_name": region_name})             
 +
    def xmlrpc_start_simulator(self):
 +
    os.chdir('/opt/opensim/simulator')
 +
    cmd = "screen -dmS simulator mono /opt/opensim/simulator/OpenSim.exe"
 +
    os.system(cmd)
 +
    def xmlrpc_ping_simulator(self):
 +
    try:
 +
      print self.radminGW.admin_simhealth({"password": self.radminpw})
 +
    except xmlrpclib.Error, v:
 +
      if str(v)[1:14] == "ProtocolError":
 +
      return 0
 +
      elif str(v)[1:14] == "Fault -32601:":
 +
      return 1
 +
      else:
 +
      return 0
 +
    def xmlrpc_stop_ugaim(self):
 +
    cmd = "screen -S user -p 0 -X stuff $'shutdown\n'"
 +
    os.system(cmd)
 +
    cmd = "screen -S grid -X -p 0 stuff $'shutdown\n'"
 +
    os.system(cmd) 
 +
    cmd = "screen -S asset -X -p 0 stuff $'shutdown\n'"
 +
    os.system(cmd)   
 +
    cmd = "screen -S inventory -p 0 -X stuff $'shutdown\n'"
 +
    os.system(cmd)
 +
    cmd = "screen -S messaging -p 0 -X stuff $'shutdown\n'"
 +
    os.system(cmd)       
 +
    def xmlrpc_start_ugaim(self):
 +
    os.chdir('/opt/opensim/simulator')
 +
    cmd = "screen -dmS user mono /opt/opensim/simulator/OpenSim.Grid.UserServer.exe"
 +
    os.system(cmd)
 +
    time.sleep(4)       
 +
    cmd = "screen -dmS grid mono /opt/opensim/simulator/OpenSim.Grid.GridServer.exe"
 +
    os.system(cmd)
 +
    time.sleep(3)
 +
    cmd = "screen -dmS asset mono /opt/opensim/simulator/OpenSim.Grid.AssetServer.exe"
 +
    os.system(cmd)
 +
    time.sleep(3)
 +
    cmd = "screen -dmS inventory mono /opt/opensim/simulator/OpenSim.Grid.InventoryServer.exe"
 +
    os.system(cmd)
 +
    cmd = "screen -dmS messaging mono /opt/opensim/simulator/OpenSim.Grid.MessagingServer.exe"
 +
    os.system(cmd)                           
 +
    def xmlrpc_ping_ugaim(self):  
 +
    os.chdir('/opt/opensim/simulator')
 +
    cmd = "screen -dmS simulator mono /opt/opensim/simulator/OpenSim.exe"
 +
    os.system(cmd)         
 +
    def xmlrpc_ping(self):
 +
    return 1       
 +
    def xmlrpc_fault(self):
 +
        raise xmlrpc.Fault(123, "The fault procedure is faulty.")
  
<pre>
 
[opensim]
 
server = http://127.0.0.1:9000/
 
password = secret
 
</pre>
 
  
then all you need to do to shutdown your OpenSim server is invoke <code>shutdown.py</code>
+
class RPCDaemon(Daemon):
 +
    def run(self):
 +
    reactor.listenTCP(9996, server.Site(OSAdmin()))
 +
    reactor.run()
  
the goal is to extend <code>shutdown.py</code> into a remote console of sorts.
+
if __name__ == "__main__":
 +
daemon = RPCDaemon('/tmp/rpcMoo.pid')
 +
if len(sys.argv) == 2:
 +
if 'start' == sys.argv[1]:
 +
daemon.start()
 +
elif 'stop' == sys.argv[1]:
 +
daemon.stop()
 +
elif 'restart' == sys.argv[1]:
 +
daemon.restart()
 +
else:
 +
print "Unknown command"
 +
sys.exit(2)
 +
sys.exit(0)
 +
else:
 +
print "usage: %s start|stop|restart" % sys.argv[0]
 +
sys.exit(2)
  
''(adapted from [http://xyzzyxyzzy.net/2008/04/29/console-less-opensim/ xyzzy xyzzy])'''
+
</source>

Latest revision as of 07:07, 9 October 2013


[edit] Background

OpenSimulator uses consoles for each service it's running. It was already possible to run the services(UGAIMR) in the background, while still being able to access the consoles. On Linux we used the convenient screen tool for this. The ideal situation would be though, to run most commands from an external source like a webportal. The region service has an XMLRPC interface, that's called remote admin. While remote admin offers controls to some extent, it's still impossible to let's say shutdown the region service, and restart it from a webpage. This would require a seperate xmlrpc daemon, which has control over the OpenSimulator services. In python, it's quite easy to make such a xmlrpc daemon as you will see below.

[edit] Concepts

XMLRPC is a powerful, yet simple protocol that's supported by most scripting languages. From all scripting languages, python has the best options to actually create a standalone daemon, that can handle remote xmlrpc calls. With such a standalone daemon, it's simple to link systemcommands to xmlrpc calls. Each simulator would have such a daemon, allowing the controling webportal(like WiXTD) to issue commands to each daemon. This would allow total control from a webportal, like service rebooting, rolling restarts, rolling updates, ugaim on/off, etc. Currently the code below doesn't have any security, but it can be extended to be really secure(SSL, host-based dispatching). It's focussed on a linux platform at the moment, but can be easily modified to work on a windows platform as well.

[edit] MooRPC

This code is part of the moo-package on the forge. The script depends on the following custom libraries: pycurl, twisted, daemon and confObj. It also uses linux-commands, screen and the directory-structure from Moo currently. You're free to modify the code.

[edit] Source

#!/usr/bin/python
# MooRPC, XMLRPC controller for WiXTD
# BSD-Licensed
# Copyright (c) <2009>, <Jeroen van Veen, OS-Networks>
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of the <organization> nor the
# names of its contributors may be used to endorse or promote products
# derived from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY <OS-Networks> ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL <OS-Networks> BE LIABLE FOR ANY
# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
from twisted.web import xmlrpc, server
from twisted.internet import reactor
import socket, os, sys, time, xmlrpclib, pycurl
import xmlrpclib, pycurl
import subprocess
sys.path.append("./lib")
from daemon import Daemon
from configobj import ConfigObj
 
try:
    import signal
    from signal import SIGPIPE, SIG_IGN
    signal.signal(signal.SIGPIPE, signal.SIG_IGN)
except ImportError:
    pass
try:
    from cStringIO import StringIO
except ImportError:
    from StringIO import StringIO
 
class CURLTransport(xmlrpclib.Transport):
    xmlrpc_h = [ "Content-Type: text/xml" ]
 
    def __init__(self, username=None, password=None):
        self.c = pycurl.Curl()
        self.c.setopt(pycurl.POST, 1)
        self.c.setopt(pycurl.NOSIGNAL, 1)
        self.c.setopt(pycurl.CONNECTTIMEOUT, 30)
        self.c.setopt(pycurl.HTTPHEADER, self.xmlrpc_h)
        if username != None and password != None:
            self.c.setopt(pycurl.USERPWD, '%s:%s' % (username, password))
        self._use_datetime = False
 
    def request(self, host, handler, request_body, verbose=0):
        b = StringIO()
        self.c.setopt(pycurl.URL, 'http://%s%s' % (host, handler))
        self.c.setopt(pycurl.POSTFIELDS, request_body)
        self.c.setopt(pycurl.WRITEFUNCTION, b.write)
        self.c.setopt(pycurl.VERBOSE, verbose)
        self.verbose = verbose
        try:
           self.c.perform()
        except pycurl.error, v:
            raise xmlrpclib.ProtocolError(
                host + handler,
                v[0], v[1], None
                )
        b.seek(0)
        return self.parse_response(b)
 
class OSAdmin(xmlrpc.XMLRPC):
    config = ConfigObj('settings.ini')
    radminhost = config['RPC']['radminhost']
    radminpw = config['RPC']['radminpw']
    radminGW = xmlrpclib.ServerProxy(radminhost,transport=CURLTransport())    
    def xmlrpc_admin_broadcast(self,message): 
     return self.radminGW.admin_broadcast({"password": self.radminpw,   "message": message})       
    def xmlrpc_admin_create_region(self,region_id,region_name,region_x,region_y,listen_port,region_master_first,region_master_last,region_master_password,listen_ip,external_address,allow_alternate_ports,lastmap_uuid,lastmap_refresh):     
     return self.radminGW.admin_create_region({"password": self.radminpw,"region_id": region_id,"region_name": region_name,"region_x":region_x,"region_y":region_y,"listen_port":listen_port,"region_master_first":region_master_first,"region_master_last":region_master_last,"region_master_password":region_master_password,"listen_ip":listen_ip,"external_address":external_address,"allow_alternate_ports":allow_alternate_ports,"lastmap_uuid":lastmap_uuid,"lastmap_refresh":lastmap_refresh})       
    def xmlrpc_admin_delete_region(self,region_name):     
     return self.radminGW.admin_delete_region({"password": self.radminpw,"region_name": region_name})       
    def xmlrpc_admin_shutdown(self):     
     return self.radminGW.admin_shutdown({"password": self.radminpw}) 
    def xmlrpc_admin_region_query(self,region_name):     
     return self.radminGW.admin_region_query({"password": self.radminpw,"region_name": region_name})               
    def xmlrpc_start_simulator(self): 
     os.chdir('/opt/opensim/simulator')
     cmd = "screen -dmS simulator mono /opt/opensim/simulator/OpenSim.exe"
     os.system(cmd)
    def xmlrpc_ping_simulator(self):
     try:
      print self.radminGW.admin_simhealth({"password": self.radminpw})
     except xmlrpclib.Error, v:
      if str(v)[1:14] == "ProtocolError":
       return 0
      elif str(v)[1:14] == "Fault -32601:":
       return 1
      else:
       return 0
    def xmlrpc_stop_ugaim(self): 
     cmd = "screen -S user -p 0 -X stuff $'shutdown\n'"
     os.system(cmd)
     cmd = "screen -S grid -X -p 0 stuff $'shutdown\n'"
     os.system(cmd)  
     cmd = "screen -S asset -X -p 0 stuff $'shutdown\n'"
     os.system(cmd)    
     cmd = "screen -S inventory -p 0 -X stuff $'shutdown\n'"
     os.system(cmd) 
     cmd = "screen -S messaging -p 0 -X stuff $'shutdown\n'"
     os.system(cmd)        
    def xmlrpc_start_ugaim(self): 
     os.chdir('/opt/opensim/simulator')
     cmd = "screen -dmS user mono /opt/opensim/simulator/OpenSim.Grid.UserServer.exe"
     os.system(cmd) 
     time.sleep(4)         
     cmd = "screen -dmS grid mono /opt/opensim/simulator/OpenSim.Grid.GridServer.exe"
     os.system(cmd)
     time.sleep(3)
     cmd = "screen -dmS asset mono /opt/opensim/simulator/OpenSim.Grid.AssetServer.exe"
     os.system(cmd)
     time.sleep(3)
     cmd = "screen -dmS inventory mono /opt/opensim/simulator/OpenSim.Grid.InventoryServer.exe"
     os.system(cmd) 
     cmd = "screen -dmS messaging mono /opt/opensim/simulator/OpenSim.Grid.MessagingServer.exe"
     os.system(cmd)                             
    def xmlrpc_ping_ugaim(self): 
     os.chdir('/opt/opensim/simulator')
     cmd = "screen -dmS simulator mono /opt/opensim/simulator/OpenSim.exe"
     os.system(cmd)          
    def xmlrpc_ping(self):
     return 1         
    def xmlrpc_fault(self):
        raise xmlrpc.Fault(123, "The fault procedure is faulty.")
 
 
class RPCDaemon(Daemon):
    def run(self):
     reactor.listenTCP(9996, server.Site(OSAdmin()))
     reactor.run()
 
if __name__ == "__main__":
	daemon = RPCDaemon('/tmp/rpcMoo.pid')
	if len(sys.argv) == 2:
		if 'start' == sys.argv[1]:
			daemon.start()
		elif 'stop' == sys.argv[1]:
			daemon.stop()
		elif 'restart' == sys.argv[1]:
			daemon.restart()
		else:
			print "Unknown command"
			sys.exit(2)
		sys.exit(0)
	else:
		print "usage: %s start|stop|restart" % sys.argv[0]
		sys.exit(2)
Personal tools
General
About This Wiki