Revision as of 10:02, 2 October 2008

Overview

As OpenSim matures, we are extremely interested in adding more automated verification into the OpenSim source tree. Test exist not only to prevent bugs from creeping in, but also to provide fair warning that the behavior of the system has changed, and tests may need updating.

In OpenSim today we use NUnit tests. Our conventions are:

• Tests should not exist inside runtime assemblies, as this makes nunit a production requirement
• Tests should be in .Tests.dll assemblies. For instance, the tests for OpenSim.Data.SQLite.dll should be in the OpenSim.Data.SQLite.Tests.dll assembly. This allows for easy removal of test assemblies in products.
• Tests should be as close to the code as possible, but not intermingled. So the tests for OpenSim/Data/SQLite should be in OpenSim/Data/SQLite/Tests/. Through the use of the Exclude keyword in prebuild.xml you can ensure that directory is part of OpenSim.Data.SQLite.Tests.dll and not OpenSim.Data.SQLite.dll.
• Tests should be able to run safely in a production environment. That means that care must be taken not to damage data on the machine that it is being run.

Writing New Tests

Getting started with NUnit

Integration into the Build Tree

Good / Bad Test practices

Running Tests

Goal

To create a system of automatic building, testing, and simulation so we can have better awareness of our changes to OpenSim.

Summary

To properly test OpenSim, we will need to address a number of objectives:

• Crossplatform Support
• Build Stability
• Code Test
• Code Performance
• Runtime Performance
• Code Style
• Web site results
• Communication

We will be using a number of in-house technologies as well as 3rd party (NUnit, Nant, etc.) to address our challenges. We will strive to maintain all our testing tools in a package that any one can use on any computers with any operating system. For convience, a centralized site for this testing result information on opensim will be created on http://testing.ambientunion.com .

People

• Daedius Moskvitch ( daedius @@@@ daedius.com)

Testing Objectives

Crossplatform Support

There are a number of objectives that should report on which operating system they ran

• Build Stability - Make sure all our builds compile on every operating system
• Code Test - Make sure all our tests run on every operating system
• Code Performance - See how our functions run on every operating system
• Runtime Performance - See how operating systems actually run opensim

Build Stability

Ensure that the build is intact.

Code Test

Use NUnit, NCover

Code Performance

Need to ponder

Runtime Performance

Statistics Server

Code Introspection

Use Gendarme

Web site results

Results of our testing should be accessible web service. This web service will be available via a maintained plugin for build bot so when running a build bot master, one only need a website that can access the web service and properly show it how they wish. Our web service needs to be able to hand off a variety of information:

• Realtime status information of operations
• Crossplatform Support information on tests that have been run for some spiffy grids
• Build results for showing us what happens when things break
• Code test results for showing us what tasts ran
• Code performance for showing us how all our profiled code worked
• Runtime Performance statistics information for bar graphs and charts
• Code Style results for showing which code structure names are non-standard

Communication

We should be able to notify the right people of various important changes (or not so important if they wish) via Email, IRC, etc.

Technologies

• Statistics Server
• NanoBot