OpenSimulator - User contributions [en]

Development Team

2010-06-08T09:38:00Z

Mikem: /* Active OpenSim Core Developers */

__NOTOC__
{{Template:Quicklinks}}

[[Technical Reference | Technical Reference]] -> [[Technical Reference/terms | Terms]] -> [[Development_Team | Core Development Team]]

== Active OpenSim Core Developers ==
These people have commit access to our central SVN server and are [http://www.ohloh.net/projects/4753/contributors regular contributors] to the codebase.
** '''Only voted in developers should be listed here, please do not list yourself'''
(please add in as much info as you like for your name)

<table border="1" cellpadding="2" cellspacing="0" class="sortable">
<tr>
<th>Photo </th>
<th>IRC Nick </th>
<th>Name</th>
<th>SL Avatar</th>
<th>Other Grid</th>
<th>Time Zone (UTC)</th>
<th>Org</th>
<th>Areas of Interest</th>
</tr>

<tr>
<td />
<td>[[User:MW |MW ]]</td>
<td>Darren</td>
<td>Wright Juran</td>
<td></td>
<td>0</td>
<td></td>
<td>Everything</td>
</tr>
<tr>
<td />
<td>[[User:Adam Frisby|Adam Frisby]]</td>
<td>Adam Frisby</td>
<td>Adam Zaius</td>
<td></td>
<td>+8</td>
<td>DeepThink Pty Ltd</td>
<td>Terrain, Performance</td>
</tr>
<tr>
<td />
<td>[[User:MingChen|MingChen]]</td>
<td>Mike/Michael Ortman</td>
<td>Ming Chen</td>
<td></td>
<td>-6 (-5 in Summer)</td>
<td>DeepThink Pty Ltd</td>
<td>Estate/Parcel Support/Modules/Keeping things all neat and tidy.</td>
</tr>
<tr>
<td />
<td>[[User:SeanDague|sdague]]</td>
<td>Sean Dague</td>
<td>Neas Bade</td>
<td></td>
<td>-5</td>
<td>IBM</td>
<td>Database, Linux, Testing, Misc</td>
</tr>
<tr>
<td />
<td>[[User:Tedd|Tedd]]</td>
<td>Tedd Hansen</td>
<td>Tedd Maa</td>
<td></td>
<td>+1</td>
<td>Tedd Hansen</td>
<td>Programming/Scripting/Architecture</td>
</tr>
<tr>
<td />
<td>ckrinke</td>
<td>Charles Krinke</td>
<td>Charlesk Bing</td>
<td></td>
<td>-8</td>
<td> </td>
<td>Reliability/Grid servers/ll-functions</td>
</tr>
<tr>
<td />
<td>[[User:chi11ken|chi11ken]]</td>
<td>Jeff Ames</td>
<td>Chillken Proto</td>
<td> </td>
<td>+9</td>
<td>[http://www.genkii.com Genkii]</td>
<td> </td>
</tr>
<tr>
<td />
<td>[[User:adjohn|adjohn]]</td>
<td>Adam Johnson</td>
<td>Zeuz Zenovka</td>
<td> </td>
<td>+9</td>
<td>[http://www.genkii.com Genkii]</td>
<td> </td>
</tr>
<tr>
<td />
<td>[[User:joha1|joha1]]</td>
<td>Johan Berntsson</td>
<td>Joppi Brandenburg</td>
<td> </td>
<td>+9</td>
<td>[http://www.3di.jp/en/ 3Di Inc, Japan] http://www.3di.jp/en/</td>
<td>Performance, packet handling/libSL</td>
</tr>
<tr>
<td />
<td>[[User:Teravus|Teravus]]</td>
<td>Daniel Olivares</td>
<td>Teravus Ousley</td>
<td></td>
<td>-5</td>
<td>W3z</td>
<td>Physics & Admin tools, A working sim.</td>
</tr>
<tr>
<td />
<td>[[User:justincc|justincc]]</td>
<td>Justin Clark-Casey</td>
<td>Lulworth Beaumont</td>
<td>Justin Clark-Casey (all other grids)</td>
<td>0</td>
<td>[http://justincc.org/blog justincc's OpenSim blog]</td>
<td>Grid, performance & reliability, inventory (avatar and object), assets, scenes, OARs, etc.</td>
</tr>
<tr>
<td />
<td>[[User:DrScofield|drscofld]]</td>
<td>Dirk Husemann</td>
<td>Dr Scofield</td>
<td></td>
<td>+1</td>
<td>[http://xyzzyxyzzy.net/ xyzzyxyzzy.net]</td>
<td>Reliability, networking protocols, inventory, assets, remote control, voice, and pretty much everything else :-)</td>
</tr>
<tr>
<td />
<td>[[User:dahlia|dahlia]]</td>
<td>T. Hoff</td>
<td>Dahlia Trimble</td>
<td></td>
<td>-8 / -7</td>
<td>Independent</td>
<td>Collision geometry, various math and physics issues, occasional bug fixes and random enhancements</td>
</tr>
<tr>
<td></td>
<td>[[User:Mikem|mikem]]</td>
<td>Mike Mazur</td>
<td> </td>
<td> </td>
<td>+9</td>
<td>Independent</td>
<td>Patches, scripting improvements, LSL compiler</td>
</tr>
<tr>
<td></td>
<td>[[User:Melanie_T|Melanie_T]]</td>
<td>Melanie</td>
<td>Melanie Milland</td>
<td> </td>
<td>+1</td>
<td>Independent</td>
<td>Scripting, Prims/Scene, Life, The Universe, and Everything</td>
</tr>
<tr>
<td></td>
<td>[[User:HomerHorwitz|homerh]]</td>
<td>Homer Horwitz</td>
<td>Homer Horwitz</td>
<td> </td>
<td>+2</td>
<td>Independent</td>
<td>Rev. engineering, "now, that's funny" problems, but still interested in all parts of it</td>
</tr>
<tr>
<td></td>
<td>[[User:Diva|Diva]]</td>
<td>Crista Lopes</td>
<td>Diva Canto</td>
<td>Crista Lopes / Diva Canto</td>
<td>-8</td>
<td>University of California, Irvine</td>
<td>Everything, except databases</td>
</tr>
<tr>
<td />
<td>[[User:nlin|nlin]]</td>
<td>N Lin</td>
<td>Standard Drucker</td>
<td> </td>
<td>+9</td>
<td>[http://www.3di.jp/en/ 3Di Inc, Japan] http://www.3di.jp/en/</td>
<td>Physics, scripting, more to come</td>
</tr>
<tr>
<td />
<td>[[User:arthursv|arthursv]]</td>
<td>Arthur Valadares</td>
<td>Arthur Valadares</td>
<td>NONE</td>
<td>-3</td>
<td>IBM</td>
<td>Unit testing, database plugins, bug fixes, general </td>
</tr>

</table>

== Retired OpenSim core developers ==

These people are core developers who have transcended our mortal plane (i.e. they are no longer active).

<table border="1" cellpadding="2" cellspacing="0" class="sortable">
<tr>
<th>Photo </th>
<th>IRC Nick </th>
<th>Name</th>
<th>SL Avatar</th>
<th>Other Grid</th>
<th>Time Zone (UTC)</th>
<th>Org</th>
<th>Areas of Interest</th>
</tr>
<tr>
<td />
<td>[[User:babblefrog|babblefrog]]</td>
<td>Brian McBee</td>
<td>Dogen Coldstream</td>
<td>Babblefrog Ballistic (osgrid)</td>
<td>-8</td>
<td>Disorganized</td>
<td> </td>
</tr>
<tr>
<td />
<td>[[User:danx0r|danx0r]]</td>
<td>Dan Miller</td>
<td>Albert Pascal</td>
<td></td>
<td>-8</td>
<td>squiggle.com</td>
<td>PHEEZIKS; everything</td>
</tr>
<tr>
<td />
<td>Tleiades</td>
<td> </td>
<td>Tleiades Hax</td>
<td></td>
<td>+1</td>
<td> </td>
<td>Grid servers/Database</td>
</tr>
<tr>
<td />
<td>[[User:Darok|Darok]]</td>
<td> </td>
<td>Darok Kaminski</td>
<td></td>
<td>+1</td>
<td> </td>
<td>Physics engines (especially BulletX)</td>
</tr>
<tr>
<td />
<td>Gareth / Gwen</td>
<td>Gareth Nelson</td>
<td>Gareth Ellison</td>
<td>Gareth Nelson (on everywhere but SL)</td>
<td>BST (UTC+1)</td>
<td>Litesim Ltd</td>
<td>Grid servers, sim border crossing, avatar animations</td>
</tr>
<tr>
<td />
<td>[[User:dalien|dalien]]</td>
<td>Dalien Talbot</td>
<td>Dalien Talbot</td>
<td></td>
<td>+1</td>
<td>Mostly TCP-based</td>
<td>Small fixes; rev.eng./prototyping; nightlies; git-keeper </td>
</tr>
<tr>
<td />
<td>[[Alondria]]</td>
<td></td>
<td>Alondria LeFay</td>
<td>Alondria LeFay (OSGrid)</td>
<td>-8</td>
<td>Independent</td>
<td>Implementation of LSL functions and other scripting tidbits.</td>
</tr>
<tr>
<td> </td>
<td>[[User:lbsa71|lbsa71]]</td>
<td>Stefan Andersson</td>
<td>Tribal Skytower</td>
<td>OSG:Stefan Andersson TN:Stefan Andersson</td>
<td>+1</td>
<td>[http://tribalmedia.se/ Tribal Media AB]</td>
<td>Web Integration</td>
</tr>
</table>

== Additional Developers/Testers/Contributors ==

These people have contributed bug reports, patches or other contributions to OpenSim. '''New comers please add yourself to bottom of the list!'''

{| cellspacing="0" cellpadding="2" border="1" class="wikitable sortable"
|-
! IRC Nick
! Name
! SL Avatar
! Other Grid
! Time Zone (UTC)
! Org
! Areas of Interest
|-
| [[User:Nebadon|Nebadon]]
| Michael Cerquoni
| Nebadon Izumi
| Nebadon Izumi
| -7 Arizona
| Oni Kenkon Creations
| Building, Scripting, Testing
|-
| [[User:Jtclark48|jclark4]]
| Jay Clark
| Jay Clarke
|
| -5
| IBM
| Physics, Grid Host, AI, Scripting, Testing
|-
| [[User:AdamStevenson|BigFootAg]]
| Adam Stevenson
| Adamus Petrov
|
| -6
| Texas A&M University
| AI, Skynet, Evolving Systems, Biology
|-
| [[User:Jeff1564|Jeff1564]]
| Jeff
| Potter Taurog
| Potter Taurog
| -8
| http://myopengrid.com
| Building, Scripting, Testing
|-
| Rock_Vacirca
| Colin Withers
| Rock Vacirca
|  
| +1
| http://rock-vacirca.blogspot.com
| Testing, building, scripting, maintaining an opensim blog.
|-
| simsim
| caocao
|  
|  
| +9
|  
| Testing whole functions of OpenSim system,working with OpenSim-Engine,reporting on OpenSim
|-
| [[User:Vicero Lambert|Vicero Lambert]]
|  
|  
|
|  
|  
|  
|-
| [[User:Magi|Magi]]
| Andy Agnew
| Magi Merlin
|
| +10
| Spun Pty Ltd
| 3D Web Integration, Database stuff and playing with the odds and ends box.
|-
| [[User:ClarkZone|ClarkZone]]
| Troy Admin(@ClarkZone)
| Troy Childs
| Troy Admin (ClarkZone)
| -5
| Http://clarkzone.dyndns.org
| Tester and Grid Host
|-
| [[User:Aiaustin|aiaustin]]
| Ai Austin
| Ai Austin
| Ai AIAI (AIAI Grid)
| +0
| AIAI, University of Edinburgh http://www.aiai.ed.ac.uk/~ai/ http://vue.ed.ac.uk/openvue/
| Windows Vista tests Content testing Use of multiple VWs
|-
| Marc Manders
| Marc Manders
|  
|  
| +6
| marcmanders@gmail.com
| Creative features
|-
| [[User:Balthazar|balthazar]]
| Trevor Brooks
| Balthazar Sin
|
| -5
| None
| Terrains, testing and some small coding tasks
|-
| [[User:Jimbo2120|jimbo2120]]
| Michael Osias
| Illuminous Beltran
|
| -5
| IBM
| Grid, AI, Skynet, coding and testing
|-
| ZeroPoint
|  
| Guilderoy Dench
|
| -5
|  
| Programming/Database
|-
| [[User:DerekTang|DerekTang]]
| Derek Tang
| Derek Timeless
| Derek Tang (ChineseGrid)
| +8
| http://ChineseGrid.net
| Running a public WINDOWS sim for testing, Docs, Helping Chinese users to enjoy OpenSim; building Chinese OpenSim communities. In construction...
|-
| [[User:TayB|TayB]]
| Earl B
| Taylor Boyau
|
| -10
| ViziGrid
| Grid Host,Networking,Contributions & Testing.
|-
| [[User:JamieDav|JamieDav]]
| Jamie David
| Jamie David
|
| +7
| Forum
| Grid, Sim, Avitar, Functionality
|-
| [[User:Krtaylor|Krtaylor]]
| Kurt Taylor
| Kurt Stringer
|
| -6
| IBM
| Grid, Networking, Monitoring, Scripting, Inventory, Testing
|-
| [[User:Nink|Nink]]
| Peter Finn
| Nink Noonan
|
| -5
| IBM
| Disruptive Influence.
|-
| [[User:Bruce|Bruce]]
| Bruce Meerson
| Bruce Meerson
|
| +8
| HiPiHi
| Watching.
|-
| [[User:Darb|DarbD]]
| Brian B. Quinn
| Darb Dabney
| regions near Marin
| PST/SLT (-7 or -8)
| County of Marin, California http://blog.simgis.com
| LiDAR-based sculpties, real-world terrain, pursuit of civic paraverses
|-
| [[CharlieO]]
| Dan
| Charlie Omega
|
|  
|  
| Mild coding/tweaking/simple feature adds, Stress testing/break stuff, Testing limits of existing code. Making sure [[LSL Status]] is up to date
|-
| oobscure
|  
| Opensource Obscure
|
| +1
| http://www.opensim.it
| Running a public Linux sim for testing, Docs, Helping italian users, Building opensim communities, Watching
|-
| pitman
| Mike Pitman
| Rez Tone
|
|  
| IBM
| Scientific visualization schemes, virt world product design, persistant workspaces, virt world based big biz
|-
| Shenlei
| Shenlei Winkler
| Shenlei Flasheart, Shenlei Winkler
|
|  
| Fashion Research Institute
| Product Design and Development, Apparel industry, and o yes, I wrote the book ;)
|-
| cmu
| Christopher Mumme
| Snook Destiny
|
| +1
| http://www.cmu-develop.de/ and research group "Collaboration Systems and CSCW" at Clausthal University of Technology
| Testing OpenSim, working with OpenSim-Engine, reporting on OpenSim
|-
| [[Silpol]]
| Andriy Tymchenko
| Andy Tir
|
| EET (+2/3)
| http://silpol.blogspot.com/ (also visible at Nokia)
| Highly uncoordinated mess with elements of palace games, under-table diplomacy, rebellion, coup d'état and mutiny. optionally pirate
|-
| [[User:Grumly|Grumly]]
|
| Forest Klaar
| Grumly TheBear
| GMT+1
| .NET MCAD Dev/Arch/Trainer http://www.devoteam.com
| Trying to get into OpenSim code for now. Particularly interrested in data persistence. blog (Hello, Avatar!): http://lslblog.free.fr
|-
| [[DaTwitch]]
| James G. Stallings II
| Lazarus Longstaff
| Hiro Protagonist (OSGrid)
| -5
| House Husband
| OSGrid Region owner, OSGrid Operator, Forum Admin, sometime wiki editor
|-
| gryc
| Gryc Ueusp
| Gryc Uriza
| Gryc Uriza(OSGrid)
| -6
|
| PHP scripting, web interfaces, interconnectivity, cross-platformedness
|-
| [[User:Phrearch|Phrearch]]
| Jeroen van Veen
| Phrearch Miles
| Phrearch Miles(OSGrid)
| Amsterdam/Paris
|
| HWIOS, WiXTD, Wikidoc, Moo, User interfaces
|-
| [[User:Burnman|Burnman]]
| Allen Wilkins
| Burnman Bedlam
| Sid Green (United Grid)
| Boston, USA
| United Grid
| Testing, testing, and more testing! Getting familiar with the source, interested in all aspects of the project.
|-
| [[User:Krisbfunk|krisbfunk]]
| Kris Bulman
| Krisbfunk Vought
| Krisbfunk Nocturnal(OSGrid)
| PE, Canada (-4)
| Edactive Technologies NocturnalEye Productions UPEI
| Currently: Testing, bug reports, wiki updating, building on OSGrid
|-
| [[User:HashBox|HashBox]]
|  
| Sibariel Darkstone
| Sibariel Darkstone (OSGrid)
| New Zealand (+12)
|
| Testing, bug reports, and updating the wiki.
|-
| [[User:Kinoc|Kinoc]]
| Kino Coursey
| Daxxon Jaxxon
| Daxxon Kinoc (OSgrid)
| -6
| Daxtron Laboratories http://www.daxtron.com University of North Texas
| AI, Semantic web, Ontologies, Natural Laanguage Processing, Cyc, Bots, NPC
|-
| [[User:Trapuh|trapuh]]
| Pedro Ribeiro
| Vaiten Forder
|
| GMT
| University Student, Escola Superior de Educação de Viseu, Portugal
| Testing, eventual bug reports and wiki. Music, web/digital arts and php+sql.
|-
| [[User:SonicViz|SonicViz]]
| Paul Cohen
| Komuso Tokugawa
|
| +9
| Http://sonicviz.com
| Audio/Music, Interactive Music, Control Protocols, Interfaces, VisualFX, Procedural animation/Generative systems + testing and general dev
|-
| [[User:Mokele|mokele]]
| Scott Norman
| Mokelembembe Mokeev
|
| -8 (Southern California)
| Web Developer (PHP and MySQL)
| Interested in seeing running on PowerPC Macs which it is. So, when I can, I'll compile and test on PowerPC Mac (PowerBook G4) and submit reports and then update the wiki if need on installing on Mac. Also have a Ubuntu 7.10 server that I can do testing on too.
|-
| [[User:Devalnor|devalnor]]
| Devalnor
| M. Watkin
|
| +1 (Belgium)
|
| Small Patch code, bug reports, and updating the wiki.
|-
| [[User:Ezekiel|Ezekiel]]
| Ezekiel
| Ezekiel Zabelin
|
| +1
| http://www.yosims.com
| Concepts, business aspects of virtual worlds - web developer (PHP, MySQL, Javascript, LSL)
|-
| [[User:Buggmaster|Buggmaster]]
| Mike D
| Bug Master
| None
| -8
| http://www.adultmetaverse.com
| Grid, Data/Web PHP/PERL/MySQL
|-
| [[User:Nixnerd|nixnerd]]
|
| Dangerously Moody
| None
| GMT
| http://www.integratedtechnologies.eu
| Cross Platform Testing, Feedback, Bug Reporting
|-
| [[User:MoHax|mohax]]
| Mo Hax
| Mo Hax
|
| -5 Eastern
| IBM
| Testing, Feedback, Content Contributions, Bug Reporting, Documenting, Development
|-
| [[User:Webmage|webmage]]
| webmage
| Leyla Masala
| Web Mage
| +1
| IBM
| Testing, terrain
|-
| [[User:NLStitch|NLStitch]]
| Marijn Oosterveld
| Stitch Seale
| NYA
| GMT +1 Amsterdam
| Twingate Systems (http://www.twingate.nl) HanzeHogeschool Groningen, Netherlands
| Programming, Photography, AI
|-
| [[User:Ideia Boa|Ideia Boa]]
| Joao Lopes
| Ideia Boa
| Ideia Boa or Boa Ideia in some grids
| GTM+1 Stockholm/Sweden
| WorldSimTERRA - Virtual World that speaks Portuguese too http://www.worldsimterra.com
| Testing and more testing! Updating the original wiki and translating the OpenSim Wiki into Portuguese and reporting on OpenSim
|-
| [[User:Lulurun|lulurun]]
| liu
| 
| 
| +9
| 3Di Inc, Japan http://www.3di.jp
| Patches, openid, server performance, UGAI
|-
| [[User:Carlosroundel|Carlosrounde]]
| Carlosroundel
| Carlos Roundel
| 
| +1
| Cyberlandia Italy http://www.cyberlandia.net
| Grid, programmer, database, tester
|-
| [[User:Mikebert|Mikebert]]
| Michael Strunck
| Mikebert Miles
| Mikebert M34
| +1
| OpenSIM Wiki, Germany http://www.opensim.de
| German Wiki, Translater, Server Performance (Linux/Windows), Tester, Feedback, Bug Reporting, Server-Hosting
|-
| [[User:Fly-man-|Fly-Man-]]
| Laurence
|
| Fly Man
| +1
|
| Testing, OpenSimSearch, OpenSimProfile
|-
| Taoki
| Mircea Kitsune / Taoki Vixen
| Mircea Kitsune (OSGrid) / Mircea Lobo (LL grid)
| 
| GMT +2
| 
| Usually testing and bug reporting but I also make smaller patches where I know what to do.
|-
| [[User:Patnad|Patnad]]
| Patrick
| Patnad Babii
| Patnad Babii (OSGrid)
| GMT -5
| RezzMe Technologies http://www.rezzme.com
| Bug testing and reporting, I code C# and have submitted a few patches
|-
| [[User:^DarkMan|^DarkMan]]
| Brian Adair
| Patrick Ouachita
| Brian Adair | Patrick Meta
| -6 CST
| RealMetaLife | B&H Networking
| Building, Scripting, Testing, etc.
|-
| [[User:Tlaukkan|Tommi Laukkanen]]
| Tommi Laukkanen
|  
| Tommi Laukkanen
| +2 GMT
| http://www.bubblecloud.org
| Protocols ([http://www.bubblecloud.org MXP]), NHibernate, Scrip API, Map Generation, Bug Fixes, Grid Hosting
|-
| [[User:Mystical|Mystical]]
| Kevin Tweedy
| Mystical Demina
| Mystical Demina
| -5
| Extreme Reality Grid http://www.XRGrid.com
| Windows Communication Framework, Windows Workflow,Entity Framework, MSSQL Enhancements,Commerce, Content,DotNetNuke based portal, development services
|-
| [[User:Godfrey|Godfrey]]
| Jeff Lee
| Warin Cascabel
|
| -5 (EST5EDT)
|
| Testing, minor bugfixes. Scripting, building, animating
|-
| Jamenai
| Christopher Händler
| Jamenai Luik
| Jamenai Luik
| +1
| Playneko Grid | XIMDEX Jamenai http://www.playneko.de http://www.ximdex.de
| Performance,Bug Reporting, Hosting, Grid-Owner,(PHP, MySQL, Perl, JavaScript, LSL)
|-
| [[User:Bikcmp|bikcmp]]
| Jason
| Jake1500 Allen
| Jason Helios (The Helios Grid)
| EST
| Blue Software
| Search, groups, land, and currency
|-
| [[User:Mark.malewski|Slipaway]]
| Mark Malewski
| Chris Rock
|
| -6 (-5 during summer - CDT)
| NexTECH / Joopla
| Web development & systems integration, terrain, WIKI documentation, tutorials, testing, bug reporting and feedback.
|-
| barakademi
| Steve Topp
| barakademi Barzane
| same avi on baragrid OSgrid Grid4us sciencesim
| utc+1 (CET) paris
| http://xbot-sl.barakademi.org http://vps.barakademi.org/oswi http://vps.barakademi.org/oswi/loginscreen.php
| Music LiveMusic MetaverseMusic Opensim Libomv Mono-2.4 Linux (suse,debian,ubuntu) Admin Scripting Automating Development Intergration php mysql bash nant +++
|-
| [[User:RemedyTomm|RemedyTomm]]
| Tom Grimshaw
| Tomm Remedy
| KGrid: Casper Warden OSGrid: Tomm Remedy
| UTC+0 (BST)
| Remedy Communications
| Texture pipeline, Groups, ObjectUpdates
|-
| [[User:Robert d|robert_d]]
| Robert Dzikowski
|
| OSGrid: robert_d 13
| UTC+1
| [http://blog.rd-it.net http://blog.rd-it.net]
| Region Modules, Tutorials
|-
| [[User:Snoopy2|Snoopy2]]
| Snoopy Pfeffer
| Snoopy Pfeffer
| Snoopy Pfeffer
|
| [http://www.3dmetaverse.com/ http://www.3dmetaverse.com/]
| Region Hosting, Open Metaverse, 3D Web, server and viewers, service management
|-
| john_
| John Moyer
| VAJohn GeekSquad or Matthew Kendal
|
| -5
| Best Buy/Geek Squad
| Tester
|-
| [[User:W!cKeD|_WicKeD]]
| Maik
| Maik Galaxy
| El Diablo
| +1 Germany
| Creatio Inc. / [http://www.OpenSimGerman.us/ OpenSimGerman.us]
| German Support, Translator, Building, Scripting, Testing, Hosting
|-
| [[User:Stevie Wakowski|Stevie Wakowksi]]
| Steve Roberts
| Stevie Wakowski
|
| +10 Australia
| IBM
| OpenSim builds, Linux, Modrex, bug reporting, evangalist for OpenSim in business applications.
|-
[[User:Revolution]]
| Revolution
| Matthew
| Revolution Smythe
| Revolution Smythe
| -6 Central USA
| None
| Script engine, physics engine, general odd bugs, interesting and odd things
|- 
|}

[[Category:Main]] [[Category:Development]] [[Category:Tech_Reference]] [[Category:Help]]

Talk:Automated Testing

2009-03-12T08:55:16Z

Mikem:

I crossed out a paragraph about naming tests a certain way in order to make sure they're run in a certain order because they depend on each other. This is bad practice. All tests should be able to run independently of one another. For any environment setup, use the SetUp and TearDown NUnit attributes.

[[User:Mikem|Mikem]] 01:01, 9 October 2008 (PDT)

I changed this back, because it's the way the unit tests work today. The "theory" that no test method should depend on any other one tends to force unit tests into very shallow coverage because you would need massive code duplication to test anything deep. My experience has shown that imposing the "no test should really on another test" generates much worse test cases.

If you want to go through the unit tests and adjust them to all have Setup and Teardown methods, be my guest. :) However it really just makes things much more difficult than they need to be, for lots of effort, and no real benefit.

---

While my experience hasn't yet shown that test which depend on other tests are the ultimate evil, I'm confident that one should strive to avoid such a situation. The argument against chained tests stating that massive code duplication is necessary to test anything deep isn't convincing because good use of test fixture SetUp/TearDown eliminates most of the duplication.

Chained tests probably do have their place in extremely complicated systems, however I'm not convinced the components of OpenSim that are being tested have ''not'' reached such a level of complexity.

[[User:Mikem|Mikem]] 08:54, 12 March 2009 (UTC)

Talk:Automated Testing

2009-03-12T08:54:31Z

Mikem: /* = */

I crossed out a paragraph about naming tests a certain way in order to make sure they're run in a certain order because they depend on each other. This is bad practice. All tests should be able to run independently of one another. For any environment setup, use the SetUp and TearDown NUnit attributes.

[[User:Mikem|Mikem]] 01:01, 9 October 2008 (PDT)

I changed this back, because it's the way the unit tests work today. The "theory" that no test method should depend on any other one tends to force unit tests into very shallow coverage because you would need massive code duplication to test anything deep. My experience has shown that imposing the "no test should really on another test" generates much worse test cases.

If you want to go through the unit tests and adjust them to all have Setup and Teardown methods, be my guest. :) However it really just makes things much more difficult than they need to be, for lots of effort, and no real benefit.

---

While my experience hasn't yet shown that test which depend on other tests are the ultimate evil, I'm confident that one should strive to avoid such a situation. The argument against chained tests stating that massive code duplication is necessary to test anything deep isn't convincing because good use of test fixture SetUp/TearDown eliminates most of the duplication.

Chained tests probably do have their place in extremely complicated systems, however I'm not convinced the components of OpenSim that are being tested have reached such a level of complexity.

[[User:Mikem|Mikem]] 08:54, 12 March 2009 (UTC)

Talk:Automated Testing

2009-03-12T08:54:17Z

Mikem:

I crossed out a paragraph about naming tests a certain way in order to make sure they're run in a certain order because they depend on each other. This is bad practice. All tests should be able to run independently of one another. For any environment setup, use the SetUp and TearDown NUnit attributes.

[[User:Mikem|Mikem]] 01:01, 9 October 2008 (PDT)

I changed this back, because it's the way the unit tests work today. The "theory" that no test method should depend on any other one tends to force unit tests into very shallow coverage because you would need massive code duplication to test anything deep. My experience has shown that imposing the "no test should really on another test" generates much worse test cases.

If you want to go through the unit tests and adjust them to all have Setup and Teardown methods, be my guest. :) However it really just makes things much more difficult than they need to be, for lots of effort, and no real benefit.

===

While my experience hasn't yet shown that test which depend on other tests are the ultimate evil, I'm confident that one should strive to avoid such a situation. The argument against chained tests stating that massive code duplication is necessary to test anything deep isn't convincing because good use of test fixture SetUp/TearDown eliminates most of the duplication.

Chained tests probably do have their place in extremely complicated systems, however I'm not convinced the components of OpenSim that are being tested have reached such a level of complexity.

[[User:Mikem|Mikem]] 08:54, 12 March 2009 (UTC)

Talk:Automated Testing

2009-03-12T08:53:06Z

Mikem: /* Chained tests are not necessary */ new section

I crossed out a paragraph about naming tests a certain way in order to make sure they're run in a certain order because they depend on each other. This is bad practice. All tests should be able to run independently of one another. For any environment setup, use the SetUp and TearDown NUnit attributes.

[[User:Mikem|Mikem]] 01:01, 9 October 2008 (PDT)

I changed this back, because it's the way the unit tests work today. The "theory" that no test method should depend on any other one tends to force unit tests into very shallow coverage because you would need massive code duplication to test anything deep. My experience has shown that imposing the "no test should really on another test" generates much worse test cases.

If you want to go through the unit tests and adjust them to all have Setup and Teardown methods, be my guest. :) However it really just makes things much more difficult than they need to be, for lots of effort, and no real benefit.

== Chained tests are not necessary ==

While my experience hasn't yet shown that test which depend on other tests are the ultimate evil, I'm confident that one should strive to avoid such a situation. The argument against chained tests stating that massive code duplication is necessary to test anything deep isn't convincing because good use of test fixture SetUp/TearDown eliminates most of the duplication.

Chained tests probably do have their place in extremely complicated systems, however I'm not convinced the components of OpenSim that are being tested have reached such a level of complexity.

Automated Testing

2009-03-12T08:47:32Z

Mikem:

As OpenSim matures, we are extremely interested in adding more automated verification into the OpenSim source tree. Test exist not only to prevent bugs from creeping in, but also to provide fair warning that the behavior of the system has changed, and tests may need updating.

In OpenSim today we use NUnit tests. Our conventions are:
# Tests should '''not''' exist inside runtime assemblies, as this makes nunit a production requirement
# Tests should be in .Tests.dll assemblies. For instance, the tests for OpenSim.Data.SQLite.dll should be in the OpenSim.Data.SQLite.Tests.dll assembly. This allows for easy removal of test assemblies in products.
# Tests should be as close to the code as possible, but not intermingled. So the tests for OpenSim/Data/SQLite should be in OpenSim/Data/SQLite/Tests/. Through the use of the '''Exclude''' keyword in prebuild.xml you can ensure that directory is part of OpenSim.Data.SQLite.Tests.dll and not OpenSim.Data.SQLite.dll. See exclude clarification in writing unit tests section.
# Tests testing a class should be grouped into a test class file called xxxTest.cs, where xxx is the name of the class that is being tested.
# Tests should be able to run safely in a production environment. That means that care must be taken not to damage data on the machine that it is being run.
# Tests should be deterministic in other words repeatable. Avoid randomness in tests. See good and bad testing practices below.

== Core Functionality Missing Unit Tests ==

This is a list of functionality which is not covered by unit tests and is identified as highly desireable test target:

# Database Modules (These are mysql tables)
## region ban
## land
## landaccesslist

== Good / Bad Test practices ==
Creating good tests is an art, not a science. Tests are useful by how many bugs they find or how many bugs they avoid. Things you should think about in creating good tests is:
* Throwing edge cases, like 0, "", or Null at parameters. This ensures that people functions are hardened against incomplete data. Many of our crashes come from the lack of this hardening showing up at just the wrong time.
* Use stateful testing to build up complex scenarios. This is more useful than just cursory get / set calls.
* Random tests are not a good idea. We need test results to be deterministic. In other words tests need to be repeatable. If you want to test for a range it is good idea to make separate tests for min and max values. Random values in fields can fail randomly. When something goes wrong for example in database schema the developer will not necessarily notice if the stored values are random. On the other hand its hard to troubleshoot randomly failing tests as you dont know which specific value caused the failure.
* Tests should be independent and should not rely on another test being run, passing or failing. An excerpt from [http://xunitpatterns.com/Principles%20of%20Test%20Automation.html#Independent%20Test xUnit Patterns]:
<blockquote>If tests are interdependent and (even worse) order dependent, we will be depriving ourselves of the useful feedback test failures provide. Interacting Tests [...] tend to fail in a group. The failure of a test that moved the [subject under test] into the state required by the dependent test will lead to the failure of the dependent test too. With both tests failing, how can we tell if it is because of a problem in code that both rely on in some way or is it a problem in code that only the first relies on. With both tests failing we can't tell. We are only talking about two tests here. Imagine how much worse this is with tens or hundreds of tests.</blockquote>
* Only one function of the subject under test should be tested in one test. When testing a database access object, for example, write separate tests for creating DB entries, updating them and removing them.
* Do not use the subject under test to set up the state for the test or to verify the result. Use a different method. When testing a database access object, for example, use raw SQL to insert the initial data into the DB, then run the method being tested. To verify if the operation was successful, use raw SQL again to verify the DB changed as expected.

== Writing Tests ==

See [http://www.nunit.org/index.php?p=quickStart&r=2.4 NUnit Quick Start] for an introduction to unit testing with NUnit.

Writing a new unit test is pretty easy, and very helpful in increasing the stability of opensim by nailing down bugs. I'm going to present an example here of SQLite Asset testing to show how simple such a test case is to write. The actual in tree SQLite Asset tests are a little different because the code was factored out so that it was easily applied to any database driver, so don't be concerned with the fact that what you see here isn't in the tree.

Exclude clarification: Make sure your master project (not the test project) has an entry for files like the following so that the test code is not included in the master project dll:

<pre>
<Files>
<Match pattern="*.cs" recurse="true">
<Exclude name="Tests" pattern="Tests" />
</Match>
</Files>
</pre>

=== NUnit Conventions ===
An NUnit test suite:
* is a class with a default constructor (takes no arguments)
* has public methods that are tests
* uses annotations to determine what are tests
* runs it's tests in '''alphabetical order by method name'''

An NUnit test method:
* must be public
* must return void
* must take no arguments
* is successful if no exception or assert is thrown while running it

The run order is important if you want to have early tests that setup some complicated state (like creating objects), and have later tests remove or update that state. For that reason I find it very helpful to name all test methods '''Txxx_somename''' where '''xxx''' is a number between 000 and 999. That guaruntees no surprises in run order.

=== Fixture Setup / Teardown ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[TestFixtureSetUp]
public void Init()
{
uuid1 = UUID.Random();
uuid2 = UUID.Random();
uuid3 = UUID.Random();
name1 = "asset one";
name2 = "asset two";
name3 = "asset three";

asset1 = new byte[100];
asset1.Initialize();
file = Path.GetTempFileName() + ".db";
connect = "URI=file:" + file + ",version=3";
db = new SQLiteAssetData();
db.Initialise(connect);
}

[TestFixtureTearDown]
public void Cleanup()
{
db.Dispose();
System.IO.File.Delete(file);
}
</source>

In the case of testing something like the database layer, we have to actually attempt to store / retrieve things from a database. Following from rule #4 of good tests, we want to make sure not to touch the production databases to run our tests, so during startup we generate a temporary file name which is guaranteed not to be an existing file on the system, and use that as our database file name. By running db.Initialize() the OpenSim migration code will correctly populate that database with the latest schema.

Once we are done with the tests we want to make sure we aren't leaving garbage temp files on the user's system. So we remove that file we created.

During setup we also create a set of state variables, such as 3 uuids, 3 strings, and a data block. You could have always just stuck these inline, but variables are there for a reason, so use them.

=== Test Setup / Teardown ===

What's missing in [[Example Test SQLite Assets]] are individual test Setup and Teardown methods. These methods allow each test to be completely self sufficient without the code duplication needed to set up the test environment at the start of each test.

Let's assume the <code>SQLiteAssetData</code> class provided a <code>FetchAsset()</code> method and a <code>UpdateAsset()</code> method. Since every test should be independent of any other test, and <code>FetchAsset()</code> and <code>UpdateAsset()</code> should be tested in separate tests, that means each test would need to create its own entries in the asset table in order to succeed. You may have something like this (see [[Example Test SQLite Assets#This Test is Flawed|Example Test SQLite Assets]] for an explanation of <code>sqldb.executeSQL()</code>):

<source lang="csharp">[Test]
public void T101_TestFetchAsset()
{
AssetBase a1 = new AssetBase(...);
sqldb.executeSQL("INSERT INTO assets VALUES({0}, ...)", a1.uuid, ...);

AssetBase a1_actual = db.FetchAsset(a1.uuid);

Assert.Equal(a1_actual.uuid, a1.uuid);
Assert.Equal(a1_actual.Name, a1.Name);
// etc
}

[Test]
public void T103_TestUpdateAsset()
{
AssetBase a1 = new AssetBase(...);
sqldb.executeSQL("INSERT INTO assets VALUES({0}, ...)", a1.uuid, ...);

a1.Name = "new name";

db.UpdateAsset(a1.uuid, a1);

AssetBase a1_actual = sqldb.executeSQL("SELECT * FROM assets WHERE uuid = {0}", a1.uuid);

Assert.Equal(a1_actual.uuid, a1.uuid);
Assert.Equal(a1_actual.Name, a1.Name);
// etc
}
</source>

You will note that both tests have the same code at the top in which they create an entry in the assets table. This duplicate code can be factored out into a Setup method, which is called before every test is executed (assume <code>a1</code> is a class attribute):

<source lang="csharp">
[SetUp]
public void SetUp()
{
a1 = new AssetBase(...);
sqldb.executeSQL("INSERT INTO assets VALUES({0}, ...)", a1.uuid, ...);
}

[TearDown]
public void TearDown()
{
// clean up after ourselves so the next test has a clean DB to start with
sqldb.executeSQL("DELETE FROM assets");
}

[Test]
public void T101_TestFetchAsset()
{
AssetBase a1_actual = db.FetchAsset(a1.uuid);

Assert.Equal(a1_actual.uuid, a1.uuid);
Assert.Equal(a1_actual.Name, a1.Name);
// etc
}

[Test]
public void T103_TestUpdateAsset()
{
a1.Name = "new name";

db.UpdateAsset(a1.uuid, a1);

AssetBase a1_actual = sqldb.executeSQL("SELECT * FROM assets WHERE uuid = {0}", a1.uuid);

Assert.Equal(a1_actual.uuid, a1.uuid);
Assert.Equal(a1_actual.Name, a1.Name);
// etc
}
</source>

Also note the <code>TearDown()</code> method; it is called after each test has run, regardless whether the test passed or failed. It deletes all the entries in the <code>assets</code> table so that there is no leftover data in the database to interfere with the next test.

==== Multiple Setup Methods ====

Not all setup and teardown must happen in methods declared [SetUp] and [TearDown]. It may be useful to provide methods which perform part of the setup and to call them from whichever test may need it:

<source lang="csharp">
private AssetBase InsertAssetWithRandomData(UUID assetUuid)
{
AssetBase asset = new AssetBase(assetUuid);
asset.Name = somethingRandom();
asset.Data = somethingRandom();

sqldb.executeSQL("INSERT INTO assets VALUES({0}, ...)", asset.uuid, ...);

return asset;
}

[Test]
public void T201_TestNeedsTwoAssets()
{
AssetBase a1 = InsertAssetWithRandomData(uuid1);
AssetBase a2 = InsertAssetWithRandomData(uuid2);
// etc
}

[Test]
public void T203_TestNeedsFiveAssets()
{
AssetBase a1 = InsertAssetWithRandomData(uuid1);
AssetBase a2 = InsertAssetWithRandomData(uuid2);
AssetBase a3 = InsertAssetWithRandomData(uuid3);
AssetBase a4 = InsertAssetWithRandomData(uuid4);
AssetBase a5 = InsertAssetWithRandomData(uuid5);
// etc
}
</source>

Notice that <code>InsertAssetWithRandomData</code> is <code>private</code> as it's only called from within the class.

=== Asserts ===
You will see scattered through the code '''Assert.That(...)'''. These will throw an exception if the condition is not valid. This format of assertions is called the [http://www.nunit.org/index.php?p=constraintModel&r=2.4 Constraint Model] in NUnit, and provides a large number of tests with the flavor of:
* Assert.That(foo, Is.Null)
* Assert.That(foo, Is.Not.Null)
* Assert.That(foo, Is.True)
* Assert.That(foo, Is.EqualTo(bar))
* Assert.That(foo, Text.Matches( "*bar*" ))

Of note, Is.EqualTo uses the Equals function of foo, so this can only be used on objects that are IComparable. Most of the OpenSim base objects are not, so you'll have to compare fields manually in tests.

For the complete set of conditions you can use see [http://www.nunit.org/index.php?p=constraintModel&r=2.4 the Constraint Model NUnit documentation]. While there is another syntax for tests, the Constraint Model is preferred as it is far more human readable.

=== Simple Negative Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">[Test]
public void T001_LoadEmpty()
{
Assert.That(db.ExistsAsset(uuid1), Is.False);
Assert.That(db.ExistsAsset(uuid2), Is.False);
Assert.That(db.ExistsAsset(uuid3), Is.False);
}
</source>

Test T001 is an example of a simple negative test. We assume a new database will not have any of those assets in them. While the value of this test may look low, it does provide a baseline in ensuring that the database connection is there, that these return false correctly, and that no other exception is thrown. Negative tests are a good way to force bounds conditions and ensure that not only does it ''return what you expect'' it also ''doesn't return what you don't expect''. Thought of another way, it ensures your code is somewhat defensive in nature, not coughing on bad or unexpected data.

=== Simple Positive Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[Test]
public void T010_StoreSimpleAsset()
{
AssetBase a1 = new AssetBase(uuid1, name1);
AssetBase a2 = new AssetBase(uuid2, name2);
AssetBase a3 = new AssetBase(uuid3, name3);
a1.Data = asset1;
a2.Data = asset1;
a3.Data = asset1;

db.CreateAsset(a1);
db.CreateAsset(a2);
db.CreateAsset(a3);

AssetBase a1a = db.FetchAsset(uuid1);
Assert.That(a1a.ID, Is.EqualTo(uuid1));
Assert.That(a1a.Name, Is.EqualTo(name1));

AssetBase a2a = db.FetchAsset(uuid2);
Assert.That(a2a.ID, Is.EqualTo(uuid2));
Assert.That(a2a.Name, Is.EqualTo(name2));

AssetBase a3a = db.FetchAsset(uuid3);
Assert.That(a3a.ID, Is.EqualTo(uuid3));
Assert.That(a3a.Name, Is.EqualTo(name3));
}
</source>
T010 is an example of a simple positive test. In it we create and store 3 assets (ensuring no exceptions), then load those 3 assets back from the database and ensure the fields are correct. Because AssetBase is not IComparible we just check the ID and Name fields with equals tests. If any of the Asserts fail, the whole test fails.

=== Stateful Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[Test]
public void T011_ExistsSimpleAsset()
{
Assert.That(db.ExistsAsset(uuid1), Is.True);
Assert.That(db.ExistsAsset(uuid2), Is.True);
Assert.That(db.ExistsAsset(uuid3), Is.True);
}
</source>

T011 is an example of a stateful test, because it requires the state created by T010 (i.e. the creation of those 3 objects). In order to test any kind of complicated scenario you will find that you need to use stateful tests to build up various amounts of state (testing along the way), then manipulating and possibly tearing it down. Without doing this you can't do truly deep testing of function in any complex environment. This example isn't very stateful (I tried to pick an easy example), but it should give you some ideas.

=== Speculative Tests ===
Speculative tests are tests that might or might not apply in a given situation. MySQL testing in the OpenSim tree is done by speculative testing, the tests will only run if there is a properly configured database, otherwise they will not be run. If you execute '''Assert.Ignore()''' in a '''Test''' the test will end and be ignored. If you run '''Assert.Ignore()''' in the '''TestFixtureSetup''' all tests in the test fixture will be skipped and ignored.

Speculative testing lets you create tests that require certain preconditions to be met which you can't guarantee on all platforms/configuration, and are an important part of deep testing.

== Adding Tests to the Tree ==
As we said previously all tests for assembly OpenSim.Foo (in directory OpenSim/Foo) should:
* be in assembly OpenSim.Foo.Tests.dll
* not be in the OpenSim.Foo.dll assembly
* be in OpenSim/Foo/Tests directory

Also, if you have created a new test assembly you must add references to it in both ''.nant/local.include'' and ''.nant/bamboo.build'' to ensure that the assembly is added to the automated bamboo runs as well as the nant test target.

== Executing Tests ==

=== Bamboo ===
On every commit to opensim all the tests are run on a [http://opensimulator.org:8085 Bamboo build server on opensimulator.org]. The process takes about 5 minutes to build, test, and report the results back out on #opensim-dev via the osmantis bot.

=== Nant ===
You can manually run all the tests for OpenSim on your system by running '''nant test''' as a nant target. This will run all the tests that are in the tree, though some speculative tests might be ignored if your platform does not have the right features or configuration to run these tests.

=== NUnit Console ===
If you only want to run tests for one assembly you can do that using the NUnit Console. On Linux just run '''nunit-console2 OpenSim.Foo.Tests.dll''' and it will run only the tests for OpenSim.Foo.Tests.dll. If you are only making changes to 1 dll and just want a quick sanity check, this is the fastest way to do that.

=== Debugging Tests ===
There is a special page dedicated to this. See [[Debugging Unit Tests]].

== Learning More ==
You should definitely read the documentation at the [http://www.nunit.org/index.php?p=documentation NUnit homepage] if you want to know more about testing. It's a very good reference for all the APIs in NUnit that you can use for creating tests.

== Code Coverage ==

A prototype has been included using monocov, which has a profile built-in in mono. Instructions for using monocov can be found on the Mono homepage, in [http://www.mono-project.com/Code_Coverage Code Coverage] section. Unfortunately nunit2 and nant do not support code coverage with monocov (only some other proprietary code coverages), so there is no <monocov> tag. The solution was to implement it using many nunit-console and on <exec> tags.

ATTENTION: Code coverage only works with mono 1.2.x , any other version will most likely not work. Code coverage development is being put on hold for now until it supports newer mono versions.

=== Running ===

Download [http://primates.ximian.com/~lupus/monocov-0.2.tar.gz monocov] and [http://prdownloads.sourceforge.net/nunit/NUnit-2.4.8-net-2.0.zip?download nunit-console] if your nunit-console does not work. To test if it works (my Ubuntu's Hardy version did not), just run one of the tests with nunit-console.

Install monocov (./configure, ./make install, there could be some minor conflicts, I had to add to the compiling line -I/usr/include/mono-1.0/) and make sure the working nunit-console.exe is in /usr/lib/nunit/nunit-console.exe. Now just run nant test-cov and it will generate .cov files and HTML directories in the cov directory. The .cov files can be seen by running monocov on them, and have the same information as the HTML directories.

== Testing Todo ==

=== Coverage ===

A prototype has been done and documented. Now we must keep a look if monocov will evolve to support newer mono versions.

[[Category:Development]]
[[Category:Testing]]

== Links to More Information on Unit Testing ==

* [http://www.nunit.org/index.php?p=quickStart&r=2.4 NUnit Quick Start]
* [http://xunitpatterns.com/ xUnit Patterns book homepage], with lots of information on good practices, patterns, code smells, etc.
* There is a lot of information on unit testing at the [http://c2.com/cgi/wiki?search=unittest Cunningham & Cunningham, Inc wiki at c2.com]

Automated Testing

2009-03-12T08:42:54Z

Mikem: /* Writing Tests */

Automated Testing

2009-03-12T08:32:29Z

Mikem: /* Test Setup / Teardown */

As OpenSim matures, we are extremely interested in adding more automated verification into the OpenSim source tree. Test exist not only to prevent bugs from creeping in, but also to provide fair warning that the behavior of the system has changed, and tests may need updating.

In OpenSim today we use NUnit tests. Our conventions are:
# Tests should '''not''' exist inside runtime assemblies, as this makes nunit a production requirement
# Tests should be in .Tests.dll assemblies. For instance, the tests for OpenSim.Data.SQLite.dll should be in the OpenSim.Data.SQLite.Tests.dll assembly. This allows for easy removal of test assemblies in products.
# Tests should be as close to the code as possible, but not intermingled. So the tests for OpenSim/Data/SQLite should be in OpenSim/Data/SQLite/Tests/. Through the use of the '''Exclude''' keyword in prebuild.xml you can ensure that directory is part of OpenSim.Data.SQLite.Tests.dll and not OpenSim.Data.SQLite.dll. See exclude clarification in writing unit tests section.
# Tests testing a class should be grouped into a test class file called xxxTest.cs, where xxx is the name of the class that is being tested.
# Tests should be able to run safely in a production environment. That means that care must be taken not to damage data on the machine that it is being run.
# Tests should be deterministic in other words repeatable. Avoid randomness in tests. See good and bad testing practices below.

== Core Functionality Missing Unit Tests ==

This is a list of functionality which is not covered by unit tests and is identified as highly desireable test target:

# Database Modules (These are mysql tables)
## region ban
## land
## landaccesslist

== Good / Bad Test practices ==
Creating good tests is an art, not a science. Tests are useful by how many bugs they find or how many bugs they avoid. Things you should think about in creating good tests is:
* Throwing edge cases, like 0, "", or Null at parameters. This ensures that people functions are hardened against incomplete data. Many of our crashes come from the lack of this hardening showing up at just the wrong time.
* Use stateful testing to build up complex scenarios. This is more useful than just cursory get / set calls.
* Random tests are not a good idea. We need test results to be deterministic. In other words tests need to be repeatable. If you want to test for a range it is good idea to make separate tests for min and max values. Random values in fields can fail randomly. When something goes wrong for example in database schema the developer will not necessarily notice if the stored values are random. On the other hand its hard to troubleshoot randomly failing tests as you dont know which specific value caused the failure.
* Tests should be independent and should not rely on another test being run, passing or failing. An excerpt from [http://xunitpatterns.com/Principles%20of%20Test%20Automation.html#Independent%20Test xUnit Patterns]:
<blockquote>If tests are interdependent and (even worse) order dependent, we will be depriving ourselves of the useful feedback test failures provide. Interacting Tests [...] tend to fail in a group. The failure of a test that moved the [subject under test] into the state required by the dependent test will lead to the failure of the dependent test too. With both tests failing, how can we tell if it is because of a problem in code that both rely on in some way or is it a problem in code that only the first relies on. With both tests failing we can't tell. We are only talking about two tests here. Imagine how much worse this is with tens or hundreds of tests.</blockquote>
* Only one function of the subject under test should be tested in one test. When testing a database access object, for example, write separate tests for creating DB entries, updating them and removing them.
* Do not use the subject under test to set up the state for the test or to verify the result. Use a different method. When testing a database access object, for example, use raw SQL to insert the initial data into the DB, then run the method being tested. To verify if the operation was successful, use raw SQL again to verify the DB changed as expected.

== Writing Tests ==
Writing a new unit test is pretty easy, and very helpful in increasing the stability of opensim by nailing down bugs. I'm going to present an example here of SQLite Asset testing to show how simple such a test case is to write. The actual in tree SQLite Asset tests are a little different because the code was factored out so that it was easily applied to any database driver, so don't be concerned with the fact that what you see here isn't in the tree.

Exclude clarification: Make sure your master project (not the test project) has an entry for files like the following so that the test code is not included in the master project dll:

<pre>
<Files>
<Match pattern="*.cs" recurse="true">
<Exclude name="Tests" pattern="Tests" />
</Match>
</Files>
</pre>

=== NUnit Conventions ===
An NUnit test suite:
* is a class with a default constructor (takes no arguments)
* has public methods that are tests
* uses annotations to determine what are tests
* runs it's tests in '''alphabetical order by method name'''

An NUnit test method:
* must be public
* must return void
* must take no arguments
* is successful if no exception or assert is thrown while running it

The run order is important if you want to have early tests that setup some complicated state (like creating objects), and have later tests remove or update that state. For that reason I find it very helpful to name all test methods '''Txxx_somename''' where '''xxx''' is a number between 000 and 999. That guaruntees no surprises in run order.

==== Fixture Setup / Teardown ====

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[TestFixtureSetUp]
public void Init()
{
uuid1 = UUID.Random();
uuid2 = UUID.Random();
uuid3 = UUID.Random();
name1 = "asset one";
name2 = "asset two";
name3 = "asset three";

asset1 = new byte[100];
asset1.Initialize();
file = Path.GetTempFileName() + ".db";
connect = "URI=file:" + file + ",version=3";
db = new SQLiteAssetData();
db.Initialise(connect);
}

[TestFixtureTearDown]
public void Cleanup()
{
db.Dispose();
System.IO.File.Delete(file);
}
</source>

In the case of testing something like the database layer, we have to actually attempt to store / retrieve things from a database. Following from rule #4 of good tests, we want to make sure not to touch the production databases to run our tests, so during startup we generate a temporary file name which is guaranteed not to be an existing file on the system, and use that as our database file name. By running db.Initialize() the OpenSim migration code will correctly populate that database with the latest schema.

Once we are done with the tests we want to make sure we aren't leaving garbage temp files on the user's system. So we remove that file we created.

During setup we also create a set of state variables, such as 3 uuids, 3 strings, and a data block. You could have always just stuck these inline, but variables are there for a reason, so use them.

==== Test Setup / Teardown ====

What's missing in [[Example Test SQLite Assets]] are individual test Setup and Teardown methods. These methods allow each test to be completely self sufficient without the code duplication needed to set up the test environment at the start of each test.

Let's assume the <code>SQLiteAssetData</code> class provided a <code>FetchAsset()</code> method and a <code>UpdateAsset()</code> method. Since every test should be independent of any other test, and <code>FetchAsset()</code> and <code>UpdateAsset()</code> should be tested in separate tests, that means each test would need to create its own entries in the asset table in order to succeed. You may have something like this (see [[Example Test SQLite Assets#This Test is Flawed|Example Test SQLite Assets]] for an explanation of <code>sqldb.executeSQL()</code>):

<source lang="csharp">[Test]
public void T101_TestFetchAsset()
{
AssetBase a1 = new AssetBase(...);
sqldb.executeSQL("INSERT INTO assets VALUES({0}, ...)", a1.uuid, ...);

AssetBase a1_actual = db.FetchAsset(a1.uuid);

Assert.Equal(a1_actual.uuid, a1.uuid);
Assert.Equal(a1_actual.Name, a1.Name);
// etc
}

[Test]
public void T103_TestUpdateAsset()
{
AssetBase a1 = new AssetBase(...);
sqldb.executeSQL("INSERT INTO assets VALUES({0}, ...)", a1.uuid, ...);

a1.Name = "new name";

db.UpdateAsset(a1.uuid, a1);

AssetBase a1_actual = sqldb.executeSQL("SELECT * FROM assets WHERE uuid = {0}", a1.uuid);

Assert.Equal(a1_actual.uuid, a1.uuid);
Assert.Equal(a1_actual.Name, a1.Name);
// etc
}
</source>

You will note that both tests have the same code at the top in which they create an entry in the assets table. This duplicate code can be factored out into a Setup method, which is called before every test is executed (assume <code>a1</code> is a class attribute):

<source lang="csharp">
[SetUp]
public void SetUp()
{
a1 = new AssetBase(...);
sqldb.executeSQL("INSERT INTO assets VALUES({0}, ...)", a1.uuid, ...);
}

[TearDown]
public void TearDown()
{
// clean up after ourselves so the next test has a clean DB to start with
sqldb.executeSQL("DELETE FROM assets");
}

[Test]
public void T101_TestFetchAsset()
{
AssetBase a1_actual = db.FetchAsset(a1.uuid);

Assert.Equal(a1_actual.uuid, a1.uuid);
Assert.Equal(a1_actual.Name, a1.Name);
// etc
}

[Test]
public void T103_TestUpdateAsset()
{
a1.Name = "new name";

db.UpdateAsset(a1.uuid, a1);

AssetBase a1_actual = sqldb.executeSQL("SELECT * FROM assets WHERE uuid = {0}", a1.uuid);

Assert.Equal(a1_actual.uuid, a1.uuid);
Assert.Equal(a1_actual.Name, a1.Name);
// etc
}
</source>

Also note the <code>TearDown()</code> method; it is called after each test has run, regardless whether the test passed or failed. It deletes all the entries in the <code>assets</code> table so that there is no leftover data in the database to interfere with the next test.

Not all setup and teardown must happen in methods declared [SetUp] and [TearDown]. It may be useful to provide methods which perform part of the setup and to call them from whichever test may need it:

<source lang="csharp">
private AssetBase InsertAssetWithRandomData(UUID assetUuid)
{
AssetBase asset = new AssetBase(assetUuid);
asset.Name = somethingRandom();
asset.Data = somethingRandom();

sqldb.executeSQL("INSERT INTO assets VALUES({0}, ...)", asset.uuid, ...);

return asset;
}

[Test]
public void T201_TestNeedsTwoAssets()
{
AssetBase a1 = InsertAssetWithRandomData(uuid1);
AssetBase a2 = InsertAssetWithRandomData(uuid2);
// etc
}

[Test]
public void T203_TestNeedsFiveAssets()
{
AssetBase a1 = InsertAssetWithRandomData(uuid1);
AssetBase a2 = InsertAssetWithRandomData(uuid2);
AssetBase a3 = InsertAssetWithRandomData(uuid3);
AssetBase a4 = InsertAssetWithRandomData(uuid4);
AssetBase a5 = InsertAssetWithRandomData(uuid5);
// etc
}
</source>

Notice that <code>InsertAssetWithRandomData</code> is <code>private</code> as it's only called from within the class.

==== Asserts ====
You will see scattered through the code '''Assert.That(...)'''. These will throw an exception if the condition is not valid. This format of assertions is called the [http://www.nunit.org/index.php?p=constraintModel&r=2.4 Constraint Model] in NUnit, and provides a large number of tests with the flavor of:
* Assert.That(foo, Is.Null)
* Assert.That(foo, Is.Not.Null)
* Assert.That(foo, Is.True)
* Assert.That(foo, Is.EqualTo(bar))
* Assert.That(foo, Text.Matches( "*bar*" ))

Of note, Is.EqualTo uses the Equals function of foo, so this can only be used on objects that are IComparable. Most of the OpenSim base objects are not, so you'll have to compare fields manually in tests.

For the complete set of conditions you can use see [http://www.nunit.org/index.php?p=constraintModel&r=2.4 the Constraint Model NUnit documentation]. While there is another syntax for tests, the Constraint Model is preferred as it is far more human readable.

=== Simple Negative Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">[Test]
public void T001_LoadEmpty()
{
Assert.That(db.ExistsAsset(uuid1), Is.False);
Assert.That(db.ExistsAsset(uuid2), Is.False);
Assert.That(db.ExistsAsset(uuid3), Is.False);
}
</source>

Test T001 is an example of a simple negative test. We assume a new database will not have any of those assets in them. While the value of this test may look low, it does provide a baseline in ensuring that the database connection is there, that these return false correctly, and that no other exception is thrown. Negative tests are a good way to force bounds conditions and ensure that not only does it ''return what you expect'' it also ''doesn't return what you don't expect''. Thought of another way, it ensures your code is somewhat defensive in nature, not coughing on bad or unexpected data.

=== Simple Positive Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[Test]
public void T010_StoreSimpleAsset()
{
AssetBase a1 = new AssetBase(uuid1, name1);
AssetBase a2 = new AssetBase(uuid2, name2);
AssetBase a3 = new AssetBase(uuid3, name3);
a1.Data = asset1;
a2.Data = asset1;
a3.Data = asset1;

db.CreateAsset(a1);
db.CreateAsset(a2);
db.CreateAsset(a3);

AssetBase a1a = db.FetchAsset(uuid1);
Assert.That(a1a.ID, Is.EqualTo(uuid1));
Assert.That(a1a.Name, Is.EqualTo(name1));

AssetBase a2a = db.FetchAsset(uuid2);
Assert.That(a2a.ID, Is.EqualTo(uuid2));
Assert.That(a2a.Name, Is.EqualTo(name2));

AssetBase a3a = db.FetchAsset(uuid3);
Assert.That(a3a.ID, Is.EqualTo(uuid3));
Assert.That(a3a.Name, Is.EqualTo(name3));
}
</source>
T010 is an example of a simple positive test. In it we create and store 3 assets (ensuring no exceptions), then load those 3 assets back from the database and ensure the fields are correct. Because AssetBase is not IComparible we just check the ID and Name fields with equals tests. If any of the Asserts fail, the whole test fails.

=== Stateful Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[Test]
public void T011_ExistsSimpleAsset()
{
Assert.That(db.ExistsAsset(uuid1), Is.True);
Assert.That(db.ExistsAsset(uuid2), Is.True);
Assert.That(db.ExistsAsset(uuid3), Is.True);
}
</source>

T011 is an example of a stateful test, because it requires the state created by T010 (i.e. the creation of those 3 objects). In order to test any kind of complicated scenario you will find that you need to use stateful tests to build up various amounts of state (testing along the way), then manipulating and possibly tearing it down. Without doing this you can't do truly deep testing of function in any complex environment. This example isn't very stateful (I tried to pick an easy example), but it should give you some ideas.

=== Speculative Tests ===
Speculative tests are tests that might or might not apply in a given situation. MySQL testing in the OpenSim tree is done by speculative testing, the tests will only run if there is a properly configured database, otherwise they will not be run. If you execute '''Assert.Ignore()''' in a '''Test''' the test will end and be ignored. If you run '''Assert.Ignore()''' in the '''TestFixtureSetup''' all tests in the test fixture will be skipped and ignored.

Speculative testing lets you create tests that require certain preconditions to be met which you can't guarantee on all platforms/configuration, and are an important part of deep testing.

== Adding Tests to the Tree ==
As we said previously all tests for assembly OpenSim.Foo (in directory OpenSim/Foo) should:
* be in assembly OpenSim.Foo.Tests.dll
* not be in the OpenSim.Foo.dll assembly
* be in OpenSim/Foo/Tests directory

Also, if you have created a new test assembly you must add references to it in both ''.nant/local.include'' and ''.nant/bamboo.build'' to ensure that the assembly is added to the automated bamboo runs as well as the nant test target.

== Executing Tests ==

=== Bamboo ===
On every commit to opensim all the tests are run on a [http://opensimulator.org:8085 Bamboo build server on opensimulator.org]. The process takes about 5 minutes to build, test, and report the results back out on #opensim-dev via the osmantis bot.

=== Nant ===
You can manually run all the tests for OpenSim on your system by running '''nant test''' as a nant target. This will run all the tests that are in the tree, though some speculative tests might be ignored if your platform does not have the right features or configuration to run these tests.

=== NUnit Console ===
If you only want to run tests for one assembly you can do that using the NUnit Console. On Linux just run '''nunit-console2 OpenSim.Foo.Tests.dll''' and it will run only the tests for OpenSim.Foo.Tests.dll. If you are only making changes to 1 dll and just want a quick sanity check, this is the fastest way to do that.

=== Debugging Tests ===
There is a special page dedicated to this. See [[Debugging Unit Tests]].

== Learning More ==
You should definitely read the documentation at the [http://www.nunit.org/index.php?p=documentation NUnit homepage] if you want to know more about testing. It's a very good reference for all the APIs in NUnit that you can use for creating tests.

== Code Coverage ==

A prototype has been included using monocov, which has a profile built-in in mono. Instructions for using monocov can be found on the Mono homepage, in [http://www.mono-project.com/Code_Coverage Code Coverage] section. Unfortunately nunit2 and nant do not support code coverage with monocov (only some other proprietary code coverages), so there is no <monocov> tag. The solution was to implement it using many nunit-console and on <exec> tags.

ATTENTION: Code coverage only works with mono 1.2.x , any other version will most likely not work. Code coverage development is being put on hold for now until it supports newer mono versions.

=== Running ===

Download [http://primates.ximian.com/~lupus/monocov-0.2.tar.gz monocov] and [http://prdownloads.sourceforge.net/nunit/NUnit-2.4.8-net-2.0.zip?download nunit-console] if your nunit-console does not work. To test if it works (my Ubuntu's Hardy version did not), just run one of the tests with nunit-console.

Install monocov (./configure, ./make install, there could be some minor conflicts, I had to add to the compiling line -I/usr/include/mono-1.0/) and make sure the working nunit-console.exe is in /usr/lib/nunit/nunit-console.exe. Now just run nant test-cov and it will generate .cov files and HTML directories in the cov directory. The .cov files can be seen by running monocov on them, and have the same information as the HTML directories.

== Testing Todo ==

=== Coverage ===

A prototype has been done and documented. Now we must keep a look if monocov will evolve to support newer mono versions.

[[Category:Development]]
[[Category:Testing]]

Automated Testing

2009-03-12T08:23:18Z

Mikem:

As OpenSim matures, we are extremely interested in adding more automated verification into the OpenSim source tree. Test exist not only to prevent bugs from creeping in, but also to provide fair warning that the behavior of the system has changed, and tests may need updating.

In OpenSim today we use NUnit tests. Our conventions are:
# Tests should '''not''' exist inside runtime assemblies, as this makes nunit a production requirement
# Tests should be in .Tests.dll assemblies. For instance, the tests for OpenSim.Data.SQLite.dll should be in the OpenSim.Data.SQLite.Tests.dll assembly. This allows for easy removal of test assemblies in products.
# Tests should be as close to the code as possible, but not intermingled. So the tests for OpenSim/Data/SQLite should be in OpenSim/Data/SQLite/Tests/. Through the use of the '''Exclude''' keyword in prebuild.xml you can ensure that directory is part of OpenSim.Data.SQLite.Tests.dll and not OpenSim.Data.SQLite.dll. See exclude clarification in writing unit tests section.
# Tests testing a class should be grouped into a test class file called xxxTest.cs, where xxx is the name of the class that is being tested.
# Tests should be able to run safely in a production environment. That means that care must be taken not to damage data on the machine that it is being run.
# Tests should be deterministic in other words repeatable. Avoid randomness in tests. See good and bad testing practices below.

== Core Functionality Missing Unit Tests ==

This is a list of functionality which is not covered by unit tests and is identified as highly desireable test target:

# Database Modules (These are mysql tables)
## region ban
## land
## landaccesslist

== Good / Bad Test practices ==
Creating good tests is an art, not a science. Tests are useful by how many bugs they find or how many bugs they avoid. Things you should think about in creating good tests is:
* Throwing edge cases, like 0, "", or Null at parameters. This ensures that people functions are hardened against incomplete data. Many of our crashes come from the lack of this hardening showing up at just the wrong time.
* Use stateful testing to build up complex scenarios. This is more useful than just cursory get / set calls.
* Random tests are not a good idea. We need test results to be deterministic. In other words tests need to be repeatable. If you want to test for a range it is good idea to make separate tests for min and max values. Random values in fields can fail randomly. When something goes wrong for example in database schema the developer will not necessarily notice if the stored values are random. On the other hand its hard to troubleshoot randomly failing tests as you dont know which specific value caused the failure.
* Tests should be independent and should not rely on another test being run, passing or failing. An excerpt from [http://xunitpatterns.com/Principles%20of%20Test%20Automation.html#Independent%20Test xUnit Patterns]:
<blockquote>If tests are interdependent and (even worse) order dependent, we will be depriving ourselves of the useful feedback test failures provide. Interacting Tests [...] tend to fail in a group. The failure of a test that moved the [subject under test] into the state required by the dependent test will lead to the failure of the dependent test too. With both tests failing, how can we tell if it is because of a problem in code that both rely on in some way or is it a problem in code that only the first relies on. With both tests failing we can't tell. We are only talking about two tests here. Imagine how much worse this is with tens or hundreds of tests.</blockquote>
* Only one function of the subject under test should be tested in one test. When testing a database access object, for example, write separate tests for creating DB entries, updating them and removing them.
* Do not use the subject under test to set up the state for the test or to verify the result. Use a different method. When testing a database access object, for example, use raw SQL to insert the initial data into the DB, then run the method being tested. To verify if the operation was successful, use raw SQL again to verify the DB changed as expected.

== Writing Tests ==
Writing a new unit test is pretty easy, and very helpful in increasing the stability of opensim by nailing down bugs. I'm going to present an example here of SQLite Asset testing to show how simple such a test case is to write. The actual in tree SQLite Asset tests are a little different because the code was factored out so that it was easily applied to any database driver, so don't be concerned with the fact that what you see here isn't in the tree.

Exclude clarification: Make sure your master project (not the test project) has an entry for files like the following so that the test code is not included in the master project dll:

<pre>
<Files>
<Match pattern="*.cs" recurse="true">
<Exclude name="Tests" pattern="Tests" />
</Match>
</Files>
</pre>

=== NUnit Conventions ===
An NUnit test suite:
* is a class with a default constructor (takes no arguments)
* has public methods that are tests
* uses annotations to determine what are tests
* runs it's tests in '''alphabetical order by method name'''

An NUnit test method:
* must be public
* must return void
* must take no arguments
* is successful if no exception or assert is thrown while running it

The run order is important if you want to have early tests that setup some complicated state (like creating objects), and have later tests remove or update that state. For that reason I find it very helpful to name all test methods '''Txxx_somename''' where '''xxx''' is a number between 000 and 999. That guaruntees no surprises in run order.

==== Fixture Setup / Teardown ====

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[TestFixtureSetUp]
public void Init()
{
uuid1 = UUID.Random();
uuid2 = UUID.Random();
uuid3 = UUID.Random();
name1 = "asset one";
name2 = "asset two";
name3 = "asset three";

asset1 = new byte[100];
asset1.Initialize();
file = Path.GetTempFileName() + ".db";
connect = "URI=file:" + file + ",version=3";
db = new SQLiteAssetData();
db.Initialise(connect);
}

[TestFixtureTearDown]
public void Cleanup()
{
db.Dispose();
System.IO.File.Delete(file);
}
</source>

In the case of testing something like the database layer, we have to actually attempt to store / retrieve things from a database. Following from rule #4 of good tests, we want to make sure not to touch the production databases to run our tests, so during startup we generate a temporary file name which is guaranteed not to be an existing file on the system, and use that as our database file name. By running db.Initialize() the OpenSim migration code will correctly populate that database with the latest schema.

Once we are done with the tests we want to make sure we aren't leaving garbage temp files on the user's system. So we remove that file we created.

During setup we also create a set of state variables, such as 3 uuids, 3 strings, and a data block. You could have always just stuck these inline, but variables are there for a reason, so use them.

==== Test Setup / Teardown ====

What's missing in [[Example Test SQLite Assets]] are individual test Setup and Teardown methods. Let's assume the <code>SQLiteAssetData</code> class provided a <code>FetchAsset()</code> method and a <code>UpdateAsset()</code> method. Since every test should be independent of any other test, and <code>FetchAsset()</code> and <code>UpdateAsset()</code> should be tested in separate tests, that means each test would need to create its own entries in the asset table in order to succeed. You may have something like this (see [[Example Test SQLite Assets#This Test is Flawed|Example Test SQLite Assets]] for an explanation of <code>sqldb.executeSQL()</code>):

<source lang="csharp">[Test]
public void T101_TestFetchAsset()
{
AssetBase a1 = new AssetBase(...);
sqldb.executeSQL("INSERT INTO assets VALUES({0}, ...)", a1.uuid, ...);

AssetBase a1_actual = db.FetchAsset(a1.uuid);

Assert.Equal(a1_actual.uuid, a1.uuid);
Assert.Equal(a1_actual.Name, a1.Name);
// etc
}

[Test]
public void T103_TestUpdateAsset()
{
AssetBase a1 = new AssetBase(...);
sqldb.executeSQL("INSERT INTO assets VALUES({0}, ...)", a1.uuid, ...);

a1.Name = "new name";

db.UpdateAsset(a1.uuid, a1);

AssetBase a1_actual = sqldb.executeSQL("SELECT * FROM assets WHERE uuid = {0}", a1.uuid);

Assert.Equal(a1_actual.uuid, a1.uuid);
Assert.Equal(a1_actual.Name, a1.Name);
// etc
}
</source>

You will note that both tests have the same code at the top in which they create an entry in the assets table. This duplicate code can be factored out into a Setup method, which is called before every test is executed (assume <code>a1</code> is a class attribute):

<source lang="csharp">
[SetUp]
public void SetUp()
{
a1 = new AssetBase(...);
sqldb.executeSQL("INSERT INTO assets VALUES({0}, ...)", a1.uuid, ...);
}

[TearDown]
public void TearDown()
{
// clean up after ourselves so the next test has a clean DB to start with
sqldb.executeSQL("DELETE FROM assets");
}

[Test]
public void T101_TestFetchAsset()
{
AssetBase a1_actual = db.FetchAsset(a1.uuid);

Assert.Equal(a1_actual.uuid, a1.uuid);
Assert.Equal(a1_actual.Name, a1.Name);
// etc
}

[Test]
public void T103_TestUpdateAsset()
{
a1.Name = "new name";

db.UpdateAsset(a1.uuid, a1);

AssetBase a1_actual = sqldb.executeSQL("SELECT * FROM assets WHERE uuid = {0}", a1.uuid);

Assert.Equal(a1_actual.uuid, a1.uuid);
Assert.Equal(a1_actual.Name, a1.Name);
// etc
}
</source>

Also note the <code>TearDown()</code> method; it is called after each test has run, regardless whether the test passed or failed. It deletes all the entries in the <code>assets</code> table so that there is no leftover data in the database to interfere with the next test.

==== Asserts ====
You will see scattered through the code '''Assert.That(...)'''. These will throw an exception if the condition is not valid. This format of assertions is called the [http://www.nunit.org/index.php?p=constraintModel&r=2.4 Constraint Model] in NUnit, and provides a large number of tests with the flavor of:
* Assert.That(foo, Is.Null)
* Assert.That(foo, Is.Not.Null)
* Assert.That(foo, Is.True)
* Assert.That(foo, Is.EqualTo(bar))
* Assert.That(foo, Text.Matches( "*bar*" ))

Of note, Is.EqualTo uses the Equals function of foo, so this can only be used on objects that are IComparable. Most of the OpenSim base objects are not, so you'll have to compare fields manually in tests.

For the complete set of conditions you can use see [http://www.nunit.org/index.php?p=constraintModel&r=2.4 the Constraint Model NUnit documentation]. While there is another syntax for tests, the Constraint Model is preferred as it is far more human readable.

=== Simple Negative Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">[Test]
public void T001_LoadEmpty()
{
Assert.That(db.ExistsAsset(uuid1), Is.False);
Assert.That(db.ExistsAsset(uuid2), Is.False);
Assert.That(db.ExistsAsset(uuid3), Is.False);
}
</source>

Test T001 is an example of a simple negative test. We assume a new database will not have any of those assets in them. While the value of this test may look low, it does provide a baseline in ensuring that the database connection is there, that these return false correctly, and that no other exception is thrown. Negative tests are a good way to force bounds conditions and ensure that not only does it ''return what you expect'' it also ''doesn't return what you don't expect''. Thought of another way, it ensures your code is somewhat defensive in nature, not coughing on bad or unexpected data.

=== Simple Positive Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[Test]
public void T010_StoreSimpleAsset()
{
AssetBase a1 = new AssetBase(uuid1, name1);
AssetBase a2 = new AssetBase(uuid2, name2);
AssetBase a3 = new AssetBase(uuid3, name3);
a1.Data = asset1;
a2.Data = asset1;
a3.Data = asset1;

db.CreateAsset(a1);
db.CreateAsset(a2);
db.CreateAsset(a3);

AssetBase a1a = db.FetchAsset(uuid1);
Assert.That(a1a.ID, Is.EqualTo(uuid1));
Assert.That(a1a.Name, Is.EqualTo(name1));

AssetBase a2a = db.FetchAsset(uuid2);
Assert.That(a2a.ID, Is.EqualTo(uuid2));
Assert.That(a2a.Name, Is.EqualTo(name2));

AssetBase a3a = db.FetchAsset(uuid3);
Assert.That(a3a.ID, Is.EqualTo(uuid3));
Assert.That(a3a.Name, Is.EqualTo(name3));
}
</source>
T010 is an example of a simple positive test. In it we create and store 3 assets (ensuring no exceptions), then load those 3 assets back from the database and ensure the fields are correct. Because AssetBase is not IComparible we just check the ID and Name fields with equals tests. If any of the Asserts fail, the whole test fails.

=== Stateful Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[Test]
public void T011_ExistsSimpleAsset()
{
Assert.That(db.ExistsAsset(uuid1), Is.True);
Assert.That(db.ExistsAsset(uuid2), Is.True);
Assert.That(db.ExistsAsset(uuid3), Is.True);
}
</source>

T011 is an example of a stateful test, because it requires the state created by T010 (i.e. the creation of those 3 objects). In order to test any kind of complicated scenario you will find that you need to use stateful tests to build up various amounts of state (testing along the way), then manipulating and possibly tearing it down. Without doing this you can't do truly deep testing of function in any complex environment. This example isn't very stateful (I tried to pick an easy example), but it should give you some ideas.

=== Speculative Tests ===
Speculative tests are tests that might or might not apply in a given situation. MySQL testing in the OpenSim tree is done by speculative testing, the tests will only run if there is a properly configured database, otherwise they will not be run. If you execute '''Assert.Ignore()''' in a '''Test''' the test will end and be ignored. If you run '''Assert.Ignore()''' in the '''TestFixtureSetup''' all tests in the test fixture will be skipped and ignored.

Speculative testing lets you create tests that require certain preconditions to be met which you can't guarantee on all platforms/configuration, and are an important part of deep testing.

== Adding Tests to the Tree ==
As we said previously all tests for assembly OpenSim.Foo (in directory OpenSim/Foo) should:
* be in assembly OpenSim.Foo.Tests.dll
* not be in the OpenSim.Foo.dll assembly
* be in OpenSim/Foo/Tests directory

Also, if you have created a new test assembly you must add references to it in both ''.nant/local.include'' and ''.nant/bamboo.build'' to ensure that the assembly is added to the automated bamboo runs as well as the nant test target.

== Executing Tests ==

=== Bamboo ===
On every commit to opensim all the tests are run on a [http://opensimulator.org:8085 Bamboo build server on opensimulator.org]. The process takes about 5 minutes to build, test, and report the results back out on #opensim-dev via the osmantis bot.

=== Nant ===
You can manually run all the tests for OpenSim on your system by running '''nant test''' as a nant target. This will run all the tests that are in the tree, though some speculative tests might be ignored if your platform does not have the right features or configuration to run these tests.

=== NUnit Console ===
If you only want to run tests for one assembly you can do that using the NUnit Console. On Linux just run '''nunit-console2 OpenSim.Foo.Tests.dll''' and it will run only the tests for OpenSim.Foo.Tests.dll. If you are only making changes to 1 dll and just want a quick sanity check, this is the fastest way to do that.

=== Debugging Tests ===
There is a special page dedicated to this. See [[Debugging Unit Tests]].

== Learning More ==
You should definitely read the documentation at the [http://www.nunit.org/index.php?p=documentation NUnit homepage] if you want to know more about testing. It's a very good reference for all the APIs in NUnit that you can use for creating tests.

== Code Coverage ==

A prototype has been included using monocov, which has a profile built-in in mono. Instructions for using monocov can be found on the Mono homepage, in [http://www.mono-project.com/Code_Coverage Code Coverage] section. Unfortunately nunit2 and nant do not support code coverage with monocov (only some other proprietary code coverages), so there is no <monocov> tag. The solution was to implement it using many nunit-console and on <exec> tags.

ATTENTION: Code coverage only works with mono 1.2.x , any other version will most likely not work. Code coverage development is being put on hold for now until it supports newer mono versions.

=== Running ===

Download [http://primates.ximian.com/~lupus/monocov-0.2.tar.gz monocov] and [http://prdownloads.sourceforge.net/nunit/NUnit-2.4.8-net-2.0.zip?download nunit-console] if your nunit-console does not work. To test if it works (my Ubuntu's Hardy version did not), just run one of the tests with nunit-console.

Install monocov (./configure, ./make install, there could be some minor conflicts, I had to add to the compiling line -I/usr/include/mono-1.0/) and make sure the working nunit-console.exe is in /usr/lib/nunit/nunit-console.exe. Now just run nant test-cov and it will generate .cov files and HTML directories in the cov directory. The .cov files can be seen by running monocov on them, and have the same information as the HTML directories.

== Testing Todo ==

=== Coverage ===

A prototype has been done and documented. Now we must keep a look if monocov will evolve to support newer mono versions.

[[Category:Development]]
[[Category:Testing]]

Automated Testing

2009-03-12T08:00:38Z

Mikem: /* Setup / Teardown */

As OpenSim matures, we are extremely interested in adding more automated verification into the OpenSim source tree. Test exist not only to prevent bugs from creeping in, but also to provide fair warning that the behavior of the system has changed, and tests may need updating.

In OpenSim today we use NUnit tests. Our conventions are:
# Tests should '''not''' exist inside runtime assemblies, as this makes nunit a production requirement
# Tests should be in .Tests.dll assemblies. For instance, the tests for OpenSim.Data.SQLite.dll should be in the OpenSim.Data.SQLite.Tests.dll assembly. This allows for easy removal of test assemblies in products.
# Tests should be as close to the code as possible, but not intermingled. So the tests for OpenSim/Data/SQLite should be in OpenSim/Data/SQLite/Tests/. Through the use of the '''Exclude''' keyword in prebuild.xml you can ensure that directory is part of OpenSim.Data.SQLite.Tests.dll and not OpenSim.Data.SQLite.dll. See exclude clarification in writing unit tests section.
# Tests testing a class should be grouped into a test class file called xxxTest.cs, where xxx is the name of the class that is being tested.
# Tests should be able to run safely in a production environment. That means that care must be taken not to damage data on the machine that it is being run.
# Tests should be deterministic in other words repeatable. Avoid randomness in tests. See good and bad testing practices below.

== Core Functionality Missing Unit Tests ==

This is a list of functionality which is not covered by unit tests and is identified as highly desireable test target:

# Database Modules (These are mysql tables)
## region ban
## land
## landaccesslist

== Good / Bad Test practices ==
Creating good tests is an art, not a science. Tests are useful by how many bugs they find or how many bugs they avoid. Things you should think about in creating good tests is:
* Throwing edge cases, like 0, "", or Null at parameters. This ensures that people functions are hardened against incomplete data. Many of our crashes come from the lack of this hardening showing up at just the wrong time.
* Use stateful testing to build up complex scenarios. This is more useful than just cursory get / set calls.
* Random tests are not a good idea. We need test results to be deterministic. In other words tests need to be repeatable. If you want to test for a range it is good idea to make separate tests for min and max values. Random values in fields can fail randomly. When something goes wrong for example in database schema the developer will not necessarily notice if the stored values are random. On the other hand its hard to troubleshoot randomly failing tests as you dont know which specific value caused the failure.
* Tests should be independent and should not rely on another test being run, passing or failing. An excerpt from [http://xunitpatterns.com/Principles%20of%20Test%20Automation.html#Independent%20Test xUnit Patterns]:
<blockquote>If tests are interdependent and (even worse) order dependent, we will be depriving ourselves of the useful feedback test failures provide. Interacting Tests [...] tend to fail in a group. The failure of a test that moved the [subject under test] into the state required by the dependent test will lead to the failure of the dependent test too. With both tests failing, how can we tell if it is because of a problem in code that both rely on in some way or is it a problem in code that only the first relies on. With both tests failing we can't tell. We are only talking about two tests here. Imagine how much worse this is with tens or hundreds of tests.</blockquote>
* Only one function of the subject under test should be tested in one test. When testing a database access object, for example, write separate tests for creating DB entries, updating them and removing them.
* Do not use the subject under test to set up the state for the test or to verify the result. Use a different method. When testing a database access object, for example, use raw SQL to insert the initial data into the DB, then run the method being tested. To verify if the operation was successful, use raw SQL again to verify the DB changed as expected.

== Writing Tests ==
Writing a new unit test is pretty easy, and very helpful in increasing the stability of opensim by nailing down bugs. I'm going to present an example here of SQLite Asset testing to show how simple such a test case is to write. The actual in tree SQLite Asset tests are a little different because the code was factored out so that it was easily applied to any database driver, so don't be concerned with the fact that what you see here isn't in the tree.

Exclude clarification: Make sure your master project (not the test project) has an entry for files like the following so that the test code is not included in the master project dll:

<pre>
<Files>
<Match pattern="*.cs" recurse="true">
<Exclude name="Tests" pattern="Tests" />
</Match>
</Files>
</pre>

=== NUnit Conventions ===
An NUnit test suite:
* is a class with a default constructor (takes no arguments)
* has public methods that are tests
* uses annotations to determine what are tests
* runs it's tests in '''alphabetical order by method name'''

An NUnit test method:
* must be public
* must return void
* must take no arguments
* is successful if no exception or assert is thrown while running it

The run order is important if you want to have early tests that setup some complicated state (like creating objects), and have later tests remove or update that state. For that reason I find it very helpful to name all test methods '''Txxx_somename''' where '''xxx''' is a number between 000 and 999. That guaruntees no surprises in run order.

==== Fixture Setup / Teardown ====

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[TestFixtureSetUp]
public void Init()
{
uuid1 = UUID.Random();
uuid2 = UUID.Random();
uuid3 = UUID.Random();
name1 = "asset one";
name2 = "asset two";
name3 = "asset three";

asset1 = new byte[100];
asset1.Initialize();
file = Path.GetTempFileName() + ".db";
connect = "URI=file:" + file + ",version=3";
db = new SQLiteAssetData();
db.Initialise(connect);
}

[TestFixtureTearDown]
public void Cleanup()
{
db.Dispose();
System.IO.File.Delete(file);
}
</source>

In the case of testing something like the database layer, we have to actually attempt to store / retrieve things from a database. Following from rule #4 of good tests, we want to make sure not to touch the production databases to run our tests, so during startup we generate a temporary file name which is guaranteed not to be an existing file on the system, and use that as our database file name. By running db.Initialize() the OpenSim migration code will correctly populate that database with the latest schema.

Once we are done with the tests we want to make sure we aren't leaving garbage temp files on the user's system. So we remove that file we created.

During setup we also create a set of state variables, such as 3 uuids, 3 strings, and a data block. You could have always just stuck these inline, but variables are there for a reason, so use them.

==== Asserts ====
You will see scattered through the code '''Assert.That(...)'''. These will throw an exception if the condition is not valid. This format of assertions is called the [http://www.nunit.org/index.php?p=constraintModel&r=2.4 Constraint Model] in NUnit, and provides a large number of tests with the flavor of:
* Assert.That(foo, Is.Null)
* Assert.That(foo, Is.Not.Null)
* Assert.That(foo, Is.True)
* Assert.That(foo, Is.EqualTo(bar))
* Assert.That(foo, Text.Matches( "*bar*" ))

Of note, Is.EqualTo uses the Equals function of foo, so this can only be used on objects that are IComparable. Most of the OpenSim base objects are not, so you'll have to compare fields manually in tests.

For the complete set of conditions you can use see [http://www.nunit.org/index.php?p=constraintModel&r=2.4 the Constraint Model NUnit documentation]. While there is another syntax for tests, the Constraint Model is preferred as it is far more human readable.

=== Simple Negative Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">[Test]
public void T001_LoadEmpty()
{
Assert.That(db.ExistsAsset(uuid1), Is.False);
Assert.That(db.ExistsAsset(uuid2), Is.False);
Assert.That(db.ExistsAsset(uuid3), Is.False);
}
</source>

Test T001 is an example of a simple negative test. We assume a new database will not have any of those assets in them. While the value of this test may look low, it does provide a baseline in ensuring that the database connection is there, that these return false correctly, and that no other exception is thrown. Negative tests are a good way to force bounds conditions and ensure that not only does it ''return what you expect'' it also ''doesn't return what you don't expect''. Thought of another way, it ensures your code is somewhat defensive in nature, not coughing on bad or unexpected data.

=== Simple Positive Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[Test]
public void T010_StoreSimpleAsset()
{
AssetBase a1 = new AssetBase(uuid1, name1);
AssetBase a2 = new AssetBase(uuid2, name2);
AssetBase a3 = new AssetBase(uuid3, name3);
a1.Data = asset1;
a2.Data = asset1;
a3.Data = asset1;

db.CreateAsset(a1);
db.CreateAsset(a2);
db.CreateAsset(a3);

AssetBase a1a = db.FetchAsset(uuid1);
Assert.That(a1a.ID, Is.EqualTo(uuid1));
Assert.That(a1a.Name, Is.EqualTo(name1));

AssetBase a2a = db.FetchAsset(uuid2);
Assert.That(a2a.ID, Is.EqualTo(uuid2));
Assert.That(a2a.Name, Is.EqualTo(name2));

AssetBase a3a = db.FetchAsset(uuid3);
Assert.That(a3a.ID, Is.EqualTo(uuid3));
Assert.That(a3a.Name, Is.EqualTo(name3));
}
</source>
T010 is an example of a simple positive test. In it we create and store 3 assets (ensuring no exceptions), then load those 3 assets back from the database and ensure the fields are correct. Because AssetBase is not IComparible we just check the ID and Name fields with equals tests. If any of the Asserts fail, the whole test fails.

=== Stateful Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[Test]
public void T011_ExistsSimpleAsset()
{
Assert.That(db.ExistsAsset(uuid1), Is.True);
Assert.That(db.ExistsAsset(uuid2), Is.True);
Assert.That(db.ExistsAsset(uuid3), Is.True);
}
</source>

T011 is an example of a stateful test, because it requires the state created by T010 (i.e. the creation of those 3 objects). In order to test any kind of complicated scenario you will find that you need to use stateful tests to build up various amounts of state (testing along the way), then manipulating and possibly tearing it down. Without doing this you can't do truly deep testing of function in any complex environment. This example isn't very stateful (I tried to pick an easy example), but it should give you some ideas.

=== Speculative Tests ===
Speculative tests are tests that might or might not apply in a given situation. MySQL testing in the OpenSim tree is done by speculative testing, the tests will only run if there is a properly configured database, otherwise they will not be run. If you execute '''Assert.Ignore()''' in a '''Test''' the test will end and be ignored. If you run '''Assert.Ignore()''' in the '''TestFixtureSetup''' all tests in the test fixture will be skipped and ignored.

Speculative testing lets you create tests that require certain preconditions to be met which you can't guarantee on all platforms/configuration, and are an important part of deep testing.

== Adding Tests to the Tree ==
As we said previously all tests for assembly OpenSim.Foo (in directory OpenSim/Foo) should:
* be in assembly OpenSim.Foo.Tests.dll
* not be in the OpenSim.Foo.dll assembly
* be in OpenSim/Foo/Tests directory

Also, if you have created a new test assembly you must add references to it in both ''.nant/local.include'' and ''.nant/bamboo.build'' to ensure that the assembly is added to the automated bamboo runs as well as the nant test target.

== Executing Tests ==

=== Bamboo ===
On every commit to opensim all the tests are run on a [http://opensimulator.org:8085 Bamboo build server on opensimulator.org]. The process takes about 5 minutes to build, test, and report the results back out on #opensim-dev via the osmantis bot.

=== Nant ===
You can manually run all the tests for OpenSim on your system by running '''nant test''' as a nant target. This will run all the tests that are in the tree, though some speculative tests might be ignored if your platform does not have the right features or configuration to run these tests.

=== NUnit Console ===
If you only want to run tests for one assembly you can do that using the NUnit Console. On Linux just run '''nunit-console2 OpenSim.Foo.Tests.dll''' and it will run only the tests for OpenSim.Foo.Tests.dll. If you are only making changes to 1 dll and just want a quick sanity check, this is the fastest way to do that.

=== Debugging Tests ===
There is a special page dedicated to this. See [[Debugging Unit Tests]].

== Learning More ==
You should definitely read the documentation at the [http://www.nunit.org/index.php?p=documentation NUnit homepage] if you want to know more about testing. It's a very good reference for all the APIs in NUnit that you can use for creating tests.

== Code Coverage ==

A prototype has been included using monocov, which has a profile built-in in mono. Instructions for using monocov can be found on the Mono homepage, in [http://www.mono-project.com/Code_Coverage Code Coverage] section. Unfortunately nunit2 and nant do not support code coverage with monocov (only some other proprietary code coverages), so there is no <monocov> tag. The solution was to implement it using many nunit-console and on <exec> tags.

ATTENTION: Code coverage only works with mono 1.2.x , any other version will most likely not work. Code coverage development is being put on hold for now until it supports newer mono versions.

=== Running ===

Download [http://primates.ximian.com/~lupus/monocov-0.2.tar.gz monocov] and [http://prdownloads.sourceforge.net/nunit/NUnit-2.4.8-net-2.0.zip?download nunit-console] if your nunit-console does not work. To test if it works (my Ubuntu's Hardy version did not), just run one of the tests with nunit-console.

Install monocov (./configure, ./make install, there could be some minor conflicts, I had to add to the compiling line -I/usr/include/mono-1.0/) and make sure the working nunit-console.exe is in /usr/lib/nunit/nunit-console.exe. Now just run nant test-cov and it will generate .cov files and HTML directories in the cov directory. The .cov files can be seen by running monocov on them, and have the same information as the HTML directories.

== Testing Todo ==

=== Coverage ===

A prototype has been done and documented. Now we must keep a look if monocov will evolve to support newer mono versions.

[[Category:Development]]
[[Category:Testing]]

Example Test SQLite Assets

2009-03-12T08:00:02Z

Mikem: /* An Example Test - SQLite Assets */

=== An Example Test - SQLite Assets ===
<source lang="csharp">
using System;
using System.IO;
using System.Collections.Generic;
using NUnit.Framework;
using NUnit.Framework.SyntaxHelpers;
using OpenSim.Framework;
using OpenSim.Data.Tests;
using OpenSim.Data.SQLite;
using OpenSim.Region.Environment.Scenes;
using OpenMetaverse;

namespace OpenSim.Data.SQLite.Tests
{
[TestFixture]
public class SQLiteAssetTest
{
public string file;
public string connect;
public AssetDataBase db;
public UUID uuid1;
public UUID uuid2;
public UUID uuid3;
public string name1;
public string name2;
public string name3;
public byte[] asset1;

[TestFixtureSetUp]
public void Init()
{
uuid1 = UUID.Random();
uuid2 = UUID.Random();
uuid3 = UUID.Random();
name1 = "asset one";
name2 = "asset two";
name3 = "asset three";

asset1 = new byte[100];
asset1.Initialize();
file = Path.GetTempFileName() + ".db";
connect = "URI=file:" + file + ",version=3";
db = new SQLiteAssetData();
db.Initialise(connect);
}

[TestFixtureTearDown]
public void Cleanup()
{
db.Dispose();
System.IO.File.Delete(file);
}

[Test]
public void T001_LoadEmpty()
{
Assert.That(db.ExistsAsset(uuid1), Is.False);
Assert.That(db.ExistsAsset(uuid2), Is.False);
Assert.That(db.ExistsAsset(uuid3), Is.False);
}

[Test]
public void T010_StoreSimpleAsset()
{
AssetBase a1 = new AssetBase(uuid1, name1);
AssetBase a2 = new AssetBase(uuid2, name2);
AssetBase a3 = new AssetBase(uuid3, name3);
a1.Data = asset1;
a2.Data = asset1;
a3.Data = asset1;

db.CreateAsset(a1);
db.CreateAsset(a2);
db.CreateAsset(a3);

AssetBase a1a = db.FetchAsset(uuid1);
Assert.That(a1a.ID, Is.EqualTo(uuid1));
Assert.That(a1a.Name, Is.EqualTo(name1));

AssetBase a2a = db.FetchAsset(uuid2);
Assert.That(a2a.ID, Is.EqualTo(uuid2));
Assert.That(a2a.Name, Is.EqualTo(name2));

AssetBase a3a = db.FetchAsset(uuid3);
Assert.That(a3a.ID, Is.EqualTo(uuid3));
Assert.That(a3a.Name, Is.EqualTo(name3));
}

[Test]
public void T011_ExistsSimpleAsset()
{
Assert.That(db.ExistsAsset(uuid1), Is.True);
Assert.That(db.ExistsAsset(uuid2), Is.True);
Assert.That(db.ExistsAsset(uuid3), Is.True);
}
}
}
</source>

You can see 4 of the important annotations here:
* TestFixture - this class is a test suite
* TestFixtureSetup - this code is always run before any of the tests are executed
* TestFixtureTearDown - this code is always run after the tests are done executing, even if they fail.
* Test - this method is a test

== This Test is Flawed ==

There is also a flaw in the tests above, namely the subject under test (the <code>SQLiteAssetData</code> object <code>db</code>) is used in the tests as well as the fixture setup. Consider the following scenario: the <code>SQLiteAssetData</code> has a caching mechanism keeps track of DB rows in memory. When rows are fetched, they are stored into the cache. When new rows are inserted or existing rows updated, the cache is updated, and eventually the cache is synced to disk.

Note that in the test above, all interactions with the database are handled by this <code>SQLiteAssetData</code> object. First, it's used to create the database and tables:

<source lang="csharp">
[TestFixture]
public class SQLiteAssetTest
{
// ...
public AssetDataBase db;

[TestFixtureSetUp]
public void Init()
{
// ...
connect = "URI=file:" + Path.GetTempFileName() + ".db,version=3";
db = new SQLiteAssetData();
db.Initialise(connect);
}

[TestFixtureTearDown]
public void Cleanup()
{
db.Dispose();
// ...
}
</source>

Then, in <code>T010_StoreSimpleAsset()</code>, <code>db.CreateAsset()</code> is called, with the intent to persist the asset to the database on disk. However to verify whether this operation succeeded, <code>db.FetchAsset()</code> is called:

<source lang="csharp">
[Test]
public void T010_StoreSimpleAsset()
{
// ...
db.CreateAsset(a1);
db.CreateAsset(a2);
db.CreateAsset(a3);

AssetBase a1a = db.FetchAsset(uuid1);
Assert.That(a1a.ID, Is.EqualTo(uuid1));
Assert.That(a1a.Name, Is.EqualTo(name1));

AssetBase a2a = db.FetchAsset(uuid2);
Assert.That(a2a.ID, Is.EqualTo(uuid2));
Assert.That(a2a.Name, Is.EqualTo(name2));

AssetBase a3a = db.FetchAsset(uuid3);
Assert.That(a3a.ID, Is.EqualTo(uuid3));
Assert.That(a3a.Name, Is.EqualTo(name3));
}
}
</source>

Now assume a change introduced a bug that prevent the cache from being flushed to disk '''ever'''. The tests above would never discover this bug!

=== How to Fix the Flaw ===

Another means of populating the database and verifying the success of the operations performed by <code>SQLiteAssetData</code> object should be used instead. For example:

<source lang="csharp">
[TestFixture]
public class SQLiteAssetTest
{
// ...
public string filename, connect;
public AssetDataBase db;

[TestFixtureSetUp]
public void Init()
{
// ...
filename = Path.GetTempFileName() + ".db";
connect = "URI=file:" + filename + ",version=3";

SQLiteDB sqldb = new SQLiteDBAdapter(connect); // ficticious adapter interfacing directly with SQLite database
sqldb.createTable(new SQLiteTable(...));
sqldb.executeSQL("INSET INTO assets VALUES(...)");

db = new SQLiteAssetData();
}

[TestFixtureTearDown]
public void Cleanup()
{
db.Dispose();
System.IO.File.Delete(filename);
// ...
}
</source>

Similarly, to verify that the <code>SQLiteAssetData</code> object creates and retrieves assets properly, test <code>T010_StoreSimpleAsset()</code> might be broken up into two tests:

<source lang="csharp">
[Test]
public void T010_StoreSimpleAsset()
{
AssetBase a1, a2, a3;
// initialize a1, a2 and a3

db.CreateAsset(a1);
db.CreateAsset(a2);
db.CreateAsset(a3);

SQLiteDB sqldb = new SQLiteDBAdapter(connect); // ficticious adapter interfacing directly with SQLite database
AssetBase a1_actual = sqldb.executeSQL("SELECT * FROM assets WHERE uuid={0}", a1.uuid);

Assert.Equals(a1_actual.uuid, a1.uuid);
Assert.Equals(a1_actual.Name, a1.Name);
// etc
}

[Test]
public void T011_FetchSimpleAsset()
{
AssetBase a1 = new AssetBase(uuid1, name1, ...);

SQLiteDB sqldb = new SQLiteDBAdapter(connect); // ficticious adapter interfacing directly with SQLite database
AssetBase a1_actual = sqldb.executeSQL("INSERT INTO assets VALUES({0}, ...)", a1.uuid, ...);
// etc

// ...
AssetBase a1a = db.FetchAsset(uuid1);
Assert.That(a1a.ID, Is.EqualTo(uuid1));
Assert.That(a1a.Name, Is.EqualTo(name1));

AssetBase a2a = db.FetchAsset(uuid2);
Assert.That(a2a.ID, Is.EqualTo(uuid2));
Assert.That(a2a.Name, Is.EqualTo(name2));

AssetBase a3a = db.FetchAsset(uuid3);
Assert.That(a3a.ID, Is.EqualTo(uuid3));
Assert.That(a3a.Name, Is.EqualTo(name3));
}
}
</source>

This way <code>T010_StoreSimpleAsset</code> tests <code>SQLiteAssetData.CreateAsset()</code> '''only''' and <code>T011_FetchSimpleAsset()</code> tests <code>SQLiteAssetData.FetchAsset()</code> '''only'''. If there are any problems in either of those methods or all of them, these tests will discover that there is a problem.

Automated Testing

2009-03-12T07:07:13Z

Mikem: /* Setup / Teardown */

As OpenSim matures, we are extremely interested in adding more automated verification into the OpenSim source tree. Test exist not only to prevent bugs from creeping in, but also to provide fair warning that the behavior of the system has changed, and tests may need updating.

In OpenSim today we use NUnit tests. Our conventions are:
# Tests should '''not''' exist inside runtime assemblies, as this makes nunit a production requirement
# Tests should be in .Tests.dll assemblies. For instance, the tests for OpenSim.Data.SQLite.dll should be in the OpenSim.Data.SQLite.Tests.dll assembly. This allows for easy removal of test assemblies in products.
# Tests should be as close to the code as possible, but not intermingled. So the tests for OpenSim/Data/SQLite should be in OpenSim/Data/SQLite/Tests/. Through the use of the '''Exclude''' keyword in prebuild.xml you can ensure that directory is part of OpenSim.Data.SQLite.Tests.dll and not OpenSim.Data.SQLite.dll. See exclude clarification in writing unit tests section.
# Tests testing a class should be grouped into a test class file called xxxTest.cs, where xxx is the name of the class that is being tested.
# Tests should be able to run safely in a production environment. That means that care must be taken not to damage data on the machine that it is being run.
# Tests should be deterministic in other words repeatable. Avoid randomness in tests. See good and bad testing practices below.

== Core Functionality Missing Unit Tests ==

This is a list of functionality which is not covered by unit tests and is identified as highly desireable test target:

# Database Modules (These are mysql tables)
## region ban
## land
## landaccesslist

== Good / Bad Test practices ==
Creating good tests is an art, not a science. Tests are useful by how many bugs they find or how many bugs they avoid. Things you should think about in creating good tests is:
* Throwing edge cases, like 0, "", or Null at parameters. This ensures that people functions are hardened against incomplete data. Many of our crashes come from the lack of this hardening showing up at just the wrong time.
* Use stateful testing to build up complex scenarios. This is more useful than just cursory get / set calls.
* Random tests are not a good idea. We need test results to be deterministic. In other words tests need to be repeatable. If you want to test for a range it is good idea to make separate tests for min and max values. Random values in fields can fail randomly. When something goes wrong for example in database schema the developer will not necessarily notice if the stored values are random. On the other hand its hard to troubleshoot randomly failing tests as you dont know which specific value caused the failure.
* Tests should be independent and should not rely on another test being run, passing or failing. An excerpt from [http://xunitpatterns.com/Principles%20of%20Test%20Automation.html#Independent%20Test xUnit Patterns]:
<blockquote>If tests are interdependent and (even worse) order dependent, we will be depriving ourselves of the useful feedback test failures provide. Interacting Tests [...] tend to fail in a group. The failure of a test that moved the [subject under test] into the state required by the dependent test will lead to the failure of the dependent test too. With both tests failing, how can we tell if it is because of a problem in code that both rely on in some way or is it a problem in code that only the first relies on. With both tests failing we can't tell. We are only talking about two tests here. Imagine how much worse this is with tens or hundreds of tests.</blockquote>
* Only one function of the subject under test should be tested in one test. When testing a database access object, for example, write separate tests for creating DB entries, updating them and removing them.
* Do not use the subject under test to set up the state for the test or to verify the result. Use a different method. When testing a database access object, for example, use raw SQL to insert the initial data into the DB, then run the method being tested. To verify if the operation was successful, use raw SQL again to verify the DB changed as expected.

== Writing Tests ==
Writing a new unit test is pretty easy, and very helpful in increasing the stability of opensim by nailing down bugs. I'm going to present an example here of SQLite Asset testing to show how simple such a test case is to write. The actual in tree SQLite Asset tests are a little different because the code was factored out so that it was easily applied to any database driver, so don't be concerned with the fact that what you see here isn't in the tree.

Exclude clarification: Make sure your master project (not the test project) has an entry for files like the following so that the test code is not included in the master project dll:

<pre>
<Files>
<Match pattern="*.cs" recurse="true">
<Exclude name="Tests" pattern="Tests" />
</Match>
</Files>
</pre>

=== NUnit Conventions ===
An NUnit test suite:
* is a class with a default constructor (takes no arguments)
* has public methods that are tests
* uses annotations to determine what are tests
* runs it's tests in '''alphabetical order by method name'''

An NUnit test method:
* must be public
* must return void
* must take no arguments
* is successful if no exception or assert is thrown while running it

The run order is important if you want to have early tests that setup some complicated state (like creating objects), and have later tests remove or update that state. For that reason I find it very helpful to name all test methods '''Txxx_somename''' where '''xxx''' is a number between 000 and 999. That guaruntees no surprises in run order.

==== Setup / Teardown ====

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[TestFixtureSetUp]
public void Init()
{
uuid1 = UUID.Random();
uuid2 = UUID.Random();
uuid3 = UUID.Random();
name1 = "asset one";
name2 = "asset two";
name3 = "asset three";

asset1 = new byte[100];
asset1.Initialize();
file = Path.GetTempFileName() + ".db";
connect = "URI=file:" + file + ",version=3";
db = new SQLiteAssetData();
db.Initialise(connect);
}

[TestFixtureTearDown]
public void Cleanup()
{
db.Dispose();
System.IO.File.Delete(file);
}
</source>

In the case of testing something like the database layer, we have to actually attempt to store / retrieve things from a database. Following from rule #4 of good tests, we want to make sure not to touch the production databases to run our tests, so during startup we generate a temporary file name which is guaranteed not to be an existing file on the system, and use that as our database file name. By running db.Initialize() the OpenSim migration code will correctly populate that database with the latest schema.

Once we are done with the tests we want to make sure we aren't leaving garbage temp files on the user's system. So we remove that file we created.

During setup we also create a set of state variables, such as 3 uuids, 3 strings, and a data block. You could have always just stuck these inline, but variables are there for a reason, so use them.

==== Asserts ====
You will see scattered through the code '''Assert.That(...)'''. These will throw an exception if the condition is not valid. This format of assertions is called the [http://www.nunit.org/index.php?p=constraintModel&r=2.4 Constraint Model] in NUnit, and provides a large number of tests with the flavor of:
* Assert.That(foo, Is.Null)
* Assert.That(foo, Is.Not.Null)
* Assert.That(foo, Is.True)
* Assert.That(foo, Is.EqualTo(bar))
* Assert.That(foo, Text.Matches( "*bar*" ))

Of note, Is.EqualTo uses the Equals function of foo, so this can only be used on objects that are IComparable. Most of the OpenSim base objects are not, so you'll have to compare fields manually in tests.

For the complete set of conditions you can use see [http://www.nunit.org/index.php?p=constraintModel&r=2.4 the Constraint Model NUnit documentation]. While there is another syntax for tests, the Constraint Model is preferred as it is far more human readable.

=== Simple Negative Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">[Test]
public void T001_LoadEmpty()
{
Assert.That(db.ExistsAsset(uuid1), Is.False);
Assert.That(db.ExistsAsset(uuid2), Is.False);
Assert.That(db.ExistsAsset(uuid3), Is.False);
}
</source>

Test T001 is an example of a simple negative test. We assume a new database will not have any of those assets in them. While the value of this test may look low, it does provide a baseline in ensuring that the database connection is there, that these return false correctly, and that no other exception is thrown. Negative tests are a good way to force bounds conditions and ensure that not only does it ''return what you expect'' it also ''doesn't return what you don't expect''. Thought of another way, it ensures your code is somewhat defensive in nature, not coughing on bad or unexpected data.

=== Simple Positive Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[Test]
public void T010_StoreSimpleAsset()
{
AssetBase a1 = new AssetBase(uuid1, name1);
AssetBase a2 = new AssetBase(uuid2, name2);
AssetBase a3 = new AssetBase(uuid3, name3);
a1.Data = asset1;
a2.Data = asset1;
a3.Data = asset1;

db.CreateAsset(a1);
db.CreateAsset(a2);
db.CreateAsset(a3);

AssetBase a1a = db.FetchAsset(uuid1);
Assert.That(a1a.ID, Is.EqualTo(uuid1));
Assert.That(a1a.Name, Is.EqualTo(name1));

AssetBase a2a = db.FetchAsset(uuid2);
Assert.That(a2a.ID, Is.EqualTo(uuid2));
Assert.That(a2a.Name, Is.EqualTo(name2));

AssetBase a3a = db.FetchAsset(uuid3);
Assert.That(a3a.ID, Is.EqualTo(uuid3));
Assert.That(a3a.Name, Is.EqualTo(name3));
}
</source>
T010 is an example of a simple positive test. In it we create and store 3 assets (ensuring no exceptions), then load those 3 assets back from the database and ensure the fields are correct. Because AssetBase is not IComparible we just check the ID and Name fields with equals tests. If any of the Asserts fail, the whole test fails.

=== Stateful Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[Test]
public void T011_ExistsSimpleAsset()
{
Assert.That(db.ExistsAsset(uuid1), Is.True);
Assert.That(db.ExistsAsset(uuid2), Is.True);
Assert.That(db.ExistsAsset(uuid3), Is.True);
}
</source>

T011 is an example of a stateful test, because it requires the state created by T010 (i.e. the creation of those 3 objects). In order to test any kind of complicated scenario you will find that you need to use stateful tests to build up various amounts of state (testing along the way), then manipulating and possibly tearing it down. Without doing this you can't do truly deep testing of function in any complex environment. This example isn't very stateful (I tried to pick an easy example), but it should give you some ideas.

=== Speculative Tests ===
Speculative tests are tests that might or might not apply in a given situation. MySQL testing in the OpenSim tree is done by speculative testing, the tests will only run if there is a properly configured database, otherwise they will not be run. If you execute '''Assert.Ignore()''' in a '''Test''' the test will end and be ignored. If you run '''Assert.Ignore()''' in the '''TestFixtureSetup''' all tests in the test fixture will be skipped and ignored.

Speculative testing lets you create tests that require certain preconditions to be met which you can't guarantee on all platforms/configuration, and are an important part of deep testing.

== Adding Tests to the Tree ==
As we said previously all tests for assembly OpenSim.Foo (in directory OpenSim/Foo) should:
* be in assembly OpenSim.Foo.Tests.dll
* not be in the OpenSim.Foo.dll assembly
* be in OpenSim/Foo/Tests directory

Also, if you have created a new test assembly you must add references to it in both ''.nant/local.include'' and ''.nant/bamboo.build'' to ensure that the assembly is added to the automated bamboo runs as well as the nant test target.

== Executing Tests ==

=== Bamboo ===
On every commit to opensim all the tests are run on a [http://opensimulator.org:8085 Bamboo build server on opensimulator.org]. The process takes about 5 minutes to build, test, and report the results back out on #opensim-dev via the osmantis bot.

=== Nant ===
You can manually run all the tests for OpenSim on your system by running '''nant test''' as a nant target. This will run all the tests that are in the tree, though some speculative tests might be ignored if your platform does not have the right features or configuration to run these tests.

=== NUnit Console ===
If you only want to run tests for one assembly you can do that using the NUnit Console. On Linux just run '''nunit-console2 OpenSim.Foo.Tests.dll''' and it will run only the tests for OpenSim.Foo.Tests.dll. If you are only making changes to 1 dll and just want a quick sanity check, this is the fastest way to do that.

=== Debugging Tests ===
There is a special page dedicated to this. See [[Debugging Unit Tests]].

== Learning More ==
You should definitely read the documentation at the [http://www.nunit.org/index.php?p=documentation NUnit homepage] if you want to know more about testing. It's a very good reference for all the APIs in NUnit that you can use for creating tests.

== Code Coverage ==

A prototype has been included using monocov, which has a profile built-in in mono. Instructions for using monocov can be found on the Mono homepage, in [http://www.mono-project.com/Code_Coverage Code Coverage] section. Unfortunately nunit2 and nant do not support code coverage with monocov (only some other proprietary code coverages), so there is no <monocov> tag. The solution was to implement it using many nunit-console and on <exec> tags.

ATTENTION: Code coverage only works with mono 1.2.x , any other version will most likely not work. Code coverage development is being put on hold for now until it supports newer mono versions.

=== Running ===

Download [http://primates.ximian.com/~lupus/monocov-0.2.tar.gz monocov] and [http://prdownloads.sourceforge.net/nunit/NUnit-2.4.8-net-2.0.zip?download nunit-console] if your nunit-console does not work. To test if it works (my Ubuntu's Hardy version did not), just run one of the tests with nunit-console.

Install monocov (./configure, ./make install, there could be some minor conflicts, I had to add to the compiling line -I/usr/include/mono-1.0/) and make sure the working nunit-console.exe is in /usr/lib/nunit/nunit-console.exe. Now just run nant test-cov and it will generate .cov files and HTML directories in the cov directory. The .cov files can be seen by running monocov on them, and have the same information as the HTML directories.

== Testing Todo ==

=== Coverage ===

A prototype has been done and documented. Now we must keep a look if monocov will evolve to support newer mono versions.

[[Category:Development]]
[[Category:Testing]]

Automated Testing

2009-03-12T07:03:41Z

Mikem: /* Asserts */

As OpenSim matures, we are extremely interested in adding more automated verification into the OpenSim source tree. Test exist not only to prevent bugs from creeping in, but also to provide fair warning that the behavior of the system has changed, and tests may need updating.

In OpenSim today we use NUnit tests. Our conventions are:
# Tests should '''not''' exist inside runtime assemblies, as this makes nunit a production requirement
# Tests should be in .Tests.dll assemblies. For instance, the tests for OpenSim.Data.SQLite.dll should be in the OpenSim.Data.SQLite.Tests.dll assembly. This allows for easy removal of test assemblies in products.
# Tests should be as close to the code as possible, but not intermingled. So the tests for OpenSim/Data/SQLite should be in OpenSim/Data/SQLite/Tests/. Through the use of the '''Exclude''' keyword in prebuild.xml you can ensure that directory is part of OpenSim.Data.SQLite.Tests.dll and not OpenSim.Data.SQLite.dll. See exclude clarification in writing unit tests section.
# Tests testing a class should be grouped into a test class file called xxxTest.cs, where xxx is the name of the class that is being tested.
# Tests should be able to run safely in a production environment. That means that care must be taken not to damage data on the machine that it is being run.
# Tests should be deterministic in other words repeatable. Avoid randomness in tests. See good and bad testing practices below.

== Core Functionality Missing Unit Tests ==

This is a list of functionality which is not covered by unit tests and is identified as highly desireable test target:

# Database Modules (These are mysql tables)
## region ban
## land
## landaccesslist

== Good / Bad Test practices ==
Creating good tests is an art, not a science. Tests are useful by how many bugs they find or how many bugs they avoid. Things you should think about in creating good tests is:
* Throwing edge cases, like 0, "", or Null at parameters. This ensures that people functions are hardened against incomplete data. Many of our crashes come from the lack of this hardening showing up at just the wrong time.
* Use stateful testing to build up complex scenarios. This is more useful than just cursory get / set calls.
* Random tests are not a good idea. We need test results to be deterministic. In other words tests need to be repeatable. If you want to test for a range it is good idea to make separate tests for min and max values. Random values in fields can fail randomly. When something goes wrong for example in database schema the developer will not necessarily notice if the stored values are random. On the other hand its hard to troubleshoot randomly failing tests as you dont know which specific value caused the failure.
* Tests should be independent and should not rely on another test being run, passing or failing. An excerpt from [http://xunitpatterns.com/Principles%20of%20Test%20Automation.html#Independent%20Test xUnit Patterns]:
<blockquote>If tests are interdependent and (even worse) order dependent, we will be depriving ourselves of the useful feedback test failures provide. Interacting Tests [...] tend to fail in a group. The failure of a test that moved the [subject under test] into the state required by the dependent test will lead to the failure of the dependent test too. With both tests failing, how can we tell if it is because of a problem in code that both rely on in some way or is it a problem in code that only the first relies on. With both tests failing we can't tell. We are only talking about two tests here. Imagine how much worse this is with tens or hundreds of tests.</blockquote>
* Only one function of the subject under test should be tested in one test. When testing a database access object, for example, write separate tests for creating DB entries, updating them and removing them.
* Do not use the subject under test to set up the state for the test or to verify the result. Use a different method. When testing a database access object, for example, use raw SQL to insert the initial data into the DB, then run the method being tested. To verify if the operation was successful, use raw SQL again to verify the DB changed as expected.

== Writing Tests ==
Writing a new unit test is pretty easy, and very helpful in increasing the stability of opensim by nailing down bugs. I'm going to present an example here of SQLite Asset testing to show how simple such a test case is to write. The actual in tree SQLite Asset tests are a little different because the code was factored out so that it was easily applied to any database driver, so don't be concerned with the fact that what you see here isn't in the tree.

Exclude clarification: Make sure your master project (not the test project) has an entry for files like the following so that the test code is not included in the master project dll:

<pre>
<Files>
<Match pattern="*.cs" recurse="true">
<Exclude name="Tests" pattern="Tests" />
</Match>
</Files>
</pre>

=== NUnit Conventions ===
An NUnit test suite:
* is a class with a default constructor (takes no arguments)
* has public methods that are tests
* uses annotations to determine what are tests
* runs it's tests in '''alphabetical order by method name'''

An NUnit test method:
* must be public
* must return void
* must take no arguments
* is successful if no exception or assert is thrown while running it

The run order is important if you want to have early tests that setup some complicated state (like creating objects), and have later tests remove or update that state. For that reason I find it very helpful to name all test methods '''Txxx_somename''' where '''xxx''' is a number between 000 and 999. That guaruntees no surprises in run order.

==== Setup / Teardown ====
In the case of testing something like the database layer, we have to actually attempt to store / retrieve things from a database. Following from rule #4 of good tests, we want to make sure not to touch the production databases to run our tests, so during startup we generate a temporary file name which is guaranteed not to be an existing file on the system, and use that as our database file name. By running db.Initialize() the OpenSim migration code will correctly populate that database with the latest schema.

Once we are done with the tests we want to make sure we aren't leaving garbage temp files on the user's system. So we remove that file we created.

During setup we also create a set of state variables, such as 3 uuids, 3 strings, and a data block. You could have always just stuck these inline, but variables are there for a reason, so use them.

==== Asserts ====
You will see scattered through the code '''Assert.That(...)'''. These will throw an exception if the condition is not valid. This format of assertions is called the [http://www.nunit.org/index.php?p=constraintModel&r=2.4 Constraint Model] in NUnit, and provides a large number of tests with the flavor of:
* Assert.That(foo, Is.Null)
* Assert.That(foo, Is.Not.Null)
* Assert.That(foo, Is.True)
* Assert.That(foo, Is.EqualTo(bar))
* Assert.That(foo, Text.Matches( "*bar*" ))

Of note, Is.EqualTo uses the Equals function of foo, so this can only be used on objects that are IComparable. Most of the OpenSim base objects are not, so you'll have to compare fields manually in tests.

For the complete set of conditions you can use see [http://www.nunit.org/index.php?p=constraintModel&r=2.4 the Constraint Model NUnit documentation]. While there is another syntax for tests, the Constraint Model is preferred as it is far more human readable.

=== Simple Negative Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">[Test]
public void T001_LoadEmpty()
{
Assert.That(db.ExistsAsset(uuid1), Is.False);
Assert.That(db.ExistsAsset(uuid2), Is.False);
Assert.That(db.ExistsAsset(uuid3), Is.False);
}
</source>

Test T001 is an example of a simple negative test. We assume a new database will not have any of those assets in them. While the value of this test may look low, it does provide a baseline in ensuring that the database connection is there, that these return false correctly, and that no other exception is thrown. Negative tests are a good way to force bounds conditions and ensure that not only does it ''return what you expect'' it also ''doesn't return what you don't expect''. Thought of another way, it ensures your code is somewhat defensive in nature, not coughing on bad or unexpected data.

=== Simple Positive Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[Test]
public void T010_StoreSimpleAsset()
{
AssetBase a1 = new AssetBase(uuid1, name1);
AssetBase a2 = new AssetBase(uuid2, name2);
AssetBase a3 = new AssetBase(uuid3, name3);
a1.Data = asset1;
a2.Data = asset1;
a3.Data = asset1;

db.CreateAsset(a1);
db.CreateAsset(a2);
db.CreateAsset(a3);

AssetBase a1a = db.FetchAsset(uuid1);
Assert.That(a1a.ID, Is.EqualTo(uuid1));
Assert.That(a1a.Name, Is.EqualTo(name1));

AssetBase a2a = db.FetchAsset(uuid2);
Assert.That(a2a.ID, Is.EqualTo(uuid2));
Assert.That(a2a.Name, Is.EqualTo(name2));

AssetBase a3a = db.FetchAsset(uuid3);
Assert.That(a3a.ID, Is.EqualTo(uuid3));
Assert.That(a3a.Name, Is.EqualTo(name3));
}
</source>
T010 is an example of a simple positive test. In it we create and store 3 assets (ensuring no exceptions), then load those 3 assets back from the database and ensure the fields are correct. Because AssetBase is not IComparible we just check the ID and Name fields with equals tests. If any of the Asserts fail, the whole test fails.

=== Stateful Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[Test]
public void T011_ExistsSimpleAsset()
{
Assert.That(db.ExistsAsset(uuid1), Is.True);
Assert.That(db.ExistsAsset(uuid2), Is.True);
Assert.That(db.ExistsAsset(uuid3), Is.True);
}
</source>

T011 is an example of a stateful test, because it requires the state created by T010 (i.e. the creation of those 3 objects). In order to test any kind of complicated scenario you will find that you need to use stateful tests to build up various amounts of state (testing along the way), then manipulating and possibly tearing it down. Without doing this you can't do truly deep testing of function in any complex environment. This example isn't very stateful (I tried to pick an easy example), but it should give you some ideas.

=== Speculative Tests ===
Speculative tests are tests that might or might not apply in a given situation. MySQL testing in the OpenSim tree is done by speculative testing, the tests will only run if there is a properly configured database, otherwise they will not be run. If you execute '''Assert.Ignore()''' in a '''Test''' the test will end and be ignored. If you run '''Assert.Ignore()''' in the '''TestFixtureSetup''' all tests in the test fixture will be skipped and ignored.

Speculative testing lets you create tests that require certain preconditions to be met which you can't guarantee on all platforms/configuration, and are an important part of deep testing.

== Adding Tests to the Tree ==
As we said previously all tests for assembly OpenSim.Foo (in directory OpenSim/Foo) should:
* be in assembly OpenSim.Foo.Tests.dll
* not be in the OpenSim.Foo.dll assembly
* be in OpenSim/Foo/Tests directory

Also, if you have created a new test assembly you must add references to it in both ''.nant/local.include'' and ''.nant/bamboo.build'' to ensure that the assembly is added to the automated bamboo runs as well as the nant test target.

== Executing Tests ==

=== Bamboo ===
On every commit to opensim all the tests are run on a [http://opensimulator.org:8085 Bamboo build server on opensimulator.org]. The process takes about 5 minutes to build, test, and report the results back out on #opensim-dev via the osmantis bot.

=== Nant ===
You can manually run all the tests for OpenSim on your system by running '''nant test''' as a nant target. This will run all the tests that are in the tree, though some speculative tests might be ignored if your platform does not have the right features or configuration to run these tests.

=== NUnit Console ===
If you only want to run tests for one assembly you can do that using the NUnit Console. On Linux just run '''nunit-console2 OpenSim.Foo.Tests.dll''' and it will run only the tests for OpenSim.Foo.Tests.dll. If you are only making changes to 1 dll and just want a quick sanity check, this is the fastest way to do that.

=== Debugging Tests ===
There is a special page dedicated to this. See [[Debugging Unit Tests]].

== Learning More ==
You should definitely read the documentation at the [http://www.nunit.org/index.php?p=documentation NUnit homepage] if you want to know more about testing. It's a very good reference for all the APIs in NUnit that you can use for creating tests.

== Code Coverage ==

A prototype has been included using monocov, which has a profile built-in in mono. Instructions for using monocov can be found on the Mono homepage, in [http://www.mono-project.com/Code_Coverage Code Coverage] section. Unfortunately nunit2 and nant do not support code coverage with monocov (only some other proprietary code coverages), so there is no <monocov> tag. The solution was to implement it using many nunit-console and on <exec> tags.

ATTENTION: Code coverage only works with mono 1.2.x , any other version will most likely not work. Code coverage development is being put on hold for now until it supports newer mono versions.

=== Running ===

Download [http://primates.ximian.com/~lupus/monocov-0.2.tar.gz monocov] and [http://prdownloads.sourceforge.net/nunit/NUnit-2.4.8-net-2.0.zip?download nunit-console] if your nunit-console does not work. To test if it works (my Ubuntu's Hardy version did not), just run one of the tests with nunit-console.

Install monocov (./configure, ./make install, there could be some minor conflicts, I had to add to the compiling line -I/usr/include/mono-1.0/) and make sure the working nunit-console.exe is in /usr/lib/nunit/nunit-console.exe. Now just run nant test-cov and it will generate .cov files and HTML directories in the cov directory. The .cov files can be seen by running monocov on them, and have the same information as the HTML directories.

== Testing Todo ==

=== Coverage ===

A prototype has been done and documented. Now we must keep a look if monocov will evolve to support newer mono versions.

[[Category:Development]]
[[Category:Testing]]

Automated Testing

2009-03-12T07:02:09Z

Mikem:

As OpenSim matures, we are extremely interested in adding more automated verification into the OpenSim source tree. Test exist not only to prevent bugs from creeping in, but also to provide fair warning that the behavior of the system has changed, and tests may need updating.

In OpenSim today we use NUnit tests. Our conventions are:
# Tests should '''not''' exist inside runtime assemblies, as this makes nunit a production requirement
# Tests should be in .Tests.dll assemblies. For instance, the tests for OpenSim.Data.SQLite.dll should be in the OpenSim.Data.SQLite.Tests.dll assembly. This allows for easy removal of test assemblies in products.
# Tests should be as close to the code as possible, but not intermingled. So the tests for OpenSim/Data/SQLite should be in OpenSim/Data/SQLite/Tests/. Through the use of the '''Exclude''' keyword in prebuild.xml you can ensure that directory is part of OpenSim.Data.SQLite.Tests.dll and not OpenSim.Data.SQLite.dll. See exclude clarification in writing unit tests section.
# Tests testing a class should be grouped into a test class file called xxxTest.cs, where xxx is the name of the class that is being tested.
# Tests should be able to run safely in a production environment. That means that care must be taken not to damage data on the machine that it is being run.
# Tests should be deterministic in other words repeatable. Avoid randomness in tests. See good and bad testing practices below.

== Core Functionality Missing Unit Tests ==

This is a list of functionality which is not covered by unit tests and is identified as highly desireable test target:

# Database Modules (These are mysql tables)
## region ban
## land
## landaccesslist

== Good / Bad Test practices ==
Creating good tests is an art, not a science. Tests are useful by how many bugs they find or how many bugs they avoid. Things you should think about in creating good tests is:
* Throwing edge cases, like 0, "", or Null at parameters. This ensures that people functions are hardened against incomplete data. Many of our crashes come from the lack of this hardening showing up at just the wrong time.
* Use stateful testing to build up complex scenarios. This is more useful than just cursory get / set calls.
* Random tests are not a good idea. We need test results to be deterministic. In other words tests need to be repeatable. If you want to test for a range it is good idea to make separate tests for min and max values. Random values in fields can fail randomly. When something goes wrong for example in database schema the developer will not necessarily notice if the stored values are random. On the other hand its hard to troubleshoot randomly failing tests as you dont know which specific value caused the failure.
* Tests should be independent and should not rely on another test being run, passing or failing. An excerpt from [http://xunitpatterns.com/Principles%20of%20Test%20Automation.html#Independent%20Test xUnit Patterns]:
<blockquote>If tests are interdependent and (even worse) order dependent, we will be depriving ourselves of the useful feedback test failures provide. Interacting Tests [...] tend to fail in a group. The failure of a test that moved the [subject under test] into the state required by the dependent test will lead to the failure of the dependent test too. With both tests failing, how can we tell if it is because of a problem in code that both rely on in some way or is it a problem in code that only the first relies on. With both tests failing we can't tell. We are only talking about two tests here. Imagine how much worse this is with tens or hundreds of tests.</blockquote>
* Only one function of the subject under test should be tested in one test. When testing a database access object, for example, write separate tests for creating DB entries, updating them and removing them.
* Do not use the subject under test to set up the state for the test or to verify the result. Use a different method. When testing a database access object, for example, use raw SQL to insert the initial data into the DB, then run the method being tested. To verify if the operation was successful, use raw SQL again to verify the DB changed as expected.

== Writing Tests ==
Writing a new unit test is pretty easy, and very helpful in increasing the stability of opensim by nailing down bugs. I'm going to present an example here of SQLite Asset testing to show how simple such a test case is to write. The actual in tree SQLite Asset tests are a little different because the code was factored out so that it was easily applied to any database driver, so don't be concerned with the fact that what you see here isn't in the tree.

Exclude clarification: Make sure your master project (not the test project) has an entry for files like the following so that the test code is not included in the master project dll:

<pre>
<Files>
<Match pattern="*.cs" recurse="true">
<Exclude name="Tests" pattern="Tests" />
</Match>
</Files>
</pre>

=== NUnit Conventions ===
An NUnit test suite:
* is a class with a default constructor (takes no arguments)
* has public methods that are tests
* uses annotations to determine what are tests
* runs it's tests in '''alphabetical order by method name'''

An NUnit test method:
* must be public
* must return void
* must take no arguments
* is successful if no exception or assert is thrown while running it

The run order is important if you want to have early tests that setup some complicated state (like creating objects), and have later tests remove or update that state. For that reason I find it very helpful to name all test methods '''Txxx_somename''' where '''xxx''' is a number between 000 and 999. That guaruntees no surprises in run order.

==== Setup / Teardown ====
In the case of testing something like the database layer, we have to actually attempt to store / retrieve things from a database. Following from rule #4 of good tests, we want to make sure not to touch the production databases to run our tests, so during startup we generate a temporary file name which is guaranteed not to be an existing file on the system, and use that as our database file name. By running db.Initialize() the OpenSim migration code will correctly populate that database with the latest schema.

Once we are done with the tests we want to make sure we aren't leaving garbage temp files on the user's system. So we remove that file we created.

During setup we also create a set of state variables, such as 3 uuids, 3 strings, and a data block. You could have always just stuck these inline, but variables are there for a reason, so use them.

==== Asserts ====
You will see scattered through the code '''Assert.That(...)'''. These will throw an exception if the condition is not valid. This format of assertions is called the [http://www.nunit.org/index.php?p=constraintModel&r=2.4 Contraint Model] in NUnit, and provides a large number of tests with the flavor of:
* Assert.That(foo, Is.Null)
* Assert.That(foo, Is.Not.Null)
* Assert.That(foo, Is.True)
* Assert.That(foo, Is.EqualTo(bar))
* Assert.That(foo, Text.Matches( "*bar*" ))

Of note, Is.EqualTo uses the Equals function of foo, so this can only be used on objects that are IComparable. Most of the OpenSim base objects are not, so you'll have to compare fields manually in tests.

For the complete set of conditions you can use see [http://www.nunit.org/index.php?p=constraintModel&r=2.4 the Contraint Model NUnit documentation]. While there is another syntax for tests, the Constraint Model is preferred as it is far more human readable.

=== Simple Negative Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">[Test]
public void T001_LoadEmpty()
{
Assert.That(db.ExistsAsset(uuid1), Is.False);
Assert.That(db.ExistsAsset(uuid2), Is.False);
Assert.That(db.ExistsAsset(uuid3), Is.False);
}
</source>

Test T001 is an example of a simple negative test. We assume a new database will not have any of those assets in them. While the value of this test may look low, it does provide a baseline in ensuring that the database connection is there, that these return false correctly, and that no other exception is thrown. Negative tests are a good way to force bounds conditions and ensure that not only does it ''return what you expect'' it also ''doesn't return what you don't expect''. Thought of another way, it ensures your code is somewhat defensive in nature, not coughing on bad or unexpected data.

=== Simple Positive Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[Test]
public void T010_StoreSimpleAsset()
{
AssetBase a1 = new AssetBase(uuid1, name1);
AssetBase a2 = new AssetBase(uuid2, name2);
AssetBase a3 = new AssetBase(uuid3, name3);
a1.Data = asset1;
a2.Data = asset1;
a3.Data = asset1;

db.CreateAsset(a1);
db.CreateAsset(a2);
db.CreateAsset(a3);

AssetBase a1a = db.FetchAsset(uuid1);
Assert.That(a1a.ID, Is.EqualTo(uuid1));
Assert.That(a1a.Name, Is.EqualTo(name1));

AssetBase a2a = db.FetchAsset(uuid2);
Assert.That(a2a.ID, Is.EqualTo(uuid2));
Assert.That(a2a.Name, Is.EqualTo(name2));

AssetBase a3a = db.FetchAsset(uuid3);
Assert.That(a3a.ID, Is.EqualTo(uuid3));
Assert.That(a3a.Name, Is.EqualTo(name3));
}
</source>
T010 is an example of a simple positive test. In it we create and store 3 assets (ensuring no exceptions), then load those 3 assets back from the database and ensure the fields are correct. Because AssetBase is not IComparible we just check the ID and Name fields with equals tests. If any of the Asserts fail, the whole test fails.

=== Stateful Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[Test]
public void T011_ExistsSimpleAsset()
{
Assert.That(db.ExistsAsset(uuid1), Is.True);
Assert.That(db.ExistsAsset(uuid2), Is.True);
Assert.That(db.ExistsAsset(uuid3), Is.True);
}
</source>

T011 is an example of a stateful test, because it requires the state created by T010 (i.e. the creation of those 3 objects). In order to test any kind of complicated scenario you will find that you need to use stateful tests to build up various amounts of state (testing along the way), then manipulating and possibly tearing it down. Without doing this you can't do truly deep testing of function in any complex environment. This example isn't very stateful (I tried to pick an easy example), but it should give you some ideas.

=== Speculative Tests ===
Speculative tests are tests that might or might not apply in a given situation. MySQL testing in the OpenSim tree is done by speculative testing, the tests will only run if there is a properly configured database, otherwise they will not be run. If you execute '''Assert.Ignore()''' in a '''Test''' the test will end and be ignored. If you run '''Assert.Ignore()''' in the '''TestFixtureSetup''' all tests in the test fixture will be skipped and ignored.

Speculative testing lets you create tests that require certain preconditions to be met which you can't guarantee on all platforms/configuration, and are an important part of deep testing.

== Adding Tests to the Tree ==
As we said previously all tests for assembly OpenSim.Foo (in directory OpenSim/Foo) should:
* be in assembly OpenSim.Foo.Tests.dll
* not be in the OpenSim.Foo.dll assembly
* be in OpenSim/Foo/Tests directory

Also, if you have created a new test assembly you must add references to it in both ''.nant/local.include'' and ''.nant/bamboo.build'' to ensure that the assembly is added to the automated bamboo runs as well as the nant test target.

== Executing Tests ==

=== Bamboo ===
On every commit to opensim all the tests are run on a [http://opensimulator.org:8085 Bamboo build server on opensimulator.org]. The process takes about 5 minutes to build, test, and report the results back out on #opensim-dev via the osmantis bot.

=== Nant ===
You can manually run all the tests for OpenSim on your system by running '''nant test''' as a nant target. This will run all the tests that are in the tree, though some speculative tests might be ignored if your platform does not have the right features or configuration to run these tests.

=== NUnit Console ===
If you only want to run tests for one assembly you can do that using the NUnit Console. On Linux just run '''nunit-console2 OpenSim.Foo.Tests.dll''' and it will run only the tests for OpenSim.Foo.Tests.dll. If you are only making changes to 1 dll and just want a quick sanity check, this is the fastest way to do that.

=== Debugging Tests ===
There is a special page dedicated to this. See [[Debugging Unit Tests]].

== Learning More ==
You should definitely read the documentation at the [http://www.nunit.org/index.php?p=documentation NUnit homepage] if you want to know more about testing. It's a very good reference for all the APIs in NUnit that you can use for creating tests.

== Code Coverage ==

A prototype has been included using monocov, which has a profile built-in in mono. Instructions for using monocov can be found on the Mono homepage, in [http://www.mono-project.com/Code_Coverage Code Coverage] section. Unfortunately nunit2 and nant do not support code coverage with monocov (only some other proprietary code coverages), so there is no <monocov> tag. The solution was to implement it using many nunit-console and on <exec> tags.

ATTENTION: Code coverage only works with mono 1.2.x , any other version will most likely not work. Code coverage development is being put on hold for now until it supports newer mono versions.

=== Running ===

Download [http://primates.ximian.com/~lupus/monocov-0.2.tar.gz monocov] and [http://prdownloads.sourceforge.net/nunit/NUnit-2.4.8-net-2.0.zip?download nunit-console] if your nunit-console does not work. To test if it works (my Ubuntu's Hardy version did not), just run one of the tests with nunit-console.

Install monocov (./configure, ./make install, there could be some minor conflicts, I had to add to the compiling line -I/usr/include/mono-1.0/) and make sure the working nunit-console.exe is in /usr/lib/nunit/nunit-console.exe. Now just run nant test-cov and it will generate .cov files and HTML directories in the cov directory. The .cov files can be seen by running monocov on them, and have the same information as the HTML directories.

== Testing Todo ==

=== Coverage ===

A prototype has been done and documented. Now we must keep a look if monocov will evolve to support newer mono versions.

[[Category:Development]]
[[Category:Testing]]

Automated Testing

2009-03-12T07:01:14Z

Mikem: /* Stateful Tests */

As OpenSim matures, we are extremely interested in adding more automated verification into the OpenSim source tree. Test exist not only to prevent bugs from creeping in, but also to provide fair warning that the behavior of the system has changed, and tests may need updating.

In OpenSim today we use NUnit tests. Our conventions are:
# Tests should '''not''' exist inside runtime assemblies, as this makes nunit a production requirement
# Tests should be in .Tests.dll assemblies. For instance, the tests for OpenSim.Data.SQLite.dll should be in the OpenSim.Data.SQLite.Tests.dll assembly. This allows for easy removal of test assemblies in products.
# Tests should be as close to the code as possible, but not intermingled. So the tests for OpenSim/Data/SQLite should be in OpenSim/Data/SQLite/Tests/. Through the use of the '''Exclude''' keyword in prebuild.xml you can ensure that directory is part of OpenSim.Data.SQLite.Tests.dll and not OpenSim.Data.SQLite.dll. See exclude clarification in writing unit tests section.
# Tests testing a class should be grouped into a test class file called xxxTest.cs, where xxx is the name of the class that is being tested.
# Tests should be able to run safely in a production environment. That means that care must be taken not to damage data on the machine that it is being run.
# Tests should be deterministic in other words repeatable. Avoid randomness in tests. See good and bad testing practices below.

== Core Functionality Missing Unit Tests ==

This is a list of functionality which is not covered by unit tests and is identified as highly desireable test target:

# Database Modules (These are mysql tables)
## region ban
## land
## landaccesslist

== Good / Bad Test practices ==
Creating good tests is an art, not a science. Tests are useful by how many bugs they find or how many bugs they avoid. Things you should think about in creating good tests is:
* Throwing edge cases, like 0, "", or Null at parameters. This ensures that people functions are hardened against incomplete data. Many of our crashes come from the lack of this hardening showing up at just the wrong time.
* Use stateful testing to build up complex scenarios. This is more useful than just cursory get / set calls.
* Random tests are not a good idea. We need test results to be deterministic. In other words tests need to be repeatable. If you want to test for a range it is good idea to make separate tests for min and max values. Random values in fields can fail randomly. When something goes wrong for example in database schema the developer will not necessarily notice if the stored values are random. On the other hand its hard to troubleshoot randomly failing tests as you dont know which specific value caused the failure.
* Tests should be independent and should not rely on another test being run, passing or failing. An excerpt from [http://xunitpatterns.com/Principles%20of%20Test%20Automation.html#Independent%20Test xUnit Patterns]:
<blockquote>If tests are interdependent and (even worse) order dependent, we will be depriving ourselves of the useful feedback test failures provide. Interacting Tests [...] tend to fail in a group. The failure of a test that moved the [subject under test] into the state required by the dependent test will lead to the failure of the dependent test too. With both tests failing, how can we tell if it is because of a problem in code that both rely on in some way or is it a problem in code that only the first relies on. With both tests failing we can't tell. We are only talking about two tests here. Imagine how much worse this is with tens or hundreds of tests.</blockquote>
* Only one function of the subject under test should be tested in one test. When testing a database access object, for example, write separate tests for creating DB entries, updating them and removing them.
* Do not use the subject under test to set up the state for the test or to verify the result. Use a different method. When testing a database access object, for example, use raw SQL to insert the initial data into the DB, then run the method being tested. To verify if the operation was successful, use raw SQL again to verify the DB changed as expected.

== Writing Tests ==
Writing a new unit test is pretty easy, and very helpful in increasing the stability of opensim by nailing down bugs. I'm going to present an example here of SQLite Asset testing to show how simple such a test case is to write. The actual in tree SQLite Asset tests are a little different because the code was factored out so that it was easily applied to any database driver, so don't be concerned with the fact that what you see here isn't in the tree.

Exclude clarification: Make sure your master project (not the test project) has an entry for files like the following so that the test code is not included in the master project dll:

<pre>
<Files>
<Match pattern="*.cs" recurse="true">
<Exclude name="Tests" pattern="Tests" />
</Match>
</Files>
</pre>

=== NUnit Conventions ===
An NUnit test suite:
* is a class with a default constructor (takes no arguments)
* has public methods that are tests
* uses annotations to determine what are tests
* runs it's tests in '''alphabetical order by method name'''

An NUnit test method:
* must be public
* must return void
* must take no arguments
* is successful if no exception or assert is thrown while running it

The run order is important if you want to have early tests that setup some complicated state (like creating objects), and have later tests remove or update that state. For that reason I find it very helpful to name all test methods '''Txxx_somename''' where '''xxx''' is a number between 000 and 999. That guaruntees no surprises in run order.

==== Setup / Teardown ====
In the case of testing something like the database layer, we have to actually attempt to store / retrieve things from a database. Following from rule #4 of good tests, we want to make sure not to touch the production databases to run our tests, so during startup we generate a temporary file name which is guaranteed not to be an existing file on the system, and use that as our database file name. By running db.Initialize() the OpenSim migration code will correctly populate that database with the latest schema.

Once we are done with the tests we want to make sure we aren't leaving garbage temp files on the user's system. So we remove that file we created.

During setup we also create a set of state variables, such as 3 uuids, 3 strings, and a data block. You could have always just stuck these inline, but variables are there for a reason, so use them.

==== Asserts ====
You will see scattered through the code '''Assert.That(...)'''. These will throw an exception if the condition is not valid. This format of assertions is called the [http://www.nunit.org/index.php?p=constraintModel&r=2.4 Contraint Model] in NUnit, and provides a large number of tests with the flavor of:
* Assert.That(foo, Is.Null)
* Assert.That(foo, Is.Not.Null)
* Assert.That(foo, Is.True)
* Assert.That(foo, Is.EqualTo(bar))
* Assert.That(foo, Text.Matches( "*bar*" ))

Of note, Is.EqualTo uses the Equals function of foo, so this can only be used on objects that are IComparable. Most of the OpenSim base objects are not, so you'll have to compare fields manually in tests.

For the complete set of conditions you can use see [http://www.nunit.org/index.php?p=constraintModel&r=2.4 the Contraint Model NUnit documentation]. While there is another syntax for tests, the Constraint Model is preferred as it is far more human readable.

=== Simple Negative Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">[Test]
public void T001_LoadEmpty()
{
Assert.That(db.ExistsAsset(uuid1), Is.False);
Assert.That(db.ExistsAsset(uuid2), Is.False);
Assert.That(db.ExistsAsset(uuid3), Is.False);
}
</source>

Test T001 is an example of a simple negative test. We assume a new database will not have any of those assets in them. While the value of this test may look low, it does provide a baseline in ensuring that the database connection is there, that these return false correctly, and that no other exception is thrown. Negative tests are a good way to force bounds conditions and ensure that not only does it ''return what you expect'' it also ''doesn't return what you don't expect''. Thought of another way, it ensures your code is somewhat defensive in nature, not coughing on bad or unexpected data.

=== Simple Positive Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[Test]
public void T010_StoreSimpleAsset()
{
AssetBase a1 = new AssetBase(uuid1, name1);
AssetBase a2 = new AssetBase(uuid2, name2);
AssetBase a3 = new AssetBase(uuid3, name3);
a1.Data = asset1;
a2.Data = asset1;
a3.Data = asset1;

db.CreateAsset(a1);
db.CreateAsset(a2);
db.CreateAsset(a3);

AssetBase a1a = db.FetchAsset(uuid1);
Assert.That(a1a.ID, Is.EqualTo(uuid1));
Assert.That(a1a.Name, Is.EqualTo(name1));

AssetBase a2a = db.FetchAsset(uuid2);
Assert.That(a2a.ID, Is.EqualTo(uuid2));
Assert.That(a2a.Name, Is.EqualTo(name2));

AssetBase a3a = db.FetchAsset(uuid3);
Assert.That(a3a.ID, Is.EqualTo(uuid3));
Assert.That(a3a.Name, Is.EqualTo(name3));
}
</source>
T010 is an example of a simple positive test. In it we create and store 3 assets (ensuring no exceptions), then load those 3 assets back from the database and ensure the fields are correct. Because AssetBase is not IComparible we just check the ID and Name fields with equals tests. If any of the Asserts fail, the whole test fails.

=== Stateful Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[Test]
public void T011_ExistsSimpleAsset()
{
Assert.That(db.ExistsAsset(uuid1), Is.True);
Assert.That(db.ExistsAsset(uuid2), Is.True);
Assert.That(db.ExistsAsset(uuid3), Is.True);
}
</source>

T011 is an example of a stateful test, because it requires the state created by T010 (i.e. the creation of those 3 objects). In order to test any kind of complicated scenario you will find that you need to use stateful tests to build up various amounts of state (testing along the way), then manipulating and possibly tearing it down. Without doing this you can't do truly deep testing of function in any complex environment. This example isn't very stateful (I tried to pick an easy example), but it should give you some ideas.

=== Speculative Tests ===
Speculative tests are tests that might or might not apply in a given situation. MySQL testing in the OpenSim tree is done by speculative testing, the tests will only run if there is a properly configured database, otherwise they will not be run. If you execute '''Assert.Ignore()''' in a '''Test''' the test will end and be ignored. If you run '''Assert.Ignore()''' in the '''TestFixtureSetup''' all tests in the test fixture will be skipped and ignored.

Speculative testing lets you create tests that require certain preconditions to be met which you can't guarantee on all platforms/configuration, and are an important part of deep testing.

=== An Example Test - SQLite Assets ===
<source lang="csharp">
using System;
using System.IO;
using System.Collections.Generic;
using NUnit.Framework;
using NUnit.Framework.SyntaxHelpers;
using OpenSim.Framework;
using OpenSim.Data.Tests;
using OpenSim.Data.SQLite;
using OpenSim.Region.Environment.Scenes;
using OpenMetaverse;

namespace OpenSim.Data.SQLite.Tests
{
[TestFixture]
public class SQLiteAssetTest
{
public string file;
public string connect;
public AssetDataBase db;
public UUID uuid1;
public UUID uuid2;
public UUID uuid3;
public string name1;
public string name2;
public string name3;
public byte[] asset1;

[TestFixtureSetUp]
public void Init()
{
uuid1 = UUID.Random();
uuid2 = UUID.Random();
uuid3 = UUID.Random();
name1 = "asset one";
name2 = "asset two";
name3 = "asset three";

asset1 = new byte[100];
asset1.Initialize();
file = Path.GetTempFileName() + ".db";
connect = "URI=file:" + file + ",version=3";
db = new SQLiteAssetData();
db.Initialise(connect);
}

[TestFixtureTearDown]
public void Cleanup()
{
db.Dispose();
System.IO.File.Delete(file);
}

[Test]
public void T001_LoadEmpty()
{
Assert.That(db.ExistsAsset(uuid1), Is.False);
Assert.That(db.ExistsAsset(uuid2), Is.False);
Assert.That(db.ExistsAsset(uuid3), Is.False);
}

[Test]
public void T010_StoreSimpleAsset()
{
AssetBase a1 = new AssetBase(uuid1, name1);
AssetBase a2 = new AssetBase(uuid2, name2);
AssetBase a3 = new AssetBase(uuid3, name3);
a1.Data = asset1;
a2.Data = asset1;
a3.Data = asset1;

db.CreateAsset(a1);
db.CreateAsset(a2);
db.CreateAsset(a3);

AssetBase a1a = db.FetchAsset(uuid1);
Assert.That(a1a.ID, Is.EqualTo(uuid1));
Assert.That(a1a.Name, Is.EqualTo(name1));

AssetBase a2a = db.FetchAsset(uuid2);
Assert.That(a2a.ID, Is.EqualTo(uuid2));
Assert.That(a2a.Name, Is.EqualTo(name2));

AssetBase a3a = db.FetchAsset(uuid3);
Assert.That(a3a.ID, Is.EqualTo(uuid3));
Assert.That(a3a.Name, Is.EqualTo(name3));
}

[Test]
public void T011_ExistsSimpleAsset()
{
Assert.That(db.ExistsAsset(uuid1), Is.True);
Assert.That(db.ExistsAsset(uuid2), Is.True);
Assert.That(db.ExistsAsset(uuid3), Is.True);
}
}
}
</source>

You can see 4 of the important annotations here:
* TestFixture - this class is a test suite
* TestFixtureSetup - this code is always run before any of the tests are executed
* TestFixtureTearDown - this code is always run after the tests are done executing, even if they fail.
* Test - this method is a test

== Adding Tests to the Tree ==
As we said previously all tests for assembly OpenSim.Foo (in directory OpenSim/Foo) should:
* be in assembly OpenSim.Foo.Tests.dll
* not be in the OpenSim.Foo.dll assembly
* be in OpenSim/Foo/Tests directory

Also, if you have created a new test assembly you must add references to it in both ''.nant/local.include'' and ''.nant/bamboo.build'' to ensure that the assembly is added to the automated bamboo runs as well as the nant test target.

== Executing Tests ==

=== Bamboo ===
On every commit to opensim all the tests are run on a [http://opensimulator.org:8085 Bamboo build server on opensimulator.org]. The process takes about 5 minutes to build, test, and report the results back out on #opensim-dev via the osmantis bot.

=== Nant ===
You can manually run all the tests for OpenSim on your system by running '''nant test''' as a nant target. This will run all the tests that are in the tree, though some speculative tests might be ignored if your platform does not have the right features or configuration to run these tests.

=== NUnit Console ===
If you only want to run tests for one assembly you can do that using the NUnit Console. On Linux just run '''nunit-console2 OpenSim.Foo.Tests.dll''' and it will run only the tests for OpenSim.Foo.Tests.dll. If you are only making changes to 1 dll and just want a quick sanity check, this is the fastest way to do that.

=== Debugging Tests ===
There is a special page dedicated to this. See [[Debugging Unit Tests]].

== Learning More ==
You should definitely read the documentation at the [http://www.nunit.org/index.php?p=documentation NUnit homepage] if you want to know more about testing. It's a very good reference for all the APIs in NUnit that you can use for creating tests.

== Code Coverage ==

A prototype has been included using monocov, which has a profile built-in in mono. Instructions for using monocov can be found on the Mono homepage, in [http://www.mono-project.com/Code_Coverage Code Coverage] section. Unfortunately nunit2 and nant do not support code coverage with monocov (only some other proprietary code coverages), so there is no <monocov> tag. The solution was to implement it using many nunit-console and on <exec> tags.

ATTENTION: Code coverage only works with mono 1.2.x , any other version will most likely not work. Code coverage development is being put on hold for now until it supports newer mono versions.

=== Running ===

Download [http://primates.ximian.com/~lupus/monocov-0.2.tar.gz monocov] and [http://prdownloads.sourceforge.net/nunit/NUnit-2.4.8-net-2.0.zip?download nunit-console] if your nunit-console does not work. To test if it works (my Ubuntu's Hardy version did not), just run one of the tests with nunit-console.

Install monocov (./configure, ./make install, there could be some minor conflicts, I had to add to the compiling line -I/usr/include/mono-1.0/) and make sure the working nunit-console.exe is in /usr/lib/nunit/nunit-console.exe. Now just run nant test-cov and it will generate .cov files and HTML directories in the cov directory. The .cov files can be seen by running monocov on them, and have the same information as the HTML directories.

== Testing Todo ==

=== Coverage ===

A prototype has been done and documented. Now we must keep a look if monocov will evolve to support newer mono versions.

[[Category:Development]]
[[Category:Testing]]

Automated Testing

2009-03-12T06:59:13Z

Mikem: /* Simple Positive Tests */

As OpenSim matures, we are extremely interested in adding more automated verification into the OpenSim source tree. Test exist not only to prevent bugs from creeping in, but also to provide fair warning that the behavior of the system has changed, and tests may need updating.

In OpenSim today we use NUnit tests. Our conventions are:
# Tests should '''not''' exist inside runtime assemblies, as this makes nunit a production requirement
# Tests should be in .Tests.dll assemblies. For instance, the tests for OpenSim.Data.SQLite.dll should be in the OpenSim.Data.SQLite.Tests.dll assembly. This allows for easy removal of test assemblies in products.
# Tests should be as close to the code as possible, but not intermingled. So the tests for OpenSim/Data/SQLite should be in OpenSim/Data/SQLite/Tests/. Through the use of the '''Exclude''' keyword in prebuild.xml you can ensure that directory is part of OpenSim.Data.SQLite.Tests.dll and not OpenSim.Data.SQLite.dll. See exclude clarification in writing unit tests section.
# Tests testing a class should be grouped into a test class file called xxxTest.cs, where xxx is the name of the class that is being tested.
# Tests should be able to run safely in a production environment. That means that care must be taken not to damage data on the machine that it is being run.
# Tests should be deterministic in other words repeatable. Avoid randomness in tests. See good and bad testing practices below.

== Core Functionality Missing Unit Tests ==

This is a list of functionality which is not covered by unit tests and is identified as highly desireable test target:

# Database Modules (These are mysql tables)
## region ban
## land
## landaccesslist

== Good / Bad Test practices ==
Creating good tests is an art, not a science. Tests are useful by how many bugs they find or how many bugs they avoid. Things you should think about in creating good tests is:
* Throwing edge cases, like 0, "", or Null at parameters. This ensures that people functions are hardened against incomplete data. Many of our crashes come from the lack of this hardening showing up at just the wrong time.
* Use stateful testing to build up complex scenarios. This is more useful than just cursory get / set calls.
* Random tests are not a good idea. We need test results to be deterministic. In other words tests need to be repeatable. If you want to test for a range it is good idea to make separate tests for min and max values. Random values in fields can fail randomly. When something goes wrong for example in database schema the developer will not necessarily notice if the stored values are random. On the other hand its hard to troubleshoot randomly failing tests as you dont know which specific value caused the failure.
* Tests should be independent and should not rely on another test being run, passing or failing. An excerpt from [http://xunitpatterns.com/Principles%20of%20Test%20Automation.html#Independent%20Test xUnit Patterns]:
<blockquote>If tests are interdependent and (even worse) order dependent, we will be depriving ourselves of the useful feedback test failures provide. Interacting Tests [...] tend to fail in a group. The failure of a test that moved the [subject under test] into the state required by the dependent test will lead to the failure of the dependent test too. With both tests failing, how can we tell if it is because of a problem in code that both rely on in some way or is it a problem in code that only the first relies on. With both tests failing we can't tell. We are only talking about two tests here. Imagine how much worse this is with tens or hundreds of tests.</blockquote>
* Only one function of the subject under test should be tested in one test. When testing a database access object, for example, write separate tests for creating DB entries, updating them and removing them.
* Do not use the subject under test to set up the state for the test or to verify the result. Use a different method. When testing a database access object, for example, use raw SQL to insert the initial data into the DB, then run the method being tested. To verify if the operation was successful, use raw SQL again to verify the DB changed as expected.

== Writing Tests ==
Writing a new unit test is pretty easy, and very helpful in increasing the stability of opensim by nailing down bugs. I'm going to present an example here of SQLite Asset testing to show how simple such a test case is to write. The actual in tree SQLite Asset tests are a little different because the code was factored out so that it was easily applied to any database driver, so don't be concerned with the fact that what you see here isn't in the tree.

Exclude clarification: Make sure your master project (not the test project) has an entry for files like the following so that the test code is not included in the master project dll:

<pre>
<Files>
<Match pattern="*.cs" recurse="true">
<Exclude name="Tests" pattern="Tests" />
</Match>
</Files>
</pre>

=== NUnit Conventions ===
An NUnit test suite:
* is a class with a default constructor (takes no arguments)
* has public methods that are tests
* uses annotations to determine what are tests
* runs it's tests in '''alphabetical order by method name'''

An NUnit test method:
* must be public
* must return void
* must take no arguments
* is successful if no exception or assert is thrown while running it

The run order is important if you want to have early tests that setup some complicated state (like creating objects), and have later tests remove or update that state. For that reason I find it very helpful to name all test methods '''Txxx_somename''' where '''xxx''' is a number between 000 and 999. That guaruntees no surprises in run order.

==== Setup / Teardown ====
In the case of testing something like the database layer, we have to actually attempt to store / retrieve things from a database. Following from rule #4 of good tests, we want to make sure not to touch the production databases to run our tests, so during startup we generate a temporary file name which is guaranteed not to be an existing file on the system, and use that as our database file name. By running db.Initialize() the OpenSim migration code will correctly populate that database with the latest schema.

Once we are done with the tests we want to make sure we aren't leaving garbage temp files on the user's system. So we remove that file we created.

During setup we also create a set of state variables, such as 3 uuids, 3 strings, and a data block. You could have always just stuck these inline, but variables are there for a reason, so use them.

==== Asserts ====
You will see scattered through the code '''Assert.That(...)'''. These will throw an exception if the condition is not valid. This format of assertions is called the [http://www.nunit.org/index.php?p=constraintModel&r=2.4 Contraint Model] in NUnit, and provides a large number of tests with the flavor of:
* Assert.That(foo, Is.Null)
* Assert.That(foo, Is.Not.Null)
* Assert.That(foo, Is.True)
* Assert.That(foo, Is.EqualTo(bar))
* Assert.That(foo, Text.Matches( "*bar*" ))

Of note, Is.EqualTo uses the Equals function of foo, so this can only be used on objects that are IComparable. Most of the OpenSim base objects are not, so you'll have to compare fields manually in tests.

For the complete set of conditions you can use see [http://www.nunit.org/index.php?p=constraintModel&r=2.4 the Contraint Model NUnit documentation]. While there is another syntax for tests, the Constraint Model is preferred as it is far more human readable.

=== Simple Negative Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">[Test]
public void T001_LoadEmpty()
{
Assert.That(db.ExistsAsset(uuid1), Is.False);
Assert.That(db.ExistsAsset(uuid2), Is.False);
Assert.That(db.ExistsAsset(uuid3), Is.False);
}
</source>

Test T001 is an example of a simple negative test. We assume a new database will not have any of those assets in them. While the value of this test may look low, it does provide a baseline in ensuring that the database connection is there, that these return false correctly, and that no other exception is thrown. Negative tests are a good way to force bounds conditions and ensure that not only does it ''return what you expect'' it also ''doesn't return what you don't expect''. Thought of another way, it ensures your code is somewhat defensive in nature, not coughing on bad or unexpected data.

=== Simple Positive Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">
[Test]
public void T010_StoreSimpleAsset()
{
AssetBase a1 = new AssetBase(uuid1, name1);
AssetBase a2 = new AssetBase(uuid2, name2);
AssetBase a3 = new AssetBase(uuid3, name3);
a1.Data = asset1;
a2.Data = asset1;
a3.Data = asset1;

db.CreateAsset(a1);
db.CreateAsset(a2);
db.CreateAsset(a3);

AssetBase a1a = db.FetchAsset(uuid1);
Assert.That(a1a.ID, Is.EqualTo(uuid1));
Assert.That(a1a.Name, Is.EqualTo(name1));

AssetBase a2a = db.FetchAsset(uuid2);
Assert.That(a2a.ID, Is.EqualTo(uuid2));
Assert.That(a2a.Name, Is.EqualTo(name2));

AssetBase a3a = db.FetchAsset(uuid3);
Assert.That(a3a.ID, Is.EqualTo(uuid3));
Assert.That(a3a.Name, Is.EqualTo(name3));
}
</source>
T010 is an example of a simple positive test. In it we create and store 3 assets (ensuring no exceptions), then load those 3 assets back from the database and ensure the fields are correct. Because AssetBase is not IComparible we just check the ID and Name fields with equals tests. If any of the Asserts fail, the whole test fails.

=== Stateful Tests ===
T011 is an example of a stateful test, because it requires the state created by T010 (i.e. the creation of those 3 objects). In order to test any kind of complicated scenario you will find that you need to use stateful tests to build up various amounts of state (testing along the way), then manipulating and possibly tearing it down. Without doing this you can't do truly deep testing of function in any complex environment. This example isn't very stateful (I tried to pick an easy example), but it should give you some ideas.

=== Speculative Tests ===
Speculative tests are tests that might or might not apply in a given situation. MySQL testing in the OpenSim tree is done by speculative testing, the tests will only run if there is a properly configured database, otherwise they will not be run. If you execute '''Assert.Ignore()''' in a '''Test''' the test will end and be ignored. If you run '''Assert.Ignore()''' in the '''TestFixtureSetup''' all tests in the test fixture will be skipped and ignored.

Speculative testing lets you create tests that require certain preconditions to be met which you can't guarantee on all platforms/configuration, and are an important part of deep testing.

=== An Example Test - SQLite Assets ===
<source lang="csharp">
using System;
using System.IO;
using System.Collections.Generic;
using NUnit.Framework;
using NUnit.Framework.SyntaxHelpers;
using OpenSim.Framework;
using OpenSim.Data.Tests;
using OpenSim.Data.SQLite;
using OpenSim.Region.Environment.Scenes;
using OpenMetaverse;

namespace OpenSim.Data.SQLite.Tests
{
[TestFixture]
public class SQLiteAssetTest
{
public string file;
public string connect;
public AssetDataBase db;
public UUID uuid1;
public UUID uuid2;
public UUID uuid3;
public string name1;
public string name2;
public string name3;
public byte[] asset1;

[TestFixtureSetUp]
public void Init()
{
uuid1 = UUID.Random();
uuid2 = UUID.Random();
uuid3 = UUID.Random();
name1 = "asset one";
name2 = "asset two";
name3 = "asset three";

asset1 = new byte[100];
asset1.Initialize();
file = Path.GetTempFileName() + ".db";
connect = "URI=file:" + file + ",version=3";
db = new SQLiteAssetData();
db.Initialise(connect);
}

[TestFixtureTearDown]
public void Cleanup()
{
db.Dispose();
System.IO.File.Delete(file);
}

[Test]
public void T001_LoadEmpty()
{
Assert.That(db.ExistsAsset(uuid1), Is.False);
Assert.That(db.ExistsAsset(uuid2), Is.False);
Assert.That(db.ExistsAsset(uuid3), Is.False);
}

[Test]
public void T010_StoreSimpleAsset()
{
AssetBase a1 = new AssetBase(uuid1, name1);
AssetBase a2 = new AssetBase(uuid2, name2);
AssetBase a3 = new AssetBase(uuid3, name3);
a1.Data = asset1;
a2.Data = asset1;
a3.Data = asset1;

db.CreateAsset(a1);
db.CreateAsset(a2);
db.CreateAsset(a3);

AssetBase a1a = db.FetchAsset(uuid1);
Assert.That(a1a.ID, Is.EqualTo(uuid1));
Assert.That(a1a.Name, Is.EqualTo(name1));

AssetBase a2a = db.FetchAsset(uuid2);
Assert.That(a2a.ID, Is.EqualTo(uuid2));
Assert.That(a2a.Name, Is.EqualTo(name2));

AssetBase a3a = db.FetchAsset(uuid3);
Assert.That(a3a.ID, Is.EqualTo(uuid3));
Assert.That(a3a.Name, Is.EqualTo(name3));
}

[Test]
public void T011_ExistsSimpleAsset()
{
Assert.That(db.ExistsAsset(uuid1), Is.True);
Assert.That(db.ExistsAsset(uuid2), Is.True);
Assert.That(db.ExistsAsset(uuid3), Is.True);
}
}
}
</source>

You can see 4 of the important annotations here:
* TestFixture - this class is a test suite
* TestFixtureSetup - this code is always run before any of the tests are executed
* TestFixtureTearDown - this code is always run after the tests are done executing, even if they fail.
* Test - this method is a test

== Adding Tests to the Tree ==
As we said previously all tests for assembly OpenSim.Foo (in directory OpenSim/Foo) should:
* be in assembly OpenSim.Foo.Tests.dll
* not be in the OpenSim.Foo.dll assembly
* be in OpenSim/Foo/Tests directory

Also, if you have created a new test assembly you must add references to it in both ''.nant/local.include'' and ''.nant/bamboo.build'' to ensure that the assembly is added to the automated bamboo runs as well as the nant test target.

== Executing Tests ==

=== Bamboo ===
On every commit to opensim all the tests are run on a [http://opensimulator.org:8085 Bamboo build server on opensimulator.org]. The process takes about 5 minutes to build, test, and report the results back out on #opensim-dev via the osmantis bot.

=== Nant ===
You can manually run all the tests for OpenSim on your system by running '''nant test''' as a nant target. This will run all the tests that are in the tree, though some speculative tests might be ignored if your platform does not have the right features or configuration to run these tests.

=== NUnit Console ===
If you only want to run tests for one assembly you can do that using the NUnit Console. On Linux just run '''nunit-console2 OpenSim.Foo.Tests.dll''' and it will run only the tests for OpenSim.Foo.Tests.dll. If you are only making changes to 1 dll and just want a quick sanity check, this is the fastest way to do that.

=== Debugging Tests ===
There is a special page dedicated to this. See [[Debugging Unit Tests]].

== Learning More ==
You should definitely read the documentation at the [http://www.nunit.org/index.php?p=documentation NUnit homepage] if you want to know more about testing. It's a very good reference for all the APIs in NUnit that you can use for creating tests.

== Code Coverage ==

A prototype has been included using monocov, which has a profile built-in in mono. Instructions for using monocov can be found on the Mono homepage, in [http://www.mono-project.com/Code_Coverage Code Coverage] section. Unfortunately nunit2 and nant do not support code coverage with monocov (only some other proprietary code coverages), so there is no <monocov> tag. The solution was to implement it using many nunit-console and on <exec> tags.

ATTENTION: Code coverage only works with mono 1.2.x , any other version will most likely not work. Code coverage development is being put on hold for now until it supports newer mono versions.

=== Running ===

Download [http://primates.ximian.com/~lupus/monocov-0.2.tar.gz monocov] and [http://prdownloads.sourceforge.net/nunit/NUnit-2.4.8-net-2.0.zip?download nunit-console] if your nunit-console does not work. To test if it works (my Ubuntu's Hardy version did not), just run one of the tests with nunit-console.

Install monocov (./configure, ./make install, there could be some minor conflicts, I had to add to the compiling line -I/usr/include/mono-1.0/) and make sure the working nunit-console.exe is in /usr/lib/nunit/nunit-console.exe. Now just run nant test-cov and it will generate .cov files and HTML directories in the cov directory. The .cov files can be seen by running monocov on them, and have the same information as the HTML directories.

== Testing Todo ==

=== Coverage ===

A prototype has been done and documented. Now we must keep a look if monocov will evolve to support newer mono versions.

[[Category:Development]]
[[Category:Testing]]

Automated Testing

2009-03-12T06:57:07Z

Mikem: /* Simple Negative Tests */

As OpenSim matures, we are extremely interested in adding more automated verification into the OpenSim source tree. Test exist not only to prevent bugs from creeping in, but also to provide fair warning that the behavior of the system has changed, and tests may need updating.

In OpenSim today we use NUnit tests. Our conventions are:
# Tests should '''not''' exist inside runtime assemblies, as this makes nunit a production requirement
# Tests should be in .Tests.dll assemblies. For instance, the tests for OpenSim.Data.SQLite.dll should be in the OpenSim.Data.SQLite.Tests.dll assembly. This allows for easy removal of test assemblies in products.
# Tests should be as close to the code as possible, but not intermingled. So the tests for OpenSim/Data/SQLite should be in OpenSim/Data/SQLite/Tests/. Through the use of the '''Exclude''' keyword in prebuild.xml you can ensure that directory is part of OpenSim.Data.SQLite.Tests.dll and not OpenSim.Data.SQLite.dll. See exclude clarification in writing unit tests section.
# Tests testing a class should be grouped into a test class file called xxxTest.cs, where xxx is the name of the class that is being tested.
# Tests should be able to run safely in a production environment. That means that care must be taken not to damage data on the machine that it is being run.
# Tests should be deterministic in other words repeatable. Avoid randomness in tests. See good and bad testing practices below.

== Core Functionality Missing Unit Tests ==

This is a list of functionality which is not covered by unit tests and is identified as highly desireable test target:

# Database Modules (These are mysql tables)
## region ban
## land
## landaccesslist

== Good / Bad Test practices ==
Creating good tests is an art, not a science. Tests are useful by how many bugs they find or how many bugs they avoid. Things you should think about in creating good tests is:
* Throwing edge cases, like 0, "", or Null at parameters. This ensures that people functions are hardened against incomplete data. Many of our crashes come from the lack of this hardening showing up at just the wrong time.
* Use stateful testing to build up complex scenarios. This is more useful than just cursory get / set calls.
* Random tests are not a good idea. We need test results to be deterministic. In other words tests need to be repeatable. If you want to test for a range it is good idea to make separate tests for min and max values. Random values in fields can fail randomly. When something goes wrong for example in database schema the developer will not necessarily notice if the stored values are random. On the other hand its hard to troubleshoot randomly failing tests as you dont know which specific value caused the failure.
* Tests should be independent and should not rely on another test being run, passing or failing. An excerpt from [http://xunitpatterns.com/Principles%20of%20Test%20Automation.html#Independent%20Test xUnit Patterns]:
<blockquote>If tests are interdependent and (even worse) order dependent, we will be depriving ourselves of the useful feedback test failures provide. Interacting Tests [...] tend to fail in a group. The failure of a test that moved the [subject under test] into the state required by the dependent test will lead to the failure of the dependent test too. With both tests failing, how can we tell if it is because of a problem in code that both rely on in some way or is it a problem in code that only the first relies on. With both tests failing we can't tell. We are only talking about two tests here. Imagine how much worse this is with tens or hundreds of tests.</blockquote>
* Only one function of the subject under test should be tested in one test. When testing a database access object, for example, write separate tests for creating DB entries, updating them and removing them.
* Do not use the subject under test to set up the state for the test or to verify the result. Use a different method. When testing a database access object, for example, use raw SQL to insert the initial data into the DB, then run the method being tested. To verify if the operation was successful, use raw SQL again to verify the DB changed as expected.

== Writing Tests ==
Writing a new unit test is pretty easy, and very helpful in increasing the stability of opensim by nailing down bugs. I'm going to present an example here of SQLite Asset testing to show how simple such a test case is to write. The actual in tree SQLite Asset tests are a little different because the code was factored out so that it was easily applied to any database driver, so don't be concerned with the fact that what you see here isn't in the tree.

Exclude clarification: Make sure your master project (not the test project) has an entry for files like the following so that the test code is not included in the master project dll:

<pre>
<Files>
<Match pattern="*.cs" recurse="true">
<Exclude name="Tests" pattern="Tests" />
</Match>
</Files>
</pre>

=== NUnit Conventions ===
An NUnit test suite:
* is a class with a default constructor (takes no arguments)
* has public methods that are tests
* uses annotations to determine what are tests
* runs it's tests in '''alphabetical order by method name'''

An NUnit test method:
* must be public
* must return void
* must take no arguments
* is successful if no exception or assert is thrown while running it

The run order is important if you want to have early tests that setup some complicated state (like creating objects), and have later tests remove or update that state. For that reason I find it very helpful to name all test methods '''Txxx_somename''' where '''xxx''' is a number between 000 and 999. That guaruntees no surprises in run order.

==== Setup / Teardown ====
In the case of testing something like the database layer, we have to actually attempt to store / retrieve things from a database. Following from rule #4 of good tests, we want to make sure not to touch the production databases to run our tests, so during startup we generate a temporary file name which is guaranteed not to be an existing file on the system, and use that as our database file name. By running db.Initialize() the OpenSim migration code will correctly populate that database with the latest schema.

Once we are done with the tests we want to make sure we aren't leaving garbage temp files on the user's system. So we remove that file we created.

During setup we also create a set of state variables, such as 3 uuids, 3 strings, and a data block. You could have always just stuck these inline, but variables are there for a reason, so use them.

==== Asserts ====
You will see scattered through the code '''Assert.That(...)'''. These will throw an exception if the condition is not valid. This format of assertions is called the [http://www.nunit.org/index.php?p=constraintModel&r=2.4 Contraint Model] in NUnit, and provides a large number of tests with the flavor of:
* Assert.That(foo, Is.Null)
* Assert.That(foo, Is.Not.Null)
* Assert.That(foo, Is.True)
* Assert.That(foo, Is.EqualTo(bar))
* Assert.That(foo, Text.Matches( "*bar*" ))

Of note, Is.EqualTo uses the Equals function of foo, so this can only be used on objects that are IComparable. Most of the OpenSim base objects are not, so you'll have to compare fields manually in tests.

For the complete set of conditions you can use see [http://www.nunit.org/index.php?p=constraintModel&r=2.4 the Contraint Model NUnit documentation]. While there is another syntax for tests, the Constraint Model is preferred as it is far more human readable.

=== Simple Negative Tests ===

See [[Example Test SQLite Assets]] for this code snipped in context.

<source lang="csharp">[Test]
public void T001_LoadEmpty()
{
Assert.That(db.ExistsAsset(uuid1), Is.False);
Assert.That(db.ExistsAsset(uuid2), Is.False);
Assert.That(db.ExistsAsset(uuid3), Is.False);
}
</source>

Test T001 is an example of a simple negative test. We assume a new database will not have any of those assets in them. While the value of this test may look low, it does provide a baseline in ensuring that the database connection is there, that these return false correctly, and that no other exception is thrown. Negative tests are a good way to force bounds conditions and ensure that not only does it ''return what you expect'' it also ''doesn't return what you don't expect''. Thought of another way, it ensures your code is somewhat defensive in nature, not coughing on bad or unexpected data.

=== Simple Positive Tests ===
T010 is an example of a simple positive test. In it we create and store 3 assets (ensuring no exceptions), then load those 3 assets back from the database and ensure the fields are correct. Because AssetBase is not IComparible we just check the ID and Name fields with equals tests. If any of the Asserts fail, the whole test fails.

=== Stateful Tests ===
T011 is an example of a stateful test, because it requires the state created by T010 (i.e. the creation of those 3 objects). In order to test any kind of complicated scenario you will find that you need to use stateful tests to build up various amounts of state (testing along the way), then manipulating and possibly tearing it down. Without doing this you can't do truly deep testing of function in any complex environment. This example isn't very stateful (I tried to pick an easy example), but it should give you some ideas.

=== Speculative Tests ===
Speculative tests are tests that might or might not apply in a given situation. MySQL testing in the OpenSim tree is done by speculative testing, the tests will only run if there is a properly configured database, otherwise they will not be run. If you execute '''Assert.Ignore()''' in a '''Test''' the test will end and be ignored. If you run '''Assert.Ignore()''' in the '''TestFixtureSetup''' all tests in the test fixture will be skipped and ignored.

Speculative testing lets you create tests that require certain preconditions to be met which you can't guarantee on all platforms/configuration, and are an important part of deep testing.

=== An Example Test - SQLite Assets ===
<source lang="csharp">
using System;
using System.IO;
using System.Collections.Generic;
using NUnit.Framework;
using NUnit.Framework.SyntaxHelpers;
using OpenSim.Framework;
using OpenSim.Data.Tests;
using OpenSim.Data.SQLite;
using OpenSim.Region.Environment.Scenes;
using OpenMetaverse;

namespace OpenSim.Data.SQLite.Tests
{
[TestFixture]
public class SQLiteAssetTest
{
public string file;
public string connect;
public AssetDataBase db;
public UUID uuid1;
public UUID uuid2;
public UUID uuid3;
public string name1;
public string name2;
public string name3;
public byte[] asset1;

[TestFixtureSetUp]
public void Init()
{
uuid1 = UUID.Random();
uuid2 = UUID.Random();
uuid3 = UUID.Random();
name1 = "asset one";
name2 = "asset two";
name3 = "asset three";

asset1 = new byte[100];
asset1.Initialize();
file = Path.GetTempFileName() + ".db";
connect = "URI=file:" + file + ",version=3";
db = new SQLiteAssetData();
db.Initialise(connect);
}

[TestFixtureTearDown]
public void Cleanup()
{
db.Dispose();
System.IO.File.Delete(file);
}

[Test]
public void T001_LoadEmpty()
{
Assert.That(db.ExistsAsset(uuid1), Is.False);
Assert.That(db.ExistsAsset(uuid2), Is.False);
Assert.That(db.ExistsAsset(uuid3), Is.False);
}

[Test]
public void T010_StoreSimpleAsset()
{
AssetBase a1 = new AssetBase(uuid1, name1);
AssetBase a2 = new AssetBase(uuid2, name2);
AssetBase a3 = new AssetBase(uuid3, name3);
a1.Data = asset1;
a2.Data = asset1;
a3.Data = asset1;

db.CreateAsset(a1);
db.CreateAsset(a2);
db.CreateAsset(a3);

AssetBase a1a = db.FetchAsset(uuid1);
Assert.That(a1a.ID, Is.EqualTo(uuid1));
Assert.That(a1a.Name, Is.EqualTo(name1));

AssetBase a2a = db.FetchAsset(uuid2);
Assert.That(a2a.ID, Is.EqualTo(uuid2));
Assert.That(a2a.Name, Is.EqualTo(name2));

AssetBase a3a = db.FetchAsset(uuid3);
Assert.That(a3a.ID, Is.EqualTo(uuid3));
Assert.That(a3a.Name, Is.EqualTo(name3));
}

[Test]
public void T011_ExistsSimpleAsset()
{
Assert.That(db.ExistsAsset(uuid1), Is.True);
Assert.That(db.ExistsAsset(uuid2), Is.True);
Assert.That(db.ExistsAsset(uuid3), Is.True);
}
}
}
</source>

You can see 4 of the important annotations here:
* TestFixture - this class is a test suite
* TestFixtureSetup - this code is always run before any of the tests are executed
* TestFixtureTearDown - this code is always run after the tests are done executing, even if they fail.
* Test - this method is a test

== Adding Tests to the Tree ==
As we said previously all tests for assembly OpenSim.Foo (in directory OpenSim/Foo) should:
* be in assembly OpenSim.Foo.Tests.dll
* not be in the OpenSim.Foo.dll assembly
* be in OpenSim/Foo/Tests directory

Also, if you have created a new test assembly you must add references to it in both ''.nant/local.include'' and ''.nant/bamboo.build'' to ensure that the assembly is added to the automated bamboo runs as well as the nant test target.

== Executing Tests ==

=== Bamboo ===
On every commit to opensim all the tests are run on a [http://opensimulator.org:8085 Bamboo build server on opensimulator.org]. The process takes about 5 minutes to build, test, and report the results back out on #opensim-dev via the osmantis bot.

=== Nant ===
You can manually run all the tests for OpenSim on your system by running '''nant test''' as a nant target. This will run all the tests that are in the tree, though some speculative tests might be ignored if your platform does not have the right features or configuration to run these tests.

=== NUnit Console ===
If you only want to run tests for one assembly you can do that using the NUnit Console. On Linux just run '''nunit-console2 OpenSim.Foo.Tests.dll''' and it will run only the tests for OpenSim.Foo.Tests.dll. If you are only making changes to 1 dll and just want a quick sanity check, this is the fastest way to do that.

=== Debugging Tests ===
There is a special page dedicated to this. See [[Debugging Unit Tests]].

== Learning More ==
You should definitely read the documentation at the [http://www.nunit.org/index.php?p=documentation NUnit homepage] if you want to know more about testing. It's a very good reference for all the APIs in NUnit that you can use for creating tests.

== Code Coverage ==

A prototype has been included using monocov, which has a profile built-in in mono. Instructions for using monocov can be found on the Mono homepage, in [http://www.mono-project.com/Code_Coverage Code Coverage] section. Unfortunately nunit2 and nant do not support code coverage with monocov (only some other proprietary code coverages), so there is no <monocov> tag. The solution was to implement it using many nunit-console and on <exec> tags.

ATTENTION: Code coverage only works with mono 1.2.x , any other version will most likely not work. Code coverage development is being put on hold for now until it supports newer mono versions.

=== Running ===

Download [http://primates.ximian.com/~lupus/monocov-0.2.tar.gz monocov] and [http://prdownloads.sourceforge.net/nunit/NUnit-2.4.8-net-2.0.zip?download nunit-console] if your nunit-console does not work. To test if it works (my Ubuntu's Hardy version did not), just run one of the tests with nunit-console.

Install monocov (./configure, ./make install, there could be some minor conflicts, I had to add to the compiling line -I/usr/include/mono-1.0/) and make sure the working nunit-console.exe is in /usr/lib/nunit/nunit-console.exe. Now just run nant test-cov and it will generate .cov files and HTML directories in the cov directory. The .cov files can be seen by running monocov on them, and have the same information as the HTML directories.

== Testing Todo ==

=== Coverage ===

A prototype has been done and documented. Now we must keep a look if monocov will evolve to support newer mono versions.

[[Category:Development]]
[[Category:Testing]]

Example Test SQLite Assets

2009-03-12T06:54:13Z

Mikem: New page: === An Example Test - SQLite Assets === <source lang="csharp"> using System; using System.IO; using System.Collections.Generic; using NUnit.Framework; using NUnit.Framework.SyntaxHelpers; ...

Automated Testing

2009-03-12T06:50:06Z

Mikem: /* Good / Bad Test practices */

As OpenSim matures, we are extremely interested in adding more automated verification into the OpenSim source tree. Test exist not only to prevent bugs from creeping in, but also to provide fair warning that the behavior of the system has changed, and tests may need updating.

In OpenSim today we use NUnit tests. Our conventions are:
# Tests should '''not''' exist inside runtime assemblies, as this makes nunit a production requirement
# Tests should be in .Tests.dll assemblies. For instance, the tests for OpenSim.Data.SQLite.dll should be in the OpenSim.Data.SQLite.Tests.dll assembly. This allows for easy removal of test assemblies in products.
# Tests should be as close to the code as possible, but not intermingled. So the tests for OpenSim/Data/SQLite should be in OpenSim/Data/SQLite/Tests/. Through the use of the '''Exclude''' keyword in prebuild.xml you can ensure that directory is part of OpenSim.Data.SQLite.Tests.dll and not OpenSim.Data.SQLite.dll. See exclude clarification in writing unit tests section.
# Tests testing a class should be grouped into a test class file called xxxTest.cs, where xxx is the name of the class that is being tested.
# Tests should be able to run safely in a production environment. That means that care must be taken not to damage data on the machine that it is being run.
# Tests should be deterministic in other words repeatable. Avoid randomness in tests. See good and bad testing practices below.

== Core Functionality Missing Unit Tests ==

This is a list of functionality which is not covered by unit tests and is identified as highly desireable test target:

# Database Modules (These are mysql tables)
## region ban
## land
## landaccesslist

== Good / Bad Test practices ==
Creating good tests is an art, not a science. Tests are useful by how many bugs they find or how many bugs they avoid. Things you should think about in creating good tests is:
* Throwing edge cases, like 0, "", or Null at parameters. This ensures that people functions are hardened against incomplete data. Many of our crashes come from the lack of this hardening showing up at just the wrong time.
* Use stateful testing to build up complex scenarios. This is more useful than just cursory get / set calls.
* Random tests are not a good idea. We need test results to be deterministic. In other words tests need to be repeatable. If you want to test for a range it is good idea to make separate tests for min and max values. Random values in fields can fail randomly. When something goes wrong for example in database schema the developer will not necessarily notice if the stored values are random. On the other hand its hard to troubleshoot randomly failing tests as you dont know which specific value caused the failure.
* Tests should be independent and should not rely on another test being run, passing or failing. An excerpt from [http://xunitpatterns.com/Principles%20of%20Test%20Automation.html#Independent%20Test xUnit Patterns]:
<blockquote>If tests are interdependent and (even worse) order dependent, we will be depriving ourselves of the useful feedback test failures provide. Interacting Tests [...] tend to fail in a group. The failure of a test that moved the [subject under test] into the state required by the dependent test will lead to the failure of the dependent test too. With both tests failing, how can we tell if it is because of a problem in code that both rely on in some way or is it a problem in code that only the first relies on. With both tests failing we can't tell. We are only talking about two tests here. Imagine how much worse this is with tens or hundreds of tests.</blockquote>
* Only one function of the subject under test should be tested in one test. When testing a database access object, for example, write separate tests for creating DB entries, updating them and removing them.
* Do not use the subject under test to set up the state for the test or to verify the result. Use a different method. When testing a database access object, for example, use raw SQL to insert the initial data into the DB, then run the method being tested. To verify if the operation was successful, use raw SQL again to verify the DB changed as expected.

== Writing Tests ==
Writing a new unit test is pretty easy, and very helpful in increasing the stability of opensim by nailing down bugs. I'm going to present an example here of SQLite Asset testing to show how simple such a test case is to write. The actual in tree SQLite Asset tests are a little different because the code was factored out so that it was easily applied to any database driver, so don't be concerned with the fact that what you see here isn't in the tree.

Exclude clarification: Make sure your master project (not the test project) has an entry for files like the following so that the test code is not included in the master project dll:

<pre>
<Files>
<Match pattern="*.cs" recurse="true">
<Exclude name="Tests" pattern="Tests" />
</Match>
</Files>
</pre>

=== NUnit Conventions ===
An NUnit test suite:
* is a class with a default constructor (takes no arguments)
* has public methods that are tests
* uses annotations to determine what are tests
* runs it's tests in '''alphabetical order by method name'''

An NUnit test method:
* must be public
* must return void
* must take no arguments
* is successful if no exception or assert is thrown while running it

The run order is important if you want to have early tests that setup some complicated state (like creating objects), and have later tests remove or update that state. For that reason I find it very helpful to name all test methods '''Txxx_somename''' where '''xxx''' is a number between 000 and 999. That guaruntees no surprises in run order.

==== Setup / Teardown ====
In the case of testing something like the database layer, we have to actually attempt to store / retrieve things from a database. Following from rule #4 of good tests, we want to make sure not to touch the production databases to run our tests, so during startup we generate a temporary file name which is guaranteed not to be an existing file on the system, and use that as our database file name. By running db.Initialize() the OpenSim migration code will correctly populate that database with the latest schema.

Once we are done with the tests we want to make sure we aren't leaving garbage temp files on the user's system. So we remove that file we created.

During setup we also create a set of state variables, such as 3 uuids, 3 strings, and a data block. You could have always just stuck these inline, but variables are there for a reason, so use them.

==== Asserts ====
You will see scattered through the code '''Assert.That(...)'''. These will throw an exception if the condition is not valid. This format of assertions is called the [http://www.nunit.org/index.php?p=constraintModel&r=2.4 Contraint Model] in NUnit, and provides a large number of tests with the flavor of:
* Assert.That(foo, Is.Null)
* Assert.That(foo, Is.Not.Null)
* Assert.That(foo, Is.True)
* Assert.That(foo, Is.EqualTo(bar))
* Assert.That(foo, Text.Matches( "*bar*" ))

Of note, Is.EqualTo uses the Equals function of foo, so this can only be used on objects that are IComparable. Most of the OpenSim base objects are not, so you'll have to compare fields manually in tests.

For the complete set of conditions you can use see [http://www.nunit.org/index.php?p=constraintModel&r=2.4 the Contraint Model NUnit documentation]. While there is another syntax for tests, the Constraint Model is preferred as it is far more human readable.

=== Simple Negative Tests ===
Test T001 is an example of a simple negative test. We assume a new database will not have any of those assets in them. While the value of this test may look low, it does provide a baseline in ensuring that the database connection is there, that these return false correctly, and that no other exception is thrown. Negative tests are a good way to force bounds conditions and ensure that not only does it ''return what you expect'' it also ''doesn't return what you don't expect''. Thought of another way, it ensures your code is somewhat defensive in nature, not coughing on bad or unexpected data.

=== Simple Positive Tests ===
T010 is an example of a simple positive test. In it we create and store 3 assets (ensuring no exceptions), then load those 3 assets back from the database and ensure the fields are correct. Because AssetBase is not IComparible we just check the ID and Name fields with equals tests. If any of the Asserts fail, the whole test fails.

=== Stateful Tests ===
T011 is an example of a stateful test, because it requires the state created by T010 (i.e. the creation of those 3 objects). In order to test any kind of complicated scenario you will find that you need to use stateful tests to build up various amounts of state (testing along the way), then manipulating and possibly tearing it down. Without doing this you can't do truly deep testing of function in any complex environment. This example isn't very stateful (I tried to pick an easy example), but it should give you some ideas.

=== Speculative Tests ===
Speculative tests are tests that might or might not apply in a given situation. MySQL testing in the OpenSim tree is done by speculative testing, the tests will only run if there is a properly configured database, otherwise they will not be run. If you execute '''Assert.Ignore()''' in a '''Test''' the test will end and be ignored. If you run '''Assert.Ignore()''' in the '''TestFixtureSetup''' all tests in the test fixture will be skipped and ignored.

Speculative testing lets you create tests that require certain preconditions to be met which you can't guarantee on all platforms/configuration, and are an important part of deep testing.

=== An Example Test - SQLite Assets ===
<source lang="csharp">
using System;
using System.IO;
using System.Collections.Generic;
using NUnit.Framework;
using NUnit.Framework.SyntaxHelpers;
using OpenSim.Framework;
using OpenSim.Data.Tests;
using OpenSim.Data.SQLite;
using OpenSim.Region.Environment.Scenes;
using OpenMetaverse;

namespace OpenSim.Data.SQLite.Tests
{
[TestFixture]
public class SQLiteAssetTest
{
public string file;
public string connect;
public AssetDataBase db;
public UUID uuid1;
public UUID uuid2;
public UUID uuid3;
public string name1;
public string name2;
public string name3;
public byte[] asset1;

[TestFixtureSetUp]
public void Init()
{
uuid1 = UUID.Random();
uuid2 = UUID.Random();
uuid3 = UUID.Random();
name1 = "asset one";
name2 = "asset two";
name3 = "asset three";

asset1 = new byte[100];
asset1.Initialize();
file = Path.GetTempFileName() + ".db";
connect = "URI=file:" + file + ",version=3";
db = new SQLiteAssetData();
db.Initialise(connect);
}

[TestFixtureTearDown]
public void Cleanup()
{
db.Dispose();
System.IO.File.Delete(file);
}

[Test]
public void T001_LoadEmpty()
{
Assert.That(db.ExistsAsset(uuid1), Is.False);
Assert.That(db.ExistsAsset(uuid2), Is.False);
Assert.That(db.ExistsAsset(uuid3), Is.False);
}

[Test]
public void T010_StoreSimpleAsset()
{
AssetBase a1 = new AssetBase(uuid1, name1);
AssetBase a2 = new AssetBase(uuid2, name2);
AssetBase a3 = new AssetBase(uuid3, name3);
a1.Data = asset1;
a2.Data = asset1;
a3.Data = asset1;

db.CreateAsset(a1);
db.CreateAsset(a2);
db.CreateAsset(a3);

AssetBase a1a = db.FetchAsset(uuid1);
Assert.That(a1a.ID, Is.EqualTo(uuid1));
Assert.That(a1a.Name, Is.EqualTo(name1));

AssetBase a2a = db.FetchAsset(uuid2);
Assert.That(a2a.ID, Is.EqualTo(uuid2));
Assert.That(a2a.Name, Is.EqualTo(name2));

AssetBase a3a = db.FetchAsset(uuid3);
Assert.That(a3a.ID, Is.EqualTo(uuid3));
Assert.That(a3a.Name, Is.EqualTo(name3));
}

[Test]
public void T011_ExistsSimpleAsset()
{
Assert.That(db.ExistsAsset(uuid1), Is.True);
Assert.That(db.ExistsAsset(uuid2), Is.True);
Assert.That(db.ExistsAsset(uuid3), Is.True);
}
}
}
</source>

You can see 4 of the important annotations here:
* TestFixture - this class is a test suite
* TestFixtureSetup - this code is always run before any of the tests are executed
* TestFixtureTearDown - this code is always run after the tests are done executing, even if they fail.
* Test - this method is a test

== Adding Tests to the Tree ==
As we said previously all tests for assembly OpenSim.Foo (in directory OpenSim/Foo) should:
* be in assembly OpenSim.Foo.Tests.dll
* not be in the OpenSim.Foo.dll assembly
* be in OpenSim/Foo/Tests directory

Also, if you have created a new test assembly you must add references to it in both ''.nant/local.include'' and ''.nant/bamboo.build'' to ensure that the assembly is added to the automated bamboo runs as well as the nant test target.

== Executing Tests ==

=== Bamboo ===
On every commit to opensim all the tests are run on a [http://opensimulator.org:8085 Bamboo build server on opensimulator.org]. The process takes about 5 minutes to build, test, and report the results back out on #opensim-dev via the osmantis bot.

=== Nant ===
You can manually run all the tests for OpenSim on your system by running '''nant test''' as a nant target. This will run all the tests that are in the tree, though some speculative tests might be ignored if your platform does not have the right features or configuration to run these tests.

=== NUnit Console ===
If you only want to run tests for one assembly you can do that using the NUnit Console. On Linux just run '''nunit-console2 OpenSim.Foo.Tests.dll''' and it will run only the tests for OpenSim.Foo.Tests.dll. If you are only making changes to 1 dll and just want a quick sanity check, this is the fastest way to do that.

=== Debugging Tests ===
There is a special page dedicated to this. See [[Debugging Unit Tests]].

== Learning More ==
You should definitely read the documentation at the [http://www.nunit.org/index.php?p=documentation NUnit homepage] if you want to know more about testing. It's a very good reference for all the APIs in NUnit that you can use for creating tests.

== Code Coverage ==

A prototype has been included using monocov, which has a profile built-in in mono. Instructions for using monocov can be found on the Mono homepage, in [http://www.mono-project.com/Code_Coverage Code Coverage] section. Unfortunately nunit2 and nant do not support code coverage with monocov (only some other proprietary code coverages), so there is no <monocov> tag. The solution was to implement it using many nunit-console and on <exec> tags.

ATTENTION: Code coverage only works with mono 1.2.x , any other version will most likely not work. Code coverage development is being put on hold for now until it supports newer mono versions.

=== Running ===

Download [http://primates.ximian.com/~lupus/monocov-0.2.tar.gz monocov] and [http://prdownloads.sourceforge.net/nunit/NUnit-2.4.8-net-2.0.zip?download nunit-console] if your nunit-console does not work. To test if it works (my Ubuntu's Hardy version did not), just run one of the tests with nunit-console.

Install monocov (./configure, ./make install, there could be some minor conflicts, I had to add to the compiling line -I/usr/include/mono-1.0/) and make sure the working nunit-console.exe is in /usr/lib/nunit/nunit-console.exe. Now just run nant test-cov and it will generate .cov files and HTML directories in the cov directory. The .cov files can be seen by running monocov on them, and have the same information as the HTML directories.

== Testing Todo ==

=== Coverage ===

A prototype has been done and documented. Now we must keep a look if monocov will evolve to support newer mono versions.

[[Category:Development]]
[[Category:Testing]]

How to create a dynamic plugin

2009-03-10T02:06:20Z

Mikem:

== Quick Start ==

See [[Dynamic Plugin Quickstart]] for a (hopefully) short introduction.

== Current state of Plugin loading ==

The .NET runtime system has always made reflecting and introspecting of assemblies at runtime very easy to do, making dynamic loading of modules possible with very little effort.

This has unfortunately lead to a situation in OpenSim where historically module loading was hard-coded and done by hand, using copy-and-paste code. Clearly this is a suboptimal situation.

Effort was made to find a reusable, cross-platform dynamic assembly loader that could be used as-is, instead of reinventing our own. The only current candidate is Mono.Addins. MS's .NET runtime has its own Addin class, however as of this writing it is not yet implemented in Mono.

A class called PluginLoader has been created to thinly wrap Mono.Addins, and the assemblies loaded by this class have been standardized on the IPlugin interface hierarchy. With the exception of RegionModules, all dynamically loaded assemblies should be called Plugins, and inherit from this base class.

Currently the following interfaces have been converted to IPlugin:
# OpenSim.IApplicationPlugin
# OpenSim.Grid.GridServer.IGridPlugin
# OpenSim.Data.IGridDataPlugin
# OpenSim.Data.ILogDataPlugin

And the following Assemblies are being loaded by PluginLoader:
# OpenSim.ApplicationPlugins.LoadRegions.LoadRegionsPlugin
# OpenSim.ApplicationPlugins.Rest.Regions.RestRegionPlugin
# OpenSim.ApplicationPlugins.Rest.Inventory.RestHandler
# OpenSim.ApplicationPlugins.RemoteController.RemoteAdminPlugin
# OpenSim.Data.MySQL.MySQLGridData
# OpenSim.Data.MySQL.MySQLLogData

== Why Mono.Addins ==

Mono.Addins offers features outside of being upstream maintained by the Mono project:
* logically splits up module interfaces from implementations in a cross platform way through the use of text-based "extension points" and "addin manifests"
* lazily loads assemblies only when finally necessary
* tracks module dependencies
* allows customize the addin manifest XML
* automatically handles module sharing across application domains
* provides support for downloading assemblies from remote repositories

For more information, please read the [http://www.mono-project.com/Mono.Addins_FAQ Mono.Addins FAQ]

== Mono.Addin concepts ==

Although effort has been made to make PluginLoader independent of the module loading mechanism, there are some concepts that should be understood when learning how to learn the system.

A more detailed (and '''''authoritative'''''!) explanation can be found [http://www.mono-project.com/Mono.Addins_FAQ#Using_Mono.Addins here] and [http://www.mono-project.com/Introduction_to_Mono.Addins here].

=== Separating Male and Female ===

A good system provides a means of loosely coupling two connected parts with a common interface so that one can be changed easily without affecting the other.

The assembly that consumes a service is called a ''host'', and the provider of a service is called an ''addin''.

A service that can be split into producer/consumer role is called an ''extension''. An extension consists of an ''extension point'' which is essentially a text "path" that give a hierarchical name to the service a consumer requires, and an ''extension node'' which represents an object that can provide that service. If a host asks for the addin(s) corresponding to extension point "/Foo", and an addin extension node claims to implement "/Foo", then the two can be joined to make a functioning whole.

All OpenSim extension points start at the root "/OpenSim". If you wished to create a new service "Foo", you might choose to name it "/OpenSim/Foo".

In operation the consumer of a service, the host, requests that the system load all addins that claim to extend an extension point, and a list is returned to the consumer, who can choose to load any subset of those providers, the addins, into memory.

=== Describing an Addin ===

Clearly metadata must be associated with both host and addin, in our case this is an XML file that has the extension *.addin.xml, called a ''manifest''.

In practice, the host manifest and addin manifest are so similar as to be almost exactly the same. The principle difference is that host manifests tell Mono.Addins where to find abstract interfaces, and addin manifests tell Mono.Addins where to find concrete implementations of those interfaces.

The manifest has 3 critical parts
# Name and Version information this is used to check inter-addin dependencies
# Filename of assemblies where the consumer interfaces or provider implementations can be found this is needed to know which assemblies must be searched
# Extension Point path and Fully Qualified .NET Class Name of the consumer interface or provider implementation this is needed to load the actual classes into memory

The way to tell a host manifest from an addin manifest is that host manifests will *always* contain an ''isroot=true'' attribute in the <Addin> tag. However this is complicated by the fact that an addin can act as host for another addin in a recursive manner.

=== Building a model ===

When the PluginLoader is first created within the host, Mono.Addins will search the directory the host was executed and search for all files that end in *.addin.xml. It uses the information to build a ''registry'' about the addins in addin-db-${VERSION}/ directory. The purpose of the registry is to collect all information once, then lazily load assemblies into memory only at the last moment when it is precisely known exactly which class is required by the user.

== How to make your own plugin ==

# Decide on the service you wish to separate into provider and consumer. Ex: "LartMe"
# Create an extension point to identify that service. Ex: "/OpenSim/LartMe"
# Create an consumer interface that has the methods needed, and derive it from OpenSim.Framework.IPlugin. Ex: "OpenSim.ILart"
## If that interface needs a constructor that takes parameters, you will also have to create a class derived from PluginInitialiserBase that can act as a [http://en.wikipedia.org/wiki/Closure_(computer_science) closure] for calling a parameterized constructor. See IApplicationPlugin for an example. Ex: "OpenSim.LartInitializer"
# Write a provider class that implements that interface. Ex: "OpenSim.LartPlugin"
# Write a host manifest. See OpenSim.addin.xml as an example.
## Ensure that /Addin@isroot=true
## Ensure that /Addin@id is set to a unique name. Ex: "LartMe"
## Ensure that /Addin/Runtime/Import@assembly exists for each assembly that either hosts the application (host.EXE) or defines an addin consumer interface (Lart.dll) used by the host
## Ensure that /Addin/ExtensionPoint@path is set. Ex: "<ExtensionPoint path="/OpenSim/LartMe">..."
## Ensure that /Addin/ExtensionPoint/ExtensionNode@name is set to "Plugin". This is a custom derived node type (OpenSim.Framework.PluginExtensionNode).
## Ensure that /Addin/ExtensionPoint/ExtensionNode@type is set to "OpenSim.Framework.PluginExtensionNode"
## Ensure that /Addin/ExtensionPoint/ExtensionNode@objectType is set to the fully qualified name of the consumer interface found in the above assembly import. Ex: "OpenSim.ILart"
# Write a addin manifest. See LoadRegionsPlugin.addin.xml as an example.
## Ensure that /Addin/Runtime/Import@assembly exists for the assembly that implements the producer class.
## Ensure that /Addin/Dependencies/Addin@id exists and is the same as the id of the host manifest. Ex: "Lart"
## Ensure that /Addin/ExtensionPoint@path is set and is the same as that of the host manifest. Ex: "<ExtensionPoint path="/OpenSim/LartMe">..."
## Ensure that /Addin/Extension/Plugin@type is set to the fully qualified name of the implementing provider class. Ex: "OpenSim.LartPlugin"
## Optionally, if you wish to discriminate on "provider" you can set /Addin/Extension/Plugin@provider
# When you need to load the producer, create a PluginLoader of the type defined by the consumer interface. Ex: "new PluginLoader <ILart> ();"
## If the provider requires a Initializer, it should be passed to the PluginLoader constructor. All plugins will be initialized using this initializer. Ex: "new PluginLoader <ILart> (new LartInitializer ("l4rt"));"
# Call PluginLoader.Load using the required extension point. Ex: "loader.Load ("/OpenSim/LartMe")"
# Instead of putting the plugin manifest into the bin/ directory, it is cleaner to embed it into the plugin dll by specifying buildAction="EmbeddedResource" in prebuild.xml to the Files section of your project
## eg <Match pattern="*.addin.xml" path="Resources" buildAction="EmbeddedResource" recurse="true"/>
## For host manifests, it usually makes more sense to have them in the bin/ directory (they would otherwise have to be embedded into a number of dlls and exes)

All loaded plugins can now be found in "loader.Plugins" and used as necessary.

If that looks like a lot of work, consider that outside of the normal work of splitting a class into interface and implementation, its really mostly just boilerplate, and can be accomplished by just comparing how things are currently implemented within OpenSim.

== Filtering and Constraining ==

When you ask to load an extension point, you are potentially bringing a lot of assemblies into memory. Before you do that, you will most likely want to have a say about how many assemblies you expect, and which one you need.

You can do this by assigning a IPluginConstraint or IPluginFilter to an extension point.

An IPluginConstraint is applied each time Load() is called on an extension point, but the behavior of the constraint is up to the implementor. It is assumed that a constraint will ask something of Mono.Addins, and throw an exception if it is not happy with the answer.

=== Loading Critical Plugins ===

For some applications we expect precisely a certain amount of plugins to be loaded. Often exactly one. PluginCountConstraint implements precisely this:
<pre>
int n = 1;
PluginLoader <ILart> loader = new PluginLoader();
loader.AddConstraint ("/OpenSim/LartMe", new PluginCountConstraint (n));
loader.Load ("/OpenSim/LartMe");
</pre>

If the above code is run, and more or less than n plugin could be loaded, the constraint will throw a PluginConstraintViolatedException.

=== Loading Named Plugins ===

Often the reason why more than one plugin is available to load is because we wish to afford choice to the user as to which plugin he prefers, exposed in a configuration file. We read from the file the user's preference, and want to only load the desired plugin.

If we have some addin manifests that look like these:
<pre>
<Addin id="OpenSim.Lart" version="0.1">
<Runtime>
<Import assembly="LartCorp.dll"/>
</Runtime>
<Dependencies>
<Addin id="OpenSim" version="0.5" />
</Dependencies>
<Extension path = "/OpenSim/LartMe">
<Plugin provider="LartCorp" type="Corp.LartMatic" />
</Extension>
</Addin>
...
<Addin id="OpenSim.Lart" version="0.1">
<Runtime>
<Import assembly="OpenSim.Lart.dll"/>
</Runtime>
<Dependencies>
<Addin id="OpenSim" version="0.5" />
</Dependencies>
<Extension path = "/OpenSim/LartMe">
<Plugin provider="OpenLart" type="OpenSim.LartPlugin" />
</Extension>
</Addin>
</pre>

And loading code that looks like this:
<pre>
PluginLoader <ILart> loader = new PluginLoader();
loader.AddConstraint ("/OpenSim/LartMe", new PluginProviderFilter ("LartCorp"));
loader.Load ("/OpenSim/LartMe");
</pre>

Only the "LartCorp" plugin will be loaded.

Dynamic Plugin Quickstart

2009-03-10T02:06:02Z

Mikem:

A quick guide to writing plugins. Please refer to [[How to create a dynamic plugin]] for more in-depth information.

== Defining Extension Points ==

Extension points describe the location in the program where external modules can be plugged-in at runtime. Extension points can be defined in one of two ways:

* decorating interfaces, classes or methods with attributes in the source code
* embedding an XML file, called a manifest, as a resource in an assembly

In OpenSim, the preferred method is to embed an XML manifest in the assembly. Below is an example manifest defining one extension point in the AssetInventoryServer:

<source lang="xml"><Addin id="OpenSim.Grid.AssetInventoryServer" isroot="true" version="0.1">
<Runtime>
<Import assembly="OpenSim.Grid.AssetInventoryServer.exe" />
<Import assembly="OpenSim.Framework.dll" />
</Runtime>
<ExtensionPoint path="/OpenSim/AssetInventoryServer/AssetStorageProvider">
<ExtensionNode name="Plugin"
type="OpenSim.Framework.PluginExtensionNode"
objectType="OpenSim.Grid.AssetInventoryServer.IAssetStorageProvider" />
</ExtensionPoint>
</Addin></source>

This manifest contains the following elements:

* '''Addin'''
** '''id''': an identifier describing this addin, used for dependencies in other addins.
** '''isroot''': set to '''true''' if this manifest does not define any ''extension nodes'', otherwise can be left out completely.
** '''version''': a version number, also used for dependencies in other addins.
* '''Runtime''': a list of assemblies that need to be imported by Mono.Addins as it builds the plugin repository. In the example, the <code>OpenSim.Grid.AssetInventoryServer.exe</code> assembly contains the definition for <code>OpenSim.Grid.AssetInventoryServer.IAssetStorageProvider</code> interface, while <code>OpenSim.Framework.dll</code> contains a definition for <code>OpenSim.Framework.PluginExtensionNode</code>.
* '''ExtensionPoint'''
** '''path''': the location of the extension point in the extension point tree (see [http://www.mono-project.com/Introduction_to_Mono.Addins#Extension_Paths Extension Paths]).
* '''ExtensionNode'''
** '''name''': a name describing this extension node.
** '''type''': a class that handles plugin loading, this should probably be <code>OpenSim.Framework.PluginExtensionNode</code> unless you know what you're doing.
** '''objectType''': an instance of this object will be loaded into this extension point at runtime. This is probably what your plugin will instantiate/subclass/implement.

At this stage, all that is needed is <code>OpenSim.Grid.AssetInventoryServer.IAssetStorageProvider</code> defined and present in <code>OpenSim.Grid.AssetInventoryServer.exe</code>.

'''PLEASE NOTE''': embedding XML manifests in .exe assemblies doesn't work. The .addin.xml file must reside in a directory that Mono.Addins will scan for plugins. Currently in OpenSim this is the current working directory (<code>bin/</code>). Ideally all applications would consist of an .exe assembly and a .dll assembly which contains the embedded XML manifest.

== Defining Extensions ==

Extensions that implement extension points are defined in a similar way. Below is an example corresponding XML file describing a plugin for the above extension point:

<source lang="xml"><Addin id="OpenSim.Grid.AssetInventoryServer.Plugins.OpenSim" version="0.1">
<Runtime>
<Import assembly="OpenSim.Grid.AssetInventoryServer.Plugins.OpenSim.dll" />
<Import assembly="OpenSim.Data.dll" />
</Runtime>

<Dependencies>
<Addin id="OpenSim.Grid.AssetInventoryServer" version="0.1" />
</Dependencies>

<ExtensionPoint path = "/OpenSim/AssetData">
<ExtensionNode name="Plugin"
type="OpenSim.Framework.PluginExtensionNode"
objectType="OpenSim.Data.IAssetDataPlugin" />
</ExtensionPoint>

<Extension path="/OpenSim/AssetInventoryServer/AssetStorageProvider">
<Plugin id="OpenSimAssetStorage"
provider="OpenSim.Grid.AssetInventoryServer.Plugins.OpenSim.dll"
type="OpenSim.Grid.AssetInventoryServer.Plugins.OpenSim.OpenSimAssetStoragePlugin" />
</Extension>
</Addin></source>

The elements (not in the same order as in the file) are:

* '''Addin'''
** '''id''': a string identifying this plugin, can be used for dependencies in other plugins.
** '''version''': a version of this plugin, also used for dependencies in other plugins.
* '''Runtime''': a list of assemblies to import at runtime while Mono.Addins builds up the plugin registry. In this case, <code>OpenSim.Grid.AssetInventoryServer.Plugins.OpenSim.dll</code> contains the definition for <code>OpenSim.Grid.AssetInventoryServer.Plugins.OpenSim.OpenSimAssetStoragePlugin</code>, and <code>OpenSim.Framework.dll</code> contains a definition for <code>OpenSim.Framework.PluginExtensionNode</code>.
* '''Dependencies''': describes other addin(s) required to be loaded before this addin can be loaded. The '''id''' and '''version''' is used.
* '''Extension'''
** '''path''': the path along the extension point tree to the extension point which this extension implements.
** '''Plugin'''
*** '''id''': a string identifying this particular extension. Pretty much an arbitrary string.
*** '''provider''': another property of the extension, which can be an arbitrary string. In OpenSim, the convention seems to be to place the assembly name here which contains this plugin.
*** '''type''': the fully qualified class name to be instantiated which implements the extension. In the code, this class will inherit from/implement the class/interface specified in '''objectType''' attribute of the '''ExtensionPath'''.
* '''NOTE''': this XML file also defines another extension point, meaning that this plugin also takes another plugin. Note the '''isroot''' attribute in the topmost '''Addin''' element is omitted.

== Source Code ==

The relevant code which ends up in <code>OpenSim.Grid.AssetInventoryServer.exe</code> and loads the AssetStorage plugin looks something like this:

<source lang="csharp">private IAssetStorageProvider LoadAssetStorageProvider(string addinPath)
{
PluginLoader<IAssetStorageProvider> loader = new PluginLoader<IAssetStorageProvider>();
loader.Add(addinPath);

try
{
loader.Load();
}
catch (PluginNotInitialisedException e)
{
// log the error, clean up, exit if needed, etc
}

return loader.Plugin;
}

// Then, from elsewhere in the code:
IAssetStorageProvider StorageProvider = LoadAssetStorageProvider("/OpenSim/AssetInventoryServer/AssetStorageProvider");</source>

The <code>PluginLoader</code> class lives in <code>OpenSim.Framework</code>; have a look through the code for more plugin loading options.

Dynamic Plugin Quickstart

2009-03-10T02:05:11Z

Mikem: /* Defining Extension Points */

A quick guide to writing plugins. Please refer to [[How to create a dynamic plugin]] for more in-depth information.

=== Defining Extension Points ===

Extension points describe the location in the program where external modules can be plugged-in at runtime. Extension points can be defined in one of two ways:

* decorating interfaces, classes or methods with attributes in the source code
* embedding an XML file, called a manifest, as a resource in an assembly

In OpenSim, the preferred method is to embed an XML manifest in the assembly. Below is an example manifest defining one extension point in the AssetInventoryServer:

<source lang="xml"><Addin id="OpenSim.Grid.AssetInventoryServer" isroot="true" version="0.1">
<Runtime>
<Import assembly="OpenSim.Grid.AssetInventoryServer.exe" />
<Import assembly="OpenSim.Framework.dll" />
</Runtime>
<ExtensionPoint path="/OpenSim/AssetInventoryServer/AssetStorageProvider">
<ExtensionNode name="Plugin"
type="OpenSim.Framework.PluginExtensionNode"
objectType="OpenSim.Grid.AssetInventoryServer.IAssetStorageProvider" />
</ExtensionPoint>
</Addin></source>

This manifest contains the following elements:

* '''Addin'''
** '''id''': an identifier describing this addin, used for dependencies in other addins.
** '''isroot''': set to '''true''' if this manifest does not define any ''extension nodes'', otherwise can be left out completely.
** '''version''': a version number, also used for dependencies in other addins.
* '''Runtime''': a list of assemblies that need to be imported by Mono.Addins as it builds the plugin repository. In the example, the <code>OpenSim.Grid.AssetInventoryServer.exe</code> assembly contains the definition for <code>OpenSim.Grid.AssetInventoryServer.IAssetStorageProvider</code> interface, while <code>OpenSim.Framework.dll</code> contains a definition for <code>OpenSim.Framework.PluginExtensionNode</code>.
* '''ExtensionPoint'''
** '''path''': the location of the extension point in the extension point tree (see [http://www.mono-project.com/Introduction_to_Mono.Addins#Extension_Paths Extension Paths]).
* '''ExtensionNode'''
** '''name''': a name describing this extension node.
** '''type''': a class that handles plugin loading, this should probably be <code>OpenSim.Framework.PluginExtensionNode</code> unless you know what you're doing.
** '''objectType''': an instance of this object will be loaded into this extension point at runtime. This is probably what your plugin will instantiate/subclass/implement.

At this stage, all that is needed is <code>OpenSim.Grid.AssetInventoryServer.IAssetStorageProvider</code> defined and present in <code>OpenSim.Grid.AssetInventoryServer.exe</code>.

'''PLEASE NOTE''': embedding XML manifests in .exe assemblies doesn't work. The .addin.xml file must reside in a directory that Mono.Addins will scan for plugins. Currently in OpenSim this is the current working directory (<code>bin/</code>). Ideally all applications would consist of an .exe assembly and a .dll assembly which contains the embedded XML manifest.

=== Defining Extensions ===

Extensions that implement extension points are defined in a similar way. Below is an example corresponding XML file describing a plugin for the above extension point:

<source lang="xml"><Addin id="OpenSim.Grid.AssetInventoryServer.Plugins.OpenSim" version="0.1">
<Runtime>
<Import assembly="OpenSim.Grid.AssetInventoryServer.Plugins.OpenSim.dll" />
<Import assembly="OpenSim.Data.dll" />
</Runtime>

<Dependencies>
<Addin id="OpenSim.Grid.AssetInventoryServer" version="0.1" />
</Dependencies>

<ExtensionPoint path = "/OpenSim/AssetData">
<ExtensionNode name="Plugin"
type="OpenSim.Framework.PluginExtensionNode"
objectType="OpenSim.Data.IAssetDataPlugin" />
</ExtensionPoint>

<Extension path="/OpenSim/AssetInventoryServer/AssetStorageProvider">
<Plugin id="OpenSimAssetStorage"
provider="OpenSim.Grid.AssetInventoryServer.Plugins.OpenSim.dll"
type="OpenSim.Grid.AssetInventoryServer.Plugins.OpenSim.OpenSimAssetStoragePlugin" />
</Extension>
</Addin></source>

The elements (not in the same order as in the file) are:

* '''Addin'''
** '''id''': a string identifying this plugin, can be used for dependencies in other plugins.
** '''version''': a version of this plugin, also used for dependencies in other plugins.
* '''Runtime''': a list of assemblies to import at runtime while Mono.Addins builds up the plugin registry. In this case, <code>OpenSim.Grid.AssetInventoryServer.Plugins.OpenSim.dll</code> contains the definition for <code>OpenSim.Grid.AssetInventoryServer.Plugins.OpenSim.OpenSimAssetStoragePlugin</code>, and <code>OpenSim.Framework.dll</code> contains a definition for <code>OpenSim.Framework.PluginExtensionNode</code>.
* '''Dependencies''': describes other addin(s) required to be loaded before this addin can be loaded. The '''id''' and '''version''' is used.
* '''Extension'''
** '''path''': the path along the extension point tree to the extension point which this extension implements.
** '''Plugin'''
*** '''id''': a string identifying this particular extension. Pretty much an arbitrary string.
*** '''provider''': another property of the extension, which can be an arbitrary string. In OpenSim, the convention seems to be to place the assembly name here which contains this plugin.
*** '''type''': the fully qualified class name to be instantiated which implements the extension. In the code, this class will inherit from/implement the class/interface specified in '''objectType''' attribute of the '''ExtensionPath'''.
* '''NOTE''': this XML file also defines another extension point, meaning that this plugin also takes another plugin. Note the '''isroot''' attribute in the topmost '''Addin''' element is omitted.

=== Source Code ===

The relevant code which ends up in <code>OpenSim.Grid.AssetInventoryServer.exe</code> and loads the AssetStorage plugin looks something like this:

<source lang="csharp">private IAssetStorageProvider LoadAssetStorageProvider(string addinPath)
{
PluginLoader<IAssetStorageProvider> loader = new PluginLoader<IAssetStorageProvider>();
loader.Add(addinPath);

try
{
loader.Load();
}
catch (PluginNotInitialisedException e)
{
// log the error, clean up, exit if needed, etc
}

return loader.Plugin;
}

// Then, from elsewhere in the code:
IAssetStorageProvider StorageProvider = LoadAssetStorageProvider("/OpenSim/AssetInventoryServer/AssetStorageProvider");</source>

The <code>PluginLoader</code> class lives in <code>OpenSim.Framework</code>; have a look through the code for more plugin loading options.

Dynamic Plugin Quickstart

2009-03-10T02:02:21Z

Mikem: New page: A quick guide to writing plugins. Please refer to How to create a dynamic plugin for more in-depth information. === Defining Extension Points === Extension points describe the locati...

A quick guide to writing plugins. Please refer to [[How to create a dynamic plugin]] for more in-depth information.

=== Defining Extension Points ===

Extension points describe the location in the program where external modules can be plugged-in at runtime. Extension points can be defined in one of two ways:

* decorating interfaces, classes or methods with attributes in the source code
* embedding an XML file, called a manifest, as a resource in an assembly

In OpenSim, the preferred method is to embed an XML manifest in the assembly. Below is an example manifest defining one extension point in the AssetInventoryServer:

<source lang="xml"><Addin id="OpenSim.Grid.AssetInventoryServer" isroot="true" version="0.1">
<Runtime>
<Import assembly="OpenSim.Grid.AssetInventoryServer.exe" />
<Import assembly="OpenSim.Framework.dll" />
</Runtime>
<ExtensionPoint path="/OpenSim/AssetInventoryServer/AssetStorageProvider">
<ExtensionNode name="Plugin"
type="OpenSim.Framework.PluginExtensionNode"
objectType="OpenSim.Grid.AssetInventoryServer.IAssetStorageProvider" />
</ExtensionPoint>
</Addin></source>

This manifest contains the following elements:

* '''Addin'''
** '''id''': an identifier describing this addin, used for dependencies in other addins.
** '''isroot''': set to '''true''' if this manifest does not define any ''extension nodes'', otherwise can be left out completely.
** '''version''': a version number, also used for dependencies in other addins.
* '''Runtime''': a list of assemblies that need to be imported by Mono.Addins as it builds the plugin repository. In the example, the <code>OpenSim.Grid.AssetInventoryServer.exe</code> assembly contains the definition for <code>OpenSim.Grid.AssetInventoryServer.IAssetStorageProvider</code> interface, while <code>OpenSim.Framework.dll</code> contains a definition for <code>OpenSim.Framework.PluginExtensionNode</code>.
* '''ExtensionPoint'''
** '''path''': the location of the extension point in the extension point tree (see [http://www.mono-project.com/Introduction_to_Mono.Addins#Extension_Paths Extension Paths]).
* '''ExtensionNode'''
** '''name''': a name describing this extension node.
** '''type''': a class that handles plugin loading, this should probably be <code>OpenSim.Framework.PluginExtensionNode</code> unless you know what you're doing.
** '''objectType''': an instance of this object will be loaded into this extension point at runtime. This is probably what your plugin will instantiate/subclass/implement.

At this stage, all that is needed is <code>OpenSim.Grid.AssetInventoryServer.IAssetStorageProvider</code> defined and present in <code>OpenSim.Grid.AssetInventoryServer.exe</code>.

=== Defining Extensions ===

Extensions that implement extension points are defined in a similar way. Below is an example corresponding XML file describing a plugin for the above extension point:

<source lang="xml"><Addin id="OpenSim.Grid.AssetInventoryServer.Plugins.OpenSim" version="0.1">
<Runtime>
<Import assembly="OpenSim.Grid.AssetInventoryServer.Plugins.OpenSim.dll" />
<Import assembly="OpenSim.Data.dll" />
</Runtime>

<Dependencies>
<Addin id="OpenSim.Grid.AssetInventoryServer" version="0.1" />
</Dependencies>

<ExtensionPoint path = "/OpenSim/AssetData">
<ExtensionNode name="Plugin"
type="OpenSim.Framework.PluginExtensionNode"
objectType="OpenSim.Data.IAssetDataPlugin" />
</ExtensionPoint>

<Extension path="/OpenSim/AssetInventoryServer/AssetStorageProvider">
<Plugin id="OpenSimAssetStorage"
provider="OpenSim.Grid.AssetInventoryServer.Plugins.OpenSim.dll"
type="OpenSim.Grid.AssetInventoryServer.Plugins.OpenSim.OpenSimAssetStoragePlugin" />
</Extension>
</Addin></source>

The elements (not in the same order as in the file) are:

* '''Addin'''
** '''id''': a string identifying this plugin, can be used for dependencies in other plugins.
** '''version''': a version of this plugin, also used for dependencies in other plugins.
* '''Runtime''': a list of assemblies to import at runtime while Mono.Addins builds up the plugin registry. In this case, <code>OpenSim.Grid.AssetInventoryServer.Plugins.OpenSim.dll</code> contains the definition for <code>OpenSim.Grid.AssetInventoryServer.Plugins.OpenSim.OpenSimAssetStoragePlugin</code>, and <code>OpenSim.Framework.dll</code> contains a definition for <code>OpenSim.Framework.PluginExtensionNode</code>.
* '''Dependencies''': describes other addin(s) required to be loaded before this addin can be loaded. The '''id''' and '''version''' is used.
* '''Extension'''
** '''path''': the path along the extension point tree to the extension point which this extension implements.
** '''Plugin'''
*** '''id''': a string identifying this particular extension. Pretty much an arbitrary string.
*** '''provider''': another property of the extension, which can be an arbitrary string. In OpenSim, the convention seems to be to place the assembly name here which contains this plugin.
*** '''type''': the fully qualified class name to be instantiated which implements the extension. In the code, this class will inherit from/implement the class/interface specified in '''objectType''' attribute of the '''ExtensionPath'''.
* '''NOTE''': this XML file also defines another extension point, meaning that this plugin also takes another plugin. Note the '''isroot''' attribute in the topmost '''Addin''' element is omitted.

=== Source Code ===

The relevant code which ends up in <code>OpenSim.Grid.AssetInventoryServer.exe</code> and loads the AssetStorage plugin looks something like this:

<source lang="csharp">private IAssetStorageProvider LoadAssetStorageProvider(string addinPath)
{
PluginLoader<IAssetStorageProvider> loader = new PluginLoader<IAssetStorageProvider>();
loader.Add(addinPath);

try
{
loader.Load();
}
catch (PluginNotInitialisedException e)
{
// log the error, clean up, exit if needed, etc
}

return loader.Plugin;
}

// Then, from elsewhere in the code:
IAssetStorageProvider StorageProvider = LoadAssetStorageProvider("/OpenSim/AssetInventoryServer/AssetStorageProvider");</source>

The <code>PluginLoader</code> class lives in <code>OpenSim.Framework</code>; have a look through the code for more plugin loading options.

How to create a dynamic plugin

2009-01-27T03:27:18Z

Mikem: /* How to make your own plugin */

== Current state of Plugin loading ==

The .NET runtime system has always made reflecting and introspecting of assemblies at runtime very easy to do, making dynamic loading of modules possible with very little effort.

This has unfortunately lead to a situation in OpenSim where historically module loading was hard-coded and done by hand, using copy-and-paste code. Clearly this is a suboptimal situation.

Effort was made to find a reusable, cross-platform dynamic assembly loader that could be used as-is, instead of reinventing our own. The only current candidate is Mono.Addins. MS's .NET runtime has its own Addin class, however as of this writing it is not yet implemented in Mono.

A class called PluginLoader has been created to thinly wrap Mono.Addins, and the assemblies loaded by this class have been standardized on the IPlugin interface hierarchy. With the exception of RegionModules, all dynamically loaded assemblies should be called Plugins, and inherit from this base class.

Currently the following interfaces have been converted to IPlugin:
# OpenSim.IApplicationPlugin
# OpenSim.Grid.GridServer.IGridPlugin
# OpenSim.Data.IGridDataPlugin
# OpenSim.Data.ILogDataPlugin

And the following Assemblies are being loaded by PluginLoader:
# OpenSim.ApplicationPlugins.LoadRegions.LoadRegionsPlugin
# OpenSim.ApplicationPlugins.Rest.Regions.RestRegionPlugin
# OpenSim.ApplicationPlugins.Rest.Inventory.RestHandler
# OpenSim.ApplicationPlugins.RemoteController.RemoteAdminPlugin
# OpenSim.Data.MySQL.MySQLGridData
# OpenSim.Data.MySQL.MySQLLogData

== Why Mono.Addins ==

Mono.Addins offers features outside of being upstream maintained by the Mono project:
* logically splits up module interfaces from implementations in a cross platform way through the use of text-based "extension points" and "addin manifests"
* lazily loads assemblies only when finally necessary
* tracks module dependencies
* allows customize the addin manifest XML
* automatically handles module sharing across application domains
* provides support for downloading assemblies from remote repositories

For more information, please read the [http://www.mono-project.com/Mono.Addins_FAQ Mono.Addins FAQ]

== Mono.Addin concepts ==

Although effort has been made to make PluginLoader independent of the module loading mechanism, there are some concepts that should be understood when learning how to learn the system.

A more detailed (and '''''authoritative'''''!) explanation can be found [http://www.mono-project.com/Mono.Addins_FAQ#Using_Mono.Addins here] and [http://www.mono-project.com/Introduction_to_Mono.Addins here].

=== Separating Male and Female ===

A good system provides a means of loosely coupling two connected parts with a common interface so that one can be changed easily without affecting the other.

The assembly that consumes a service is called a ''host'', and the provider of a service is called an ''addin''.

A service that can be split into producer/consumer role is called an ''extension''. An extension consists of an ''extension point'' which is essentially a text "path" that give a hierarchical name to the service a consumer requires, and an ''extension node'' which represents an object that can provide that service. If a host asks for the addin(s) corresponding to extension point "/Foo", and an addin extension node claims to implement "/Foo", then the two can be joined to make a functioning whole.

All OpenSim extension points start at the root "/OpenSim". If you wished to create a new service "Foo", you might choose to name it "/OpenSim/Foo".

In operation the consumer of a service, the host, requests that the system load all addins that claim to extend an extension point, and a list is returned to the consumer, who can choose to load any subset of those providers, the addins, into memory.

=== Describing an Addin ===

Clearly metadata must be associated with both host and addin, in our case this is an XML file that has the extension *.addin.xml, called a ''manifest''.

In practice, the host manifest and addin manifest are so similar as to be almost exactly the same. The principle difference is that host manifests tell Mono.Addins where to find abstract interfaces, and addin manifests tell Mono.Addins where to find concrete implementations of those interfaces.

The manifest has 3 critical parts
# Name and Version information this is used to check inter-addin dependencies
# Filename of assemblies where the consumer interfaces or provider implementations can be found this is needed to know which assemblies must be searched
# Extension Point path and Fully Qualified .NET Class Name of the consumer interface or provider implementation this is needed to load the actual classes into memory

The way to tell a host manifest from an addin manifest is that host manifests will *always* contain an ''isroot=true'' attribute in the <Addin> tag. However this is complicated by the fact that an addin can act as host for another addin in a recursive manner.

=== Building a model ===

When the PluginLoader is first created within the host, Mono.Addins will search the directory the host was executed and search for all files that end in *.addin.xml. It uses the information to build a ''registry'' about the addins in addin-db-${VERSION}/ directory. The purpose of the registry is to collect all information once, then lazily load assemblies into memory only at the last moment when it is precisely known exactly which class is required by the user.

== How to make your own plugin ==

# Decide on the service you wish to separate into provider and consumer. Ex: "LartMe"
# Create an extension point to identify that service. Ex: "/OpenSim/LartMe"
# Create an consumer interface that has the methods needed, and derive it from OpenSim.Framework.IPlugin. Ex: "OpenSim.ILart"
## If that interface needs a constructor that takes parameters, you will also have to create a class derived from PluginInitialiserBase that can act as a [http://en.wikipedia.org/wiki/Closure_(computer_science) closure] for calling a parameterized constructor. See IApplicationPlugin for an example. Ex: "OpenSim.LartInitializer"
# Write a provider class that implements that interface. Ex: "OpenSim.LartPlugin"
# Write a host manifest. See OpenSim.addin.xml as an example.
## Ensure that /Addin@isroot=true
## Ensure that /Addin@id is set to a unique name. Ex: "LartMe"
## Ensure that /Addin/Runtime/Import@assembly exists for each assembly that either hosts the application (host.EXE) or defines an addin consumer interface (Lart.dll) used by the host
## Ensure that /Addin/ExtensionPoint@path is set. Ex: "<ExtensionPoint path="/OpenSim/LartMe">..."
## Ensure that /Addin/ExtensionPoint/ExtensionNode@name is set to "Plugin". This is a custom derived node type (OpenSim.Framework.PluginExtensionNode).
## Ensure that /Addin/ExtensionPoint/ExtensionNode@type is set to "OpenSim.Framework.PluginExtensionNode"
## Ensure that /Addin/ExtensionPoint/ExtensionNode@objectType is set to the fully qualified name of the consumer interface found in the above assembly import. Ex: "OpenSim.ILart"
# Write a addin manifest. See LoadRegionsPlugin.addin.xml as an example.
## Ensure that /Addin/Runtime/Import@assembly exists for the assembly that implements the producer class.
## Ensure that /Addin/Dependencies/Addin@id exists and is the same as the id of the host manifest. Ex: "Lart"
## Ensure that /Addin/ExtensionPoint@path is set and is the same as that of the host manifest. Ex: "<ExtensionPoint path="/OpenSim/LartMe">..."
## Ensure that /Addin/Extension/Plugin@type is set to the fully qualified name of the implementing provider class. Ex: "OpenSim.LartPlugin"
## Optionally, if you wish to discriminate on "provider" you can set /Addin/Extension/Plugin@provider
# When you need to load the producer, create a PluginLoader of the type defined by the consumer interface. Ex: "new PluginLoader <ILart> ();"
## If the provider requires a Initializer, it should be passed to the PluginLoader constructor. All plugins will be initialized using this initializer. Ex: "new PluginLoader <ILart> (new LartInitializer ("l4rt"));"
# Call PluginLoader.Load using the required extension point. Ex: "loader.Load ("/OpenSim/LartMe")"
# Instead of putting the plugin manifest into the bin/ directory, it is cleaner to embed it into the plugin dll by specifying buildAction="EmbeddedResource" in prebuild.xml to the Files section of your project
## eg <Match pattern="*.addin.xml" path="Resources" buildAction="EmbeddedResource" recurse="true"/>
## For host manifests, it usually makes more sense to have them in the bin/ directory (they would otherwise have to be embedded into a number of dlls and exes)

All loaded plugins can now be found in "loader.Plugins" and used as necessary.

If that looks like a lot of work, consider that outside of the normal work of splitting a class into interface and implementation, its really mostly just boilerplate, and can be accomplished by just comparing how things are currently implemented within OpenSim.

== Filtering and Constraining ==

When you ask to load an extension point, you are potentially bringing a lot of assemblies into memory. Before you do that, you will most likely want to have a say about how many assemblies you expect, and which one you need.

You can do this by assigning a IPluginConstraint or IPluginFilter to an extension point.

An IPluginConstraint is applied each time Load() is called on an extension point, but the behavior of the constraint is up to the implementor. It is assumed that a constraint will ask something of Mono.Addins, and throw an exception if it is not happy with the answer.

=== Loading Critical Plugins ===

For some applications we expect precisely a certain amount of plugins to be loaded. Often exactly one. PluginCountConstraint implements precisely this:
<pre>
int n = 1;
PluginLoader <ILart> loader = new PluginLoader();
loader.AddConstraint ("/OpenSim/LartMe", new PluginCountConstraint (n));
loader.Load ("/OpenSim/LartMe");
</pre>

If the above code is run, and more or less than n plugin could be loaded, the constraint will throw a PluginConstraintViolatedException.

=== Loading Named Plugins ===

Often the reason why more than one plugin is available to load is because we wish to afford choice to the user as to which plugin he prefers, exposed in a configuration file. We read from the file the user's preference, and want to only load the desired plugin.

If we have some addin manifests that look like these:
<pre>
<Addin id="OpenSim.Lart" version="0.1">
<Runtime>
<Import assembly="LartCorp.dll"/>
</Runtime>
<Dependencies>
<Addin id="OpenSim" version="0.5" />
</Dependencies>
<Extension path = "/OpenSim/LartMe">
<Plugin provider="LartCorp" type="Corp.LartMatic" />
</Extension>
</Addin>
...
<Addin id="OpenSim.Lart" version="0.1">
<Runtime>
<Import assembly="OpenSim.Lart.dll"/>
</Runtime>
<Dependencies>
<Addin id="OpenSim" version="0.5" />
</Dependencies>
<Extension path = "/OpenSim/LartMe">
<Plugin provider="OpenLart" type="OpenSim.LartPlugin" />
</Extension>
</Addin>
</pre>

And loading code that looks like this:
<pre>
PluginLoader <ILart> loader = new PluginLoader();
loader.AddConstraint ("/OpenSim/LartMe", new PluginProviderFilter ("LartCorp"));
loader.Load ("/OpenSim/LartMe");
</pre>

Only the "LartCorp" plugin will be loaded.

AssetServer/DeveloperDocs

2009-01-15T02:53:06Z

Mikem:

=Developer Documentation=

This documentation gives an overview of the distributed asset server inner workings. The asset server uses ExtensionLoader, which implements the [http://en.wikipedia.org/wiki/Inversion_of_control Inversion of Control] design methodology. The result is a series of distinct components referred to in the documentation as interfaces or extensions. Each interface exposes a series of functions and that may be implemented in very different ways by different extensions. Generally, each interface is assigned a single extension through a configuration file*. Most functions return a StorageResponse value, a simple enumeration of the possible success or failure values of a request. Many asset functions use the common Metadata class which is explained below, and inventory functions use the common InventoryFolder and InventoryItem classes.

* An exception to this rule is the MemcacheStorage provider, which stacks on top of another storage provider to provide scalable caching.

==Code Diagram==

The following diagram shows the interaction between the different interfaces, frontends, and external resources.

''WARNING:'' The model has changed to require frontends to contact the authorization provider instead of passing authorization tokens to the backend. This allows insecure frontends such as the OpenSimFrontend and OpenSimInventoryFrontend to function alongside secured frontends. It also allows partial security such as the BrowseFrontend, where asset IDs are shown but no additional metadata. This diagram needs to be updated to reflect the changes.

[[Image:DistributedAssetServer-UML.png]]

==Metadata==

All assets share a basic set of common metadata. Storage backends should make a best attempt to support the entire common metadata set, and use default values for unsupported fields. The common metadata structure is listed below:

<pre>public class Metadata
{
public UUID ID;
public string Name;
public string Description;
public DateTime CreationDate;
public string ContentType;
public byte[] SHA1;
public bool Temporary;
public Dictionary<string, Uri> Methods;
public OSDMap ExtraData;
}</pre>

===Content-Type===

Asset type is distinguished with an Internet Media Type (MIME type). For backward compatibility with OpenSim's Second Life(tm) implementation, a set of nonstandard content types map to SL asset types. For reference, here is the function that does the conversion:

<pre>public static string SLAssetTypeToExtraData(int assetType)
{
switch (assetType)
{
case 0: return "image/jp2";
case 1: return "application/ogg";
case 2: return "application/x-metaverse-callingcard";
case 3: return "application/x-metaverse-landmark";
case 5: return "application/x-metaverse-clothing";
case 6: return "application/x-metaverse-primitive";
case 7: return "application/x-metaverse-notecard";
case 10: return "application/x-metaverse-lsl";
case 11: return "application/x-metaverse-lso";
case 12: return "image/tga";
case 13: return "application/x-metaverse-bodypart";
case 17: return "audio/x-wav";
case 19: return "image/jpeg";
case 20: return "application/x-metaverse-animation";
case 21: return "application/x-metaverse-gesture";
case 22: return "application/x-metaverse-simstate";
default: return "application/octet-stream";
}
}</pre>

===Methods===

A mapping of strings to URIs holds all of the available methods for an asset. The only common method between all assets is "data", although access level restrictions may prevent users from seeing any methods at all. The method URIs do not necessarily map to resources on the same server that hosts the metadata.

===ExtraData===

The SimpleStorage field is used to store extended values that can be serialized by a frontend interface without any additional knowledge of the data contained in it. The SimpleStorage backend defines two additional metadata fields for JPEG2000 textures: components and layer_ends. components is an integer type holding the number of components (red, green, blue, alpha, etc.) for a texture. layer_ends is an array of integers denoting byte positions in the file where each quality layer ends. The data is stored as an Open Structured Data (OSD) map, allowing frontends to enumerate over the collection of values and serialize to a wire format without prior knowledge of the structure.

==Storage==

Storage providers are responsible for accessing the metadata and file data for assets. Example storage providers include local file storage (OpenSim), using the Amazon Simple Storage Service to host data and associated metadata (AmazonS3Storage), an existing AssetServer.Grid.OpenSim.exe MySQL database (OpenSimMySQLStorage), or a caching storage provider (MemcacheStorage) that works in conjunction with another storage provider. Storage providers must implement the IStorageProvider interface.

<pre>public interface IStorageProvider
{
BackendResponse TryFetchMetadata(UUID assetID, out Metadata metadata);
BackendResponse TryFetchData(UUID assetID, out byte[] assetData);
BackendResponse TryFetchDataMetadata(UUID assetID, out Metadata metadata, out byte[] assetData);
BackendResponse TryCreateAsset(Metadata metadata, byte[] assetData);
BackendResponse TryCreateAsset(Metadata metadata, byte[] assetData, out UUID assetID);
int ForEach(Action<Metadata> action, int start, int count);
}</pre>

The StorageResponse enum has the possible values Success, NotFound, AuthNeeded, and Failure.

==Inventory==

Inventory providers are similar to storage providers but implement a set of interface functions for dealing with inventory items. Inventory is a hierarchical (folder-based) storage system for small inventory item files that contain additional metadata and link to assets. Currently the only inventory backend is SimpleInventory, using file-system based storage for inventory items and folders.

<pre>public interface IInventoryProvider
{
BackendResponse TryFetchItem(Uri owner, UUID itemID, out InventoryItem item);
BackendResponse TryFetchFolder(Uri owner, UUID folderID, out InventoryFolder folder);
BackendResponse TryFetchFolderContents(Uri owner, UUID folderID, out InventoryCollection contents);
BackendResponse TryFetchFolderList(Uri owner, out List<InventoryFolder> folders);
BackendResponse TryFetchInventory(Uri owner, out InventoryCollection inventory);

BackendResponse TryFetchActiveGestures(Uri owner, out List<InventoryItem> gestures);

BackendResponse TryCreateItem(Uri owner, InventoryItem item);
BackendResponse TryCreateFolder(Uri owner, InventoryFolder folder);
BackendResponse TryCreateInventory(Uri owner, InventoryFolder rootFolder);

BackendResponse TryDeleteItem(Uri owner, UUID itemID);
BackendResponse TryDeleteFolder(Uri owner, UUID folderID);
BackendResponse TryPurgeFolder(Uri owner, UUID folderID);
}</pre>

The TryFetchActiveGestures() interface function is currently required by OpenSim, but may be removed in a future release.

==Frontends==

Frontends provide the HTTP interface(s) that users, simulators, and client applications communicate with. Frontends do not implement any interface because no other code depends on them. Unlike the interfaces, many frontends can run at the same time, serving up the same backend content through several different protocols. Source code files implementing frontends typically end with "...Frontend.cs". The current frontends include a browsing interface for paging through the assets with a web browser, an OpenSim asset frontend to replace OpenSim.Grid.AssetServer.exe, an OpenSim inventory frontend to replace OpenSim.Grid.InventoryServer.exe, and a reference frontend that uses the new JSON-based wire protocol.

==Authentication==

Authentication extensions provide a mapping from authentication tokens to identifiers (URIs). The authentication provider may provide a frontend for authenticating with the asset server (such as the OpenIdAuth extension), but the function interfaces are typically only called by authorization providers to convert authentication tokens to identifiers before performing an authorization check.

<pre>public interface IAuthenticationProvider
{
void AddIdentifier(UUID authToken, Uri identifier);
bool RemoveIdentifier(UUID authToken);
bool TryGetIdentifier(UUID authToken, out Uri identifier);
}</pre>

Authentication providers should expire UUID to URI mappings after some amount of inactivity.

==Authorization==

Authorization providers determine what actions are allowed for assets and inventory trees. Given an authentication token, an authorization provider will contact the authentication provider and retrieve an identifier. Based on the identifier, the requested resource, and a set of rules in the authorization provider, access is either denied or granted. These interface functions are called from storage providers where a failed authorization translates into a return code of AuthNeeded. Frontend providers can deliver this message with an HTML message, an HTTP response code, or any number of different ways.

<pre>public interface IAuthorizationProvider
{
bool IsMetadataAuthorized(UUID authToken, UUID assetID);
bool IsDataAuthorized(UUID authToken, UUID assetID);
bool IsCreateAuthorized(UUID authToken);

bool IsInventoryReadAuthorized(UUID authToken, Uri owner);
bool IsInventoryWriteAuthorized(UUID authToken, Uri owner);
}</pre>

= See Also =

* [[AssetServerProposal|Asset Server Proposal]]
* [[AssetServerProposal/ClientDocs|Client Documentation]]

AssetServer/ClientDocs

2009-01-15T02:51:46Z

Mikem:

=Client Documentation=

Through the variety of extensions that ship with the asset server, several wire protocol formats are supported. Connectors for existing OpenSim grid protocols have been added wherever possible, along with new frontends that support web browsers and extensible metadata through JSON.

==Reference Asset Protocol==

In the previous OpenSim asset model, asset requests and asset data proxied through simulators to a backend asset service. This required sending asset data over UDP and adding unnecessary bandwidth overhead to simulators.

[[Image:Client-sim-assetserver-original.png]]
Previous OpenSim asset flow diagram

[[Image:Client-sim-assetserver-distributed.png]]
Distributed Asset Service flow diagram

===Wire Format===

To support the new metadata features and extensible content methods in the asset server, a new wire protocol is being designed. This is referred to in the documentation as the Reference Asset Protocol, and is currently using JSON as the transport format. JSON provides a lightweight markup and is loosely typed, allowing extended metadata to be transferred without breaking compatibility with existing or lightweight clients. Although the markup is pure JSON, some additional "types" are implemented by prefixing strings with type identifiers. This allows data to easily be mapped between [[OpenStructuredData]] and JSON. The types and their prefixes are as follows:

{|
|+ New JSON Types
|-
|uuid:: || UUID, 128-bit unique identifier
|-
|uri:: || Uniform resource identifier
|-
|date:: || ISO-8601 numeric encoding in UTC
|-
|b64:: || Base64-encoded binary data
|}

===Fetching Asset Metadata===

Asset data may be located on the asset server, or scattered all over the world at various URLs. To find the location of the data for each asset, you must first fetch the asset metadata which can be retrieved at a fixed location. The URL for metadata is http://[someserver.com]/[assetid]/metadata. An example request:

<pre>GET /c228d1cf-4b5d-4ba8-84f4-899a0796aa97/metadata HTTP/1.1
Host: assets.virtualworld.com
Date: Mon, 22 Sep  2008 12:00:00 GMT
Authorization: OpenGrid ce9a6995-a737-4206-bae8-910cae310e94</pre>

The Authorization header is used to pass an authentication token (a login cookie) to the server. This is covered in detail in the Authentication section.

An example response:

<pre>{
"id":"uuid::c228d1cf-4b5d-4ba8-84f4-899a0796aa97",
"name":"Default Avatar",
"description":"",
"creation_date":"date::2008-11-26T00:49:48.83Z",
"type":"image/jp2",
"sha1":"b64::3ftHyRoIfVYSeumfxRcSk4LZVVU=",
"temporary":false,
"methods":{
"data":"uri::http://assets.virtualworld.com/c228d1cf-4b5d-4ba8-84f4-899a0796aa97/data"
},
"extra_data":{
"components":3,
"layer_ends":[
580,
1987,
7983,
31969,
36041
]
}
}</pre>

In this case the metadata is for a texture (type of "image/jp2", JPEG2000 compressed image). The methods map contains a single key/value pair for "data", the method used to fetch the actual asset data. The extra_data structure is also used in this example, to carry the number of components in the JPEG2000 data and the byte offsets of each quality layer in the texture.

===Fetching Asset Data===

Once the location of the asset data has been retrieved with a metadata request, a simple HTTP GET will download the content. The Authorization header is still required to fetch content that is not publicly accessible. In a future release, the asset server will support partial downloads with the standard HTTP Range header.

===Creating an Asset===

To create an asset, a simple JSON structure containing the asset metadata and data is POSTed to /createasset. Creating an asset and replacing an existing asset are done through the same interface. The "id" field in the POST data is optional. Specifying a UUID for the new asset may impose additional security restrictions, as this is generally reserved for system users who have been granted permission to replace existing assets. An example POST of a text file called note (specifying a UUID in the request) is shown here:

<pre>POST /createasset HTTP/1.1
Host: assets.virtualworld.com
Date: Mon, 22 Sep  2008 12:00:00 GMT
Authorization: OpenGrid http://users.virtualworld.com/users/jhurliman ce9a6995-a737-4206-bae8-910cae310e94
Content-Type: application/json
Content-Length: 225

{
"id":"uuid::c228d1cf-4b5d-4ba8-84f4-899a0796aa97",
"name":"note",
"description":"A small note containing the word testing",
"type":"text/plain",
"temporary":false,
"data":"b64::dGVzdGluZw=="
}</pre>

An HTTP status code 201 (Created) indicates success; all other status code indicate a failure to create the asset. On success, the return data will be a JSON map containing the key "id" with a value of the new asset's ID. If a failure occurred, the HTTP status description may contain more information. An example of a successful response:

<pre>{ "id":"uuid::c228d1cf-4b5d-4ba8-84f4-899a0796aa97" }</pre>

====Content-Types====

The asset server treats all stored content as binary blobs with content types. Some content types can have special handlers that expose additional methods or metadata, such as image/jp2 returning extended metadata for the number of image components and byte offsets of the quality layers. However, no special handling is needed for storing and serving up data. To store ShockWave Flash files, use the content-type application/x-shockwave-flash and upload a .swf file.

To maintain compatibility with existing virtual worlds, MIME types have been defined for the existing OpenSim asset types. It is important that the correct content-type is used for uploads. The following table shows the mapping between OpenSim assets, commonly used file extensions, and content-types. Note that some of the formats used already have recognized MIME types. Anything starting with application/x-metaverse is ad hoc.

{|
! OpenSim Type
! File Extensions
! Content-Type
! Notes
|-
| 0 || jp2,j2c,texture || image/jp2 || JPEG2000 texture. Not actually JP2 files, only the codestream. LRCP ordering
|-
| 1 || ogg,sound || application/ogg || OGG Vorbis audio
|-
| 2 || callingcard || application/x-metaverse-callingcard || OpenSim calling card. Custom/undocumented text format
|-
| 3 || landmark || application/x-metaverse-landmark || OpenSim landmark. Custom/undocumented text format
|-
| 5 || clothing || application/x-metaverse-clothing || OpenSim clothing. Custom/undocumented text format
|-
| 6 || primitive || application/x-metaverse-primitive || OpenSim primitive. Custom/undocumented XML format
|-
| 7 || notecard || application/x-metaverse-notecard || OpenSim notecard. Custom/undocumented text format
|-
| 10 || lsl || application/x-metaverse-lsl || OpenSim LSL script
|-
| 11 || lso || application/x-metaverse-lso || OpenSim compiled script binary. Currently unused
|-
| 12 || tga || image/tga || Targa image. Currently unused
|-
| 13 || bodypart || application/x-metaverse-bodypart || OpenSim bodypart. Custom/undocumented text format
|-
| 17 || wav || audio/x-wav || WAV audio file
|-
| 19 || jpg,jpeg || image/jpeg || JPEG texture
|-
| 20 || animation || application/x-metaverse-animation || OpenSim animation. Custom/undocumented binary format
|-
| 21 || gesture || application/x-metaverse-gesture || OpenSim gesture. Custom/undocumented format
|-
| 22 || simstate || application/x-metaverse-simstate || OpenSim simulator state. Currently unused
|-
| -1 || (none,unknown) || application/octet-stream || Arbitrary or unrecognized binary data
|}

==Reference Inventory Protocol==

The inventory reference protocol is a work in progress. Leave notes on the discussion page for this document or start a thread on the [https://lists.berlios.de/mailman/listinfo/opensim-dev opensim-dev] mailing list if you'd like to be a part of the process.

==OpenSim Asset Protocol==

The OpenSim.Grid.AssetServer.exe protocol uses a REST interface for retrieving and storing assets. Assets are fetched with a GET to /assets/[assetid] or /assets/[assetid]?texture. The additional ?texture parameter is optional and does not change the response in any way. Asset data and metadata are returned in an XML format, with the asset data base64 encoded as an XML element. To upload, the same XML data structure that is received from a GET is POSTed to /assets/. The XML data is serialized from a class in OpenSim code, therefore the structure is subject to change. The current XML serialization is undocumented outside of source code.

==OpenSim Inventory Protocol==

OpenSim.Grid.InventoryServer.exe supports both a REST-like interface and an XML-RPC interface. Although the former interface shares some similar properties with a REST interface, it defines an additional messaging layer using HTTP POSTs of XML data for all requests. The REST-like interface is actually somewhere between SOAP and XML-RPC.

An example request, passing a single UUID argument in with session information:

<pre><?xml version="1.0"?>
<RestSessionObjectOfGuid xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<SessionID>070b4811-0a57-4fcd-bb3a-a03fbe9106d2</SessionID>
<AvatarID>634ada51-3151-4e6d-a989-309f89f740e0</AvatarID>
<Body>3207551c-da01-49ec-bb6e-f24f9ab36dcb</Body>
</RestSessionObjectOfGuid></pre>

Another example, passing a single UUID argument with no session information:

<pre><?xml version="1.0"?>
<guid>3207551c-da01-49ec-bb6e-f24f9ab36dcb</guid></pre>

==Authentication==

If an asset is protected, an identity must be proven by a client (and that identity authorized to access the asset) before access is granted. Internally the asset server stores identities as URLs. A number of different methods of proving client identity can be used depending on the setup of the asset server. The reference asset server has OpenID login support at http://[yourserver.com]/authenticate. This is a simple OpenID login process that can happen over the reference JSON protocol, or through web forms. If the OpenID provider being used is setup with a well formed login method (such as the current OpenSim user server), the entire process can happen without user intervention. Once the OpenID authentication has completed, a UUID token will be returned to the client. Optionally, a cookie will be returned as well containing the same token. This token is a temporary proof of identity that can be used in the Authorization header when requesting assets.

Instead of directly logging into the asset server, a common grid setup may be that the login server contacts an asset server that it knows about to inform it of a successful login and create a temporary session token. This token could be passed back to the client to avoid having to login to both the login server and asset server. Another setup may involve the client using an SSL client certificate signed by an authority that the asset server trusts. The different methods of authenticating allow flexibility between creating a loose, decentralized set of grids and services and a tightly coupled closed or semi-closed grid.

= See Also =

* [[AssetServerProposal|Asset Server Proposal]]
* [[AssetServerProposal/DeveloperDocs|Developer Documentation]]

AssetServer

2009-01-15T02:50:55Z

Mikem:

{{proposal}}

=Asset Server Proposal=

==Motivation==

Many virtual worlds are being developed independently with a wide variety of different architectures and protocols. However, one common component across these worlds is an asset service. An asset service is the means of storing and serving static content that populates the world; anything from textures and 3D meshes to clothing definitions can be stored. The current generation of virtual world asset services are implemented as proprietary, behind-the-simulator services that are not directly available to a user.

We believe there are shortcomings with this approach. Specifically:

# Assets are tied to a single grid. Wide availability of content is critical to the growth of virtual worlds, and the inability to share content between grids limits the ability of small providers. Having an asset server closely tied to a grid limits the ability of many small grids to reach critical mass.
# The current vendor-specific nature of asset storage limits the ability of third-party value added services. Offline content creation, asset archiving, specialized caching, and web service integration are all services that could be added with a more open approach.
# With assets "locked up" in proprietary asset servers, the ability to combine and re-purpose content is limited. For instance, specialized asset processing tuned to mobile applications.
# An asset server closely tied to a grid limits the content creator and owner's ability to manage asset lifecycle. An asset owner may wish to have their assets stored, backed up, and tracked from their own servers rather than handing over control to a grid.

==Vision==

We envision virtual world grids modeled after the current World Wide Web, with millions of independent administrative domains. Content will be spread across millions of small, independent domains as well as aggregated on large domains supporting millions of users. A rich community of value added services and the free and open exchange of content will weave the network together, much as the Web 2.0 movement is tying the web together today. Every organization can choose to host an asset service or connect with any number of third party asset providers. Additionally, the content rights decisions are placed in the hands of the content hosts. With the proper authentication users are free to move assets to wherever they roam.

Data services will become as important as data hosting itself. Just as search engines and content portals have changed how we use the web, services that can plug into a common interface in asset hosting will change how we use virtual worlds. Auditing services can provide an approach to rights management and traffic analytics. Existing caching techniques and services that have been built for today's web content can be leveraged for delivery of rich virtual world content.

==Approach==

Most implementations, including Second Life® and its open source derivatives, have opaque asset hosting that is a part of the world itself. The opaque asset hosting model has a choke point at the simulator level. Asset downloads make up the bulk of virtual world bandwidth requirements, and in the current Second Life model all assets must be proxied through the simulator.

In the proposed architecture, control of assets is moved from grid owners to content owners to allow cross-world assets with content authorization management. Protocol enhancements could include better support for content rights, asset hosting as an independent service, asset migration between worlds, and scalable performance by allowing centralized servers to be removed from the equation.

Our approach is to start simple by proposing common interfaces and paths for extensibility. By working with the community to refine these interfaces and explore use cases, we can outline specifications and minimum requirements documents. The short-term goals are to consolidate on a common metadata format with core features for version one, and build a reference asset service that plugs into the OpenSim UGAIM model.

===Asset Hosting as An Independent Service===

The current model of asset hosting approaches trust and permissions from the viewpoint of a grid. Users authenticate with a grid login service, upload assets to grid asset storage, and set permissions that apply to other users of that grid. By moving authorization to the asset storage, an asset service no longer needs to be tied to a specific grid and can serve anyone who is properly authenticated. This opens up the possibility of web service access to assets or asset metadata, or other use cases that do not directly involve logging in to a virtual world.

===Leveraging Existing Infrastructure===

With asset serving separated from grid-specific code, the solution looks similar to existing solutions. Proven infrastructure for asset lookups, authentication, serving metadata, and content distribution can be used as building blocks.

===Scalability===

The difficult research in scalable content distribution has already been done and implemented across the web in various different implementations. The goal is to reuse that existing knowledge for virtual world assets, adding optional features such as identity-based permission levels or new caching strategies.

==Terminology==

* Client: From the asset server's point of view, the networked program that is fetching or storing an asset.
* Asset Server: The network service that presents an interface allowing fetching and storing of asset items.
* Asset: A typed blob of data. The actual definition of asset content is left to the asset creator.
* Asset Metadata: Information stored along with the asset blob which includes type-specific information, ownership information, creation information (creation data), license, and anything else useful in identifying and controlling the asset data.
* Asset ID: A 128-bit identifier for content in a virtual world. These identifiers may or may not follow RFC 4122; the specific implementation is out of scope of the asset service. Asset IDs only have context in the grid they were created in.
* Authentication Token: A session cookie that is temporarily stored on the Asset Server and the Client that proves authentication and may grant some level of authorization. These cookies take the same UUID form as Asset IDs.
* Grid: For the purposes of this document, a grid is defined as the set of services where a particular Asset ID has contextual meaning.

==Protocol Overview==

# The end user retrieves an authorization token to connect to an asset service. This can be returned by directly connecting to an asset service and supplying OpenID credentials, or an out of band method such as logging into a virtual world grid server.
# The end user does a GET operation, supplying the credentials from step 1, and requests the metadata for an asset ID.
# The asset service performs a lookup step which resolves the ID to metadata which includes the location of the content, a list of supported REST operations on the content, and additional information, such as the dimensions of a texture.
# The end user has information on different valid operations on the asset, and in the most common scenario will choose to use the fetch operation to download the asset.

All user-facing services must be available over standard HTTP 1.1, and must adopt at least one common metadata format.

==Asset Metadata Request==

An end user must discover the asset service through some means. The specific means of discovery is out of scope of this document but could for example be built into a client application, provided during grid login, or published on a webpage. To request asset metadata, an HTTP 1.1 GET is made to the asset service with the asset ID and metadata operation specified. For example, a request might look like:

<pre>GET /89556747-24cb-43ed-920b-47caed15465f/metadata HTTP/1.1
Host: assets.virtualworld.com
Date: Mon, 22 Sep  2008 12:00:00 GMT
Authorization: OpenGrid ce9a6995-a737-4206-bae8-910cae310e94</pre>

If the asset is a protected resource the Authorization header is required. If the resource is publicly accessible, no Authorization header is needed and a normal HTTP GET (such as one generated by a web browser) can be used.

==Authentication==

Access to the asset service methods is controlled with a authentication token, a UUID. This UUID is a non-guessable, temporary token assigned to an end user at the end of a successful authentication process. This token could be granted by authentication through a virtual world login server (where the login server is responsible for transmitting the authentication token to the asset server), or through a direct login to the asset service itself using OpenID. Temporary tokens must only be given out to authenticated end users, and should be expired similar to HTTP cookies.

If the asset service supports direct logins, it may respond to unauthorized requests with a response that includes information about authenticating with OpenID.

Once an authenticated request for an asset has been made, the asset service determines if the authenticated user has authorization for the requested content. If a request is unauthorized, a 403 Forbidden response must be sent back to the end user, optionally including information about authenticating with OpenID so the user can choose to authenticate with another identity.

==Lookup==

Once a request has been authorized, a lookup is performed. This could be as simple as a map<UUID,metadata> or a more scalable solution such as a distributed hash table. The lookup resolves the asset metadata and asset location, and together with the authorization can determine the list of supported actions that are returned in metadata.

==Metadata==

The metadata provides information on supported methods for the resource and information about the resource itself (such as the size and dimensions of a texture, or byte boundaries of quality layers in a JPEG2000 file). One of the goals of this research is to work with the community to develop a common metadata format for requests and responses. The current metadata reference implementation uses JSON for the wire protocol. See [[AssetServerProposal/ClientDocs]] for more information.

The metadata is broken up into three parts. Required fields, methods, and extra fields.

The required fields are basic attributes that all assets share. These are:

* id - UUID of the asset
* name - Name of the asset
* description - Description of the asset
* creation_date - Timestamp when the asset was created
* type - HTTP Content-Type of the asset
* sha1 - The SHA-1 hash of the asset data
* temporary - Boolean, whether the asset is temporary or not

The methods section is a collection of allowed methods for the asset. The only required method is data, however this method may not be present in the list of returned methods if the client has permission to access an assets metadata but not the data. Each method is a string (such as "data") that maps to a URL. The URL may exist on the same server as the metadata, but this is not required.

The extra data section is a free-form version of the required fields section. Any fields can be defined, and is most commonly used to add extended data for specific types of assets.

==Data==

Once the metadata has been retrieved, if a data method exists the asset data can be fetched with a simple HTTP GET on the asset data URL. This allows the actual data to optionally be stored on a simple web server, a content distribution network, a third party storage provider, etc.

= See Also =

* [[AssetServerProposal/ClientDocs|Client Documentation]]
* [[AssetServerProposal/DeveloperDocs|Developer Documentation]]

Build Instructions

2008-11-11T03:11:23Z

Mikem: /* RedHat Enterprise Linux 5 */

[[Category:Users]]
This page covers building OpenSim from source code on multiple platforms. Please help us keep this page up to date as the project progresses.

==Download from SVN==
Check out the [[Download]] Section

==MS Windows==

OpenSim requires either the .Net framework version 2.0, or the latest Mono. It supports the following compilers:
* [http://msdn2.microsoft.com/en-us/express/aa700756.aspx Microsoft Visual C# Express Edition] (note: not Visual C++)
* [http://www.mono-project.com/ mono]

Note for people who just downloaded the sources from http://dist.opensimulator.org/ (the "Downloads" link on the left) be advised that some important things are missing (like MySQL template scripts). For such features, you must download using svn!

Additional note: any Microsoft C# Express edition should work (2005 or 2008)

=== Building ===

* In the top-level directory, run the '<tt>runprebuild.bat</tt>' file. This will create a VS2005 solution file, a nant build file and a '<tt>compile.bat</tt>' file.
* If you prefer VS2008, run the '<tt>runprebuild2008.bat</tt>' instead.

* Open the resulting sln file with visual studio and build it there, or
* Run the '<tt>compile.bat</tt>' file. This will build the executable using MSBuild.
* if you prefer to use nant, run nant in the same top-level directory. This will build the executables.

If you don't care about physics (walking on prims, etc), ignore the rest of this section.

=== Running ===

Recent versions of OpenSim come without an <tt>OpenSim.ini</tt> file. Copy the <tt>OpenSim.ini.example</tt> file to <tt>OpenSim.ini</tt> before making any changes.

Double-click on the <tt>OpenSim.exe</tt> executable file in the <tt>bin</tt> directory. This will start up OpenSim in standalone mode.

The debugger in VS2005 C# may be used to step through the code. For those that use a Cygwin shell, you may find that one or more dll's have permissions that cause problems running. Most find that a "<tt>chmod 777 *</tt>" from the <tt>bin</tt> directory solves this.

Physics can be invoked by adding the appropriate line to the [Startup] section of <tt>OpenSim.ini</tt>. For ODE, that would be:

physics = OpenDynamicsEngine

You can also add a command line option to a shortcut, or run from a command prompt with:

-physics=OpenDynamicsEngine

'''''Windows Vista'''''

To run on Windows Vista, you must first disable Windows Firewall. Under the new "Start" button of Vista, select "Control panel". Then double-click "Windows Firewall". In the window that pops up, on the left column, select "Turn Windows Firewall on or off". You will have to give permission for this to run, then select the option "Off (not recommended)". Click "OK" and exit from the Windows Firewall window.

If you have McAfee SecurityCenter, see the description below.

Once all the security features are disabled, right click on <tt>OpenSim.exe</tt> and select "Run as administrator". This will pop up a window asking permission, select "Allow". Your OpenSim server should run in a DOS-like window and accept connections.

'''''McAfee Security'''''

McAfee Security does not allow applications to listen on ports not explicitly specified. You have two options: 1) disable firewall protection all together, 2) enable <tt>OpenSim.exe</tt> to be able to open ports.

''Disable firewall''

Open McAfee SecurityCenter. Select "Internet & Network". In the lower left corner is a small link to "Configure...". Select this. In the right side of the window, select the bar that says "Firewall protection is enabled". Here you can select "Off".

''Enable <tt>OpenSim.exe</tt> to open ports''

Open McAfee SecurityCenter. Select "Internet & Network". In the lower left corner is a small link to "Configure...". Select this. In the right side of the window, select the bar that says "Firewall protection is enabled". Select the "Advanced..." button. This will pop up a new window.

In the new window, on the left side, select "Program Permissions." In the middle on the right side of the window, select the "Add Allowed Program" button. Use the browser that pops up to find the OpenSim executable and select it.

Finally, select "OK" and exit the McAfee SecurityCenter window.

==Linux/Mac OS X/FreeBSD==

The easiest plaform to get running on the Linux side is Ubuntu 8.04, 32bit. This is what most of the developers running Linux use. If you are looking for the quick path, start there.

=== Ubuntu 8.04 / 8.10 ===

For Ubuntu 7.10 users '''you need''' to upgrade your mono to 1.9.1.

You can use the built in packages for mono. However, for better performance, you may want to [http://xyzzyxyzzy.net/2008/05/08/updated-mono-build-script-for-hardy-heron-and-mono-191/ upgrade mono to 1.9.1] ([http://tempvariable.blogspot.com/2008/04/installing-mono-191-on-ubuntu-804-hardy.html Other simple method])

sudo apt-get install subversion nant mono-gmcs libmono-microsoft8.0-cil \
libmono-system-runtime2.0-cil libgdiplus libmono-i18n2.0-cil ruby
svn co http://opensimulator.org/svn/opensim/trunk opensim
cd opensim
./runprebuild.sh
nant

=== openSUSE 10.3 and 11 ===

Install an openSUSE 11 or 10.3 with its default options, add the online repositories
when finished installing do an online update with all the latest packages.

In yast install these packages, for running Opensim in standalone mode.
(there is a slight diffrence between 10.3 and 11 but following should be same)
subversion
nant
mono-jscript
- check that mono-core is installed

If you just want to use SQLite then jump to last section
within this post.

* Optional mysql - for Opensim running in Grid mode:
Install these mysql packages via yast
mysql
mysql-client
mysql-administrator
mysql-gui-tools
mysql-query-browser

Before building create the mysql database.
/etc/init.d/mysql start
mysql -u root -p -h localhost
(when asked for password just hit enter)

mysql> create database opensim;
mysql> quit

set the configuration in bin/mysql_connection.ini
Or on later builds set the connection string inside bin/OpenSim.ini

Build after installation of above in bash terminal. i save it in /opt

su -
cd /opt
svn co http://opensimulator.org/svn/opensim/trunk opensim
cd opensim
./runprebuild.sh
nant

After this you should be able to continue on starting the diffrent Servers, look in the mysql-config section,or
just run your OpenSim as a Standalone. By - eagleFX

=== Mac OS X 10.5/10.4 ===
* OpenSim is now working on PowerPC Macs! Thanks to DrScofield and those who helped him. Current nightly builds for PowerPC are not working, not sure about Intel so use the 0.5 Build. OpenSim works on Intel Macs. I'm testing on PowerBook G4. Tested these step on 10.5, but not 10.4 but should work --[[User:Mokele|Mokele]] 22:36, 14 February 2008 (PST)
* Install XCode Developers Tools from DVD/CD Installation Disk or download from http://developer.apple.com/. You have to create an Apple account to access the downloads if you don't have an Apple account.
* Install X11 for 10.4 from the Optional Install from the DVD/CD Installation Disk. X11 for 10.5 is installed by default.
* Install Mono 1.2.5 from http://ftp.novell.com/pub/mono/archive/1.2.5/macos-10-universal/5/MonoFramework-1.2.5_5.macos10.novell.universal.dmg and in Terminal or X11 edit the .profile file and add the following line:
export PKG_CONFIG_PATH="/Library/Frameworks/Mono.framework/Versions/Current/lib/pkgconfig/:${PKG_CONFIG_PATH}"
* Compile OpenSim
svn co http://opensimulator.org/svn/opensim/tags/0.5.0-release opensim
cd opensim
./runprebuild.sh
nant

* Download and Compile libopenjpeg-libsl-2.1.2.0.dylib and libsecondlife.dll
* libopenjpeg-libsl-2.1.2.0.dylib:
svn co http://opensimulator.org/svn/opensim-libs/libsl1550 opensim-libs
cd opensim-libs/openjpeg-libsl
make -f Makefile.osx
cp libopenjpeg-libsl-2.1.2.0.dylib ../../opensim/bin
* Note: The Makefile that creates the libopenjpeg-libsl-2.1.2.0.so does not compile on PowerPC, but works properly on Intel Macs. Looks like a gcc issue with compile options.

* libsecondlife.dll: (for PowerPC Only, see details on this step [http://xyzzyxyzzy.net/2008/02/12/installing-opensim-on-powerpcor-of-eggs-and-virtual-worlds installing OpenSim on PowerPC…or: of eggs and virtual worlds])
cd .. (back into opensim-libs)
nant
cp bin/libsecondlife.dll ../opensim/bin

* Edit the libsecondlife.dll.config (PowerPC Only). Remove the cpu="x86" tag in the last dllmap line.

=== FreeBSD 6.2 ===
su
cd /usr/ports/devel/subversion/ && make install clean (you may also need to rebuild apr-svn if this step fails)
cd /usr/ports/lang/mono/ && make install clean
cd /usr/ports/devel/nant/ && make install clean
cd /usr/ports/databases/sqlite3/ && make install clean
cd /usr/ports/x11-toolkits/libgdiplus/ && make install clean
cd /opensim/installation/directory/
svn co http://opensimulator.org/svn/opensim/trunk opensim
cd opensim
./runprebuild.sh
nant

Note: [http://opensimulator.org/wiki/OpenSim:FAQ#System.DllNotFoundException:_..2Flibopenjpeg-libsl-2.1.2.0.so|Follow the instructions on the FAQ to fix the]
"System.DllNotFoundException: ./libopenjpeg-libsl-2.1.2.0.so" issue, but use "gmake" instead of "make"

For ODE Physics you must do the following:
cd /usr/ports/graphics/libGL/ && make install clean
cd /usr/ports/graphics/libGLU/ && make install clean
cd /opensim/installation/directory/
svn co http://opensimulator.org/svn/opensim-libs/trunk opensim-libs
cd opensim-libs/unmanaged/OpenDynamicsEngine2/
sh autogen.sh
./configure --enable-shared --enable-release --disable-demos
make
mv ./ode/src/libode.so /opensim/installation/directory/opensim/bin/

=== RedHat Enterprise Linux 4 ===
sudo vi /etc/yum.repos.d/mono.repo

[mono]
name=Mono for rhel-4-i386 (stable)
baseurl=http://ftp.novell.com/pub/mono/download-stable/rhel-4-i386/
enabled=1
gpgcheck=0

sudo yum install mono-complete monodoc-core nant
svn co svn co http://opensimulator.org/svn/opensim/trunk opensim
cd opensim
./runprebuild.sh
nant

=== RedHat Enterprise Linux 5 ===

The instructions below also work on other RedHat Linux flavors such as CentOS or maybe Fedora.

1. Put the [http://download.opensuse.org/repositories/Mono/RHEL_5/Mono.repo Mono.repo] file in the /etc/yum.repo.d/ directory:
$ sudo su -
$ cd /etc/yum.repos.d/
$ wget http://download.opensuse.org/repositories/Mono/RHEL_5/Mono.repo
Naturally use the most [http://download.opensuse.org/repositories/Mono up-to-date link for your distribution].

2. Install Mono and related tools with yum:
$ yum install mono nant mono-jscript mono-nunit
Make sure to use nunit-console2 to run your tests.

=== Fedora 5 ===
* I needed to build latest mono and nant from sources to build OpenSim successfully, the ones available in yum repository didn't work so I had to uninstall and build and configure the packages.

For detailed instructions go [http://ruakuu.blogspot.com/2008/06/installing-and-configuring-opensim-on.html here]

=== Debian 4 ===

The following packages and their dependencies are required to run OpenSim on a default Debian 4 netinstall:
* mono
* libmono-corlib2.0-cil
* libmono-sqlite2.0-cil
* libmono-system-web2.0-cil
* libmono-microsoft8.0-cil
* libmono-system-runtime2.0-cil

=== 64bit ===
Please note that only 32bit binaries are provided in the bin/ directory of subversion. If you want to use 64bit, you'll need to rebuild these shared objects. See [[Installing and running on x86-64]]

=== Physics (Open Dynamics Engine ODE) ===
As installed from svn, ODE will work on most 32 bit platforms. If you get an ODE-related crash, and/or a libode.so not found type of error, you will need to build libode from source.

Remove <tt>libode.so</tt> from the <tt>./bin</tt> folder. (Note that subsequent svn updates may replace it again; best fix is to copy your built <tt>libode.so</tt> to <tt>bin</tt>). Do NOT remove <tt>ode.net.dll</tt>! Download the latest source from:

svn co http://opensimulator.org/svn/opensim-libs/trunk/unmanaged/OpenDynamicsEngine

OpenSim requires a couple of patches on top of ODE which are not yet included upstream. When compiling, make sure to use the following configure options:

--with-trimesh=gimpact
--enable-shared

Make sure the configure script confirms these choices, and always compile with single precision (I believe that's the default). Try <code> make -k </code> if you get errors relating to drawstuff, test*, or openGL. <code> make install </code> should put <tt>libode.so</tt> in the proper place (usually <tt>/usr/local/lib</tt>), and it should be seen by opensim (<tt>ode.net.dll</tt>)

===Running===
Recent versions of OpenSim come without an <tt>OpenSim.ini</tt> file. Copy the <tt>OpenSim.ini.example</tt> file to <tt>OpenSim.ini</tt> before making any changes.
<code>
cd bin
mono OpenSim.exe
</code>

Note: if you are running a 32bit Server such as Ubuntu 8.0.4 you need the alternative launcher:

<code>
mono OpenSim.32BitLaunch.exe
</code>

* To invoke ODE, add the option:
-physics=OpenDynamicsEngine
to the <tt>mono OpenSim.exe</tt> line

or add <code> physics = OpenDynamicsEngine </code> to the [Startup] section of <tt>OpenSim.ini</tt>. Same deal for other physics engines, when available.

On mono 1.2.6, some distributions may see
Unhandled Exception: System.NotSupportedException: CodePage 1252 not supported
on startup when using mysql. This can be resolved by installing the package libmono-i18n2.0-cil (see http://bugs.mysql.com/bug.php?id=33938).

=== Additional Items ===

* [[GC_NO_EXPLICIT|GC NO EXPLICIT]] - Enable Large Heap in Mono, this has been known to help performance and stability

[[Category:Users]]

OpenSim 0.6 IClientAPI

2008-11-10T01:28:41Z

Mikem: /* OnNewClient vs OnClientConnect */

{{content}}
This page is under construction.
=Porting Guide=
==OnNewClient vs OnClientConnect==
* OnNewClient is deprecated, please use OnClientConnect instead.
** Rationale: OnNewClient passes IClientAPI in it's parameter, IClientAPI is being deprecated.
** IClientAPI Changes: IClientCore may not contain all the interfaces you want just yet, but if it does (see the list below for what's been ported so far), you should switch to OnClientConnect and then use IClientCore.[Try]Get<IClientInterface>().Method() instead of IClientAPI.Method();

==Instant Message==
* Dropping fromAgentSession from SendInstantMessage (both overloaded versions)
** Rationale: Session ID is both linden specific, and something that shouldnt be transmitted to users anyway ('''security''').
** IClientAPI changes: Insert this parameter into your packets manually if you need it, LLClientView.cs contains a reference to sessionID already. Be advised that this represents a security risk (giving the users one of the two shared secrets).
* Dropping imSessionID from SendInstantMessage (both overloaded versions)
** Rationale: IM Session ID's are unique to SL and we just multiplex the two users ID's to form this so it's static anyway.
** IClientAPI changes: Multiplex the ID inside your own implementations vs relying on the module to do it for you.
==Bluebox==
* Dropping sessionID from SendBlueBoxMessage
** Rationale: Session ID is both linden specific, and something that shouldnt be transmitted to users anyway ('''security''').
** IClientAPI changes: Insert this parameter into your packets manually if you need it, LLClientView.cs contains a reference to sessionID already. Be advised that this represents a security risk (giving the users one of the two shared secrets).
==Chat==
* Dropping SendChatMessage(bytes[]...) - the overload with string still works however.
** Rationale: We arent using this, and sending the bytes raw instead of a string is poor methodology.
** IClientAPI Changes: Drop this method if you arent using it.

=Example Conversion=
==InstantMessageModule.cs==
===Old===
private void OnNewClient(IClientAPI client)
{
client.OnInstantMessage += OnInstantMessage;
}

===New===
void OnClientConnect(IClientCore client)
{
IClientIM clientIM;
if(client.TryGet(out clientIM))
{
clientIM.OnInstantMessage += OnInstantMessage;
}
}
===Comments===
Isnt this messier? I see more code! What's this IClientIM thing?

The short answer here is we're now using multiple interfaces to represent clients. Rather than dictating that every client must obey IClientAPI.cs - we can say clients can pick and choose the bits and pieces they want to implement.

The extra code is mostly in checking that the client supports Instant Messaging before attempting to handle IM's for it. If the client doesnt support that interface, it will lead to a crash if you try use it.

OpenSim 0.6 IClientAPI

2008-11-10T01:28:22Z

Mikem: /* OnNewClient vs OnClientConnect */

{{content}}
This page is under construction.
=Porting Guide=
==OnNewClient vs OnClientConnect==
* OnNewClient is deprecated, please use OnClientConnect instead.
** Rationale: OnNewClient passes IClientAPI in it's parameter, IClientAPI is being depreciated.
** IClientAPI Changes: IClientCore may not contain all the interfaces you want just yet, but if it does (see the list below for what's been ported so far), you should switch to OnClientConnect and then use IClientCore.[Try]Get<IClientInterface>().Method() instead of IClientAPI.Method();

==Instant Message==
* Dropping fromAgentSession from SendInstantMessage (both overloaded versions)
** Rationale: Session ID is both linden specific, and something that shouldnt be transmitted to users anyway ('''security''').
** IClientAPI changes: Insert this parameter into your packets manually if you need it, LLClientView.cs contains a reference to sessionID already. Be advised that this represents a security risk (giving the users one of the two shared secrets).
* Dropping imSessionID from SendInstantMessage (both overloaded versions)
** Rationale: IM Session ID's are unique to SL and we just multiplex the two users ID's to form this so it's static anyway.
** IClientAPI changes: Multiplex the ID inside your own implementations vs relying on the module to do it for you.
==Bluebox==
* Dropping sessionID from SendBlueBoxMessage
** Rationale: Session ID is both linden specific, and something that shouldnt be transmitted to users anyway ('''security''').
** IClientAPI changes: Insert this parameter into your packets manually if you need it, LLClientView.cs contains a reference to sessionID already. Be advised that this represents a security risk (giving the users one of the two shared secrets).
==Chat==
* Dropping SendChatMessage(bytes[]...) - the overload with string still works however.
** Rationale: We arent using this, and sending the bytes raw instead of a string is poor methodology.
** IClientAPI Changes: Drop this method if you arent using it.

=Example Conversion=
==InstantMessageModule.cs==
===Old===
private void OnNewClient(IClientAPI client)
{
client.OnInstantMessage += OnInstantMessage;
}

===New===
void OnClientConnect(IClientCore client)
{
IClientIM clientIM;
if(client.TryGet(out clientIM))
{
clientIM.OnInstantMessage += OnInstantMessage;
}
}
===Comments===
Isnt this messier? I see more code! What's this IClientIM thing?

The short answer here is we're now using multiple interfaces to represent clients. Rather than dictating that every client must obey IClientAPI.cs - we can say clients can pick and choose the bits and pieces they want to implement.

The extra code is mostly in checking that the client supports Instant Messaging before attempting to handle IM's for it. If the client doesnt support that interface, it will lead to a crash if you try use it.

Build Instructions

2008-11-04T22:52:27Z

Mikem:

[[Category:Users]]
This page covers building OpenSim from source code on multiple platforms. Please help us keep this page up to date as the project progresses.

==Download from SVN==
Check out the [[Download]] Section

==MS Windows==

OpenSim requires either the .Net framework version 2.0, or the latest Mono. It supports the following compilers:
* [http://msdn2.microsoft.com/en-us/express/aa700756.aspx Microsoft Visual C# Express Edition] (note: not Visual C++)
* [http://www.mono-project.com/ mono]

Note for people who just downloaded the sources from http://dist.opensimulator.org/ (the "Downloads" link on the left) be advised that some important things are missing (like MySQL template scripts). For such features, you must download using svn!

Additional note: any Microsoft C# Express edition should work (2005 or 2008)

=== Building ===

* In the top-level directory, run the '<tt>runprebuild.bat</tt>' file. This will create a VS2005 solution file, a nant build file and a '<tt>compile.bat</tt>' file.
* If you prefer VS2008, run the '<tt>runprebuild2008.bat</tt>' instead.

* Open the resulting sln file with visual studio and build it there, or
* Run the '<tt>compile.bat</tt>' file. This will build the executable using MSBuild.
* if you prefer to use nant, run nant in the same top-level directory. This will build the executables.

If you don't care about physics (walking on prims, etc), ignore the rest of this section.

=== Running ===

Recent versions of OpenSim come without an <tt>OpenSim.ini</tt> file. Copy the <tt>OpenSim.ini.example</tt> file to <tt>OpenSim.ini</tt> before making any changes.

Double-click on the <tt>OpenSim.exe</tt> executable file in the <tt>bin</tt> directory. This will start up OpenSim in standalone mode.

The debugger in VS2005 C# may be used to step through the code. For those that use a Cygwin shell, you may find that one or more dll's have permissions that cause problems running. Most find that a "<tt>chmod 777 *</tt>" from the <tt>bin</tt> directory solves this.

Physics can be invoked by adding the appropriate line to the [Startup] section of <tt>OpenSim.ini</tt>. For ODE, that would be:

physics = OpenDynamicsEngine

You can also add a command line option to a shortcut, or run from a command prompt with:

-physics=OpenDynamicsEngine

'''''Windows Vista'''''

To run on Windows Vista, you must first disable Windows Firewall. Under the new "Start" button of Vista, select "Control panel". Then double-click "Windows Firewall". In the window that pops up, on the left column, select "Turn Windows Firewall on or off". You will have to give permission for this to run, then select the option "Off (not recommended)". Click "OK" and exit from the Windows Firewall window.

If you have McAfee SecurityCenter, see the description below.

Once all the security features are disabled, right click on <tt>OpenSim.exe</tt> and select "Run as administrator". This will pop up a window asking permission, select "Allow". Your OpenSim server should run in a DOS-like window and accept connections.

'''''McAfee Security'''''

McAfee Security does not allow applications to listen on ports not explicitly specified. You have two options: 1) disable firewall protection all together, 2) enable <tt>OpenSim.exe</tt> to be able to open ports.

''Disable firewall''

Open McAfee SecurityCenter. Select "Internet & Network". In the lower left corner is a small link to "Configure...". Select this. In the right side of the window, select the bar that says "Firewall protection is enabled". Here you can select "Off".

''Enable <tt>OpenSim.exe</tt> to open ports''

Open McAfee SecurityCenter. Select "Internet & Network". In the lower left corner is a small link to "Configure...". Select this. In the right side of the window, select the bar that says "Firewall protection is enabled". Select the "Advanced..." button. This will pop up a new window.

In the new window, on the left side, select "Program Permissions." In the middle on the right side of the window, select the "Add Allowed Program" button. Use the browser that pops up to find the OpenSim executable and select it.

Finally, select "OK" and exit the McAfee SecurityCenter window.

==Linux/Mac OS X/FreeBSD==

The easiest plaform to get running on the Linux side is Ubuntu 8.04, 32bit. This is what most of the developers running Linux use. If you are looking for the quick path, start there.

=== Ubuntu 8.04 / 8.10 ===

For Ubuntu 7.10 users '''you need''' to upgrade your mono to 1.9.1.

You can use the built in packages for mono. However, for better performance, you may want to [http://xyzzyxyzzy.net/2008/05/08/updated-mono-build-script-for-hardy-heron-and-mono-191/ upgrade mono to 1.9.1] ([http://tempvariable.blogspot.com/2008/04/installing-mono-191-on-ubuntu-804-hardy.html Other simple method])

sudo apt-get install subversion nant mono-gmcs libmono-microsoft8.0-cil \
libmono-system-runtime2.0-cil libgdiplus libmono-i18n2.0-cil ruby
svn co http://opensimulator.org/svn/opensim/trunk opensim
cd opensim
./runprebuild.sh
nant

=== openSUSE 10.3 and 11 ===

Install an openSUSE 11 or 10.3 with its default options, add the online repositories
when finished installing do an online update with all the latest packages.

In yast install these packages, for running Opensim in standalone mode.
(there is a slight diffrence between 10.3 and 11 but following should be same)
subversion
nant
mono-jscript
- check that mono-core is installed

If you just want to use SQLite then jump to last section
within this post.

* Optional mysql - for Opensim running in Grid mode:
Install these mysql packages via yast
mysql
mysql-client
mysql-administrator
mysql-gui-tools
mysql-query-browser

Before building create the mysql database.
/etc/init.d/mysql start
mysql -u root -p -h localhost
(when asked for password just hit enter)

mysql> create database opensim;
mysql> quit

set the configuration in bin/mysql_connection.ini
Or on later builds set the connection string inside bin/OpenSim.ini

Build after installation of above in bash terminal. i save it in /opt

su -
cd /opt
svn co http://opensimulator.org/svn/opensim/trunk opensim
cd opensim
./runprebuild.sh
nant

After this you should be able to continue on starting the diffrent Servers, look in the mysql-config section,or
just run your OpenSim as a Standalone. By - eagleFX

=== Mac OS X 10.5/10.4 ===
* OpenSim is now working on PowerPC Macs! Thanks to DrScofield and those who helped him. Current nightly builds for PowerPC are not working, not sure about Intel so use the 0.5 Build. OpenSim works on Intel Macs. I'm testing on PowerBook G4. Tested these step on 10.5, but not 10.4 but should work --[[User:Mokele|Mokele]] 22:36, 14 February 2008 (PST)
* Install XCode Developers Tools from DVD/CD Installation Disk or download from http://developer.apple.com/. You have to create an Apple account to access the downloads if you don't have an Apple account.
* Install X11 for 10.4 from the Optional Install from the DVD/CD Installation Disk. X11 for 10.5 is installed by default.
* Install Mono 1.2.5 from http://ftp.novell.com/pub/mono/archive/1.2.5/macos-10-universal/5/MonoFramework-1.2.5_5.macos10.novell.universal.dmg and in Terminal or X11 edit the .profile file and add the following line:
export PKG_CONFIG_PATH="/Library/Frameworks/Mono.framework/Versions/Current/lib/pkgconfig/:${PKG_CONFIG_PATH}"
* Compile OpenSim
svn co http://opensimulator.org/svn/opensim/tags/0.5.0-release opensim
cd opensim
./runprebuild.sh
nant

* Download and Compile libopenjpeg-libsl-2.1.2.0.dylib and libsecondlife.dll
* libopenjpeg-libsl-2.1.2.0.dylib:
svn co http://opensimulator.org/svn/opensim-libs/libsl1550 opensim-libs
cd opensim-libs/openjpeg-libsl
make -f Makefile.osx
cp libopenjpeg-libsl-2.1.2.0.dylib ../../opensim/bin
* Note: The Makefile that creates the libopenjpeg-libsl-2.1.2.0.so does not compile on PowerPC, but works properly on Intel Macs. Looks like a gcc issue with compile options.

* libsecondlife.dll: (for PowerPC Only, see details on this step [http://xyzzyxyzzy.net/2008/02/12/installing-opensim-on-powerpcor-of-eggs-and-virtual-worlds installing OpenSim on PowerPC…or: of eggs and virtual worlds])
cd .. (back into opensim-libs)
nant
cp bin/libsecondlife.dll ../opensim/bin

* Edit the libsecondlife.dll.config (PowerPC Only). Remove the cpu="x86" tag in the last dllmap line.

=== FreeBSD 6.2 ===
su
cd /usr/ports/devel/subversion/ && make install clean (you may also need to rebuild apr-svn if this step fails)
cd /usr/ports/lang/mono/ && make install clean
cd /usr/ports/devel/nant/ && make install clean
cd /usr/ports/databases/sqlite3/ && make install clean
cd /usr/ports/x11-toolkits/libgdiplus/ && make install clean
cd /opensim/installation/directory/
svn co http://opensimulator.org/svn/opensim/trunk opensim
cd opensim
./runprebuild.sh
nant

Note: [http://opensimulator.org/wiki/OpenSim:FAQ#System.DllNotFoundException:_..2Flibopenjpeg-libsl-2.1.2.0.so|Follow the instructions on the FAQ to fix the]
"System.DllNotFoundException: ./libopenjpeg-libsl-2.1.2.0.so" issue, but use "gmake" instead of "make"

For ODE Physics you must do the following:
cd /usr/ports/graphics/libGL/ && make install clean
cd /usr/ports/graphics/libGLU/ && make install clean
cd /opensim/installation/directory/
svn co http://opensimulator.org/svn/opensim-libs/trunk opensim-libs
cd opensim-libs/unmanaged/OpenDynamicsEngine2/
sh autogen.sh
./configure --enable-shared --enable-release --disable-demos
make
mv ./ode/src/libode.so /opensim/installation/directory/opensim/bin/

=== RedHat Enterprise Linux 4 ===
sudo vi /etc/yum.repos.d/mono.repo

[mono]
name=Mono for rhel-4-i386 (stable)
baseurl=http://ftp.novell.com/pub/mono/download-stable/rhel-4-i386/
enabled=1
gpgcheck=0

sudo yum install mono-complete monodoc-core nant
svn co svn co http://opensimulator.org/svn/opensim/trunk opensim
cd opensim
./runprebuild.sh
nant

=== RedHat Enterprise Linux 5 ===

The instructions below also work on other RedHat Linux flavors such as CentOS or maybe Fedora.

1. Put the [http://download.opensuse.org/repositories/Mono/RHEL_5/Mono.repo Mono.repo] file in the /etc/yum.repo.d/ directory:
$ sudo su -
$ cd /etc/yum.repo.d/
$ wget http://download.opensuse.org/repositories/Mono/RHEL_5/Mono.repo
Naturally use the most [http://download.opensuse.org/repositories/Mono up-to-date link for your distribution].

2. Install Mono and related tools with yum:
$ yum install mono nant mono-jscript mono-nunit
Make sure to use nunit-console2 to run your tests.

=== Fedora 5 ===
* I needed to build latest mono and nant from sources to build OpenSim successfully, the ones available in yum repository didn't work so I had to uninstall and build and configure the packages.

For detailed instructions go [http://ruakuu.blogspot.com/2008/06/installing-and-configuring-opensim-on.html here]

=== Debian 4 ===

The following packages and their dependencies are required to run OpenSim on a default Debian 4 netinstall:
* mono
* libmono-corlib2.0-cil
* libmono-sqlite2.0-cil
* libmono-system-web2.0-cil
* libmono-microsoft8.0-cil
* libmono-system-runtime2.0-cil

=== 64bit ===
Please note that only 32bit binaries are provided in the bin/ directory of subversion. If you want to use 64bit, you'll need to rebuild these shared objects. See [[Installing and running on x86-64]]

=== Physics (Open Dynamics Engine ODE) ===
As installed from svn, ODE will work on most 32 bit platforms. If you get an ODE-related crash, and/or a libode.so not found type of error, you will need to build libode from source.

Remove <tt>libode.so</tt> from the <tt>./bin</tt> folder. (Note that subsequent svn updates may replace it again; best fix is to copy your built <tt>libode.so</tt> to <tt>bin</tt>). Do NOT remove <tt>ode.net.dll</tt>! Download the latest source from:

svn co http://opensimulator.org/svn/opensim-libs/trunk/unmanaged/OpenDynamicsEngine2

OpenSim requires a couple of patches on top of ODE which are not yet included upstream. When compiling, make sure to use the following configure options:

--with-trimesh=gimpact
--enable-shared

Make sure the configure script confirms these choices, and always compile with single precision (I believe that's the default). Try <code> make -k </code> if you get errors relating to drawstuff, test*, or openGL. <code> make install </code> should put <tt>libode.so</tt> in the proper place (usually <tt>/usr/local/lib</tt>), and it should be seen by opensim (<tt>ode.net.dll</tt>)

===Running===
Recent versions of OpenSim come without an <tt>OpenSim.ini</tt> file. Copy the <tt>OpenSim.ini.example</tt> file to <tt>OpenSim.ini</tt> before making any changes.
cd bin
mono OpenSim.exe

* To invoke ODE, add the option:
-physics=OpenDynamicsEngine
to the <tt>mono OpenSim.exe</tt> line

or add <code> physics = OpenDynamicsEngine </code> to the [Startup] section of <tt>OpenSim.ini</tt>. Same deal for other physics engines, when available.

On mono 1.2.6, some distributions may see
Unhandled Exception: System.NotSupportedException: CodePage 1252 not supported
on startup when using mysql. This can be resolved by installing the package libmono-i18n2.0-cil (see http://bugs.mysql.com/bug.php?id=33938).

=== Additional Items ===

* [[GC_NO_EXPLICIT|GC NO EXPLICIT]] - Enable Large Heap in Mono, this has been known to help performance and stability

[[Category:Users]]

LLUDP Dissector

2008-10-14T02:40:18Z

Mikem:

== LLUDP protocol dissector ==
On this page you will find the Lua code for a wireshark protocol dissector that can parse the message_template.msg file and use that information to decode all the message fields from the Linden UDP protocol.

== Installing ==
* Requires wireshark with Lua 5.1.x support. [http://wiki.wireshark.org/Lua See this page for getting wireshark to support lua]

=== on Linux ===
* Copy all four source files into ~/.wireshark
* edit ''/etc/wireshark/init.lua'' (or equivalent on your system) and change ''disable_lua'' to ''false'' (default is true)
* If you need to run wireshark as the root user or using sudo then you will need to edit the scripts into one file by replacing the dofile("script.lua") calls with the contents of file between the quotes.
* The other method is to add your user account to the correct group (on Gentoo it is group "wireshark") that will allow your non-root user to capture packets.

=== on Windows ===
* Copy all four source files into your user profiles directory

'''Vista*'''

C:\Users\<username>\AppData\Roaming\Wireshark

'''XP/2000'''

C:\Documents and Settings\<username>\Application Data\Wireshark

* Edit C:\Program Files\Wireshark\init.lua and change ''disable_lua'' to ''false'' (default is true)

* *Note: I have only tested this on Windows XP

== LLUDP preferences ==
There are three preferences that can be changed from wiresharks "Preferences" dialog:
* Message template file: Full path to the message_template.msg file used to decode message name & details from the packets. On windows XP/Vista use double backslash '\\' instead of single blackslash '\' to separate directories (Example "C:\\Program Files\\SecondLife\\app_settings\\message_template.msg").
* UDP port range start: First UDP port to mark as LLUDP packets. (default 13000)
* UDP port range end: Last UDP port to mark as LLUDP packets. (default 13050)

If your OpenSim regions are using ports 9000-9050 range then change the UDP port range.

== Description of source files ==
* "init.lua" -- simple script that loads the "lludp.lua" script.
* "lludp.lua" -- contains the code that decodes each packet header and decompresses zero-encoded packets. This file uses wireshark only functions for accessing packet bytes and building a tree of information from each packet.
* "llmessage.lua" -- contains the message_template.msg file parser the decodes the tokens from the lexer into an tree of tables containing all details about each message/block/variable from the template file. This file only has pure lua code.
* "lexer.lua" -- contains the template file lexer. This lexer knows how to tokenize the template file into the follow tokens: IDENTIFIER, NUMBER, COMMENT, EOL. The stream of tokens produced by this lexer is parsed by the "llmessage.lua" file. This file only has pure lua code.

=== File "init.lua" ===
<source lang="lua">

-- register http to handle tcp ports 8000-8010 and 9000
do
local tcp_port_table = DissectorTable.get("tcp.port")
local http_dissector = tcp_port_table:get_dissector(80)
for port = 8000,8010 do
tcp_port_table:add(port,http_dissector)
end
tcp_port_table:add(9000,http_dissector)
end

-- Load lludp protocol dissector.
dofile("lludp.lua")

</source>

=== File "lludp.lua" ===
<source lang="lua">
--[[
BSD-Licensed:
Copyright (c) 2008, Robert G. Jakabosky <bobby@sharedrealm.com>
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
* Neither the name of the SharedRealm nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

]]

dofile("llmessage.lua")

-- cache globals to local for speed.
local str_format=string.format

-- test if ByteArray only has printable ASCII character and ends with '\0'
local allowed_special = {
[0] = true, -- null
[9] = true, -- tab
[10] = true, -- new line
[13] = true, -- carriage return
}
local function is_string(bytes)
local c
local max = bytes:len() - 1
for i=0,max do
c = bytes:get_index(i)
if c >= 127 then -- not ascii character
return false
elseif c < 32 then -- control character range
if not allowed_special[c] then
-- control characters between NULL and Space
return false
elseif c == 0 and i < max then
-- null byte only allowed at end.
return false
end
end
end
return true
end

-- lludp protocol example
-- declare our protocol
local lludp_proto = Proto("lludp","LLUDP","LindenLabs UDP Protocol")

-- setup preferences
lludp_proto.prefs["template_file"] =
Pref.string("Message template file", "message_template.msg", "Message template file")
lludp_proto.prefs["udp_port_start"] =
Pref.string("UDP port range start", "13000", "First UDP port to decode as this protocol")
lludp_proto.prefs["udp_port_end"] =
Pref.string("UDP port range end", "13050", "Last UDP port to decode as this protocol")
-- current preferences settings.
local current_settings = {
template_file = "",
udp_port_start = -1,
udp_port_end = -1,
}
-- current list of parsed messages.
local message_details = nil

-- setup protocol fields.
lludp_proto.fields = {}
local fds = lludp_proto.fields
fds.flags = ProtoField.new("Flags", "lludp.flags", "FT_UINT8", nil, "BASE_HEX", "0xFF")
fds.flags_zero = ProtoField.new("Zero", "lludp.flags.zero", "FT_UINT8", nil, "BASE_HEX", "0x80")
fds.flags_reliable = ProtoField.new("Reliable", "lludp.flags.rel", "FT_UINT8", nil, "BASE_HEX", "0x40")
fds.flags_resent = ProtoField.new("Resent", "lludp.flags.res", "FT_UINT8", nil, "BASE_HEX", "0x20")
fds.flags_ack = ProtoField.new("Ack", "lludp.flags.ack", "FT_UINT8", nil, "BASE_HEX", "0x10")
fds.sequence = ProtoField.new("Sequence", "lludp.sequence", "FT_UINT32", nil, "BASE_DEC")
fds.extra_len = ProtoField.new("Extra length", "lludp.extra_len", "FT_UINT8", nil, "BASE_DEC")
fds.extra_bytes = ProtoField.new("Extra header", "lludp.extra_bytes", "FT_BYTES", nil, "BASE_HEX")
fds.msg_id = ProtoField.new("Message ID", "lludp.msg.id", "FT_UINT32", nil, "BASE_HEX")
fds.msg_name = ProtoField.new("Message name", "lludp.msg.name", "FT_STRINGZ", nil)
fds.msg = ProtoField.new("Message body", "lludp.msg", "FT_BYTES", nil, "BASE_HEX")
fds.acks_count = ProtoField.new("Acks count", "lludp.acks_count", "FT_UINT8", nil, "BASE_DEC")
fds.acks = ProtoField.new("Acks", "lludp.acks", "FT_UINT32", nil, "BASE_DEC")
fds.block_count = ProtoField.new("Block count", "lludp.block_count", "FT_UINT8", nil, "BASE_DEC")
fds.block = ProtoField.new("Block", "lludp.block", "FT_BYTES", nil, "BASE_HEX")
fds.var_fixed = ProtoField.new("Fixed blob", "lludp.var.fixed", "FT_BYTES", nil, "BASE_HEX")
fds.var_variable = ProtoField.new("Variable blob", "lludp.var.variable", "FT_BYTES", nil, "BASE_HEX")
fds.var_string = ProtoField.new("String", "lludp.var.string", "FT_STRINGZ", nil)
fds.var_u8 = ProtoField.new("U8", "lludp.var.u8", "FT_UINT8", nil, "BASE_DEC")
fds.var_u16 = ProtoField.new("U16", "lludp.var.u16", "FT_UINT16", nil, "BASE_DEC")
fds.var_u32 = ProtoField.new("U32", "lludp.var.u32", "FT_UINT32", nil, "BASE_DEC")
fds.var_u64 = ProtoField.new("U64", "lludp.var.u64", "FT_UINT64", nil, "BASE_DEC")
fds.var_s8 = ProtoField.new("S8", "lludp.var.s8", "FT_INT8", nil, "BASE_DEC")
fds.var_s16 = ProtoField.new("S16", "lludp.var.s16", "FT_INT16", nil, "BASE_DEC")
fds.var_s32 = ProtoField.new("S32", "lludp.var.s32", "FT_INT32", nil, "BASE_DEC")
fds.var_s64 = ProtoField.new("S64", "lludp.var.s64", "FT_INT64", nil, "BASE_DEC")
fds.var_f32 = ProtoField.new("F32", "lludp.var.f32", "FT_FLOAT", nil, "BASE_DEC")
fds.var_f64 = ProtoField.new("F64", "lludp.var.f64", "FT_DOUBLE", nil, "BASE_DEC")
fds.var_llvector3 = ProtoField.new("LLVector3", "lludp.var.llvector3", "FT_BYTES", nil, "BASE_HEX")
fds.var_llvector3d = ProtoField.new("LLVector3d", "lludp.var.llvector3d", "FT_BYTES", nil, "BASE_HEX")
fds.var_llvector4 = ProtoField.new("LLVector4", "lludp.var.llvector4", "FT_BYTES", nil, "BASE_HEX")
fds.var_llquaternion = ProtoField.new("LLQuaternion", "lludp.var.llquaternion", "FT_BYTES", nil, "BASE_HEX")
fds.var_lluid = ProtoField.new("LLUID", "lludp.var.lluid", "FT_BYTES", nil, "BASE_HEX")
fds.var_bool = ProtoField.new("BOOL", "lludp.var.bool", "FT_UINT8", nil, "BASE_DEC")
fds.var_ipaddr = ProtoField.new("IPADDR", "lludp.var.ipaddr", "FT_IPv4", nil, "BASE_DEC")
fds.var_ipport = ProtoField.new("IPPORT", "lludp.var.ipport", "FT_UINT16", nil, "BASE_DEC")
-- variable type handlers.
local variable_handlers = {
Fixed = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add_le(fds.var_fixed, rang)
if len <= 4 then
ti:set_text(str_format("%s: 0x%08x",var.name, rang:uint()))
else
ti:set_text(str_format("%s: length=%d, Blob:%s", var.name, len, tostring(rang)))
end
end,
Variable = function(block_tree, buffer, offset, len, var)
local is_data = false
-- try to guess if this field is text.
if var.name:find("Data") then
-- this is a data find.
is_data = true
end
local str_rang = buffer(offset + var.count_length, len - var.count_length)
local bytes = str_rang:bytes()
if not is_data and is_string(bytes) then
local str = str_rang:string()
local ti = block_tree:add(fds.var_string, buffer(offset, len), str)
ti:set_text(str_format("%s: %s", var.name, str))
else
local rang = buffer(offset, len)
local ti = block_tree:add_le(fds.var_variable, rang)
if len <= 4 then
ti:set_text(str_format("%s: 0x%08x",var.name, rang:uint()))
else
ti:set_text(str_format("%s: length=%d, Blob:%s", var.name, len, tostring(rang)))
end
end
end,
U8 = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add_le(fds.var_u8, rang)
ti:set_text(str_format("%s: %d", var.name, rang:le_uint()))
end,
U16 = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add_le(fds.var_u16, rang)
ti:set_text(str_format("%s: %d", var.name, rang:le_uint()))
end,
U32 = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add_le(fds.var_u32, rang)
ti:set_text(str_format("%s: %d", var.name, rang:le_uint()))
end,
U64 = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add_le(fds.var_u64, rang)
ti:set_text(str_format("%s: 0x%s",var.name, tostring(rang)))
end,
S8 = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add_le(fds.var_s8, rang)
local num = rang:le_uint()
if num > 127 then num = num - 256 end
ti:set_text(str_format("%s: %d",var.name, num))
end,
S16 = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add_le(fds.var_s16, rang)
local num = rang:le_uint()
if num > 32768 then num = num - 65536 end
ti:set_text(str_format("%s: %d",var.name, num))
end,
S32 = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add_le(fds.var_s32, rang)
local num = rang:le_uint()
if num > 2147483648 then num = num - 4294967296 end
ti:set_text(str_format("%s: %d",var.name, num))
end,
S64 = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add_le(fds.var_s64, rang)
ti:set_text(str_format("%s: 0x%s",var.name, tostring(rang)))
end,
F32 = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add_le(fds.var_f32, rang)
ti:set_text(str_format("%s: %f", var.name, rang:le_float()))
end,
F64 = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add_le(fds.var_f64, rang)
ti:set_text(str_format("%s: %f", var.name, rang:le_float()))
end,
LLVector3 = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add_le(fds.var_llvector3, rang)
-- parse LLVector3
local x,y,z
x = buffer(offset + 0,4):le_float()
y = buffer(offset + 4,4):le_float()
z = buffer(offset + 8,4):le_float()
-- display
ti:set_text(str_format("%s: <%f,%f,%f>", var.name, x, y, z))
end,
LLVector3d = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add_le(fds.var_llvector3d, rang)
-- parse LLVector3d
local x,y,z
x = buffer(offset + 0,8):le_float()
y = buffer(offset + 8,8):le_float()
z = buffer(offset + 16,8):le_float()
-- display
ti:set_text(str_format("%s: <%f,%f,%f>", var.name, x, y, z))
end,
LLVector4 = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add_le(fds.var_llvector4, rang)
-- parse LLVector4
local x,y,z,s
x = buffer(offset + 0,4):le_float()
y = buffer(offset + 4,4):le_float()
z = buffer(offset + 8,4):le_float()
s = buffer(offset + 12,4):le_float()
-- display
ti:set_text(str_format("%s: <%f,%f,%f,%f>", var.name, x, y, z, s))
end,
LLQuaternion = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add_le(fds.var_llquaternion, rang)
-- parse LLQuaternion
local x,y,z,w
x = buffer(offset + 0,4):le_float()
y = buffer(offset + 4,4):le_float()
z = buffer(offset + 8,4):le_float()
-- calculate W
w = 1 - (x * x) - (y * y) - (z * z)
if w > 0 then
w = math.sqrt(w)
else
w = 0
end
-- display
ti:set_text(str_format("%s: <%f,%f,%f,%f>", var.name, x, y, z, w))
end,
LLUUID = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add_le(fds.var_lluid, rang)
local str = tostring(rang)
str = str:sub(1,8) .. '-' ..
str:sub(9,12) .. '-' .. str:sub(13,16) .. '-' ..
str:sub(17,20) .. '-' .. str:sub(21)
ti:set_text(str_format("%s: %s", var.name, str))
end,
BOOL = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add_le(fds.var_bool, rang)
local val = "false"
if rang:le_uint() > 0 then
val = "true"
end
ti:set_text(str_format("%s: %s", var.name, val))
end,
IPADDR = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add(fds.var_ipaddr, rang)
ti:set_text(str_format("%s: %s", var.name, tostring(rang:ipv4())))
end,
IPPORT = function(block_tree, buffer, offset, len, var)
local rang = buffer(offset, len)
local ti = block_tree:add(fds.var_ipport, rang)
ti:set_text(str_format("%s: %d", var.name, rang:uint()))
end,
}

-- un-register lludp to handle udp port range
local function unregister_udp_port_range(start_port, end_port)
if not start_port or start_port <= 0 or not end_port or end_port <= 0 then
return
end
udp_port_table = DissectorTable.get("udp.port")
for port = start_port,end_port do
udp_port_table:remove(port,lludp_proto)
end
end

-- register lludp to handle udp port range
local function register_udp_port_range(start_port, end_port)
if not start_port or start_port <= 0 or not end_port or end_port <= 0 then
return
end
udp_port_table = DissectorTable.get("udp.port")
for port = start_port,end_port do
udp_port_table:add(port,lludp_proto)
end
end

-- handle preferences changes.
function lludp_proto.init(arg1, arg2)
local old_start, old_end
local new_start, new_end
-- check if preferences have changed.
for pref_name,old_v in pairs(current_settings) do
local new_v = lludp_proto.prefs[pref_name]
if new_v ~= old_v then
if pref_name == "template_file" then
-- load & parse message_template.msg file.
local file = new_v
if file and file:len() > 0 then
local new_details = parse_template(file)
if new_details then
message_details = new_details
end
end
elseif pref_name == "udp_port_start" then
old_start = old_v
new_start = new_v
elseif pref_name == "udp_port_end" then
old_end = old_v
new_end = new_v
end
-- save new value.
current_settings[pref_name] = new_v
end
end
-- un-register old port range
if old_start and old_end then
unregister_udp_port_range(tonumber(old_start), tonumber(old_end))
end
-- register new port range.
if new_start and new_end then
register_udp_port_range(tonumber(new_start), tonumber(new_end))
end
end

-- parse flag bits.
local FLAG_ZER = 4
local FLAG_REL = 3
local FLAG_RES = 2
local FLAG_ACK = 1
local flag_names = {"ACK", "RES", "REL", "ZER"}
local bits_lookup = {
{},
{1},
{2},
{2,1},
{3},
{3,1},
{3,2},
{3,2,1},
{4},
{4,1},
{4,2},
{4,2,1},
{4,3},
{4,3,1},
{4,3,2},
{4,3,2,1},
}
local function parse_flags(flags)
flags = (flags / 16) + 1
local bit_list = bits_lookup[flags]
local bits = {}
local names = ""
for _, bit in ipairs(bit_list) do
bits[bit] = true
if names:len() > 0 then
names = names .. ", "
end
names = names .. flag_names[bit]
end
return bits, names
end

local function grow_buff(buff, size)
local old_size = buff:len()
if old_size > size then return end
-- buffer needs to grow
buff:set_size(size)
-- fill new space with zeros
for i = old_size,size-1 do
buff:set_index(i, 0)
end
end

local function zero_decode(zero_buf)
local out_buf = ByteArray.new()
local zero_off = 0
local zero_len = zero_buf:len()
local out_size = 0
local out_off = 0
local b
-- pre-allocate
grow_buff(out_buf, zero_len)
out_size = zero_len
-- zero expand
repeat
b = zero_buf:get_index(zero_off)
if b == 0 then
-- get zero count
local count = zero_buf:get_index(zero_off + 1)
if count == 0 then count = 255 end
out_off = out_off + count
-- fill zeros
if out_off > out_size then
out_size = out_off + 128
grow_buff(out_buf, out_size)
end
zero_off = zero_off + 2
else
if out_off >= out_size then
out_size = out_off + (zero_len - zero_off) + 4
grow_buff(out_buf, out_size)
end
-- copy non-zero bytes.
out_buf:set_index(out_off,b)
zero_off = zero_off + 1
out_off = out_off + 1
end
until zero_off == zero_len
-- truncate to real size.
out_buf:set_size(out_off)

return out_buf:tvb("Decompressed Data")
end

local function parse_msg_id(buff)
local b = buff:get_index(0)
local msg_id = b
local msg_id_len = 1
if b == 255 then
b = buff:get_index(1)
msg_id = msg_id * 256 + b
msg_id_len = 2
if b == 255 then
b = buff:get_index(2)
msg_id = msg_id * 256 + b
b = buff:get_index(3)
msg_id = msg_id * 256 + b
msg_id_len = 4
end
end
return msg_id, msg_id_len
end

-- get message name.
local function get_msg_name(msg_id)
-- check that we have message details
if message_details == nil then
return str_format("0x%08x", msg_id)
end
-- find message name from id.
local msg = message_details.msgs[msg_id]
-- Invalid message id
if msg == nil then
return str_format("0x%08x", msg_id)
end
return msg.name
end

-- calculate length a block.
local function get_block_length(msg_buffer, start_offset, block)
-- check if bock is fixed length.
if block.fixed_length then
return block.min_length
end
-- parse block's variables to calculate total block length.
local offset = start_offset
local rang
for _,var in ipairs(block) do
local len = 0
if var.has_count then
-- variable with length bytes.
len = var.count_length
--print(var.name, offset, ", len:", len)
rang = msg_buffer(offset, len)
len = len + rang:le_uint()
--print(var.name, var.count_length, ", total:", len)
else
-- fixed length variable
len = var.length
--print(var.name, ", total:", len)
end
offset = offset + len
end
return (offset - start_offset)
end

-- build block tree
local function build_block_tree(msg_buffer, block_tree, start_offset, block)
local offset = start_offset
local rang
-- parse block's variables
for _,var in ipairs(block) do
local len = 0
if var.has_count then
-- variable with length bytes.
len = var.count_length
rang = msg_buffer(offset, len)
len = len + rang:le_uint()
else
-- fixed length variable
len = var.length
end
-- get variable's type field.
local handler = variable_handlers[var.type]
-- parse variable.
if handler then
handler(block_tree, msg_buffer, offset, len, var)
end
offset = offset + len
end
return (offset - start_offset)
end

-- buid message tree
local function build_msg_tree(msg_buffer, msg_tree, msg_id)
local offset = 0
local rang
-- check that we have message details
if message_details == nil then
msg_tree:set_text(str_format("Message Id: 0x%08x", msg_id))
return nil
end
-- find message name from id.
local msg = message_details.msgs[msg_id]
-- Invalid message id
if msg == nil then
msg = str_format("Invalid message id: 0x%08x", msg_id)
msg_tree:add_expert_info(PI_MALFORMED, PI_ERROR, msg)
msg_tree:set_text(msg)
return nil
end
-- skip message id bytes.
offset = msg.id_length
-- set message name.
msg_tree:set_text(msg.name .. ":")
-- proccess message blocks
for _,block in ipairs(msg) do
local count = block.count
if count == nil then
-- parse count byte.
rang = msg_buffer(offset,1)
count = rang:uint()
msg_tree:add(fds.block_count,rang)
offset = offset + 1
end
-- print("block name: ", block.name, count)
for n=1,count do
local block_len = get_block_length(msg_buffer, offset, block)
-- parse block
rang = msg_buffer(offset, block_len)
local block_tree = msg_tree:add(fds.block,rang)
if count > 1 then
block_tree:set_text(str_format("%s: %d of %d",block.name,n,count))
else
block_tree:set_text(block.name)
end
-- parse block variables.
build_block_tree(msg_buffer, block_tree, offset, block)
offset = offset + block_len
end
end
return msg.name
end

-- packet dissector
function lludp_proto.dissector(buffer,pinfo,tree)
local rang,offset
pinfo.cols.protocol = "LLUDP"
local lludp_tree = tree:add(lludp_proto,buffer(),"Linden UDP Protocol")
-- Flags byte.
offset = 0
rang = buffer(offset,1)
local flags = rang:uint()
local flags_bits, flags_list = parse_flags(flags)
flags_tree = lludp_tree:add(fds.flags, rang)
flags_tree:set_text("Flags: " .. str_format('0x%02X (%s)', flags, flags_list))
flags_tree:add(fds.flags_zero, rang)
flags_tree:add(fds.flags_reliable, rang)
flags_tree:add(fds.flags_resent, rang)
flags_tree:add(fds.flags_ack, rang)
offset = offset + 1
-- Sequence number 4 bytes.
rang = buffer(offset,4)
local sequence = rang:uint()
lludp_tree:add(fds.sequence, rang)
offset = offset + 4
-- Extra header length.
rang = buffer(offset,1)
local extra_length = rang:uint()
lludp_tree:add(fds.extra_len,rang)
offset = offset + 1
-- Extra header data.
if extra_length > 0 then
rang = buffer(offset, extra_length)
lludp_tree:add(fds.extra_bytes, rang)
offset = offset + extra_length
end
-- Appended Acks. count
local acks_bytes = 0
local acks_count = 0
if flags_bits[FLAG_ACK] then
rang = buffer(buffer:len() - 1, 1)
acks_count = rang:uint()
acks_bytes = (acks_count * 4) + 1
end
-- Zero Decode
local msg_len = (buffer:len() - acks_bytes) - offset
if flags_bits[FLAG_ZER] then
msg_buffer=zero_decode(buffer(offset,msg_len):bytes())
msg_len = msg_buffer:len()
offset = 0
else
msg_buffer = buffer(offset, msg_len):tvb()
offset = 0
end
-- Message ID
local msg_id, msg_id_len = -1, 4
if msg_id_len > msg_len then
msg_id_len = msg_len
end
msg_id, msg_id_len = parse_msg_id(msg_buffer(offset, msg_id_len):bytes())
rang = msg_buffer(offset, msg_id_len)
lludp_tree:add(fds.msg_id, rang)
local msg_name = get_msg_name(msg_id)
if msg_name == nil then
msg_name = str_format("0x%08x", msg_id)
else
lludp_tree:add(fds.msg_name, rang, msg_name)
end
-- Message body.
rang = msg_buffer(offset, msg_len)
local msg_tree = lludp_tree:add(fds.msg, rang)
build_msg_tree(msg_buffer, msg_tree, msg_id)
-- Appended Acks. list.
if flags_bits[FLAG_ACK] then
local acks_off = buffer:len()
rang = buffer(acks_off - 1, 1)
acks_off = acks_off - acks_bytes
local acks_tree = lludp_tree:add(fds.acks_count, rang)
for i = 1,acks_count do
rang = buffer(acks_off,4)
acks_tree:add(fds.acks, rang)
acks_off = acks_off + 4
end
end
-- Info column
pinfo.cols.info = str_format('[%s] Seq=%u Type=%s', flags_list, sequence, msg_name)
end

-- register lludp to handle udp ports 9000-9003
register_udp_port_range(9000,9003)
</source>

=== File "llmessage.lua" ===
<source lang="lua">
--[[
BSD-Licensed:
Copyright (c) 2008, Robert G. Jakabosky <bobby@sharedrealm.com>
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
* Neither the name of the SharedRealm nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

]]

dofile("lexer.lua")

local lexer
local cur_token = nil
local cur_token_str = nil

local function get_token(skip_tokens)
repeat
token = lexer.get_token()
if token ~= nil then
cur_token = token[1]
cur_token_str = token[2]
end
until token == nil or not skip_tokens[cur_token]
return token
end

local function run_parser(parser)
local state = parser.init()
if parser.skip_tokens == nil then parser.skip_tokens = {} end
while get_token(parser.skip_tokens) do
-- check what the parser is expecting next.
if state.expect then
-- check expected type
if state.expect ~= cur_token then
error(string.format("state.expected token '%s' instead of '%s'",
TokenNames[state.expect], TokenNames[cur_token]))
end
-- reset expect field
state.expect = nil
end
if state.expect_str then
-- check expected string
if state.expect_str ~= cur_token_str then
error(string.format("state.expected token '%s' instead of '%s'",
state.expect_str, cur_token_str))
end
-- reset expect_str field
state.expect_str = nil
end
-- get handler function for current token.
local f = parser[cur_token]
if f ~= nil then
local ret = f(state)
if ret then
-- praser finished.
return ret
end
elseif parser.unhandled_error then
error(string.format("unhandled token '%s' when paring '%s'\n", cur_token_str, parser.name))
end
end
return parser.eof(state)
end

-- Known variable types and there fixed length.
-- length == -1, requires a number after the type that is the length of count field
-- length == -2, requires a number after the type that is the fixed variable length.
VariableTypes = {
Null = 0,
Fixed = -2,
Variable = -1,
U8 = 1,
U16 = 2,
U32 = 4,
U64 = 8,
S8 = 1,
S16 = 2,
S32 = 4,
S64 = 8,
F32 = 4,
F64 = 8,
LLVector3 = 12,
LLVector3d = 24,
LLVector4 = 16,
LLQuaternion = 12,
LLUUID = 16,
BOOL = 1,
IPADDR = 4,
IPPORT = 2,
}

--
-- Variable parser
--
local variable_parser = {
name = "variable",
unhandled_error = false,
skip_tokens = {[Token.EOL] = true},
init = function()
return {
name = "<MISSING VARIABLE NAME>",
type = "Null",
has_count = false,
length = 0,
expect = Token.IDENTIFIER,
expect_field = "name",
required = 2
}
end,
[Token.IDENTIFIER] = function(state)
if state.expect_field == "name" then
state.name = cur_token_str
state.expect = Token.IDENTIFIER
state.expect_field = "type"
state.required = state.required - 1
elseif state.expect_field == "type" then
state.type = cur_token_str
state.length = VariableTypes[state.type]
if state.length == nil then
error("Unknown variable type: " .. cur_token_str)
elseif state.length == -1 or state.length == -2 then
state.expect = Token.NUMBER
else
state.required = state.required - 1
end
else
error(string.format("unhandled variable identifier: %s\n",cur_token_str))
end
return nil
end,
[Token.NUMBER] = function(state)
if state.expect_field == "type" then
if state.length == -1 then
-- variable field length uses embedded count field
state.has_count = true
state.count_length = tonumber(cur_token_str)
state.length = nil
elseif state.length == -2 then
-- fixed field length
state.length = tonumber(cur_token_str)
end
state.required = state.required - 1
else
error(string.format("unhandled variable number: %s\n",cur_token_str))
end
return nil
end,
["{"] = function(state)
error("sub block not allowed in variable block")
end,
["}"] = function(state)
if state.required > 0 then
error("missing " .. state.required .. " fields")
end
-- clean state.
state.required = nil
state.expect = nil
state.expect_field = nil
return state
end,
eof = function(state)
error("missing '}' token at end of variable: " .. state.name)
end,
}

-- Block Quantities
local BlockQuantity = {
Single = 1,
Variable = -1,
Multiple = -2,
}

--
-- Block parser
--
local block_parser = {
name = "block",
unhandled_error = false,
skip_tokens = {[Token.EOL] = true},
init = function()
return {
name = "<MISSING BLOCK NAME>",
quantity = "Single",
count = 0,
min_length = 0,
fixed_length = true,
expect = Token.IDENTIFIER,
expect_field = "name",
required = 2
}
end,
[Token.IDENTIFIER] = function(state)
if state.expect_field == "name" then
state.name = cur_token_str
state.expect = Token.IDENTIFIER
state.expect_field = "quantity"
state.required = state.required - 1
elseif state.expect_field == "quantity" then
state.quantity = cur_token_str
state.count = BlockQuantity[cur_token_str]
if state.count == nil then
error("Unknown block quantity: " .. cur_token_str)
elseif state.count == -2 then
state.expect_field = "count"
state.expect = Token.NUMBER
else
if state.count == -1 then
state.has_count = true
state.count_length = 1
state.count = nil
end
state.required = state.required - 1
end
else
error(string.format("unhandled block identifier: %s\n",cur_token_str))
end
return nil
end,
[Token.NUMBER] = function(state)
if state.expect_field == "count" then
state.count = tonumber(cur_token_str)
state.required = state.required - 1
else
error(string.format("unhandled block number: %s\n",cur_token_str))
end
return nil
end,
["{"] = function(state)
local variable = run_parser(variable_parser)
table.insert(state,variable)
-- add length of fixed length variables to minimal length of block.
if variable.has_count then
state.min_length = state.min_length + variable.count_length
state.fixed_length = false
else
state.min_length = state.min_length + variable.length
end
end,
["}"] = function(state)
if state.required > 0 then
error("missing " .. state.required .. " fields")
end
-- clean state.
state.required = nil
state.expect = nil
state.expect_field = nil
return state
end,
eof = function(state)
error("missing '}' token at end of block: " .. state.name)
end,
}

--
-- Message parser
--
local message_parser = {
name = "message",
unhandled_error = false,
skip_tokens = {[Token.EOL] = true},
init = function()
-- create state
return {
name = "<MISSING MESSAGE NAME>",
expect = Token.IDENTIFIER,
expect_field = "name",
fixed_length = true,
min_length = 0,
required = 5
}
end,
[Token.IDENTIFIER] = function(state)
if state.expect_field == "name" then
state.name = cur_token_str
state.expect = Token.IDENTIFIER
state.expect_field = "frequency"
state.required = state.required - 1
elseif state.expect_field == "frequency" then
state.frequency = cur_token_str
state.expect = Token.NUMBER
state.required = state.required - 1
elseif state.expect_field == "trust" then
state.trust = cur_token_str
state.expect = Token.IDENTIFIER
state.expect_field = "compression"
state.required = state.required - 1
elseif state.expect_field == "compression" then
state.compression = cur_token_str
state.required = state.required - 1
else
error(string.format("unhandled message identifier: %s\n",cur_token_str))
end
return nil
end,
[Token.NUMBER] = function(state)
if state.expect_field == "frequency" then
state.number = tonumber(cur_token_str)
state.expect = Token.IDENTIFIER
state.expect_field = "trust"
state.required = state.required - 1
-- create true message id from frequency and message number
local freq = state.frequency
if freq == "High" then
-- High is already correct.
state.id = state.number
state.id_length = 1
elseif freq == "Medium" then
state.id = tonumber("0xFF" .. string.format("%02X", state.number))
state.id_length = 2
elseif freq == "Low" then
state.id = tonumber("0xFFFF" .. string.format("%04X", state.number))
state.id_length = 4
else
-- Fixed is already correct.
state.id = state.number
state.id_length = 4
end
else
error(string.format("unhandled message number: %s\n",cur_token_str))
end
return nil
end,
["{"] = function(state)
local block = run_parser(block_parser)
table.insert(state,block)
-- add min length of block to minimal length of message
local min_length = block.min_length
if block.has_count then
-- add one byte for the block count
min_length = min_length + 1
state.fixed_length = false
else
-- if block is not fixed length then message can't be fixed length.
if not block.fixed_length then
state.fixed_length = false
end
min_length = min_length * block.count
end
state.min_length = state.min_length + min_length
end,
["}"] = function(state)
if state.required > 0 then
error("missing " .. state.required .. " fields")
end
-- clean state.
state.required = nil
state.expect = nil
state.expect_field = nil
return state
end,
eof = function(state)
error("missing '}' token at end of message: " .. state.name)
end,
}

--
-- Template file parser
--
local template_parser = {
name = "message_template",
unhandled_error = false,
init = function()
return {
version = 0,
msg_count = 0,
msgs = {}
}
end,
[Token.IDENTIFIER] = function(state)
-- handle version
if cur_token_str == "version" then
state.expect = Token.NUMBER
state.last_ident = cur_token_str
else
error(string.format("unknown template identifier: %s\n",cur_token_str))
end
return nil
end,
[Token.NUMBER] = function(state)
-- handle version number
if state.last_ident == "version" then
state.last_ident = nil
state.version = tonumber(cur_token_str)
-- check version number
if state.version ~= 2 then
error("invalid verion: " .. state.version)
end
else
error(string.format("unhandled template number: %s\n",cur_token_str))
end
return nil
end,
["{"] = function(state)
local message = run_parser(message_parser)
state.msg_count = state.msg_count + 1
state.msgs[message.id] = message
end,
["}"] = function(state)
error(string.format("unhandled '%s' token",cur_token_str))
end,
eof = function(state)
return state
end,
}

function parse_template(file)
-- create lexer
local status, ret = pcall(get_lexer,file)
if not status then
ret = string.format("LLUDP: Failed parse file into tokens: %s\n%s\n", file, ret)
error(ret, 0)
return nil
end
lexer = ret
-- parse template file
local status, ret = pcall(run_parser,template_parser)
if not status then
ret = string.format("LLUDP: Failed parsing on line %s:%d: '%s'\n%s\n",
file, lexer.get_line_number(), lexer.get_line(), ret)
error(ret, 0)
return nil
end
io.write("finished parsing: " .. file .. "\n")
-- return list of messages parsed from file.
return ret
end

--parse_template("message_template.msg")
--print_tokens("message_template.msg")

</source>

=== File "lexer.lua" ===
<source lang="lua">
--[[
BSD-Licensed:
Copyright (c) 2008, Robert G. Jakabosky <bobby@sharedrealm.com>
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
* Neither the name of the SharedRealm nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

]]

-- Token types.
Token = {
NONE = -1,
IDENTIFIER = -2,
NUMBER = -3,
COMMENT = -6,
EOL = -7,
["{"] = "{",
["}"] = "}",
}
TokenNames = {
[-1] = "NONE",
[-2] = "IDENTIFIER",
[-3] = "NUMBER",
[-6] = "COMMENT",
[-7] = "EOL",
["{"] = "{",
["}"] = "}",
}

-- parse line into array of tokens.
local function default_parse_tokens(line)
local tokens = {}
local comment = nil
-- check for a comment on this line.
local idx = line:find("//")
if idx ~= nil then
comment = {Token.COMMENT, line:sub(idx)}
-- remove comment from line
line = line:sub(1,idx - 1)
end

-- split line into tokens using white-space as token delimitator
for tok in line:gmatch("%s?([^%s]+)") do
local tok_type = Token.NONE
-- check for number
if tonumber(tok) ~= nil then
tok_type = Token.NUMBER
elseif Token[tok] then
-- token is same as type
tok_type = Token[tok]
else
-- token is an identifier
tok_type = Token.IDENTIFIER
end
table.insert(tokens,{tok_type,tok})
end
-- insert comment token.
if comment ~= nil then
table.insert(tokens,comment)
end
-- add token to mark the end of this line
table.insert(tokens,{Token.EOL, ""})
return tokens
end

function get_lexer(file, parse_tokens)
-- use the default line tokenizer if one is not provided
if parse_tokens == nil then parse_tokens = default_parse_tokens end
-- next/current line code
local line_num = 0
local line = nil
local next_line = io.lines(file)
-- parse line tokens code
local get_next_token = nil
local next_tokens = function ()
local f, tokens, idx
repeat
line_num = line_num + 1
line = next_line()
if line == nil then return nil end
tokens = parse_tokens(line)
until tokens ~= nil
-- create get_next_toekn function from table iterator
f, tokens, idx = ipairs(tokens)
get_next_token = function()
idx, token = f(tokens, idx)
return token
end
return tokens
end

-- get first group of tokens
if next_tokens() == nil then
-- error reading file or empty file
return nil
end
-- build lexer table.
local lexer = {
get_token = function ()
local token
repeat
token = get_next_token()
if token == nil then
-- get next group of tokens
if next_tokens() == nil then
-- end of file.
return nil
end
end
until token ~= nil
return token
end,
get_line_number = function() return line_num end,
get_line = function() return line end
}
return lexer
end

function print_tokens(file)
local lexer = get_lexer(file)
local num = -1
while true do
local tok = lexer.get_token()
if tok == nil then
break
end
if num ~= lexer.get_line_number() then
num = lexer.get_line_number()
io.write("\n")
io.write(string.format("%d: ",num))
end
io.write(string.format("%s ",tok[2]))
end
io.write("\n")
end

</source>

Talk:Automated Testing

2008-10-09T08:01:46Z

Mikem:

Talk:Automated Testing

2008-10-09T08:01:33Z

Mikem:

Talk:Automated Testing

2008-10-09T07:58:36Z

Mikem: New page: I crossed out a paragraph about naming tests a certain way in order to make sure they're run in a certain order because they depend on each other. This is bad practice. All tests should be...

Automated Testing

2008-10-09T07:17:36Z

Mikem: /* NUnit Conventions */

Automated Testing

2008-10-09T07:06:38Z

Mikem: /* Writing New Tests */

Security vulnerability brought by non-check inventory service

2008-08-12T06:57:03Z

Mikem: /* *NOTE* */

== Problem ==

With the following conditions, one can simply take over the full control(CRUD) of other user's inventory.
# InventoryServer is exposed to the public.
# user's UUID is given

Simply describe in the following figure:
*InventoryServer is a normal http server, the normal way to use it is:
**user get the authentication from UserServer
**user control its inventory through RegionServer
*But since the InventoryServer accepts any request without check if the user is authenticated, or, even it does not check if the request is from a RegionServer.
*So, if you know other users' UUID, you can send CRUD http requests directly to the InventoryServer without login.

[[Image:secure_inventory_1.PNG]]

And [[Avatar_portability_version_2|AvatarPortability]] needs a public inventory server,
so we have to make a secure one.

== Solution ==

* every inventory operation packet contains a "session_id" field, but it is never used.
* so, a secure inventory service could be like this
[[Image:secure_inventory_2.PNG]]
* "session_id" is a important information, that is(should be) only transfered in a login session.
**"expect_user" transfer "session_id" from UserServer to RegionServer only when the authentication is OK, so "expect_user" is safe.
**method, such like "get_agent_by_uuid" is very dangerous.

== Configuration ==

=== RegionServer side ===
* in OpenSim.ini, [Network] section,
inventory_server_url = http://127.0.0.1:8004
'''secure_inventory_server''' = true / false
* if the inventory server specified by "inventory_server_url" is a "secure" inventory server, set "secure_inventory_server = true". Then, inventory requests from the region server will have the user's session_id attached
* else, set "secure_inventory_server = false". In this case, session_id is not attached to inventory requests.
** Setting secure_inventory_server to false is only useful when you want your region server to connect to an '''old inventory server''' which does not expect a session_id.

=== InventoryServer side ===
* in InventoryServer_Config.xml,
'''session_lookup''' = true / false
(* for session_lookup please also refer the picture above)
* if you want the inventory server to validate each incoming inventory request by session_id, set session_lookup = true
* else, set session_lookup = false
** Setting session_lookup to false makes inventory server accept any request from any client.

=== *NOTE* ===
* Regardless of whether '''session_lookup''' is true or false, '''new inventory server''' requires a session_id in every inventory request. If you want your region server to connect to a '''new inventory server''', you should always set '''secure_inventory_server = true''' in OpenSim.ini.
** Here '''new inventory server''' means inventory server after SVN revision 5600.

Security vulnerability brought by non-check inventory service

2008-08-12T06:54:30Z

Mikem: /* InventoryServer side */

== Problem ==

With the following conditions, one can simply take over the full control(CRUD) of other user's inventory.
# InventoryServer is exposed to the public.
# user's UUID is given

Simply describe in the following figure:
*InventoryServer is a normal http server, the normal way to use it is:
**user get the authentication from UserServer
**user control its inventory through RegionServer
*But since the InventoryServer accepts any request without check if the user is authenticated, or, even it does not check if the request is from a RegionServer.
*So, if you know other users' UUID, you can send CRUD http requests directly to the InventoryServer without login.

[[Image:secure_inventory_1.PNG]]

And [[Avatar_portability_version_2|AvatarPortability]] needs a public inventory server,
so we have to make a secure one.

== Solution ==

* every inventory operation packet contains a "session_id" field, but it is never used.
* so, a secure inventory service could be like this
[[Image:secure_inventory_2.PNG]]
* "session_id" is a important information, that is(should be) only transfered in a login session.
**"expect_user" transfer "session_id" from UserServer to RegionServer only when the authentication is OK, so "expect_user" is safe.
**method, such like "get_agent_by_uuid" is very dangerous.

== Configuration ==

=== RegionServer side ===
* in OpenSim.ini, [Network] section,
inventory_server_url = http://127.0.0.1:8004
'''secure_inventory_server''' = true / false
* if the inventory server specified by "inventory_server_url" is a "secure" inventory server, set "secure_inventory_server = true". Then, inventory requests from the region server will have the user's session_id attached
* else, set "secure_inventory_server = false". In this case, session_id is not attached to inventory requests.
** Setting secure_inventory_server to false is only useful when you want your region server to connect to an '''old inventory server''' which does not expect a session_id.

=== InventoryServer side ===
* in InventoryServer_Config.xml,
'''session_lookup''' = true / false
(* for session_lookup please also refer the picture above)
* if you want the inventory server to validate each incoming inventory request by session_id, set session_lookup = true
* else, set session_lookup = false
** Setting session_lookup to false makes inventory server accept any request from any client.

=== *NOTE* ===
* no matter '''session_lookup''' is true or false, '''new inventoryserver''' requires session_id in every inventory request. if you want your regionserver to connect to a '''new inventoryserver''', you should always set '''secure_inventory_server = true''' in OpenSim.ini.
* here '''new inventoryserver''' means inventoryserver after svn revision 5600.

Security vulnerability brought by non-check inventory service

2008-08-12T06:52:42Z

Mikem: /* RegionServer side */

== Problem ==

With the following conditions, one can simply take over the full control(CRUD) of other user's inventory.
# InventoryServer is exposed to the public.
# user's UUID is given

Simply describe in the following figure:
*InventoryServer is a normal http server, the normal way to use it is:
**user get the authentication from UserServer
**user control its inventory through RegionServer
*But since the InventoryServer accepts any request without check if the user is authenticated, or, even it does not check if the request is from a RegionServer.
*So, if you know other users' UUID, you can send CRUD http requests directly to the InventoryServer without login.

[[Image:secure_inventory_1.PNG]]

And [[Avatar_portability_version_2|AvatarPortability]] needs a public inventory server,
so we have to make a secure one.

== Solution ==

* every inventory operation packet contains a "session_id" field, but it is never used.
* so, a secure inventory service could be like this
[[Image:secure_inventory_2.PNG]]
* "session_id" is a important information, that is(should be) only transfered in a login session.
**"expect_user" transfer "session_id" from UserServer to RegionServer only when the authentication is OK, so "expect_user" is safe.
**method, such like "get_agent_by_uuid" is very dangerous.

== Configuration ==

=== RegionServer side ===
* in OpenSim.ini, [Network] section,
inventory_server_url = http://127.0.0.1:8004
'''secure_inventory_server''' = true / false
* if the inventory server specified by "inventory_server_url" is a "secure" inventory server, set "secure_inventory_server = true". Then, inventory requests from the region server will have the user's session_id attached
* else, set "secure_inventory_server = false". In this case, session_id is not attached to inventory requests.
** Setting secure_inventory_server to false is only useful when you want your region server to connect to an '''old inventory server''' which does not expect a session_id.

=== InventoryServer side ===
* in InventoryServer_Config.xml,
'''session_lookup''' = true / false
(* for the session_lookup please also refer the above picture.)
* if you want inventory server to validate the incoming session_id, set session_lookup = true
* else, set session_lookup = false
** this makes inventory server accept any request, just like before.

=== *NOTE* ===
* no matter '''session_lookup''' is true or false, '''new inventoryserver''' requires session_id in every inventory request. if you want your regionserver to connect to a '''new inventoryserver''', you should always set '''secure_inventory_server = true''' in OpenSim.ini.
* here '''new inventoryserver''' means inventoryserver after svn revision 5600.