[Opensim-dev] Suggestion for project organization.

[Justin Clark-Casey]

Michael Cortez wrote:
> I'd like to make a suggestion about how the OpenSim project is setup
> in trunk for building and get some feedback from ya'll.
> 
> Currently if a developer would like to work on a new module for
> OpenSim, they have to first decide if they want to work inside trunk
> (building their module along with OpenSim in the same build) or if
> they want to create a completely separate project that just links to
> the DLLs produced from OpenSim's trunk.  I don't know about others,
> but I personally tend to want to build and debug using the entire
> trunk.

Personally I much prefer to build against DLLs, since it makes the build process much simpler and doesn't require 
inevitably messy insertions of projects into the OpenSim solution.  Indeed, I did this for the Parallel Selves IM Bridge 
if anybody is interested in seeing what the minimum DLL set is.

> 
> So my new module experience is usually something along the following lines:
> 
> * Checkout trunk
> * Prebuild
> * Load up the project in Visual Studio, add a new project, or simply
> add a new file to one of the existing projects (Core or Optional
> Modules)
> * Add any additional references to the existing or new project that
> are needed for my new code
> * Work on my code for a while.
> * Copy out the relevant files to my own separate code repository
> 
> Then at some point I end up updating the checkout with latest trunk,
> running prebuild, load up the solution file and cuss because the extra
> bits I added for my project are no longer present in the solution
> (were overwritten by prebuild.)  At this point I usually then go
> modify the huge (and growing) prebuild.xml file to add my own project
> in there, so it'll be generated along with the trunk projects each
> time I prebuild.
> 
> I've also noticed a lot of feature creep of things into trunk of
> things that are really optional and not required for either building
> or running of OpenSim by a lot of people utilizing trunk -=- but they
> are things being worked on by the core developers and are definitely
> "nice to have" things.
> 
> Here is my suggestion:
> 
> 1) Create a new top level directory in at /trunk/OpenSimModules to
> contain individual module projects that are not required/core modules
> 
> 2) Create an empty Modules directory at /trunk/OpenSim/Modules/ where
> nested SVN projects can be checked out to
> 
> 3) Each module that is optional, and not truly required for every
> OpenSim instance can slowly be migrated into /trunk/OpenSimModules/
> for example /trunk/OpenSimModules/FlotsamAssetCache  -- each of these
> modules would have their own mini, self contained, prebuild included
> in their project directory
> 
> 4) Patch prebuild so that it allows a wildcard to be specified in the
> include path, so the following would be valid:  <? include
> file="modules/*/prebuild.xml" ?>  [Note: I have a working hack/patch
> that would allow this]
> 
> 5) Modify the main prebuild.xml file located at
> /trunk/OpenSim/prebuild.xml to use this new expanded include
> directive.
> 
> 
> To use the repository in this new configuration, you would first
> checkout a copy of trunk/OpenSim -- or a stable build of your choice
> -- you then look through the list of modules within
> trunk/OpenSimModules for modules that you would like to use with your
> OpenSim instance.  You then do a nested checkout of those individual
> modules into your local working directory at /OpenSim/Modules/* {svn
> fully supports nested working copies like this.}  You then run
> prebuild as normal, which will now scan the OpenSimModules directory
> and automatically add any found projects to the nant and Visual Studio
> solution/build files.  You then run nant or open the Visual Studio
> solution file as normal.
> 
> 
> What does this accomplish?
> 
> *  It separates out optional modules so that we would be working in an
> environment that is more like a "base kernel" with "modules" that you
> add in to make distributions, aka the "Linux Distribution model."
> 
> * Separating out all the non-required modules makes the core
> trunk/OpenSim code easier to understand by new developers.
> 
> * This separation will also create a clear dividing line between what
> is "core" and what isn't
> 
> * It will be easier to "drop-in" 3rd party modules that are not stored
> in trunk, for compilation and debugging.  GForge projects could simply
> be directly checked out into the /OpenSim/Modules/ folder and would
> automatically be included during prebuild with no further
> configuration other then just running prebuild again.
> 
> * All default builds of OpenSim directly out of trunk/OpenSim will be
> slim/light weight distributions because the optional stuff won't be
> there.
> 
> * Individual modules could now be tagged and assigned version numbers.
>  They are also separately checked out and updated, thus making it a
> lot easier to update individual modules from the main SVN repository
> without also pulling down updates to the core or other modules and
> vice'versa {for example if you know the latest AssetCacheX is broken,
> but you want to retrieve a core update, you can update core without
> also updating that particular cache.}
> 
> 
> The downsides?
> 
> I could only think of two, so this is where I need the most feedback.
> The first being that it requires a little more effort on the part of
> developers because you have to do multiple SVN checkouts to get a
> "full featured" copy of OpenSim's source.   The second being the
> migration of the existing code, and the inevitable discovery of some
> code that's too highly coupled to move and the re-factoring necessary
> to do so.

I think that doing multiple source code checkouts to build OpenSim would be too onerous for the core modules.

The optional modules are arguably different.  Some time back Dr Scofield did separate these out into 
OpenSim/Region/CoreModules and OpenSim/Region/OptionalModules.  However, there was but there was a general hesitation to 
follow through on the last step of moving the optional modules out of core.  afair, the feeling at the time was that the 
inconvenience caused by interfaces in flux was too high, though I would say that the core developers should feel the 
same pain as third party developers, as this would then motivate them to reduce it.

Migrating the existing code is a refactoring debt that has to be paid off some time so I wouldn't regard this as a negative.

> 
> 
> Parting notes:
> 
> I'd also suggest doing the same for all the Grid & Stand-alone related
> components, leaving only the bare essentials in trunk/OpenSim
> necessary to make a grid attached instance along with the core
> components needed by all modules and grid services.  Thus we would
> have something like trunk/OpenSimServers or trunk/OpenSimServices
> where the grid services (Asset Server, User Server, etc) would be
> moved to, and they in-turn would be checked out into a
> trunk/OpenSim/Server/ directory.
> 
> 
> Thoughts, comments?

Michael, I would say that all this certainly makes sense in general - I think it was always the intention for OpenSim to 
eventually be a 'kernel' with an orbiting ring of modules.  I can't really comment on the specifics without seeing the 
ideas in action.

But I suspect that it's probably still a little too early to be moving anything that looks like a core module or service.

-- 
justincc
Justin Clark-Casey
http://justincc.wordpress.com