[Opensim-dev] switch gears a bit, thoughts on the state of the database in OpenSim

[Stefan Andersson]

Sean,
 
may I then revoke my earlier suggestion to drop the OpenSim.Data.Base, and instead suggest we concentrate on that, since that WAS a stab at getting reasonably cheap and customizable cross-db functionality? Converting current db providers to the Base framework really should be a breeze.
It does give us things like seamless LLUUID, vector and rotation handling, as well as the possibility to write specialized queries. It also isolates connection factoring so that that we can have much more detailed control over it for various environments.
 
If the migration framework could be added to that, I think we would have something really simple, compact and efficient that will last us until we get something better on rails?
Best regards,Stefan AnderssonTribal Media AB Join the 3d web revolution : http://tribalnet.se/ 



> Date: Thu, 28 Aug 2008 15:06:59 -0400> From: sdague at gmail.com> To: opensim-dev at lists.berlios.de> Subject: [Opensim-dev] switch gears a bit, thoughts on the state of the database in OpenSim> > Over the past couple of months I've been working on an nhibernate> provider in between random bug fixes and other clean ups. For those> that don't know, nhibernate is a generic db mapping layer that helps> smooth out differences between database engines.> > There are a few things I found in this process.> > First off, it took progressively longer to do the Asset -> Inventory ->> User (never quite finished) -> Region (very partial) stores for> nhibernate. Those paying attention will note that basically that> corresponds to the reverse chronological order of those> interfaces/objects. As we go back in time to how our objects were> defined, cruft levels and inconsistencies spring up. When mapping all> of this out with custom code, these just get code around with more> custom code. When trying to push through a more standard interface,> these inconsistencies become much more evident and time sinking.> > Secondly, a lot of our older objects definitely don't follow best> practices of having properties, and a lot of refactoring was done along> the way.> > Thirdly, after tuning some of the mysql tables, the nhibernate bits> definitely seemed a bit slower than they should be. We are using 1.2> (2.0 is soon to be out). While this is most likely something we can> actually make better, there is an bit of a hit in the naive case for> doing this approach.> > Forthly, nhibernate can't do automated table upgrades (even in 2.0).> The lack of this was the reason for writing the Migrations framework> which seems to be working out quite well across all our data sources.> This was something I was hoping to get for free which we didn't, sadly.> This also has let us do data optimization (like the fixed length> uuids), and add indexes sanely.> > Finally, prevailing feeling in the community right now is that we've got> sqlite for easy startup, and most people are using mysql for serious> work. I think with moving towards addons we'll actually be able to have> out of tree drop ins for other data sources.> > All of this has led me to a slight change of gears.... I'm going to be> putting the nhibernate stuff on the back burner for now and focus on a> few other things that I think will be of more near term use (and will> make completion of nh or other db drivers easier).> > * Some clean up of the scene object model. We iterate over entities> way too often in scene, which is an issue from both program correctness> and performance.> * The rest of the UUID cleanup in the existing db drivers. We're down> to 2 formats for UUID, but we really need to get to one. That's a chunk> of time.> * Normalizing the various data driver interfaces. It's sort of amazing> how different each of those is in both flavor and naming convention,> which causes substantial confusion.> * Experiment with the possibility of doing SOP recursively and getting> rid of SOG. Before anyone freaks out, this will just be an experiment,> and will be presented to the group for review prior to anything that> extensive going into trunk.> > I think that a lot of this is pretty critical right now for a couple of> reasons. Firstly, that doing a new db driver requires a bit too much> work, which is bad for all of us. Secondly, if we are going to actually> standardize on an XML3 export format (which I think is a really> important thing to do this fall), it would be really good if some of the> base objects we use today we a bit more normalized and encapsulated.> > Anyway, I wanted to at least let folks know where my head was at, and> what I'm planning on focusing in the near term (given time as the> biggest limited resource). If anyone else wants to dig in on what's> left on nhibernate to do, I'd be happy to point you in the right> direction. I've learned a lot about the way the system works, and most> of the gotchas. Finishing it off isn't rocket science, it's just time.> > -Sean> > -- > Sean Dague / Neas Bade> sdague at gmail.com> http://dague.net> > 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20080829/ae3aaad1/attachment-0001.html>