[Opensim-dev] Proposal: Implement a de-duplicating core ROBUST asset service

[M.E. Verhagen]

2012/3/9 Wade Schuette <wade.schuette at gmail.com>

>  M.E.
>
> You provided a link to an excellent article with many great points and a
> compelling argument for keeping images in a separate file system based on
> both I/O and database considerations.     I yield to overwhelming
> experience on that.
>
> So,  how best to architect the format, indexing, and evolution of the
> many-terabyte file system for the particular situation in hand, virtual
> world assets?
>
> Omg many terrabyte file system, that is a lot. Hoever opensim databases
quicky grow to multiple gigabytes ...

And as the opensim worlds are user generated worlds, lots of people do not
really care about things like small textures or small meshes.

In the virtual words assets there are a few cases in wich a lot of data has
to be pushed. When users are opening their inventory and when they arrive
at a region. (in case of events sometimes 20+ people are arriving at the
same time)

When the assets are small in size, maybe less than a 100k then it would not
be issue if they are stored in a database. But when a database has to push
lots of mb's (a heavy textured and meshed sim can be a multiple of 100mb's)
than it would probably be better when the assets are fetched from an file
somewhere in a file system (maybe even on another server to preserver
bandwidth)

I think it would be good when large assets are stored in a filesystem, and
there is just a refence to that location.



> Given the localization, clustering and clumping nature of related
> textures,  it makes me wonder if a tree-structured index system, instead of
> a relational-database flat-table structure, would make the most sense --
> ie, some language like MUMPS.
>
> Has someone else already solved this type of  problem in the literature
> and practice?
>
> Your thoughts on that?
>
> Wade
>
>
>
> On 3/9/12 1:33 AM, M.E. Verhagen wrote:
>
> this is a bit of an old article:
> http://mysqldatabaseadministration.blogspot.com/2008/01/i-will-not-blob.html
>
>  qoute: 'The concept of databases is generally suited to storing a very
> large number of objects that are small in size. Here, both "very large" and
> "small" are relative to each industry and requirements of the system. When
> you look at overall picture, file systems are more suited to handling large
> objects, especially if the large object consists of an image.'
>
>
>
>
> _______________________________________________
> Opensim-dev mailing listOpensim-dev at lists.berlios.dehttps://lists.berlios.de/mailman/listinfo/opensim-dev
>
>
>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev
>
>


-- 
Groningen en Hannover Opensims:   secondlife://meverhagen.nl:8002:Hannover
ZW/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20120309/778bcc55/attachment-0001.html>