[Opensim-users] Announcement of inventory tool (MyInventory), mostly of interest to grid operators/grid nauts
Snowcrash Short
snowcrash.short at gmail.com
Fri Nov 16 07:34:54 UTC 2012
On Fri, Nov 16, 2012 at 6:23 AM, Justin Clark-Casey <
jjustincc at googlemail.com> wrote:
> On 15/11/12 11:31, Snowcrash Short wrote:
>
>> Yes, adding the entries will make it easier to access the backing
>> web-services, although there isn't that much of a
>> performance gain going directly to the web-services, it does make it
>> possible for MyInventory to avoid duplicating
>> already existing assets, imagine this scenario:
>>
>> Users A and B downloads an .iar file, both decide to upload the same
>> inventory item from the .iar file, using UDP/CAPS
>> there is no way to know if the asset already exists in the asset db of
>> the grid and/or specify the asset id, hence if
>> the asset store isn't "single instance" or "deduplicating", - which my
>> cursory read of the asset code in opensim
>> suggests to me it isnt' - two identical streams of binary data may be
>> stored in the asset database.
>>
>> Using direct access to the asset service it is possible to detect this
>> duplication and simply create inventory items
>> referring to an existing asset id. Thus avoiding duplication of assets.
>> Currently this is done using 'GET' since 'HEAD'
>> always seems to report 200.
>>
>
> Hi Snowcrash. There is an experimental deduplicating asset service in
> core called XAssetService [1]. This appears to work fine - I haven't had a
> chance to progress it since I wanted to think through how migration would
> (or would not) work.
>
> I noticed the code for an XAsset service, but also noted the comments in
the code. I sincerely hope to see it go live soon. The asset service in
MyInventory is "Deduplicating" or as I call it "single instance" and in my
testing I have encountered quite a few duplicates.
> My chief reason for doing this was that I was interested myself in
> providing an external optional "load iar" mechanism which did not rely on
> client interface operations and did not rely on exposing the internal
> service port. I was reluctant to provide another avenue for uploading data
> before some kind of default deduplication was in place.
>
> There is also an assets/<asset-id>/metadata URL available that would
> return asset ID if one is willing to rely on this. Unfortunately, this is
> not yet documented.
>
> That could be very useful, currently I'm using GET on the asset service in
order to determine if the asset already exists, which is a bit overkill,
ideally the web service would support "HEAD" but a meta data check
would/should work equally well.
I'm also interested in providing a license/doc mechanism in IARs/OARs,
> chiefly so that anybody can add doc a license that could be displayed on
> reloading the IAR/OAR. Unfortunately, stretching this into the main body
> of the system itself would be considerably more complicated.
>
In my implementation, the asset document is for most parts a small document
containing a reference to something I have named a "bytestream". The byte
stream has an array Creator/LicenceId properties which identifies the
license the creator has chosen (currently set to default to:
"proprietary-closed".
>
> I would have to counsel anybody to think very carefully before opening up
> their internal asset and inventory services. As Diva says, it may be better
> to use the HG gatekeepers to these services though that may involve
> compromise.
>
> I'm not terribly familiar with the terminology used by the core, to
somebody with my intelligence or lack thereof the number of interfaces and
their differences are a bit bewildering (but hey, my code is bewildering
too). The webservices I rely on can be found in the config files under
[AssetService] and [InventoryService]. E.g.
AssetServerURI = "http://assets.osgrid.org"
InventoryServerURI = "http://inventory.osgrid.org"
as recorded in the GridCommon.ini config file distributed with the OSGrid
edition of open simulator.
The use of these URL's was a relative latecomer to the project once I
realized that the opensim asset store wasn't single instance. If - and that
is a big if - this tool becomes popular it could lead to a lot of duplicate
assets, I decided that MyInventory needed a way to lessen the impact it
will have on the infrastructure servers backing opensim.
I'm very happy to see experimentation in this field because it's a major
> issue. And it's up to individuals to make these decisions. But I would
> say OpenSimulator itself will remain with default config options very
> similar to what we have now (e.g. no service exposure).
>
> On a final note - it's will be much easier if you make your code
> BSD/MIT/Apache before releasing for interaction with existing codebases
> with a similar license. AGPL is a wildly different license.
>
> Once the code is more stable the licence will be modified to BSD. AGPL was
selected by me in the hopes that any changes would flow back to the
original project (I know it doesn't have to) and prevent wild forking until
the dust has settled a bit. I know that this won't deter the hardcore black
hatters, but they already have quite a toolset at their disposal.
[1] http://opensimulator.org/wiki/**Feature_Proposals/**
> Deduplicating_Asset_Service<http://opensimulator.org/wiki/Feature_Proposals/Deduplicating_Asset_Service>
>
>
>> best regards
>> Snowcrash
>>
>>
>> On Thu, Nov 15, 2012 at 11:55 AM, Melanie <melanie at t-data.com <mailto:
>> melanie at t-data.com>> wrote:
>>
>> Apparently, the extra entries are for more efficient upload. Unless
>> a grid specifies the policy they want, open download and the
>> inefficient upload mechanism will be in effect for it.
>>
>> Melanie
>>
>> On 15/11/2012 11:32, InuYasha Meiji wrote:
>> > Sorry, this is what I get for writing email at 5:31am. So you are
>> > saying I would have to give you the gridinventory service link in
>> order
>> > to choose to suport this service. Ok then I guess I am goo dfor
>> now,
>> > just won't give that info out. Sorry I got confused.
>> >
>> > Inu.
>> >
>> >
>> > On 11/15/2012 4:44 AM, Snowcrash Short wrote:
>> >> Hi
>> >>
>> >> I've been working on a client side tool for decentralizing user
>> >> inventories, which I will release as an open source tool in two
>> weeks,
>> >> some of the features may be relevant to grid operators.
>> >>
>> >> The basic premise of the tool is that the inventory and the
>> backing
>> >> assets of the inventory items really should be controlled by the
>> user.
>> >> The tool is born out of a frustration of having visited a number
>> of
>> >> grids. Each visit to a new grid presents me with an empty
>> inventory,
>> >> and I can then spend time searching for suitable item, clothing,
>> >> attachments and other accessories.
>> >>
>> >> For this purpose I have created a tool which will allow me to
>> backup
>> >> my inventory to a local cache and then upload the contents to
>> another
>> >> grid.
>> >>
>> >> If my tool becomes popular, both the upload and download
>> mechanisms
>> >> may have some impact on the grid-operators, hence this email to
>> serve
>> >> as a notice.
>> >>
>> >> The basic architecture is pretty simple, consisting of a number of
>> >> import agents, which can import the users inventory and backing
>> assets
>> >> to a local database, and a number of upload agents which can
>> upload
>> >> inventory content to a specific account.
>> >>
>> >> Backup/Import
>> >> There are two import agents, one which will import .iar files and
>> one
>> >> which works very much like I believe "Stored Inventory" works,
>> which
>> >> can backup the inventory of an avatars inventory. Avatar
>> backup/Import
>> >> is governed by a policy. Currently there are two policies,
>> >> one complying with a very restrictive interpretation of the Linden
>> >> Labs policy on backups, and a completely unrestricted policy,
>> where
>> >> anything that can be downloaded will be downloaded.
>> >>
>> >> When a new account is registered in MyInventory it checks if the
>> >> account is for a Linden Lab grid and limits the choices of
>> policies to
>> >> policies suitable for LL's TOS, I cannot and do not know if other
>> >> grids have similar policies, I can well imagine that Avination
>> has a
>> >> similar restrictions, and would like similar logic implemented to
>> >> restrict the download. Any grid operator which would like to have
>> >> backup governed by a more restrictive policy are invited to
>> notify me
>> >> and I will attempt to implement the policy prior to the first
>> release
>> >> of the source code. or supply patches at a later time.
>> >>
>> >> Upload/Export
>> >> MyInventory supports two mechanisms for uploading inventory
>> >> content, traditional upload using UDP/CAPS and direct access to
>> the
>> >> inventory and asset web-services.
>> >> Due to limitations in the UDP/CAPS protocol each upload will
>> create
>> >> new assets, and as of my latest read of the Open Simulator code
>> the
>> >> asset store does not support "single instance assets", i.e. it
>> does
>> >> not use a checksum to verify if the asset already exists, for this
>> >> reason MyInventory prefers to upload using direct access to asset
>> and
>> >> inventory web-services.
>> >>
>> >> I would propose that the grids which chooses to support
>> MyInventory
>> >> augment their "GridInfoService" entries with the url's for the
>> asset
>> >> and inventory web-services, e.g.
>> >>
>> >> [GridInfoService]
>> >> assets = http://assets.osgrid.org
>> >> inventory = http://inventory.osgrid.org
>> >>
>> >> Best regards
>> >> Snowcrash
>> >>
>> >>
>> >> ______________________________**_________________
>> >> Opensim-users mailing list
>> >> Opensim-users at lists.berlios.de <mailto:Opensim-users at lists.**
>> berlios.de <Opensim-users at lists.berlios.de>>
>>
>> >> https://lists.berlios.de/**mailman/listinfo/opensim-users<https://lists.berlios.de/mailman/listinfo/opensim-users>
>> >
>> >
>> ______________________________**_________________
>> Opensim-users mailing list
>> Opensim-users at lists.berlios.de <mailto:Opensim-users at lists.**
>> berlios.de <Opensim-users at lists.berlios.de>>
>>
>> https://lists.berlios.de/**mailman/listinfo/opensim-users<https://lists.berlios.de/mailman/listinfo/opensim-users>
>>
>>
>>
>>
>> ______________________________**_________________
>> Opensim-users mailing list
>> Opensim-users at lists.berlios.de
>> https://lists.berlios.de/**mailman/listinfo/opensim-users<https://lists.berlios.de/mailman/listinfo/opensim-users>
>>
>>
>
> --
> Justin Clark-Casey (justincc)
> OSVW Consulting
> http://justincc.org
> http://twitter.com/justincc
>
> ______________________________**_________________
> Opensim-users mailing list
> Opensim-users at lists.berlios.de
> https://lists.berlios.de/**mailman/listinfo/opensim-users<https://lists.berlios.de/mailman/listinfo/opensim-users>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-users/attachments/20121116/bc3feddf/attachment.html>
More information about the Opensim-users
mailing list