[Opensim-dev] Proposal to eliminate the name, description and invType fields from the assets db

[Justin Clark-Casey]

Stefan Andersson wrote:
> Again, I don't see why we need to have assetId === sha-1 hash - if the 
> only thing we want to accomplish with it is de-duping, we just make sure 
> to send the hash along, and any context can choose to use it to de-dupe 
> or not.
>  
> And to trust it or not.
>  
> Obviously, you wouldn't trust an assetId generated by an 
> untrusted source to represent the asset binary anyway, so quite a lot of 
> re-hashing and checking will be done along the way anyway.

I think that whether there's a trust issue or not depends on where you 
calculate the hash.  If it's calculated at the asset service then 
there's no issue.  Even if it's calculated externally, then you could 
always force a recalculation if there appears to be a hash collision 
when an upload is attempted.

I do think that the dupe issue is going to loom larger if we were to get 
to an intergrid architecture of federated asset databases with lots of 
people loading in material at different points.  The number of 
duplicates for a popular texture, say, would steadily increase over time 
without any realistic way of co-ordinating reaping within the federation.

I know flexibility is our watchword, but I wonder if this isn't one 
design point where making efforts to be as bendy as possible isn't too 
costly in terms of adding extra layers of indirection or passing more 
information around a network than is really needed.

Another good thing about asset uuid hashes is that they can potentially 
make the uuids disappear into the background when integrating with 
external programs (and the rest of the web).  For instance, suppose that 
a user wants to design a complex object or entire scene in some external 
program, and periodically upload different versions to their region 
server (grid connected or not).  If our hashes are random, either the 
user or their program has to worry about generating new random ones for 
each changed prim in their building/vehicle/elaborate clothing design. 
For OpenSim to be reliable we have to assume that they will forget and 
attempt to upload changed assets with existing uuids.  OpenSim might 
then need to drag down large amounts of existing data from the asset 
service to run internal hashing to know whether it needs to generate new 
uuids or not, or force every single imported prim to have a new uuid 
(which will really push up the dupe levels).

If asset uuids are hashes, then they can disappear from the user and 
program's view.  Outside of OpenSim, there is no need to manipulate 
uuids in this scenario - collections of objects or scenes can be 
structured in any way that the external program pleases.  Instead of 
getting asset uuids on import, the importing process calculates the 
necessary uuids from data, and can ask the asset service whether these 
already exist without needing to actually obtain the blobs.

I think that it's possible to experiment with the hashing idea without 
requiring the whole system to change.  Indeed, because of the very 
import scenario outlined above, I may well try using it for importing 
object archives.  This shouldn't impact the rest of the system since, of 
course, the chances of collision are small.

> 
> Best regards,
> Stefan Andersson
> Tribal Media AB
>  
> Join the 3d web revolution : http://tribalnet.se/
>  
> 
> 
> 
>     ------------------------------------------------------------------------
>     Date: Mon, 23 Jun 2008 15:59:39 -0400
>     From: teravus at gmail.com
>     To: opensim-dev at lists.berlios.de
>     Subject: Re: [Opensim-dev] Proposal to eliminate the name,
>     description and invType fields from the assets db table
> 
>     I'm not convinced the SHA1 UUID will be extensible enough to allow
>     for future plans.   The idea is to not assume anything with a UUID. 
>     Adding assumptions increases scalability at the cost of extensibility.
>      
>     Best Regards
>      
>     Teravus
> 
>      
>     On 6/23/08, *Melanie* <melanie at t-data.com
>     <mailto:melanie at t-data.com>> wrote:
> 
>         Hi,
> 
>         Sean Dague wrote:
>          > I don't understand how your approach deals with sniffing the
>         wire or the
>          > cache, finding the asset UUID, and just embedding that in you
>         items or
>          > scripts.
> 
>         It doesn't. Because that approach is not enjoying blanket coverage
>         in the DMCA. Means, in the case of UUID sniffing, each _item_ is a
>         separate violation, subject to a separate takedown order. Yes, one
>         per prim!
> 
>         If the texture is re-uploaded, blanket protection applies.
> 
>         The law isn't always common sense - in fact, it mostly diverges from
>         common sense!
> 
>          > Anyway, it's a little moot for right now, as the current
>         system is
>          > there.  But I really think the way forward involves us using
>         a hashing
>          > strategy.  Once we dig in there, we'll see how hard it would
>         be to make
>          > UUID generation runtime configurable.  And keep this in mind.
> 
>         I'm happy if that concern is not forgotten. That is all I wanted,
>         recognition of this as a valid concern, so no decisions are made
>         that could ultimately lead to an entire grid being judicially
>         shut down.
> 
> 
>         Melanie
>         _______________________________________________
>         Opensim-dev mailing list
>         Opensim-dev at lists.berlios.de <mailto:Opensim-dev at lists.berlios.de>
>         https://lists.berlios.de/mailman/listinfo/opensim-dev
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev


-- 
justincc
Justin Clark-Casey
http://justincc.wordpress.com