Chat log from the meeting on 2008-02-17

Revision as of 04:27, 21 May 2009

 

Home

 

Download

 

News

 

Support

 

Admin

 

Dev

 

Screen Shots

 

Grid List

 

Bugs

Languages:

English

Justincc 14:08, 19 February 2008 (PST)

[10:59] Charles Krinkeb: Morning, Chillken, Justin
[11:00] Charles Krinkeb: hi, Nebadon
[11:00] You: Hello Charles, Chillken
[11:00] You: Finrod, cua
[11:00] Finrod Meriman: hello
[11:00] cua Curie: Hello )
[11:01] Charles Krinkeb: 8 avatars, 975VIRT, 859RES, 25% CPU, 86% MEM
[11:01] Nebadon Izumi: hmm
[11:01] Charles Krinkeb: r3526
[11:01] Nebadon Izumi: look at the Description of Wright Plaza
[11:01] You: I see the memory issue remains
[11:02] Nebadon Izumi: looks like Wright Plaza is suffering the (nobody) or (waiting) owner also
[11:02] Nebadon Izumi: same as my region
[11:02] You: I see Estate: Plaza Builder
[11:02] You: same for Estate Owner
[11:03] You: Hello Ter. Is it just me, or has everyone adopted my unique clothing style?
[11:04] cua Curie: afk a moment
[11:04] Teravus Ousley: heh
[11:04] Finrod Meriman: hey teravus
[11:04] Nebadon Izumi: hmm
[11:04] Finrod Meriman: anything else interesting happen at AWG?
[11:04] Nebadon Izumi: crash #3 for me
[11:05] Teravus Ousley: yes, talked breifly about what we didn't talk about
[11:05] Hiro Protagonist: butt-load of Linden peeps there :)
[11:05] Teravus Ousley: like what happens when you log-out
[11:05] Finrod Meriman: i saw that there were a bunch of lindens there
[11:05] Hiro Protagonist: the real meeting is probably happening now lmao
[11:05] Finrod Meriman: most of ice house
[11:06] Charles Krinkeb: 10 avatars, 1159VIRT, 848RES, 28%CPU, 85%MEM. No scripts will run.
[11:06] Finrod Meriman: should i be able to create new shapes etc?
[11:06] Hiro Protagonist: create them in inventory first Fin
[11:06] Ahzzmandius Werribee: k, i'm present now.
[11:06] Hiro Protagonist: then wear and mod
[11:06] Finrod Meriman: yeah... i have been
[11:06] Finrod Meriman: nothing showing in the inventory after the create
[11:06] Teravus Ousley: Yeah the memory point and the crash point are probably going to be the topics today
[11:07] You: Hello MW
[11:07] More Wright: hey
[11:07] You: looks liek we're lagging already
[11:07] Teravus Ousley: .. probably among the things that would be great to accomplish.. is common threads amongst the crashes and mem issues.
[11:07] More Wright: hmm , this is the first time in quite a while, I've seen the grey ruths
[11:07] Nebadon Izumi: yea i doubt the sims gonna be stable for long
[11:07] Nebadon Izumi: yea everyone is grey for me too
[11:08] Charles Krinkeb: How shall we proceed today?
[11:08] Ahzzmandius Werribee: there's 5 clothed avies. 5 gray for me.
[11:08] Neas Bade: hey all
[11:08] You: Teravus, do you have an idea what the common threads are (if any)?
[11:08] You: hey Neas
[11:08] More Wright: I think we should proceed by testing Nebadon's hippo launcher
[11:09] Neas Bade: Neb didn't write a bus launcher yet?
[11:09] Teravus Ousley: lol
[11:09] More Wright: no that is next
[11:09] Nebadon Izumi: hehe
[11:09] Teravus Ousley: I don't know. Has anyone done any testing to see if the crashes occur under POS or basic physics?
[11:09] Nebadon Izumi: i tried making a hippo launcher last night
[11:09] You: I think we have an unhealthy obsession with busses
[11:09] Neas Bade: pending downloads is going through the roof here
[11:09] Nebadon Izumi: but forgot that linked stuff doesnt turn physical well
[11:09] Ahzzmandius Werribee: sme here on pending.
[11:09] Ahzzmandius Werribee: 67k
[11:10] Charles Krinkeb: Would I be correct in assuming that regaining stability and script functionality is probably fairly important?
[11:10] You: charles: yes
[11:10] Neas Bade: interesting thing I saw this morning, use of osSetDynamicTextureURL is awesome, but causes an interesting issue
[11:10] Neas Bade: which is that we never clean in memory cache
[11:10] Nebadon Izumi: also notice how this region shows no Owner on the About Land general tab
[11:10] You: neas: yes, that's true
[11:10] Nebadon Izumi: this prevents the owner from parceling
[11:10] Nebadon Izumi: and setting media URL
[11:10] Neas Bade: so those cause one of the memory run aways
[11:10] Nebadon Izumi: and pretty much all aspects of sim managment
[11:11] Neas Bade: Justin: that's why tardis crashed this morning
[11:11] You: Nebadon: I see Plaza Builder as the owner
[11:11] Nebadon Izumi: really
[11:11] Charles Krinkeb: Wonder why the owner is not displaying anymore?
[11:11] You: Neas: ah ha - normally it is not such an issue, but it would be with you
[11:11] Nebadon Izumi: i see (nobody)
[11:11] Nebadon Izumi: i see this in my region too
[11:11] Nebadon Izumi: and i cant do anything in that region management wise
[11:11] You: Even now, wright Plaza only has about 20mb of asset data in memory
[11:11] Neas Bade: I had 12 people pounding on surfaces that were loading textures of webpages that I was scraping through a little rails app
[11:11] Nebadon Izumi: hey Tedd
[11:11] You: nebadon: interesting, we have different experiences
[11:12] Nebadon Izumi: yea very Justin
[11:12] Tedd Maa: maaahn... here are soooo many chicks
[11:12] Nebadon Izumi: lol
[11:12] You: hi Tedd
[11:12] Teravus Ousley: Hello :D
[11:12] You: Could Pending Downloads be connected with our memory isuse?
[11:12] You: For some reason I thought it was just a rogue stat, but perhaps not
[11:13] Tedd Maa: there, I'm a man again
[11:13] Teravus Ousley: It's possible.. the count gets decreased when the 'texture complete' callback gets called
[11:13] You: if we're never completing then we would probably hold on to that object in memory forever
[11:13] Teravus Ousley: it gets increased when a texture request starts
[11:13] Nebadon Izumi: interestingly wen people log i i see them with colors
[11:13] Nebadon Izumi: then a few minutes later
[11:13] Nebadon Izumi: they turn grey
[11:13] Teravus Ousley: yep, we're at about crash swap zone
[11:13] You: yep
[11:13] Ahzzmandius Werribee: ;-p
[11:14] You: anybody have any info on the current segfault issue?
[11:14] Neas Bade: woot drift
[11:14] Neas Bade: justin, I just pulled trunk again, seeing how we are doing with it
[11:14] Neas Bade: I haven't crashed yet, but I'm still at < 30 minutes run time
[11:14] Neas Bade: it tends to happen in the 30 - 60 realm
[11:15] You: yeah, I'll have to leave my sim running for a long so that I experience your pain
[11:15] Charles Krinkeb: The bad news is my client died. The good news is my client died.
[11:15] Nebadon Izumi: woot crash #4
[11:15] Teravus Ousley: Well, if anyone could test to see if we get the segfault issue on Basic or Pos physics, that would help me some.
[11:15] Charles Krinkeb: At least the sim is still running.
[11:16] Teravus Ousley: :D
[11:16] Chris D: I was having segfaults with r3505. Since updating to r3522 - r3529, no segfaults so far.
[11:16] More Wright: charles, the db persistance of avatar appearances isn't enabled in this region is it?
[11:16] You: The fact that we've seen it both on Windows and Linux suggests that it isn't vm related
[11:17] Ahzzmandius Werribee: are the sims that are getting the segfaults using SQLite?
[11:17] Charles Krinkeb: I dont remember. Do you remember the .ini setting?
[11:17] Chris D: I use MySQL.
[11:17] Tedd Maa: ah.. I'm not using sqlite and don't get segfaults...
[11:17] Neas Bade: Ahzz, both mysql and sqlite
[11:17] Ahzzmandius Werribee: ok. the SQLite driver appears to NOT be thread safe.
[11:17] Tedd Maa: I onyl get segfaults during shutdown
[11:17] More Wright: appearance_persist =
[11:17] Ahzzmandius Werribee: I ran into that during abuse testing and that resulted int he inventory table patch for SQLite.
[11:17] Mircea Kitsune: hello everyone :)
[11:17] Stef Fawcett: Hi everyone.
[11:18] Charles Krinkeb: Mw. Shall I set it to true after this meeting?
[11:18] You: Ahzzmandius: yeah, that was good
[11:18] Teravus Ousley: hello
[11:18] Hiro Protagonist: hello MK, Stef
[11:18] Neas Bade: right, but those races tended to show up as exceptions
[11:18] You: hi stef
[11:18] Ahzzmandius Werribee: tedd, I see that as well, during mdb runs it pointed at attempts to write to the console after log4net was shutdown.
[11:18] Tedd Maa: SQLite doesn't work on Vista64 either, so we shouldn't use it... (Powerreasoning)
[11:18] Neas Bade: this has no exception
[11:18] Hiro Protagonist: Ahzz: I've actually seen that on the console before
[11:18] You: yes, it's those thread aborts
[11:18] Teravus Ousley: .
[11:18] Ahzzmandius Werribee: and I continually see times where something writes after log4net was stopped and it faults.
[11:18] Neas Bade: yeh, that's the latest issue. We need to track down where a log4net close call is
[11:18] Nebadon Izumi: sigh
[11:18] You: possibly we should allow thread to exit 'naturally'?
[11:19] Tedd Maa: no, I think its my fault... ScriptEngine does some descructor logging
[11:19] You: threads
[11:19] Ahzzmandius Werribee: justin, just set it as background and on program exit it'll get disposed.
[11:19] Neas Bade: it will get disposed on shutdown regardless :)
[11:19] Ahzzmandius Werribee: :)
[11:19] Hiro Protagonist: <osgridMonkey> 14:19:12 up 77 days, 19:56, 7 users, load average: 0.00, 0.04, 0.00 <osgridMonkey> OpenSim.Grid.UserServer.exe
[842] CPU%: 0.0 MEM%: 2.2 CPU time: 0:05 <osgridMonkey> OpenSim.Grid.GridServer.exe
[869] CPU%: 0.0 MEM%: 2.0 CPU time: 0:05 <osgridMonkey> OpenSim.Grid.AssetServer.exe
[884] CPU%: 0.0 MEM%: 3.3 CPU time: 0:10 <osgridMonkey> OpenSim.Grid.InventoryServer.exe
[933] CPU%: 0.0 MEM%: 4.8 CPU time: 0:14 <osgridMonkey> mono
[8946] CPU%: 0.0 MEM%: 0.0 CPU time: 0:00
[11:19] Charles Krinkeb: Anyone *not* notice the fountain stopped?
[11:19] You: Ahzzmandius: these are threads that need to be disposed of while running
[11:20] Nebadon Izumi: yea its stooped
[11:20] Nebadon Izumi: stopped
[11:20] More Wright: charles, no in grid mode, leave avatar appearance turned off (set to false), hope to have it working for grid mode soon
[11:20] Nebadon Izumi: no scripts are running here at all
[11:20] Ahzzmandius Werribee: Justin, k.
[11:20] Neas Bade: I think the log4net bug is actually someone shutting it down manually, hwich shouldn't happen
[11:20] Hiro Protagonist: 24 main agents
[11:20] Neas Bade goes hunting
[11:20] Neas Bade: 24 main agents is pretty good :)
[11:20] Ahzzmandius Werribee: :)
[11:20] Nebadon Izumi: stats show 108 active scripts though
[11:20] Hiro Protagonist: only count 15 though
[11:21] Teravus Ousley: heh, by now my packet limiting code will have kicked in :D
[11:21] Hiro Protagonist: 0 *
[11:21] You: oh yes, that's interesting, 108
[11:21] Hiro Protagonist: o ips tho
[11:21] Ahzzmandius Werribee: this mornings work on the outbound packet queues is showing some promise.
[11:21] Charles Krinkeb: 18 avatars with "show users" from the console.
[11:21] Ahzzmandius Werribee: I've gotten the packet flow harmonic broken up and better response times by directly sending outbound instead of going through the PacketQueue SendQueue.
[11:21] You: I think we should investigate the constantly increasing pending downloads
[11:21] Hiro Protagonist: time dialation and FPS are nothing short of outstanding, even by LL standards
[11:21] Ahzzmandius Werribee: on throttles that are free to send.
[11:21] Charles Krinkeb: 1593 VIRT, 743 RES, 52% CPU, 74% MEM, 0 Scripts.
[11:22] Neas Bade: MW, over the weekend I was thinking about inventory and asset things, and was wonderin about moving UUIDs to v5, which is SHA1 based instead of Random
[11:22] Neas Bade: I wanted to sort out what you thought about it, and get that out for discussion
[11:22] Ahzzmandius Werribee: Neas, what is the SHA1 input based on?
[11:22] Neas Bade: I was thinking that the UUID of assets should be the SHA1 of their contnet
[11:22] Ahzzmandius Werribee: won't that collide on identical assets?
[11:22] Neas Bade: so that if 2 assets were actually the same asset, they would have the same id
[11:22] Hiro Protagonist: interesting
[11:23] Neas Bade: that's sort of the point
[11:23] You: ahzzmandius: yes, which is what we want
[11:23] Finrod Meriman: there are a bunch of content addressable storage papers that touch on that
[11:23] You: ping
[11:24] Teravus Ousley: pong
[11:24] Charles Krinkeb: 18 avatars, 1875VIRT, 889RES, 6% CPU, 89%MEM, 0 Scripts
[11:24] You: neas: it certainly sounds like a good idea
[11:24] Ahzzmandius Werribee: hm
[11:24] Krisbfunk Nocturnal: definitely some avatars sinking and floating here
[11:25] Teravus Ousley: yep

At this point the sim crashed as it was out of memory. The main discussion continued on IRC

[19:25] <Sir_Ahzz> O_o holy floating avatars batman
[19:25] <nebadon> i dont ever recall crashing on my or others regions
[19:25] <daTwitch> maybe the number of prims?
[19:25] <daTwitch> how many prims there these days?
[19:25] <nebadon> not sure
[19:25] *** krtaylor quit (Read error: 110 (Connection timed out))
[19:25] <nebadon> i cant get in
[19:25] <nebadon> i think the sim is hosed
[19:26] <daTwitch> I'm damped
[19:26] <Ter_afk> I would have to agree based on what I see
[19:26] <daTwitch> locked in type anim
[19:26] <sdague> it's taking > 1 minute for a chat message to go through
[19:26] <justinccWork> charles - time to reboot?
[19:26] <ckrinke> I think we went over the edge.
[19:26] <sdague> it's still alive, but holy crap barely
[19:26] <Sir_Ahzz> *sigh*
[19:26] <ckrinke> I will restart
[19:26] <Sir_Ahzz> and I so wanted to ask about type 5 and my idea for intergrid. 8-(
[19:27] <sdague> :)
[19:27] <ckrinke> er, maybe. CPU=262%, having trouble getting bash control.
[19:27] <sdague> well, one sec
[19:27] <sdague> heh
[19:27] <Sir_Ahzz> lol
[19:27] <daTwitch> ruhroh
[19:27] <sdague> yeh, just segfaulted my test environment as well
[19:27] <ckrinke> May as well continue here. I have to wait for the bash to become available.
[19:27] <Sir_Ahzz> killing my client.
[19:28] <Tedd1> renice opensim on startup .. it will have approx same amount of CPU to play with, but bash will work :)
[19:28] <Sir_Ahzz> ok, will the type 5 using sha1 preclude my using the mac field as a routing mechanism?
[19:28] <justinccWork> Ter_afk: I'm going to prioritize investigating the pending downloads issue above making inventory squirts on region crossings async
[19:28] <justinccWork> but really we need to profile
[19:28] <ckrinke> restarting the plaza.
[19:29] <Ter_afk> ok. have a look at where it increases and decreases the count. Make sure I'm doing it right.
[19:29] <Ter_afk> I think I am, however I'm open to having made a mistake
[19:29] *** krisbfunk_ (n=chatzill@postgresql.vre.upei.ca) joined
[19:29] <_MW> hmm, not sure if we don't need to allow duplicate asset data but with different id, think we have spoke about this before, and there are some cases where its required, but can't think right now ( a bit distracted with family things now)
[19:29] <ckrinke> Well, we got 18 inworld.
[19:30] <justinccWork> _MW: those would be very intersting cases to know, since on the surface one wouldn't think so...
[19:30] <Tedd1> if we could have a way of choosing both it would be great ... at least in scripting terms to be able to initialize same .dll twice
[19:30] <Ter_afk> TextureDownloadModule, TextureRequest to add 1 to the count, TextureSent to decrease the count by 1
[19:30] <sdague> ok, so scripting is an interesting case
[19:31] <justinccWork> Ter_afk: it wouldn't surprise me if the problem isn't more fundamental and your stats are okay
[19:31] <_MW> yeah, maybe I'm mis-remembering, the first thought was things like when you want to allow different users to own their own copy of a base item. like clothes item, but then I remember that would be done through inventory and as soon as one person modified it they would get a new asset item created, so it didn't effect the ohers
[19:31] <_MW> so I guess I remembering wrongly
[19:32] <sdague> right, if it is modified, then it automatically gets a new id
[19:32] <justinccWork> _MW: yep
[19:32] <_MW> yup
[19:32] <sdague> because it is sha1 of content
[19:32] <Tedd1> for objects that would require that (if you thing in SL terms) it would be great ... anything you use twice or more (lamps, chairs, bullets for guns, etc) could use same code and share memory (even L2 cache)
[19:32] <sdague> right, plus it means you don't fetch the same thing multiple times on the network
[19:32] <Tedd1> *think in SL terms
[19:32] <Sir_Ahzz> wait, but if that is identified by content, wouldn't having two copies of a chair in two positions make that two items automaticly?
[19:32] <Tedd1> yeah, that too
[19:33] <Tedd1> and won't have to initialize or compile it twice
[19:33] <justinccWork> well, to be honest we should be covering such situations already
[19:33] <justinccWork> all those chairs should be referring to the same asset chair
[19:33] <justinccWork> though in practice this may not currently be the case
[19:33] <sdague> Sir_Ahzz: no, because the asset is different from the prim metadata
[19:33] <_MW> but doesn't sometimes the client generate the id's of items it created and uploaded, I really can't remember if it does (damn my memory)
[19:33] <Sir_Ahzz> ah
[19:33] <sdague> honestly, the big win ends up in texture space
[19:33] <Sir_Ahzz> gotcha.
[19:33] <justinccWork> _MW: I haven't seen that personally
[19:33] <sdague> _MW: you may be right, I'm not sure
[19:33] <Sir_Ahzz> I'm not familar enough with asset vs prim vs inventory just yet then.
[19:34] <sdague> it's one of the reasons to stick that out here as a thought
[19:34] <sdague> I'm also going to poke Zero about it for Linden space
[19:34] <sdague> he may have more reasons why it won't work, but I'd like to figure out if there is any truly unworkable reasons there
[19:34] <Sir_Ahzz> is it possible to still reserve the mac bits in the UUID for my routing idea?
[19:34] <justinccWork> sdague: remind me, how is this good for textures?
[19:34] <_MW> yeah guess I'm thinking of the transaction ids , err but aren't some items ids meant to be the transactions ids combined with the users secureSessionID, remember that is used somewhere, just was so long ago
[19:35] <sdague> that's type2, right?
[19:35] <justinccWork> _MW: oh good point.....
[19:35] <Sir_Ahzz> sdague: the mac? type 1.
[19:35] <Tedd1> it would affect (future?) viewer, because it would only need shapes/textures once in gfx mem
[19:35] <justinccWork> _MW: yeah, I saw that in the code just recently
[19:35] <sdague> justinccWork: because if the texture is really the same, but multiple people have it, then it is still one thing
[19:35] <Sir_Ahzz> all UUIDS have the reserved space for type identification.
[19:35] <sdague> you don't get copies
[19:35] <sdague> right, this would be type5
[19:35] <justinccWork> sdague: oh, so these would be independent uploads?
[19:36] <_MW> I have to run, will try to get back in a bit
[19:36] <sdague> Sir_Ahzz: if the sha1 of 2 assets is the same, they are the same thing content wise
[19:36] <sdague> I'm not sure why you would care about MAC
[19:36] <Sir_Ahzz> ok, but I have to be able to extract that MAC id in order to do the routing. 8-(
[19:36] <Ter_afk> presumably you would also have permissions stored there.. or have a record for each object with a set of permissions anyway.. . wouldn't the avatar owner's UUID be stored anyway?
[19:37] <Sir_Ahzz> I wanted to use the mac as a sort of "source" locator. so we dont' have to add extra data to a UUID beyond the 128 bits for locating the source.
[19:37] <sdague> Sir_Ahzz: I think you start getting into an issue there where you've removed too many bits
[19:37] <sdague> then you have to actually do type2 allocators
[19:38] <sdague> otherwise you'll get collisions
[19:38] <Sir_Ahzz> there's 2^48 unique mac addys available in a UUID type 1
[19:38] <justinccWork> Is Tedd around?
[19:38] *** krisbfunk quit (Connection timed out)
[19:38] <Tedd1> yes
[19:38] <justinccWork> any idea why scripts have stopped working on wright plaza (putting my ckrinke mask on for a moment)
[19:38] <Sir_Ahzz> hm. if the type1 is followed, then colissions are eliminated due ot the timestamp.
[19:38] <Sir_Ahzz> but that precludes content based id.
[19:38] <Sir_Ahzz> *sigh*
[19:38] <sdague> yes, but sha1 is type5
[19:39] <sdague> it needs all but the reserved bits
[19:39] <Sir_Ahzz> right, which takes over the entire 128 bits if memory is right.
[19:39] <Tedd1> justinccWork; and they haven't been stopped?
[19:39] <sdague> otherwise you don't have enough bits for the hash
[19:39] <Sir_Ahzz> and that would definitly preclude storing the source ID in there.
[19:39] <nebadon> Tedd1 positive they arent stopped
[19:39] <justinccWork> Tedd1: I don't believe so
[19:39] <sdague> yes
[19:39] <Sir_Ahzz> which means routing asets would have to have extra data attached to a UUID makign it larger than 128bits.
[19:40] <ckrinke> Scripts on Wright Plaza have not been stopped. They all refuse to run after r3503
[19:40] <Tedd1> hmm.. so they just don't execute? I need to add some debug messages then to figure it out
[19:40] <sdague> yes, or you have a reverse directory
[19:40] <sdague> basically just do bittorrent kinds of things
[19:40] <sdague> "who has this bit"
[19:40] <nebadon> im wondering if its a result of some of the UDP changes maybe?
[19:40] <Sir_Ahzz> there's jsut no way around having portable assets without expanding the size of uuid with sha 8-(
[19:41] <Sir_Ahzz> and that would seriously break things in the protocol.
[19:41] <nebadon> what will happen to existing assets if we change UUID formats?
[19:41] <justinccWork> nothing
[19:41] <nebadon> cool
[19:41] <justinccWork> errrrr
[19:41] <Sir_Ahzz> neb slim chance of collision.
[19:41] <justinccWork> actually allows some synapses to fire
[19:41] <nebadon> just how new UUIDs are generated?
[19:41] <Sir_Ahzz> but there's already that chance by using purely random uuids.
[19:41] <sdague> you'd probably need to run a script over them to move to new UUID format
[19:41] <justinccWork> probably nothing
[19:42] <sdague> Sir_Ahzz: if SHA1 has random collisions in it, all of cryptography breaks
[19:42] <nebadon> ouch
[19:42] <sdague> nebadon: it could be done server side
[19:42] <Sir_Ahzz> sdague: i'm refering to a random generated uuid, not sha1
[19:42] <nebadon> yea
[19:42] <justinccWork> sdague: you can think of a place where change the UUIDs would require that?
[19:42] <nebadon> might take a while
[19:42] <sdague> right, but if you can randomly generate a UUID that is a SHA1 of something legitimate in the amount of time we've built assets in opensim, you've also broken all of cryptography :)
[19:42] <Sir_Ahzz> sdague: and yes, it's admited that sha1 DOES have that chance, and I vaguely recall a paper on attempts at exploiting that.
[19:43] <nebadon> what about using Binary16
[19:43] <nebadon> is that still going to be how we store them?
[19:43] <Sir_Ahzz> it's jsut that the chance of colission is so low that it's deemed "impossible"
[19:43] <sdague> they managed to weaken out like 8 bits of sha1
[19:43] <sdague> right, heat death of the universe impossible
[19:43] <Sir_Ahzz> basicly.
[19:43] <sdague> content addressable filesystems have existed for 30 years that use this model
[19:44] <justinccWork> it sounds like one could experimentally try this out just by replacing all the new LLUUID() calls
[19:44] <Sir_Ahzz> yes but the volume is nowhere near getting colission potential to a meaningfull level.
[19:44] <justinccWork> or altering the libsecondlife code underneath that
[19:44] <Sir_Ahzz> justin, yes.
[19:44] <sdague> yep
[19:44] <sdague> need to actually work up a type5 implementation
[19:44] <Sir_Ahzz> I was planning on creating a IUUID interface and attachign it to a new class that extends LLUUID
[19:44] <Sir_Ahzz> so you could easilly switch types as a plugin. ;)
[19:45] <Sir_Ahzz> sdague: would having a sha1 field in the asset for detecting identicals be acceptable comprimise to using it as the basis of the uuid?
[19:46] <Sir_Ahzz> so that we leave uuid generation type as selectable?
[19:46] <Sir_Ahzz> and not prevent comapcting identical assets?
[19:46] <sdague> Sir_Ahzz: one of the reasons that you want it as the actual UUID field is that the client won't pull a UUID if it already has it
[19:47] <sdague> otherwise you still dupe on the network
[19:47] <Sir_Ahzz> just check against the local cache's sha1
[19:47] <Sir_Ahzz> the same as if you checked against the UUID
[19:47] <sdague> that requires client changes
[19:47] <Sir_Ahzz> hm.
[19:47] <sdague> and how is it going to know that?
[19:47] <Sir_Ahzz> could be caught server side on asset use
[19:47] <sdague> it requires protocol changes as well
[19:48] <sdague> you don't know what the client has
[19:48] <Sir_Ahzz> the server has to do all create/store yes?
[19:48] <Sir_Ahzz> who assigns the asset uuid on copyign a prim?
[19:48] <sdague> yeh, but it doesn't know if the client actually got things, or if it lost them and needs them again
[19:49] <sdague> so, difference between assets and some of the other meta data
[19:49] <Sir_Ahzz> I need more detail on the client/server relationship 8-(
[19:50] <Sir_Ahzz> I'll chase that info down after I finish up with my current work.
[19:50] <Sir_Ahzz> and get back to you on my thoughts.
[19:50] <sdague> Assets are 8 fields: UUID, Name, Description, a couple other fields that wouldn't matter here, and Data
[19:51] <justinccWork> are name and description really necessary?
[19:52] <Sir_Ahzz> i'm guessing that when you copy a prim in the sim, it creates two prim entities and 2 assets, the assets being identical.
[19:52] <Sir_Ahzz> you're trying to make those two prim entities share the same asset.
[19:52] <krtaylor_> also what about Local ID
[19:53] <Sir_Ahzz> the client requests the prim entity and the sset of each prim?
[19:53] <justinccWork> that will happen already
[19:53] <Sir_Ahzz> or just the prim and the asset comes with it?
[19:53] <justinccWork> both items point to the same entry in the asset database (in principle)
[19:53] <Sir_Ahzz> ok.
[19:54] <Sir_Ahzz> so where is the duplicate entity then that this sha makes into one?
[19:54] <Sir_Ahzz> because I"m not seeing it with what I know.
[19:54] <sdague> osSetDynamicTextureURL is a good instance
[19:54] <justinccWork> well, I'm assuming that if a client uploaded an image which already existed
[19:54] <sdague> yep
[19:54] <justinccWork> then we wouldn't create another one
[19:54] <justinccWork> or, yes, osSetDynamicTextureURL :)
[19:54] <sdague> or you rebake, and you are actually wearing all the same stuff
[19:54] <justinccWork> where you're uploaded a new webpage image every 15 seconds (hypotheetically)
[19:54] <Sir_Ahzz> but isn't that a distinctly new entity from the client's viewpoint?
[19:54] <Sir_Ahzz> different creation date
[19:55] <sdague> or the 1500 blank assets that are in my db
[19:55] <justinccWork> so if that webpage hasn't changed, you don't comit another asset to the db?
[19:55] <sdague> justinccWork: right
[19:55] <lbsa71> Is there office hours now?
[19:55] <sdague> though that doesn't actually go to the db now, just fills up the in memory cache
[19:55] <Sir_Ahzz> hm.
[19:55] <sdague> lbsa71: yeh
[19:55] <justinccWork> yes, tho all the tech talk is here
[19:55] <lbsa71> Did it start an hour ago?
[19:56] <sdague> Sir_Ahzz: at some point do a 'select name, count(name) from assets group by name order by count(name)'
[19:56] <justinccWork> afraid so
[19:56] <lbsa71> Argh
[19:56] <lbsa71> *sigh*
[19:56] <justinccWork> sdague: heh, the immortal memory cache :)
[19:56] <Sir_Ahzz> hm.
[19:57] <ckrinke> Thats Ok, Lbsa, your sage advice is greatly appreciated.
[19:57] <lbsa71> Uh?
[19:57] <lbsa71> On what topic?
[19:57] <nebadon> most of the meeting has occured here in iRC
[19:57] <Sir_Ahzz> sdague: ok, i'm goign to step out of the uuid discussion till I get a better understanding of their creation, assignment, etc.
[19:58] <lbsa71> Sage advice = opinionated geekiness?
[19:58] <krtaylor_> Local ID is used often instead of LLUUID when refering to prims, would that need to be rewritten?
[19:58] <Sir_Ahzz> localID is mapped to the uuid in region
[19:58] <Sir_Ahzz> hm.
[19:58] <nebadon> http://nebadon2025.com/opensim/viewtopic.php?f=3&t=9
[19:58] * Sir_Ahzz shakes his head. this hurts.
[19:58] <ckrinke> Nah, Lbsa. It looks like we have stability issues, crashes, memory leaks, scripts dont work anymore, all the usual.
[19:58] <justinccWork> krtaylor: don't think so _ i think those are only local to the client server session
[19:59] <lbsa71> If you're discussing assets, I'd say we should step back from being hell bent on having everything stored in one cache, and perhaps create micro-caches
[19:59] <lbsa71> or just keep local copies
[19:59] <ckrinke> But it was nice to have the water running in the fountain for a week or so.
[19:59] <Sir_Ahzz> :)
[19:59] <justinccWork> lbsa71: discussing sdague's idea of running a sha1 over asset data to generate uuids
[20:00] <Sir_Ahzz> so far as stability, i'm suspicious that the work i've been doing in UDP has triggered the exposure of existing race conditions with locks.
[20:00] <os_ant> A simple question: I must understand opensim+libsecondlife for my thesis, When I can start?
[20:00] <lbsa71> Well, actually, I'd say what we need is a) a directed effort and b) some process to give us piece of mind to execute it.
[20:00] <justinccWork> os_ant: Right away?
[20:00] <os_ant> I want understand about data structures, basic concept...
[20:00] <os_ant> justinccWork, ?
[20:01] <justinccWork> os_ant: you can start anytime you like, all the source code is open!
[20:01] <nebadon> i would say thats up to you os_ant
[20:01] <nebadon> we dont hold classes
[20:01] <nebadon> hehe
[20:01] <nebadon> everyone learns at their own pace
[20:01] <nebadon> start by reading the entire wiki
[20:01] <nebadon> http://www.opensimulator.org
[20:02] <ckrinke> Three Times
[20:02] <nebadon> lol
[20:02] <os_ant> ok... I means: there is a page, a tutorial... about libsecondlife data structures...
[20:02] <nebadon> libsecondlife is a seperate project
[20:02] <os_ant> ???
[20:02] <krtaylor_> that is slower for some of us than others...
[20:03] <os_ant> but I must understand libsecondlife before opensim, right?
[20:03] <ckrinke> Althought libsecondlife is a seperate project, we use the libsecondlife.dll, so understanding libsecondlife will help you here. Just jump into the source.
[20:03] <os_ant> opensim is based on libsecondlife
[20:03] <nebadon> depends on your goals
[20:03] <lbsa71> sdague what would be the central win of using sha hashes?
[20:03] <nebadon> libsecondlife is a viewer really
[20:04] <daTwitch> it's a very sexy thesis topic os_ant, everybody's doing it it would seem - but they have to do their own work
[20:04] <nebadon> if you want to start with libsecondlife visit their website
[20:04] <nebadon> http://www.libsecondlife.org/wiki/Main_Page
[20:05] <daTwitch> if you wish to understand opensim at a doctoral level, grokking libsecondlife first is not a bad idea
[20:06] <os_ant> for example: assets are stored in LLUUID that is part of libsecondlife, right?
[20:08] <justinccWork> LLUUID is just the identifier for the asset - opensim stores the actual data
[20:10] <Sir_Ahzz> _MW: *ping*
[20:10] <lbsa71> Ok, to comment on the sha approach; I think we should use different strategies for different content types _and_ contexts
[20:10] <Sir_Ahzz> lbsa sounds good.
[20:10] <lbsa71> the dynamictexture - should it really be persisted at all?
[20:11] <lbsa71> and, should it be cached in more than one version anyway?
[20:11] <lbsa71> and if not, why not just have that texture saved in a separate space?
[20:12] <lbsa71> One approach would be to have 'soft asset refs', ie, an asset type that forwards a request to another asset
[20:13] <lbsa71> so that an assets content could actually change
[20:13] <Sir_Ahzz> interesting
[20:13] <lbsa71> now, that's normally not a good idea, so the other approach would be to have 'layered' caches
[20:14] <lbsa71> I've already suggested that we have a chain of providers, so that we can go from having to load all assets, to being able to look them up on demand
[20:15] <_MW> problem with using asset refs/assets that can change data, is the client cache, having the contents of a asset change, will mean that different users will likely see different things (say for a texture), as there is no way to force the client to refresh a texture that it has in its cache
[20:15] <_MW> thats why if we want to change the texture image of a prim, we actually have to set a different id
[20:15] <lbsa71> ie, you'd first have a look in the in-mem cache; if there's nothing there, you look at the dynamic cache; if there's nothing there, you look in the local asset db, if not there, you look at local assetsets,
[20:16] <lbsa71> if not there, you look at remote db, which would probably do the exact same chain over again
[20:16] <lbsa71> _MW yeah, I know; hence my reluctance
[20:16] <lbsa71> But it's a fact, I think, that we need to turn the asset cache approach around
[20:16] <_MW> yeah I agree we want different caches/sources for the assets
[20:17] <lbsa71> it's just weird that we should have to load the whole asset set regardless of whether anybody actually are going to use them.
[20:17] <_MW> do we load all assets?, I guess we do in the db layer
[20:17] <lbsa71> The dynamic asset cache could have another set of keys, so that the texture/asset key is not the same key as is identifing the dynamic texture
[20:17] <justinccWork> isn't this is a fairly small case atm? We only load library assets
[20:18] <lbsa71> so, if you want to update it, you update the dynamic texture key, which removes the old version from the cache prior to adding a new one
[20:18] <lbsa71> that's an 'only' that takes like a minute or so of startup time; and for large-scale applications using large assetsets...
[20:19] *** kinoc quit (Read error: 110 (Connection timed out))
[20:19] <justinccWork> yeah, it's shit - but if we checked for whether they were already in the asset db we could eliminate that time
[20:19] <lbsa71> Also, at least last I looked, at least SQLite actually stores and commits the library assets?
[20:20] <lbsa71> Which is just weird
[20:20] <lbsa71> The whole thing should be on demand
[20:20] <lbsa71> with an escalation chain
[20:20] <justinccWork> putting them in the db means we don't put them into memory until they are demanded
[20:21] <+CIA-44> opensim: justincc * r3533 OpenSim/ (3 files in 3 dirs): Remove "Loading inventory" messages from item inventory loads
[20:21] <lbsa71> On another note; do we do local persistence cacheing on remote assets?
[20:21] <justinccWork> I like the sha1 idea because it's nice and simple
[20:21] *** krtaylor_ quit ("Leaving" )
[20:21] <_MW> I think what lbsa71 means is the sqlite provider reads the whole db into memory (or at least it did, not sure if thats still the case)
[20:21] <justinccWork> lbsa71: not yet, though I think we should
[20:21] <justinccWork> _MW: I didn't think it did
[20:21] <justinccWork> could be wrong though
[20:23] <lbsa71> Well, consider the case with baked avatar textures; you only need one, and you don't need to persist it.
[20:23] <lbsa71> So, something somewhere should have that reference to that baked texture
[20:24] <_MW> yeah doesn't look like it does now, so maybe I'm thinking of one of the other sqlite db providers (like region storage ) which used to (going back a number of months)
[20:24] <lbsa71> the weird thing is if we get a new baked texture, we still need the old one, in case there's transfers in progress
[20:24] <lbsa71> But we definitively should have some way of chucking non-reffed assets
[20:25] <lbsa71> non-reffed in-mem assets
[20:25] <justinccWork> if we move to sha1 the problem disappears for now
[20:25] <lbsa71> By the way, am I right in that the asset data does not contain the asset id?
[20:26] <justinccWork> when I've looked on osgrid, the ammount of memory actually being used by assets in cache is dwarfed by other things
[20:26] <lbsa71> yes, I know
[20:26] <lbsa71> but that's probably because no regions survive even 24 hours
[20:26] <daTwitch> justinCC: like what, if I may ask?
[20:26] <_MW> lbsa71, yeah generally the data doesn't contain the id (can't think of any cases where it does)
[20:26] <justinccWork> it was just an issue because of sdague's osTextureUrl (not quite right) coolness
[20:26] <lbsa71> so, one could, in theory, have a sha on the data safely
[20:26] <justinccWork> automatically updating a webpage on a prim every n seconds
[20:26] <daTwitch> ahhh
[20:26] <lbsa71> and have several assets ref the same data sha
[20:27] <justinccWork> daTwitch: yes, that's the rub :) not sure yet....
[20:27] <lbsa71> But, as I said, I think THE solution is to have the dynamic urls properly managed