[11:01 AM] Andrew Hellershanks: Hello, everyone.
[11:02 AM] Andrew Hellershanks: Hm... Low attendance at the moment. I wonder if the winter storms have anything to do with it.
[11:03 AM] Joe Magarac: Hi all. Just checking in, but can't stay too long.
[11:03 AM] Joe Magarac: I'm still working on that region impostor thing. Results soon. Code here: https://github.com/John-Nagle/maptools
[11:04 AM] Joe Magarac: Takes way too much web server machinery to make it go.
[11:04 AM] Vincent.Sylvester @hg.zetaworlds.com: I keep telling you it's a waste of effort, but you do you
[11:05 AM] Joe Magarac: It looks kind of cool in SL. In OS, mainland isn't big enough.
[11:05 AM] Vincent.Sylvester @hg.zetaworlds.com: Cuga, any news on the crashes?
[11:06 AM] Cuga.Rajal @rajal.org:9000: No crashes since I updated to the lates code, with all the new socket stuff
[11:06 AM] Ubit Umarov: nice horse
[11:06 AM] Cuga.Rajal @rajal.org:9000: No way to tell if the offending request actually came in
[11:06 AM] Ubit Umarov: yeap code change hides it
[11:06 AM] Cuga.Rajal @rajal.org:9000: But if that crash never happens again its fine with me
[11:07 AM] Vincent.Sylvester @hg.zetaworlds.com: Running release code as a honeypot for it looks like still running so I must not be on the target list
[11:07 AM] Cuga.Rajal @rajal.org:9000: If I wanted to try to capture it I would need to use the build from Dec 8 or before
[11:07 AM] Ubit Umarov: well about last week code changes..
[11:08 AM] Ubit Umarov: tehy where mostly about mantis 9230
[11:09 AM] Ubit Umarov: in same cases, with recent code changes some prim inventoiry itens where not saved
[11:09 AM] Ubit Umarov: saves in region db i mean
[11:10 AM] Ubit Umarov: specially in the unlink case
[11:10 AM] Vincent.Sylvester @hg.zetaworlds.com: It does explain why I saw a script go missing, but the trouble with linking and unlinking continues to be a difficult nut to crack
[11:10 AM] Ubit Umarov: well still is spargetti code
[11:11 AM] Ubit Umarov: and inventory thread lock is also not nice
[11:11 AM] Ubit Umarov: that is not the right way to use upgradable locks
[11:12 AM] Ubit Umarov: sadly that bad lock code is used all over maybe also on modules done by others
[11:13 AM] Ubit Umarov: nothing should actually dive in the inventory directly
[11:13 AM] Vincent.Sylvester @hg.zetaworlds.com: The part about the scene and db going out of sync troubles me. It may only be things left behind that were meant to be removed anyways, but that doesn't meant it can't be problematic
[11:13 AM] Ubit Umarov: should only use the code in sop inventory api
[11:14 AM] Vincent.Sylvester @hg.zetaworlds.com: The most direct approach would be linking and unlinking be directly dependent on db write, but that would be slow and lock the scene
[11:14 AM] Lyr.Lobo @cc.opensimulator.org:8002: Hello *grins*
[11:14 AM] Ubit Umarov: well out of sync is a must to reduce saves
[11:14 AM] Ubit Umarov: you do not want to save a SOG on every frame
[11:14 AM] Ubit Umarov: sopping the region doing it..
[11:15 AM] Ubit Umarov: so desync is a must
[11:15 AM] Ubit Umarov: as long ther is no loss
[11:15 AM] Andrew Hellershanks: Hello, Lyr
[11:15 AM] Cuga.Rajal @rajal.org:9000: There are some use cases where unlinking is part of their normal use and they depend on prim contents immediastely
[11:15 AM] Ubit Umarov: and currently loss mostly only happens if you close the region not doing shutdown or backup comands
[11:16 AM] Lyr.Lobo @cc.opensimulator.org:8002: /me grins
[11:16 AM] Ubit Umarov: prim contents are on region
[11:16 AM] Cuga.Rajal @rajal.org:9000: Like for example, "burning an effigy" where a script delinks prims one at a time and the unlinked prim's script makes it go physics and changes particles
[11:16 AM] Ubit Umarov: we are talking about saving into the region DB
[11:16 AM] Vincent.Sylvester @hg.zetaworlds.com: The crux of the issue is that save doesn't check for "is to be deleted" or the sop straight up doesn't have anything to signal that deletion so it just skips removal from the db
[11:17 AM] Cuga.Rajal @rajal.org:9000: That might be an expreme case but its a popular one at the Groovyverse grid
[11:17 AM] Vincent.Sylvester @hg.zetaworlds.com: I fear in order to resolve that even more variables needed and routines to check them, just making every heavier and heavier
[11:18 AM] Ubit Umarov: well it needs to be async.. just with no holes ( except the need to run backup or shutdown )
[11:18 AM] Vincent.Sylvester @hg.zetaworlds.com: Haven't had the time or head space to look, but I'm sure there is way to do that, just the part of being clever about it and not making even more of a mess might be a bit difficult
[11:18 AM] Cuga.Rajal @rajal.org:9000: async but with triggers to make some things hapen faster?
[11:19 AM] Vincent.Sylvester @hg.zetaworlds.com: Mostly just that it remembers to do certain things
[11:19 AM] Vincent.Sylvester @hg.zetaworlds.com: The issue at the moment is that deletions are skipped
[11:19 AM] Ubit Umarov: cuga i say again.. it is save changes to DB
[11:19 AM] Ubit Umarov: region db in this case
[11:19 AM] Cuga.Rajal @rajal.org:9000: is that a problem?
[11:19 AM] Ubit Umarov: region state is updated in time
[11:19 AM] Ubit Umarov: so all see teh cahnges etc
[11:20 AM] Ubit Umarov: ( except on delete where prim collision stay active for a while )
[11:20 AM] Ubit Umarov: nothing special on this
[11:21 AM] Ubit Umarov: it is working so far
[11:21 AM] Ubit Umarov: this last issue was added a few months ago with the changes to try to reduce unnecessary writes to the DB
[11:21 AM] Ubit Umarov: a missinf call to flag the need to save on on case
[11:22 AM] Ubit Umarov: now if there are cases where things stay on db.. well something to debug and fix
[11:22 AM] Vincent.Sylvester @hg.zetaworlds.com: Check mantis 9228 if you want to know what's happening in detail. Short version is that backup routine seems to skip certain deletions when unlink is involved leading to fresh boot from db data to put those objects back. It's writing to disk the change of the root prim, but not the links being removed as I think it just doesn't keep track of that deletion properly. If you don't instantly delete them then the changed link status gets written first after which deletion works fine. It's when that link status is skipped and the scene directly deletes them the data goes out of sync
[11:23 AM] Vincent.Sylvester @hg.zetaworlds.com: It's lacking the "this was unlinked" when doing deletes so it never finds the correct thing to remove
[11:24 AM] Cuga.Rajal @rajal.org:9000: I suspect this problem has been around for a long time
[11:24 AM] Ubit Umarov: in fact unlink does delete the linkset that lost prims
[11:24 AM] Ubit Umarov: and does it sync
[11:24 AM] Ubit Umarov: a bad thing to change on TODO one day
[11:25 AM] Ubit Umarov: since the objects are marked dirty, they are saved back a bit later
[11:25 AM] Vincent.Sylvester @hg.zetaworlds.com: I wrote a routine to compare the db and scene and it does detect the problem, so now I gotta figure out how to get it back into sync, but I think that'll require a new var to track that additional status, which ideally I don't want to add
[11:26 AM] Ubit Umarov: teh first thing obj delete code does is to take objkects out of automatic back..
[11:27 AM] Ubit Umarov: sure race conditons on multitask are not rare.. and it is possible there is one somewhere
[11:28 AM] Ubit Umarov: anyway... region data store does need a deep change
[11:28 AM] Ubit Umarov: current sure has issues and does not scale well
[11:29 AM] Ubit Umarov: but it is (almost ?) working
[11:29 AM] Vincent.Sylvester @hg.zetaworlds.com: I have been testing that new primitems routine for two months now and so far it looks to be working well. First thought that script missing was my fault and it kinda is, but at least seems unrelated to primitems themselves
[11:30 AM] Vincent.Sylvester @hg.zetaworlds.com: Still not the ideal solution and a true delete routine is still a better option than rewrite on all changes, but small steps
[11:30 AM] Ubit Umarov: wel i do not want to read every item back to detect changes
[11:30 AM] Cuga.Rajal @rajal.org:9000: Maybe the new primitems routine will rduce DB activity enough you ca do more with the delete thing?
[11:31 AM] Cuga.Rajal @rajal.org:9000: brb, seeing 10-second chat lag
[11:31 AM] Lyr.Lobo @cc.opensimulator.org:8002: sure, Cuga
[11:31 AM] Ubit Umarov: instead the inventory code needs to trak those changes
[11:31 AM] Ubit Umarov: as it does better now
[11:33 AM] Ubit Umarov: another reason for all the direct access to prim inventory may need to be only via its api.. not direct scans as we still do in lsl etc
[11:34 AM] Vincent.Sylvester @hg.zetaworlds.com: Certainly needs a better approach to doing the tasks it needs to perform rather than crawling things and detecting changes it should just make the required change directly yeah
[11:35 AM] Ubit Umarov: guess teh flag Hasgroupchanged needs to be a bit field tellin what major block did change, not just a blind bool
[11:35 AM] Vincent.Sylvester @hg.zetaworlds.com: The has inventory changed too in that sense
[11:35 AM] Ubit Umarov: like already partially did on the need to send updates to user
[11:35 AM] Vincent.Sylvester @hg.zetaworlds.com: But should be a direct api call to add, remove, update and whatever else it needs to do
[11:36 AM] Ubit Umarov: those sued to be used just as a bool.. all or nothing
[11:36 AM] Ubit Umarov: not some cases state what changed
[11:36 AM] Ubit Umarov: ( no idea why we are having chat lag grrr )
[11:37 AM] Vincent.Sylvester @hg.zetaworlds.com: In a sense it needs to be a queue of things to do that it can do async on the db while the scene keeps direct track of it so shutdown just flushes the changes to disk
[11:37 AM] Ubit Umarov: no no need for a queue
[11:37 AM] Vincent.Sylvester @hg.zetaworlds.com: No not directly, but via the update types
[11:37 AM] Ubit Umarov: queues will make GC get mad and eat all your ram
[11:37 AM] Ubit Umarov: :)
[11:38 AM] Vincent.Sylvester @hg.zetaworlds.com: A pseudo queue
[11:38 AM] Vincent.Sylvester @hg.zetaworlds.com: An item existing with the flag "to be deleted" basically
[11:38 AM] Ubit Umarov: kinfa for now prims get into 3 dbs..
[11:38 AM] Ubit Umarov: shapes db, inventory db and rest db :)
[11:39 AM] Ubit Umarov: at least thsoe 3 should have flags
[11:40 AM] Ubit Umarov: as is a change on inv will for store of every thing
[11:40 AM] Ubit Umarov: only inv has own flag now
[11:41 AM] Ubit Umarov: but this changes are a pain and error prone :(
[11:41 AM] Vincent.Sylvester @hg.zetaworlds.com: Well it currently also nukes everything and writes all the items again, which is certainly worse than reading, comparing to scene and only writing changes, but technically also not the way you are meant to do it
[11:42 AM] Vincent.Sylvester @hg.zetaworlds.com: Need to take small steps and test yeah :)
[11:42 AM] Ubit Umarov: well save all may cost almost same as save just one parameter
[11:42 AM] Ubit Umarov: actually
[11:42 AM] Ubit Umarov: just more wasted bandwith
[11:43 AM] Vincent.Sylvester @hg.zetaworlds.com: It's the disks that are killed by the writing, when it deletes all prim items and writes them all again
[11:43 AM] Ubit Umarov: but most dbs only read and store entire rows
[11:44 AM] Vincent.Sylvester @hg.zetaworlds.com: This also creates race conditions as the backup process due to delete all and write all taking so long that the scene sets another change so when it gets to the end it has more changes to write yet
[11:44 AM] Vincent.Sylvester @hg.zetaworlds.com: It keeps writing the same things over and over
[11:44 AM] Vincent.Sylvester @hg.zetaworlds.com: I see that with some pmac stuff, backup command taking 20+ minutes
[11:44 AM] Ubit Umarov: well good thing all os do caceh disk operations also :)
[11:45 AM] Ubit Umarov: ( so their store is also async and you may lose data if you do not do shutdown )
[11:46 AM] Vincent.Sylvester @hg.zetaworlds.com: Doing that to reserve grid coordinates... there actually is a token based reservation system already kinda built into that, it may not work seeing as the registration currently has no way to send that, seems that part was never written, but grid service has the validation code for that
[11:47 AM] Vincent.Sylvester @hg.zetaworlds.com: Not that hard to rewrite that slightly and make a reservation system with tokens in regions ini
[11:47 AM] Vincent.Sylvester @hg.zetaworlds.com: The token field in db already exists
[11:47 AM] Ubit Umarov: i kinda lost the issue transition :)
[11:48 AM] Cuga.Rajal @rajal.org:9000: Sorry, I need to run, RL
[11:48 AM] Vincent.Sylvester @hg.zetaworlds.com: Internet is unhappy today huh, my last messages all came in out of order
[11:50 AM] Andrew Hellershanks: You'll need to start numbering them. ;)
[11:50 AM] Vincent.Sylvester @hg.zetaworlds.com: In other news I pretty much finished the hypergrid stats page: https://operations.zetamex.com/grid_stats Unless someone has an idea what else to add
[11:51 AM] Ubit Umarov: cookies to get users?
[11:51 AM] Ubit Umarov: :)
[11:51 AM] Ubit Umarov: well most grids are for adults only
[11:51 AM] Lyr.Lobo @cc.opensimulator.org:8002: /me chuckles, feeling her seat rise and fall
[11:51 AM] Lyr.Lobo @cc.opensimulator.org:8002: Dino crossing
[11:51 AM] Ubit Umarov: byt guess all others will lose french kids soon
[11:52 AM] Ubit Umarov: wel no idea if france will consider sl and a opensim as social media
[11:52 AM] Ubit Umarov: or australia
[11:53 AM] Vincent.Sylvester @hg.zetaworlds.com: We have direct messaging and open chats so... ofcom already wants all the uk folks out of it
[11:53 AM] Andrew Hellershanks: adults need cookies too
[11:53 AM] Lyr.Lobo @cc.opensimulator.org:8002: Wow, Tampa, that is a lot of work
[11:53 AM] Ubit Umarov: man watch that diabetes
[11:54 AM] Vincent.Sylvester @hg.zetaworlds.com: Not really, it's just very basic code
[11:54 AM] Andrew Hellershanks: Are the hours grid up time?
[11:55 AM] Vincent.Sylvester @hg.zetaworlds.com: On the main page it's how many hours of stats are reported, the uptime is in the known grids list
[11:55 AM] Vincent.Sylvester @hg.zetaworlds.com: In percentage since they became known to the system
[11:56 AM] Vincent.Sylvester @hg.zetaworlds.com: It's all done hourly so you take the join date and availability percentage to get a rough hour number. I could add the actual count in there somewhere
[11:56 AM] Vincent.Sylvester @hg.zetaworlds.com: https://operations.zetamex.com/grid_stats?stat=known-available#
[11:58 AM] Andrew Hellershanks: That is a lot of stats for a lot of grids. One minor formatting comment. Should the % numbers all be on the second line? Currently the second and third % are on the first line with the stats hours.
[11:58 AM] Ubit Umarov: not sure the tow columns on that stats page is a good idea
[11:58 AM] Ubit Umarov: think one grid per line whould be more readable
[11:59 AM] Vincent.Sylvester @hg.zetaworlds.com: Depending on your monitor resolution it may look weird, it's a bit of a pain to get this dynamically scaled
[11:59 AM] Lyr.Lobo @cc.opensimulator.org:8002: ah, yuo mean displaying 3 columns of grids
[11:59 AM] Lyr.Lobo @cc.opensimulator.org:8002: moving to one grid per row
[11:59 AM] Vincent.Sylvester @hg.zetaworlds.com: There are nearly 5000 grids, that would be one long page
[11:59 AM] Lyr.Lobo @cc.opensimulator.org:8002: Yes, thought so *grins*
[12:00 PM] Lyr.Lobo @cc.opensimulator.org:8002: I figured it out once I saw it compressing in the grid columns 2 and 3
[12:00 PM] Andrew Hellershanks: The number of columns on that page depends on your screen width. I'm seeing 3 columns.
[12:00 PM] Lyr.Lobo @cc.opensimulator.org:8002: same here
[12:00 PM] Ubit Umarov: ok its 3
[12:00 PM] Ubit Umarov: hmm odd entries for osgrid there also ?
[12:01 PM] Vincent.Sylvester @hg.zetaworlds.com: It's cause they have not been reporting stats for that long
[12:01 PM] Vincent.Sylvester @hg.zetaworlds.com: Only started a few weeks ago
[12:01 PM] Lyr.Lobo @cc.opensimulator.org:8002: yes, still updating
[12:01 PM] Lyr.Lobo @cc.opensimulator.org:8002: great work, Tampa
[12:01 PM] Ubit Umarov: wel guess page should tell viewers do do one region per row
[12:01 PM] Ubit Umarov: browsers
[12:02 PM] Vincent.Sylvester @hg.zetaworlds.com: I only have 1920x1080 so on 4k it may look a bit funky
[12:02 PM] Ubit Umarov: ok rl calls
[12:02 PM] Lyr.Lobo @cc.opensimulator.org:8002: /me grins
[12:02 PM] Lyr.Lobo @cc.opensimulator.org:8002: Many thanks for the hard work, Tampa
[12:02 PM] Lyr.Lobo @cc.opensimulator.org:8002: ooh, I'm having a wee bit of chat lag, too
[12:02 PM] Lyr.Lobo @cc.opensimulator.org:8002: looks fine here
[12:02 PM] Vincent.Sylvester @hg.zetaworlds.com: Formatting the data and making it scale dynamically clashes at times
[12:03 PM] Lyr.Lobo @cc.opensimulator.org:8002: Cuga, you are not alone
[12:03 PM] Andrew Hellershanks: I see a search bar in the top left corner of my screen. It is in the way of the text behind it.
[12:03 PM] Lyr.Lobo @cc.opensimulator.org:8002: understandable
[12:04 PM] Vincent.Sylvester @hg.zetaworlds.com: ctrl + mouse wheel scale it down slightly and it should be all visible
[12:04 PM] Vincent.Sylvester @hg.zetaworlds.com: It's cause the search is stickied at the top, can't make that aware of the other elements
[12:05 PM] Andrew Hellershanks: A sad statistic for osgrid. 157,000 or so registered users and less than 4,000 active users.
[12:05 PM] Lyr.Lobo @cc.opensimulator.org:8002: I like it there...easy to find
[12:05 PM] Ubit Umarov: ( rl calls)
[12:05 PM] Ubit Umarov: ( cya next week )
[12:05 PM] Lyr.Lobo @cc.opensimulator.org:8002: see you next week
[12:05 PM] Andrew Hellershanks: ok, Ubit. See you next week.
[12:05 PM] Lyr.Lobo @cc.opensimulator.org:8002: bye Andrew! Take care, everyone
[12:05 PM] Vincent.Sylvester @hg.zetaworlds.com: I suspect some grids suffer from bot registrations as there are some that have similar weird numbers
[12:06 PM] Andrew Hellershanks: ok, bye Lyr
[12:06 PM] Lyr.Lobo @cc.opensimulator.org:8002: Many thanks, Tampa!
[12:06 PM] Lyr.Lobo @cc.opensimulator.org:8002: have a great week!
