(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
[12:11 PM PT] Ubit Umarov: welcome :)
[12:12 PM PT] Ubit Umarov: bout last week code changes... just a few cosmetics :)
[12:13 PM PT] Vincent.Sylvester @hg.zetaworlds.com: Also fixed an issue with acquire time of prim inventory items
[12:13 PM PT] Ubit Umarov: yeah was abotu to say that :P
[12:13 PM PT] Lyr Lobo grins and waves
[12:14 PM PT] Ubit Umarov: welcome :)
[12:14 PM PT] Lyr Lobo: Thanks - just got out of class
[12:14 PM PT] Ubit Umarov: by spec it is the time when the item is added to a prim. On same cases it was set has the item real creation time
[12:16 PM PT] Ubit Umarov: as not has :P
[12:16 PM PT] Ubit Umarov: also a little change to avoid a potencial null ref on groups code
[12:17 PM PT] Vincent.Sylvester @hg.zetaworlds.com: The real big thing that happened last week was the discovery of a rather strange illusive problem in groups. We had a mantis report about a group locking up becoming unusable and I received two reports last week of similar nature. Upon investigating found entries in rolememberships not belonging to an actual role. Basically someone is in a role that doesn't exist
[12:17 PM PT] Vincent.Sylvester @hg.zetaworlds.com: Code now handles that gracefully on permission checks, but will output a warning just in case
[12:17 PM PT] Vincent.Sylvester @hg.zetaworlds.com: The big question is how that happened in the first place
[12:18 PM PT] Ubit Umarov: you broke yr dbs, ofc
[12:18 PM PT] Ubit Umarov: :p
[12:18 PM PT] Vincent.Sylvester @hg.zetaworlds.com: I have found about 50 entries from groups dating back as far as 5 years to as recent as last month, seemingly random
[12:19 PM PT] Ubit Umarov: we are talking about groups v2 roles btw
[12:19 PM PT] Vincent.Sylvester @hg.zetaworlds.com: So far I have asked around and was told that no roles were deleted on those groups so the delete routine might be fine. I got no other failures in the logs as far as I have looked not really knowing what to look for
[12:20 PM PT] Vincent.Sylvester @hg.zetaworlds.com: Suspect it's a silent failure, maybe related to group cache or not, fishing in the dark a bit
[12:20 PM PT] Vincent.Sylvester @hg.zetaworlds.com: I left a sql query on mantis that checks for orphaned rolemembership entries
[12:21 PM PT] Vincent.Sylvester @hg.zetaworlds.com: Groups code is a total mess. The permission check it used to fail on first retrieves the roles a avatar is part of and then runs another query per role to get the associated permissions ... instead of just joining that at sql level and parsing it, which likely would be faster and less heavy
[12:21 PM PT] Cuga.Rajal @rajal.org:9000: do you have a link or mantis bug #? I would like to check mine
[12:22 PM PT] Vincent.Sylvester @hg.zetaworlds.com: http://opensimulator.org/mantis/view.php?id=9170
[12:22 PM PT] Ubit Umarov: mantis 9170
[12:22 PM PT] Cuga.Rajal @rajal.org:9000: ty :)
[12:23 PM PT] Vincent.Sylvester @hg.zetaworlds.com: You can also sql join to get the group names of the broken roles if you want to investigate. Maybe the smoking gun can be found in log entries or even reproduction steps, but probably have to dig through groups code
[12:23 PM PT] Vincent.Sylvester @hg.zetaworlds.com: At least it seems like something old enough that it was not introduced by the switch in runtimes
[12:23 PM PT] Ubit Umarov: the code change does not address the missing roles, but the null ref created by them, and does log when one is found
[12:25 PM PT] Vincent.Sylvester @hg.zetaworlds.com: Before that it would see missing powers as the query returned null, so it would fail there and never continue checking the other roles thus potentially locking you out of making any changes as it never saw your permission to do so
[12:25 PM PT] Cuga.Rajal @rajal.org:9000: glad that's fixed
[12:26 PM PT] Vincent.Sylvester @hg.zetaworlds.com: It's kinda weird that this popped up now in such a short time. I suspect perhaps the order of the returned roles might have changed under dotnet or something like that
[12:26 PM PT] Vincent.Sylvester @hg.zetaworlds.com: It is an old problem though, cause I have a broken role on a group that is 12 years old now
[12:27 PM PT] Cuga.Rajal @rajal.org:9000: could add an ORDER BY to query
[12:27 PM PT] Ubit Umarov: guess roles are not much used in opensim
[12:27 PM PT] Gavin.Hird @grid.xmir.org:8002: I think there has been a bit of group code changes in the viewers lately so maybe that surfaces an old issue
[12:27 PM PT] Ubit Umarov: and also several use flotsam groups...
[12:28 PM PT] Vincent.Sylvester @hg.zetaworlds.com: Just need a Groups V3 at this point, don't think much can be salvaged from the V2 mess
[12:28 PM PT] Ubit Umarov: talking about viewers.. how are yours going, gavin anfdjoe ?
[12:29 PM PT] Gavin.Hird @grid.xmir.org:8002: I published another update to the preview a wee ago
[12:30 PM PT] Ubit Umarov: :)
[12:30 PM PT] Gavin.Hird @grid.xmir.org:8002: having brisk downloads of almost 24k for the Mac version and 6500 for the Windows version
[12:30 PM PT] Gavin.Hird @grid.xmir.org:8002: not a wee
[12:31 PM PT] Cuga.Rajal @rajal.org:9000: Any arm64 port on the horizon Gavin?
[12:31 PM PT] Gavin.Hird @grid.xmir.org:8002: so the time has come to add back all the Opensim code and enable it all
[12:31 PM PT] Gavin.Hird @grid.xmir.org:8002: no ta chance
[12:31 PM PT] Cuga.Rajal @rajal.org:9000: kk, just checkin :)
[12:31 PM PT] Gavin.Hird @grid.xmir.org:8002: but the Windows version will run under Windows for Arm
[12:32 PM PT] Cuga.Rajal @rajal.org:9000: ooh and I have that on a VM
[12:32 PM PT] Gavin.Hird @grid.xmir.org:8002: that works too
[12:32 PM PT] Gavin.Hird @grid.xmir.org:8002: at least in VMware
[12:32 PM PT] Cuga.Rajal @rajal.org:9000: I'll try it in Parallels
[12:33 PM PT] Cuga.Rajal @rajal.org:9000: they support OpenGL 3.2!
[12:33 PM PT] Gavin.Hird @grid.xmir.org:8002: The new viewers really need 4.2
[12:33 PM PT] Ubit Umarov: oh btw if you didn't notice, we did release 0.9.3.0
[12:34 PM PT] Gavin.Hird @grid.xmir.org:8002: when was that?
[12:34 PM PT] Ubit Umarov: nov 8th
[12:34 PM PT] Gavin.Hird @grid.xmir.org:8002: congrats
[12:34 PM PT] Cuga.Rajal @rajal.org:9000: I saw! congrats. Hopefully that will bring more grids up to recent standards
[12:34 PM PT] Gavin.Hird @grid.xmir.org:8002: when is the release party?
[12:34 PM PT] Ubit Umarov: ty
[12:35 PM PT] Ubit Umarov: it was on last week meeting :)
[12:35 PM PT] Gavin.Hird @grid.xmir.org:8002: bummer
[12:35 PM PT] Vincent.Sylvester @hg.zetaworlds.com: If you plan to upgrade to the release you may want to update straight to master, no particular reason ;)
[12:36 PM PT] Joe Magarac: Was there a meeting last week? OSGrid was down?
[12:36 PM PT] Ubit Umarov: yeah you missied all those cookies vincent had there a few years ago
[12:36 PM PT] Cuga.Rajal @rajal.org:9000: master is already ahead of release
[12:36 PM PT] Ubit Umarov: yes we had it at Zeta backup site
[12:36 PM PT] Vincent.Sylvester @hg.zetaworlds.com: NOTE: If there are problems with the Dev Outreach region the meeting will be held in Hurliman Plaza. Should OSGrid be unavailable the meeting might be on http://hg.zetaworlds.com:80/OpenSim or http://grid.xmir.org:8002/barcola
[12:37 PM PT] Cuga.Rajal @rajal.org:9000: making note for next time
[12:37 PM PT] Vincent.Sylvester @hg.zetaworlds.com: All the logs are on the wiki if you want to read back
[12:37 PM PT] Vincent.Sylvester @hg.zetaworlds.com: http://opensimulator.org/wiki/Office_hours
[12:39 PM PT] Ubit Umarov: and yes the code changes we spoke about are on master that is 0.9.3.1 Dev
[12:40 PM PT] Vincent.Sylvester @hg.zetaworlds.com: In other news, still working on migrations code. Have moved to sqlite and pulled some hairs out trying to get that to behave as the connector struggles with more modern queries. mysql so far seems to be working fine, postgres yet to be tested
[12:47 PM PT] Ubit Umarov: our sqlite seems to be from 2018
[12:47 PM PT] Ubit Umarov: not that old :p
[12:49 PM PT] Vincent.Sylvester @hg.zetaworlds.com: I have not checked when exactly it was added, but it seems unable to process the pseudo queries for pragma_table_info() as it doesn't recognize that as a function, only PRAGMA table_info() works, but that cannot be cast to anything so no further things can be done with it. Basically have to process the returns in code rather than doing it all in one go directly in sql
[12:49 PM PT] Vincent.Sylvester @hg.zetaworlds.com: Ends up being a lot more code than just processing show indexes from table as mysql does
[12:49 PM PT] Vincent.Sylvester @hg.zetaworlds.com: Not impossible, just annoying to get right with the relations
[12:49 PM PT] Ubit Umarov: well sqlite only supports a subset of sql... like name tells
[12:50 PM PT] Gavin.Hird @grid.xmir.org:8002: don't let details like that get in the way of your efforts
[12:50 PM PT] Ubit Umarov: :)
[12:50 PM PT] Vincent.Sylvester @hg.zetaworlds.com: I try to keep the schema data as unified as possible between things to make it easier to understand them
[12:51 PM PT] Vincent.Sylvester @hg.zetaworlds.com: It's mostly an issue with the indexes, the rest is fine
[12:51 PM PT] Gavin.Hird @grid.xmir.org:8002: back to groups; yes there hasd been viewer code changes lately
[12:51 PM PT] Gavin.Hird @grid.xmir.org:8002: unsure if that is causing the behavior you see
[12:52 PM PT] Gavin.Hird @grid.xmir.org:8002: they may even use AIS in the new code
[12:52 PM PT] Ubit Umarov: ais is inventory
[12:52 PM PT] Vincent.Sylvester @hg.zetaworlds.com: Anything in groups changed that requires backend changes?
[12:52 PM PT] Gavin.Hird @grid.xmir.org:8002: and groups deals with inventory
[12:53 PM PT] Gavin.Hird @grid.xmir.org:8002: dunno Vincent
[12:53 PM PT] Gavin.Hird @grid.xmir.org:8002: might
[12:53 PM PT] Ubit Umarov: not really...
[12:55 PM PT] Ubit Umarov: well current role creation code does a few independent changes to the the DB
[12:56 PM PT] Ubit Umarov: so a fail in middle may leave incomplete data, like missing roles..
[12:57 PM PT] Vincent.Sylvester @hg.zetaworlds.com: It must be adding the rolemembership first though and then actual role creation fails, odd
[12:57 PM PT] Ubit Umarov: guess a more modern imp would use transactions
[12:57 PM PT] Ubit Umarov: so all aborted on a fail
[12:57 PM PT] Vincent.Sylvester @hg.zetaworlds.com: I only looked at deletion code as I suspected that, but been told from a broken group that they never deleted a role
[12:58 PM PT] Vincent.Sylvester @hg.zetaworlds.com: Groups sql interface is horribly inefficient, but digging into that and replacing things isn't so easy due to the custom types it also uses
[12:58 PM PT] Ubit Umarov: at least we could blame the DB devs on a partial fail.. :P
[12:58 PM PT] Gavin.Hird @grid.xmir.org:8002: deleting a role should not be possible as long as it is assigned to a group member
[12:59 PM PT] Vincent.Sylvester @hg.zetaworlds.com: Yeah I suspected that the removal of the member worked in cache, but not in db, so it got stuck there, but if they never deleted a role...
[1:00 PM PT] Vincent.Sylvester @hg.zetaworlds.com: So must be on creation or something in update getting messed up
[1:00 PM PT] Gavin.Hird @grid.xmir.org:8002: I guess it will be messed up if a group role is deleted and there are still members of the role
[1:01 PM PT] Lyr Lobo: Shared the draft OSCC Dec 7-8 schedule as an image before the meeting ends
[1:01 PM PT] Vincent.Sylvester @hg.zetaworlds.com: I'm tempted to remove the cache from groups to remove that as variable in this
[1:02 PM PT] Gavin.Hird @grid.xmir.org:8002: sounds like a plan
[1:03 PM PT] Vincent.Sylvester @hg.zetaworlds.com: It's somewhat problematic anyways if you run multiple group instances, cause the cache is only in memory and I have found it sometimes not updating when you make permission changes even though it changed in the db
[1:03 PM PT] Vincent.Sylvester @hg.zetaworlds.com: Probably will make it slower though
[1:03 PM PT] Gavin.Hird @grid.xmir.org:8002: how often is group data retrieved anyway?
[1:04 PM PT] Vincent.Sylvester @hg.zetaworlds.com: That said I was under the impression that groups with hundreds of members would just be unusable, but they seem to hold up fine for now
[1:04 PM PT] Vincent.Sylvester @hg.zetaworlds.com: It's only pulled when you request it, then it is cached for like an hour or so
[1:04 PM PT] Ubit Umarov: well a lot actually
[1:04 PM PT] Ubit Umarov: since roles define rights to modify a lot of things inc objects
[1:04 PM PT] Vincent.Sylvester @hg.zetaworlds.com: It tries to update the cache when you make changes so that both db and cache are in sync
[1:05 PM PT] Vincent.Sylvester @hg.zetaworlds.com: On permission changes I have seen that fail though, yet to work out why
[1:05 PM PT] Gavin.Hird @grid.xmir.org:8002: shouldn't it just invalidate the cache entry on write to db and fore a pull from the db on next read?
[1:06 PM PT] Gavin.Hird @grid.xmir.org:8002: force*
[1:06 PM PT] Vincent.Sylvester @hg.zetaworlds.com: I suspect it tries to update the cache to reduce the database load as it otherwises uses a lot of queries to get all that data
[1:07 PM PT] Gavin.Hird @grid.xmir.org:8002: sure, but writes probalby don't happen that often
[1:07 PM PT] Vincent.Sylvester @hg.zetaworlds.com: A full group update basically pulls select * from all the groups tables based on the group id and then processes all that internally, hence it being a bit slow waiting for all the db stuff to finish
[1:08 PM PT] Vincent.Sylvester @hg.zetaworlds.com: I have db sitting on rather fast drives and a lot of memory for table caches so the memory cache in groups probably not even necessary, but this stuff was written when spinning disks were the norm
[1:08 PM PT] Vincent.Sylvester @hg.zetaworlds.com: More efficient sql and bit more modern code and it might just be fine, but that's still a lot of work
[1:08 PM PT] Ubit Umarov: yeah caches not need with remote tcp connections..
[1:09 PM PT] Ubit Umarov: well whatever..
[1:09 PM PT] Lyr Lobo: Many thanks for the meeting! I must dash back to class.
[1:09 PM PT] Lyr Lobo: Have a great week
[1:09 PM PT] Cuga.Rajal @rajal.org:9000: tc Lyr :)
[1:09 PM PT] Ubit Umarov: Cya Lyr :)
[1:09 PM PT] Gavin.Hird @grid.xmir.org:8002: you too
[1:09 PM PT] Motoko.Karu @login.digiworldz.com:8002: Bye Lyr
[1:09 PM PT] Vincent.Sylvester @hg.zetaworlds.com: I have a group with over 1000 members that I suspected would just break the moment I tried to send a notice through, but it worked just fine and sent all the notices within a minute
[1:10 PM PT] Vincent.Sylvester @hg.zetaworlds.com: So by that metric don't really need a cache heh
[1:10 PM PT] Gavin.Hird @grid.xmir.org:8002: also the data might be cached on the db server
[1:11 PM PT] Vincent.Sylvester @hg.zetaworlds.com: Processing a sql query joining a few tables to get specific data you can just parse directly is going to work better than doing multiple queries and processing in code
[1:11 PM PT] Gavin.Hird @grid.xmir.org:8002: if you run explain on the queries you might see where there are bottlenecks
[1:12 PM PT] Gavin.Hird @grid.xmir.org:8002: agreed
[1:12 PM PT] Kayaker Magic: Question: I'm using fsassets on my little grids, I hear MySQL guru's say that blobs are better than files, so should I switch to the assets version?
[1:12 PM PT] Ubit Umarov: gurus say many things
[1:13 PM PT] Vincent.Sylvester @hg.zetaworlds.com: A database technically is files too
[1:13 PM PT] Ubit Umarov: and fs is a bot more than just use files
[1:14 PM PT] Gavin.Hird @grid.xmir.org:8002: You may want to look into materialized views for the group data.
[1:14 PM PT] Ubit Umarov: like the try to reduce duplication of contents
[1:14 PM PT] Ubit Umarov: ( possible not that effective.. but at least something )
[1:15 PM PT] Gavin.Hird @grid.xmir.org:8002: A materialized view is a duplicate data table created by combining data from multiple existing tables for faster data retrieval.
[1:15 PM PT] Vincent.Sylvester @hg.zetaworlds.com: I been fiddling with indexes and various ideas regarding that yeah. Ultimately I want to explore different database style so a group retrieve is just get group by id and get back something already ready to send, like a document style db
[1:15 PM PT] Gavin.Hird @grid.xmir.org:8002: storage is cheap
[1:15 PM PT] Motoko.Karu @login.digiworldz.com:8002: heading back to RL... good to see everyone.. take care
[1:16 PM PT] Ubit Umarov: have fun
[1:16 PM PT] Motoko.Karu @login.digiworldz.com:8002: *waves*
[1:16 PM PT] Cuga.Rajal @rajal.org:9000: tc Moto
[1:16 PM PT] Ubit Umarov: well we are pass the hour anyway
[1:16 PM PT] Ubit Umarov: any other opensim issue?
[1:16 PM PT] Gavin.Hird @grid.xmir.org:8002: not on my end
[1:16 PM PT] Kayaker Magic: Repeat question: Is this PBR helmet here open source? Can I have a copy to test my PBR regions?
[1:17 PM PT] Ubit Umarov: oh it is open source
[1:17 PM PT] Ubit Umarov: part of the gltf demos
[1:17 PM PT] Kayaker Magic: Please mark it copy then!
[1:17 PM PT] Vincent.Sylvester @hg.zetaworlds.com: Things like avatars, profiles, inventory and groups are all candidates for that sort of thing as they are hinged on the uuid primarily so you don't need other data in a table. Just finding a document db with a permissible license, C# connector and a structure that works is not so easy
[1:17 PM PT] Kayaker Magic: Better yet, FULL PERM
[1:17 PM PT] Vincent.Sylvester @hg.zetaworlds.com: mongodb looked okay until I saw the license
[1:18 PM PT] Ubit Umarov: well open source does not mean free :P
[1:18 PM PT] Gavin.Hird @grid.xmir.org:8002: :-)
[1:18 PM PT] Ubit Umarov: but sure.. it is full perms now :)
[1:18 PM PT] Vincent.Sylvester @hg.zetaworlds.com: apache has one that looks good, but the c# connector is a bit suspect
[1:18 PM PT] Cuga.Rajal @rajal.org:9000: pay someone for their hard work importing and texturing
[1:19 PM PT] Cuga.Rajal @rajal.org:9000: Thanks Ubit.. gong to try it in my region
[1:19 PM PT] Cuga.Rajal @rajal.org:9000: too
[1:19 PM PT] Kayaker Magic: Me also!
[1:20 PM PT] Ubit Umarov: https://github.com/KhronosGroup/glTF-Sample-Models.git
[1:20 PM PT] Kayaker Magic: But I have chores in RL, expecting a storm tomorrow need to batten down the hatches.
[1:20 PM PT] Ubit Umarov: think it is there
[1:20 PM PT] Cuga.Rajal @rajal.org:9000: oh cool thanks
[1:20 PM PT] Kayaker Magic: Buy all!
[1:20 PM PT] Gavin.Hird @grid.xmir.org:8002: Right. I need to dash too
[1:20 PM PT] Cuga.Rajal @rajal.org:9000: Yes same here, huge storm coming west coast
[1:21 PM PT] Ubit Umarov: hope to see u all next week, if not before :)
[1:21 PM PT] Gavin.Hird @grid.xmir.org:8002: have a great week
