(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
[11:10 AM] Ubit Umarov: welcome :)
[11:10 AM] Vincent.Sylvester @hg.zetaworlds.com: Anyways shall we begin the meeting
[11:10 AM] Gavin.Hird @grid.xmir.org:8002: I have one of the original translucent Mac keyboards that was launched in 2000 with USB 1. Still works
[11:11 AM] Lyr.Lobo @cc.opensimulator.org:8002: very cool, Gavin
[11:11 AM] Andrew Hellershanks: I'm multi-tasking. It is time to start.
[11:11 AM] Gavin.Hird @grid.xmir.org:8002: I also have the original iPod with a 5 Gb harddrive
[11:11 AM] Gavin.Hird @grid.xmir.org:8002: mechanical
[11:11 AM] Andrew Hellershanks: I'm typing under difficulties as my cat is in front of my monitor partially blocking my view.
[11:11 AM] Ubit Umarov: well on code changes in added a patch with bugs from vicent..
[11:12 AM] Vincent.Sylvester @hg.zetaworlds.com: Some of it is even working code this time, I know shocker
[11:12 AM] Ubit Umarov: and from vincent also
[11:13 AM] Ubit Umarov: wel i didnt test :p
[11:13 AM] Ubit Umarov: made a few changes here and there, cosmetics mostly
[11:14 AM] Vincent.Sylvester @hg.zetaworlds.com: I did, well for the most part anyways. It's a pain with the different image formats. Basically I expanded the TIFF loader to use full 32bit floating points and be able to load some lesser formats as well. As TIFF can be used in most photo editors and it supports the format we use for heightmaps it seems like a good option. PNG and the others round or cutoff data because their format doesn't support 32 bit floats
[11:14 AM] Vincent.Sylvester @hg.zetaworlds.com: I also added some way to check if the terrain is so tall that a specific loader would cut off at 256 meters
[11:15 AM] Vincent.Sylvester @hg.zetaworlds.com: The big difference is that I shifted the the conversion to pixel grayscale down 256 points to account for negative terrain, so existing TIFF files might load much lower now
[11:16 AM] Vincent.Sylvester @hg.zetaworlds.com: Think the benefit of finally having a format that is one to one with heightmaps internally and can be easily edited is worth it though
[11:19 AM] Vincent.Sylvester @hg.zetaworlds.com: This was originally part of a larger rework of the entire maptile system, generators and map service, but I pulled it out of there to make it easier to digest :)
[11:20 AM] Vincent.Sylvester @hg.zetaworlds.com: Now the only open thing left is that primitems stuff that Ubit and I have to eventually agree on how to accomplish
[11:21 AM] Vincent.Sylvester @hg.zetaworlds.com: In other commit news also did a fix for Postgresql
[11:21 AM] Vincent.Sylvester @hg.zetaworlds.com: Apparently some code was trying to compare a string to a uuid or rather guid and that blew up logins
[11:21 AM] Ubit Umarov: ahh yes there was that also
[11:22 AM] Vincent.Sylvester @hg.zetaworlds.com: Went through postgres wrapper and checked it for that. Logins work again. Have not really checked much else. There was also a small bug in XAssets that got fixed
[11:22 AM] Vincent.Sylvester @hg.zetaworlds.com: Probably cause no one uses that it wasn't found
[11:23 AM] Ubit Umarov: originaly uuids where stored as char36 like on mysql
[11:23 AM] Vincent.Sylvester @hg.zetaworlds.com: In postgres some still are, like on regionextra
[11:23 AM] Ubit Umarov: but then there was a change to use native uuid
[11:23 AM] Ubit Umarov: but not all code got the needed changes..
[11:24 AM] Vincent.Sylvester @hg.zetaworlds.com: I think it was around postgres 15 or so they forced type comparison to be equal types, so comparing a string to a uuid blew up. Originally this came along with the certificate issue that was fixed a while back
[11:24 AM] Vincent.Sylvester @hg.zetaworlds.com: As I was trying to update it and check for newer postgres versions I ran into that login issue, but at the time thought it was the connector not the wrapper code
[11:26 AM] Vincent.Sylvester @hg.zetaworlds.com: I'm not really a fan of postgres but since we are keeping it, might as well get it working
[11:26 AM] Vincent.Sylvester @hg.zetaworlds.com: With sqlite now back working as well
[11:27 AM] Andrew Hellershanks: Good work.
[11:27 AM] Vincent.Sylvester @hg.zetaworlds.com: I still have that rewrite on the migration pending that eventually should really go in there, but it still needs some fine tuning and testing
[11:28 AM] Gavin.Hird @grid.xmir.org:8002: I like postgres, but it take a bit to get used to
[11:28 AM] Gavin.Hird @grid.xmir.org:8002: what version did you test on you said?
[11:28 AM] Vincent.Sylvester @hg.zetaworlds.com: Supposedly it has ways we could use to make some queries a lot faster, but that's database tuning territory I'm not that familiar with
[11:29 AM] Vincent.Sylvester @hg.zetaworlds.com: 16 was originally one I couldn't get working, now tested with 18 and works
[11:29 AM] Vincent.Sylvester @hg.zetaworlds.com: So latest release
[11:29 AM] Gavin.Hird @grid.xmir.org:8002: it has Vince
[11:30 AM] Vincent.Sylvester @hg.zetaworlds.com: That side of things has been neglected a lot and was originally written rather hastily from the looks of things so putting some effort into that, as everything is based on that data, is well worth it
[11:30 AM] Gavin.Hird @grid.xmir.org:8002: cool - I should upgrade from 15
[11:31 AM] Vincent.Sylvester @hg.zetaworlds.com: I thought you moved to maria?
[11:31 AM] Gavin.Hird @grid.xmir.org:8002: me?
[11:31 AM] Vincent.Sylvester @hg.zetaworlds.com: Yea
[11:31 AM] Gavin.Hird @grid.xmir.org:8002: for a short period and then back to MySQL proper
[11:32 AM] Gavin.Hird @grid.xmir.org:8002: or not back to, but to
[11:32 AM] Vincent.Sylvester @hg.zetaworlds.com: Ah. Well if you have some optimizations we can stuff into postgres I'm all ears
[11:33 AM] Gavin.Hird @grid.xmir.org:8002: there is materialized views which could be used many places rather than torture the tables like we do now
[11:34 AM] Vincent.Sylvester @hg.zetaworlds.com: Inventory and groups I presume?
[11:35 AM] Andrew Hellershanks: Is that similar to the Views you can get with MySQL?
[11:35 AM] Vincent.Sylvester @hg.zetaworlds.com: I been trying some things in maria, but couldn't really tell if it was faster or not
[11:35 AM] Vincent.Sylvester @hg.zetaworlds.com: I just throw a lot of memory at it for table caching, that seems to help the most
[11:36 AM] Gavin.Hird @grid.xmir.org:8002: not sure if the community version of MySql has materialzed views
[11:36 AM] Gavin.Hird @grid.xmir.org:8002: have to check
[11:36 AM] Gavin.Hird @grid.xmir.org:8002: Yes Vincent, but also for search
[11:37 AM] Gavin.Hird @grid.xmir.org:8002: you aggregate data from multiple tables in a materialized view and then run the such on it
[11:37 AM] Gavin.Hird @grid.xmir.org:8002: search*
[11:38 AM] Vincent.Sylvester @hg.zetaworlds.com: The reason that is currently insanely slow is because the tables are not the same charset, making any join operations require data conversion which takes forever
[11:38 AM] Vincent.Sylvester @hg.zetaworlds.com: Unify them all to the same charset and inner join will be less than a second
[11:38 AM] Vincent.Sylvester @hg.zetaworlds.com: Banged my head against a wall for an hour on that one
[11:39 AM] Gavin.Hird @grid.xmir.org:8002: ok, but the material view is updated on write, so it is a fast as a table search
[11:39 AM] Vincent.Sylvester @hg.zetaworlds.com: Oh yeah that's definitely still faster
[11:40 AM] Andrew Hellershanks: I see different charsets on tables now and then. Don't know how that seems to reoccur at times.
[11:40 AM] Vincent.Sylvester @hg.zetaworlds.com: I unified everything to uft8mb4 and called it a day
[11:41 AM] Andrew Hellershanks: Why mb4 vs one of the unicode ones?
[11:42 AM] Vincent.Sylvester @hg.zetaworlds.com: Far as I know it can handle all the latin1 stuff and even though it's pretty heavy on data it should allow for all the weird stuff the db sees
[11:43 AM] Vincent.Sylvester @hg.zetaworlds.com: The latin stuff would be breaking on some unicode too, utf8mb3 as well, mb4 seems to have no issues with that stuff
[11:43 AM] Vincent.Sylvester @hg.zetaworlds.com: Search found 33 results completed in: 0.04070019721984863 seconds
[11:43 AM] Vincent.Sylvester @hg.zetaworlds.com: I'd say that's fast enough for a search query with 4 inner joins
[11:44 AM] Gavin.Hird @grid.xmir.org:8002: there is this thing called scaling ;-)
[11:45 AM] Vincent.Sylvester @hg.zetaworlds.com: This is the websearch on zw, which has to search 3500+ users, dozens of blog posts, places, classifieds and such
[11:45 AM] Vincent.Sylvester @hg.zetaworlds.com: Not that there is a ton of data to search to begin with, parcel sales and events are empty
[11:46 AM] Gavin.Hird @grid.xmir.org:8002: I meant concurrency with many users, such as a big grid
[11:46 AM] Gavin.Hird @grid.xmir.org:8002: and the db being hammered by other queries and writes at the same time
[11:47 AM] Vincent.Sylvester @hg.zetaworlds.com: Just throw more hardware at the problem /s
[11:47 AM] Vincent.Sylvester @hg.zetaworlds.com: I think at the usual OpenSim scales it'll not have any trouble, but I could try benchmarking it
[11:47 AM] Gavin.Hird @grid.xmir.org:8002: db writes are usually quite slow regardless of hardware
[11:48 AM] Vincent.Sylvester @hg.zetaworlds.com: And a disk killer
[11:48 AM] Gavin.Hird @grid.xmir.org:8002: yep
[11:48 AM] Andrew Hellershanks: @Vincent, ok. I thought the unicode ones were like that too.
[11:50 AM] Vincent.Sylvester @hg.zetaworlds.com: utf8mb4 general_ci everywhere, fields and table. So far I haven't seen anything break. It does use more data, but with it being uniform so many queries are faster and there is lots of useful information to gain joining things. With different charsets those queries can take minutes
[11:50 AM] Ubit Umarov: there is no point on using utf9 on many things
[11:50 AM] Ubit Umarov: or even utf8
[11:51 AM] Ubit Umarov: uuids only use ascii
[11:51 AM] Andrew Hellershanks: ok. I'm just using the unicode for my standalone as I don't have to deal with characters outside those of English.
[11:51 AM] Ubit Umarov: also several other things are ascii only by spec
[11:51 AM] Gavin.Hird @grid.xmir.org:8002: sorry, do we support emojies?
[11:52 AM] Ubit Umarov: so utb8 is pure waste
[11:52 AM] Vincent.Sylvester @hg.zetaworlds.com: Yeah a lot of unicode icons are now in
[11:52 AM] Lyr.Lobo @cc.opensimulator.org:8002: UTF-8
[11:52 AM] Ubit Umarov: yes utf-8 also :)
[11:52 AM] Lyr.Lobo @cc.opensimulator.org:8002: /me nods and smiles
[11:53 AM] Vincent.Sylvester @hg.zetaworlds.com: I found the uniformity easier to use for special queries and really it doesn't waste that much extra space. A tradeoff I'm willing to make
[11:54 AM] Ubit Umarov: yeah 4 times more space... slower compares.. etc
[11:54 AM] Gavin.Hird @grid.xmir.org:8002: in Postgres record are 8k anyway so usually there are plenty of space
[11:54 AM] Ubit Umarov: sure some dbs devs also thing all should waste cpu on mb4 and make crap code on other options
[11:55 AM] Vincent.Sylvester @hg.zetaworlds.com: A lot of slow behavior isn't really down to the mb4 use though and would actually benefit joining data on the db level rather than in code
[11:55 AM] Ubit Umarov: not really gavin.Hird
[11:56 AM] Ubit Umarov: it is 8k for every thing but blobs
[11:56 AM] Gavin.Hird @grid.xmir.org:8002: yes
[11:56 AM] Gavin.Hird @grid.xmir.org:8002: and they are stored separate from the rest of the record
[11:56 AM] Gavin.Hird @grid.xmir.org:8002: can even go on their own disks
[11:56 AM] Ubit Umarov: we did had size issues on same tables already
[11:56 AM] Vincent.Sylvester @hg.zetaworlds.com: You can fine tune these things till the cows come home, but really for most things you just want it compatible and not worry about any minor chance in unicode immediately exploding your db
[11:56 AM] Andrew Hellershanks: We are a few minutes before the top of the hour. Let's hold the DB topics for just a moment.
[11:57 AM] Ubit Umarov: so move all to mb4 is just plain wrong
[11:57 AM] Andrew Hellershanks: We are just about a week and a half from the OSCC weekend.
[11:57 AM] Vincent.Sylvester @hg.zetaworlds.com: The table scale issue has somewhat gone away as the reduction in data sizes implemented into most db software has reduced record size slightly
[11:57 AM] Ubit Umarov: problems are on still incoerent tables
[11:57 AM] Ubit Umarov: and ofc the mysql mb3
[11:58 AM] Ubit Umarov: that does not support some chars ppl decide to need :p
[11:58 AM] Vincent.Sylvester @hg.zetaworlds.com: mb3 is just useless when it comes to char support
[11:58 AM] Ubit Umarov: well it covers most uses
[11:58 AM] Vincent.Sylvester @hg.zetaworlds.com: Annoyingly not all and those edge cases keep popping up
[11:58 AM] Andrew Hellershanks: Wait a moment please.I'm checking with Lyr whether there is anything she wants to say about the OSCC now that we are almost there.
[11:59 AM] Lyr.Lobo @cc.opensimulator.org:8002: we have 3 live music and dances on Friday, Dec 5th beginning at Noon Pacific and the conference opens Saturday at 7 am with the Core Dev panel. The VIP is at 11:45 am. I gave you my view of the schedule. The official program is online at https://conference.opensimulator.org/
[11:59 AM] Lyr.Lobo @cc.opensimulator.org:8002: Thank you, dear friends. I am sorry that I must dash. We will have a live stream on YouTube on the AvaconInc channel and welcome you to join us. It is a community event and you make our community possible *hugs*
[11:59 AM] Lyr.Lobo @cc.opensimulator.org:8002: We'll use Discord for voice and pipe it into the music stream
[12:00 PM] Lyr.Lobo @cc.opensimulator.org:8002: and to YouTube.
[12:00 PM] Ubit Umarov: oh my rl calls also :)
[12:00 PM] Lyr.Lobo @cc.opensimulator.org:8002: yes
[12:00 PM] Lyr.Lobo @cc.opensimulator.org:8002: take care...see you next week
[12:00 PM] Lyr.Lobo @cc.opensimulator.org:8002: We love you and thank you for your hard work.
[12:00 PM] Lyr.Lobo @cc.opensimulator.org:8002: /me cheers
[12:01 PM] Gavin.Hird @grid.xmir.org:8002: cheers
[12:01 PM] Lyr.Lobo @cc.opensimulator.org:8002: have a great week! Join us at any of the events and celebrate with us
[12:01 PM] Lyr.Lobo @cc.opensimulator.org:8002: hehe
[12:01 PM] Lyr.Lobo @cc.opensimulator.org:8002: have fun! I'm heading to class *waves*
[12:01 PM] Vincent.Sylvester @hg.zetaworlds.com: I did talk to FS this week as well regarding that long gone interests tab in the profile as I was hoping it could make a return in order to use the data in there for building a translator right into OpenSim, but it seems that's unlikely as the work involved in getting that back is just too much
[12:01 PM] Vincent.Sylvester @hg.zetaworlds.com: That leaves the question what to do with the data in the database though
[12:02 PM] Andrew Hellershanks: I was planning to ask a question about voice in Firestorm. I have not been able to talk in world for the past two years.
[12:02 PM] Ubit Umarov: ( bte main performance issues of mb4 is on indexes.. where thay are int32 basicly )
[12:02 PM] Andrew Hellershanks: I'm running Linux Mint 21.2
[12:02 PM] Vincent.Sylvester @hg.zetaworlds.com: Andrew, voice is still there far as I can tell Vivox haven't turned it off
[12:03 PM] Vincent.Sylvester @hg.zetaworlds.com: You may need to update your viewer, there were some issues with voice on unix based systems
[12:03 PM] Vincent.Sylvester @hg.zetaworlds.com: The Welcome region over on zw has some parcels with voice if you want to test it
[12:06 PM] Andrew Hellershanks: I've tried all the voice setting options in Firestorm. None work when I tested at the Staff Zone in the OSCC grid. I can hear just fine.
[12:06 PM] Ubit Umarov: ok ppl my rl calls. See you all next week if not before >(
[12:06 PM] Ubit Umarov: :)
[12:06 PM] Gavin.Hird @grid.xmir.org:8002: I should go too
[12:06 PM] Gavin.Hird @grid.xmir.org:8002: have a good week everyone
