Chat log from the meeting on 2022-01-18

From OpenSimulator

Jump to: navigation, search
[11:01] Gavin.Hird Hi Andrew
[11:01] Andrew Hellershanks: Hello, everone.
[11:02] Ubit Umarov: Oh here cames Mr Andrew White Hellershanks
[11:02] Andrew Hellershanks: White is my middle name.  ;)
[11:02] Gavin.Hird winter pale
[11:02] Ubit Umarov: food and drinks here are so old
[11:02] Selby.Evans hi everyone
[11:02] Andrew Hellershanks: I'm trying to blend in with the foot and a half of snow we have outside.
[11:02] Ubit Umarov: and bet did come from even older regions
[11:03] Jamie.Jordan Hi Selby
[11:03] Ubit Umarov: oh? no snow here
[11:03] Gavin.Hird right, Ubit - bit-rot of the food is not goo
[11:03] Ubit Umarov: mb gavins has a bot mor esnow
[11:03] Gavin.Hird not god
[11:03] Gavin.Hird good*
[11:03] Ubit Umarov: or a bit more..
[11:03] Gavin.Hird Gavin have 0 mm snowcover
[11:04] Ubit Umarov: ok seems we are having a typos issue :)
[11:04] Gavin.Hird 5°C too
[11:04] Vincent.Sylvester Gavin have you seen the object cache bug fix?
[11:04] Gavin.Hird I have compiled it
[11:04] Gavin.Hird so any day now
[11:04] Gavin.Hird :-)
[11:04] Vincent.Sylvester Yay!
[11:05] Gavin.Hird maybe Friday
[11:05] Ubit Umarov: remember at opensim a prim is only considered for cache 15mins after last change/rez
[11:05] Ubit Umarov: at sl is less that 5min
[11:05] Ubit Umarov: used to be 1 hour here..
[11:05] Ubit Umarov: well is 1 hour on older versions
[11:06] Ubit Umarov: so better rez a few prims i case you need to test more than one and wiat just 15min
[11:06] Vincent.Sylvester I turned the caps off for it entirely on some regions to test and I struggle to see what exactly it otherwise does really in terms of performance, sure it caches prims so they display faster, but I find it difficult to really "see" that difference
[11:07] Ubit Umarov: Ohh its big
[11:07] Ubit Umarov: on return to a region
[11:07] Vincent.Sylvester I don't doubt that, especially the slower the connection is and such
[11:07] Ubit Umarov: very visible
[11:08] Ubit Umarov: ofc now and then fs chokes rezzing prims
[11:08] Ubit Umarov: and now and then, no idea why, some stay trasparent until touched
[11:09] Vincent.Sylvester Perhaps testing on var regions with 50000 objects probably fills it up or something as well at least default cache size isn't very big
[11:09] Ubit Umarov: also culling by distance is viewer side
[11:09] Ubit Umarov: ( where the bug was.. )
[11:10] Vincent.Sylvester I was probably expecting more like back in the day waiting half an hour for a region to load fully or something with it turned off
[11:10] Ubit Umarov: thats a bad aspect of the object cache
[11:10] Ubit Umarov: viewer wants to get all the prims on a region at start
[11:11] Gavin.Hird if not culling will be rather random
[11:11] Ubit Umarov: i did soem work on server side culling even o top of that
[11:11] Ubit Umarov: option there
[11:11] Ubit Umarov: but is heavy and did not finished
[11:11] Ubit Umarov: just SL never considered regions with 120K prims
[11:12] Ubit Umarov: but on other hand opensim never did culling as SL regions did
[11:12] Ubit Umarov: so.. same thing
[11:12] Vincent.Sylvester Was starting to lose my sanity over this thinking it was some obscure packet loss somewhere or something
[11:12] Gavin.Hird they rather have 1000 prims(meses) with 3 millions tries each to contend with
[11:13] Vincent.Sylvester It going all the way to LL and then being half a decade old code is just comical, the cherry on top
[11:13] Ubit Umarov: aurora did had culling
[11:13] Ubit Umarov: but also heavy
[11:14] Ubit Umarov: interest lists per user.. looked at all the time..
[11:14] Vincent.Sylvester Speaking of prims, been digging about today and found an old linkset with xml encoding utf 16 followed by utf 8 data
[11:15] Vincent.Sylvester That based on some idea or should I just ignore that as bad formatting?
[11:16] Andrew Hellershanks: Sounds odd for it to have two different string encodings.
[11:16] Gavin.Hird possibly written and updated by two different viewers
[11:16] Andrew Hellershanks: Could be.
[11:17] Vincent.Sylvester It's also missing a bunch of parameters in the xml like particle system and such things
[11:17] Ubit Umarov: xml is a crap
[11:17] Gavin.Hird doe the thing rez?
[11:17] Ubit Umarov: and opensim use of has not been the best
[11:17] Ubit Umarov: and opensim use of it has not been the best
[11:18] Ubit Umarov: also ms documentation on it is confusing
[11:18] Ubit Umarov: so.. yeah  utf-16 may had shown up on oars
[11:18] Ubit Umarov: at least on the first line
[11:18] Ubit Umarov: while actual data was utf-8
[11:19] Vincent.Sylvester Haven't tried, I just found the asset data being flagged as malformed by the xml parsing so looking at it the linkset xml itself claimed utf16 encoding in the parameter of the header, but the parts xml all said utf8, weird double structure in there
[11:19] Ubit Umarov: use of BOM also not that clear
[11:19] Vincent.Sylvester It should be an inventory item so I might be able to find and rez it
[11:19] Gavin.Hird double structure - like DNA?
[11:20] Vincent.Sylvester Well like one xml open and close tag around a bunch of others for each link part
[11:20] Ubit Umarov: well and possible several oar formats outthere now
[11:20] Ubit Umarov: think inworldz had its own variant?
[11:20] Vincent.Sylvester Original creator is from osgrid back when that other profiles system was in use
[11:21] Ubit Umarov: i think the utf-16 on the header line is ignored if there is a BOM
[11:22] Vincent.Sylvester Forcing utf8 encoding it parsed just fine though so at least the rest of it seems proper
[11:22] Vincent.Sylvester Surprises me that it probably works as asset in OpenSim like that
[11:23] Ubit Umarov: guess that utf16 is still there even on new oars
[11:23] Ubit Umarov: let me xee
[11:23] Vincent.Sylvester There are certainly some whacky data structures in OpenSim
[11:24] Vincent.Sylvester I recall decoding parcel blob data was a fun trip into bits and byte land
[11:24] Gavin.Hird xee - is that a new xml keyword?
[11:24] Ubit Umarov: <SceneObjectGroup><SceneObjectPart xmlns:xsi="" xmlns:xsd="">
[11:24] Ubit Umarov: no idea why we thave that xsi bs
[11:25] Ubit Umarov: <?xml version="1.0" encoding="utf-16"?>
[11:26] Ubit Umarov: <?xml version="1.0" encoding="utf-16"?>
<archive major_version="0" minor_version="8">
[11:26] Ubit Umarov: well this was broken from start
[11:26] Vincent.Sylvester Thing is the data in there is utf8 most likely, because most of it is now
[11:26] Ubit Umarov: can't change now
[11:26] Ubit Umarov: utf-16 is a stupid encode
[11:26] Ubit Umarov: but windows and linux went for it
[11:27] Ubit Umarov: like many early adopters of unicode
[11:27] Ubit Umarov: at a point utf-16 was the entire unicode
[11:27] Ubit Umarov: unicode was so clever that decided 16bits was all that would be needed
[11:28] Andrew Hellershanks: Yup. 64k should be enough for anyone. :)
[11:28] Ubit Umarov: yeah when present all xml headers ssay utf8
[11:28] Ubit Umarov: opoe utf-16
[11:28] Ubit Umarov: hmm let me see a file in binary
[11:29] Vincent.Sylvester lxml.etree.XMLSyntaxError: Document labelled UTF-16 but has UTF-8 content, line 1, column 38
[11:30] Vincent.Sylvester Thankfully can always tell it to force utf8 in that case
[11:31] Ubit Umarov: file is utf8
[11:31] Vincent.Sylvester Just really odd given it was also missing normal parameters for prims, I'd imagine those would eventually get added when reloading the asset via an archive, but doesn't seem to be the case either
[11:31] Ubit Umarov: in fact ascii
[11:31] Ubit Umarov: one byte per char
[11:31] Ubit Umarov: and no BOM
[11:32] Ubit Umarov: no idea how it works LOL
[11:32] Ubit Umarov: ms xml crap
[11:32] Ubit Umarov: 3c 3f 78 6d 6c 20 76 65 72 73 69 6f 6e 3d 22 31 2e 30 22 20 65 6e 63 6f 64 69 6e 67 3d 22 75 74 66 2d 31 36 22 3f 3e 0d 0a 3c 61 72 63 68 69 76 65 20 6d 61 6a 6f 72 5f 76 65 72 73 69 6f 6e 3d
[11:32] Vincent.Sylvester I imagine if proper structure was enforced half the stuff in world wouldn't load anymore
[11:32] Ubit Umarov: see? simple plain ascii
[11:34] Ubit Umarov: guess on read we do force utf8
[11:34] Vincent.Sylvester I figured as much given the parser said it was all utf8 anyways
[11:35] Vincent.Sylvester Got stuck on something else after that, but that's a different story for another time
[11:35] Ubit Umarov: utf-16 also has bit order issues lol
[11:35] Ubit Umarov: byte order
[11:36] Ubit Umarov: xml is so well done protocoll
[11:36] Ubit Umarov: for a crap made on knees
[11:36] Vincent.Sylvester I been making more modules lately and found the old dreaded thread stuck on shutdown gets worse the more modules I add
[11:37] Andrew Hellershanks: I thought that problem went away as of 0.9
[11:37] Vincent.Sylvester I thought so too
[11:37] Vincent.Sylvester It is starting to look a bit like the reason for that is usage or data contained in those threads
[11:37] Vincent.Sylvester Given I add modules that put a lot of data out into the system tend to make the most impact
[11:38] Vincent.Sylvester So I wonder if the cause is some form of memory leak or other orphaned data that fails to clear
[11:38] Vincent.Sylvester Especially when I do a lot of testing with the modules thus cause more data to go back and forth
[11:39] Vincent.Sylvester On master I still see some red during shutdown on random threads not shutting down cleanly, though only on mono
[11:40] Vincent.Sylvester on windows .net doing tests in vscode it all works splendidly
[11:40] Ubit Umarov: mono has issues on threads stop
[11:40] Ubit Umarov: lets them stuck on some things
[11:41] Ubit Umarov: in same cases can't even kill then the wai it wants, then gets stuck
[11:41] Jagga Meredith whispers: -[[[[[[[[[
[11:41] Ubit Umarov: ...the way...
[11:42] Vincent.Sylvester One can always check if the simulator responds to "pings" and nuke it, but of course not ideal either
[11:42] Vincent.Sylvester It surprises me that bug is still in mono after all this time
[11:42] Vincent.Sylvester Would assume we can't be the only ones to see it
[11:42] Gavin.Hird mono is not being updated any more
[11:42] Ubit Umarov: well mono is now dead
[11:42] Gavin.Hird yes
[11:42] Ubit Umarov: on its down hill
[11:43] Ubit Umarov: ms made all the devel it needs to canabilze parts of it to dotnet
[11:43] Andrew Hellershanks: Vincent, to do that properly requires more than one ping and enough time between them to have allowed the instance to have saved prim data to the data base so that it doesn't get killed while it is still in the process of shutting down.
[11:43] Ubit Umarov: now..  basicly dead
[11:43] Kayaker Magic: so how is progress on mono's dotnet on Linux?
[11:43] Ubit Umarov: no updates in almost one year
[11:43] Ubit Umarov: who cares..
[11:44] Ubit Umarov: opensim is a huge pain to covert to dotnet
[11:44] Ubit Umarov: we aill lose some features
[11:44] Ubit Umarov: Xengine is one
[11:44] Ubit Umarov: even Yengine may have issues
[11:45] Ubit Umarov: bitmap operations need to be converted to use other external lib, because removed from .net
[11:46] Ubit Umarov: that measn no map, no dynamic textures, even no lludp textures fetch some viewers still use
[11:46] Ubit Umarov: ann and no sculpt maps
[11:47] Ubit Umarov: .net5 and 6 is just a huge breaking change
[11:47] Ubit Umarov: and .net framework 4.8 is suposed to last several years still
[11:47] Ubit Umarov: problem is  mono
[11:47] Ubit Umarov: how it lost all devs
[11:48] Vincent.Sylvester Last commit two weeks ago it says
[11:48] Vincent.Sylvester I mean thankfully is open source, but yeah
[11:48] Gavin.Hird also no usable 64-bit arm version of mono
[11:48] Ubit Umarov: well no release since Feb last here
[11:48] Ubit Umarov: yeah no new things on mono
[11:49] Ubit Umarov: ms will slow start killing it to "sell" dotnet
[11:49] Vincent.Sylvester It got so much traction 5 and 6 flew out the door quickly and then MS killed it by buying it
[11:49] Ubit Umarov: also ms does not care
[11:49] Ubit Umarov: who does use linux and mono on the users universe?
[11:49] Ubit Umarov: 0.001% of all users?
[11:50] Vincent.Sylvester Er that reminds me, Gavin, you saw the mantis on postgres not working, were you able to reproduce that?
[11:50] Gavin.Hird I did not see that
[11:51] Kayaker Magic: Lots of the commercial OpenSim grids use Linux/Mono. Lots of the mid-sized users (between grids and sim-on a stick) use Linux/Mono
[11:51] Vincent.Sylvester 8959 is the id iirc
[11:51] Gavin.Hird postgres don't run on mono higher than
[11:52] Gavin.Hird without a larger rewrite
[11:52] Ubit Umarov: yes and opensim users are like 0.000000001% of all users :p
[11:52] Ubit Umarov: all ms users..
[11:52] Vincent.Sylvester Well I was able to get OpenSim to load with just one error by updating the Npgsql and supply two additional dlls to it
[11:52] Vincent.Sylvester Though as Ubit pointed out those are a bad idea
[11:53] Gavin.Hird you can get it to laod, but try logging in
[11:53] Vincent.Sylvester Yeah there is an issue with user accounts parsing uuid field
[11:54] Vincent.Sylvester I'm not familiar with postgres so attempting to fix that....
[11:54] Gavin.Hird as I said, a rather large rewrite
[11:54] Ubit Umarov: our current Npgsql.dll fails to run on last stable mono??
[11:54] Gavin.Hird yes
[11:54] Ubit Umarov whispers: why i  hell??
[11:54] Ubit Umarov: should not
[11:55] Vincent.Sylvester You have to use version 3.2.7 of it and supply two additional dlls then it kinda works
[11:55] Gavin.Hird you have your work cut out in front of you ;-)
[11:55] Gavin.Hird except 3.2.7 is proably already unsupported too
[11:55] Ubit Umarov: vince that is changinf to a .net5 or standard lib
[11:55] Vincent.Sylvester Tell me about it I looked at the Npgsql source to remove the need for said dlls...
[11:56] Vincent.Sylvester OpenSim has clean and well structured code in comparison
[11:56] Ubit Umarov: possible can't anymore
[11:56] Gavin.Hird current supported versions are in the 4.high or even 5 versions
[11:56] Vincent.Sylvester 4.1.9 is total nightmare, 8 dlls that wants
[11:56] Vincent.Sylvester All manners of random things
[11:56] Ubit Umarov: like all religion kids, possible they just blindly moved to .net5/6 and relatives
[11:57] Gavin.Hird nuget(?) install
[11:57] Vincent.Sylvester It's total bloatware, the latest version is 8 times the size of the one OpenSim uses
[11:58] Ubit Umarov: vincent main software job is to force users to by more powerfull machines
[11:58] Vincent.Sylvester Space heaters
[11:58] Gavin.Hird it actually does more like supporting new features in postgres 13 and 14
[11:58] Ubit Umarov: ofc is is larger and needs more cpu to do the same.. that is what software is all abotu :p
[11:59] Andrew Hellershanks: :)
[11:59] Andrew Hellershanks: We are at the top of the hour.
[12:00] Vincent.Sylvester If the postgres connector code was a bit easier to understand I might be inclined to try to fix that uuid error just bending over and embedding the two dlls in there, but I feel I will go down a rabbit hole of further errors down the line
[12:00] Ubit Umarov: what uuid error?
[12:00] Andrew Hellershanks: One small bit of OpenSim news. The biggest change this past week was saving some nanoseconds by handling the checks for empty strings in a more efficient manner.
[12:00] Gavin.Hird it is a casting error in
[12:00] Gavin.Hird casting error with libomv (or what it is called)
[12:01] Ubit Umarov: libomv has nothing to do with pgsql
[12:01] Gavin.Hird you keep saying that
[12:01] Gavin.Hird but it does
[12:01] Gavin.Hird it cannot do a cast
[12:01] Gavin.Hird with uuid as it is implemented in libomv
[12:01] Ubit Umarov: wel then the cast needs a workaround
[12:02] Ubit Umarov: ut has own uuid?
[12:02] Ubit Umarov: possible the casts need the full namespace spec
[12:02] Ubit Umarov: uuid is also a native type of that hign?
[12:02] Ubit Umarov: thing?
[12:03] Ubit Umarov: or is it ms guid ?
[12:03] Vincent.Sylvester
[12:04] Vincent.Sylvester Error loading plugin OpenSim.Services.Interfaces.IUserAccountService from OpenSim.Services.UserAccountService.dll. Exception: Can't convert .NET type >
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.InvalidCastException: Can't convert .NET type OpenMetaverse.UUID to PostgreSQL uu>
[12:04] Gavin.Hird I think it has to do with ms guid, but O cannot recall exactly right now
[12:04] Ubit Umarov: Can't convert .NET type OpenMetaverse.UUID to PostgreSQL uu
[12:04] Ubit Umarov: uu ?
[12:04] Vincent.Sylvester id
[12:04] Gavin.Hird uuid
[12:04] Ubit Umarov: it tells uu
[12:04] Ubit Umarov: :)
[12:04] Gavin.Hird uuid is native dataype in postgres
[12:05] Ubit Umarov: ok then isolation is needed
[12:05] Ubit Umarov: one needs to have full namespace spec
[12:06] Ubit Umarov: why that is a issue with new mono and not older.. well..  no idea
[12:06] Gavin.Hird deprecations
[12:06] Gavin.Hird or actually removal of support in npgsql
[12:07] Gavin.Hird but then BOOM - mono was not updated any more
[12:07] Ubit Umarov: PGSQLGenericTableHandler
[12:08] Ubit Umarov: well that is a "clever" to it all thing
[12:08] Ubit Umarov: not that simple do fix
[12:08] Gavin.Hird actaully it is pretty dumb and needs to be rewriten
[12:08] Gavin.Hird it does not support upsert, so there are issues with some tables bacause of that
[12:09] Ubit Umarov: else if (m_Fields[name].GetValue(row) is UUID)
                            UUID uuid = UUID.Zero;

                            UUID.TryParse(reader[name].ToString(), out uuid);
                            m_Fields[name].SetValue(row, uuid);
[12:09] Ubit Umarov: guess m_Fields[name].GetValue(row) is UUID is other uuid
[12:09] Vincent.Sylvester With mariadb getting uuid field type soon might be good to revisit the connectors in general is my thinking on this, squeeze some performance out of them perhaps :)
[12:10] Ubit Umarov: then code goes via string to convert it to openmetaverse :)
[12:10] Ubit Umarov: uuid on opensim and dbs is a mess
[12:10] Ubit Umarov: some are just done as string
[12:10] Jamie.Jordan Have a great week yall
[12:11] Ubit Umarov: and not just on the db.. even on main opensim code
[12:11] Andrew Hellershanks: ok, Jamie. You too. Thanks for dropping by
[12:11] Ubit Umarov: and worse
[12:11] Gavin.Hird the postgres tables are partly native uuid and partly varchar
[12:11] Ubit Umarov: some where born as uuid and now have more things
[12:11] Ubit Umarov: like the uui
[12:11] Ubit Umarov: ie  uuid+ gridhost etc
[12:12] Andrew Hellershanks: Right.
[12:12] Gavin.Hird Griduser UserID is also a gem
[12:13] Ubit Umarov: hard to fix that mess now
[12:13] Ubit Umarov: it is on a lot of places
[12:13] Gavin.Hird :-)
[12:14] Ubit Umarov: so use of dbs native uuid is not that great
[12:14] Ubit Umarov: and on same that natice is just a 36 chars string and with the useless '-'
[12:14] Ubit Umarov: that native..
[12:15] Gavin.Hird the native actually support 3 different implementations of uuid
[12:17] Gavin.Hird PostgreSQL also accepts the following alternative forms for input: use of upper-case digits, the standard format surrounded by braces, omitting some or all hyphens, adding a hyphen after any group of four digits.
[12:19] Gavin.Hird PostgreSQL provides storage and comparison functions for UUIDs, but the core database does not include any function for generating UUIDs, because no single algorithm is well suited for every application
[12:20] Ubit Umarov: well no idea that crash needs some debug to find where is the types mix
[12:21] Ubit Umarov: cmd.Parameters.Add(m_database.CreateParameter(fields[i], keys[i], m_FieldTypes[fields[i]]));
[12:21] Ubit Umarov: guess here
[12:22] Ubit Umarov: yeha guess it is there
[12:27] Andrew Hellershanks: Almost half past the hour. Any last minute items for today?
[12:28] Gavin.Hird got the macOS version of the viewer building on the latest Xcode. Now the Intel version also compiles on an Apple Silicon machine
[12:29] Andrew Hellershanks: Sounds good, Gavin.
[12:29] Gavin.Hird it compiled faster than on any of the Intel kit I have
[12:29] Andrew Hellershanks: :)
[12:30] Andrew Hellershanks: Time to wrap the meeting for today.
[12:30] Gavin.Hird warp
[12:30] Kayaker Magic: Yup, I need to run.
[12:30] Andrew Hellershanks: Good luck with your work on Postgres issues. Postgres support doesn't see a lot of love.
[12:30] Kayaker Magic: bye all!
[12:30] Andrew Hellershanks: Thank you all for coming. See you again next week.
Personal tools
About This Wiki