Chat log from the meeting on 2019-03-19

From OpenSimulator

Jump to: navigation, search

[11:01] Kayaker Magic: Welcome Andrew!
[11:02] Selby.Evans hi everyone
[11:02] Gavin.Hird Hi Andres
[11:02] Andrew Hellershanks: Hello, everyone.
[11:03] Gavin.Hird Andrew - do I have autoreplace turned on today??
[11:03] Bill Blight waves
[11:03] Andrew Hellershanks: autoreplace?
[11:03] Gavin.Hird in the viewer
[11:03] Kayaker Magic whispers: waves and beaches!
[11:03] Andrew Hellershanks: I have no idea. I don't know what that is.
[11:03] Andrew Hellershanks: wb, Kayaker.
[11:04] Gavin.Hird I think I typed Andrew and it came out as Andres
[11:04] Leighton.Marjoram hi all
[11:05] Ubit Umarov: hi
[11:05] Gavin.Hird Hi Ubit
[11:05] Andrew Hellershanks: Gavin, yes it did.
[11:05] Kayaker Magic: Vincent, time to bring up your issue: the Important People are here!
[11:06] Andrew Hellershanks: The time change hasn't thrown people off this week.
[11:07] Andrew Hellershanks: I had some distractions this week so I didn't finish all the work on the OSSL wiki pages but I'm about half way done.
[11:07] Kayaker Magic: California is working on getting off the daylight wastings sytem.
[11:08] Vincent.Sylvester Okay so I ran into a bit of an issue yesterday updating my databases. I use Mariadb instead of MySQL. Mariadb has tightened restrictions on db queries regarding boolean sets. Meaning inserting a true or false is no longer possible for some fields, instead it has to be 0 or 1. I am not yet sure how this is handled in regards to the queries opensim sends, but it may be a good as time as ever to take a look at the database stuff, especially since postgres is not maintained and revise the system a bit. New changes in mysql, mariadb have also enabled possible performance gains that could be of great benefit.
[11:09] Andrew Hellershanks: Which version of MariaDB introduced that change?
[11:09] Gavin.Hird why does it has to be 0 or 1? Sounds pretty non.standard to me
[11:09] Vincent.Sylvester It happened on the move from 10.0 to 10.3
[11:11] Andrew Hellershanks: ok. I have one machine using MariaDB but it is an older version so I haven't seen that issue.
[11:12] Vincent.Sylvester I just started reading changelogs yesterday, but the changes already seem promising to bring performance improvements if they are taken advantage of. Along the lines there are such changes to the "standard" of SQL and how much it allows. I suppose allowing true or false was a legacy thing that is now removed. Again, not sure if this was just with this particular pdo thing, it may not affect opensim at all depending on how the fields are setup.
[11:13] Gavin.Hird some database systems can only store 0 and 1 for a boolean filed, but some allow it to be undtermined, meaning not initialized
[11:15] Vincent.Sylvester The change I made was to the local, temporary and enabled fields in the asset server.
[11:15] Vincent.Sylvester They are setup as tinyint length 1 so I guess true is a bit longer than 1
[11:18] Andrew Hellershanks: I was just looking for a mention of the change in the MariaDB site but can't find. If the DB was storing the text version of true or false before and now it is not that could lead to some issues.
[11:19] Vincent.Sylvester I think it just no longer converts true false to 1 0 on the commit, instead it takes all statements literal at which point it will fail.
[11:19] Andrew Hellershanks: If the field is defined as tinyint(1) it shouldn't have allowed the text representations unless a conversion was being done on the fly to make it easier for humans to read.
[11:19] Vincent.Sylvester It may have been a change in 10.2 since that is a major version change.
[11:19] Andrew Hellershanks: Vincent, that is probably whats going on.
[11:22] Vincent.Sylvester So the question is how opensim handles this, if it too uses true false for fields that actually have 1 0 in them and relies on the conversion or if all db queries are in accordance to the fieldset. I have not updated any regions to run on newer mariadb since I don't want to brick them. If my power company can stop being weird I will try to setup a vm to test on, in the meantime though. since postgres and the new mysql all over some performance gains, having a look at this stuff may be a good idea regardless.
[11:24] Andrew Hellershanks: Kayaker, were there some particular queries where you noticed the problem?
[11:24] Vincent.Sylvester Especially from what I read mysql and mariadb now having some form of concurrent queries that is apparently new and faster than before. Haven't gotten to the specifics yet, but if there really is something that could reduce query time then it definitely is something worth looking at. Mesh is ever growing, inventory and assets are under massive loads, anything that can be done to reduce that, even just a few percent, is probably noticeable at this point :)
[11:24] Gavin.Hird the PGSQL is using code like Convert.ToBoolean
[11:25] Gavin.Hird so I suppose it is the implementation of that library (NPGSQL) that matters and now exactly how it is stored in the db
[11:28] Gavin.Hird in Postgres both bool and uuid are native datatypes
[11:28] Vincent.Sylvester I myself am guilty of using true false when really the field asks for 01, I never gave it much thought, probably because LSL gives no fucks about that normally, but really it is probably good practice to obey fieldsets regardless.
[11:29] Andrew Hellershanks: It is worth investigating this. I was doing a quick check and found a few tables using tinyint(1). I didn't immediately notice where true/false is being used in queries but I suspect it will require digging a little deeper in to the code than just doing a quick search.
[11:29] Bill Blight: Well technically LSL does care the TRUE/FALSE text bools are just constants for 1/0
[11:30] Andrew Hellershanks nods
[11:31] Vincent.Sylvester Kinda same issue we had with mesh, allowing the broken stuff far too long and then cause trouble when actual standard is imposed.
[11:32] Andrew Hellershanks: First step will be to locate all the boolean fields.
[11:33] Gavin.Hird first quest is this another non-standard SQL extension in Mariadb?
[11:33] Bill Blight: just a quick glance shows that they are tinyint or int
[11:33] Gavin.Hird what is the justification to not support somethign that is actually si in the SQL standard?
[11:33] Gavin.Hird is*
[11:34] Bill Blight: I'd agree with Gavin on that one
[11:34] Bill Blight: and what would this change break in existing infrastructures
[11:34] Andrew Hellershanks: I found 10 tinyint(1) references in the DB migration files.
[11:35] Vincent.Sylvester I can only guess as to the reasons for the change, I simply observed it and found my asset server suddenly blowing up trying to fetch stuff that wasn't expected
[11:35] Ubit Umarov: i really don't ubderstand what is this talk about
[11:35] Ubit Umarov: boolean is tinyint
[11:36] Gavin.Hird it is about the definition of 1 and 0
[11:36] Ubit Umarov: on mariadb it is just a alias
[11:36] Ubit Umarov: "These types are synonyms for TINYINT(1). A value of zero is considered false. Non-zero values are considered true:"
[11:36] Ubit Umarov: so what are you talking about ???
[11:36] Ubit Umarov: gezzz
[11:37] Kayaker Magic: Wouldn't the correct thing to do would be to always use some abstration, like TRUE and FALSE in the code, and have a way to define that to whatever the underlying database needed?
[11:38] Ubit Umarov: yes it should kayaker
[11:38] Gavin.Hird agreed
[11:38] Andrew Hellershanks: Ubit, MariaDB isn't handling true and false as values being passed for storage in boolean fields seems to be the issue.
[11:38] Ubit Umarov: but .. who knows what clever ppl think
[11:39] Ubit Umarov: ( "" on that clever :p )
[11:39] Bill Blight: are there even any actual boolean fields in Opensimulator, or are they all INT/TINYINT ?
[11:39] Ubit Umarov: well let us know if you see a issue on opensim
[11:39] Ubit Umarov: most filelds are tinyint since ever
[11:39] Andrew Hellershanks: Does anyone have some simple SQL queries that can show the problem?
[11:40] Bill Blight: I think you'd have to create the problem first, in this case
[11:40] Andrew Hellershanks: I found only one field that is of type boolean.
[11:40] Andrew Hellershanks: in the OS migrations files.
[11:40] Ubit Umarov: and it is also or should be, a language connector job to provide proper convertion btw types
[11:40] Andrew Hellershanks: agreed.
[11:41] Bill Blight: agreed
[11:41] Ubit Umarov: but DB ppl are so clever.. whatever
[11:41] Gavin.Hird it should be the db connector's job to convert
[11:41] Bill Blight: so if MariaDB has made a change that is outside, the MySQL spec , we can't really adopt it without possibly causing issues for people using actual mysql
[11:41] Ubit Umarov: utf8 is 3 bytes.. we all should know that..
[11:41] Ubit Umarov: :)
[11:41] Vincent.Sylvester Don't get hung up on me calling them boolean, that is what I read in the changelog "boolean type fields like (list of them all)" the issue arose with asset server which is based around sras and fsassets, so those potentially have that flaw in them.
[11:42] Vincent.Sylvester When you give tinyint(1) a true, it fails, give it a 1 and it works
[11:42] Vincent.Sylvester So searching the code for all queries that push or retrieve fields with tinyints and see if truefalse or 01 is expected/used
[11:43] Ubit Umarov: im not going to upgrade my mariadb now
[11:43] Vincent.Sylvester The db has 01 in it regardless so that should be what is sent to it, relying on a conversion from truefalse to 01 to happen, well doesn't that also a tiny amount of nanoseconds to each query ;)
[11:43] Andrew Hellershanks: Before digging in to the OS code queries it would be easier to set up a quick test DB with a table containing a boolean field and run a few SQL queries to verify the problem.
[11:44] Ubit Umarov: let us know issues if you upgrade
[11:44] Vincent.Sylvester I will setup a vm with 10.0 and some regions, then upgrade to 10.3 and see if I get red text
[11:45] Vincent.Sylvester Though how long until I get to that I can't say
[11:45] Andrew Hellershanks: If you are going to setup a VM just do some simpler tests first.
[11:46] Leighton.Marjoram quick question for my own clarity what would your asset server be looking for that has a true/false state I am little confused
[11:46] Andrew Hellershanks: Checking for temporary assets, perhaps.
[11:46] Vincent.Sylvester $this->put_stmt = $this->pdo->prepare('INSERT INTO '.DB_ASSETS_TABLE_NAME.' (id, asset_type, sha256, name, description, local, temporary, base_dir, created_at, updated_at, enabled) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)'); $this->put_stmt->execute([
$asset->local ? 1 : 0,
$asset->temporary ? 1 : 0,
$asset->enabled ? 1 : 0
[11:47] Leighton.Marjoram ah ok thank you
[11:47] Vincent.Sylvester noticed 1: 0 was true : false before
[11:47] Ubit Umarov: vincent you did not rebaked
[11:47] Gavin.Hird he is grey
[11:48] Gavin.Hird so is Andrew
[11:48] Andrew Hellershanks: Not to me I'm not. :)
[11:48] Kayaker Magic: On to another issue:
[11:48] Kayaker Magic: I'm having a problem with newer versions of OpenSim getting better (more restrictive) at detecting bad mesh prims. This is good, I'm happy to remove those prims or try to fix them. But during region startup I get warning messages that do not help me find the offending mesh. Is there a way I can find out the name or location of the offending prims?
[11:49] Gavin.Hird isn't there a log message with the region uuid and the asset id?
[11:49] Ubit Umarov: new? those things are as old as
[11:49] Andrew Hellershanks: There are several types of error messages that are reported at a low level where the error doesn't propagate up to where you can get information about the offending object.
[11:50] Ubit Umarov: fs did add some restrictions, then flags to disable them
[11:50] Vincent.Sylvester By default that part of startup does not "know" which one that is, but you can add some code that does spam the console with that info yes, it may even just be commented in the code somewhere.
[11:50] Vincent.Sylvester I think I made a patch some time ago to catch the termination that happens on really bad mesh
[11:50] Bill Blight: the errors you are seeing are most likely from warp3d
[11:50] Bill Blight: about mesh rendering on the map
[11:50] Vincent.Sylvester I think Ubit put the same code into master to stop said termination
[11:51] Ubit Umarov: think it was code to try to see the problematic prim
[11:51] Ubit Umarov: not just "you have a bad mesh"
[11:51] Ubit Umarov: :)
[11:52] Vincent.Sylvester There is a line that is commented in there I think I uncommented that and got like every mesh name uuid it ran through
[11:52] Bill Blight: or you just turn your logging level down
[11:52] Bill Blight: :P
[11:53] Bill Blight: and you don't see the errors
[11:53] Bill Blight: :P
[11:53] Andrew Hellershanks: I have log level at DEBUG and don't see UUID's of bad prims.
[11:53] Ubit Umarov: not all cases are possible to chow
[11:54] Ubit Umarov: or show
[11:54] Andrew Hellershanks: It would be useful to have UUID's reported. Same thing for bad textures.
[11:54] Bill Blight: is that even feasible during map generation
[11:55] Andrew Hellershanks: When bad items are found at a low level some default value gets passed up. I've seen a case of 'null' being passed up the chain. I suspect that the higher level code then skips further processing without reporting on the unexpected(?) value.
[11:55] Vincent.Sylvester OpenSim\Region\PhysicsModules\ubOdeMeshing\Meshmerizer.cs has a bunch of commented lines that output various things about the meshes it processes, just uncomment them all and watch the console for the warnings it spits out hehe
[11:56] Andrew Hellershanks: I suspect that will spam the console.
[11:56] Vincent.Sylvester Yes, yes it will :D
[11:57] Andrew Hellershanks: We are almost at the top of the hour. Is there anyone who needs to leave shortly who has a question they wanted to ask?
[11:58] Bill Blight: I try not to get too hung up on console errors if the sim runs fine ... LOL ,, I'd spend all my time chasing things that are not actually causing usage problems ..  :P
[11:58] Gavin.Hird Not a question: Updated viewers are at
[11:58] Andrew Hellershanks: Thanks, Gavin.
[11:58] Gavin.Hird tha macOS version no will not crash a machine with Intel graphics and T2 processor
[11:58] Vincent.Sylvester Well bad mesh can cause increased resource usage if there are broken physics constantly screwing about, especially if you create physical vehicles and leave them sitting around.
[11:59] Gavin.Hird the RLV windows version sia major RLV refresh
[11:59] Gavin.Hird is a*
[11:59] Vincent.Sylvester It also runs a whole lot better than Firestorm, like almost twice the fps :D
[11:59] Andrew Hellershanks: I thought of some viewer issue just now I was going to comment on but I've forgotten what it was.  :P
[11:59] Cuga Rajal: Dayturn has never run on my Mac system
[12:00] Gavin.Hird what Mac do you have?
[12:00] Cuga Rajal: Macbook pro 13" 2014, latest High Sierra
[12:00] Bill Blight: If you are leaving non-used physical vehicles sitting around and they are still physical, and causing issues, you need to re-visit your scripts as much as you need to fix the physics mesh
[12:00] Gavin.Hird OK, try the new build of today
[12:00] Cuga Rajal: I get an error when unpacking, says app is damaged and must be deleted
[12:00] Cuga Rajal: ok will try
[12:01] Cuga Rajal: checking now
[12:01] Vincent.Sylvester Just beware the windows version may get caught by antivirus because it is built on a vm, rest assured that is a false-positive, I checked :)
[12:01] Vincent.Sylvester *throws stones at firewall*
[12:02] Gavin.Hird Cuga, look in Library/Application Support and remove the Dayturn folder and also the KokuaOS folder if you have one
[12:02] Cuga Rajal: I still get message Dayturn” is damaged and can’t be opened. You should move it to the Trash.
[12:02] Cuga Rajal: Wont launch
[12:02] Andrew Hellershanks: How are you downloading the file, Cuga?
[12:02] Gavin.Hird what is your gatekeeper setting like?
[12:03] Cuga Rajal: The DMG appears to sdownload fine, and I copy the app from the DMG to the desktop
[12:03] Gavin.Hird it shall not run from the desktop
[12:03] Gavin.Hird really
[12:03] Gavin.Hird put it in Applications
[12:03] Cuga Rajal: OK I'll move it to apps
[12:04] Cuga Rajal: I put it in /Applications, right click -> Open, still get damaged message
[12:04] Gavin.Hird very strange
[12:04] Cuga Rajal: Maybe a high sierra thing?
[12:04] Gavin.Hird it is even code signed with an Apple dev cert
[12:04] Gavin.Hird no
[12:05] Gavin.Hird do you have MacKeeper on your system?
[12:05] Cuga Rajal: Not a code signing issue
[12:05] Gavin.Hird or some other antivirus shit
[12:05] Cuga Rajal: Ihave no antivisrus
[12:05] Gavin.Hird MacKeeper?
[12:05] Cuga Rajal: Debug message would be useful
[12:05] Cuga Rajal: Nope
[12:06] Andrew Hellershanks: Is there an sha or md5 sum on the file that can be checked to make sure the file was downloaded correctly?
[12:06] Gavin.Hird long sisnce you verified your disk?
[12:06] Cuga Rajal: downloaded it from your link, the DMG has its own internal crc
[12:06] Vincent.Sylvester Maybe start in terminal gives out an error?
[12:06] Cuga Rajal: I can try
[12:06] Gavin.Hird both the DMG and the app and plugins are signed
[12:07] Cuga Rajal: You mean the Open command, or running the binary inside?
[12:07] Gavin.Hird use disk utils and verify the disk as it sounds like there might be some corruption
[12:07] Cuga Rajal: I do that regularly
[12:08] Cuga Rajal: Ive downloaded diff versions many times, always same result, never have any other system problems
[12:08] Cuga Rajal: No other app gives me that error, except apps from 10 years ago
[12:08] Gavin.Hird open Console and look at the system log when it opens
[12:08] Cuga Rajal: ok
[12:09] Cuga Rajal: I am trying to run macos binary at cl
[12:09] Cuga Rajal: will see what happens
[12:09] Cuga Rajal: Oh, it launches when I do ./Dayturn
[12:09] Cuga Rajal: from MacOS folder
[12:09] Cuga Rajal: Lots of console spew
[12:10] Cuga Rajal: But wont run when double-clicking on icon, or using right click->Open
[12:10] Gavin.Hird very strange
[12:10] Cuga Rajal: Is console spew useful? I can copy it into a NC
[12:10] Gavin.Hird we had that with Kokua some years ago
[12:11] Cuga Rajal: preparing NC with console output
[12:12] Gavin.Hird did it generate a log file in ~Libary/Application Support/Dayturn/logs?
[12:12] Cuga Rajal: lemme check
[12:13] Cuga Rajal: There is a log, 39k
[12:13] Cuga Rajal: one launch
[12:14] Cuga Rajal: Want it?
[12:15] Gavin.Hird actually it would be the same you sent me in the NC
[12:15] Cuga Rajal: ok
[12:15] Andrew Hellershanks: If there isn't anything further for todays meeting I'll call this one to a close and let Gavin and Cuga sort out the problem using the latest Mac viewer.
[12:15] Kayaker Magic: I gota run anyway. Good meeting!
[12:16] Cuga Rajal: Isnt NC limit 32k?
[12:16] Andrew Hellershanks: See you all again next week.

Personal tools
About This Wiki