Chat log from the meeting on 2016-06-28

[11:05] George Equus: So quiet here...
[11:07] Lucy Afarensis: Is there a meeting today ?
[11:08] George Equus: Should be I suppose
[11:09] Robert Adams: there is a meeting and here we are
[11:09] Lucy Afarensis: Here we are then
[11:10] Gavin.Hird @grid.xmir.org:8002: yup yup
[11:10] George Equus: indeed we are
[11:10] Gavin.Hird @grid.xmir.org:8002: anyone having fun yet this week?
[11:11] Lucy Afarensis: It has been quiet
[11:11] Lucy Afarensis: for me
[11:11] Billy.Bradshaw @hg.viewtwo.net:8600: I was unable to leave a sim running 0.8.1, either to a 0.8.2 or .9.0
[11:12] Billy.Bradshaw @hg.viewtwo.net:8600: is that correct?
[11:12] Billy.Bradshaw @hg.viewtwo.net:8600: I did not know another 8.1 to try that option
[11:12] Lucy Afarensis: They crashed ?
[11:13] Billy.Bradshaw @hg.viewtwo.net:8600: No, all ok just said something about blocked, should have made a note
[11:13] Billy.Bradshaw @hg.viewtwo.net:8600: I had to logout to recover
[11:16] Gavin.Hird @grid.xmir.org:8002: Inspired by the European Football Championship i set out ot make a mesh football. Once done I found a script that made it kickable. The problem was a tiny kick sent it across the sim, and trying to find it back I discovered Areas Search was broken, so I had to fix that in the viewer. – One thing led to the other
[11:19] Billy.Bradshaw @hg.viewtwo.net:8600: @Robert, can the collision frame around an avatar be any rectangle or has it to be a rectangle with perpendicular angles?
[11:21] Robert Adams: the current collision shape of an avatar in BulletSim is a rectangle that's as tall as the avatar
[11:22] Robert Adams: I've played with more complex mesh shapes but it's not turned on
[11:22] Billy.Bradshaw @hg.viewtwo.net:8600: ok, thanks
[11:24] Sheera Khan: I really enjoy the activity in the mantis and git as seen last weeks :-)
[11:25] Robert Adams: it is being worked on :)
[11:25] Sheera Khan: finally there is some progress again :-)
[11:26] Lucy Afarensis: I have been getting some strange errors in my sim consoles lately
[11:26] Robert Adams: like what, Lucy?
[11:27] Lucy Afarensis: May I past them in Chat ?
[11:27] Lucy Afarensis: 2016-06-24 10:25:53,512 WARN (1) - OpenSim.Region.Framework.Scenes.SceneObjectGroup [SCENE OBJECT GROUP]: SavedScriptState element had no UUID in object Ice Fae Forest Foot - Left Lower Leg*
2016-06-24 10:25:54,411 WARN (1) - OpenSim.Region.Framework.Scenes.SceneObjectGroup [SCENE OBJECT GROUP]: SavedScriptState element had no UUID in object Ice Fae Forest Foot - Right Lower Leg*
2016-06-24 10:25:56,993 WARN (1) - OpenSim.Region.Framework.Scenes.SceneObjectGroup [SCENE OBJECT GROUP]: SavedScriptState element had no UUID in object Maleficent's brown wings
[11:27] Gavin.Hird @grid.xmir.org:8002: the http server update caused some grief, but there was an update today, and so far it seems more well-behaved
[11:27] Lucy Afarensis: None of this is on my sims
[11:28] Lucy Afarensis: I dozens of lines of this stuff
[11:28] Gavin.Hird @grid.xmir.org:8002: I see some of these too for one particular object that is not even rezzed anywhere
[11:28] George Equus: Ice Fae Forest Foot - Left Lower Leg seem very familiar, had that in my console for ages but now it is gone
[11:28] George Equus: Forgot what caused it,,,
[11:29] Sheera Khan: could it be located on a sim next to yours?
[11:29] Lucy Afarensis: No I have looked
[11:29] Gavin.Hird @grid.xmir.org:8002: for my part it is not rezzed anywhere at all
[11:29] Gavin.Hird @grid.xmir.org:8002: it has been at some time
[11:30] Gavin.Hird @grid.xmir.org:8002: but it exist in the asset server as xml
[11:31] George Equus: Some form of lingering DB corrupted items maybe?
[11:31] Robert Adams: Lucy, you might try cleaning the script cache and letting things recompile... the directory is usually bin/ScriptEngines/
[11:31] Gavin.Hird @grid.xmir.org:8002: probalby some in-world object pointing where it should not be pointing
[11:31] Lucy Afarensis: I am seeing the same errors in a sim I have on Franco grid
[11:32] Lucy Afarensis: I try that Robert Thanks
[11:33] Gavin.Hird @grid.xmir.org:8002: Is the script cache scanned at regular intervals as the message is pretty periodic
[11:33] Gavin.Hird @grid.xmir.org:8002: probably once every hour
[11:33] Robert Adams: there have also been some complaints about XML serialization problems... but there should be other errors if that was the problem
[11:34] Lucy Afarensis: I always see it when rebooting
[11:36] Gavin.Hird @grid.xmir.org:8002: Microsoft announced availablility of .NET 1.0 for Linux and macOS yesterday. Does that perhaps mean we can do away with mono?
[11:36] Gavin.Hird @grid.xmir.org:8002: not .NET 1.0 but the 1.0 of the open version
[11:36] Billy.Bradshaw @hg.viewtwo.net:8600: I am testing core 1.0 on redhat at present, our intention is to replace mono
[11:37] Simulator Version v0.5 ruft: OpenSim 0.9.0.0 Dev bc4e5d8: 2016-06-26 14:40:26 +0100 (Unix/Mono)
[11:37] Robert Adams: need to decide if we're supporting both or what
[11:37] Gavin.Hird @grid.xmir.org:8002: what is the experience so far?
[11:38] Billy.Bradshaw @hg.viewtwo.net:8600: some complications on configurations but seems to run the basic standalone
[11:38] Gavin.Hird @grid.xmir.org:8002: ok
[11:38] Billy.Bradshaw @hg.viewtwo.net:8600: I am travelling at present so progress is slow
[11:38] Snakedance Moonwing: holy batfarts is nebadon :)
[11:38] George Equus: Long time no see
[11:39] Lucy Afarensis: ☻
[11:39] Nebadon Izumi: guess my wave gesture isnt on lol
[11:39] Robert Adams: time to dance!!
[11:39] Sheera.Khan @hypergrid.org:8002: Welcome back on board :-)
[11:40] Billy.Bradshaw @hg.viewtwo.net:8600: music Maestro
[11:40] Nebadon Izumi: thanks, try to be here when I have some time, summer is not so busy now, everyone taking vacations :)
[11:40] Ubit Umarov: ( bullet :p )
[11:40] Snakedance Moonwing: i am not a dev btw :)
[11:41] Snakedance Moonwing: just a curious resident :)
[11:41] Nebadon Izumi: This meeting is for everyone :)
[11:41] Snakedance Moonwing: :)
[11:41] Snakedance Moonwing: is nice to see you again neb :)
[11:42] Nebadon Izumi: you too :)
[11:42] Snakedance Moonwing: :P
[11:42] Nebadon Izumi: I have been kind of lurking around for a bit
[11:42] Nebadon Izumi: im still using opensim on a daily basis for my job
[11:42] George Equus: Upsala?
[11:42] Nebadon Izumi: finally got my home servers sorted out and my osgrid regions back up
[11:42] Nebadon Izumi: Ya, a lot of our projects are in Sweden
[11:43] Nebadon Izumi: a few in the US now too
[11:43] George Equus: :)
[11:43] Nebadon Izumi: i traveled to University of Utah last month to train some students
[11:43] Nebadon Izumi: how to build in OpenSim
[11:43] Snakedance Moonwing: spiffy :)
[11:43] George Equus: Like the model of Upsala Domkyrka!
[11:43] Billy.Bradshaw @hg.viewtwo.net:8600: info: there is a DNS issue with later mono developments, had a patch applied recently that reverts some of the problem code but not all. Latest 4.4.0 has the patch
[11:43] Nebadon Izumi: nice
[11:43] Nebadon Izumi: i just upgraded to 4.4.1
[11:44] Nebadon Izumi: maybe i should upgrade the plaza boxes
[11:44] Billy.Bradshaw @hg.viewtwo.net:8600: patch is in 4.4.1
[11:44] Nebadon Izumi: ok that is good to know
[11:44] Nebadon Izumi: i will uprade plaza08 and plaza09 later
[11:44] Gavin.Hird @grid.xmir.org:8002: I get 3x CPU use on 4.4.1 over 4.2.3.4
[11:44] Nebadon Izumi: dang
[11:45] Nebadon Izumi: ive not seen anything like that
[11:45] Billy.Bradshaw @hg.viewtwo.net:8600: the issue can appear under specific circumstance, yet to be resolved
[11:45] Nebadon Izumi: let me see what version we are running here now
[11:45] Gavin.Hird @grid.xmir.org:8002: the release notes says that certain kinds of locking is 3 times more expensive than earlier versions
[11:45] Gavin.Hird @grid.xmir.org:8002: perhaps that has something to do with it
[11:46] Billy.Bradshaw @hg.viewtwo.net:8600: The issue is with reverse DNS, not all ISP play ball
[11:46] Nebadon Izumi: ah that's right we reverted back we have 3.12.1 running here on plaza08
[11:46] Billy.Bradshaw @hg.viewtwo.net:8600: as they say, your mileage will vary
[11:46] Nebadon Izumi: 3.2.3 on plaza09
[11:46] Nebadon Izumi: yea
[11:46] Nebadon Izumi: 4.4.1 has been running quite well for me locally
[11:47] Nebadon Izumi: i havent done a ton of testing though, but its been solid
[11:47] Nebadon Izumi: im also running a fairly new kernel too though
[11:47] Nebadon Izumi: 4.1.12 kernel
[11:47] Nebadon Izumi: opensuse leap 42.1
[11:47] Lucy Afarensis: How can you determine what version of mono you have ?
[11:47] Nebadon Izumi: type "mono -V"
[11:47] Gavin.Hird @grid.xmir.org:8002: mono --version
[11:48] Lucy Afarensis: cool
[11:48] Snakedance Moonwing: sits contently listening to the smart people :)
[11:49] Billy.Bradshaw @hg.viewtwo.net:8600: benefits of later mono are really apparent with sgen improvements
[11:49] Nebadon Izumi: yea, I have seen slightly higher CPU at my 2 physics var regions
[11:49] Nebadon Izumi: Sisyphus and Newton
[11:49] Nebadon Izumi: sit at about 50% of a single core sitting idle no visitors
[11:49] Nebadon Izumi: they are neighbors here on OSgrid
[11:50] Nebadon Izumi: someone mentioned that the problem may have something to do with 2 vars being neighbors
[11:50] Nebadon Izumi: i havent really tested that fully though
[11:50] Billy.Bradshaw @hg.viewtwo.net:8600: How many sripts prims for that figure ned?
[11:50] Nebadon Izumi: let me look
[11:50] Robert Adams: I'm looking into CPU and BulletSim... it does use more than it should on an idle sim
[11:50] Nebadon Izumi: 7945 prims and 753 scripts
[11:50] Billy.Bradshaw @hg.viewtwo.net:8600: of course other factors come into play like cpu speed etc
[11:50] Nebadon Izumi: Sisyphus runs Bulletsim
[11:50] Robert Adams: there do seem to be a lot of ways to use up CPU even on a pretty quiet sim
[11:50] Nebadon Izumi: and Newton runs Ubode
[11:51] Nebadon Izumi: Ubode sits at like 40% of a single core
[11:51] Nebadon Izumi: and Bullet around 50%
[11:51] Nebadon Izumi: they flucuate a bit though
[11:51] Nebadon Izumi: they both probably have active physical prims
[11:51] Nebadon Izumi: like some chains
[11:51] Nebadon Izumi: and few spheres rolling around probably
[11:51] Nebadon Izumi: but nothing major
[11:51] Robert Adams: chains are constant work :))
[11:52] Nebadon Izumi: yea however I dont recall it being so high in the past
[11:52] Nebadon Izumi: with the same load
[11:52] Nebadon Izumi: I havent changed those regions much in a long while
[11:52] Robert Adams: I think there are some other things using up CPU that shouldn't be.... I'm adding some tools to find out where time is being spent
[11:52] Nebadon Izumi: cool
[11:52] Nebadon Izumi: look forward to testing
[11:52] Robert Adams: cpu is higher than it should be
[11:53] Gavin.Hird @grid.xmir.org:8002: as stated before I find that idle sim CPU both with ubODe and Bullet is 50% higher on 0.9 over 0.8.2.1
[11:53] Nebadon Izumi: ya that sounds pretty correct
[11:53] Nebadon Izumi: i sort of recall these regions hovering around 20-30% at most
[11:53] Nebadon Izumi: with 1 avatar it shoots up to around 80-85%
[11:53] Nebadon Izumi: even if avatar is just standing not moving
[11:53] Gavin.Hird @grid.xmir.org:8002: seeing the same
[11:54] Billy.Bradshaw @hg.viewtwo.net:8600: some people report higher memory usage on linux, but they should be aware that linux caches files in memory as a function of the kernel, you can clear the cached files if you wish
[11:54] Nebadon Izumi: yea
[11:55] Nebadon Izumi: for me memory seems better
[11:55] Billy.Bradshaw @hg.viewtwo.net:8600: like so
[11:55] Nebadon Izumi: than previous versions
[11:55] Billy.Bradshaw @hg.viewtwo.net:8600: free -m && sync && echo 3 > /proc/sys/vm/drop_caches && free -m
[11:55] Nebadon Izumi: that could be mono itself though
[11:55] Gavin.Hird @grid.xmir.org:8002: if you use something like glances you get a good idea of what's going on with cpu and memory
[11:55] Nebadon Izumi: like you said better sgen
[11:55] Nebadon Izumi: actually now I think about it too
[11:55] Nebadon Izumi: with the C# bullet on my raspberry pi
[11:55] Nebadon Izumi: cpu is incredibly higher
[11:55] Nebadon Izumi: i forgot about that
[11:56] Nebadon Izumi: it's averaging 85-100% of a single core
[11:56] Nebadon Izumi: when sitting idle
[11:56] Gavin.Hird @grid.xmir.org:8002: with 4.4.1?
[11:56] Nebadon Izumi: that is alot higher
[11:56] Nebadon Izumi: yea raspberry pi is running 4.4.1 too
[11:56] Billy.Bradshaw @hg.viewtwo.net:8600: mono go into some detail as to the sgen improvement with 4.4.0 release notes
[11:56] Gavin.Hird @grid.xmir.org:8002: right, because that is what I see for idle sim with 4.4.1
[11:56] Robert Adams: the PI implementation must have been using the software version of Bullet... that will be CPU intensive
[11:56] Billy.Bradshaw @hg.viewtwo.net:8600: and a locking issue with the mac
[11:57] Nebadon Izumi: the pi is very underpowered and it is also running the C# bullesimXNA
[11:57] Nebadon Izumi: but still on older opensim the cpu was much much lower
[11:57] Ubit Umarov: hmm my empty var is taking 0% cpu in Win
[11:57] Gavin.Hird @grid.xmir.org:8002: macOS version behaves a bit different as it mostly use LLVm and not the mono JIT
[11:57] Nebadon Izumi: there is a noticable feeling of lag too when i drive the vehicle around on the pi as well
[11:57] Nebadon Izumi: it wasnt horrible but it was noticeable change
[11:58] Billy.Bradshaw @hg.viewtwo.net:8600: I have a modest welcome region which rarely goes above 5% cpu
[12:00] Gavin.Hird @grid.xmir.org:8002: a simulator with 11 regions used to be 10-12% in 0.8.1. With 0.9 and mono 4.4.1 it is about 90%
[12:00] Gavin.Hird @grid.xmir.org:8002: 0.8.2.1
[12:00] Robert Adams: I have also noticed a big difference between Windows and Linux cpu... haven't found out why that would make a difference though
[12:02] Gavin.Hird @grid.xmir.org:8002: here is one of the messages we talked about earlier:
[12:02] Gavin.Hird @grid.xmir.org:8002: 21:02:08 - [SCENE OBJECT GROUP]: SavedScriptState element had no UUID in object ZHAO Anim Overrider - SP 0.7 (full perm)
[12:03] Gavin.Hird @grid.xmir.org:8002: not rezzed anywhere
[12:04] Lucy Afarensis: I have dozens of lines of this in my logs
[12:04] Robert Adams: Gavin, that error come out of the script deserializer (complaining about things not found in the XML)... are you on mono?
[12:04] Gavin.Hird @grid.xmir.org:8002: yep
[12:04] Lucy Afarensis: yes
[12:05] Gavin.Hird @grid.xmir.org:8002: but why is it complaining about something that is not rezzed and probably has not been for years
[12:05] Lucy Afarensis: none of this was ever on mine
[12:06] Nebadon Izumi: I have seen things spew serialization errors on console before and the object in world is actually invisible
[12:06] Nebadon Izumi: mostly here on this region heh
[12:06] Nebadon Izumi: its been a while since i have seen that happen though
[12:07] Lucy Afarensis: Iwas wondering if they might be part of an npc on a dance ball
[12:07] Nebadon Izumi: i guess that is a possibility
[12:08] Gavin.Hird @grid.xmir.org:8002: even if the NPC is not in-world?
[12:08] Robert Adams: don't know why it would be an old object... the code paths seem to be about rezzing an object
[12:08] Gavin.Hird @grid.xmir.org:8002: only as part of the description I mean
[12:08] Nebadon Izumi: hmm probably not
[12:08] Nebadon Izumi: unless it happens when you are making a OAR or something maybe
[12:09] Gavin.Hird @grid.xmir.org:8002: as far as I can see it came in via an IAR
[12:09] Nebadon Izumi: npcs can spew all kinds of wierd errors on IAR import and exports
[12:09] Gavin.Hird @grid.xmir.org:8002: because the name in the asset table is From IAR
[12:09] Nebadon Izumi: but generally it wont persist those kind of errors
[12:09] Nebadon Izumi: usually you just see it during the import/export
[12:09] Nebadon Izumi: if its constantly happening that means it's in the sim or perhaps an attachment maybe
[12:10] Gavin.Hird @grid.xmir.org:8002: it is every hour - most likely as part of the flotsamcache update
[12:10] Nebadon Izumi: hmm
[12:10] Gavin.Hird @grid.xmir.org:8002: I'll see if I can find it stuck somewhere in world
[12:11] Nebadon Izumi: ya see if you can track it down by location or to a particular primid
[12:11] Nebadon Izumi: may be able to find coordinates in the database
[12:11] Nebadon Izumi: I've seen wierd things happen where root prim gets erased
[12:11] Nebadon Izumi: but all the child prims remain
[12:11] Nebadon Izumi: and cause wierd issues
[12:11] Nebadon Izumi: wierd spurious console messages
[12:11] Nebadon Izumi: those tend to also be invisible in world
[12:11] Gavin.Hird @grid.xmir.org:8002: I can only find it in the asset table in the DB
[12:12] Nebadon Izumi: but can still cause problems with the scene
[12:12] Nebadon Izumi: in the Robust asset table?
[12:12] Gavin.Hird @grid.xmir.org:8002: yes
[12:12] Nebadon Izumi: hrmm
[12:12] Nebadon Izumi: prim inventory uuids are stored as a blob right?
[12:13] Gavin.Hird @grid.xmir.org:8002: yes
[12:13] Nebadon Izumi: those may not be so easily searchable on a database query
[12:13] Gavin.Hird @grid.xmir.org:8002: but you can regex the blob - at least Postgres can, so it exist in there
[12:13] Nebadon Izumi: you could try saving an oar
[12:13] Nebadon Izumi: and loading it into a standalone
[12:13] Nebadon Izumi: and see if you get similar behavior
[12:14] Lucy Afarensis: I have done and it show up
[12:14] Lucy Afarensis: in the standalone
[12:14] Nebadon Izumi: you have anything running on script timer?
[12:14] Nebadon Izumi: try disable script engine in the standalone
[12:15] Nebadon Izumi: and see if it goes away
[12:15] Lucy Afarensis: will do
[12:15] Nebadon Izumi: worth a shot
[12:15] Nebadon Izumi: see if you can isolate what is causing it to happen every hour
[12:16] Nebadon Izumi: if no change there
[12:16] Nebadon Izumi: try disable flotsam cache
[12:16] Nebadon Izumi: and see if that has any change
[12:17] Gavin.Hird @grid.xmir.org:8002: can do that
[12:17] Nebadon Izumi: maybe grep the fcache folder too
[12:17] Nebadon Izumi: for the uuid
[12:18] Robert Adams: if you have the UUID, you can do a 'show asset UUID' on the console
[12:19] Nebadon Izumi: ah ya forgot about that command
[12:20] Lucy Afarensis: Error says no UUID
[12:20] Lucy Afarensis: SavedScriptState element had no UUID in object Maleficent's brown wings
[12:20] Gavin.Hird @grid.xmir.org:8002: the only place I can find the name / description "ZHAO Anim Overrider - SP 0.7 (full perm)" is if I do a regex search inside the XML in the asset table and thereby get the asset UUID
[12:20] Gavin.Hird @grid.xmir.org:8002: it is not rezzed anywhere
[12:20] Nebadon Izumi: strange
[12:21] Gavin.Hird @grid.xmir.org:8002: very strange
[12:21] Nebadon Izumi: maybe those wings are inside a box or a vendor or something?
[12:21] Gavin.Hird @grid.xmir.org:8002: they are in a box encoded in the XML
[12:21] Nebadon Izumi: a box inside a box
[12:22] Lucy Afarensis: I have never seen that term before
[12:22] Ubit Umarov: that is an AO, it's an attachment
[12:22] Ubit Umarov: or maybe in an attachment..
[12:22] Gavin.Hird @grid.xmir.org:8002: the AO is inside a box of other items made by Arcadia
[12:23] Gavin.Hird @grid.xmir.org:8002: it was imported via IAR from OSGrid
[12:24] Gavin.Hird @grid.xmir.org:8002: I can't rezz it because there is no corresponding inventory item
[12:24] Nebadon Izumi: yea unfortunately Gavin it just may be one of those old unfixable assets
[12:24] Gavin.Hird @grid.xmir.org:8002: I can just delete the asset record :-))
[12:25] Nebadon Izumi: ya that might be best Gavin
[12:25] Nebadon Izumi: if it's not something you are worried about
[12:25] Nebadon Izumi: if it never worked then no major loss
[12:25] Nebadon Izumi: unfortunately OSgrid is chock full of old bad assets
[12:25] Gavin.Hird @grid.xmir.org:8002: no loss at all
[12:26] Nebadon Izumi: things that came in via Second Innventory especially bad
[12:26] Lucy Afarensis: I have an inventory full of those
[12:26] Nebadon Izumi: no doubt i created 100s of bad assets myself
[12:26] Nebadon Izumi: lol
[12:26] Gavin.Hird @grid.xmir.org:8002: but the interesting thing is why does the server even chew over it at regular intervals?
[12:26] Gavin.Hird @grid.xmir.org:8002: seems like wasted work
[12:26] Nebadon Izumi: yea i doubt it has much impact
[12:26] Nebadon Izumi: it could just be fcache
[12:26] Lucy Afarensis: Thanks for talking about this
[12:27] Nebadon Izumi: we have opensim spit out a lot of messages for debugging purposes
[12:27] Ubit Umarov: odd is that means region thinks script is running so needs to save its state
[12:27] Lucy Afarensis: it has been bugging me
[12:27] Ubit Umarov: hmm i guess
[12:27] Gavin.Hird @grid.xmir.org:8002: but does the fcache traverse the entire asset table?
[12:27] Nebadon Izumi: no
[12:27] Nebadon Izumi: i cant think of anything that does that
[12:28] Gavin.Hird @grid.xmir.org:8002: I hope not :-)
[12:28] Nebadon Izumi: there must be some obscure reference some where in the region
[12:29] Nebadon Izumi: bye all, thanks for coming