Chat log from the meeting on 2007-11-27

[11:01] Teravus Ousley: new people seem to be less gray to me [11:01] Neas Bade: yes, agreed

[11:01] Kurt Stringer: Heoo all

[11:02] Kurt Stringer: lol

[11:02] Teravus Ousley: Hai Kurt

[11:02] Justin Clark-Casey: hey kurt

[11:02] Kurt Stringer: hello all

[11:02] Neas Bade: hey

[11:02] Adam Frisby: lo

[11:04] Teravus Ousley: 0x0 textures.. asset server is definately hosed

[11:04] Teravus Ousley: .

[11:04] Adam Frisby: Woot

[11:04] Adam Frisby: Just like the main grid. :D

[11:05] Neas Bade: yeh, we need to get this off of the grid asset server, which is just very broken right now

[11:06] Adam Frisby: Or add in a bit of intelligent caching

[11:06] Adam Frisby: a sim should have a copy of every texture in prims in it

[11:06] Adam Frisby: err on prims

[11:06] Adam Frisby: ditto for common assets, eg the moon / sun, etc.

[11:09] More Wright: is no one talking, or can I just not hear anything?

[11:09] Teravus Ousley: no one is talking

[11:09] Teravus Ousley: we're all changing our appearance :P

[11:09] Adam Johnson: We are telling secrets about you ;)

[11:09] Teravus Ousley: .

[11:09] Adam Frisby: Heh

[11:09] Neas Bade: :)

[11:09] More Wright: ahh cool

[11:10] Adam Frisby: All secrets about me, like how I'm going to completely stack myself on the slopes in Japan this weekend, courtesy of Adam.

[11:10] Adam Frisby: hehe

[11:10] More Wright: you two going sking?

[11:10] Adam Frisby: Yeah

[11:10] More Wright: and the rest of us wasn't invited?

[11:10] Adam Johnson: yeah, gonna hopefully not break my neck. ;)

[11:11] Adam Johnson: You are all welcome anytime. :P

[11:11] Adam Frisby: Well, if your in Japan. hehe.

[11:11] Adam Johnson: Indeed. haha

[11:12] Adam Frisby: Anyway

[11:12] Adam Frisby: Agenda?

[11:12] Neas Bade: I had some proposals on the list

[11:12] Neas Bade: given that Adam J is here, I'd suggest we start with the packet overhead findings the 3Di team found

[11:13] Neas Bade: I think that would be very interesting

[11:13] Adam Johnson: Heh, sure.

[11:13] Teravus Ousley: very reasonable

[11:13] Adam Johnson: Though it is opening a fun topic involving libsl.

[11:13] Neas Bade: opening fun topics is always allowed :)

[11:13] Adam Johnson: Well, do we push on with .3 of libsl, or completely do away with it?

[11:14] Adam Frisby: I had a chat with Adam earlier this evening about that latter option -- having classes of each packets may be a very inefficient way of handling things.

[11:14] Adam Frisby: It might be better to have a: byte[] buildXYZPacket(A, B, C, D); function for every packet.

[11:15] Teravus Ousley: shh, nobody type.. .. my boss is looking over my shoulder.. j/k

[11:16] Adam Frisby: Hehe

[11:16] Adam Johnson: So, I guess the big question on 3DI's end would be, how much work will each option involve.

[11:16] Neas Bade: right

[11:17] Neas Bade: I think the issue that was hit with a previous attempt at 0.3 update was a bunch of fields got marked private that busted opensim

[11:17] Adam Frisby: 90% of libsls codebase is generated procedurally by another application, we should be able to modify that to produce the buildPacket equivilents pretty easily.

[11:18] Adam Frisby: But, there is extra stuff which we use like their layerdata encoder/decoders.

[11:18] Teravus Ousley: They also have 'all' of their packets in one CS file.. _packets.cs

[11:18] More Wright: that is what is built precedurally

[11:18] Adam Johnson: and the openjpeg bits as well.

[11:19] Adam Frisby: True, although I made the suggestion to p0pe about moving their OpenJpeg to a seperate project called OpenJpeg.net or something

[11:19] More Wright: they as far as I know have a application that takes the message_template file and produces the _packet.cs file from it

[11:19] Adam Frisby: since there's more than just libsl using that now

[11:19] Adam Johnson: True

[11:20] Adam Frisby: Hrrm

[11:20] Teravus Ousley: odd, does the message template have enough information to do that?

[11:20] Adam Frisby: These performance issues -- they should be hitting libsl too right?

[11:20] Adam Frisby: Tervaus: yes.

[11:20] Adam Johnson: They are actually..

[11:20] Adam Frisby: libsl might appreciate switching to buildPacket equivilents internally if we can demonstrate a big performance increase.

[11:20] Teravus Ousley wonders what message template they are looking at :P

[11:20] Adam Johnson: I notice these spikes on login/teleports in MovableLife

[11:21] More Wright: Tervaus, the one included with the SL client

[11:21] Adam Frisby: msg_template.msg

[11:21] Adam Frisby: Open it in notepad

[11:21] Teravus Ousley: hmm.. I did.. but it didn't seem to have enough info specifically on packet payloads.

[11:21] Teravus Ousley shrugs..

[11:21] Teravus Ousley: maybe they figured out something..

[11:22] Adam Frisby: We dont use anything more complex than msg_template in opensim itself

[11:22] Teravus Ousley: anyway.. different topic.. back to performance

[11:22] Adam Frisby: we have to figure out the payloads too

[11:23] Adam Johnson: Indeed.

[11:23] Teravus Ousley: so the main bog is not dealing with them as byte arrays the majority of the time?

[11:25] Adam Johnson: The main bog is coming from GC, because we keep creating new packets and never reuse them.

[11:25] Adam Frisby: Hrrm.

[11:25] Adam Frisby: Create a circular list of each packet type for re-use. Mark as "in use" / "not in use"?

[11:25] Adam Johnson: On login the number of packets created is pretty huge.. and if you check out heapbuddy, you can see that the GC is pretty intense at that point.

[11:25] Adam Frisby: after it's been sent, reset the use flag

[11:25] Teravus Ousley: Adam beat me to it :P

[11:26] Adam Frisby: Question: did you guys find any other performance issues while you were profiling?

[11:27] Adam Johnson: So far, that was the only one that we have found.. but I think we need to do some more analyzing.

[11:27] Adam Johnson: and, it would be good to have a custom profiler created...

[11:27] Adam Frisby: I've got one I've got semi-completed, the catch is it's windows-only since it relies on WMI to get profiling information.

[11:27] Adam Johnson: That would still be of use to us I think.

[11:28] Adam Frisby: Alright, I'll send you guys a copy -- it's designed for grid information mainly.

[11:28] Adam Frisby: health / status / etc.

[11:28] Adam Johnson: Okay great, we have been mainly dealing with sandbox, but will be doing some grid tests shortly.

[11:30] Adam Frisby: Alright, well - seems like we should look at re-using rather than disposing of packets

[11:30] Adam Frisby: MW: This is probably something you have the most experience with, how long do you think it will take?

[11:30] Adam Johnson: Yep, I think it will fix a lot of the login cap problems

[11:31] Neas Bade: it would be good to get some statistic counts on packet types being used. Teravus, does the throttling code keep track of that at the moment?

[11:31] Teravus Ousley: It keeps track of only the general types used for throttling

[11:31] Teravus Ousley: .. and only in the amount necessary to make sure the throttles are honored

[11:32] Teravus Ousley lists the throttle types..

[11:32] Adam Frisby: Hrrm, I wouldnt mind seeing that charted, since I think a few of the types (eg land) come in big spikes.

[11:32] Teravus Ousley: Land packets.. wind packets, cloud packets, asset packets, texture packets, task packets, ..

[11:32] Teravus Ousley: Resend packets..

[11:33] Teravus Ousley: You could easily hook a routine to find out how much of which.

[11:34] Neas Bade: if no one else is diving in there, I will. It would be a good excuse to actually get more wrapped up in the throttle code

[11:34] Neas Bade: on a first pass, those statistics will at least show us if there are a few packet types that we could reuse that would save us a lot

[11:34] Teravus Ousley: I'm sure the throttle can also be optimized

[11:34] Teravus Ousley: :D

[11:35] Adam Johnson: We did notice you could cearly see the throttling working when examing the heapbuddy info

[11:35] Adam Johnson: clearly even

[11:36] Adam Frisby: Hrrm.

[11:36] Teravus Ousley: that code is in two parts.. ClientView.cs.. and in ClientView.PacketQueue

[11:36] Adam Frisby: Adam: were these tests done under a Debug or Release version out of curiosity?

[11:36] Teravus Ousley: .. the part in ClientView.PacketQueue is in the method, 'OutPacket'

[11:37] Adam Johnson: version of what?

[11:37] Teravus Ousley: the part in ClientView is in throttle_Timer_Elapsed

[11:37] Adam Frisby: Was it compiled under a debug or release version?

[11:38] Justin Clark-Casey: hi nebadon

[11:38] Adam Johnson: Ah, not sure.. Will ask Johan tomorrow for you though.

[11:38] Teravus Ousley: Hey nebadon. I think the asset server is hosed... if you have not already reset it.

[11:38] Adam Frisby: If it was under Debug, think you could ask him what the performance is like under a Release?

[11:38] Adam Frisby: thanks. :)

[11:38] Nebadon Izumi: i havent looked

[11:38] Nebadon Izumi: i havnet seen textures here in while now

[11:38] Adam Johnson: MW, you there?

[11:38] Nebadon Izumi: its been a few days since ive seen the textures load here

[11:41] Teravus Ousley wonders if we're disecting the clientview code now :D

[11:41] Neas Bade: I am, sorry got distracted :)

[11:42] Adam Johnson: No problem.. So do you have a kind of guestimate of how much work is involved with ditching libsl?

[11:43] Neas Bade: I think that's a confident no

[11:44] Adam Frisby: Heh yeah

[11:44] Nebadon Izumi: neas last week you said we should be using Sqlite over Grid Assets

[11:44] Nebadon Izumi: i did some testing on my grid

[11:44] Nebadon Izumi: and i got the exact same results sith sqlite

[11:44] Neas Bade: really?

[11:44] Neas Bade: interesting

[11:44] Neas Bade: I've not seen any drops with asset_database = sqlite

[11:45] Nebadon Izumi: hmm

[11:45] Adam Frisby: We totally need to make ODE primitive bounciness a opensim.ini parameter

[11:45] Nebadon Izumi: i keep freezing up

[11:46] More Wright: Adam Johnson, am here now?

[11:46] Nebadon Izumi: but as i was saying in grid mode for me i had better results with Grid assets than i did with sqlite

[11:46] Nebadon Izumi: sqlite performance seems uber slow in comparison

[11:46] Adam Johnson: Oh, speaking of Physics. chi11ken and I were discussing possibly replacing basic physics with POS, and making it configurable.

[11:46] Neas Bade: well, there is a different issue there

[11:46] Adam Johnson: I think danx0r had similar thoughts about this.

[11:47] Neas Bade: on my todo list to do some ado.net gutting to get sqlite performance up

[11:47] Nebadon Izumi: i was less likely to see textures with sqlite

[11:47] Neas Bade: I think most of the sqlite perf issues is actually the ado.net layer being slow

[11:47] Nebadon Izumi: im talking when 1000's of prims are in the picture

[11:49] Teravus Ousley: yep. I think we'll also have better luck with everything once we get our draw distance based prim update limitor... however it being like this now is good.. because it gives us a chance to really beat our interfaces to smitherenes with load

[11:49] Adam Frisby: Oh --- quick issue actually.

[11:49] Adam Frisby: I'm still getting "Remote Host Closed Connection" issues with remoting under .NET/Win32

[11:50] Adam Frisby: We need to put a reconnect in on the remoting handler --- DeepThink which is near here keeps dropping off the radar

[11:50] Adam Frisby: although opensim.exe is still there & active

[11:50] Nebadon Izumi: yes

[11:50] Nebadon Izumi: my windows server on my mostly linux grid

[11:50] Nebadon Izumi: either crashes all the time

[11:50] Nebadon Izumi: or is not visible when it is running

[11:51] Optik Philtr: my old sindows server use to do that too till i fixed it

[11:51] More Wright: better yet, lets delete all the grid code, then it will force someone to rewrite it and no mater what I think they would have to make a better job ;)

[11:51] Teravus Ousley: There's also likely an issue with events .. and notifying of children agents.

[11:51] Nebadon Izumi: what did you do to fix it?

[11:51] Nebadon Izumi: lol

[11:51] Optik Philtr: I changed over to a hosted tektonic linux box

[11:51] Nebadon Izumi: oh

[11:52] Adam Frisby: haha MW

[11:52] Nebadon Izumi: lol

[11:52] Adam Frisby: Good idea.

[11:52] Adam Johnson: lol

[11:52] Adam Frisby: I'm still massively in favour of rewriting the grid code.

[11:52] Optik Philtr: like was said earlier there are some documented anomolies with ado.net

[11:52] Adam Frisby: I'd like to revive that topic on the mailing lists.

[11:52] Nebadon Izumi: yea from my testing so far OpenSIM seems much more stable on Linux

[11:52] Optik Philtr: documented as in we can see them regularly

[11:52] Adam Frisby: Nebadon: It's probably because it gets more testing under Linux.

[11:52] Adam Frisby: Still only a few of us using it on Windows

[11:53] Nebadon Izumi: yea I have a 1 windows server

[11:53] Adam Johnson: I agree, and would like to put someone on that task, but the suits just don't understand..

[11:53] Neas Bade: well I kept throwing in my $0.02 on the grid stuff :)

[11:53] Optik Philtr: now i still have my stand alone grid services grid on windows...the huge one

[11:53] Optik Philtr: it has no big issues

[11:53] Nebadon Izumi: yea im speaking about Grid Mode

[11:53] Nebadon Izumi: with windows

[11:53] Nebadon Izumi: standalone it runs great

[11:54] More Wright: a topic that isn't for now, but maybe over the next few months is: now that vs 2008 is released and mono 2.2.6 says c# 3.0 is nearly complete, will swapping to c# 3.0 bring anything useful/ that we would like to use?

[11:54] Nebadon Izumi: 2.2.6

[11:54] Nebadon Izumi: ?

[11:54] Adam Frisby: WCF looks potentially nice.

[11:54] Adam Frisby: LinQ does too

[11:54] Neas Bade: I think we'd have to wait until at least June for that

[11:55] Nebadon Izumi: will there be a free version of 2008?

[11:55] Neas Bade: if it doesn't run on the latest ubuntu, the barrier to run on linux is higher than it should be

[11:55] Nebadon Izumi: thats the question

[11:55] Neas Bade: and we're still on 1.2.4 in latest ubuntu

[11:55] More Wright: 1.2.6

[11:55] Nebadon Izumi: you can downloading the 1.2.5.1 installer from the mono-project website

[11:56] Neas Bade: you can, but then you loose a lot of the other linkage to the system

[11:56] Nebadon Izumi: I did it on Sabayon

[11:56] Neas Bade: breaking package management on Linux is like forcing people into dos prompts on windows

[11:56] Mircea Kitsune: hi

[11:56] Nebadon Izumi: and it seemed to work well

[11:56] More Wright: and yeah there is a free VS 2008 express already, but yeah if 1.2.6 isn't usable yet for most people it will have to wait

[11:57] Adam Frisby: Speaking of VS 2008

[11:57] Nebadon Izumi: is 1.2.6 even available?

[11:57] Adam Frisby: Anyone here have access to VS 2008 Team Edition?

[11:57] Adam Frisby: I've only got Pro

[11:57] Mircea Kitsune: did the meeting just beginned or is ending?

[11:57] Mircea Kitsune: *begin

[11:57] Adam Frisby: Probably ending soonish

[11:57] Adam Frisby: anyway -- anyone have VS 2008 Team Ed?

[11:57] More Wright: nope

[11:58] Adam Frisby: There's a new "Code Review" function in it which helps isolate bad sections of code

[11:58] Kurt Stringer: nope - Im on express edition

[11:58] Adam Johnson: no

[11:58] Adam Frisby: I'd love for someone to run it across the codebase and then post a copy of the report to the mailing list

[11:58] Nebadon Izumi: I dont even see Mono 1.2.6 available for download on the mono-project website

[11:58] More Wright: there is a 90 day trail of the team edition though

[11:58] Neas Bade: I wonder if it is as good as the gramar checker ;)

[11:58] Neas Bade: mono 1.2.6 isn't released yet

[11:58] Adam Frisby: Neas: well it isolates things like needlessly complex functions & classes

[11:59] Adam Frisby: which, if we at least look at, might be able to nail down some potentially buggy areas of the code.

[11:59] More Wright: Neas Bade, last time I looked mono 1.2.6 was set for a begining of december release

[11:59] Optik Philtr thinks it should be run on the client

[12:00] Kurt Stringer: interesting conversation today - I have to leave now = bye everyone

[12:00] Neas Bade: yeh, got to start diving into packet code

[12:00] Teravus Ousley: bye

[12:00] Mircea Kitsune: too bad i missed it :( anyway, like i said on irc, ill do some teleporting tests soon

[12:00] Justin Clark-Casey: bye kurt

[12:00] Teravus Ousley: It's not that scary really

[12:00] Mircea Kitsune: see ya

[12:00] More Wright: bye

[12:00] Teravus Ousley: .. Just have to understand what it's doing :D

[12:00] Adam Frisby: Yeah I should be heading to bed

[12:01] Adam Johnson: Me too

[12:01] Adam Johnson: Ugh

[12:01] Adam Frisby: need to be up by lunch today. heh, 5AM already

[12:01] Adam Johnson: ditto

[12:01] Teravus Ousley: .. the BlockingQueue is keeping the clientloop thread from sucking up resources.. and managing the traffing inbound and outbound

[12:01] Adam Frisby: Alright, I'll see you guys around later. :)

[12:01] Teravus Ousley: take care Adam :D

[12:01] Teravus Ousley: sleep well

[12:02] Adam Johnson: I'll see you guys tomorrow in irc. :)

[12:02] Teravus Ousley: when there's packets in PacketQueue(the blocking queue).. the clientloop runs

[12:02] Adam Johnson: MW, mind helping us a bit.. we may be starting the process of geting rid of libsl

[12:03] Mircea Kitsune: well im leaving back to my sim now (tries a teleport) see you around :)

[12:04] More Wright: Adam, yeah I'll help all I can, I'm a bit pressed for free time the last few months and is likely to last until january at least but I'll help where I can

[12:04] Adam Johnson: Okay thanks. :)

[12:05] Adam Johnson: Anways, talk to you guys later.

[12:05] Justin Clark-Casey: chhers

[12:08] More Wright: yeah I need to go too, see you in irc

[12:08] Teravus Ousley: k, take care

[12:08] Teravus Ousley poofs also