Chat log from the meeting on 2007-11-27

[11:01] Teravus Ousley: new people seem to be less gray to me [11:01] Neas Bade: yes, agreed [11:01] Kurt Stringer: Heoo all [11:02] Kurt Stringer: lol [11:02] Teravus Ousley: Hai Kurt [11:02] Justin Clark-Casey: hey kurt [11:02] Kurt Stringer: hello all [11:02] Neas Bade: hey [11:02] Adam Frisby: lo [11:04] Teravus Ousley: 0x0 textures.. asset server is definately hosed [11:04] Teravus Ousley: . [11:04] Adam Frisby: Woot [11:04] Adam Frisby: Just like the main grid. :D [11:05] Neas Bade: yeh, we need to get this off of the grid asset server, which is just very broken right now [11:06] Adam Frisby: Or add in a bit of intelligent caching [11:06] Adam Frisby: a sim should have a copy of every texture in prims in it [11:06] Adam Frisby: err on prims [11:06] Adam Frisby: ditto for common assets, eg the moon / sun, etc. [11:09] More Wright: is no one talking, or can I just not hear anything? [11:09] Teravus Ousley: no one is talking [11:09] Teravus Ousley: we're all changing our appearance :P [11:09] Adam Johnson: We are telling secrets about you ;) [11:09] Teravus Ousley: . [11:09] Adam Frisby: Heh [11:09] Neas Bade: :) [11:09] More Wright: ahh cool [11:10] Adam Frisby: All secrets about me, like how I'm going to completely stack myself on the slopes in Japan this weekend, courtesy of Adam. [11:10] Adam Frisby: hehe [11:10] More Wright: you two going sking? [11:10] Adam Frisby: Yeah [11:10] More Wright: and the rest of us wasn't invited? [11:10] Adam Johnson: yeah, gonna hopefully not break my neck. ;) [11:11] Adam Johnson: You are all welcome anytime. :P [11:11] Adam Frisby: Well, if your in Japan. hehe. [11:11] Adam Johnson: Indeed. haha [11:12] Adam Frisby: Anyway [11:12] Adam Frisby: Agenda? [11:12] Neas Bade: I had some proposals on the list [11:12] Neas Bade: given that Adam J is here, I'd suggest we start with the packet overhead findings the 3Di team found [11:13] Neas Bade: I think that would be very interesting [11:13] Adam Johnson: Heh, sure. [11:13] Teravus Ousley: very reasonable [11:13] Adam Johnson: Though it is opening a fun topic involving libsl. [11:13] Neas Bade: opening fun topics is always allowed :) [11:13] Adam Johnson: Well, do we push on with .3 of libsl, or completely do away with it? [11:14] Adam Frisby: I had a chat with Adam earlier this evening about that latter option -- having classes of each packets may be a very inefficient way of handling things. [11:14] Adam Frisby: It might be better to have a: byte[] buildXYZPacket(A, B, C, D); function for every packet. [11:15] Teravus Ousley: shh, nobody type.. .. my boss is looking over my shoulder.. j/k [11:16] Adam Frisby: Hehe [11:16] Adam Johnson: So, I guess the big question on 3DI's end would be, how much work will each option involve. [11:16] Neas Bade: right [11:17] Neas Bade: I think the issue that was hit with a previous attempt at 0.3 update was a bunch of fields got marked private that busted opensim [11:17] Adam Frisby: 90% of libsls codebase is generated procedurally by another application, we should be able to modify that to produce the buildPacket equivilents pretty easily. [11:18] Adam Frisby: But, there is extra stuff which we use like their layerdata encoder/decoders. [11:18] Teravus Ousley: They also have 'all' of their packets in one CS file.. _packets.cs [11:18] More Wright: that is what is built precedurally [11:18] Adam Johnson: and the openjpeg bits as well. [11:19] Adam Frisby: True, although I made the suggestion to p0pe about moving their OpenJpeg to a seperate project called OpenJpeg.net or something [11:19] More Wright: they as far as I know have a application that takes the message_template file and produces the _packet.cs file from it [11:19] Adam Frisby: since there's more than just libsl using that now [11:19] Adam Johnson: True [11:20] Adam Frisby: Hrrm [11:20] Teravus Ousley: odd, does the message template have enough information to do that? [11:20] Adam Frisby: These performance issues -- they should be hitting libsl too right? [11:20] Adam Frisby: Tervaus: yes. [11:20] Adam Johnson: They are actually.. [11:20] Adam Frisby: libsl might appreciate switching to buildPacket equivilents internally if we can demonstrate a big performance increase. [11:20] Teravus Ousley wonders what message template they are looking at :P [11:20] Adam Johnson: I notice these spikes on login/teleports in MovableLife [11:21] More Wright: Tervaus, the one included with the SL client [11:21] Adam Frisby: msg_template.msg [11:21] Adam Frisby: Open it in notepad [11:21] Teravus Ousley: hmm.. I did.. but it didn't seem to have enough info specifically on packet payloads. [11:21] Teravus Ousley shrugs.. [11:21] Teravus Ousley: maybe they figured out something.. [11:22] Adam Frisby: We dont use anything more complex than msg_template in opensim itself [11:22] Teravus Ousley: anyway.. different topic.. back to performance [11:22] Adam Frisby: we have to figure out the payloads too [11:23] Adam Johnson: Indeed. [11:23] Teravus Ousley: so the main bog is not dealing with them as byte arrays the majority of the time? [11:25] Adam Johnson: The main bog is coming from GC, because we keep creating new packets and never reuse them. [11:25] Adam Frisby: Hrrm. [11:25] Adam Frisby: Create a circular list of each packet type for re-use. Mark as "in use" / "not in use"? [11:25] Adam Johnson: On login the number of packets created is pretty huge.. and if you check out heapbuddy, you can see that the GC is pretty intense at that point. [11:25] Adam Frisby: after it's been sent, reset the use flag [11:25] Teravus Ousley: Adam beat me to it :P [11:26] Adam Frisby: Question: did you guys find any other performance issues while you were profiling? [11:27] Adam Johnson: So far, that was the only one that we have found.. but I think we need to do some more analyzing. [11:27] Adam Johnson: and, it would be good to have a custom profiler created... [11:27] Adam Frisby: I've got one I've got semi-completed, the catch is it's windows-only since it relies on WMI to get profiling information. [11:27] Adam Johnson: That would still be of use to us I think. [11:28] Adam Frisby: Alright, I'll send you guys a copy -- it's designed for grid information mainly. [11:28] Adam Frisby: health / status / etc. [11:28] Adam Johnson: Okay great, we have been mainly dealing with sandbox, but will be doing some grid tests shortly. [11:30] Adam Frisby: Alright, well - seems like we should look at re-using rather than disposing of packets [11:30] Adam Frisby: MW: This is probably something you have the most experience with, how long do you think it will take? [11:30] Adam Johnson: Yep, I think it will fix a lot of the login cap problems [11:31] Neas Bade: it would be good to get some statistic counts on packet types being used. Teravus, does the throttling code keep track of that at the moment? [11:31] Teravus Ousley: It keeps track of only the general types used for throttling [11:31] Teravus Ousley: .. and only in the amount necessary to make sure the throttles are honored [11:32] Teravus Ousley lists the throttle types.. [11:32] Adam Frisby: Hrrm, I wouldnt mind seeing that charted, since I think a few of the types (eg land) come in big spikes. [11:32] Teravus Ousley: Land packets.. wind packets, cloud packets, asset packets, texture packets, task packets, .. [11:32] Teravus Ousley: Resend packets.. [11:33] Teravus Ousley: You could easily hook a routine to find out how much of which. [11:34] Neas Bade: if no one else is diving in there, I will. It would be a good excuse to actually get more wrapped up in the throttle code [11:34] Neas Bade: on a first pass, those statistics will at least show us if there are a few packet types that we could reuse that would save us a lot [11:34] Teravus Ousley: I'm sure the throttle can also be optimized [11:34] Teravus Ousley: :D [11:35] Adam Johnson: We did notice you could cearly see the throttling working when examing the heapbuddy info [11:35] Adam Johnson: clearly even [11:36] Adam Frisby: Hrrm. [11:36] Teravus Ousley: that code is in two parts.. ClientView.cs.. and in ClientView.PacketQueue [11:36] Adam Frisby: Adam: were these tests done under a Debug or Release version out of curiosity? [11:36] Teravus Ousley: .. the part in ClientView.PacketQueue is in the method, 'OutPacket' [11:37] Adam Johnson: version of what? [11:37] Teravus Ousley: the part in ClientView is in throttle_Timer_Elapsed [11:37] Adam Frisby: Was it compiled under a debug or release version? [11:38] Justin Clark-Casey: hi nebadon [11:38] Adam Johnson: Ah, not sure.. Will ask Johan tomorrow for you though. [11:38] Teravus Ousley: Hey nebadon. I think the asset server is hosed... if you have not already reset it. [11:38] Adam Frisby: If it was under Debug, think you could ask him what the performance is like under a Release? [11:38] Adam Frisby: thanks. :) [11:38] Nebadon Izumi: i havent looked [11:38] Nebadon Izumi: i havnet seen textures here in while now [11:38] Adam Johnson: MW, you there? [11:38] Nebadon Izumi: its been a few days since ive seen the textures load here [11:41] Teravus Ousley wonders if we're disecting the clientview code now :D [11:41] Neas Bade: I am, sorry got distracted :) [11:42] Adam Johnson: No problem.. So do you have a kind of guestimate of how much work is involved with ditching libsl? [11:43] Neas Bade: I think that's a confident no [11:44] Adam Frisby: Heh yeah [11:44] Nebadon Izumi: neas last week you said we should be using Sqlite over Grid Assets [11:44] Nebadon Izumi: i did some testing on my grid [11:44] Nebadon Izumi: and i got the exact same results sith sqlite [11:44] Neas Bade: really? [11:44] Neas Bade: interesting [11:44] Neas Bade: I've not seen any drops with asset_database = sqlite [11:45] Nebadon Izumi: hmm [11:45] Adam Frisby: We totally need to make ODE primitive bounciness a opensim.ini parameter [11:45] Nebadon Izumi: i keep freezing up [11:46] More Wright: Adam Johnson, am here now? [11:46] Nebadon Izumi: but as i was saying in grid mode for me i had better results with Grid assets than i did with sqlite [11:46] Nebadon Izumi: sqlite performance seems uber slow in comparison [11:46] Adam Johnson: Oh, speaking of Physics. chi11ken and I were discussing possibly replacing basic physics with POS, and making it configurable. [11:46] Neas Bade: well, there is a different issue there [11:46] Adam Johnson: I think danx0r had similar thoughts about this. [11:47] Neas Bade: on my todo list to do some ado.net gutting to get sqlite performance up [11:47] Nebadon Izumi: i was less likely to see textures with sqlite [11:47] Neas Bade: I think most of the sqlite perf issues is actually the ado.net layer being slow [11:47] Nebadon Izumi: im talking when 1000's of prims are in the picture [11:49] Teravus Ousley: yep. I think we'll also have better luck with everything once we get our draw distance based prim update limitor... however it being like this now is good.. because it gives us a chance to really beat our interfaces to smitherenes with load [11:49] Adam Frisby: Oh --- quick issue actually. [11:49] Adam Frisby: I'm still getting "Remote Host Closed Connection" issues with remoting under .NET/Win32 [11:50] Adam Frisby: We need to put a reconnect in on the remoting handler --- DeepThink which is near here keeps dropping off the radar [11:50] Adam Frisby: although opensim.exe is still there & active [11:50] Nebadon Izumi: yes [11:50] Nebadon Izumi: my windows server on my mostly linux grid [11:50] Nebadon Izumi: either crashes all the time [11:50] Nebadon Izumi: or is not visible when it is running [11:51] Optik Philtr: my old sindows server use to do that too till i fixed it [11:51] More Wright: better yet, lets delete all the grid code, then it will force someone to rewrite it and no mater what I think they would have to make a better job ;) [11:51] Teravus Ousley: There's also likely an issue with events .. and notifying of children agents. [11:51] Nebadon Izumi: what did you do to fix it? [11:51] Nebadon Izumi: lol [11:51] Optik Philtr: I changed over to a hosted tektonic linux box [11:51] Nebadon Izumi: oh [11:52] Adam Frisby: haha MW [11:52] Nebadon Izumi: lol [11:52] Adam Frisby: Good idea. [11:52] Adam Johnson: lol [11:52] Adam Frisby: I'm still massively in favour of rewriting the grid code. [11:52] Optik Philtr: like was said earlier there are some documented anomolies with ado.net [11:52] Adam Frisby: I'd like to revive that topic on the mailing lists. [11:52] Nebadon Izumi: yea from my testing so far OpenSIM seems much more stable on Linux [11:52] Optik Philtr: documented as in we can see them regularly [11:52] Adam Frisby: Nebadon: It's probably because it gets more testing under Linux. [11:52] Adam Frisby: Still only a few of us using it on Windows [11:53] Nebadon Izumi: yea I have a 1 windows server [11:53] Adam Johnson: I agree, and would like to put someone on that task, but the suits just don't understand.. [11:53] Neas Bade: well I kept throwing in my $0.02 on the grid stuff :) [11:53] Optik Philtr: now i still have my stand alone grid services grid on windows...the huge one [11:53] Optik Philtr: it has no big issues [11:53] Nebadon Izumi: yea im speaking about Grid Mode [11:53] Nebadon Izumi: with windows [11:53] Nebadon Izumi: standalone it runs great [11:54] More Wright: a topic that isn't for now, but maybe over the next few months is: now that vs 2008 is released and mono 2.2.6 says c# 3.0 is nearly complete, will swapping to c# 3.0 bring anything useful/ that we would like to use? [11:54] Nebadon Izumi: 2.2.6 [11:54] Nebadon Izumi: ? [11:54] Adam Frisby: WCF looks potentially nice. [11:54] Adam Frisby: LinQ does too [11:54] Neas Bade: I think we'd have to wait until at least June for that [11:55] Nebadon Izumi: will there be a free version of 2008? [11:55] Neas Bade: if it doesn't run on the latest ubuntu, the barrier to run on linux is higher than it should be [11:55] Nebadon Izumi: thats the question [11:55] Neas Bade: and we're still on 1.2.4 in latest ubuntu [11:55] More Wright: 1.2.6 [11:55] Nebadon Izumi: you can downloading the 1.2.5.1 installer from the mono-project website [11:56] Neas Bade: you can, but then you loose a lot of the other linkage to the system [11:56] Nebadon Izumi: I did it on Sabayon [11:56] Neas Bade: breaking package management on Linux is like forcing people into dos prompts on windows [11:56] Mircea Kitsune: hi [11:56] Nebadon Izumi: and it seemed to work well [11:56] More Wright: and yeah there is a free VS 2008 express already, but yeah if 1.2.6 isn't usable yet for most people it will have to wait [11:57] Adam Frisby: Speaking of VS 2008 [11:57] Nebadon Izumi: is 1.2.6 even available? [11:57] Adam Frisby: Anyone here have access to VS 2008 Team Edition? [11:57] Adam Frisby: I've only got Pro [11:57] Mircea Kitsune: did the meeting just beginned or is ending? [11:57] Mircea Kitsune: *begin [11:57] Adam Frisby: Probably ending soonish [11:57] Adam Frisby: anyway -- anyone have VS 2008 Team Ed? [11:57] More Wright: nope [11:58] Adam Frisby: There's a new "Code Review" function in it which helps isolate bad sections of code [11:58] Kurt Stringer: nope - Im on express edition [11:58] Adam Johnson: no [11:58] Adam Frisby: I'd love for someone to run it across the codebase and then post a copy of the report to the mailing list [11:58] Nebadon Izumi: I dont even see Mono 1.2.6 available for download on the mono-project website [11:58] More Wright: there is a 90 day trail of the team edition though [11:58] Neas Bade: I wonder if it is as good as the gramar checker ;) [11:58] Neas Bade: mono 1.2.6 isn't released yet [11:58] Adam Frisby: Neas: well it isolates things like needlessly complex functions & classes [11:59] Adam Frisby: which, if we at least look at, might be able to nail down some potentially buggy areas of the code. [11:59] More Wright: Neas Bade, last time I looked mono 1.2.6 was set for a begining of december release [11:59] Optik Philtr thinks it should be run on the client [12:00] Kurt Stringer: interesting conversation today - I have to leave now = bye everyone [12:00] Neas Bade: yeh, got to start diving into packet code [12:00] Teravus Ousley: bye [12:00] Mircea Kitsune: too bad i missed it :( anyway, like i said on irc, ill do some teleporting tests soon [12:00] Justin Clark-Casey: bye kurt [12:00] Teravus Ousley: It's not that scary really [12:00] Mircea Kitsune: see ya [12:00] More Wright: bye [12:00] Teravus Ousley: .. Just have to understand what it's doing :D [12:00] Adam Frisby: Yeah I should be heading to bed [12:01] Adam Johnson: Me too [12:01] Adam Johnson: Ugh [12:01] Adam Frisby: need to be up by lunch today. heh, 5AM already [12:01] Adam Johnson: ditto [12:01] Teravus Ousley: .. the BlockingQueue is keeping the clientloop thread from sucking up resources.. and managing the traffing inbound and outbound [12:01] Adam Frisby: Alright, I'll see you guys around later. :) [12:01] Teravus Ousley: take care Adam :D [12:01] Teravus Ousley: sleep well [12:02] Adam Johnson: I'll see you guys tomorrow in irc. :) [12:02] Teravus Ousley: when there's packets in PacketQueue(the blocking queue).. the clientloop runs [12:02] Adam Johnson: MW, mind helping us a bit.. we may be starting the process of geting rid of libsl [12:03] Mircea Kitsune: well im leaving back to my sim now (tries a teleport) see you around :) [12:04] More Wright: Adam, yeah I'll help all I can, I'm a bit pressed for free time the last few months and is likely to last until january at least but I'll help where I can [12:04] Adam Johnson: Okay thanks. :) [12:05] Adam Johnson: Anways, talk to you guys later. [12:05] Justin Clark-Casey: chhers [12:08] More Wright: yeah I need to go too, see you in irc [12:08] Teravus Ousley: k, take care [12:08] Teravus Ousley poofs also