[Opensim-users] MySQL on seperate server?
Karen Palen
karen_palen at yahoo.com
Thu Feb 18 21:38:49 UTC 2010
Hi,
I was deliberately being as neutral as I could in my tone since I do NOT want to start some "Fanboy" flamewar!
I do notice that a defrag operation is standard in all Windows versions and certainly does make a huge difference in file access.
I also take note that "defrag" is NOT a Linux utility and I have not experienced as problem due to its lack. My only real experience for potential fragmentation was in copying/moving a Ubuntu 9.04 install to another machine - about the crudest form of "defrag" there is!
I was careful to use a "file by file" copy and not "sector by sector" for exactly this reason. The old file system had been in heavy use for nearly a year, but I measured zero difference in hard drive performance!
From that I conclude that Linux does not experience a fragmentation problem.
You are welcome to debate the reasons for this, I see that as unproductive.
Karen
--- On Thu, 2/18/10, Tedd Hansen <tedd at konge.net> wrote:
> From: Tedd Hansen <tedd at konge.net>
> Subject: Re: [Opensim-users] MySQL on seperate server?
> To: opensim-users at lists.berlios.de
> Date: Thursday, February 18, 2010, 8:31 AM
> Hi
>
> NTFS filesystem does have all the features of ext3/ext4 +
> much more (except maybe live resize of fs), and have had it
> for at least 10 years. NTFS has been way ahead of its time
> for well over a decade. The journal is the part that keeps
> your data safe (introduced in ext3) in event of a crash.
> BOTH filesystems have pre-allocation of space when you open
> a file for write, which combats fragmentation as long as you
> have a fair amount of free space on your disk. But Windows
> 2008 also has prioritized IO and background defragmenting
> which allows it to organize things in the background without
> affecting the services you are running. Anyway you should
> have enough memory to not care too much about fragmentation,
> it doesn't cost much and gives a good boost.
>
> When that is said, the kernel does not matter that much
> anymore. That is a 90's discussion. Threading, memory
> handling, disk IO, all are competitive on all platforms
> today. Some advances are made now and then, but all the
> others are quick to follow.
>
> I highly recommend the book "Modern Operating Systems" for
> anyone who wants to dig in deep in OS' in general.
>
> Considerations to be made (from the top of my head):
> 1) Of course; how much memory, memory
> speed/bandwidth(bridge) and how many MHz's do you pack...
> 2) How much L2-cache do you have (quite important)?
> 3) How many physical CPU's do you have (cores share L2
> cache)?
> 4) How many memory channels do you have (simultaneous
> memory access)?
> 5) How are these channels mapped to the CPU's?
> 6) Do the software you run know how to utilize the
> CPU2PhysicalMem(data) mapping correctly (MS-SQL does, MySQL
> does not afaik)?
> 7) Are you forcing many CPU-intensive components to share
> memory bandwith (regions, several db components, etc)?
> Remember that CPU-intensive tasks usually involve modifying
> some amount of memory.
> 8) What other tasks are you doing on this computer which
> affects #7.
>
> Because of the fact that OpenSim is a CPU intensive package
> which uses a lot of code and memory you can run out of
> L2-cache on larger setups, hence you get a trashing behavior
> on treads.
> So splitting OpenSim over multiple machines (regions,
> databases, etc) will yield better performance. Putting too
> much on the same machine will after a certain point cause an
> exponential decrease in speed (trashing behavior).
>
> The exact point at where your installation will start
> trashing is highly dependent on hardware and number of
> objects ... but some general experience in this area would
> be very interesting.
> OpenSim/3D worlds are hw-intensive applications, and
> squeezing the most out of them will be an issue for anyone
> who plan to have more than a handful of users. (What is the
> TCO of 1k visitors, 10k, 100k, etc.)
>
> For kernel hackers...
> If you have a dedicated OpenSim-server you could try to
> change the default Hz of the kernel. Currently it's at 250,
> optimized for multimedia such as video display.
> Linus in e-mail explained it as "1kHz is also good in that
> it makes it easy to convert both to USER_HZ and to ms/ns.
> But maybe something like 250Hz would be better - still high
> enough that things like multimedia (which really wants
> higher frequencies in order to be able to sleep for
> fractional video-frames) should be happy, low enough that we
> use less CPU."
> You can set this to 100 to avoid an additional 150 task
> switches and MMU-reprograms per second. This would allow
> each thread to work longer on the data it has retrieved from
> memory before a switch occurs.
> This can be done for Windows as well with a simple registry
> change and reboot.
> Linux: http://kerneltrap.org/node/5411
> Windows: http://doc.sch130.nsc.ru/www.sysinternals.com/ntw2k/freeware/frob.shtml
>
> Definition of trashing = http://en.wikipedia.org/wiki/Thrash_(computer_science)
>
> BR,
> Tedd
>
> -----Original Message-----
> From: opensim-users-bounces at lists.berlios.de
> [mailto:opensim-users-bounces at lists.berlios.de]
> On Behalf Of Karen Palen
> Sent: 18. februar 2010 15:21
> To: opensim-users at lists.berlios.de
> Subject: Re: [Opensim-users] MySQL on seperate server?
>
> Zonja did not specify, but I am guessing that he is using
> Windows since he needed to de-fragment his HD. The Linux
> ext3/4 file system takes care of that automatically in the
> journalling.
>
> Inevitably there will be many differences between a Windows
> based system and a Linux based one, if only because the two
> OSes are so different in their kernals.
>
> If you haven't already tried it, you should ge the best
> efficiency with one of the Linux server distros.
>
> I have played with Ubuntu server and it is much "leaner"
> than the desktop version. The Kernel is optimised for
> database type applications rather than giving a responsive
> GUI.
>
> I would assume that the other Linux server versions are
> similar, but I really don't know enough about them to
> compare. Any of them should come with all of the basic tools
> (LAMP, MySQL support etc) so it should boil down to which
> one fits your needs the best.
>
> There are hundreds of "standard" tests for performance, but
> as far as I know there is nothing yet that is oriented to
> OpenSim. One of the MySQL benchmarks should give you some
> good insight though.
>
> Karen
>
> --- On Thu, 2/18/10, Impalah Shenzhou <impalah at gmail.com>
> wrote:
>
> > From: Impalah Shenzhou <impalah at gmail.com>
> > Subject: Re: [Opensim-users] MySQL on seperate
> server?
> > To: opensim-users at lists.berlios.de
> > Date: Thursday, February 18, 2010, 5:02 AM
> > Then the problem are in my computers
> > :-P or in the tests we both did.
> >
> > I got better performance separating db from
> > opensim servers, but I don't know if prim
> > size/modifications/colors/texturing/scripting have
> some
> > influence in the results we both got.
> >
> > Any standard test for performance?
> > 2010/2/18 Zonja Capalini
> > <zonja.capalini at gmail.com>
> >
> > I'm
> > using a Quad Core Q6600 @ 2.40 GHz with 3 GB of RAM
> and two
> > 372 GB HDsfor Condensation Land. Condensation runs
> two
> > Opensim instances (8 and 3 sims respectively)
> > in a serverless grid configuration AND the MySQL
> > database. I have the OS in one HD and MySQL
> > and Opensim in the other HD, and my startup times are
> > < 2 mins even if I havesims with > 10000
> > prims. When I log in with 512 m draw distance, I
> experience
> > some lagfor 15 seconds or so (on a 10 Mb ADSL
> > line) while everything is downloaded,
> >
> > then no lag. Tried it with four avatars, no
> > lag.
> > Defragmenting the HDs every so often does *a
> > lot* to improve performance.
> > ymmv
> >
> > /Zonja
> >
> > On Thu, Feb 18, 2010 at 11:26 AM,
> > Impalah Shenzhou <impalah at gmail.com>
> > wrote:
> >
> >
> > I arrive late, as usual :-)
> >
> > When a region has around 12000 primitives, with
> > a db server on the same machine you will have hell on
> earth
> > or something like that.
> > Boot up time is around 10 minutes and is
> > impossible to do anything but using zoom. Impossible
> to
> > move.
> >
> >
> >
> > I have tested this with 3 machines (4Gb ram,
> > 500Gb hd RAID, Intel Core Duo 1,73, Intel Core duo 2,3
> and
> > amd64) running windows XP, Vista and Windows 7-64
> > bits.
> > I was using grid mode (grid server on separate
> > computer... no ROBUST or UGAIM, it uses a PHP ugaim
> > system).
> >
> >
> >
> > Separating Mysql to another machine on the same
> > local network (100 Mbps nc) I could move, build, and
> even
> > another avatar could enter.
> > Exporting db and running it on separate server
> > with Linux (ubuntu server 9.04) I could fill the
> region with
> > 5000 more prims and 2 avatars could work smooth.
> >
> >
> >
> > Running opensim on Linux (amd64, 4gb ram, ubuntu
> > server 9.04 64 bits) and db on separate linux server
> 2
> > avatars could work smoothly and filling the region
> with
> > 25000 prims (no simple prims, but different sizes,
> > texturized and scripted).
> >
> >
> >
> > Anyway, that were my "home"
> > experiences... My conclusion: separate db, don't use
> > windows.
> > Greetings
> >
> >
> >
> >
> >
> > 2010/2/16 Steve <steveamigauk at yahoo.co.uk>
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Thank you, although this doesn't tie in to my
> bandwidth
> > usage reports :/
> >
> >
> >
> > I have 60Gig peak, 240 Gig off peak allowance with my
> > provider
> > download. No limit upload. I have had the sim up and
> > running for a week
> > 24/7 (one region only - and 24/7 apart from a
> > OpenSimulator version
> > upgrade and the odd restart), using my net connection
> as
> > normal on my
> > main PC.
> >
> >
> >
> > My usage for 1 - 15 Feb is 11.78 Gig D/L (which
> includes 3
> > Gig HD vid
> > from BBC iPlayer) and 2 Gig upload.
> >
> >
> >
> > I was interested in the OP question because I have a
> SQL
> > server sat on
> > my web hosts servers doing nada, but it looks like I
> will
> > gain neither
> > performance nor speed by trying to use it, nor will I
> save
> > any
> > bandwidth.
> >
> >
> >
> > This is a real learning experience.
> >
> >
> >
> > Steve
> >
> >
> >
> > On 16/02/2010 8:15 PM, John Mieske wrote:
> > Oh I might add, that is 20 -
> > 40KBytes Upload one way and
> > 20 - 80 KB download one way. I thought I would
> meantion
> > that.
> > John
> >
> >
> >
> > On Tue, Feb 16, 2010 at 3:14 PM,
> > John Mieske
> > <johnmieske at gmail.com>
> > wrote:
> >
> > here
> > is an example of what happened to me on a few Grids I
> have
> > tested. Per
> > AV I am uploading / downloading roughly 40KBytes a
> second.
> > Not kBits.
> > That is full load with everything going on. So I am
> giving
> > the most I
> > have seen in this case. That is roughly 320 KBit
> upload and
> > download
> > just for one avatar. So in theory, you should be able
> to
> > push less then
> > 40K IF your avatars are not really doing all of what
> they
> > want them to.
> > Example : Voice.
> >
> >
> >
> > So with a 1 Meg upload, your
> looking at around 3
> > Avatars
> > smoothly. If your going to go with 2 Meg Upload which
> some
> > of us have,
> > then your could go to about 7 Avatars smoothly. Again,
> this
> > is all
> > dependant on what your USERS are doing. If all are
> building
> > and in
> > Voice all at once, then 1 Meg = 3 Avatars smoothly.
> >
> >
> >
> > Keep in mind, this is a test I
> did myself on 5
> > Grids I have
> > tested. Each with different ISPs. So if your going to
> run a
> > Grid you
> > also have to account that the region that ties into
> your
> > main region
> > ALSO has to accomodate for the extra communication
> between
> > each region
> > that is not on the same computer / ISP. AND they are
> > constantly 24 / 7
> > communicating. So if you have a bandwidth limit, you
> could
> > be in
> > trouble. At a Previous version of Sim world, we had
> 500
> > Megabytes
> > transfer for about a weeks worth of time. And that is
> with
> > only 8
> > regions and VERY little activity. The limit ? 1
> > Terribyte. So in one
> > week, we ate up half our bandwidth limit. ( Thats all
> the
> > HOST allowed
> > at the time )
> >
> >
> >
> > Bottom line, you MUST do the
> math to see if this
> > is even
> > possible to keep running before spending a lot of
> > money.
> >
> >
> >
> > John
> >
> >
> >
> >
> >
> > On Tue, Feb 16, 2010 at 2:41
> > PM, Steve <steveamigauk at yahoo.co.uk>
> > wrote:
> >
> >
> >
> > So out of interest, what are the upload limits per
> avatar
> > for example.
> > I have just shy of 1 Mb upload speed, and have
> happily
> > accommodated 8
> > Avatars (after one in particular who was throwing up
> masses
> > of errors
> > left). How far could I, in theory, push it?
> >
> >
> >
> > Steve
> >
> >
> >
> >
> >
> > On 15/02/2010 11:00 PM, Ken G. wrote:
> > Alright, thanks Daz
> for
> > answering my
> > question and thanks
> > to John and Master_Mirage for the input. I am just a
> hobby
> > user of
> > opensim, connected to OSGrid, hoping that maybe there
> was a
> > way to
> > improve my mediocre connection :)
> >
> >
> >
> > Ken aka Key
> >
> >
> >
> >
> >
> > On Mon, Feb 15,
> 2010 at
> > 3:00 PM, John
> > Mieske <johnmieske at gmail.com>
> > wrote:
> >
> > I
> > have
> > to agree with Darren here. If your doing this at home,
> you
> > might
> > as well have both on the same computer. Your internet
> in
> > general will
> > dictate what you can do. So if your running this at
> home,
> > hang up the
> > idea of two computer region serving because no one
> will see
> > a
> > difference when they already bogg down from a slow
> internet
> > connection
> > to your region. I have seen some Regions who run them
> at
> > home with the
> > best of intentions only shut down days later because
> they
> > realize their
> > internet only allowed 2 avatars to run smoothly. When
> they
> > got to 3,
> > their sim started to crash. It is all about your
> upload
> > speed. However,
> > if you got a FAT PIPE ( not to smoke hahaha ) then yea
> you
> > definitely
> > might want to try the way I suggested. Again
> everything in
> > OpenSim is
> > based on HOW you run your Grid / Regions. Nothing is
> 100%
> > set one way.
> > I only offered a suggestion to those who could use
> this
> > benefit. And
> > there are already some servers that has been setup
> this way
> > and they
> > are smooth.
> >
> >
> >
> > John
> >
> >
> >
> >
> >
> >
> > On Mon,
> Feb 15, 2010 at 3:39 PM, Darren
> > Williams <darren at dmmhosting.co.uk>
> > wrote:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> Ken G asked:
> >
> >
> >
> Dumb question from a
> > casual opensim
> > user who
> > is not fully
> > clued in on it's workings, that popped into my head
> > after reading John
> > Mieske's
> > Space Grid Station post:
> >
> >
> >
> > Is it feasible and sensible while running opensim on a
> home
> > PC
> > connection to be
> > able to use a MySQL database on a shared web hosting
> > server, such as
> > Hostgator
> > for example? If possible, would this type of setup
> improve
> > performance
> > of a
> > lightly used standalone or grid-connected sim?
> >
> > I haven't put much thought into this idea, just
> wanted
> > to blurt it out
> > and see
> > if it's worth pursuing.
> >
> >
> >
> Ken,
> >
> >
> You
> > shouldn’t
> > receive any benefit at all doing the above.
> > In fact it will probably make things worse.
> >
> > If
> I’m
> > reading this
> > right then the region server which you
> > have running locally would then be pulling/pushing the
> data
> > from across
> > the net
> > on the mysql server, back to your machine. It would
> > probably run worse.
> >
> >
> Daz
> >
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> >
> > Opensim-users mailing list
> >
> > Opensim-users at lists.berlios.de
> >
> > https://lists.berlios.de/mailman/listinfo/opensim-users
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > --
> >
> > John Mieske / Winword Exonar
> >
> > http://johnmieske.org
> >
> > Space Grid Station
> >
> >
> >
> >
> >
> > _______________________________________________
> >
> > Opensim-users mailing list
> >
> > Opensim-users at lists.berlios.de
> >
> > https://lists.berlios.de/mailman/listinfo/opensim-users
> >
> >
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> > Opensim-users mailing list
> > Opensim-users at lists.berlios.de
> > https://lists.berlios.de/mailman/listinfo/opensim-users
> >
> >
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> >
> > Opensim-users mailing list
> >
> > Opensim-users at lists.berlios.de
> >
> > https://lists.berlios.de/mailman/listinfo/opensim-users
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > --
> >
> > John Mieske / Winword Exonar
> >
> > http://johnmieske.org
> >
> > Space Grid Station
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > --
> >
> > John Mieske / Winword Exonar
> >
> > http://johnmieske.org
> >
> > Space Grid Station
> >
> >
> > _______________________________________________
> > Opensim-users mailing list
> > Opensim-users at lists.berlios.de
> > https://lists.berlios.de/mailman/listinfo/opensim-users
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> >
> > Opensim-users mailing list
> >
> > Opensim-users at lists.berlios.de
> >
> > https://lists.berlios.de/mailman/listinfo/opensim-users
> >
> >
> >
> >
> >
> > _______________________________________________
> >
> > Opensim-users mailing list
> >
> > Opensim-users at lists.berlios.de
> >
> > https://lists.berlios.de/mailman/listinfo/opensim-users
> >
> >
> >
> >
> >
> > _______________________________________________
> >
> > Opensim-users mailing list
> >
> > Opensim-users at lists.berlios.de
> >
> > https://lists.berlios.de/mailman/listinfo/opensim-users
> >
> >
> >
> >
> >
> > -----Inline Attachment Follows-----
> >
> > _______________________________________________
> > Opensim-users mailing list
> > Opensim-users at lists.berlios.de
> > https://lists.berlios.de/mailman/listinfo/opensim-users
> >
>
>
>
> _______________________________________________
> Opensim-users mailing list
> Opensim-users at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-users
>
> _______________________________________________
> Opensim-users mailing list
> Opensim-users at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-users
>
More information about the Opensim-users
mailing list