[Opensim-dev] Memory cache
Mic Bowman
cmickeyb at gmail.com
Thu May 21 19:06:21 UTC 2009
so removing the asset cache from a "reasonable" number of regions has
not changed the number of requests to the asset server. it sure sounds
like asset cache was not absorbing many of the asset requests.
regarding the performance... nothing concrete.. but the way the
process queue function in the cache code looks to be written right
now, it would process at most five asset requests then sleep for half
a second before it processed any others. if the cache was not
absorbing any requests... then the best you're going to get is 10
assets per second. which for a complex scene might take awhile.
--mic
On Thu, May 21, 2009 at 11:57 AM, Nebadon Izumi <nebadon2025 at gmail.com> wrote:
> osgrid has seen 0% increase in traffic to the asset server.
>
> http://assets.osgrid.org/stats
>
> have a look for yourself.
>
>
> On Thu, May 21, 2009 at 11:55 AM, Melanie <melanie at t-data.com> wrote:
>>
>> Indeed, it might be it's not really working....
>>
>> Melanie
>>
>>
>> Mic Bowman wrote:
>> > is there any instrumentation in the asset cache about hit rate? the
>> > fact that there is no measurable increase reported on the osgrid asset
>> > server with the caches turned off suggests that the cache isn't really
>> > absorbing many hits (either because its not working the way its
>> > expected to or because the re-use isn't significant).
>> >
>> > --mic
>> >
>> >
>> > On Thu, May 21, 2009 at 11:43 AM, <diva at metaverseink.com> wrote:
>> >> The cache, as configured, has 32K asset slots, which is a lot. So it
>> >> takes a while until the cache releases some objects for the GC to
>> >> collect. I think there's something else going on with that cache --
>> >> probably too many locking conflicts between threads, or something like
>> >> that. Whatever it is, it's much heavier than fetching the assets from
>> >> the remote asset server, which is amazing. And another amazing thing is
>> >> that remote fetching of so many assets at the same time (whenever a
>> >> client connects) has such good observable performance for the user.
>> >>
>> >> If the purpose of the cache is to go easy on the asset server (not
>> >> necessarily improve the observable performance, which is already very
>> >> good without the mem cache) it might make more sense to use a file
>> >> system / local DB instead of a memory cache.
>> >>
>> >> Stefan Andersson wrote:
>> >>> If the cache is releasing objects, I think we might well be seeing
>> >>> LOH fragmentation.
>> >>>
>> >>> http://msdn.microsoft.com/en-us/magazine/cc534993.aspx
>> >>>
>> >>> The solution would then be to break the binaries apart into byte cucks
>> >>> smaller than the LOH trshold.
>> >>>
>> >>> Best regards,
>> >>> Stefan Andersson
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> > From: adam at deepthink.com.au
>> >>> > To: opensim-dev at lists.berlios.de
>> >>> > Date: Wed, 20 May 2009 17:21:51 -0400
>> >>> > Subject: Re: [Opensim-dev] Memory cache
>> >>> >
>> >>> > We are using an external caching library I believe, it could be
>> >>> sitting in there.
>> >>> >
>> >>> > Adam
>> >>> >
>> >>> > > -----Original Message-----
>> >>> > > From: opensim-dev-bounces at lists.berlios.de [mailto:opensim-dev-
>> >>> > > bounces at lists.berlios.de] On Behalf Of Melanie
>> >>> > > Sent: Wednesday, 20 May 2009 1:56 PM
>> >>> > > To: opensim-dev at lists.berlios.de
>> >>> > > Subject: Re: [Opensim-dev] Memory cache
>> >>> > >
>> >>> > > The surprise really is that the cache would be a performance loss
>> >>> > > instead of a performance gain. What is there that makes it so
>> >>> slow?
>> >>> > >
>> >>> > > Melanie
>> >>> > >
>> >>> > > Frisby, Adam wrote:
>> >>> > > > As chief engineer aboard the USS OSgrid, I might want to
>> >>> recommend
>> >>> > > against this.
>> >>> > > >
>> >>> > > > Not having the sims cache assets will mean that every asset
>> >>> request
>> >>> > > will hit the core asset server which in turn will result in
>> >>> higher
>> >>> > > bandwidth requirements for it in the long run. It doesn't appear
>> >>> to
>> >>> > > have made a huge impact on our bandwidth charts for this week yet
>> >>> -
>> >>> > > however I wouldn't be surprised if it did if everyone did this.
>> >>> > > >
>> >>> > > > Adam
>> >>> > > >
>> >>> > > > From: opensim-dev-bounces at lists.berlios.de [mailto:opensim-dev-
>> >>> > > bounces at lists.berlios.de] On Behalf Of Nebadon Izumi
>> >>> > > > Sent: Wednesday, 20 May 2009 10:19 AM
>> >>> > > > To: diva at metaverseink.com; opensim-dev at lists.berlios.de
>> >>> > > > Subject: Re: [Opensim-dev] Memory cache
>> >>> > > >
>> >>> > > > Yay for mistakes that end in gains!!! of course if anyone asks
>> >>> i
>> >>> > > planned this ;P anyway i thought people would like to see my ini
>> >>> > > changes.. this is what lead to the discovery:
>> >>> > > >
>> >>> > > > [OpenSim.ini]
>> >>> > > > ; The following is the configuration section for the new style
>> >>> > > grid servers
>> >>> > > > ; If you don't know what this is, don't enable it. It will eat
>> >>> > > your data,
>> >>> > > > ; format your hard drive and make all meat in your fridge
>> >>> spoil.
>> >>> > > > ; You have been warned.
>> >>> > > > ; Some of this is starting to work!
>> >>> > > > [Modules]
>> >>> > > > ; Choose one
>> >>> > > > ;AssetServices = "LocalAssetServicesConnector"
>> >>> > > > AssetServices = "RemoteAssetServicesConnector"
>> >>> > > > ;AssetServices = "HGAssetBroker"
>> >>> > > >
>> >>> > > > ; If you don't want asset caching in the regions, comment this
>> >>> > > > AssetCaching = "CoreAssetCache" ;<<<---*** SEE ERROR HERE
>> >>> ***---
>> >>> > > ->>>
>> >>> > > >
>> >>> > > > ; Choose one
>> >>> > > > ;UserServices = "LocalUserServicesConnector"
>> >>> > > > UserServices = "RemoteUserServicesConnector"
>> >>> > > >
>> >>> > > >
>> >>> > > > [AssetService]
>> >>> > > > ; Parameters for local assets, formerly known as "standalone"
>> >>> > > > LocalServiceModule =
>> >>> > > "OpenSim.Services.AssetService.dll:AssetService"
>> >>> > > > StorageProvider = "OpenSim.Data.SQLite.dll"
>> >>> > > > ;StorageProvider = "OpenSim.Data.MySQL.dll"
>> >>> > > > ;ConnectionString = "Data
>> >>> Source=localhost;Database=opensim;User
>> >>> > > ID=opensim;Password=opensim;"
>> >>> > > > DefaultAssetLoader =
>> >>> > > "OpenSim.Framework.AssetLoader.Filesystem.dll"
>> >>> > > > AssetLoaderArgs = "assets/AssetSets.xml"
>> >>> > > >
>> >>> > > > ; Parameters for remote assets, formerly known as "grid"
>> >>> > > > AssetServerURI = "http://assets.osgrid.org:8003/"
>> >>> > > >
>> >>> > > > ; Paremeters for the Hypergrid connector
>> >>> > > >
>> >>> > > > ;; Parameters for the HG Broker
>> >>> > > > ; Use this one if you have a standalone grid
>> >>> > > > ;LocalGridAssetService =
>> >>> > > "OpenSim.Services.AssetService.dll:AssetService"
>> >>> > > > ; Use this one if this sim is connected to a grid-wide asset
>> >>> > > server
>> >>> > > > ;LocalGridAssetService =
>> >>> > > "OpenSim.Services.Connectors.dll:AssetServiceConnector"
>> >>> > > > ;HypergridAssetService =
>> >>> > > "OpenSim.Services.AssetService.dll:HGAssetService"
>> >>> > > >
>> >>> > > > [AssetCache]
>> >>> > > > ; Number of buckets for assets
>> >>> > > > CacheBuckets = 32768
>> >>> > > >
>> >>> > > > [UserService]
>> >>> > > > ;LocalServiceModule =
>> >>> > > "OpenSim.Services.UserService.dll:UserService"
>> >>> > > >
>> >>> > > > [/OpenSim.ini]
>> >>> > > >
>> >>> > > > Please test this out and see if you see gains too and let us
>> >>> know.
>> >>> > > >
>> >>> > > > Neb
>> >>> > > >
>> >>> > > >
>> >>> > > >
>> >>> > > >
>> >>> ---------------------------------------------------------------------
>> >>> > > ---
>> >>> > > >
>> >>> > > > _______________________________________________
>> >>> > > > Opensim-dev mailing list
>> >>> > > > Opensim-dev at lists.berlios.de
>> >>> > > > https://lists.berlios.de/mailman/listinfo/opensim-dev
>> >>> > > _______________________________________________
>> >>> > > Opensim-dev mailing list
>> >>> > > Opensim-dev at lists.berlios.de
>> >>> > > https://lists.berlios.de/mailman/listinfo/opensim-dev
>> >>> > _______________________________________________
>> >>> > Opensim-dev mailing list
>> >>> > Opensim-dev at lists.berlios.de
>> >>> > https://lists.berlios.de/mailman/listinfo/opensim-dev
>> >>>
>> >>>
>> >>>
>> >>> ------------------------------------------------------------------------
>> >>>
>> >>> _______________________________________________
>> >>> Opensim-dev mailing list
>> >>> Opensim-dev at lists.berlios.de
>> >>> https://lists.berlios.de/mailman/listinfo/opensim-dev
>> >> _______________________________________________
>> >> Opensim-dev mailing list
>> >> Opensim-dev at lists.berlios.de
>> >> https://lists.berlios.de/mailman/listinfo/opensim-dev
>> >>
>> > _______________________________________________
>> > Opensim-dev mailing list
>> > Opensim-dev at lists.berlios.de
>> > https://lists.berlios.de/mailman/listinfo/opensim-dev
>> >
>> >
>> _______________________________________________
>> Opensim-dev mailing list
>> Opensim-dev at lists.berlios.de
>> https://lists.berlios.de/mailman/listinfo/opensim-dev
>
>
>
> --
> Michael Emory Cerquoni - Nebadon Izumi @ http://osgrid.org
>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev
>
>
More information about the Opensim-dev
mailing list