[Opensim-dev] Memory cache
Melanie
melanie at t-data.com
Thu May 21 20:50:05 UTC 2009
What code are you looking at? You mightbe looking at the _old_ cache
(which is removed in trunk)
Melanie
Mic Bowman wrote:
> so removing the asset cache from a "reasonable" number of regions has
> not changed the number of requests to the asset server. it sure sounds
> like asset cache was not absorbing many of the asset requests.
>
> regarding the performance... nothing concrete.. but the way the
> process queue function in the cache code looks to be written right
> now, it would process at most five asset requests then sleep for half
> a second before it processed any others. if the cache was not
> absorbing any requests... then the best you're going to get is 10
> assets per second. which for a complex scene might take awhile.
>
> --mic
>
>
> On Thu, May 21, 2009 at 11:57 AM, Nebadon Izumi <nebadon2025 at gmail.com> wrote:
>> osgrid has seen 0% increase in traffic to the asset server.
>>
>> http://assets.osgrid.org/stats
>>
>> have a look for yourself.
>>
>>
>> On Thu, May 21, 2009 at 11:55 AM, Melanie <melanie at t-data.com> wrote:
>>>
>>> Indeed, it might be it's not really working....
>>>
>>> Melanie
>>>
>>>
>>> Mic Bowman wrote:
>>> > is there any instrumentation in the asset cache about hit rate? the
>>> > fact that there is no measurable increase reported on the osgrid asset
>>> > server with the caches turned off suggests that the cache isn't really
>>> > absorbing many hits (either because its not working the way its
>>> > expected to or because the re-use isn't significant).
>>> >
>>> > --mic
>>> >
>>> >
>>> > On Thu, May 21, 2009 at 11:43 AM, <diva at metaverseink.com> wrote:
>>> >> The cache, as configured, has 32K asset slots, which is a lot. So it
>>> >> takes a while until the cache releases some objects for the GC to
>>> >> collect. I think there's something else going on with that cache --
>>> >> probably too many locking conflicts between threads, or something like
>>> >> that. Whatever it is, it's much heavier than fetching the assets from
>>> >> the remote asset server, which is amazing. And another amazing thing is
>>> >> that remote fetching of so many assets at the same time (whenever a
>>> >> client connects) has such good observable performance for the user.
>>> >>
>>> >> If the purpose of the cache is to go easy on the asset server (not
>>> >> necessarily improve the observable performance, which is already very
>>> >> good without the mem cache) it might make more sense to use a file
>>> >> system / local DB instead of a memory cache.
>>> >>
>>> >> Stefan Andersson wrote:
>>> >>> If the cache is releasing objects, I think we might well be seeing
>>> >>> LOH fragmentation.
>>> >>>
>>> >>> http://msdn.microsoft.com/en-us/magazine/cc534993.aspx
>>> >>>
>>> >>> The solution would then be to break the binaries apart into byte cucks
>>> >>> smaller than the LOH trshold.
>>> >>>
>>> >>> Best regards,
>>> >>> Stefan Andersson
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> > From: adam at deepthink.com.au
>>> >>> > To: opensim-dev at lists.berlios.de
>>> >>> > Date: Wed, 20 May 2009 17:21:51 -0400
>>> >>> > Subject: Re: [Opensim-dev] Memory cache
>>> >>> >
>>> >>> > We are using an external caching library I believe, it could be
>>> >>> sitting in there.
>>> >>> >
>>> >>> > Adam
>>> >>> >
>>> >>> > > -----Original Message-----
>>> >>> > > From: opensim-dev-bounces at lists.berlios.de [mailto:opensim-dev-
>>> >>> > > bounces at lists.berlios.de] On Behalf Of Melanie
>>> >>> > > Sent: Wednesday, 20 May 2009 1:56 PM
>>> >>> > > To: opensim-dev at lists.berlios.de
>>> >>> > > Subject: Re: [Opensim-dev] Memory cache
>>> >>> > >
>>> >>> > > The surprise really is that the cache would be a performance loss
>>> >>> > > instead of a performance gain. What is there that makes it so
>>> >>> slow?
>>> >>> > >
>>> >>> > > Melanie
>>> >>> > >
>>> >>> > > Frisby, Adam wrote:
>>> >>> > > > As chief engineer aboard the USS OSgrid, I might want to
>>> >>> recommend
>>> >>> > > against this.
>>> >>> > > >
>>> >>> > > > Not having the sims cache assets will mean that every asset
>>> >>> request
>>> >>> > > will hit the core asset server which in turn will result in
>>> >>> higher
>>> >>> > > bandwidth requirements for it in the long run. It doesn't appear
>>> >>> to
>>> >>> > > have made a huge impact on our bandwidth charts for this week yet
>>> >>> -
>>> >>> > > however I wouldn't be surprised if it did if everyone did this.
>>> >>> > > >
>>> >>> > > > Adam
>>> >>> > > >
>>> >>> > > > From: opensim-dev-bounces at lists.berlios.de [mailto:opensim-dev-
>>> >>> > > bounces at lists.berlios.de] On Behalf Of Nebadon Izumi
>>> >>> > > > Sent: Wednesday, 20 May 2009 10:19 AM
>>> >>> > > > To: diva at metaverseink.com; opensim-dev at lists.berlios.de
>>> >>> > > > Subject: Re: [Opensim-dev] Memory cache
>>> >>> > > >
>>> >>> > > > Yay for mistakes that end in gains!!! of course if anyone asks
>>> >>> i
>>> >>> > > planned this ;P anyway i thought people would like to see my ini
>>> >>> > > changes.. this is what lead to the discovery:
>>> >>> > > >
>>> >>> > > > [OpenSim.ini]
>>> >>> > > > ; The following is the configuration section for the new style
>>> >>> > > grid servers
>>> >>> > > > ; If you don't know what this is, don't enable it. It will eat
>>> >>> > > your data,
>>> >>> > > > ; format your hard drive and make all meat in your fridge
>>> >>> spoil.
>>> >>> > > > ; You have been warned.
>>> >>> > > > ; Some of this is starting to work!
>>> >>> > > > [Modules]
>>> >>> > > > ; Choose one
>>> >>> > > > ;AssetServices = "LocalAssetServicesConnector"
>>> >>> > > > AssetServices = "RemoteAssetServicesConnector"
>>> >>> > > > ;AssetServices = "HGAssetBroker"
>>> >>> > > >
>>> >>> > > > ; If you don't want asset caching in the regions, comment this
>>> >>> > > > AssetCaching = "CoreAssetCache" ;<<<---*** SEE ERROR HERE
>>> >>> ***---
>>> >>> > > ->>>
>>> >>> > > >
>>> >>> > > > ; Choose one
>>> >>> > > > ;UserServices = "LocalUserServicesConnector"
>>> >>> > > > UserServices = "RemoteUserServicesConnector"
>>> >>> > > >
>>> >>> > > >
>>> >>> > > > [AssetService]
>>> >>> > > > ; Parameters for local assets, formerly known as "standalone"
>>> >>> > > > LocalServiceModule =
>>> >>> > > "OpenSim.Services.AssetService.dll:AssetService"
>>> >>> > > > StorageProvider = "OpenSim.Data.SQLite.dll"
>>> >>> > > > ;StorageProvider = "OpenSim.Data.MySQL.dll"
>>> >>> > > > ;ConnectionString = "Data
>>> >>> Source=localhost;Database=opensim;User
>>> >>> > > ID=opensim;Password=opensim;"
>>> >>> > > > DefaultAssetLoader =
>>> >>> > > "OpenSim.Framework.AssetLoader.Filesystem.dll"
>>> >>> > > > AssetLoaderArgs = "assets/AssetSets.xml"
>>> >>> > > >
>>> >>> > > > ; Parameters for remote assets, formerly known as "grid"
>>> >>> > > > AssetServerURI = "http://assets.osgrid.org:8003/"
>>> >>> > > >
>>> >>> > > > ; Paremeters for the Hypergrid connector
>>> >>> > > >
>>> >>> > > > ;; Parameters for the HG Broker
>>> >>> > > > ; Use this one if you have a standalone grid
>>> >>> > > > ;LocalGridAssetService =
>>> >>> > > "OpenSim.Services.AssetService.dll:AssetService"
>>> >>> > > > ; Use this one if this sim is connected to a grid-wide asset
>>> >>> > > server
>>> >>> > > > ;LocalGridAssetService =
>>> >>> > > "OpenSim.Services.Connectors.dll:AssetServiceConnector"
>>> >>> > > > ;HypergridAssetService =
>>> >>> > > "OpenSim.Services.AssetService.dll:HGAssetService"
>>> >>> > > >
>>> >>> > > > [AssetCache]
>>> >>> > > > ; Number of buckets for assets
>>> >>> > > > CacheBuckets = 32768
>>> >>> > > >
>>> >>> > > > [UserService]
>>> >>> > > > ;LocalServiceModule =
>>> >>> > > "OpenSim.Services.UserService.dll:UserService"
>>> >>> > > >
>>> >>> > > > [/OpenSim.ini]
>>> >>> > > >
>>> >>> > > > Please test this out and see if you see gains too and let us
>>> >>> know.
>>> >>> > > >
>>> >>> > > > Neb
>>> >>> > > >
>>> >>> > > >
>>> >>> > > >
>>> >>> > > >
>>> >>> ---------------------------------------------------------------------
>>> >>> > > ---
>>> >>> > > >
>>> >>> > > > _______________________________________________
>>> >>> > > > Opensim-dev mailing list
>>> >>> > > > Opensim-dev at lists.berlios.de
>>> >>> > > > https://lists.berlios.de/mailman/listinfo/opensim-dev
>>> >>> > > _______________________________________________
>>> >>> > > Opensim-dev mailing list
>>> >>> > > Opensim-dev at lists.berlios.de
>>> >>> > > https://lists.berlios.de/mailman/listinfo/opensim-dev
>>> >>> > _______________________________________________
>>> >>> > Opensim-dev mailing list
>>> >>> > Opensim-dev at lists.berlios.de
>>> >>> > https://lists.berlios.de/mailman/listinfo/opensim-dev
>>> >>>
>>> >>>
>>> >>>
>>> >>> ------------------------------------------------------------------------
>>> >>>
>>> >>> _______________________________________________
>>> >>> Opensim-dev mailing list
>>> >>> Opensim-dev at lists.berlios.de
>>> >>> https://lists.berlios.de/mailman/listinfo/opensim-dev
>>> >> _______________________________________________
>>> >> Opensim-dev mailing list
>>> >> Opensim-dev at lists.berlios.de
>>> >> https://lists.berlios.de/mailman/listinfo/opensim-dev
>>> >>
>>> > _______________________________________________
>>> > Opensim-dev mailing list
>>> > Opensim-dev at lists.berlios.de
>>> > https://lists.berlios.de/mailman/listinfo/opensim-dev
>>> >
>>> >
>>> _______________________________________________
>>> Opensim-dev mailing list
>>> Opensim-dev at lists.berlios.de
>>> https://lists.berlios.de/mailman/listinfo/opensim-dev
>>
>>
>>
>> --
>> Michael Emory Cerquoni - Nebadon Izumi @ http://osgrid.org
>>
>> _______________________________________________
>> Opensim-dev mailing list
>> Opensim-dev at lists.berlios.de
>> https://lists.berlios.de/mailman/listinfo/opensim-dev
>>
>>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev
>
>
More information about the Opensim-dev
mailing list