[Opensim-dev] Memory cache

[Imaze Rhiano]

Hi!

I have done some research about asset server. I used lates trunk from 
this morning.

Here are my findings:
1. CoreAssetCache's CacheBuckets configuration is not working properly. 
Cache's size is limited to 1024 buckets, because of bug or by design. So 
"default" configuration value 32768 - didn't never happen.
    See:  private void SetSize(int newSize) in Cache.cs
             lock (m_Index)
            {
                if (Count <= Size)
                    return;
                .....
             }
    During initialization phase, Count is 0 - so initial size value 1024 
is used. (And Count never can't grow up bigger than Size)

2. GlynnTucker cache is basically Hashtable that contains all assets  
they are never removed from cache unless assets is exlusively removed by 
calling Expire method. It definately can provide huge performance impact 
for short time - but eventually your server is going to run out of  memory.

3. To evaluate different cache's performance I first tried to do some 
testing in stand alone SIM. Unfortunately single client, just didn't 
provide enough caching to really to measure performance, so I adopted 
different strategy. I did write simple Window$ console application with 
4 different test cases.

Test Case 1 "EnumerateAllTest": Just add assets to cache by predefined 
order and then try to get them from cache in same order. (Basically 
caches were pretty much guranteed to forgot first few thousands assets.)
Test Case 2 "GetRandom":  Try to get random asset from cache, and if 
asset is not found, add it to there.
Test Case 3 "GetRandomMoreFrequentTest": Try to get random asset from 
cache, and if asset is not found, add it to there. However, this time 
there 1000 assets that are more likely asked than other assets (80% chance).
Test Case 4 "GetRandomMoreFrequentWithExpireTest": Same as Test Case 3 - 
except that there is 25% chance to remove (expire) random assets from 
cache for each item retrieve iteration. Remove assets didn't actually 
need to be in cache.

 From these test cases, case 4 is maybe most realistic. Assets are 
requested randomly, but some of assets are requested more often. And 
sometimes assets are also expired explicitely.

Test program did measure used time for each test case and cache hit rate 
(how frequently cache actually did return requested assets). I also did 
write two additional caches - Cenome Assets Cache (this is very basic 
memory cache coming from commercial database project that I am currently 
working on) and simple Dictionary based cache (to see difference between 
real caches and maximal possible performance). There was 100 000 
randomly generated "assets" with data length between 393 170 and 10 
bytes - total asset size 7 361 310 300 bytes.

Here are performance results:

TIME
                                      Test Case
                                                1                
2               3                        4
CoreAssetCache                      6.552s       33.446s     1m 
10.668s        1m 16.222s
GlynnTuckerAssetCache          0.359s       0.811s       
3.292s                7.691s
CenomeAssetsCache               0.156s       0.764s       
2.309s                4.165s
DictionaryAssets                      0.203s       0.468s       
2.231s                3.900s

HIT COUNT
                                      Test Case
                                                1                
2               3                        4
CoreAssetCache                      1.0%          1.0%         
55.1%               55.0%
GlynnTuckerAssetCache          100%         75.4%       95.1% 
               83.3%
CenomeAssetsCache               4.0%          5.9%         
81.0%                80.8%
DictionaryAssets                      100%         75.4%       
95.1%                83.3%

As you can see, CoreAssetCache is 15-20 times slower than more optimal 
solutions. And GlynnTucker assets cache hit rate is optimal - because it 
is storing all assets - and never forget them.

4. To find out why, CoreAssetsCache's performance is so poor compared to 
other caches, I did use JetBrain's dotTrace profiler. For profiling, I 
did lower assets count to 10000 and reduced iterations to lower 
(profiling is taking lot's of time) - so profiling measurements are not 
directly comparable with performance measurements.

Major cause for CoreAssetsCache bad performance was 
"OpenSim.Framework.Cache.Store(String, Object, Type, Object [])" methods 
line "if (m_Index.Contains(new CacheItemBase(index)))". About 75% of 
time in Test Case 4 was used in this line. m_index is 
List<CacheItemBase> object. Contains call is basically linear search 
algorithm - instead of Contains method, BinarySearch should have been used.

I thing that CoreAssetsCache class might be salvageable, but it really 
need some serious refactoring - like why there is two collection of keys 
m_Index and m_Loopup? It might be easier to either implement new cache 
class or maybe use CenomeCache as base of new implementation.

5. There is "some" weirdness in other parts of code:
- Cache class is also used directly in FriendsModule and 
LandManagementModule
- There is interface called: IAssetCache that is used in several places 
and modules, however just one Test class is implementing interface 
(TestAssetCache) (CoreAssetsCache, GlynnTuckerAssetCache and 
CenomeAssetsCache are implementing IImprovedAssetCache interface)

You can (hopefully) download test program from 
http://imazer.x10hosting.com/CacheTesting.zip
Please note CenomeAssetCache is not ready for production - it is missing 
thread safety. (And I don't have update rights to Open SIM's version 
control system.)
This was first day when I took serious look to Open SIMs code - so I 
might be wrong several things here...

- Imaze Rhiano

PS: It is possible that I could contribute licence to Cenome database 
for Open SIM project. Database is designed to store BLOB like objects 
with full transaction and threading support to single file. However - 
database is closed source and still under development. If closed source 
implementation is not acceptable - then alternative is B-tree variant 
implemation that I might be able to release to open source.