[Opensim-dev] Search server DB schema

[Charles Krinke]

Dear Diva:

I like your ideas and they seem worth pursuing. Interesting enough, I am in Irvine also and just down the street from you. Perhaps we will meet and discuss this in person sometime.

You might find it interesting that OSGrid is controlled from a series of servers at 15 Bayside near the corner of Culver and Alton.

Charles


----- Original Message ----
From: Diva Canto <diva at metaverseink.com>
To: opensim-dev at lists.berlios.de
Sent: Monday, February 4, 2008 3:42:26 PM
Subject: Re: [Opensim-dev] Search server DB schema

Hello opensim-developers,

First of all, thanks for putting opensim together, this is what we all 
needed! I just signed up for this list, so apologies if my comments are 
out of place. I did set up my own OpenSim, and it's great! (in spite of 
all the fights i've been having with mono on Mac, but that's another thread)

Over the past 10 months or so, a colleague and I have developed an 
independent search engine for LL's SL, which can be accessed here: 
http://slbrowser.com The engine does not require special access to grid 
databases, it uses bots to collect inworld information, one sim at a 
time. We use libsecondlife. It has been working continuously for the 
past 6 months; we crawl the grid twice a week with only 14 bots, and 
have been able to find appropriate heuristics for many things.

In general search can be looked at in two ways, and these are *not* 
incompatible: it can be a basic administrative function -- you want to 
know the data you serve; or it can be a basic user function -- you want 
people to be able find things. The first type of search is really 
simple: stick a DB, and you've solved it. This works fairly well for 
small amounts of data, and for data that is fairly constrained. The 
second type of search is a lot more powerful, but it's not so simple, 
because you want to rank the huge amount of results in a semantically 
meaningful way. With SLBrowser, we've followed the latter. Crawling the 
live sims gives us a lot of useful aggregate information that we use to 
experiment with ranking in much more interesting ways. We don't use a 
relational DB on the backend, we use Lucene.

I would hate that OpenSim follows Linden Lab's steps with search without 
taking advantage of the lessons that even they already learned -- that 
relational schemas are not appropriate for modern information retrieval. 
I'll be happy to help setting up this basic search service with Lucene, 
rather than with a relational DB. Lucene is, essentially, a highly 
optimized database for text search. For example, issues like this
" I'm going to assume name == varchar(63) and description == 
varchar(127), but it might be easier to just set everything to 
varchar(255) for flexibility."
are a non-problem in Lucene -- you can use as little or as much text as 
you want in a field, you don't need to hard-code that.

Can I help with plugging a Lucene-based search for OpenSim, please? (the 
thought of having a relational DB serving text search makes me shiver :-)
I've never participated in an Open Source project as such, so I'm not 
sure how the process is. I did contribute to OS projects before -- 
aspectj.org, co-founder, and more recently contributed plugins to XWiki 
with one of my students.

Let me know.

Crista Lopes / Diva Canto
School of Information and Computer Sciences
University of California, Irvine



_______________________________________________
Opensim-dev mailing list
Opensim-dev at lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20080205/33121441/attachment-0001.html>