[Opensim-dev] Search server DB schema

[Sean Dague]

On Mon, Feb 04, 2008 at 03:42:26PM -0800, Diva Canto wrote:
> In general search can be looked at in two ways, and these are *not* 
> incompatible: it can be a basic administrative function -- you want to 
> know the data you serve; or it can be a basic user function -- you want 
> people to be able find things. The first type of search is really 
> simple: stick a DB, and you've solved it. This works fairly well for 
> small amounts of data, and for data that is fairly constrained. The 
> second type of search is a lot more powerful, but it's not so simple, 
> because you want to rank the huge amount of results in a semantically 
> meaningful way. With SLBrowser, we've followed the latter. Crawling the 
> live sims gives us a lot of useful aggregate information that we use to 
> experiment with ranking in much more interesting ways. We don't use a 
> relational DB on the backend, we use Lucene.
> 
> I would hate that OpenSim follows Linden Lab's steps with search without 
> taking advantage of the lessons that even they already learned -- that 
> relational schemas are not appropriate for modern information retrieval. 
> I'll be happy to help setting up this basic search service with Lucene, 
> rather than with a relational DB. Lucene is, essentially, a highly 
> optimized database for text search. For example, issues like this
> " I'm going to assume name == varchar(63) and description == 
> varchar(127), but it might be easier to just set everything to 
> varchar(255) for flexibility."
> are a non-problem in Lucene -- you can use as little or as much text as 
> you want in a field, you don't need to hard-code that.
> 
> Can I help with plugging a Lucene-based search for OpenSim, please? (the 
> thought of having a relational DB serving text search makes me shiver :-)
> I've never participated in an Open Source project as such, so I'm not 
> sure how the process is. I did contribute to OS projects before -- 
> aspectj.org, co-founder, and more recently contributed plugins to XWiki 
> with one of my students.

+1 on doing this via Lucene.  My experiences with Lucene have always
been good, and I think that is the right way to approach the search
problem.

	-Sean

-- 
__________________________________________________________________

Sean Dague                                       Mid-Hudson Valley
sean at dague dot net                            Linux Users Group
http://dague.net                                 http://mhvlug.org

There is no silver bullet.  Plus, werewolves make better neighbors
than zombies, and they tend to keep the vampire population down.
__________________________________________________________________
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20080206/82f96a84/attachment-0001.pgp>