[Opensim-dev] search server Re: Opensim-dev Digest, Vol 6, Issue 5
Dalien Talbot
dalienta at gmail.com
Mon Feb 4 16:02:42 UTC 2008
On Feb 4, 2008 12:08 PM, Sakai Openlife <sakai at openlifegrid.com> wrote:
> I think this search idea also re-inforces my earlier call (already in
> Mantis) that parcel data should be held grid side. Rather than the current
> position of in the local prim store.
>
in other words you are suggesting all the websites should hold their data in
google cache as their primary content repository ?
how does yahoo get the content then ? we don't want monopolies :-)
indeed the search servers can and should have a *secondary* copy of the
information.
But making them hold the *primary* copy IMHO makes that yet another
bottleneck and maintenance load, alongside with the UGA.
I'm talking about a server to hold a few dozen millions of records and
~10-100 requests per second
/d
>
> However the option of it being local should still be there as not every
> application of the OpenSimulator will be an 'SL Like' style Grid.<-Perhaps
> flagged in the .ini
>
> It may also be beneficial to have a 'parcel' just for handling this things
> much like we have a grid server, as not to overwhelm existing services
> such
> as GRID or USER.
>
> Sakai Openlife
>
> -----Original Message-----
> From: opensim-dev-bounces at lists.berlios.de
> [mailto:opensim-dev-bounces at lists.berlios.de] On Behalf Of
> opensim-dev-request at lists.berlios.de
> Sent: Monday, 4 February 2008 9:02 PM
> To: opensim-dev at lists.berlios.de
> Subject: Opensim-dev Digest, Vol 6, Issue 5
>
> Send Opensim-dev mailing list submissions to
> opensim-dev at lists.berlios.de
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.berlios.de/mailman/listinfo/opensim-dev
> or, via email, send a message with subject or body 'help' to
> opensim-dev-request at lists.berlios.de
>
> You can reach the person managing the list at
> opensim-dev-owner at lists.berlios.de
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Opensim-dev digest..."
>
>
> Today's Topics:
>
> 1. Search server DB schema (David Wendt JR.)
> 2. Re: Search server DB schema (Justin Clark-Casey)
> 3. Re: Search server DB schema (Dalien Talbot)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sun, 3 Feb 2008 10:19:35 -0800 (PST)
> From: "David Wendt JR." <dcrkid at yahoo.com>
> Subject: [Opensim-dev] Search server DB schema
> To: Opensim-dev mailing list <opensim-dev at lists.berlios.de>
> Message-ID: <171180.25639.qm at web58314.mail.re3.yahoo.com>
> Content-Type: text/plain; charset="us-ascii"
>
> Hello, OpenSim developers.
>
> I have recently taken up OpenSim development. I'm writing plugins to
> allow Search to work. I need your opinion on a database schema for storing
> the indexed search information for a hypothetical (so far) new grid
> service
> "SearchServer".
>
> I figure out for searching "Places", "Popular Places", and "Land Sales"
> we will need the following fields: Parcel UUID, Name, Description,
> LandArea,
> Traffic, Region UUID, Region Maturity Flag, Region Parent Estate ID, Loc
> X,
> Loc Y, Loc Z, Image UUID, ForSaleStatus, Category, ForSalePrice,
> AuctionID,
> Owner UUID, OwnerIsGroup Flag. Some of these fields won't be implemented
> for
> a long time (especially ParentEstates) but it's good to have those anyway.
>
> The "Classifieds" search tab requires these fields: Entry UUID, Name,
> Description, Publisher UUID, Publisher Parent Estate ID, Image UUID,
> ClassifiedPrice, AutoRenewFlag, Category, Region UUID, Loc X, Loc Y, Loc
> Z,
> MatureFlag.
>
> The "Events" calendar search tab requires these fields: Event UUID,
> Name, Description, Publisher UUID, Publisher Parent Estate ID, Mature
> Content Flag, Event Date, Event Length, Region UUID, Loc X, Loc Y, Loc Z,
> Cover Charge. The Event calendar also supports notifying people who
> opt-in,
> so it would also require another table with Event UUID and Subscribing
> Avatar UUID.
>
> New Search (which I don't forsee being able to implement until way
> later) also requires sim object data. Object UUID, Parent Parcel UUID,
> Name,
> ForSaleStatus, ForSalePrice, Loc X, Loc Y, Loc Z seem to be good enough.
>
> Now I need your opinion on this schema. Is it a good idea to have each
> type of searchable object in a separate table? Or would it be more
> feasible
> to have all the relevant fields in a single table "search" with an
> "objecttype" enum to differentiate between the types?
>
> From,
> KMeist Hax
>
>
>
>
>
>
> ____________________________________________________________________________
> ________
> Be a better friend, newshound, and
> know-it-all with Yahoo! Mobile. Try it now.
> http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
>
> https://lists.berlios.de/pipermail/opensim-dev/attachments/20080203/15b30aed
> /attachment-0001.html
>
> ------------------------------
>
> Message: 2
> Date: Mon, 04 Feb 2008 08:29:26 +0000
> From: Justin Clark-Casey <jjustincc at googlemail.com>
> Subject: Re: [Opensim-dev] Search server DB schema
> To: opensim-dev at lists.berlios.de
> Message-ID: <47A6CCE6.4020205 at googlemail.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Hi David,
>
> Personally, my instinct would be to have one table to hold all the
> common fields between these searchable objects, and separate tables to
> hold the details specific to each.
>
> However, this opinion is formed more by my desire for neatness (not
> having lots of redundant attributes for search objects where certain
> fields don't apply) than any knowledge about how this may affect
> performance.
>
> Regards,
>
> --
> justincc
>
> David Wendt JR. wrote:
> > Hello, OpenSim developers.
> >
> > I have recently taken up OpenSim development. I'm writing plugins
> > to allow Search to work. I need your opinion on a database schema for
> > storing the indexed search information for a hypothetical (so far) new
> > grid service "SearchServer".
> >
> > I figure out for searching "Places", "Popular Places", and "Land
> > Sales" we will need the following fields: Parcel UUID, Name,
> > Description, LandArea, Traffic, Region UUID, Region Maturity Flag,
> > Region Parent Estate ID, Loc X, Loc Y, Loc Z, Image UUID,
> > ForSaleStatus, Category, ForSalePrice, AuctionID, Owner UUID,
> > OwnerIsGroup Flag. Some of these fields won't be implemented for a
> > long time (especially ParentEstates) but it's good to have those anyway.
> >
> > The "Classifieds" search tab requires these fields: Entry UUID,
> > Name, Description, Publisher UUID, Publisher Parent Estate ID, Image
> > UUID, ClassifiedPrice, AutoRenewFlag, Category, Region UUID, Loc X,
> > Loc Y, Loc Z, MatureFlag.
> >
> > The "Events" calendar search tab requires these fields: Event
> > UUID, Name, Description, Publisher UUID, Publisher Parent Estate ID,
> > Mature Content Flag, Event Date, Event Length, Region UUID, Loc X, Loc
> > Y, Loc Z, Cover Charge. The Event calendar also supports notifying
> > people who opt-in, so it would also require another table with Event
> > UUID and Subscribing Avatar UUID.
> >
> > New Search (which I don't forsee being able to implement until way
> > later) also requires sim object data. Object UUID, Parent Parcel UUID,
> > Name, ForSaleStatus, ForSalePrice, Loc X, Loc Y, Loc Z seem to be good
> > enough.
> >
> > Now I need your opinion on this schema. Is it a good idea to have
> > each type of searchable object in a separate table? Or would it be
> > more feasible to have all the relevant fields in a single table
> > "search" with an "objecttype" enum to differentiate between the types?
> >
> > From,
> > KMeist Hax
> >
> > ------------------------------------------------------------------------
> > Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try
> > it now.
> >
> <
> http://us.rd.yahoo.com/evt=51733/*http://mobile.yahoo.com/;_ylt=Ahu06i62sR8
> HDtDypao8Wcj9tAcJ%20>
> >
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > Opensim-dev mailing list
> > Opensim-dev at lists.berlios.de
> > https://lists.berlios.de/mailman/listinfo/opensim-dev
> >
>
>
> --
> justincc
> Justin Clark-Casey
>
>
>
> ------------------------------
>
> Message: 3
> Date: Mon, 4 Feb 2008 09:59:37 +0100
> From: "Dalien Talbot" <dalienta at gmail.com>
> Subject: Re: [Opensim-dev] Search server DB schema
> To: opensim-dev at lists.berlios.de
> Message-ID:
> <970c50810802040059p6d7389dav6b6dc6db80154f37 at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> David,
>
> just curious - have you played with the non-relational methods of doing
> the
> search ?
>
> I've had a very good success with Xapian library - and it has bindings for
> anything and everything (although - yick. this would mean one more native
> code lib in the code base).
>
> But the upside is that it would allow for more flexible "google-like"
> searches, imho - which is the very big plus. And you can allow quite a few
> custom tags to be included into boolean search - even though it is not
> very
> fast to index, the convenience of working with it was extremely pleasing.
>
> Another interesting beast is sphinx - I did not test it personally, but
> from
> what they write here:
>
>
> http://www.mysqlperformanceblog.com/2007/07/23/sphinx-going-beyond-full-text
> -search/
>
> It looks quite promising scalability-wise.
>
> An engine which was theoretically very good (at least according to the
> benchmarks that I've read) was zettair. However, playing with the code
> showed the quality of it is really "alpha" - sprinkled with assert's - and
> not without a few bugs too :-) - so probably more of theoretical
> interest.
>
> /d
>
>
>
>
> On Feb 3, 2008 7:19 PM, David Wendt JR. <dcrkid at yahoo.com> wrote:
>
> > Hello, OpenSim developers.
> >
> > I have recently taken up OpenSim development. I'm writing plugins to
> > allow Search to work. I need your opinion on a database schema for
> storing
> > the indexed search information for a hypothetical (so far) new grid
> service
> > "SearchServer".
> >
> > I figure out for searching "Places", "Popular Places", and "Land
> > Sales" we will need the following fields: Parcel UUID, Name,
> Description,
> > LandArea, Traffic, Region UUID, Region Maturity Flag, Region Parent
> Estate
> > ID, Loc X, Loc Y, Loc Z, Image UUID, ForSaleStatus, Category,
> ForSalePrice,
> > AuctionID, Owner UUID, OwnerIsGroup Flag. Some of these fields won't be
> > implemented for a long time (especially ParentEstates) but it's good to
> have
> > those anyway.
> >
> > The "Classifieds" search tab requires these fields: Entry UUID,
> Name,
> > Description, Publisher UUID, Publisher Parent Estate ID, Image UUID,
> > ClassifiedPrice, AutoRenewFlag, Category, Region UUID, Loc X, Loc Y, Loc
> Z,
> > MatureFlag.
> >
> > The "Events" calendar search tab requires these fields: Event UUID,
> > Name, Description, Publisher UUID, Publisher Parent Estate ID, Mature
> > Content Flag, Event Date, Event Length, Region UUID, Loc X, Loc Y, Loc
> Z,
> > Cover Charge. The Event calendar also supports notifying people who
> opt-in,
> > so it would also require another table with Event UUID and Subscribing
> > Avatar UUID.
> >
> > New Search (which I don't forsee being able to implement until way
> > later) also requires sim object data. Object UUID, Parent Parcel UUID,
> Name,
> > ForSaleStatus, ForSalePrice, Loc X, Loc Y, Loc Z seem to be good enough.
> >
> > Now I need your opinion on this schema. Is it a good idea to have
> each
> > type of searchable object in a separate table? Or would it be more
> feasible
> > to have all the relevant fields in a single table "search" with an
> > "objecttype" enum to differentiate between the types?
> >
> > From,
> > KMeist Hax
> >
> > ------------------------------
> > Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try
> it
> >
> now.<
> http://us.rd.yahoo.com/evt=51733/*http://mobile.yahoo.com/;_ylt=Ahu06i6
> 2sR8HDtDypao8Wcj9tAcJ>
> >
> > _______________________________________________
> > Opensim-dev mailing list
> > Opensim-dev at lists.berlios.de
> > https://lists.berlios.de/mailman/listinfo/opensim-dev
> >
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
>
> https://lists.berlios.de/pipermail/opensim-dev/attachments/20080204/4c13fe39
> /attachment-0001.html
>
> ------------------------------
>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev
>
>
> End of Opensim-dev Digest, Vol 6, Issue 5
> *****************************************
>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20080204/8779905b/attachment-0001.html>
More information about the Opensim-dev
mailing list