[Opensim-dev] Introduction of a sitemap style system for users and land

[Grid Master]

The other thing to consider, is with all these new privacy laws, and even
some existing ones, scraping and republishing data from a website you don't
own, could be a grey area as far as the letter of the law is concerned.
 Having the data pushed to you is going to mean that the person is giving
their consent willingly.



On Wed, Jul 4, 2018 at 9:28 AM, Cinder Roxley <cinder at alchemyviewer.org>
wrote:

> Have you seen the functionality in Core that drives MetaverseInk’s web
> search already?
> http://www.metaverseink.com/blog/news/metaverse-ink-search-for-opensims/ I
> would suggest leveraging that for land instead. Crawling seems like a waste
> of resources when a grid could simply push the data to you in a format
> easily consumable. A module could be developed that pushes other updates to
> the search as well. (Does events work in Core? I don’t remember.) Failing
> that, a module that provides the information via a common api makes a lot
> more sense than scraping webpages.
>
> I wrote a search module a while back, I could open source it and you could
> build upon it with an external api that would provide consumers with
> relevant data in json or whatnot. Although, I still consider a push data
> model to the the more efficient design than poll and scrape.
>
> On July 4, 2018 at 7:56:44 AM, Alan Johnston (
> alan.johnston at allvirtualsocial.com) wrote:
>
> I did not receive the reply I think I set my options wrong in the mailman
> but I did see the reply in the online archives, Haravikk The point of
> sitemaps is you submit then to a search engine for a search engine to have
> a set of URLs for them to look up and get the information from. Its a map
> to the site so to speak, this is good for such things as social sites, that
> wont have links on the main site that point to every possible page there
> is, as an example if you go to https://www.allvirtualsocial.com you will
> not see my post I posted on there a few weeks ago or the post about me
> creating android apps that are now available on the google play, but if you
> look at the sitemap https://www.allvirtualsocial.com/sitemap.xml you will
> see the link in there, although search engines dont use them for the main
> system to provide ranking, they use it to know where to crawl on the site,
> especially for pages that are not that obvious to “spiders”, so site maps
> are more common than people think, but are used primarily for search
> engines, this is why I have thought about this a lot. The concept would be
> to provide the search engine with the information it needs to provide
> search results to a set type, and would make be able to provide the nest
> answers it can give. I have a lot of work to do on the system including the
> bots, I’m thinking of making a 3 bot system 1 that does sites in the
> general sense, I.e. my social site, grids websites, news sites like Hyper
> Grid News, Blog sites related to virtual worlds and so forth, one bot to do
> site map collection for such sites, and another bot to do the land and
> users maps in grids, I’m going to install OS on my system some point today
> and check out some things on the system, and going to make a PHP sitemap
> Implementation of landmaps. And look at something when it comes to
> usermaps, though I would need to speak to someone within OS and Viewers to
> see if a thought is possible though I’m doubtful, this is something I am
> hopeing that can either be a module (and someone can help me create a
> opensource module of this) or even have it as a core integration, and have
> it with some config options so then people can choose to opt in or out of
> it.
> The search engine is a long way off from being finished, I have yet to
> fully create the bots, I’m working on the first bot, and working on making
> it so it follows robots.txt and robots META tags, and working on the back
> end the options of what to use for the back end, elastic search, DB system
> like MySQL or alike, or even a custom engine design, I have a lot of work
> ahead of me and I’m working hard on it, I’m also working hard on the page
> ranking algorithm aswell which will be hard to do. If people are wanting to
> help with this I would be very happy to accept help on this project in any
> form.
>
> Sent from Mail for Windows 10
>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at opensimulator.org
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at opensimulator.org
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
>