lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Smiley (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (SOLR-2155) Geospatial search using geohash prefixes
Date Mon, 03 Jan 2011 15:22:46 GMT

    [ https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12976608#action_12976608
] 

David Smiley edited comment on SOLR-2155 at 1/3/11 10:21 AM:
-------------------------------------------------------------

Hi Bill.  I think you are referring to Grant Ingersoll. Yes, this patch was developed to be
compatible with his efforts (which are not his alone).  They are applied "on top" of the existing
work, so to speak, enhancing the performance of the Lucene query returned by solr.GeoHashField
by using a completely different algorithm.  As such, you can use the existing query syntax
documented on the wiki http://wiki.apache.org/solr/SpatialSearch#bbox_-_Bounding-box_filter.
 Simply apply this patch to trunk, and use the solr.GeoHashField type to store your "lat,lon"
formatted data just like an example might show for solr.LatLonType.  Of course, with this
field type, you can supply as many values as you want.  You needn't know anything about geohashes.

This work is for GeoHashField, not for LatLonType.  These both have a similar feel for the
user, in which you provide a "lat,lon".  The difference is in the implementation.  solr.PointType
is non-geospatial--it's for a cartesian (i.e. x,y) plane.

An update on my work lately:

Since the posting of this patch in October, I've created my own query parser.  It handles
point-distance, a specific lat-lon box (instead of being forced to pick the box encompassing
a point-distance, which is what the bbox query does), and point-in-polygon searches.  The
polygon support required more than query parsing, of course, but figuring out how to actually
implement it.  I'm using the JTS library (LGPL open-source).

I have been evaluating the performance of my patch compared to LatLonType.  LatLonType scales
to match basically any number of points matched due to the underlying Trie based numeric fields
introduced in solr v1.4.  My geohash field doesn't scale as well... so if your query matches,
say, less than 100k distinct points then you'll meet or exceed LatLonType's performance (in
the default configuration).  However it's linear from there on out: 1000k (1M) points takes
10x as long as 100k whereas for LatLonType it's something like 1.5x as long.  I've been thinking
about some ideas to scale better and I hope to try it this week.  If you don't expect a query
to match more than ~100k points then the current patch will serve you well.

      was (Author: dsmiley):
    Hi Bill.  I think you are referring to Grant Ingersoll. Yes, this patch was developed
to be compatible with his efforts (which are not his alone).  They are applied "on top" of
the existing work, so to speak, enhancing the performance of the Lucene query returned by
solr.GeoHashField by using a completely different algorithm.  As such, you can use the existing
query syntax documented on the wiki http://wiki.apache.org/solr/SpatialSearch#bbox_-_Bounding-box_filter.
 Simply apply this patch to trunk, and use the solr.GeoHashField type to store your "lat,lon"
formatted data just like an example might show for solr.LatLonType.  Of course, with this
field type, you can supply as many values as you want.  You needn't know anything about geohashes.

This work is for GeoHashField, not for LatLonType.  These both have a similar feel for the
user, in which you provide a "lat,lon".  The difference is in the implementation.  solr.PointType
is non-geospatial--it's for a cartesian (i.e. x,y) plane.

An update on my work lately:

Since the posting of this patch in October, I've created my own query parser.  It handles
point-distance, a specific lat-lon box (instead of being forced to pick the box encompassing
a point-distance, which is what the bbox query does), and point-in-polygon searches.  The
polygon support required more than query parsing, of course, but figuring out how to actually
implement it.  I'm using the JTS library (LGPL open-source).

I have been evaluating the performance of my patch compared to LatLonType.  LatLonType scales
to match basically any number of points matched due to the underlying Trie based numeric fields
introduced in solr v1.4.  My geohash field doesn't scale as well... so if your query matches,
say, less than 10k distinct points then you'll meet or exceed LatLonType's performance (in
the default configuration).  However it's linear from there on out: 100k points takes 10x
as long as 10k whereas for LatLonType it's something like 1.5x as long.  I've been thinking
about some ideas to scale better and I hope to try it this week.  If you don't expect a query
to match more than ~10k points then the current patch will serve you well.
  
> Geospatial search using geohash prefixes
> ----------------------------------------
>
>                 Key: SOLR-2155
>                 URL: https://issues.apache.org/jira/browse/SOLR-2155
>             Project: Solr
>          Issue Type: Improvement
>            Reporter: David Smiley
>         Attachments: GeoHashPrefixFilter.patch
>
>
> There currently isn't a solution in Solr for doing geospatial filtering on documents
that have a variable number of points.  This scenario occurs when there is location extraction
(i.e. via a "gazateer") occurring on free text.  None, one, or many geospatial locations might
be extracted from any given document and users want to limit their search results to those
occurring in a user-specified area.
> I've implemented this by furthering the GeoHash based work in Lucene/Solr with a geohash
prefix based filter.  A geohash refers to a lat-lon box on the earth.  Each successive character
added further subdivides the box into a 4x8 (or 8x4 depending on the even/odd length of the
geohash) grid.  The first step in this scheme is figuring out which geohash grid squares cover
the user's search query.  I've added various extra methods to GeoHashUtils (and added tests)
to assist in this purpose.  The next step is an actual Lucene Filter, GeoHashPrefixFilter,
that uses these geohash prefixes in TermsEnum.seek() to skip to relevant grid squares in the
index.  Once a matching geohash grid is found, the points therein are compared against the
user's query to see if it matches.  I created an abstraction GeoShape extended by subclasses
named PointDistance... and CartesianBox.... to support different queried shapes so that the
filter need not care about these details.
> This work was presented at LuceneRevolution in Boston on October 8th.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message