lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adrien Grand <jpou...@gmail.com>
Subject Re: Need suggestions on implementing a custom query (offload R-tree filter to fully in-memory) on Lucene-8.3
Date Wed, 04 Dec 2019 08:14:19 GMT
Are you sure you are comparing apples to apples? The first paragraph
mentions a range filter, which would be LatLonPoint#newBoxQuery, but
then you mentioned LatLonPoint#newDistanceQuery, which is
significantly more costly due to the need to compute distances.

If you plan to combine text queries with your geo queries, I'd also
advise to index both with LatLonPoint and LatLonDocValuesField, and
then use IndexOrDocValuesQuery at query time. Typically something like
this:

```
Query textQuery = ...;
Query latLonPointQuery = LatLonPoint.newBoxQuery("poi", www, xxx, yyy, zzz);
Query latLonDocValuesQuery =
LatLonDocValuesField.newSlowBoxQuery("poi", www, xxx, yyy, zzz);
Query poiQuery = new IndexOrDocValuesQuery(latLonPointQuery,
latLonDocValuesQuery);
Query query = new BooleanQuery.Builder()
    .add(textQuery, Occur.MUST)
    .add(poiQuery, Occur.FILTER)
    .build();
```

On Wed, Dec 4, 2019 at 5:31 AM 小鱼儿 <ctengctsh@gmail.com> wrote:
>
> Background: i need to implement a document indexing and search for
> POIs(point of interest) under LBS scene. A POI has name, address, and
> location(LatLonPoint), and i want to combine a text query with a
> geo-spatial 2d range filter.
>
> The problem is, when i first build a native in-memory index which use a
> simple BitSet as DocIDSet type and STRTree class from the famous JTS lib, i
> get 20ms/1000qps perf metrics with 1w8 POIs on my laptop(Windows 7 x64, use
> mmap codec). But when i use Lucene-8.3 to implement the same
> functionality(which use LatLonPoint.newDistanceQuery which seems use the
> default BKD tree index), i only get 150ms/130qps which is a very bad
> degrade?
>
> So my idea is, can i do a custom filter query, which builds a fully
> in-memory R-tree index to boost the spatial2d range filter performance? I
> need to access Lucene's internal DocIDSet class so i can do a fast merge
> with no scoring needed. Hope this will improve the query performance.
>
> Any suggestions?



-- 
Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message