lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Smiley (JIRA)" <>
Subject [jira] [Assigned] (LUCENE-5056) Indexing non-point shapes close to the poles doesn't scale
Date Wed, 20 Nov 2013 19:42:37 GMT


David Smiley reassigned LUCENE-5056:

    Assignee: David Smiley

> Indexing non-point shapes close to the poles doesn't scale
> ----------------------------------------------------------
>                 Key: LUCENE-5056
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: modules/spatial
>    Affects Versions: 4.3
>            Reporter: Hal Deadman
>            Assignee: David Smiley
>         Attachments: indexed circle close to the pole.png
> From: [~hdeadman]
> We are seeing an issue where certain shapes are causing Solr to use up all available
heap space when a record with one of those shapes is indexed. We were indexing polygons where
we had the points going clockwise instead of counter-clockwise and the shape would be so large
that we would run out of memory. We fixed those shapes but we are seeing this circle eat up
about 700MB of memory before we get an OutOfMemory error (heap space) with a 1GB JVM heap.
> Circle(3.0 90 d=0.0499542757922153)
> Google Earth can't plot that circle either, maybe it is invalid or too close to the north
pole due to the latitude of 90, but it would be nice if there was a way for shapes to be validated
before they cause an OOM error.
> The objects (4.5 million) are all GeohashPrefixTree$GhCell objects in an ArrayList owned
by PrefixTreeStrategy$CellTokenStream.
> Is there anyway to have a max number of cells in a shape before it is considered too
large and is not indexed? Is there a geo library that could validate the shape as being reasonably
sized and bounded before it is processed?
> We are currently using Solr 4.1.
> <fieldType name="location_rpt" class="solr.SpatialRecursivePrefixTreeFieldType"
> spatialContextFactory="com.spatial4j.core.context.jts.JtsSpatialContextFactory"
> geo="true" distErrPct="0.025" maxDistErr="0.000009" units="degrees" />

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message