lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicholas Knize (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (LUCENE-7179) GeoPoint and LatLonPoint test data should quantize once
Date Tue, 05 Apr 2016 15:51:25 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-7179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15226503#comment-15226503
] 

Nicholas Knize edited comment on LUCENE-7179 at 4/5/16 3:51 PM:
----------------------------------------------------------------

bq. I think if we are changing it to use 64-bits, then we should also fix the current rounding
and overflow issues too?

In the Lucene-7165 patch the encoding tolerance is changed to be consistent 1e-7. Indeed the
instability occurs > 1e-7. You can see the issue present itself in {{scaleLon}} when the
32bit quantization is "truncated" to a long. Overflow is not a problem because the positive
values are mapped using {{(0x1L << 32) - 1}} so 360.0 correctly maps to 0xFFFFFFFF /
4294967295. Not sure what you mean by "fix the rounding issue"?


was (Author: nknize):
bq. I think if we are changing it to use 64-bits, then we should also fix the current rounding
and overflow issues too?

In the Lucene-7165 patch the encoding tolerance is changed to be consistent 1e-7. Indeed the
instability occurs > 1e-7. You can see the issue present itself in {{scaleLon}} when the
32bit quantization is "truncated" to a long. Overflow is not a problem because the positive
values are mapped using {{0x1L << 32) - 1}} so 360.0 correctly maps to 0xFFFFFFFF /
4294967295. Not sure what you mean by "fix the rounding issue"?

> GeoPoint and LatLonPoint test data should quantize once
> -------------------------------------------------------
>
>                 Key: LUCENE-7179
>                 URL: https://issues.apache.org/jira/browse/LUCENE-7179
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Nicholas Knize
>
> {{LatLonPoint}} and {{GeoPointField}} tests pre quantizes test data to ensure consistency
with indexed (encoded) data. The pre quantized data then becomes indexed, undergoing another
quantization. To guarantee numerical stability this should be changed such that the test data
is quantized after indexing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message