lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Johannes Kloos (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENE-8666) NPE in o.a.l.codecs.perfield.PerFieldPostingsFormat
Date Thu, 31 Jan 2019 14:29:00 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Johannes Kloos updated LUCENE-8666:
-----------------------------------
    Environment: 
Running on Unix, using a git checkout close to master.

h2. Steps to reproduce
 * Build commit ea2c8ba of Solr as described in the section below.
 * Build the films collection as described below.
 * Start the server using the command {{“./bin/solr start -f -p 8983 -s /tmp/home”}}
 * Request the URL above.

h2. Compiling the server

{noformat}
git clone https://github.com/apache/lucene-solr
cd lucene-solr
git checkout ea2c8ba
ant compile
cd solr
ant server
{noformat}

h2. Building the collection

We followed Exercise 2 from the quick start tutorial ([http://lucene.apache.org/solr/guide/7_5/solr-tutorial.html#exercise-2])
- for reference, I have attached a copy of the database.

{noformat}
mkdir -p /tmp/home
echo '<?xml version="1.0" encoding="UTF-8" ?><solr></solr>' > /tmp/home/solr.xml
{noformat}

In one terminal start a Solr instance in foreground:

{noformat}
./bin/solr start -f -p 8983 -s /tmp/home
{noformat}

In another terminal, create a collection of movies, with no shards and no replication:

{noformat}
bin/solr create -c films
curl -X POST -H 'Content-type:application/json' --data-binary '\{"add-field": {"name":"name",
"type":"text_general", "multiValued":false, "stored":true}}' http://localhost:8983/solr/films/schema
curl -X POST -H 'Content-type:application/json' --data-binary '{"add-copy-field" : {"source":"*","dest":"_text_"}}'
[http://localhost:8983/solr/films/schema]
./bin/post -c films example/films/films.json
{noformat}

  was:
* 
Running on Unix, using a recent git checkout of master and the films example database.
h2. Steps to reproduce
 * Build commit ea2c8ba of Solr as described in the section below.
 * Build the films collection as described below.
 * Start the server using the command “./bin/solr start -f -p 8983 -s /tmp/home”
 * Request the URL above.

h2. Compiling the server

git clone [https://github.com/apache/lucene-solr]
 cd lucene-solr
 git checkout ea2c8ba
 ant compile
 cd solr
 ant server


h2. Building the collection

We followed Exercise 2 from the SOLR quick start tutorial ([http://lucene.apache.org/solr/guide/7_5/solr-tutorial.html#exercise-2]).
The attached file (home.zip) gives the contents of folder /tmp/home that you will obtain by
following the steps below.
{{}}{{mkdir -p /tmp/home}}
{{ echo '<?xml version="1.0" encoding="UTF-8" ?><solr></solr>' > /tmp/home/solr.xml}}
 

In one terminal start a Solr instance in foreground:

{{./bin/solr start -f -p 8983 -s /tmp/home}}

 

In another terminal, create a collection of movies, with no shards and no replication:
{{bin/solr create -c films}}
{{ curl -X POST -H 'Content-type:application/json' --data-binary '\{"add-field": {"name":"name",
"type":"text_general", "multiValued":false, "stored":true}}' [http://localhost:8983/solr/films/schema]}}}}
{{curl -X POST -H 'Content-type:application/json' --data-binary '{"add-copy-field" : {"source":"*","dest":"_text_"}}{{'
[http://localhost:8983/solr/films/schema]}}'}}
{{./bin/post -c films example/films/films.json}}
{{ }}


> NPE in o.a.l.codecs.perfield.PerFieldPostingsFormat 
> ----------------------------------------------------
>
>                 Key: LUCENE-8666
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8666
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/codecs
>    Affects Versions: 7.5, master (9.0)
>         Environment: Running on Unix, using a git checkout close to master.
> h2. Steps to reproduce
>  * Build commit ea2c8ba of Solr as described in the section below.
>  * Build the films collection as described below.
>  * Start the server using the command {{“./bin/solr start -f -p 8983 -s /tmp/home”}}
>  * Request the URL above.
> h2. Compiling the server
> {noformat}
> git clone https://github.com/apache/lucene-solr
> cd lucene-solr
> git checkout ea2c8ba
> ant compile
> cd solr
> ant server
> {noformat}
> h2. Building the collection
> We followed Exercise 2 from the quick start tutorial ([http://lucene.apache.org/solr/guide/7_5/solr-tutorial.html#exercise-2])
- for reference, I have attached a copy of the database.
> {noformat}
> mkdir -p /tmp/home
> echo '<?xml version="1.0" encoding="UTF-8" ?><solr></solr>' > /tmp/home/solr.xml
> {noformat}
> In one terminal start a Solr instance in foreground:
> {noformat}
> ./bin/solr start -f -p 8983 -s /tmp/home
> {noformat}
> In another terminal, create a collection of movies, with no shards and no replication:
> {noformat}
> bin/solr create -c films
> curl -X POST -H 'Content-type:application/json' --data-binary '\{"add-field": {"name":"name",
"type":"text_general", "multiValued":false, "stored":true}}' http://localhost:8983/solr/films/schema
> curl -X POST -H 'Content-type:application/json' --data-binary '{"add-copy-field" : {"source":"*","dest":"_text_"}}'
[http://localhost:8983/solr/films/schema]
> ./bin/post -c films example/films/films.json
> {noformat}
>            Reporter: Johannes Kloos
>            Priority: Minor
>              Labels: diffblue, newdev, patch-available
>         Attachments: 0001-Fix-NullPointerException.patch, home.zip
>
>
> Requesting this URL in SOLR gives a 500 error with a stack trace pointing to Lucene:
> {{http://localhost:8983/solr/films/select?q=\{!complexphrase}genre:"-om*"}}
> The stack trace is (cut down to the reasonably relevant part):
> {{java.lang.NullPointerException\n\tat java.util.TreeMap.getEntry(TreeMap.java:347)
> at java.util.TreeMap.get(TreeMap.java:278)
> at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.terms(PerFieldPostingsFormat.java:311)
> at org.apache.lucene.index.CodecReader.terms(CodecReader.java:106)
> at org.apache.lucene.index.FilterLeafReader.terms(FilterLeafReader.java:351)
> at org.apache.lucene.index.ExitableDirectoryReader$ExitableFilterAtomicReader.terms(ExitableDirectoryReader.java:91)
> at org.apache.lucene.search.spans.SpanNearQuery$SpanNearWeight.getSpans(SpanNearQuery.java:208)
> at org.apache.lucene.search.spans.SpanNotQuery$SpanNotWeight.getSpans(SpanNotQuery.java:127)
> at org.apache.lucene.search.spans.SpanWeight.scorer(SpanWeight.java:135)
> at org.apache.lucene.search.spans.SpanWeight.scorer(SpanWeight.java:46)
> at org.apache.lucene.search.Weight.bulkScorer(Weight.java:177)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:649)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:443)
> at org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:200)
> at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1604)}}{{The
error is actually a bit deeper and can be traced back to the o.a.l.queryparser.complexPhrase.ComplexPhraseQueryParser
class.}}
> Handling this query involves constructing a SpanQuery, which happens in the rewrite method
of ComplexPhraseQueryParser. In particular, the expression is decomposed into a BooleanQuery,
which has exactly one clause, namely the negative clause -genre:”om*”. The rewrite method
then further transforms this into a SpanQuery; in this case, it goes into the path that handles
complex queries with both positive and negative clauses. It extracts the subset of positive
clauses - note that this set of clauses is empty for this query. The positive clauses are
then combined into a SpanNearQuery (around line 340), which is then used to build a SpanNotQuery.
Further down the line, the field attribute of the SpanNearQuery is accessed and used as an
index into a TreeMap. But since we had an empty set of positive clauses, the SpanNearQuery
does not have its field attribute set, so we get a null here - this leads to an exception.
A possible fix would be to detect the situation where we have an empty set of positive clauses
and include a single synthetic clause that matches either everything or nothing. See attached
file 0001-Fix-NullPointerException.patch.
> This bug was found using [Diffblue Microservices Testing|http://www.diffblue.com/labs].
Find more information on this [test campaign|https://www.diffblue.com/blog/2018/12/19/diffblue-microservice-testing-a-sneak-peek-at-our-early-product-and-results].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message