lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Høydahl <jan....@cominvent.com>
Subject Re: Inconsistent results with solr admin ui and solrj
Date Tue, 16 Aug 2016 14:27:44 GMT
I’m not sure of the root cause for your problem.
Solr is built to stay in sync automatically, so there is no need to script anything in that
regard.
There may be something with your environement, network, ZooKeeper setup or similar that caused
the state you were in. I would need to dig further into the system to diagnose, such as looking
at state.json files, live_nodes znode etc. But really, this should JustWork™ :)

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

> 16. aug. 2016 kl. 15.58 skrev Pranaya Behera <pranaya.behera@igp.com>:
> 
> Hi,
>     I did as you said, now it is coming ok.
> And what are the things to look for while checking about these kind of issues, such as
mismatch count, lukerequest not returning all the fields etc. The doc sync is one, how can
I programmatically use the info and sync them ? Is there any method in solrj?
> 
> On 16/08/16 14:50, Jan Høydahl wrote:
>> Hi,
>> 
>> There is clearly something wrong when your two replicas are not in sync. Could you
go to the “Cloud->Tree” tab of admin UI and look in the overseer queue whether you
find signs of stuck jobs or something?
>> Btw - what warnings do you see in the logs? Anything repeatedly popping up?
>> 
>> I would also try the following:
>> 1. Take down the node hosting replica 1 (assuming that replica2 is the correct, most
current)
>> 2. Manually empty the data folder
>> 3. Take the node up again
>> 4. Verify that a full index recovery happens, and that they get back in sync
>> 5. Run your indexing procedure.
>> 6. Verify that both replicas are still in sync
>> 
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>> 
>>> 16. aug. 2016 kl. 06.51 skrev Pranaya Behera <pranaya.behera@igp.com>:
>>> 
>>> Hi,
>>> a.) Yes index is static, not updated live. We index new documents over old documents
by this sequesce, deleteall docs, add 10 freshly fetched from db, after adding all the docs
to cloud instance, commit. Commit happens only once per collection,
>>> b.) I took one shard and below are the results for the each replica, it has 2
replica.
>>> Replica - 2
>>> Last Modified: 33 minutes ago
>>> Num Docs: 127970
>>> Max Doc: 127970
>>> Heap Memory Usage: -1
>>> Deleted Docs: 0
>>> Version: 14530
>>> Segment Count: 5
>>> Optimized: yes
>>> Current: yes
>>> Data:  /var/solr/data/product_shard1_replica2/data
>>> Index: /var/solr/data/product_shard1_replica2/data/index.20160816040537452
>>> Impl:  org.apache.solr.core.NRTCachingDirectoryFactory
>>> 
>>> Replica - 1
>>> Last Modified: about 19 hours ago
>>> Num Docs: 234013
>>> Max Doc: 234013
>>> Heap Memory Usage: -1
>>> Deleted Docs: 0
>>> Version: 14272
>>> Segment Count: 7
>>> Optimized: yes
>>> Current: no
>>> Data:  /var/solr/data/product_shard1_replica1/data
>>> Index: /var/solr/data/product_shard1_replica1/data/index
>>> Impl:  org.apache.solr.core.NRTCachingDirectoryFactory
>>> 
>>> c.) With the admin ui: if I query for all, *:* it gives different numFound each
time.
>>> e.g.
>>> 1.
>>> 
>>> |{ "responseHeader":{ "zkConnected":true, "status":0, "QTime":7, "params":{ "q":"*:*",
"indent":"on", "wt":"json", "_":"1471322871767"}}, "response":{"numFound":452300,"start":0,"maxScore":1.0,
2. |
>>> |{ "responseHeader":{ "zkConnected":true, "status":0, "QTime":23, "params":{
"q":"*:*", "indent":"on", "wt":"json", "_":"1471322871767"}}, "response":{"numFound":574013,"start":0,"maxScore":1.0,
This is queried live from the solr instances. |
>>> 
>>> It happens with any type of queries, if I search in parent document or search
through child documents to get parents. sorting is used in both cases but with different field,
while doingblock join query sortingis on the child document field, otherwise on the parent
document field.
>>> 
>>> d.) I dont find any errors in the logs. All warnings only.
>>> 
>>> On 14/08/16 02:56, Jan Høydahl wrote:
>>>> Could it be that your cluster is not in sync, so that when Solr picks three
nodes, results will vary depending on what replica answers?
>>>> 
>>>> A few questions:
>>>> 
>>>> a) Is your index static, i.e. not being updated live?
>>>> b) Can you try to go directly to the core menu of both replicas for each
shard, and compare numDocs / maxDocs for each? Both replicas in each shard should have same
count.
>>>> c) What are you querying on and sorting by? Does it happen with only one
query and sorting?
>>>> d) Are there any errors in the logs?
>>>> 
>>>> If possible, please share some queries, responses, config, screenshots etc.
>>>> 
>>>> --
>>>> Jan Høydahl, search solution architect
>>>> Cominvent AS - www.cominvent.com
>>>> 
>>>>> 13. aug. 2016 kl. 12.10 skrev Pranaya Behera <pranaya.behera@igp.com>:
>>>>> 
>>>>> Hi,
>>>>>    I am running solr 6.1.0 with solrcloud. We have 3 instance of zookeeper
and 3 instance of solrcloud. All three of them are active and up. One collection has 3 shards,
each shard has 2 replicas.
>>>>> 
>>>>> Everytime query whether from solrj or admin ui, getting inconsistent
results. e.g.
>>>>> 1. numFound is always fluctuating.
>>>>> 2. facet count shows the count for a field, filter query on that field
gets 0 results.
>>>>> 3. luke requests work(not sure whether gives correct info of all the
dynamic field) on per shard not on collection when invoked from curl but doesnt work when
called from solrj.
>>>>> 4. admin ui shows expanded results, same query goes from solrj, getExpandedResults()
gives 0 docs.
>>>>> 
>>>>> What would be cause of all this ? Any pointer to look for an error anything
in the logs.
>> 
> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message