lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Intermittent error 401 with JSON Facet query to retrieve count all collections
Date Wed, 01 May 2019 02:03:14 GMT
Patches welcome!

> On Apr 30, 2019, at 8:17 PM, Zheng Lin Edwin Yeo <edwinyeozl@gmail.com> wrote:
> 
> Hi,
> 
> Any expected timeline on when we can solve this bug?
> 
> Regards,
> Edwin
> 
> On Fri, 26 Apr 2019 at 00:21, Zheng Lin Edwin Yeo <edwinyeozl@gmail.com>
> wrote:
> 
>> Thanks Jan.
>> 
>> I have created a bug in JIRA under
>> https://issues.apache.org/jira/browse/SOLR-13421
>> 
>> Regards,
>> Edwin
>> 
>> On Thu, 25 Apr 2019 at 19:53, Jan Høydahl <jan.asf@cominvent.com> wrote:
>> 
>>> Please create a bug for this in JIRA. I think that HttpShardHandler needs
>>> to handle shards from shards param the same as the auto-computed shards it
>>> gets from ZK, but have not looked into it in detail yet.
>>> 
>>> --
>>> Jan Høydahl, search solution architect
>>> Cominvent AS - www.cominvent.com
>>> 
>>>> 25. apr. 2019 kl. 09:15 skrev Zheng Lin Edwin Yeo <edwinyeozl@gmail.com
>>>> :
>>>> 
>>>> Regarding the issue, we have found that if we put only one collections
>>> in
>>>> the shards (can be any collection), there will not be error, and it can
>>>> always produce the correct output. If we put 2 collections, there will
>>> not
>>>> be error 90% of the time (only 10% of the time the issue will occur).
>>>> 
>>>> However, once we put 3 or more collections (can be any of the
>>> collections),
>>>> this issue will keep occurring.
>>>> 
>>>> This seems more like a bug, since the error mostly occurs when we put 3
>>> or
>>>> more collections, but it does not give error when we put only one or
>>> two of
>>>> the three collections, and we do not face this issue in the earlier
>>> version
>>>> of Solr.
>>>> 
>>>> Appreciate if anyone can advice on this weird scenario?
>>>> 
>>>> Thank you.
>>>> 
>>>> Regards,
>>>> Edwin
>>>> 
>>>> 
>>>> 
>>>> On Wed, 24 Apr 2019 at 15:20, Zheng Lin Edwin Yeo <edwinyeozl@gmail.com
>>>> 
>>>> wrote:
>>>> 
>>>>> Hi,
>>>>> 
>>>>> 
>>>>> I am using the below JSON Facet to retrieve the count of all the
>>> different
>>>>> collections in one query.
>>>>> 
>>>>> 
>>>>> 
>>> https://localhost:8983/solr/collection1/select?q=testing&shards=https://localhost:8983/solr/collection1,https://localhost:8983/solr/collection2,https://localhost:8983/solr/collection3,https://localhost:8983/solr/collection4,https://localhost:8983/solr/collection5,https://localhost:8983/solr/collection6&rows=0&json.facet={categories
>>> :
>>>>> {type : terms,field : content_type,limit : 100}}
>>>>> 
>>>>> 
>>>>> Previously, in Solr 7.6 and Solr 7.7, this query can work correctly
>>> and we
>>>>> are able to produce the correct output.
>>>>> 
>>>>> {
>>>>> "responseHeader":
>>>>> {     "zkConnected":true,     "status":0,     "QTime":24}
>>>>> 
>>>>> ,
>>>>> "response":
>>>>> {"numFound":41200,"start":0,"maxScore":12.993215,"docs":[]   }
>>>>> 
>>>>> ,
>>>>> "facets":{
>>>>>   "count":41200,
>>>>>   "categories":{
>>>>>     "buckets":[
>>>>> {           "val":"collection1",           "count":26213}
>>>>> 
>>>>> ,
>>>>> 
>>>>> {           "val":"collection2",           "count":12075}
>>>>> 
>>>>> ,
>>>>> 
>>>>> {           "val":"collection3",           "count":1947}
>>>>> 
>>>>> ,
>>>>> 
>>>>> {           "val":"collection4",           "count":850}
>>>>> 
>>>>> ,
>>>>> 
>>>>> {           "val":"collection5",           "count":111}
>>>>> 
>>>>> ,
>>>>> 
>>>>> {           "val":"collection6",           "count":4}
>>>>> 
>>>>> ]}}}
>>>>> 
>>>>> 
>>>>> However, in the new Solr 8.0.0, this query can only work occasionally.
>>>>> Most of the time, we will get the following error of 'Error 401 require
>>>>> authentication':
>>>>> 
>>>>> {
>>>>> "responseHeader":
>>>>> {     "zkConnected":true,     "status":401,     "QTime":11}
>>>>> 
>>>>> ,
>>>>> "error":{
>>>>>   "metadata":[
>>>>> 
>>>>> 
>>> "error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException",
>>>>> 
>>>>> 
>>> "root-error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException"],
>>>>>   "msg":"Error from server at null: Expected mime type
>>>>> application/octet-stream but got text/html. <html>\n<head>\n<meta
>>>>> http-equiv=\"Content-Type\"
>>>>> content=\"text/html;charset=utf-8\"/>\n<title>Error 401 require
>>>>> authentication</title>\n</head>\n<body><h2>HTTP
ERROR
>>> 401</h2>\n<p>Problem
>>>>> accessing /solr/collection6/select. Reason:\n<pre>    require
>>>>> authentication</pre></p>\n</body>\n</html>\n",
>>>>>   "code":401}}
>>>>> 
>>>>> This issue does not occur in Solr 7.6 and Solr 7.7, even though I have
>>> set
>>>>> up the same authentication for all the versions.
>>>>> 
>>>>> What could be the issue that causes this?
>>>>> 
>>>>> 
>>>>> Below is the format of my security.json:
>>>>> 
>>>>> {
>>>>> "authentication":
>>>>> 
>>>>> {    "blockUnknown": true,    "class":"solr.BasicAuthPlugin",
>>>>> "credentials":
>>>>> {"user1":"hyHXXuJSqcZdNgdSTGUvrQZRpqrYFUQ2ffmlWQ4GUTk=
>>>>> E0w3/2FD+rlxulbPm2G7i9HZqT+2gMBzcyJCcGcMWwA="}
>>>>> 
>>>>> },
>>>>> "authorization":
>>>>> 
>>>>> {    "class":"solr.RuleBasedAuthorizationPlugin",    "user-role":
>>>>> {"user1":"admin"}
>>>>> 
>>>>> ,
>>>>>  "permissions":[
>>>>> {"name":"security-edit",                   "role":"admin"}
>>>>> 
>>>>> ]
>>>>> }}
>>>>> 
>>>>> 
>>>>> Regards,
>>>>> Edwin
>>>>> 
>>>>> 
>>>>> On Mon, 22 Apr 2019 at 09:37, Zheng Lin Edwin Yeo <
>>> edwinyeozl@gmail.com>
>>>>> wrote:
>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> Anyone has experienced this or have any insights of this?
>>>>>> 
>>>>>> Regards,
>>>>>> Edwin
>>>>>> 
>>>>>> On Thu, 18 Apr 2019 at 18:04, Zheng Lin Edwin Yeo <
>>> edwinyeozl@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> Is there possibility that this could be a bug in the new Solr
8.0.0?
>>>>>>> 
>>>>>>> Since I do not face the issue in the earlier version, and I have
not
>>>>>>> changed any configuration in this new version. My data in Solr
8.0.0
>>> is
>>>>>>> freshly re-index directly in Solr 8.0.0, not upgraded from earlier
>>> version.
>>>>>>> 
>>>>>>> Regards,
>>>>>>> Edwin
>>>>>>> 
>>>>>>> On Thu, 18 Apr 2019 at 10:10, Zheng Lin Edwin Yeo <
>>> edwinyeozl@gmail.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Hi Jason,
>>>>>>>> 
>>>>>>>> The same problem still persist after restarting my Solr nodes.
The
>>> only
>>>>>>>> time the problem didn't occur is when I disabled the basic
>>> authentication.
>>>>>>>> 
>>>>>>>> I have tried with a few "/select?q=*:*", and they do not
exhibit the
>>>>>>>> same problem. Even the similar query with only 1 shard does
not
>>> have the
>>>>>>>> problem.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>> https://localhost:8983/solr/collection1/select?q=testing&shards=https://localhost:8983/solr/collection1&rows=0&json.facet={categories
>>>>>>>> : {type : terms,field : content_type,limit : 100}}
>>>>>>>> 
>>>>>>>> 
>>>>>>>> It is only when there are 2 or more shards, that the problem
occur.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>> https://localhost:8983/solr/collection1/select?q=testing&shards=https://localhost:8983/solr/collection1,https://localhost:8983/solr/collection2&rows=0&json.facet={categories
>>>>>>>> : {type : terms,field : content_type,limit : 100}}
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Regards,
>>>>>>>> Edwin
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Thu, 18 Apr 2019 at 01:15, Jason Gerlowski <
>>> gerlowskija@gmail.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Agreed, I'd be surprised if this behavior was specific
to JSON
>>>>>>>>> Faceting.  Though I'm surprised it's happening at all,
so...
>>>>>>>>> 
>>>>>>>>> Anyway, that's easy for you to test though.  Try a few
>>> "/select?q=*:*"
>>>>>>>>> queries and see whether they also exhibits this behavior.
 One
>>> other
>>>>>>>>> question: does the behavior persist after restarting
your Solr
>>> nodes?
>>>>>>>>> 
>>>>>>>>> Good luck,
>>>>>>>>> 
>>>>>>>>> Jason
>>>>>>>>> 
>>>>>>>>> On Wed, Apr 17, 2019 at 4:05 AM Zheng Lin Edwin Yeo
>>>>>>>>> <edwinyeozl@gmail.com> wrote:
>>>>>>>>>> 
>>>>>>>>>> Hi,
>>>>>>>>>> 
>>>>>>>>>> For your info, I have enabled basic authentication
and SSL in all
>>>>>>>>> the 3
>>>>>>>>>> versions, and I'm not sure if the issue is more on
the
>>>>>>>>> authentication side
>>>>>>>>>> instead of the JSON Facet query?
>>>>>>>>>> 
>>>>>>>>>> Regards,
>>>>>>>>>> Edwin
>>>>>>>>>> 
>>>>>>>>>> On Wed, 17 Apr 2019 at 06:54, Zheng Lin Edwin Yeo
<
>>>>>>>>> edwinyeozl@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Hi Jason,
>>>>>>>>>>> 
>>>>>>>>>>> Yes, that is correct.
>>>>>>>>>>> 
>>>>>>>>>>> Below is the format of my security.json. I have
changed the
>>> masked
>>>>>>>>>>> password for security purposes.
>>>>>>>>>>> 
>>>>>>>>>>> {
>>>>>>>>>>> "authentication":{
>>>>>>>>>>>  "blockUnknown": true,
>>>>>>>>>>>  "class":"solr.BasicAuthPlugin",
>>>>>>>>>>> 
>>>>>>>>> 
>>> "credentials":{"user1":"hyHXXuJSqcZdNgdSTGUvrQZRpqrYFUQ2ffmlWQ4GUTk=
>>>>>>>>>>> E0w3/2FD+rlxulbPm2G7i9HZqT+2gMBzcyJCcGcMWwA="}
>>>>>>>>>>> },
>>>>>>>>>>> "authorization":{
>>>>>>>>>>>  "class":"solr.RuleBasedAuthorizationPlugin",
>>>>>>>>>>>  "user-role":{"user1":"admin"},
>>>>>>>>>>>  "permissions":[{"name":"security-edit",
>>>>>>>>>>>                 "role":"admin"}]
>>>>>>>>>>> }}
>>>>>>>>>>> 
>>>>>>>>>>> Regards,
>>>>>>>>>>> Edwin
>>>>>>>>>>> 
>>>>>>>>>>> On Tue, 16 Apr 2019 at 23:12, Jason Gerlowski
<
>>>>>>>>> gerlowskija@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Hi Edwin,
>>>>>>>>>>>> 
>>>>>>>>>>>> To clarify what you're running into:
>>>>>>>>>>>> 
>>>>>>>>>>>> - on 7.6, this query works all the time
>>>>>>>>>>>> - on 7.7 this query works all the time
>>>>>>>>>>>> - on 8.0, this query works the first time
you run it, but
>>>>>>>>> subsequent
>>>>>>>>>>>> runs return a 401 error?
>>>>>>>>>>>> 
>>>>>>>>>>>> Is that correct?  It might be helpful for
others if you could
>>>>>>>>> share
>>>>>>>>>>>> your security.json.
>>>>>>>>>>>> 
>>>>>>>>>>>> Best,
>>>>>>>>>>>> 
>>>>>>>>>>>> Jason
>>>>>>>>>>>> 
>>>>>>>>>>>> On Mon, Apr 15, 2019 at 10:40 PM Zheng Lin
Edwin Yeo
>>>>>>>>>>>> <edwinyeozl@gmail.com> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I am using the below JSON Facet to retrieve
the count of all
>>> the
>>>>>>>>>>>> different
>>>>>>>>>>>>> collections in one query.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>> https://localhost:8983/solr/collection1/select?q=testing&shards=https://localhost:8983/solr/collection1,https://localhost:8983/solr/collection2,https://localhost:8983/solr/collection3,https://localhost:8983/solr/collection4,https://localhost:8983/solr/collection5,https://localhost:8983/solr/collection6&rows=0&json.facet={categories
>>>>>>>>>>>>> : {type : terms,field : content_type,limit
: 100}}
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Previously, in Solr 7.6 and Solr 7.7,
this query can work
>>>>>>>>> correctly and
>>>>>>>>>>>> we
>>>>>>>>>>>>> are able to produce the correct output.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> {
>>>>>>>>>>>>> "responseHeader":{
>>>>>>>>>>>>>   "zkConnected":true,
>>>>>>>>>>>>>   "status":0,
>>>>>>>>>>>>>   "QTime":24},
>>>>>>>>>>>>> 
>>>>>>>>> 
>>> "response":{"numFound":41200,"start":0,"maxScore":12.993215,"docs":[]
>>>>>>>>>>>>> },
>>>>>>>>>>>>> "facets":{
>>>>>>>>>>>>>   "count":41200,
>>>>>>>>>>>>>   "categories":{
>>>>>>>>>>>>>     "buckets":[{
>>>>>>>>>>>>>         "val":"collection1",
>>>>>>>>>>>>>         "count":26213},
>>>>>>>>>>>>>       {
>>>>>>>>>>>>>         "val":"collection2",
>>>>>>>>>>>>>         "count":12075},
>>>>>>>>>>>>>       {
>>>>>>>>>>>>>         "val":"collection3",
>>>>>>>>>>>>>         "count":1947},
>>>>>>>>>>>>>       {
>>>>>>>>>>>>>         "val":"collection4",
>>>>>>>>>>>>>         "count":850},
>>>>>>>>>>>>>       {
>>>>>>>>>>>>>         "val":"collection5",
>>>>>>>>>>>>>         "count":111},
>>>>>>>>>>>>>       {
>>>>>>>>>>>>>         "val":"collection6",
>>>>>>>>>>>>>         "count":4}]}}}
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> However, in the new Solr 8.0.0, this
query can only work once.
>>>>>>>>>>>>> Subsequently, we will get the following
error of 'require
>>>>>>>>>>>> authentication':
>>>>>>>>>>>>> 
>>>>>>>>>>>>> {
>>>>>>>>>>>>> "responseHeader":{
>>>>>>>>>>>>>   "zkConnected":true,
>>>>>>>>>>>>>   "status":401,
>>>>>>>>>>>>>   "QTime":11},
>>>>>>>>>>>>> "error":{
>>>>>>>>>>>>>   "metadata":[
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>> "error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException",
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>> "root-error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException"],
>>>>>>>>>>>>>   "msg":"Error from server at null: Expected
mime type
>>>>>>>>>>>>> application/octet-stream but got text/html.
>>>>>>>>> <html>\n<head>\n<meta
>>>>>>>>>>>>> http-equiv=\"Content-Type\"
>>>>>>>>>>>>> content=\"text/html;charset=utf-8\"/>\n<title>Error
401 require
>>>>>>>>>>>>> authentication</title>\n</head>\n<body><h2>HTTP
ERROR
>>>>>>>>>>>> 401</h2>\n<p>Problem
>>>>>>>>>>>>> accessing /solr/collection6/select. Reason:\n<pre>
   require
>>>>>>>>>>>>> authentication</pre></p>\n</body>\n</html>\n",
>>>>>>>>>>>>>   "code":401}}
>>>>>>>>>>>>> 
>>>>>>>>>>>>> This issue does not occur in Solr 7.6
and Solr 7.7, even though
>>>>>>>>> I have
>>>>>>>>>>>> set
>>>>>>>>>>>>> up the same authentication for all the
versions.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> What could be the issue that causes this?
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Edwin
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>> 
>>> 


Mime
View raw message