lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zheng Lin Edwin Yeo <edwinye...@gmail.com>
Subject Re: Intermittent error 401 with JSON Facet query to retrieve count all collections
Date Wed, 01 May 2019 00:17:17 GMT
Hi,

Any expected timeline on when we can solve this bug?

Regards,
Edwin

On Fri, 26 Apr 2019 at 00:21, Zheng Lin Edwin Yeo <edwinyeozl@gmail.com>
wrote:

> Thanks Jan.
>
> I have created a bug in JIRA under
> https://issues.apache.org/jira/browse/SOLR-13421
>
> Regards,
> Edwin
>
> On Thu, 25 Apr 2019 at 19:53, Jan Høydahl <jan.asf@cominvent.com> wrote:
>
>> Please create a bug for this in JIRA. I think that HttpShardHandler needs
>> to handle shards from shards param the same as the auto-computed shards it
>> gets from ZK, but have not looked into it in detail yet.
>>
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>>
>> > 25. apr. 2019 kl. 09:15 skrev Zheng Lin Edwin Yeo <edwinyeozl@gmail.com
>> >:
>> >
>> > Regarding the issue, we have found that if we put only one collections
>> in
>> > the shards (can be any collection), there will not be error, and it can
>> > always produce the correct output. If we put 2 collections, there will
>> not
>> > be error 90% of the time (only 10% of the time the issue will occur).
>> >
>> > However, once we put 3 or more collections (can be any of the
>> collections),
>> > this issue will keep occurring.
>> >
>> > This seems more like a bug, since the error mostly occurs when we put 3
>> or
>> > more collections, but it does not give error when we put only one or
>> two of
>> > the three collections, and we do not face this issue in the earlier
>> version
>> > of Solr.
>> >
>> > Appreciate if anyone can advice on this weird scenario?
>> >
>> > Thank you.
>> >
>> > Regards,
>> > Edwin
>> >
>> >
>> >
>> > On Wed, 24 Apr 2019 at 15:20, Zheng Lin Edwin Yeo <edwinyeozl@gmail.com
>> >
>> > wrote:
>> >
>> >> Hi,
>> >>
>> >>
>> >> I am using the below JSON Facet to retrieve the count of all the
>> different
>> >> collections in one query.
>> >>
>> >>
>> >>
>> https://localhost:8983/solr/collection1/select?q=testing&shards=https://localhost:8983/solr/collection1,https://localhost:8983/solr/collection2,https://localhost:8983/solr/collection3,https://localhost:8983/solr/collection4,https://localhost:8983/solr/collection5,https://localhost:8983/solr/collection6&rows=0&json.facet={categories
>> :
>> >> {type : terms,field : content_type,limit : 100}}
>> >>
>> >>
>> >> Previously, in Solr 7.6 and Solr 7.7, this query can work correctly
>> and we
>> >> are able to produce the correct output.
>> >>
>> >> {
>> >>  "responseHeader":
>> >> {     "zkConnected":true,     "status":0,     "QTime":24}
>> >>
>> >> ,
>> >>  "response":
>> >> {"numFound":41200,"start":0,"maxScore":12.993215,"docs":[]   }
>> >>
>> >> ,
>> >>  "facets":{
>> >>    "count":41200,
>> >>    "categories":{
>> >>      "buckets":[
>> >> {           "val":"collection1",           "count":26213}
>> >>
>> >> ,
>> >>
>> >> {           "val":"collection2",           "count":12075}
>> >>
>> >> ,
>> >>
>> >> {           "val":"collection3",           "count":1947}
>> >>
>> >> ,
>> >>
>> >> {           "val":"collection4",           "count":850}
>> >>
>> >> ,
>> >>
>> >> {           "val":"collection5",           "count":111}
>> >>
>> >> ,
>> >>
>> >> {           "val":"collection6",           "count":4}
>> >>
>> >> ]}}}
>> >>
>> >>
>> >> However, in the new Solr 8.0.0, this query can only work occasionally.
>> >> Most of the time, we will get the following error of 'Error 401 require
>> >> authentication':
>> >>
>> >> {
>> >>  "responseHeader":
>> >> {     "zkConnected":true,     "status":401,     "QTime":11}
>> >>
>> >> ,
>> >>  "error":{
>> >>    "metadata":[
>> >>
>> >>
>> "error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException",
>> >>
>> >>
>> "root-error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException"],
>> >>    "msg":"Error from server at null: Expected mime type
>> >> application/octet-stream but got text/html. <html>\n<head>\n<meta
>> >> http-equiv=\"Content-Type\"
>> >> content=\"text/html;charset=utf-8\"/>\n<title>Error 401 require
>> >> authentication</title>\n</head>\n<body><h2>HTTP
ERROR
>> 401</h2>\n<p>Problem
>> >> accessing /solr/collection6/select. Reason:\n<pre>    require
>> >> authentication</pre></p>\n</body>\n</html>\n",
>> >>    "code":401}}
>> >>
>> >> This issue does not occur in Solr 7.6 and Solr 7.7, even though I have
>> set
>> >> up the same authentication for all the versions.
>> >>
>> >> What could be the issue that causes this?
>> >>
>> >>
>> >> Below is the format of my security.json:
>> >>
>> >> {
>> >> "authentication":
>> >>
>> >> {    "blockUnknown": true,    "class":"solr.BasicAuthPlugin",
>> >> "credentials":
>> >> {"user1":"hyHXXuJSqcZdNgdSTGUvrQZRpqrYFUQ2ffmlWQ4GUTk=
>> >> E0w3/2FD+rlxulbPm2G7i9HZqT+2gMBzcyJCcGcMWwA="}
>> >>
>> >> },
>> >> "authorization":
>> >>
>> >> {    "class":"solr.RuleBasedAuthorizationPlugin",    "user-role":
>> >> {"user1":"admin"}
>> >>
>> >> ,
>> >>   "permissions":[
>> >> {"name":"security-edit",                   "role":"admin"}
>> >>
>> >> ]
>> >> }}
>> >>
>> >>
>> >> Regards,
>> >> Edwin
>> >>
>> >>
>> >> On Mon, 22 Apr 2019 at 09:37, Zheng Lin Edwin Yeo <
>> edwinyeozl@gmail.com>
>> >> wrote:
>> >>
>> >>> Hi,
>> >>>
>> >>> Anyone has experienced this or have any insights of this?
>> >>>
>> >>> Regards,
>> >>> Edwin
>> >>>
>> >>> On Thu, 18 Apr 2019 at 18:04, Zheng Lin Edwin Yeo <
>> edwinyeozl@gmail.com>
>> >>> wrote:
>> >>>
>> >>>> Is there possibility that this could be a bug in the new Solr 8.0.0?
>> >>>>
>> >>>> Since I do not face the issue in the earlier version, and I have
not
>> >>>> changed any configuration in this new version. My data in Solr 8.0.0
>> is
>> >>>> freshly re-index directly in Solr 8.0.0, not upgraded from earlier
>> version.
>> >>>>
>> >>>> Regards,
>> >>>> Edwin
>> >>>>
>> >>>> On Thu, 18 Apr 2019 at 10:10, Zheng Lin Edwin Yeo <
>> edwinyeozl@gmail.com>
>> >>>> wrote:
>> >>>>
>> >>>>> Hi Jason,
>> >>>>>
>> >>>>> The same problem still persist after restarting my Solr nodes.
The
>> only
>> >>>>> time the problem didn't occur is when I disabled the basic
>> authentication.
>> >>>>>
>> >>>>> I have tried with a few "/select?q=*:*", and they do not exhibit
the
>> >>>>> same problem. Even the similar query with only 1 shard does
not
>> have the
>> >>>>> problem.
>> >>>>>
>> >>>>>
>> >>>>>
>> https://localhost:8983/solr/collection1/select?q=testing&shards=https://localhost:8983/solr/collection1&rows=0&json.facet={categories
>> >>>>> : {type : terms,field : content_type,limit : 100}}
>> >>>>>
>> >>>>>
>> >>>>> It is only when there are 2 or more shards, that the problem
occur.
>> >>>>>
>> >>>>>
>> >>>>>
>> https://localhost:8983/solr/collection1/select?q=testing&shards=https://localhost:8983/solr/collection1,https://localhost:8983/solr/collection2&rows=0&json.facet={categories
>> >>>>> : {type : terms,field : content_type,limit : 100}}
>> >>>>>
>> >>>>>
>> >>>>> Regards,
>> >>>>> Edwin
>> >>>>>
>> >>>>>
>> >>>>> On Thu, 18 Apr 2019 at 01:15, Jason Gerlowski <
>> gerlowskija@gmail.com>
>> >>>>> wrote:
>> >>>>>
>> >>>>>> Agreed, I'd be surprised if this behavior was specific to
JSON
>> >>>>>> Faceting.  Though I'm surprised it's happening at all, so...
>> >>>>>>
>> >>>>>> Anyway, that's easy for you to test though.  Try a few
>> "/select?q=*:*"
>> >>>>>> queries and see whether they also exhibits this behavior.
 One
>> other
>> >>>>>> question: does the behavior persist after restarting your
Solr
>> nodes?
>> >>>>>>
>> >>>>>> Good luck,
>> >>>>>>
>> >>>>>> Jason
>> >>>>>>
>> >>>>>> On Wed, Apr 17, 2019 at 4:05 AM Zheng Lin Edwin Yeo
>> >>>>>> <edwinyeozl@gmail.com> wrote:
>> >>>>>>>
>> >>>>>>> Hi,
>> >>>>>>>
>> >>>>>>> For your info, I have enabled basic authentication and
SSL in all
>> >>>>>> the 3
>> >>>>>>> versions, and I'm not sure if the issue is more on the
>> >>>>>> authentication side
>> >>>>>>> instead of the JSON Facet query?
>> >>>>>>>
>> >>>>>>> Regards,
>> >>>>>>> Edwin
>> >>>>>>>
>> >>>>>>> On Wed, 17 Apr 2019 at 06:54, Zheng Lin Edwin Yeo <
>> >>>>>> edwinyeozl@gmail.com>
>> >>>>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Hi Jason,
>> >>>>>>>>
>> >>>>>>>> Yes, that is correct.
>> >>>>>>>>
>> >>>>>>>> Below is the format of my security.json. I have
changed the
>> masked
>> >>>>>>>> password for security purposes.
>> >>>>>>>>
>> >>>>>>>> {
>> >>>>>>>> "authentication":{
>> >>>>>>>>   "blockUnknown": true,
>> >>>>>>>>   "class":"solr.BasicAuthPlugin",
>> >>>>>>>>
>> >>>>>>
>> "credentials":{"user1":"hyHXXuJSqcZdNgdSTGUvrQZRpqrYFUQ2ffmlWQ4GUTk=
>> >>>>>>>> E0w3/2FD+rlxulbPm2G7i9HZqT+2gMBzcyJCcGcMWwA="}
>> >>>>>>>> },
>> >>>>>>>> "authorization":{
>> >>>>>>>>   "class":"solr.RuleBasedAuthorizationPlugin",
>> >>>>>>>>   "user-role":{"user1":"admin"},
>> >>>>>>>>   "permissions":[{"name":"security-edit",
>> >>>>>>>>                  "role":"admin"}]
>> >>>>>>>> }}
>> >>>>>>>>
>> >>>>>>>> Regards,
>> >>>>>>>> Edwin
>> >>>>>>>>
>> >>>>>>>> On Tue, 16 Apr 2019 at 23:12, Jason Gerlowski <
>> >>>>>> gerlowskija@gmail.com>
>> >>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Hi Edwin,
>> >>>>>>>>>
>> >>>>>>>>> To clarify what you're running into:
>> >>>>>>>>>
>> >>>>>>>>> - on 7.6, this query works all the time
>> >>>>>>>>> - on 7.7 this query works all the time
>> >>>>>>>>> - on 8.0, this query works the first time you
run it, but
>> >>>>>> subsequent
>> >>>>>>>>> runs return a 401 error?
>> >>>>>>>>>
>> >>>>>>>>> Is that correct?  It might be helpful for others
if you could
>> >>>>>> share
>> >>>>>>>>> your security.json.
>> >>>>>>>>>
>> >>>>>>>>> Best,
>> >>>>>>>>>
>> >>>>>>>>> Jason
>> >>>>>>>>>
>> >>>>>>>>> On Mon, Apr 15, 2019 at 10:40 PM Zheng Lin Edwin
Yeo
>> >>>>>>>>> <edwinyeozl@gmail.com> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>> Hi,
>> >>>>>>>>>>
>> >>>>>>>>>> I am using the below JSON Facet to retrieve
the count of all
>> the
>> >>>>>>>>> different
>> >>>>>>>>>> collections in one query.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>
>> https://localhost:8983/solr/collection1/select?q=testing&shards=https://localhost:8983/solr/collection1,https://localhost:8983/solr/collection2,https://localhost:8983/solr/collection3,https://localhost:8983/solr/collection4,https://localhost:8983/solr/collection5,https://localhost:8983/solr/collection6&rows=0&json.facet={categories
>> >>>>>>>>>> : {type : terms,field : content_type,limit
: 100}}
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> Previously, in Solr 7.6 and Solr 7.7, this
query can work
>> >>>>>> correctly and
>> >>>>>>>>> we
>> >>>>>>>>>> are able to produce the correct output.
>> >>>>>>>>>>
>> >>>>>>>>>> {
>> >>>>>>>>>>  "responseHeader":{
>> >>>>>>>>>>    "zkConnected":true,
>> >>>>>>>>>>    "status":0,
>> >>>>>>>>>>    "QTime":24},
>> >>>>>>>>>>
>> >>>>>>
>> "response":{"numFound":41200,"start":0,"maxScore":12.993215,"docs":[]
>> >>>>>>>>>>  },
>> >>>>>>>>>>  "facets":{
>> >>>>>>>>>>    "count":41200,
>> >>>>>>>>>>    "categories":{
>> >>>>>>>>>>      "buckets":[{
>> >>>>>>>>>>          "val":"collection1",
>> >>>>>>>>>>          "count":26213},
>> >>>>>>>>>>        {
>> >>>>>>>>>>          "val":"collection2",
>> >>>>>>>>>>          "count":12075},
>> >>>>>>>>>>        {
>> >>>>>>>>>>          "val":"collection3",
>> >>>>>>>>>>          "count":1947},
>> >>>>>>>>>>        {
>> >>>>>>>>>>          "val":"collection4",
>> >>>>>>>>>>          "count":850},
>> >>>>>>>>>>        {
>> >>>>>>>>>>          "val":"collection5",
>> >>>>>>>>>>          "count":111},
>> >>>>>>>>>>        {
>> >>>>>>>>>>          "val":"collection6",
>> >>>>>>>>>>          "count":4}]}}}
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> However, in the new Solr 8.0.0, this query
can only work once.
>> >>>>>>>>>> Subsequently, we will get the following
error of 'require
>> >>>>>>>>> authentication':
>> >>>>>>>>>>
>> >>>>>>>>>> {
>> >>>>>>>>>>  "responseHeader":{
>> >>>>>>>>>>    "zkConnected":true,
>> >>>>>>>>>>    "status":401,
>> >>>>>>>>>>    "QTime":11},
>> >>>>>>>>>>  "error":{
>> >>>>>>>>>>    "metadata":[
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>
>> "error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException",
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>
>> "root-error-class","org.apache.solr.client.solrj.impl.Http2SolrClient$RemoteSolrException"],
>> >>>>>>>>>>    "msg":"Error from server at null: Expected
mime type
>> >>>>>>>>>> application/octet-stream but got text/html.
>> >>>>>> <html>\n<head>\n<meta
>> >>>>>>>>>> http-equiv=\"Content-Type\"
>> >>>>>>>>>> content=\"text/html;charset=utf-8\"/>\n<title>Error
401 require
>> >>>>>>>>>> authentication</title>\n</head>\n<body><h2>HTTP
ERROR
>> >>>>>>>>> 401</h2>\n<p>Problem
>> >>>>>>>>>> accessing /solr/collection6/select. Reason:\n<pre>
   require
>> >>>>>>>>>> authentication</pre></p>\n</body>\n</html>\n",
>> >>>>>>>>>>    "code":401}}
>> >>>>>>>>>>
>> >>>>>>>>>> This issue does not occur in Solr 7.6 and
Solr 7.7, even though
>> >>>>>> I have
>> >>>>>>>>> set
>> >>>>>>>>>> up the same authentication for all the versions.
>> >>>>>>>>>>
>> >>>>>>>>>> What could be the issue that causes this?
>> >>>>>>>>>>
>> >>>>>>>>>> Regards,
>> >>>>>>>>>> Edwin
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>
>> >>>>>
>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message