lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anshum Gupta <ans...@anshumgupta.net>
Subject Re: Re:Re: Implementing security.json is breaking ADDREPLICA
Date Tue, 24 Nov 2015 17:31:21 GMT
Yes, it certainly is a PKI issue.

On Tue, Nov 24, 2015 at 7:59 AM, Oakley, Craig (NIH/NLM/NCBI) [C] <
craig.oakley@nih.gov> wrote:

> Thank you for the reply
>
> Trying those exact commands, I'm still getting the same issue
> tar xvzf /net/sybdev11/export/home/sybase/Distr/Solr/solr-5.3.1.tgz
> tar xvzf /net/sybdev11/export/home/sybase/Distr/Solr/zookeeper-3.4.6.tar.gz
> cd zookeeper-3.4.6/
> bin/zkServer.sh start zoo_sample.cfg
> cd ..
> solr-5.3.1/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181
> -cmd putfile /security.json
> PREVsolr-5.3.1/server/scripts/cloud-scripts/security.json
> solr-5.3.1/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181
> -cmd list
> solr-5.3.1/bin/solr start -e cloud -z localhost:2181
> cd solr-5.3.1/
> bin/solr stop -p 7574
> bin/solr start -c -p 7574 -s "example/cloud/node2/solr" -z localhost:2181
> tail -f example/cloud/node2/logs/solr.log
>
> The -cmd list shows
> / (2)
> DATA:
>
>  /zookeeper (1)
>  DATA:
>
>  /security.json (0)
>  DATA:
>
>  {"authentication":{"class":"solr.BasicAuthPlugin","credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}},"authorization":{"class":"solr.RuleBasedAuthorizationPlugin","user-role":
>      {"solr":["admin"]}
>
>      ,"permissions":[
>      {"name":"security-edit","role":"admin"}
>
>      ]}}
>
>
> While the output of tail contains
> ERROR - 2015-11-24 10:45:54.796; [c:gettingstarted s:shard1 r:core_node4
> x:gettingstarted_shard1_replica1] org.apache.solr.common.SolrException;
> Error while trying to recover.:java.util.concurrent.ExecutionException:
> org.apache.http.ParseException: Invalid content type:
>         at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>         at
> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:598)
>         at
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:361)
>         at
> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
> Caused by: org.apache.http.ParseException: Invalid content type:
>         at org.apache.http.entity.ContentType.parse(ContentType.java:273)
>         at
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:512)
>         at
> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:270)
>         at
> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:266)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
>
>
> -----Original Message-----
> From: Anshum Gupta [mailto:anshum@anshumgupta.net]
> Sent: Monday, November 23, 2015 7:24 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
>
> Yes, I see the same issue. I'll update the JIRA and drill down. Thanks.
>
> On Mon, Nov 23, 2015 at 4:18 PM, Anshum Gupta <anshum@anshumgupta.net>
> wrote:
>
> > To restart solr, you should instead use something like:
> > bin/solr start -c -p 8983 -s "example/cloud/node1/solr" -z localhost:2181
> > or
> > bin/solr start -c -p 7574 -s "example/cloud/node2/solr" -z localhost:2181
> >
> > I've seen others report the same exception but never ran into this one
> > myself. Let me try this out.
> >
> >
> >
> > On Mon, Nov 23, 2015 at 2:55 PM, Oakley, Craig (NIH/NLM/NCBI) [C] <
> > craig.oakley@nih.gov> wrote:
> >
> >> FWIW
> >>
> >> I am getting fairly consistent results that if I follow the SOLR-8326
> >> procedure just up through the step of "solr-5.3.1/bin/solr start -e
> cloud
> >> -z localhost:2181": if I then stop just one node (either "./solr stop -p
> >> 7574" or "./solr stop -p 8983") and then restart that same node (using
> the
> >> command suggested by "solr-5.3.1/bin/solr start -e cloud -z
> >> localhost:2181"), then the solr.log for the stopped-and-restarted node
> gets
> >> such stack traces as
> >> ERROR - 2015-11-23 21:49:28.663; [c:gettingstarted s:shard2 r:core_node3
> >> x:gettingstarted_shard2_replica2] org.apache.solr.common.SolrException;
> >> Error while trying to recover.:java.util.concurrent.ExecutionException:
> >> org.apache.http.ParseException: Invalid content type:
> >>         at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> >>         at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> >>         at
> >>
> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:598)
> >>         at
> >>
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:361)
> >>         at
> >> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
> >> Caused by: org.apache.http.ParseException: Invalid content type:
> >>         at
> org.apache.http.entity.ContentType.parse(ContentType.java:273)
> >>         at
> >>
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:512)
> >>         at
> >>
> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:270)
> >>         at
> >>
> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:266)
> >>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>         at
> >>
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
> >>         at
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>         at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>         at java.lang.Thread.run(Thread.java:745)
> >>
> >> While the node which stayed up the whole time starts getting such stack
> >> traces as
> >> ERROR - 2015-11-23 21:57:46.019; [c:gettingstarted s:shard2 r:core_node3
> >> x:gettingstarted_shard2_replica2]
> >> org.apache.solr.security.PKIAuthenticationPlugin; Invalid time r?
> >> java.lang.NumberFormatException: For input string: "r?"
> >>         at
> >>
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> >>         at java.lang.Long.parseLong(Long.java:589)
> >>         at java.lang.Long.parseLong(Long.java:631)
> >>         at
> >>
> org.apache.solr.security.PKIAuthenticationPlugin.doAuthenticate(PKIAuthenticationPlugin.java:128)
> >>         at
> >>
> org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:252)
> >>         at
> >>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:186)
> >>         at
> >>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
> >>         at
> >>
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> >>         at
> >>
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> >>         at
> >>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> >>         at
> >>
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> >>         at
> >>
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> >>         at
> >>
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> >>         at
> >>
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> >>         at
> >>
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> >>         at
> >>
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> >>         at
> >>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> >>         at
> >>
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
> >>         at
> >>
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> >>         at
> >>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> >>         at org.eclipse.jetty.server.Server.handle(Server.java:499)
> >>         at
> >> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> >>         at
> >>
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> >>         at
> >>
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
> >>         at
> >>
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> >>         at
> >>
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> >>         at java.lang.Thread.run(Thread.java:745)
> >>
> >> In this case the string is just "r?", but usually it is a longer string
> >> of control characters.
> >>
> >> If I shutdown _both_ nodes and restart _one_, and then allow it to be
> >> "Waiting until we see more replicas up" until it recognizes itself as
> >> leader, and _then_ restart the other node -- in this case it
> successfully
> >> starts.
> >>
> >> Is there some necessary environment tweaking? The symptoms seem similar
> >> whether I use the security.json from SOLR-8326 or the security.json from
> >> the Wiki (with the comma repositioned).
> >>
> >>
> >>
> >> -----Original Message-----
> >> From: Oakley, Craig (NIH/NLM/NCBI) [C]
> >> Sent: Friday, November 20, 2015 6:59 PM
> >> To: 'solr-user@lucene.apache.org' <solr-user@lucene.apache.org>
> >> Subject: RE: Re:Re: Implementing security.json is breaking ADDREPLICA
> >>
> >> Thanks
> >>
> >> It seems to work when there is no security.json, so perhaps there's some
> >> typo in the initial version.
> >>
> >> I notice that the version you sent is different from the documentation
> at
> >>
> cwiki.apache.org/confluence/display/solr/Authentication+and+Authorization+Plugins
> >> in that the Wiki version has "permissions" before "user-role": I also
> >> notice that (at least as of right this moment) the Wiki version has a
> comma
> >> at the end of '"user-role":{"solr":"admin"},' even though it is at the
> end:
> >> and I notice that the Wiki version seems to lack a comma between the
> >> "permissions" section and the "user-role" section. I just now also
> noticed
> >> that the version you sent has '"user-role":{"solr":["admin"]}' (with
> square
> >> brackets) whereas the Wiki does not have square brackets.
> >>
> >> The placement of the comma definitely looks wrong in the Wiki at the
> >> moment (though perhaps someone might correct the Wiki before too long).
> >> Other than that, I don’t know whether the order and/or the square
> brackets
> >> make a difference. I can try with different permutations.
> >>
> >> Thanks again
> >>
> >> P.S. for the record, the Wiki currently has
> >> {
> >> "authentication":{
> >>    "class":"solr.BasicAuthPlugin",
> >>    "credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
> >> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
> >> },
> >> "authorization":{
> >>    "class":"solr.RuleBasedAuthorizationPlugin",
> >>    "permissions":[{"name":"security-edit",
> >>       "role":"admin"}]
> >>    "user-role":{"solr":"admin"},
> >> }}
> >>
> >> -----Original Message-----
> >> From: Anshum Gupta [mailto:anshum@anshumgupta.net]
> >> Sent: Friday, November 20, 2015 6:18 PM
> >> To: solr-user@lucene.apache.org
> >> Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
> >>
> >> This seems unrelated and more like a user error somewhere. Can you just
> >> follow the steps, without any security settings i.e. not even uploading
> >> security.json and see if you still see this? Sorry, but I don't have
> >> access
> >> to the code right now, I try and look at this later tonight.
> >>
> >> On Fri, Nov 20, 2015 at 3:07 PM, Oakley, Craig (NIH/NLM/NCBI) [C] <
> >> craig.oakley@nih.gov> wrote:
> >>
> >> > Thank you for opening SOLR-8326
> >> >
> >> > As a side note, in the procedure you listed, even before adding the
> >> > collection-admin-edit authorization, I'm already hitting trouble:
> >> stopping
> >> > and restarting a node results in the following
> >> >
> >> > INFO  - 2015-11-20 22:48:41.275; [c:solr8326 s:shard2 r:core_node4
> >> > x:solr8326_shard2_replica1] org.apache.solr.cloud.RecoveryStrategy;
> >> > Publishing state of core solr8326_shard2_replica1 as recovering,
> leader
> >> is
> >> > http://{IP-address-redacted}:8983/solr/solr8326_shard2_replica2/ and
> I
> >> am
> >> > http://{IP-address-redacted}:7574/solr/solr8326_shard2_replica1/
> >> > INFO  - 2015-11-20 22:48:41.275; [c:solr8326 s:shard2 r:core_node4
> >> > x:solr8326_shard2_replica1] org.apache.solr.cloud.ZkController;
> >> publishing
> >> > state=recovering
> >> > INFO  - 2015-11-20 22:48:41.278; [c:solr8326 s:shard1 r:core_node3
> >> > x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy;
> >> > Publishing state of core solr8326_shard1_replica1 as recovering,
> leader
> >> is
> >> > http://{IP-address-redacted}:8983/solr/solr8326_shard1_replica2/ and
> I
> >> am
> >> > http://{IP-address-redacted}:7574/solr/solr8326_shard1_replica1/
> >> > INFO  - 2015-11-20 22:48:41.280; [c:solr8326 s:shard1 r:core_node3
> >> > x:solr8326_shard1_replica1] org.apache.solr.cloud.ZkController;
> >> publishing
> >> > state=recovering
> >> > INFO  - 2015-11-20 22:48:41.282; [c:solr8326 s:shard2 r:core_node4
> >> > x:solr8326_shard2_replica1] org.apache.solr.cloud.RecoveryStrategy;
> >> Sending
> >> > prep recovery command to http://{IP-address-redacted}:8983/solr;
> >> > WaitForState:
> >> >
> >>
> action=PREPRECOVERY&core=solr8326_shard2_replica2&nodeName={IP-address-redacted}%3A7574_solr&coreNodeName=core_node4&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true
> >> > INFO  - 2015-11-20 22:48:41.289; [   ]
> >> > org.apache.solr.common.cloud.ZkStateReader$8; A cluster state change:
> >> > WatchedEvent state:SyncConnected type:NodeDataChanged
> >> > path:/collections/solr8326/state.json for collection solr8326 has
> >> occurred
> >> > - updating... (live nodes size: 2)
> >> > INFO  - 2015-11-20 22:48:41.290; [c:solr8326 s:shard1 r:core_node3
> >> > x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy;
> >> Sending
> >> > prep recovery command to http://{IP-address-redacted}:8983/solr;
> >> > WaitForState:
> >> >
> >>
> action=PREPRECOVERY&core=solr8326_shard1_replica2&nodeName={IP-address-redacted}%3A7574_solr&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true
> >> > INFO  - 2015-11-20 22:48:41.291; [   ]
> >> > org.apache.solr.common.cloud.ZkStateReader; Updating data for solr8326
> >> to
> >> > ver 25
> >> > ERROR - 2015-11-20 22:48:41.298; [c:solr8326 s:shard2 r:core_node4
> >> > x:solr8326_shard2_replica1] org.apache.solr.common.SolrException;
> Error
> >> > while trying to recover.:java.util.concurrent.ExecutionException:
> >> > org.apache.http.ParseException: Invalid content type:
> >> >         at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> >> >         at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> >> >         at
> >> >
> >>
> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:598)
> >> >         at
> >> >
> >>
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:361)
> >> >         at
> >> > org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
> >> > Caused by: org.apache.http.ParseException: Invalid content type:
> >> >         at
> >> org.apache.http.entity.ContentType.parse(ContentType.java:273)
> >> >         at
> >> >
> >>
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:512)
> >> >         at
> >> >
> >>
> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:270)
> >> >         at
> >> >
> >>
> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:266)
> >> >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> >         at
> >> >
> >>
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
> >> >         at
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> >         at
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> >         at java.lang.Thread.run(Thread.java:745)
> >> >
> >> > INFO  - 2015-11-20 22:48:41.298; [   ]
> >> > org.apache.solr.common.cloud.ZkStateReader$8; A cluster state change:
> >> > WatchedEvent state:SyncConnected type:NodeDataChanged
> >> > path:/collections/solr8326/state.json for collection solr8326 has
> >> occurred
> >> > - updating... (live nodes size: 2)
> >> > ERROR - 2015-11-20 22:48:41.298; [c:solr8326 s:shard2 r:core_node4
> >> > x:solr8326_shard2_replica1] org.apache.solr.cloud.RecoveryStrategy;
> >> > Recovery failed - trying again... (4)
> >> > INFO  - 2015-11-20 22:48:41.300; [c:solr8326 s:shard2 r:core_node4
> >> > x:solr8326_shard2_replica1] org.apache.solr.cloud.RecoveryStrategy;
> Wait
> >> > 32.0 seconds before trying to recover again (5)
> >> > ERROR - 2015-11-20 22:48:41.300; [c:solr8326 s:shard1 r:core_node3
> >> > x:solr8326_shard1_replica1] org.apache.solr.common.SolrException;
> Error
> >> > while trying to recover.:java.util.concurrent.ExecutionException:
> >> > org.apache.http.ParseException: Invalid content type:
> >> >         at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> >> >         at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> >> >         at
> >> >
> >>
> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:598)
> >> >         at
> >> >
> >>
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:361)
> >> >         at
> >> > org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
> >> > Caused by: org.apache.http.ParseException: Invalid content type:
> >> >         at
> >> org.apache.http.entity.ContentType.parse(ContentType.java:273)
> >> >         at
> >> >
> >>
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:512)
> >> >         at
> >> >
> >>
> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:270)
> >> >         at
> >> >
> >>
> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:266)
> >> >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> >         at
> >> >
> >>
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
> >> >         at
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> >         at
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> >         at java.lang.Thread.run(Thread.java:745)
> >> >
> >> > ERROR - 2015-11-20 22:48:41.318; [c:solr8326 s:shard1 r:core_node3
> >> > x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy;
> >> > Recovery failed - trying again... (4)
> >> > INFO  - 2015-11-20 22:48:41.318; [   ]
> >> > org.apache.solr.common.cloud.ZkStateReader; Updating data for solr8326
> >> to
> >> > ver 26
> >> > INFO  - 2015-11-20 22:48:41.319; [c:solr8326 s:shard1 r:core_node3
> >> > x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy;
> Wait
> >> > 32.0 seconds before trying to recover again (5)
> >> >
> >> >
> >> > I would not be surprised if this were to be some unrelated issue (the
> >> > symptoms are quite different)
> >> >
> >> >
> >> >
> >> > Thanks again
> >> >
> >> >
> >> > -----Original Message-----
> >> > From: Anshum Gupta [mailto:anshum@anshumgupta.net]
> >> > Sent: Friday, November 20, 2015 1:31 PM
> >> > To: solr-user@lucene.apache.org
> >> > Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
> >> >
> >> > Collections API were available before November of 2014, if that is
> when
> >> you
> >> > took the class. However, it was only with Solr 5.0 (released in Feb
> >> 2015)
> >> > that the only supported mechanism to create a collection was
> restricted
> >> to
> >> > Collections API.
> >> >
> >> > Here are the list of steps that you'd need to run to see that things
> are
> >> > fine for you without the read permission:
> >> > * Untar and setup Solr, don't start it yet
> >> > * Start clean zookeeper
> >> > * Put the security.json in zk, without anything other than a
> >> security-edit
> >> > permission. Find the content of the file below. Upload it using your
> >> own zk
> >> > client or through the solr script:
> >> > > solr-5.3.1/server/scripts/cloud-scripts/zkcli.sh -zkhost
> >> localhost:2181
> >> > -cmd putfile /security.json ~/security.json
> >> >
> >> > security.json:
> >> >
> >> >
> >>
> {"authentication":{"class":"solr.BasicAuthPlugin","credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
> >> >
> >> >
> >>
> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}},"authorization":{"class":"solr.RuleBasedAuthorizationPlugin","user-role":{"solr":["admin"]},"permissions":[{"name":"security-edit","role":"admin"}]}}
> >> >
> >> > * Start solr:
> >> > > solr-5.3.1/bin/solr start -e cloud -z localhost:2181
> >> >
> >> > You would need to key in a few things e.g. #nodes and ports, leave
> them
> >> at
> >> > the default values of 2 nodes and 8983/7574, unless you want to run
> >> Solr on
> >> > a different port. Then let it create a default collection to just make
> >> sure
> >> > that everything works fine.
> >> >
> >> > * Add the collection-admin-edit command:
> >> > > curl --user solr:SolrRocks
> >> > http://localhost:8983/solr/admin/authorization
> >> > -H 'Content-type:application/json' -d '{"set-permission" :
> >> > {"name":"collection-admin-edit", "role":"admin"}}'
> >> >
> >> > At this point, everything should be working fine. Restarting the nodes
> >> >  should also work fine. You can try 2 things at this point:
> >> > 1. Create a new collection with 1 shard and 1 replica and then try
> >> adding a
> >> > replica, here's how:
> >> > > curl --user solr:SolrRocks
> >> >
> >> >
> >>
> http://localhost:8983/solr/admin/collections?action=CREATE&name=testcollection&collection.configName=gettingstarted&numShards=1
> >> >
> >> > > curl --user solr:SolrRocks
> >> >
> >> >
> >>
> http://localhost:8983/solr/admin/collections?action=ADDREPLICA&collection=testcollection&shard=shard1
> >> >
> >> > This should work fine.
> >> >
> >> > 2. After this, try restarting the solr cluster. Here's how you can do
> >> so,
> >> > assuming you didn't change any of the defaults and you are running zk
> on
> >> > localhost:2181. If not, just change those values below:
> >> > > bin/solr stop -all
> >> >
> >> > After this, check that Solr was actually stopped. I'd also suggest you
> >> tail
> >> > the logs on both nodes when they are coming up to see any errors, if
> >> any.
> >> > The logs would be here: example/cloud/node1/logs/solr.log
> >> > and example/cloud/node2/logs/solr.log
> >> >
> >> > > bin/solr start -c -p 8983 -s "example/cloud/node1/solr" -z
> >> localhost:2181
> >> > > bin/solr start -c -p 7574 -s "example/cloud/node2/solr" -z
> >> localhost:2181
> >> >
> >> > If you get to this checkpoint fine, try adding a read permission.
> >> > Add a permission:
> >> > > curl --user solr:SolrRocks
> >> > http://localhost:8983/solr/admin/authorization
> >> > -H 'Content-type:application/json' -d '{"set-permission" :
> >> {"name":"read",
> >> > "role":"read"}}'
> >> >
> >> > Add a user:
> >> > > curl --user solr:SolrRocks
> >> > http://localhost:8983/solr/admin/authentication
> >> > -H 'Content-type:application/json' -d '{"set-user" :
> >> > {"solrread":"solrRocks"}}'
> >> >
> >> > Assign a role to the user:
> >> > >curl --user solr:SolrRocks
> >> http://localhost:8983/solr/admin/authorization
> >> > -H 'Content-type:application/json' -d '{"set-user-role" :
> >> > {"solrread":["read"]}}'
> >> >
> >> > After this, you should start having issues with ADDREPLICA.
> >> > Also, as you would at this point have a collection with a shard that
> >> has a
> >> > replication factor > 1 (remember the ADDREPLICA we did earlier), you
> >> would
> >> > have issues when you restart the cluster again using the steps I
> >> mentioned
> >> > above.
> >> >
> >> >
> >> > Can you confirm this? I guess I'll just use this text to create a new
> >> JIRA
> >> > now.
> >> >
> >> >
> >> > On Fri, Nov 20, 2015 at 10:04 AM, Oakley, Craig (NIH/NLM/NCBI) [C] <
> >> > craig.oakley@nih.gov> wrote:
> >> >
> >> > > Thank you again for the reply.
> >> > >
> >> > > Below is the Email I was about to send prior to your reply a moment
> >> ago:
> >> > > shall I try again without "read" in the security.json?
> >> > >
> >> > >
> >> > >
> >> > > The Collections API method was not discussed in the "Unleashed"
> class
> >> at
> >> > > the conference in DC in 2014 (probably because it was not yet
> >> available),
> >> > > so I was using the method I knew.
> >> > >
> >> > > I have now tried again using admin/collections?action=CREATE (using
> >> > > different port numbers to avoid confusion from the failed previous
> >> > > attempts: the previously created nodes had been shutdown and their
> >> > > core.properties files renamed so as not to be discovered), but the
> >> > results
> >> > > are the same:
> >> > > INFO  - 2015-11-20 16:56:25.283; [c:xmpl3 s:shard1 r:core_node2
> >> > > x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy;
> >> Starting
> >> > > Replication Recovery.
> >> > > INFO  - 2015-11-20 16:56:25.284; [c:xmpl3 s:shard1 r:core_node2
> >> > > x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy;
> Begin
> >> > > buffering updates.
> >> > > INFO  - 2015-11-20 16:56:25.284; [c:xmpl3 s:shard1 r:core_node2
> >> > > x:xmpl3_shard1_replica2] org.apache.solr.update.UpdateLog; Starting
> to
> >> > > buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
> >> > > INFO  - 2015-11-20 16:56:25.284; [c:xmpl3 s:shard1 r:core_node2
> >> > > x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy;
> >> > Attempting
> >> > > to replicate from http://
> >> > > {IP-address-redacted}:4685/solr/xmpl3_shard1_replica1/.
> >> > > ERROR - 2015-11-20 16:56:25.292; [c:xmpl3 s:shard1 r:core_node2
> >> > > x:xmpl3_shard1_replica2] org.apache.solr.common.SolrException; Error
> >> > while
> >> > > trying to
> >> > >
> >> >
> >>
> recover:org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
> >> > > Error from server at http://
> >> > {IP-address-redacted}:4685/solr/xmpl3_shard1_replica1:
> >> > > Expected mime type application/octet-stream but got text/html.
> <html>
> >> > > <head>
> >> > > <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
> >> > > <title>Error 401 Unauthorized request, Response code: 401</title>
> >> > > </head>
> >> > > <body><h2>HTTP ERROR 401</h2>
> >> > > <p>Problem accessing /solr/xmpl3_shard1_replica1/update. Reason:
> >> > > <pre>    Unauthorized request, Response code:
> >> > > 401</pre></p><hr><i><small>Powered by
Jetty://</small></i><hr/>
> >> > >
> >> > > </body>
> >> > > </html>
> >> > >
> >> > >         at
> >> > >
> >> >
> >>
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:528)
> >> > >         at
> >> > >
> >> >
> >>
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
> >> > >         at
> >> > >
> >> >
> >>
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
> >> > >         at
> >> > >
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
> >> > >         at
> >> > >
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
> >> > >         at
> >> > >
> >> >
> >>
> org.apache.solr.cloud.RecoveryStrategy.commitOnLeader(RecoveryStrategy.java:207)
> >> > >         at
> >> > >
> >> >
> >>
> org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:147)
> >> > >         at
> >> > >
> >> >
> >>
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:437)
> >> > >         at
> >> > >
> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
> >> > >
> >> > > INFO  - 2015-11-20 16:56:25.292; [c:xmpl3 s:shard1 r:core_node2
> >> > > x:xmpl3_shard1_replica2] org.apache.solr.update.UpdateLog; Dropping
> >> > > buffered updates FSUpdateLog{state=BUFFERING, tlog=null}
> >> > > ERROR - 2015-11-20 16:56:25.293; [c:xmpl3 s:shard1 r:core_node2
> >> > > x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy;
> >> Recovery
> >> > > failed - trying again... (2)
> >> > > INFO  - 2015-11-20 16:56:25.293; [c:xmpl3 s:shard1 r:core_node2
> >> > > x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy;
> Wait
> >> 8.0
> >> > > seconds before trying to recover again (3)
> >> > >
> >> > >
> >> > > Below is a list of the steps I took.
> >> > >
> >> > > ./zkcli.sh --zkhost localhost:4545 -cmd makepath /solr/xmpl3
> >> > > ./zkcli.sh --zkhost localhost:4545/solr/xmpl3 -cmd putfile
> >> /security.json
> >> > > ~/solr/security151119a.json
> >> > > ./zkcli.sh --zkhost localhost:4545/solr/xmpl3 -cmd upconfig -confdir
> >> > > ../../solr/configsets/basic_configs/conf -confname xmpl3
> >> > > cd ../../../bin/
> >> > > ./solr -c -p 4695 -d
> >> ~dbman/solr/straight531outofbox/solr-5.3.1/server/
> >> > -z
> >> > > localhost:4545/solr/xmpl3 -s
> >> > > ~dbman/solr/straight531outofbox/solr-5.3.1/example/solr
> >> > > ./solr -c -p 4685 -d
> >> ~dbman/solr/straight531outofbox/solr-5.3.1/server/
> >> > -z
> >> > > localhost:4545/solr/xmpl3 -s
> >> > > ~dbman/solr/straight531outofbox/solr-5.3.1/server/solr
> >> > > curl -u solr:SolrRocks '
> >> > >
> >> >
> >>
> http://nosqltest11:4685/solr/admin/collections?action=CREATE&name=xmpl3&numShards=1&replicationFactor=1&createNodeSet={IP-address-redacted}:4685_solr
> >> > > '
> >> > > curl -u solr:SolrRocks '
> >> > >
> >> >
> >>
> http://nosqltest11:4685/solr/admin/collections?action=ADDREPLICA&collection=xmpl3&shard=shard1&node={IP-address-redacted}:4695_solr&wt=json&indent=true
> >> > > '
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > Can you provide a list of steps to take in an out-of-the-box
> directory
> >> > > tree whereby ADDREPLICA _will_ work with security.json already in
> >> place?
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > -----Original Message-----
> >> > > From: Anshum Gupta [mailto:anshum@anshumgupta.net]
> >> > > Sent: Thursday, November 19, 2015 3:44 PM
> >> > > To: solr-user@lucene.apache.org
> >> > > Subject: Re: Re:Re: Implementing security.json is breaking
> ADDREPLICA
> >> > >
> >> > > I'll try out what you did later in the day, as soon as I get time
> but
> >> why
> >> > > exactly are you creating cores manually? Seems like you manually
> >> create a
> >> > > core and the try to add a replica. Can you try using the Collections
> >> API
> >> > to
> >> > > create a collection?
> >> > >
> >> > > Starting Solr 5.0, the only supported way to create a new collection
> >> is
> >> > via
> >> > > the Collections API. Creating a core would lead to a collection
> >> creation
> >> > > but that's not really supported. It was just something that was done
> >> when
> >> > > there were no Collections API.
> >> > >
> >> > >
> >> > > On Thu, Nov 19, 2015 at 12:36 PM, Oakley, Craig (NIH/NLM/NCBI) [C]
<
> >> > > craig.oakley@nih.gov> wrote:
> >> > >
> >> > > > I tried again with the following security.json, but the results
> were
> >> > the
> >> > > > same:
> >> > > >
> >> > > > {
> >> > > >   "authentication":{
> >> > > >     "class":"solr.BasicAuthPlugin",
> >> > > >     "credentials":{
> >> > > >       "solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
> >> > > > Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c=",
> >> > > >       "solruser":"VgZX1TAMNHT2IJikoGdKtxQdXc+MbNwfqzf89YqcLEE=
> >> > > > 37pPWQ9v4gciIKHuTmFmN0Rv66rnlMOFEWfEy9qjJfY="},
> >> > > >     "":{"v":9}},
> >> > > >   "authorization":{
> >> > > >     "class":"solr.RuleBasedAuthorizationPlugin",
> >> > > >     "user-role":{
> >> > > >       "solr":[
> >> > > >         "admin",
> >> > > >         "read",
> >> > > >         "xmpladmin",
> >> > > >         "xmplgen",
> >> > > >         "xmplsel"],
> >> > > >       "solruser":[
> >> > > >         "read",
> >> > > >         "xmplgen",
> >> > > >         "xmplsel"]},
> >> > > >     "permissions":[
> >> > > >       {
> >> > > >         "name":"security-edit",
> >> > > >         "role":"admin"},
> >> > > >       {
> >> > > >         "name":"xmpl_admin",
> >> > > >         "collection":"xmpl",
> >> > > >         "path":"/admin/*",
> >> > > >         "role":"xmpladmin"},
> >> > > >       {
> >> > > >         "name":"xmpl_sel",
> >> > > >         "collection":"xmpl",
> >> > > >         "path":"/select/*",
> >> > > >         "role":null},
> >> > > >       {
> >> > > >          "name":"all-admin",
> >> > > >          "collection":null,
> >> > > >          "path":"/*",
> >> > > >          "role":"xmplgen"},
> >> > > >       {
> >> > > >          "name":"all-core-handlers",
> >> > > >          "path":"/*",
> >> > > >          "role":"xmplgen"}],
> >> > > >     "":{"v":42}}}
> >> > > >
> >> > > > -----Original Message-----
> >> > > > From: Oakley, Craig (NIH/NLM/NCBI) [C]
> >> > > > Sent: Thursday, November 19, 2015 1:46 PM
> >> > > > To: 'solr-user@lucene.apache.org' <solr-user@lucene.apache.org>
> >> > > > Subject: RE: Re:Re: Implementing security.json is breaking
> >> ADDREPLICA
> >> > > >
> >> > > > I note that the thread called "Security Problems" (most recent
> post
> >> by
> >> > > > Nobel Paul) seems like it may help with much of what I'm trying
to
> >> do.
> >> > I
> >> > > > will see to what extent that may help.
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Anshum Gupta
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > Anshum Gupta
> >> >
> >>
> >>
> >>
> >> --
> >> Anshum Gupta
> >>
> >
> >
> >
> > --
> > Anshum Gupta
> >
>
>
>
> --
> Anshum Gupta
>



-- 
Anshum Gupta

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message