lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yatir Ben Shlomo <yat...@outbrain.com>
Subject solrCloud zookeepr related excpetions
Date Wed, 25 Aug 2010 09:23:21 GMT
Hi I am running a zookeeper ensemble of 3 zookeeper instances
and established a solrCloud to work with it (2 masters , 2 slaves)
on each master machine I have 2 shards (4 shards in total)
on one of the masters I keep noticing ZooKeeper related exceptions which I can't understand:
One appears to be  TIME OUT in (ClientCnxn.java):906
And the other is java.lang.IllegalArgumentException: Path cannot be null (PathUtils.java:45)

Here are my logs (I set the log level to FINE on zookeeper package)

 Anyone can identify the issue?



FINE: Reading reply sessionid:0x12a97312613010b, packet:: clientPath:null serverPath:null
finished:false header:: -8,101  replyHeader:: -8,-1,0  request:: 30064776552,v{'/collections},v{},v{'/collections/ENPwl/shards/ENPWL1,'/collections/ENPwl/shards/ENPWL4,'/collections/ENPwl/shards/ENPWL2,'/collections,'/collections/ENPwl/shards/ENPWL3,'/collections/ENPwlMaster/shards/ENPWLMaster_3,'/collections/ENPwlMaster/shards/ENPWLMaster_4,'/live_nodes,'/collections/ENPwlMaster/shards/ENPWLMaster_1,'/collections/ENPwlMaster/shards/ENPWLMaster_2}
 response:: null
Aug 25, 2010 5:18:19 AM org.apache.log4j.Category debug
FINE: Reading reply sessionid:0x12a97312613010b, packet:: clientPath:null serverPath:null
finished:false header:: 540,8  replyHeader:: 540,-1,0  request:: '/collections,F  response::
v{'ENPwl,'ENPwlMaster}
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.log4j.Category error
SEVERE: Error while calling watcher
java.lang.IllegalArgumentException: Path cannot be null
        at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:45)
        at org.apache.zookeeper.ZooKeeper.getChildren(zookeeper:ZooKeeper.java):1196)
        at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:200)
        at org.apache.solr.common.cloud.ZkStateReader$5.process(ZkStateReader.java:315)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(zookeeper:ClientCnxn.java):425)
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWL3 in collection:ENPwl
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWL4 in collection:ENPwl
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWL1 in collection:ENPwl
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.cloud.ZkController$2 process
INFO: Updating live nodes:org.apache.solr.common.cloud.SolrZkClient@55308275
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Updating live nodes from ZooKeeper...
Aug 25, 2010 5:18:19 AM org.apache.log4j.Category debug
FINE: Reading reply sessionid:0x12a97312613010b, packet:: clientPath:null serverPath:null
finished:false header:: 541,8  replyHeader:: 541,-1,0  request:: '/live_nodes,F  response::
v{'ob1078.nydc1.outbrain.com:8983_solr2,'ob1078.nydc1.outbrain.com:8983_solr1,'ob1061.nydc1.outbrain.com:8983_solr2,'ob1062.nydc1.outbrain.com:8983_solr1,'ob1062.nydc1.outbrain.com:8983_solr2,'ob1061.nydc1.outbrain.com:8983_solr1,'ob1077.nydc1.outbrain.com:8983_solr2,'ob1077.nydc1.outbrain.com:8983_solr1}
Aug 25, 2010 5:18:19 AM org.apache.log4j.Category error
SEVERE: Error while calling watcher
java.lang.IllegalArgumentException: Path cannot be null
        at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:45)
        at org.apache.zookeeper.ZooKeeper.getChildren(zookeeper:ZooKeeper.java):1196)
        at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:200)
        at org.apache.solr.cloud.ZkController$2.process(ZkController.java:321)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(zookeeper:ClientCnxn.java):425)
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ConnectionManager process
INFO: Watcher org.apache.solr.common.cloud.ConnectionManager@339bb448 name:ZooKeeperConnection
Watcher:zook1:2181,zook2:2181,zook3:2181 got event WatchedEvent: Server state change. New
state: Disconnected path:null type:None
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWLMaster_1 in collection:ENPwlMaster
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWLMaster_3 in collection:ENPwlMaster
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWL2 in collection:ENPwl
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWLMaster_2 in collection:ENPwlMaster
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWLMaster_4 in collection:ENPwlMaster
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$5 process
INFO: Detected a new or removed collection
Aug 25, 2010 5:18:19 AM org.apache.log4j.Category debug
FINE: Reading reply sessionid:0x12a97312613010b, packet:: clientPath:null serverPath:null
finished:false header:: 542,8  replyHeader:: 542,-1,0  request:: '/collections,F  response::
v{'ENPwl,'ENPwlMaster}
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.log4j.Category error
SEVERE: Error while calling watcher
java.lang.IllegalArgumentException: Path cannot be null
        at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:45)
        at org.apache.zookeeper.ZooKeeper.getChildren(zookeeper:ZooKeeper.java):1196)
        at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:200)
        at org.apache.solr.common.cloud.ZkStateReader$5.process(ZkStateReader.java:315)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(zookeeper:ClientCnxn.java):425)
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWL3 in collection:ENPwl
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWL4 in collection:ENPwl
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWL1 in collection:ENPwl
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.cloud.ZkController$2 process
INFO: Updating live nodes:org.apache.solr.common.cloud.SolrZkClient@55308275
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Updating live nodes from ZooKeeper...
Aug 25, 2010 5:18:19 AM org.apache.log4j.Category debug
FINE: Reading reply sessionid:0x12a97312613010b, packet:: clientPath:null serverPath:null
finished:false header:: 543,8  replyHeader:: 543,-1,0  request:: '/live_nodes,F  response::
v{'ob1078.nydc1.outbrain.com:8983_solr2,'ob1078.nydc1.outbrain.com:8983_solr1,'ob1061.nydc1.outbrain.com:8983_solr2,'ob1062.nydc1.outbrain.com:8983_solr1,'ob1062.nydc1.outbrain.com:8983_solr2,'ob1061.nydc1.outbrain.com:8983_solr1,'ob1077.nydc1.outbrain.com:8983_solr2,'ob1077.nydc1.outbrain.com:8983_solr1}
Aug 25, 2010 5:18:19 AM org.apache.log4j.Category error
SEVERE: Error while calling watcher
java.lang.IllegalArgumentException: Path cannot be null
        at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:45)
        at org.apache.zookeeper.ZooKeeper.getChildren(zookeeper:ZooKeeper.java):1196)
        at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:200)
        at org.apache.solr.cloud.ZkController$2.process(ZkController.java:321)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(zookeeper:ClientCnxn.java):425)
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ConnectionManager process
INFO: Watcher org.apache.solr.common.cloud.ConnectionManager@339bb448 name:ZooKeeperConnection
Watcher:zook1:2181,zook2:2181,zook3:2181 got event WatchedEvent: Server state change. New
state: SyncConnected path:null type:None
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWLMaster_1 in collection:ENPwlMaster
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWLMaster_3 in collection:ENPwlMaster
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWL2 in collection:ENPwl
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.log4j.Category info
INFO: Attempting connection to server zook2/192.168.252.62:2181
Aug 25, 2010 5:18:19 AM org.apache.log4j.Category info
INFO: Priming connection to java.nio.channels.SocketChannel[connected local=/192.168.252.77:54892
remote=zook2/192.168.252.62:2181]
Aug 25, 2010 5:18:19 AM org.apache.log4j.Category info
INFO: Server connection successful
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.log4j.Category error
SEVERE: Error while calling watcher
java.lang.IllegalArgumentException: Path cannot be null
        at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:45)
        at org.apache.zookeeper.ZooKeeper.getChildren(zookeeper:ZooKeeper.java):1196)
        at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:200)
        at org.apache.solr.common.cloud.ZkStateReader$5.process(ZkStateReader.java:315)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(zookeeper:ClientCnxn.java):425)
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWL1 in collection:ENPwl
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWLMaster_2 in collection:ENPwlMaster
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.cloud.ZkController$2 process
INFO: Updating live nodes:org.apache.solr.common.cloud.SolrZkClient@29d03e78
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Updating live nodes from ZooKeeper...
Aug 25, 2010 5:18:19 AM org.apache.log4j.Category error
SEVERE: Error while calling watcher
java.lang.IllegalArgumentException: Path cannot be null
        at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:45)
        at org.apache.zookeeper.ZooKeeper.getChildren(zookeeper:ZooKeeper.java):1196)
        at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:200)
        at org.apache.solr.cloud.ZkController$2.process(ZkController.java:321)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(zookeeper:ClientCnxn.java):425)
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWLMaster_4 in collection:ENPwlMaster
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWL4 in collection:ENPwl
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWLMaster_1 in collection:ENPwlMaster
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWL3 in collection:ENPwl
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWL2 in collection:ENPwl
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ConnectionManager process
INFO: Watcher org.apache.solr.common.cloud.ConnectionManager@40bb2bc3 name:ZooKeeperConnection
Watcher:zook1:2181,zook2:2181,zook3:2181 got event WatchedEvent: Server state change. New
state: SyncConnected path:null type:None
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWLMaster_3 in collection:ENPwlMaster
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$5 process
INFO: Detected a new or removed collection
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.log4j.Category error
SEVERE: Error while calling watcher
java.lang.IllegalArgumentException: Path cannot be null
        at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:45)
        at org.apache.zookeeper.ZooKeeper.getChildren(zookeeper:ZooKeeper.java):1196)
        at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:200)
        at org.apache.solr.common.cloud.ZkStateReader$5.process(ZkStateReader.java:315)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(zookeeper:ClientCnxn.java):425)
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWL1 in collection:ENPwl
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWLMaster_2 in collection:ENPwlMaster
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.cloud.ZkController$2 process
INFO: Updating live nodes:org.apache.solr.common.cloud.SolrZkClient@29d03e78
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Updating live nodes from ZooKeeper...
Aug 25, 2010 5:18:19 AM org.apache.log4j.Category error
SEVERE: Error while calling watcher
java.lang.IllegalArgumentException: Path cannot be null
        at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:45)
        at org.apache.zookeeper.ZooKeeper.getChildren(zookeeper:ZooKeeper.java):1196)
        at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:200)
        at org.apache.solr.cloud.ZkController$2.process(ZkController.java:321)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(zookeeper:ClientCnxn.java):425)
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWLMaster_4 in collection:ENPwlMaster
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWL4 in collection:ENPwl
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$4 process
INFO: Detected a shard change under ShardId:ENPWLMaster_1 in collection:ENPwlMaster
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader$6 process
INFO: Notified of CloudState change
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Cloud state update for ZooKeeper already scheduled
Aug 25, 2010 5:18:19 AM org.apache.solr.cloud.ZkController$2 process
INFO: Updating live nodes:org.apache.solr.common.cloud.SolrZkClient@29d03e78
Aug 25, 2010 5:18:19 AM org.apache.solr.common.cloud.ZkStateReader updateCloudState
INFO: Updating live nodes from ZooKeeper...
Aug 25, 2010 5:18:22 AM org.apache.log4j.Category debug
FINE: Got ping response for sessionid:0x12a97312613010b after 0ms

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message