nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Venkatesh Bodapati <venkatesh.bodap...@inndata.in>
Subject Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi
Date Fri, 06 May 2016 15:03:10 GMT
2016-05-06 20:01:33,201 INFO [pool-18-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile
Repository
2016-05-06 20:01:33,473 INFO [pool-18-thread-1]
org.wali.MinimalLockingWriteAheadLog
org.wali.MinimalLockingWriteAheadLog@26650de4 checkpointed with 0 Records
and 0 Swap Files in 134 milliseconds (Stop-the-world time = 26
milliseconds, Clear Edit Logs time = 37 millis), max Transaction ID 268027
2016-05-06 20:01:33,473 INFO [pool-18-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
Repository with 0 records in 135 milliseconds
2016-05-06 20:02:30,775 INFO [Flow Service Tasks Thread-1]
o.a.nifi.controller.StandardFlowService Saved flow controller
org.apache.nifi.controller.FlowController@2c4e3ffb // Another save pending
= false
2016-05-06 20:02:30,958 INFO [pool-26-thread-18]
o.a.h.h.zookeeper.RecoverableZooKeeper Process
identifier=hconnection-0xaa14407 connecting to ZooKeeper
ensemble=localhost:2181
2016-05-06 20:02:30,958 INFO [pool-26-thread-18]
org.apache.zookeeper.ZooKeeper Initiating client connection,
connectString=localhost:2181 sessionTimeout=90000
watcher=hconnection-0xaa144070x0, quorum=localhost:2181, baseZNode=/hbase
2016-05-06 20:02:30,975 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
2016-05-06 20:02:30,977 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Socket connection established to localhost/
127.0.0.1:2181, initiating session
2016-05-06 20:02:31,081 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Session establishment complete on server
localhost/127.0.0.1:2181, sessionid = 0x154865d23ca000c, negotiated timeout
= 60000
2016-05-06 20:02:32,657 INFO [pool-26-thread-19]
o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] to run with 1 threads
2016-05-06 20:02:32,658 INFO [pool-26-thread-17]
o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
PutHBaseCell[id=04a3e270-375c-4f2d-9e93-ac53a70765d8] to run with 1 threads
2016-05-06 20:02:32,659 INFO [pool-26-thread-20]
o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
PutHBaseJSON[id=dc18a7a7-dd2f-484c-aa27-4a6f0de65920] to run with 1 threads
2016-05-06 20:02:32,676 ERROR [Timer-Driven Process Thread-6]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:33,678 ERROR [Timer-Driven Process Thread-4]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:34,679 ERROR [Timer-Driven Process Thread-10]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:35,680 ERROR [Timer-Driven Process Thread-2]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:36,681 ERROR [Timer-Driven Process Thread-10]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:37,683 ERROR [Timer-Driven Process Thread-10]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:38,684 ERROR [Timer-Driven Process Thread-3]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:39,686 ERROR [Timer-Driven Process Thread-4]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:40,690 ERROR [Timer-Driven Process Thread-7]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:41,691 ERROR [Timer-Driven Process Thread-5]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:42,693 ERROR [Timer-Driven Process Thread-9]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:42,703 INFO [pool-26-thread-21]
o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
AttributesToJSON[id=bbd65940-8ba8-4c97-8658-3ba4ce835cf3] to run with 1
threads
2016-05-06 20:02:42,714 INFO [pool-26-thread-22]
o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
EvaluateJsonPath[id=aa7e0bb0-9948-48eb-9464-301c3b014392] to run with 1
threads
2016-05-06 20:02:42,718 INFO [pool-26-thread-23]
o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
InvokeHTTP[id=cd2576bd-5d03-4961-9547-48249b86aadd] to run with 1 threads
2016-05-06 20:02:42,724 INFO [pool-26-thread-24]
o.a.k.clients.producer.ProducerConfig ProducerConfig values:
value.serializer = class
org.apache.kafka.common.serialization.ByteArraySerializer
key.serializer = class
org.apache.kafka.common.serialization.ByteArraySerializer
block.on.buffer.full = false
retry.backoff.ms = 100
buffer.memory = 5242880
batch.size = 200
metrics.sample.window.ms = 30000
metadata.max.age.ms = 300000
receive.buffer.bytes = 32768
timeout.ms = 30000
max.in.flight.requests.per.connection = 5
bootstrap.servers = [localhost:9092]
metric.reporters = []
client.id = NiFi-24337410-7f3e-4fe0-8da4-5772399635e3
compression.type = none
retries = 0
max.request.size = 1048576
send.buffer.bytes = 131072
acks = 0
reconnect.backoff.ms = 10
linger.ms = 5000
metrics.num.samples = 2
metadata.fetch.timeout.ms = 30000

2016-05-06 20:02:42,729 INFO [pool-26-thread-1]
o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
SplitJson[id=857b6d94-c163-495a-8145-e5903830c02e] to run with 1 threads
2016-05-06 20:02:42,790 INFO [Flow Service Tasks Thread-2]
o.a.nifi.controller.StandardFlowService Saved flow controller
org.apache.nifi.controller.FlowController@2c4e3ffb // Another save pending
= false
2016-05-06 20:02:42,800 WARN [pool-26-thread-24]
o.a.k.clients.producer.ProducerConfig The configuration value.serializer =
class org.apache.kafka.common.serialization.ByteArraySerializer was
supplied but isn't a known config.
2016-05-06 20:02:42,801 WARN [pool-26-thread-24]
o.a.k.clients.producer.ProducerConfig The configuration key.serializer =
class org.apache.kafka.common.serialization.ByteArraySerializer was
supplied but isn't a known config.
2016-05-06 20:02:42,801 INFO [pool-26-thread-24]
o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
PutKafka[id=24337410-7f3e-4fe0-8da4-5772399635e3] to run with 1 threads
2016-05-06 20:02:43,698 ERROR [Timer-Driven Process Thread-9]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:44,457 INFO [Timer-Driven Process Thread-4]
o.a.nifi.processors.standard.SplitJson
SplitJson[id=857b6d94-c163-495a-8145-e5903830c02e] Split
StandardFlowFileRecord[uuid=bd250e09-7c81-4371-838b-b64319ee2792,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1462543677866-4, container=default,
section=4], offset=888295,
length=791977],offset=0,name=10310751916797,size=791977] into 1000 FlowFiles
2016-05-06 20:02:44,700 ERROR [Timer-Driven Process Thread-10]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:45,839 ERROR [Timer-Driven Process Thread-8]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:45,884 INFO [Provenance Maintenance Thread-2]
o.a.n.p.PersistentProvenanceRepository Created new Provenance Event Writers
for events starting with ID 123674
2016-05-06 20:02:46,840 ERROR [Timer-Driven Process Thread-1]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:47,324 INFO [Provenance Repository Rollover Thread-2]
o.a.n.p.PersistentProvenanceRepository Successfully merged 16 journal files
(1554 records) into single Provenance Log File
./provenance_repository/122120.prov in 1451 milliseconds
2016-05-06 20:02:47,325 INFO [Provenance Repository Rollover Thread-2]
o.a.n.p.PersistentProvenanceRepository Successfully Rolled over Provenance
Event file containing 1338 records
2016-05-06 20:02:47,841 ERROR [Timer-Driven Process Thread-9]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:48,843 ERROR [Timer-Driven Process Thread-1]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:49,844 ERROR [Timer-Driven Process Thread-5]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:50,845 ERROR [Timer-Driven Process Thread-3]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:53,021 ERROR [Timer-Driven Process Thread-8]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:54,023 ERROR [Timer-Driven Process Thread-9]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:55,024 ERROR [Timer-Driven Process Thread-7]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:56,026 ERROR [Timer-Driven Process Thread-6]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:57,027 ERROR [Timer-Driven Process Thread-9]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:58,029 ERROR [Timer-Driven Process Thread-1]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:59,031 ERROR [Timer-Driven Process Thread-1]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:59,559 INFO [NiFi Web Server-369]
o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
InvokeHTTP[id=cd2576bd-5d03-4961-9547-48249b86aadd] to run
2016-05-06 20:02:59,577 INFO [NiFi Web Server-17]
o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
AttributesToJSON[id=bbd65940-8ba8-4c97-8658-3ba4ce835cf3] to run
2016-05-06 20:02:59,819 INFO [NiFi Web Server-21]
o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
SplitJson[id=857b6d94-c163-495a-8145-e5903830c02e] to run
2016-05-06 20:02:59,822 INFO [NiFi Web Server-20]
o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
PutHBaseJSON[id=dc18a7a7-dd2f-484c-aa27-4a6f0de65920] to run
2016-05-06 20:02:59,869 INFO [NiFi Web Server-19]
o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] to run
2016-05-06 20:02:59,889 INFO [NiFi Web Server-18]
o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
PutKafka[id=24337410-7f3e-4fe0-8da4-5772399635e3] to run
2016-05-06 20:02:59,910 INFO [NiFi Web Server-461]
o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
EvaluateJsonPath[id=aa7e0bb0-9948-48eb-9464-301c3b014392] to run
2016-05-06 20:02:59,914 INFO [NiFi Web Server-369]
o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
PutHBaseCell[id=04a3e270-375c-4f2d-9e93-ac53a70765d8] to run
2016-05-06 20:03:00,068 INFO [Flow Service Tasks Thread-1]
o.a.nifi.controller.StandardFlowService Saved flow controller
org.apache.nifi.controller.FlowController@2c4e3ffb // Another save pending
= false
2016-05-06 20:03:15,923 INFO [Provenance Maintenance Thread-2]
o.a.n.p.PersistentProvenanceRepository Created new Provenance Event Writers
for events starting with ID 128123
2016-05-06 20:03:16,584 INFO [Provenance Repository Rollover Thread-1]
o.a.n.p.PersistentProvenanceRepository Successfully merged 16 journal files
(4449 records) into single Provenance Log File
./provenance_repository/123674.prov in 698 milliseconds
2016-05-06 20:03:16,584 INFO [Provenance Repository Rollover Thread-1]
o.a.n.p.PersistentProvenanceRepository Successfully Rolled over Provenance
Event file containing 2770 records
2016-05-06 20:03:33,473 INFO [pool-18-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile
Repository
2016-05-06 20:03:33,638 INFO [pool-18-thread-1]
org.wali.MinimalLockingWriteAheadLog
org.wali.MinimalLockingWriteAheadLog@26650de4 checkpointed with 0 Records
and 0 Swap Files in 163 milliseconds (Stop-the-world time = 32
milliseconds, Clear Edit Logs time = 35 millis), max Transaction ID 270109
2016-05-06 20:03:33,638 INFO [pool-18-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
Repository with 0 records in 164 milliseconds
2016-05-06 20:03:34,057 INFO [FileSystemRepository Workers Thread-2]
o.a.n.c.repository.FileSystemRepository Successfully archived 1 Resource
Claims for Container default in 26 millis
2016-05-06 20:05:33,638 INFO [pool-18-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile
Repository
2016-05-06 20:05:33,782 INFO [pool-18-thread-1]
org.wali.MinimalLockingWriteAheadLog
org.wali.MinimalLockingWriteAheadLog@26650de4 checkpointed with 0 Records
and 0 Swap Files in 143 milliseconds (Stop-the-world time = 43
milliseconds, Clear Edit Logs time = 31 millis), max Transaction ID 270109
2016-05-06 20:05:33,782 INFO [pool-18-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
Repository with 0 records in 143 milliseconds
2016-05-06 20:07:33,782 INFO [pool-18-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile
Repository
2016-05-06 20:07:33,927 INFO [pool-18-thread-1]
org.wali.MinimalLockingWriteAheadLog
org.wali.MinimalLockingWriteAheadLog@26650de4 checkpointed with 0 Records
and 0 Swap Files in 143 milliseconds (Stop-the-world time = 30
milliseconds, Clear Edit Logs time = 37 millis), max Transaction ID 270109
2016-05-06 20:07:33,927 INFO [pool-18-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
Repository with 0 records in 144 milliseconds
2016-05-06 20:09:33,927 INFO [pool-18-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile
Repository
2016-05-06 20:09:34,298 INFO [pool-18-thread-1]
org.wali.MinimalLockingWriteAheadLog
org.wali.MinimalLockingWriteAheadLog@26650de4 checkpointed with 0 Records
and 0 Swap Files in 370 milliseconds (Stop-the-world time = 31
milliseconds, Clear Edit Logs time = 31 millis), max Transaction ID 270109
2016-05-06 20:09:34,299 INFO [pool-18-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
Repository with 0 records in 371 milliseconds
2016-05-06 20:10:16,217 INFO [Provenance Maintenance Thread-3]
o.a.n.p.lucene.DeleteIndexAction Deleted Indices for Expired Provenance
File ./provenance_repository/62087.prov.gz from 1 index files; 0 documents
removed
2016-05-06 20:10:16,217 INFO [Provenance Maintenance Thread-3]
o.a.n.p.PersistentProvenanceRepository Successfully performed Expiration
Action org.apache.nifi.provenance.lucene.DeleteIndexAction@58b024f8 on
Provenance Event file ./provenance_repository/62087.prov.gz in 479 millis
2016-05-06 20:10:16,217 INFO [Provenance Maintenance Thread-3]
o.a.n.p.lucene.DeleteIndexAction Removed expired Provenance Event file
./provenance_repository/62087.prov.gz
2016-05-06 20:10:16,217 INFO [Provenance Maintenance Thread-3]
o.a.n.p.lucene.DeleteIndexAction Removed expired Provenance
Table-of-Contents file ./provenance_repository/toc/62087.toc
2016-05-06 20:10:16,218 INFO [Provenance Maintenance Thread-3]
o.a.n.p.PersistentProvenanceRepository Successfully performed Expiration
Action org.apache.nifi.provenance.expiration.FileRemovalAction@7f90b2b2 on
Provenance Event file ./provenance_repository/62087.prov.gz in 351860 nanos
2016-05-06 20:10:51,454 INFO [Provenance Maintenance Thread-1]
o.a.n.p.lucene.DeleteIndexAction Deleted Indices for Expired Provenance
File ./provenance_repository/62090.prov.gz from 1 index files; 0 documents
removed
2016-05-06 20:10:51,454 INFO [Provenance Maintenance Thread-1]
o.a.n.p.PersistentProvenanceRepository Successfully performed Expiration
Action org.apache.nifi.provenance.lucene.DeleteIndexAction@58b024f8 on
Provenance Event file ./provenance_repository/62090.prov.gz in 187 millis
2016-05-06 20:10:51,454 INFO [Provenance Maintenance Thread-1]
o.a.n.p.lucene.DeleteIndexAction Removed expired Provenance Event file
./provenance_repository/62090.prov.gz
2016-05-06 20:10:51,455 INFO [Provenance Maintenance Thread-1]
o.a.n.p.lucene.DeleteIndexAction Removed expired Provenance
Table-of-Contents file ./provenance_repository/toc/62090.toc
2016-05-06 20:10:51,455 INFO [Provenance Maintenance Thread-1]
o.a.n.p.PersistentProvenanceRepository Successfully performed Expiration
Action org.apache.nifi.provenance.expiration.FileRemovalAction@7f90b2b2 on
Provenance Event file ./provenance_repository/62090.prov.gz in 256807 nanos
2016-05-06 20:11:34,299 INFO [pool-18-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile
Repository
2016-05-06 20:11:34,468 INFO [pool-18-thread-1]
org.wali.MinimalLockingWriteAheadLog
org.wali.MinimalLockingWriteAheadLog@26650de4 checkpointed with 0 Records
and 0 Swap Files in 169 milliseconds (Stop-the-world time = 31
milliseconds, Clear Edit Logs time = 31 millis), max Transaction ID 270109
2016-05-06 20:11:34,469 INFO [pool-18-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
Repository with 0 records in 169 milliseconds
2016-05-06 20:13:34,469 INFO [pool-18-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile
Repository
2016-05-06 20:13:50,482 INFO [pool-18-thread-1]
org.wali.MinimalLockingWriteAheadLog
org.wali.MinimalLockingWriteAheadLog@26650de4 checkpointed with 0 Records
and 0 Swap Files in 15975 milliseconds (Stop-the-world time = 14990
milliseconds, Clear Edit Logs time = 599 millis), max Transaction ID 270109
2016-05-06 20:13:50,483 INFO [pool-18-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
Repository with 0 records in 15976 milliseconds
2016-05-06 20:15:42,638 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Unable to read additional data from server
sessionid 0x154865d23ca000c, likely server has closed socket, closing
socket connection and attempting reconnect
2016-05-06 20:15:43,965 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
2016-05-06 20:15:46,268 WARN [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Session 0x154865d23ca000c for server null,
unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_101]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
~[na:1.7.0_101]
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
2016-05-06 20:15:47,822 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
2016-05-06 20:15:47,823 WARN [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Session 0x154865d23ca000c for server null,
unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_101]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
~[na:1.7.0_101]
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
2016-05-06 20:15:49,458 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
2016-05-06 20:15:49,458 WARN [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Session 0x154865d23ca000c for server null,
unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_101]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
~[na:1.7.0_101]
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
2016-05-06 20:15:50,483 INFO [pool-18-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile
Repository
2016-05-06 20:15:50,945 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
2016-05-06 20:15:50,946 WARN [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Session 0x154865d23ca000c for server null,
unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_101]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
~[na:1.7.0_101]
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
2016-05-06 20:15:51,449 INFO [pool-18-thread-1]
org.wali.MinimalLockingWriteAheadLog
org.wali.MinimalLockingWriteAheadLog@26650de4 checkpointed with 0 Records
and 0 Swap Files in 966 milliseconds (Stop-the-world time = 775
milliseconds, Clear Edit Logs time = 54 millis), max Transaction ID 270109
2016-05-06 20:15:51,449 INFO [pool-18-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
Repository with 0 records in 966 milliseconds
2016-05-06 20:15:52,722 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
2016-05-06 20:15:52,724 WARN [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Session 0x154865d23ca000c for server null,
unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_101]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
~[na:1.7.0_101]
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
2016-05-06 20:15:54,375 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
2016-05-06 20:15:54,376 WARN [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Session 0x154865d23ca000c for server null,
unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_101]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
~[na:1.7.0_101]
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
2016-05-06 20:15:55,589 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
2016-05-06 20:15:55,589 WARN [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Session 0x154865d23ca000c for server null,
unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_101]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
~[na:1.7.0_101]
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
2016-05-06 20:15:57,128 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
2016-05-06 20:15:57,129 WARN [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Session 0x154865d23ca000c for server null,
unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_101]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
~[na:1.7.0_101]
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
2016-05-06 20:15:58,892 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
2016-05-06 20:15:58,892 WARN [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Session 0x154865d23ca000c for server null,
unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_101]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
~[na:1.7.0_101]
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
2016-05-06 20:16:00,526 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
2016-05-06 20:16:00,527 WARN [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Session 0x154865d23ca000c for server null,
unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_101]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
~[na:1.7.0_101]
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
2016-05-06 20:16:02,368 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
2016-05-06 20:16:02,368 WARN [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Session 0x154865d23ca000c for server null,
unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_101]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
~[na:1.7.0_101]
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
2016-05-06 20:16:03,869 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
2016-05-06 20:16:04,016 WARN [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Session 0x154865d23ca000c for server null,
unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_101]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
~[na:1.7.0_101]
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
2016-05-06 20:16:05,599 INFO [Thread-1] org.apache.nifi.NiFi Initiating
shutdown of Jetty web server...
2016-05-06 20:16:05,787 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
2016-05-06 20:16:09,161 WARN [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Session 0x154865d23ca000c for server null,
unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_101]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
~[na:1.7.0_101]
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
2016-05-06 20:16:10,684 INFO [Thread-1]
org.eclipse.jetty.server.ServerConnector Stopped ServerConnector@1ae01111
{HTTP/1.1}{0.0.0.0:8080}
2016-05-06 20:16:11,052 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
2016-05-06 20:16:13,906 WARN [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Session 0x154865d23ca000c for server null,
unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_101]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
~[na:1.7.0_101]
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
~[zookeeper-3.4.6.jar:3.4.6-1569965]

On Fri, May 6, 2016 at 8:16 PM, Bryan Bende <bbende@gmail.com> wrote:

> Looks like it is not logging the full stacktrace by default...
>
> Can you edit NIFI_HOME/conf/logback.xml and add a line like this:
>
> <logger name="org.apache.nifi.hbase" level="DEBUG"/>
>
> Wait about 15-30 seconds and then see what is in nifi-app.log
>
> On Fri, May 6, 2016 at 10:37 AM, Venkatesh Bodapati <
> venkatesh.bodapati@inndata.in> wrote:
>
>> This is  whole stacktrace in nifi-app.log.
>>
>> 2016-05-06 20:01:33,201 INFO [pool-18-thread-1]
>> o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile
>> Repository
>> 2016-05-06 20:01:33,473 INFO [pool-18-thread-1]
>> org.wali.MinimalLockingWriteAheadLog
>> org.wali.MinimalLockingWriteAheadLog@26650de4 checkpointed with 0
>> Records and 0 Swap Files in 134 milliseconds (Stop-the-world time = 26
>> milliseconds, Clear Edit Logs time = 37 millis), max Transaction ID 268027
>> 2016-05-06 20:01:33,473 INFO [pool-18-thread-1]
>> o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
>> Repository with 0 records in 135 milliseconds
>> 2016-05-06 20:02:30,775 INFO [Flow Service Tasks Thread-1]
>> o.a.nifi.controller.StandardFlowService Saved flow controller
>> org.apache.nifi.controller.FlowController@2c4e3ffb // Another save
>> pending = false
>> 2016-05-06 20:02:30,958 INFO [pool-26-thread-18]
>> o.a.h.h.zookeeper.RecoverableZooKeeper Process
>> identifier=hconnection-0xaa14407 connecting to ZooKeeper
>> ensemble=localhost:2181
>> 2016-05-06 20:02:30,958 INFO [pool-26-thread-18]
>> org.apache.zookeeper.ZooKeeper Initiating client connection,
>> connectString=localhost:2181 sessionTimeout=90000
>> watcher=hconnection-0xaa144070x0, quorum=localhost:2181, baseZNode=/hbase
>> 2016-05-06 20:02:30,975 INFO
>> [pool-26-thread-18-SendThread(localhost:2181)]
>> org.apache.zookeeper.ClientCnxn Opening socket connection to server
>> localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
>> (unknown error)
>> 2016-05-06 20:02:30,977 INFO
>> [pool-26-thread-18-SendThread(localhost:2181)]
>> org.apache.zookeeper.ClientCnxn Socket connection established to localhost/
>> 127.0.0.1:2181, initiating session
>> 2016-05-06 20:02:31,081 INFO
>> [pool-26-thread-18-SendThread(localhost:2181)]
>> org.apache.zookeeper.ClientCnxn Session establishment complete on server
>> localhost/127.0.0.1:2181, sessionid = 0x154865d23ca000c, negotiated
>> timeout = 60000
>> 2016-05-06 20:02:32,657 INFO [pool-26-thread-19]
>> o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] to run with 1 threads
>> 2016-05-06 20:02:32,658 INFO [pool-26-thread-17]
>> o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
>> PutHBaseCell[id=04a3e270-375c-4f2d-9e93-ac53a70765d8] to run with 1 threads
>> 2016-05-06 20:02:32,659 INFO [pool-26-thread-20]
>> o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
>> PutHBaseJSON[id=dc18a7a7-dd2f-484c-aa27-4a6f0de65920] to run with 1 threads
>> 2016-05-06 20:02:32,676 ERROR [Timer-Driven Process Thread-6]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:33,678 ERROR [Timer-Driven Process Thread-4]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:34,679 ERROR [Timer-Driven Process Thread-10]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:35,680 ERROR [Timer-Driven Process Thread-2]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:36,681 ERROR [Timer-Driven Process Thread-10]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:37,683 ERROR [Timer-Driven Process Thread-10]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:38,684 ERROR [Timer-Driven Process Thread-3]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:39,686 ERROR [Timer-Driven Process Thread-4]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:40,690 ERROR [Timer-Driven Process Thread-7]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:41,691 ERROR [Timer-Driven Process Thread-5]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:42,693 ERROR [Timer-Driven Process Thread-9]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:42,703 INFO [pool-26-thread-21]
>> o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
>> AttributesToJSON[id=bbd65940-8ba8-4c97-8658-3ba4ce835cf3] to run with 1
>> threads
>> 2016-05-06 20:02:42,714 INFO [pool-26-thread-22]
>> o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
>> EvaluateJsonPath[id=aa7e0bb0-9948-48eb-9464-301c3b014392] to run with 1
>> threads
>> 2016-05-06 20:02:42,718 INFO [pool-26-thread-23]
>> o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
>> InvokeHTTP[id=cd2576bd-5d03-4961-9547-48249b86aadd] to run with 1 threads
>> 2016-05-06 20:02:42,724 INFO [pool-26-thread-24]
>> o.a.k.clients.producer.ProducerConfig ProducerConfig values:
>> value.serializer = class
>> org.apache.kafka.common.serialization.ByteArraySerializer
>> key.serializer = class
>> org.apache.kafka.common.serialization.ByteArraySerializer
>> block.on.buffer.full = false
>> retry.backoff.ms = 100
>> buffer.memory = 5242880
>> batch.size = 200
>> metrics.sample.window.ms = 30000
>> metadata.max.age.ms = 300000
>> receive.buffer.bytes = 32768
>> timeout.ms = 30000
>> max.in.flight.requests.per.connection = 5
>> bootstrap.servers = [localhost:9092]
>> metric.reporters = []
>> client.id = NiFi-24337410-7f3e-4fe0-8da4-5772399635e3
>> compression.type = none
>> retries = 0
>> max.request.size = 1048576
>> send.buffer.bytes = 131072
>> acks = 0
>> reconnect.backoff.ms = 10
>> linger.ms = 5000
>> metrics.num.samples = 2
>> metadata.fetch.timeout.ms = 30000
>>
>> 2016-05-06 20:02:42,729 INFO [pool-26-thread-1]
>> o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
>> SplitJson[id=857b6d94-c163-495a-8145-e5903830c02e] to run with 1 threads
>> 2016-05-06 20:02:42,790 INFO [Flow Service Tasks Thread-2]
>> o.a.nifi.controller.StandardFlowService Saved flow controller
>> org.apache.nifi.controller.FlowController@2c4e3ffb // Another save
>> pending = false
>> 2016-05-06 20:02:42,800 WARN [pool-26-thread-24]
>> o.a.k.clients.producer.ProducerConfig The configuration value.serializer =
>> class org.apache.kafka.common.serialization.ByteArraySerializer was
>> supplied but isn't a known config.
>> 2016-05-06 20:02:42,801 WARN [pool-26-thread-24]
>> o.a.k.clients.producer.ProducerConfig The configuration key.serializer =
>> class org.apache.kafka.common.serialization.ByteArraySerializer was
>> supplied but isn't a known config.
>> 2016-05-06 20:02:42,801 INFO [pool-26-thread-24]
>> o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
>> PutKafka[id=24337410-7f3e-4fe0-8da4-5772399635e3] to run with 1 threads
>> 2016-05-06 20:02:43,698 ERROR [Timer-Driven Process Thread-9]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:44,457 INFO [Timer-Driven Process Thread-4]
>> o.a.nifi.processors.standard.SplitJson
>> SplitJson[id=857b6d94-c163-495a-8145-e5903830c02e] Split
>> StandardFlowFileRecord[uuid=bd250e09-7c81-4371-838b-b64319ee2792,claim=StandardContentClaim
>> [resourceClaim=StandardResourceClaim[id=1462543677866-4, container=default,
>> section=4], offset=888295,
>> length=791977],offset=0,name=10310751916797,size=791977] into 1000 FlowFiles
>> 2016-05-06 20:02:44,700 ERROR [Timer-Driven Process Thread-10]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:45,839 ERROR [Timer-Driven Process Thread-8]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:45,884 INFO [Provenance Maintenance Thread-2]
>> o.a.n.p.PersistentProvenanceRepository Created new Provenance Event Writers
>> for events starting with ID 123674
>> 2016-05-06 20:02:46,840 ERROR [Timer-Driven Process Thread-1]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:47,324 INFO [Provenance Repository Rollover Thread-2]
>> o.a.n.p.PersistentProvenanceRepository Successfully merged 16 journal files
>> (1554 records) into single Provenance Log File
>> ./provenance_repository/122120.prov in 1451 milliseconds
>> 2016-05-06 20:02:47,325 INFO [Provenance Repository Rollover Thread-2]
>> o.a.n.p.PersistentProvenanceRepository Successfully Rolled over Provenance
>> Event file containing 1338 records
>> 2016-05-06 20:02:47,841 ERROR [Timer-Driven Process Thread-9]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:48,843 ERROR [Timer-Driven Process Thread-1]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:49,844 ERROR [Timer-Driven Process Thread-5]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:50,845 ERROR [Timer-Driven Process Thread-3]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:53,021 ERROR [Timer-Driven Process Thread-8]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:54,023 ERROR [Timer-Driven Process Thread-9]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:55,024 ERROR [Timer-Driven Process Thread-7]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:56,026 ERROR [Timer-Driven Process Thread-6]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:57,027 ERROR [Timer-Driven Process Thread-9]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:58,029 ERROR [Timer-Driven Process Thread-1]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:59,031 ERROR [Timer-Driven Process Thread-1]
>> org.apache.nifi.hbase.GetHBase
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
>> from HBase due to java.net.ConnectException: Connection refused
>> 2016-05-06 20:02:59,559 INFO [NiFi Web Server-369]
>> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
>> InvokeHTTP[id=cd2576bd-5d03-4961-9547-48249b86aadd] to run
>> 2016-05-06 20:02:59,577 INFO [NiFi Web Server-17]
>> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
>> AttributesToJSON[id=bbd65940-8ba8-4c97-8658-3ba4ce835cf3] to run
>> 2016-05-06 20:02:59,819 INFO [NiFi Web Server-21]
>> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
>> SplitJson[id=857b6d94-c163-495a-8145-e5903830c02e] to run
>> 2016-05-06 20:02:59,822 INFO [NiFi Web Server-20]
>> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
>> PutHBaseJSON[id=dc18a7a7-dd2f-484c-aa27-4a6f0de65920] to run
>> 2016-05-06 20:02:59,869 INFO [NiFi Web Server-19]
>> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
>> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] to run
>> 2016-05-06 20:02:59,889 INFO [NiFi Web Server-18]
>> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
>> PutKafka[id=24337410-7f3e-4fe0-8da4-5772399635e3] to run
>> 2016-05-06 20:02:59,910 INFO [NiFi Web Server-461]
>> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
>> EvaluateJsonPath[id=aa7e0bb0-9948-48eb-9464-301c3b014392] to run
>> 2016-05-06 20:02:59,914 INFO [NiFi Web Server-369]
>> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
>> PutHBaseCell[id=04a3e270-375c-4f2d-9e93-ac53a70765d8] to run
>> 2016-05-06 20:03:00,068 INFO [Flow Service Tasks Thread-1]
>> o.a.nifi.controller.StandardFlowService Saved flow controller
>> org.apache.nifi.controller.FlowController@2c4e3ffb // Another save
>> pending = false
>> 2016-05-06 20:03:15,923 INFO [Provenance Maintenance Thread-2]
>> o.a.n.p.PersistentProvenanceRepository Created new Provenance Event Writers
>> for events starting with ID 128123
>> 2016-05-06 20:03:16,584 INFO [Provenance Repository Rollover Thread-1]
>> o.a.n.p.PersistentProvenanceRepository Successfully merged 16 journal files
>> (4449 records) into single Provenance Log File
>> ./provenance_repository/123674.prov in 698 milliseconds
>> 2016-05-06 20:03:16,584 INFO [Provenance Repository Rollover Thread-1]
>> o.a.n.p.PersistentProvenanceRepository Successfully Rolled over Provenance
>> Event file containing 2770 records
>> 2016-05-06 20:03:33,473 INFO [pool-18-thread-1]
>> o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile
>> Repository
>> 2016-05-06 20:03:33,638 INFO [pool-18-thread-1]
>> org.wali.MinimalLockingWriteAheadLog
>> org.wali.MinimalLockingWriteAheadLog@26650de4 checkpointed with 0
>> Records and 0 Swap Files in 163 milliseconds (Stop-the-world time = 32
>> milliseconds, Clear Edit Logs time = 35 millis), max Transaction ID 270109
>> 2016-05-06 20:03:33,638 INFO [pool-18-thread-1]
>> o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
>> Repository with 0 records in 164 milliseconds
>> 2016-05-06 20:03:34,057 INFO [FileSystemRepository Workers Thread-2]
>> o.a.n.c.repository.FileSystemRepository Successfully archived 1 Resource
>> Claims for Container default in 26 millis
>>
>> On Fri, May 6, 2016 at 7:50 PM, Bryan Bende <bbende@gmail.com> wrote:
>>
>>> Ok, thanks for sharing that...
>>>
>>> Can you look in NIFI_HOME/logs/nifi-app.log and see if you can find that
>>> error message that says "Failed to receive data from HBase due to",
>>> and then there should be a whole stack trace that goes with that.
>>>
>>> If you could paste that whole stacktrace here that would be helpful.
>>>
>>> Thanks,
>>>
>>> Bryan
>>>
>>> On Fri, May 6, 2016 at 10:14 AM, Venkatesh Bodapati <
>>> venkatesh.bodapati@inndata.in> wrote:
>>>
>>>> I am running NiFi, Hbase and Zookeeper in same machine.
>>>>
>>>> This my Hbase-site.xml :
>>>> <configuration>
>>>> <property>
>>>>                 <name>hbase.master</name>
>>>>                 <value>localhost:60000</value>
>>>>         </property>
>>>>         <property>
>>>>                 <name>hbase.rootdir</name>
>>>>                 <value>hdfs://localhost:8020/hbase</value>
>>>>         </property>
>>>>         <property>
>>>>                 <name>hbase.cluster.distributed</name>
>>>>                 <value>true</value>
>>>>         </property>
>>>>         <property>
>>>>                 <name>hbase.zookeeper.quorum</name>
>>>>                 <value>localhost</value>
>>>>         </property>
>>>>         <property>
>>>>                 <name>dfs.replication</name>
>>>>                 <value>1</value>
>>>>         </property>
>>>>         <property>
>>>>                 <name>hbase.zookeeper.property.clientPort</name>
>>>>                 <value>2181</value>
>>>>         </property>
>>>>         <property>
>>>>                 <name>hbase.zookeeper.property.dataDir</name>
>>>>                 <value>/usr/local/hadoop/hbase-1.1.2/zookeeper</value>
>>>>         </property>
>>>>         <property>
>>>>                 <name>zookeeper.znode.parent</name>
>>>>                 <value>/hbase</value>
>>>>         </property>
>>>> </configuration>
>>>>
>>>> and my Hbase-client Service properties like this :
>>>> Hadoop Configuration FilesInfo  :
>>>> /usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hbase-1.1.2/conf/hbase-site.xml
>>>> ZooKeeper QuorumInfo            : No value set
>>>> ZooKeeper Client PortInfo        : No value set
>>>> ZooKeeper ZNode ParentInfo   : /hbase
>>>> HBase Client RetriesInfo           : 1
>>>>
>>>> On Fri, May 6, 2016 at 7:13 PM, Bryan Bende <bbende@gmail.com> wrote:
>>>>
>>>>> Can you describe your setup a little bit more, things like...
>>>>>
>>>>> Is NiFi running on the same machine that HBase and ZooKeeper are
>>>>> running on (it doesn't have to be, just want to understand the setup) ?
>>>>>
>>>>> What values do you have in hbase-site.xml for properties like...
>>>>> - hbase.zookeeper.quorum
>>>>> - hbase.zookeeper.property.clientPort
>>>>>
>>>>> Here is a template that is working for me:
>>>>>
>>>>> https://gist.githubusercontent.com/bbende/9457b5ed261e6eeb0f98995a5a2699e0/raw/38e050828c2545cb50e623d6dcf45dbbc9ad1d9d/FunWithHBaseUpdated.xml
>>>>>
>>>>> I am running NiFi on the HDP Sandbox so everything is local for me.
>>>>>
>>>>>
>>>>> On Fri, May 6, 2016 at 8:54 AM, Venkatesh Bodapati <
>>>>> venkatesh.bodapati@inndata.in> wrote:
>>>>>
>>>>>>   " Missing Row id failure, routing to failure " issue solved .
>>>>>> Still i got the "failed to receive data from Hbase due to java net
>>>>>> connection exception :Connection Refused" error in GetHbase Processor.  I
>>>>>> update Hbase-Site.xml and remove remaining properties like
>>>>>> " Zookeeper Quorum Info,Zookeeper Client Port Info . still i got the same
>>>>>> error in GetHbase processor.
>>>>>>
>>>>>> On Thu, May 5, 2016 at 9:07 PM, Bryan Bende <bbende@gmail.com> wrote:
>>>>>>
>>>>>>> I was looking at this a little more and instead of needing the
>>>>>>> RouteOnAttribute, I think you can just update EvaluateJsonPath properties...
>>>>>>>
>>>>>>> Based on the JSON I see coming in, they should be:
>>>>>>>
>>>>>>> email = $.email
>>>>>>> firstName = $.name.first
>>>>>>> lastName = $.name.last
>>>>>>> ssn = $.id.value
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, May 5, 2016 at 11:30 AM, Bryan Bende <bbende@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> For HBaseClient, are you sure your ZooKeeper being used by HBase is
>>>>>>>> running on localhost:2181?
>>>>>>>>
>>>>>>>> Typically you don't really need to set the three ZooKeeper
>>>>>>>> properties, and you can instead just set the hbase-site.xml in the config
>>>>>>>> resources.
>>>>>>>>
>>>>>>>> For example, my Hadoop Configuration Resources is set as:
>>>>>>>> /etc/hadoop/conf/core-site.xml,/etc/hbase/conf/hbase-site.xml
>>>>>>>>
>>>>>>>> And then I don't have anything specified for ZooKeeper QuorumInfo,
>>>>>>>> etc., because hbase-site.xml is providing all that information.
>>>>>>>>
>>>>>>>> The example template requires a Users table, so from Hbase shell:
>>>>>>>> create 'Users', {NAME => 'cf'}
>>>>>>>>
>>>>>>>> Then the template has a slight problem where it is using
>>>>>>>> EvaluteJsonPath to extract an attribute called "ssn" and using that for the
>>>>>>>> row id... the problem is a lot of the ssn values are empty so they are
>>>>>>>> failing.
>>>>>>>>
>>>>>>>> If you stick a RouteOnAttribute right after EvaluateJsonPath, and
>>>>>>>> add a property "matched" = ${ssn:isEmpty():not()} that will filter out all
>>>>>>>> the empty SSNs, then connect RouteOnAttribute's matched relationship to the
>>>>>>>> HBase processors.
>>>>>>>>
>>>>>>>> Lets get the inserting side working and then I would expect the
>>>>>>>> GetHBase to work since they use the same underlying connection.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Bryan
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, May 5, 2016 at 10:59 AM, Venkatesh Bodapati <
>>>>>>>> venkatesh.bodapati@inndata.in> wrote:
>>>>>>>>
>>>>>>>>> I am working on "Fun_With_Hbase.xml" Template. In this i will send
>>>>>>>>> data to hbase table, but i will not get data from hbase. I will get the
>>>>>>>>> "failed to receive data from hbase due to java net connection exception
>>>>>>>>> :Connection Refused" error in GetHbase Processor and In
>>>>>>>>> PutHbaseCell,PutHbaseJson Processor i got error Like This " Missing Row id
>>>>>>>>> failure, routing to failure ".
>>>>>>>>>
>>>>>>>>> This is my HbaseClient Properties :
>>>>>>>>>
>>>>>>>>> Hadoop Configuration FilesInfo  :
>>>>>>>>> /usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/hdfs-site.xml
>>>>>>>>> ZooKeeper QuorumInfo            : localhost:2181
>>>>>>>>> ZooKeeper Client PortInfo        : 2181
>>>>>>>>> ZooKeeper ZNode ParentInfo   : /hbase
>>>>>>>>> HBase Client RetriesInfo           : 1
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> How do i get data from Hbase Table in Nifi,Any suggesions please.
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message