kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prabhu V <vpra...@gmail.com>
Subject kafka-connect-hdfs failure due to corrupt WAL file
Date Tue, 02 Aug 2016 22:12:24 GMT

I am using kafka-connect-hdfs in a 2 nodes and one of the nodes had to be
rebooted when the process was running.

Upon restart the process fails with

16/08/02 21:43:30 ERROR hdfs.TopicPartitionWriter: Recovery failed at state
org.apache.kafka.connect.errors.ConnectException: java.io.IOException:
Filesystem closed
        at io.confluent.connect.hdfs.wal.FSWAL.apply(FSWAL.java:131)

I looked into the code and looks like when the WAL log file may be
corrupt/unreadeable, it causes this issue.

Can I delete the WAL files to fix this issue ? I dont mind duplicate events.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message