nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Gresock <jgres...@gmail.com>
Subject Re: Recovering corrupt flowfile repo
Date Wed, 21 Nov 2018 00:26:01 GMT
Ok, I finally found the jar and got it working using java -cp instead of
java -jar.  However, I suspect this procedure might not meet my particular
case, because it's looking for parition-* directories, and I don't have
those.  Perhaps the flowfile repo implementation I'm using is different
from the one for which that tool (
https://github.com/apache/nifi/blob/rel/nifi-1.6.0/nifi-toolkit/nifi-toolkit-flowfile-repo/src/main/java/org/apache/nifi/toolkit/repos/flowfile/RepairCorruptedFileEndings.java)
applies:

# My configuration
nifi.flowfile.repository.implementation=org.apache.nifi.controller.repository.WriteAheadFlowFileRepository
nifi.flowfile.repository.wal.implementation=org.apache.nifi.wali.SequentialAccessWriteAheadLog

So, here is the actual error I'm getting, in case anyone can help me work
through it.  I don't want to give up on the flowfile repo, because this was
production data.

2018-11-20 20:07:53,901 INFO [main]
o.a.n.wali.SequentialAccessWriteAheadLog Successfully recovered 249151
records and 10 swap files from Snapshot at
/data/nifi/flowfile_repository/checkpoint with Max Transaction ID of
4233134189 in 8593 milliseconds. Now recovering records from 1 journal files
2018-11-20 20:07:53,907 INFO [main] o.a.nifi.wali.LengthDelimitedJournal
Recovering records from journal
/data/nifi/flowfile_repository/journals/4233134190.journal
2018-11-20 20:07:54,592 INFO [main] o.a.nifi.wali.LengthDelimitedJournal
6.78% of the way finished recovering journal
/data/nifi/flowfile_repository/journals/4233134190.journal, having
recovered 15845 updates

... skipping some lines

018-11-20 20:08:05,363 INFO [main] o.a.nifi.wali.LengthDelimitedJournal
88.09% of the way finished recovering journal
/data/nifi/flowfile_repository/journals/4233134190.journal, having
recovered 322778 updates
2018-11-20 20:08:06,054 INFO [main] o.a.nifi.wali.LengthDelimitedJournal
94.89% of the way finished recovering journal
/data/nifi/flowfile_repository/journals/4233134190.journal, having
recovered 352084 updates
2018-11-20 20:08:06,576 ERROR [main]
o.a.nifi.controller.StandardFlowService Failed to load flow from cluster
due to: org.apache.nifi.cluster.ConnectionException: Failed to connect node
to cluster due to: java.io.IOException: Expected to read a Sentinel Byte of
'1' but got a value of '64' instead
org.apache.nifi.cluster.ConnectionException: Failed to connect node to
cluster due to: java.io.IOException: Expected to read a Sentinel Byte of
'1' but got a value of '64' instead
        at
org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:946)
        at
org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:516)
        at
org.apache.nifi.web.server.JettyServer.start(JettyServer.java:872)
        at org.apache.nifi.NiFi.<init>(NiFi.java:157)
        at org.apache.nifi.NiFi.<init>(NiFi.java:71)
        at org.apache.nifi.NiFi.main(NiFi.java:292)
Caused by: java.io.IOException: Expected to read a Sentinel Byte of '1' but
got a value of '64' instead
        at
org.apache.nifi.repository.schema.SchemaRecordReader.readRecord(SchemaRecordReader.java:65)
        at
org.apache.nifi.controller.repository.SchemaRepositoryRecordSerde.deserializeRecord(SchemaRepositoryRecordSerde.java:124)
        at
org.apache.nifi.controller.repository.SchemaRepositoryRecordSerde.deserializeEdit(SchemaRepositoryRecordSerde.java:109)
        at
org.apache.nifi.controller.repository.SchemaRepositoryRecordSerde.deserializeEdit(SchemaRepositoryRecordSerde.java:46)
        at
org.apache.nifi.wali.LengthDelimitedJournal.recoverRecords(LengthDelimitedJournal.java:335)
        at
org.apache.nifi.wali.SequentialAccessWriteAheadLog.recoverRecords(SequentialAccessWriteAheadLog.java:198)
        at
org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.loadFlowFiles(WriteAheadFlowFileRepository.java:545)
        at
org.apache.nifi.controller.FlowController.initializeFlow(FlowController.java:746)
        at
org.apache.nifi.controller.StandardFlowService.initializeController(StandardFlowService.java:956)
        at
org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:928)
        ... 5 common frames omitted
2018-11-20 20:08:06,576 INFO [main] o.a.n.c.c.node.NodeClusterCoordinator
ip-172-31-55-35.ec2.internal:8443 requested disconnection from cluster due
to org.apache.nifi.cluster.ConnectionException: Failed to connect node to
cluster due to: java.io.IOException: Expected to read a Sentinel Byte of
'1' but got a value of '64' instead

Thanks,
Joe


On Tue, Nov 20, 2018 at 8:18 PM Joe Gresock <jgresock@gmail.com> wrote:

> Hi,
>
> I'm trying to restore a corrupt flowfile repo using these instructions:
> https://community.hortonworks.com/content/supportkb/149943/errorjavaioioexception-expected-to-read-a-sentinel.html
>
> I get basically the same error noted on that post.  However, the
> instructions reference a jar called
>
> nifi-toolkit-flowfile-repo-1.2.0-SNAPSHOT-jar-with-dependencies.jar
>
> I downloaded nifi-toolkit 1.6.0 (since that's the nifi version I'm using), and I see
a nifi-toolkit-flowanalyzer-1.6.0.jar,
> but nothing called nifi-toolkit-flowfile-repo.
>
> Does anyone know where this tool lives now?
>
> Thanks,
>
> Joe
>
>
> --
> I know what it is to be in need, and I know what it is to have plenty.  I
> have learned the secret of being content in any and every situation,
> whether well fed or hungry, whether living in plenty or in want.  I can
> do all this through him who gives me strength.    *-Philippians 4:12-13*
>


-- 
I know what it is to be in need, and I know what it is to have plenty.  I
have learned the secret of being content in any and every situation,
whether well fed or hungry, whether living in plenty or in want.  I can do
all this through him who gives me strength.    *-Philippians 4:12-13*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message