samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lhaiesp <>
Subject [GitHub] samza pull request #633: SAMZA-1870: hdfs offset comparator to handle end of...
Date Tue, 11 Sep 2018 15:51:03 GMT
GitHub user lhaiesp opened a pull request:

    SAMZA-1870: hdfs offset comparator to handle end of stream offset

    This happens particularly when using HDFS as a bootstrap stream:
    org.apache.samza.SamzaException: Invalid offset for MultiFileHdfsReader: END_OF_STREAM
    at org.apache.samza.system.hdfs.reader.MultiFileHdfsReader.getCurFileIndex(
    at org.apache.samza.system.hdfs.HdfsSystemAdmin.offsetComparator(
    at org.apache.samza.system.chooser.BootstrappingChooser.choose(BootstrappingChooser.scala:204)
    at org.apache.samza.system.chooser.DefaultChooser.choose(DefaultChooser.scala:294)
    at org.apache.samza.system.SystemConsumers.choose(SystemConsumers.scala:210)
    at org.apache.samza.task.AsyncRunLoop.chooseEnvelope(
    at org.apache.samza.runtime.LocalContainerRunner.main(

You can merge this pull request into a Git repository by running:

    $ git pull master

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #633
commit 42b80cc34a999955b79997494fe078f8024c9c2c
Author: Hai Lu <halu@...>
Date:   2018-09-11T15:49:51Z

    hdfs offset comparator to handle end of stream offset



View raw message