flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5518) HadoopInputFormat throws NPE when close() is called before open()
Date Tue, 17 Jan 2017 08:42:26 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15825652#comment-15825652

ASF GitHub Bot commented on FLINK-5518:

GitHub user jakubhavlik opened a pull request:


    [FLINK-5518] [input/output formats] Add null value check.

    Thanks for contributing to Apache Flink. Before you open your pull request, please take
the following check list into consideration.
    If your changes take all of the items into account, feel free to open your pull request.
For more information and/or questions please refer to the [How To Contribute guide](http://flink.apache.org/how-to-contribute.html).
    In addition to going through the list, please provide a meaningful description of your
    - [ ] General
      - The pull request references the related JIRA issue ("[FLINK-XXX] Jira title text")
      - The pull request addresses only one issue
      - Each commit in the PR has a meaningful commit message (including the JIRA id)
    - [ ] Documentation
      - Documentation has been added for new functionality
      - Old documentation affected by the pull request has been updated
      - JavaDoc for public methods has been added
    - [ ] Tests & Build
      - Functionality added by the pull request is covered by tests
      - `mvn clean verify` has been executed successfully locally or a Travis build has passed

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/jakubhavlik/flink havlikj/flink_5518

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3133
commit 984b50fa0d85ab42c47692413d0bad331b363cab
Author: Jakub Havlik <jakub.havlik@brain-station.com>
Date:   2017-01-17T07:26:07Z

    [FLINK-5518] [input/output formats] Add null value check.


> HadoopInputFormat throws NPE when close() is called before open()
> -----------------------------------------------------------------
>                 Key: FLINK-5518
>                 URL: https://issues.apache.org/jira/browse/FLINK-5518
>             Project: Flink
>          Issue Type: Bug
>          Components: Batch Connectors and Input/Output Formats
>    Affects Versions: 1.2.0, 1.1.4
>            Reporter: Jakub Havlik
>              Labels: beginner, easyfix, newbie
> When developing a simple Flink applications reading ORC files it crashes with NullPointerException
when number of instances/executor threads is higher then the number of files because it is
trying to close a HadoopInputFormat which is trying to close RecordReader which was not yet
initialized as there is no file for which it should have been opened. The issue is caused
> {code:java}
> public void run(SourceContext<OUT> ctx) throws Exception {
>     try {
> ...
>         while (isRunning) {
> 	    format.open(splitIterator.next());
> ...
>     } finally {
> 	format.close();
> ...
>     }
> {code}
> in file {{InputFormatSourceFunction.java}} which calls
> {code:java}
> public void close() throws IOException {
>     // enforce sequential close() calls
>     synchronized (CLOSE_MUTEX) {
>         this.recordReader.close();
>     }
> }
> {code}
> from {{HadoopInputFormatBase.java}}.
> As there is just this one implementation of the {{close()}} method it may be enough just
to add a null check for the {{this.recordReader}} in there.

This message was sent by Atlassian JIRA

View raw message