hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12994) Specify PositionedReadable, add contract tests, fix problems
Date Thu, 07 Apr 2016 19:15:25 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15230857#comment-15230857

Steve Loughran commented on HADOOP-12994:

As an aside, I was thinking we could just modify {{FSInputStreamread(long position, byte[]
buffer, int offset, int length))}} to catch an EOF exception and downgrade.

With azure, there's now two classes that handle EOF specially; I think swift will need it
too. Put it all in one place, and there's one code path that everything picks up.

*I can't see how, on any FS, catching an EOF exception on the seek or read and downgrading
to a -1 would be a mistake*

Remember: the param validation of seeking < 0 has already taken place, so the only way
an EOF could be raised is if they went off the end and the FS raises EOFs here. For which
catch and downgrade is the required action

> Specify PositionedReadable, add contract tests, fix problems
> ------------------------------------------------------------
>                 Key: HADOOP-12994
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12994
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs
>    Affects Versions: 2.8.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: HADOOP-12994-001.patch, HADOOP-12994-002.patch, HADOOP-12994-003.patch
> Some work on S3a has shown up that there aren't tests catching regressions in readFully,
reviewing the documentation shows that its specification could be improved.
> # review the spec
> # review the implementations
> # add tests (proposed: to the seek contract; streams which support seek should support
positioned readable)
> # fix code, where it differs significantly from HDFS or LocalFS

This message was sent by Atlassian JIRA

View raw message