hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6900) FileSystem#listLocatedStatus API needs changes to indicate complete set of error conditions
Date Wed, 04 Aug 2010 23:40:18 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12895470#action_12895470

Suresh Srinivas commented on HADOOP-6900:

Iterator#hasNext() has to indicate the following error conditions:
# next element exists - returns true
# next element does not exist - returns false
# could not get status for the next element, because a directory that was available before
iteration started does not exist any more.
# could not talk to the remote server to get status.
# remote server throws IOException due to an error while getting the status.

For the last 3 conditions, currently RuntimeException is thrown. Throwing RTE forces application
to catch it, to handle the error condition. This results in application catching other RTE
as well. Worse still, an application that does not catch RTE might exit.

The problem is, Java Iterator interface is not suitable for implementation that use RPC underneath.
We should have a different interface, which could indicate error conditions more clearly.

Additionally I feel when a directory gets deleted in the middle of iteration, we should ignore
it, or add provision to the iterator to ignore it.

> FileSystem#listLocatedStatus API needs changes to indicate complete set of error conditions
> -------------------------------------------------------------------------------------------
>                 Key: HADOOP-6900
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6900
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Suresh Srinivas
> HDFS-6870 introduced FileSystem#listLocatedStatus(), that returns an iterator to iterate
through LocatedFileStatus for files under a directory or recursively under a sub-directory.
Through the iterator application does not get all the error conditions.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message