flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shashank Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8182) Unable to read hdfs file system directory(which contains sub directories) recursively
Date Fri, 01 Dec 2017 10:25:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274215#comment-16274215
] 

Shashank Agarwal commented on FLINK-8182:
-----------------------------------------

Actually it's unable to access listing after the change to Debug logs its printing continues
following logs :


{code}
15:49:44,713 DEBUG org.apache.hadoop.ipc.Client                                  - Connecting
to {masked}/{masked}::8020
15:49:45,302 DEBUG org.apache.hadoop.ipc.Client                                  - IPC Client
(428622004) connection to {masked}/{masked}:8020 from shashank: starting, having connections
1
15:49:45,307 DEBUG org.apache.hadoop.ipc.Client                                  - IPC Client
(428622004) connection to {masked}/{masked}::8020 from shashank sending #0
15:49:47,292 DEBUG org.apache.hadoop.ipc.Client                                  - IPC Client
(428622004) connection to {masked}/{masked}::8020 from shashank got value #0
15:49:47,292 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine                       - Call: getFileInfo
took 2598ms
15:49:47,320 DEBUG org.apache.hadoop.ipc.Client                                  - IPC Client
(428622004) connection to {masked}/{masked}::8020 from shashank sending #1
15:49:47,454 DEBUG org.apache.flink.runtime.taskmanager.TaskManager              - Sending
heartbeat to JobManager
15:49:48,278 DEBUG org.apache.hadoop.ipc.Client                                  - IPC Client
(428622004) connection to {masked}/{masked}::8020 from shashank got value #1
15:49:48,279 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine                       - Call: getListing
took 958ms
15:49:48,303 DEBUG org.apache.hadoop.ipc.Client                                  - IPC Client
(428622004) connection to {masked}/{masked}::8020 from shashank sending #2
{code}


> Unable to read hdfs file system directory(which contains sub directories) recursively
 
> ---------------------------------------------------------------------------------------
>
>                 Key: FLINK-8182
>                 URL: https://issues.apache.org/jira/browse/FLINK-8182
>             Project: Flink
>          Issue Type: Improvement
>          Components: Streaming
>    Affects Versions: 1.3.2
>            Reporter: Shashank Agarwal
>
> Unable to read hdfs file system directory(which contains subdirectories) recursively,
It works fine when a single directory contains only files but when the directory contains
subdirectories it dosesn't read subdirectory files.
> {code}
> streamExecutionEnvironment.readTextFile("HDFS path")
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message