sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-3225) Mainframe module FTP listing parser should cater for larger datasets on disk
Date Tue, 24 Oct 2017 13:17:00 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16216889#comment-16216889
] 

Hudson commented on SQOOP-3225:
-------------------------------

FAILURE: Integrated in Jenkins build Sqoop-hadoop200 #1132 (See [https://builds.apache.org/job/Sqoop-hadoop200/1132/])
SQOOP-3225: Mainframe module FTP listing parser should cater for larger (bogi: [https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=e8588e243ea3f158193b4decfe5884620ec0f021])
* (edit) src/java/org/apache/sqoop/util/MainframeFTPClientUtils.java
* (edit) src/test/org/apache/sqoop/util/TestMainframeFTPClientUtils.java


> Mainframe module FTP listing parser should cater for larger datasets on disk
> ----------------------------------------------------------------------------
>
>                 Key: SQOOP-3225
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3225
>             Project: Sqoop
>          Issue Type: Improvement
>            Reporter: Chris Teoh
>            Assignee: Chris Teoh
>            Priority: Minor
>         Attachments: SQOOP-3225.patch
>
>
> The parser can handle smaller datasets but when a dataset reaches a certain size, the
FTP listing changes such that the current FTP parser doesn't read the listing correctly. Current
workaround is to move the datasets to tape. Sample is below of some tape based datasets followed
by a large dataset followed by a small dataset:-
> {{ftp> ls
> 227 Entering Passive Mode (Xx,XX,XXX,XXX,XX,XXX)
> 125 List started OK
> Volume Unit    Referred Ext Used Recfm Lrecl BlkSz Dsorg Dsname
> Migrated                                                G1622V00
> Migrated                                                G1623V00
> Migrated                                                G1624V00
> Migrated                                                G1625V00
> Migrated                                                G1626V00
> Migrated                                                G1627V00
> Migrated                                                G1628V00
> Migrated                                                G1629V00
> Migrated                                                G1630V00
> Migrated                                                G1631V00
> Migrated                                                G1632V00
> Migrated                                                G1633V00
> Migrated                                                G1634V00
> Migrated                                                G1635V00
> Migrated                                                G1636V00
> Migrated                                                G1637V00
> Migrated                                                G1638V00
> Migrated                                                G1639V00
> Migrated                                                G1640V00
> xxxx39 3390   2017/08/02  633540  FB     847 27951  PS  G1641V00
> xxxx09 3390   2017/08/02  1   15  FB     132 27984  PS  OUTPUT}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message