sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Teoh (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SQOOP-3326) Mainframe FTP listing for GDG should filter out non-GDG datasets in a heterogeneous listing
Date Tue, 29 May 2018 00:36:00 GMT

     [ https://issues.apache.org/jira/browse/SQOOP-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Teoh updated SQOOP-3326:
------------------------------
    Description: 
The FTP listing will automatically assume the first file in the listing is the most recent
GDG file. This is a problem when there are mixed datasets in the listing that the FTP listing
doesn't filter these out.

 

GDG base is : HLQ.ABC.DEF.AB15HUP

 

The sequential dataset in the middle of the GDG member listing is : HLQ.ABC.DEF.AB15HUP.DATA

 

The pattern for listing GDG members should be : <<GDG base name>>.G\d\{4}V\d\{2}

 

 Sample below:-

   Menu  Options  View  Utilities  Compilers  Help                             

 ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss

 DSLIST - Data Sets Matching HLQ.ABC.DEF.GDGBASE               Row 1 of 8

 Command ===>                                                  Scroll ===> PAGE

                                                                                

 Command - Enter "/" to select action                  Message           Volume

 -------------------------------------------------------------------------------

          HLQ.ABC.DEF.GDGBASE                                      ??????

          HLQ.ABC.DEF.GDGBASE.DUMMY                                SHT331

          HLQ.ABC.DEF.GDGBASE.G0034V00                             H19761

          HLQ.ABC.DEF.GDGBASE.G0035V00                             H81751

          HLQ.ABC.DEF.GDGBASE.G0035V00.COPY                        SHT337

          HLQ.ABC.DEF.GDGBASE.G0036V00                             H73545

          HLQ.ABC.DEF.GDGBASE.G0037V00                             G10987

          HLQ.ABC.DEF.GDGBASE.HELLO                                SHT33A

 ***************************** End of Data Set list ****************************


ftp> open some.machine.network.zxc.au

Connected to some.machine.network.zxc.au (11.22.33.44).

220-TCPFTP01 IBM FTP CS V2R1 at some.machine.network.zxc.au, 00:12:29 on 2018-05-29.

220 Connection will close if idle for more than 10 minutes.

Name (some.machine.network.zxc.au:someuser):

331 Send password please.

Password:

230 someuser is logged on.  Working directory is "someuser.".

Remote system type is MVS.

ftp> cd  'HLQ.ABC.DEF.GDGBASE'

250 "HLQ.ABC.DEF.GDGBASE." is the working directory name prefix.

ftp> dir

227 Entering Passive Mode (11,22,33,44,55,66)

125 List started OK

Volume Unit    Referred Ext Used Recfm Lrecl BlkSz Dsorg Dsname

H19761 Tape                                             G0034V00

H81751 Tape                                             G0035V00

H73545 Tape                                             G0036V00

G10987 Tape                                             G0037V00

SHT331 3390   **NONE**    1   15  VB     114 27998  PS  DUMMY

SHT337 3390   **NONE**    1   15  VB     114 27998  PS  G0035V00.COPY

SHT33A 3390   **NONE**    1   15  VB     114 27998  PS  HELLO

250 List completed successfully.

ftp>

  was:
The FTP listing will automatically assume the first file in the listing is the most recent
GDG file. This is a problem when there are mixed datasets in the listing that the FTP listing
doesn't filter these out.

 

GDG base is : HLQ.ABC.DEF.AB15HUP

 

The sequential dataset in the middle of the GDG member listing is : HLQ.ABC.DEF.AB15HUP.DATA

 

The pattern for listing GDG members should be : <<GDG base name>>.G\d\{4}V\d\{2}

 

 

  Menu  Options  View  Utilities  Compilers  Help                             

ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss

DSLIST - Data Sets Matching HLQ.ABC.DEF.AB15HUP Empty data set or member

Command ===>                                                 
Scroll ===> PAGE

                                                                               

Command - Enter "/" to select action                  Message          
Volume

-------------------------------------------------------------------------------

         HLQ.ABC.DEF.AB15HUP                                     
??????

         HLQ.ABC.DEF.AB15HUP.DATA                                 ZXC344+

         HLQ.ABC.DEF.AB15HUP.G0007V00                            
H54924

         HLQ.ABC.DEF.AB15HUP.G0008V00                            
G54837

         HLQ.ABC.DEF.AB15HUP.G0009V00                            
G53709

         HLQ.ABC.DEF.AB15HUP.G0010V00                            
G27559

***************************** End of Data Set list ****************************


> Mainframe FTP listing for GDG should filter out non-GDG datasets in a heterogeneous listing
> -------------------------------------------------------------------------------------------
>
>                 Key: SQOOP-3326
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3326
>             Project: Sqoop
>          Issue Type: Improvement
>            Reporter: Chris Teoh
>            Assignee: Chris Teoh
>            Priority: Minor
>
> The FTP listing will automatically assume the first file in the listing is the most recent
GDG file. This is a problem when there are mixed datasets in the listing that the FTP listing
doesn't filter these out.
>  
> GDG base is : HLQ.ABC.DEF.AB15HUP
>  
> The sequential dataset in the middle of the GDG member listing is : HLQ.ABC.DEF.AB15HUP.DATA
>  
> The pattern for listing GDG members should be : <<GDG base name>>.G\d\{4}V\d\{2}
>  
>  Sample below:-
>    Menu  Options  View  Utilities  Compilers  Help                             
>  ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
>  DSLIST - Data Sets Matching HLQ.ABC.DEF.GDGBASE               Row 1 of 8
>  Command ===>                                                  Scroll ===> PAGE
>                                                                                 
>  Command - Enter "/" to select action                  Message           Volume
>  -------------------------------------------------------------------------------
>           HLQ.ABC.DEF.GDGBASE                                      ??????
>           HLQ.ABC.DEF.GDGBASE.DUMMY                                SHT331
>           HLQ.ABC.DEF.GDGBASE.G0034V00                             H19761
>           HLQ.ABC.DEF.GDGBASE.G0035V00                             H81751
>           HLQ.ABC.DEF.GDGBASE.G0035V00.COPY                        SHT337
>           HLQ.ABC.DEF.GDGBASE.G0036V00                             H73545
>           HLQ.ABC.DEF.GDGBASE.G0037V00                             G10987
>           HLQ.ABC.DEF.GDGBASE.HELLO                                SHT33A
>  ***************************** End of Data Set list ****************************
> ftp> open some.machine.network.zxc.au
> Connected to some.machine.network.zxc.au (11.22.33.44).
> 220-TCPFTP01 IBM FTP CS V2R1 at some.machine.network.zxc.au, 00:12:29 on 2018-05-29.
> 220 Connection will close if idle for more than 10 minutes.
> Name (some.machine.network.zxc.au:someuser):
> 331 Send password please.
> Password:
> 230 someuser is logged on.  Working directory is "someuser.".
> Remote system type is MVS.
> ftp> cd  'HLQ.ABC.DEF.GDGBASE'
> 250 "HLQ.ABC.DEF.GDGBASE." is the working directory name prefix.
> ftp> dir
> 227 Entering Passive Mode (11,22,33,44,55,66)
> 125 List started OK
> Volume Unit    Referred Ext Used Recfm Lrecl BlkSz Dsorg Dsname
> H19761 Tape                                             G0034V00
> H81751 Tape                                             G0035V00
> H73545 Tape                                             G0036V00
> G10987 Tape                                             G0037V00
> SHT331 3390   **NONE**    1   15  VB     114 27998  PS  DUMMY
> SHT337 3390   **NONE**    1   15  VB     114 27998  PS  G0035V00.COPY
> SHT33A 3390   **NONE**    1   15  VB     114 27998  PS  HELLO
> 250 List completed successfully.
> ftp>



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message