sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mariappan Asokan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-1272) Support importing mainframe sequential datasets
Date Sat, 14 Jun 2014 23:21:01 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031734#comment-14031734

Mariappan Asokan commented on SQOOP-1272:

      Thanks for your suggestion.  I was able to send a review request.  To answer your questions:
Any reason MainframeManager inherits from the deprecated com.cloudera.sqoop.manager.ConnManager
and not from org.apache.sqoop.manager.ConnManager? I noticed you also use the deprecated SqoopOptions.
I will change MainframeManager to inherit from org.apache.sqoop.manager.ConnManager.  It is
not a problem.  However, I still have to use deprecated SqoopOptions because importing to
HBase or Accumulo requires using HBaseImportJob or AccumuloImportJob respectively.  The constructors
for these classes can take only deprecated SqoopOptions.
MainframeFTPClientUtils depends on org.apache.commons.net. I didn't see commons-net added
as a dependency to ivy.xml.
When the dependencies for Apache Hadoop jar files are picked up, commons-net is picked up
automatically.  If you think it is a good idea to update ivy.xml in Sqoop to make it independent
of that, I will add it to ivy.xml.

    To answer your questions:
Can we add documentation (stored in src/docs/user) for the new tool?
Good suggestion.  I will add it in the next version of the patch.
Can move files Mainframe* from org.apache.sqoop.mapreduce.* to a special sub-package mainframe
forming org.apache.sqoop.mapreduce.mainframe?
I thought about it initially.  However, it was not clear whether I should do it considering
the existing directory structure.  There are several *ImportJob.java, *InputFormat.java, and
*Mapper.java classes already in org.apache.sqoop.mapreduce.  Sure, I can create org.apache.sqoop.mapreduce.mainframe
if it avoids cluttering of org.apache.sqoop.mapreduce.

> Support importing mainframe sequential datasets
> -----------------------------------------------
>                 Key: SQOOP-1272
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1272
>             Project: Sqoop
>          Issue Type: New Feature
>          Components: connectors
>    Affects Versions: 1.4.4
>            Reporter: Mariappan Asokan
>              Labels: Mainframe
>             Fix For: 1.4.5
>         Attachments: 1272.patch, MainframeImport.pdf, MainframeImport.pdf
> There is a growing need to move data from mainframe to HDFS.  This Jira proposes to enhance
Sqoop to support moving a set of sequential mainframe datasets to HDFS.  The attached document
describes a design for this enhancement.

This message was sent by Atlassian JIRA

View raw message