sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mariappan Asokan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-1272) Support importing mainframe sequential datasets
Date Sat, 14 Jun 2014 23:21:01 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031734#comment-14031734
] 

Mariappan Asokan commented on SQOOP-1272:
-----------------------------------------

Gwen,
      Thanks for your suggestion.  I was able to send a review request.  To answer your questions:
{quote}
Any reason MainframeManager inherits from the deprecated com.cloudera.sqoop.manager.ConnManager
and not from org.apache.sqoop.manager.ConnManager? I noticed you also use the deprecated SqoopOptions.
{quote}
I will change MainframeManager to inherit from org.apache.sqoop.manager.ConnManager.  It is
not a problem.  However, I still have to use deprecated SqoopOptions because importing to
HBase or Accumulo requires using HBaseImportJob or AccumuloImportJob respectively.  The constructors
for these classes can take only deprecated SqoopOptions.
{quote}
MainframeFTPClientUtils depends on org.apache.commons.net. I didn't see commons-net added
as a dependency to ivy.xml.
{quote}
When the dependencies for Apache Hadoop jar files are picked up, commons-net is picked up
automatically.  If you think it is a good idea to update ivy.xml in Sqoop to make it independent
of that, I will add it to ivy.xml.

Jarcec,
    To answer your questions:
{quote}
Can we add documentation (stored in src/docs/user) for the new tool?
{quote}
Good suggestion.  I will add it in the next version of the patch.
{quote}
Can move files Mainframe* from org.apache.sqoop.mapreduce.* to a special sub-package mainframe
forming org.apache.sqoop.mapreduce.mainframe?
{quote}
I thought about it initially.  However, it was not clear whether I should do it considering
the existing directory structure.  There are several *ImportJob.java, *InputFormat.java, and
*Mapper.java classes already in org.apache.sqoop.mapreduce.  Sure, I can create org.apache.sqoop.mapreduce.mainframe
if it avoids cluttering of org.apache.sqoop.mapreduce.

> Support importing mainframe sequential datasets
> -----------------------------------------------
>
>                 Key: SQOOP-1272
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1272
>             Project: Sqoop
>          Issue Type: New Feature
>          Components: connectors
>    Affects Versions: 1.4.4
>            Reporter: Mariappan Asokan
>              Labels: Mainframe
>             Fix For: 1.4.5
>
>         Attachments: 1272.patch, MainframeImport.pdf, MainframeImport.pdf
>
>
> There is a growing need to move data from mainframe to HDFS.  This Jira proposes to enhance
Sqoop to support moving a set of sequential mainframe datasets to HDFS.  The attached document
describes a design for this enhancement.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message