sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-1532) Sqoop2: Support Sqoop on Spark Execution Engine
Date Tue, 15 Dec 2015 07:13:46 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15057480#comment-15057480

ASF GitHub Bot commented on SQOOP-1532:

Github user jarcec commented on the pull request:

    Sqoop project do not accept pull requests at this point. Would you mind attaching the
patch the JIRA itself?

> Sqoop2: Support Sqoop on Spark Execution Engine
> -----------------------------------------------
>                 Key: SQOOP-1532
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1532
>             Project: Sqoop
>          Issue Type: Improvement
>            Reporter: Veena Basavaraj
>            Assignee: Veena Basavaraj
>             Fix For: 2.0.0
> The current execution engine supported in sqoop is MR.
> The goal if this ticket is to support sqoop jobs ( map only and map+reduce ) to run on
spark environment.
> It should at the minimum support running on the standalone spark cluster and then subsequently
work with YARN/mesos.
> High level goals
> 1. Hook up with the connector apis to provide the basic load/ extract to the spark RDD.
> 2. Implementation of the Sqoop RDD to support extraction from different data sources
. The design proposal will discuss the alternatives on how this can be achieved.
> 3. Optimizing the loading/writing ( re-use/ refactor the consumer thread code to be agnostic
of the hadoop output format)

This message was sent by Atlassian JIRA

View raw message