sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Veena Basavaraj (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SQOOP-1532) Sqoop2: Support Sqoop on Spark Execution Engine
Date Mon, 23 Mar 2015 22:08:53 GMT

     [ https://issues.apache.org/jira/browse/SQOOP-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Veena Basavaraj updated SQOOP-1532:
-----------------------------------
    Fix Version/s:     (was: 1.99.6)
                   2.0.0

> Sqoop2: Support Sqoop on Spark Execution Engine
> -----------------------------------------------
>
>                 Key: SQOOP-1532
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1532
>             Project: Sqoop
>          Issue Type: Improvement
>            Reporter: Veena Basavaraj
>            Assignee: Veena Basavaraj
>             Fix For: 2.0.0
>
>
> The current execution engine supported in sqoop is MR.
> The goal if this ticket is to support sqoop jobs ( map only and map+reduce ) to run on
spark environment.
> It should at the minimum support running on the standalone spark cluster and then subsequently
work with YARN/mesos.
> High level goals
> 1. Hook up with the connector apis to provide the basic load/ extract to the spark RDD.
> 2. Implementation of the Sqoop RDD to support extraction from different data sources
. The design proposal will discuss the alternatives on how this can be achieved.
> 3. Optimizing the loading/writing ( re-use/ refactor the consumer thread code to be agnostic
of the hadoop output format)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message