sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Richard (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-1853) Sqoop2: Kafka connector supporting FROM direction
Date Mon, 03 Aug 2015 04:19:05 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14651402#comment-14651402
] 

Richard commented on SQOOP-1853:
--------------------------------

[~gwenshap], got your point. I have drafted a [Design doc | https://cwiki.apache.org/confluence/display/SQOOP/Kafka+Connector+%28From+side%29+Design].
Please help to review.
You are true that SimpleConsumer should be used instead of HighLevelConsumer to make it easy
to support the further features (partition and incremental). Actually, my initial thought
is that the first patch focuses on a workable solution and integrate into Sqoop, then handle
the partition and incremental in the following patches. So I have not thought much on the
further feature. I will upload a patch according to your comment soon.
For the new consumer API, what is the timeline? when will it release? We are interested in
using it. But as the task of Kafka connector (From Side) is target on 20th Aug, I am not sure
whether it could catch up. Will it be a huge change from SimpleConsumer to this new consumer?
Shall we implement it with SimpleConsumer now and change to the new consumer when it is released?

> Sqoop2: Kafka connector supporting FROM direction
> -------------------------------------------------
>
>                 Key: SQOOP-1853
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1853
>             Project: Sqoop
>          Issue Type: Sub-task
>    Affects Versions: 1.99.6
>            Reporter: Gwen Shapira
>            Assignee: Richard
>             Fix For: 1.99.7
>
>         Attachments: SQOOP-1853.0.patch, SQOOP-1853.1.patch, SQOOP-1853.2.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message