sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abraham Elmahrek" <...@cloudera.com>
Subject Re: Review Request 35591: Sqoop-1853: Kafka connector supporting FROM direction
Date Fri, 19 Jun 2015 00:37:03 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35591/#review88473
-----------------------------------------------------------



connector/connector-kafka/src/main/java/org/apache/sqoop/connector/kafka/KafkaExtractor.java
(line 33)
<https://reviews.apache.org/r/35591/#comment141011>

    Replace with full imports.



connector/connector-kafka/src/main/java/org/apache/sqoop/connector/kafka/KafkaExtractor.java
(line 55)
<https://reviews.apache.org/r/35591/#comment141013>

    Debug?



connector/connector-kafka/src/main/java/org/apache/sqoop/connector/kafka/KafkaExtractor.java
(line 57)
<https://reviews.apache.org/r/35591/#comment141012>

    Debug?



connector/connector-kafka/src/main/java/org/apache/sqoop/connector/kafka/KafkaExtractor.java
(line 71)
<https://reviews.apache.org/r/35591/#comment141029>

    Schemaless is assumed here. Maybe the user could provie a schema? If not, maybe a follow
up Jira?



connector/connector-kafka/src/main/java/org/apache/sqoop/connector/kafka/KafkaPartitioner.java
(line 42)
<https://reviews.apache.org/r/35591/#comment141016>

    Topic not being set. Also, I think you can partition based on the number of partitions
in a kafka topic?



connector/connector-kafka/src/main/java/org/apache/sqoop/connector/kafka/configuration/FromJobConfiguration.java
(line 29)
<https://reviews.apache.org/r/35591/#comment141018>

    Can you create a follow up thread for CDC with Kafka? We should be able to remember the
offset easily as well.


- Abraham Elmahrek


On June 18, 2015, 2:33 a.m., richard zhou wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35591/
> -----------------------------------------------------------
> 
> (Updated June 18, 2015, 2:33 a.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Repository: sqoop-sqoop2
> 
> 
> Description
> -------
> 
> This is an initial patch. It may not work currently. Please check whether this design
of framework is fine or not.
> 
> 
> Diffs
> -----
> 
>   connector/connector-kafka/src/main/java/org/apache/sqoop/connector/kafka/KafkaConnector.java
e3eafe87f7376031d79fec76739fd9e648df872b 
>   connector/connector-kafka/src/main/java/org/apache/sqoop/connector/kafka/KafkaExtractor.java
PRE-CREATION 
>   connector/connector-kafka/src/main/java/org/apache/sqoop/connector/kafka/KafkaFromDestroyer.java
PRE-CREATION 
>   connector/connector-kafka/src/main/java/org/apache/sqoop/connector/kafka/KafkaFromInitializer.java
PRE-CREATION 
>   connector/connector-kafka/src/main/java/org/apache/sqoop/connector/kafka/KafkaLoader.java
ad9ec34755e36985078e880a0a076c46337ef411 
>   connector/connector-kafka/src/main/java/org/apache/sqoop/connector/kafka/KafkaPartition.java
PRE-CREATION 
>   connector/connector-kafka/src/main/java/org/apache/sqoop/connector/kafka/KafkaPartitioner.java
PRE-CREATION 
>   connector/connector-kafka/src/main/java/org/apache/sqoop/connector/kafka/KafkaUtils.java
PRE-CREATION 
>   connector/connector-kafka/src/main/java/org/apache/sqoop/connector/kafka/configuration/FromJobConfig.java
PRE-CREATION 
>   connector/connector-kafka/src/main/java/org/apache/sqoop/connector/kafka/configuration/FromJobConfiguration.java
PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/35591/diff/
> 
> 
> Testing
> -------
> 
> Passed compile, but it still needs to be tested in the real cluster.
> 
> 
> Thanks,
> 
> richard zhou
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message