spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mykidong <mykid...@gmail.com>
Subject Re: KafkaUtils and specifying a specific partition
Date Fri, 13 Mar 2015 00:24:15 GMT
If you want to use another kafka receiver instead of current spark kafka 
receiver, You can see this:
https://github.com/mykidong/spark-kafka-simple-consumer-receiver/blob/master/src/main/java/spark/streaming/receiver/kafka/KafkaReceiverUtils.java

You can handle to get just the stream from the specified partition.

- Kidong.


------ Original Message ------
From: "ColinMc [via Apache Spark User List]" 
<ml-node+s1001560n22018h83@n3.nabble.com>
To: "mykidong" <mykidong@gmail.com>
Sent: 2015-03-13 오전 1:58:08
Subject: KafkaUtils and specifying a specific partition

>Hi,
>
>How do you use KafkaUtils to specify a specific partition? I'm writing 
>customer Marathon jobs where a customer is given 1 partition in a topic 
>in Kafka. The job will get the partition from our database for that 
>customer and use that to get the messages for that customer.
>
>I misinterpreted KafkaUtils when creating the stream and didn't know 
>that it was the number of partitions per topic in the map.
>
>If KafkaUtils doesn't support this, is there another Spark API call for 
>Kafka that supports this?
>
>--------------------------------------------------------------------------------
>If you reply to this email, your message will be added to the 
>discussion below:
>http://apache-spark-user-list.1001560.n3.nabble.com/KafkaUtils-and-specifying-a-specific-partition-tp22018.html
>To unsubscribe from Apache Spark User List, click here.
>NAML



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/KafkaUtils-and-specifying-a-specific-partition-tp22018p22023.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Mime
View raw message