spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Dutrow <dan.dut...@gmail.com>
Subject Different Kafka createDirectStream implementations
Date Tue, 08 Sep 2015 16:44:55 GMT
The two methods of createDirectStream appear to have different
implementations, the second checks the offset.reset flags and does some
error handling while the first does not. Besides the use of a
messageHandler, are they intended to be used in different situations?

def createDirectStream[ K: ClassTag, V: ClassTag, KD <: Decoder[K]:
ClassTag,
VD <: Decoder[V]: ClassTag,* R: ClassTag] *
( ssc: StreamingContext, kafkaParams: Map[String, String], fromOffsets:
Map[TopicAndPartition, Long], * messageHandler: MessageAndMetadata[K, V] =>
R *
):

def createDirectStream[ K: ClassTag, V: ClassTag, KD <: Decoder[K]:
ClassTag, VD <: Decoder[V]: ClassTag]
( ssc: StreamingContext, kafkaParams: Map[String, String], topics:
Set[String] )
-- 
Dan📱

Mime
View raw message