spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From venki-kratos <ve...@thekratos.com>
Subject NoSuchMethodError in KafkaReciever
Date Sun, 02 Mar 2014 04:26:32 GMT
I am trying to user code similar to following :
public  JavaPairDStream<String, String> openStream() {
	    HashMap<String, String> kafkaParams = Maps.newHashMap();
	    kafkaParams.put(ZK_CONNECT,kafkaConfig.getString(ZK_CONNECT));
	   
kafkaParams.put(CONSUMER_GRP_ID,kafkaConfig.getString(CONSUMER_GRP_ID));
	    
	    Map<String,Integer> topicMap = Maps.newHashMap();
	    topicMap.put(kafkaConfig.getString(ZK_TOPIC),
kafkaConfig.getInteger(CONSUMER_THREAD_COUNT, 1));
	    JavaPairDStream<String, String> inputStream = 
	    		KafkaUtils.createStream(streamingContext, String.class, String.class,
	    						StringDecoder.class, 
	    						StringDecoder.class,
	    						kafkaParams, topicMap, StorageLevel.MEMORY_AND_DISK_SER_2());		
	    return inputStream;
	}

I have spark-streaming_2.10-0.9.0-incubating.jar and
spark-streaming-kafka_2.10-0.9.0-incubating.jar
in the classpath using POM and m2e in Eclipse. JVM version is set to 1.6

I get the following error, 

14/03/02 09:29:15 INFO kafka.KafkaReceiver: Connected to localhost:2181
14/03/02 09:29:15 ERROR kafka.KafkaReceiver: Error receiving data
java.lang.NoSuchMethodException:
java.lang.Object.<init>(kafka.utils.VerifiableProperties)
	at java.lang.Class.getConstructor0(Class.java:2763)
	at java.lang.Class.getConstructor(Class.java:1693)
	at
org.apache.spark.streaming.kafka.KafkaReceiver.onStart(KafkaInputDStream.scala:108)
	at
org.apache.spark.streaming.dstream.NetworkReceiver.start(NetworkInputDStream.scala:126)
	at
org.apache.spark.streaming.scheduler.NetworkInputTracker$ReceiverExecutor$$anonfun$8.apply(NetworkInputTracker.scala:173)
	at
org.apache.spark.streaming.scheduler.NetworkInputTracker$ReceiverExecutor$$anonfun$8.apply(NetworkInputTracker.scala:169)
	at
org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:884)
	at
org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:884)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:109)
	at org.apache.spark.scheduler.Task.run(Task.scala:53)
	at .....

This is similar to code in JavaKafkaStreamSuite.testKafkaStream. I find that
the kafka jar - kafka_2.10-0.8.0 does have such a constructor. 

What is going wrong? Can someone help solve this mystery and help with my
misery? Basically stuck for last 2 days - as I am a Java Guy and would like
to develop downstream code in Java



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/NoSuchMethodError-in-KafkaReciever-tp2209.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message