spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From maddenpj <madde...@gmail.com>
Subject Re: Spark Streaming unable to handle production Kafka load
Date Wed, 24 Sep 2014 21:28:38 GMT
Another update, actually it just hit me my problem is probably right here:

https://gist.github.com/maddenpj/74a4c8ce372888ade92d#file-gistfile1-scala-L22

I'm creating a JDBC connection on every record, that's probably whats
killing the performance. I assume the fix is just broadcast the connection
pool? 



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-unable-to-handle-production-Kafka-load-tp15077p15081.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message