spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Su She <suhsheka...@gmail.com>
Subject [X-post] Saving SparkSQL result RDD to Cassandra
Date Thu, 09 Jul 2015 21:24:00 GMT
Hello All,

I also posted this on the Spark/Datastax thread, but thought it was also
50% a spark question (or mostly a spark question).

I was wondering what is the best practice to saving streaming Spark SQL (
https://github.com/Intel-bigdata/spark-streamingsql/blob/master/src/main/scala/org/apache/spark/sql/streaming/examples/KafkaDDL.scala)
results to Cassandra?

The query looks like this:

 streamSqlContext.sql(
      """
        |SELECT t.word, COUNT(t.word)
        |FROM (SELECT * FROM t_kafka) OVER (WINDOW '9' SECONDS, SLIDE '3'
SECONDS) AS t
        |GROUP BY t.word
      """.stripMargin)
      .foreachRDD { r => r.toString()}.map(x =>
x.split(",")).map(x=>data(x(0),x(1))).saveToCassandra("demo", "sqltest")

I’m getting a message saying map isn’t a member of Unit.

I thought since I'm converting it to a string I can call a map/save to
Cassandra function there, but it seems like I can't call map after
r.toString()?

Please let me know if this is possible and what is the best way of doing
this. Thank you for the help!

-Su

Mime
View raw message