spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prateek ." <prat...@aricent.com>
Subject RE: Saving RDD into cassandra keyspace.
Date Fri, 10 Jul 2015 11:41:30 GMT
Hi,

Thanks Todd..the link is really helpful to get started. ☺

-Prateek


From: Todd Nist [mailto:tsindotg@gmail.com]
Sent: Friday, July 10, 2015 4:43 PM
To: Prateek .
Cc: user@spark.apache.org
Subject: Re: Saving RDD into cassandra keyspace.

I would strongly encourage you to read the docs at, they are very useful in getting up and
running:

https://github.com/datastax/spark-cassandra-connector/blob/master/doc/0_quick_start.md

For your use case shown above, you will need to ensure that you include the appropriate version
of the spark-cassandra-connectore assembly jar when you submit the job.  The version you use
should correspond to the version of Spark you are running.   In addition, you will want to
ensure that you set the "spark.cassandra.connection.host" as shown below, prior to creating
the SparkContext.


val conf = new SparkConf(true)

   .set("spark.cassandra.connection.host", "127.0.0.1")

HTH

-Todd


On Fri, Jul 10, 2015 at 5:24 AM, Prateek . <prateek@aricent.com<mailto:prateek@aricent.com>>
wrote:
Hi,

I am beginner to spark , I want save the word and its count to cassandra keyspace, I wrote
the following code

import org.apache.spark.SparkContext
import org.apache.spark.SparkContext._
import org.apache.spark.SparkConf
import com.datastax.spark.connector._

object SparkWordCount {
  def main(args: Array[String]) {
    val sc = new SparkContext(new SparkConf().setAppName("Spark Count"))
    val tokenized = sc.textFile(args(0)).flatMap(_.split(" "))
    val wordCounts = tokenized.map((_, 1)).reduceByKey(_ + _)
    wordCounts.saveToCassandra("sparkdata", "words", SomeColumns("word", "count"));

  }
and did spark-submit. The code doesn’t work ( may be some very basic error because I am
new to it).I know there is datastax cassandra connector but how to make connection?
What all things I am missing in my code?

Thanks





"DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of
the individual to whom it is addressed. It may contain privileged or confidential information
and should not be circulated or used for any purpose other than for what it is intended. If
you have received this message in error, please notify the originator immediately. If you
are not the intended recipient, you are notified that you are strictly prohibited from using,
copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility
for loss or damage arising from the use of the information transmitted by this email including
damage from virus."

"DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of
the individual to whom it is addressed. It may contain privileged or confidential information
and should not be circulated or used for any purpose other than for what it is intended. If
you have received this message in error, please notify the originator immediately. If you
are not the intended recipient, you are notified that you are strictly prohibited from using,
copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility
for loss or damage arising from the use of the information transmitted by this email including
damage from virus."
Mime
View raw message