spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Baptiste Onofré ...@nanthrax.net>
Subject Re: DStream Transformation to save JSON in Cassandra 2.1
Date Mon, 05 Oct 2015 14:27:51 GMT
Hi Prateek,

I see two ways:

- using Cassandra CQL to adapt the RDD in the DStream to Cassandra
- using a Cassandra converter

You have a couple of code snippet in the examples. Let me know if you 
need a code sample.

Regards
JB

On 10/05/2015 04:14 PM, Prateek . wrote:
> Hi,
>
> I am beginner in Spark , this is sample data I get from Kafka stream:
>
> {"id": "9f5ccb3d5f4f421392fb98978a6b368f","coordinate":{"ax":"1.20","ay":"3.80","az":"9.90","oa":"8.03","ob":"8.8","og":"9.97"}}
>
>    val lines = KafkaUtils.createStream(ssc, zkQuorum, group, topicMap).map(_._2)
>    val jsonf = lines.map(JSON.parseFull(_)).map(_.get.asInstanceOf[scala.collection.immutable.Map[String,
Any]])
>
>    I am getting a, DSTream[Map[String,Any]]. I need to store each coordinate values in
the below Cassandra schema
>
> CREATE TABLE iotdata.coordinate (
>      id text PRIMARY KEY, ax double, ay double, az double, oa double, ob double, oz double
> )
>
> For this what transformations I need to apply before I execute saveToCassandra().
>
> Thank You,
> Prateek
>
>
> "DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use
of the individual to whom it is addressed. It may contain privileged or confidential information
and should not be circulated or used for any purpose other than for what it is intended. If
you have received this message in error, please notify the originator immediately. If you
are not the intended recipient, you are notified that you are strictly prohibited from using,
copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility
for loss or damage arising from the use of the information transmitted by this email including
damage from virus."
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>

-- 
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message