spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Selvam Raman <>
Subject Cassandra timestamp to spark Date field
Date Mon, 05 Sep 2016 19:29:19 GMT
Hi All,

As per datastax report Cassandra to spark type
timestamp Long, java.util.Date, java.sql.Date, org.joda.time.DateTime

Please help me with your input.

I have a Cassandra table with 30 fields. Out of it 3 are timestamp.

I read cassandratable using sc.cassandraTable
= CassandraTableScanRDD[9] at RDD at CassandraRDD.scala:15]

then I have converted to row of rdd

*val* exis_repair_fact = sqlContext.createDataFrame( =>

in schema fields I have mentioned timestamp as

*StructField*("shipped_datetime", *DateType*),

when I try to show the result, it throws java.util.Date can not convert to

so how can I solve the issue.

First I have converted cassandrascanrdd to

Selvam Raman
"லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"

View raw message