spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "vijay.bvp" <bvpsa...@gmail.com>
Subject Re: Type Casting Error in Spark Data Frame
Date Wed, 31 Jan 2018 08:50:25 GMT
formatted
=============
Assuming MessageHelper.sqlMapping schema is correctly mapped with input json 
(it would help if the schema and sample json is shared) 

here is explode function with dataframes similar functionality is available
with SQL 

import sparkSession.implicits._ 
import org.apache.spark.sql.functions._ 
val routeEventDF=dataDF.select($"messageId"
,explode($"messageData").alias("MessageData")) 
                                     .select($"messageId",
$"MessageData.unixTime",$"MessageData.packetID", 
                                                $"MessageData.messageID") 
routeEventDF.show 


thanks 
Vijay



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message