val rdd = sparkContext.parallelize((1 to 100).map(i => Person(i.toString)))
val sparkDataFrame = sqlContext.createDataFrame(rdd)
The second line throws an exception:
java.lang.ClassCastException: ....types.PersonUDT cannot be cast to org.apache.spark.sql.types.StructType
I looked into the code in SQLContext.scala and it seems that the code requires UDT to be extending StructType but in fact it extends UserDefinedType which extends directly DataType.
I'm not sure whether it is a bug or I just don't know how to use UDTs.
Do you have any suggestions how to solve this? I based my UDT on ExamplePointUDT but it seems to be incorrect. Is there a working example for UDT?
Thank you for the reply in advance!