spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Divya Gehlot <divya.htco...@gmail.com>
Subject Spark 1.5.2 : change datatype in programaticallly generated schema
Date Fri, 04 Mar 2016 07:58:20 GMT
Hi,

I am generating schema programatically  as show below
val schemaFile = sc.textFile("/TestDivya/Spark/cars.csv")
 val schemaString = schemaFile.first()
val schema =
      StructType(Array(
        schemaString.split(" ").map(fieldName => StructField(fieldName,
StringType(), true))))

I want to change the datatype of year column to int  and for many other
columns as my schemaString is huge with more than 100+ columns
any suggestions?


Thanks,
Divya

Mime
View raw message