spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeetendra Gangele <gangele...@gmail.com>
Subject creating data warehouse with Spark and running query with Hive
Date Thu, 20 Aug 2015 05:53:30 GMT
HI All,

I have a data in HDFS partition with Year/month/data/event_type. And I am
creating a hive tables with this data, this data is in JSON so I am using
json serve and creating hive tables.
 below is the code
  val jsonFile =
hiveContext.read.json("hdfs://localhost:9000/housing/events_real/category=Impressions/date=1007465766/*")
    jsonFile.toDF().printSchema()
    jsonFile.write.saveAsTable("JsonFileTable")
    jsonFile.toDF().printSchema()
    val events = hiveContext.sql("SELECT category, uid FROM JsonFileTable")
    events.map(e => "Event: " + e).collect().foreach(println)

saveAstable  failing with Error saying MKDir failed to create the directory
 ,anybody has any idea?

Mime
View raw message