spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From haridass saisriram <>
Subject SparkSQL: Reading data from hdfs and storing into multiple paths
Date Thu, 01 Oct 2015 21:11:08 GMT

  I am trying to find a simple example to read a data file on HDFS. The
file has the following format
a , b  , c ,yyyy,mm

I would like to read this file and store it in HDFS partitioned by year and
month. Something like this

I want to specify the "/path/to/hdfs/" and yyyy/mm should be populated
automatically based on those columns. Could some one point me in the right

Thank you,
Sri Ram

View raw message