spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Silvio Fiorito <silvio.fior...@granturing.com>
Subject Re: save schemardd to hive
Date Tue, 02 Sep 2014 12:42:58 GMT
You can use saveAsTable or do an INSERT SparkSQL statement as well in case
you need other Hive query features, like partitioning.

On 9/2/14, 6:54 AM, "centerqi hu" <centerqi@gmail.com> wrote:

>I got it
>
>import org.apache.spark.{SparkConf, SparkContext}
>import org.apache.spark.sql._
>val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc)
>import hiveContext._
>val usermeta =  hql("  SELECT userid,idlist from meta WHERE
>day='2014-08-01' limit 1000")
>case class SomeClass(name:String,idlist:String)
>val scmm = usermeta.map(t=>{SomeClass(t(0).toString,t(1).toString
>+"iddddddddd")})
>val good = createSchemaRDD(scmm)
>good.saveAsTable("meta_test")
>
>2014-09-02 17:50 GMT+08:00 centerqi hu <centerqi@gmail.com>:
>>  I want to save schemardd to hive
>>
>> val usermeta =  hql("  SELECT userid,idlist from usermeta WHERE
>> day='2014-08-01' limit 1000")
>> case class SomeClass(name:String,idlist:String)
>> val schemardd = 
>>usermeta.map(t=>{SomeClass(t(0).toString,t(1).toString)})
>>
>> How to save  schemardd to hive?
>>
>> Thanks
>>
>> --
>> centerqi@gmail.com|齐忠
>
>
>
>-- 
>centerqi@gmail.com|齐忠
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>For additional commands, e-mail: user-help@spark.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Mime
View raw message