spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From centerqi <cente...@gmail.com>
Subject Re: save schemardd to hive
Date Tue, 02 Sep 2014 13:33:08 GMT
 

Thank you very much, also can do this, it seems that I know too little about RDD


> 在 2014年9月2日,21:22,Silvio Fiorito <silvio.fiorito@granturing.com> 写道:
> 
> Once you’ve registered an RDD as a table, you can use it in SparkSQL
> statements:
> 
> myrdd.registerAsTable(“my_table”)
> 
> hql(“FROM my_table INSERT INTO TABLE my_other_table”)
> 
> 
> On 9/2/14, 9:18 AM, "centerqi" <centerqi@gmail.com> wrote:
> 
>>>>> schemardd can exec SQL stamens?
>> 
>> 
>> 
>>> 在 2014年9月2日,20:42,Silvio Fiorito <silvio.fiorito@granturing.com>
写道:
>>> 
>>> You can use saveAsTable or do an INSERT SparkSQL statement as well in
>>> case
>>> you need other Hive query features, like partitioning.
>>> 
>>>> On 9/2/14, 6:54 AM, "centerqi hu" <centerqi@gmail.com> wrote:
>>>> 
>>>> I got it
>>>> 
>>>> import org.apache.spark.{SparkConf, SparkContext}
>>>> import org.apache.spark.sql._
>>>> val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc)
>>>> import hiveContext._
>>>> val usermeta =  hql("  SELECT userid,idlist from meta WHERE
>>>> day='2014-08-01' limit 1000")
>>>> case class SomeClass(name:String,idlist:String)
>>>> val scmm = usermeta.map(t=>{SomeClass(t(0).toString,t(1).toString
>>>> +"iddddddddd")})
>>>> val good = createSchemaRDD(scmm)
>>>> good.saveAsTable("meta_test")
>>>> 
>>>> 2014-09-02 17:50 GMT+08:00 centerqi hu <centerqi@gmail.com>:
>>>>> I want to save schemardd to hive
>>>>> 
>>>>> val usermeta =  hql("  SELECT userid,idlist from usermeta WHERE
>>>>> day='2014-08-01' limit 1000")
>>>>> case class SomeClass(name:String,idlist:String)
>>>>> val schemardd =
>>>>> usermeta.map(t=>{SomeClass(t(0).toString,t(1).toString)})
>>>>> 
>>>>> How to save  schemardd to hive?
>>>>> 
>>>>> Thanks
>>>>> 
>>>>> --
>>>>> centerqi@gmail.com|齐忠
>>>> 
>>>> 
>>>> 
>>>> -- 
>>>> centerqi@gmail.com|齐忠
>>>> 
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>>>> For additional commands, e-mail: user-help@spark.apache.org
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>>> For additional commands, e-mail: user-help@spark.apache.org
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
> 

Mime
View raw message