spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pralabh Kumar <pralabhku...@gmail.com>
Subject Re: Are there any alternatives to Hive "stored by" clause as Spark 2.0 does not support it
Date Thu, 08 Feb 2018 16:04:50 GMT
Hi Jacek

https://cwiki.apache.org/confluence/display/Hive/StorageHandlers

The motivation is to make it possible to allow Hive to access data stored
and managed by other systems in a modular, extensible fashion.


I have hive script which have custom storage handler , something like this


create table


   1. CREATE EXTERNAL TABLE $temp_output
   2. (
   3. data String
   4. )
   5. STORED BY 'ABCStorageHandler' LOCATION '$table_location'
   TBLPROPERTIES (
   6.
   7. );


when I migrate to Spark it says STORED BY operation is not permitted.

Regards
Pralabh Kumar

On Thu, Feb 8, 2018 at 6:28 PM, Jacek Laskowski <jacek@japila.pl> wrote:

> Hi,
>
> Since I'm new to Hive, what does `stored by` do? I might help a bit in
> Spark if I only knew a bit about Hive :)
>
> Pozdrawiam,
> Jacek Laskowski
> ----
> https://about.me/JacekLaskowski
> Mastering Spark SQL https://bit.ly/mastering-spark-sql
> Spark Structured Streaming https://bit.ly/spark-structured-streaming
> Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
> Follow me at https://twitter.com/jaceklaskowski
>
> On Thu, Feb 8, 2018 at 7:25 AM, Pralabh Kumar <pralabhkumar@gmail.com>
> wrote:
>
>> Hi
>>
>> Spark 2.0 doesn't support stored by . Is there any alternative to achieve
>> the same.
>>
>>
>>
>

Mime
View raw message