spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From HARSH TAKKAR <takkarha...@gmail.com>
Subject Re: Unable to write data from Spark into a Hive Managed table
Date Tue, 20 Aug 2019 11:00:44 GMT
Please refere to the following documentation on how to write data into hive
in hdp3.1

https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.1.0/integrating-hive/content/hive_hivewarehouseconnector_for_handling_apache_spark_data.html



Harsh

On Fri, 9 Aug, 2019, 10:21 PM Mich Talebzadeh, <mich.talebzadeh@gmail.com>
wrote:

> Check your permissioning.
>
> Can you do insert select from external table into Hive managed table
> created by spark?
>
> //
> // Need to create and populate target ORC table transactioncodes_ll in
> database accounts.in Hive
> //
> HiveContext.sql("use accounts")
> //
> // Drop and create table transactioncodes_ll
> //
> spark.sql("DROP TABLE IF EXISTS accounts.transactioncodes_ll")
> var sqltext = ""
> sqltext = """
> CREATE TABLE accounts.transactioncodes_ll (
> id                         Int
> ,transactiontype           String
> ,description               String
> )
> COMMENT 'from Hive external table'
> STORED AS ORC
> TBLPROPERTIES ( "orc.compress"="ZLIB" )
> """
> HiveContext.sql(sqltext)
> //
> // Put data in Hive table. Clean up is already done
> //
> sqltext = """
> INSERT INTO TABLE accounts.transactioncodes_ll
> SELECT
>           id
>         , TRIM(transactiontype)
>         , TRIM(description)
> FROM <HIVE_EXTERNAL_TABLE>
> """
> spark.sql(sqltext)
>
> HTH
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Fri, 9 Aug 2019 at 12:04, Debabrata Ghosh <mailfordebu@gmail.com>
> wrote:
>
>> Hi ,
>>               I am using Hortonworks Data Platform 3.1. I am unable to
>> write data from Spark into a Hive Managed table but am able to do so in a
>> Hive External table.
>>
>>               Would you please help get me with a resolution.
>>
>> Thanks,
>> Debu
>>
>

Mime
View raw message