sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suresh Kumar Sethuramaswamy <rock...@gmail.com>
Subject Re: hive CLI not started while using Sqoop import from oracle to hive
Date Tue, 04 Jun 2019 15:30:39 GMT
Glad that it worked.

Thanks for the update Kirill

On Tue, Jun 4, 2019, 11:20 AM K Makeev <k.makeev@gmail.com wrote:

> Suresh,
>
> copying hive-site.xml to current dir worked well. Now sqoop hive import
> works properly.
>
> Thanks a lot
>
> going to find out why sqoop doesn't see hive configuration. Setting
> --hive-home or HIVE_CONF_DIR doesn't helps.
>
> Kirill
>
>
>
> On Tue, Jun 4, 2019 at 5:37 PM Suresh Kumar Sethuramaswamy <
> rockssk@gmail.com> wrote:
>
>> If you are using Cloudera Manager or Ambari,try deploying client
>> configuration.
>>
>> If not copy the hive-site.xml and place it in the current working
>> directory
>>
>> On Tue, Jun 4, 2019, 10:14 AM K Makeev <k.makeev@gmail.com wrote:
>>
>>> Hi, Suresh
>>>
>>> hive-metastore seems to have no info about sqoop trying to do anything.
>>> Found nothing.
>>>
>>> when using just hive command to create table - hive-metastore has all
>>> logs, SQL statement, usernames and etc.
>>>
>>> Kirill
>>>
>>> On Tue, Jun 4, 2019 at 2:57 PM Suresh Kumar Sethuramaswamy <
>>> rockssk@gmail.com> wrote:
>>>
>>>> Couple of things to check
>>>>
>>>> 1. Check if there is a hive-site.xml placed mistakenly in the current
>>>> working directory, if so remove it, else it might overwrite system config
>>>>
>>>> 2. Get the hive-metastore log for any clues while sqoop import happens
>>>>
>>>> Regards
>>>> Suresh
>>>>
>>>> On Tue, Jun 4, 2019, 7:16 AM K Makeev <k.makeev@gmail.com wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> Trying to import simple test table from oracle to hive using sqoop.
>>>>>
>>>>> Sqoop successfully fetches oracle data, puts them to HDFS, but when it
>>>>> comes to hive it doesn't seem to work. Sqoop writes no errors or warnings
>>>>> to output. It jus quits.
>>>>>
>>>>> Here is my command line
>>>>>
>>>>> sqoop import --connect jdbc:oracle:thin:@host:1521:SID --username
>>>>> username --password password --delete-target-dir --table TEST_SQOOP
>>>>> --hive-import --hive-database db -m 1 -verbose
>>>>>
>>>>> Here is the end of verbose output
>>>>>
>>>>> 19/06/03 18:28:02 DEBUG hive.TableDefWriter: Create statement: CREATE
>>>>> TABLE IF NOT EXISTS `db`.`TEST_SQOOP` ( `A` DOUBLE, `B` STRING, `D` STRING)
>>>>> COMMENT 'Imported by sqoop on 2019/06/03 18:28:02' ROW FORMAT DELIMITED
>>>>> FIELDS TERMINATED BY '\001' LINES TERMINATED BY '\012' STORED AS TEXTFILE
>>>>> 19/06/03 18:28:02 DEBUG hive.TableDefWriter: Load statement: LOAD DATA
>>>>> INPATH 'hdfs://hadoop/user/airflow/TEST_SQOOP' INTO TABLE `db`.`TEST_SQOOP`
>>>>> 19/06/03 18:28:02 DEBUG ipc.Client: The ping interval is 60000 ms.
>>>>> 19/06/03 18:28:02 DEBUG ipc.Client: Connecting to hadoop-m1/ip:8020
>>>>> 19/06/03 18:28:02 DEBUG ipc.Client: IPC Client (2071364564) connection
>>>>> to hadoop-m1/ip:8020 from airflow: starting, having connections 2
>>>>> 19/06/03 18:28:02 DEBUG ipc.Client: IPC Client (2071364564) connection
>>>>> to hadoop-m1/ip:8020 from airflow sending #779
>>>>> 19/06/03 18:28:02 DEBUG ipc.Client: IPC Client (2071364564) connection
>>>>> to hadoop-m1/ip:8020 from airflow got value #779
>>>>> 19/06/03 18:28:02 DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took
>>>>> 69ms
>>>>> 19/06/03 18:28:02 INFO hive.HiveImport: Loading uploaded data into Hive
>>>>> 19/06/03 18:28:02 DEBUG hive.HiveImport: Using in-process Hive
>>>>> instance.
>>>>> 19/06/03 18:28:02 DEBUG util.SubprocessSecurityManager: Installing
>>>>> subprocess security manager
>>>>>
>>>>> Logging initialized using configuration in
>>>>> jar:file:/usr/hdp/2.5.3.0-37/hive/lib/hive-common-1.2.1000.2.5.3.0-37.jar!/hive-log4j.properties
>>>>>
>>>>> Nothing happens in hive
>>>>>
>>>>> If i run hive manually through hive command i'm able to execute Create
>>>>> Table/LOAD DATA INPATH statements without problems.
>>>>>
>>>>> Why sqoop doesn't do its hive job?
>>>>>
>>>>>

Mime
View raw message