sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From p...@insufficient-light.com
Subject Re: cannot see hive table after import
Date Tue, 19 Nov 2013 22:44:22 GMT
It looks like the answer lies here:

13/11/19 15:02:16 WARN tool.BaseSqoopTool: It seems that you've specified
at least one of following:
13/11/19 15:02:16 WARN tool.BaseSqoopTool:      --hive-home
13/11/19 15:02:16 WARN tool.BaseSqoopTool:      --hive-overwrite
13/11/19 15:02:16 WARN tool.BaseSqoopTool:      --create-hive-table
13/11/19 15:02:16 WARN tool.BaseSqoopTool:      --hive-table
13/11/19 15:02:16 WARN tool.BaseSqoopTool:      --hive-partition-key
13/11/19 15:02:16 WARN tool.BaseSqoopTool:      --hive-partition-value
13/11/19 15:02:16 WARN tool.BaseSqoopTool:      --map-column-hive
13/11/19 15:02:16 WARN tool.BaseSqoopTool: Without specifying parameter
--hive-import. Please note that
13/11/19 15:02:16 WARN tool.BaseSqoopTool: those arguments will not be
used in this session. Either
13/11/19 15:02:16 WARN tool.BaseSqoopTool: specify --hive-import to apply
them correctly or remove them
13/11/19 15:02:16 WARN tool.BaseSqoopTool: from command line to remove
this warning.

Perhaps you should try re-running the command you used, adding the
--hive-import parameter.


> Just started to use Hadoop, Hive, Sqoop today. Sorry for any stupid
> questions.
>
> I managed to run a sqoop-import command. The command output says "Hive
> import complete". But I cannot see the table within hive. Hive command
> "show tables" show nothing.
>
> My Sqoop command is: 
> /bin/sqoop-import --connect jdbc:mysql://xxx.com/yyyy --table user_entity
> --username uuuuuu --password ppppp --hive-table
> h_user_entity --create-hive-table
>>
> My Hive command is:
> show tables;
> Sqoop command output is:
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
>>Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
>>Please set $HCAT_HOME to the root of your HCatalog installation.
>>Warning: $HADOOP_HOME is deprecated.
>>
>>
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: Setting your password on the
>> command-line is insecure. Consider using -P instead.
>>13/11/19 15:02:16 INFO tool.BaseSqoopTool: Using Hive-specific delimiters
>> for output. You can override
>>13/11/19 15:02:16 INFO tool.BaseSqoopTool: delimiters with
>> --fields-terminated-by, etc.
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: It seems that you've specified
>> at least one of following:
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-home
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-overwrite
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --create-hive-table
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-table
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-partition-key
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-partition-value
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --map-column-hive
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: Without specifying parameter
>> --hive-import. Please note that
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: those arguments will not be
>> used in this session. Either
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: specify --hive-import to apply
>> them correctly or remove them
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: from command line to remove
>> this warning.
>>13/11/19 15:02:16 INFO tool.BaseSqoopTool: Please note that --hive-home,
>> --hive-partition-key, 
>>13/11/19 15:02:16 INFO tool.BaseSqoopTool:  hive-partition-value and
>> --map-column-hive options are 
>>13/11/19 15:02:16 INFO tool.BaseSqoopTool:  are also valid for HCatalog
>> imports and exports
>>13/11/19 15:02:16 INFO manager.MySQLManager: Preparing to use a MySQL
>> streaming resultset.
>>13/11/19 15:02:16 INFO tool.CodeGenTool: Beginning code generation
>>13/11/19 15:02:16 INFO manager.SqlManager: Executing SQL statement:
>> SELECT t.* FROM `user_entity` AS t LIMIT 1
>>13/11/19 15:02:16 INFO manager.SqlManager: Executing SQL statement:
>> SELECT t.* FROM `user_entity` AS t LIMIT 1
>>13/11/19 15:02:16 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
>> /data/hadoop/current
>>Note:
>> /tmp/sqoop-root/compile/39e8d314afcc1b81c1cc50a6c3d396b1/user_entity.java
>> uses or overrides a deprecated API.
>>Note: Recompile with -Xlint:deprecation for details.
>>13/11/19 15:02:17 INFO orm.CompilationManager: Writing jar file:
>> /tmp/sqoop-root/compile/39e8d314afcc1b81c1cc50a6c3d396b1/user_entity.jar
>>13/11/19 15:02:17 WARN manager.MySQLManager: It looks like you are
>> importing from mysql.
>>13/11/19 15:02:17 WARN manager.MySQLManager: This transfer can be faster!
>> Use the --direct
>>13/11/19 15:02:17 WARN manager.MySQLManager: option to exercise a
>> MySQL-specific fast path.
>>13/11/19 15:02:17 INFO manager.MySQLManager: Setting zero DATETIME
>> behavior to convertToNull (mysql)
>>13/11/19 15:02:17 INFO mapreduce.ImportJobBase: Beginning import of
>> user_entity
>>13/11/19 15:02:18 INFO db.DataDrivenDBInputFormat: BoundingValsQuery:
>> SELECT MIN(`id`), MAX(`id`) FROM `user_entity`
>>13/11/19 15:02:18 WARN db.TextSplitter: Generating splits for a textual
>> index column.
>>13/11/19 15:02:18 WARN db.TextSplitter: If your database sorts in a
>> case-insensitive order, this may result in a partial import or duplicate
>> records.
>>13/11/19 15:02:18 WARN db.TextSplitter: You are strongly encouraged to
>> choose an integral split column.
>>13/11/19 15:02:19 INFO mapred.JobClient: Running job:
>> job_201311181910_0007
>>13/11/19 15:02:20 INFO mapred.JobClient:  map 0% reduce 0%
>>13/11/19 15:02:29 INFO mapred.JobClient:  map 16% reduce 0%
>>13/11/19 15:02:32 INFO mapred.JobClient:  map 33% reduce 0%
>>13/11/19 15:02:35 INFO mapred.JobClient:  map 50% reduce 0%
>>13/11/19 15:02:39 INFO mapred.JobClient:  map 66% reduce 0%
>>13/11/19 15:02:42 INFO mapred.JobClient:  map 83% reduce 0%
>>13/11/19 15:02:43 INFO mapred.JobClient:  map 100% reduce 0%
>>13/11/19 15:02:44 INFO mapred.JobClient: Job complete:
>> job_201311181910_0007
>>13/11/19 15:02:44 INFO mapred.JobClient: Counters: 18
>>13/11/19 15:02:44 INFO mapred.JobClient:   Job Counters 
>>13/11/19 15:02:44 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=38087
>>13/11/19 15:02:44 INFO mapred.JobClient:     Total time spent by all
>> reduces waiting after reserving slots (ms)=0
>>13/11/19 15:02:44 INFO mapred.JobClient:     Total time spent by all maps
>> waiting after reserving slots (ms)=0
>>13/11/19 15:02:44 INFO mapred.JobClient:     Launched map tasks=6
>>13/11/19 15:02:44 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
>>13/11/19 15:02:44 INFO mapred.JobClient:   File Output Format Counters 
>>13/11/19 15:02:44 INFO mapred.JobClient:     Bytes Written=85103032
>>13/11/19 15:02:44 INFO mapred.JobClient:   FileSystemCounters
>>13/11/19 15:02:44 INFO mapred.JobClient:     HDFS_BYTES_READ=825
>>13/11/19 15:02:44 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=389460
>>13/11/19 15:02:44 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=85103032
>>13/11/19 15:02:44 INFO mapred.JobClient:   File Input Format Counters 
>>13/11/19 15:02:44 INFO mapred.JobClient:     Bytes Read=0
>>13/11/19 15:02:44 INFO mapred.JobClient:   Map-Reduce Framework
>>13/11/19 15:02:44 INFO mapred.JobClient:     Map input records=387423
>>13/11/19 15:02:44 INFO mapred.JobClient:     Physical memory (bytes)
>> snapshot=787578880
>>13/11/19 15:02:44 INFO mapred.JobClient:     Spilled Records=0
>>13/11/19 15:02:44 INFO mapred.JobClient:     CPU time spent (ms)=22420
>>13/11/19 15:02:44 INFO mapred.JobClient:     Total committed heap usage
>> (bytes)=1205403648
>>13/11/19 15:02:44 INFO mapred.JobClient:     Virtual memory (bytes)
>> snapshot=6980870144
>>13/11/19 15:02:44 INFO mapred.JobClient:     Map output records=387423
>>13/11/19 15:02:44 INFO mapred.JobClient:     SPLIT_RAW_BYTES=825
>>13/11/19 15:02:44 INFO mapreduce.ImportJobBase: Transferred 81.1606 MB in
>> 26.1394 seconds (3.1049 MB/sec)
>>13/11/19 15:02:44 INFO mapreduce.ImportJobBase: Retrieved 387423 records.
>>13/11/19 15:02:44 INFO manager.SqlManager: Executing SQL statement:
>> SELECT t.* FROM `user_entity` AS t LIMIT 1
>>13/11/19 15:02:44 INFO hive.HiveImport: Removing temporary files from
>> import process: hdfs://localhost:9000/user/root/user_entity/_logs
>>13/11/19 15:02:44 INFO hive.HiveImport: Loading uploaded data into Hive
>>13/11/19 15:02:46 INFO hive.HiveImport: 
>>13/11/19 15:02:46 INFO hive.HiveImport: Logging initialized using
>> configuration in
>> jar:file:/data/hive/hive-0.12.0/lib/hive-common-0.12.0.jar!/hive-log4j.properties
>>13/11/19 15:02:51 INFO hive.HiveImport: OK
>>13/11/19 15:02:51 INFO hive.HiveImport: Time taken: 5.221 seconds
>>13/11/19 15:02:51 INFO hive.HiveImport: Loading data to table
>> default.h_user_entity
>>13/11/19 15:02:52 INFO hive.HiveImport: Table default.h_user_entity
>> stats: [num_partitions: 0, num_files: 7, num_rows: 0, total_size:
>> 85103032, raw_data_size: 0]
>>13/11/19 15:02:52 INFO hive.HiveImport: OK
>>13/11/19 15:02:52 INFO hive.HiveImport: Time taken: 0.636 seconds
>>13/11/19 15:02:52 INFO hive.HiveImport: Hive import complete.
>>13/11/19 15:02:52 INFO hive.HiveImport: Export directory is empty,
>> removing it.
>
> Any help is highly appreciated.
>
> -Kevin



Mime
View raw message