sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pratik khadloya <tispra...@gmail.com>
Subject Parquet does not support varchar. See HIVE-6384
Date Fri, 19 Sep 2014 19:08:42 GMT
Currently hive does not support the BINARY data type for parquet formatted
files.
For now is it safe to map the varchar types to string type in hive?

This is because i am running into the following error:

create table `default`.`test` (
     `test_id` bigint,
     `type` int,
     `name` varchar(255),
     `value` varchar(2048))
STORED AS PARQUET
14/09/19 14:31:47 INFO hcat.SqoopHCatUtilities: Executing external HCatalog
CLI process with args :-f,/tmp/hcat-script-1411151507302
14/09/19 14:31:52 INFO hcat.SqoopHCatUtilities: Launching Job 1 out of 0
14/09/19 14:31:52 INFO hcat.SqoopHCatUtilities: FAILED: Execution Error,
return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
java.lang.UnsupportedOperationException: *Parquet does not support varchar.
See HIVE-6384*
14/09/19 14:31:52 ERROR tool.ImportTool: Encountered IOException running
import job: java.io.IOException: HCat exited with status 1
     at
org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.executeExternalHCatProgram(SqoopHCatUtilities.java:1113)
     at
org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.launchHCatCli(SqoopHCatUtilities.java:1062)
     at
org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.createHCatTable(SqoopHCatUtilities.java:595)



I can work around this by using: --map-column-hive name=string
Is there a generic way to tell sqoop to "map all varchar columns to string"?


Thanks,
~Pratik

Mime
View raw message