sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tolliver, Johnny S." <tolli...@ornl.gov>
Subject Re: import error: column name '........pg.dropped.16........' not in table
Date Mon, 27 Feb 2012 16:06:41 GMT
On Feb 25, 2012, at 1:11 AM, Jarek Jarcec Cecho wrote:

> would you mind running sqoop again with parameter --verbose and sending entire output?

Sure. With --verbose on the create-hive-table command, I get the following:

[hduser@cmsvm01 ~]$ sqoop create-hive-table --connect jdbc:postgresql://cmsgp/kdi_big --table
kdi_eligibility --username xxxxxx -P --verbose --fields-terminated-by ','
12/02/27 10:06:07 DEBUG tool.BaseSqoopTool: Enabled debug logging.
Enter password: 
12/02/27 10:06:18 DEBUG sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory
12/02/27 10:06:18 DEBUG sqoop.ConnFactory: Trying ManagerFactory: com.cloudera.sqoop.manager.DefaultManagerFactory
12/02/27 10:06:18 DEBUG manager.DefaultManagerFactory: Trying with scheme: jdbc:postgresql:
12/02/27 10:06:18 INFO manager.SqlManager: Using default fetchSize of 1000
12/02/27 10:06:18 DEBUG sqoop.ConnFactory: Instantiated ConnManager com.cloudera.sqoop.manager.PostgresqlManager@39e87719
12/02/27 10:06:19 INFO hive.HiveImport: Loading uploaded data into Hive
12/02/27 10:06:19 DEBUG hive.HiveImport: Hive.inputTable: kdi_eligibility
12/02/27 10:06:19 DEBUG hive.HiveImport: Hive.outputTable: kdi_eligibility
12/02/27 10:06:19 DEBUG manager.SqlManager: No connection paramenters specified. Using regular
API for making connection.
12/02/27 10:06:19 DEBUG manager.SqlManager: Using fetchSize for next query: 1000
12/02/27 10:06:19 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM "kdi_eligibility"
AS t LIMIT 1
12/02/27 10:06:19 WARN hive.TableDefWriter: Column link_key had to be cast to a less precise
type in Hive
12/02/27 10:06:19 WARN hive.TableDefWriter: Column birth_dt had to be cast to a less precise
type in Hive
12/02/27 10:06:19 WARN hive.TableDefWriter: Column death_dt had to be cast to a less precise
type in Hive
12/02/27 10:06:19 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
java.lang.NullPointerException
        at com.cloudera.sqoop.hive.TableDefWriter.getCreateTableStmt(TableDefWriter.java:151)
        at com.cloudera.sqoop.hive.HiveImport.importTable(HiveImport.java:193)
        at com.cloudera.sqoop.tool.CreateHiveTableTool.run(CreateHiveTableTool.java:60)
        at com.cloudera.sqoop.Sqoop.run(Sqoop.java:146)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:182)
        at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:221)
        at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:230)
        at com.cloudera.sqoop.Sqoop.main(Sqoop.java:239)


And with --verbose on the import command, this:


[hduser@cmsvm01 ~]$ sqoop import --connect jdbc:postgresql://cmsgp/kdi_big --username xxxxxx
-P --table kdi_eligibility --hive-import -m 1 --direct --verbose
12/02/27 10:04:41 DEBUG tool.BaseSqoopTool: Enabled debug logging.
Enter password: 
12/02/27 10:04:57 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You
can override
12/02/27 10:04:57 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
12/02/27 10:04:57 DEBUG sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory
12/02/27 10:04:57 DEBUG sqoop.ConnFactory: Trying ManagerFactory: com.cloudera.sqoop.manager.DefaultManagerFactory
12/02/27 10:04:57 DEBUG manager.DefaultManagerFactory: Trying with scheme: jdbc:postgresql:
12/02/27 10:04:57 INFO manager.SqlManager: Using default fetchSize of 1000
12/02/27 10:04:57 DEBUG sqoop.ConnFactory: Instantiated ConnManager com.cloudera.sqoop.manager.DirectPostgresqlManager@1bbb60c3
12/02/27 10:04:57 INFO tool.CodeGenTool: Beginning code generation
12/02/27 10:04:57 DEBUG manager.SqlManager: No connection paramenters specified. Using regular
API for making connection.
12/02/27 10:04:57 DEBUG manager.SqlManager: Using fetchSize for next query: 1000
12/02/27 10:04:57 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM "kdi_eligibility"
AS t LIMIT 1
12/02/27 10:04:58 ERROR tool.ImportTool: Imported Failed: Column name '........pg.dropped.16........'
not in table


And, for reference, the table is defined as follows:

CREATE TABLE kdi_eligibility (
        state varchar(2) NOT NULL,
        fyq varchar(5) NULL,
        link_key numeric(8, 0) NOT NULL,
	rectype varchar(1) NULL,
        ident varchar(20) NULL,
	xxx varchar(1) NULL,
        county varchar(3) NULL,
	zip varchar(5) NULL,
        hhh varchar(12) NULL,
	casenum varchar(12) NULL,
        birth_dt datetime NULL,
	death_dt datetime NULL,
        idnum varchar(9) NULL,
	s_indicator varchar(1) NULL,
        e_indicator varchar(1) NULL,
	xrefnum varchar(12) NULL
)


Thanks!


Mime
View raw message