sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Muthu Pandi <muthu1...@gmail.com>
Subject Exporting to non standard databases
Date Mon, 25 Aug 2014 04:55:28 GMT
Hi every one,
               Is it possible to export data from HDFS to vetica.I
successfully imported data from vertica to HDFS using sqoop.When i Export i
get the following error.Please give me solution as soon as possible.this is
the error which am getting,

*WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver
however appropriate connection manager is not being set (via
--connection-manager). Sqoop is going to fall back to
org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly
which connection manager should be used next time.*

will above will have impact.How do i create connection manager for vertica.

My sqoop query is

sqoop export  --driver com.vertica.jdbc.Driver --connect
"jdbc:vertica://X.X.X.X:5433/dbname" --username username -P --table
tablename  -m 1 --export-dir /ver.txt  --input-fields-terminated-by ','
--lines-terminated-by '\n'


and am using hadoop2.4.1 and sqoop1.4.4 version



14/08/25 10:12:57 WARN tool.BaseSqoopTool: Setting your password on the
command-line is insecure. Consider using -P instead.
14/08/25 10:12:57 WARN sqoop.ConnFactory: Parameter --driver is set to an
explicit driver however appropriate connection manager is not being set
(via --connection-manager). Sqoop is going to fall back to
org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly
which connection manager should be used next time.
14/08/25 10:12:57 INFO manager.SqlManager: Using default fetchSize of 1000
14/08/25 10:12:57 INFO tool.CodeGenTool: Beginning code generation
14/08/25 10:12:58 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM AGE AS t WHERE 1=0
14/08/25 10:12:58 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM AGE AS t WHERE 1=0
14/08/25 10:12:58 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
/opt/hadoop-2.4.1
Note: /tmp/sqoop-hadoop2/compile/f0a438b0e6007fea6da95808e5f9468e/AGE.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
14/08/25 10:13:02 INFO orm.CompilationManager: Writing jar file:
/tmp/sqoop-hadoop2/compile/f0a438b0e6007fea6da95808e5f9468e/AGE.jar
14/08/25 10:13:03 INFO mapreduce.ExportJobBase: Beginning export of AGE
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/opt/hadoop-2.4.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/hbase/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
14/08/25 10:13:03 INFO Configuration.deprecation: mapred.jar is deprecated.
Instead, use mapreduce.job.jar
14/08/25 10:13:06 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM AGE AS t WHERE 1=0
14/08/25 10:13:06 INFO Configuration.deprecation:
mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
mapreduce.reduce.speculative
14/08/25 10:13:06 INFO Configuration.deprecation:
mapred.map.tasks.speculative.execution is deprecated. Instead, use
mapreduce.map.speculative
14/08/25 10:13:06 INFO Configuration.deprecation: mapred.map.tasks is
deprecated. Instead, use mapreduce.job.maps
14/08/25 10:13:06 INFO client.RMProxy: Connecting to ResourceManager at
nn01/10.10.200.74:8032
14/08/25 10:13:27 INFO input.FileInputFormat: Total input paths to process
: 1
14/08/25 10:13:27 INFO input.FileInputFormat: Total input paths to process
: 1
14/08/25 10:13:27 INFO mapreduce.JobSubmitter: number of splits:1
14/08/25 10:13:28 INFO mapreduce.JobSubmitter: Submitting tokens for job:
job_1408941947869_0001
14/08/25 10:13:30 INFO impl.YarnClientImpl: Submitted application
application_1408941947869_0001
14/08/25 10:13:30 INFO mapreduce.Job: The url to track the job:
http://nn01:8088/proxy/application_1408941947869_0001/
14/08/25 10:13:30 INFO mapreduce.Job: Running job: job_1408941947869_0001
14/08/25 10:13:44 INFO mapreduce.Job: Job job_1408941947869_0001 running in
uber mode : false
14/08/25 10:13:44 INFO mapreduce.Job:  map 0% reduce 0%
14/08/25 10:14:11 INFO mapreduce.Job:  map 100% reduce 0%
14/08/25 10:14:11 INFO mapreduce.Job: Task Id :
attempt_1408941947869_0001_m_000000_0, Status : FAILED
Error: java.io.IOException: java.sql.SQLException: [Vertica][VJDBC](4856)
ERROR: Syntax error at or near ","
 at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
 at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
 at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.sql.SQLException: [Vertica][VJDBC](4856) ERROR: Syntax
error at or near ","
 at com.vertica.util.ServerErrorData.buildException(Unknown Source)
at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown Source)
 at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
 at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
at com.vertica.jdbc.SPreparedStatement.<init>(Unknown Source)
 at com.vertica.jdbc.VerticaPreparedStatementImpl.<init>(Unknown Source)
at com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown
Source)
 at com.vertica.jdbc.SConnection.prepareStatement(Unknown Source)
at
org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
 at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
 ... 8 more
Caused by: com.vertica.util.ServerException: [Vertica][VJDBC](4856) ERROR:
Syntax error at or near ","
... 20 more

14/08/25 10:14:12 INFO mapreduce.Job:  map 0% reduce 0%
14/08/25 10:14:25 INFO mapreduce.Job: Task Id :
attempt_1408941947869_0001_m_000000_1, Status : FAILED
Error: java.io.IOException: java.sql.SQLException: [Vertica][VJDBC](4856)
ERROR: Syntax error at or near ","
 at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
 at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
 at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.sql.SQLException: [Vertica][VJDBC](4856) ERROR: Syntax
error at or near ","
 at com.vertica.util.ServerErrorData.buildException(Unknown Source)
at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown Source)
 at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
 at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
at com.vertica.jdbc.SPreparedStatement.<init>(Unknown Source)
 at com.vertica.jdbc.VerticaPreparedStatementImpl.<init>(Unknown Source)
at com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown
Source)
 at com.vertica.jdbc.SConnection.prepareStatement(Unknown Source)
at
org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
 at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
 ... 8 more
Caused by: com.vertica.util.ServerException: [Vertica][VJDBC](4856) ERROR:
Syntax error at or near ","
... 20 more

14/08/25 10:14:34 INFO mapreduce.Job: Task Id :
attempt_1408941947869_0001_m_000000_2, Status : FAILED
Error: java.io.IOException: java.sql.SQLSyntaxErrorException:
[Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
 at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
 at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
 at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.sql.SQLSyntaxErrorException: [Vertica][VJDBC](4856) ERROR:
Syntax error at or near ","
 at com.vertica.util.ServerErrorData.buildException(Unknown Source)
at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown Source)
 at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
 at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
at com.vertica.jdbc.common.SPreparedStatement.<init>(Unknown Source)
 at com.vertica.jdbc.jdbc4.S4PreparedStatement.<init>(Unknown Source)
at com.vertica.jdbc.VerticaJdbc4PreparedStatementImpl.<init>(Unknown Source)
 at com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown
Source)
at com.vertica.jdbc.common.SConnection.prepareStatement(Unknown Source)
 at
org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
 at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
... 8 more
Caused by: com.vertica.support.exceptions.SyntaxErrorException:
[Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
 ... 21 more

14/08/25 10:14:43 INFO mapreduce.Job:  map 100% reduce 0%
14/08/25 10:14:44 INFO mapreduce.Job: Job job_1408941947869_0001 failed
with state FAILED due to: Task failed task_1408941947869_0001_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

14/08/25 10:14:45 INFO mapreduce.Job: Counters: 9
Job Counters
Failed map tasks=4
 Launched map tasks=4
Other local map tasks=3
Rack-local map tasks=1
 Total time spent by all maps in occupied slots (ms)=53177
Total time spent by all reduces in occupied slots (ms)=0
 Total time spent by all map tasks (ms)=53177
Total vcore-seconds taken by all map tasks=53177
Total megabyte-seconds taken by all map tasks=54453248
14/08/25 10:14:45 WARN mapreduce.Counters: Group FileSystemCounters is
deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
14/08/25 10:14:45 INFO mapreduce.ExportJobBase: Transferred 0 bytes in
99.0548 seconds (0 bytes/sec)
14/08/25 10:14:45 WARN mapreduce.Counters: Group
org.apache.hadoop.mapred.Task$Counter is deprecated. Use
org.apache.hadoop.mapreduce.TaskCounter instead
14/08/25 10:14:45 INFO mapreduce.ExportJobBase: Exported 0 records.
14/08/25 10:14:45 ERROR tool.ExportTool: Error during export: Export job
failed!



*Regards Muthupandi.K*

[image: Picture (Device Independent Bitmap)]

Mime
View raw message