sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abraham Elmahrek <...@cloudera.com>
Subject Re: Sqoop 1.4.4 export to sql server
Date Thu, 04 Dec 2014 19:33:37 GMT
Hey there,

Could you provide an example snippet of the content you're exporting?

-Abe

On Tue, Dec 2, 2014 at 2:49 PM, sai chaitanya tirumerla <tirumerla@gmail.com
> wrote:

> Hi,
>
> My sqoop export command is as follows
>
> sqoop export \
> --connect  "xxxxxx" \
> --username "xxxxxx" \
> --password "xxxxxxxxx" \
> --table "abcd" \
> --columns
> accnt_duns_num,accnt_firm_name,business_name,ceo_first_name,ceo_full_name,ceo_last_name,ceo_middle_name,country_name_50,county_name_45,fax_nbr_nt_fmtd,first_exec_first_name,first_exec_last_name,first_exec_mid_name,mail_addr_2,mail_addr_64,phone_16,primary_addr_1,primary_addr_2,sec_exec_first_name,sec_exec_last_name,sec_exec_mid_name,second_name,state_prov_name_50,third_exec_first_name,third_exec_last_name,third_exec_mid_name
> \
> --export-dir "$S3_PATH/abcd" \
> --escaped-by '\\' --enclosed-by '"' --fields-terminated-by ',' \
> --update-key accnt_duns_num
>
> Here is the full error stack trace for the error above
>
> 14/12/02 22:38:28 INFO tool.CodeGenTool: Beginning code generation
> 14/12/02 22:38:28 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM [t_cde_duns_master] AS t WHERE 1=0
> 14/12/02 22:38:28 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
> /home/hadoop
> Note:
> /tmp/sqoop-hadoop/compile/f782b6393504f22cee53e80c8b63e1c1/t_cde_duns_master.java
> uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 14/12/02 22:38:30 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-hadoop/compile/f782b6393504f22cee53e80c8b63e1c1/t_cde_duns_master.jar
> 14/12/02 22:38:30 INFO mapreduce.ExportJobBase: Beginning export of
> t_cde_duns_master
> 14/12/02 22:38:30 INFO Configuration.deprecation: mapred.jar is
> deprecated. Instead, use mapreduce.job.jar
> 14/12/02 22:38:31 INFO guice.EmrFSBaseModule: Consistency disabled, using
> com.amazon.ws.emr.hadoop.fs.s3n.S3NativeFileSystem as FileSystem
> implementation.
> 14/12/02 22:38:32 INFO fs.EmrFileSystem: Using
> com.amazon.ws.emr.hadoop.fs.s3n.S3NativeFileSystem as filesystem
> implementation
> 14/12/02 22:38:32 INFO Configuration.deprecation:
> mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.reduce.speculative
> 14/12/02 22:38:33 INFO s3n.S3NativeFileSystem: listStatus
> s3://datamasking/bfix/odsg/cde/dbo/t_cde_duns_master/masked with recursive
> false
> 14/12/02 22:38:33 INFO s3n.S3NativeFileSystem: Opening
> 's3://datamasking/bfix/odsg/cde/dbo/t_cde_duns_master/masked/43679ac8-bb94-47da-9474-0c62782ebf2d-000000.gz'
> for reading
> 14/12/02 22:38:33 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 14/12/02 22:38:33 INFO Configuration.deprecation: mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
> 14/12/02 22:38:33 INFO client.RMProxy: Connecting to ResourceManager at /
> 10.37.70.235:9022
> 14/12/02 22:38:34 INFO s3n.S3NativeFileSystem: listStatus
> s3://datamasking/bfix/odsg/cde/dbo/t_cde_duns_master/masked with recursive
> false
> 14/12/02 22:38:34 INFO input.FileInputFormat: Total input paths to process
> : 6
> 14/12/02 22:38:34 INFO s3n.S3NativeFileSystem: listStatus
> s3://datamasking/bfix/odsg/cde/dbo/t_cde_duns_master/masked with recursive
> false
> 14/12/02 22:38:34 INFO input.FileInputFormat: Total input paths to process
> : 6
> 14/12/02 22:38:34 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
> from the embedded binaries
> 14/12/02 22:38:34 INFO lzo.LzoCodec: Successfully loaded & initialized
> native-lzo library [hadoop-lzo rev 77cfa96225d62546008ca339b7c2076a3da91578]
> 14/12/02 22:38:35 INFO mapreduce.JobSubmitter: number of splits:3
> 14/12/02 22:38:35 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1415156847621_2303
> 14/12/02 22:38:35 INFO impl.YarnClientImpl: Submitted application
> application_1415156847621_2303
> 14/12/02 22:38:35 INFO mapreduce.Job: The url to track the job:
> http://10.37.70.235:9046/proxy/application_1415156847621_2303/
> 14/12/02 22:38:35 INFO mapreduce.Job: Running job: job_1415156847621_2303
> 14/12/02 22:38:42 INFO mapreduce.Job: Job job_1415156847621_2303 running
> in uber mode : false
> 14/12/02 22:38:42 INFO mapreduce.Job:  map 0% reduce 0%
> 14/12/02 22:38:49 INFO mapreduce.Job: Task Id :
> attempt_1415156847621_2303_m_000000_0, Status : FAILED
> Error: java.io.IOException: Can't export data, please check task tracker
> logs
>         at
> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
>         at
> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
>         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
>         at
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:775)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> Caused by: java.io.IOException:
> com.microsoft.sqlserver.jdbc.SQLServerException: The value is not set for
> the parameter number 2.
>         at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:220)
>         at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:46)
>         at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:646)
>         at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
>         at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
>         at
> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:84)
>         ... 10 more
> Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The value is
> not set for the parameter number 2.
>         at
> com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:190)
>         at
> com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.buildParamTypeDefinitions(SQLServerPreparedStatement.java:260)
>         at
> com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.buildPreparedStrings(SQLServerPreparedStatement.java:219)
>         at
> com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doPrepExec(SQLServerPreparedStatement.java:612)
>         at
> com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatementBatch(SQLServerPreparedStatement.java:1282)
>         at
> com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtBatchExecCmd.doExecute(SQLServerPreparedStatement.java:1209)
>         at
> com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:5696)
>         at
> com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1715)
>         at
> com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:180)
>         at
> com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:155)
>         at
> com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeBatch(SQLServerPreparedStatement.java:1173)
>         at
> org.apache.sqoop.mapreduce.AsyncSqlOutputFormat$AsyncSqlExecThread.run(AsyncSqlOutputFormat.java:231)
>
> 14/12/02 22:38:49 INFO mapreduce.Job: Task Id :
> attempt_1415156847621_2303_m_000002_0, Status : FAILED
> Error: java.io.IOException: Can't export data, please check task tracker
> logs
>         at
> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
>         at
> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
>         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
>         at
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:775)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> Caused by: java.io.IOException:
> com.microsoft.sqlserver.jdbc.SQLServerException: The value is not set for
> the parameter number 2.
>         at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:220)
>         at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:46)
>         at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:646)
>         at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
>         at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
>         at
> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:84)
>         ... 10 more
>
>
> Any help would be appreciated.
>
> Thanks
>
>
>
>
>
> On Tue, Dec 2, 2014 at 2:43 PM, sai chaitanya tirumerla <
> tirumerla@gmail.com> wrote:
>
>> Hi,
>>
>> I am trying to export s3 data into sql server but it keeps on failing
>> with error :
>>
>> "The value is not set for the parameter number 2"
>>
>> Not sure what the error is about everything is specified within the sqoop
>> export command though.
>>
>> The full error stack trace
>>
>>
>>
>>
>

Mime
View raw message