sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rekha Joshi (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SQOOP-1237) sqoop export of hdfs file with empty lines causes TextExportMapper.map to fail
Date Tue, 19 Nov 2013 09:59:22 GMT
Rekha Joshi created SQOOP-1237:

             Summary: sqoop export of hdfs file with empty lines causes TextExportMapper.map
to fail
                 Key: SQOOP-1237
                 URL: https://issues.apache.org/jira/browse/SQOOP-1237
             Project: Sqoop
          Issue Type: Improvement
          Components: sqoop2-client
    Affects Versions: 1.4.3
            Reporter: Rekha Joshi
            Priority: Minor

When the hdfs file coming from different sources show empty lines, it causes break in sqoop.And
the options -input-null-string do not work.
This can be workaround by applying sed -i '/^$/d' <file> on the hdfs file.

However it would be nice TextExportMapper can ignore blank lines., possibly by -ignore_blanks
true option

Sqoop: 1.4.3 (cdh 4.3.1)
command: sqoop export -Dmapred.job.queue.name=<queue_name>--connect <connection>
--username <username> --password <password> --table <table> --input-fields-terminated-by
"|" --input-lines-terminated-by \\n --export-dir <export_dir> --input-null-string '\\N'
--input-null-non-string '\\N' 

error:java.io.IOException: Can't export data, please check task tracker logs
	at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
	at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)
	at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
	at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.util.NoSuchElementException
	at java.util.AbstractList$Itr.next(AbstractList.java:350)

This message was sent by Atlassian JIRA

View raw message