sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abraham Elmahrek (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SQOOP-1661) Sqoop2: Intermediate data format text null handling
Date Wed, 05 Nov 2014 08:51:33 GMT

     [ https://issues.apache.org/jira/browse/SQOOP-1661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Abraham Elmahrek updated SQOOP-1661:
------------------------------------
    Description: 
Error:
{noformat}
Error: org.apache.sqoop.common.SqoopException: MAPRED_EXEC_0017:Error occurs during extractor
run at org.apache.sqoop.job.mr.SqoopMapper.run(SqoopMapper.java:99) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554) at
org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162) Caused by: org.apache.sqoop.common.SqoopException:
MAPRED_EXEC_0013:Cannot write to the data writer at org.apache.sqoop.job.mr.SqoopMapper$SqoopMapDataWriter.writeContent(SqoopMapper.java:148)
at org.apache.sqoop.job.mr.SqoopMapper$SqoopMapDataWriter.writeArrayRecord(SqoopMapper.java:122)
at org.apache.sqoop.connector.jdbc.GenericJdbcExtractor.extract(GenericJdbcExtractor.java:62)
at org.apache.sqoop.connector.jdbc.GenericJdbcExtractor.extract(GenericJdbcExtractor.java:31)
at org.apache.sqoop.job.mr.SqoopMapper.run(SqoopMapper.java:94) ... 7 more Caused by: org.apache.sqoop.common.SqoopException:
INTERMEDIATE_DATA_FORMAT_0002:An error has occurred while escaping a row. - null null 0 null
at org.apache.sqoop.connector.idf.CSVIntermediateDataFormat.escapeStrings(CSVIntermediateDataFormat.java:332)
at org.apache.sqoop.connector.idf.CSVIntermediateDataFormat.escapeArray(CSVIntermediateDataFormat.java:299)
at org.apache.sqoop.connector.idf.CSVIntermediateDataFormat.setObjectData(CSVIntermediateDataFormat.java:243)
at org.apache.sqoop.job.mr.SqoopMapper$SqoopMapDataWriter.writeContent(SqoopMapper.java:143)
... 11 more Container killed by the ApplicationMaster. Container killed on request. Exit code
is 143 Container exited with a non-zero exit code 143
{noformat}

It seems like both NULL strings aren't handled properly.

  was:
Jobs should be able to define what the null values look like. Also, the following issues exist:

Simple table:
{noformat}
+-------+-------------+------+-----+---------+-------+
| Field | Type        | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| id    | int(11)     | NO   | PRI | 0       |       |
| name  | varchar(35) | YES  |     | NULL    |       |
+-------+-------------+------+-----+---------+-------+
{noformat}

Content:
{noformat}
+----+------+
| id | name |
+----+------+
|  1 |      |
|  2 | test |
+----+------+
2 rows in set (0.00 sec)
{noformat}

Error:
{noformat}
Error: org.apache.sqoop.common.SqoopException: MAPRED_EXEC_0017:Error occurs during extractor
run at org.apache.sqoop.job.mr.SqoopMapper.run(SqoopMapper.java:99) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554) at
org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162) Caused by: org.apache.sqoop.common.SqoopException:
MAPRED_EXEC_0013:Cannot write to the data writer at org.apache.sqoop.job.mr.SqoopMapper$SqoopMapDataWriter.writeContent(SqoopMapper.java:148)
at org.apache.sqoop.job.mr.SqoopMapper$SqoopMapDataWriter.writeArrayRecord(SqoopMapper.java:122)
at org.apache.sqoop.connector.jdbc.GenericJdbcExtractor.extract(GenericJdbcExtractor.java:62)
at org.apache.sqoop.connector.jdbc.GenericJdbcExtractor.extract(GenericJdbcExtractor.java:31)
at org.apache.sqoop.job.mr.SqoopMapper.run(SqoopMapper.java:94) ... 7 more Caused by: org.apache.sqoop.common.SqoopException:
INTERMEDIATE_DATA_FORMAT_0002:An error has occurred while escaping a row. - null null 0 null
at org.apache.sqoop.connector.idf.CSVIntermediateDataFormat.escapeStrings(CSVIntermediateDataFormat.java:332)
at org.apache.sqoop.connector.idf.CSVIntermediateDataFormat.escapeArray(CSVIntermediateDataFormat.java:299)
at org.apache.sqoop.connector.idf.CSVIntermediateDataFormat.setObjectData(CSVIntermediateDataFormat.java:243)
at org.apache.sqoop.job.mr.SqoopMapper$SqoopMapDataWriter.writeContent(SqoopMapper.java:143)
... 11 more Container killed by the ApplicationMaster. Container killed on request. Exit code
is 143 Container exited with a non-zero exit code 143
{noformat}

It seems like both NULL and EMPTY strings aren't handled properly.


> Sqoop2: Intermediate data format text null handling
> ---------------------------------------------------
>
>                 Key: SQOOP-1661
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1661
>             Project: Sqoop
>          Issue Type: Bug
>          Components: sqoop2-framework
>    Affects Versions: 1.99.4
>            Reporter: Abraham Elmahrek
>            Assignee: Abraham Elmahrek
>             Fix For: 1.99.4
>
>         Attachments: SQOOP-1661.0.patch
>
>
> Error:
> {noformat}
> Error: org.apache.sqoop.common.SqoopException: MAPRED_EXEC_0017:Error occurs during extractor
run at org.apache.sqoop.job.mr.SqoopMapper.run(SqoopMapper.java:99) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554) at
org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162) Caused by: org.apache.sqoop.common.SqoopException:
MAPRED_EXEC_0013:Cannot write to the data writer at org.apache.sqoop.job.mr.SqoopMapper$SqoopMapDataWriter.writeContent(SqoopMapper.java:148)
at org.apache.sqoop.job.mr.SqoopMapper$SqoopMapDataWriter.writeArrayRecord(SqoopMapper.java:122)
at org.apache.sqoop.connector.jdbc.GenericJdbcExtractor.extract(GenericJdbcExtractor.java:62)
at org.apache.sqoop.connector.jdbc.GenericJdbcExtractor.extract(GenericJdbcExtractor.java:31)
at org.apache.sqoop.job.mr.SqoopMapper.run(SqoopMapper.java:94) ... 7 more Caused by: org.apache.sqoop.common.SqoopException:
INTERMEDIATE_DATA_FORMAT_0002:An error has occurred while escaping a row. - null null 0 null
at org.apache.sqoop.connector.idf.CSVIntermediateDataFormat.escapeStrings(CSVIntermediateDataFormat.java:332)
at org.apache.sqoop.connector.idf.CSVIntermediateDataFormat.escapeArray(CSVIntermediateDataFormat.java:299)
at org.apache.sqoop.connector.idf.CSVIntermediateDataFormat.setObjectData(CSVIntermediateDataFormat.java:243)
at org.apache.sqoop.job.mr.SqoopMapper$SqoopMapDataWriter.writeContent(SqoopMapper.java:143)
... 11 more Container killed by the ApplicationMaster. Container killed on request. Exit code
is 143 Container exited with a non-zero exit code 143
> {noformat}
> It seems like both NULL strings aren't handled properly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message