Hi here!

I tried to run a sqoop export command on Amazon EMR. I succeed running it on a small dataset, but with a bigger one, I get this error on the cluster:

Error: java.io.IOException: org.postgresql.util.PSQLException: Tried to end inactive copy

    at org.apache.sqoop.mapreduce.postgresql.PostgreSQLCopyExportMapper.cleanup(PostgreSQLCopyExportMapper.java:145)

    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:160)

    at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)

    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:773)

    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)

    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)

    at java.security.AccessController.doPrivileged(Native Method)

    at javax.security.auth.Subject.doAs(Subject.java:415)

    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)

    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:170)

Caused by: org.postgresql.util.PSQLException: Tried to end inactive copy

    at org.postgresql.core.v3.QueryExecutorImpl.endCopy(QueryExecutorImpl.java:814)

    at org.postgresql.core.v3.CopyInImpl.endCopy(CopyInImpl.java:59)

    at org.apache.sqoop.mapreduce.postgresql.PostgreSQLCopyExportMapper.cleanup(PostgreSQLCopyExportMapper.java:142)

    ... 9 more


and here my command : ./sqoop export --direct --connect jdbc:postgresql://xxxx --username xxxx --table updates_test --export-dir s3://xxx  -P --fields-terminated-by ',' --enclosed-by '\"' --null-string '' --null-non-string ''

I was wondering if I missed anything , or if anyone here have any clue ?

Thank you so much for your help.

Eric