sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gwen Shapira <gshap...@cloudera.com>
Subject Re: Issue with sqoop merge command
Date Mon, 04 Aug 2014 17:38:56 GMT
It says that the output directory exists, can you verify if thats the case?

Sqoop merge takes 3 directory parameters:
--new-data - directory with the new data set to merge (in case of
duplicates we keep the values from this directory)
--onto - directory with existing data set to merge (some of the values
here will be replaced by new values)
--target-dir - this directory should not exist before running Sqoop
merge. The results will go here.

Gwen

On Mon, Aug 4, 2014 at 9:51 AM,  <shouvanik.haldar@accenture.com> wrote:
> Hi,
>
>
>
> I am getting a weird error this morning! When running sqoop merge command
>
>
>
> Deleted s3://xxx-xx-xxx/data/xxxx/xxxx/xxxx/merged
>
> merge folder exists so deleted 0
>
>
>
> Next,
>
>
>
> Merge command fails with the error
>
>
>
>
>
> 14/08/04 11:39:24 ERROR security.UserGroupInformation:
> PriviledgedActionException as:hadoop (auth:SIMPLE)
> cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
> s3://xx-xx-xx/xx/xxxx/xxx/xxx/merged already exists
>
> 14/08/04 11:39:24 ERROR tool.MergeTool: Encountered IOException running
> import job: org.apache.hadoop.mapred.FileAlreadyExistsException: Output
> directory s3://xx-xx-xx/xx/xxxx/xxxx/xxxx/merged already exists
>
>                at
> org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:146)
>
>                at
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:456)
>
>                at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:342)
>
>                at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
>
>                at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
>
>                at java.security.AccessController.doPrivileged(Native Method)
>
>                at javax.security.auth.Subject.doAs(Subject.java:415)
>
>                at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
>
>                at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
>
>                at
> org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286)
>
>                at
> org.apache.sqoop.mapreduce.JobBase.runJob(JobBase.java:312)
>
>                at
> org.apache.sqoop.mapreduce.MergeJob.runMergeJob(MergeJob.java:138)
>
>                at org.apache.sqoop.tool.MergeTool.run(MergeTool.java:57)
>
>                at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>
>                at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>
>                at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>
>                at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>
>                at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>
>                at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>
>
>
> 11:39:24: sqoopMergeStatus 1
>
> 11:39:24: 0
>
> 11:39:24: sqoop Merge Failed  for XXXX
>
>
>
> Any idea, how to fix it?
>
>
> ________________________________
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited. Where allowed by
> local law, electronic communications with Accenture and its affiliates,
> including e-mail and instant messaging (including content), may be scanned
> by our systems for the purposes of information security and assessment of
> internal compliance with Accenture policy.
> ______________________________________________________________________________________
>
> www.accenture.com

Mime
View raw message