hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-2605) Hadoop Streaming (StreamJob) does not delete temporary job/package jar
Date Sat, 18 Jun 2011 17:19:47 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Harsh J resolved MAPREDUCE-2605.

       Resolution: Not A Problem
    Fix Version/s: 0.22.0


Per {{StreamJob#packageJobJar()}} in 0.22 and trunk, the file is created as a Java Temp File
and is also asked to be deleted-on-exit (unless debug is turned on). Please reopen if you
feel this does not work as intended.

Fix appears to be in 0.22 version, so marking that as the fix version. Unfortunately, revision
history appears to end at some move performed long ago, so I can't find what exact JIRA was
this fix related to.

Resolving as Not A Problem (anymore). But please re-open in case am wrong with something.

> Hadoop Streaming (StreamJob) does not delete temporary job/package jar
> ----------------------------------------------------------------------
>                 Key: MAPREDUCE-2605
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2605
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 0.20.2
>            Reporter: Greg Wittel
>            Priority: Minor
>             Fix For: 0.22.0
>   Original Estimate: 5m
>  Remaining Estimate: 5m
> The streaming job driver (org.apache.hadoop.streaming.StreamJob) does not delete the
temporary JAR file it generates after a job completes.  Without the fix, /var/tmp fills up
 with streaming job jars until they get wiped.
> The jar name is stored in the class variable 'jar_'.  The JAR is generated in 'packageJobJar()'
and the name stored in jar_.
> Fix:  run()/submitAndMonitorJob() should clean up the jar_ file when done.  Or the JAR
could be generatd as a tempfile and cleaned up automatically.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message