hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohith Sharma K S (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-6889) FileSystem leak when Job#submit() used when ATS1.5 enabled
Date Mon, 15 May 2017 06:15:04 GMT
Rohith Sharma K S created MAPREDUCE-6889:

             Summary: FileSystem leak when Job#submit() used when ATS1.5 enabled
                 Key: MAPREDUCE-6889
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6889
             Project: Hadoop Map/Reduce
          Issue Type: Bug
            Reporter: Rohith Sharma K S
            Assignee: Rohith Sharma K S

ATS1.5 uses FileSystemTimelineWriter which creates FS object on every writer initialization.
If writer is not closed, then there is possibility of OOM see YARN-5438 fixes closing FS object.

TimelineClient is used by YarnClient. So all the user who uses YarnClient with ATS1.5 need
to stop service properly. Otherwise there is big chance of FS object leak. 

Of course MR uses YARN client submit job. If MR do not stop YarnClient then there is FS object

JobClient provides a API to stop all these service using *JobClient#close*. But many MR clients
uses *Job* object to submit a job. But  do not stop started services by default. So, Job class
should provide a API to close the services very similar to JobClient#close. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

View raw message