hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brandon Scheller (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-7127) Add aggregated webservice endpoint to fetch all tasks including their attempts
Date Fri, 03 Aug 2018 18:08:00 GMT
Brandon Scheller created MAPREDUCE-7127:
-------------------------------------------

             Summary: Add aggregated webservice endpoint to fetch all tasks including their
attempts
                 Key: MAPREDUCE-7127
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7127
             Project: Hadoop Map/Reduce
          Issue Type: New Feature
            Reporter: Brandon Scheller


We have a usecase where we would like to poll Hadoop for various running Tasks and display
info on each individual task attempt to the user. 

On large clusters with many tasks, this is an issue because using the current webservice APIs
we would have to make many repeat calls to the same task attempts endpoints. 

To improve performance, we have added additional endpoints to HS and AM webservices that will
fetch aggregated data on hadoop tasks including their task attempts for a job. 

This would look like 
- taskDescriptions webservice endpoint to AMWebServices 
@Path("/jobs/{jobid}/taskDescriptions") 

- describeTasks webservice endpoint to HSWebServices 
@Path("/mapreduce/jobs/{jobid}/describeTasks") 

It would be similar to the current getJobTasks() in HS and AM webservices except it would
include each of the task attempts for each task.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org


Mime
View raw message