hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yesha Vora (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-8913) Add helper scripts to launch MaWo App to run Hadoop unit tests on Hadoop Cluster
Date Thu, 18 Oct 2018 21:46:00 GMT
Yesha Vora created YARN-8913:
--------------------------------

             Summary: Add helper scripts to launch MaWo App to run Hadoop unit tests on Hadoop
Cluster
                 Key: YARN-8913
                 URL: https://issues.apache.org/jira/browse/YARN-8913
             Project: Hadoop YARN
          Issue Type: Sub-task
            Reporter: Yesha Vora


MaWo application can be used to run Hadoop UT faster in a Hadoop cluster.
 Develop helper scripts to orchestrate end-to-end workflow for running Hadoop UT using MaWo
app.

Pre-requisite:
 * A Hadoop Cluster with HDFS and YARN installed
 * Enable Docker on YARN feature

 

Helper-scripts
 * MaWo_Driver
 ** create a docker image with latest hadoop source code
 ** create payload to MaWo app (This is input to mawo app where Each MaWo Task = UT execution
of each Hadoop Module)
 ** Upload payload file to HDFS
 ** Update MaWo-Launch.json to resolve RM_HOST / Docker Image etc dynamically
 ** Launch MaWo app in Hadoop cluster



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org


Mime
View raw message