hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Ahuja (Jira)" <j...@apache.org>
Subject [jira] [Created] (YARN-10123) Error message around yarn app -stop/start can be improved to highlight that an implementation at framework level is needed for the stop/start functionality to work
Date Mon, 10 Feb 2020 07:06:00 GMT
Siddharth Ahuja created YARN-10123:
--------------------------------------

             Summary: Error message around yarn app -stop/start can be improved to highlight
that an implementation at framework level is needed for the stop/start functionality to work
                 Key: YARN-10123
                 URL: https://issues.apache.org/jira/browse/YARN-10123
             Project: Hadoop YARN
          Issue Type: Improvement
          Components: client, documentation
    Affects Versions: 3.2.1
            Reporter: Siddharth Ahuja


A "stop" on a YARN application fails with the below error:

{code}
# yarn app -stop application_1581294743321_0002 -appTypes SPARK
20/02/10 06:24:27 INFO client.RMProxy: Connecting to ResourceManager at c3224-node2.squadron.support.hortonworks.com/172.25.34.128:8050
20/02/10 06:24:27 INFO client.AHSProxy: Connecting to Application History server at c3224-node2.squadron.support.hortonworks.com/172.25.34.128:10200
Exception in thread "main" java.lang.IllegalArgumentException: App admin client class name
not specified for type SPARK
        at org.apache.hadoop.yarn.client.api.AppAdminClient.createAppAdminClient(AppAdminClient.java:76)
        at org.apache.hadoop.yarn.client.cli.ApplicationCLI.run(ApplicationCLI.java:579)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
        at org.apache.hadoop.yarn.client.cli.ApplicationCLI.main(ApplicationCLI.java:123)
{code}

>From https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/AppAdminClient.java#L76,
it seems that this is because user does not have the setting:

{code}
yarn.application.admin.client.class.SPARK set up in the client configuration.
{code}

However, even if this setting is present, we still need to have an implementation available
for the application type. From my internal discussions - Jobs don't have a notion of stop
/ resume functionality at YARN level. If some apps like Spark need it, it has to be implemented
at those framework's level.

Therefore, the above error message is a bit misleading in that, even if "yarn.application.admin.client.class.SPARK"
is supplied (or for that matter - yarn.application.admin.client.class.MAPREDUCE), if there
is no implementation actually available underneath to handle the stop/start functionality
then, we will fail again, albeit with a different error here: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/AppAdminClient.java#L85.

As such, maybe this error message can be improved to say something like:

{code}
Exception in thread "main" java.lang.IllegalArgumentException: App admin client class name
not specified for type SPARK, is there an implementation even available for this framework?
{code}

or something similar.

Further, documentation around "-stop" and "-start" options will need to be improved here ->
https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YarnCommands.html#application_or_app
as it does not mention anything about having an implementation at the framework level for
the YARN stop/start command to succeed.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org


Mime
View raw message