hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-14124) Spark app name should be in line with MapReduce app name when using Hive On Spark
Date Thu, 07 Jul 2016 16:37:11 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15366367#comment-15366367
] 

Xuefu Zhang commented on HIVE-14124:
------------------------------------

Well, this is by design. Hive on Spark reuses the AM for the whole user session, so the app
name isn't mapped to a single user query.

> Spark app name should be in line with MapReduce app name when using Hive On Spark
> ---------------------------------------------------------------------------------
>
>                 Key: HIVE-14124
>                 URL: https://issues.apache.org/jira/browse/HIVE-14124
>             Project: Hive
>          Issue Type: Improvement
>          Components: Spark
>    Affects Versions: 1.1.0
>            Reporter: Thomas Scott
>            Priority: Minor
>
> When using the spark execution engine the jobs submitted to YARN are submitted with name
"Hive On Spark" whereas in mr  execution engine the name contains the query executed. This
is overrideable via spark.app.name but it should automatically fill out the query executed
in line with the mr engine.
> Example:
> set hive.execution.engine=spark;
> Select count(*) from sometable; 
>  
> -> Launched YARN Job description: Hive On Spark
> set hive.execution.engine=mr;
> Select count(*) from sometable; 
>  
> -> Launched YARN Job description: Select count(*) from sometable



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message