hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (HIVE-24059) Llap external client - Initial changes for running in cloud environment
Date Mon, 31 Aug 2020 08:41:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-24059?focusedWorklogId=476436&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-476436
]

ASF GitHub Bot logged work on HIVE-24059:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 31/Aug/20 08:40
            Start Date: 31/Aug/20 08:40
    Worklog Time Spent: 10m 
      Work Description: ShubhamChaurasia commented on a change in pull request #1418:
URL: https://github.com/apache/hive/pull/1418#discussion_r479982693



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java
##########
@@ -559,12 +566,22 @@ private SplitResult getSplits(JobConf job, TezWork work, Schema schema,
Applicat
         // 4. Make location hints.
         SplitLocationInfo[] locations = makeLocationHints(hints.get(i));
 
+        // 5. populate info about llap daemons(to help client submit request and read data)
+        LlapDaemonInfo[] llapDaemonInfos = populateLlapDaemonInfos(job, locations);
+
+        // 6. Generate JWT for external clients if it's a cloud deployment
+        String jwt = "";
+        if (LlapUtil.isCloudDeployment()) {
+          JwtHelper jwtHelper = new JwtHelper(SessionState.getSessionConf());
+          jwt = jwtHelper.buildJwtForLlap(applicationId);

Review comment:
       done

##########
File path: llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/ContainerRunnerImpl.java
##########
@@ -342,6 +352,33 @@ public SubmitWorkResponseProto submitWork(SubmitWorkRequestProto request)
throws
         .build();
   }
 
+  // if request is coming from llap external client, verify the JWT
+  // as of now, JWT contains applicationId
+  private void verifyJwtForExternalClient(SubmitWorkRequestProto request, String applicationIdString,
+      String fragmentIdString) {
+    LOG.info("Checking if request[{}] is from llap external client in a cloud based deployment",
applicationIdString);
+    if (request.getIsExternalClientRequest() && LlapUtil.isCloudDeployment()) {
+      LOG.info("Llap external client request - {}, verifying JWT", applicationIdString);
+      Preconditions.checkState(request.hasJwt(), "JWT not found in request, fragmentId: "
+ fragmentIdString);
+
+      JwtHelper jwtHelper = new JwtHelper(getConfig());
+      Jws<Claims> claimsJws;
+      try {
+        claimsJws = jwtHelper.parseClaims(request.getJwt());
+      } catch (JwtException e) {
+        LOG.error("Cannot verify JWT provided with the request, fragmentId: {}, {}", fragmentIdString,
e);
+        throw e;
+      }
+
+      String appIdInJwt = (String) claimsJws.getBody().get(JwtHelper.LLAP_EXT_CLIENT_APP_ID);
+      // this should never happen ideally.
+      Preconditions.checkState(appIdInJwt.equals(applicationIdString),

Review comment:
       done




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 476436)
    Time Spent: 1h 20m  (was: 1h 10m)

> Llap external client - Initial changes for running in cloud environment
> -----------------------------------------------------------------------
>
>                 Key: HIVE-24059
>                 URL: https://issues.apache.org/jira/browse/HIVE-24059
>             Project: Hive
>          Issue Type: Sub-task
>          Components: llap
>            Reporter: Shubham Chaurasia
>            Assignee: Shubham Chaurasia
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> Please see problem description in https://issues.apache.org/jira/browse/HIVE-24058
> Initial changes include - 
> 1. Moving LLAP discovery logic from client side to server (HS2 / get_splits) side.
> 2. Opening additional RPC port in LLAP Daemon.
> 3. JWT Based authentication on this port.
> cc [~prasanth_j] [~jdere] [~anishek] [~thejas]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message