atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (Jira)" <>
Subject [jira] [Commented] (ATLAS-3779) Inmemory JAASConfig issue in Atlas
Date Wed, 27 May 2020 08:57:00 GMT


ASF subversion and git services commented on ATLAS-3779:

Commit b873dd96ed32ed4731f50ea991bb51c2b43a8a9b in atlas's branch refs/heads/branch-2.0 from
Jayendra Parab
[;h=b873dd9 ]

ATLAS-3779 : Refactoring Kafka in-memory JAASConfig in Atlas.

(cherry picked from commit 61abecac22ef3e9341a07be6d5354bf246544a3b)

> Inmemory JAASConfig issue in Atlas
> ----------------------------------
>                 Key: ATLAS-3779
>                 URL:
>             Project: Atlas
>          Issue Type: Bug
>            Reporter: Mayank Jain
>            Assignee: Mayank Jain
>            Priority: Major
>             Fix For: trunk, 3.0.0
> Spark uses Kafka as source and sink in secure cluster. The test creates a JAAS file like
> {code:java}
> KafkaClient {
> required
>   debug=true
>   useKeyTab=true
>   storeKey=true
>   keyTab="/xxx/keytabs/systest.keytab"
>   useTicketCache=false
>   serviceName="kafka"
>   principal="systest@GCE.EXAMPLE.COM";
> };
> {code}
> As one can see serviceName is set properly.
> Then the test pass the JAAS file to Spark's driver + executor as well:
> {code:java}
> "--conf \""
> "--conf \""
> {code}
> Later on SAC + atlas makes some magic in the background with the Jvm JAAS configuration.
As a result Spark is not able to create consumer for processing data:
> {code:java}
> Caused by: java.lang.IllegalArgumentException: No serviceName defined in either JAAS
or Kafka config
> {code}
> When I've turned off SAC then all the problem gone away.
> Atlas replaces the JVM global JAAS configuration with InMemoryJAASConfiguration once
Atlas configuration is initialized. InMemoryJAASConfiguration has an old JAAS config as "parent"
but Atlas config takes precedence which is unexpected.
> We never want to let Atlas to overwrite existing JAAS configuration if there's a conflict.
(I believe most endpoints using Atlas client as a library would agree with this.) This may
be achieved via swapping precedence for "parent" vs "Atlas config" in InMemoryJAASConfiguration,
but I have no idea the change would be safe to Atlas side. In any way, Atlas should at least
provide a config to let "parent" take precedence for the conflict.

This message was sent by Atlassian Jira

View raw message