flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eron Wright <eronwri...@gmail.com>
Subject Re: multiple users per flink deployment
Date Thu, 03 Aug 2017 00:10:56 GMT
One of the key challenges is isolation, eg. ensuring that one job cannot
access the credentials of another.  The easiest solution today is to use
the YARN deployment mode, with a separate app per job.  Meanwhile,
improvements being made under the FLIP-6 banner for 1.4+ are lying
groundwork for a multiuser experience.

Hope this helps!

On Aug 2, 2017 8:29 AM, "Georg Heiler" <georg.kf.heiler@gmail.com> wrote:

> Thanks for the overview.
> Currently a single flink cluster seems to run all tasks with the same
> user. I would want to be able to run each flink job as a separate user
> instead.
> The update for separate read/write users is nice though.
> Tzu-Li (Gordon) Tai <tzulitai@apache.org> schrieb am Mi. 2. Aug. 2017 um
> 10:59:
>> Hi,
>> There’s been quite a few requests on this recently on the mailing lists
>> and also mentioned by some users offline, so I think we may need to start
>> with plans to probably support this.
>> I’m CC’ing Eron to this thread to see if he has any thoughts on this, as
>> he was among the first authors driving the Kerberos support in Flink.
>> I’m not really sure if such a feature support makes sense, given that all
>> jobs of a single Flink deployment have full privileges and therefore no
>> isolation in between.
>> Related question: what external service are you trying to authenticate to
>> with different users?
>> If it is Kafka and perhaps you have different users for the consumer /
>> producer, that will be very soon available in 1.3.2, which includes a
>> version bump to Kafka 0.10 that allows multiple independent users within
>> the same JVM through dynamic JAAS configuration.
>> See this mail thread [1] for more detail on that.
>> Cheers,
>> Gordon
>> [1] http://apache-flink-user-mailing-list-archive.2336050.
>> n4.nabble.com/Kafka-0-10-jaas-multiple-clients-td12831.html#a13317
>> On 1 August 2017 at 6:16:08 PM, Georg Heiler (georg.kf.heiler@gmail.com)
>> wrote:
>> Hi,
>> flink currently only seems to support a single kerberos ticket for
>> deployment. Are there plans to support different users per each job?
>> regards,
>> Georg

View raw message