From dev-return-19307-apmail-nifi-dev-archive=nifi.apache.org@nifi.apache.org Thu Jun 13 17:30:56 2019 Return-Path: X-Original-To: apmail-nifi-dev-archive@minotaur.apache.org Delivered-To: apmail-nifi-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by minotaur.apache.org (Postfix) with SMTP id 8270019F9B for ; Thu, 13 Jun 2019 17:30:56 +0000 (UTC) Received: (qmail 71451 invoked by uid 500); 13 Jun 2019 17:30:51 -0000 Delivered-To: apmail-nifi-dev-archive@nifi.apache.org Received: (qmail 71407 invoked by uid 500); 13 Jun 2019 17:30:51 -0000 Mailing-List: contact dev-help@nifi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@nifi.apache.org Delivered-To: mailing list dev@nifi.apache.org Received: (qmail 71395 invoked by uid 99); 13 Jun 2019 17:30:50 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 13 Jun 2019 17:30:50 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 66FA21A49E4 for ; Thu, 13 Jun 2019 17:30:50 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.8 X-Spam-Level: X-Spam-Status: No, score=0.8 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_REPLY=1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id BIn-iwJqlZDk for ; Thu, 13 Jun 2019 17:30:48 +0000 (UTC) Received: from mail-oi1-f195.google.com (mail-oi1-f195.google.com [209.85.167.195]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 939BD5FB07 for ; Thu, 13 Jun 2019 17:30:48 +0000 (UTC) Received: by mail-oi1-f195.google.com with SMTP id y6so15066483oix.2 for ; Thu, 13 Jun 2019 10:30:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :content-transfer-encoding; bh=/ARlUpL+mCvm8+hIMRFDhd4llYtLmv+jW8M5Qb3/GF0=; b=T/a11N5W3rVyqLeXPBtLyps3Wsb8dnRbOwJa3iu3d5Vt7ZWz3ZzTHLqLEChDzidn1Z CygZOHQ96xCOK0EgvUsF1HTFnaOlhuoEL6bEdSjAyznOsr5Crnb4fzHuR8XmDST9Zih7 PZUJM5sSmhl8X9un/YOnN3rBjfPryTOBtRTGoR8zcNpDuWPf5f5cVKn3zk/L9dpf8D6j wwSDTc6pDw2wI8YxYHcxTkcWog27lLzEbe/rGqa37v+l54vgsOlmsGvMXN99ZD/83oPb jt0xxRfhXYjmqR5Krcd3O5VmdN7DkoPNW9EizdKeTz35l9Huheq6l+lCd3FlgAa3EtfC rjGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:content-transfer-encoding; bh=/ARlUpL+mCvm8+hIMRFDhd4llYtLmv+jW8M5Qb3/GF0=; b=eLYhobSwFbGnKZJB/JRHGnUYXAI26F2Z2dExH/UiDFgJNFxoLciOpoI2K4t3a1EnUf mfOXEvzpr0FU56Iq3dwETBXR8GiwzaskjZ3Mzy9GuhsKzHBWF7lRSxfKJxmxmCCTwy/x 4JtzPzJEnWUZhPCCbeD5Qv2rMzfCXUzehLS/dLk+UEIWXjMNCPMtcXwI9fh7V0SjZ+Z/ 8G3EfaywzMMWa2rY0JX4ZP3uhBrR/Mo9MJ40YNKb1ACdvnJXipjjUBh0MFYlvwkf0Pod 3p55U/zaZIyXlt7eIAT595McWFoUhpos5iT21JZ8OpwfIz/u0G74dcXrBQzTf8NS3D/7 VP4g== X-Gm-Message-State: APjAAAWcpGc1tTdwqstnJMRqEvau4YSwYff30m1vyg5SkHiZEzdXBOsW yJKGerfkH4GnTL1ZuR2WpG8zIcZ3KWjL4h9Xt1s33TnM X-Google-Smtp-Source: APXvYqyu2ngA3V+TC275xX0FckYBPgdd9oRYtPdNmAqpNhjRJyn7xmAvPA65gT7l1wenpDDgZ1+1zYgRyHeYy86EAi8= X-Received: by 2002:aca:7c5:: with SMTP id 188mr3607483oih.70.1560447047590; Thu, 13 Jun 2019 10:30:47 -0700 (PDT) MIME-Version: 1.0 References: <54A4345D-377C-43D8-80BC-60043605E89C@weeksconsulting.us> <3ED79A2B-793D-4EFA-A4BC-AA248DB61534@apache.org> <4EBA74C9-B9F7-4E2E-807A-BDD2C18035A3@weeksconsulting.us> In-Reply-To: From: Bryan Bende Date: Thu, 13 Jun 2019 13:30:36 -0400 Message-ID: Subject: Re: NiFi Toolkit CLI Token Creation To: dev@nifi.apache.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Just to further elaborate, within the CLI there are commands that work against registry and commands that work against NiFi. For registry commands, they use the Java client that is provided by registry [1]. For NiFi commands, there is mini client developed as need with in the CLI [2]. None of these client calls currently have any concept of a JWT/token. In order to do the kerberos auth correctly across both systems, I think both of these clients would need to be updated to support a method that called the /access/kerberos end point to obtain a token, and then also provide a way to pass back that token on future requests. It would likely be the CLI's job to store that token somewhere (in memory for interactive shell, or on filesystem for individual executions) and pass it back on each request. In order to call the /access/kerberos end-point there also needs to be code in the client that handles the negotiation to provide the kerberos credentials that are present from having done a kinit. Long story short, Andy's first suggest would be a much easier option with no code changes. [1] https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/= nifi-registry-client [2] https://github.com/apache/nifi/tree/master/nifi-toolkit/nifi-toolkit-cl= i/src/main/java/org/apache/nifi/toolkit/cli/impl/client On Thu, Jun 13, 2019 at 1:28 PM Andy LoPresto wrote: > > You=E2=80=99ll probably have to write (minimal) code to expose the Client= Builder constructor/factory methods to the part that parses command-line ar= guments. > > Andy LoPresto > alopresto@apache.org > alopresto.apache@gmail.com > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > On Jun 13, 2019, at 10:27 AM, Shawn Weeks w= rote: > > > > Is there a way to pass 2 currently? Because you can get the token via c= url like I=E2=80=99m currently doing? > > > > Thanks > > Shawn > > > > Sent from my iPhone > > > >> On Jun 13, 2019, at 12:21 PM, Andy LoPresto wro= te: > >> > >> I see a couple choices here: > >> > >> 1. Use the CA to generate and sign a new certificate for deployments. = This certificate would not be as sensitive as the server certificate, as yo= u can put stricter permissions on that identity within the NiFi access cont= rols, and the cert would be issued for a DN that cannot be used to imperson= ate the server itself. Use this certificate to authenticate for deployment = activities. > >> 2. Manually extract the user=E2=80=99s JWT from the Developer Tools in= your browser and pass that into the CLI. This token expires regularly, so = you will need to continually update it. > >> 3. Build the Kerberos implementation of the authentication aspects of = the CLI toolkit. > >> > >> Andy LoPresto > >> alopresto@apache.org > >> alopresto.apache@gmail.com > >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >> > >>> On Jun 13, 2019, at 10:00 AM, Shawn Weeks = wrote: > >>> > >>> For our organization the server certificate is considered sensitive a= nd not available to the users who need to deploy to NiFi. Actual authentica= tion to NiFi is handled through Knox and our SSO Service so the end user ne= ver deals with SSL or has access to a certificate. Originally I started dow= n the path of writing a bunch of tools based on NiPyAPI to handle deploymen= ts but since the CLI already does that I was hoping to save some work. Curr= ently we do several other things via rest using the Kerberos Token. > >>> > >>> As I looked through the tool kit CLI I was seeing that auth token bei= ng passed into all the rest calls so I was hoping I could hijack wherever t= hat was being generated via 2way ssl and add an option to call Kerberos ins= tead to get the token. When I say token I mean the auth bearer token that y= ou can get from a post request to /access/kerberos in NiFi and /access/toke= n/Kerberos in NiFi registry. > >>> > >>> Thanks > >>> Shawn > >>> > >>> =EF=BB=BFOn 6/12/19, 12:06 PM, "Bryan Bende" wrote= : > >>> > >>> I meant to say that you obviously could generate certs for CLI users= , but I > >>> was just mentioning an alternative where you can proxy an identity. > >>> > >>> Right now the CLI never obtains a token because it is all cert based= . > >>> > >>>> On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende wrot= e: > >>>> > >>>> Right now the idea is that whoever is running the CLI would have acc= ess to > >>>> a NiFi server certificate and then you can proxy any user you want. = There > >>>> should be examples of this in the readme or toolkit guide. > >>>> > >>>> Supporting Kerberos auth was something I wanted to do, but it=E2=80= =99s definitely > >>>> not a trivial effort. > >>>> > >>>> On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto > >>>> wrote: > >>>> > >>>>> Shawn, > >>>>> > >>>>> I=E2=80=99m not sure I understand your question. > >>>>> > >>>>> I am in the process of refactoring the TLS Toolkit to integrate wit= h > >>>>> public certificate authorities, so in the near future it will be ea= sier to > >>>>> use certificates signed by external authorities rather than self-si= gned. > >>>>> > >>>>> My understanding is that you are talking about the CLI Toolkit rath= er > >>>>> than the TLS Toolkit, but your reference to =E2=80=9Ctoken=E2=80=9D= was ambiguous, so I=E2=80=99m > >>>>> going to proceed with the understanding that you are referring to t= he JWT > >>>>> token used to identify an authenticated user when communicating wit= h the > >>>>> NiFi API. > >>>>> > >>>>> You may want to look at JerseyNiFiClient [1], which has methods for > >>>>> getting various clients given an authentication token. > >>>>> > >>>>> You can create the token via the POST /access/kerberos API [2]. > >>>>> > >>>>> [1] > >>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolki= t-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/Jerse= yNiFiClient.java#L163 > >>>>> < > >>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolki= t-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/Jerse= yNiFiClient.java#L163 > >>>>>> > >>>>> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html < > >>>>> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html> > >>>>> > >>>>> Andy LoPresto > >>>>> alopresto@apache.org > >>>>> alopresto.apache@gmail.com > >>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >>>>> > >>>>>> On Jun 12, 2019, at 9:39 AM, Shawn Weeks > >>>>> wrote: > >>>>>> > >>>>>> I work in an environment reluctant to create self signed ssl > >>>>> certificates and I=E2=80=99m looking at the feasibility of having t= he toolkit cli > >>>>> authenticate via Kerberos. I was expecting it to be as simple as ad= ding > >>>>> another way to get the authentication token but I=E2=80=99m having = trouble figuring > >>>>> out exactly when the token is created. I see lots of references to = it after > >>>>> it=E2=80=99s been created. > >>>>>> > >>>>>> Thanks > >>>>>> Shawn > >>>>> > >>>>> -- > >>>> Sent from Gmail Mobile > >>>> > >>> -- > >>> Sent from Gmail Mobile > >>> > >>> > >> >