nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy LoPresto <alopre...@apache.org>
Subject Re: Problems getting invokeHTTP/PutSQL to work
Date Wed, 21 Jun 2017 23:05:16 GMT
Can you please share a screenshot of the SSLContextService properties again? There should be
only four values populated:

> Truststore Filename: /etc/pki/java/cacerts
> Truststore Password: “changeit”
> Truststore Type: JKS
> SSL Protocol: TLS
I do not believe Amazon changes the default password for cacerts, but you can verify it is
correct by running this command:

$ keytool -keystore /etc/pki/java/cacerts -list -v -storepass changeit

It will either succeed and display the trusted certificate entries or warn you that the password
is incorrect.

Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Jun 21, 2017, at 5:02 PM, Karsten Zakarias <KZA@foss.dk> wrote:
> 
> Thanks James and Andy,
> 
> I located the cacerts. So far so good.
> I still get an error.
> 
> Br,
> Karsten.
> <image001.jpg>
> 
> <image002.jpg>
> 
> Best regards / Med venlig hilsen
> 
> Karsten Zakarias
> Product Manager, M.Sc.
> IT & Digital Business
> 
> 
> 
> <image003.jpg>
> Phone:
> Mobile:
> E-mail:
> +45 4820 8514
> +45 2761 6920
> KZA@foss.dk <mailto:KZA@foss.dk>
> 
> 
> <image004.jpg>
> 
> FOSS Analytical A/S - Foss Allé 1 DK-3400 Hillerød DK
> FOSS provides and supports dedicated, rapid and accurate analytical solutions, which
analyze and control the quality and production of agricultural and food products, for the
enhancement of our customers’ business and better food quality for consumers.
> 
> 
> From: James Wing [mailto:jvwing@gmail.com <mailto:jvwing@gmail.com>]
> Sent: 21. juni 2017 15:27
> To: users@nifi.apache.org <mailto:users@nifi.apache.org>
> Subject: Re: Problems getting invokeHTTP/PutSQL to work
> 
> As Andy mentioned below, the StandardSSLContextService's Truststore Filename should probably
be configured to point to a file named 'cacerts', unless you have a custom truststore to use.
 I believe /etc/pki/java/cacerts is typical path for an EC2 instance.  The path to the JRE
is not sufficient.
> 
> Thanks,
> 
> James
> 
> On Wed, Jun 21, 2017 at 1:54 AM, Karsten Zakarias <KZA@foss.dk <mailto:KZA@foss.dk>>
wrote:
> Hi Andy,
> 
> Thanks for your reply. This is how far I’ve gotten.
> 
> My invokeHTTP configuration looks like this:
> <image007.png>
> 
> My ContextService looks like this:
> <image008.png>
> 
> As my Apache Nifi is on an EC2 instance I ssh in and tried to locate the Java Runtime
Environment and this is what I found under /usr/lib/jvm. I don’t know if that is the correct
location:
> <image010.png>
> 
> This is the ContextService error:
> <image011.jpg>
> 
> Where am I getting it wrong?
> 
> 
> Best regards / Med venlig hilsen
> 
> Karsten Zakarias
> Product Manager, M.Sc.
> IT & Digital Business
> 
> 
> 
> <image003.jpg>
> Phone:
> Mobile:
> E-mail:
> +45 4820 8514
> +45 2761 6920
> KZA@foss.dk <mailto:KZA@foss.dk>
> 
> 
> <image004.jpg>
> 
> FOSS Analytical A/S - Foss Allé 1 DK-3400 Hillerød DK
> FOSS provides and supports dedicated, rapid and accurate analytical solutions, which
analyze and control the quality and production of agricultural and food products, for the
enhancement of our customers’ business and better food quality for consumers.
> 
> 
> From: Andy LoPresto [mailto:alopresto@apache.org <mailto:alopresto@apache.org>]
> Sent: 20. juni 2017 15:37
> To: users@nifi.apache.org <mailto:users@nifi.apache.org>
> Subject: Re: Problems getting invokeHTTP/PutSQL to work
> 
> Hi Karsten,
> 
> Are you getting a specific error message when using InvokeHTTP or PutSQL? I don’t have
an Airtable account, but it looks like a pretty straightforward REST API over HTTPS. From
your screenshot, I believe you need to configure an SSLContextService in order to validate
the certificate presented by Airtable. You should configure it as shown below. The reason
is that NiFi by default does not trust any certificates. When it tries to make a request to
Airtable, Airtable tries to negotiate an encrypted TLS connection and NiFi attempts to verify
the server’s identity using the presented certificate chain. Without a “truststore”
(a collection of trusted certificates), it fails to do this. Java provides a default truststore
to allow these connections to succeed.
> 
> Steps for creating & linking SSLContextService:
> 
> 1. From the Operate palette, click the gear icon (“Configuration”) and switch to
the “Controller Services” tab. You can also skip directly there from the InvokeHTTP properties
tab by opening the dropdown for “SSL Context Service” and selecting “Create new service…"
> 2. Create a new StandardSSLContextService. This controller service provides a common
interface to the TLS (nee SSL) settings that various components use.
> 3. You do not need to populate the first four values (the “keystore” values) unless
Airtable requires client authentication for TLS, which I highly doubt. Instead, populate the
following fields:
>             Truststore Filename: <path to your Java Runtime Environment’s “cacerts”
file>
>             Truststore Password: “changeit”
>             Truststore Type: JKS
>             SSL Protocol: TLS
> 4. Save the controller service and “enable” it by clicking the lightning icon on
the far right.
> 5. Once this service is enabled, your processor should be valid (as shown by a red “stopped”
icon rather than a yellow “caution” icon).
> 6. For debugging, I would also recommend the following settings:
>             Put Response Body in Attribute: “response” <— This will put the
response body into an attribute so you can view it with a LogAttribute processor or by examining
the flowfile in the queue
>             Always Output Response: true <— This will force a flowfile to be generated
even if the HTTP status is one that does not send a response body
> 7. You provided a custom request header with your API key, but you named it “Authentication”
instead of “Authorization”. You’ll need to change this to “Authorization”.
> 8. You may need to set the Content Type header to explicitly be “application/json”.
The mime type is correctly set coming out of the AttributesToJSON processor, so I doubt this
is the issue.
> 
> There may be an Airtable tool for viewing the incoming requests and determining why the
request body could not be parsed. If not, I suggest setting up a simple HTTP server on your
local device and temporarily pointing the request there (or copying the InvokeHTTP processor
and doing it in parallel) to debug the request. You can also use tools like Paw or Postman
to craft POST calls to Airtable manually until you are successful and compare to the output
of the InvokeHTTP processor — you can enable DEBUG level logging for it by modifying your
$NIFI_HOME/conf/logback.xml file and adding the line:
> 
> <logger name=“org.apache.nifi.processors.standard.InvokeHTTP" level="DEBUG"/>
> 
> Hope this helps. Please let us know if this solves your issues and if you encounter any
further trouble.
> 
> 
> <image012.png>
> 
> 
> Andy LoPresto
> alopresto@apache.org <mailto:alopresto@apache.org>
> alopresto.apache@gmail.com <mailto:alopresto.apache@gmail.com>
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> 
> On Jun 20, 2017, at 5:52 AM, Karsten Zakarias <KZA@foss.dk <mailto:KZA@foss.dk>>
wrote:
> 
> Hi all,
> 
> I’m new to Apache Nifi and having problems writing a POST/PUT request to an external
db. Any advice/links to material and prior threats would be greatly appreciated.
> 
> My setup is the following:
> -          I have BatchIQ Apache Nifi running on an AWS EC2 instance.
> -          A GetFTP processor ingests csv files
> -          Each value is extracted using regex and combined into a JSON object using
an AttributesToJSON processor
> -          I now want to use a InvokeHTTP to write the json object to an airtable db
and a PutSQL to write to a SQL db.
> 
> -          How do I setup InvokeHTTP correctly.
> 
> -          How do I configure PutSQL correctly on a remote instance with a database driver.
> 
> 
> Thanks a lot in advance!
> Karsten.
> 
> 
> AttributesToJSON:
> <image003.png>
> 
> InvokeHTTP:
> <image001.png><image002.png>
> <image004.png>
> 
> 
> PutSQL:
> <image006.png>
> <image007.png>
> 
> 
> 
> Best regards / Med venlig hilsen
> 
> Karsten Zakarias
> Product Manager, M.Sc.
> IT & Digital Business
> 
> 
> 
> <image005.jpg>
> Phone:
> Mobile:
> E-mail:
> +45 4820 8514
> +45 2761 6920
> KZA@foss.dk <mailto:KZA@foss.dk>
> 
> 
> <image008.jpg>
> 
> FOSS Analytical A/S - Foss Allé 1 DK-3400 Hillerød DK
> FOSS provides and supports dedicated, rapid and accurate analytical solutions, which
analyze and control the quality and production of agricultural and food products, for the
enhancement of our customers’ business and better food quality for consumers.
> 
> 


Mime
View raw message