nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Wing <jvw...@gmail.com>
Subject Re: PutS3Object error
Date Thu, 03 Nov 2016 18:42:36 GMT
The short answer is no, PutS3Object does not currently support a direct
equivalent of the AWS CLI's --no-verify-ssl option.  There is an option to
provide your own SSLContextService, if you need to establish trust with
your proxy server (maybe, I'm not sure).

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.ssl.StandardSSLContextService/index.html

Can you share a bit more about your use case and proxy setup?  I know there
are other NiFi installations using proxy servers against S3, and I do not
believe they have had this problem.

Last, I believe I foolishly stated in an earlier email that the AWS CLI was
a good comparison tool, but I might have to flip-flop now that we're
bringing proxy settings and SSL verification into the picture.  Are you
sure the CLI is using your proxy similarly?

Thanks,

James

On Thu, Nov 3, 2016 at 5:58 AM, John Burns <jzburns@gmail.com> wrote:

> Hi,
>
> I have a workflow that compresses an file then invokes PutS3Object to
> store in an S3 bucket. This processor works fine in a non-proxy
> environment,  where PutS3Object is parameterised correctly with the proxy
> settings, but in a proxy environment I get the following error shown in the
> stack trace.
>
> Testing from the AWS cli tools, I need to use the --no-verify-ssl
> parameter:
>
> aws s3 ls --no-verify-ssl s3://nifibucket/
>
> Is there an equivalent "--no-verify-ss"for the PutS3Object processor?
>
> Thanks
>
> John
>
>
> ERROR [Timer-Driven Process Thread-10] o.a.nifi.processors.aws.s3.PutS3Object
> PutS3Object[id=26ea1644-0158-1000-be29-271b59722ea4] Failed to put
> StandardFlowFileRecord[uuid=72488dde-07c8-4236-8116-bd8b34d93716,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1478122984174-68,
> container=default, section=68], offset=233361, length=34033],offset=0,name=bbctext.gz,size=34033]
> to Amazon S3 due to com.amazonaws.AmazonClientException: Unable to
> execute HTTP request: sun.security.validator.ValidatorException: PKIX
> path building failed: sun.security.provider.certpath.SunCertPathBuilderException:
> unable to find valid certification path to requested target: com.amazonaws.AmazonClientException:
> Unable to execute HTTP request: sun.security.validator.ValidatorException:
> PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException:
> unable to find valid certification path to requested target
> 2016-11-03 12:49:50,876 ERROR [Timer-Driven Process Thread-10]
> o.a.nifi.processors.aws.s3.PutS3Object
> com.amazonaws.AmazonClientException: Unable to execute HTTP request:
> sun.security.validator.ValidatorException: PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to
> find valid certification path to requested target
>         at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:706)
> ~[aws-java-sdk-core-1.11.8.jar:na]
>         at com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:447)
> ~[aws-java-sdk-core-1.11.8.jar:na]
>         at com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:409)
> ~[aws-java-sdk-core-1.11.8.jar:na]
>         at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:358)
> ~[aws-java-sdk-core-1.11.8.jar:na]
>         at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3787)
> ~[aws-java-sdk-s3-1.11.8.jar:na]
>         at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1399)
> ~[aws-java-sdk-s3-1.11.8.jar:na]
>         at org.apache.nifi.processors.aws.s3.PutS3Object$1.process(PutS3Object.java:451)
> ~[nifi-aws-processors-1.0.0.jar:1.0.0]
>         at org.apache.nifi.controller.repository.
> StandardProcessSession.read(StandardProcessSession.java:1880) ~[na:na]
>         at org.apache.nifi.controller.repository.
> StandardProcessSession.read(StandardProcessSession.java:1851) ~[na:na]
>         at org.apache.nifi.processors.aws.s3.PutS3Object.onTrigger(PutS3Object.java:401)
> ~[nifi-aws-processors-1.0.0.jar:1.0.0]
>         at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> [nifi-api-1.0.0.jar:1.0.0]
>         at org.apache.nifi.controller.StandardProcessorNode.onTrigger(
> StandardProcessorNode.java:1064) [nifi-framework-core-1.0.0.jar:1.0.0]
>         at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.
> call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.0.0.
> jar:1.0.0]
>         at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.
> call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.0.0.
> jar:1.0.0]
>         at org.apache.nifi.controller.scheduling.
> TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
> [nifi-framework-core-1.0.0.jar:1.0.0]
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_60]
>         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_60]
>         at java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_60]
>         at java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_60]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_60]
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.ja
>

Mime
View raw message