nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew McDonald <amcdon...@ccri.com>
Subject Re: How do you use a custom region?
Date Fri, 30 Nov 2018 16:19:33 GMT
Setting line 289 in AbstractAWSProcessor.java to

this.client.setEndpoint(urlstr);

worked for me.

Regards,Andrew

On 11/30/18 09:12, Andrew McDonald wrote:
>
> This is on an air-gapped system so I'll type in the essential part
>
> Failed to receive messages from Amazon SQS due to 
> com.amazonaws.services.sqs.model.AmazonSQSException: Credential should 
> be scoped to a valid region, not 'us-east-1'. (Service: AmazonSQS; 
> Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID: 
> xxxxx-xxxx-xxxx-xxxx-xxxxxxxx).
>
> I believe that line 289 in AbstractAWSProcessor.java is setting the 
> wrong regionId
>
> this.client.setEndpoint(urlstr, this.client.getServiceName(), 
> this.region.getName());
>
> The amazon docs for the third parameter (regionId) state:
>
> * @param regionId * The ID of the region in which this service resides 
> AND the * overriding region for signing purposes.
>
> This would explain the SignatureDoesNotMatch error.
>
> The properties for the sqs processor work for nifi 1.4.0.
>
> Regards, Andrew
>
>
> On 11/29/18 23:22, Sivaprasanna wrote:
>> Andrew, Can you please share the complete error message and also show 
>> us how the properties look for the GetSQS processor?
>>
>> Thanks.
>>
>> On Fri, Nov 30, 2018 at 3:45 AM Andrew McDonald <amcdonald@ccri.com 
>> <mailto:amcdonald@ccri.com>> wrote:
>>
>>     So I've installed nifi 1.8.0 and endpoint override didn't help
>>
>>     The GetSQS is throwing and exception
>>
>>         com.amazonaws.services.sqs.model.AmazonSQSException:
>>         Credential should be scoped to a valid region, not 'us-east-1'.
>>
>>     I'm guessing that the override endpoint is not overriding the
>>     region b/c 'us-east-1' is not my region.
>>
>>     Regards, Andrew
>>
>>     On 11/29/18 02:09, Sivaprasanna wrote:
>>>     Yeah. This was added in 1.8.0 for SQS. However, the reason why a
>>>     custom enum was added on the NiFi side[1] was to have a proper
>>>     readable region instead of just the region code i.e., Asia
>>>     Pacific (Singapore) instead of ap-southeast-1. However, I raised
>>>     a request later to the AWS Java SDK team to have a readable
>>>     name. It was added recently, if I remember correctly. So ideally
>>>     this enum has to be removed and the one on the official AWS SDK
>>>     has to be leveraged completely. I have created a Jira[2] and
>>>     started working on it. I'll raise a PR soon.
>>>
>>>     [1] https://issues.apache.org/jira/browse/NIFI-5129
>>>     [2] https://issues.apache.org/jira/browse/NIFI-5850
>>>
>>>     Thanks,
>>>
>>>     On Thu, Nov 29, 2018 at 2:20 AM Andrew McDonald
>>>     <amcdonald@ccri.com <mailto:amcdonald@ccri.com>> wrote:
>>>
>>>         zenfenan  added this later to 1.8.0, yay!
>>>
>>>         On 11/28/18 14:23, Andrew McDonald wrote:
>>>>
>>>>
>>>>         This workaround doesn't work for sqs because it doesn't
>>>>         have endpoint override URL property
>>>>
>>>>
>>>>         On 11/28/18 12:16, Michael Moser wrote:
>>>>>         Greetings!  This JIRA ticket [1] describes the recommended
>>>>>         work around for AWS regions that aren't in the list.
>>>>>
>>>>>         -- Mike
>>>>>
>>>>>         [1] - https://issues.apache.org/jira/browse/NIFI-4523
>>>>>
>>>>>
>>>>>
>>>>>         On Wed, Nov 28, 2018 at 11:44 AM Jon Logan
>>>>>         <jmlogan@buffalo.edu <mailto:jmlogan@buffalo.edu>>
wrote:
>>>>>
>>>>>             Andrew,
>>>>>
>>>>>             I know there's a few regions not in the list. I'm not
>>>>>             sure which region you're targeting, but at least for
>>>>>             the case of one of the new regions, I submitted a PR
>>>>>             for this. I haven't dug into it deeply, but it seems
>>>>>             like a better way to do this might be to remove the
>>>>>             enum entirely and get the region list via the AWS API,
>>>>>             or allow a free-form entry.
>>>>>
>>>>>             https://github.com/apache/nifi/pull/3187
>>>>>
>>>>>
>>>>>             Jon
>>>>>
>>>>>             On Wed, Nov 28, 2018 at 10:35 AM Andrew McDonald
>>>>>             <amcdonald@ccri.com <mailto:amcdonald@ccri.com>>
wrote:
>>>>>
>>>>>                 I'm trying to upgrade from 1.4.0 to 1.7.1 but the
>>>>>                 s3 processors can not
>>>>>                 be initialized.
>>>>>
>>>>>                 Nifi 1.7.1 uses 1.11.319 and is throwing an
>>>>>                 IllegalArgumentException: no
>>>>>                 region provided
>>>>>
>>>>>                 The region I'm using isn't in the enum, so is it
>>>>>                 possible to use a
>>>>>                 custom region?
>>>>>
>>>>>                 Regards,Andrew
>>>>>
>>>>>

Mime
View raw message